Skip to Main Content
  • Questions
  • How to export xml data ( clob column ) to a csv file

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, Venkatraman.

Asked: June 29, 2016 - 2:11 pm UTC

Last updated: July 08, 2016 - 12:11 pm UTC

Version: 11.2.0.4.0

Viewed 10K+ times! This question is

You Asked

Hi,

Let say, I have a table employee ( id number, name varchar2, notes clob ).
I use dbms_util to write the data to a file as csv.

The id (number) and name( Varchar2) columns are writing to csv file as expected. But when it comes to notes (clob-xml data ), it writes only " <?xml version= " and other characters are not coming.

Could you please let me know, how can I write xml data to csv file.

Thank you.

and Connor said...

What is "dbms_util" ? That isn't one of our packages.

It strikes me as very odd to write XML as a single chunk to a CSV file... or is your intention to parse out the XML into components as well ?

In the latter case, you might need some xslt to do that, eg

http://stackoverflow.com/questions/17314062/how-to-convert-xml-to-csv-using-xsl


Rating

  (6 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Venkatraman Muralidharan, June 30, 2016 - 1:48 pm UTC

Hi, Thank you.
I don't want to parse the xml data. I want to write the columns into csv which I understand from
https://asktom.oracle.com/pls/asktom/f?p=100:11:0::NO::P11_QUESTION_ID:88212348059

Now I am able to write all fields into csv file. But now the issue is, while writing the xml data into csv file by chunking ( looping ), without using utl_file.new_line(file); it throws error "ORA-29285: file write error".
Could you please let me know, how to overcome this error.
Connor McDonald
July 01, 2016 - 2:22 am UTC

Paste in your code (or a smaller test case) so we can see it.

code

Venkatraman Muralidharan, July 01, 2016 - 2:07 pm UTC

Hi, please find below the code.
I am getting the write error. If I use utl_file.new_line, then it completes, but the xml data writes to next line and which is not my requirement and makes the csv file incorrect.

DECLARE
dirPath VARCHAR2(101) := '/u01/Utl_dir';
g_sep VARCHAR2(5) := ',';
Lv_cursor binary_INTEGER;
p_query VARCHAR2(32767) := 'SELECT * FROM table WHERE';
Lv_col_cnt PLS_INTEGER;
Lv_desc_tab DBMS_SQL.desc_tab;
Lv_file UTL_FILE.file_type;
l_offset NUMBER :=1;
l_columnValClob CLOB;
l_clobLen pls_integer;
l_amount pls_integer;
l_buffer CLOB;
lv_buffer VARCHAR2(32767);
FileName VARCHAR2(101) := 'sample';
Lv_rows PLS_INTEGER;
BEGIN
Lv_file := utl_file.fopen(DirPath,FileName||'.csv','w', 32767);
Lv_cursor := DBMS_SQL.open_cursor;
DBMS_SQL.parse(Lv_cursor, p_query, DBMS_SQL.native);
DBMS_SQL.describe_columns(Lv_cursor, Lv_col_cnt, Lv_desc_tab);
FOR i IN 1 .. Lv_col_cnt
LOOP
IF Lv_desc_tab(i).col_type = 112 THEN
dbms_sql.define_column ( Lv_cursor , i , l_columnValClob );
ELSE
dbms_sql.define_column ( Lv_cursor , i , Lv_buffer , 32767 );
END IF;
--Modification 1 ends for the xml / clob
END LOOP;
dbms_output.put_line('555');
Lv_rows := DBMS_SQL.execute(Lv_cursor);
FOR i IN 1 .. Lv_col_cnt
LOOP
IF i > 1 THEN
UTL_FILE.put(Lv_file, g_sep);
END IF;
IF Lv_desc_tab(i).col_type = 112 THEN
BEGIN
l_offset := 1;
dbms_sql.column_value ( Lv_cursor , i , l_columnValClob );
l_clobLen := DBMS_LOB.GETLENGTH(l_columnValClob);
LOOP
l_buffer:=DBMS_LOB.substr(l_columnValClob,l_amount,l_offset);
UTL_FILE.put(Lv_file, l_buffer);
l_offset := l_offset + l_amount;
l_clobLen := l_clobLen-l_amount;
utl_file.fflush(Lv_file);
EXIT
WHEN l_clobLen <= 0;
END LOOP;
EXCEPTION
WHEN NO_DATA_FOUND THEN
UTL_FILE.put(Lv_file, '"');
END;
ELSE
DBMS_SQL.COLUMN_VALUE(Lv_cursor, i, Lv_buffer);
UTL_FILE.put(Lv_file, '"'||Lv_buffer||'"');
-- utl_file.fflush(Lv_file);
END IF;
END LOOP;
UTL_FILE.fclose(Lv_file);
EXCEPTION
WHEN OTHERS THEN
NULL;
END ;
Chris Saxon
July 04, 2016 - 6:25 am UTC

If your data exceed 32k in a single line, you are going to have issues using UTL_FILE.

You *could* workaround that by writing the data out in raw format, eg

https://asktom.oracle.com/pls/apex/f?p=100:11:::NO:RP:P11_QUESTION_ID:9525301800346126803

but then *you* become responsible for all characterset control etc etc, because we're now just sending out bytes without any consideration for what kind of the data that is.

Venkatraman Muralidharan, July 04, 2016 - 2:01 pm UTC

Thank you. I will check and let you know.

Venkatraman Muralidharan, July 05, 2016 - 7:21 am UTC

Hi, I tried to write the data using RAW.
But still I am getting it in 3 lines ( 3 cells ) in csv file which was supposed to be in one cell.
The length of my clob data ( xml data ) is : 96539
Please find below my code,
DECLARE
Lv_cursor PLS_INTEGER;
Lv_file UTL_FILE.file_type;
P_SubstrVal VARCHAR2(4000) ;
lv_file_name VARCHAR2 (100):='testing.csv';
P_Directory VARCHAR2(101) :='/u01/Utl_dir';
crlf VARCHAR2(2) := chr(10)||chr(13);
l_columnValClob CLOB;
P_offset NUMBER :=1;
P_amount NUMBER := 3000;
l_clobLen NUMBER :=0;
BEGIN
Lv_cursor := DBMS_SQL.open_cursor;
SELECT finder_set_xml
INTO l_columnValClob
FROM finder_set
WHERE finder_set_xid ='GTMINABOX_GTM_TRANSACTION';
l_clobLen := DBMS_LOB.GETLENGTH(l_columnValClob);
Lv_file := utl_file.fopen(P_Directory,lv_file_name,'wb');
WHILE P_offset < l_clobLen
LOOP
P_SubstrVal := dbms_lob.substr(l_columnValClob,P_amount,P_offset);
utl_file.put_raw(Lv_file,utl_raw.cast_to_raw(P_SubstrVal));
P_offset:=P_offset+P_amount;
END LOOP;
UTL_FILE.fclose(Lv_file);
EXCEPTION
WHEN OTHERS THEN
NULL;
END ;
Connor McDonald
July 06, 2016 - 2:34 am UTC

Does your XML contain either chr(10) or chr(13) ?

If it does, you would need to strip that out first, eg

SQL> set serverout on
SQL> declare
  2    l_xml varchar2(1000) :=
  3  q'{<?xml version='1.0'  encoding='UTF8' ?>
  4  <ROW>
  5    <COLUMN NAME="EMPNO"><![CDATA[7369]]></COLUMN>
  6    <COLUMN NAME="ENAME"><![CDATA[SMITH]]></COLUMN>
  7    <COLUMN NAME="JOB"><![CDATA[CLERK]]></COLUMN>
  8    <COLUMN NAME="MGR"><![CDATA[7902]]></COLUMN>
  9    <COLUMN NAME="HIREDATE"><![CDATA[17-12-1980]]></COLUMN>
 10    <COLUMN NAME="SAL"><![CDATA[800]]></COLUMN>
 11    <COLUMN NAME="COMM"><![CDATA[]]></COLUMN>
 12    <COLUMN NAME="DEPTNO"><![CDATA[20]]></COLUMN>
 13  </ROW>}';
 14
 15  begin
 16     dbms_output.put_line('--------------------------');
 17     dbms_output.put_line(l_xml);
 18     dbms_output.put_line('--------------------------');
 19     dbms_output.put_line(replace(replace(l_xml,chr(10)),chr(13)));
 20     dbms_output.put_line('--------------------------');
 21  end;
 22  /
--------------------------
<?xml version='1.0'  encoding='UTF8' ?>
<ROW>
  <COLUMN NAME="EMPNO"><![CDATA[7369]]></COLUMN>
  <COLUMN NAME="ENAME"><![CDATA[SMITH]]></COLUMN>
  <COLUMN NAME="JOB"><![CDATA[CLERK]]></COLUMN>
  <COLUMN NAME="MGR"><![CDATA[7902]]></COLUMN>
  <COLUMN NAME="HIREDATE"><![CDATA[17-12-1980]]></COLUMN>
  <COLUMN NAME="SAL"><![CDATA[800]]></COLUMN>
  <COLUMN NAME="COMM"><![CDATA[]]></COLUMN>
  <COLUMN NAME="DEPTNO"><![CDATA[20]]></COLUMN>
</ROW>
--------------------------
<?xml version='1.0'  encoding='UTF8' ?><ROW>  <COLUMN NAME="EMPNO"><![CDATA[7369]]></COLUMN>  <COLUMN NAME="ENAME"><![CDATA[SMITH]]></COLUMN>
NAME="MGR"><![CDATA[7902]]></COLUMN>  <COLUMN NAME="HIREDAT.....
--------------------------

PL/SQL procedure successfully completed.


Venkatraman Muralidharan, July 07, 2016 - 10:32 am UTC

Hi,

Yes, I checked but there are no new line characters or line feeds. Any way I replaced but still they are breaking into new line.

I have noticed that, approximately after 32500 characters it breaks into a newline and again after approximately around 65200 characters it writes into a new line.
The new line splits the word and write into a new line.

For e.g. ( a data with word as 'transactions' ) first line ends with 'Trans' and the next line starts with 'ctions'. It misses the letter 'a'
Connor McDonald
July 08, 2016 - 3:26 am UTC

I think it must be something with your data. I just did this:

SQL> drop table t purge;

Table dropped.

SQL> create table t ( x clob );

Table created.

SQL>
SQL> declare
  2    c clob;
  3  begin
  4    insert into t values ( empty_clob())
  5    returning x into c;
  6
  7    c := q'{<?xml version='1.0'  encoding='UTF8' ?>}<RESULTS>}';
  8    loop
  9      c := c || '<ROW><COLUMN NAME="EMPNO"><![CDATA[7369]]></COLUMN><COLUMN NAME="ENAME"><![CDATA[SMITH]]></COLUMN><COLUMN NAME="JOB"><![CDATA[CLERK]]></COLUMN><COLUMN NAME="MGR"><![CDATA[7902]]></COLUMN></ROW>';
 10      exit when length(c) > 90000;
 11    end loop;
 12    c := c || '</RESULTS>';
 13
 14    update t set x = c;
 15    commit;
 16  end;
 17  /

PL/SQL procedure successfully completed.

SQL> select length(x), x from t;

 LENGTH(X) X
---------- --------------------------------------------------------------------------------
     90190 <?xml version='1.0'  encoding='UTF8' ?>}<RESULTS><ROW><COLUMN NAME="EMPNO"><![CD

1 row selected.

SQL>
SQL> DECLARE
  2    Lv_cursor PLS_INTEGER;
  3    Lv_file UTL_FILE.file_type;
  4    P_SubstrVal VARCHAR2(4000) ;
  5    lv_file_name VARCHAR2 (100):='testing.csv';
  6    P_Directory VARCHAR2(101) :='TEMP';
  7    crlf VARCHAR2(2) := chr(10)||chr(13);
  8    l_columnValClob CLOB;
  9    P_offset NUMBER :=1;
 10    P_amount NUMBER := 3000;
 11    l_clobLen NUMBER :=0;
 12  BEGIN
 13    Lv_cursor := DBMS_SQL.open_cursor;
 14    SELECT x
 15    INTO l_columnValClob
 16    FROM t;
 17
 18    l_clobLen := DBMS_LOB.GETLENGTH(l_columnValClob);
 19    Lv_file := utl_file.fopen(P_Directory,lv_file_name,'wb');
 20    WHILE P_offset < l_clobLen
 21    LOOP
 22      P_SubstrVal := dbms_lob.substr(l_columnValClob,P_amount,P_offset);
 23      utl_file.put_raw(Lv_file,utl_raw.cast_to_raw(P_SubstrVal));
 24      P_offset:=P_offset+P_amount;
 25    END LOOP;
 26    UTL_FILE.fclose(Lv_file);
 27  END ;
 28  /

PL/SQL procedure successfully completed.

SQL>
SQL>


C:\temp>wc -l testing.csv
0 testing.csv

C:\temp>awk "{print length($0)}" testing.csv
90190

C:\temp>cat testing.csv
<?xml version='1.0' encoding='UTF8' ?>}<RESULTS><ROW><COLUMN NAME="EMPNO">...

So

- it is "zero" lines long (ie, there are not cr/lf in there).
- it is 90910 bytes, same as the clob
- the data looks correct.

Venkatraman Muralidharan, July 08, 2016 - 8:37 am UTC

Hi,

The code was working correctly. Since the size of cell in excel is limited to 32767, the data result was split into new line. So I assumed that there was new line character. When I opened the same in other editor, it is good and works as expected.

Thank you so much.

Regards,
Venkat
Connor McDonald
July 08, 2016 - 12:11 pm UTC

Glad you got to the bottom of it.

More to Explore

CLOB/BLOB/etc

Complete documentation on Securefiles and Large Objects here