Skip to Main Content
  • Questions
  • write to file in PL/SQL and sort that file

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question.

Asked: December 11, 2003 - 7:50 am UTC

Last updated: September 13, 2004 - 3:49 pm UTC

Version: 9.0.2

Viewed 1000+ times

You Asked

Tom --

I was wondering if you could provide an example of how to read from a file in PL/SQL, then sort it by, say, two fields - first name and zip code and read the result into variables.

thank you in advance!

AM

and Tom said...

don't use PLSQL!

use an EXTERNAL TABLE. it can't get any easier then this:




ops$tkyte@ORA9IR2> create or replace directory data_dir as '/tmp/'
2 /

Directory created.

ops$tkyte@ORA9IR2> host flat scott/tiger emp > /tmp/emp.dat

ops$tkyte@ORA9IR2> host head /tmp/emp.dat
7369,SMITH,CLERK,7902,17-DEC-80,800,,20
7499,ALLEN,SALESMAN,7698,20-FEB-81,1600,300,30
7521,WARD,SALESMAN,7698,22-FEB-81,1250,500,30
7566,JONES,MANAGER,7839,02-APR-81,2975,,20
7654,MARTIN,SALESMAN,7698,28-SEP-81,1250,1400,30
7698,BLAKE,MANAGER,7839,01-MAY-81,2850,,30
7782,CLARK,MANAGER,7839,09-JUN-81,2450,,10
7788,SCOTT,ANALYST,7566,19-APR-87,3000,,20
7839,KING,PRESIDENT,,17-NOV-81,5000,,10
7844,TURNER,SALESMAN,7698,08-SEP-81,1500,0,30

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> create table external_table
2 (EMPNO NUMBER(4) ,
3 ENAME VARCHAR2(10),
4 JOB VARCHAR2(9),
5 MGR NUMBER(4),
6 HIREDATE DATE,
7 SAL NUMBER(7, 2),
8 COMM NUMBER(7, 2),
9 DEPTNO NUMBER(2)
10 )
11 ORGANIZATION EXTERNAL
12 ( type oracle_loader
13 default directory data_dir
14 access parameters
15 ( fields terminated by ',' )
16 location ('emp.dat')
17 )
18 /

Table created.

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> select empno, ename from external_table order by empno;

EMPNO ENAME
---------- ----------
7369 SMITH
7499 ALLEN
7521 WARD
7566 JONES
7654 MARTIN
7698 BLAKE
7782 CLARK
7788 SCOTT
7839 KING
7844 TURNER
7876 ADAMS
7900 JAMES
7902 FORD
7934 MILLER

14 rows selected.

ops$tkyte@ORA9IR2> select empno, ename from external_table order by ename;

EMPNO ENAME
---------- ----------
7876 ADAMS
7499 ALLEN
7698 BLAKE
7782 CLARK
7902 FORD
7900 JAMES
7566 JONES
7839 KING
7654 MARTIN
7934 MILLER
7788 SCOTT
7369 SMITH
7844 TURNER
7521 WARD

14 rows selected.



Rating

  (21 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

write to file in PL/SQL and sort that file

A reader, December 11, 2003 - 8:43 am UTC

Thank you for response. Do you think your approch will work if the name of the file is actually supposed to be supplied by user as a parameter to this program, part of which is supposed to read data from the file(they do it by the means of an Oracle Form, which passes parameters to our programs)?
Although I do realize that reading a file with PL/SQL is like trying to open a can of peas with a hammer, I could not think of another way to incorporate this task in a program (which is going to be run by the user as descirbed above)..

thanx for great response!

anya

Tom Kyte
December 11, 2003 - 8:53 am UTC

that is a horse of a different color.

is this a FORMS application running on a CLIENT that is to read a file on the CLIENT machine and not the SERVER?

write to file in PL/SQL and sort that file

A reader, December 11, 2003 - 9:06 am UTC

The way our applicaton is set up is that we have Oracle Form that the users access to submitt parameters to various jobs (Pro*C, SQL, SQRs, PL/SQLs). There is actually a shell script that is connected to this form that passes these parms to various programs. Anyway, by the means of various includes that we wrote we get these parms from the shell.
The user wants to use this form as they always do to submitt a name of the file and its directory (/userdir/filename.dat) as one of the parameters of this job. So that the job (supposed to be written in PL/SQL) can read the records from the file (which is by the way not delemeted by anything) and try to match these records with the existing records in the database. So, the task here is to try to minimize the work for the user..I dont think they will be able to load this file into external tables :)

Tom Kyte
December 11, 2003 - 9:59 am UTC

you'll have to get the file from the USERS system to the SERVER.

how do you plan on doing that? else the SERVER (where the job runs I assume) will not be able to read said file.

write to file in PL/SQL and sort that file

A reader, December 11, 2003 - 10:03 am UTC

Usually the user keeps this file in her/his Unix directory and they want to specify this dir/filename as one of the parameters. We actually used to write SQRs that would read the file from the specified dir and then sort it and process it. However, I do not see how this will be feasible in PL/SQL...

Tom Kyte
December 11, 2003 - 10:18 am UTC

as much as I hate doing ddl in plsql -- you might consider it here.

how many times a day will this happen.

write to file in PL/SQL and sort that file

anya, December 11, 2003 - 10:28 am UTC

The user will be submitting a job once a week actually..

Tom Kyte
December 11, 2003 - 12:17 pm UTC

just use execute immediate to create a table, use it, execute immediate to drop it.

A reader, December 11, 2003 - 3:49 pm UTC

Tom, I never used external table feature before. Is it possible to put it all in PL/SQL program? Can you give an example of how to put the data into table using external table (inside the PL/SQL program)?

Tom Kyte
December 11, 2003 - 5:41 pm UTC

declare
l_cursor sys_refcursor;

begin
execute immediate 'create table t' || userenv('sessionid') || '....... organization external';

execute immediate open l_cursor for 'select * from t' || userenv('sessionid') || ' order by whatever';

process l-cursor


execute immediate 'drop table t' || userenv( 'sessionid' );

end;



write to file in PL/SQL and sort that file

A reader, December 12, 2003 - 2:02 pm UTC

Tom --

one last thing, can you provide an example on how to copy a file from one directory to another form within PL/SQL? The thing is that our users want to submit the directory/filename as parameter to a PL/SQL package. Fine, but in order to use UTL_FILE procedures utl_dir has to be specifed in init.ora file (or in 9i as I understand it you can use create direcrtory..). The problem is that we do not want to specify these directories in the init.ora because different users might run the program with different directories and we dont want to put them all in init.ora.
So, I was thinking that my PL/SQL package would have a procedure to copy the file form whatever directory the user specifies and to put it in the directory that is specified in init.ora file - this way UTL_FILE package can do its majic there.

thank you!

Tom Kyte
December 13, 2003 - 11:17 am UTC

you'd most likely be looking at a java stored procedure to do this.

plsql isn't appropriate right now for doing a file copy.

bfiles cannot be written to
utl_file does text only in 8i

think about your question -- if plsql could move a file from somewhere it isn't allowed to read -- to somewhere it is, of what use would be restricting where it could read!!!!

with a java sp, you can use dbms_java to grant read/write on specific directories/files very granularly at runtime without restarting.

write to file in PL/SQL and sort that file

anya, December 14, 2003 - 10:05 am UTC

That's a good suggestion. 
I am on 9i and I tried the following:

ORA-29283: invalid file operation with UTL_FILE

My configurations: AIX 4.3 
Oracle9i Enterprise Edition Release 9.2.0.1.0 - 64bit Production 
With the Partitioning, OLAP and Oracle Data Mining options 
JServer Release 9.2.0.1.0 - Production 



1. 

SQL> CREATE OR REPLACE DIRECTORY gwcustom as '/utl/oracle_data'; 

Directory created. 

SQL> GRANT READ ON DIRECTORY gwcustom to public; 

Grant succeeded. 

2. 

SQL> CREATE OR REPLACE DIRECTORY test_dir as '/5u/tst000/tstaog/test'; 

Directory created. 

SQL> GRANT READ ON DIRECTORY test_dir to public; 

Grant succeeded. 

3. 

SQL> select * from dba_directories; 

OWNER DIRECTORY_NAME 
------------------------------ ------------------------------ 
DIRECTORY_PATH 
-------------------------------------------------------------------------------- 
SYS GWCUSTOM 
utl/oracle_data 

SYS DATAHOME_FINAID 
/jobout/faid 

SYS DATA_DIR 
/tmp/ 


OWNER DIRECTORY_NAME 
------------------------------ ------------------------------ 
DIRECTORY_PATH 
-------------------------------------------------------------------------------- 
SYS TEST_DIR 
/5u/tst000/tstaog/test 


When I cd to the /url/oracle_data it is there - I see the directory, but when I cd to 
this directory it is not there! 

3. Trying to take a file from a directory (say my home dir) and copy it to GWCUSTOM directory (created by using create dir command) 
I try the following: 

declare 
file_suffix VARCHAR2 (100) 
:= TO_CHAR (SYSDATE, 'YYYYMMDDHHMISS'); 
begin 

UTL_FILE.fcopy ( 
src_location => '/my_home_dir/home', 
src_filename => 'test.dat', 
dest_location => 'GWCUSTOM', 
dest_filename => 'anya' 
|| file_suffix 

|| '.dat' 
); 

end; 

I get the following error: 

declare 
* 
ERROR at line 1: 
ORA-29283: invalid file operation 
ORA-06512: at "SYS.UTL_FILE", line 18 
ORA-06512: at "SYS.UTL_FILE", line 1064 
ORA-06512: at line 36 

Any ideas about what can be wrong here? 

Also, I could not find anywhere in the documentation whether you  have to create both directories by  create directory command (the one you are taking the file from and the one you are copying it to). My understanding was that I have to just specify one dir with this command and that I will be able to copy files there from wherever I want...
I searched metalink and it seems that 80% of the people do not have any luck with this new 9i feature and 20% do....


 

Tom Kyte
December 14, 2003 - 10:48 am UTC

quote:

"When I cd to the /url/oracle_data it is there - I see the directory, but when I 
cd to this directory it is not there! "

HUH?

but in anycase, your query shows "no leading / on utl/oracle"  so, I think the command you cut and pasted here is differen then the one you actually executed OR the query cut and paste was done wrong -- something does not add up there.


You use DIRECTORY objects in utl_file in 9i - not "real paths", why would we use "directory objects" in one place and "real paths" in another place?  You use directory objects -- or a path specified in UTL_FILE_DIR (the init.ora parameter)

I sort of disagree with your 80/20 -- for you see, the vast majority of users never file a tar in the first place!


But here is a simple, teeny example showing you the basics here.  But, I don't know why you would do this (the copy) since you need to either

a) have utl_file_dir set (meaning plsql could already read it)
b) have a directory object created (meaning plsql could already read it)




ops$tkyte@ORA920> create or replace directory my_home as '/home/tkyte'
  2  /
 
Directory created.
 
ops$tkyte@ORA920> create or replace directory tmp_dir as '/tmp'
  2  /
 
Directory created.
 
ops$tkyte@ORA920>
ops$tkyte@ORA920> !ls -l /tmp/test.dat /home/tkyte/test.dat
ls: /tmp/test.dat: No such file or directory
ls: /home/tkyte/test.dat: No such file or directory
 
ops$tkyte@ORA920> !echo hello world > /home/tkyte/test.dat
 
ops$tkyte@ORA920> !ls -l /tmp/test.dat /home/tkyte/test.dat
ls: /tmp/test.dat: No such file or directory
-rw-rw-r--    1 tkyte    tkyte          12 Dec 14 10:50 /home/tkyte/test.dat
 
ops$tkyte@ORA920>
ops$tkyte@ORA920> begin
  2     UTL_FILE.fcopy ( src_location => 'MY_HOME',
  3                      src_filename => 'test.dat',
  4                      dest_location => 'TMP_DIR',
  5                      dest_filename => 'test.dat' );
  6  end;
  7  /
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA920>
ops$tkyte@ORA920> !ls -l /tmp/test.dat /home/tkyte/test.dat
-rw-rw-r--    1 tkyte    tkyte          12 Dec 14 10:50 /home/tkyte/test.dat
-rw-rw-r--    1 ora920   ora920         12 Dec 14 10:50 /tmp/test.dat
 
ops$tkyte@ORA920> !cat /tmp/test.dat
hello world
 
ops$tkyte@ORA920>
 

write to file in PL/SQL and sort that file

anya, December 15, 2003 - 7:58 am UTC

sorry, it was late..What I meant to say is that when I query dba_directories, I see that my dir got created, but when I cd there, I get "no such file or directory"..Am I totally off track here?
Most importanlty, do you have to create (with create dir command - I am on 9i) BOTH directories - the one you are copying files from and the one you are copying files to? I thought ( and, again, I could have been totally wrong). The thing is that the directory from which I will be copying will be different every time the program is run...I thought it was possible, am I wrong?

Tom Kyte
December 15, 2003 - 9:56 am UTC

how do you "cd" from within the database?

createing a directory object creates a MAPPING in the database. It does nothing t the file system. You weren't expecting it to create a file system directory were you?

the directory can be different every time AS LONG AS

a) the directory exists in the file system
b) you issued the create directory command

write to file in PL/SQL and sort that file

A reader, December 15, 2003 - 10:31 am UTC

I was expecting a real dir..ok, thank you, Tom!!!!

Deleting after file is processed

Ahmad, December 15, 2003 - 10:32 am UTC

Tom,

I have a similar situation and I am using your example described at 
http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:439619916584
 and would like to delete the file when its processed. I looked in metalink and found Note:190311.1 and got the following  java procedure :

CREATE OR REPLACE AND RESOLVE JAVA SOURCE NAMED "DelFile" AS
import java.io.*;   
  public class  DelFile    
  {   
   public static void remove(String filename)     
   {        File myfile = new File(filename);   
     try  
     {  
       System.out.println("Deleting ..... "+filename );
       boolean b = myfile.delete();  
     }  
     catch (Exception e) 
     {  
      System.out.println(e);  
     }  
    }  
  }
/

and created a pl/sql wrapper called deletefile
and my DBA has given me appropriate privilege ie 

EXEC Dbms_Java.Grant_Permission('SCHEMA-NAME', 'java.io.FilePermission', '/export/data/*', ' delete');

and when I tried to delte a file from /export/data , file is not being deleted. I did

SQL> exec dbms_java.set_output(1000);
SQL> exec deletefile('/export/data/test1.csv');
Deleting...../export/data/test1.csv


I can see this message , however file is not being deleted in the unix directory.

Any idea what may be wrong here. We have 9i-9.0.1.4.0

Thanks

 

Tom Kyte
December 15, 2003 - 10:42 am UTC

I hate code that catches and hides all exceptions.

get rid of the try/catch block. let whatever error is happening propagate up.


at the very least, put a system.out.println AFTER the delete saying "it appears to have worked"

I did but no effect

Ahmad, December 15, 2003 - 11:21 am UTC

I get rid of try catch and System.out.println("Deleting ..... "+filename ) and still same effects. It does not delete. My oracle unix login has capability to read/delete/write files in the directory. I donot understand whats the problem?

Tom Kyte
December 15, 2003 - 3:44 pm UTC

what is the boolean being returned?

and do you want to just try utl_file.fremove?

(i didn't want to get RID of the system.out.printlns -- i wanted MORE of them

reader, December 16, 2003 - 2:53 am UTC

Tom, Why you use word "then" instead of "THAN" I think you missed the lecture when your teacher was teaching about than.

we have Rel 1- 9i

A reader, December 16, 2003 - 9:01 am UTC

Tom,

we cannot use utl_file.fremove, since we have rel 1 - 9i. The boolean variable returns FALSE. So its not deleting the file. Any idea what should be the problem?

Thanks

Ahmad


Tom Kyte
December 16, 2003 - 11:33 am UTC

permissions.

what are the permissions on the file itself in the OS (can Oracle erase that file)

probably not.  consider


ops$tkyte@ORA9IR2> CREATE OR REPLACE AND RESOLVE JAVA SOURCE NAMED "DelFile" AS
  2  import java.io.*;
  3    public class     DelFile
  4    {
  5     public static void remove(String filename)     throws Exception
  6     {
  7     File myfile = new File(filename);
  8
  9         System.out.println("Deleting ..... "+filename );
 10         if ( myfile.delete() )
 11            System.out.println( "success" );
 12         else
 13            System.out.println( "failure" );
 14    }
 15   }
 16  /
                                                                                             
Java created.
                                                                                             
ops$tkyte@ORA9IR2> create or replace procedure delfile( p_string in varchar2 )
  2  as language java
  3  name 'DelFile.remove( java.lang.String )';
  4  /
 
Procedure created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> EXEC Dbms_Java.Grant_Permission(user, 'java.io.FilePermission', '/tmp/*', ' delete');
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> @connect /
ops$tkyte@ORA9IR2> set termout off
ops$tkyte@ORA9IR2> REM GET afiedt.buf NOLIST
ops$tkyte@ORA9IR2> set termout on
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> !echo Hello > /tmp/t1.dat
 
ops$tkyte@ORA9IR2> !ls -l /tmp/t?.dat
-rw-rw-r--    1 tkyte    tkyte           6 Dec 16 11:43 /tmp/t1.dat
-rw-rw-r--    1 ora9ir2  ora9ir2         6 Dec 16 11:43 /tmp/t3.dat
 
ops$tkyte@ORA9IR2> exec dbms_java.set_output(100000);
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> exec delfile( '/tmp/t1.dat' );
Deleting ..... /tmp/t1.dat
failure
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> exec delfile( '/tmp/t3.dat' );
Deleting ..... /tmp/t3.dat
success
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> !ls -l /tmp/t?.dat
-rw-rw-r--    1 tkyte    tkyte           6 Dec 16 11:43 /tmp/t1.dat
 


t3.dat -- gone, t1.dat -- still there.  if I su'ed to ora9ir2 -- it cannot erase /tmp/t1.dat, not allowed.


[tkyte@localhost tkyte]$ msu - ora9ir2
[ora9ir2@localhost ora9ir2]$ rm /tmp/t1.dat
rm: remove write-protected regular file `/tmp/t1.dat'? y
rm: cannot remove `/tmp/t1.dat': Operation not permitted
[ora9ir2@localhost ora9ir2]$


so, must be a perm problem at the OS level, remember the java proc is running as ORACLE.

 

permission is Ok

Ahmad, December 16, 2003 - 11:42 am UTC

Yes, I can telnet and remove the file in the directory.

Tom Kyte
December 16, 2003 - 1:47 pm UTC

do it like I did then -- show us.

ls -l

lets see everything like I did (showing this WORKS when you can in fact ERASE the file)

show us the userid under which the TNSLISTENER is running if you connect over the network (since the dedicated server will run as that user)

show us everything we need to see that show this "can work"

I am not sure if Oracle has permission

Ahmad, December 16, 2003 - 11:46 am UTC

Tom,

I will ask our DBA if Oracle has permission to delete file in the directory? I can howevere, delete with mu userid.

Thanks



Tom Kyte
December 16, 2003 - 1:48 pm UTC

whats the "mu" userid?

checking with DBA

Ahmad, December 16, 2003 - 2:03 pm UTC

I am checking with my DBA, he is outside somewhere. I donot think Oracle has permission on my directory.
"mu" is a typo ,it is my userid.

Also I donot hasve sqlplus access in UNIX environment , I can only do it from windows so I cannot perform your example ie !echo Hello > /export/home/nyssis/staging/t1.dat.

But I am pretty sure Oracle does not have permission , so you are saying that Oracle must be given OS level permission on the directory ie /export/home/nyssis/staging in my case?

I will make sure this and will update to you.

Thanks for all your help.



Tom Kyte
December 16, 2003 - 2:32 pm UTC

yes, that is exactly what I'm saying. java will not give you the ability to do something you cannot normally do. oracle isn't allowed to erase your files, java won't let you do it (the OS stops it)

thanks it work

Ahmad, December 17, 2003 - 7:55 am UTC

Tom,
Thank you and it works. However, I have one question. I was able to read the file using DBMS_JAVA and as you described at the thread </code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:439619916584 <code> and no permission was given to Oracle to read files from my directory. I just granted read permission to my user/schema to read the files. Why this works withought granting read permission to Oracle and delete does't work?

Ahmad

Tom Kyte
December 18, 2003 - 7:05 am UTC

because Oracle did in fact have the permission apparently.

we do not, cannot bypass the OS here. The user the dedicated server or shared server you were using was running as HAD access to the file.

log into the OS and try it, you'll see.

getting data from client side flat files and saved into database

M Faisal, January 14, 2004 - 10:23 am UTC

Tom your information is very helpful. Can you clear more. how we will do it when flat file is in client machine and we get file name and directory through forms and want to process that file in our database. I have come to know from this subject we can't use
1- External tables because files are in client machine and we can only give directory information where the server is installed.
2- UTL_FIle because the same problem as above.

We can use SQL*Loader then we have to install SQL*Loader on each client machine what if we are using forms over the internet.
I think we can use TEXT_IO forms pl/sql package but i am not sured how good it is.
Is it possible we can save flat file data in CLOB and save it in our database and from there we can process it.

Cau you suggest any other method and clear my concept regarding above subject.


Tom Kyte
January 14, 2004 - 3:58 pm UTC

you'll use TEXT_IO, but you'll have DIY (do it yourself)

Need your Opinion !!!

Riaz Shahid, May 05, 2004 - 1:32 pm UTC

Tom !

What if i want to add a column (and subsequently assign it some value, say sysdate, at the time of data loading) to external table that is not the in file which contains data?

Tom Kyte
May 05, 2004 - 3:03 pm UTC

views are perfect for that.

create view et_view
as
select sysdate the_date, et.*
from external_table et;



Is this pseudo-code?

Michael Stelly, September 09, 2004 - 5:13 pm UTC

In a follow-up response on how to place the code in a pl/sql program, you posted the following:

declare
--> l_cursor sys_refcursor;

begin

...your 'open cursor' code...

--> process l-cursor

... end of your code...
);

end;

-------
I'm a novice with ref cursors. Is this how they are declared and run (ie, the lines marked with -->) or is that just short-hand. I'm not familiar with the sys_refcursor data type or the process command. Am I just reading this too literally? If so, what would be the proper declaration and call? Thanks for your time.

Tom Kyte
September 09, 2004 - 6:26 pm UTC

"process l-cursor" was pure psuedo code saying "fetch and do whatever here"

sys_refcursor is a datatype in 9i and above.  in 8i you would:


ops$tkyte@ORA9IR2> declare
  2      type rc is ref cursor;
  3      l_cursor rc;
  4      l_data   varchar2(20);
  5  begin
  6      open l_cursor for 'select * from dual';
  7      loop
  8          fetch l_cursor into l_data;
  9          exit when l_cursor%notfound;
 10
 11          dbms_output.put_line( 'This represents processing this row ' || l_data );
 12      end loop;
 13      close l_cursor;
 14  end;
 15  /
This represents processing this row X
 
PL/SQL procedure successfully completed.
 
 

Can you copy a file from machine to machine using the database?

A reader, September 13, 2004 - 2:31 pm UTC

I don't want to use FTP or a mapped drive to copy a file from my one database server to another. The one database is on a laptop and is disconnected frequently from the network.

How would you do this?

Thanks!!

Tom Kyte
September 13, 2004 - 3:49 pm UTC

there is utl_file and/or bfiles.
you can access a bfile, and use put_raw in utl_file to write it.


there is of course java stored procedures - read remote, write local.


but, it would be best if they weren't files but CLOBS and BLOBS, then a simple sql query would move them.



More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library