Skip to Main Content
  • Questions
  • PL/SQL programming to write to file in batches of 2 million rows

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Naveed.

Asked: May 22, 2018 - 11:05 am UTC

Last updated: April 20, 2020 - 9:47 am UTC

Version: oracle 11G

Viewed 10K+ times! This question is

You Asked

I have been assigned to the task.
Task :- In one table , I have 10 million records and I need to export table data into a CSV files/Text files but the condition is that(I need to export into 5 files ,each file should contain 2 million records)
1) 1 to 2 million records should be inserted into 1 file
2) after 2 million records inserted , it has to create a separate file to insert 2 million records to 4 millions record and so on the process should continue until 10 million records

Note :- This has to be solve by using procedures only(PL/SQL)

and Chris said...

Here's one approach:

- Bulk collect the rows into an array. 2 million will probably blow out your memory limits. So set the limit to something (much) smaller.
- Keep a counter of how many rows you've processed
- When this count is a multiple of 2 million, close your current file and open a new one

Which would look something like:

declare
  cursor cur is 
    select * from ...;
  
  type cur_rec is table of cur%rowtype index by pls_integer;
  recs cur_rec;
  
  row_count pls_integer := 0;
begin
  open cur;
  
  loop
  
    if mod ( row_count, 2000000 ) = 0 then
      /* close old file, open new */
    end if;    
    
    fetch cur bulk collect 
    into recs limit 1000;
    
    exit when recs.count = 0;
    
    row_count := row_count + recs.count;
    
    /* write to the file */
    
  end loop;
    
  close cur;
end;
/


There are plenty of examples of how to use UTL_file to write CSV files on this site and the web generally, e.g.:

https://oracle-base.com/articles/9i/generating-csv-files#custom-plsql
https://asktom.oracle.com/pls/asktom/f?p=100:11:0::NO::P11_QUESTION_ID:88212348059

Rating

  (2 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Jaganath, January 20, 2020 - 9:08 am UTC

Hi Tom,

Thanks for the help. It worked as expected.

Thank you so much
Chris Saxon
January 20, 2020 - 10:53 am UTC

You're welcome

nisha, April 20, 2020 - 6:14 am UTC

Can u give the complete code
Chris Saxon
April 20, 2020 - 9:47 am UTC

The "complete code" depends a lot on the specifics of your environment; which table you're querying, files you're writing to, etc.

The template above is a good starting point. I'm sure you can figure out how to write this yourself. Go on, try!

More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library