Skip to Main Content
  • Questions
  • Insert Large set of data from View to Table faster

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, Rashmi Ranjan.

Asked: September 22, 2017 - 4:12 am UTC

Last updated: September 25, 2017 - 3:43 am UTC

Version: NA

Viewed 1000+ times

You Asked

Hi Tom,

Good morning.
We have a dynamic view which is growing day by day due to the combinations are increasing.

SO we have decided to create a table from the view , for which we have created a procedure which runs fine but it takes lot of time like 3 hour 40 minutes to insert all the records from view to table.


Here is the code i am using for the insert the reccords.
FOR i IN 1..vn_loopctr LOOP
vn_cnt := vn_cnt + 1;
V_SQL := 'INSERT INTO ' || COMB_TABLE_NAME || ' SELECT /*+ parallel (c,15) */ c.* FROM ' || VS_COPY_TABLE|| ' c WHERE c.comb_id between ' || VN_LAST || ' and ' || CHUNK_SIZE * VN_CNT ;
EXECUTE IMMEDIATE V_SQL;
COMMIT;
v_sql := 'analyze table '||COMB_TABLE_NAME||' compute statistics';
EXECUTE IMMEDIATE V_SQL;


vn_last := (CHUNK_SIZE * vn_cnt) + 1;

v_sql := 'INSERT INTO ' || TARGET_TABLE || ' SELECT /*+ parallel (pd,15) */ pd.* FROM ' || BIEO_TABLE || ' pd';
execute immediate v_sql;
commit;


v_sql:= 'truncate table ' || COMB_TABLE_NAME;

execute immediate v_sql;
commit;

END LOOP;

Only the insert section seems to be taking very long time. for each chuck it is taking 4 min. and we have to process 144 chunks.

Kindly suggest if we do do something to run it faster than this.

Regards,
Giri

and Connor said...

1 - get some genuine timings
============================

For example

FOR i IN 1..vn_loopctr LOOP
dbms_output.put_line(systimestamp);
vn_cnt := vn_cnt + 1;
V_SQL := 'INSERT INTO ' || COMB_TABLE_NAME || ' SELECT /*+ parallel (c,15) */ c.* FROM ' || VS_COPY_TABLE|| ' c WHERE c.comb_id between ' || VN_LAST || ' and ' || CHUNK_SIZE * VN_CNT ;
EXECUTE IMMEDIATE V_SQL;
COMMIT; 
dbms_output.put_line(systimestamp);
v_sql := 'analyze table '||COMB_TABLE_NAME||' compute statistics'; 
EXECUTE IMMEDIATE V_SQL;


So you can get elapsed times for every component. Check out Logger for something more rigourous

https://github.com/OraOpenSource/Logger


2 - get some good trace data
============================
When you've located the part that is running slowly, add this:

dbms_monitor.session_trace_enable(waits=>true,binds=>true);
[then your slow code]
dbms_monitor.session_trace_disable;

3 - examine the tkprof formatted trace file
============================
From here you'll get executions plan, sql elapsed times etc and you'll be equipped to do some tuning

This will give you the best data to work with. Some things that look like potential options

a) parallel dml (your queries are parallel but your inserts are not)
b) direct load using the APPEND hint
c) dont use "analyze", its obsolete
d) use dbms_stat.set_table_stats using the cumulative sql%rowcounts to avoid a full dbms_stats run

Rating

  (2 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Thank you for quick response

Giri Giri, September 22, 2017 - 4:56 am UTC


I am new to advance PL/SQL

Giri Giri, September 22, 2017 - 5:18 am UTC

Hi,

I am new to this forum and pl.sql advance .

Please help me what i need to do.

regards,
Giri
Connor McDonald
September 25, 2017 - 3:43 am UTC

Have you done 1, 2 and 3 ?

More to Explore

Performance

Get all the information about database performance in the Database Performance guide.