Skip to Main Content
  • Questions
  • Tune update pl/sql code with bulk collect/FORALL/LIMIT

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question.

Asked: April 27, 2010 - 10:43 am UTC

Last updated: April 28, 2010 - 12:55 pm UTC

Version: 10.2.0.3

Viewed 1000+ times

You Asked

Hi Tom:

I have a procedure that update a table with 200 millions of rows. Here is the original code:

PROCEDURE updt_tab

FOR rec IN (SELECT *
from TABLE1 V
left outer join TABLE2 S
on V.polnum = S.s7poln
AND V.polren = S.s7renn
AND V.polseq = S.s7seqn
AND V.vehnum = S.s7vehn
AND V.linecvg = S.s7lcvg) LOOP

update TABLE1 P
set P.CEDEDPOL = rec.s7ceding
where P.polnum = rec.polnum
and P.polren = rec.polren
and P.polseq = rec.polseq
and P.vehnum = rec.vehnum
and P.linecvg = rec.linecvg
and P.sat = 'ABC';

update TABLE1 P
set P.CEDEDCVG = rec.s7cedecvg
where P.polnum = rec.polnum
and P.polren = rec.polren
and P.polseq = rec.polseq
and P.vehnum = rec.vehnum
and P.linecvg = rec.linecvg
and P.sat = 'ABC';

END LOOP;
COMMIT;

EXCEPTION
WHEN OTHERS THEN
NULL;

END UPDT_TAB;

Above procedure tables many hours to run, to make it run faster, I used bulk collect and forall and limit(so I don't run out of PGA memory) to rewrite the code:

PROCEDURE updt_tab

IS
CURSOR table1_cur
IS
SELECT *
from TABLE1 V
left outer join TABLE2 S
on V.polnum = S.s7poln
AND V.polren = S.s7renn
AND V.polseq = S.s7seqn
AND V.vehnum = S.s7vehn
AND V.linecvg = S.s7lcvg;

TYPE table1_collect is TABLE OF table1_cur%ROWTYPE
INDEX BY PLS_INTEGER;
table1_col table1_collect;

BEGIN
OPEN table1_cur
LOOP
FETCH table1_cur
BULK COLLECT INTO table1_col LIMIT 500;

FORALL indx IN 1 .. table1_col.COUNT
LOOP
update table1 P
set P.CEDEDPOL = rec.s7ceding
where P.polnum = rec.polnum
and P.polren = rec.polren
and P.polseq = rec.polseq
and P.vehnum = rec.vehnum
and P.linecvg = rec.linecvg
and P.sat = 'ABC';
update table1 P
set P.CEDEDCVG = rec.s7cedecvg
where P.polnum = rec.polnum
and P.polren = rec.polren
and P.polseq = rec.polseq
and P.vehnum = rec.vehnum
and P.linecvg = rec.linecvg
and P.sat = 'ABC';

END LOOP;
EXIT WHEN table1_col.COUNT < 500;

END LOOP;
CLOSE table1_cur;
COMMIT;
EXCEPTION
WHEN OTHERS THEN
NULL;

END UPDT_TAB;

Do you think the rewritten code will perform much faster? I have yet to test it once our testing environment becomes available. Any issues that I might run into? Your expert insights is very much appreciated.

Thank you.

JS

and Tom said...

I know how to make the original procedure run MUCH faster!!!! A lot faster!!

Just code it as follows:

PROCEDURE updt_tab
<b>/*</b>
FOR rec IN (SELECT *
                from TABLE1 V
                left outer join TABLE2 S
                on V.polnum = S.s7poln
                AND V.polren = S.s7renn
                AND V.polseq = S.s7seqn
                AND V.vehnum = S.s7vehn
                AND V.linecvg = S.s7lcvg) LOOP
                               
update TABLE1 P
      set P.CEDEDPOL = rec.s7ceding
      where P.polnum = rec.polnum
      and P.polren = rec.polren
      and P.polseq = rec.polseq
      and P.vehnum = rec.vehnum
      and P.linecvg = rec.linecvg
      and P.sat = 'ABC';

update TABLE1 P
      set P.CEDEDCVG = rec.s7cedecvg
      where P.polnum = rec.polnum
      and P.polren = rec.polren
      and P.polseq = rec.polseq
      and P.vehnum = rec.vehnum
      and P.linecvg = rec.linecvg
      and P.sat = 'ABC';
   
END LOOP;
COMMIT;<b>
*/  -- we don't need that code at all!!!!!
  NULL;</b>
  EXCEPTION
  WHEN OTHERS THEN
  NULL;

END UPDT_TAB; 



I am not joking.

I am not kidding.

I am not being sarcastic.

I am being 100% honest in my tuning effort.


For you see, with that original code - the follow outcomes are all perfectly acceptable:

a) all rows are read, all rows are updated in both tables and we commit. and then we return.

b) NO rows are read - the first row caused a failure, we do not commit. and then we return

c) 5,000,000 rows are read - this executes 10,000,000 updates, we fail on reading row 5,000,001, we do not commit. and then we return.


Since (a) is acceptable AND (b) is acceptable and (c) is acceptable - I'll just go with (b) (it is by far the most performant!!!) and you are tuned *beyond belief*.

Pretty amazing how easy tuning is - isn't it??? You can go from hours to just milliseconds just like that...

Here read this:
http://www.drmaciver.com/2010/04/the-best-way-to-handle-exceptions/?

I just read it today - your exception handling code makes me feel very afraid for you. I usually just say:

http://asktom.oracle.com/pls/asktom/asktom.search?p_string=%22i+hate+your+code%22

"i hate your code", but I was feeling verbose today...


Your new code will not compile.
Why are there two updates???????? you are updating the same record(s), just different columns???? doesn't anyone learn SQL anymore?


There is no reason this code should not be:

merge into table1 p
using (SELECT *
  from TABLE1 V
  left outer join TABLE2 S
  on V.polnum = S.s7poln
  AND V.polren = S.s7renn
  AND V.polseq = S.s7seqn
  AND V.vehnum = S.s7vehn
  AND V.linecvg = S.s7lcvg) rec
on (P.polnum = rec.polnum
      and P.polren = rec.polren
      and P.polseq = rec.polseq
      and P.vehnum = rec.vehnum
      and P.linecvg = rec.linecvg
      and P.sat = 'ABC')
when matched then update set P.CEDEDPOL = rec.s7ceding, P.CEDEDCVG = rec.s7cedecvg ;


there should be NO procedural code

there should be NO commit (the client should control that, not a little procedure, it is not smart enough to know "we are truly done"

there should definitely be no "when others then null"

Rating

  (3 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

A reader, April 27, 2010 - 3:16 pm UTC

Thank you, Tom! That merge statement is very efficient and simple. Unfortunately we have many similar PL/SQL procedures and packages that run for days like my original posting, so you'll see more questions from me...

Thanks again!

JS
Tom Kyte
April 28, 2010 - 7:25 am UTC

The answer to all of your questions from me will be:

remove the procedural code, replace it with a single sql statment.

over and over and over again.

And comment all when others handlers you have, they are seriously evil and are a BUG waiting to happen - I'm not kidding even a little bit on that - your when others, GET RID OF THEM - first, today, right now.

FORALL ?

v.hariharaputhran, April 28, 2010 - 12:18 pm UTC

Looks like trying even without compiling the code, Will this work by any chance

FORALL indx IN 1 .. table1_col.COUNT
LOOP
update table1 P
set P.CEDEDPOL = rec.s7ceding
where P.polnum = rec.polnum
and P.polren = rec.polren
and P.polseq = rec.polseq
and P.vehnum = rec.vehnum
and P.linecvg = rec.linecvg
and P.sat = 'ABC';
update table1 P
set P.CEDEDCVG = rec.s7cedecvg
where P.polnum = rec.polnum
and P.polren = rec.polren
and P.polseq = rec.polseq
and P.vehnum = rec.vehnum
and P.linecvg = rec.linecvg
and P.sat = 'ABC';

Having more than one DML under a FORALL,

Regards
V.Hari
Tom Kyte
April 28, 2010 - 12:55 pm UTC

no, it won't, that is why I wrote:

Your new code will not compile.

FORALL

V.Hariharaputhran, April 28, 2010 - 1:45 pm UTC

Definetly not my code :), just commented on the real post.

Really appreciate your patience and passion Mr.oracle.

Regards
V.Hari

More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library