Skip to Main Content
  • Questions
  • performance issues using FOR UPDATE and WHERE CURRENT OF

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Sita.

Asked: January 21, 2009 - 1:49 pm UTC

Last updated: January 30, 2009 - 4:14 pm UTC

Version: 9.2.0

Viewed 1000+ times

You Asked

Hi Tom,
I have read your blog " http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:546822742166" which talks about commit exact fetch scenarios.
I have a requirement where in I need to update the same table on which my cursor is based. Please find the below mentioned pseudo code:

DECLARE
CURSOR cur_recs
IS
SELECT * FROM ra_interface_lines WHERE org_id = 81 FOR UPDATE OF term_id;
BEGIN
FOR c_recs IN cur_recs LOOP
validations...
UPDATE ra_interface_lines
SET term_id = l_term_id (calculated value during validations)
WHERE CURRENT OF cur_recs;
END LOOP;
EXCEPTION
WHEN OTHERS THEN
displaying the error message...
END;

I have not used commit and rollback in this piece of code. If lot of data has to be updated, will this code create any performance issue? If yes can you please suggest the ways to avoid those issues.

Awaiting for your reply.

Regards,
Sita

and Tom said...

if you absolutely, totally cannot do your validations in sql (try it, you might find you can...) then I'd prefer you to 'bulk up' the code

ops$tkyte%ORA9IR2> create table t as select * from all_users where rownum <= 11;

Table created.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> select * from t;

USERNAME                          USER_ID CREATED
------------------------------ ---------- ---------
B                                     107 09-DEC-08
DEMO                                   98 06-AUG-08
USER2                                  88 16-APR-08
USER1                                  87 16-APR-08
PERFSTAT                               86 21-DEC-07
OPS$TKYTE                             101 24-NOV-08
BIG_TABLE                              63 15-OCT-07
A                                     106 09-DEC-08
SCOTT                                  60 06-DEC-03
QS_CS                                  59 06-DEC-03
QS_CB                                  58 06-DEC-03

11 rows selected.

ops$tkyte%ORA9IR2> declare
  2      cursor c is select rowid rid, t.* from t for update;
  3
  4      type input_array is table of c%rowtype;
  5      type rid_array is table of rowid index by binary_integer;
  6      type vc_array is table of t.username%type index by binary_integer;
  7
  8      l_data      input_array;
  9      l_rids      rid_array;
 10      l_usernames vc_array;
 11  begin
 12      open c;
 13      loop
 14          fetch c bulk collect into l_data <b>limit 100;</b>
 15
 16          for i in 1 .. l_data.count
 17          loop
 18              l_rids(i) := l_data(i).rid;
 19              l_usernames(i) := lower(l_data(i).username); -- your validation
 20          end loop;<b>
 21          forall i in 1 .. l_data.count
 22              update t set username = l_usernames(i) where rowid = l_rids(i);
 23</b>
 24          exit when c%notfound;
 25      end loop;
 26          close c;
 27  end;
 28  /

PL/SQL procedure successfully completed.

ops$tkyte%ORA9IR2> select * from t;

USERNAME                          USER_ID CREATED
------------------------------ ---------- ---------
b                                     107 09-DEC-08
demo                                   98 06-AUG-08
user2                                  88 16-APR-08
user1                                  87 16-APR-08
perfstat                               86 21-DEC-07
ops$tkyte                             101 24-NOV-08
big_table                              63 15-OCT-07
a                                     106 09-DEC-08
scott                                  60 06-DEC-03
qs_cs                                  59 06-DEC-03
qs_cb                                  58 06-DEC-03

11 rows selected.


Rating

  (5 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Thanks for such an useful logic

Sita Yallapu, January 21, 2009 - 2:25 pm UTC

Hi,
Thanks lot. Let me try using this logic.
Thanks once again,
Sita

Review on FOR UPDATE

Dinesh, January 23, 2009 - 1:20 am UTC

Hi Tom,

I am ver much thankful for your idea of explaining the FOR UPDATE clause.

Need some more clarification

Sita Yallapu, January 27, 2009 - 4:48 am UTC

Hi Tom,

I have implemented the logic suggested by you and it works fine, while testing I got a doubt. Even in the logic suggested by you, I am updating the same table without closing rhe cursor on the table. Would not it create any performance issue or ora-01555 error? And do I need to use COMMIT explicitly in the code?

Thanks,
Sita

Tom Kyte
January 28, 2009 - 8:36 am UTC

committing in the loop would be a reason for causing an ora-1555!!!!


if you commit in the for loop, you can expect:

a) increased chance of 1555
b) increased redo log generation
c) increased undo generation
d) longer processing times
e) possibly corrupt data since when you fail (WHEN, not if, your code will fail some day) you leave some of the data processed, some of the data NOT processed, and no clue as to which was processed and which was not.


You can fix (e) by write a ton more code - so you can restart your code after it fails safely, but you will still have a, b, c, and d as fact.

Increased chance of ora-1555

Kunal, January 30, 2009 - 1:41 am UTC

Hi Tom,

I was reading this thread and was unable to figure out the reason for "Increased chance of ora-1555" as you have mentioned.
I know the concept of delayed clean out but some how unable to relate that to "Increased chance of ora-1555".
Can you please explain?
Tom Kyte
January 30, 2009 - 2:58 pm UTC

I will assume you mean:


...
if you commit in the for loop, you can expect:

a) increased chance of 1555
...


a typical loop looks like this:

for x in (select * from t where some_condition )
loop
   do some process
   update t set something = something_else where pk = x.pk;
   commit;
end loop;



Ok, so the query "select * from t where some_condition" must be done read consistently. It cannot see ANY change that was not committed when it began. It cannot see the changes you make in the loop, nor can it see changes made by other transactions.

Any time it hits a block that contains a change - a change introduced or uncommitted since it began processing - it will use UNDO to rollback that change.

So, let's say "where some_condition" uses an index to read the table. The first row in your result set is on block 101 in the table (the index tells us that). Unfortunately for you - the last row in your result set is on block 101 as well (rows are sorted in the index, not so in the table). In between your first row and last row are 10,000 other rows you will process.

You update the first row, generate undo for it and commit.

When you commit, you know what you do? You make the undo you generate available for reuse.

Over the course of the next 10,000 updates - we decide to reuse, overwrite, blow away, make disappeared the undo you generated for block 101. We can, you told us we could by committing.

So, when we get to the last record - we find the block has changed - we need the undo but the undo no longer exists.

Bamm ora-1555

Worse: all of the rows EXCEPT for the last row have been updated.

But you don't know that - you just know "we got an error", you would need to write A LOT more code to make your process restartable - it isn't as I coded it above, what you have now is not only an ora-1555, but a mess in your database.

If you want to commit in the cursor for loop you have to

a) expect increased odds of 1555 - whether or not you are updating the table you are reading - heck we can get ora-1555 on read only tablespaces if you just commit too darn frequently:
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:895410916429

b) expect to write at least 10 times as much code, because you have to - you MUST - figure out a way to make it so that when your code fails (when, not if) you can restart it. This typically involves some other table, extra columns, something to ensure you are restartable.


so, you can commit in a for loop, just be ready to handle a and b.

SAKS, January 30, 2009 - 8:27 am UTC

Hi Tom,

I am facing this issue with UPDATE FOR, while executing this Package it hangs, maybe because of FOR UPDATE, when I try to execute the below select statement from cursor along For Update it hangs. This package is actually called from the JOB which runs every days at 7:00 in system test environment.

CREATE OR REPLACE Package PFECODER.PFECODER_MAILER
IS

-- The array maillist must be defined as follows:
maillist pfecoder_generic_mailer.mailList;

ROCEDURE send_mail;


END; -- Package Specification pfecoder_mailer
/

CREATE OR REPLACE Package Body PFECODER.PFECODER_MAILER
IS
Procedure send_mail
is

v_directory varchar2(100);
v_mail_attachment varchar2(1);
v_dir_length integer default 0;
v_file_name varchar2(100);
v_SQLErrMsg varchar2(150);

NoFromEmailAddress Exception;
NoToEmailAddress Exception;
NoSubjectOrMessage Exception;
v_outmsg varchar2(2000) := null;

v_temp_str varchar2(2000) := null;
v_cntr number := 0;
v_start_pos number :=1;
v_end_pos number :=0;
v_clean_dur number :=30;

cursor c_rpt_cursor is select TO_ADDRESS,
FROM_ADDRESS,
SUBJECT,
file_or_text,
IS_FILE,
SEND_AS_ATTACHMENT,
ERROR_MSG
FROM pfecoder_email_queue
where sent_ts is null
for update;

BEGIN
for rpt in c_rpt_cursor loop
BEGIN

if rpt.TO_ADDRESS is null then
raise NoToEmailAddress;
elsif rpt.FROM_ADDRESS is null then
raise NoFromEmailAddress;
elsif rpt.SUBJECT is null and rpt.file_or_text is null then
raise NoSubjectOrMessage;
end if;

v_temp_str := rpt.TO_ADDRESS || ',';

loop
if v_temp_str is null then
exit;
end if;

v_end_pos := instr(v_temp_str,',');
v_cntr := v_cntr +1;
maillist(v_cntr) := rtrim(substr(v_temp_str,v_start_pos, v_end_pos-1));
v_temp_str := rtrim(substr(v_temp_str,v_end_pos+1, length(v_temp_str)));
end loop;
v_cntr := 0;
v_start_pos :=1;
v_end_pos :=0;
v_temp_str := null;

select instr(rpt.file_or_text, '/', -1, 1) into v_dir_length from dual;


if v_dir_length > 0 then
select substr(rpt.file_or_text,1,v_dir_length - 1) into v_directory from dual;
select substr(rpt.file_or_text, v_dir_length + 1) into v_file_name from dual;
end if;

pfecoder_generic_mailer.send_message
(rpt.FROM_ADDRESS,
maillist,
v_directory,
v_file_name,
rpt.subject,
'Y',
v_outmsg);
if rpt.error_msg is not null then
update pfecoder_email_queue
set error_msg = null where current of c_rpt_cursor;
end if;
else
v_directory := null;
pfecoder_generic_mailer.send_message
(rpt.FROM_ADDRESS,
maillist,
v_directory,
rpt.file_or_text,
rpt.subject,
v_outmsg);

if rpt.error_msg is not null then
update pfecoder_email_queue
set error_msg = null where current of c_rpt_cursor;
end if;
end if;

maillist.delete;

update pfecoder_email_queue
set sent_ts = sysdate,
error_msg = v_outmsg
where current of c_rpt_cursor;

begin
select p_value into v_clean_dur
from pfecoder_parameters
where p_key = 'CLEAN_EMAIL_LOG' and
rownum < 2;

exception
when others then
dbms_output.put_line(SQLERRM);
end;

begin
delete from pfecoder_email_queue
where sent_ts < sysdate-v_clean_dur;
exception
when others then
dbms_output.put_line(SQLERRM);
end;

EXCEPTION

WHEN UTL_TCP.NETWORK_ERROR THEN
v_SQLErrMsg := SUBSTR(SQLERRM, 1, 150);
update pfecoder_email_queue
set ERROR_MSG = 'Email error :' || v_SQLErrMsg||', Network Error'
where current of c_rpt_cursor;

WHEN NoFromEmailAddress THEN
v_SQLErrMsg := SUBSTR(SQLERRM, 1, 150);

update pfecoder_email_queue
set ERROR_MSG = 'Email error :' || v_SQLErrMsg||', No From email address'
where current of c_rpt_cursor;

WHEN NoSubjectOrMessage THEN
v_SQLErrMsg := SUBSTR(SQLERRM, 1, 150);
update pfecoder_email_queue
set ERROR_MSG = 'Email error :' || v_SQLErrMsg||', No Subject and no Message'
where current of c_rpt_cursor;

WHEN NoToEmailAddress THEN
v_SQLErrMsg := SUBSTR(SQLERRM, 1, 150);
update pfecoder_email_queue
set ERROR_MSG = 'Email error :' || v_SQLErrMsg||', No To email address'
where current of c_rpt_cursor;

WHEN OTHERS THEN
v_SQLErrMsg := SUBSTR(SQLERRM, 1, 150);
update pfecoder_email_queue
set ERROR_MSG = 'Email error :' || v_SQLErrMsg
where current of c_rpt_cursor;
END;

end loop;

commit;
END;

END; -- Package Body pfecoder_mailer
/

I am stuck with this issue since few days. Thanks for your help in advance.
Tom Kyte
January 30, 2009 - 4:14 pm UTC

 exception
 when others then
    dbms_output.put_line(SQLERRM);
 end;


I hate your code

http://asktom.oracle.com/pls/ask/search?p_string=%22i+hate+your+code%22

start over, rewrite this entire procedure - I'm dead serious, the error handling is actually error INDUCING.


why would you do this?
select substr(rpt.file_or_text,1,v_dir_length - 1) into 
v_directory from dual;


instead of

  v_directory := substr(rpt.file_or_text,1,v_dir_length - 1);


Ok, way too many things wrong here - no to mention that this is "production code" and yet, here I am, looking up the definition of instr to be sure I can reverse engineer it (eg: comments? logic flow?)


And it is really long - why no procedures/functions in the package to make it modular and understandable?

There are logic issues, I don't even know what to say about, consider that set error msg to null in one update and then follow it by setting to v_outmsg right after???

Please don't tell me you have a column name "P_KEY" - given that you also have a coding convention of p_ for parameter names????? sigh, p_value too - not too good of an idea... You'll ALWAYS WANT TO USE A CORRELATION NAME AND DOT NOTATION WITH THAT, ALWAYS.

This is stunning:
  begin
   select p_value into v_clean_dur
   from pfecoder_parameters
   where p_key = 'CLEAN_EMAIL_LOG' and
         rownum < 2;

 exception
 when others then
    dbms_output.put_line(SQLERRM);
 end;


one would presume that p_key is a primary key, so one wonders about the rownum clause and one really wonders about that truly bad when others :(


You will want to ensure you have an index that starts with SENT_TS and has at least one attribute after it that is NOT NULL. For example, an index like:

create index sent_ts_idx on pfecoder_mail_queue(sent_ts,0)

would be useful - so the "where sent_ts is null" can do a full scan efficiently (this by the way is probably your 'hang', it isn't a hang, you are waiting for the full scan of this probably large table with lots of legacy records in it to finish....) see
http://asktom.oracle.com/Misc/something-about-nothing.html
for details on what I'm talking about



I'm going to do something I don't often do, I'm going to rewrite this from scratch as an example...




<b>now, I had to stub out this package - just to compile the real package..</b>

ops$tkyte%ORA10GR2> create or replace package pfecoder_generic_mailer
  2  as
  3      type mailList is table of varchar2(200) index by binary_integer;
  4
  5      procedure send_message
  6                  (p_from in varchar2, p_maillist in mailList, p_dir in varchar2,
  7                   p_file in varchar2, p_subject in varchar2, p_flag in varchar2,
  8                   p_outmsg out varchar2);
  9
 10  end;
 11  /

Package created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> CREATE OR REPLACE Package PFECODER_MAILER
  2  IS<b>
  3      -- rule #1
  4      -- never use global variables unless and until you have to
  5      -- rule #2
  6      -- you never "have" to
  7      -- therefore, that nasty awkward global you had is history...
  8</b>
  9      -- this is the main and only entry point to this package, it
 10      -- is designed to be called by blah blah blah...... and so on
 11      -- basically, it
 12      -- a) flags as an error all emails in the queue without a to/from or body
 13      -- b) processes all emails in the queue with a sent_ts is null
 14      -- c) cleans out old emails based on the parameter CLEAN_EMAIL_LOG
 15      --    in the parameters table
 16      procedure send_mail;
 17  END;
 18  /

Package created.

<b>Note in the following how each routine is small, bit sized, and pretty clear as to precisely what it does.  They each fit on a screen.  This is really important, especially if you ever want someone to look at your code, you have to make it easy for our brains to understand.  It took me way (far far far) too long to figure out what the heck was happening in this code.</b>

ops$tkyte%ORA10GR2> CREATE OR REPLACE Package Body PFECODER_MAILER
  2  IS
  3
  4  -- internal routine, probably actually belongs in the pfecoder_generic_mailer
  5  -- package.  Turns a string into a mailList
  6
  7  function string_to_table( p_str in varchar2 ) return pfecoder_generic_mailer.mailList
  8  is
  9      l_str   long default p_str || ',';
 10      l_n     number;
 11      l_data  pfecoder_generic_mailer.mailList;
 12  begin
 13      loop
 14          l_n := instr( l_str, ',' );
 15          exit when (nvl(l_n,0) = 0);
 16          l_data( l_data.count+1 ) := ltrim(rtrim(substr(l_str,1,l_n-1)));
 17          l_str := substr( l_str, l_n+1 );
 18      end loop;
 19      return l_data;
 20  end;
 21
 22
 23  -- routine to take an input string, find the last '/' in it
 24  -- if the '/' exists - then break into DIR and FILENAME
 25  -- else, set DIR to null and FILENAME to string
 26
 27  procedure split_into_dir_and_file
 28  ( p_str in varchar2,
 29    p_dir out varchar2,
 30    p_file out varchar2 )
 31  is
 32      l_n number := instr( p_str, '/', -1, 1 );
 33  begin
 34      if ( l_n > 0 )
 35      then
 36          p_dir := substr( p_str, 1, l_n-1 );
 37          p_file := substr( p_str, l_n+1 );
 38      else
 39          p_dir := null;  -- not necessary, it is null, but more clear
 40          p_file := p_str;
 41      end if;
 42  end;
 43
 44  -- based on the setting int he parameters table
 45  -- remove all old emails...
 46
 47  procedure clean_up_old_emails
 48  is
 49  begin
 50      delete from pfecoder_email_queue
 51       where sent_ts < (select sysdate-pfecoder_parameters.p_value
 52                          from pfecoder_parameters
 53                         where pfecoder_parameters.p_key = 'CLEAN_EMAIL_LOG');
 54  end;
 55
 56  -- instead of procedural row by row code, let us just avoid hitting emails
 57  -- we know we cannot send.  Flag them as errors.  We will not process them in the
 58  -- main body below - the main body will take our where clause and negate it, what
 59  -- we flag - they skip!
 60
 61  procedure flag_as_errors_bad_emails
 62  is
 63  begin
 64      update pfecoder_email_queue
 65         set error_msg = case when to_address is null
 66                              then 'Email error : NoToEmailAdress, No From email address'
 67                              when from_address is null
 68                              then 'Email error : No From Address'
 69                              when subject is null and file_or_text is null
 70                              then 'Email error : No Subject and no Message'
 71                          end
 72       where sent_ts is null
 73         and ((to_address is null) or (from_address is null) or (subject is null and file_or_text is null));
 74  end;
 75
 76
 77  Procedure send_mail
 78  is
 79
 80  v_SQLErrMsg           varchar2(150);
 81  v_file_name           varchar2(100);
 82  v_directory           varchar2(100);
 83  v_mailList            pfecoder_generic_mailer.mailList;
 84  v_outmsg              varchar2(4000);
 85
 86  -- notice we only select records we have a chance of actually
 87  -- processing, the and NOT () takes care of that for us.
 88
 89  cursor c_rpt_cursor is select TO_ADDRESS,
 90                                FROM_ADDRESS,
 91                                SUBJECT,
 92                                file_or_text,
 93                                IS_FILE,
 94                                SEND_AS_ATTACHMENT,
 95                                ERROR_MSG
 96                           FROM pfecoder_email_queue
 97                          where sent_ts is null
 98                            and NOT ((to_address is null) or
 99                                     (from_address is null) or
100                                     (subject is null and file_or_text is null))
101                            for update;
102
103  BEGIN
104      flag_as_errors_bad_emails;
105
106      for rpt in c_rpt_cursor
107      loop
108          v_mailList := string_to_table( rpt.to_address );
109          split_into_dir_and_file( rpt.file_or_text, v_directory, v_file_name );
110
111          begin
112              pfecoder_generic_mailer.send_message
113                 (rpt.FROM_ADDRESS, v_maillist, v_directory,
114                  v_file_name, rpt.subject, 'Y', v_outmsg);
115
116              update pfecoder_email_queue
117                 set sent_ts = sysdate,
118                     error_msg = v_outmsg
119                  where current of c_rpt_cursor;
120          exception
121                 when utl_tcp.network_error then
122                     v_SQLErrMsg := SUBSTR(SQLERRM, 1, 150);
123                     update pfecoder_email_queue
124                        set ERROR_MSG = 'Email error :' || v_SQLErrMsg||', Network Error'
125                      where current of c_rpt_cursor;
126          end;
127      end loop;
128
129      clean_up_old_emails;
130  end send_mail;
131
132
133
134  END; -- Package Body pfecoder_mailer
135  /

Package body created.



Now, if it was my code, I would have added calls to dbms_applicaiton.set_session_longops (search for that please) to the code in order to be able to monitor it's progress from another session.....

Also, I left the commit out on purpose, really. Only the client knows if this should be committed or not.

More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library