Skip to Main Content

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Jon.

Asked: January 21, 2002 - 3:15 pm UTC

Last updated: June 30, 2006 - 12:48 pm UTC

Version: 8.1.6

Viewed 1000+ times

You Asked

We generate packages to do standard data access to our tables. My question relates to the UPDATE part of the data access packages (the generated package updates by primary key only).

The invoker of the procedure passes in a "map" record and a "change" record. The "map" record consists of one boolean field for each column on the table. It is set to FALSE if no update is to occur, and TRUE if the "change" record contains a value which needs to be reset on the row.

Up until now, we have used these to build a dynamic SQL statement which is then executed. But this means each SQL statement is new and reparsing, etc. must occur. The alternative would be to retrieve the record from the database (remembering this is by PK, so it is fast) and build a SQL statement that would contain a SET statement for EVERY column, except the primary key column(s). Columns which are to change (known from the "map" record) would be set to the new values, and columns which are not changed would be set to equal the value on the existing record. This alternative was challenged on the basis that even if the columns are being set to their current value, there could be a serious impact on the indexes, including potentially severe fragmenting if there are a relatively large number of indexes on the table.

I would like to take advantage of bind variables (read your book and got your message!) which we could achieve using the alternative update, but I have to know if this index concern is real or not.

Ignore the overhead of going and getting the record by PK, as we do that in either case, for another reason.

and Tom said...

Well, in addition to the index issue (which is true) there is the issue that your REDO generation will go way up as well.

You are between the proverbial rock and hard place.

Now, I would say that you should be able to use bind variables here regardless!!!

You will have a finite number of statements -- say a table has an ID and columns C1, C2, C3. the only updates you should EVER generate are:

update t set c1 = :c1 where id = :id;update t set c1 = :c2 where id = :id;update t set c1 = :c3 where id = :id;update t set c1 = :c1, c2 = :c2 where id = :id;
update t set c1 = :c1, c3 = :c3 where id = :id;
update t set c2 = :c2, c3 = :c3 where id = :id;
update t set c1 = :c1, c2 = :c2, c3 = :c3 where id = :id;

You have a finite number SQL statements that will be generated for ANY table ever (thats the important point here!) Oracle Forms can take this approach for example -- you can either update all columns (default) or just the ones that changed (then it constructs the proper SQL like above)

It would be NICE if you could avoid the softparse, but if you set session_cached_cursors to some good number, you'll avoid alot of that overhead as well.

So, use bind variables!

Make sure to generate the update in a predicable fashion (eg: never "set c1 = :c1, c2 = :c2" and later "set c2 = :c2, c1 = :c1" - do the columns in the same order)

use session_cached_cursors



Rating

  (10 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

How will the above lead to index fragmentation, can you please explain briefly

Kulguru, January 22, 2002 - 9:36 am UTC

How will the above lead to index fragmentation, can you please explain briefly.

Tom Kyte
January 22, 2002 - 2:27 pm UTC

well look at that.

testing is showing that the index update is ignored -- its not doing it (it optimized it right out of there). another case of conventional wisdom out the window....

So, that (the index) is not a concern -- but the row data in the table, that is, it'll generate gobs of extra redo.

scott@ORA817DEV.US.ORACLE.COM> update emp set empno = empno;

14 rows updated.


Statistics
----------------------------------------------------------
2 recursive calls
20 db block gets
2 consistent gets
0 physical reads
3648 redo size
830 bytes sent via SQL*Net to client
554 bytes received via SQL*Net from client
3 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
14 rows processed

scott@ORA817DEV.US.ORACLE.COM> update emp set empno = empno, ename = ename, job = job, mgr = mgr, hiredate = hiredate, sal = sal, comm = comm, deptno = deptno;

14 rows updated.


Statistics
----------------------------------------------------------
0 recursive calls
18 db block gets
1 consistent gets
0 physical reads
5104 redo size
845 bytes sent via SQL*Net to client
653 bytes received via SQL*Net from client
3 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
14 rows processed

scott@ORA817DEV.US.ORACLE.COM>

So -- nix the index stuff (thanks for making me really look at it) but remember the extra redo generated.

You were clear but the solution is not helpful

Jon, January 22, 2002 - 12:15 pm UTC

This is viable for a table with a very small number of columns but not otherwise. If a table has 2 columns plus a PK, the solutions are (read "1" as "set c1 = :1"; read "12" as "set c1 = :1, c2 = :2", etc.):

1
2
12

For 3 columns,
1
2
3
12
13
23
123

For 4 columns,
1
2
3
4
12
13
14
23
24
34
123
124
134
234
1234

So for "n" columns, we have (2 to the nth) - 1 statements.

If we have 39 columns, we have 5.498E+11 Statements!! Are we stuck between the rock and the hard place? It seems like it.

Tom Kyte
January 22, 2002 - 12:48 pm UTC

Yes, if you want to selectively update the columns -- tradeoff -- DISK vs CPU+RAM.

Me, I would probably opt for the disk hit personally.

One other comment on this. Will your end users really generate 5.4E11 STATEMENTS (i think not). there will be some small working set that you will generate, it'll be small, it'll be finite. As long as you make the sql in a predicable fashion -- using bind variables -- you'll be just fine.

Mio, January 22, 2002 - 1:11 pm UTC

How about

update t
set
c1 = decode(:changed1, 1, :c1, c1),
c2 = decode(:changed2, 1, :C2, C2)
...
/
?

Tom Kyte
January 22, 2002 - 2:32 pm UTC

that will have the effect of updating every column -- greatly increasing the REDO. Yes, you can do it -- but be aware of the consequences.

Well, yes, but remember this is code generation

Jon, January 22, 2002 - 3:35 pm UTC

So while the users may only typically update a relatively small portion of the columns, how does one extract which those are from the data dictionary?

By the way, thanks for checking again on the index update/fragmentation. That was in dispute and was the crux of my question. But of course, now we have the REDO issue. So in your opinion, if you had the choice of building the UPDATE statement and executing it dynamically, or updating all the columns and incurring the REDO, which would you do?

Tom Kyte
January 22, 2002 - 6:04 pm UTC

Well, you don't have to get it from the data dictionary -- you don't have to PRE-GENERATE these, just generate them in a predicable fashion.

You say you pass a map record and change record and build a sql statement based on that. That is fine, you should do something like this (psuedo-ish code, won't necessarily compile)


create package body dyn_update_table_t
as
g_cursor number := dbms_sql.open_cursor;

procedure do_udpate( map_rec in ..., changes in ... )
is
l_stmt long default 'update t set ';

procedure add_set( map_field in number, cname in varchar2 )
is
begin
if ( map_field = 1 ) then
l_stmt := l_stmt || field_name || '= :' || field_name ||',';
end if;
end;
procedure do_bind( map_field in number, cname in varchar2, val in number )
is
begin
if ( map_field = 1 ) then
dbms_sql.bind_variable( ... );
end if;
end;

procedure do_bind( map_field in number, cname in varchar2, val in date )
is
begin
if ( map_field = 1 ) then
dbms_sql.bind_variable( ... );
end if;
end;
procedure do_bind( map_field in number, cname in varchar2, val in varchar2 )
is
begin
if ( map_field = 1 ) then
dbms_sql.bind_variable( ... );
end if;
end;
begin
add_set( map_rec.c1, 'c1' );
add_set( map_rec.c2, 'c2' );
...
add_set( map_rec.cN, 'cN' );

l_stmt := rtrim(l_stmt,',') || ' where pk = :id';
dbms_sql.parse( g_cursor, l_stmt );
add_bind( map_rec.c1, 'c1', changes.c1 );
add_bind( map_rec.c2, 'c2', changes.c2 );
...
add_bind( map_rec.cN, 'cN', changes.cN );

dbms_sql.execute( g_cursor );
end;

end;
/

You won't avoid the soft parse, but you'll use binds, the sqlstmts are generated in a predicable fashion (they will be reused if two people update the same set of columns) and by using session_cached_cursors, you'll avoid much of the soft parse overhead.

Now, if I had the CHOICE, and the amount of redo generated was acceptable -- I would use the static sql suggested by someone else:

update t
set
c1 = decode(map_rec.c1, 1, changes.c1, c1),
c2 = decode(map_rec.c2, 1, changes.c2, C2)
...;



only you can answer 'is the amount of extra redo acceptable'.



Thanks

A reader, January 23, 2002 - 10:00 am UTC

Thanks, Tom. I was hoping to avoid the soft parses, too, but as that isn't practical without incurring the REDO cost, you've offered a very reasonable alternative.

As always, you've been clear, thorough and PATIENT!!

Alternative - better, worse or no difference?

Jon, February 25, 2002 - 8:41 am UTC

Tom,

Could you comment on this alternative suggested by Chris Weiss of PureCarbon and print in Quest-Pipelines newsletter?
</code> http://www.quest-pipelines.com/newsletter-v3/0202_D.htm <code>

And welcome back - we missed you!

Tom Kyte
February 25, 2002 - 10:30 am UTC

It is not any different from what i said:

You will have a finite number of statements -- say a table has an ID and columns
C1, C2, C3. the only updates you should EVER generate are:

update t set c1 = :c1 where id = :id;
update t set c1 = :c2 where id = :id;
update t set c1 = :c3 where id = :id;
update t set c1 = :c1, c2 = :c2 where id = :id;
update t set c1 = :c1, c3 = :c3 where id = :id;
update t set c2 = :c2, c3 = :c3 where id = :id;
update t set c1 = :c1, c2 = :c2, c3 = :c3 where id = :id;


In fact, it is just an IMPLEMENTATION of what I said. It is not an alternative, its the same thing!



I guess you've answered my question; I saw if as different

Jon, February 25, 2002 - 11:13 am UTC

I guess I saw his building all the potential bind variables in a declare in l_sql_string and then executing:

EXECUTE IMMEDIATE l_sql_string
USING p_first_name, p_last_name, p_middle_name, p_dob, p_termination, p_id

as being different than building just the update statement in l_stmt and doing

dbms_sql.parse( g_cursor, l_stmt );
add_bind( map_rec.c1, 'c1', changes.c1 );
add_bind( map_rec.c2, 'c2', changes.c2 );
...
add_bind( map_rec.cN, 'cN', changes.cN );

dbms_sql.execute( g_cursor );


If you're saying there's no effective difference, I guess that answers my question. Thanks.

Tom Kyte
February 25, 2002 - 2:09 pm UTC

It is "different" -- it's a different implementation of the same thing. Another way of doing it -- there must be hundreds of ways to do the same thing.

There is a certain elagance to their approach, a certain readability to the one I laid out. Cursory testing shows a very slight performance edge in the "elegant" approach.

What about this approach?

Jack Mulhollan, September 01, 2004 - 7:30 pm UTC

Tom,

What about this approach? Over time you could put the elsifs in the order of the most frequently occuring column combinations. Also, you could have one update procedure per table, regardless of which columns the application made available for update on various screens. Is this viable? What are the drawbacks? How many columns would you consider too many for this approach? Thanks.

- Jack


procedure update_row
(
p_contact_id in int,
p_bitmap in varchar2,
p_name in varchar2,
p_address in varchar2
)
is
begin

if p_bitmap = 'YY' then
update temp_02_tb
set name = p_name, address = p_address
where contact_id = p_contact_id;
commit;
elsif p_bitmap = 'YN' then
update temp_02_tb
set name = p_name
where contact_id = p_contact_id;
commit;
elsif p_bitmap = 'NY' then
update temp_02_tb
set address = p_address
where contact_id = p_contact_id;
commit;
end if;

end update_row;



Tom Kyte
September 01, 2004 - 9:01 pm UTC

except for the "commit", it would be fine for a small set of columns (quickly explodes into lots of elseif)

CASE would be used in 9i...

(the commit comment is because I firmly believe commit and rollback should have been left out of the plsql language -- it would be a limitation about 0.1% of the times but prevent thousands of bugs in the meantime...)

Then how do I commit?

Jack Mulhollan, September 02, 2004 - 5:10 pm UTC

Tom,

Help me understand why not to include the commit. I assume a user executes this procedure from within an application (clicks an update button). The application may be a client-server application that hold a connection to the database (such as in my SQL*Plus example below), or it may be a stateless application (like a web app) that gets a connection just long enough to execute the procedure. If the commit is left out of the procedure, what happens to the update in each case? How and when do I commit the update?

- Jack

execute contacts_pkg.update_row(3,'NY',null,'123 Main Street');


Tom Kyte
September 02, 2004 - 9:44 pm UTC

Ok, imagine the future where the application takes your nice routine to update this row AND an update to some *other row* - but you made your update_row "atomic" all by itself.

Oh no, i cannot reuse your code, because YOUR CODE had the audacity to say "i'm it, i'm the king, you call me, I commit -- anything you do after me is someone elses problem, don't even THINK about reusing me cause I'll mess your transaction up big time"


To me - the CLIENT application (that stateless thing in the middle tier) is the ONLY thing on this planet that *knows*

a) when the transaction is complete
b) what is ok to fail and what is ok to succeed

you, the coder of this little tiny subroutine, have no idea what the requirements will be in 3 years from now.

Maybe I need to:

a) update this address row
b) and modify that audit trail
c) and update this queue to propagate the change back to the mainframe

but if you commit in (a), well, forget about using your code, you are the most unsafe thing possible.


the only thing -- repeat ONLY THING -- that knows "commit should happen here" is the client invoking you. They might be stateless - but they do most certainly have the ability to execute as many of you as they want at a time - in a transaction.



Commit/Rollback in stored procedure

JHT, June 30, 2006 - 11:26 am UTC

Tom,

I agree with your comment with not using commit and rollback in a stored procedure because the stored procedure should not infer what the calling program has done previous to calling the stored procedure.

However, when I have a stored procedure that executes several different SQL updates/inserts/deletes, I like to create a savepoint at the very beginning of my stored procedure and "rollback to that savepoint" if an error occurs during one of the steps. Basically, I believe my stored proc should only undo what the stored proc has done if an error occurs. Your thoughts on this? Thanks.

Tom Kyte
June 30, 2006 - 12:48 pm UTC

my thought is - 

Oracle already does this, it is the way procedures work.


ops$tkyte@ORA10GR2> create table t ( x int check (x>0) );
 
Table created.
 
ops$tkyte@ORA10GR2>
ops$tkyte@ORA10GR2> create or replace procedure p
  2  as
  3  begin
  4          insert into t values ( 1 );
  5          insert into t values ( 2 );
  6          insert into t values ( 3 );
  7          dbms_output.put_line( 'I have inserted 3 rows...' );
  8          insert into t values ( 0 );
  9  end;
 10  /
 
Procedure created.
 
ops$tkyte@ORA10GR2>
ops$tkyte@ORA10GR2> insert into t values ( 100 );
 
1 row created.
 
ops$tkyte@ORA10GR2> select * from t;
 
         X
----------
       100
 
ops$tkyte@ORA10GR2> exec p;
I have inserted 3 rows...
BEGIN p; END;
 
*
ERROR at line 1:
ORA-02290: check constraint (OPS$TKYTE.SYS_C0020965) violated
ORA-06512: at "OPS$TKYTE.P", line 8
ORA-06512: at line 1
 
 
ops$tkyte@ORA10GR2> select * from t;
 
         X
----------
       100
 
ops$tkyte@ORA10GR2> rollback;
 
Rollback complete.
 
ops$tkyte@ORA10GR2> select * from t;
 
no rows selected
 
 

More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library