Skip to Main Content

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question.

Asked: February 01, 2007 - 11:22 am UTC

Last updated: August 22, 2007 - 2:16 pm UTC

Version: 10.2.0.1.0

Viewed 1000+ times

You Asked

Hi Tom

I recently came across a dotnet application (with IIS app server and against 10g database) where the developers had used optimistic locking (get a version id from table, update the required row and then update this version id as well) to claim that this was the only way they could make sure that same record was not updated by many users at the same time. I believe that this can be done using FOR UPDATE clause rather than writing unnecessary code to create artificial locking. Am I right in saying that?

Please help me in understanding it further.

Session A:


select salary from employee where employee_id = 1000 for update;

At this point,

Session B: select salary from employee where employee_id = 1000 for update;

What will happen now? Will Session B have to wait until Session A completes the transaction? Or will Session B get an exception saying that it has been locked by Session A?

Another scenario:

Session A : selects a record FOR UPDATE and leaves the desk for some time while

Session B: selects the same record.

Session A completes the transaction. So will Session B have to wait all this time to get an error message?

Thanks as always

and Tom said...

there are two ways n-tier applications access the database:

a) stateful. They hold a connection for a long period of time - in fact over many "web pages"

b) stateless. They hold a connection for a very very short period of time, maybe less than the time it takes to even generate a single html page.

Most applications today use b) in my experience. In this fashion, an end user is consuming resources only when they are actually 'active' in the database. They grab a connection, use it and release it.

If you have a stateless connection to the database - you cannot use pessimistic (for update) locking. You lose the lock, optimistic is the only way to go.

So, they are probably doing the right thing.


In my last book - Expert Oracle Database Architecture - I wrote about this extensively - a small excerpt:



.....
Optimistic or Pessimistic Locking?

So which method is best? In my experience, pessimistic locking works very well in Oracle (but perhaps not in other databases) and has many advantages over optimistic locking. However, it requires a stateful connection to the database, like a client/server connection. This is because locks are not held across connections. This single fact makes pessimistic locking unrealistic in many cases today. In the past, with client/server applications and a couple dozen or hundred users, it would have been my first and only choice. Today, however, optimistic concurrency control is what I would recommend for most applications. Having a connection for the entire duration of a transaction is just too high a price to pay.

Of the methods available:

There are many methods of implementing optimistic concurrency control. We¿ve discussed one whereby the application will store all of the before images of the row in the application itself. In the following sections, we¿ll explore three others, namely
* Using a special column that is maintained by a database trigger or application code to tell us the ¿version¿ of the record
* Using a checksum or hash that was computed using the original data
* Using the new Oracle 10g feature ORA_ROWSCN


which do I use? I tend to use the version column approach with a timestamp column. It gives me the extra information ¿when was this row last updated¿ in a long-term sense. So it adds value in that way. It is less computationally expensive than a hash or checksum, and it doesn¿t run into the issues potentially encountered with a hash or checksum when processing LONG, LONG RAW, CLOB, BLOB, and other very large columns.

If I had to add optimistic concurrency controls to a table that was still being used with a pessimistic locking scheme (e.g., the table was accessed in both client/server applications and over the Web), I would opt for the ORA_ROWSCN approach. The reason is that the existing legacy application might not appreciate a new column appearing, or even if we took the additional step of hiding the extra column, we might not appreciate the overhead of the necessary trigger to maintain it. The ORA_ROWSCN technique would be nonintrusive and lightweight in that respect (well, after we get over the table re-creation, that is).

The hashing/checksum approach is very database independent, especially if we compute the hashes or checksums outside of the database. However, by performing the computations in the middle tier rather than the database, we will incur higher resource usage penalties, in terms of CPU usage and network transfers.
....

Rating

  (12 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

version column approach

Yang, February 01, 2007 - 9:56 pm UTC

What I understant about "version column approach" is:

1 Read Column 1#;
2 v_timestamp := 1#.timestamp;
3 Do things...
4 Read Record 1#;
5 if v_timestamp = 1#.timestamp then
6 Do the update;
7 Else
8 The record's been touched by another session,
Do other stuffs;
9 end if;

Is that right? If it is, what if Record #1 was changed by another session after code line 4? If it is not, could you please give an example of "version column approach"?
Thanks a lot.

Tom Kyte
February 02, 2007 - 10:41 am UTC

<quote src = Expert Oracle Database Architecture>

Optimistic Locking Using a Version Column

This is a simple implementation that involves adding a single column to each database table you wish to protect from lost updates. This column is generally either a NUMBER or DATE/TIMESTAMP column. It is typically maintained via a row trigger on the table, which is responsible for incrementing the NUMBER column or updating the DATE/TIMESTAMP column every time a row is modified.

The application you want to implement optimistic concurrency control would need only to save the value of this additional column, not all of the before images of the other columns. The application would only need to verify that the value of this column in the database at the point when the update is requested matches the value that was initially read out. If these values are the same, then the row has not been updated.

Let¿s look at an implementation of optimistic locking using a copy of the SCOTT.DEPT table. We could use the following Data Definition Language (DDL) to create the table:
ops$tkyte@ORA10G> create table dept
  2  ( deptno     number(2),
  3    dname      varchar2(14),
  4    loc        varchar2(13),
  5    last_mod   timestamp with time zone
  6               default systimestamp
  7               not null,
  8    constraint dept_pk primary key(deptno)
  9  )
 10  /
Table created.


Then we INSERT a copy of the DEPT data into this table:
ops$tkyte@ORA10G> insert into dept( deptno, dname, loc )
  2  select deptno, dname, loc
  3    from scott.dept;
4 rows created.
 
ops$tkyte@ORA10G> commit;
Commit complete.


That code re-creates the DEPT table, but with an additional LAST_MOD column that uses the TIMESTAMP WITH TIME ZONE datatype (available in Oracle9i and above). We have defined this column to be NOT NULL so that it must be populated, and its default value is the current system time.

This TIMESTAMP datatype has the highest precision in Oracle, typically going down to the microsecond (millionth of a second). For an application that involves user think time, this level of precision on the TIMESTAMP is more than sufficient, as it is highly unlikely that the process of the database retrieving a row and a human looking at it, modifying it, and issuing the update back to the database could take place within a fraction of a second. The odds of two people reading and modifying the same row in the same fraction of a second are very small indeed.

Next, we need way of maintaining this value. We have two choices: either the application can maintain the LAST_MOD column by setting its value to to SYSTIMESTAMP when it updates a record or a trigger/stored procedure can maintain it. Having the application maintain LAST_MOD is definitely more performant than a trigger-based approach, since a trigger will add additional processing on part of Oracle to the modification. However, this does mean that you are relying on all of the applications to maintain LAST_MOD consistently in all places that table is modified . So, if each application is responsible for maintaining this field, it needs to consistently verify that the LAST_MOD column was not changed and set the LAST_MOD column to the current SYSTIMESTAMP. For example, if an application queries the row where DEPTNO=10

ops$tkyte@ORA10G> variable deptno   number
ops$tkyte@ORA10G> variable dname    varchar2(14)
ops$tkyte@ORA10G> variable loc      varchar2(13)
ops$tkyte@ORA10G> variable last_mod varchar2(50)

ops$tkyte@ORA10G> begin
  2      :deptno := 10;
  3      select dname, loc, last_mod
  4        into :dname,:loc,:last_mod
  5        from dept
  6       where deptno = :deptno;
  7  end;
  8  /
PL/SQL procedure successfully completed.
which we can see is currently
ops$tkyte@ORA10G> select :deptno dno, :dname dname, :loc loc, :last_mod lm
  2    from dual;
 
       DNO DNAME      LOC      LM
---------- ---------- -------- -----------------------------------
        10 ACCOUNTING NEW YORK 25-APR-05 10.54.00.493380 AM -04:00 


it would use this next update statement to modify the information . The last line does the very important check to make sure the timestamp has not changed and uses the built-in function TO_TIMESTAMP_TZ (tz is short for time zone ) to convert the string we saved in from the select back into the proper datatype. Additionally, line 3 of the update updates the LAST_MOD column to be the current time if the row is found to be updated:

ops$tkyte@ORA10G> update dept
  2     set dname = initcap(:dname),
  3         last_mod = systimestamp
  4   where deptno = :deptno
  5     and last_mod = to_timestamp_tz(:last_mod);
1 row updated.


As you can see, one row was updated¿the row of interest. We updated the row by primary key (DEPTNO) and verified that the LAST_MOD column had not been modified by any other session between the time we read it first and the time we did the update. If we were to try to update that same record again, using the same logic, but without retrieving the new LAST_MOD value, we would observe the following:
ops$tkyte@ORA10G> update dept
  2     set dname = upper(:dname),
  3         last_mod = systimestamp
  4   where deptno = :deptno
  5     and last_mod = to_timestamp_tz(:last_mod);
0 rows updated.


Notice how 0 rows updated is reported this time because the predicate on LAST_MOD was not satisfied. While DEPTNO 10 still exists, the value at the moment we wish to update no longer matches the timestamp value at the moment we queried the row. So, the application knows based on the fact that no rows were modified that the data has been changed in the database¿and it must now figure out what it wants to do about that.

You would not rely on each application to maintain this field for a number of reasons. For one, it adds code to an application, and it is code that must be repeated and correctly implemented anywhere this table is modified. In a large application, that could be in many places. Furthermore, every application developed in the future must also conform to these rules. There are many chances to ¿miss¿ a spot in the application code and not have this field properly used. So, if the application code itself is not to be made responsible for maintaining this LAST_MOD field, then I believe that the application should not be made responsible for checking this LAST_MOD field either (if it can do the check, it can certainly do the update!). So, in this case, I suggest encapsulating the update logic in a stored procedure and not allowing the application to update the table directly at all. If it cannot be trusted to maintain the value in this field, then it cannot be trusted to check it properly either. So, the stored procedure would take as inputs the bind variables we used in the previous updates and do exactly the same update. Upon detecting that zero rows were updated, the stored procedure could raise an exception back to the client to let the client know the update had, in effect, failed.

An alternate implementation uses a trigger to maintain this LAST_MOD field, but for something as simple as this, my recommendation is to avoid the trigger and let the DML take care of it. Triggers introduce a measurable amount of overhead, and in this case they would be unnecessary.
</quote>

Optimistic locking

Bo Brunsgaard, February 02, 2007 - 5:55 am UTC

Yang:

Instead of re-reading in step 4 and then testing in step 5, why not simply:

4)
update xyz
         set   ....
            ,  timestamp = new_timestamp_or_whatever 
       where   key = #1.key
        and    timestamp = #1.timestamp


That way, noone would update the row between re-reading and updating.

If the update updates no rows, then the row must have been either changed or deleted by another session

Bo Brunsgaard

Time lapse between read and update

Venky, February 02, 2007 - 6:38 am UTC

Thanks Tom.

So what will happen if another user reads the same row for a different field (or may be the same field) update? How will versioning prevent this situation?

In a stateless connection, User1 already got the version number from the table field and is taking few minutes to check other fields before "committing" the transaction from his screen. While he is doing this, user2 must have got the same version number as user1 to do an update on the same row.

Now, user1's update statement will see if the row still has the same version number that he obtained at the start of his transaction and then increments the version number and successfully updates it. But user2 when committing his transaction, the current version number would be different from what he obtained at the beginning of his transaction. So the update will have to fail after he took 15 minutes to type in all fields on the screen, check them and click the "UPDATE" button.

So, how is this feasible?

On the other hand, if user2 had something like this:

select employee_id, salary from employee for update nowait;

he would have got an exception straightaway which would have been translated in plain English (something like "This employee is being updated by different user right now. Try later").

Please don't get me wrong. I am just trying to understand it completely. So, whether it is stateful or stateless connection, I believe that pessimistic locking is easy to implement and also very useful in multi-user environment.

Thanks as always.
Tom Kyte
February 02, 2007 - 11:01 am UTC

it is NOT useful to use "for update" in a stateless environment, in general the sequence is:

page 1: grab connection, read out the version# and other data, release connection (transaction ends - any and all locks disappear)

onto
page 2: grab connection, try to update that row with that version#, release connection


the problem is locks "go away", they are not held.

Yang, February 02, 2007 - 9:18 am UTC

Bo Brunsgaard,Thank you for your help. Actually, right after I posted my review, I found an solution which is exactly the same as yours when I was reading some articla about ORA_ROWSCN. Thanks anyway.

For Venky's first question, I think you can try to find some information about "Virtual Private Database". You can specify the access privilege on particular colums to particular user by using this VPD mechanism. Then the concurrent updates(by different users) of different columns in the same row will not affect each other.

Same table same row same column

Venky, February 02, 2007 - 10:06 am UTC

Yang

My concern is about updating same field, same row and same table. And I want to convince myself between this optmistic and pessimistic locking.

Sure, code using optimistic locking can be ported across any database but pessimistic locking can be applies to only very advanced technology database like Oracle.

Locking unleashed

Venky, February 02, 2007 - 10:55 am UTC

Thanks Tom.
----------------------------
Two things:

1) I am just copying the same information again from my last post so as not to miss your attention.
2) Why am I not receiving email when there's a response to me question?

---------------------------

So what will happen if another user reads the same row for a different field (or may be the same field) update? How will versioning prevent this situation?

In a stateless connection, User1 already got the version number from the table field and is taking few minutes to check other fields before "committing" the transaction from his screen. While he is doing this, user2 must have got the same version number as user1 to do an update on the same row.

Now, user1's update statement will see if the row still has the same version number that he obtained at the start of his transaction and then increments the version number and successfully updates it. But user2 when committing his transaction, the current version number would be different from what he obtained at the beginning of his transaction. So the update will have to fail after he took 15 minutes to type in all fields on the screen, check them and click the "UPDATE" button.

So, how is this feasible?

On the other hand, if user2 had something like this:

select employee_id, salary from employee for update nowait;

he would have got an exception straightaway which would have been translated in plain English (something like "This employee is being updated by different user right now. Try later").

Please don't get me wrong. I am just trying to understand it completely. So, whether it is stateful or stateless connection, I believe that pessimistic locking is easy to implement and also very useful in multi-user environment.

Thanks as always.

Tom Kyte
February 02, 2007 - 1:36 pm UTC

2) because this is not your question? or your email was wrong.



read the text please - it should be obvious how the versioning prevents it.

You read out the "version value" and other data.
When you update, you include the original "version value" in your predicate.
If someone else updated the record - the version value will not match, zero rows updated!

The two users will never see the same version number upon update since only one of them can update at a time.

Optmistic locking cuts down productivity

venky, February 05, 2007 - 4:46 am UTC

So, I am then correct in repeating that optimistic locking will cut down productivity as the attempted update transactions by users user2.......usern will fail after spending few precious minutes to type in every field on the screen only to realise that it was updated by user1. Pessimistic locking will be better off then!

Thanks
Tom Kyte
February 05, 2007 - 9:36 am UTC

no, it is doubtful that your end users would be bumping into each other frequently - think about your applications. What are the odds that two people work on the same record at the same time? If you say "high", then I'd say "you have a really bad design, let us fix that". In general - this is the exception - not the rule.

In order to use pessimistic locking you need to maintain 'state', locks in the database.

That increases HUGELY the amount of 'server' you need. That in turn decreases productivity.

That increases the amount of code you have to make - because now you have to time out sessions, clean up dead sessions, etc - to release locks (locks held by sessions that don't really exist anymore - meaning no one can modify that record - meaning productivity is really down). So, score minus one for end users and programmers.

And so on. In a multi-tiered environment - I'm going to go with optimistic locking everytime. Only in client server would pessimistic make sense.

Thanks for the crisp reply

Venky, February 06, 2007 - 9:12 am UTC

Thank you Tom as always

Custom pessimistic locking

Robert, July 01, 2007 - 6:57 am UTC

Hi Tim,

I've been reading your site for many years and would like to say that I find this site very useful and thought provoking. Anyway..

The points mentioned about stateless and stateful connections as to why FOR UPDATE locks cannot be held are obvious, and thus the natural implementation of pessimistic locking can not be used. However, I strongly agree with many of the posters to this thread who have suggested that optimistic locking can be a pain for the end user.

One solution that I used a few months ago in a dev. environment, to test its usability, involved a custom process lock table that was joined to the database managed session table. I did not use Table API's or any such variance as I believe in the process driven model (or some other name) rather than table-driven.

This process model was also used for auditing certain conditions. Just on auditing - I find it inadequate to use triggers for auditing because of the fact that triggers are associated to tables and not procedures. For example for someone to tell me that a columnC was updated in tableA on dateA is absolutely useless. I may have four procedures that update various subsets of tableA and of these subsets, three include columnC. Now, a trigger can not tell me which process updated columnC. Anyway, I'm digressing so I apologise.

Back to locking. I used both optimistic and a custom simple locking model similar to pessimistic, which I'll call PO. Optimistic was implemented through a versioning column. The following objects were used to implement the PO locking model:

* Session table (used for other purposes also). Table consisted of SessionID, UserID, StartTime, and a few other columns.
* Process table: Contained definitions of logical processes that the database developers defined. These were in no way associated to tables. Remember - no Table APIs. An example process was "Update CompanyTaxCodes". I don't understand how people can use the expression Table APIs - This process involved many PL/SQL calculations, scans, auditing inserts and updates, a few performance and statistics recordings and finally, the transaction data updates. So in total, a minimum of seven or so tables were used.
* 2 Oracle DB Jobs:
* Job 1: Periodically expire sessions from the session table. We used a middle tier (.NET) only to render the HTML pages, which occasionally used data from the database. We felt .NET was better suited to managing the graphics, graphs, HTML, CSS etc.
* Job 2: Periodically, through the minimal factor of the total allocated lock time out of the process, expire process locks for a session.

The process simply involved:

- User1 wishes to execute process p1 (some random update process which the business have requested be handled in a manner similar to pessimistic locking).
- The corresponding stored procedure for p1, say sp1, attempts to update the lock process table row for p1 with the sessionID value of User1 as stored in the Session table. Again, we use no middle tier so all activity goes through the central session table.
- If update fails, report back to the user that the current process, p1, for the record r1 is being performed by another user. At this stage, we inserted a few records (again, not possible with Table APIs) into some auditing table to report to management and other users, locking activity.
- If the update succeeded, the user now has exclusive access to this process for this record or subset.
- User2 tries to invoke p1 for r1 and causes the event described above to occur.
- User1 finishes the update and execute p1. part2 (call it what you like - I mention this briefly at the end) to signal the end of the update. The process locks table is updated to remove the process lock flag held by User1.
- Throughout this process, the Database job has been polling the process locks table checking for timeouts.

Just a suggestion that led to a quick development prototype. It actually worked rather well. Of course, it cannot replace Oracle's pessimistic locking and we never had that as our goal. However, for certain processes it worked very well.
Tom Kyte
July 03, 2007 - 8:54 am UTC

seems you are running a stored procedure, in which case pessimistic locking can and should be used - you had no need of this do it yourself lock table at all!

you had a single transaction from start to finish, there was no need for this artificial table.


My apologies for the mispelling of your name.

Robert, July 01, 2007 - 6:59 am UTC

Tom.
Tom Kyte
July 03, 2007 - 8:52 am UTC

no worries, I make the same typo myself a surprising number of times ;)

How about performance impact?

MNguy, August 22, 2007 - 10:57 am UTC

Hi Tom,

This is indeed very useful as always. I want to addres the potential performance impact of having the optimistic locking in place with the added column. Say, if the table is fairly big and growing, will not updates with where clause of primary key and last_mod field incur more range scans in the absence of a index on primary key and last_mod field) and that too preferably unique one?

Thanks in advance,
Tom Kyte
August 22, 2007 - 2:16 pm UTC

there will be no absence of an index on the primary key??? Not sure what you mean there.

Lock the table with Update

Sowmindra, October 13, 2009 - 12:28 am UTC

Hi Tom,

It has been wonderful checking this site and gaining knowledge all around.

For exclusive control and locking the record, we thought of the below approach and found it to be successful, I would appreciate, if you can suggest if its ok or not.

Assuming that two users working on same screen (Using EMP table) and after clicking submit on the screen, then the following will be done.
VersionNo is a column used in DB to control the version.

Update EMP set VersionNo = VersionNo+1 where VersionNo=:p_VersionNo;

If rows_updated=0 then
show exclusive control error to user
end if;

The above statement will be the first statement and all users performing same operation will wait at this statement.
If first user is successful, then for subsequent users, this will fail.
We noticed that there could be a chance of dead lock because of this, so we need to update the table carefully, by checking impacts properly on other operations.

Thanks in advance

More to Explore

CLOB/BLOB/etc

Complete documentation on Securefiles and Large Objects here