-- Thanks for the question regarding "consistent gets -- Very puzzling", version 8.1.6

Submitted on 28-Jun-2001 16:59 Central time zone

Tom's latest followup | Bookmark | Bottom

Last updated 24-Jul-2009 9:23

You Asked

Tom:

create table test( a int);
begin
for i in 1..10000 loop
insert into test values (i);
end loop;
end;

set autotrace on
select count(0) from test;

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   SORT (AGGREGATE)
   2    1     TABLE ACCESS (FULL) OF 'TEST'

Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         20  consistent gets
          0  physical reads
          0  redo size
        369  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed


select * from test where a=10;

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'TEST'




Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         21  consistent gets
          0  physical reads
          0  redo size
        360  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed


select * from test;
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'TEST'




Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
        686  consistent gets
          0  physical reads
          0  redo size
     185864  bytes sent via SQL*Net to client
      74351  bytes received via SQL*Net from client
        668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10001  rows processed



question:
1.
when query count(0), what exactly did oracle do? since we don't have index, did oracle 
went to every block and query the number?
if so, why consistent gets is much lower that select * from test?
both are full scan table.

2.
why select * from test have much higher consistent gets than
select * from test where a=10?
since there are no index, oracle need to go to every block to get the value, it should be 
same consistent gets, although the first one return more rows, but it does't matter.

Regards,

and we said...

q1) select count(0) is just like 

select count(*)
  from ( select 0 from t )
/

yes, oracle went to each block to find the rows to give you a zero

q2) its a side effect of your arraysize.  You must have an 8k blocksize cause I 
reproduced this exactly.

We were expecting about 20 consistent gets right?  Well, the default array size in 
sqlplus is 15 rows / fetch.  10000/15 = 666.66666.  Well, 666+20 = 686 -- whoah there -- 
686 is our consistent gets!

Thats what happened.  When you fetched 15 rows, Oracle paused, gave you the data.  When 
you went back to get the next 15, it got the buffer again to resume your query. 

Watch what happens with different array sizes, starting with 15:


ops$tkyte@8i> select * from test;

10000 rows selected.

Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
        686  consistent gets
          0  physical reads
          0  redo size
     108813  bytes sent via SQL*Net to client
      46265  bytes received via SQL*Net from client
        668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10000  rows processed

ops$tkyte@8i> set arraysize 1000
ops$tkyte@8i> select * from test;

10000 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         30  consistent gets
          0  physical reads
          0  redo size
      86266  bytes sent via SQL*Net to client
        942  bytes received via SQL*Net from client
         11  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10000  rows processed

ops$tkyte@8i> set arraysize 5000
ops$tkyte@8i> select * from test;

10000 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         22  consistent gets
          0  physical reads
          0  redo size
     149793  bytes sent via SQL*Net to client
        382  bytes received via SQL*Net from client
          3  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10000  rows processed



Other interesting thing to note is that as the array size gets too large -- the amount of 
data transferred goes up.  there is a diminishing marginal return on the array size so 
don't go OVERBOARD.

followup to comment one

Set the arraysize to some constant as it will not matter for single row fetches whether 
it is too big but having it too small for lots of rows (NOT just full scans -- lots of 
rows) does impact you.

Reviews

June 29, 2001 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

does that mean when we do full table scen we want array size a little bit bigger, when query by 
rowid, we want arraysize smaller?

Tested it but got different results April 30, 2002 - 10pm Central time zone

Bookmark | Bottom | Top

Reviewer: ZS from Australia

I am a bit confused now, though you explained it well, but I tested it in my envirnment and got 
different results.
My query returns 122 rows with a select *.

Arraysize of 15 --> number of consistent gets was=1051 (122/15=8) which means 1043 + 8 = 1051 (good 
so far)
so I would assume according to the example above that when I do count(*) number of consistent gets 
would be 1043, right? BUT 
Arrayszie of 15 with count(*) --> consistent gets = 920 I dont understand where this comes from?

I did more testing:
Arraysize of 122 with select * --> consistent gets = 1043
Arraysize of 122 with select count(*) --> consistent gets = 920

Arrayszie of 5000 with select * --> consistent gets = 1043
Arrayszie of 5000 with select count(*) --> consistent gets = 920

I would appreciate if you can explain.
thanks,
ZS

Followup May 1, 2002 - 7am Central time zone:

The count(*) returns how many rows?  1.

The array size does even begin to kick in.  That statement is processed in ONE call.

Also, the count(*) may very well *never be touching the table*.  It might be just using an index.

Consider:

ops$tkyte@ORA817DEV.US.ORACLE.COM> create table t as select * from all_objects;

Table created.

ops$tkyte@ORA817DEV.US.ORACLE.COM> 
ops$tkyte@ORA817DEV.US.ORACLE.COM> 
ops$tkyte@ORA817DEV.US.ORACLE.COM> exec show_space( 'T' )
Free Blocks.............................0
Total Blocks............................320
Total Bytes.............................2621440
Unused Blocks...........................4
Unused Bytes............................32768
Last Used Ext FileId....................7
Last Used Ext BlockId...................40969
Last Used Block.........................60

PL/SQL procedure successfully completed.

so the table consumes about 316 blocks..


ops$tkyte@ORA817DEV.US.ORACLE.COM> select * from t;
22905 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
       1823  consistent gets
         46  physical reads
          0  redo size
    2704019  bytes sent via SQL*Net to client
     169811  bytes received via SQL*Net from client
       1528  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      22905  rows processed


22905/15 = 1527+316 = 1843 which is about our consistent gets (what we were expecting)....

ops$tkyte@ORA817DEV.US.ORACLE.COM> select count(*) from t;


Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        319  consistent gets
         46  physical reads
          0  redo size
        369  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

And the count(*), since it was done in a SINGLE CALL -- no arraysize -- only did a consistent get 
on each block ONE TIME.  Hence it did 319 consistent gets (which is about the size of the table)

Now, adding a primary key to the table:


ops$tkyte@ORA817DEV.US.ORACLE.COM> alter table t add constraint t_pk primary key(object_id);

Table altered.

ops$tkyte@ORA817DEV.US.ORACLE.COM> analyze table t compute statistics;

Table analyzed.

ops$tkyte@ORA817DEV.US.ORACLE.COM> select count(*) from t;


Statistics
----------------------------------------------------------
         79  recursive calls
          4  db block gets
         82  consistent gets
          1  physical reads
          0  redo size
        369  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
          1  rows processed

we can see that the consistent gets go way down.  why?  because the optimizer counted the rows in 
the index, not the table in this case.

May 1, 2002 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi tom,

 we calculate hit ratio like 
(1-(phycial_read/(db_block_get+consistance get))

but according to your reply here i think consistent gets or db_block_get is physical read (it is 
not logical).

Can you explian about this.

Followup May 1, 2002 - 9am Central time zone:

db_block_gets + consistent_gets = LOGICAL IO
physical_reads                  = PHYSICAL IO


I don't see where I might have mislead you on that here -- can you point out which part of the 
above indicated to you that they were physical IO?

May 1, 2002 - 9am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom

I think i am confused here about local i/o and physical i/o.
Sorry about that(it's not your fault).

But I am confused here.

If i issue query first time. Noting will be in memory So
db_block_get and consistant get will be physical read.
(am i Right?)

If i issue query second time.lots of block will be in memory.Here it will be logical i/o.

Am i right here?

Thanks.

Followup May 1, 2002 - 9am Central time zone:

You might be right, you might not be.   Some other query and/or operation may have cached the data 
already.


On the second point, you might be right, you might not be.  It depends on 

o the amount of data touched
o the size of the buffer cache
o what else is going on in the system
o the use of various features/functions in the database (eq: parallel query)

In general, yes, you would expect in many cases that the second execution would be almost all 
logical io.

Reader May 1, 2002 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Consistenet gets is based upon re-constructing a block
for consistent read. Hence it is a function of only the
number of db_blocks to be read.

If you say that it is altered by the arraysize, do you
suggest that, due to arraysize, some blocks are read
muliple times and hence some blocks have > 1 
consistent read in the process

Thanks

Followup May 1, 2002 - 10am Central time zone:

No, you are wrong in your statement.

A consistent get is a block gotten in read consistent mode (point in time mode).  It MAY or MAY NOT 
involve reconstruction (rolling back).

Db Block Gets are CURRENT mode gets -- blocks read "as of right now".


Some blocks are processed more then once, yes, the blocks will have more then 1 consistent read in 
the process.  Consider:


ops$tkyte@ORA817DEV.US.ORACLE.COM> create table t as select * from all_objects;

Table created.

ops$tkyte@ORA817DEV.US.ORACLE.COM> exec show_space( 'T')
Free Blocks.............................0
Total Blocks............................320
Total Bytes.............................2621440
Unused Blocks...........................4
Unused Bytes............................32768
Last Used Ext FileId....................7
Last Used Ext BlockId...................40969
Last Used Block.........................60

PL/SQL procedure successfully completed.

Table has 316 blocks, 22,908 rows..


ops$tkyte@ORA817DEV.US.ORACLE.COM> set autotrace traceonly statistics;

ops$tkyte@ORA817DEV.US.ORACLE.COM> set arraysize 15
ops$tkyte@ORA817DEV.US.ORACLE.COM> select * from t;
22908 rows selected.

here with an array size of 15, we expect

22908/15 + 316 = 1843 consistent mode gets.  db block gets -- they were for performing the FULL 
SCAN, they had nothing to do with the data itself we selected


Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
       1824  consistent gets
        170  physical reads
          0  redo size
    2704448  bytes sent via SQL*Net to client
     169922  bytes received via SQL*Net from client
       1529  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      22908  rows processed

ops$tkyte@ORA817DEV.US.ORACLE.COM> set arraysize 100
ops$tkyte@ORA817DEV.US.ORACLE.COM> select * from t;

22908 rows selected.

Now, with 100 as the arraysize, we expect

22908/100 + 316 = 545 consistent mode gets.

Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        546  consistent gets
        180  physical reads
          0  redo size
    2557774  bytes sent via SQL*Net to client
      25844  bytes received via SQL*Net from client
        231  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      22908  rows processed

ops$tkyte@ORA817DEV.US.ORACLE.COM> set arraysize 1000
ops$tkyte@ORA817DEV.US.ORACLE.COM> select * from t;

22908 rows selected.

now, with arraysize = 1000, we expect:

22908/1000+316 = 338 consistent mode gets...

Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        342  consistent gets
        222  physical reads
          0  redo size
    2534383  bytes sent via SQL*Net to client
       2867  bytes received via SQL*Net from client
         24  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      22908  rows processed




so yes, the blocks are gotten in consistent mode MORE THEN ONCE when the array fetch size is 
lower then the number of rows to be retrieved in this case

This is because we'll be 1/2 way through processing a block -- have enough rows to return to the 
client -- and we'll give UP that block.  When they ask for the next N rows, we need to get that 
halfway processed block again and pick up where we left off.

how does this formula works? May 1, 2002 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

22908/15 + 316 = 1843 consistent mode gets.  db block gets -- they were for 
performing the FULL SCAN, they had nothing to do with the data itself we 
selected


how did you get 22908/15 + 315 would be the total consistent gets?

Followup May 1, 2002 - 3pm Central time zone:

read the original question / answer.  I went through it there.

In this case, the numbers:

22908 = number of rows
15 = arraysize.

22908/15 = number of times you'll fetch

so, that is the number of times you'll have to read a block more then once.

315 was the number of blocks in the table (in a perfect world, we would do 315 consistent gets.  
With an arraysize of 15 and 22908 rows, we'll add 22908/15 consistent gets to that total)...

I got that srt of idea but I was confused May 1, 2002 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi

So a low arraysize in sql*plus can actually distort autotrace stats and affects performance?

So I guess autotrace is not that accurate or TKPROF will do the same?

How can we set arraysize let's for example in Developer Reports or AD HOC Tools like Business 
Objects to improve the select * from table_a like queries performance

Followup May 1, 2002 - 9pm Central time zone:

No -- it won't "distort them".

You can use the array size in sqlplus to see what your APP would do.  

Array size by default is 15 in plus
10 in JDBC
2 in pro*c
1 in OCI
??? in odbc (no idea, never use it)

It is NOT distorted -- it is reality!  It shows what really happens as you use different array 
sizes to fetch.

TKPROF -- that would show exactly the same information.  The same -- from jdbc to plus to pro*c 
(sqlplus is JUST an oci application)


with reports see:

ARRAYSIZE (RWBLD60)

Description ARRAYSIZE is the size of the array in kilobytes for use with ORACLE array processing.
Syntax
[ARRAYSIZE=]n


with business objects -- refer to their documentation.

arent consistent gets in blocks? May 1, 2002 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

arent consistent gets calculated in blocks? if that was the case how can we say 
number_of_rows/arraysize + total_blocks_read would be the total consistent gets? 

number_of_rows/arraysize dont give us in blocks does it?

The numbers suggests it is correct but this is really confusing since I cannot find it anywhere 
documentated

Followup May 1, 2002 - 9pm Central time zone:

It is because we need to get a block more then ONCE.

Suppose we do that simple "select * from t"

Suppose every block has 16 rows on it.

Suppose we have an array size of 15.


We do the first fetch.  That'll get block 1 (one consistent get) and get the first 15 rows and 
return.  

Now, we do the second fetch.  That'll get block 1 again (two consistent gets) and then get block 2 
to get the next 14 rows (three consistent gets).  We return.

We do the 3rd fetch.  We get block 2 again (four consistent gets) to get the remaining 2 rows and 
then get block 3 (five consistent gets) to get the next 13 rows.

And so on.  We end up getting every block 2 times -- because we stopped processing it partway 
through to return the result to the client.


Hence, if you full scan a table with N rows and use an array fetch of M rows, you stand a change of 
getting the blocks an extra N/M times.

from your last example May 2, 2002 - 3am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom

from your last example it would be true if we have 16 rows per block, what if we have 10 rows per 
block? We would just visit the block once right? 
If we have 46 rows instead then first block would be visited 4 or 5 times?

first time to get 15 rows
second to get another 15
their to get another 15
and fourth to get the remaining one
fifth the full table scan visit, I am not sure if fifth should exist but since all your examples 
add a previos visit for the Full Table Scan...

Finally I dont understand why a large arraysize generates more network traffic, I thought it must 
decrease but we see the oppsite

Followup May 2, 2002 - 7am Central time zone:

Ok, 10 rows/block:

on first fetch we:
   get block 1  (1 consistent get)   10 rows
   get block 2  (2 consistent gets)   5 rows / 5 left

on second fetch we:
   get block 2 (3 consistent gets) 5 rows / 0 left
   get block 3 (4 consistent gets) 5 rows / 5 left

....

and so on.  We visit EVERY OTHER block 2 times (the N/M still holds here.  If I have 100 rows on 10 
blocks and an array size of 10 -- I'll add 10 consistent gets for a grand total of 20 consistent 
gets).

I don't get :-( May 2, 2002 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: Igor from France

Hi

"We were expecting about 20 consistent gets right?  Well, the default array size 
in sqlplus is 15 rows / fetch.  10000/15 = 666.66666.  Well, 666+20 = 686 -- 
whoah there -- 686 is our consistent gets!"

Why 20? I also don't get point for "666+20 = ... " 
 
 Hope to get answer

                                         Igor

Followup May 3, 2002 - 10am Central time zone:

Sorry --- I've explained this beyond my best ability at this point.  I've explained many many 
times.  This is my last try:

The table has 20 blocks in it.
You would expect 20 consistent gets on a full scan.
But, with an array size of 15 we will revisit some blocks MORE THEN ONCE.

In fact, we might revisit upto 10000/15 = 666 blocks more then ONCE.

This is because we "start" and "stop" the query in the middle of a block.  (see the two followups 
immediately above this).


If you full scan a table T that has N blocks and R rows and you use an array fetch size of A, we 
will typically perform the following number of consistent gets:

N + R/A


We have to read N blocks (that should be obvious)
We might revisit R/A blocks more then once (since we stop in the middle)

How should we tune this? October 14, 2002 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A Reader from USA

Hi Tom,
A full table scan of a table with 770555 rows takes more than 2 minutes to get the results. It 
looks that both the "consistent gets" and the "physical reads" are high for a 13 data block gets. 
Where should we start to tune the database/query?  We have to use the query.  

Thank you in advance.


SQL> select * from TAB1;

770555 rows selected.

Elapsed: 00:02:147.52

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1447 Card=752634 Byt
          es=161816310)

   1    0   TABLE ACCESS (FULL) OF 'TAB1' (Cost=1447 Card=752
          634 Bytes=161816310)





Statistics
----------------------------------------------------------
          0  recursive calls
         13  db block gets
      74096  consistent gets
      21627  physical reads
          0  redo size
   55033565  bytes sent via SQL*Net to client
    5702495  bytes received via SQL*Net from client
      51372  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     770555  rows processed

Followup October 14, 2002 - 3pm Central time zone:

13 db block gets are blocks gotten in CURRENT mode -- they were blocks that told us were all of the 
other blocks were!  (they were the segment headers).

set arraysize 100

and try again in sqlplus (and watch your consistent gets start to go down)


Now, also bear in mind -- this is not two minutes just to read the blocks.  This is two minutes to 

o read the blocks
o send them over the network
o have sqlplus process them

consistent gets January 4, 2003 - 9pm Central time zone

Bookmark | Bottom | Top

Reviewer: Steve from NJ, USA

Hi Tom,

1) When you talk about tuning SQL, you said just reducing 
    LIO (

http://asktom.oracle.com/pls/ask/f?p=4950:8:1814295::NO::F4950_P8_DISPLAYID,F4950_P8_CRITERIA:674945
4952894,%7Blios%7D

. So increase ArraySize is one way
to reduce LIO ( consistent gets). Is this right?    

2) what is the relationship between number of "consistent gets" + "db block gets" and number of 
current (tkprof output)?  Based on the formular you ive us in this question: consistent gets = 
N+R/A, so current = N + R/A ?   

3) what is relationship  the between physical reads and disk (tkprof output)?


4) Is the following show_space system procedure? if that is one you wrote, May I have that? 

ops$tkyte@ORA817DEV.US.ORACLE.COM> exec show_space( 'T' )
Free Blocks.............................0
Total Blocks............................320
Total Bytes.............................2621440
Unused Blocks...........................4
Unused Bytes............................32768
Last Used Ext FileId....................7
Last Used Ext BlockId...................40969
Last Used Block.........................60

Thanks  

Steve

Followup January 5, 2003 - 9am Central time zone:

1) in some cases it'll reduce the LIO's yes (in many cases).  In all cases -- using a healthy but 
not "overweight" arraysize is recommended - in all cases.

2) not sure what N+r/a is (hopefully that is not A=arraysize cause it doesn't go that way -- the 
arraysize can have an effect - I used a case where I could predicate the effect but there are many 
more cases where

  a) it'll not have an effect
  b) it'll have an effect but a wholly unpredicable one.

db block gets (autotrace) are current (tkprof)
consistent gets(autotrace) are query (tkprof)

3) one to one - physical reads(autotrace) is disk (tkprof)

4)

http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:231414051079

January 5, 2003 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: Dave from Ohio

To respond to "A Reader"s question earlier, in business Objects the array fetch size is defined as 
part of the named connection used for universes or free-hand SQL.

Low - Hanging Tuning Fruit ? January 12, 2003 - 7am Central time zone

Bookmark | Bottom | Top

Reviewer: Jerry from Washington, DC

I work for a company that buys hardware that we grow out of, not into, so I am always on the 
lookout for tuning ideas, both in the app, and to some extent, the instance (even though I am a 
developer).

You mention some apps and their default arraysizes.  We run PeopleSoft on Oracle, which uses COBOL 
and SQR on a midtier UNIX app server.  Should/could we try and tune those array sizes for any gain? 
 If so, would that be in Net8?  I note that OCI has an array size of 1.

Thanks for your time.  Can't tell you how much I've learned from your site.  It is part of my 
Sunday routine.


"
Array size by default is 15 in plus
10 in JDBC
2 in pro*c
1 in OCI
"

Followup January 12, 2003 - 9am Central time zone:

It would be a question for peoplesoft to answer -- they wrote the code.  OCI can do anysize, they 
would have had to of programmed support for more than one row at a time.

Why should the SQL PLUS array size affect the plan? January 13, 2003 - 12am Central time zone

Bookmark | Bottom | Top

Reviewer: Tony from India

Why should the SQL PLUS array size affect the statistics/plan? In general, we tune queries in Sql 
Plus but the query is invoked from the middle tier. I don't understand the relavence of array size 
in Sql Plus?. Only middle tier component's array size matters. We use COM component. How to set the 
array size in COM object?

Followup January 13, 2003 - 7am Central time zone:

the arraysize doesn't affect the plan.

they arraysize DOES affect the statistics (it would be a shame if it didn't!)

The arraysize in sqlplus demonstrates that "hey, I should look at my client and set the bulk fetch 
size appropriately".  Pretty much all tools have the ability to do it.

A com object is like saying "I use CORBA, how to set arraysize in CORBA".  Doesn't make sense.  COM 
isn't a "language", COM doesn't do sql.  You look at the language you are using.  If it is an open 
systems language like C, Java -- I'lll be glad to tell you.  If it is a closed language like VB -- 
you'll have to refer to the docs for that -- I've no idea as I've never considered programming with 
them.

Why should the SQL PLUS array size affect the plan? January 13, 2003 - 1am Central time zone

Bookmark | Bottom | Top

Reviewer: Tony from India

Continuation of my earlier question..

    What I understand from your explanation is that only client side array size has to be set 
appropriately to minimize the consistent gets. Nothing to do with the database. Am I correct? 
Developers are least bothered about the array size of the middle tier. Now I understand the 
importance.

Sorry if my question is repeatative.

Followup January 13, 2003 - 8am Central time zone:

Developers are the ONLY bothered people with regards to this.  They are responsible for making sure 
the batch fetch size in their client applications is set to a reasonable number (i like 100 
personally)

For Ref Cursor also? January 13, 2003 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: Tony from India

We extensively use ref cursor in our application.
Do we have to set array size even if refcursor is used?

Followup January 13, 2003 - 12pm Central time zone:

yup.  if you want the client to fetch N rows at a time where N is some number you wish to control.

Consistent Gets - Why LIO? January 13, 2003 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: Jerry from Washington, DC USA

After reading Gorman's "THE SEARCH FOR INTELLIGENT LIFE IN THE COST-BASED OPTIMIZER" paper 
referenced on your site, a question about consistent gets came to mind.

If cgets are LIO, but involve constructing a read-consistent image of a block, wouldn't that 
involve referencing rbs, which in my mind would (or at least could) involve PIO?

Followup January 13, 2003 - 1pm Central time zone:

Nope, rollback is to be found in the buffer cache just like anything else.  It is just a rollback 
SEGMENT after all -- just another segment (like an index segment).

Extra CR / CU ? February 18, 2003 - 1am Central time zone

Bookmark | Bottom | Top

Reviewer: Amit from Dallas, USA

Hi Tom-

During testing of above examples- I came across this:
I built 2 tables - both containing a single row. 
One is created and populated with CTAS, the other is created empty, and the row is inserted and 
commited. 
Both tables have one extent consisting of 2 8K blocks.

1. Why does oracle need to perform 5 CR, 12 CU while scanning the 1st table (containing 2 blocks)?
2. Why does oracle need to perform 1 CR, 4 CU while scanning the 2nd table (containing 2 blocks)?


create table t1ctas storage (initial 1k next 1k pctincrease 0 ) tablespace users 
as select rownum n from dual
/
create table t1ins storage (initial 1k next 1k pctincrease 0)  tablespace users 
as select * from t1ctas where 1=0
/
insert into t1ins  select rownum n from dual
/
commit;

set arraysize 100;
alter session set events '10046  trace name context forever, level 1';

select * from t1ctas            -- CR=5  CU=12
/
select * from t1ins             -- CR=1  CU=4
/
select * from t1ctas            -- CR=10 CU=24
union all select * from t1ctas 
/
select * from t1ins             -- CR=8  CU=2
union all select * from t1ins  
/
select * from t1ctas            -- CR=6  CU=16
union all select * from t1ins   
/
exit;




*** SESSION ID:(11.1483) 2003-02-17 23:22:29.809
APPNAME mod='SQL*Plus' mh=3669949024 act='' ah=4029777240
=====================
PARSING IN CURSOR #1 len=70 dep=0 uid=37 oct=42 lid=37 tim=1283779 hv=1445189883 ad='5302b64'
alter session set events '10046  trace name context forever, level 1'
END OF STMT
EXEC #1:c=1,e=38,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=3,tim=1283784
=====================
PARSING IN CURSOR #1 len=47 dep=0 uid=37 oct=3 lid=37 tim=1283805 hv=4134657170 ad='5310df8'
select * from t1ctas            -- CR=5  CU=12
END OF STMT
PARSE #1:c=0,e=8,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=3,tim=1283811
EXEC #1:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=3,tim=1283812
FETCH #1:c=0,e=2,p=1,cr=5,cu=12,mis=0,r=1,dep=0,og=3,tim=1283816
FETCH #1:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=3,tim=1283818
STAT #1 id=1 cnt=1 pid=0 pos=0 obj=40021 op='TABLE ACCESS FULL T1CTAS '
=====================
PARSING IN CURSOR #1 len=46 dep=0 uid=37 oct=3 lid=37 tim=1283840 hv=740128398 ad='530d86c'
select * from t1ins             -- CR=1  CU=4
END OF STMT
PARSE #1:c=0,e=8,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=3,tim=1283846
EXEC #1:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=3,tim=1283847
FETCH #1:c=0,e=0,p=0,cr=1,cu=4,mis=0,r=1,dep=0,og=3,tim=1283849
FETCH #1:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=3,tim=1283851
STAT #1 id=1 cnt=1 pid=0 pos=0 obj=40022 op='TABLE ACCESS FULL T1INS '
=====================
PARSING IN CURSOR #1 len=78 dep=0 uid=37 oct=3 lid=37 tim=1283877 hv=3911934752 ad='5313e54'
select * from t1ctas            -- CR=10 CU=24
union all select * from t1ctas
END OF STMT
PARSE #1:c=0,e=9,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=3,tim=1283884
EXEC #1:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=3,tim=1283886
FETCH #1:c=0,e=0,p=0,cr=5,cu=12,mis=0,r=1,dep=0,og=3,tim=1283888
FETCH #1:c=0,e=0,p=0,cr=5,cu=12,mis=0,r=1,dep=0,og=3,tim=1283890
STAT #1 id=1 cnt=2 pid=0 pos=0 obj=0 op='UNION-ALL '
STAT #1 id=2 cnt=1 pid=1 pos=1 obj=40021 op='TABLE ACCESS FULL T1CTAS '
STAT #1 id=3 cnt=1 pid=1 pos=2 obj=40021 op='TABLE ACCESS FULL T1CTAS '
=====================
PARSING IN CURSOR #1 len=76 dep=0 uid=37 oct=3 lid=37 tim=1283923 hv=3592920098 ad='530ac48'
select * from t1ins             -- CR=8  CU=2
union all select * from t1ins
END OF STMT
PARSE #1:c=0,e=11,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=3,tim=1283931
EXEC #1:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=3,tim=1283933
FETCH #1:c=0,e=0,p=0,cr=1,cu=4,mis=0,r=1,dep=0,og=3,tim=1283935
FETCH #1:c=0,e=0,p=0,cr=1,cu=4,mis=0,r=1,dep=0,og=3,tim=1283937
STAT #1 id=1 cnt=2 pid=0 pos=0 obj=0 op='UNION-ALL '
STAT #1 id=2 cnt=1 pid=1 pos=1 obj=40022 op='TABLE ACCESS FULL T1INS '
STAT #1 id=3 cnt=1 pid=1 pos=2 obj=40022 op='TABLE ACCESS FULL T1INS '
=====================
PARSING IN CURSOR #1 len=77 dep=0 uid=37 oct=3 lid=37 tim=1283969 hv=1374823634 ad='52ffcd8'
select * from t1ctas            -- CR=6  CU=16
union all select * from t1ins
END OF STMT
PARSE #1:c=0,e=10,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=3,tim=1283977
EXEC #1:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=3,tim=1283979
FETCH #1:c=0,e=0,p=0,cr=5,cu=12,mis=0,r=1,dep=0,og=3,tim=1283981
FETCH #1:c=0,e=0,p=0,cr=1,cu=4,mis=0,r=1,dep=0,og=3,tim=1283983
XCTEND rlbk=0, rd_only=1
STAT #1 id=1 cnt=2 pid=0 pos=0 obj=0 op='UNION-ALL '
STAT #1 id=2 cnt=1 pid=1 pos=1 obj=40021 op='TABLE ACCESS FULL T1CTAS '
STAT #1 id=3 cnt=1 pid=1 pos=2 obj=40022 op='TABLE ACCESS FULL T1INS '

Followup February 18, 2003 - 7am Central time zone:

it just does in 8i for some reason.  In 9i, it does not.

Note that most people would never see this as a single row table created via CTAS is not very 
common.  if you do a "real sized" table -- you'll see the additional header gets (the db block get 
-- current mode gets) but averaged out over a large full scan, it is less than "noise"

April 3, 2003 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: Robert from Memphis, USA

how do I know how many blocks my query read? April 17, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi

How do I determine how many database blocks were read in my query from autotrace utility or TKPROF?

Followup April 17, 2003 - 3pm Central time zone:

consitent gets:

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   SORT (AGGREGATE)
   2    1     TABLE ACCESS (FULL) OF 'TEST'

Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         20  consistent gets


or in tkprof query + db block gets.

the thing is, doesnt that deoend on arraysize? April 18, 2003 - 4am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi

I can see you get different consistent gets if we use different array size, so how can that be the 
number of data blocks...?

I mean I can see 100000 consistent gets or 10000 consistent gets for same query, does that mean my 
query reads different number of blocks????

Followup April 18, 2003 - 12pm Central time zone:

yes it does -- by definition. 

consistent gets = logical IO's = number of block gets.

it does not equal the number of DISTINCT blocks you processed if that is what you mean, it means 
the number of block gets (which require cpu cycles and latches = locks and so on)

April 18, 2003 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom,

 I am working on oracle from last 6 year. I am reading your site from begining (Aug 2000).

But when it comes to consistant get and db block get
I am more confused then ever before. Please help help!!!

You said 
 1) "A consistent get is a block gotten in read consistent mode (point in time mode). 
 It MAY or MAY NOT involve reconstruction (rolling back)."

What do you mean by "point in time mode" (Hearing this word from last couple of years but lot 
confusing...

What do you mean by "It MAY or MAY NOT involve reconstruction (rolling back)."

Please elobarate in this issue

Thnaks

Followup April 18, 2003 - 12pm Central time zone:

if you've been reading since then -- you've heard me say:

"if you read the concepts guide AND
you retain 10% of that information YOU'LL HAVE
90% more knowledge about Oracle then most people out there..."

http://download-west.oracle.com/docs/cd/B10501_01/server.920/a96524/c21cnsis.htm#2570

all about multi-versioning.

All queries done by Oracle are done "at a point in time", consistently -- read that (read the 
entire book in fact) and you'll get an understanding of the single most important concept to 
understand about Oracle ever.

If you have my book "Expert one on one Oracle" -- i cover it in gory detail in there as well.

Would dbms_jobs be affected by arraysize? April 18, 2003 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: Shiv from UK

Tom,

Thanks in advance.
Can a  dbms_job be optimized using arraysize? if so how does one set it. or it doesn't matter since 
it doesn't run as a client. 

cheers
Shiv

Followup April 18, 2003 - 12pm Central time zone:

PLSQL can be optimized using BULK COLLECT (array fetching) and FORALL (bulk modification)

search the PLSQL manual's table of contents for those terms and read on....

regarding number of blocks to read April 18, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi

you said that consistent gets = LIO = block gets

I guess you mean the times Oracle goes to fetch the blocks, but I was asking the number of physical 
blocks Oracle needs to read! How how many times but rather, how many blocks.

For example to satisfy a query Oracle needs to read 100000 database blocks (8k, so 8k*100000 around 
800MB), from where we can get this statistics??? Consistent gets is the number of times we need to 
fetch

I guess it´s 

arraysize*number of consistent gets but if we dont know the arraysize?

Followup April 18, 2003 - 3pm Central time zone:

Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         20  consistent gets
          0  physical reads

that would be that number.

it might be 0, it might be 20000000, it might be 22 to read 100,000 database blocks in answer to a 
query (for PIO's)

from your first example April 19, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

hello

from your first example

ops$tkyte@8i> select * from test;

10000 rows selected.

Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
        686  consistent gets
          0  physical reads
          0  redo size
     108813  bytes sent via SQL*Net to client
      46265  bytes received via SQL*Net from client
        668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10000  rows processed

ops$tkyte@8i> set arraysize 1000
ops$tkyte@8i> select * from test;

10000 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         30  consistent gets
          0  physical reads
          0  redo size
      86266  bytes sent via SQL*Net to client
        942  bytes received via SQL*Net from client
         11  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10000  rows processed

one is 696 consistent gets and the other 30, 4 db block gets in both cases

did Oracle read 4 database blocks only to fetch 10000 rows?!?!?!

Followup April 19, 2003 - 5pm Central time zone:

db block gets are blocks gotten in CURRENT (as opposed to consisten) mode.  They are block gets we 
did to discover the extent map for the table (to know what to read).  We get that in "current, as 
of right now" mode instead of "in consistent read, as of the point in time the query began" mode.

Oracle did 4 current mode gets in both cases to figure out what blocks to read.

In the first example, it did 686 logical IO's
In the second, only 30

It did no physical IO.

Reader April 19, 2003 - 9pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom,

If Oracle wants to read the extent map from the
segment header, why there are 4 db block gets instead of 1.

In my following test I get 0 db block gets

I have created table t as select * from dba_objects;

SQL> select count(*) from t;

  COUNT(*)
----------
     29387

<bounced the database>

SQL> get afiedt.buf
  1  set termout off
  2  set autotrace on
  3  spool c:\tmp\t.log
  4  select * from t;
  5* spool off
SQL> start afiedt.buf

Excerpt from spool file=>
=========================
Statistics
------------------------------------
        188  recursive calls                        
          0  db block gets                   
       2359  consistent gets          
        402  physical reads              
          0  redo size          
    2049661  bytes sent via SQL*Net to client
      22048  bytes received via SQL*Net from client
       1961  SQL*Net roundtrips to/from client
          2  sorts (memory) 
          0  sorts (disk) 
      29387  rows processed


Thanks

Followup April 20, 2003 - 8am Central time zone:

because -- just because.  In different releases -- different count.

In your example, you didn't do a full table scan -- apples to toaster oven comparision.

You are using the CBO
You have a primary key on T
It read the index using an index full scan.

Or you are using Oracle9i and new features like ASSM...

Or ....


Let me put it this way -- if you see 4 there, and you did a simple full table scan -- then the 4 db 
block gets are what I said they where.

I did not say "you shall always see 4 there", rather, that when you do -- thats what they are.

from your first example April 20, 2003 - 2am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

sorry I didnt make the question clear.... I really wanted to ask how many database blocks (in 
memory or disk whatever) did Oracle read in order to satisfy your query and not how many LIO or 
PIO. 

Definitely the number of blocks visited is not number of consistent gets since consistent get 
varies with arraysize, so in your example how many blocks did Oracle need to read?

Followup April 20, 2003 - 9am Central time zone:

what is the difference?  

What do  you mean by "how many blocks did Oracle need to read".

To me that is one of two things:

o how many block gets did Oracle do in order to satisfy my query.  That is logical IO

o how many blocks did Oracle physically read from disk in order to satisfy my query.  That is 
phyiscal IO.


Those are the only two numbers that mean anything.  Those are the only two numbers you can get. 
 

That there are 100 blocks in the table, and we did 1,000 Logical IO's to read it -- of those two 
numbers the only relevant one is "1,000".

the difference is April 20, 2003 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

hi

the difference is I wanted to know how many blocks did Oracle need to read in order to satisfy a 
query, I wanted to know how much MB or GB did Oracle need to read. Consistents gets are fine but 
for example if I see a huge consistent gets but I did not know the arraysize someone might think 
geez it's read tons to satisfy this query, but it was probably because a low arraysize, who knows? 
If we dont know arraysize it's hard to determine how "large" or how "bad" is that high number of 
consistent gets. So since consistent gets is determined by the arraysize I wanteds to know some 
value which does not depend on another parameter, for example the number of database blocks needed 
to read. I guess for a same query with same execution plan Oracle should read X number of database 
blocks which would YMB (database block size * number of database blocks). I am not that conmcerned 
about PIO at this time, I was wondering how do we know the number of database blocks Oracle needs 
to read, so for example from that determine how much db cache is my query using, we definitely 
cannot calculate that with consistent get right?

Followup April 20, 2003 - 2pm Central time zone:

need to read -- that is PIO

need to process that is LIO

nothing else is relevant nor obtainable.

TKPROF would tell you if arraysize is a factor -- divide fetches into rows - if the number is 1, 
they aren't using array fetches.

A FACTOR in consistent gets is arraysize.

ARRAYSIZE does not determine consistent gets.

April 21, 2003 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom,

Consistant get means reconstructing block from rollback segment. Am I right ?

If so then it is reading rollback segment which resides on tablespace and it doing PIO then why we 
call it LIO ???

If not then throght some light on this.

Thanks for excellent web site!!!

Followup April 21, 2003 - 8am Central time zone:

consistent gets means getting a block "as of a point in time" and MAY require read asides to the 
rollback segment in order to reconstruct the block.  emphasis on MAY.

the RBS is just a segment (that is what the S is for).  It resides in the cache just like an index, 
table or anything else does.


Here is an example showing the "read asides" to the rbs.  It shows that we MAY require a read to 
the rbs by running the same exact query -- but doing lots of work in between:


ops$tkyte@ORA920> create table t ( x int );

Table created.

ops$tkyte@ORA920> insert into t values ( 1 );

1 row created.

ops$tkyte@ORA920> commit;

Commit complete.

ops$tkyte@ORA920>
ops$tkyte@ORA920>
ops$tkyte@ORA920> variable x refcursor
ops$tkyte@ORA920> variable y refcursor
ops$tkyte@ORA920>
ops$tkyte@ORA920> alter session set sql_trace=true;

Session altered.

ops$tkyte@ORA920> begin
  2     open :x for select * from t X;
  3     open :y for select * from t Y;
  4  end;
  5  /

PL/SQL procedure successfully completed.

ops$tkyte@ORA920> print x

         X
----------
         1

ops$tkyte@ORA920>
ops$tkyte@ORA920> begin
  2      for i in 1 .. 10000
  3      loop
  4          update t set x = i;
  5          commit;
  6      end loop;
  7  end;
  8  /

PL/SQL procedure successfully completed.

ops$tkyte@ORA920>
ops$tkyte@ORA920> print y

         X
----------
         1

ops$tkyte@ORA920>


So, they return the same exact information but TKPROF shows us:

SELECT * from t X

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        1      0.00       0.00          0          7          0           1
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        3      0.00       0.00          0          7          0           1

Rows     Row Source Operation
-------  ---------------------------------------------------
      1  TABLE ACCESS FULL T (cr=7 r=0 w=0 time=162 us)
********************************************************************************
SELECT * from t Y

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        1      0.04       0.06          0      10002          0           1
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        3      0.04       0.06          0      10002          0           1

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 83     (recursive depth: 1)

Rows     Row Source Operation
-------  ---------------------------------------------------
      1  TABLE ACCESS FULL OBJ#(34399) (cr=10002 r=0 w=0 time=60325 us)


that it took 10k consistent gets for Y -- versus 7 for X -- both queries return the same data 
-- it is just that for the first fetch from Y we had to "undo" the 10,000 updates that happened 
between the OPEN of Y and the fetch.

April 21, 2003 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom

what do you think about DB2 calim on this

http://www-

3.ibm.com/software/data/pubs/papers/readconsistency/readconsistency.pdf

Followup April 21, 2003 - 9am Central time zone:

I like this quote:

<quote>
The reality is that with Oracle, as with any other database, you design and code your application 
with an understanding of the underlying isolation and concurrency model
</quote>

I've been saying that for years myself...


but then they say:

<quote>
No other database vendor has implemented Oracle's Multi Version Read Consistency isolation nor has 
it proven to be a performance advantage in industry standard, ISV or real life customer benchmark.
</quote>

well, they sort of need to do their homework -- I would like us to be the only ones with this 
feature -- but we ain't.  RDB (which we bought a while ago) had it for a loooonnnnnggg time.  
Borlands Interbase (someone bought that I believe) did.  Heck, mysql does it.  So, to make us 
look "unique", that is very nice of them -- but not really true.  And given that I don't even see 
DB2 in the top 10 tpc-C results -- even on AIX for goodness sake -- don't see where they are going 
with this benchmark concept (oh thats right, they don't actually run the tpc-c's, not even on their 
own machines)....  Ok, going to the SAP website

http://www.sap.com/benchmark/index.asp?content=http://www.sap.com/benchmark/ato2tier.asp

<code>

looks like the top ones in this ISV benchmark (run by a company that doesn't necessarily promote Oracle) would be .... (not DB2).

Oh, i see way down at the bottom they refer to a read only benchmark -- a tpc-h. How appropriate in a paper describing CONCURRENCY CONTROLS that affect reads/writes at the same time, (not).

Oh I see they say we bypass our MV in the tpc-c because we use serializable. Boy -- just shows a TOTAL and FUNDEMENTAL lack of understanding on their part of how MV works in Oracle -- they totally 100% missed the boat. Our serializable is even MORE depenedent on multi-versions then our read committed is!!!

Now, they say:

<quote>
With Oracle, the data you see in any query is the data as it existed at the start of that query. This does not conform to any ANSI standard isolation level, nor is it the way other RDBMSs work.
</quote>

Wrong and Wrong (2 for 2). ANSI describes isolation in terms of "phenomena".
They take three phenomena and create four isolation levels based on the
existence, or lack thereof, of the above phenomena. They are:

Isolation Level Dirty Read Non-Repeatable Read Phantom Read
Read Uncommitted Permitted Permitted Permitted
Read Committed Permitted Permitted
Repeatable Read Permitted
Serializable

Oracle supports explicitly two of the above isolation levels as they are defined
-- read committed and serializable. This doesn't tell the whole story however.
The SQL standard was attempting to set up isolation levels that would permit
various degrees of consistency for queries performed in each level. Repeatable
read is the isolation level they claim you must be at to get a read consistent
result from a query. Read committed does not give you consistent results. Read
uncommitted is the level to use to get non-blocking reads.

In Oracle -- Serializable and Repeatable Read are synonymous. Read uncommitted with its dirty read is unTHINKABLE.

but wait -- lets see what DB2 has to say in their technical documentation - not their marketecture documentation:

<quote>
IBM Redbook for DB2

3.3.1.1 Strategy 1 - Conversation Integrity (Not Recommended)

A commit point is created at the end of the conversation. The lock duration may include user think time. Therefore, this strategy should not be used in applications where other users need concurrent access to the tables.

Read-only application might get away with this strategy in some cases. S-locks are normally released before commit point, but an S-locked page or row cannot be updated by another process and an S-lock may have a very long duration. It may include user think time.

We do not recommend this strategy even for read-only applications, although it may be the most convenient one from a programming point of view.
</quote>

what they are talking about there is "getting the right answer". In DB2 getting the right answer from a query that accesses more then one record requires leaving shared read locks behind -- that is sort of a bummer in a multi-user environment isn't it.

or, going onto the read uncommitted level they are so proud of (apparently)

<quote>
1.3.2.1 Case 2 - Without Locking

1. User A updates value 100 to 0 into the DB2 table column.
2. User B reads the updated value 0 and makes program decisions based on that value.
3. User A cancels the process and changes value 0 back to value 100 for the
DB2 table column.

This case shows that, without locking, User B made a wrong program decision, and because of that the business data in the database is potentially corrupted

</quote>

well, thats not too brilliant.

Of course, I've already addressed the "hey no one else does it like this" (which is a really poor argument don't you think. Is IBM a company that would jump off a bridge because everyone else does?? I don't get where they are going with that little tidbit of inaccuracy)

It is funny that they continue on to bash our detour into the RBS to read around locks -- when they themselves would STOP. "Bad Oracle -- they use extra CPU, Good IBM -- we just stop you cold, no wasting of a resource you cannot put in the bank and use later, no siree bob!"

Great -- they get into a technical discussion of how Oracle manages rollback and manage to totally muck it up in the first paragraph:

<quote SRC=IBM accuracy=LOW>
Rollback segments are simply disk space set aside to store old images of data. They are physically a set of operating system files

inaccurate -- they are a segment that may span tons of files/logical volumes

of a predefined size

inaccurate -- they can grow and shrink

that work in a circular fashion.

accurate to a point -- but they get it wrong in the following

That is, old data is put into rbs 1 followed by rbs 2 and so on. When all of the rollback segments are full, the process wraps back around to the first segment.

wrong wrong wrong -- so wrong.

Undo Tablespaces in Oracle 9i work in a similar fashion except that the information is stored in an Oracle managed tablespace which is made up of operating system data files so the process is almost identical. There is a DBA management burden to configure and maintain rollback segments.

lets look below, after this quote for a tidbit on that

How big should your rollback segments be? What if you have a single transaction that does not fit into your rollback segments? What happens if you run out of space?

in IBM with fixed sized logs that cannot wrap (as opposed to Oracle) -- how big should your logs be? what if you have a single transaction that doesn't fit into your logs? what if you run out of space? POT calling KETTLE black here?

</quote>

Ok, so Oracle unduly burdens you with RBS? How about:

<quote src= IBM udb release notes>

If maxlocks is set too low, lock escalation happens when there is still enough lock space for other concurrent applications. If maxlocks is set too high, a few applications can consume most of the lock space, and other applications will have to perform lock escalation. The need for lock escalation in this case results in poor concurrency.�
</quote>

hey, whats this lock escalation thing and why didn't they talk about this "feature" in their marketecture paper? What undue burden does this "feature" place on the DB2 dba.

Oh yeah, they forget to mention in 9i using undo tablespaces, the dba sets up an undo retention period and -- oh yeah, thats it.

continueing on:

<quote>
Well running out of space is not a problem...Oracle simply cancels your transaction! Yes that's right, if there is not enough rollback segment space for your transaction, the transaction fails. ...
</quote>

How evil that

April 21, 2003 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom,

DB2 page contain statement like

"Oracle's concurrency model is page based(block) not row based
In Oracle, the SCN is stored in the header of each data block (a.k.a. page).
So if any record on that block is modified, the SCN for that block is updated.
If a transaction is looking for record 5 on block 106, it may have to clone and
reconstruct several different version of block 106 even if record 5 has never
changed."


If there is five row in a block and One get changed and we are reading rows which are not changed. 
According to DB2 document it has to reconstruct image from rollback segment
fot the rows which haven't changed .

If this is the case then it is not right..

Great Explaination April 22, 2003 - 12am Central time zone

Bookmark | Bottom | Top

Reviewer: Sam

I ponder, what Mr. DB2 has to say on this..

db block gets on update. May 6, 2003 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: Jerry from Washington, DC

I think this is my favorite thread.  One question came up regarding the statistics for db block 
gets during an update.  It seems to me that db block gets very closely approximates the number of 
rows involved in an update, even though it is a block.  

Could Oracle be accessing or updating the block header for each row updated? 

Here is my working example (8.1.7 on AIX):



SQL> create table t as select * from all_objects;

Table created.

SQL>
SQL> exec show_space('T');
Free Blocks.............................0
Total Blocks............................900
Total Bytes.............................7372800
Unused Blocks...........................29
Unused Bytes............................237568
Last Used Ext FileId....................31
Last Used Ext BlockId...................4033
Last Used Block.........................16

PL/SQL procedure successfully completed.

SQL>
SQL> update t set object_id = object_id;

67118 rows updated.

SQL>
SQL> commit;

Commit complete.

SQL>
SQL> set autotrace on exp stat
SQL> set timing on
SQL>
SQL> /* once again after parsing/caching */
SQL>
SQL> update t set object_id = object_id;

67118 rows updated.

Elapsed: 00:00:45.01

Execution Plan
----------------------------------------------------------
   0      UPDATE STATEMENT Optimizer=CHOOSE
   1    0   UPDATE OF 'T'
   2    1     TABLE ACCESS (FULL) OF 'T'




Statistics
----------------------------------------------------------
          0  recursive calls
      68751  db block gets
        979  consistent gets
          0  physical reads
   16924764  redo size
        545  bytes sent via SQL*Net to client
        458  bytes received via SQL*Net from client
          6  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
      67118  rows processed

SQL>
SQL> commit;

Commit complete.

Elapsed: 00:00:00.00
SQL>
SQL> set autotrace off
SQL> set timing off
SQL> insert into t select * from t;

67118 rows created.

SQL>
SQL> commit;

Commit complete.

SQL>
SQL> exec show_space('T');
Free Blocks.............................1
Total Blocks............................1740
Total Bytes.............................14254080
Unused Blocks...........................0
Unused Bytes............................0
Last Used Ext FileId....................31
Last Used Ext BlockId...................4443
Last Used Block.........................60

PL/SQL procedure successfully completed.

SQL>
SQL> set autotrace on exp stat
SQL> set timing on
SQL> update t set object_id = object_id;

134236 rows updated.

Elapsed: 00:01:04.05

Execution Plan
----------------------------------------------------------
   0      UPDATE STATEMENT Optimizer=CHOOSE
   1    0   UPDATE OF 'T'
   2    1     TABLE ACCESS (FULL) OF 'T'




Statistics
----------------------------------------------------------
          0  recursive calls
     137468  db block gets
       1739  consistent gets
          0  physical reads
   33839832  redo size
        546  bytes sent via SQL*Net to client
        458  bytes received via SQL*Net from client
          6  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
     134236  rows processed

SQL>
SQL> commit;

Followup May 6, 2003 - 2pm Central time zone:

yes it does -- don't forget to include indexes that could be on the table as well....

May 7, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: Arash from Canada

Hi Tom,

  I can't say how much I learnt from your site. I did understand how to calculate 'Consistent Gets' 
for 'SELECT' statement and also what is 'db block read' in update statement. But I don't know how 
to calculate or what is the number of 'Consistent Gets' in update statement. I attached the 
example:

D820> exec show_space('T');
Free Blocks.............................5                                                           

Total Blocks............................164                                                         

Total Bytes.............................1343488                                                     

Unused Blocks...........................1                                                           

Unused Bytes............................8192                                                        

Last Used Ext FileId....................4                                                           

Last Used Ext BlockId...................38555                                                       

Last Used Block.........................15                                                          


PL/SQL procedure successfully completed.

D820> update t set object_id = object_id;

12016 rows updated.


Execution Plan
----------------------------------------------------------                                          

   0      UPDATE STATEMENT Optimizer=CHOOSE                                                         

   1    0   UPDATE OF 'T'                                                                           

   2    1     TABLE ACCESS (FULL) OF 'T'                                                            





Statistics
----------------------------------------------------------                                          

       5706  recursive calls                                                                        

      13109  db block gets                                                                          

       3823  consistent gets                                                                        

          0  physical reads                                                                         

    2922208  redo size                                                                              

        485  bytes sent via SQL*Net to client                                                       

        837  bytes received via SQL*Net from client                                                 

          6  SQL*Net roundtrips to/from client                                                      

          1  sorts (memory)                                                                         

          0  sorts (disk)                                                                           

      12016  rows processed                                                                         

------------------------------------------------------
D820> select * from t;

12016 rows selected.


Execution Plan
----------------------------------------------------------                                          

   0      SELECT STATEMENT Optimizer=CHOOSE                                                         

   1    0   TABLE ACCESS (FULL) OF 'T'                                                              





Statistics
----------------------------------------------------------                                          

          0  recursive calls                                                                        

          3  db block gets                                                                          

        242  consistent gets                                                                        

          0  physical reads                                                                         

          0  redo size                                                                              

    1676901  bytes sent via SQL*Net to client                                                       

      10132  bytes received via SQL*Net from client                                                 

         83  SQL*Net roundtrips to/from client                                                      

          0  sorts (memory)                                                                         

          0  sorts (disk)                                                                           

      12016  rows processed                                                                         

Why 'Consistent Gets' is higher in update statement than select !? The same arraysize(150), same 
table, same datablocks...

I would appreciate it if you elaborate this more.
Thanks for your time,
Arash

Followup May 8, 2003 - 9am Central time zone:

an UPDATE (and delete for that matter - and sometimes an insert) have two components -- a 
consistent READ part and a modification part.

inline views will be useful here to see this, instead of coding:

update t set object_id = object_id;


say you wrote:

update
  ( select * 
      from t )
 set object_id = object_id;

the part in "bold" is done as a consistent read -- the update, modification part regets the block 
in current mode to actually update the data.

Why must this be? well, consider what would happen if:

create table t ( x int );
create t_idx index on t(x);


update t set x = x+100 where x > 5;

Now, say that update used t_idx to get the data, to find rows where x >5.  Now, we get the first 
row -- x = 5, we update to 105 and update the index et.al.  Now, we continue one and eventually we 
get into rows where x > 100.... We find a row 105 -- we already updated (but do not realize this in 
current mode, only in consistent read mode).  We would make it 205 and so on.

You would have an infinite loop.  Don't laugh -- sqlserver (MS and Sybase) both have the ability 
for this to happen.  I remember the first time I saw it happen -- I wrote a program to read a table 
and generate a complex update.  I wrote the update to stdout.  I had the update go into isql, 
something like this:


isql (with a query) | filter_to_generate_updates | isql (to do the updates)


that table had 5000 rows in it.  without consistent reads -- and since their update will sometimes 
be a "delete+insert" -- what happened was row 1 became row 5001 and so on.  when we got to row 5000 
instead of stopping, it kept reading.

25,000 plus updates later I killed it -- really confused.  took a while to figure out what exactly 
had happened.

May 8, 2003 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Thanks Tom.

May 8, 2003 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom,
Excellent example.
Could you explain 

"modification part regets the block in current mode to actually update the data."

Thanks

Followup May 9, 2003 - 12pm Central time zone:

say you:

create table t ( x int );

insert into t values ( 1 );
insert into t values ( 2 );
commit;


update t set x = x+1;

Oracle will

a) read T in consistent read mode as of the point in time the update was started
b) that'll get the single block T is on in consistent mode.
c) Oracle will see the first row -- x=1 -- and say "hey, I gotta update that"
d) Oracle will get that block in current mode, modify it and let go of it
e) Oracle will see the second row -- x=2 -- and say "hey...."
f) Oracle will get that block in current mode, modify and let go of it.


consider:

ops$tkyte@ORA920> create table t ( x int );
Table created.

ops$tkyte@ORA920> insert into t select rownum from all_users;
46 rows created.

ops$tkyte@ORA920> commit;
Commit complete.

ops$tkyte@ORA920> set autotrace traceonly statistics
ops$tkyte@ORA920> update t set x = x where rownum <= 1;

Statistics
----------------------------------------------------------
          2  db block gets
          7  consistent gets
          1  rows processed

ops$tkyte@ORA920> update t set x = x where rownum <= 2;

Statistics
----------------------------------------------------------
          2  db block gets
          7  consistent gets
          2  rows processed

ops$tkyte@ORA920> update t set x = x where rownum <= 5;

Statistics
----------------------------------------------------------
          5  db block gets
          7  consistent gets
          5  rows processed

ops$tkyte@ORA920> update t set x = x where rownum <= 10;

Statistics
----------------------------------------------------------
         10  db block gets
          7  consistent gets
         10  rows processed

ops$tkyte@ORA920> update t set x = x where rownum <= 20;

Statistics
----------------------------------------------------------
         20  db block gets
          7  consistent gets
         20  rows processed


The number of consistent gets is "consistent" (we full scanned the table in consistent read mode 
each time).  The number of db block (current mode) gets goes up as the rows modified goes up.

May 9, 2003 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom,
Now I am getting clear on this.

One other question

When you update table number of DB BLOCK GET goes up and up. In your example if you update 5 rows 
DB BLOCK GET would be 5, for 10 rows it wold be 10 and for 20 it would be 20 and so on.....

What does that means when you update "update t set x = x where rownum <= 20" ? 

Is table spread across 20 blocks or Table spread across 20 block in buffer cahce?

Could you explain this?
Thanks,

Followup May 9, 2003 - 3pm Central time zone:

that table (very very small table) would have been on a single block.  the same block was gotten 
over and over and over.

ops$tkyte@ORA920> create table t ( x int );

Table created.

ops$tkyte@ORA920> insert into t select rownum from all_users;

46 rows created.

ops$tkyte@ORA920> select dbms_rowid.rowid_block_number(rowid), count(*) from t
  2  group by dbms_rowid.rowid_block_number(rowid);

DBMS_ROWID.ROWID_BLOCK_NUMBER(ROWID)   COUNT(*)
------------------------------------ ----------
                                 584         46


there was one block, block 584...

May 9, 2003 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

I got it

Thanks

May 12, 2003 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom,

  Here is the summary of what I've understood uptill  now.
  update ( select * from t ) set object_id = object_id; 
  select "select * from t" would do consistent read . (Ie If query start at 10 SCN then it will 
read all block below 10 SCN.
  If the block is not available then it will reconstruct image from rollback segment.)
  Update portition would do CURRENT READ (ie It won't care about SCN it will get latest value for 
the block)
  Please confirm this.

 Thanks

Followup May 12, 2003 - 10am Central time zone:

yes, confirmed.

May 28, 2003 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom, 

  I understood what you mean by giving the following example : 
"
create table t ( x int );
create t_idx index on t(x);


update t set x = x+100 where x > 5;

Now, say that update used t_idx to get the data, to find rows where x >5.  Now, 
we get the first row -- x = 5, we update to 105 and update the index et.al.  
Now, we continue one and eventually we get into rows where x > 100.... We find a 
row 105 -- we already updated (but do not realize this in current mode, only in 
consistent read mode).  We would make it 205 and so on.
"

But I couldn't simulate in MSSQL 2000 and Sybase. I mean it works fine. It doesn't work like 
forever loop. It just dose the update.

example:
set transaction isolation level read uncommitted

create table t (x integer)

begin
declare @i integer;
SET @i=1;
while @i<1000
begin
  insert into t(x) values(@i)
  SET @i = @i + 1
end
end

select * from t

begin transaction
update t
set x = x+5
where x> 100

(899 row(s) affected)

Could you please tell me what cause to have this weird problem "You would have an infinite loop.  
Don't laugh -- sqlserver (MS and Sybase) both have the ability for this to happen. " ?

Thanks for your time

Followup May 28, 2003 - 7pm Central time zone:

process 1 reads and prints to stdout

process 2 converts data into update statement

process 3 updates data

( using a pipe, i had isql | awk | isql ).  If the row is updated via a DELETE+INSERT and there are 
no clustered indexes, you may will be able to reproduce this as the insert tends to go at the end 
of the table.

unfortunately, I don't have either of those products installed (well, actually one won't install on 
any computer I have, no DLLs you know)...

You can see a similar effect by doing this

o create a large table with "known data" -- something that will take a couple of seconds to full 
scan.

o make sure you know the "first" record that will be hit in the full scan and the "last record".  
eg, in Oracle I might:

insert into this_table ( some known value );
insert into this_table select lots of data;
insert into this_table ( some other known value );

Now, make sure one of the columns is a number you can "sum up".

o have two sessions ready to go

o in session one "select sum(of_that_number) from table",  remember that number.  also, remember 
what value row 1 has.

o while that is running, but before it is done do these updates in session 2

  update table set of_that_number = 0 where <predicate to get the first row>;
  update table set of_that_number = <value that was in row1> where <last row>;


o make sure the updates run after the first row was read, but before the last.  Tell me, is the sum 
you get an answer that ever existed in your database at any point in time?  The results can be even 
more amusing if

   o the update to zero is to some number other then zero (say 5)
   o the update to row 1 causes it to move to the end of the table (or 
     even the middle) as it might (might not, they got better at that over time)


you can end up reading row 1 more then once giving you really ambigous results.

Many changes to a block in different SCNs July 1, 2003 - 4am Central time zone

Bookmark | Bottom | Top

Reviewer: Oren from Israel

Hi Tom.

Suppose a specific data block has been changed at SCNs 10, 13, 15 and 20.
And suppose that a query that started at SCN 14 needs a record from this block (when the block is 
already "signed" as SCN 20).

So the block should be reconstructed from the rollback segment that contains the "SCN 15" change 
AND from the rollback segment that contains the "SCN 13" change, right?
Now, how does Oracle know which rollback segments it should read? Does it go over all the existing 
rollback segments and look for that block? (I guess this would be too expensive)

And if the rollback segment that contains the "SCN 13" change has already been overridden, we'll 
get "snapshot too old", right?
But assuming the rollback segment that contains the "SCN 10" change has not been overridden yet, 
how does Oracle know NOT to reconstruct the block from the "SCN 10" change (and to give ora-1555 
instead)?
Is there some kind of a list for each data blcok, containing all the SCNs (or rollback segment 
slots) that changed that block?

Does my question indicate I misunderstand something crucial in multi-versioning :-( ?
Many thanks,
Oren.

Followup July 1, 2003 - 8am Central time zone:

the block header has this information.  the transaction header in the block itself has this 
information.

SP2-0612: Error generating AUTOTRACE report when arraysize is reduced..???? July 1, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from MN, USA

RKPD01> create table test                    
  2  ( x char(2000),
  3    y char(2000),
  4    z char(2000),
  5    a char(1000));

Table created.


RKPD01> begin
  2  for i in 1..10 loop
  3  insert into test values(chr(i), chr(i+1), chr(i+2), chr(i+3));
  4  end loop;
  5  end;
  6  /

PL/SQL procedure successfully completed.

RKPD01> commit;       

Commit complete.

RKPD01> select count(*) from test;


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   SORT (AGGREGATE)
   2    1     TABLE ACCESS (FULL) OF 'TEST'



Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         12  consistent gets
          0  physical reads
          0  redo size
        367  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed


RKPD01> show array         
arraysize 15
RKPD01> select * from test;

10 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'TEST'




Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         12  consistent gets
          0  physical reads
          0  redo size
      71296  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         10  rows processed

RKPD01> set arraysize 10
RKPD01> select * from test;

10 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'TEST'




Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         12  consistent gets
          0  physical reads
          0  redo size
      71296  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         10  rows processed

RKPD01> set arraysize 5
RKPD01> select * from test;
SP2-0612: Error generating AUTOTRACE report

10 rows selected.

SP2-0612: Error generating AUTOTRACE report

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'TEST'



SP2-0612: Error generating AUTOTRACE STATISTICS report
RKPD01> select * from test;
SP2-0612: Error generating AUTOTRACE report

10 rows selected.

SP2-0612: Error generating AUTOTRACE report

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'TEST'



SP2-0612: Error generating AUTOTRACE STATISTICS report
RKPD01> set arraysize 100
RKPD01> select * from test 
  2
RKPD01> /

10 rows selected.
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'TEST'




Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
         12  consistent gets
          0  physical reads
          0  redo size
      71296  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         10  rows processed

RKPD01> set arraysize 5
RKPD01> /
SP2-0612: Error generating AUTOTRACE report

10 rows selected.

SP2-0612: Error generating AUTOTRACE report

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'TEST'



SP2-0612: Error generating AUTOTRACE STATISTICS report 

as you see i m getting this error when i reduce the arraysize to a low value but not when i set it 
sufficiently high.. what is the reason for the same....am i doing something wrong here?? Oracle 
8.1.7

Followup July 1, 2003 - 3pm Central time zone:

looks like a small issue with 8i sqlplus and autotrace with arraysizes below 9.

not a "big deal"

Thanks for the quick response :) July 1, 2003 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from MN, USA

more the consistent gets more the usage of cpu? Thanks. August 9, 2003 - 5pm Central time zone

Bookmark | Bottom | Top

Reviewer: Reader

Followup August 10, 2003 - 2pm Central time zone:

more cpu, more latches (which are locks) more work.

SO, SETTING THE ARRAY SIZE IS IMPORTANT TO MINIMIZE THE CPU USAGE? THANKS. August 10, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: reader

Followup August 10, 2003 - 3pm Central time zone:

IT IS IMPORTANT for performance (less network roundtrips), for scalability (less latching), and for 
resource utilization (less resources used).

yes....

why I am not seeing values......? August 11, 2003 - 9pm Central time zone

Bookmark | Bottom | Top

Reviewer: reader

Please see below. Am I missing something? Why can't I see the statistics? Thanks for your time.

SQL> @$ORACLE_HOME/rdbms/admin/utlxplan
SQL> rem
SQL> rem $Header: utlxplan.sql 29-oct-2001.20:28:58 mzait Exp $ xplainpl.sql
SQL> rem
SQL> Rem Copyright (c) 1988, 2001, Oracle Corporation.  All rights reserved.
SQL> Rem NAME
SQL> REM    UTLXPLAN.SQL
SQL> Rem  FUNCTION
SQL> Rem  NOTES
SQL> Rem  MODIFIED
SQL> Rem        mzait   10/26/01  - add keys and filter predicates to the plan e
SQL> Rem        ddas    05/05/00  - increase length of options column
SQL> Rem        ddas    04/17/00  - add CPU, I/O cost, temp_space columns
SQL> Rem        mzait   02/19/98 -  add distribution method column
SQL> Rem        ddas    05/17/96 -  change search_columns to number
SQL> Rem        achaudhr        07/23/95 -  PTI: Add columns partition_{start, }
SQL> Rem        glumpkin        08/25/94 -  new optimizer fields
SQL> Rem        jcohen  11/05/93 -  merge changes from branch 1.1.710.1 - 9/24
SQL> Rem        jcohen  09/24/93 - #163783 add optimizer column
SQL> Rem        glumpkin        10/25/92 -  Renamed from XPLAINPL.SQL
SQL> Rem        jcohen  05/22/92 - #79645 - set node width to 128 (M_XDBI in ge)
SQL> Rem        rlim    04/29/91 -         change char to varchar2
SQL> Rem   Peeler        10/19/88 - Creation
SQL> Rem
SQL> Rem This is the format for the table that is used by the EXPLAIN PLAN
SQL> Rem statement.     The explain statement requires the presence of this
SQL> Rem table in order to store the descriptions of the row sources.
SQL>
SQL> create table PLAN_TABLE (
  2          statement_id    varchar2(30),
  3          timestamp       date,
  4          remarks         varchar2(80),
  5          operation       varchar2(30),
  6          options         varchar2(255),
  7          object_node     varchar2(128),
  8          object_owner    varchar2(30),
  9          object_name     varchar2(30),
 10          object_instance numeric,
 11          object_type     varchar2(30),
 12          optimizer       varchar2(255),
 13          search_columns  number,
 14          id              numeric,
 15          parent_id       numeric,
 16          position        numeric,
 17          cost            numeric,
 18          cardinality     numeric,
 19          bytes           numeric,
 20          other_tag       varchar2(255),
 21          partition_start varchar2(255),
 22          partition_stop  varchar2(255),
 23          partition_id    numeric,
 24          other           long,
 25          distribution    varchar2(30),
 26          cpu_cost        numeric,
 27          io_cost         numeric,
 28          temp_space      numeric,
 29          access_predicates varchar2(4000),
 30          filter_predicates varchar2(4000));

Table created.

SQL>
SQL> set autotrace traceonly statistics
SQL> select * from t;

6197 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          0  consistent gets
          0  physical reads
          0  redo size
          0  bytes sent via SQL*Net to client
          0  bytes received via SQL*Net from client
          0  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
       6197  rows processed

Followup August 12, 2003 - 8am Central time zone:

you are logged in as sys.

You should never use sys, system, or ANY of the oracle accounts for anything.  they are "ours"

Create your own account to play with.

Why the difference in consistent gets? September 5, 2003 - 5am Central time zone

Bookmark | Bottom | Top

Reviewer: Tony from INDIA

I have two tables, SMALL and BIG

SQL> select count(*) from small;

    COUNT(*)
    ----------
     104

SQL> select count(*) from big;

  COUNT(*)
   ----------
     34486

I issued the query SELECT COUNT(*) FROM small, big (without join) in RBO and got different 
consistent gets and elapse time when the table order was changed. Why the consistent get was 
different when table order changed? Why the consistent get was different for CBO also? Plans are 
given below: Please clear my doubt.

RBO:
-------
SQL> select /*+ rule */ count(*) from small, big;

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=HINT: RULE
   1    0   SORT (AGGREGATE)
   2    1     NESTED LOOPS
   3    2       TABLE ACCESS (FULL) OF 'BIG'
   4    2       TABLE ACCESS (FULL) OF 'SMALL'



Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     172822  consistent gets
          0  physical reads
          0  redo size
        382  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed


SQL> select /*+ rule */ count(*) from big,small;

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=HINT: RULE
   1    0   SORT (AGGREGATE)
   2    1     NESTED LOOPS
   3    2       TABLE ACCESS (FULL) OF 'SMALL'
   4    2       TABLE ACCESS (FULL) OF 'BIG'


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
      40773  consistent gets
          0  physical reads
          0  redo size
        382  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

CBO:
------

SQL> select count(*) from small, big;

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=4058 Card=1)
   1    0   SORT (AGGREGATE)
   2    1     MERGE JOIN (CARTESIAN) (Cost=4058 Card=3586544)
   3    2       TABLE ACCESS (FULL) OF 'SMALL' (Cost=2 Card=104)
   4    2       BUFFER (SORT) (Cost=4056 Card=34486)
   5    4         TABLE ACCESS (FULL) OF 'BIG' (Cost=39 Card=34486)

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        397  consistent gets
          0  physical reads
          0  redo size
        382  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
          1  rows processed

Followup September 5, 2003 - 3pm Central time zone:

the plan is:

  for x in ( select * from T1 )
    for y in ( select * from T2 )
        output x+y
    end loop
  end loop

from the rbo.  The formula for the consistent gets will then be:

number_of_blocks_in_t1 + number_of_rows_in_t1 * number_of_blocks_in_t2

so, here are some sample numbers to play with:

ops$tkyte@ORA920> declare
  2      l_small_rows   number := 104;
  3      l_big_rows     number := 34486;
  4
  5      type numArray is table of number;
  6
  7      l_small_blocks numArray := numArray( 1,     2,    4,    8,   16 );
  8      l_big_blocks   numArray := numArray( 300, 600, 1200, 2400, 4800 );
  9
 10      l_small_drives number;
 11      l_big_drives number;
 12  begin
 13      for i in 1 .. l_small_blocks.count
 14      loop
 15          for j in 1 .. l_big_blocks.count
 16          loop
 17              dbms_output.put_line( 'small blocks = ' || l_small_blocks(i) ||
 18                                    ' big blocks = ' || l_big_blocks(i) );
 19
 20              l_small_drives := (l_small_blocks(i) + l_small_rows*l_big_blocks(j));
 21              l_big_drives   := (l_big_blocks(i) + l_big_rows*l_small_blocks(j));
 22              dbms_output.put_line( 'small drives = ' || l_small_drives ||
 23                                    ' big drives = ' || l_big_drives ||
 24                                    ' difference = ' || abs( l_big_drives-l_small_drives ) );
 25          end loop;
 26      end loop;
 27  end;
 28  /
small blocks = 1 big blocks = 300
small drives = 31201 big drives = 34786 difference = 3585
small blocks = 1 big blocks = 300
small drives = 62401 big drives = 69272 difference = 6871
small blocks = 1 big blocks = 300
small drives = 124801 big drives = 138244 difference = 13443
small blocks = 1 big blocks = 300
small drives = 249601 big drives = 276188 difference = 26587
small blocks = 1 big blocks = 300
small drives = 499201 big drives = 552076 difference = 52875
small blocks = 2 big blocks = 600
small drives = 31202 big drives = 35086 difference = 3884
small blocks = 2 big blocks = 600
small drives = 62402 big drives = 69572 difference = 7170
small blocks = 2 big blocks = 600
small drives = 124802 big drives = 138544 difference = 13742
small blocks = 2 big blocks = 600
small drives = 249602 big drives = 276488 difference = 26886
small blocks = 2 big blocks = 600
small drives = 499202 big drives = 552376 difference = 53174
small blocks = 4 big blocks = 1200
small drives = 31204 big drives = 35686 difference = 4482
small blocks = 4 big blocks = 1200
small drives = 62404 big drives = 70172 difference = 7768
small blocks = 4 big blocks = 1200
small drives = 124804 big drives = 139144 difference = 14340
small blocks = 4 big blocks = 1200
small drives = 249604 big drives = 277088 difference = 27484
small blocks = 4 big blocks = 1200
small drives = 499204 big drives = 552976 difference = 53772
small blocks = 8 big blocks = 2400
small drives = 31208 big drives = 36886 difference = 5678
small blocks = 8 big blocks = 2400
small drives = 62408 big drives = 71372 difference = 8964
small blocks = 8 big blocks = 2400
small drives = 124808 big drives = 140344 difference = 15536
small blocks = 8 big blocks = 2400
small drives = 249608 big drives = 278288 difference = 28680
small blocks = 8 big blocks = 2400
small drives = 499208 big drives = 554176 difference = 54968
small blocks = 16 big blocks = 4800
small drives = 31216 big drives = 39286 difference = 8070
small blocks = 16 big blocks = 4800
small drives = 62416 big drives = 73772 difference = 11356
small blocks = 16 big blocks = 4800
small drives = 124816 big drives = 142744 difference = 17928
small blocks = 16 big blocks = 4800
small drives = 249616 big drives = 280688 difference = 31072
small blocks = 16 big blocks = 4800
small drives = 499216 big drives = 556576 difference = 57360
 
PL/SQL procedure successfully completed.
 


As you can see -- the number of consistent gets for that plan is radically affected by the NUMBER 
OF BLOCKS in the table.  


the cbo plan -- it is not even remotely comparable.  but it basically read BIG once, sorted it and 
read it from temp (temp reads are NOT consistent gets)

Client Arraysize won't affect the consistent gets? September 6, 2003 - 3am Central time zone

Bookmark | Bottom | Top

Reviewer: Tony from India

Thanks for my question on "Why the difference in consistent gets?". You have given a new formula to 
calculate consistent gets. But you have mentioned on many occations that consistent get (logical 
read, if I'm not wrong) depends on client array size, in my case SQL Plus array size. Please 
clarify.

Followup September 6, 2003 - 9am Central time zone:

clarity is in the original answer and followups??

that was the crux of the first answer on this page.  do you have a specific question?

More consistent gets Less elapsed time. Why? September 6, 2003 - 3am Central time zone

Bookmark | Bottom | Top

Reviewer: Tony from India

[Continuation of the above question....]

When consistent get is more, elapsed time is less. And
When consistent get is less, elapsed time is more for the above query plan for the question on "Why 
the difference in consistent gets?"

Why elapsed time is inversely proportional to consistent gets in my case?. Please clear my doubt.

Followup September 6, 2003 - 9am Central time zone:

give me a "for example" to work with at least.

Why elapsed time is high when consistent get is low? September 8, 2003 - 1am Central time zone

Bookmark | Bottom | Top

Reviewer: Tony from India

Why elapsed time is high when consistent get is low and elapsed time is low when consistent get is 
high?

For example:

SQL>  select count(*) from small;

  COUNT(*)
   ----------
     104

SQL> select count(*) from big;

  COUNT(*)
   ----------
   34486

SQL> select /*+ rule */ count(*) from small, big;

Elapsed: 00:00:01.08

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=HINT: RULE
   1    0   SORT (AGGREGATE)
   2    1     NESTED LOOPS
   3    2       TABLE ACCESS (FULL) OF 'BIG'
   4    2       TABLE ACCESS (FULL) OF 'SMALL'

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     172822  consistent gets
          0  physical reads
          0  redo size
        382  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

SQL>  select /*+ rule */ count(*) from big, small;

Elapsed: 00:00:02.00

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=HINT: RULE
   1    0   SORT (AGGREGATE)
   2    1     NESTED LOOPS
   3    2       TABLE ACCESS (FULL) OF 'SMALL'
   4    2       TABLE ACCESS (FULL) OF 'BIG'


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
      40773  consistent gets
          0  physical reads
          0  redo size
        382  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

Followup September 8, 2003 - 11am Central time zone:

use tkprof.  

first, the autotrace stuff takes time and is counted.

second, you are seeing the total elapsed time -- including sqlplus "screen paint", network, etc

use tkprof to find out for real what is taking place.

Outstanding September 10, 2003 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: Jim from MA

Tom, you're still the one!  This thread is rockin!

Setting arraysize for an oci app October 6, 2003 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: Ma$e

Tom:

Following up on the arraysize discussion.

I have an app that connects to the oracle d/b via oci calls. How can I test the effect of arraysize 
for this app ?
Does arraysize even come into consideration for an oci app ?

Thanks again for your help.

Ma$e

Followup October 6, 2003 - 11am Central time zone:

yes, it makes sense/comes into consideration.

Look at the OCIDefine functions -- valuep can be an array.

Can't find OCIDefine function calls October 6, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: Ma$e

Tom:

I searched for OCIDefine function calls in the source code library and found no hits. Chances are 
pretty good that we are using the OCI 7 calls. I tried looking for its equivalent name from the 
"old days" and did not get anywhere.

Would you know what this function call was known in its previous incarnation ?

Thanks in advance.

Ma$e

Followup October 6, 2003 - 2pm Central time zone:

ocidefinebypos

for example in 8 and above

see $ORACLE_HOME/rdbms/demo/oci06.c for "old style" array fetch example

Sorry I also forgot to ask another Q. October 6, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: Ma$e

How can the end user set the array size for an oci app without access to the source code?

Or in other words:

We are s/w developpers: How can we give the option to the user of setting the value for this to the 
end user?

Can it be set via a logon trigger or some other means, for the user that we use to establish an oci 
connection to the d/b with ?

Thanks
Ma$e

Followup October 6, 2003 - 2pm Central time zone:

you, as developers, would have a parameter file with configuration options that the end user could 
edit.


it is a CLIENT thing, cannot be done in the server.

What was I thinking.... October 6, 2003 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom:

Absolutely correct. The arraysize setting is a client side setting. What was I thinking....

So let me understand. The application would open the parameter file and assign a value to an 
internal variable which we would pass to the oci function call as our array size (simillar to the 
init.ora)!!! You can tell I'm a DBA.

Any other possible way?

Since this is unlikely to happen unless I can demonstrate to the developpers (yet another battle 
between a DBA and the other guys) that this will benefit the end user in a tangible form that the 
developpers can understand. So I need to be able to demonstrate without any shadow of doubt of the 
benefits for a user settable array size.


I found a "C" programming example on metalink:

http://metalink.oracle.com/metalink/plsql/ml2_documents.showFrameDocument?p_database_id=NOT&p_id=738
23.1

However this example does not demonstrate "roi" for different arraysize that the developpers can 
understand.

Can you help (please wear both hats - developper and DBA) ?


Thanks

Ma$e

Followup October 6, 2003 - 3pm Central time zone:

the ROI?

I do that all of the time with runstats and sqlplus. I did it in chapter 2 of my new book
"Effective Oracle By Design". I do it in my "whats wrong presentation" i give from time to time.
Here is short excerpt showing the effects of array fetching on logical IO's from the book:

<quote>
Array Size Effects

The array size is the number of rows fetched (or sent, in the case of inserts, updates, and
deletes) by the server at a time. It can have a dramatic effect on performance.

To demonstrate the effects of array size, we'll run the same query a couple of times and look at
just the consistent get differences between runs:

ops$tkyte@ORA920> create table t as select * from all_objects;
Table created.

ops$tkyte@ORA920> set autotrace traceonly statistics;

ops$tkyte@ORA920> set arraysize 2
ops$tkyte@ORA920> select * from t;
29352 rows selected.

Statistics
----------------------------------------------------------
14889 consistent gets

Note how one half of 29,352 (rows fetched) is very close to 14,889, the number of consistent gets.
Every row we fetched from the server actually caused it to send two rows back. So, for every two
rows of data, we needed to do a logical I/O to get the data. Oracle got a block, took two rows from
it, and sent it to SQL*Plus. Then SQL*Plus asked for the next two rows, and Oracle got that block
again or got the next block, if we had already fetched the data, and returned the next two rows,
and so on.

Next, let's increase the array size:

ops$tkyte@ORA920> set arraysize 5
ops$tkyte@ORA920> select * from t;
29352 rows selected.

Statistics
----------------------------------------------------------
6173 consistent gets

Now, 29,352 divided by 5 is about 5,871, and that would be the least amount of consistent gets we
would be able to achieve (the actual observed number of consistent gets is slightly higher). All
that means is sometimes in order to get two rows, we needed to get two blocks: we got the last row
from one block and the first row from the next block.

Let's increase the array size again:

ops$tkyte@ORA920> set arraysize 10
ops$tkyte@ORA920> select * from t;
29352 rows selected.

Statistics
----------------------------------------------------------
3285 consistent gets

ops$tkyte@ORA920> set arraysize 15
ops$tkyte@ORA920> select * from t;
29352 rows selected.

Statistics
----------------------------------------------------------
2333 consistent gets

ops$tkyte@ORA920> set arraysize 100
ops$tkyte@ORA920> select * from t;
29352 rows selected.

Statistics
----------------------------------------------------------
693 consistent gets

ops$tkyte@ORA920> set arraysize 5000
ops$tkyte@ORA920> select * from t;
29352 rows selected.

Statistics
----------------------------------------------------------
410 consistent gets

ops$tkyte@ORA920> set autotrace off

As you can see, as the array size goes up, the number of consistent gets goes down. So, does that
mean you should set your array size to 5,000, as in this last test? Absolutely not.

If you notice, the overall number of consistent gets has not dropped dramatically between array
sizes of 100 and 5,000. However, the amount of RAM needed on the client and server has gone up with
the increased array size. The client must now be able to cache 5,000 rows. Not only that, but it
makes our performance look choppy: The server works really hard and fast to get 5,000 rows, then
the client works really hard and fast to process 5,000 rows, then the server, then the client, and
so on. It would be better to have more of a stream of information flowing: Ask for 100 rows, get
100 rows, ask for 100, process 100, and so on. That way, both the client and server are more or
less continuously processing data, rather than the processing occurring in small bursts.
</quote>

and to put it in terms they (coders) will understand -- array fetching goes faster. On:

http://asktom.oracle.com/~tkyte/flat/index.html

I have a pro*c program that accepts an array size as input.  Look at the difference between single 
row fetching:

[tkyte@tkyte-pc array_flat]$ time ./array_flat userid=big_table/big_table 'sqlstmt=select * from 
big_table where rownum <= 100000' arraysize=1 > /dev/null
 
Connected to ORACLE as user: big_table/big_table
 
Unloading 'select * from big_table where rownum <= 100000'
Array size = 1
ID,OWNER,OBJECT_NAME,SUBOBJECT_NAME,OBJECT_ID,DATA_OBJECT_ID,OBJECT_TYPE,CREATED,LAST_DDL_TIME,TIMES
TAMP,STATUS,TEMPORARY,GENERATED,SECONDARY
100000 rows extracted
 
real    1m10.250s
user    0m30.852s
sys     0m2.365s
[tkyte@tkyte-pc array_flat]$ !!
time ./array_flat userid=big_table/big_table 'sqlstmt=select * from big_table where rownum <= 
100000' arraysize=1 > /dev/null
 
Connected to ORACLE as user: big_table/big_table
 
Unloading 'select * from big_table where rownum <= 100000'
Array size = 1
ID,OWNER,OBJECT_NAME,SUBOBJECT_NAME,OBJECT_ID,DATA_OBJECT_ID,OBJECT_TYPE,CREATED,LAST_DDL_TIME,TIMES
TAMP,STATUS,TEMPORARY,GENERATED,SECONDARY
100000 rows extracted
 
real    1m13.662s
user    0m20.514s
sys     0m0.910s

versus array fetching:


[tkyte@tkyte-pc array_flat]$ time ./array_flat userid=big_table/big_table 'sqlstmt=select * from 
big_table where rownum <= 100000' arraysize=100 > /dev/null                                         
                                                                                                    
               
Connected to ORACLE as user: big_table/big_table
 
Unloading 'select * from big_table where rownum <= 100000'
Array size = 100
ID,OWNER,OBJECT_NAME,SUBOBJECT_NAME,OBJECT_ID,DATA_OBJECT_ID,OBJECT_TYPE,CREATED,LAST_DDL_TIME,TIMES
TAMP,STATUS,TEMPORARY,GENERATED,SECONDARY
100000 rows extracted
 
real    0m6.145s
user    0m3.816s
sys     0m0.211s
[tkyte@tkyte-pc array_flat]$
[tkyte@tkyte-pc array_flat]$ !!
time ./array_flat userid=big_table/big_table 'sqlstmt=select * from big_table where rownum <= 
100000' arraysize=100 > /dev/null
 
Connected to ORACLE as user: big_table/big_table
 
Unloading 'select * from big_table where rownum <= 100000'
Array size = 100
ID,OWNER,OBJECT_NAME,SUBOBJECT_NAME,OBJECT_ID,DATA_OBJECT_ID,OBJECT_TYPE,CREATED,LAST_DDL_TIME,TIMES
TAMP,STATUS,TEMPORARY,GENERATED,SECONDARY
100000 rows extracted
 
real    0m5.491s
user    0m3.795s
sys     0m0.230s
[tkyte@tkyte-pc array_flat]$


that should raise an eyebrow or two -- hey, you mean we can make our code run faster with array 
fetching!

How can I thank you.... October 7, 2003 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: Ma$e

Tom:

There is so much value in your response. I can't than you enough for this. I'm still digesting your 
response.

One thing for sure I am going to read your book first before asking you questions that have been 
answered there. I feel bad. I do own both your books. However, it's your comments that preceede or 
follow your quotes that are invaluble.

Thanks one again. May God bless you.

Ma$e

Clarification October 27, 2003 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

In your response to one of the reader's queries, you had mentioned a case where a table has 100 
rows across 10 blocks with an array size of 10 and the total consistent gets is 20. Can you explain 
me how you arrived at 20. My calculation is as follows :

100 rows
10 rows/block
Array size 10

First fetch (1 consistent get) - 10 rows/0 rows left
Second fetch (2 consistent gets) - 10 rows/0 rows left
....
Tenth fetch (10 consistent gets) - 10 rows/0 rows left

My question is that since the number of rows/block is the same as the array size in the above 
example, it should do 10 consistent gets exactly. Can you explain me how you arrived at 20 
consistent gets instead 10.Is Oracle revisiting blocks again or how you have arrived at 20 
consistent block gets?

Followup October 27, 2003 - 1pm Central time zone:

it picks up where it left off -- on the last block.  you know it had 10 rows, I know it had 10 
rows, but the database just said "get me 10 and remember where we stopped".  it didn't look 
necessarily to see that "hey, there isn't an 11th row, we should pick up on the NEXT block next 
time"

with the full scan extent map reading -- it is actually "more" then 20

consistent gets and db block gets October 30, 2003 - 7am Central time zone

Bookmark | Bottom | Top

Reviewer: Ivan from China

Tom - 

  I am confused about db block gets and consistent gets. 

  1. db block gets can only be concerned for full table    
     scan. Why ?
     Why it is not related to the data we selected ? 
    

  2. Consistent gets can be two things( Pleas clarify ) 
     i. from buffer cache if buffer cache has the data. For 
        example,
        Person A selected the table T. After, Person B 
        selected the table T again. 
        So Person B can get the data in memory. This can be 
        called consistent gets.

     ii. from rollback segment.
         Person A selected a row in table A. But that row 
         was updated by Person B.
         So Person A needs to get the data from rollback 
         segment. It will cause 
         two things to happen. 
         Frist ,  Physical reads  - from rollback segmet 
         into buffer cache. 
         Second,  consistent gets - reading the data from 
         buffer cache.    
        
    
Ivan

Followup October 30, 2003 - 7am Central time zone:

1) it is not concerned only for full table scans.  It is a mode blocks are gotten in in many cases.

blocks are either gotten in current mode (db block gets) or consistent read mode (consistent gets). 
 We get the blocks in the mode we need them.  If you do not know what "consistent gets" means -- 
please see the concepts guide and read the concurrency control sections -- and learn about 
multi-versioning and read consistency.

2) consistent gets ALWAYS come from the buffer cache.  

rollback is buffered in the cache.  

but yes, a consistent get may well involved reading rollback data for the affected segment.

meaning of current mode October 30, 2003 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: Ivan from China

Tom 
    I am not still clear about the current mode. 
 
    Please clarify whether I am correct . 

    Assume every block has a time in it. It will 
    record the most updated time if it is modified.

    For example, at time t, Person A select * from t.
    
    Consisten mode means getting the block wiht time <= t.

    If we need  to get a block with time > t, we need to    
    roll it back to get the consistent data.

    Since we are performing full table scan, we need 
    currnet mode(the block with most updated time) to tell
    us where is the segment header.

Followup October 30, 2003 - 11am Central time zone:

current mode = get the block as it exists right now.

consistent mode = get the block as it existed at some point in the past.

Total Reads vs Memory reads November 3, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

In the book "Performance Tuning Tips and Techniques(TUSC)", Rich indicates that the Total reads = 
"Consistent Gets" + "Db Block Gets" and the Memory reads = Total Reads - Disk Reads. My 
understanding is Total Reads = Logical I/O + Physical I/O where Logical I/O = Consistent Gets + Db 
Block Gets. 
    a. Can you please clarify me as to what he means by Memory reads.
    b. Since Physical I/O is initiated by a Logical I/O, the memory reads are only those reads that 
do not include the physical I/o part of it.

Please clarify

Followup November 4, 2003 - 7am Central time zone:

that is wrong.

A physical IO performed in order to read a block into the cache will be followed by a subsequent 
logical IO to get the block out of the buffer cache.  We can observe this via something like this:

big_table@ORA920LAP> alter tablespace users offline;

Tablespace altered.

big_table@ORA920LAP> alter tablespace users online;

Tablespace altered.

big_table@ORA920LAP>
big_table@ORA920LAP> select blocks from user_tables where table_name = 'BIG_TABLE';

    BLOCKS
----------
     13804

big_table@ORA920LAP> set autotrace on
big_table@ORA920LAP> select /*+ FULL(big_table) */ count(*) from big_table;

  COUNT(*)
----------
   1000000


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1329 Card=1)
   1    0   SORT (AGGREGATE)
   2    1     TABLE ACCESS (FULL) OF 'BIG_TABLE' (Cost=1329 Card=998400)




Statistics
----------------------------------------------------------
        458  recursive calls
          0  db block gets
      13769  consistent gets
      13647  physical reads
          0  redo size
        379  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          6  sorts (memory)
          0  sorts (disk)
          1  rows processed

big_table@ORA920LAP> set autotrace off


The alter's invalidate the buffers in the cache, ensuring a PIO.  As you can see, if we use the 
formula:

memory reads = total reads - disk reads

we'd get it wrong.  memory reads = consistent gets+db block gets.


I don't know what he means by memory reads, sometimes information taken out of context is hard to 
interpret.  I don't know if it is a "mistake" or just taken out of context.

There are other nuances to consider as well -- direct IO done to TEMP (reads and writes) will 
appear as PIO but not as LIO -- that is, you can have a query that does more PIO then LIO (rare, 
but can happen) due to this.


I've been asked from time to time to put out "challenges".  As I'm pressed for time this morning -- 
I'll try one.

In the following output -- we always process 10,000 or 20,000 rows.  Only difference is a sort to 
disk.  Soooooo tell me -- why the big differences in LIO's and PIO's between the two???

but it shows that the above formulas are not really accurate. some PIO's are followed by LIO's -- 
this is the "normal" case.  some PIO's (direct reads) are not (hash joins, sorts may well result in 
this)

big_table@ORA920LAP> alter session set workarea_size_policy=manual;

Session altered.

big_table@ORA920LAP> alter session set sort_area_size = 65537;

Session altered.

big_table@ORA920LAP>
big_table@ORA920LAP> set autotrace  traceonly
big_table@ORA920LAP> select * from big_table where rownum <= 10000;

10000 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1329 Card=10000 Bytes=940000)
   1    0   COUNT (STOPKEY)
   2    1     TABLE ACCESS (FULL) OF 'BIG_TABLE' (Cost=1329 Card=998400 Bytes=93849600)




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        794  consistent gets
        134  physical reads
          0  redo size
     773075  bytes sent via SQL*Net to client
       7825  bytes received via SQL*Net from client
        668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10000  rows processed

big_table@ORA920LAP> select * from big_table where rownum <= 10000 order by 1,2,3,4,5,6,7,8;

10000 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=82617 Card=10000 Bytes=940000)
   1    0   SORT (ORDER BY) (Cost=82617 Card=10000 Bytes=940000)
   2    1     COUNT (STOPKEY)
   3    2       TABLE ACCESS (FULL) OF 'BIG_TABLE' (Cost=1329 Card=998400 Bytes=93849600)




Statistics
----------------------------------------------------------
          0  recursive calls
          2  db block gets
        138  consistent gets
        257  physical reads
          0  redo size
     773086  bytes sent via SQL*Net to client
       7825  bytes received via SQL*Net from client
        668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          1  sorts (disk)
      10000  rows processed

big_table@ORA920LAP> select * from big_table where rownum <= 20000;

20000 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1329 Card=20000 Bytes=1880000)
   1    0   COUNT (STOPKEY)
   2    1     TABLE ACCESS (FULL) OF 'BIG_TABLE' (Cost=1329 Card=998400 Bytes=93849600)




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
       1591  consistent gets
        257  physical reads
          0  redo size
    1570644  bytes sent via SQL*Net to client
      15162  bytes received via SQL*Net from client
       1335  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      20000  rows processed

big_table@ORA920LAP> select * from big_table where rownum <= 20000 order by 1,2,3,4,5,6,7,8;

20000 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=82617 Card=20000 Bytes=1880000)
   1    0   SORT (ORDER BY) (Cost=82617 Card=20000 Bytes=1880000)
   2    1     COUNT (STOPKEY)
   3    2       TABLE ACCESS (FULL) OF 'BIG_TABLE' (Cost=1329 Card=998400 Bytes=93849600)




Statistics
----------------------------------------------------------
          0  recursive calls
          3  db block gets
        279  consistent gets
        536  physical reads
          0  redo size
    1570607  bytes sent via SQL*Net to client
      15162  bytes received via SQL*Net from client
       1335  SQL*Net roundtrips to/from client
          0  sorts (memory)
          1  sorts (disk)
      20000  rows processed

big_table@ORA920LAP> set autotrace off

Consistent Gets November 3, 2003 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

In one of the discussions for Consistent gets, you mention that 
"it took 10k consistent gets for Y -- versus 7 for X -- both queries return the same data -- it is 
just that for the first fetch from Y we had to "undo" the 10,000 updates that happened between the 
OPEN of Y and the fetch."

You mention that we need to reduce the number of "consistent gets" in order to fine tune a query. 
In this example, there is no problem with our query because it just reads one record from table. 
However, the "consistent gets" figure is so huge because of the UPDATE and the way Oracle uses to 
reconstruct a consistent image and so we might try to determine as to why this query made such huge 
consistent gets to return just 1 row. Given this, how do we differentiate between :
 a. A really bad query with a large number of "consistent gets"
 b. A single-row query that shows a large number of "consistent gets" just because of an UPDATE 
that happened between the OPEN and FETCH (as in the example above).
 c. What other trace result might indicate that there is no problem with the query, but it is just 
because of the UPDATE that happened in between and how can we reduce the consistent gets in the 
above case

Thanks

Followup November 4, 2003 - 7am Central time zone:

this sort of shows why the outline I had in chapter 10 of Expert one on one Oracle is so important.

FIRST 

a) tune in isolation. get that app really tuned in single user mode.  here you are isolated from 
external events like this

THEN

b) tune in multi-user situations -- test to scale.  then side effects from other users will become 
apparent (locking, blocking, things like this)

and lastly

c) tune the "system" after A and B are done.


In an OLTP environment -- for the query to be really affected by this consistent read mechanism 
means you are running a really really long query in a system with tons of updates.  Normally the 
queries would be short and sweet -- the odds of having to undo 10,000 updates would be exceedingly 
small.

In a read mostly environment -- it won't happen of course.

So, it is more of a "mixed workload" environment and would only be suspected in the execution of 
long running queries against active tables (eg: your KNOWLEDGE of the charactertistics of the 
system will be crucial in looking at information)

Colocated vs Disorganized November 5, 2003 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ USA

Tom,

I read your book Expert One-on-One Oracle on the topic on indexes where you explain about how
physical placement of data affects consistent gets. I have the following understanding and I have
the following questions :

Understanding :

Non-SQL*Plus environment, processing 1 row at a time, data colocated :

Assume that we want to retreive 100 rows ie x between 1 and 100 and that the index and table blocks
are not in buffer cache. Oracle reads the index block which constitutes 1 physical I/O (in turn a 1
logical I/O). Since each rowid in the index block points to the same data block, Oracle will read
the first data block from disk and returns the first row which constitutes 1 physical I/O (in turn
1 logical I/O). It gets the next record from the index block and since the data is colocated it
finds the next record in the same table block as the first one and so this time it performs just 1
logical I/O. Continuing in this manner, Oracle will at the best case perform N+1 logical reads (1
index block read and N table block read) for retreiving N colocated rows

Questions :

1. Is my understanding correct
2. In that example in your book, Oracle performs 2909 logical I/Os for getting 20000 rows in
colocated table. Can you explain me how Oracle performed 2909 consistent reads?
3. The only difference in the case of colocated and disorganised is that Oracle may have to perform
lot of physical reads in the case of disorganized table because the index entry may not point to a
row in the same physical block. However, colocated table will perform only logical I/O since the
data are closer to each other. Is it correct?
4. Is the autotrace got using SQL*PLus which uses an arraysize of 15?

Followup November 5, 2003 - 6pm Central time zone:

1) well, it'll get the index block to find the table block, and then get the index block to find 
the next table block, and so on.

ops$tkyte@ORA920LAP> set echo on
ops$tkyte@ORA920LAP> /*
DOC>drop table t;
DOC>create table t ( x int, y int );
DOC>create index t_idx on t(x);
DOC>insert into t select rownum , rownum from all_objects;
DOC>*/
ops$tkyte@ORA920LAP>
ops$tkyte@ORA920LAP> alter session set sql_trace=true;

Session altered.

ops$tkyte@ORA920LAP> begin
  2          for x in ( select * from t where x < 100 )
  3          loop
  4                  null;
  5          end loop;
  6  end;
  7  /

PL/SQL procedure successfully completed.

ops$tkyte@ORA920LAP> begin
  2          for x in ( select * from t where x < 200 )
  3          loop
  4                  null;
  5          end loop;
  6  end;
  7  /

PL/SQL procedure successfully completed.

select * from t where x < 100

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch      100      0.00       0.00          0        200          0          99
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total      102      0.00       0.00          0        200          0          99

Rows     Row Source Operation
-------  ---------------------------------------------------
     99  TABLE ACCESS BY INDEX ROWID T (cr=200 r=0 w=0 time=1748 us)
     99   INDEX RANGE SCAN T_IDX (cr=101 r=0 w=0 time=984 us)(object id 41082)
********************************************************************************
select * from t where x < 200

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch      200      0.00       0.03          0        400          0         199
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total      202      0.00       0.03          0        400          0         199

Rows     Row Source Operation
-------  ---------------------------------------------------
    199  TABLE ACCESS BY INDEX ROWID T (cr=400 r=0 w=0 time=29098 us)
    199   INDEX RANGE SCAN T_IDX (cr=201 r=0 w=0 time=27490 us)(object id 41082)


so, 100 rows -- 200 consistent gets.  200 rows -- 400 consistent gets.  2 consistent gets per 
row....


2) well -- it is a matter of the number of rows per block, the array size and such.  It just "was" 
in this case.  do that test case and play with the arraysize - set it up and watch that go down. 
set it down and watch it go up.

3) no, no it isn't.  you need to look closer at the example.  look at the consistent gets between 
colocated and disorganized.  It is all about the LOGICAL IO, even more so then physical.  they may 
well perform the same amount of physical io.

4) yes...

N + 1 gets November 6, 2003 - 7am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

Then could you pls. explain me under what situations we would get a best case of N + 1 gets to 
retreive N rows. Is it that the best case will always be 2 * N?

Help November 6, 2003 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

If Oracle is anyway going to do 2 consistent gets per row even if the data is colocated, then can 
you explain me what difference it makes if the data is colocated or disorganized. The only thing if 
the data is colocated is that the table block will already be in the buffer cache. But if the data 
is disorganized the index entry may point to the table block not in the buffer cache and so Oracle 
will perform a physical I/O and then a logical I/O. (So an extra physical I/O). If that is the 
case, why so much difference between consistent gets if the data is colocated and disorganized. In 
that case, the difference should only be in the physical reads and not "consistent gets" (since 
consistent gets is performed in both cases to access the block from buffer cache). Guess, I am 
still not understanding clearly what consistent gets means. Pls explain
Does consistent gets increase each time the same block in the buffer cache is visited again or only 
if a new block is brought over into the buffer cache?

Consistent Gets - Misleading November 6, 2003 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

I have a table called LOT_LEVEL_POSITION in our development environment and it has a non-unique 
index LOT1 on 3 columns 
(POSITION_ID,SECURITY_ALIAS,LOT_NUMBER). There are around 63191 rows in that table in our 
development database and around 53214898 on our production m/c. I ran the following 2 queries on 
the development database and here are the autotrace output.

select position_id,security_alias,lot_number,orig_lot_number,update_date,update_source
  from holdingdbo.lot_level_position llp
 where 1 < (select count(*)
              from holdingdbo.lot_level_position i
             where i.position_id = llp.position_id
               and i.security_alias = llp.security_alias
               and i.lot_number = llp.lot_number
)

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     116325  consistent gets
         36  physical reads
          0  redo size
     377716  bytes sent via SQL*Net to client
      22640  bytes received via SQL*Net from client
        607  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
       9080  rows processed

select position_id,security_alias,lot_number,orig_lot_number,update_date,update_source
  from holdingdbo.lot_level_position llp
 where (position_id,security_alias,lot_number) in (
select  position_id,security_alias,lot_number
  from holdingdbo.lot_level_position
 group by position_id,security_alias,lot_number
 having count(*) > 1
)

Statistics
----------------------------------------------------------
          0  recursive calls
          6  db block gets
       3064  consistent gets
       3195  physical reads
          0  redo size
     377716  bytes sent via SQL*Net to client
      22640  bytes received via SQL*Net from client
        607  SQL*Net roundtrips to/from client
          0  sorts (memory)
          1  sorts (disk)
       9080  rows processed

Based on the above results, I extrapolated the "consistent gets" (which was a guess on my part) to 
my production database volume. Surprisingly, the first query which showed more consistent gets on 
my development database ran more quickly on the production database very quickly and the second 
query which showed less consistent gets ran more slowly and took almost 00:59:3541.73 to complete.

1. Is my extrapolation approach correct. If not how can I simulate this on my development 
environment. I want to mimic my production volumes.
2. You insist on reducing logical I/O and so is "consistent gets" alone reliable or any other 
factors to be considered (such as disk reads, db block gets, sorts etc)
3. In my second query, why my physical reads greater than logical reads (3195 > 3064 + 6)
4. Why my first query was faster on production database even though it showed a large number of 
"consistent gets" on my development database based on which I did my extrapolation
5. What other tuning ratios I need to use and any other tuning tricks/tips/figures I need to look 
for in my trace.
5. Is cost figures reliable and can I assume queries with lower costs will be faster and any 
formula I can use as to why Oracle arrived at "x number of consistent gets"

Followup November 6, 2003 - 5pm Central time zone:

queries scale in many different ways.

there are queries that have the same number of CG's (consistent gets) regardless of the volume of 
data (eg: select * from t where pk = :x)

there are queries that will use N*CG's -- where N is some multiple of the data.  So, if you have 
100,000 rows, it'll do 1*CG, if you have 200,000 rows it'll do 200,000 rows

There are other queries that will use exp(CG) meaning -- it'll get worse and worse and worse in a 
linear/exponetial fashion as the volume increases.

N+1 best case vs 2*N November 6, 2003 - 10pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

In one of your discussions, you mentioned that the best case consistent reads is N+1 and worst case 
is 2*N. Also, in the earlier example, you mentioned that Oracle will perform 2*(number of rows) 
"consistent gets" when the rows are accessed using the index-then-table approach. Can you let me 
know when the N+1 consistent gets (the best case) occur

Followup November 7, 2003 - 8am Central time zone:

i didn't mention that.

someone else did.

i showed it to be "not correct"

Consistent Gets November 6, 2003 - 10pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

You mentioned that the number of CGs is data volume dependent. ie. queries that may be N*CGs or 
exponential. So how do we rely on CGs and how do we write queries that are data volume independent

Followup November 7, 2003 - 8am Central time zone:

you cannot -- it is the nature of the query itself!!  it is a characteristic of the underlying data 
and how it works.

Ok, lets say you have a table:

create table emp ( empno int primary key, ename varchar2(25), deptno int );


very simple.  The rules are:

a) there are never more then 100 depts in any company, EVER.  data is somewhat distributed equally 
over deptnos 

b) empno is unique

c) we know NOTHING about ename really


Well, the query:

select * from emp where empno = :x

will scale with pretty much the SAME number of CG's regardless of the number of rows.  There will 
be 2/4 LIO's against an index followed by a table access by index rowid.  You'll have between 3/5 
LIO's for that query regardless of whether you have 10,000 or 10,000,000,000 rows.


The query:

select * from emp where deptno = :x

will scale somewhat linearly with respect to the volume of data.  In general, the number of rows 
returned will be total_number_of_rows_in_table/100.  If you have 10,000 rows, you would expect 100 
returned value.  10,000,000,000 rows -- 100,000,000 rows.  the number of CG's will go up in a 
somewhat predicable fashion as the number of rows increases.


The query

select * from emp where ename = :x

is a sort of an "unpredicable" query.  Say :x = 'SMITH'.  Well, That might return 1 row, all rows, 
most rows, some rows, many rows from the table (smith is a very popular name).  It could be that 
SMITH is not a popular name in small companies located in California.  It could be that SMITH is 
the *most* popular name for small companies located in Texas.  So, this query -- even with small 
tables could return wildly different CG's in different implementations -- due solely to data skew.

does that make sense?

Now, consider 

select ..
  from t1, t2, t3, t4, t5 ....
 where ......


and you can see, it gets harder to generalize.  IF YOU KNOW your data and what it means, you can 
generalize that.  


I guess one way to do this would be

a) set up your tables
b) fill them with a bit of data
c) analyze & explain plan your queries
d) fill them with a bit more data
e) goto c/d a couple of times


then, you can look at the row sources to see how the queries are going to scale.  for example, 
let's run the above scenario using "emp"


ops$tkyte@ORA920PC> create table emp ( empno int primary key, ename varchar2(25), deptno int );
 
Table created.
 
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> create index emp_ename_idx on emp(ename);
 
Index created.
 
ops$tkyte@ORA920PC> create index emp_deptno_idx on emp(deptno);
 
Index created.
 
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> insert into emp
  2  select object_id, owner, mod(rownum,100)
  3    from (select * from all_objects order by reverse(object_name) )
  4   where rownum <= 100;
 
100 rows created.
 
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> variable x number
ops$tkyte@ORA920PC> variable y varchar2(25)
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> analyze table emp compute statistics;
 
Table analyzed.
 
ops$tkyte@ORA920PC> set autotrace traceonly explain
ops$tkyte@ORA920PC> select * from emp where empno = :x;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=1 Bytes=10)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=1 Card=1 Bytes=10)
   2    1     INDEX (UNIQUE SCAN) OF 'SYS_C007176' (UNIQUE)
 
 
 
ops$tkyte@ORA920PC> select * from emp where deptno = :x;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1 Bytes=10)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=2 Card=1 Bytes=10)
   2    1     INDEX (RANGE SCAN) OF 'EMP_DEPTNO_IDX' (NON-UNIQUE) (Cost=1 Card=1)
 
 
 
ops$tkyte@ORA920PC> select * from emp where ename = :y;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=20 Bytes=200)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=2 Card=20 Bytes=200)
   2    1     INDEX (RANGE SCAN) OF 'EMP_ENAME_IDX' (NON-UNIQUE) (Cost=1 Card=20)
 
 
 
ops$tkyte@ORA920PC> set autotrace off
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> insert into emp
  2  select object_id, owner, mod(rownum,100)
  3    from (select *
  4            from all_objects
  5                   where not exists (select null
  6                                       from emp
  7                                                          where empno=object_id)
  8           order by reverse(object_name) )
  9   where rownum <= 900;
 
900 rows created.
 
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> analyze table emp compute statistics;
 
Table analyzed.
 
ops$tkyte@ORA920PC> set autotrace traceonly explain
ops$tkyte@ORA920PC> select * from emp where empno = :x;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1 Bytes=10)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=2 Card=1 Bytes=10)
   2    1     INDEX (UNIQUE SCAN) OF 'SYS_C007176' (UNIQUE) (Cost=1 Card=1000)
 
 
 
ops$tkyte@ORA920PC> select * from emp where deptno = :x;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=10 Bytes=100)
   1    0   TABLE ACCESS (FULL) OF 'EMP' (Cost=2 Card=10 Bytes=100)
 
 
 
ops$tkyte@ORA920PC> select * from emp where ename = :y;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=59 Bytes=590)
   1    0   TABLE ACCESS (FULL) OF 'EMP' (Cost=2 Card=59 Bytes=590)
 
 
 
ops$tkyte@ORA920PC> set autotrace off
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> insert into emp
  2  select object_id, owner, mod(rownum,100)
  3    from (select *
  4            from all_objects
  5                   where not exists (select null
  6                                       from emp
  7                                                          where empno=object_id)
  8           order by reverse(object_name) )
  9   where rownum <= 9000;
 
9000 rows created.
 
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> analyze table emp compute statistics;
 
Table analyzed.
 
ops$tkyte@ORA920PC> set autotrace traceonly explain
ops$tkyte@ORA920PC> select * from emp where empno = :x;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1 Bytes=11)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=2 Card=1 Bytes=11)
   2    1     INDEX (UNIQUE SCAN) OF 'SYS_C007176' (UNIQUE) (Cost=1 Card=10000)
 
 
 
ops$tkyte@ORA920PC> select * from emp where deptno = :x;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=4 Card=100 Bytes=1100)
   1    0   TABLE ACCESS (FULL) OF 'EMP' (Cost=4 Card=100 Bytes=1100)
 
 
 
ops$tkyte@ORA920PC> select * from emp where ename = :y;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=4 Card=294 Bytes=3234)
   1    0   TABLE ACCESS (FULL) OF 'EMP' (Cost=4 Card=294 Bytes=3234)
 
 
 
ops$tkyte@ORA920PC> set autotrace off


so, we see:


    rows  =>    100         1,000            10,000
empno = :x        1             1                 1
deptno =:x        1            10               100
ename = :y       20            59               294



now, you see how each of those will "scale".


It is the nature of the data itself.

Card in Execution Plan November 7, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

So does that mean the Card= in the Execution plan section is the same as "consistent gets". If not 
what does "Card=" in the Execution plan mean and does the figures in your output like 
1,10,59,20,100,294 show the number of "consistent gets". If not what do these figures mean?


    rows  =>    100         1,000            10,000
empno = :x        1             1                 1
deptno =:x        1            10               100
ename = :y       20            59               294

Pl. explain

Followup November 7, 2003 - 5pm Central time zone:

No, card = cardinality = estimated number of rows.

consistent gets and card may be miles and miles apart.

multiblock read making difference? November 22, 2003 - 5pm Central time zone

Bookmark | Bottom | Top

Reviewer: Sudhir from Lexington, KY

Tom,

In your test case when you used example:

"big_table@ORA920LAP> select * from big_table where rownum <= 10000;

10000 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1329 Card=10000 Bytes=940000)
   1    0   COUNT (STOPKEY)
   2    1     TABLE ACCESS (FULL) OF 'BIG_TABLE' (Cost=1329 Card=998400 
Bytes=93849600)




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        794  consistent gets
        134  physical reads
          0  redo size
     773075  bytes sent via SQL*Net to client
       7825  bytes received via SQL*Net from client
        668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10000  rows processed

big_table@ORA920LAP> select * from big_table where rownum <= 10000 order by 
1,2,3,4,5,6,7,8;

10000 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=82617 Card=10000 Bytes=940000)
   1    0   SORT (ORDER BY) (Cost=82617 Card=10000 Bytes=940000)
   2    1     COUNT (STOPKEY)
   3    2       TABLE ACCESS (FULL) OF 'BIG_TABLE' (Cost=1329 Card=998400 
Bytes=93849600)




Statistics
----------------------------------------------------------
          0  recursive calls
          2  db block gets
        138  consistent gets
        257  physical reads
          0  redo size
     773086  bytes sent via SQL*Net to client
       7825  bytes received via SQL*Net from client
        668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          1  sorts (disk)
      10000  rows processed
"

Somehow I feel, having order by is making some difference in the way db_file_multiblock_read is 
being used in one and not in the other. Is that why? I didn't even think 'order by' and 'rownum <= 
10000' are valid combinations (for most cases)

As always, thanks for creative solutions.

Followup November 22, 2003 - 6pm Central time zone:

multi-block-read count affects:

a) the cost of the full scan as seen by the CBO
b) the performance of the full scan if we have to do physical IO


it will not in this case affect consistent gets....

All I wanted to do was show

1) getting 10,000 rows
2) getting 10,000 rows and sorting them


so, rownum was OK for this demo.

Tom, I am missing the point, please help! November 22, 2003 - 7pm Central time zone

Bookmark | Bottom | Top

Reviewer: Sudhir from Lexington, KY

Without sort 

794  consistent gets

with sort

138  consistent gets

whats going on? Thats why I was thinking of multiblock issue.

If you can clear the confusion, Thanks in advance.

Followup November 22, 2003 - 10pm Central time zone:

sort = read all - sort it, put in into private memory, retrieve it

no sort = read it bit by bit by bit as you hit it.


the sort was like a "bulk fetch with arraysize = ALL ROWS at once", less consistent gets overall 
since we "consistently got them" all at once - instead of row by row by row.

(it was part of the 'challenge')

amazing! November 22, 2003 - 10pm Central time zone

Bookmark | Bottom | Top

Reviewer: Sudhir from Lexington, KY USA

Tom,

In this case is it likely that the operation with sort may have run faster than one without sort? 
And do you think there is something like a pattern here that may be useful in general.

Again, thanks for the solution.

Followup November 23, 2003 - 8am Central time zone:

No, do not think about using a "sort" as a "pattern"

Use order by when you need sorted data.

It used more memory in the server process, it did more work to get the first row then the last, you 
waited longer to start seeing data, it used perhaps more CPU.

Thank you! EOM November 23, 2003 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: Sudhir

db block gets+consistent gets > Total Number of Blocks for a table. January 8, 2004 - 5pm Central time zone

Bookmark | Bottom | Top

Reviewer: Sami from NJ,USA

CREATE TABLE MY_LOOKUP1
(
  LOOKUPID      NUMBER(8)    NOT NULL,
  SERVICEID       NUMBER(8)    NOT NULL,
  LOOKUPABBREV  VARCHAR2(20) NOT NULL,
  LOOKUPDESC    VARCHAR2(40) NOT NULL,
  LANGUAGEID    NUMBER(8)    DEFAULT 30    NOT NULL,
  ACTIVEFLAG    VARCHAR2(1)  DEFAULT 'Y'   NOT NULL,
  CREATEDATE    DATE         DEFAULT NULL  NOT NULL,
  UPDATEDATE    DATE         DEFAULT NULL  NOT NULL
)
TABLESPACE MY_DATA_TS
PCTUSED    40
PCTFREE    10
INITRANS   1
MAXTRANS   255
STORAGE    (
            INITIAL          2K
            NEXT             2K
            MINEXTENTS       1
            MAXEXTENTS       505
            PCTINCREASE      50
            FREELISTS        1
            FREELIST GROUPS  1
            BUFFER_POOL      DEFAULT
           )
LOGGING 
NOCACHE
NOPARALLEL;

INSERT INTO MY_LOOKUP1 select * from MY_LOOKUP;
5 rows inserted.


ANALYZE TABLE MY_LOOKUP1 COMPUTE STATISTICS FOR TABLE FOR ALL INDEXES FOR ALL INDEXED COLUMNS;
Table Analyzed.


SQL> select blocks,empty_blocks,num_rows,avg_row_len from  dba_tables where 
TABLE_NAME='MY_LOOKUP1';
 
    BLOCKS EMPTY_BLOCKS   NUM_ROWS AVG_ROW_LEN
---------- ------------ ---------- -----------
         1            0          5          50


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=5 Bytes=250)
   1    0   TABLE ACCESS (FULL) OF 'MY_LOOKUP1' (Cost=1 Card=5 Bytes=250)
 
 
Statistics
----------------------------------------------------------
          0  recursive calls
          4  db block gets
          2  consistent gets
          0  physical reads
          0  redo size
       1085  bytes sent via SQL*Net to client
        431  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          5  rows processed


Questions:
##########

TABLE STATISTICS from DBA_TABLES are below:
===========================================
NUM_ROW    =5
BLOCKS    =1
EMPTY_BLOCKS=0
AVE_ROW_LEN=50

(Q1) BLOCKS=1 (This segment has only one block). 
It does mean that I have all my records in ONE block. Right?

(Q2) If (Q1) correct, why does statistics show 
    4  db block gets
    2  consistent gets
I have everything in ONE block, why am I getting 6 (4+2) blocks.

(Q3)Bytes=250 in Explain Plan which is AVE_ROW_LEN * NUM_ROW. Correct?


Tom,Kinly enlighten me the relationship between DBA_TABLES statistics number and AutoTrace 
Statistics.


Thanks,
Sami

Followup January 8, 2004 - 8pm Central time zone:

for a full scan, we read the segment header to get the extent map.  you are seeing that extra read 
done to figure out what to read for you.

Consistent Gets are still very puzzling January 9, 2004 - 5am Central time zone

Bookmark | Bottom | Top

Reviewer: Ik from Bangalore, India

Tom,

I always used to think that 'Consistent gets' happens only when Oracle tries to provide read
consistency. That is when you have uncommitted transactions.

(1) Iam confused here between current mode gets (buffer reads) and consistent gets. In the original
question, why is there the need for read consistency? Every read should be current mode, right?

(2)
I did the same test as the person who posted the original question here. I got the same results.

Now, i dropped the table (tes) and now used 2 sessions. Inserted those 10000 records in one session
(NO COMMIT) and then queried from another SQLPlus session. Here is what i got.

select * from tes
/
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
10040 consistent gets
0 physical reads
832 redo size
90 bytes sent via SQL*Net to client
242 bytes received via SQL*Net from client
1 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
0 rows processed

Consistent gets have shot up. Is that because transaction was left uncommitted by the other
session? Can you please explain why the numbers are high.

(3) Unrelated to the above 2 questions - I was trying to artificially shoot up the consistent reads
by forcing an index access. It seems that the optimizer won't budge from its stand. It goes for FTS
always. Why so?

create unique index idx_test on tes(a)

select /*+ INDEX(tes idx_test)*/
*
from
tes
Operation Object Name Rows Bytes Cost TQ In/Out PStart PStop

SELECT STATEMENT 82 2
TABLE ACCESS FULL TES 82 1 K 2

Thank you very much for the valuable service you are doing to us Oracle developers.

Thank You.

Followup January 9, 2004 - 8am Central time zone:

we use consistent gets to get a block consistently (as of a point in time) it may or may NOT 
require a read aside to the RBS but the block is gotten "consistently" regardless.

1) no, we ALWAYS get the blocks in consistent read mode.  You cannot tell until you get to the 
block if the data is "good enough for you".  You always process in CR mode therefore.

2) yes, it is read consistency there.  Your query read each block and had to roll back all of the 
inserts on that block in order to see that "hey, this block is actually EMPTY as far as I'm 
concerned"

3) the index is probably on a NULLABLE column.

ops$tkyte@ORA920PC> create table t ( x int, y int );
 
Table created.
 
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> create index test_idx on t(x);
 
Index created.
 
ops$tkyte@ORA920PC>
ops$tkyte@ORA920PC> set autotrace traceonly explain
ops$tkyte@ORA920PC> select /*+ index( t test_idx ) */ * from t;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=82 Bytes=2132)
   1    0   TABLE ACCESS (FULL) OF 'T' (Cost=2 Card=82 Bytes=2132)
 
 
 
ops$tkyte@ORA920PC> alter table t modify x NOT NULL;
 
Table altered.
 
ops$tkyte@ORA920PC> select /*+ index( t test_idx ) */ * from t;
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=826 Card=82 Bytes=2132)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'T' (Cost=826 Card=82 Bytes=2132)
   2    1     INDEX (FULL SCAN) OF 'TEST_IDX' (NON-UNIQUE) (Cost=26 Card=82)
 
 
 
ops$tkyte@ORA920PC> set autotrace off


since entirely null keys are not put into b*tree indexes -- we cannot use the index if X allowed 
nulls (you would not get the right answer since the index does not contain all of the rows in the 
table)

Still come confusion January 20, 2004 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

I have two environments integration (SPI1) and Benchmark (BMK1) and I have the following query 
running on the same environments.

SELECT /*+ ORDERED USE_NL(pcd,p) INDEX(pd) */
 p.position_id,                      pd.position_detail_id,            pd.Alt_Market_Value,
pd.Alt_Market_Value_Income,  pd.Alt_Book_Value,            pcd.Alt_GAAP_Cost_Value,
pd.Base_FX_Unrealized_GL,     pd.Alt_Base_Unrealized_GL,    pd.Alt_Base_FX_Unrealized_GL,
pd.Alt_Accrued_Income,         pd.Local_to_sec_basis_fx_rate,    pd.Income_Receivable_Local,
pd.Income_Receivable_Base,      pd.Alt_Income_Receivable
FROM holdingdbo.position_detail pd,
     holdingdbo.position_cost_detail pcd,         
     holdingdbo.position p
WHERE pd.update_date        >= '17-JAN-04'
  AND    p.position_id            = pd.position_id
  AND p.src_intfc_inst      = 17        
AND pd.position_detail_id = pcd.position_detail_id

I have the following statistics information :

BMK1
====
                               Clus.
Table      Index    Blocks    Rows     Factor    Col
T1         I1    15848    876751     225197    Col1
           I2    15848    876751     832570    COl2
           I2    15848    876751     832570    Col3
           I2    15848    876751     832570    Col4
           I3    15848    876751     242262    Col5
                    
T2       I11    1086082    92017221 1809257  Col1
T2       I12    1086082    92017221 25658678 Col2
                    
T3       I21    1670338    60198216 7946507  Col1
       I22    1670338    60198216 59152225 Col2
       I23    1670338    60198216 30525320 Col3
       I24    1670338    60198216 19431172 Col4
       I24    1670338    60198216 19431172 Col2
       I24    1670338    60198216 19431172 Col5
       I24    1670338    60198216 19431172 Col6
       I24    1670338    60198216 19431172 Col7

SPI1
====

Table    Index    Blocks    Rows    Clus Factor Col
T1    I1    196587    557024    372737        Col1
    I2    196587    557024    547845        Col2
    I2    196587    557024    547845        Col3
    I2    196587    557024    547845        Col4
    I3    196587    557024    410857        Col5
                    
T2    I11    904315    38124980 3716600    Col1
    I12    904315    38124980 32805420    Col2
                    
T3    I21    2247043    39646070 11968130    Col1
    I22    2247043    39646070 38879830    Col2
    I23    2247043    39646070 33992900    Col3
    I24    2247043    39646070 26692950    Col4
    I24    2247043    39646070 26692950    Col2
    I24    2247043    39646070 26692950    Col5
    I24    2247043    39646070 26692950    Col6
    I24    2247043    39646070 26692950    Col7

The above query executed in the above two environments produced two different timings. My questions 
are :

1. Is hints always good to put in a query
2. If the execution plans are same in the two environments, what are the other factors to look for 
in the two environments (besides the num rows, blocks, clustering factor as above) that contribute 
to this time difference. ie. how to find the exact cause of this time difference
3. How do we check if all the tables/indexes etc are analyzed
4. If the tools like AUTOTRACE, TKPROF, SQL_TRACE etc are used in both the environments, what would 
be the sequence of steps (like a checklist for myself) that could help me find the exact cause of 
the problem. 
5. I read your books on these chapters, but when it comes to practical tuning cases as above, I am 
not sure what could be the best possible way to tackle such problems.
6. How to infer from consistent gets/physical reads etc from TKPROF/AUTOTRACE output

A detailed explanation on this would be of great help

Thanks

Followup January 20, 2004 - 2pm Central time zone:

1) no
2) wait events -- use 10046 level 12 trace event (search for 10046 on this site if that doesn't 
mean anything to you) and see what it is waiting for.

and remember, two machines differ in thousands of ways.  

3) queries against the data dictionary, last analyzed is in there as well as sample sizes

4) since I don't know what they "problem"  is or even if there is one, a checklist won't help.  it 
is usually "obvious".  do you have a 9i tkprof with wait events that one could look at.

5) show us a tkprof - it should be somewhat "obvious"

6) you do not infer from tkprof -- it is spelled out (there is a physical reads column, there are 
current and query mode (logical) columns.  they are told to you.

from autotrace, again, they are spelled out -- they are right there.  consistent gets, db block 
gets ( logical ios) and physical reads.  no infering, just reading.

Pls explain January 20, 2004 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from NJ, USA

Tom,

In your response to the above response, you mentioned that it could differ in thousands of ways. I 
would like to get some information from you as to a couple of ways that the two could be different. 
Please provide with a list that is very significant ( I understand that the list is huge, but at 
least would be helpful if you could list a couple of them that I could easily check)

Followup January 20, 2004 - 10pm Central time zone:

nope, as i said -- show us the tkprof and it should be "obvious"

why network packets increases? February 7, 2004 - 3am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi

Why the amount of data transferred goes up when you set a big arraysize? Has it got something to do 
with tcp/ip packet size?

Followup February 7, 2004 - 2pm Central time zone:

goes up and down.  40k is "noise" as far as I'm concerned.  test was really skinny, not a good 
test.

we'll just attribute it to gremlins this time....


bigger, fatter table:

ops$tkyte@ORA9IR2> set autotrace traceonly statistics;
ops$tkyte@ORA9IR2> select * from all_objects;
 
29512 rows selected.
 
 
Statistics
----------------------------------------------------------
    2097689  bytes sent via SQL*Net to client
 
ops$tkyte@ORA9IR2> set arraysize 100
ops$tkyte@ORA9IR2> /
 
29512 rows selected.
 
 
Statistics
----------------------------------------------------------
    1895377  bytes sent via SQL*Net to client
 
ops$tkyte@ORA9IR2> set arraysize 1000
ops$tkyte@ORA9IR2> /
 
29512 rows selected.
 
 
Statistics
----------------------------------------------------------
    1863191  bytes sent via SQL*Net to client
 
ops$tkyte@ORA9IR2> set arraysize 5000
ops$tkyte@ORA9IR2> /
 
29512 rows selected.
 
 
Statistics
----------------------------------------------------------
    1860287  bytes sent via SQL*Net to client

February 19, 2004 - 7am Central time zone

Bookmark | Bottom | Top

Reviewer: Alison Tonner from Scotland

Hi Tom, thanks for all the information provided in the rest of this thread.

I was wondering if you could clarify something.

When I change the optimizer mode from first_rows_10 to first_rows_100 the cost of the plan goes up, 
even though the consistant_gets and physical reads goes down.

I do see that the cardinallity gets higher but still not sure why the cost goes up instead of down?

SYSTEM-ORCL->alter session set optimizer_mode = first_rows_10;

Session altered.

Elapsed: 00:00:00.00
SYSTEM-ORCL->select * from customers c, sales s
  2  where c.cust_id = s.cust_id
  3  and cust_last_name = 'Smith'
  4  and s.time_id <= '31-DEC-98';

359 rows selected.

Elapsed: 00:06:01.01

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=FIRST_ROWS (Cost=22 Card=10 Bytes=2890)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'CUSTOMERS' (Cost=22 Card=1 Bytes=130)
   2    1     NESTED LOOPS (Cost=22 Card=10 Bytes=2890)
   3    2       PARTITION RANGE (ITERATOR)
   4    3         TABLE ACCESS (FULL) OF 'SALES' (Cost=3 Card=201 Bytes=31959)
   5    2       BITMAP CONVERSION (TO ROWIDS)
   6    5         BITMAP AND
   7    6           BITMAP CONVERSION (FROM ROWIDS)
   8    7             INDEX (RANGE SCAN) OF 'CUSTOMERS_PK' (UNIQUE)
   9    6           BITMAP CONVERSION (FROM ROWIDS)
  10    9             INDEX (RANGE SCAN) OF 'CUST_CUST_LAST_NAME_IDX' (NON-UNIQUE) (Cost=1 
Card=641)




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     877531  consistent gets
       2731  physical reads
          0  redo size
      21137  bytes sent via SQL*Net to client
        756  bytes received via SQL*Net from client
         25  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        359  rows processed

SYSTEM-ORCL->rem Now with FIRST_ROWS_100
SYSTEM-ORCL->pause

SYSTEM-ORCL->alter session set optimizer_mode = first_rows_100;

Session altered.

Elapsed: 00:00:00.00
SYSTEM-ORCL->select * from customers c, sales s
  2  where c.cust_id = s.cust_id
  3  and cust_last_name = 'Smith'
  4  and s.time_id <= '31-DEC-98';

359 rows selected.

Elapsed: 00:00:01.09

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=FIRST_ROWS (Cost=61 Card=100 Bytes=15900)
   1    0   TABLE ACCESS (BY LOCAL INDEX ROWID) OF 'SALES' (Cost=61)
   2    1     NESTED LOOPS (Cost=61 Card=100 Bytes=15900)
   3    2       TABLE ACCESS (BY INDEX ROWID) OF 'CUSTOMERS' (Cost=5 Card=1 Bytes=130)
   4    3         INDEX (RANGE SCAN) OF 'CUST_CUST_LAST_NAME_IDX' (NON-UNIQUE) (Cost=3 Card=1)
   5    2       PARTITION RANGE (ITERATOR)
   6    5         BITMAP CONVERSION (TO ROWIDS)
   7    6           BITMAP AND
   8    7             BITMAP MERGE
   9    8               BITMAP INDEX (RANGE SCAN) OF 'SALES_TIME_BIX'
  10    7             BITMAP INDEX (SINGLE VALUE) OF 'SALES_CUST_BIX'




Statistics
----------------------------------------------------------
         14  recursive calls
          0  db block gets
       3191  consistent gets
        497  physical reads
          0  redo size
      20130  bytes sent via SQL*Net to client
        756  bytes received via SQL*Net from client
         25  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        359  rows processed

Followup February 19, 2004 - 11am Central time zone:

you cannot compare the cost of two different plans under two different settings of optimizer 
parameters.

doesn't make sense.  


the optimizer parameters cause the costing model to "change", that is their goal.  by changing them 
-- you changed the method by which the numbers are assigned.  It would be expected that the costs 
are "different", the way these parameters work in general is by changing the costing model.

Now, your test is a little invalid.  You asked "please optimize to get the first 10 rows ASAP" and 
then fetched all 359.  You "lied" to the optimizer ;)  Use this:

alter session set sql_trace=true;
declare
   cnt number := 0;
begin
  for x in ( select * from customers c, sales s
    where c.cust_id = s.cust_id
    and cust_last_name = 'Smith'
    and s.time_id <= TO_DATE( '31-DEC-1998' ) )  /* pet peeve of mine */
  loop
     cnt := cnt + 1;
     exit when (cnt=10);
  end loop;
end;
/

and use tkprof to analyze the outcome with different optimizer settings.

February 19, 2004 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: Alison

Thanks - that's all I needed to know

Just the idea that they are apples and oranges and can't be compared in that way makes sence.

Thanks as usuall for your fast, consise reply.

Low LIO != Fast responce?? March 10, 2004 - 7am Central time zone

Bookmark | Bottom | Top

Reviewer: Prasad from UK

Hi Tom,

The above discussion more than clearly says what the "consistent gets" is all about.
We consider it as a part of LIO. And you always suggests for lowering LIOs. So i did a test case 
where I lowered the LIO (by increasing the arraysize 15...1000...4000) But as i lowered the LIO my 
execution(Elapsed) time increased instead.How is it? Am I missing something??

here is the test i carried out.
----------------------------------------
create table t as select * from all_objects;

Table created.

exec show_space( 'T' )

Free Blocks.............................0
Total Blocks............................137
Total Bytes.............................1122304
Unused Blocks...........................25
Unused Bytes............................204800
Last Used Ext FileId....................61
Last Used Ext BlockId...................1677
Last Used Block.........................20


set timing on

select * from t;
8704 rows selected.

Elapsed: 00:00:18.82

Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        689  consistent gets
          0  physical reads
          0  redo size
     780354  bytes sent via SQL*Net to client
      47343  bytes received via SQL*Net from client
        582  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
       8704  rows processed


set arraysize 1000

select * from t;
8704 rows selected.


Elapsed: 00:00:19.18

Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        124  consistent gets
         99  physical reads
          0  redo size
     732878  bytes sent via SQL*Net to client
       1011  bytes received via SQL*Net from client
         10  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
       8704  rows processed


set arraysize 4000

select * from t;
8704 rows selected.

Elapsed: 00:00:24.57

Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        118  consistent gets
          0  physical reads
          0  redo size
     732380  bytes sent via SQL*Net to client
        525  bytes received via SQL*Net from client
          4  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
       8704  rows processed

Followup March 10, 2004 - 9am Central time zone:

I've also said that there are diminishing marginal returns to everything.  I found an arraysize of 
about 100 is the "sweet spot" for array fetching in general.

see

http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:12226527910608#15916220123225

yes, the goal is to reduce LIO's in general when tuning a query.  array fetching is a tool but only 
to a point.

Perfect March 10, 2004 - 9am Central time zone

Bookmark | Bottom | Top

Reviewer: Prasad from UK

set arraysize 100

select * from t;
8664 rows selected.

Elapsed: 00:00:12.74

Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        201  consistent gets
        111  physical reads
          0  redo size
     735856  bytes sent via SQL*Net to client
       7329  bytes received via SQL*Net from client
         88  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
       8664  rows processed

Thanks
    Prasad

Does analytics functions speed up a FTS? April 28, 2004 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: Marcio from Brazil

I noted this, when had to catch total (count(*)) from a table to inform application_info -- so when 
put AF on select x, y, z, a, count(*) over () from t instead of just  select * from t, got reduce 
my LIOs.

Do you know why?

Consider test!
~~~~~~~~~~~~~~
drop table t;

create global temporary table t ( x number, y number, z number, a number);

insert into t 
select rownum, rownum+1, rownum+2, null
  from all_objects
union all
select rownum, rownum+1, rownum+2, null
  from all_objects
/

alter session set sql_trace = true;

begin
    for x in ( select x, y, z, a from t )
    loop
        null;
    end loop;
end;
/

begin
    for x in ( select x, y, z, a, count(*) over () cnt from t )
    loop
        null;
    end loop;
end;
/

alter session set sql_trace = false;

------------------------------------------
Got this:

begin
    for x in ( select x, y, z, a from t )
    loop
        null;
    end loop;
end;

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.10       0.07          0          0          1           0
Execute      1      7.20       7.77          0          0          0           1
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        2      7.30       7.85          0          0          1           1

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 288  
********************************************************************************

select x, y, z, a 
from
 t 


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch    49567      3.58       3.17          0      49573          0       49566
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total    49569      3.58       3.18          0      49573          0       49566

and

begin
    for x in ( select x, y, z, a, count(*) over () cnt from t )
    loop
        null;
    end loop;
end;

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.02       0.01          0          0          0           0
Execute      1      6.69       7.50          0          0          0           1
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        2      6.71       7.51          0          0          0           1

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 288  
********************************************************************************

select x, y, z, a, count(*) over () cnt 
from
 t 


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch    49567      3.68       3.22          0        135          0       49566
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total    49569      3.68       3.22          0        135          0       49566


LIOs - from 49573 to 135 
Even on sql*plus look:

synth_base@MRP920> select x, y, z, a from t;

49566 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'T'




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        626  consistent gets
          0  physical reads
          0  redo size
     989711  bytes sent via SQL*Net to client
       5948  bytes received via SQL*Net from client
        497  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      49566  rows processed

synth_base@MRP920> select x, y, z, a, count(*) over () cnt from t;

49566 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   WINDOW (BUFFER)
   2    1     TABLE ACCESS (FULL) OF 'T'




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        135  consistent gets
          0  physical reads
          0  redo size
     989767  bytes sent via SQL*Net to client
       5948  bytes received via SQL*Net from client
        497  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
      49566  rows processed

Seems have same behavior on permanent tables as well.
Thank you.

Followup April 28, 2004 - 7pm Central time zone:

the sort caused them to be read all at one -- on the FIRST fetch.

do this:


begin
    for x in ( select x, y, z, a from t )
    loop
        exit;
    end loop;
end;
/

begin
    for x in ( select x, y, z, a, count(*) over () cnt from t )
    loop
        exit;
    end loop;
end;
/

instead and you'll see what I mean.  reads from "temp" are not LIOs.

it is as if you set arraysize = infinity.

I would *not* recommend this approach (in fact, i'm on record time and time and time and time and 
time again saying 'counting the hits is "not a good thing" (tm)') at all, not even a tiny bit.

show me the link please! April 29, 2004 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: Marcio from Brazil

Well, I'd like show in dbms_application_info n thru m, where m is count(*) from table -- I can't 
use bulk collect.
Would you show me where (link) you point it out?

Followup April 29, 2004 - 10am Central time zone:

er? what link -- not really sure what you mean (nor would I agree with "i can't use bulk collect")

Consistent gets + db_block_gets April 29, 2004 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: reader from USA

Tom,

YOUR HELP ....

create table test as select * from all_objects;
set autotrace on 

Free Blocks.............................0
Total Blocks............................390
Total Bytes.............................3194880
Unused Blocks...........................76
Unused Bytes............................622592
Last Used Ext FileId....................162
Last Used Ext BlockId...................114342
Last Used Block.........................54

PL/SQL procedure successfully completed.
1. CASE A
select count(*) from test;


  COUNT(*)
----------
     27187

Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        317  consistent gets
        313  physical reads
          0  redo size
        203  bytes sent via SQL*Net to client
         90  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed


2.CASE B 
update test1 set owner='TEST' (Session A) 

          2  recursive calls
      28089  db block gets
        374  consistent gets
        179  physical reads
    6900888  redo size
        534  bytes sent via SQL*Net to client
        580  bytes received via SQL*Net from client
          6  SQL*Net roundtrips to/from client
          3  sorts (memory)
          0  sorts (disk)
      27187  rows processed

3.CASE C 
select count(*) from test1;(session A) 

      0  recursive calls
      4  db block gets
    318  consistent gets
      0  physical reads
      0  redo size
    204  bytes sent via SQL*Net to client
    187  bytes received via SQL*Net from client
      3  SQL*Net roundtrips to/from client
      0  sorts (memory)
      0  sorts (disk)
      1  rows processed

update test1 set owner='TEST' (from a different session) (session B) 
NOT COMMITED YET 

4.CASE D 
select count(*) from test1;(session A) 

         0  recursive calls
         4  db block gets
     27875  consistent gets
        50  physical reads
     16328  redo size
       203  bytes sent via SQL*Net to client
       187  bytes received via SQL*Net from client
         3  SQL*Net roundtrips to/from client
         0  sorts (memory)
         0  sorts (disk)
         1  rows processed


1.why will db_block_gets go down from 12 to 4 From case A to case D 
  I read from your site that it the info about were will we get the blocks eg segment headers 
  But segment header is one .Does it have to read it multiple times or is it something else .

2.Consistent gets is going up in case D -which is fine because it has to rebuild the information 
using undo.
  But is the redo size > 0 for a select .
  Also ,why is the consistent read so high is it one block per row of undo info read.

3.Why is the db_block_get so high is case B .
  what info is oracle getting  here and from where ?

your answers are highly appreciated !

THANKS .

Followup April 29, 2004 - 2pm Central time zone:

1) it just does, don't know, it seems to happen due to the CTAS (if you create empty, insert 
append, it doesn't happen). 

2) the redo size >0 is block cleanout (if you have expert one on one oracle, I go into this in 
detail)

the consistent gets are high just due to the RBS processing.

3) every row update will result in a CURRENT MODE read -- and db block gets are current mode reads. 
 we do a consistent read to find the rows, and then a current mode read to update it.

select vs select for update June 17, 2004 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

I know that selects return you read consistent
data as of the point of time the select started.
Is that true for "select .. for update" as well?
Are there any differences between the two in terms
of read consistency?

Thank you!

Followup June 17, 2004 - 2pm Central time zone:

it'll be just like an update (subject to restart).  It will be read consistent, perhaps not with 
respect to the point in time at which it was submitted, but at some point in time between the time 
it started and the time it completed.

see

http://asktom.oracle.com/~tkyte/wc.html

thanx! June 17, 2004 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Buffer gets/exec and consistent gets June 17, 2004 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: Arun Gupta from Harrisburg, PA USA

Tom,
Are the Buffer gets/exec in a statspack report for a query related to consistent gets in autotrace 
statistics or query in TKPROF report?
Thanks

Followup June 17, 2004 - 6pm Central time zone:

in tkprof they are at the level of the application,that particular execution.

in a statspack they are system wide, over "every" possible tkprof....

so tkprof is a detailed view, for a particular session whereas statspack is over all sessions for 
that period of time.

Buffer gets/exec and consistent gets June 18, 2004 - 6am Central time zone

Bookmark | Bottom | Top

Reviewer: Arun Gupta from Harrisburg, PA USA

Tom,
Actually, in a load test conducted on one of our applications, a particular query showed 9 million 
buffer gets/exec in statspack report. Tuning the query in isolation, I could not reproduce the 
scenario. I was using TKPROF and could only get 167,000 consistent gets. Reading through this 
thread, I understand that insert/update/delete against base tables can increase the consistent gets 
if they happen between opening of a cursor and when the results are fetched. This is very much 
possible in our load test which stresses the application quite heavily. We use ref cursors to get 
results. Can this be a possible explanation of what might have happened?

Thanks

Followup June 18, 2004 - 10am Central time zone:

sure, if the table was really "hot" and you were updating lots of indexed columns and the query 
used lots and lots of indexes and/or the query fetched a row and worked on it a while and then 
fetched a row

sure.  seems extreme, but yes.

Still not clear about indexes. July 15, 2004 - 1am Central time zone

Bookmark | Bottom | Top

Reviewer: aru from New Zealand

Hi Tom,

I am still not clear about the way we will access the datablocks through the indexes. The part 
which confuses me is that when you have proved in this thread that the way the datablock is 
accesses is as you said:-
"Followup:  
1) well, it'll get the index block to find the table block, and then get the 
index block to find the next table block, and so on."

Now in this case the arraysize is immaterial because only one data row is accessed in this way.That 
also goes to say that the minimum IO one can have while accessing rows THROUGH an index is:
(N * 2) where N is number of rows accessed. If 100 rows accessed then MINIMUM IO will be 200. 100 
for the index block gets and 100 for the datablock gets for corresponding rowid's. 

IF this is now true then there is something definitely missing from this thread that no one has 
either understood or has not thought about.
 
Also Tom please explain why there is so much difference in the output's here:
----------------------

SQL> select * from t where x < 100;(table created as per this thread):

99 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'COLO'
   2    1     INDEX (RANGE SCAN) OF 'COLO_X_IDX' (NON-UNIQUE)




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
         13  consistent gets
          0  physical reads
          0  redo size
       2626  bytes sent via SQL*Net to client
        865  bytes received via SQL*Net from client
          6  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         99  rows processed


And from tkprof it is :- 
Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 45  
****************************************

select * 
from
 colo where x < 100 


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch      100      0.00       0.00          0        200          0          99
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total      102      0.00       0.00          0        200          0          99

Rows     Row Source Operation
-------  ---------------------------------------------------
     99  TABLE ACCESS BY INDEX ROWID T (cr=200 r=0 w=0 time=1748 us)
     99   INDEX RANGE SCAN T_IDX (cr=101 r=0 w=0 time=984 us)(object id 41082)


Shows 200. Difference of 200 - 13 = 187 is too much for me to even try thinking what's going on 
here. HELP!!!!!!!!

Thanks YET again,
ARU.

Followup July 15, 2004 - 12pm Central time zone:

using sqlplus -- we have array fetching going on.

Lets say the table data was ordered with respect to the index (eg: index on x, table data is sorted
by x).

now, we ask for x < 100.

we are using sqlplus -- arraysize defaults to 15.

we as for first row (we are doing to get 15). we read an index block, locate row one. we get the
table block for row 1 (and don't give it up).. we locate row 2 (still have the index block we
need) and get table block for row 2 -- discover "we already have that block in our session -- cool,
just use it" and so on. we get the first 15 rows with 2 LIOS

suppose we did that row by row. we would have 2 LIOS per row or 30 LIOs for the first 15.

That is how array fetching can not only improve wall clock response time (takes less time to do
2lios and transfer 15 rows in bulk then it does to do 30 lios and transfer a row by row by row) but
also scalability - since the LIO's involve latching.

try this out:

ops$tkyte@ORA9IR2> drop table t;
Table dropped.

ops$tkyte@ORA9IR2> create table t ( x int primary key, y int );
Table created.

ops$tkyte@ORA9IR2> insert into t select rownum, rownum from all_objects where rownum <= 101;
101 rows created.

ops$tkyte@ORA9IR2> set autotrace traceonly statistics
ops$tkyte@ORA9IR2> set arraysize 2
ops$tkyte@ORA9IR2> select * from t where x < 100;

99 rows selected.

Statistics
----------------------------------------------------------
101 consistent gets

ops$tkyte@ORA9IR2> set arraysize 10
22 consistent gets
ops$tkyte@ORA9IR2> set arraysize 20
12 consistent gets
ops$tkyte@ORA9IR2> set arraysize 50
6 consistent gets
ops$tkyte@ORA9IR2> set arraysize 100
4 consistent gets

the bottom was edited - i just reran the same query with different array sizes.

Why redo size? December 23, 2004 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Regarding the autotrace stats, why would a SELECT statement generate redo entries? See

no rows selected

Elapsed: 00:07:37.65

Statistics
----------------------------------------------------------
          7  recursive calls
          0  db block gets
    1725310  consistent gets
     132101  physical reads
     247884  redo size
        867  bytes sent via SQL*Net to client
        456  bytes received via SQL*Net from client
          1  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          0  rows processed

[I am using true temporary tablespaces, auto memoty allocation using pga_aggregate_target]

Followup December 23, 2004 - 12pm Central time zone:

"block cleanouts" (search for that) could cause it.  

and I see 7 recursive calls -- could be contributing factors as well.

ops$tkyte@ORA9IR2> set autotrace traceonly statistics;
ops$tkyte@ORA9IR2> select rownum from all_objects;
 
28713 rows selected.
 
 
Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     134211  consistent gets
          0  physical reads
          0  redo size
     399714  bytes sent via SQL*Net to client
      21553  bytes received via SQL*Net from client
       1916  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      28713  rows processed
 
ops$tkyte@ORA9IR2> select s.nextval from all_objects;
 
28713 rows selected.
 
 
Statistics
----------------------------------------------------------
      20104  recursive calls
       4351  db block gets
     135647  consistent gets
          0  physical reads
    1214332  redo size
     409812  bytes sent via SQL*Net to client
      21553  bytes received via SQL*Net from client
       1916  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      28713  rows processed
 
ops$tkyte@ORA9IR2> set autotrace off


there were in this case updates to SEQ$ executed as "autonomous transactions" that caused the 
redo

Thanks December 23, 2004 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

"there were in this case updates to SEQ$ executed as "autonomous transactions" that caused the 
redo"

Hm, so a select from a sequence causes a "autonomous transaction". Sequences have been around since 
at least Oracle 7.3, but autonomous transactions were "invented" in Oracle 8 and up! Guess Oracle 
always had them, but didnt "expose" them to the general public until v8?

Any other features you know of like this that are used internally but not available to us?

Thanks

Followup December 23, 2004 - 3pm Central time zone:

autonomous transactions have been in the server forever and an age ago.

they were simply exposed in 815.

Want to guess how long "flashback query" has been around?


There are thousands, perhaps millions, of things "internal" that are not exposed.

How many blocks are from rollback segment? April 28, 2005 - 8pm Central time zone

Bookmark | Bottom | Top

Reviewer: Sami

Dear Tom,

From autotrace or tkprof, Is there a way to identify, how many blocks are retrived from "rollback 
segment" to get the consistent results.

How many blocks are retrived from the rollback segment in the below example?

Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        319  consistent gets
         46  physical reads
          0  redo size
        369  bytes sent via SQL*Net to client
        425  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

Followup April 28, 2005 - 9pm Central time zone:

no.
only thing you can say is somewhere between 0 and 318.

Statistics over a database link ... May 6, 2005 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: VKOUL from WA USA

Pls. go through the following ...

===================================================
DATABASE 1 START
===================================================
SQL> create table a as select * from all_objects;

Table created.

SQL> select count(*) FROM a;

  COUNT(*)
----------
    188088

SQL> create table b as select * from a;

Table created.

SQL> select count(*) from b;

  COUNT(*)
----------
    188088

SQL> 
SQL> set autotrace trace explain statistics
SQL> 
SQL> SELECT *
  2  FROM   a, b
  3  WHERE  a.object_name = b.object_name;

2832576 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   MERGE JOIN
   2    1     SORT (JOIN)
   3    2       TABLE ACCESS (FULL) OF 'B'
   4    1     SORT (JOIN)
   5    4       TABLE ACCESS (FULL) OF 'A'




Statistics
----------------------------------------------------------
          0  recursive calls
         84  db block gets
       5228  consistent gets
       9286  physical reads
          0  redo size
  100467544  bytes sent via SQL*Net to client
    2077721  bytes received via SQL*Net from client
     188840  SQL*Net roundtrips to/from client
          0  sorts (memory)
          2  sorts (disk)
    2832576  rows processed

SQL> 
SQL> set timing on
SQL> 
SQL> /

2832576 rows selected.

Elapsed: 00:01:34.04

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   MERGE JOIN
   2    1     SORT (JOIN)
   3    2       TABLE ACCESS (FULL) OF 'B'
   4    1     SORT (JOIN)
   5    4       TABLE ACCESS (FULL) OF 'A'




Statistics
----------------------------------------------------------
          0  recursive calls
         24  db block gets
       5228  consistent gets
       9286  physical reads
          0  redo size
  100884572  bytes sent via SQL*Net to client
    2077721  bytes received via SQL*Net from client
     188840  SQL*Net roundtrips to/from client
          0  sorts (memory)
          2  sorts (disk)
    2832576  rows processed

SQL> 
SQL> disc
Disconnected from Oracle9i Enterprise Edition Release 9.2.0.1.0 - Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.1.0 - Production
SQL> 
===================================================
DATABASE 1 END
===================================================
===================================================
DATABASE 2 START
===================================================
SQL> set autotrace trace explain statistics
SQL> select * from a@dev9idw, b@dev9idw
  2* where  a.object_name = b.object_name;

2832576 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT (REMOTE) Optimizer=CHOOSE
   1    0   MERGE JOIN
   2    1     SORT (JOIN)
   3    2       TABLE ACCESS (FULL) OF 'B'                             DEV9IDW.
                                                                       ESD1.WA.
                                                                       LCL

   4    1     SORT (JOIN)
   5    4       TABLE ACCESS (FULL) OF 'A'                             DEV9IDW.
                                                                       ESD1.WA.
                                                                       LCL





Statistics
----------------------------------------------------------
          7  recursive calls
          1  db block gets
          4  consistent gets
          0  physical reads
        212  redo size
  132465052  bytes sent via SQL*Net to client
    2077717  bytes received via SQL*Net from client
     188840  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
    2832576  rows processed

SQL> 
SQL> set timing on
SQL> 
SQL> /

2832576 rows selected.

Elapsed: 00:03:16.03

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT (REMOTE) Optimizer=CHOOSE
   1    0   MERGE JOIN
   2    1     SORT (JOIN)
   3    2       TABLE ACCESS (FULL) OF 'B'                             DEV9IDW.
                                                                       ESD1.WA.
                                                                       LCL

   4    1     SORT (JOIN)
   5    4       TABLE ACCESS (FULL) OF 'A'                             DEV9IDW.
                                                                       ESD1.WA.
                                                                       LCL





Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          0  consistent gets
          0  physical reads
          0  redo size
  132075604  bytes sent via SQL*Net to client
    2077717  bytes received via SQL*Net from client
     188840  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
    2832576  rows processed

SQL> 
SQL> disc
Disconnected from Oracle9i Enterprise Edition Release 9.2.0.1.0 - Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.1.0 - Production
SQL> 
===================================================
DATABASE 2 END
===================================================

Q1 : Why is there too much difference on consistent gets
& physical reads between two databases

Q2 : Is consistent gets & physical reads from the remote database or local database

Q3 : If all tables belong to remote database, is there ever a possibility that optimizer will 
change plan to bring data across over the dblink and then join the tables together. ( but in this 
example it does not show that)

Thanks

Followup May 6, 2005 - 1pm Central time zone:

q1) one time query is done here, next time over there (remote).  autotrace shows only your local 
statistics.

q2) local only

q3) yes, it can if it wants to (but you won't see the consistent gets, they happened remotely)

Is it correct??? May 27, 2005 - 4am Central time zone

Bookmark | Bottom | Top

Reviewer: Archana from India

Hi Tom,

I was just going through this discussion and come accross a formula for consistent gets. It says,

Consistent gets = number_of_rows/arraysize + total_blocks_read

Is it really true?? 
If yes, is it true for a particular version??

I tried this in 10g database but couldn't get the proper result.

Can you please help me in this regard?

Thanks in advance.

Followup May 27, 2005 - 7am Central time zone:

for the query in question -- a simple full scan.  

This example demonstrates the effect arraysize can have on logical IO.  Oracle will get and reget a 
block over and over and over.  If you are full scanning a table with 100 blocks and each block has 
100 rows and you array fetch 10 at a time, you'll get each block 10 times -- and therefore perform 
1000 logical IO's.

If you array fetch 100 at a time, you'll get each block once and performan 100 logical IO's

But, if you select count(*) from that_table -- you'll do 100 logical IO's regardless of arraysize.

There is no hard and fast formula for every query.

reader May 27, 2005 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Does the server process reads from disk to the buffer pool, and then server process then reads from 
the buffer pool to 
process the query. If so are we counting the 
number of physical reads twice

Followup May 27, 2005 - 12pm Central time zone:

no you aren't?  there was one physical IO, from disk to buffer pool.  there was one logical io from 
buffer pool

reader May 27, 2005 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Therefore if server process wants to get a block
(ex:<file1;block5>) from the disk, does it
involve 2 ios, ( 1 physical io and 1 logical io )

Followup May 27, 2005 - 3pm Central time zone:

unless we are doing a direct read, yes. A physical IO to read into the buffer cache would be 
followed by "get it from the buffer cache"

what 's going on here? June 6, 2005 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

hi tom,

we 've observed a somehow "strange" effect: an *increased* number of "query" buffers reported by 
trace files after removing the where-clause from the definition of cursor "MyCursor" (see below) -- 
although that where-clause just pointed to the very first remaining entry to be read and deleted:

declare
    cursor MyCursor( p_nMyPKVal MyTab.MyPKCol%type ) is
        select /*+ FIRST_ROWS */ RowID RID, ...
        from   MyTab
        where  MyPKCol > p_nMyPKVal order by MyPKCol ;

begin
    for i in 1 .. p_nTxnCount loop
        open MyCursor( i * p_nTxnSize - p_nTxnSize ) ;
        for i in 1 .. p_nTxnSize loop
            fetch MyCursor into MyRecord ;
            exit when MyCursor%NOTFOUND ;
            delete MyTab where RowID = MyRecord.RID ;
        end loop ;
        commit ;
        close MyCursor ;
    end loop ;
end ;
/

furthermore this increase increased with an increasing number of transactions (parameter p_nTxnSize 
below).

as an example: 1000 transactions with 50 rows for each

statistics for the version without where-clause:

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.02          0          0          0           0
Execute   1000      0.12       0.09          0          0          0           0
Fetch    50000      6.42      34.34       7233     162832          0       50000
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total    51001      6.54      34.45       7233     162832          0       50000

statistics for the version with the additional where-clause:

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute   1000      0.14       0.11          0          0          0           0
Fetch    50000      6.13      32.25       7144     117607          0       50000
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total    51001      6.27      32.36       7144     117607          0       50000

statistics were computed for table and its primary key (and only) index.

what could have happened here?
is it a matter of empty index blocks (or such)?

Followup June 6, 2005 - 12pm Central time zone:

I see the decrease, no increase?

but without a predicate, one could assume a different plan and a different plan will likely have 
different IO's associated with it.


I don't like your code at all.  You should just be issuing a DELETE, no procedural code at all, 
especially if you are concerned about reducing the amount of work performed

June 6, 2005 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

but this code snippet has just been minimized for description purposes (sorry: i shouldn't have 
mentioned that). so it 's not about deleting some rows but rather some other processing does take 
place between fetch and delete ...

and hmmm, the number of "query" buffers did *increase* when there was *no where-clause* provided 
...

unfortunately i 've to admit that the trace files didn't include the execution plans (for whatever 
reason) -- although the plans for some other queries had been there. we switched sql_trace off 
after execution and before closing the sessions ...

here are at least the *predicted* execution plans for both queries:

select /*+ FIRST_ROWS */ RowID RID, MyPKCol, MyTxtCol from MyTab order by MyPKCol
---------------------------------------------------------------------------------
   0      SELECT STATEMENT Optimizer=HINT: FIRST_ROWS (Cost=7237 Card=50000 Bytes=26100000)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'MYTAB' (Cost=7237 Card=50000 Bytes=26100000)
   2    1     INDEX (FULL SCAN) OF 'XPKMYTAB' (UNIQUE) (Cost=94 Card=50000)


select /*+ FIRST_ROWS */ RowID RID, MyPKCol, MyTxtCol from MyTab where MyPKCol > :A order by 
MyPKCol
----------------------------------------------------------------------------------------------------

   0      SELECT STATEMENT Optimizer=HINT: FIRST_ROWS (Cost=67 Card=2500 Bytes=1305000)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'MYTAB' (Cost=67 Card=2500 Bytes=1305000)
   2    1     INDEX (RANGE SCAN) OF 'XPKMYTAB' (UNIQUE) (Cost=2 Card=450)

there were lower costs calculated for the later query. but please keep in mind that the bind 
variable's value has always been set to the *lowest* existing value for column MyPKCol (remaining 
after deletes). so the optimizer should have predicted a "full range scan" ;o) for that too ...

hey, would you mind to run the following code and compare our results?

/*
-- prepare (run only once)
connect TOM/TOM
drop table MyTab ;
create table MyTab( MyPKCol number, MyTxtCol varchar2( 1000 ), constraint XPKMyTab primary key( 
MyPKCol ) ) ;
disconnect
*/

connect TOM/TOM
declare
    MyTxnCnt  constant number := 1000 ;
    MyTxnSize constant number :=   50 ;

    cursor MyCursor( p_nPKVal MyTab.MyPKCol%type ) is
        select /*+ FIRST_ROWS */ RowID RID, MyPKCol, MyTxtCol from MyTab order by MyPKCol ;

    type MyRecordType is record( RID rowid, MyPKCol number, MyTxtCol varchar2( 1000 ) ) ;
    MyRecord  MyRecordType ;

begin
    delete MyTab ;
    for i in 1 .. MyTxnCnt * MyTxnSize loop
        insert into MyTab( MyPKCol, MyTxtCol )
        values( i, lpad( i, 1000, '0' ) ) ;
    end loop ;
    commit ;
    execute immediate 'analyze table MyTab compute statistics for table for all indexes' ;

    execute immediate 'alter session set tracefile_identifier = MyTrc_' || to_char( MyTxnCnt, 
'FM000009' ) || '_' || to_char( MyTxnSize, 'FM009' ) ;
    execute immediate 'alter session set events ''10046 trace name context forever''' ;
    for i in 1 .. MyTxnCnt loop
        open MyCursor( i * MyTxnSize - MyTxnSize ) ;
        for i in 1 .. MyTxnSize loop
            fetch MyCursor into MyRecord ;
            exit when MyCursor%NOTFOUND ;
            delete MyTab where RowID = MyRecord.RID ;
        end loop ;
        commit ;
        close MyCursor ;
    end loop ;
    execute immediate 'alter session set SQL_Trace = FALSE' ;
end ;
/
disconnect TOM/TOM

connect TOM/TOM
declare
    MyTxnCnt  constant number := 1000 ;
    MyTxnSize constant number :=   50 ;

    cursor MyCursor( p_nPKVal MyTab.MyPKCol%type ) is
        select /*+ FIRST_ROWS */ RowID RID, MyPKCol, MyTxtCol from MyTab where MyPKCol > p_nPKVal 
order by MyPKCol ;

    type MyRecordType is record( RID rowid, MyPKCol number, MyTxtCol varchar2( 1000 ) ) ;

    MyRecord  MyRecordType ;

begin
    delete MyTab ;
    for i in 1 .. MyTxnCnt * MyTxnSize loop
        insert into MyTab( MyPKCol, MyTxtCol )
        values( i, lpad( i, 1000, '0' ) ) ;
    end loop ;
    commit ;
    execute immediate 'analyze table MyTab compute statistics for table for all indexes' ;

    execute immediate 'alter session set tracefile_identifier = MyTrc_' || to_char( MyTxnCnt, 
'FM000009' ) || '_' || to_char( MyTxnSize, 'FM009' ) ;
    execute immediate 'alter session set events ''10046 trace name context forever''' ;
    for i in 1 .. MyTxnCnt loop
        open MyCursor( i * MyTxnSize - MyTxnSize ) ;
        for i in 1 .. MyTxnSize loop
            fetch MyCursor into MyRecord ;
            exit when MyCursor%NOTFOUND ;
            delete MyTab where RowID = MyRecord.RID ;
        end loop ;
        commit ;
        close MyCursor ;
    end loop ;
    execute immediate 'alter session set SQL_Trace = FALSE' ;
end ;
/
disconnect TOM/TOM

Followup June 6, 2005 - 5pm Central time zone:

---------------------------------------------------------------------------------

   0      SELECT STATEMENT Optimizer=HINT: FIRST_ROWS (Cost=7237 Card=50000 
Bytes=26100000)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'MYTAB' (Cost=7237 Card=50000 
Bytes=26100000)
   2    1     INDEX (FULL SCAN) OF 'XPKMYTAB' (UNIQUE) (Cost=94 Card=50000)


select /*+ FIRST_ROWS */ RowID RID, MyPKCol, MyTxtCol from MyTab where MyPKCol > 
:A order by MyPKCol
---------------------------------------------------------------------------------
-------------------
   0      SELECT STATEMENT Optimizer=HINT: FIRST_ROWS (Cost=67 Card=2500 
Bytes=1305000)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'MYTAB' (Cost=67 Card=2500 
Bytes=1305000)
   2    1     INDEX (RANGE SCAN) OF 'XPKMYTAB' (UNIQUE) (Cost=2 Card=450)


totally different plans, I would be amazed IF they did the same IO.

No where clause -- it read the ENTIRE index (index full scan) from front to well, when you told it 
to stop.

With where clause -- it range scanned PART of the index. getting the rows immediately, skipping the 
"front" of the index.


You cannot even remotely expect the IO's from two different plans to be even remotely similar 
except by pure accident.

but the point is ... June 7, 2005 - 1am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

... that the index range to be scanned always started at the very first existing row (remaining 
after delete/commit). that 's why i called it the "full range".

nevertheless the execution plans show different ways of how the index is used -- but how could the 
*measured* IOs be any different although each and every scan (in either case: with and without 
where-clause) always started at the very first existing/remaining row?

did the where-clause approach just benefit from skipping the processing of an increasing number of 
empty index blocks (since its where-clause always pointed to the very first remaining non-empty 
one) whereas the non-where-clause approach did have to "locate" the next entry to continue with 
first?

Followup June 7, 2005 - 8am Central time zone:

the index RANGE scan sure.

the index FULL scan without the predicate -- NOT.


the index FULL SCAN does not start at the very first *existing* row, it started way over in the 
left hand side of the index which could contain deleted entries.  You have what I call a "sweeper" 
index.

http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:2913600659112#24081704877485

June 7, 2005 - 6pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

yes, that 's it: a special case of the sample you gave.

thank you very much.

why my LIO not getting reduced ? June 10, 2005 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: Parag Jayant Patankar from India

Hi Tom,

I am running SQL query with different arraysize on server itself 

SQL> show array
arraysize 15

SQL> @j

7956 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     445546  consistent gets
      11183  physical reads
          0  redo size
     559984  bytes sent via SQL*Net to client
       6481  bytes received via SQL*Net from client
        532  SQL*Net roundtrips to/from client
          5  sorts (memory)
          0  sorts (disk)
       7956  rows processed

Now I have set arraysize 5000

SQL> set arraysize 5000
SQL> @j

7956 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     445546  consistent gets
      11159  physical reads
          0  redo size
     464235  bytes sent via SQL*Net to client
        662  bytes received via SQL*Net from client
          3  SQL*Net roundtrips to/from client
          5  sorts (memory)
          0  sorts (disk)
       7956  rows processed

Now why my LIO ( Consistent gets ) not getting reduced ? my SGA is

SQL> show sga

Total System Global Area  128943960 bytes
Fixed Size                   743256 bytes
Variable Size              75497472 bytes
Database Buffers           50331648 bytes
Redo Buffers                2371584 bytes

regards & thanks
pjp

Followup June 10, 2005 - 11am Central time zone:

because it depends on how the query is processed.  In your case - I see 5 sorts in memory.  Most 
likely, the last row of the query is "found" (processed) before the FIRST row is retrieved.  
Therefore, the entire result set is in TEMP before you get the first row.  IO from TEMP is not a 
"consistent get" (doesn't need to be, we did the consistent get to put the data into temp in the 
first place).


This arraysize setting will affect LIO for queries that are "first row" type queries only -- 
generally, ones without say a GROUP BY (group by is generally an "all rows" query -- you have to 
get an process all of the rows before you get the first row back from the database.


If you have effective Oracle by Design - I cover what I call "fast return" and "slow return" 
queries in a lot more detail.

Few Doubts July 3, 2005 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: Deepak from India

Hi Tom,

I tried to test the consistent get concept on my database. The details given below...

SCOTT> set arraysize 5
SCOTT> drop table emp1;

Table dropped.

SCOTT> create table emp1
  2  as
  3  select * from emp;

Table created.

SQL> select value from v$system_parameter
  2  where name='db_block_size';

VALUE
-----------------------------------------
2048

SYSTEM> select blocks from dba_segments
  2  where owner='SCOTT'
  3  and segment_name='EMP';

    BLOCKS
----------
         5

SCOTT> set autotrace traceonly statistics
SCOTT> select count(*) from emp1;


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          3  consistent gets
          1  physical reads
          0  redo size
        379  bytes sent via SQL*Net to client
        503  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

As per you, Consistent gets should be ...

Consistent gets=14/5 (14 rows, arraysize 5) + 5 (blocks) = 8

Can you please explain the discrepancy? Where did I go wrong?

SQL> select * from emp1;

14 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          6  consistent gets
          0  physical reads
          0  redo size
       1582  bytes sent via SQL*Net to client
        525  bytes received via SQL*Net from client
          4  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         14  rows processed

Also in the above SQL query a FTS is done but the consistent gets varry.

PLEASE NOTE: No other sessions are doing any operations on the said table EMP1 and no other 
operations are going on.

Please clear my doubt. 

Thanks in advance.

Followup July 3, 2005 - 12pm Central time zone:

select count(*)  <<<<====

think about how many rows that processes, sends back.  hmmm, how many round trips happened there...


In the second case, you see exactly what I described?  There are blocks gotten just because "it is 
a full scan" and you did some consistent gets just to figure out what to full scan:

ops$tkyte@ORA9IR2> set arraysize 5
ops$tkyte@ORA9IR2> set autotrace traceonly
ops$tkyte@ORA9IR2> select * from emp where empno=0;
 
no rows selected
 
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'EMP'
 
 
 
 
Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          3  consistent gets
          0  physical reads
          0  redo size
        589  bytes sent via SQL*Net to client
        368  bytes received via SQL*Net from client
          1  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          0  rows processed
 
two of them in fact, for zero rows we did 2 gets to figure out that emp is a one block table and 
read it finding nothing...

ops$tkyte@ORA9IR2> select * from emp;
 
14 rows selected.
 
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'EMP'
 
 
 
 
Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          6  consistent gets
          0  physical reads
          0  redo size
       1582  bytes sent via SQL*Net to client
        521  bytes received via SQL*Net from client
          4  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         14  rows processed
 
ops$tkyte@ORA9IR2> set autotrace off

Now, we did 2 LIO's to figure out what to read - leaving 4 LIO's to account for.  SQLPlus (you 
can turn on trace to see this) fetches the first row AND THEN starts array fetching so

1 LIO for row 1
1 LIO for rows 2, 3, 4, 5, 6
1 LIO for rows 7, 8, 9, 10, 11
1 LIO for rows 12, 13, 14

Correction July 3, 2005 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: Deepak from India

Please correct 

SYSTEM> select blocks from dba_segments
  2  where owner='SCOTT'
  3  and segment_name='EMP';

    BLOCKS
----------
         5

to 

SYSTEM> select blocks from dba_segments
  2  where owner='SCOTT'
  3  and segment_name='EMP1';

    BLOCKS
----------
         5

Consistent Gets & DB Block Gets July 14, 2005 - 1am Central time zone

Bookmark | Bottom | Top

Reviewer: Deepak from India

Hi Tom,

You have helped me in understanding the concepts a lot. This time I am confused regarding the 
proper definition of Consistent Gets & Db Block Gets.

Few definitions say:

Consistent gets are the no. of blocks obtainted from the buffer cache in consistent mode.

Others say:

Consistent gets are Blocks obtained in consistent mode (irrespective of their source).

Please make me understand with simple examples. 

Thanks in advance...

Followup July 14, 2005 - 10am Central time zone:

consistent gets come from buffer cache.  Don't know where the "irrespective" comes in.

a consistent get may well have been preceded by a physical io, but the get is from the buffer 
cache.

July 23, 2005 - 6am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom:

Look at this

http://asktom.oracle.com/pls/ask/f?p=4950:8:10076522618138430629::NO::F4950_P8_DISPLAYID,F4950_P8_CR
ITERIA:32812348052

----------------------
ops$tkyte@ORA817DEV.US.ORACLE.COM> create table t as select * from all_objects;
Table created.

ops$tkyte@ORA817DEV.US.ORACLE.COM> set autotrace on
ops$tkyte@ORA817DEV.US.ORACLE.COM> select *
  2   from ( select owner, object_name, object_id
  3                    from t
  4                   order by object_id desc )
  5   where rownum < 10;

9 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   COUNT (STOPKEY)
   2    1     VIEW
   3    2       SORT (ORDER BY STOPKEY)
   4    3         TABLE ACCESS (FULL) OF 'T'

Statistics
----------------------------------------------------------
          0  recursive calls
         12  db block gets
        238  consistent gets
        234  physical reads
          0  redo size
        936  bytes sent via SQL*Net to client
        430  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
          9  rows processed

ops$tkyte@ORA817DEV.US.ORACLE.COM> exec show_space('T' )
Free Blocks.............................0
Total Blocks............................256
Total Bytes.............................2097152
Unused Blocks...........................21
Unused Bytes............................172032
Last Used Ext FileId....................7
Last Used Ext BlockId...................246537
Last Used Block.........................43

PL/SQL procedure successfully completed.

We can see the 238 blocks = 256     - 21     - 1           + 4
                              Total     Not    Overhead      full scan
                              Blocks    Used   for Table     read seg. hdr       
                                                      four times

----------------------
You say 238(blocks)=256-21-1+4(full scan read seg.hdr four times)
I'm puzzled about adding '4 full scan read seg.hdr four times'
How to calculate the result4?
Could you explain the formula metioned above in particular?

Thanks

Followup July 23, 2005 - 9am Central time zone:

the 4 will vary from release to release.  It was the cost of donig a full scan - to read the 
segment header to figure out what to read.

Select vs delete get information December 7, 2005 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: Rob H from Winnipeg, MB, Canada

I'm confused regarding the significant differences in gets for a select vs a delete:

SQL> select c1 from USER.TABLE1 where c1='000000042000250';

C1
---------------
000000042000250

Elapsed: 00:00:00.62

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=RULE
   1    0   INDEX (UNIQUE SCAN) OF 'IT72' (UNIQUE)




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          3  consistent gets
          2  physical reads
          0  redo size
        498  bytes sent via SQL*Net to client
        655  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

SQL> delete from USER.TABLE1  where c1='000000042000250';

1 row deleted.

Elapsed: 00:00:16.88

Execution Plan
----------------------------------------------------------
   0      DELETE STATEMENT Optimizer=RULE
   1    0   DELETE OF 'TABLE1'
   2    1     INDEX (UNIQUE SCAN) OF 'IT72' (UNIQUE)




Statistics
----------------------------------------------------------
          0  recursive calls
         51  db block gets
       6572  consistent gets
       4262  physical reads
      22288  redo size
        799  bytes sent via SQL*Net to client
        703  bytes received via SQL*Net from client
          3  SQL*Net roundtrips to/from client
          3  sorts (memory)
          0  sorts (disk)
          1  rows processed

SQL> rollback;

Rollback complete.

Elapsed: 00:00:00.04
SQL>

There are no triggers on the table, and the only constraint is a not null on the column.  There are 
approx 15 million rows in the table and it has been recently analyzed.  There is a unique index 
'IT72' on the C1 column.  Using OPTIMIZER_MODE=RULE or CHOOSE or FIRST_ROWS yields similar results. 
 

Any help with the significant difference between the two?

Followup December 8, 2005 - 1am Central time zone:

sure, I can reproduce that:

ops$tkyte@ORA9IR2> set autotrace on
ops$tkyte@ORA9IR2> select * from p where x = 1;

         X
----------
         1


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   INDEX (UNIQUE SCAN) OF 'SYS_C003797' (UNIQUE)




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          1  consistent gets
          0  physical reads
          0  redo size
        372  bytes sent via SQL*Net to client
        503  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

ops$tkyte@ORA9IR2> delete from p where x = 1;

1 row deleted.


Execution Plan
----------------------------------------------------------
   0      DELETE STATEMENT Optimizer=CHOOSE
   1    0   DELETE OF 'P'
   2    1     INDEX (UNIQUE SCAN) OF 'SYS_C003797' (UNIQUE)




Statistics
----------------------------------------------------------
        219  recursive calls
          7  db block gets
        488  consistent gets
          0  physical reads
        556  redo size
        790  bytes sent via SQL*Net to client
        798  bytes received via SQL*Net from client
          4  SQL*Net roundtrips to/from client
          5  sorts (memory)
          0  sorts (disk)
          1  rows processed

ops$tkyte@ORA9IR2> set autotrace off
ops$tkyte@ORA9IR2>



I should tell you what I did though to make that happen (like you should with me, set up a test 
case please....)


create table p ( x int primary key );
insert into p values ( 1 );
insert into p values ( 2 );
create table c
as
select 2 x, all_objects.* from all_objects;


any unindexed foreign keys sitting out there in your system?

Forgot ... December 8, 2005 - 9am Central time zone

Bookmark | Bottom | Top

Reviewer: Bob B from Mechanicsburg, PA

Tom,

I believe you forgot to add the FK from c to p above .

Followup December 8, 2005 - 11am Central time zone:

indeed, missed the 

alter table c add constraint c_fk foreign key (x) references p
/

in the cut and paste.

No Primary Key..... December 9, 2005 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: Rob H from Winnipeg MB, Canada

The only constraints on the table are check constraints....

SQL> select owner, CONSTRAINT_NAME, CONSTRAINT_TYPE, R_CONSTRAINT_NAME from dba_constraints where 
table_name ='TABLE1';

OWNER                          CONSTRAINT_NAME                C R_CONSTRAINT_NAME
------------------------------ ------------------------------ - ------------------------------
USER                           SYS_C003566                    C
USER                           SYS_C003567                    C
USER                           SYS_C003568                    C
USER                           SYS_C003569                    C
USER                           SYS_C003570                    C
USER                           SYS_C003571                    C

6 rows selected.


I thought you needed a primary key to create a foreign one.

Followup December 9, 2005 - 1pm Central time zone:

so, can you create a table, and reproduce the issue.  help me help you - give me a test case to 
work with.

all I said was "given what you've given me so far, yes, I can make a delete do lots more work...."

so, give me a test case to work with - you should be table to create a table just like table1, 
insert some data and reproduce the issue.  Then we can look at it.

Thanks December 9, 2005 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: Rob H from Winnipeg MB, Canada

Yes, I am in the process of doing so and am wondering if the datatype has something to do with it.

Varchar2(15 CHAR)

I'll continue to work on this.

Thanks

Followup December 10, 2005 - 4am Central time zone:

doubtful, more likely something hidden like auditing, fine grained access control, a missed 
trigger, a foreign key - something like that that you will find as you make the test case ;)

Indexes? December 12, 2005 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: Rob H from Winnipeg MB Canada

I haven't looked into fine grained auditing (yet) but I have looked and no foriegn keys, no 
triggers.  I did find this out about the indexes on the table:


INDEX_NAME   BLEVEL DISTINCT_KEYS LEAF_BLOCKS AVG_DATA_BLOCKS_PER_KEY
---------- -------- ------------- ----------- -----------------------
IDX1              8            48       25033                   80392
IDX2              3        213314      101890                      56
IDX3              8             8       16623                  391445
IDX4              2             3       87970                  315863
IDX5              2      15470610       93170                       
1
Now to me, IDX1, IDX3, and IDX4 could be a problem.

Followup December 12, 2005 - 4pm Central time zone:

indeed!  but how so given 1 row deleted :)

Delete causing large index reads? December 12, 2005 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: Rob H from Winnipeg MB Canada

Sorry, your response seems a little unclear (at least the "but how so given 1 row deleted").

Am I right in assuming that the large consistent and physical reads in the explain are really from 
the indexes being read and adjusted due to the single row delete?   Similarly for the redo (tons of 
archiving being done).  

I have to delete 10 million rows from a 24x7 online table.  I would like to create table as select, 
truncate, insert but this has to be done online.  It takes 6 minutes and nearly a gig of UNDO to 
remove 1,000 rows (the UNDO usage is exponential in time).   This is causing a lot of work for the 
database.  I am trying to find a way to not impact the online production while purging the data 
safely with the least amount of "work" by the database. :(

Followup December 12, 2005 - 5pm Central time zone:

I suppose it could be the indexes, but - i would have to set up something to simulate it.

       6572  consistent gets
       4262  physical reads

for 1 row deleted, they would have to be some seriously damaged indexes to do that.


Data purging via delete - we need to talk about that, there are better ways.  Tell you what you are 
deleting 10,000,000 rows - out of how many?  and by what criteria? and is this the FIRST TIME or 
the ONLY TIME you are going to do this? what is the table structure and what do the indexes look 
like on it (no tablespaces/storage, just simple ddl)

Bitmaps December 13, 2005 - 12am Central time zone

Bookmark | Bottom | Top

Reviewer: Rob H from Winnipeg MB Canada

After some review I think I found the problem.  I never bothered to check because I made the 
assumption that no one would put bitmap indexes on a table that gets 100,000 inserts a day....

And so, I think you have the answer....

Followup December 13, 2005 - 9am Central time zone:

that indeed would be it.

surprised you did not have deadlocks all over the place - everything must be running faster now.

but still, answer the questions from above:

Data purging via delete - we need to talk about that, there are better ways.  
Tell you what you are deleting 10,000,000 rows - out of how many?  and by what 
criteria? and is this the FIRST TIME or the ONLY TIME you are going to do this?

Info December 13, 2005 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: Rob H from Winnipeg MB

surprised you did not have deadlocks all over the place <- YES!

I have not dropped the indexes yet....The application "apparently" requires it. It is a black box
type app where there is columns C1..C# and tables named T1..T#.

The delete process is

Data purging via delete - we need to talk about that, there are better ways.

They are deleting 10,000,000 out of 16,000,000 (approx 50,000-100,000 rows are added daily). I
would prefer to move the "wanted" data to another table, drop the indexes, truncate the table, add
the "wanted" data back and re-create the indexes (excluding the bitmaps).

The criteria for deletion is where C6 (I think a status field, Number datatype with 3 distinct
values) and C7 < some number (where the number is datetime converted to epoch number).

I have been asked to help with this application. I did not help develop it, I have been pulled in
to alleviate problems with performance (wow, I now see so many chapters of your book coming back to
haunt me). From a design standpoint I see several glaring issues.

1. Bitmap Indexes and ORA-00060's
2. The delete is really a delete from the primary table, then 2 other "children" but yet there is
no referential integrity or constraints (why Tom, are people so scared of data integrity).
3. No one seems to know anything about the design or how any of it works.....
4. There are 101 columns on the primary table. Here are some stats from dba_tables:

NUM_ROWS EMPTY_BLOCKS AVG_SPACE AVG_ROW_LEN
---------- ------------ ---------- -----------
15469430 1216 1011 427

I *was* using a cursor to select a group of records from the primary table, delete from the 3
tables using 'execute immediate ... using ' single row by single row. This obviously generated a
stunning amount of REDO. I also could only update 1,000 rows in the table before my UNDO TBS was
nearly full. This was only an initial benchmark for a test, but lead me to the post I put above of
large consistent and physical gets for a single delete. And now you know how I got here...

Followup December 13, 2005 - 1pm Central time zone:

... "I would prefer to move the "wanted" data to another table, "....

me too. why can't we do that - it would take just a couple of minutes.


the bitmaps should be replaced with b*trees, your indexes are likely many times bigger than they 
should be and are not making things "faster"



2) because they don't know better.  They have been scared off by people that talk like they know 
what they are talking about saying integrity "slows down" databases which is complete $@#.

But how December 13, 2005 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: Rob H from Winnipeg MB Canada

How do I ensure that no records are lost during the create table and the truncate.  A trigger 
perhaps?   This table and its contents must be available 24x7 (appearently).

Followup December 13, 2005 - 5pm Central time zone:

this would be a offline operation.

or:

create new empty table
put indexes on it, grant on it, etc....

then:

rename old_table to HOLD;
rename new_table to old_table;
insert into old_table select * from hold where <keep these is true>;


it would be "offline" for the duration of the two renames only.


or (my preference)

design a new approach to this alltogether so you never are faced with this again (eg: 
partitioning), and then create a new partitioned table and do the switch...

Are "Db Block Gets" data block reads ? January 12, 2006 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: MckDBA from NY

Hello Tom,
great Site, great fan of yours.

I learnt a lot from this thread about the LIO's. Just when I think that I know it all and when I 
replay in my mind, I get lost a bit.

Need clarification on your earlier statement -
" .... db block gets are blocks gotten in CURRENT (as opposed to consisten) mode.  They are block 
gets we did to discover the extent map for the table (to know what to read).  We get that in 
"current, as of right now" mode instead of "in consistent read, as of the point in time the query 
began" mode ....."

So, lets say we are updating table T. So, the "db block gets" are not the actual reads of table T 
data blocks but more of meta blocks, which has information where to read the real Table T data 
blocks? But, the consistent gets are access to the real Table T data blocks.

Am I right in my understanding of DB Block Gets? Appreciate your response.

Thanks

Followup January 12, 2006 - 11am Central time zone:

no, I used the segment header reads as an example.  We do current mode reads in other cases as well 
-- UPDATE for example will use consistent read to FIND a block - it will use current mode reads to 
actually modify the block (you have to modify the current copy of the block after all)


ops$tkyte@ORA10GR2> create table t as select * from all_objects;

Table created.

ops$tkyte@ORA10GR2> set autotrace on statistics;
ops$tkyte@ORA10GR2> update t set object_name = lower(object_name);

49937 rows updated.


Statistics
----------------------------------------------------------
        642  recursive calls
      52769  db block gets
      51340  consistent gets
        686  physical reads
   17110600  redo size
        920  bytes sent via SQL*Net to client
        960  bytes received via SQL*Net from client
          6  SQL*Net roundtrips to/from client
          9  sorts (memory)
          0  sorts (disk)
      49937  rows processed

how about this update ? January 30, 2006 - 12am Central time zone

Bookmark | Bottom | Top

Reviewer: MckDBA from NY

Hey Tom,
I think I got it now.
SunOS, 9.2.0.5 environment.

So, I went ahead and tried a simple test to confirm my understanding.

SQL> create table test as select * from dba_objects;
Table created.

SQL> exec show_space('TEST');
Free Blocks............................. 0
Total Blocks............................ 384 <-
Total Bytes............................. 3,145,728
Total MBytes............................ 3
Unused Blocks........................... 26 <-
Unused Bytes............................ 212,992
Last Used Ext FileId.................... 1
Last Used Ext BlockId................... 39,433
Last Used Block......................... 102

So, the actual # of blocks used = 358

SQL> set autotrace traceonly statitics
SQL> select * from test;
26127 rows selected.

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
2079 consistent gets
0 physical reads
0 redo size
1933448 bytes sent via SQL*Net to client
19806 bytes received via SQL*Net from client
1743 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
26127 rows processed

I could reason out the 2097 consistent gets(from an earlier thread on this site) by calculating :

# of blocks used by table + (table rows/fetch_size)
358+26127/15 = 2100(approximately equivalent to 2079).

Then, I tried to reason out the numbers in the update statement.
SQL> update test set owner=lower(owner);

26127 rows updated.

Statistics
----------------------------------------------------------
170 recursive calls
28176 db block gets
446 consistent gets
210 physical reads
11291096 redo size
1023 bytes sent via SQL*Net to client
1047 bytes received via SQL*Net from client
4 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
26127 rows processed

From what I understand, that update would run more like

update (select * from test) set owner=lower(owner).

The "select * from test" would be consistent reads. Makes sense. The rest of the update would be
current reads.

1) So, I was expecting to see the consistent reads to be around 2100, but the actual # shown above
is 446. What am I missing ?

2) The update statement acted on 26127 rows, so I expected to see around the same number for
current reads. But the actual number was 28176. Why is there a difference of about 2000 ?

Appreciate your response

Followup January 30, 2006 - 2am Central time zone:

1) there is no "array fetching" with the update - the update is a single call, all rows processed 
(or not)

2) transaction (undo) work was taking place as well.  look at used_ublk in v$transaction next time. 
 you were generating undo.

Now im very puzzled. February 8, 2006 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: Notna from SG

Hi Tom, 

I have this query that I dont understand the results. When I set hash_join_enabled=false consistent 
gets is low but more sorts more, and executes less than 1.50 minutes. But when I set the 
hash_join_enabled to true, Consistent gets is high, no sorts, but executes faster. Can you tell me 
what's the best execution path best for my query and could you please enlighten me why?

Cheers,
NOTNA




test@tun> set autotrace traceonly explain statistics
test@tun> l
  1   select
  2            int_cap               ,
  3            ccy,
  4            int_accrued_ctd       ,
  5            auth_int_accrued_ctd  ,
  6            int_accrued_mtd       ,
  7            auth_int_accrued_mtd  ,
  8            int_adj               ,
  9            auth_int_adj          ,
 10            int_adj_mtd           ,
 11            auth_int_adj_mtd      ,
 12            int_accrued_post      ,
 13            auth_int_accrued_post ,
 14            int_adj_post          ,
 15            auth_int_adj_post
 16           from accounts act,
 17                 interest int
 18           where act.seq_key = int.seq_key
 19*          and   act.deposit_type <> 'T'
test@tun> alter session set hash_join_enabled=false;

Session altered.

Elapsed: 00:00:00.00
test@tun>  select
  2            int_cap               ,
  3            ccy,
  4            int_accrued_ctd       ,
  5            auth_int_accrued_ctd  ,
  6            int_accrued_mtd       ,
  7            auth_int_accrued_mtd  ,
  8            int_adj               ,
  9            auth_int_adj          ,
 10            int_adj_mtd           ,
 11            auth_int_adj_mtd      ,
 12            int_accrued_post      ,
 13            auth_int_accrued_post ,
 14            int_adj_post          ,
 15            auth_int_adj_post
 16           from accounts act,
 17                 interest int
 18           where act.seq_key = int.seq_key
 19           and   act.deposit_type <> 'T'
 20  /

1203824 rows selected.

Elapsed: 00:01:50.92

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=70522 Card=1691348 B
          ytes=214801196)

   1    0   PARTITION RANGE (ALL)
   2    1     MERGE JOIN (Cost=70522 Card=1691348 Bytes=214801196)
   3    2       SORT (JOIN) (Cost=15651 Card=1691348 Bytes=20296176)
   4    3         TABLE ACCESS (FULL) OF 'ACCOUNTS' (Cost=9039 Card=169
          1348 Bytes=20296176)

   5    2       SORT (JOIN) (Cost=54871 Card=2355244 Bytes=270853060)
   6    5         TABLE ACCESS (FULL) OF 'INTEREST' (Cost=3250 Card=2
          355244 Bytes=270853060)





Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     196701  consistent gets
     169011  physical reads
          0  redo size
   12587484  bytes sent via SQL*Net to client
     562055  bytes received via SQL*Net from client
      80256  SQL*Net roundtrips to/from client
        211  sorts (memory)
          0  sorts (disk)
    1203824  rows processed

test@tun> alter session set hash_join_enabled=true;

Session altered.

Elapsed: 00:00:00.06
test@tun>  select
  2            int_cap               ,
  3            ccy,
  4            int_accrued_ctd       ,
  5            auth_int_accrued_ctd  ,
  6            int_accrued_mtd       ,
  7            auth_int_accrued_mtd  ,
  8            int_adj               ,
  9            auth_int_adj          ,
 10            int_adj_mtd           ,
 11            auth_int_adj_mtd      ,
 12            int_accrued_post      ,
 13            auth_int_accrued_post ,
 14            int_adj_post          ,
 15            auth_int_adj_post
 16           from accounts act,
 17                 interest int
 18           where act.seq_key = int.seq_key
 19           and   act.deposit_type <> 'T'
 20  /

1203824 rows selected.

Elapsed: 00:01:16.67

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=14261 Card=1691348 B
          ytes=214801196)

   1    0   PARTITION RANGE (ALL)
   2    1     HASH JOIN (Cost=14261 Card=1691348 Bytes=214801196)
   3    2       TABLE ACCESS (FULL) OF 'ACCOUNTS' (Cost=9039 Card=16913
          48 Bytes=20296176)

   4    2       TABLE ACCESS (FULL) OF 'INTEREST' (Cost=3250 Card=235
          5244 Bytes=270853060)





Statistics
----------------------------------------------------------
         91  recursive calls
          0  db block gets
     275283  consistent gets
     169990  physical reads
          0  redo size
   13233434  bytes sent via SQL*Net to client
     562055  bytes received via SQL*Net from client
      80256  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
    1203824  rows processed

test@tun>

Followup February 8, 2006 - 11am Central time zone:

what happens when you 

set arraysize 100
or
set arraysize 500

in sqlplus with the hash join (the somewhat obvious answer is "likely the hash join"

the first one reads the tables once, sorts them into temp and returns the data from temp (no 
logical IO on temp)

the second one reads and hashes one of the tables - and the reads the second table and hash joins 
to the other one.  You are likely having more than 15 rows per block on the second table being 
processed - hence you are getting and "re-getting" blocks.

Increasing the array fetch size to something more appropriate for a query that returns 1.2 million 
records will reduce the LIO.




big_table@ORA10GR2> set autotrace traceonly statistics;
big_table@ORA10GR2> select * from big_table;

1000000 rows selected.


Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
      80146  consistent gets
      14061  physical reads
        608  redo size
  112310317  bytes sent via SQL*Net to client
     733711  bytes received via SQL*Net from client
      66668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
    1000000  rows processed

big_table@ORA10GR2> set arraysize 100
big_table@ORA10GR2> select * from big_table;

1000000 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
      24302  consistent gets
      13574  physical reads
        564  redo size
  104943607  bytes sent via SQL*Net to client
     110374  bytes received via SQL*Net from client
      10001  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
    1000000  rows processed

Still confused... February 8, 2006 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: NOTNA from SG

here are the results for 500 Array size :

test@tun> set autotrace traceonly explain statistics
test@tun> alter session set hash_join_enabled=false;

Session altered.

Elapsed: 00:00:00.00
test@tun> set array size 500
SP2-0268: arraysize option not a valid number
test@tun> set arraysize 500
test@tun>  select
  2       int_cap               ,
  3       ccy,
  4       int_accrued_ctd       ,
  5       auth_int_accrued_ctd  ,
  6       int_accrued_mtd       ,
  7       auth_int_accrued_mtd  ,
  8       int_adj               ,
  9       auth_int_adj          ,
 10       int_adj_mtd           ,
 11       auth_int_adj_mtd      ,
 12       int_accrued_post      ,
 13       auth_int_accrued_post ,
 14       int_adj_post          ,
 15       auth_int_adj_post
 16      from accounts act,
 17            interest int
 18      where act.seq_key = int.seq_key
 19      and   act.deposit_type <> 'T'
 20  .
test@tun> /

1203824 rows selected.

Elapsed: 00:04:28.03

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=70522 Card=1691348 B
          ytes=214801196)

   1    0   PARTITION RANGE (ALL)
   2    1     MERGE JOIN (Cost=70522 Card=1691348 Bytes=214801196)
   3    2       SORT (JOIN) (Cost=15651 Card=1691348 Bytes=20296176)
   4    3         TABLE ACCESS (FULL) OF 'ACCOUNTS' (Cost=9039 Card=169
          1348 Bytes=20296176)

   5    2       SORT (JOIN) (Cost=54871 Card=2355244 Bytes=270853060)
   6    5         TABLE ACCESS (FULL) OF 'INTEREST' (Cost=3250 Card=2
          355244 Bytes=270853060)





Statistics
----------------------------------------------------------
         96  recursive calls
          0  db block gets
    2652763  consistent gets
     212854  physical reads
    2516020  redo size
    9265994  bytes sent via SQL*Net to client
      19534  bytes received via SQL*Net from client
       2409  SQL*Net roundtrips to/from client
        213  sorts (memory)
          0  sorts (disk)
    1203824  rows processed

test@tun> alter session set hash_join_enabled=true;

Session altered.

Elapsed: 00:00:00.00
test@tun> set arraysize 500
test@tun>  select
  2       int_cap               ,
  3       ccy,
  4       int_accrued_ctd       ,
  5       auth_int_accrued_ctd  ,
  6       int_accrued_mtd       ,
  7       auth_int_accrued_mtd  ,
  8       int_adj               ,
  9       auth_int_adj          ,
 10       int_adj_mtd           ,
 11       auth_int_adj_mtd      ,
 12       int_accrued_post      ,
 13       auth_int_accrued_post ,
 14       int_adj_post          ,
 15       auth_int_adj_post
 16      from ACCOUNTS act,
 17            INTEREST int
 18      where act.seq_key = int.seq_key
 19      and   act.deposit_type <> 'T'
 20  /

1203824 rows selected.

Elapsed: 00:04:15.75

Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=14261 Card=1691348 B
          ytes=214801196)

   1    0   PARTITION RANGE (ALL)
   2    1     HASH JOIN (Cost=14261 Card=1691348 Bytes=214801196)
   3    2       TABLE ACCESS (FULL) OF 'ACCOUNTS' (Cost=9039 Card=16913
          48 Bytes=20296176)

   4    2       TABLE ACCESS (FULL) OF 'INTEREST' (Cost=3250 Card=235
          5244 Bytes=270853060)





Statistics
----------------------------------------------------------
         91  recursive calls
          0  db block gets
    2655118  consistent gets
     216586  physical reads
    2515968  redo size
    9269898  bytes sent via SQL*Net to client
      19534  bytes received via SQL*Net from client
       2409  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
    1203824  rows processed

test@tun>

So, does it mean setting the hash_join_enabled to true for this query is the correct one?

Thanks...

Followup February 9, 2006 - 4am Central time zone:

what are you confused about first of all?

I suggested 100, not 500.

Confused about Array Size February 8, 2006 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: Sajid raza from New York, NY USA

> the first one reads the tables once, sorts them into temp 
> and returns the data 
> from temp (no logical IO on temp)
>
> the second one reads and hashes one of the tables - 
> and the reads the second table and hash joins to the 
> other one. You are likely having more than 15 rows per 
> block on the second table being processed - hence you are 
> getting and "re-getting" blocks.

I have clearly missed something, but isn't the array size a feature of the client application 
(sqlplus)? Why should it affect the outcome of the second step in the hash-join, after all, isn't 
that an 'internal' operation.

Followup February 9, 2006 - 4am Central time zone:

hash joins work like this:


a) read one of the tables from start to finish and hash it.
b) wait for client to ask for some rows
c) when client asks for some rows - read some rows from table2


it is step c) that would be affected by the client array fetch size.  If the client asks for a 
single row (and lets say that table1 and table2 have a one to one relationship to eachother to make 
the math easy) AND table2 has 100 rows per block - you end up getting the first block 100 times.

If the client on the other hand array fetches 100 rows, you end up getting block one from the 
buffer cache once.

how can index scan better than full table scan in this case? February 10, 2006 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi

I have difficulties understanding why this is happening, please check the queries, I have indexed 
DEPTNO column:

SQL> select /* index(emp emp_idx1) */ *
  2  from emp where deptno in (10, 20, 30, 40);

     EMPNO ENAME      JOB              MGR HIREDATE                 SAL       COMM     DEPTNO
---------- ---------- --------- ---------- ----------------- ---------- ---------- ----------
      7782 CLARK      MANAGER         7839 19810609 00:00:00       2450                    10
      7839 KING       PRESIDENT            19811117 00:00:00       5000                    10
      6003 MILLER     CLERK           7782 19820123 00:00:00       1300                    10
      7369 SMITH      CLERK           7902 19801217 00:00:00        800                    20
      7566 JONES      MANAGER         7839 19810402 00:00:00       2975                    20
      7788 SCOTT      ANALYST         7566 19821209 00:00:00       3000                    20
      7876 ADAMS      CLERK           7788 19830112 00:00:00       1100                    20
      7902 FORD       ANALYST         7566 19811203 00:00:00       3000                    20
      7499 ALLEN      SALESMAN        7698 19810220 00:00:00       1600        300         30
      7521 WARD       SALESMAN        7698 19810222 00:00:00       1250        500         30
      7654 MARTIN     SALESMAN        7698 19810928 00:00:00       1250       1400         30
      7698 BLAKE      MANAGER         7839 19810501 00:00:00       2850                    30
      7844 TURNER     SALESMAN        7698 19810908 00:00:00       1500          0         30
      7900 JAMES      CLERK           7698 19811203 00:00:00        950                    30

14 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=14 Bytes=448)
   1    0   INLIST ITERATOR
   2    1     TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=2 Card=14 Bytes=448)
   3    2       INDEX (RANGE SCAN) OF 'EMP_IDX1' (NON-UNIQUE) (Cost=1 Card=14)

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          9  consistent gets
          0  physical reads
          0  redo size
       1278  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         14  rows processed



SQL> select * from emp;

     EMPNO ENAME      JOB              MGR HIREDATE                 SAL       COMM     DEPTNO
---------- ---------- --------- ---------- ----------------- ---------- ---------- ----------
      7369 SMITH      CLERK           7902 19801217 00:00:00        800                    20
      7499 ALLEN      SALESMAN        7698 19810220 00:00:00       1600        300         30
      7521 WARD       SALESMAN        7698 19810222 00:00:00       1250        500         30
      7566 JONES      MANAGER         7839 19810402 00:00:00       2975                    20
      7654 MARTIN     SALESMAN        7698 19810928 00:00:00       1250       1400         30
      7698 BLAKE      MANAGER         7839 19810501 00:00:00       2850                    30
      7782 CLARK      MANAGER         7839 19810609 00:00:00       2450                    10
      7788 SCOTT      ANALYST         7566 19821209 00:00:00       3000                    20
      7839 KING       PRESIDENT            19811117 00:00:00       5000                    10
      7844 TURNER     SALESMAN        7698 19810908 00:00:00       1500          0         30
      7876 ADAMS      CLERK           7788 19830112 00:00:00       1100                    20
      7900 JAMES      CLERK           7698 19811203 00:00:00        950                    30
      7902 FORD       ANALYST         7566 19811203 00:00:00       3000                    20
      6003 MILLER     CLERK           7782 19820123 00:00:00       1300                    10

14 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=4 Card=14 Bytes=448)
   1    0   TABLE ACCESS (FULL) OF 'EMP' (Cost=4 Card=14 Bytes=448)

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
         16  consistent gets
          0  physical reads
          0  redo size
       1307  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         14  rows processed


Why index scan only needs 9 consistent gets and full table more? And I am reading same number of 
rows?!

Followup February 10, 2006 - 1pm Central time zone:

your stats are likely out of date.

but you include nothing really to reproduce with - I would never expect scott.emp to have 16'ish 
blocks.

TEST RESULTS February 12, 2006 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

<code>Hi

I dropped the table and recreated, gaher the stats with both analyze and dbms_stats and getting same result. Please bear in mind my db_block_size is 4K.
I carried the test with segment space management auto and manual. It seems that using AUTO is what is causing 16 consistent gets reading the EMP table.

The strange thing is, with segment space management manual Oracle still prefers index scan when full scan is only consumes half consistent gets!

This is Oracle 9.2.0.6 32 bit on Red Hat Enterprise Linux 3

CREATE TABLE EMP
(EMPNO NUMBER(4) NOT NULL,
ENAME VARCHAR2(10),
JOB VARCHAR2(9),
MGR NUMBER(4),
HIREDATE DATE,
SAL NUMBER(7, 2),
COMM NUMBER(7, 2),
DEPTNO NUMBER(2));

INSERT INTO EMP VALUES
(7369, 'SMITH', 'CLERK', 7902,
TO_DATE('17-DEC-1980', 'DD-MON-YYYY'), 800, NULL, 20);
INSERT INTO EMP VALUES
(7499, 'ALLEN', 'SALESMAN', 7698,
TO_DATE('20-FEB-1981', 'DD-MON-YYYY'), 1600, 300, 30);
INSERT INTO EMP VALUES
(7521, 'WARD', 'SALESMAN', 7698,
TO_DATE('22-FEB-1981', 'DD-MON-YYYY'), 1250, 500, 30);
INSERT INTO EMP VALUES
(7566, 'JONES', 'MANAGER', 7839,
TO_DATE('2-APR-1981', 'DD-MON-YYYY'), 2975, NULL, 20);
INSERT INTO EMP VALUES
(7654, 'MARTIN', 'SALESMAN', 7698,
TO_DATE('28-SEP-1981', 'DD-MON-YYYY'), 1250, 1400, 30);
INSERT INTO EMP VALUES
(7698, 'BLAKE', 'MANAGER', 7839,
TO_DATE('1-MAY-1981', 'DD-MON-YYYY'), 2850, NULL, 30);
INSERT INTO EMP VALUES
(7782, 'CLARK', 'MANAGER', 7839,
TO_DATE('9-JUN-1981', 'DD-MON-YYYY'), 2450, NULL, 10);
INSERT INTO EMP VALUES
(7788, 'SCOTT', 'ANALYST', 7566,
TO_DATE('09-DEC-1982', 'DD-MON-YYYY'), 3000, NULL, 20);
INSERT INTO EMP VALUES
(7839, 'KING', 'PRESIDENT', NULL,
TO_DATE('17-NOV-1981', 'DD-MON-YYYY'), 5000, NULL, 10);
INSERT INTO EMP VALUES
(7844, 'TURNER', 'SALESMAN', 7698,
TO_DATE('8-SEP-1981', 'DD-MON-YYYY'), 1500, 0, 30);
INSERT INTO EMP VALUES
(7876, 'ADAMS', 'CLERK', 7788,
TO_DATE('12-JAN-1983', 'DD-MON-YYYY'), 1100, NULL, 20);
INSERT INTO EMP VALUES
(7900, 'JAMES', 'CLERK', 7698,
TO_DATE('3-DEC-1981', 'DD-MON-YYYY'), 950, NULL, 30);
INSERT INTO EMP VALUES
(7902, 'FORD', 'ANALYST', 7566,
TO_DATE('3-DEC-1981', 'DD-MON-YYYY'), 3000, NULL, 20);
INSERT INTO EMP VALUES
(7934, 'MILLER', 'CLERK', 7782,
TO_DATE('23-JAN-1982', 'DD-MON-YYYY'), 1300, NULL, 10);

COMMIT;

CREATE INDEX EMP_IDX1 ON EMP(DEPTNO);

CREATE TABLE XEMP TABLESPACE DMT
AS SELECT * FROM EMP;

CREATE INDEX XEMP_IDX1 ON XEMP(DEPTNO)
TABLESPACE DMT;

ANALYZE TABLE EMP COMPUTE STATISTICS;

ANALYZE TABLE XEMP COMPUTE STATISTICS;

select
TABLESPACE_NAME,
BLOCK_SIZE,
INITIAL_EXTENT,
NEXT_EXTENT,
MIN_EXTENTS,
MAX_EXTENTS,
MIN_EXTLEN,
EXTENT_MANAGEMENT,
ALLOCATION_TYPE,
SEGMENT_SPACE_MANAGEMENT
from dba_tablespaces
where tablespace_name in ('USERS', 'DMT');

TABLESPA BLOCK_SIZE INITIAL_EXTENT NEXT_EXTENT MIN_EXTENTS MAX_EXTENTS MIN_EXTLEN EXTENT_MAN ALLOCATIO SEGMEN
-------- ---------- -------------- ----------- ----------- ----------- ---------- ---------- --------- ------
DMT 4096 131072 131072 1 2147483645 131072 LOCAL UNIFORM MANUAL
USERS 4096 65536 1 2147483645 65536 LOCAL SYSTEM AUTO

select
TABLE_NAME,
TABLESPACE_NAME,
PCT_FREE,
PCT_USED,
INI_TRANS,
MAX_TRANS,
FREELISTS,
FREELIST_GROUPS,
NUM_ROWS,
BLOCKS,
EMPTY_BLOCKS,
AVG_ROW_LEN
from user_tables
where table_name in ('XEMP', 'EMP');

TABL TABLESPA PCT_FREE PCT_USED INI_TRANS MAX_TRANS FREELISTS FREELIST_GROUPS NUM_ROWS BLOCKS EMPTY_BLOCKS AVG_ROW_LEN
---- -------- ---------- ---------- ---------- ---------- ---------- --------------- ---------- ---------- ------------ -----------
EMP USERS 10 1 255 14 13 3 40
XEMP DMT 10 40 1 255 1 1 14 1 30 40

SET AUTOTRACE TRACE EXP STAT

SELECT *
FROM EMP;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=4 Card=14 Bytes=448)
1 0 TABLE ACCESS (FULL) OF 'EMP' (Cost=4 Card=14 Bytes=448)

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
16 consistent gets
0 physical reads
0 redo size
1307 bytes sent via SQL*Net to client
499 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
14 rows processed

SELECT *
FROM EMP
WHERE DEPTNO IN (10, 20, 30, 40);

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=14 Bytes=448)
1 0 INLIST ITERATOR
2 1 TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=2 Card=14 Bytes=448)
3 2 INDEX (RANGE SCAN) OF 'EMP_IDX1' (NON-UNIQUE) (Cost=1 Card=14)

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
9 consistent gets
0 physical reads
0 redo size
1278 bytes sent via SQL*Net to client
499 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
14 rows processed

SELECT *
FROM XEMP;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=14 Bytes=448)
1 0 TABLE ACCESS (FULL) OF 'XEMP' (Cost=2 Card=14 Bytes=448)

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
4 consistent gets
0 physical reads
0 redo size
1307 bytes sent via SQL*Net to client
499 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
14 rows processed

SELECT *
FROM XEMP
WHERE DEPTNO IN (10, 20, 30, 40);

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=14 Bytes=448)
1 0 INLIST ITERATOR
2 1 TABLE ACCESS (BY INDEX ROWID) OF 'XEMP' (Cost=2 Card=14 Bytes=448)
3 2 INDEX (RANGE SCAN) OF 'XEMP_IDX1' (NON-UNIQUE) (Cost=1 Card=14)

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
9 consistent gets
0 physical reads
0 redo size
1278 bytes sent via SQL*Net to client
499 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
14 rows processed

BEGIN
dbms_stats.gather_table_stats( OWNNAME => 'LSC',
TABNAME => 'EMP',
ESTIMATE_PERCENT => 99,
METHOD_OPT => 'FOR ALL COLUMNS SIZE 100',
DEGREE => 1,
CASCADE => TRUE);
END;
/

BEGIN
dbms_stats.gather_table_stats( OWNNAME => 'LSC',
TABNAME => 'XEMP',
ESTIMATE_PERCENT => 99,
METHOD_OPT => 'FOR ALL COLUMNS SIZE 100',
DEGREE => 1,
CASCADE => TRUE);
END;
/

select
TABLE_NAME,
TABLESPACE_NAME,

Followup February 12, 2006 - 5pm Central time zone:

ok, that makes sense with ASSM - it wastes space in order to increase concurrency during 
modifications.


As for the test - I would just say "way too small", this is a case of "six one way, half a dozen 
the other".  The costs are basically about the same in this case.

number of rows per block vs. arraysize February 23, 2006 - 12am Central time zone

Bookmark | Bottom | Top

Reviewer: jianhui from CA

Hi Tom,
It seems puzzling accroding to the arraysize. For example, i have a table called test(c char(750)) 
with 200 rows, and 9 rows per 8k size block. total 23 blocks.

If i set array size to 9, shouldnt there be 23 or so consistent gets, since there is no re-visit 
for each fetch. However the testing numbers tell different story, could you explain why? I was 
suspecting db_file_multiblock_read_count has something to do with that, however after I set it to 
1, the result still the same. Thanks a lot! (version is 9205)

SQL> select dbms_rowid.ROWID_BLOCK_NUMBER(rowid), count(*)
  2  from test
  3  group by dbms_rowid.ROWID_BLOCK_NUMBER(rowid);

DBMS_ROWID.ROWID_BLOCK_NUMBER(ROWID)   COUNT(*)
------------------------------------ ----------
                             1163312          9
                             1163313          9
                             1163314          9
                             1163315          9
                             1163316          9
                             1163317          9
                             1163318          9
                             1163319          9
                             1163320          9
                             1163321          9
                             1163322          9

DBMS_ROWID.ROWID_BLOCK_NUMBER(ROWID)   COUNT(*)
------------------------------------ ----------
                             1163323          9
                             1163324          9
                             1163325          9
                             1163326          9
                             1163327          9
                             1163328          9
                             1163329          9
                             1163330          9
                             1163331          9
                             1163332          9
                             1163333          9

DBMS_ROWID.ROWID_BLOCK_NUMBER(ROWID)   COUNT(*)
------------------------------------ ----------
                             1163334          2

23 rows selected.

SQL> set autotrace trace stat
SQL> set arraysize 9
SQL> select * from test;

200 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
         50  consistent gets
          0  physical reads
          0  redo size
     155838  bytes sent via SQL*Net to client
        898  bytes received via SQL*Net from client
         24  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        200  rows processed

SQL> set arraysize 200
SQL> /

200 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
         28  consistent gets
          0  physical reads
          0  redo size
     152523  bytes sent via SQL*Net to client
        656  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        200  rows processed

SQL> set arraysize 10
SQL> /

200 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
         45  consistent gets
          0  physical reads
          0  redo size
     155290  bytes sent via SQL*Net to client
        865  bytes received via SQL*Net from client
         21  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
        200  rows processed

SQL>

Followup February 23, 2006 - 7am Central time zone:

sqlplus always does a single row fetch and then starts array fetching, that'll affect this 
definitely.  So, 

arraysize 9

read 1 - read block 1 row 1
read 2 - read block 1 rows 2-9, block 2 row 1
read 3 - read block 2 rows 2-9, block 3 row 1
read 4 - read block 3 rows 2-9, block 4 row 1

So, each block is read twice in that case.

arraysize 10

read 1 - read block 1 row 1
read 2 - read block 1 rows 2-9, block 2 rows 1-2
read 3 - read block 2 rows 3-9, block 3 rows 1-3
...

and there almost every block is read twice.


sql_trace=true is your friend.  You can see the rows fetched on each call in the trace file 
directly.


I set arraysize 9 and ran a select * from t with 200 rows:

PARSING IN CURSOR #13 len=15 dep=0 uid=78 oct=3 lid=78 tim=1113963019093011 hv=1134051363 
ad='43c7cd88'
select * from t
END OF STMT
PARSE #13:c=4999,e=4824,p=0,cr=32,cu=0,mis=1,r=0,dep=0,og=1,tim=1113963019092993
EXEC #13:c=0,e=95,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=1113963019093243
FETCH #13:c=0,e=146,p=0,cr=4,cu=0,mis=0,r=1,dep=0,og=1,tim=1113963019093481
FETCH #13:c=0,e=110,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019094329
FETCH #13:c=0,e=93,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019094882
FETCH #13:c=0,e=76,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019095326
FETCH #13:c=0,e=72,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019095788
FETCH #13:c=0,e=57,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019096166
FETCH #13:c=0,e=48,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019096484
FETCH #13:c=0,e=52,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019096816
FETCH #13:c=0,e=50,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019097121
FETCH #13:c=0,e=48,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019097412
FETCH #13:c=0,e=51,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019097739
FETCH #13:c=0,e=47,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019098048
FETCH #13:c=0,e=45,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019098336
FETCH #13:c=0,e=49,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019098661
FETCH #13:c=0,e=51,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019098970
FETCH #13:c=0,e=48,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019099273
FETCH #13:c=0,e=55,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019099588
FETCH #13:c=0,e=52,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019099904
FETCH #13:c=0,e=47,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019100207
FETCH #13:c=1000,e=84,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019100546
FETCH #13:c=0,e=137,p=0,cr=4,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019101181
FETCH #13:c=0,e=81,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019101954
FETCH #13:c=1000,e=184,p=0,cr=2,cu=0,mis=0,r=9,dep=0,og=1,tim=1113963019102596
FETCH #13:c=0,e=88,p=0,cr=4,cu=0,mis=0,r=1,dep=0,og=1,tim=1113963019103128


r=1 shows the rows fetched on each call.

a simple question February 23, 2006 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from Scotland

Hi Tom,

Since I read this thread for the first time I've been trying to explain why I get 3 consistent gets 
in the following example (segment header; data block and ??.  
If I create the table as IOT I get 1. Can you please shed some light?

Connected to:
Oracle9i Release 9.2.0.6.0 - Production
JServer Release 9.2.0.6.0 - Production

SQL> create table t(x int);

Table created.

SQL> insert into t values(1);

1 row created.

SQL> set autotrace traceonly stat
SQL> select * from t;


Statistics
----------------------------------------------------------
          0  db block gets
          3  consistent gets
          0  physical reads
          0  redo size
        372  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

SQL>

Followup February 23, 2006 - 9am Central time zone:

because for the IOT we read the root block, and stop.

We don't need to read the segment header block to figure out what blocks to full scan - we just 
goto the root block and discover "there are no leaves to goto" and stop.


full scan = "read segment header to figure out what block ranges to full scan, then full scan them"

iot range scan = "read root and from root figure out what to read next - like a branch block, leaf 
block, whatever.  In the case the root = leaf, you are done in 1 IO"

re: a simple question February 23, 2006 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader from Scotland

Thanks Tom,

But in my case I only have one data block for the heap table, so 1 (segment header) + 1 (data 
block) = 2.  I've done a block dump and the HWM is on block 1. Where's the other 1?

Followup February 23, 2006 - 7pm Central time zone:

the number of IO's performed against the segment header (and even the TYPE of IO done - current or 
consistent) vary by version.

We never promised to read it just once.

Older releases would have even a higher count.

Consistent gets too high for index range scan on global temporary Table February 24, 2006 - 1am Central time zone

Bookmark | Bottom | Top

Reviewer: Guru Dhar from Bangalore, India

Hi Tom,

We have a situation that we are not able to exlain/proceed with perfrmace tuning. It is as follows. 
We have a global temporary table that gets populated and then updated in a loop. The trace for the 
insert and update is below.

INSERT INTO XXRPA_INS_TEMP (FACTOR, RISK_RATING, CUSTOMER_ID, PARTY_NUMBER, 
  ANSWER_SET_ID, SELECTION_METHOD, RISK_ASSESSMENT_ID, ZONE) SELECT 
  XRA.FACTOR, XRA.RISK_RATING, XRA.CLAIMANT_PARTY_ID, HP.PARTY_NUMBER, 
  XRA.ANSWER_SET_ID, XRA.SELECTION_METHOD, XRA.RISK_ASSESSMENT_ID, XRA.ZONE 
  FROM XXRPA_RISK_ASSESSMENT XRA, HZ_PARTIES HP, 
  XXRPA_SPSAPPL_SPS_APPLC_SETS_V XSSASV WHERE HP.PARTY_ID = 
  XRA.CLAIMANT_PARTY_ID AND XRA.ANSWER_SET_ID = XSSASV.ANSWER_SET_ID AND 
  TRUNC (XSSASV.CLAIM_SUBMISSION_DATE) BETWEEN :B3 AND :B2 AND 
  XRA.LOOKUP_CODE_SCHEME = 'SPS' AND XRA.SELECTION_METHOD IN ('ASSESSED', 
  'RANDOM', 'RISK') AND XRA.ZONE = :B1 



call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.02       0.01          0          0          0           0
Execute      7     18.63      18.53       1232    1264709      16886      116085
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        8     18.65      18.54       1232    1264709      16886      116085

Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer goal: CHOOSE
Parsing user id: 173  (APPS)   (recursive depth: 1)

Rows     Row Source Operation
-------  ---------------------------------------------------
      0  FILTER  
      0   TABLE ACCESS BY INDEX ROWID XXRPA_ANSWER_SETS 
      1    NESTED LOOPS  
      0     NESTED LOOPS  
      0      TABLE ACCESS BY INDEX ROWID XXRPA_RISK_ASSESSMENT 
      0       INDEX RANGE SCAN XXRPA_RISK_ASSESSMENT_ZONE (object id 473197)
      0      TABLE ACCESS BY INDEX ROWID HZ_PARTIES 
      0       INDEX UNIQUE SCAN HZ_PARTIES_U1 (object id 172923)
      0     INDEX RANGE SCAN XAS_ANSWER_SET_VERSION_I (object id 408101)


UPDATE XXRPA_INS_TEMP SET BAND = :B2 
WHERE
 RISK_ASSESSMENT_ID = :B1 


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute 523042  23230.25   22698.18          0  231893488     535118      523042
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total   523043  23230.25   22698.18          0  231893488     535118      523042

Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer goal: CHOOSE
Parsing user id: 173  (APPS)   (recursive depth: 1)

Rows     Row Source Operation
-------  ---------------------------------------------------
      0  UPDATE  
 189300   INDEX RANGE SCAN XXRPA_INS_TEMP_IDX1 (object id 473196)

Now, the update is via an indexed path and the indexed column has the highest selectivity (1 
row/per risk_assesment_id). Still the gets is more than 400 blocks per execution (or perr row 
update).

Can you provide us with any clues as to why this could be? Please let us know if you need any 
additional information.

Thanks for your site.

Followup February 24, 2006 - 8am Central time zone:

I will guess

band is indexed.

tell us more.  This is likely a case where  the IO's are against the index being maintained.  You 
appear to be updating every row an AVERAGE OF 4.5 TIMES!!!!  Why is that!

give us the details on the gtt, what band is datatype wise, indexes involved, the transaction 
semantics (where is the commit and such), 


Something is funny there - you executed this 523,042 times - but the row source only shows 189,300 
- we'll need more information - details as to how to reproduce this.

Consistent gets too high for index range scan on global temporary Table February 28, 2006 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: Guru Dhar from Bangalore, India

Hi Tom,
  Thanks for your response.

There is no index on the BAND column. The Only index we have is on risk_assessment_id column.

Following are the DDL scripts used for the table and index;
CREATE GLOBAL TEMPORARY TABLE XXRPA_INS_TEMP
(
  LINE_ID              NUMBER,
  HEADER_ID            NUMBER,
  ORDERED_QUANTITY     NUMBER,
  RISK_RATING          NUMBER,
  CUSTOMER_ID          NUMBER,
  PARTY_NUMBER         VARCHAR2(30 BYTE),
  RS_ZONE              NUMBER(4),
  FACTOR               NUMBER,
  REMOTE_SENSING_FLAG  VARCHAR2(1 BYTE),
  GROWER_PARTY_ID      NUMBER,
  REQUEST_DATE         DATE,
  ANSWER_SET_ID        NUMBER,
  INSPECTION_METHOD    VARCHAR2(50 BYTE),
  QUARTILE             NUMBER,
  RISK_ASSESSMENT_ID   NUMBER,
  CLAIMANT_PARTY_ID    NUMBER,
  LOOKUP_CODE_SCHEME   VARCHAR2(30 BYTE),
  SCHEME_YEAR          VARCHAR2(10 BYTE),
  SELECTION_METHOD     VARCHAR2(50 BYTE),
  ZONE                 NUMBER,
  PROCESSED_BY_RANDOM  VARCHAR2(1 BYTE),
  BAND                 NUMBER,
  REPORT_FLAG          VARCHAR2(1 BYTE),
  COUNTER              NUMBER
)
ON COMMIT DELETE ROWS;


CREATE INDEX XXRPA_INS_TEMP_IDX1 ON XXRPA_INS_TEMP
(RISK_ASSESSMENT_ID);

There is only one commit in the program after the end of the processing.

As you pointed out, there was a problem in Looping construct which we had retified.

I was trying to get the index statistics from the instance where this program was being tested for 
Performance, but unfortunately they had dropped the index and recreated it, hence index stats are 
also not available.

The issue of the row source data that does not match up with the number of executions is indeed 
puzzling and we are not able to get that to happen again..
we tried to repoduce this on the development instance, but were not able to do so.

Current Tkprof results attached:

SELECT FACTOR, RISK_ASSESSMENT_ID 
FROM
 XXRPA_INS_TEMP


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch    18079      0.51       0.41          0      18233          0       18078
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total    18081      0.51       0.41          0      18233          0       18078

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 173  (APPS)   (recursive depth: 1)

Rows     Row Source Operation
-------  ---------------------------------------------------
  18078  TABLE ACCESS FULL XXRPA_INS_TEMP 



UPDATE XXRPA_INS_TEMP SET BAND = :B2 
WHERE
 RISK_ASSESSMENT_ID = :B1 


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute  18078      2.85       2.69          0      36224      18454       18078
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total    18079      2.85       2.70          0      36224      18454       18078

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 173  (APPS)   (recursive depth: 1)

Rows     Row Source Operation
-------  ---------------------------------------------------
      0  UPDATE  
  18078   INDEX RANGE SCAN XXRPA_INS_TEMP_IDX1 (object id 407378)

Thanks.

Followup February 28, 2006 - 9am Central time zone:

looks good now, could it have been a "bad trace" file?

Bad Trace File ??? March 1, 2006 - 5am Central time zone

Bookmark | Bottom | Top

Reviewer: Guru Dhar from Bangalore, India.

<code>Hi Tom,
I am not sure what a Bad trace file is. Recently we have encountered issue related to inconsistent executiom time reported by trace file and sql Plus (Set timing on).

I had managed to save the trace file which I am attaching:

TKPROF: Release 8.0.6.3.0 - Production on Fri Jun 3 12:37:17 2005

(c) Copyright 1999 Oracle Corporation. All rights reserved.

Trace file: rpad16_ora_23719.trc
Sort options: default

********************************************************************************
count = number of times OCI procedure was executed
cpu = cpu time in seconds executing
elapsed = elapsed time in seconds executing
disk = number of physical reads of buffers from disk
query = number of buffers gotten for consistent read
current = number of buffers gotten in current mode (usually for update)
rows = number of rows processed by the fetch or execute call
********************************************************************************

alter session set sql_trace = true

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 0 0.00 0.00 0 0 0 0
Execute 1 0.00 2.76 0 0 0 0
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 1 0.00 2.76 0 0 0 0

Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 173
********************************************************************************

BEGIN amit_test_prc; END;

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 100.00 64.58 0 0 0 0
Execute 1 0.00 29.71 0 0 0 1
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 100.00 94.29 0 0 0 1

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 173
********************************************************************************

SELECT /*1*/ count(*)
from scott.emp

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 1.33 0 0 0 0
Execute 1 0.00 0.88 0 0 0 0
Fetch 1 0.00 1.76 0 3 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.00 3.97 0 3 0 1

Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 173 (recursive depth: 1)
********************************************************************************

SELECT /*2*/ count(*)
from scott.emp

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 1.52 0 0 0 0
Execute 1 0.00 0.53 0 0 0 0
Fetch 1 0.00 0.68 0 3 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.00 2.73 0 3 0 1

Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 173 (recursive depth: 1)
********************************************************************************

SELECT /*3*/ count(*)
from scott.emp

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 1.25 0 0 0 0
Execute 1 0.00 0.54 0 0 0 0
Fetch 1 0.00 1.10 0 3 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.00 2.89 0 3 0 1

Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 173 (recursive depth: 1)
********************************************************************************

SELECT /*4*/ count(*)
from scott.emp

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 1.19 0 0 0 0
Execute 1 0.00 0.50 0 0 0 0
Fetch 1 0.00 0.61 0 3 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.00 2.30 0 3 0 1

Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 173 (recursive depth: 1)
********************************************************************************

SELECT count(*)
from scott.emp

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 4 100.00 12.62 0 0 0 0
Execute 4 0.00 1.96 0 0 0 0
Fetch 4 0.00 2.79 0 12 0 4
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 12 100.00 17.37 0 12 0 4

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 173 (recursive depth: 1)
********************************************************************************

alter session set sql_trace = false

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 1.66 0 0 0 0
Execute 1 0.00 2.15 0 0 0 0
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2 0.00 3.81 0 0 0 0

Misses in library cache during parse: 0
Optimizer goal: CHOOSE
Parsing user id: 173

********************************************************************************

OVERALL TOTALS FOR ALL NON-RECURSIVE STATEMENTS

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 2 100.00 66.24 0 0 0 0
Execute 3 0.00 34.62 0 0 0 1
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 5 100.00 100.86 0 0 0 1

Misses in library cache during parse: 1

OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS

call count cpu

Followup March 1, 2006 - 8am Central time zone:

you have a tkprof that doesn't make sense - I would suspect the trace file was "improperly 
generated, broken, bad in some way shape or form"

that is what a bad trace file is.  


why are you using 8.0.6 tkprof???


are you using a really really really old tkprof against a trace file from a newer version.  I think 
so.  All bets all.  Use the right tkprof.

May 10, 2006 - 9am Central time zone

Bookmark | Bottom | Top

Reviewer: Marcel

Hi Tom,

I think that physical reads ( because of physical reads are slow/expensive) are to avoid, the same 
with
the consistent gets( because of lachtes=locks in the memory).
What are recursive calls exactly?

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
         428  consistent gets
          0  physical reads
          0  redo size
       1307  bytes sent via SQL*Net to client
        499  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         14  rows processed

Thanks 
Marcel

Followup May 10, 2006 - 9am Central time zone:

sql executed on behalf of your sql - to do space management, hard parsing, and so on.

or sql called from other sql by you (eg: the sql you execute in a plsql routine is "recursive")


and in general, if you reduce the logical IO, you'll take care of that physical IO ;)

do LIOs always matter? June 30, 2006 - 6pm Central time zone

Bookmark | Bottom | Top

Reviewer: Ryan from Reston, VA

This is going to long, but its interesting. I hope it's worth the read.

I am performing a few tests. Here is the basic framework of the tests.

Test 1:
GTT joined to Heap table on a unique index and the data is perfectly colocated(ordered and dense)

Test 2:
GTT joined to List partitioned table by a unique index. It is now two columns. It leads with the
same unique column in the first test and the second column in a partition key. The data is
perfected colocated. The index is local.

Our GTT has 5,000 rows and 20,000 rows. We are using dynamic sampling 2. 10G release 2.

The data in the GTT are perfectly colocated. All the rows in a GTT match the rows in one partition
in the list partitioned table. The base tables are identical accept for partition. Exact same data.
They have between 20 million and 200 million rows. Tables are analyzed with autosample size option.

Test Code:
1. Same queries for both. The partitioned tables use queries that include the partition key.
2. Open 1,10,20,50,100 threads. So one round of tests with 5,000 rows and another with 20,000 rows
in the GTT.
3. For 1 thread grab tkprof. Use awrpt for multiple threads. Sometimes I grabbed 10046 for multiple
threads, but it was skewing my system results.
4. No one query is the bottleneck. All of them have somewhat comparable results.
5. We tested 8 queries run in a stored procedure.
6. We bounce all relavent tablespaces between tests and flush the shared pool.

Results:
1. Elapsed time is about the same. CPU is about the same. Physical IO is a little higher for the
non-partitioned tables.
2. We noticed a 20-50% spike in LIOs for the partitioned tables depending on the number of threads
we run. When we look at the row source in the tkprof output the increase is almost entirely on
'partition list iterator'.

1. Have you seen a spike in LIO due to the partition list iterator row source?

2. If my LIOs are so much higher, how come I do not see an increase in elapsed time or CPU? We
tested this under stress with up to 100 threads. We have more waits with multiple threads for
non-partitioned tables. So this may be balancing the elapsed time. However, what about CPU? I
thought LIO translates into CPU usage?

I believe Mogens Noorgaard had a chapter in the Oak Table book about some LIOs are more costly than
others, but we don't have the granualarity to tell yet?

I don't have a production system to test the effects against something live. It's still in
development.

Followup July 1, 2006 - 7am Central time zone:

set up test case scripts?


LIO's come into play in scaling.  each LIO is a latch into the cache.  If you have lots of latch 
misses, you spin on CPU.  As you add more users - the CPU used by the queries will higher LIO's 
will tend to go up faster than those with LOW LIO's.


Some LIO's - like those done against UNDO to provide consistent read - are less costly, but the 
ones you are observing in your single user test are "the real deal"

How to distinguish one type of LIO from another. July 3, 2006 - 5pm Central time zone

Bookmark | Bottom | Top

Reviewer: Neeraj Nagpal from LA, CA

Tom,

You said that that the LIO's - like those done against UNDO to provide consistent read - are less 
costly, but the ones you are observing in your single user test are "the real deal"

Is there a direct (or indirect) way to tell one type of LIO from another. 

Thanks,
Neeraj

does insert use Buffer Cache? July 4, 2006 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi

I have a doubt about INSERT transaction management.

When we update or delete we go to the disk and find the data we are going to modify, if found read 
that block to the buffer cache.

With insert do we do the samething? I dont think so since there is no rowid to find!

TIA

Followup July 7, 2006 - 9pm Central time zone:

you have to find a rowid to associate with the row - so yes, the same thing happens.

We goto the buffer cache to inspect the freelists (manual segment space management) or the bitmap 
blocks in the table (assm - automatic segment space management) to find a block with sufficient 
free space.  We might have to read from disk into the cache these freelist/bitmap blocks and might 
have to readd from disk the block we discover having sufficient free space.

more disk reads July 6, 2006 - 5am Central time zone

Bookmark | Bottom | Top

Reviewer: Karmit from UK

Hi Tom,

I've a very puzzling situation here. We just migrated a test database to production (different bare 
metal and SAN 
- but same sort of platform/ os/ patch etc) using RMAN backup/ restore.

There is one particular operation which seems to run extremely slow (5 times slower than test 
system). Bear in mind that everything else is the same. Only one user doing the test, same spfile 
parameters etc (its a RMAN copy).

When I compared the SQL stats I see for the offending operation, the number of rows processed is 
exactly the same, the explain plan is exactly the same (even the cost is comparable) - basically 
its the same amount of data its going through.

However, I did find a few differences - the number of buffer gets, disk reads were about 30K more 
in the prod system.. and just about every read operation for this particular SQL seems to be more 
by about 30K.. I suspect thats where the "elapsed time" increase is coming from.

From a very theoretical point of view (I know you don't like this :-) - but would you be able to 
comment on as to
why would the disk reads be so much off? Its doing an index read.. saw the db seq file waits taking 
up bulk of the time - on both systems.

P.S: I had collected stats on the prod exactly the same way as I did on the test.

Followup July 8, 2006 - 9am Central time zone:

in production - are there other users (I know you said "one user doing the test" but that doesn't 
preclude other users in production) hitting this table?

disk reads July 6, 2006 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: Karmit from UK

Hi Tom,

Follow up on the above issue. I rebuilt the index and now the disk reads seem to be comparable.. 
the CPU time is actually lower on prod - but the elapsed time is significantly higher... dodgy 
disks?

=============== =============   ================
Statistic       TEST            PROD
=============== =============   ================
DiskReadPerExec 793027          637324              
BufGetsPerExec    3,541,324       3,473,893
DiskReads       793027            637324          
BufGets         3541324            3473893     
BufGetsPerRow      9.23            9.05    
BCHitRatio    77.61            81.65        
RowsProcs      383856            383856            
CPUTime         51450000        47160000          
ElapsedTime    578,209,410     2,436,869,321    
=============== =============   ================

Thanks,
Karmit

Followup July 8, 2006 - 9am Central time zone:

I don't see how rebuilding the index could affect this IF this was a simple restore (eg: they both 
would have had the same index structure no???)


tkprof it, do you have one of those (with waits included in it)

Insert Buffer Cache July 6, 2006 - 11pm Central time zone

Bookmark | Bottom | Top

Reviewer: Roderick

Insert will still go into a buffer cache. Freelist or bitmap information for the table will tell 
the process which block to insert the row into. That block will have to be read into cache first so 
it can figure out where inside the block to put it.

Insert Buffer Cache July 6, 2006 - 11pm Central time zone

Bookmark | Bottom | Top

Reviewer: Roderick

Oops. Forgot to add that a direct path insert can bypass the buffer cache though.

disk reads July 10, 2006 - 4am Central time zone

Bookmark | Bottom | Top

Reviewer: Karmit from UK

Hi,

Finally! - one of the unix admin sheepishly admitted that the type of disk being used for that 
filesystem was a slower one than supposed to be going in... its been changed now and the 
performance is back to normal.

Thanks,
Karmit

Where is SCN Stored August 23, 2006 - 9pm Central time zone

Bookmark | Bottom | Top

Reviewer: Sanji from Mahwah, New Jersey

Hello Tom,

This is an abstract from your favourite topics in 9i Concepts Guide, "Data Concurrency and
Consistency"

"As a query enters the execution stage, the current system change number (SCN) is determined. As
data blocks are read on behalf of the query, only blocks written with the observed SCN are used.
Blocks with changed data (more recent SCNs) are reconstructed from data in the rollback segments,
and the reconstructed data is returned for the query. Therefore, each query returns all committed
data with respect to the SCN recorded at the time that query execution began. Changes of other
transactions that occur during a query's execution are not observed, guaranteeing that consistent
data is returned for each query."

1> Where is the SCN Stored, once it is determined at the start of the execution stage of any query
?

For instance the following query is to be executed

update T set X = <variable>
where clause....

So, the query scans for consistent image of table 'T' based on the "where clause" result set.
Considering the "read consistent" behaviour of Oracle as mentioned above, the current SCN is
determined.

Q> From where is this current system change number determined and once determined where is it
stored for reference ?

2> "As data blocks are read on behalf of the query, only blocks written with the observed SCN are
used. Blocks with changed data (more recent SCNs) are reconstructed from data in the rollback
segments, and the reconstructed data is returned for the query."

Does the transaction table in the RBS store the SCN for all it's contents (blocks), so as to
reconstruct the read consistent block in case if it requires reconstruction ?

3> In case of updates, inserts and deletes, block are gotten in current mode.
I have read this somewhere in this site that the current mode is actually the data segment header.
I am confused as to how the blocks are gotten in current mode.

Thanks
Sanji

Followup August 27, 2006 - 7pm Central time zone:

1) it is just part of your "cursor state", just a variable as part of the program that is called 
"Oracle"

2) the SCN is "conceptual" here, it is really all about committed transaction state.  the rollback 
segment headers and the block headers themselves contain everything we need to know.

3) it used to be in older releases that the segment headers were gotten in current mode for full 
scans.  It isn't true anymore.

You'd be interested in reading:

http://tkyte.blogspot.com/2005/08/something-different-part-i-of-iii.html

http://tkyte.blogspot.com/2005/08/part-ii-seeing-restart.html

http://tkyte.blogspot.com/2005/09/part-iii-why-is-restart-important-to.html

August 31, 2006 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

How can I reduce the consistent gets,
changing arraysize didn't help?
This is the oracle version we are
using: 9.2.0.1.0

select count(*) from t;

count(*)
-------
76232

Set arraysize 100

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
       4192  consistent gets
          0  physical reads
          0  redo size
        210  bytes sent via SQL*Net to client
        277  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

Set arraysize 1000

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
       4192  consistent gets
          0  physical reads
          0  redo size
        210  bytes sent via SQL*Net to client
        277  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

Thanks in advance!

Followup August 31, 2006 - 11am Central time zone:

that'll never help for that type of a query (changing the array size).

There is one round trip to the server here - there would not be a reason to visit and revisit the 
same block over and over during this full scan.


the only way to reduce the consistent gets for that count(*) would be to reduce the number of 
blocks that must be read to get the count.


eg:


a) having a primary key. the CBO (cost based optimizer) will fast full scan an index on a NOT NULL 
column

ops$tkyte%ORA10GR2> create table t
  2  as
  3  select * from all_objects;

Table created.

ops$tkyte%ORA10GR2> exec dbms_stats.gather_table_stats(user,'T');

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> select count(*) from t;

  COUNT(*)
----------
     50026

ops$tkyte%ORA10GR2> set autotrace traceonly statistics;
ops$tkyte%ORA10GR2> select count(*) from t;


Statistics
----------------------------------------------------------
        694  consistent gets
..

ops$tkyte%ORA10GR2> set autotrace off
ops$tkyte%ORA10GR2> alter table t add constraint t_pk primary key(object_id);
Table altered.

ops$tkyte%ORA10GR2> set autotrace traceonly statistics;
ops$tkyte%ORA10GR2> select count(*) from t;


Statistics
----------------------------------------------------------
        166  consistent gets
...

b) compress the data to make it smaller

ops$tkyte%ORA10GR2> set autotrace off
ops$tkyte%ORA10GR2> alter table t drop constraint t_pk;
Table altered.

ops$tkyte%ORA10GR2> alter table t compress;

Table altered.

ops$tkyte%ORA10GR2> alter table t move;

Table altered.

ops$tkyte%ORA10GR2> exec dbms_stats.gather_table_stats(user,'T');

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> set autotrace traceonly statistics;
ops$tkyte%ORA10GR2> select count(*) from t;


Statistics
----------------------------------------------------------
        211  consistent gets
...
ops$tkyte%ORA10GR2> set autotrace off




arraysize only helps when there is more rows than the array size (and only up to a point - say 
100-500 rows maximum).


You have a single row, the entire query is entirely processed from start to finish during the first 
fetch, we are NOT revisiting blocks as you would in a select * from T query.

August 31, 2006 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Thanks for your response.
But in my context, t is a complex view. And we doing a count(*) on that view. Depending on that 
count(*) number, we are updating another table, eg:

update table t1
where col1=(select count(*) from t);

This update is giving 756908 consistent gets, what can I do to reduce these?

Followup August 31, 2006 - 1pm Central time zone:

same answer - the entire select count(*) is done in a single call.

arraysize - that cannot possibly have any effect here at all, we do not experience the "get and 
re-get the same block over and over" syndrome - because it is a single call.

Only thing you can do is tune the underlying view definition to do less work.


If T was a complex view, posing the question as you did was counter productive.

really should be:

select count(*)
 from ( select ...
          from .....
         where .....
       )

if you want anyone to ever help you make it "go faster".


the only thing you can do to "reduce" your logical IO's is what I was alluding to above - via 
tuning (eg: adding index on not null column, tuning.  compressing table, tuning.  you - likely 
"rewrite view query" if even possible.  Sometimes your query is already doing the least amount of 
work possible)

August 31, 2006 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Thanks for your excellent explanation! One last doubt, can we compress a view?

Followup August 31, 2006 - 2pm Central time zone:

you compress segments - and only if they are read only/read mostly (eg: bulk loaded from time to 
time).

so, tables the view queries - sure.
the view itself - nope.

(unless it is a materialized view, in which case I'd just make a materialized view of the 
materialized view that was the select count(*) and be done with it)

Array Size effects in PL/SQL September 9, 2006 - 3am Central time zone

Bookmark | Bottom | Top

Reviewer: Arindam Mukherjee from Kolkata, India

Respected Mr. Tom,

On reading array size effects, could you kindly let me ask you - how can I take this advantage in 
PL/SQL program when this size setting entails SQL plus environment only?
If possible, please answer me with example.

Followup September 9, 2006 - 12pm Central time zone:

arraysize is a sqlplus parameter.

In plsql

a) if you use 10g and above, then:

   for x in ( select ... )
   for x in CURSOR_NAME 

will array fetch 100 rows at a time.  

b) otherwise you use bulk collect and limit to set the arraysize for plsql

Consistent gets September 12, 2006 - 2am Central time zone

Bookmark | Bottom | Top

Reviewer: Amit from India

Hi Tom,

I am still not clear with the concept of consistent gets
To calculate buffer cache hit ratio we use
SELECT phy.value physical_reads,
     cur.value consistent_gets,
       con.value  db_block_gets,
      1-((phy.value)/((cur.value)+(con.value))) "Cache hit ratio"
     FROM   v$sysstat cur, v$sysstat con, v$sysstat phy
     WHERE  cur.name='db block gets'
     AND    con.name='consistent gets'
     AND    phy.name='physical reads'
*********************************************

Does this mean that db block gets+ consistent gets include both disk read and reads from buffer 
cache.
I am assuming buffer cache hit ratio to be total gets from buffer to total gets. i.e 
buffer/(buffer+ disk)

Please correct me if i am wrong.

Thanks
Amit

Followup September 12, 2006 - 8am Central time zone:

a logical IO (consistent get) at some point incurred a physical IO.

But we might perform 5,000 logical IO's against a given block - and only 1 physical IO (read it 
once from disk, read it 5,000 times from cache)


So, each db block get or consistent get could have done a physical IO, or not.

What you are looking at is the ratio of the number of times a block is read from the cache vs disk.

Thanks a lot. September 20, 2006 - 12am Central time zone

Bookmark | Bottom | Top

Reviewer: Amit from India

GETS - Consistent and Current September 21, 2006 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: Arindam Mukherjee from Kolkata, India

Respected Mr. Tom,

I am truly confused with Consistent Gets and Current block Read i.e. Db block gets.
Since query generally perform Consistent Gets, so can I say  Consistent gets is UNDO block 
reading.  I cant say that because you also have written in your book  retrieval of blocks from 
the buffer cache in read consistent mode.  So??
Anyway you kindly explain us in easy language  
1> What is Consistent Gets 
2> What is Current block Read i.e. Db block gets

Followup September 22, 2006 - 1am Central time zone:

consistent gets are NOT undo blocks being read.

A consistent get is a block that is gotten as of a point in time.  Say you start a query at 11:00.  
It is still running at 11:05.  At 11:05 it needs to get a block from the buffer cache.  It will 
request a consistent get on that block AS OF 11:00.  (the block is needed as of 11:00 for a 
consistent read).  If the block in the cache is "newer" than 11:00 - we will have to do some reads 
to the UNDO segment to undo the changes - but if the block in the cache is "older" than 11:00 - we 
will just read that block out of the cache and not hit the undo.

db block gets - suppose the query you were running right above at 11:00 was an UPDATE.  At 11:05 it 
did a consistent get on a block as of 11:00.  It discovered that there is a row on that block that 
it should update.  It will then request that same block in CURRENT MODE - as of "right now", not 
"as of 11:00".  We have to update the current version of the block - not an old consistent read 
version.  So we get the block AS OF 11:05 - verify the row has not changed between 11:00 and 11:05 
- and then do the update to the current mode version of the block.

want to understand this - LIO reduction December 27, 2006 - 11pm Central time zone

Bookmark | Bottom | Top

Reviewer: Ajeet

Hi Tom,
I have a query which was doing 24093 LIOs per execution.
I generated the statistics on both of the tables involved in the query using this :

begin
dbms_stats.gather_table_stats(
ownname => 'TBAADM',
tabname => 'DAILY_TRAN_HEADER_TABLE',
estimate_percent => 10,
method_opt => 'for all indexed columns size AUTO',
degree => 4,
cascade => true
);
end;
/
Same thing I did for another table in this query -table name DAILY_TRAN_DETAIL_TABLE.

and then the LIO's went down to 2 per execution.

earlier also I had generated the statistics but I did use this - to generate it.

begin
dbms_stats.gather_table_stats(
ownname => 'TBAADM',
tabname => 'DAILY_TRAN_HEADER_TABLE',
estimate_percent => 10,
method_opt => 'for all indexed columns',
degree => 4,
cascade => true
);
end;
/
so there was NO "size auto" in the method_opt.

below is the query

SELECT
TO_CHAR(DTD.TRAN_DATE,'DD-MM-YYYY HH24:MI:SS'),
DTD.TRAN_ID,
DTD.REFERRAL_ID,
DTD.PART_TRAN_SRL_NUM,
DTD.TRAN_DATE,
DTD.SVS_TRAN_ID ,
TO_CHAR(DTD.VALUE_DATE,'DD-MM-YYYY HH24:MI:SS'),
TO_CHAR(DTD.GL_DATE,'DD-MM-YYYY HH24:MI:SS')
FROM tbaadm.DTD,tbaadm.DTH
WHERE (DTH.TRAN_DATE = DTD.TRAN_DATE AND DTH.TRAN_ID = DTD.TRAN_ID AND DTD.BANK_ID = '01'
AND DTH.BANK_ID = '01'
AND (DTD.RESTRICT_MODIFY_IND <> 'T' OR DTD.RESTRICT_MODIFY_IND IS NULL )
AND DTH.INIT_SOL_ID || NULL = '0049'
AND DTD.TRAN_CRNCY_CODE || NULL = 'INR'
AND DTH.TRAN_DATE = TO_DATE( '08-05-2003' ,'DD-MM-YYYY')
AND DTH.TRAN_ID > ' MI752762'
AND DTD.ACID = 'WF112351'
AND DTD.RCRE_TIME >= TO_DATE( '27-12-2006 00:00:00' ,'DD-MM-YYYY HH24:MI:SS')
)
ORDER BY 5,2,4
/

below is the AUTOTRACE output before :

SQL> set autot on
SQL> @wfp.sql

no rows selected

Execution Plan
----------------------------------------------------------
Plan hash value: 1051266481

--------------------------------------------------------------------------------
-----------------------------

| Id | Operation | Name | Rows | By
tes | Cost (%CPU)| Time |

--------------------------------------------------------------------------------
-----------------------------

| 0 | SELECT STATEMENT | | 1 |
242 | 2 (0)| 00:00:01 |

|* 1 | TABLE ACCESS BY INDEX ROWID | DAILY_TRAN_HEADER_TABLE | 1 |
50 | 1 (0)| 00:00:01 |

| 2 | NESTED LOOPS | | 1 |
242 | 2 (0)| 00:00:01 |

|* 3 | TABLE ACCESS BY INDEX ROWID| DAILY_TRAN_DETAIL_TABLE | 1 |
192 | 1 (0)| 00:00:01 |

|* 4 | INDEX RANGE SCAN | IDX_DAILY_TRAN_DETAIL_TABLE | 1 |
| 1 (0)| 00:00:01 |

|* 5 | INDEX RANGE SCAN | IDX_DTH_TRAN_DATE_ID | 1 |
| 1 (0)| 00:00:01 |

--------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------

1 - filter("DTH"."INIT_SOL_ID"||NULL='0049' AND "DTH"."BANK_ID"='01')
3 - filter("DTD"."RCRE_TIME">=TO_DATE('2006-12-26 00:00:00', 'yyyy-mm-dd hh24
:mi:ss') AND

"DTD"."ACID"='WF112351' AND "DTD"."TRAN_CRNCY_CODE"||NULL='INR' AN
D

("DTD"."RESTRICT_MODIFY_IND"<>'T' OR "DTD"."RESTRICT_MODIFY_IND" I
S NULL))

4 - access("DTD"."TRAN_DATE"=TO_DATE('2003-05-08 00:00:00', 'yyyy-mm-dd hh24:
mi:ss') AND

"DTD"."TRAN_ID">' MI752762' AND "DTD"."BANK_ID"='01' AND "DTD"."TR
AN_ID" IS NOT NULL)

filter("DTD"."BANK_ID"='01' AND "DTD"."TRAN_ID">' MI752762' AND
"DTD"."TRAN_DATE"=TO_DATE('2003-05-08 00:00:00', 'yyyy-mm-dd hh24:
mi:ss'))

5 - access("DTH"."TRAN_ID"="DTD"."TRAN_ID" AND "DTH"."TRAN_DATE"=TO_DATE('200
3-05-08 00:00:00',

'yyyy-mm-dd hh24:mi:ss'))
filter("DTH"."TRAN_ID">' MI752762')

Statistics
----------------------------------------------------------
53 recursive calls
0 db block gets
24093 consistent gets
0 physical reads
0 redo size
912 bytes sent via SQL*Net to client
458 bytes received via SQL*Net from client
1 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
0 rows processed

SQL>

and below is the final autot ouput (after re-generating the stats using method opt => 'all indexed colums size auto'.

SQL> @wfp.sql

no rows selected

Execution Plan
----------------------------------------------------------
Plan hash value: 2041883080

--------------------------------------------------------------------------------
--------------------------

| Id | Operation | Name | Rows | Bytes
| Cost (%CPU)| Time |

--------------------------------------------------------------------------------
--------------------------

| 0 | SELECT STATEMENT | | 1 | 168
| 12 (9)| 00:00:01 |

| 1 | SORT ORDER BY | | 1 | 168
| 12 (9)| 00:00:01 |

|* 2 | TABLE ACCESS BY INDEX ROWID | DAILY_TRAN_HEADER_TABLE | 1 | 26
| 1 (0)| 00:00:01 |

| 3 | NESTED LOOPS | | 1 | 168
| 11 (0)| 00:00:01 |

|* 4 | TABLE ACCESS BY INDEX ROWID| DAILY_TRAN_DETAIL_TABLE | 1 | 142
| 10 (0)| 00:00:01 |

|* 5 | INDEX RANGE SCAN | IDX_DTD_ACCT_ID | 68 |
| 1 (0)| 00:00:01 |

|* 6 | INDEX RANGE SCAN | IDX_DTH_TRAN_DATE_ID | 1 |
| 1 (0)| 00:00:01 |

--------------------------------------------------------------------------------
--------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

2 - filter("DTH"."INIT_SOL_ID"||NULL='0049' AND "DTH"."BANK_ID"='01')
4 - filter("DTD"."RCRE_TIME">=TO_DATE('2006-12-27 00:00:00', 'yyyy-mm-dd hh24
:mi:ss') AND

"DTD"."TRAN_CRNCY_CODE"||NULL='INR' AND "DTD"."BANK_ID"='01' AND
"DTD"."TRAN_DATE"=TO_DATE('2003-05-08 00:00:00', 'yyyy-mm-dd hh24:
mi:ss') AND

("DTD"."RESTRICT_MODIFY_IND" IS NULL OR "DTD"."RESTRICT_MODIFY_IND
"<>'T') AND "DTD"."TRAN_ID">'

MI752762')
5 - access("DTD"."ACID"='WF112351')
6 - access("DTH"."TRAN_ID"="DTD"."TRAN_ID" AND "DTH"."TRAN_DATE"=TO_DATE('200
3-05-08

00:00:00', 'yyyy-mm-dd hh24:mi:ss'))
filter("DTH"."TRAN_ID">' MI752762')

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
2 consistent gets
0 physical reads
0 redo size
912 bytes sent via SQL*Net to client
458 bytes received via SQL*Net from client
1 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
0 rows processed

SQL>

Now I am very puzzeled - not able to figure out how this improvement has happened -what is there in method_opt => 'all indexed columns size auto' .

can you please explain us using the above 2 explain plans that what is the reason of this change.

Thanks
Ajeet

Followup December 28, 2006 - 9am Central time zone:

why not compare the before and after statistics, that would sort of show you "what changed"

you computed entirely different column statistics, leading to entirely different estimated cardinalities, leading to different plans.

Another consistent get v/s db block get clarification February 8, 2007 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: Mahesh from Arizona

Hi Tom,

Assume u have a table t with 1000 rows, arraysize 100 and 200 rows per block, i.e. table has data that occupies 5 blocks.

Suppose u have an update statement U1 that executes at time T, updates some rows in the third block only. A query S1 (select * from t) starts at time T+5 (update U1 has not yet commited). S1 gets the first two blocks in current mode. When S1 fetches from the third block, it finds it needs to visit a RBS block to get the third block in consistent mode, since changes from U1 are still pending. 2 questions:

1. For the S1 query, does Oracle fetch the first 2 blocks in current mode?
2. When the S1 query fetches from the third block, does the fetch from the third block and the fetch from the RBS block together count as one consistent get, or as two consistent gets?

Followup February 8, 2007 - 12pm Central time zone:

S1 would get the blocks in consistent read mode, not current mode.

1) it does not, it fetched them in consistent read mode.

2) it would be 2 consistent gets.

Blocks accessed for index updates February 9, 2007 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: Steve from Los Angeles, CA, USA

Hi Tom,
I don't believe this specific explanation has been made in this thread, but it seems relevant.
Can you please explain all the blocks that are read/written when a value is updated in an indexed column?
Thanks in advance.

Followup February 12, 2007 - 9am Central time zone:

It depends

update t set indexed_column = 5 where primary_key = 10;

well, that will likely:

a) read the root/branch/leaf blocks from the primary key index, get a rowid
b) using that rowid read a table block for that row
c) update that indexed_column on the block
d) assuming the value actually changed - that indexed_column was not 5 before - we would then
e) read the root/branch/leaf blocks of the index on indexed_column (where value=5 and rowid=that_rowid)
f) get the leaf block in current mode, modify the value
g) if the modified value caused the index block to overfill (split), we would have to then perform more IO to get the new leaf block, get the branch and root blocks of the index in current mode as well - to allow the split to take place

So, in the most basic update - reads on root/branch/leaf - write on a leaf.

in the case of a block split - more reads to get new block, and more current mode gets and writes to update the index structure.

Blocks accessed for index updates February 12, 2007 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: Steve from Los Angeles, CA, USA

Thanks so much for the info on index block access.
A few clarifications, please:

a) = consistent gets, right?
b) = current mode get?
e) = consistent gets?
g) = consistent gets to find the new leaf block, then
current gets of root/branch(es)/leaf to do the split?

Followup February 12, 2007 - 1pm Central time zone:

b) = consistent get + current get. We read the data using consistent reads (always) and then get the block in current mode to do the update.

g) = get space from freelists or ASSM bitmap blocks - not a consistent read, a space request. then current mode reads up the structure to finish the split.

current reads for select February 12, 2007 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Does current read always happen for update statements?
At 11:05 it did a consistent get of a block as of 11:00 and discovered a row in it should be updated. In this case it will request a block in CURRENT MODE. From where Oracle will request this block?

I once remember seeing 80% of 35 million LIO's were current reads. The statement was INSERT..SELECT.

How can this happen?

Thanks,

Followup February 13, 2007 - 8am Central time zone:

Oracle gets the blocks from the buffer cache (which may of course have to get the data from disk first)

ops$tkyte%ORA10GR2> create table t
  2  as
  3  select * from all_objects where 1=0;

Table created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> create index t_idx1 on t(object_id);

Index created.

ops$tkyte%ORA10GR2> create index t_idx2 on t(object_name);

Index created.

ops$tkyte%ORA10GR2> create index t_idx3 on t(owner);

Index created.

ops$tkyte%ORA10GR2> create index t_idx4 on t(object_type);

Index created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> exec dbms_monitor.session_trace_enable;

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> insert into t select * from all_objects;

50162 rows created.

tkprof will show:

insert into t select * from all_objects


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.02       0.02          0          0          0           0
Execute      1      4.50       4.57          2      70709     259323       50162
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        2      4.53       4.59          2      70709     259323       50162

Now, if you do the same thing without the indexes

insert into t select * from all_objects


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.04       0.04          0          0          0           0
Execute      1      1.60       1.56          0      66246       6839       50165
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        2      1.64       1.60          0      66246       6839       50165

It was index maintenance - you have to update that complex data structure.

refinement ;) February 12, 2007 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: A. O'Hara from The Other Side of the Pond

Of course point e) above should be

e) read the root/branch/leaf blocks of the index on indexed_column (where value=<OLD VALUE> and rowid=that_rowid)

and then "updating the value" may mean, if "value=5 and rowid=that_rowid" has to be on a different leaf block, deleting the old entry and then re-scanning the index to insert the new "value=5 and rowid=that_rowid" in another block.

Followup February 13, 2007 - 8am Central time zone:

correct - a modify of a an indexed column in the index structure is a "delete plus insert" so likely there are two leaf blocks involved as the row "moves" in the index.

thanks!

Please clarify February 14, 2007 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: Anil from UK

Hello Tom,
Iam on Oracle9i. The tables used in the query are analyzed. In first query without any hint has the consistent gets of 36147.

SELECT a.actlinkId,actionIndex,a.fieldId,assignShort,assignLong, keywordList,parameterList, sampleServer,sampleSchema
FROM actlink_set a,actlink_mapping b
WHERE a.actlinkId=b.actlinkId
AND b.schemaId=1690
ORDER BY 1 DESC,2 DESC

Execution Plan
----------------------------------------------------------
0
SELECT STATEMENT Optimizer=CHOOSE (Cost=80 Card=1535 Bytes=127405)

1 0
SORT (ORDER BY) (Cost=80 Card=1535 Bytes=127405)

2 1
NESTED LOOPS (Cost=58 Card=1535 Bytes=127405)

3 2
TABLE ACCESS (FULL) OF 'ACTLINK_SET' (Cost=58 Card=27990 Bytes=2071260)

4 2
INDEX (UNIQUE SCAN) OF 'ACTLINK_MAPPING_IND' (UNIQUE)

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
36147 consistent gets
3 physical reads
0 redo size
160161 bytes sent via SQL*Net to client
46862 bytes received via SQL*Net from client
746 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
326 rows processed

Second query query with FTS hint has consistent gets of 1478 only.

SELECT /*+ FULL(b) */ a.actlinkId,actionIndex,a.fieldId,assignShort,assignLong, keywordList,parameterList, sampleServer,sampleSchema
FROM actlink_set a,actlink_mapping b
WHERE a.actlinkId=b.actlinkId
AND b.schemaId=1690
ORDER BY 1 DESC,2 DESC

Execution Plan
----------------------------------------------------------
0
SELECT STATEMENT Optimizer=CHOOSE (Cost=86 Card=1535 Bytes=127405)

1 0
SORT (ORDER BY) (Cost=86 Card=1535 Bytes=127405)

2 1
HASH JOIN (Cost=64 Card=1535 Bytes=127405)

3 2
TABLE ACCESS (FULL) OF 'ACTLINK_MAPPING' (Cost=5 Card=95 Bytes=855)

4 2
TABLE ACCESS (FULL) OF 'ACTLINK_SET' (Cost=58 Card=27990 Bytes=2071260)

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
1478 consistent gets
3 physical reads
0 redo size
160162 bytes sent via SQL*Net to client
46862 bytes received via SQL*Net from client
746 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
326 rows processed

As you been always been suggesting not to use hint should I use hint?

ACTLINK_MAPPING = 22378 rows

ACTLINK_SET = 34737 rows

Thanks
Anil

Blocks accessed for index updates February 20, 2007 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: Steve from Los Angeles, CA, USA

Hi Tom,
It seems that a couple of things are true:

-The addition of an index will cause additional current mode gets.

-Exactly how many additional current mode gets is somewhat unpredictable.

Can you recommend general steps to be taken, to minimize the additional current mode gets (i.e. re-org the indexes, etc.)?

Thanks

Followup February 20, 2007 - 5pm Central time zone:

- if you update the indexed column, sure
- yes, not predicable in a practical sense

b*tree indexes are very resilient structures. You would not reorg them for this update activity.

April 4, 2007 - 6am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom,
The thing about increase of arraysize could decrease logical IO's, is the crux of the matter appears to be

1) When Oracle attempts to retrieve the NEXT row of a query, it checks if it has made a trip to the Client, if it does, it always does a logical IO of the current Block.

b) If it hasn't, it appears to be happy the block is like 'latched' and just gets the next row from the current block.

Both the above points relate to the TWO rows in the same block.

Now, the question is, is it correct that it needs to do a logical IO after a trip to the Client because the World could have changed in the interval? Like the client has not necessarily fetched the next row instantaneously and it NEEDS to do a logical IO to check?

Ravi

Followup April 4, 2007 - 10am Central time zone:

it does a logical IO not because things could have changed - but because it doesn't have the information anymore - it freed it, gave it up, released it.

April 23, 2007 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom,
What if I have an update statement which needs to update rows in B1, which is currently 'latched' by a running Query Q1.

1) Will the update has to wait?
2) It may be unlikely, but possibly with one block reference data table, what if there are 10 queries waiting to get that blocks' data, is there a queuing mechanism for latches as well?

Thanks

Followup April 23, 2007 - 4pm Central time zone:

what do you mean "which is currently 'latched' by a running query"

reads do not block writes
writes do not block reads

Some minor clarifications needed April 24, 2007 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: Vladimir Andreev from Germany

Hi Tom,

Here are some conclusions of mine based on what I learned about how the buffer cache works. Please confirm (or correct) them:

1. At any given moment, at most one "current mode" buffer may exist in the (global) cache for a db block; if it is currently in use (pinned) by a session, other sessions attempting a current get of the same block would block on "buffer busy".

2. A session making a current get of a block searches the cache for a current mode buffer and if not found, reads it from disk (or requests it from the global cache) into a free buffer.

3. A session making a consistent get (as of SCN=X) searches the cache for a "consistent mode" buffer with SCN>=X; if found, it pins it and rolls it back to SCN X; if not, it reads it from disk into a free buffer and then performs the rollback to SCN X.

Questions:
a) does the fact that the (single) current mode buffer is in use by a session cause other sessions that need a consistent version of the same block but must perform a physical read to create it to block on "buffer busy" (since the block is potentially in flux)?

b) Since at any time there may be many different versions of the same block, it would make sense that these versions be sorted by SCN, so that a session searching for a version with SCN>=X can quickly find the version with SCN closest to, but not smaller than X. Are the consistent mode buffers for a block ordered by SCN in the cache buffers chain?

Thanks in advance,

Flado

Followup April 24, 2007 - 1pm Central time zone:

a) don't quite follow you. if there is a current mode version in the cache, why would another session perform a physical read?

b) don't know, detail that doesn't help me day to day, only so much room for so many details :)

Hopefully related April 25, 2007 - 1am Central time zone

Bookmark | Bottom | Top

Reviewer: A.VARADARAJAN from Bangalore, India

I was told by another DBA that if I wanted to reduce my "db file sequential read" waits then one of the ways to do it is by reducing consistant gets. I, after going trough this article, doubt this. after all if the number of records per block is much higher than 15 then the same block will be accessed multiple times and pysica; reads is anyhow not going to be affected.

Followup April 25, 2007 - 9am Central time zone:

LIO = logical IO
PIO = physical IO

In general, if you watch your LIO's your PIO's will take care of themselves.

In general, reducing LIO's can reduce PIO's, the thought is you are reading TOO MANY blocks, not necessarily the same block over and over. However, even if you are reading the SAME block - if you are reading tons of extra blocks, you are likely causing some blocks to be flushed from the cache and you might have to read and re-read them from disk over and over during the processing of your query.

I would say that in general, LIO reduction is what I'm looking to accomplish and if you watch your LIO's your PIO's will take care of themselves.

explain_plan, statistics and optimizer hints May 5, 2007 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: Piyush Agarwal from Pune, India

Dear Tom,

After reading first few reviews I got some basics in consistent gets and impact of arraysize . I think all this is very essential in performance tuning. Please also explain how to reduce SQL*Net roundtrips to/from client. I think if this value is large, it affects the performance. Moreover, for Java (Sprint JDBC) how to manually set the arraysize to improve the performance as you have said for JDBC defaul value is 10. Can we increase the value on the fly on the basis of expected output of each procedure or sql in Java to enhance performance.

We have used sys_refcursors and with sql clauses in the procedures in our project. What is the impact on performance of using them. What if, it contains clob values.

Hence, do you have any document which explains each item of explain_plan, statistics and optimizer hints and their affects . How to adjust them and what are the general standards.

IF you can also throw some light here , it would help me in understanding the document which you may refer with your reply.

Regards,
Piyush

Followup May 8, 2007 - 10am Central time zone:

... SQL*Net roundtrips to/from client. I ...

make less calls - you do that by..... using larger arraysizes (although 100-500 rows at a time is a reasonable maximum, you really do not need to go beyond that)

"sprint jdbc"??? You could read the documentation for your jdbc driver to see what is available - I've shown how to do it for the connection and for individual statements in general.

the performance guide, free on otn.oracle.com, covers explain plans, statistics and how they are used and hints.

June 20, 2007 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom,

I have two scripts, the top one using Bulk collect uses only like, they both look up the same rows and bring up the same count(*) for the number of rows, but the first query does 80 consistent get, while the next one does 39K consistent get.

Does Oracle lets go of the latch even between when PLSQL fetches using SQL?

declare
type b is table of areas_of_land.aol_id%type index
by binary_integer;
a b;
begin
select aol_id bulk collect into a from areas_of_land where
aol_id between 100 and 100001 ;
dbms_output.put_line(a.count);
end;

declare
a areas_of_land.aol_id%type;
cnt number default 0;
begin
for i in (select aol_id from areas_of_land where
aol_id between 100 and 100001 ) loop
a := i.aol_id;
cnt := cnt+1;
end loop;
dbms_output.put_line(cnt);
end;

Explain plan:

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=RULE
1 0 INDEX (RANGE SCAN) OF 'AOL_PK' (UNIQUE)

Followup June 20, 2007 - 12pm Central time zone:

plsql is no different from C or VB or <any client here> in that respect.

plsql is just doing sql and making calls to the database to fetch data.

so, yes.

do NOT bulk collect that many rows, just do 100-500

June 20, 2007 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom

Why not Bulk Collect more than 500 rows?

Thanks

ravi

Followup June 20, 2007 - 1pm Central time zone:

memory and 'practicality'

you want to

loop
get something to process
process it
end loop

biting off more than you can "chew" doesn't make sense, get a bit, work a bit, get a bit, work a bit.

get a ton of stuff, allocate ton of memory, set up ton of stuff into memory (manage a HUGE structure), retrieve small bits from this huge structure, destroy structure.

get a bit of stuff, allocate a small amount of memory, .....

June 21, 2007 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom

I have a test case, where I ask a session one to wait on a row and go on to session2 and update that row a 100 times before the session1 fetches it, and Oracle accurately gets a consistent value as of start of query in Session 1.
The question is, does the rollback segments store the SCN as well, like, does Oracle trawl throgh all the 100 rollback segment values to match the SCN that it wants and outputs it (it can't find it, then its ora 1555, I guess).

Thanks

Ravi

drop table t1;

create table t1(a number);

insert into t1 values(1);

insert into t1 values(2);

select * from t1;

Session1:

begin
for i in (select a from t1 order by a asc) loop
dbms_output.put_line(i.a);
dbms_lock.sleep(7);
end loop;
end;

Session2:

DECLARE
m NUMBER;
BEGIN
FOR i IN 1 .. 100
LOOP
UPDATE t1
SET a = a + 1
WHERE a > 1;

COMMIT;
END LOOP;

SELECT COUNT ( * )
INTO m
FROM t1;
END;

Followup June 21, 2007 - 11am Central time zone:

think of undo as being a linked list. when you modify a block - we put the before image information into undo (call this U1 - so the current mode block points to U1) and the block points to that undo.

now, if you modify the block again, we put the before image information into undo again and the current mode block points to this (call this U2)

suppose a query was started before either update. It hits this block. We discover "this block is too knew and we don't see anything else in the buffer cache that is at least old enough".

So, we take this block and follow the pointer, we find U2. We apply U2 - the block we have now looks just like it did before the second update. We look at it and say "hmm, still too new". Now, we need to rollback this block too - it points to U1 (because the current mode block pointed to U1 after the first update, before the second update).

We roll it back, it is now old enough.

The transaction history is used to walk the undo history to find a block version that predates your query. The scn is not really stored there, but it can be inferred from other undo information.

This is conceptually how it happens, there are edge cases, caveats, shortcuts, lots of stuff - but conceptually - the block is undone bit by bit till we have a version old enough.

Consistent gets July 18, 2007 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: Manju from India. Pune.

Dear Tom,

I have 1000 rows in emp table when i execute
select * from emp; query statistics looks like this

My Question is : As per this formula rows/arraysize+total blocks. In my case i have to get 81 consistent gets but i dont see any thing in stats, can you pls clarify this.

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          0  consistent gets
          0  physical reads
          0  redo size
          0  bytes sent via SQL*Net to client
          0  bytes received via SQL*Net from client
          0  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
       1000  rows processed

SQL> exec show_space('EMP');
Free Blocks.............................1
Total Blocks............................15
Total Bytes.............................122880
Unused Blocks...........................11
Unused Bytes............................90112
Last Used Ext FileId....................220
Last Used Ext BlockId...................1792
Last Used Block.........................4

PL/SQL procedure successfully completed.

Regards
Manju

Followup July 18, 2007 - 10am Central time zone:

stop doing things as SYS - just stop it right now. Never use that account for anything other then when it is mandatory to do so (and it almost never is)

sys is special, it is magic, it is why you don't see anything

Consistent gets July 18, 2007 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: Manju from India. Pune.

Dear Tom,

Sorry a small change rows/arraysize+(total blocks-unused blocks)
we have to get consistent gets as 70 in out case. pls clarify.

Regards

Manju

consistent gets go up if I use TRIM in where clause July 18, 2007 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: conundrum

hi Tom,

I have a query which has 11652 consistent gets and 5977 physical reads.
This query use "rule" hint and returns the rows quickly (I only had small number of rows - 10 - to test, though).
Explain plan has a bunch of nested nested loops with one full table scan in the last nested loop and others are TABLE ACCESS (BY INDEX ROWID).

The same query, but without "rule" hint works but it gets 65582 consistent gets and 7881 physical reads.

Now when I change "where" clause and remove TRIM() from one of the "where" conditions, the execution plan shows no longer that TABLE FULL SCAN (which is a "good thing" I guess), but TABLE ACCESS (BY INDEX ROWID) etc., which means that index was used. But instead of response time goes down or at least stay the same for these 10 rows, the query took long long time so I had to kill it. It seems that the number of consistent gets went to enormous proportions and the number of physical reads went down!?
How to explain this strange behavior?

thanks,

Consistent gets July 19, 2007 - 12am Central time zone

Bookmark | Bottom | Top

Reviewer: Manju from India.Pune

Dear Tom,

Thanks for your reply, is there any reason behind that magic, can you pls explain me.that will be 
great help.

Regards

Manju.

Followup July 19, 2007 - 10am Central time zone:

yes, because we programmed it that way. that is the reason

sys is magic

DO NOT USE IT

it is that simple.

consistent reads in delete July 20, 2007 - 7am Central time zone

Bookmark | Bottom | Top

Reviewer: Robert Koltai from Budapest

Hi Tom,

I have two environments Budapest and Milano.
Deleting records in Budapest is fast, and is slow in Milano:)

******
Milano SQL TRACE:
******
call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.01       0.00          0          0          0           0
Execute      1    128.27     127.93          1    7001151      23310        2264
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        2    128.28     127.93          1    7001151      23310        2264

Rows     Row Source Operation
-------  ---------------------------------------------------
      0  DELETE  ALARMLOG (cr=7014735 pr=1 pw=0 time=129579424 us)
   2264   COUNT STOPKEY (cr=7001120 pr=0 pw=0 time=128253166 us)
   2264    NESTED LOOPS  (cr=7001120 pr=0 pw=0 time=125607833 us)
   2264     VIEW  VW_NSO_1 (cr=12152 pr=0 pw=0 time=224828 us)
   2264      SORT UNIQUE (cr=12152 pr=0 pw=0 time=215769 us)
   2264       COUNT STOPKEY (cr=12152 pr=0 pw=0 time=2093835 us)
   2264        FILTER  (cr=12152 pr=0 pw=0 time=2091568 us)
   2264         NESTED LOOPS OUTER (cr=12152 pr=0 pw=0 time=2089296 us)
   2264          NESTED LOOPS OUTER (cr=7624 pr=0 pw=0 time=2032663 us)
   2264           PARTITION RANGE ALL PARTITION: 1 2 (cr=3096 pr=0 pw=0 time=1966934 us)
   2264            INDEX FULL SCAN PK_ALARMLOG PARTITION: 1 2 (cr=3096 pr=0 pw=0 time=1964636 
us)(object id 10022)
      0           PARTITION RANGE ALL PARTITION: 1 2 (cr=4528 pr=0 pw=0 time=55810 us)
      0            INDEX RANGE SCAN LOG_STATECHANGEDEF_IDX PARTITION: 1 2 (cr=4528 pr=0 pw=0 
time=39340 us)(object id 10034)
      0          PARTITION RANGE ALL PARTITION: 1 2 (cr=4528 pr=0 pw=0 time=54239 us)
      0           INDEX RANGE SCAN LOG_MONITOREDALARM_IDX PARTITION: 1 2 (cr=4528 pr=0 pw=0 
time=38572 us)(object id 10030)
   2264     PARTITION RANGE ALL PARTITION: 1 2 (cr=6988968 pr=0 pw=0 time=123237059 us)
   2264      INDEX FULL SCAN PK_ALARMLOG PARTITION: 1 2 (cr=6988968 pr=0 pw=0 time=123161005 
us)(object id 10022)

******
Budapest SQL TRACE:
******
call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        2      0.00       0.00          0          0          0           0
Execute      2      0.31       0.34          0       3000      22373        3095
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        4      0.31       0.34          0       3000      22373        3095

Rows     Row Source Operation
-------  ---------------------------------------------------
      0  DELETE  ALARMLOG (cr=15492 pr=0 pw=0 time=837947 us)
   3095   COUNT STOPKEY (cr=6206 pr=0 pw=0 time=73586 us)
   3095    HASH JOIN RIGHT SEMI (cr=6206 pr=0 pw=0 time=67435 us)
   3095     VIEW  VW_NSO_1 (cr=6198 pr=0 pw=0 time=49564 us)
   3095      COUNT STOPKEY (cr=6198 pr=0 pw=0 time=49562 us)
   3095       FILTER  (cr=6198 pr=0 pw=0 time=46467 us)
   3095        NESTED LOOPS OUTER (cr=6198 pr=0 pw=0 time=46464 us)
   3095         NESTED LOOPS OUTER (cr=3103 pr=0 pw=0 time=24793 us)
   3095          PARTITION RANGE SINGLE PARTITION: 1 1 (cr=8 pr=0 pw=0 time=3121 us)
   3095           INDEX FULL SCAN PK_ALARMLOG PARTITION: 1 1 (cr=8 pr=0 pw=0 time=17 us)(object id 
10022)
      0          PARTITION RANGE SINGLE PARTITION: 1 1 (cr=3095 pr=0 pw=0 time=19122 us)
      0           INDEX RANGE SCAN LOG_STATECHANGEDEF_IDX PARTITION: 1 1 (cr=3095 pr=0 pw=0 
time=12116 us)(object id 10034)
      0         PARTITION RANGE SINGLE PARTITION: 1 1 (cr=3095 pr=0 pw=0 time=18884 us)
      0          INDEX RANGE SCAN LOG_MONITOREDALARM_IDX PARTITION: 1 1 (cr=3095 pr=0 pw=0 
time=11864 us)(object id 10030)
   3095     PARTITION RANGE SINGLE PARTITION: 1 1 (cr=8 pr=0 pw=0 time=6241 us)
   3095      INDEX FULL SCAN PK_ALARMLOG PARTITION: 1 1 (cr=8 pr=0 pw=0 time=3125 us)(object id 
10022)

Some data from dba_indexes, dba_extents for both enviroments:
INDEX_NAME    BLEVEL LEAF_BLOCKS DISTINCT_KEYS CLUSTERING_FACTOR   NUM_ROWS 
-------------------- ----------- ------------- ----------------- ---------- 
MIL: PK_ALARMLOG     2        29          5065              1103       5065 
BUD: PK_ALARMLOG     1         5          2095                66       2095 

SEGMENT_NAME        SEGMENT_TYPE            BYTES     BLOCKS    EXTENTS 
------------------- ------------------ ---------- ---------- ---------- 
MIL: PK_ALARMLOG    INDEX PARTITION      13631488       1664         13
MIL: PK_ALARMLOG    INDEX PARTITION      25165824       3072         24
BUD: PK_ALARMLOG    INDEX PARTITION       7340032        896          7

Note that both tables contained only few thousand records when the delete was issued.

It seems that INDEX FULL SCAN on PK_ALARMLOG is the killer, but how come that you need 6988968 cr 
in Milano, and only 8 in Budapest?
My guess is that ALARMLOG table in Milano once contained hundreds of thousands of records. 
Q1) Do you think this is the reason?
Q2) Could I have avoided the problem with some kind of storage setting for the index? (I mean to 
get the unused space freed)
Q3) Can you explain what is the 6988968 cr needed for? If 1 cr is for 1 block (8Kb), then it's 
~56GB !!
Q4) Do you advise an index rebuild?

Thanks in advance
R.

Followup July 20, 2007 - 8am Central time zone:

are your statistics up to date.

the index is being full scanned in a nested loop (over and over)

consistent reads in delete July 20, 2007 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: Robert Koltai from Budapest

WOW that a was fast reply! And inspiring!!! Thanks!

Yes I made a GATHER_SCHEMA_STATS right before executing the command, and was expecting a new 
execution plan (even flushed the share pool in an other try), but the plan is always the same in 
Milano.

Here's an other run with different number of records:
DELETE FROM alarmLog 
WHERE
 (ROWNUM <= :"SYS_B_0" ) AND logRecordId IN ( SELECT DISTINCT logRecordId 
  FROM alarmLog TAB1, OMCDBSYS.LOG_stateChangeDef TAB2, 
  OMCDBSYS.LOG_monitoredAlarm TAB3 WHERE (ROWNUM <= :"SYS_B_1" ) AND 
  tab1.logRecordId = tab2.stateChangeDefLogRecordId(+) AND tab1.logRecordId = 
  tab3.monitoredAlarmLogRecordId(+) AND :"SYS_B_2"=:"SYS_B_3" )


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1    293.89     300.18        254   15620516      52040        5065
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        2    293.89     300.18        254   15620516      52040        5065

Rows     Row Source Operation
-------  ---------------------------------------------------
      0  DELETE  ALARMLOG (cr=15689573 pr=254 pw=0 time=308662082 us)
   5065   COUNT STOPKEY (cr=15659108 pr=4 pw=0 time=291458740 us)
   5065    NESTED LOOPS  (cr=15659108 pr=4 pw=0 time=291302036 us)
   5065     VIEW  VW_NSO_1 (cr=23453 pr=4 pw=0 time=510225 us)
   5065      SORT UNIQUE (cr=23453 pr=4 pw=0 time=479833 us)
   5065       COUNT STOPKEY (cr=23453 pr=4 pw=0 time=313387 us)
   5065        FILTER  (cr=23453 pr=4 pw=0 time=308316 us)
   5065         NESTED LOOPS OUTER (cr=23453 pr=4 pw=0 time=303238 us)
   5065          NESTED LOOPS OUTER (cr=13323 pr=2 pw=0 time=179797 us)
   5065           PARTITION RANGE ALL PARTITION: 1 2 (cr=3193 pr=0 pw=0 time=26275 us)
   5065            INDEX FAST FULL SCAN PK_ALARMLOG PARTITION: 1 2 (cr=3193 pr=0 pw=0 time=21180 
us)(object id 10022)
      0           PARTITION RANGE ALL PARTITION: 1 2 (cr=10130 pr=2 pw=0 time=140604 us)
      0            INDEX RANGE SCAN LOG_STATECHANGEDEF_IDX PARTITION: 1 2 (cr=10130 pr=2 pw=0 
time=103116 us)(object id 10034)
      0          PARTITION RANGE ALL PARTITION: 1 2 (cr=10130 pr=2 pw=0 time=117327 us)
      0           INDEX RANGE SCAN LOG_MONITOREDALARM_IDX PARTITION: 1 2 (cr=10130 pr=2 pw=0 
time=83493 us)(object id 10030)
   5065     PARTITION RANGE ALL PARTITION: 1 2 (cr=15635655 pr=0 pw=0 time=287768328 us)
   5065      INDEX FULL SCAN PK_ALARMLOG PARTITION: 1 2 (cr=15635655 pr=0 pw=0 time=287600111 
us)(object id 10022)

Initially I though that ONE index full scan took this much time, but now from your reply I got the 
idea to divide cr by the number of records and in three different Milano test cases:

6988968/2264=3087
15635655/5065=3087
19355490/6270=3087

So we have a MAGIC NUMBER here:))
My guest is that all records were indexed by partition P2 of PK_ALARMLOG having 3072 blocks.

OK now I also realized that the Budapest version of the plan has HASH JOIN RIGHT SEMI instead of 
the NESTED LOOPS.

So the question remaining: Why does Oracle choose NESTED LOOPS instead of HASH?
The response lies in the statistics I guess. But the're up to date. All other init paramters that 
may affect the optimizer are the same.

Shall I open an SR or do you have a better idea?
Thanks again!
R.

Followup July 20, 2007 - 5pm Central time zone:

do you have cursor sharing on on purpose?????

get the explain plan, compare to the actuals - are the estimated row counts accurate on the database where the plan is "not good"

are they configured the same (init.ora wise)

July 23, 2007 - 8am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Tom,

I bet you have seen questions like the one from a user who ran a trace logged on as SYS, they know 
enough about consistent gets etc, do you suspect the question is simply intended to test YOU?

Or do you assume that you need to give room that they could have a genuine doubt, even though its 
hard to believe that there is a valid reason why they would not tell you their Userid of SYS?

Thanks

Followup July 23, 2007 - 8am Central time zone:

I don't know what you mean here... Not sure where you are trying to go?

consistent reads in delete - Milan problem July 23, 2007 - 9am Central time zone

Bookmark | Bottom | Top

Reviewer: Rrobert Koltai from Budapest

Hi Tom thanks for your answers!

- Yes, cursor sharing is on on purpose
- init.oras are identical

I'll check the explain plan against the actual one as soon as the Milan colleguaes load some data 
into the DB with the application.

But the real question for You is this:
- the nested loop vs. hash join accounts for a difference in the consistent reads in the 10^3 
range. (As we saw that the nested loop is executed "number of record" times and the hash join 
once:))
- The actual difference in the cr is in the 10^6 range(6988968 vs. 8)
So still 10^3 is missing.

=>
****
Why do we need to read 10^3 blocks in Milan when we read only 8 blocks in Budapest for one full 
scan of PK_ALARMLOG?
May it have something to to with the size history of the table and its PK? If so then could we 
avoid this problem somehow? (I mean could the full scan be affected by having had a lot of records 
in PK_ALARMLOG?)
The "consistent reads in one nested loop" (3087) seems to have to do something with the number of 
blocks of PK_ALARMLOG partition 2. See my first post.
****

Thanks again,
Robert

August 8, 2007 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: Alexander the ok

Hi Tom,

I'm having a problem with a query that's selecting from a view.  In development it runs well, in 
production the major difference that sticks out is consistent gets.

Before I start posting tkprofs, explain plans, and all that stuff I was hoping if you could answer 
the following question I can just figure it out myself.

I went to trace the query in prod, and it ran exactly how it does in dev.  Why would that happen?  
If I set tracing off, it runs terrible again.  That's what I would like to know first, what could 
possibly cause that.

Thanks.

Followup August 14, 2007 - 10am Central time zone:

bind peeking....

search this site for bind peeking to read lots about it.

sql_trace is like an optimizer parameter - it causes a hard parse the first time, with your bind values (not what binds were used for the "bad" plan - but the "bad" plan IS NOT A BAD PLAN, it just isn't the best plan for your current bind variable!!!!)

here is a "for example", id=1 - use index, id=99 - use full scan. Each plan is BEST for it's respective value, but there is only ONE PLAN in place. Depending on who hard parses first - you either get index or full scan in play:


ops$tkyte%ORA10GR2> create table t as select 99 id, a.* from all_objects a;
Table created.

ops$tkyte%ORA10GR2> update t set id = 1 where rownum = 1;
1 row updated.

ops$tkyte%ORA10GR2> create index t_idx on t(id);
Index created.

ops$tkyte%ORA10GR2> exec dbms_stats.gather_table_stats( user, 'T', method_opt => 'for all indexed 
columns size 254' );
PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> variable n number
ops$tkyte%ORA10GR2> set autotrace traceonly explain
ops$tkyte%ORA10GR2> select count(object_type) from t where id = 1;
------------------------------------------------------------------------
| Id  | Operation                    | Name  | Rows  | Bytes | Cost (%CP
------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |       |     1 |    14 |     2   (
|   1 |  SORT AGGREGATE              |       |     1 |    14 |
|   2 |   TABLE ACCESS BY INDEX ROWID| T     |     1 |    14 |     2   (
|*  3 |    INDEX RANGE SCAN          | T_IDX |     1 |       |     1   (
------------------------------------------------------------------------

ops$tkyte%ORA10GR2> select count(object_type) from t where id = 99;
------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes | Cost (%CPU)| Time
------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |     1 |    14 |   159   (2)| 00:00:0
|   1 |  SORT AGGREGATE    |      |     1 |    14 |            |
|*  2 |   TABLE ACCESS FULL| T    | 49888 |   682K|   159   (2)| 00:00:0
------------------------------------------------------------------------

ops$tkyte%ORA10GR2> set autotrace traceonly statistics;
ops$tkyte%ORA10GR2> exec :n := 99;
PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> select count(object_type) from t where id = :n;
Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
        711  consistent gets

ops$tkyte%ORA10GR2> exec :n := 1;
PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> select count(object_type) from t where id = :n;
Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        711  consistent gets

ops$tkyte%ORA10GR2> alter session set sql_trace=true;
Session altered.

ops$tkyte%ORA10GR2> select count(object_type) from t where id = :n;
Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
          3  consistent gets

ops$tkyte%ORA10GR2> alter session set sql_trace=false;
Session altered.

ops$tkyte%ORA10GR2> select count(object_type) from t where id = :n;
Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        711  consistent gets

interesting!!! August 14, 2007 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Your reply to the bind peek concept is very interesting.  If the explain plan generated for the 
first value of bind variable carries for all other bind values, how to avoid that?  I am interested 
in attribute that has very few distinct values.

select * from sales where region = 'North America';
select * from sales where region = 'Europe';

If I have 7 different region values in the SALES table that has more than million rows, I want the 
query to use FULL TABLE SCAN for NORTH AMERICA (constituting more than 60% of the rows) and use 
Index scan for the region AFRICA.

How to get this happen?  Avoid bind variables in this case?

thanks,

Followup August 20, 2007 - 11am Central time zone:

you would avoid binding that particular column predicate, yes.

I'll let you have seven slots in the SGA for your sql (if you had 7,000 regions, the answer would be different of course)

slots in sga? August 20, 2007 - 12pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

I did not understand about "slots in SGA".  What does it mean.

Thanks for answering my earlier question.

Followup August 22, 2007 - 10am Central time zone:

v$sql is an in memory table, it consumes space in the SGA, that is where sql is cached (amongst other v$ like structures)

each SQL you parse takes some amount of SGA memory, out of the shared pool. It gets its "slot", slice, piece, bit, chunk... of memory.

Small question August 28, 2007 - 1pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi, 
Below is attached screen shot of alert log, my query is fatching data taking 5 mins, I have small 
question on belows screen shot can you tell me is it fetching any data? 
Why it is not giving operation details in this trace file

like 
STAT #1 id=301 cnt=0 pid=300 pos=1 obj=58650 op='TABLE ACCESS BY INDEX ROWID SRCADJ_IDW_TXN_ACTY '
STAT #1 id=302 cnt=0 pid=301 pos=1 obj=58651 op='INDEX RANGE SCAN PK_SRCADJ_IDW_TXN_ACTY '
STAT #1 id=303 cnt=0 pid=229 pos=2 obj=217354 op='INDEX RANGE SCAN IX_INSTN_CNTXT_NUNQ '







screen shot from trace file
===========================

WAIT #1: nam='db file scattered read' ela= 1658 p1=4 p2=12777 p3=16
WAIT #1: nam='db file scattered read' ela= 1469 p1=4 p2=12793 p3=16
WAIT #1: nam='db file scattered read' ela= 1373 p1=4 p2=12825 p3=16
WAIT #1: nam='db file scattered read' ela= 1588 p1=4 p2=12841 p3=16
WAIT #1: nam='db file scattered read' ela= 877 p1=4 p2=12873 p3=10
FETCH #1:c=214800000,e=267706635,p=99513,cr=326241,cu=0,mis=0,r=1,dep=0,og=4,tim=10257239122658
WAIT #1: nam='SQL*Net message from client' ela= 2365 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message to client' ela= 7 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=3698,p=0,cr=0,cu=0,mis=0,r=15,dep=0,og=4,tim=10257239136352
WAIT #1: nam='SQL*Net message from client' ela= 126172 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message to client' ela= 12 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=1567,p=0,cr=0,cu=0,mis=0,r=15,dep=0,og=4,tim=10257239264915
WAIT #1: nam='SQL*Net message from client' ela= 253613 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message to client' ela= 7 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=1192,p=0,cr=0,cu=0,mis=0,r=15,dep=0,og=4,tim=10257239521115
WAIT #1: nam='SQL*Net message from client' ela= 279000 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message to client' ela= 5 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=1225,p=0,cr=0,cu=0,mis=0,r=15,dep=0,og=4,tim=10257239803503
WAIT #1: nam='SQL*Net message from client' ela= 301066 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message to client' ela= 4 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=630,p=0,cr=0,cu=0,mis=0,r=15,dep=0,og=4,tim=10257240106787
WAIT #1: nam='SQL*Net message from client' ela= 343529 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message to client' ela= 3 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=779,p=0,cr=0,cu=0,mis=0,r=15,dep=0,og=4,tim=10257240452356
WAIT #1: nam='SQL*Net message from client' ela= 358273 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message to client' ela= 4 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=839,p=0,cr=0,cu=0,mis=0,r=15,dep=0,og=4,tim=10257240812510
WAIT #1: nam='SQL*Net message from client' ela= 371427 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message to client' ela= 4 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=735,p=0,cr=0,cu=0,mis=0,r=15,dep=0,og=4,tim=10257241185629
WAIT #1: nam='SQL*Net message from client' ela= 381607 p1=1650815232 p2=1 p3=0
WAIT #1: nam='SQL*Net message to client' ela= 4 p1=1650815232 p2=1 p3=0
FETCH #1:c=0,e=1246,p=0,cr=0,cu=0,mis=0,r=3,dep=0,og=4,tim=10257241569629

Thanks in advance;

Followup September 4, 2007 - 3pm Central time zone:

the trace file snippet clearly shows data being fetched - not sure what I'm looking for?

you will not get STAT records until the cursor is CLOSED by the client application - and then only if you are still tracing.

Too slow? September 18, 2007 - 4pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch       38      8.02       7.85          0     135119          0        3625
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total       40      8.02       7.85          0     135119          0        3625


Elapsed times include waiting on following events:
  Event waited on                             Times   Max. Wait  Total Waited
  ----------------------------------------   Waited  ----------  ------------
  SQL*Net message to client                      38        0.00          0.00
  SQL*Net more data to client                   254        0.00          0.02
  SQL*Net message from client                    38        0.00          0.21
********************************************************************************

Question:

1. Doesn't 8 seconds seem very slow for retrieving 3625 rows especially since there aren't any significant wait events?

Thanks

Followup September 19, 2007 - 10am Central time zone:

select count(*) from one_hundred_quadrilion_rows;

that query - returns 1 record.
Now, looking at the table name, one might anticipate it taking some amount of time.

the number of records returned from a query is a meaningless metric when evaluating its performance.

i don't understand the comment about the wait events (and the lack thereof), it was all cpu time here. just because something didn't wait for some resource does not mean "it'll be super fast".

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        2     17.70      22.86     166872     166956          0           3
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        4     17.70      22.86     166872     166956          0           3

is that too slow? it only gets three records....


select count(*)
from
 big_table union all select count(*) from big_table union all select count(*)
  from big_table


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        2     17.70      22.86     166872     166956          0           3
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        4     17.70      22.86     166872     166956          0           3

Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 58

Rows     Row Source Operation
-------  ---------------------------------------------------
      3  UNION-ALL  (cr=166956 pr=166872 pw=0 time=11732562 us)
      1   SORT AGGREGATE (cr=55652 pr=55624 pw=0 time=11732542 us)
25000000    INDEX FAST FULL SCAN BIG_TABLE_PK (cr=55652 pr=55624 pw=0 time=25029584 us)(object id 
134235)
      1   SORT AGGREGATE (cr=55652 pr=55624 pw=0 time=5563526 us)
25000000    INDEX FAST FULL SCAN BIG_TABLE_PK (cr=55652 pr=55624 pw=0 time=25000354 us)(object id 
134235)
      1   SORT AGGREGATE (cr=55652 pr=55624 pw=0 time=5565519 us)
25000000    INDEX FAST FULL SCAN BIG_TABLE_PK (cr=55652 pr=55624 pw=0 time=25000220 us)(object id 
134235)

Now, now it doesn't look like that was too slow - it processed a TON of data.

You need to look to the details here - you did 135,119 logical IOs, I doubt it took that many blocks to hold 3625 simple records - therefore, there were aggregates - group bys, sorts, joins maybe and lots of stuff done to thousands or millions of rows - and then 3625 were produced from it.

the amount of work performed - that would be meaningful.

buffer cache, September 25, 2007 - 3pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

When a query is issued for example SELECT statement, the output is returened to the calling user 
process.  The rows that satisfies my query are returned through database buffer cache. 

If my buffer cache can hold 1000 blocks in the memory, and I issue the following statements from a 
table that is having million blocks, how does oracle work?

select * from big_table;

select count(*) from big_table; --here the output is one row

insert into another_big_table
select * from big_table;

Where does the rows store after it process from the block and before it sends it across?

For example, if I issue a statement like

select * from big_table where name='xyz' and assume there is no index on "name" column.  The 
big_table has more than a million blocks.  If my cache can hold 1000 blocks at a time, and if the 
name ='xyz' is stored in 5000 blocks, where will Oracle store the blocks that satisfies my request? 
 How does it make room for the new blocks that are coming out of the disk?

Thanks,

Followup September 26, 2007 - 9pm Central time zone:

... The rows that satisfies my query are returned
through database buffer cache. ....

that premise is wrong, or at the very least, incomplete.

there are times when the buffer cache is not used (parallel query)

and there is always TEMP where data can be paged out

and there is always that

select * from a_PETABYTE_table;

needs only to get enough block buffer space to hold A SINGLE ROW (or whatever your array fetch size is, or one block) in order to answer it. I mean, that full scan - reads a block (we need a block in the cache - sure - get it, use it). Then we need a second block - but the first block - it could be gone, might not be but so what, we don't need it anymore.

You could run with a 2 block buffer cache and answer any query. Might be slow, but it would work - you can only process A BLOCK AT A TIME - you are serial!!!

blocks, September 27, 2007 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

In continuation to the answer you have provided....
Where will the "read" block go?  Say I issue a query and if it takes 10 minutes to execute, where 
will the "read" block sit until the query completes its execution?  

thanks,

Followup September 28, 2007 - 3pm Central time zone:

you need a block. You get the block from the cache and basically copy the contents into your working storage.

It might stay in the block buffer cache for a while or not, it doesn't really matter.

will a low arraysize flood the buffer cache? September 27, 2007 - 6pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi

I wonder if a low arraysize such as 1 can flood the databae cache?

For example if a query using arraysize 1 needs to execute 100000 consistent gets, if that query is 
running in several sessions would I see some waits such as buffer busy waits, cache buffer chains?

Followup September 28, 2007 - 5pm Central time zone:

no, it won't.

let us say you have 100,000 consistent gets to fetch 100,000 rows.

Those 100,000 rows are stored 100 rows per block (assume), so there are 1,000 blocks needed.

You could have a buffer cache of TWO BLOCKS and still be able to do that.

when you ask for row one, we get block one out of the buffer cache - we might have to do physical IO to do that, but whatever, we get block one out of the cache - read row one - and send it back to the client. Now, block one is in the cache - or block one has been flushed from cache. Client asks for row two. We get block one out of the cache again (maybe reading it from disk if need be, whatever) and do it all over again.

Your query never needs more than one block to be in the cache at ANY point in time.

will a low arraysize flood the buffer cache? September 28, 2007 - 5pm Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi

From a concurreny view low arraysize MIGHT cause contention for blocks no? If I have 10 sessions 
trying to read the same 1000 blocks (as your example) 100000 times each one of them I think we 
would see waits such as buffer busy waits?

By the way, English is not my native language and I have always had problem understanding what does 
pinning buffer mean. To exact, what does pinning mean? May you please give a brief expalnation. I 
have searched you site but all explanation I have seen is about pinning objects in library cache.

Thanks a lot

Followup October 3, 2007 - 1pm Central time zone:

yes, the lower arraysize can lead to a higher number of logical IO's as you get and re-get the same block over and over - which requires latching every time you do that.

they would not see buffer busy waits really, it would be cache buffer chains latching.

but this is not "flooding" - it is latch contention.

if you pin something to the wall, it is stuck to the wall - until you unpin it.

if you pin something in oracle, it is stuck there until you unpin it.

buffer cache, October 1, 2007 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

In your previous reply, you mentioned about "working storage".  What is that working storage 
called?  Does each process has its own working storage?  How do we find the size of the working 
storage?  Is it adjustable?

Thanks,

Followup October 3, 2007 - 3pm Central time zone:

it is your pga memory - your sort areas, hash areas, storage for plsql variables, cursor pointers, and so on. It is your private session memory.

will a low arraysize flood the buffer cache? October 4, 2007 - 4am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Hi Tom

I have done a small test, I change arraysize from 15 to 1 and see consistent increases however if I 
look at x$bh.tch (buffer touch count) it does not increase.

I was expecting it to increase since I am reading the same blocks more than arraysize 15. How so?

buffers, October 4, 2007 - 11am Central time zone

Bookmark | Bottom | Top

Reviewer: A reader

Thanks for letting me know about working area.  In that case, after the block is processed, will be 
sent to private sql area before it delivers to the calling user process?  

If the SELECT statement takes 10 minutes to execute, at minute 4 if block 1 is processed and at 
minute 5 if block 10 is processed, will these blocks sit in working area till the 10th minute?

Where would blocks reside in case of INSERT statement?

consistent gets October 10, 2007 - 7pm Central time zone

Bookmark | Bottom | Top

Reviewer: Tim

I have an update statement with subquery SELECT that consumes 153 secs and the tkprof shows that it 
had to visit 463879 blocks (query=463879) to update 37864 rows and the number of current blocks is 
1140. The execution plan shows that index is used to scan the table. How do I reduce "query" value 
as I suspect it is taking much time to process consistent gets. Thanks

Followup October 14, 2007 - 10pm Central time zone:

one might guess that you are updating indexed columns themselves and they in turn need to walk the indexes...

did you do the tkprof with wait events, it would help you come to the right conclusion as to what you are waiting on.

same plan different hosts cloned environments January 4, 2008 - 10am Central time zone

Bookmark | Bottom | Top

Reviewer: Susan from Newton, MA

We have a sql statement that runs with very high consistent gets in our production environment but 
much lower consistent gets in our staging.  I ran an autotrace in both environments and the 
execution paths are the same.  Our staging environment is an exp/imp of our production system.  I'm 
wondering if re-building indexes and/or re-defining the production tables would be something to 
consider.  Any changes need to be done on-line :( Thanks for any suggestions.

Followup January 4, 2008 - 12pm Central time zone:

consider this example:

ops$tkyte%ORA10GR2> create table t as select 1 x from dual;

Table created.

ops$tkyte%ORA10GR2> variable x refcursor
ops$tkyte%ORA10GR2> variable y refcursor
ops$tkyte%ORA10GR2> variable z refcursor
ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> @trace
ops$tkyte%ORA10GR2> alter session set events '10046 trace name context forever, level 12';

Session altered.

ops$tkyte%ORA10GR2> exec open :x for select * from t t1;

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> exec open :y for select * from t t2;

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> print x

         X
----------
         1

ops$tkyte%ORA10GR2> begin
  2     for i in 1 .. 10000
  3     loop
  4          update t set x = x+1;
  5          commit;
  6     end loop;
  7  end;
  8  /

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> exec open :z for select * from t t3;

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> print y

         X
----------
         1

ops$tkyte%ORA10GR2> print z

         X
----------
     10001

simple one block table, no indexes - very trivial query.... however, tkprof shows:

SELECT * FROM T T1

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          1          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        1      0.00       0.00          0          3          0           1
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        3      0.00       0.00          0          4          0           1
********************************************************************************
SELECT * FROM T T2

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          1          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        1      0.03       0.07          0      10003          0           1
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        3      0.03       0.07          0      10004          0           1
********************************************************************************
SELECT * FROM T T3

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          1          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch        1      0.00       0.00          0          3          0           1
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        3      0.00       0.00          0          4          0           1

Your LIO's could be increased due to consistent read (we need to process undo in order to process your query).

What I would do is look at the row source operations in the tkprofs - they can show the number of IO's performed by each step. Look for wide variations between the two systems and see if anything pops out - if all of the extra IO is against a single index - that index *might* be a candidate for a rebuild.

I would NOT reorg the entire schema, no.

Thanks January 4, 2008 - 2pm Central time zone

Bookmark | Bottom | Top

Reviewer: Susan from Newton, MA

Thank you Tom that was most helpful.

January 28, 2008 - 10pm Central time zone

Bookmark | Bottom | Top

Reviewer: Aman.... from India

Hi sir,
I am trying to understand this consistent gets.some how I guess reading this thread, I am able to come some where.But still there are some doubts which I want to ask.
Here is a table creation with some data.
DB:10.2.0.1
Os:RHEL4

begin
for i in 1..10000 loop
insert into test values (i);
end loop;
end;
PL/SQL procedure successfully completed.

SQL> set autot trace stat
SQL> select count(0) from test;



Statistics
----------------------------------------------------------
          5  recursive calls
          0  db block gets
         47  consistent gets
          0  physical reads
          0  redo size
        411  bytes sent via SQL*Net to client
        385  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed
SQL> SQL> show arraysize
arraysize 15
SQL> select * from test;

10000 rows selected.


Statistics
----------------------------------------------------------
          4  recursive calls
          0  db block gets
        714  consistent gets
          0  physical reads
          0  redo size
     176815  bytes sent via SQL*Net to client
       7711  bytes received via SQL*Net from client
        668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10000  rows processed
SQL> select count(*) from test;


Statistics
----------------------------------------------------------
          4  recursive calls
          0  db block gets
         47  consistent gets
          0  physical reads
          0  redo size
        411  bytes sent via SQL*Net to client
        385  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

SQL> set autot off
SQL> select blocks from dba_segments  where segment_name='TEST';

    BLOCKS
----------
       640
        24
SQL> select count(*) from test;


Statistics
----------------------------------------------------------
          4  recursive calls
          0  db block gets
         47  consistent gets
          0  physical reads
          0  redo size
        411  bytes sent via SQL*Net to client
        385  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

SQL> set autot off
SQL> select blocks from dba_segments  where segment_name='TEST';

    BLOCKS
----------
       640
        24
SQL> select blocks,segment_name from dba_segments where segment_name='TEST' and owner='AMAN';

    BLOCKS
----------
SEGMENT_NAME
--------------------------------------------------------------------------------
        24
TEST
SQL> set autot trace stat
SQL> select * from test;

10000 rows selected.


Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        690  consistent gets
          0  physical reads
          0  redo size
     176815  bytes sent via SQL*Net to client
       7711  bytes received via SQL*Net from client
        668  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
      10000  rows processed

SQL> update test set a=1;

10000 rows updated.


Statistics
----------------------------------------------------------
        131  recursive calls
     <b> 26401  db block gets</b>
        <b>150  consistent gets</b>
          0  physical reads
    5301360  redo size
        678  bytes sent via SQL*Net to client
        555  bytes received via SQL*Net from client
          4  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
      10000  rows processed

Here are the doubts,
(1)I understood that the 690 consistent gets are coming with the calculation that there are 10000/15+24=690.That makes sense.But I am not able to get that when I am giving an update than there are some (26401 db block gets) and jst 150 consistent gets.Can you please explain how this is happening?One guess of mine is that that the segment headers are always gotten in the current mode so Oracle is updating the segment header all the time when it is doing an update on the table.But with that guess,some how it appears that there are about twice the rows updation that happened.Please explain thatwhy there are so high current gets and why there is just 150 consistent gets ?
2) What kind of tuning advice or practise can be followed by looking at the consistent and current gets?In your book also you have mentioned that these are the most important stats to look for in the autotrace but I amnot ableto make it out that what advice one can come over looking at these ?
Thanks and regards,

Followup January 29, 2008 - 6am Central time zone:

January 28, 2008 - 10pm Central time zone

Bookmark | Bottom | Top

Reviewer: Aman.... from India

Sir ,
you mentioned,
Your query never needs more than one block to be in the cache at ANY point in time.
I didnt understand that.What does this mean?
Regards
Aman....

Followup January 29, 2008 - 6am Central time zone:

sorry, I do not know how else to say it.

it means exactly what it says.

January 29, 2008 - 10pm Central time zone

Bookmark | Bottom | Top

Reviewer: Aman.... from India

1) the consistent gets are just to read the table in consistent read mode "as it exists", when we update a row - we have to get it in current mode and we do that a row at a time, and yes, there are other bits that need to be modified - undo to be created and modified - in addition to the table itself.
Hi sir,
Thanks alot for your reply.I guessI understand the point of consistent gets but I am still not clear with what has happened in the Update clause of mine.there are 10000 rows and there is an index over the column also so the number of db block gets are double+extra rows.That makes sense as the rows are getting modified so Oracle is updating the header one by one for each row modification.But what does 150 consistent gets describe?These rows were already in the cache as I didnt do any flush of buffer cache.What I was expecting that the same number of consistent gets that comes for a normal select will come or may be more than that too.But it dropped way down as compared to the normal select.Thats what I a,nto able to understand.Please explain this behaviour.
Thanks and regards.
Aman....

Followup January 30, 2008 - 9am Central time zone:

there is undo being read, there is undo being written, there are table blocks being read, table blocks being written, all kinds of activity going on. there are index blocks being read, there are index blocks being written.

reads = consistent gets
writes = current mode gets

lots of stuff going on there when you update, you are hitting more structures than you would with a simple select.

jdbc array size February 6, 2008 - 9am Central time zone

Bookmark | Bottom | Top

Reviewer: bakunian from OC, CA

Tom,

I'm dba supporting java app so could you tell me how to increase JDBC array size. Is it something 
that can be set at application, session, transaction or perhaps statement level?
And if you could include some code snippet it would great.

Thanks for your time

Followup February 6, 2008 - 9am Central time zone:

http://www.oracle.com/pls/db102/search?remark=advanced_search&word=setRowPrefetch&format=ranked&book
=&preference=

Optimize query which returns large resultsets March 24, 2008 - 6pm Central time zone

Bookmark | Bottom | Top

Reviewer: Kamal from NJ

Tom:
I have this query which returns 154721 records as output. When i tested in sqlplus, the query runs fast but it takes long time around 20 minutes to show the resultset completion on the screen. Is there any way to improve the query completion.

"25218094 bytes sent via SQL*Net to client"

cat qry.sql
===========
select distinct a.contract_enterprise_name, a.contract_customer_name, a.contract_number,
b.contract_service, a.contract_att_signature_date, a.contract_date_updated,
a.contract_update_driver, a.contract_site, b.contract_gams_chrg_to_acct_id,
a.contract_region, b.contract_currency_code, b.isocntrycode, b.contract_exchange_rate,
a.contract_customer_address, a.contract_customer_state, a.contract_customer_zip_code,
a.contract_customer_country, b.contract_contracted_country, c.contract_customer_site_name,
c.contract_site_address, c.contract_site_state, c.contract_site_zip_code, c.contract_site_country,
d.contract_be_id, d.contract_be_description, d.contract_be_category, d.contract_be_quantity,
d.contract_be_gross_rate, d.contract_be_net_rate, c.contract_site_fixed_rate, d.contract_be_discount
from GLOBAL_CUST_CONTRACTS a, GLOBAL_ISOCOUNTRY b, GLOBAL_CUST_SITE_ADDRESS c,
GLOBAL_CUST_COMP_CONTRACTS d
where a.contract_enterprise_name = 'ADVANCED MICRO DEVICES'
and a.contract_number = b.contract_number and b.contract_number = c.contract_number
and b.contract_service = c.contract_service and b.isocntrycode = c.contract_iso_country_code
and c.contract_number = d.contract_number and c.contract_iso_country_code = d.contract_iso_country_code
and c.contract_service = d.contract_service and c.contract_site_address_number = d.contract_site_address_number
/

23:11:30 SQL> @qry

154721 rows selected.

Elapsed: 00:00:37.77

Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
156224 consistent gets
0 physical reads
0 redo size
25218094 bytes sent via SQL*Net to client
851428 bytes received via SQL*Net from client
77362 SQL*Net roundtrips to/from client
1 sorts (memory)
0 sorts (disk)
154721 rows processed

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1389 Card=47419 Byte
s=13609253)

1 0 SORT (UNIQUE) (Cost=1389 Card=47419 Bytes=13609253)
2 1 TABLE ACCESS (BY INDEX ROWID) OF 'GLOBAL_CUST_COMP_CONTR
ACTS' (Cost=2 Card=261 Bytes=23229)

3 2 NESTED LOOPS (Cost=392 Card=47419 Bytes=13609253)
4 3 HASH JOIN (Cost=30 Card=181 Bytes=35838)
5 4 HASH JOIN (Cost=17 Card=181 Bytes=26788)
6 5 TABLE ACCESS (FULL) OF 'GLOBAL_CUST_CONTRACTS' (
Cost=3 Card=9 Bytes=981)

7 5 TABLE ACCESS (FULL) OF 'GLOBAL_ISOCOUNTRY' (Cost
=13 Card=25753 Bytes=1004367)

8 4 TABLE ACCESS (FULL) OF 'GLOBAL_CUST_SITE_ADDRESS'
(Cost=12 Card=25753 Bytes=1287650)

9 3 INDEX (RANGE SCAN) OF 'DC_GCCC_PK001' (UNIQUE) (Cost
=1 Card=13)

Table row counts are :
=======================
23:37:19 SQL> select count(*) from GLOBAL_CUST_COMP_CONTRACTS ;

COUNT(*)
----------
6733522

Elapsed: 00:00:11.77
23:37:41 SQL> select count(*) from GLOBAL_CUST_CONTRACTS ;

COUNT(*)
----------
1267

Elapsed: 00:00:00.08
23:37:51 SQL> select count(*) from GLOBAL_CUST_SITE_ADDRESS ;

COUNT(*)
----------
25757

Elapsed: 00:00:00.07
23:37:57 SQL> select count(*) from GLOBAL_ISOCOUNTRY ;

COUNT(*)
----------
25757

Followup March 24, 2008 - 7pm Central time zone:

154721 - that is a teeny tiny number. tiny.

"25218094 bytes sent via SQL*Net to client" is about 25mb, a small, trivial number. Unless you are on a bad/slow network of course.

you would want a tkprof to see how much time it spent in the database, what it waited on, versus how long it took your client to show you the last row, that would be very helpful.

tkprof report March 24, 2008 - 9pm Central time zone

Bookmark | Bottom | Top

Reviewer: Kamal from NJ

Tom:

I ran the same query from shell script in nohup and it completed in 2 minutes and 23 secs.

I am attaching the TKPROF report for the same query.
Can you suggest on improving query response time.

********************************************************************************

alter session SET EVENTS '10046 TRACE NAME CONTEXT FOREVER, LEVEL 12'

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 0 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 1 0.00 0.00 0 0 0 0

Misses in library cache during parse: 0
Misses in library cache during execute: 1
Optimizer goal: CHOOSE
Parsing user id: 205

Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 1 0.00 0.00
SQL*Net message from client 1 0.00 0.00
********************************************************************************

select distinct a.contract_enterprise_name,
a.contract_customer_name, a.contract_number,
b.contract_service, a.contract_att_signature_date,
a.contract_date_updated, a.contract_update_driver,
a.contract_site, b.contract_gams_chrg_to_acct_id,
a.contract_region, b.contract_currency_code,
b.isocntrycode, b.contract_exchange_rate,
a.contract_customer_address, a.contract_customer_state,
a.contract_customer_zip_code,
a.contract_customer_country,
b.contract_contracted_country,
c.contract_customer_site_name, c.contract_site_address,
c.contract_site_state, c.contract_site_zip_code,
c.contract_site_country, d.contract_be_id,
d.contract_be_description, d.contract_be_category,
d.contract_be_quantity, d.contract_be_gross_rate,
d.contract_be_net_rate, c.contract_site_fixed_rate,
d.contract_be_discount from GLOBAL_CUST_CONTRACTS a,
GLOBAL_ISOCOUNTRY b, GLOBAL_CUST_SITE_ADDRESS c,
GLOBAL_CUST_COMP_CONTRACTS d where
a.contract_enterprise_name = 'BOMBARDIER' and
a.contract_number = b.contract_number and
b.contract_number = c.contract_number and b.contract_service =
c.contract_service and b.isocntrycode = c.contract_iso_country_code and
c.contract_number= d.contract_number and
c.contract_iso_country_code =d.contract_iso_country_code and
c.contract_service =d.contract_service and
c.contract_site_address_number =d.contract_site_address_number

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.02 0.02 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 66764 9.60 10.19 0 134743 0 133525
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 66766 9.62 10.22 0 134743 0 133525

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 205

Rows Row Source Operation
------- ---------------------------------------------------
133525 SORT UNIQUE
133525 TABLE ACCESS BY INDEX ROWID GLOBAL_CUST_COMP_CONTRACTS
133976 NESTED LOOPS
450 HASH JOIN
450 HASH JOIN
25 TABLE ACCESS FULL GLOBAL_CUST_CONTRACTS
25757 TABLE ACCESS FULL GLOBAL_ISOCOUNTRY
25757 TABLE ACCESS FULL GLOBAL_CUST_SITE_ADDRESS
133525 INDEX RANGE SCAN DC_GCCC_PK001 (object id 1633499)

Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
SQL*Net message to client 66764 0.00 0.07
SQL*Net more data to client 1 0.00 0.00
SQL*Net message from client 66764 0.15 122.93
********************************************************************************

alter session SET EVENTS '10046 TRACE NAME CONTEXT OFF'
===========================================================

Indexes on the tables used in the query:
========================================
SQL> l
1 select table_name, index_name, column_name from user_ind_columns where table_name in
2 ('GLOBAL_CUST_CONTRACTS','GLOBAL_ISOCOUNTRY','GLOBAL_CUST_SITE_ADDRESS','GLOBAL_CUST_COMP_CONTRACTS')
3* order by table_name, index_name, column_position
SQL> /

TABLE_NAME INDEX_NAME COLUMN_NAME