Skip to Main Content
  • Questions
  • Does CBO considers HWM when calculating cost for a full table scan ?

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Arun.

Asked: September 25, 2002 - 12:48 pm UTC

Last updated: August 12, 2004 - 10:35 am UTC

Version: 8.1.7.3

Viewed 1000+ times

You Asked

I have a theoretical question. Does the CBO consider all blocks upto highwater mark when calculating the cost of a full table scan ? A firm yes or no answer would be fine.
Thanks


and Tom said...

yes

it has to. all of the blocks up to the HWM are going to be in the full scan.

Consider:

ops$tkyte@ORA920.US.ORACLE.COM> /*
DOC>
DOC>drop table t;
DOC>
DOC>create table t pctfree 90 pctused 5
DOC>as
DOC>select * from all_objects;
DOC>
DOC>*/

ops$tkyte@ORA920.US.ORACLE.COM> analyze table t compute statistics;
Table analyzed.

ops$tkyte@ORA920.US.ORACLE.COM> set autotrace traceonly explain;

ops$tkyte@ORA920.US.ORACLE.COM> select * from t;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=384 Card=29272 Bytes=2488120)
1 0 TABLE ACCESS (FULL) OF 'T' (Cost=384 Card=29272 Bytes=2488120)


cost = 383

ops$tkyte@ORA920.US.ORACLE.COM> set autotrace off
ops$tkyte@ORA920.US.ORACLE.COM> delete from t;

29272 rows deleted.

ops$tkyte@ORA920.US.ORACLE.COM> commit;

Commit complete.

ops$tkyte@ORA920.US.ORACLE.COM> analyze table t compute statistics;

Table analyzed.

ops$tkyte@ORA920.US.ORACLE.COM> set autotrace traceonly explain;
ops$tkyte@ORA920.US.ORACLE.COM> select * from t;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=384 Card=1 Bytes=128)
1 0 TABLE ACCESS (FULL) OF 'T' (Cost=384 Card=1 Bytes=128)

card is downgraded but COST remains the same

ops$tkyte@ORA920.US.ORACLE.COM> truncate table t;

Table truncated.

ops$tkyte@ORA920.US.ORACLE.COM> analyze table t compute statistics;

Table analyzed.

ops$tkyte@ORA920.US.ORACLE.COM> set autotrace traceonly explain
ops$tkyte@ORA920.US.ORACLE.COM> select * from t;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1 Bytes=128)
1 0 TABLE ACCESS (FULL) OF 'T' (Cost=2 Card=1 Bytes=128)


same table, same number of rows -- but the truncate lowered the HWM so.... lower cost



Rating

  (9 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Follow up question - CARD?

JC, September 25, 2002 - 2:35 pm UTC

Excellent and clear answer.

If I could, what actually is the CARD term next to the COST?


Tom Kyte
September 25, 2002 - 3:35 pm UTC

CARD - cardinality -- estimated number of rows produced by that step in the plan (output from that step in the plan)

Arun Gupta, September 25, 2002 - 2:57 pm UTC

Thanks very much Tom. Your answer helped clear many doubts.

CBO

Ashwani Singh, September 26, 2002 - 1:48 am UTC

Great Tom!!!,

Ur Answers clear a lot of doubts.
One thing i wanted to ask was that in a Paramter file , when specifying optimizer_mode. What is the diffference between all_rows and last_row.

Regards,
Ashwani Singh

Tom Kyte
September 26, 2002 - 7:37 am UTC

Easy - all_rows will let you start your database.

Setting it to last_row will not, it is not a valid value.


So, assuming you meant "whats the difference between ALL_ROWS and FIRST_ROWS optimization"....

All_rows -- optimizes the query to get you the LAST row as fast as possible. Eg, if you issue "select x,y from big_table where x is not null order by x" and there is a single index on big_table on the column X -- all_rows optimization will most likely FULL SCAN big_table, get x,y, sort and give you the result. It'll be faster then millions of logical IOs a block at a time to access the table via the index.

First_rows -- optimizes the query to get you the FIRST row as fast as it can. Given the same scenario as above -- the optimizer would most likely choose the index access and give you the first row instantly (instead of waiting for the full scan+sort). Getting that last row will take longer then the all_rows plan but the result to the program of the first row is faster.


A stored procedure would benefit from all_rows since the client doesn't get to see anything until its done (you want the best throughput).

An interactive application that issues SQL might benefit from first_rows since the client is waiting for the first 10 rows to appear on their screen.

Similar topic

Dennis, September 26, 2002 - 8:18 am UTC

Tom,

This is related to explain plans - if this is an inappropriate spot then please say so and I'll wait in the queue.

In Tkprof, you get a rows processed count. What is this representing? I used to think it was the actual number of rows for that step, however just yesterday I experienced the number of rows returned from the query (one I was tuning) to be higher than this count. Here's visual info since that is always helpful:

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.07 0.33 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2153 320.83 1196.45 2341431 2343495 60 32275
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2155 320.90 1196.78 2341431 2343495 60 32275

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 119 (TESTTUNE)

Rows Row Source Operation
------- ---------------------------------------------------
32275 HASH JOIN
418 TABLE ACCESS FULL REF_ORGANIZATION
414816 TABLE ACCESS FULL EXT_XPAYD_PAY_DETAILS

Adding a bitmap index, I saw different row counts in tkprof, which scared me into thinking that somehow the query was altered, but I spooled the query without the index and with the index, retrieving 32595 rows each time.

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.01 0.01 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2174 30.42 260.61 129576 138357 15 32595
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 2176 30.43 260.62 129576 138357 15 32595

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 119 (TESTTUNE)

Rows Row Source Operation
------- ---------------------------------------------------
32595 HASH JOIN
418 TABLE ACCESS FULL REF_ORGANIZATION
421154 INLIST ITERATOR
421154 TABLE ACCESS BY INDEX ROWID EXT_XPAYD_PAY_DETAILS
421728 BITMAP CONVERSION TO ROWIDS
158 BITMAP INDEX SINGLE VALUE

The number of rows returned by the query is 32595:
--- original ---
32595 rows selected.

Elapsed: 00:17:1028.59
15:31:22 TESTTUNE@HIP2:sca01> spool off
---
--- added bitmap
32595 rows selected.

Elapsed: 00:03:203.53
15:07:13 TESTTUNE@HIP2:sca01> spool off
---

So...why did I get a different number of rows on the original explain plan? Any ideas??


Thanks,
Dennis



Tom Kyte
September 26, 2002 - 8:31 am UTC

I can only surmise that the first trace file was truncated and not all of the information was in there. According to that -- you made

2153 FETCH calls resulting in 32275 rows (15 rows/fetch -- looks like SQLplus's array size)

The second one made 2174 FETCH calls resulting in 32595 rows -- 15 rows/fetch again.


I would say the first trace file was truncated or otherwise incomplete and the end statistics (the other 21 fetches) were "missing"

What was your max_dump_file_size and was the first trace file hitting that limit?

A reader, September 26, 2002 - 10:20 am UTC

Referring to your answer to the original question:

After the delete, the autotrace shows the same Cost and a lower Cardinality. But why is the bytes lower? If it is doing a FTS then I would have expected the bytes remained same? So what is this bytes thing actually?

Tom Kyte
September 26, 2002 - 11:30 am UTC

bytes = card (estimated returned rows) * avg row length.

after delete, card = 1, bytes returned goes down.

cost of scan, stays the same, same number of IO's would be done to read

not sure

Dennis, September 26, 2002 - 10:23 am UTC

Well the second trace file is larger. Nobody should've modified the init parameters...I'm pretty sure I was the only one in there, and it normally takes a couple of meetings and a ripped off arm to get things like that changed (I still do not understand why they think EVERY database needs to have the same init parameters...), so I don't imagine anyone would've changed it, especially since I usually make those suggestions and I don't recall losing an arm. ;)

I have 10240 for the max_dump_file_size. That is in KB, right? The first trace file (the weird one) is 3.68 MB, the other is 5.00 MB. It does say at the bottom of the first file that it's 2xxx lines in the trace file, but the tkprof output shows only 100 or so (I didn't weed out anything either, unless it does so by default (used to have to weed out the sys, that change in 9iR2? Guess I need to look that up don't I...)).

At any rate, it doesn't really matter now that you've confirmed that I wasn't hallucinating or anything. Rows processed. Anything else means that something screwy happened to the file. Works for me.

Thanks Tom, you're the greatest! Sorry I didn't respond right away, I was in a meeting...but I did check first thing when I got back (well...after I checked out my tar). You give us such prompt service, it's only right that we do the same!
Dennis

A reader, September 26, 2002 - 12:42 pm UTC

OK. So bytes is the number of bytes returned by the query and not the number of bytes read in order to service the query. That was my confusion. Thanks.

HWM & CBO

denni50, August 12, 2004 - 8:39 am UTC

Tom

I've converted our development db from RBO to CBO and
performed dbms_stats on key tables and have plans to
convert Production db after testing(so far, so good)

One of the tables is fairly large 8+ mil records with
inserts and deletes taking place on a daily basis.
(older data being archived while new data is loaded).

Since CBO uses the HWM for Full Table Scans
then it is safe to assume the area where older records
were deleted and is now empty is still being scanned
by the CBO in computing the cost of execution.

To avoid this additional cost to the plan I'm surmising
I should export/import the table after large deletes
to bring the HWM to the correct level so that the CBO can
calculate a more accurate execution plan.

Is there another way to get the HWM corrected w/o having
to do exp/imp?

thanks








Tom Kyte
August 12, 2004 - 9:30 am UTC

well, if you are doing inserts AND deletes -- the "old space" is "new space".

Unless this table just grows and grows and grows forever -- the space is being reused.


so, is there really a problem? is this table "just growing" or is it at a pretty much steady state size.

re: HWM & CBO

denni50, August 12, 2004 - 9:57 am UTC

perhaps not on a daily basis, more of a weekly basis.

We can have 20,000+ inserts on any given day with 10,000
deletes on another day(which would raise the HWM another 10,000)...however there can be days when we "remove"
a substantial amount of data(10,000-20,000) for archiving.
I suppose in the grand scheme of things this amount of
data loading and removing is immaterial to the cost of
execution plans. We are not talking about millions of records.
Was wondering if the amount of fragmentation with the inserts/deletes poses any detrimental effects.
Just trying to ensure the optimization of the database and it's performance.

thanks!

Tom Kyte
August 12, 2004 - 10:35 am UTC

i would think the severe, massive overhead of re-orging the table would far far far outweigh any nominal benefit (if any benefit at all)




More to Explore

Performance

Get all the information about database performance in the Database Performance guide.