Skip to Main Content
  • Questions
  • basic decision on sql query performance

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Scott.

Asked: January 28, 2003 - 8:51 pm UTC

Last updated: July 19, 2007 - 11:16 am UTC

Version: 8.1.7

Viewed 10K+ times! This question is

You Asked

G'day Tom

To get to a really basic example of what I'm asking, consider this statement:
select ename from bonus
where sal = nvl(:v, sal)

Forgetting any reason why you'd want to do this, would it be preferable to have (for performance reasons)
a) two statements, one used if :v is null, the other if not null
b) build a ref cursor
c) leave as is
d) something else...

For a more definitive example, consider a customer table with an acc_nbr as primary key, which provides the following execution plan
select count(*) from customers
where acc_nbr = nvl(:v,acc_nbr)
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=137 Card=1 Bytes=7)
1 0 SORT (AGGREGATE)
2 1 CONCATENATION
3 2 FILTER
4 3 INDEX (FAST FULL SCAN) OF 'CUST_PK' (UNIQUE) (Cost
5 2 FILTER
6 5 INDEX (UNIQUE SCAN) OF 'CUST_PK' (UNIQUE) (Cost=2

In essence, I guess I'm also trying to understand exactly what the concatenation (and filter) in the explain plan means.

Thanks.

and Tom said...

It is an interesting example and shows how much smarter the CBO is than the RBO. Reason 2134 to use the CBO.

consider this example:

create table t ( acc_nbr number, data char(255), constraint t_pk primary key(acc_nbr) );

insert into t
select rownum, 'x' from all_objects;

analyze table t compute statistics
for table
for all indexes
for all indexed columns;



alter session set sql_trace=true;
exec :n := null
select count(*)
from t n_is_null_cbo
where acc_nbr = nvl(:n,acc_nbr)
/
select /*+ RULE */ count(*)
from t n_is_null_rbo
where acc_nbr = nvl(:n,acc_nbr)
/
exec :n := 55
select count(*)
from t n_is_55_cbo
where acc_nbr = nvl(:n,acc_nbr)
/
select /*+ RULE */ count(*)
from t n_is_55_rbo
where acc_nbr = nvl(:n,acc_nbr)
/

Now, tkprof shows us:

select count(*) from t n_is_null_cbo where acc_nbr = nvl(:n,acc_nbr)

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 0.05 0.04 0 45 4 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 0.05 0.04 0 45 4 1

Rows Execution Plan
------- ---------------------------------------------------
0 SELECT STATEMENT GOAL: CHOOSE
1 SORT (AGGREGATE)
24098 CONCATENATION
24098 FILTER
24098 INDEX GOAL: ANALYZED (FAST FULL SCAN) OF 'T_PK' (UNIQUE)
0 FILTER
0 INDEX GOAL: ANALYZED (UNIQUE SCAN) OF 'T_PK' (UNIQUE)
********************************************************************************
select /*+ RULE */ count(*) from t n_is_null_rbo where acc_nbr = nvl(:n,acc_nbr)

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 0.06 0.06 0 895 5 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 0.06 0.06 0 895 5 1

Misses in library cache during parse: 0
Optimizer goal: RULE
Parsing user id: 605 (OPS$TKYTE)

Rows Execution Plan
------- ---------------------------------------------------
0 SELECT STATEMENT GOAL: HINT: RULE
0 SORT (AGGREGATE)
0 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'T'
********************************************************************************
select count(*) from t n_is_55_cbo where acc_nbr = nvl(:n,acc_nbr)

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.01 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 0.00 0.00 0 2 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 0.01 0.00 0 2 0 1

Rows Execution Plan
------- ---------------------------------------------------
0 SELECT STATEMENT GOAL: CHOOSE
1 SORT (AGGREGATE)
1 CONCATENATION
0 FILTER
0 INDEX GOAL: ANALYZED (FAST FULL SCAN) OF 'T_PK' (UNIQUE)
1 FILTER
1 INDEX GOAL: ANALYZED (UNIQUE SCAN) OF 'T_PK' (UNIQUE)
********************************************************************************
select /*+ RULE */ count(*) from t n_is_55_rbo where acc_nbr = nvl(:n,acc_nbr)

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 0.05 0.05 0 895 5 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 4 0.05 0.05 0 895 5 1

Misses in library cache during parse: 0
Optimizer goal: RULE
Parsing user id: 605 (OPS$TKYTE)

Rows Execution Plan
------- ---------------------------------------------------
0 SELECT STATEMENT GOAL: HINT: RULE
0 SORT (AGGREGATE)
0 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'T'

see -- the CBO did our work for us -- it rewrote the query effectively as:


select sum(cnt)
from ( select count(*) cnt from t where acc_nbr = :n
union all
select count(*) from t where :n is null )


and the query execution engine -- upon detecting the nullness (or not) of :n would execute only one or the other query in the union all. It realized that if :n is NULL -- acc_nbr = :n is never true -- don't even bother with that -- just index fast full scan and count. If :n is not null -- then the first query would run and the second would never go.....

The RBO on the other hand, it doesn't even realize that it can use the index at all for this query (or count(*)'s in general).



So, as long as you see a plan like that -- a single query is all you need. the optimizer has already in effect rewritten it as two queries for you.

Rating

  (13 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Mirjana, January 30, 2003 - 2:30 am UTC


what is cost based about it

Mikito Harakiri, January 30, 2003 - 5:52 pm UTC

select --+no_expand
* from hz_parties p
where party_id = nvl(:1, party_id)

cost 31

select --+use_concat
* from hz_parties p
where party_id = nvl(:1, party_id)

cost 33

The decision to use concatenated plan is not cost-based (even though it made by CBO, while RBO apparently doesn't implement this heuristic).

Tom Kyte
January 30, 2003 - 6:14 pm UTC

what is your point?

can you get the RBO to use this optimization (no)
does the CBO use this plan of attack naturally (yes)


I don't get what you are "saying" here. If you do not use CBO, no special optimization that obviates the need for two cursors. If you use the CBO -- special plan is generated.

That is what is "cost based about it". use the CBO and you can get it. don't use the CBO and you won't.

hz_parties is a wicked VIEW -- there is alot more then meets the eye there. If you look under the hood -- you might discover that the CBO is still doing the right thing.

correction

Mikito Harakiri, January 30, 2003 - 8:16 pm UTC

Let it be

alter session set OPTIMIZER_DYNAMIC_SAMPLING = 1;
select --+use_concat
--no_expand
* from obj$
where obj# = nvl(:1, obj# )

then. Concatenated plan costs 19 while full table scan is 17. Since execution doesn't really follow both branches (as I was happy to learn from your explanantion, btw), optimizer should cost concatenation somewhere between 2 (table access by index) and 17 (full table scan), so that concatenated plan would be indeed cheaper than FTS. I suggest filing a bug.

There are many similar cases of cost "ariphmetics" that make me uncomfortable. Although, RBO was rudimentary, it was much easier to undestand.

Tom Kyte
January 31, 2003 - 7:26 am UTC

I'm still not getting your point Mikito

Is it that you don't understand or fear the CBO

Or is it that you don't "what is cost based about it"

you set dynamic sampling to 1, you must have had an unindexed, unanalyzed table in your query. we sampled it. That changed the cost and the plan -- AS IT IS SUPPOSED TO (else -- why bother sampling eh?)

Sure, RBO -- easy to understand, 13 simple rules to live by. It was simplier before electricity, indoor plumbing, air conditioning as well -- I wouldn't go back though.

I suggest if you think it is a bug -- you file it? When you use hints, all bets are off -- you don't *know* how they work internally. You don't know what they do to play with the costs (hey -- we are back to that "you cannot compare costs across queries", "you cannot derive any sort of relationship between COST and RUNTIME" discussion aren't we??)




How much can CBO handle?

Jerry, January 30, 2003 - 10:48 pm UTC

I think the CBO has some issues in handling extreme complex queries. I had a query with several view joining together, some view with complicated table joins and computations. There are also union all in the query. In total, about ten tables are used. Sometimes, a slight change in one of the underlying table data could cause the execution plan to change from hash joins to nested loops. The query result changed from a couple of seconds to hours, sometime even hanging. I could not use hint or outln to stabilize the execution plan. The query is used in a software product where data could be very different from user to user. I am not sure one execution plan will work well in all environments. (The queries were all run immediately after stats were refreshed)

I looked at the execution plans. Oracle gave a high cost to the faster plan.

Based on your experience, how much can CBO handle? Should queries be restricted to less than five or six table joins? Should I use PL/SQL instead?


Tom Kyte
January 31, 2003 - 7:39 am UTC

In my experience

o if you have correct statistics
o if you have checked your optimizer_index_* parameters


thats it, CBO hands down -- for the most complex (in fact, the more complex -- the better). See

</code> http://groups.google.com/groups?selm=a1g2kv02r69%40drn.newsguy.com&oe=UTF-8&output=gplain <code>

for my classic example of that. Some of the tables referenced in here have 16 million + rows. There are outer joins, exists, unions, subqueries out the whazoo, calls to custom PLSQL functions for timezones and other things, queries on queries on queries and probably more than 16 tables (never counted them).

it was written for 815 (thats why the custom TZ handling)

Helena Markova, January 31, 2003 - 8:23 am UTC


But what's going on here?

Basil, May 12, 2003 - 6:13 pm UTC

In the following scenario, why isn't the CBO realizing that object_name = NVL(:p_name, object_name) is really the dominant predicate in this query? For 64 rows out of 766,000, the scan seems out of line:

SQL> create table t as select object_name, object_type from all_objects;

Table created.

SQL> insert into t select * from t;

23949 rows created.

SQL> insert into t select * from t;

47898 rows created.

SQL> insert into t select * from t;

95796 rows created.

SQL> insert into t select * from t;

191592 rows created.

SQL> insert into t select * from t;

383184 rows created.

SQL> commit;

Commit complete.

SQL> create index t_name_idx on t(object_name);

Index created.

SQL> create index t_type_idx on t(object_type)
  2  ;

Index created.

SQL> analyze index t_name_idx  compute statistics;

Index analyzed.

SQL> analyze index t_type_idx compute statistics;

Index analyzed.

SQL>
SQL> set autotrace traceonly explain statistics
SQL> variable p_name varchar2(50);
SQL> exec :p_name := 'ALL_IND_COLUMNS';

PL/SQL procedure successfully completed.

SQL> variable p_type varchar2(50);
SQL> exec :p_type := NULL;

PL/SQL procedure successfully completed.

SQL> select object_name, object_type from t
  2  where object_name like NVL(:p_name, object_name)
  3  and object_type like NVL(:p_type, object_type);

64 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (FULL) OF 'T'




Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
       4171  consistent gets
          0  physical reads
          0  redo size
       1723  bytes sent via SQL*Net to client
        547  bytes received via SQL*Net from client
          6  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         64  rows processed

 

Tom Kyte
May 12, 2003 - 7:11 pm UTC

You'd have to ask the RBO why -- you are not using the CBO.

*analyze the table*



Another doh

Basil, May 13, 2003 - 3:46 am UTC

My production system DOES analyze the table prior to manipulatng it. But here's the results from this test:

SQL> exec dbms_stats.gather_table_stats(ownname=>USER,tabname=>'t',method_opt=>'
for all indexed columns size auto', cascade=>true);

PL/SQL procedure successfully completed.

SQL> set autotrace traceonly explain statistics
SQL> variable p_name varchar2(50);
SQL> exec :p_name := 'ALL_IND_COLUMNS';

PL/SQL procedure successfully completed.

SQL> variable p_type varchar2(50);
SQL>  exec :p_type := NULL;

PL/SQL procedure successfully completed.

SQL> select object_name, object_type from t
  2  where object_name like NVL(:p_name, object_name)
  3  and object_type like NVL(:p_type, object_type);

64 rows selected.


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=509 Card=3832 Bytes=
          137952)

   1    0   CONCATENATION
   2    1     FILTER
   3    2       TABLE ACCESS (FULL) OF 'T' (Cost=254 Card=1916 Bytes=6
          8976)

   4    1     FILTER
   5    4       TABLE ACCESS (BY INDEX ROWID) OF 'T' (Cost=254 Card=19
          16 Bytes=68976)

   6    5         INDEX (RANGE SCAN) OF 'T_TYPE_IDX' (NON-UNIQUE) (Cos
          t=104 Card=36495)





Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        162  consistent gets
          1  physical reads
          0  redo size
       1723  bytes sent via SQL*Net to client
        547  bytes received via SQL*Net from client
          6  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
         64  rows processed

SQL>

 

Tom Kyte
May 13, 2003 - 7:20 am UTC

yes, so, did you read the example above?

When the bind variable is NULL -- Oracle will (has to) do the full table scan portion of the query.

When the bind variable IS NOT NULL -- Oracle will (can) do the index range scan.


Your bind variable is null, I would expect it to read the entire table.


Use TKPROF as I did.
Run the query with NULL and with a value.
You'll see the results as I have them above.

Surprised

Basil, May 21, 2003 - 1:32 am UTC

I'm only surprised that Oracle is still doing the full scan, when one of the predicates (the one with the nonnull input) serves as the driving limiter on the query.

If I'm just being dense, I'll take my lumps.

Tom Kyte
May 21, 2003 - 8:16 am UTC




when the input is NON-NULL, Oracle does not do the full scan.

when the input is NULL, Oracle does the full scan.


where is the problem with that? Please be more clear.



Two values at work

Basil, May 21, 2003 - 12:13 pm UTC

There are two tests in the predicate, one against object_name/p_name and one against object_type/p_type.

The predicate is ANDing the two tests:
WHERE object_name = NVL(:p_name, object_name)
AND object_type = NVL(:p_type,object_type)

In the case I gave, p_type is NULL, p_name is NOT.

In an AND situation like this one, if the query can reduce to an index scan for one of these (which it CAN for the object_name/p_name test), that serves as a limiting condition for the query. There will NEVER be rows found by table scanning to test object_type/p_type that are NOT returned by the index test for object_name/p_name. Put another way, the rows from the index test for object_name/p_name will always be a superset (or an exact set) of all the rows the query could return.

Statistics on the table should reflect that a mere 64 out of 760,000 rows fit the object_name/p_name restriction.

It seems to me that the total of (logical reads for object_name/p_name + logical reads to test object_type/p_type on the rows from the index scan) is quite a bit less than (full table scan logical reads) for a table with those sorts of statistics.

Again, I'm just trying to understand what's happening here. I apologize if I'm missing something completely obvious.

Tom Kyte
May 21, 2003 - 2:56 pm UTC

Oh i see -- the forest was hidden for the trees.

I guess it just isn't "smart enough" to come up with all 4 plans

p_name null and p_type null
p_name not null and p_type null
p_name null and p_type not null
p_name not null and p_type not null


you see, it quickly explodes as you add predicates. it works for the single predicate.

???

A reader, May 22, 2003 - 5:03 pm UTC

sorry, tom, but I didn't get your answer at all ...

Tom Kyte
May 23, 2003 - 8:36 am UTC

with 1 column -- the optimizer will develop "both" plans in a single plan as demonstrated above.


with 2 columns -- it does NOT develop the 4 necessary plans (with 3 the 9 and so on).

the plan is a static object that is built at hard parse time, it is invariant from then on in -- regardless of the bind variable values at run time. The query plan would literally "explode" in size if it did this for 2, 3, 4, ... n columns.



Just for the record

Basil, May 23, 2003 - 2:31 pm UTC

...what I was interested in trying to do is something like:

PROCEDURE search_proc( p_name varchar2, p_type varchar2, p_search3 varchar2, etc...)
RETURN REF CURSOR
IS
mycur REF CURSOR;
BEGIN
OPEN mycur FOR
select object_name, object_type from t
where object_name like NVL(p_name, object_name)
and object_type like NVL(p_type, object_type)
and something_else like NVL(p_search3, something_else)
...;
END search_proc;
/

The idea being a general search procedure for a web front-end that will specify 0 or more search conditions. (A NULL indicating that the condition doesn't matter.) The problem I get is that the query plan is too ugly. It's always table scanning, even when the limiting condition on the query (the non-null conditions) could make use of an index (it's a big table, WITH statistics).

Right now I'm planning to switch to a dynamically-constructed query (that includes only the non-null conditions) in conjunction with dbms_session.set_context() and sys_context(). Kind of a pain, but it performs significantly better in my tests.

multi predicates case

James Su, July 18, 2007 - 12:01 pm UTC

Dear Tom,
You said: "it works for the single predicate".

If there are more than one predicate, is it a good solution to use dynamic SQL?

Thank you.

cbo decision

Car Elcaro, July 19, 2007 - 9:26 am UTC

Tom,
Are there any chances that Oracle will used two indexes available on a table to access the record, where both indexes are specified on where clause?

For example, table t has column a, b, c, and d; Indexes created for that table are t_idx1(a,b) and t_idx2(c).

For query below, are there any chances Oracle used those indexes together (in what circumtances)

select * from t where a = <...> and b = <...> and c = <...>

Tom Kyte
July 19, 2007 - 11:16 am UTC

bitmap indexes - sure, we AND and OR and whatever them together all of the time.

ops$tkyte%ORA10GR2> create table t ( a int, b int, c int, d int );

Table created.

ops$tkyte%ORA10GR2> create bitmap index t_idx1 on t(a,b);

Index created.

ops$tkyte%ORA10GR2> create bitmap index t_idx2 on t(c);

Index created.

ops$tkyte%ORA10GR2> exec dbms_stats.set_table_stats( user, 'T', numrows => 1000000, numblks => 100000 );

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> @at
ops$tkyte%ORA10GR2> column PLAN_TABLE_OUTPUT format a72 truncate
ops$tkyte%ORA10GR2> set autotrace traceonly explain
ops$tkyte%ORA10GR2> select * from t where a = 5 and b = 6 and c = 7;

Execution Plan
----------------------------------------------------------
Plan hash value: 2685081498

------------------------------------------------------------------------
| Id  | Operation                    | Name   | Rows  | Bytes | Cost (%C
------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |        |     1 |    52 |     2
|   1 |  TABLE ACCESS BY INDEX ROWID | T      |     1 |    52 |     2
|   2 |   BITMAP CONVERSION TO ROWIDS|        |       |       |
|   3 |    BITMAP AND                |        |       |       |
|*  4 |     BITMAP INDEX SINGLE VALUE| T_IDX1 |       |       |
|*  5 |     BITMAP INDEX SINGLE VALUE| T_IDX2 |       |       |
------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   4 - access("A"=5 AND "B"=6)
   5 - access("C"=7)

ops$tkyte%ORA10GR2> set autotrace off



with "normal indexes", in this case - no. But, if D were added to one of the indexes, we can do what is known as an index join:

ops$tkyte%ORA10GR2> drop index t_idx1;

Index dropped.

ops$tkyte%ORA10GR2> drop index t_idx2;

Index dropped.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> create index t_idx1 on t(a,b);

Index created.

ops$tkyte%ORA10GR2> create index t_idx2 on t(c,d);

Index created.

ops$tkyte%ORA10GR2> exec dbms_stats.set_index_stats( user, 'T_IDX1', numrows => 1000000, NUMDIST => 10 );

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> exec dbms_stats.set_index_stats( user, 'T_IDX2', numrows => 1000000, NUMDIST => 10 );

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> @at
ops$tkyte%ORA10GR2> column PLAN_TABLE_OUTPUT format a72 truncate
ops$tkyte%ORA10GR2> set autotrace traceonly explain
ops$tkyte%ORA10GR2> select * from t where a = 5 and b = 6 and c = 7;

Execution Plan
----------------------------------------------------------
Plan hash value: 2547943377

------------------------------------------------------------------------
| Id  | Operation          | Name             | Rows  | Bytes | Cost (%C
------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |                  |     1 |    52 |     5  (
|*  1 |  VIEW              | index$_join$_001 |     1 |    52 |     5  (
|*  2 |   HASH JOIN        |                  |       |       |
|*  3 |    INDEX RANGE SCAN| T_IDX2           |     1 |    52 |     2 (1
|*  4 |    INDEX RANGE SCAN| T_IDX1           |     1 |    52 |     4  (
------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("A"=5 AND "B"=6 AND "C"=7)
   2 - access(ROWID=ROWID)
   3 - access("C"=7)
   4 - access("A"=5 AND "B"=6)

ops$tkyte%ORA10GR2> set autotrace off

More to Explore

Performance

Get all the information about database performance in the Database Performance guide.