Skip to Main Content

Breadcrumb

May 4th

Question and Answer

Tom Kyte

Thanks for the question, Andrew.

Asked: August 28, 2012 - 3:46 pm UTC

Last updated: April 22, 2013 - 7:06 pm UTC

Version: 10.2.0.5.0

Viewed 1000+ times

You Asked

Hello Tom!
I have a question about joining with collections and cardinality estimation.
For tables with index Oracle 10g suggests plan with full table scan with can be slow on large tables.
For example (p_var is a variable in a stored procedure):

/* test data */
create type n_arr as table of number;
create table t1(id number primary key, a1 number);
insert into t1 select level,level-1 from dual connect by level <= 1000000; commit;
begin dbms_stats.gather_table_stats(user,'t1',cascade => true); end;


/* Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 */
explain plan for
select * from t1 where t1.id in
 (select column_value from table(cast(:p_var as n_arr)) t2);



-----------------------------------------------------------------------------------------------------
| Id  | Operation                            | Name         | Rows  | Bytes | Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                     |              |   255 |  3060 |   541   (1)| 00:00:07 |
|   1 |  NESTED LOOPS                        |              |       |       |            |          |
|   2 |   NESTED LOOPS                       |              |   255 |  3060 |   541   (1)| 00:00:07 |
|   3 |    SORT UNIQUE                       |              |  8168 | 16336 |    29   (0)| 00:00:01 |
|   4 |     COLLECTION ITERATOR PICKLER FETCH|              |  8168 | 16336 |    29   (0)| 00:00:01 |
|*  5 |    INDEX UNIQUE SCAN                 | SYS_C0014171 |     1 |       |     1   (0)| 00:00:01 |
|   6 |   TABLE ACCESS BY INDEX ROWID        | T1           |     1 |    10 |     2   (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   5 - access("T1"."ID"=VALUE(KOKBF$))

/* Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 */

explain plan for
select * from t1 where t1.id in
 (select column_value from table(cast(:p_var as n_arr)) t2);

-------------------------------------------------------------------------------------------
| Id  | Operation                          | Name | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                   |      |     1 |    11 |   504   (3)| 00:00:07 |
|*  1 |  HASH JOIN RIGHT SEMI              |      |     1 |    11 |   504   (3)| 00:00:07 |
|   2 |   COLLECTION ITERATOR PICKLER FETCH|      |       |       |            |          |
|   3 |   TABLE ACCESS FULL                | T1   |   997K|  8765K|   476   (2)| 00:00:06 |
-------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   1 - access("T1"."ID"=VALUE(KOKBF$))



Now I use cardinality hint to set the cardinality for collections manually, but the number of elements in collection can be variable.
I tried to use dynamic sampling but in Oracle 10g it doeesn't work with collections.

/* Oracle Database 10g Enterprise Edition Release 10.2.0.5.0 */

explain plan for
select * from t1 where t1.id in
 (select /*+ cardinality(t2 10)*/ column_value from table(cast(:p_var as n_arr)) t2);

---------------------------------------------------------------------------------------------------
| Id  | Operation                           | Name        | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                    |             |    10 |   110 |    35   (3)| 00:00:01 |
|   1 |  NESTED LOOPS                       |             |    10 |   110 |    35   (3)| 00:00:01 |
|   2 |   SORT UNIQUE                       |             |       |       |            |          |
|   3 |    COLLECTION ITERATOR PICKLER FETCH|             |       |       |            |          |
|   4 |   TABLE ACCESS BY INDEX ROWID       | T1          |     1 |     9 |     2   (0)| 00:00:01 |
|*  5 |    INDEX UNIQUE SCAN                | SYS_C006586 |     1 |       |     1   (0)| 00:00:01 |
---------------------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 
   5 - access("T1"."ID"=VALUE(KOKBF$))



How oracle estimates cardinality for collections?
How can be optimized joining and filtering with collections in stored procedures?

and Tom said...

The estimate is based on the block size - you have an 8k block so about 8,000. If ou had a 4k block - about 4,000. and so on.


The cardinality hint is good for this - and the number does not have to be "exact", just representative.


If you expect most of the time to be around "10" - (anywhere from 1..100 or maybe even more) - just use 10. It does not have to be precise, it just has to be representative.

if the number of elements could vary widely and you want a different plan for say "small", "medium", and "large" collections - you can use three queries - each with a different hint (ref cursors in plsql, dynamic sql in other languages)


Rating

  (4 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Clarification

Muhammad Riaz Shahid, August 29, 2012 - 11:17 pm UTC

Tom,

Can we use ROWNUM to make this happen:

08:14:22 riaz@ORCL> ed
Wrote file afiedt.buf

  1  explain plan for
  2  select * from t1 where t1.id in
  3*  (select column_value from table(cast(:p_var as n_arr)) t2 where rownum>0)
08:14:52 riaz@ORCL> /

Explained.

08:14:52 riaz@ORCL> @explain1

PLAN_TABLE_OUTPUT
----------------------------------------------------------------------------------------------------
Plan hash value: 1699974380

----------------------------------------------------------------------------------------------------
| Id  | Operation                              | Name         | Rows  | Bytes | Cost (%CPU)| Time     |
----------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                       |              |     1 |    22 |    69   (3)| 00:00:01 |
|   1 |  NESTED LOOPS                          |              |     1 |    22 |    69   (3)| 00:00:01 |
|   2 |   VIEW                                 | VW_NSO_1     |  8168 |   103K|    24   (0)| 00:00:01 |
|   3 |    HASH UNIQUE                         |              |     1 | 16336 |            |          |
|   4 |     COUNT                              |              |       |       |            |          |
|*  5 |      FILTER                            |              |       |       |            |          |
|   6 |       COLLECTION ITERATOR PICKLER FETCH|              |       |       |            |          |
|   7 |   TABLE ACCESS BY INDEX ROWID          | T1           |     1 |     9 |     2   (0)| 00:00:01 |
|*  8 |    INDEX UNIQUE SCAN                   | SYS_C0060726 |     1 |       |     1   (0)| 00:00:01 |
----------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   5 - filter(ROWNUM>0)
   8 - access("T1"."ID"="$nso_col_1")

21 rows selected.

08:15:19 riaz@ORCL> SELECT * From v$version;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Prod
PL/SQL Release 10.2.0.1.0 - Production
CORE    10.2.0.1.0      Production
TNS for 32-bit Windows: Version 10.2.0.1.0 - Production
NLSRTL Version 10.2.0.1.0 - Production

Tom Kyte
September 10, 2012 - 5:57 pm UTC

your estimated cardinality was still 8168? what did you accomplish?

To Andrew

Raj, August 30, 2012 - 3:59 am UTC

Check this link.

http://www.oracle-developer.net/display.php?id=427

It explains the different method on how to do it.

Thanks

Raj
Tom Kyte
September 10, 2012 - 6:09 pm UTC

nice

Andrew, August 30, 2012 - 11:50 am UTC

Thanks

Debendra, April 15, 2013 - 11:51 am UTC

Hi Tom,

Instead of rownum > 0, if we set rownum <= array.count, it does helps to setup the cardinality estimates correctly.


Tom Kyte
April 22, 2013 - 7:06 pm UTC

with bind peeking - it could work - but it would be subject to bind peeking issues...

with bind peeking, it would derive the estimated the cardinality the first time - so it the collection "usually had about 500 items in it", but the first time you hard parsed - it had 1 in it, the plan would be developed for 1. If the next time you ran it - it had 500....


so, yes, bind peeking would let it default the cardinality the first time:



ops$tkyte%ORA11GR2> declare
  2          l_data sys.odciVarchar2List := sys.odciVarchar2List( 'a', 'b', 'c' );
  3          l_n number;
  4  begin
  5          l_n := l_data.count;
  6          for x in (select * from table(l_data) where rownum <= l_n )
  7          loop
  8                  null;
  9          end loop;
 10          for y in ( select plan_table_output from table(dbms_xplan.display_cursor))
 11          loop
 12                  dbms_output.put_line( y.plan_table_output );
 13          end loop;
 14  end;
 15  /
SQL_ID  752n8ycjxuxnb, child number 0
-------------------------------------
SELECT * FROM TABLE(:B1 ) WHERE ROWNUM <= :B2

Plan hash value: 1440075759

-------------------------------------------------------------------------------
| Id  | Operation                          | Name | Rows  | Bytes | Cost (%CPU)
-------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                   |      |       |       |    29 (100)
|*  1 |  COUNT STOPKEY                     |      |       |       |
|   2 |   COLLECTION ITERATOR PICKLER FETCH|      |     3 |     6 |    29   (0)
-------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(ROWNUM<=:B2)


PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR2> declare
  2          l_data sys.odciVarchar2List := sys.odciVarchar2List( 'a', 'b', 'c', 'd', 'e', 'f' );
  3          l_n number;
  4  begin
  5          l_n := l_data.count;
  6          for x in (select * from table(l_data) where rownum <= l_n )
  7          loop
  8                  null;
  9          end loop;
 10          for y in ( select plan_table_output from table(dbms_xplan.display_cursor))
 11          loop
 12                  dbms_output.put_line( y.plan_table_output );
 13          end loop;
 14  end;
 15  /
SQL_ID  752n8ycjxuxnb, child number 0
-------------------------------------
SELECT * FROM TABLE(:B1 ) WHERE ROWNUM <= :B2

Plan hash value: 1440075759

-------------------------------------------------------------------------------
| Id  | Operation                          | Name | Rows  | Bytes | Cost (%CPU)
-------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                   |      |       |       |    29 (100)
|*  1 |  COUNT STOPKEY                     |      |       |       |
|   2 |   COLLECTION ITERATOR PICKLER FETCH|      |     3 |     6 |    29   (0)
-------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(ROWNUM<=:B2)


PL/SQL procedure successfully completed.



but it would be "stuck" there. using a representative number (what I said above) or a small/med/large number as appropriate would result in less variability (someone hard parses with 1 one day and 1,000 another... different plans on different days)

More to Explore

PL/SQL demos

Check out more PL/SQL tutorials on our LiveSQL tool.

PL/SQL docs

PL/SQL reference manual from the Oracle documentation library