Skip to Main Content

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Reader.

Asked: September 02, 2004 - 10:23 am UTC

Last updated: September 10, 2019 - 11:54 am UTC

Version: 9.2.0

Viewed 10K+ times! This question is

You Asked

SQL> connect demo/demo
Connected.
SQL> create table emp as select * from scott.emp;

Table created.

Elapsed: 00:00:00.08
SQL> insert into emp (select * from emp);

14 rows created.

Elapsed: 00:00:00.00
SQL> /

28 rows created.

Elapsed: 00:00:00.00
SQL> /

56 rows created.

Elapsed: 00:00:00.00
SQL> /

112 rows created.

Elapsed: 00:00:00.00
SQL> /

224 rows created.

Elapsed: 00:00:00.00
SQL> commit;

Commit complete.

Elapsed: 00:00:00.00
SQL> select count(*) from emp;

COUNT(*)
----------
448

Elapsed: 00:00:00.00
SQL> create bitmap index idx_emp_deptno on emp(deptno) nologging;

Index created.

Elapsed: 00:00:00.00
SQL> analyze table emp compute statistics for all indexed columns size 10;

Table analyzed.

Elapsed: 00:00:00.01
SQL> set autotrace expl stat
Usage: SET AUTOT[RACE] {OFF | ON | TRACE[ONLY]} [EXP[LAIN]] [STAT[ISTICS]]
SQL> set autotrace trace expl stat
SQL> select ename, sal from emp where deptno=10;

96 rows selected.

Elapsed: 00:00:00.00

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE
1 0 TABLE ACCESS (FULL) OF 'EMP'




Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 consistent gets
0 physical reads
0 redo size
0 bytes sent via SQL*Net to client
0 bytes received via SQL*Net from client
0 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
96 rows processed

SQL> drop index idx_emp_deptno ;

Index dropped.

Elapsed: 00:00:01.00
SQL> create index idx_emp_deptno on emp(deptno);

Index created.

Elapsed: 00:00:00.00
SQL> analyze table emp compute statistics for all indexed columns size 10;

Table analyzed.

Elapsed: 00:00:00.00
SQL> select ename, sal from emp where deptno=10;

96 rows selected.

Elapsed: 00:00:00.00

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMP'
2 1 INDEX (RANGE SCAN) OF 'IDX_EMP_DEPTNO' (NON-UNIQUE)




Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 consistent gets
0 physical reads
0 redo size
0 bytes sent via SQL*Net to client
0 bytes received via SQL*Net from client
0 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
96 rows processed


Why is bitmap index not being used? But when I change it to B*Tree index, it is being used. Can you please explain ?

and Tom said...

never ever in a billion years do anything remotely similar to this as SYS.

Just don't do it!!!!

Really!

don't! SYS is special, sys is magical, sys is not an account you should ever log in with and use like this.

you tried to trick us with the connect demo/demo, but you didn't do that did you :)


don't use nologging without the expressed written consent and guidance of your dba either! bad habits can be hard to break so don't get used to doing it.




you are using the RULE BASED OPTIMIZER (autotrace proves that -- no cost, no card, NO CBO).

You computed histograms, but you never analyzed the table itself.


So, that bitmap index would never be used. AND further, the RBO using the index was "a bad idea(tm)". think about the distribution of data here -- indexes would be *evil*, horrible, bad. You've got all of 3 values! you would be getting a vast majority of the rows regardless of WHAT deptno you pick -- think about it!

so, picking up your example from the midpoint:




ops$tkyte@ORA9IR2> analyze table emp compute statistics for all indexed columns size 10;
Table analyzed.

ops$tkyte@ORA9IR2> set autotrace traceonly explain
ops$tkyte@ORA9IR2> select ename, sal from emp where deptno = 10;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE
1 0 TABLE ACCESS (FULL) OF 'EMP'


So, no cost=, card= means no CBO.

ops$tkyte@ORA9IR2> analyze table emp compute statistics FOR TABLE;

Table analyzed.

ops$tkyte@ORA9IR2> select ename, sal from emp where deptno = 10;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=96 Bytes=2112)
1 0 TABLE ACCESS (FULL) OF 'EMP' (Cost=2 Card=96 Bytes=2112)


now we have the cost/card= and we are using the CBO and but says "no index", which is totally the best plan, when we force the index:

ops$tkyte@ORA9IR2> select /*+ index( emp idx_emp_deptno ) */ ename, sal from emp where deptno = 10;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=11 Card=96 Bytes=2112)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=11 Card=96 Bytes=2112)
2 1 BITMAP CONVERSION (TO ROWIDS)
3 2 BITMAP INDEX (SINGLE VALUE) OF 'IDX_EMP_DEPTNO'

the cost of using the index is more than a full scan OF SUCH A TEENY TINY TABLE

ops$tkyte@ORA9IR2> set autotrace off
ops$tkyte@ORA9IR2> update emp set deptno = 11 where rownum < 5;

4 rows updated.

ops$tkyte@ORA9IR2> analyze table emp compute statistics for table for all indexes for all indexed columns size 254;

Table analyzed.

ops$tkyte@ORA9IR2> set autotrace traceonly explain
ops$tkyte@ORA9IR2> select ename, sal from emp where deptno = 11;

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=4 Bytes=88)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=2 Card=4 Bytes=88)
2 1 BITMAP CONVERSION (TO ROWIDS)
3 2 BITMAP INDEX (SINGLE VALUE) OF 'IDX_EMP_DEPTNO'

but even thought the table is tiny, if the index is to retrieve very few rows, it becomes the cheaper alternative

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> set autotrace off



So, get out of the mindset that "indexes should be used at all costs"....

And don't use sys. only sys could/would show up with 0 consistent gets on a query like that.

Rating

  (30 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Let me guess...

Reader, September 02, 2004 - 12:47 pm UTC

.. how did you get around knowing that he was loggin as SYS ? the statistics, perhaps ?

Tom Kyte
September 02, 2004 - 1:46 pm UTC

see my last sentence above......



RE: Let guess...

Mark A. Williams, September 02, 2004 - 1:35 pm UTC

Tom tells you right in his response:

"And don't use sys. only sys could/would show up with 0 consistent gets on a query like that."

No guessing involved.

=======================================
Mark A. Williams
Oracle DBA
Author, Professional .NET Oracle Programming
</code> http://www.apress.com/book/bookDisplay.html?bID=378 <code>

Consistent gets

Curious, September 02, 2004 - 2:32 pm UTC

you said
'only sys could/would show up with 0 consistent gets on a
query like that. '

do you mind telling why would consistent gets show up 0 for sys.

Thanks.

Tom Kyte
September 02, 2004 - 3:04 pm UTC

....SYS is special, sys is magical,.....

actually, it was an issue finding the session id that was fixed in 9205 so I won't be able to use it to "bust" people anymore but the same saying applies:

do not use sys

Still confused

curious, September 02, 2004 - 3:13 pm UTC

without taking too much of your time, I still do not follow how 0 consistent gets is linked to SYS id.

I am just trying to understand this as a DBA, since, we still have some 'database playgrounds' running 8i for various resons.

By the way, it was funny how replied to the original post.

thanks.

Tom Kyte
September 02, 2004 - 4:19 pm UTC

it was an autotrace "product issue"

it is now fixed.

autotrace statistics were always 0 for "sys"

Bitmap index not used

reader, September 06, 2004 - 9:49 am UTC

Tom,

I am very sure I connected as demo and not as SYS for the above example. And you are right, the table was not analyzed. Hence there was no COST shown. But I am very sure, I connected as demo. And I cut and paste the whole thing.

Tom Kyte
September 06, 2004 - 10:37 am UTC

well, then something is wrong with your database.

the statistics would only show ZERO like that for sys or "as sysdba" accounts.  I've never ever seen it any other way.

does it reproduce for you?


sys@ORA9IR1> set autotrace on stat
sys@ORA9IR1> select * from dual;
 
D
-
X
 
 
Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          0  consistent gets
          0  physical reads
          0  redo size
          0  bytes sent via SQL*Net to client
          0  bytes received via SQL*Net from client
          0  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed
 
scott@ORA9IR1> @connect /
ops$tkyte@ORA9IR1> set autotrace on stat
ops$tkyte@ORA9IR1> select * from dual;
 
D
-
X
 
 
Statistics
----------------------------------------------------------
          0  recursive calls
          2  db block gets
          1  consistent gets
          0  physical reads
          0  redo size
        379  bytes sent via SQL*Net to client
        503  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed
 

Bitmap index not used

reader, September 06, 2004 - 9:53 am UTC

Tom, sorry, I wanted to ask you one more question about your example.

You did,

ops$tkyte@ORA9IR2> analyze table emp compute statistics for table for all
indexes for all indexed columns size 254;

Why 254? If it was some number like 100 or 150, I would have ingored it as "trial and experimental" values. But why particularly 254 for the size?

Again, please clarify the user demo/sys issue as well. I am very sure I connected as demo for that example.

Thanks very much

Tom Kyte
September 06, 2004 - 10:38 am UTC

254 is the biggest you can get. If you have histograms, might as well have space for them all. 254 is probably the only "size" you need to use.

If you have only 5 values, it'll do 5 exactly.

If you have 500, might as well have 254 vs 100.





Bitmap index not used

Reader, September 06, 2004 - 10:47 am UTC

Thanks very much Tom for your quick response. Again, I tried to reproduce it as demo but am not able to. But I am sure I didn't trick you. Anyway, thanks about the SYS' information. Very much appreciated.

One more question regarding index. Say, a static (like a lookup)table has a column called SEX with 2 values "m" and "f". Before populating the table, I am sorting them by sex. Assume that it is a 5 million records-table. Is it good to do a b-tree indexing or bit-map indexing? Will your choice differ if the data are not sorted but skewed through out the table?

Thanks in advance

Tom Kyte
September 06, 2004 - 11:00 am UTC

my answer would depend on your questions -- the questions you ask of this data.

That you have a two valued column
With 5 millions rows


is "insufficient data" to propose any sort of solution (since there isn't a stated problem as yet!)

if you said:

and we frequently count the number of M's or F's.... might say "bitmap"

and we frequently "select * from t where sex = 'M'"... probably say "no index", might not though, would need more info as to exactly why you do that...

and we found that 1% of the data is sex='F' and we say "where sex = 'F'"... probably say "index"

and we frequently query on all of the attributes, in unknown fashions. That is, there are 100 attributes and the user clicks and picks upto 15 of them and we aggregate the results. I'd probably be looking at a bitmap index (single column indexes) on most all columns so a query such as:

where ( c1 = ? and c2 in ( ?, ?) ) or ( c4 = ? and c6 = ? ) )


could just take the bitmaps on c1, c2 -- and them, on c4/c6 -- and them and then or the results.






analyze.. all indexes vs all indexed columns

Reader.., September 06, 2004 - 11:10 am UTC

Tom, I was reading the other day the documentation about the ANALYZE command, but got confused in either, the results I got and what I read about the two parameters: FOR ALL INDEXES and FOR ALL INDEXED COLUMNS. What is the difference between either ? And If you have test case that I can see would be even better.

P.S.: Hope this question is not out of context.

Tom Kyte
September 06, 2004 - 11:59 am UTC

one analyzes indexes -- computes leaf rows, branch rows, index height, clustering factors and such.

the other analyzes the columns in the table and computes histograms for them -- it only does columns which happen to have indexes on them, ignoring the others.

you want to read:
</code> http://docs.oracle.com/docs/cd/B10501_01/server.920/a96533/stats.htm#25018 <code>



Bitmap index not used

reader, September 06, 2004 - 11:23 am UTC

Tom, once again, than you for your quick response. I quoted the above scenario to get an idea about how bitmap vs b-tree is useful on such situations. In one of our customers' site, they have a datawarehouse. For quick illustration, I cite you a simple situation. They had 5 dimension tables and one fact table. Their Location dimension has a bitmap composite index on region+territory+branch. This combination doesn't make it unique but is frequently queried in this fashion from their users.

I vaguely remember the query....

select a.customerid, a.customer_name, a.product, sum(a.quantity) from dim_customer a, fact_sales b, dim_location c, dim_time d, dim_salesman e
a.customer_id = b.customer_id and
b.regionid = c.region and
b.territory=c.territory and
b.branch=c.branch ...
.......
.......

So the DBA there wanted to index this combination as a bitmap index and he says that it would make the queries perform in lightening speed. But unfortunately, it takes 1 hour and 22 minutes to come back with 30 rows.

Could you kindly throw some light on it as I personally don't understand his logic. But the whole management and my boss want to go with his logic and explanation.


Tom Kyte
September 06, 2004 - 11:55 am UTC

without further predicates, any indexes at all would just be "getting in the way".

but that aside. a dimension table that is not joined to by a primary key?!?

Bitmap index not used

reader, September 06, 2004 - 12:02 pm UTC

Tom, I am not sure I understood your first comments

"without further predicates, any indexes at all would just be "getting in the
way".

For the second comment, yes, strangely, there is no primary key on that dimension.

Tom Kyte
September 06, 2004 - 12:43 pm UTC

with just:

select a.customerid, a.customer_name, a.product, sum(a.quantity) from
dim_customer a, fact_sales b, dim_location c, dim_time d, dim_salesman e
a.customer_id = b.customer_id and
b.regionid = c.region and
b.territory=c.territory and
b.branch=c.branch ...
.......
.......

I could not see any useful indexes.

If they had "and c.region = ? and c.territory = ? and c.branch = ?", sure, index away but that just looks so far like "full scan + full scan + hash join"

Bitmap index not used

Reader, September 06, 2004 - 12:52 pm UTC

Thanks Tom. I am not able to remember the exact query. I will post it here when I come across it.

Thank you very much for your support.

A reader, November 26, 2004 - 12:09 pm UTC

Tom,

Someone asked this in this thread:

<quote>

One more question regarding index. Say, a static (like a lookup)table has a
column called SEX with 2 values "m" and "f". Before populating the table, I am
sorting them by sex. Assume that it is a 5 million records-table. Is it good to
do a b-tree indexing or bit-map indexing? Will your choice differ if the data
are not sorted but skewed through out the table?

</quote>

I am now convinced bitmap index shouldn't be used on a table with lot of insert activity. I have a table with a similar field as above. Values "N" or "F" and it has inserts done constantly. Is it a candidate for bitmap index?

I wouldn't personally go for bitmap index, but when it has just two values always, what should I do before I decide not to go for bitmap index?




Tom Kyte
November 26, 2004 - 12:41 pm UTC

why do you want to index this column in the first place -- I need to know that.

I can say however "if you have lots of single row inserts -- a bitmap index would be the kiss of death". for concurrency reasons alone.

Bitmap index not being used

Yogesh, February 14, 2005 - 1:32 pm UTC

I have a customer details table with 24633307 rows. It has a column title, which stores values like Mr, Mrs and so on ... following is the distribution of the rows based on these titles.

MR 11837267
MRS 6913007
MISS 4280373
MS 1221240
COMPAN 156152
DR 140541
MASTER 27309
REV 9011
PROF 4713
SISTER 3163
SIR 1992
LADY 1615
LORD 913
CAPT 811
STAFF 554
FR 518
INF 474
MADAM 320
MAJOR 279
COL 267
SGT 73
CMDR 36
MSTR 6
MRS 4
GROUP 3
MIS 3
MR & MR 3
REV. 3
MR 3
MR. 2
OTHER 32652

When I use

SELECT * FROM cust_details WHERE title=<any of above>

I always get FTS. Index is not being used.

Following are the sql trace details


SELECT * FROM cust_details WHERE title='MRS'

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 1 0.00 0.00 2 1 3 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.00 0.00 2 1 3 1

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 18 (EIBTC1)

Rows Execution Plan
------- ---------------------------------------------------
0 SELECT STATEMENT GOAL: CHOOSE
0 PARTITION (CONCATENATED)
3 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'CUST_DETAILS'

I'm getting results fast, just curious to know why it is not using the index.

Secondly, in one of the replies you said,

"if you have lots of single row inserts -- a bitmap index would be the kiss of death".


In my DB, data is loaded in database using single rows, but this operation is performed nightly, so no users are using DB. Will you recommend bitmap index in this scenario?






Tom Kyte
February 14, 2005 - 5:34 pm UTC

because you probably don't have histograms so it basically knows the hi/lo (maybe) and number of distinct (maybe) values.

 
ops$tkyte@ORA9IR2> select 24633307 /31 from dual;
 
24633307/31
-----------
 794622.806


it thinks "oh about 794,622 rows for each"


(but me thinks that tkprof and explain plan are "lying to you".  there is NOT A CHANCE IN HECK that was a full scan.  It looks more like index range scan to me.....)


explain plan can be wrong for various reasons.  you should

a) enable trace
b) run query
c) EXIT SQLPLUS

then tkprof without explain=u/p, you'll get the real plan 

Bitmap index

Yogesh, February 15, 2005 - 7:38 am UTC

I followed the steps, which you mentioned. I used 'FR' instead of MRS, so screen won't flood with rows.

I got following sql trace output

SELECT *FROM customer WHERE title='FR'


call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 36 0.00 0.00 442939 498818 70 518
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 38 0.00 0.00 442939 498818 70 518

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 18

Rows Execution Plan
------- ---------------------------------------------------
0 SELECT STATEMENT GOAL: CHOOSE
0 PARTITION (CONCATENATED)
24640499 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'CUSTOMER'


Another interesting thing, I have one more bitmap index on same table on column customer_type

select customer_type, count(*) from customer group by customer_type

I got 545 distinct types,

When I use

SELECT * FROM customer WHERE customer_type='E0010'

It uses following plan, which I was expecting in above case as well.

SELECT STATEMENT Optimizer Mode=CHOOSE
PARTITION CONCATENATED
TABLE ACCESS BY LOCAL INDEX ROWID CUSTOMER
BITMAP CONVERSION TO ROWID
BITMAP INDEX SINGLE VALUE CUSTOMER_S2

How I can check if the histogram is generated for this table?

Following are the values from dba_histogram table for these two columns

TABLE COLUMN ENDPOINT_NUMBER, ENDPOINT_VALUE

CUSTOMER,CUSTOMER_TYPE, 0, 3.38497667279466E35
CUSTOMER,TITLE, 0, 0

As well, as I asked in last question, In my DB, data is loaded in database using single rows, but this operation is performed nightly, so no users are using DB. Will you recommend bitmap index in this scenario?


Tom Kyte
February 15, 2005 - 3:30 pm UTC

have you read the jonathan lewis articles on bitmapped indexes on dbazine.com?



dbazine.com

Yogesh, February 16, 2005 - 10:43 am UTC

Text from the bitmap index article

"When you are considering bitmap indexes, do not be put off by a column which has a "large" number of different values. If every value appears a "large" number of times, and if the rows for each value are reasonably clustered, then a bitmap index may be highly appropriate. In a table with 100M rows, a column with 10,000 different values could easily turn out to be a perfect candidate for a bitmap index."

If I use this as my baseline, indexes what I've created are perfect candidates for bitmap indexes. Table size 4G+, number of distinct values 600.

But, another text from same article says

"Bitmap indexes are typically useful only for queries that can use several such indexes at once.

Updates to bitmapped columns, and general insertion/deletion of data can cause serious lock contention.

Updates to bitmapped columns, and general insertion/deletion of data can degrade the quality of the indexes quite dramatically.
"

I do not use bitmap indexes together. Bitmap columns do get updated frequently. I guess which is a problem. So I'm going with B*tree index.

But I could not understand why it was not able to pick up the index.

For ex, if I use query

select * from customer where customer_type='X'

it use to do full scan and time taken was some seconds.

But when I used following query, where I explicitly specified the index hint

select /*+ INDEX(c,customer_i3)*/ * from customer c where customer_type='X'

Not only it was picking up the index, but also I was getting results in MS.

As you said in last reply,

Is it a histogram problem?

I analyzed the columns as well, but still there was no change.

Is it a problem of bucket size?


Tom Kyte
February 16, 2005 - 11:28 am UTC

<quote>
Bitmap columns do get updated frequently. 
</quote>

that is sufficient information to know "bitmap indexes would be totally inappropriate for you in this case"


you give me NOTHING to work with, no exmaple, no "this is the analyze I used", NOTHING.

by default in 9ir2, you should expect this:

ops$tkyte@ORA9IR2> begin
  2          for x in ( select * from t2 )
  3          loop
  4                  insert /*+ APPEND */ into t (title,data)
  5          select x.title, 'x'
  6            from big_table.big_table
  7           where rownum <= x.cnt;
  8                  commit;
  9          end loop;
 10  end;
 11  /
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> create index t_idx on t(title);
 
Index created.
 
ops$tkyte@ORA9IR2> exec dbms_stats.gather_table_stats( user, 'T', cascade=>true );
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> select title, count(*) from t where title is not null group by title;
 
TITLE        COUNT(*)
---------- ----------
CAPT              811
CMDR               36
COL               267
COMPAN         156152
DR             140541
FR                518
GROUP               3
INF               474
LADY             1615
LORD              913
MADAM             320
MAJOR             279
MASTER          27309
MIS                 3
MISS          4280373
MR            9000003
MR & MR             3
MR.                 2
MRS           6913011
MS            1221240
MSTR                6
OTHER           32652
PROF             4713
REV              9011
REV.                3
SGT                73
SIR              1992
SISTER           3163
STAFF             554
 
29 rows selected.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> set autotrace traceonly explain
ops$tkyte@ORA9IR2> select * from t where title = 'X';
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=4645 Card=751588 Bytes=18038112)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'T' (Cost=4645 Card=751588 Bytes=18038112)
   2    1     INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE) (Cost=1549 Card=751588)
 
 
 
ops$tkyte@ORA9IR2> select * from t where title = 'MR';
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=4645 Card=751588 Bytes=18038112)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'T' (Cost=4645 Card=751588 Bytes=18038112)
   2    1     INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE) (Cost=1549 Card=751588)
 
 
 
ops$tkyte@ORA9IR2> select * from t where title = 'MIS';
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=4645 Card=751588 Bytes=18038112)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'T' (Cost=4645 Card=751588 Bytes=18038112)
   2    1     INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE) (Cost=1549 Card=751588)
 
 
 
ops$tkyte@ORA9IR2> set autotrace off
<b>
because it basically got two buckets in the histogram, so it has imperfect information, if you instead:</b>


ops$tkyte@ORA9IR2> exec dbms_stats.gather_table_stats( user, 'T', method_opt=>'for all indexed columns size 254', cascade=>true );
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> set autotrace traceonly explain
ops$tkyte@ORA9IR2> select * from t where title = 'X';
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=4 Card=1 Bytes=24)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'T' (Cost=4 Card=1 Bytes=24)
   2    1     INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE) (Cost=3 Card=1)
 
 
 
ops$tkyte@ORA9IR2> select * from t where title = 'MR';
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=13697 Card=9000003 Bytes=216000072)
   1    0   TABLE ACCESS (FULL) OF 'T' (Cost=13697 Card=9000003 Bytes=216000072)
 
 
 
ops$tkyte@ORA9IR2> select * from t where title = 'MIS';
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE (Cost=4 Card=3 Bytes=72)
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'T' (Cost=4 Card=3 Bytes=72)
   2    1     INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE) (Cost=3 Card=3)
 
 
 
ops$tkyte@ORA9IR2> set autotrace off


<b>it changes plans based on the values asked for since it "knows"</b>


 

Bucket size

Yogesh, February 17, 2005 - 7:43 am UTC

Any specific reason for using bucket size 254?

Tom Kyte
February 17, 2005 - 9:27 am UTC

ops$tkyte@ORA9IR2> exec dbms_stats.gather_table_stats( user, 'T', method_opt=> 'for all indexed columns size 255' );
BEGIN dbms_stats.gather_table_stats( user, 'T', method_opt=> 'for all indexed columns size 255' ); END;
 
*
ERROR at line 1:
ORA-20000: Cannot parse for clause: FOR ALL INDEXED COLUMNS SIZE 255
ORA-06512: at "SYS.DBMS_STATS", line 9375
ORA-06512: at "SYS.DBMS_STATS", line 9389
ORA-06512: at line 1
 

because 255 doesn't work.....

and if there are less than 255 distinct values, it'll store just that many buckets.


I think 254 is "good", if you are going to get histograms, for the most part, why not get as much detail as possible in most cases (now, if I had thousands of columns and they have thousands of values -- i might not want that much information in my dictionary cache so there are extremes to consider) 

is optimizer biased against bitmap indexes

Stanislav, April 19, 2005 - 9:14 pm UTC

Tom, i created a 10 mil rows fact table in datawarehouse

i partition on time
and i have bitmap index on a column which has only 3000 unique columns; both table and index have computed up-to-date statistics

when i run query which has both the partition column as well as column on which i have bitmap index oracle is using full scan; when i specify index_combine hint it uses my bitmap index and of course query runs 10 times as fast with the bitmap indexep

so the question is what is going on??
i cannot attach hint to every query as it would mean changing aweful lot of code



Tom Kyte
April 20, 2005 - 6:43 am UTC

no, it is not biased.

show us the nicely formatted dbms_xplan.display output from both queries along with a tkprof and we'll see what we can say.

never mind

Stanislav, April 19, 2005 - 9:27 pm UTC

i analyzed the table using histograms and although it took twice a long as usual analyze the result paid back handsomely - optimizer now uses bitmap indexes

Tom Kyte
April 20, 2005 - 6:44 am UTC

that would have been a side effect of looking at the

a) plan, what the optimizer thought and
b) tkprof, what actually happened

we would have seen the "guesses" were way off in (a) and asked "why". Insufficient statistics would have been the guess....

(just explaining how I would have gotten from here to there)

questionon COST

sns, April 20, 2005 - 12:54 pm UTC

In one of the discussion thread, I remember you saying

"You cannot compare the cost of 2 queries with eachother. They are simply not comparable"

However, in this thread at the beginning of the discussion you have mentioned "the cost of using the index is more than a full scan OF SUCH A TEENY TINY
TABLE
"
I think you are comparing the COST of two different explain plan statements.

So what should we infer on the value of COST?
When does its value makes sense for our analysis?

Thanks,


Tom Kyte
April 20, 2005 - 9:08 pm UTC

the cost as in "the cost, the expense, the result of"

Why cant I get INDEX_COMBINE to be used

Paul, June 21, 2005 - 9:44 am UTC

I have a large table with several Bitmap indexed columns.

The 2 columns that I need to use for this query are AGE and PROB_CD. No matter what I do, the BITMAPS are not doing what I think I understand could/should be happening. The query was :

SELECT
c1,
C2,
COUNT(*)
FROM
valids_table v
data_table s
WHERE
v.Valid_Category = 'SOMEVALUE'
AND s.Prob_Cd BETWEEN v.Lo AND v.Hi
AND s.Age >= 14
GROUP BY
c1,
C2 ;

In this case, there are 9 rows from the Valid table. They convert to only 298 from the data table. I can hint it but it will use either the AGE ( 4BI ) or the PROB_CD ( 5BI ) bitmap. I decided that the between was causing the Optimizer to be unable to infer the needed stats. I changed it to:

SELECT /*+ INDEX_COMBINE( s data_table_4BI data_table_5BI ) */
c1,
C2,
COUNT(*)
FROM
data_table s
WHERE
s.Proc_Cd IN
( SELECT /*+ INDEX( s data_table_5BI ) */
DISTINCT s2.Prob_Cd
FROM
valids_table v,
data_table s2
WHERE
v.Valid_Category = 'SOMEVALUE' AND
s2.Proc_Cd BETWEEN v.Lo AND v.Hi )
AND s.Age >= 14
GROUP BY
c1,
C2 ;

But it still will not use them both and I want to understand why not. The plan, via AUTOTRACE, is:

Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=289301 Card=1)
1 0 SORT (AGGREGATE)
2 1 PX COORDINATOR
3 2 PX SEND* (QC (RANDOM)) OF ':TQ10004'
4 3 SORT* (AGGREGATE)
5 4 VIEW* (Cost=289301 Card=30599931)
6 5 SORT* (GROUP BY) (Cost=289301 Card=30599931 Bytes=1376996895)
7 6 PX RECEIVE* (Cost=289301 Card=30599931 Bytes=1376996895)
8 7 PX SEND* (HASH) OF ':TQ10003' (Cost=289301 Card=30599931 Bytes=1376996895)
9 8 SORT* (GROUP BY) (Cost=289301 Card=30599931 Bytes=1376996895)
10 9 HASH JOIN* (RIGHT SEMI) (Cost=273708 Card=30599931 Bytes=1376996895)
11 10 PX RECEIVE* (Cost=1028 Card=813003 Bytes=4878018)
12 11 PX SEND* (HASH) OF ':TQ10001' (Cost=1028 Card=813003 Bytes=4878018)
13 12 VIEW* OF 'VW_NSO_1' (VIEW) (Cost=1028 Card=813003 Bytes=4878018)
14 13 NESTED LOOPS* (Cost=1028 Card=813003 Bytes=23577087)
15 14 BUFFER* (SORT)
16 15 PX RECEIVE*
17 16 PX SEND* (BROADCAST) OF 'TQ10000'
18 17 TABLE ACCESS (FULL) OF 'valids_table ' (TABLE) (Cost=2 Card=8 Bytes=184)
19 14 PX PARTITION HASH* (ALL) (Cost=1028 Card=102912 Bytes=617472)
20 19 BITMAP CONVERSION* (TO ROWIDS) (Cost=1028 Card=102912 Bytes=617472)
21 20 BITMAP INDEX* (RANGE SCAN) OF 'data_table_5BI' (INDEX (BITMAP))
22 10 PX RECEIVE* (Cost=272625 Card=35658521 Bytes=1390682319)
23 22 PX SEND* (HASH) OF ':TQ10002' (Cost=272625 Card=35658521 Bytes=1390682319)
24 23 PX PARTITION HASH* (ALL) (Cost=272625 Card=35658521 Bytes=1390682319)
25 24 TABLE ACCESS* (BY LOCAL INDEX ROWID) OF 'data_table' (TABLE) (Cost=272625 Card=35658521 Bytes=1390682319)
26 25 BITMAP CONVERSION* (TO ROWIDS)
27 26 BITMAP INDEX* (RANGE SCAN) OF 'data_table_4BI' (INDEX (BITMAP))

I believe I am seeing this frequently, where the optimizer ignores valid but higher cost hints (although in this case it may not be.) What am I doing wrong? The number of rows actually returned is only 15 out of 40 million...

Tom Kyte
June 21, 2005 - 5:06 pm UTC

SELECT
c1,
C2,
COUNT(*)
FROM
valids_table v
data_table s
WHERE
v.Valid_Category = 'SOMEVALUE'
AND s.Prob_Cd BETWEEN v.Lo AND v.Hi
AND s.Age >= 14
GROUP BY
c1,
C2 ;

how could it index combine?? you are comparing prob_cd not to constants, but many variable values from V?????

you'd need the equivalent of a "between list iterator" like we have for inlist iterators, but there isn't one.

Tom, please read my note again

Paul, June 22, 2005 - 8:38 am UTC

If you look at both of my query examples, you will see that in the second query, which is a refinement of the first, I suspected what you said was true and modified the query to use an INLIST iterator. It still will not do a COMBINE.

Just to be very clear: I can hint the query to use EITHER bitmap index and it always runs faster than the query without. (We are having to hint everything, the optimizer thinks FTS is always the way to go.) I just can not get it to use both at the same time and use bitmap operations to arrive at the set of ROWIDS in one pass.

Tom Kyte
June 23, 2005 - 11:20 am UTC

do you have a simple setup for me to work with, create table, create index, dbms_stats.set_table_stats sort of stuff/

Not that I know what are you're "doing wrong" ...

Gabe, June 23, 2005 - 11:31 am UTC

Paul:

The hint in the subquery is likely being ignored ...
<quote>
SELECT /*+ INDEX( s data_table_5BI ) */
</quote>
"s" is out of context there ... maybe you meant "s2".

Is valid_category unique in valids_table? If not - if a data_table row could be matched against many rows in valids_table - then the next question is: what question is this query answering? ... what is all that double/triple/lots of counting of (c1,c2) pairs in data_table actually achieving?

INDEX_COMBINE

Anil, August 19, 2005 - 8:07 am UTC

Hi Tom

Would you please explain a bit about index_combine hint and its use. I checked the oracle doc but there isn't sufficient explenation abou the hint.

Appreciate your help

Thanks & Rgds
Anil

Tom Kyte
August 20, 2005 - 4:03 pm UTC

<quote src=tuning guide>
INDEX_COMBINE

The INDEX_COMBINE hint explicitly chooses a bitmap access path for the table. If no indexes are given as arguments for the INDEX_COMBINE hint, then the optimizer uses whatever Boolean combination of bitmap indexes has the best cost estimate for the table. If certain indexes are given as arguments, then the optimizer tries to use some Boolean combination of those particular bitmap indexes.

</quote>

what needs more detail?

Index Combine

Anil, August 21, 2005 - 12:41 pm UTC

Hi Tom

I had seen this bit documentation. But I dibn't really understand the concept.

Is this usefull only for bitmap indexes?

What does it mean by boolean combination?

If you don't mind , would you please explain a bit with an example.

Rgds
Anil

Tom Kyte
August 21, 2005 - 1:35 pm UTC

it is all about bitmaps, it mentions "bitmaps" repeatedly.

boolean - and/or.


bitmaps can be "anded" and "ored" together:

small snippet of this anding/oring from my new book:

<quote>
Here the bitmap index comes into play. With three small bitmap indexes, one on each of the individual columns, you will be able to satisfy all of the previous predicates efficiently. Oracle will simply use the functions AND, OR, and NOT, with the bitmaps of the three indexes together, to find the solution set for any predicate that references any set of these three columns. It will take the resulting merged bitmap, convert the 1s into rowids if necessary, and access the data (if you are just counting rows that match the criteria, Oracle will just count the 1 bits). Let’s take a look at an example. First, we’ll generate test data that matches our specified distinct cardinalities—index it and gather statistics. We’ll make use of the DBMS_RANDOM package to generate random data fitting our distribution:

ops$tkyte@ORA10G> create table t
  2  ( gender not null,
  3    location not null,
  4    age_group not null,
  5    data
  6  )
  7  as
  8  select decode( ceil(dbms_random.value(1,2)),
  9                 1, 'M',
 10                 2, 'F' ) gender,
 11         ceil(dbms_random.value(1,50)) location,
 12         decode( ceil(dbms_random.value(1,5)),
 13                 1,'18 and under',
 14                 2,'19-25',
 15                 3,'26-30',
 16                 4,'31-40',
 17                 5,'41 and over'),
 18         rpad( '*', 20, '*')
 19    from big_table.big_table
 20   where rownum <= 100000;
Table created.
 
ops$tkyte@ORA10G> create bitmap index gender_idx on t(gender);
Index created.
 
ops$tkyte@ORA10G> create bitmap index location_idx on t(location);
Index created.
 
ops$tkyte@ORA10G> create bitmap index age_group_idx on t(age_group);
Index created.
 
ops$tkyte@ORA10G> exec dbms_stats.gather_table_stats( user, 'T', cascade=>true );
PL/SQL procedure successfully completed.

Now we’ll take a look at the plans for our various ad hoc queries from earlier:

ops$tkyte@ORA10G> Select count(*)
  2    from T
  3   where gender = 'M'
  4     and location in ( 1, 10, 30 )
  5     and age_group = '41 and over';
 
Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=ALL_ROWS (Cost=5 Card=1 Bytes=13)
   1    0   SORT (AGGREGATE)
   2    1     BITMAP CONVERSION (COUNT) (Cost=5 Card=1 Bytes=13)
   3    2       BITMAP AND
   4    3         BITMAP INDEX (SINGLE VALUE) OF 'GENDER_IDX' (INDEX (BITMAP))
   5    3         BITMAP OR
   6    5           BITMAP INDEX (SINGLE VALUE) OF 'LOCATION_IDX' (INDEX (BITMAP))
   7    5           BITMAP INDEX (SINGLE VALUE) OF 'LOCATION_IDX' (INDEX (BITMAP))
   8    5           BITMAP INDEX (SINGLE VALUE) OF 'LOCATION_IDX' (INDEX (BITMAP))
   9    3         BITMAP INDEX (SINGLE VALUE) OF 'AGE_GROUP_IDX' (INDEX (BITMAP))

This example shows the power of the bitmap indexes. Oracle is able to see the location in (1,10,30) and knows to read the index on location for these three values and logically OR together the “bits” in the bitmap. It then takes that resulting bitmap and logically ANDs that with the bitmaps for AGE_GROUP='41 AND OVER' and GENDER='M'. Then a simple count of 1s and the answer is ready.
</quote>



Note: did not need to use the index_combine hint (as is true for most hints, you don't need to use them).  Oracle combined the indexes for us transparently, this is the way it is supposed to work (transparently) 

Thanks a Lot

Anil, August 21, 2005 - 2:09 pm UTC


Bitmap Index not used

Sree, March 29, 2011 - 10:34 am UTC

Tom, it has been answered many times in this forum /thread why a Bitmap index is not being used.but again,every case is differetn. So, here's my scenario and I'm trying ot understand what i'm missing..

I have a table with 2 mil and two bitmap indexed columns rows with a count on each bitmap index as

EFF_PGM_YEAR||'-'||EFF_PGM_QTR COUNT(*)
2010-1 629007
2010-2 614112
2009-3 589102
2009-4 608075


So, recreated a test scenario with small set and here are the scripts for that:
SQL> create table test_bitmap_index(year integer,qtr integer,detail_value integer);

Table created.

Elapsed: 00:00:00.07
SQL> insert into test_bitmap_index select 2010,1,rownum+100 from all_objects where rownum<=1000;

1000 rows created.

Elapsed: 00:00:01.48
SQL> insert into test_bitmap_index select 2010,2,rownum+10 from all_objects where rownum<=1000;

1000 rows created.

Elapsed: 00:00:00.70
SQL> insert into test_bitmap_index select 2011,1,rownum+1000 from all_objects where rownum<=1000;

1000 rows created.

Elapsed: 00:00:00.68
SQL> insert into test_bitmap_index select 2011,2,rownum+1000 from all_objects where rownum<=1000;

1000 rows created.

Elapsed: 00:00:00.70
SQL> commit;

Commit complete.

Elapsed: 00:00:00.00
SQL> select count(*) from test_bitmap_index;

  COUNT (*)                                                                     
----------4000                                                        
Elapsed: 00:00:00.03
SQL> create bitmap index idx_year on test_bitmap_index(year);
Index created.
Elapsed: 00:00:00.34
SQL> create bitmap index idx_qtr on test_bitmap_index(qtr);

Index created.

Elapsed: 00:00:00.46
SQL> 
SQL> exec dbms_stats.gather_table_stats('OSOSBI','TEST_BITMAP_INDEX');

PL/SQL procedure successfully completed.

Elapsed: 00:00:01.37
SQL> set linesize 1000;
SQL> select index_name,blevel,leaf_blocks,distinct_keys,num_rows from user_indexes where index_name in ('IDX_YEAR','IDX_QTR');

INDEX_NAME                         BLEVEL LEAF_BLOCKS DISTINCT_KEYS   NUM_ROWS                                                    ------------------------------ ---------- ----------- ------------- ----------                                                            IDX_QTR                                 0           1             2          2                                                           
IDX_YEAR                                0           1             2          2                                                           
Elapsed: 00:00:00.07
SQL> explain plan for select count(*) from test_bitmap_index where year=2010 and qtr=2;

Explained.

Elapsed: 00:00:00.18
SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                   
Plan hash value: 2075070854                                                                                                              -----------------------------------------------------------------------------------------
| Id  | Operation                    | Name     | Rows  | Bytes | Cost (%CPU)| Time     |                                                                                                                                                                                   -----------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT             |          |     1 |     7 |     2   (0)| 00:00:01 |                                                                                                                       |   1 |  SORT AGGREGATE              |          |     1 |     7 |            |          |                                                           |   2 |   BITMAP CONVERSION COUNT    |          |  1000 |  7000 |     2   (0)| 00:00:01 |                                                                                                                       |   3 |    BITMAP AND                |          |       |       |            |          |                                                                                                                       |*  4 |     BITMAP INDEX SINGLE VALUE| IDX_QTR  |       |       |            |          |                                                                                                                       |*  5 |     BITMAP INDEX SINGLE VALUE| IDX_YEAR |       |       |            |          |                                                           
PLAN_TABLE_OUTPUT                                             ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Predicate Information (identified by operation id):                                                           4 - access("QTR"=2)                                                      5 - access("YEAR"=2010)                                                                                                           18 rows selected.

Elapsed: 00:00:00.67
SQL> explain plan for select * from test_bitmap_index where year=2010 and qtr=2;

Explained.

Elapsed: 00:00:00.00
SQL> select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                                                                                       Plan hash value: 4148853163                                                                                                              | Id  | Operation         | Name              | Rows  | Bytes | Cost (%CPU)| Time     |                                                           |   0 | SELECT STATEMENT  |                   |  1000 | 10000 |    15   (0)| 00:00:01 |                                                           |*  1 |  TABLE ACCESS FULL| TEST_BITMAP_INDEX |  1000 | 10000 |    15   (0)| 00:00:01 |                                                           Predicate Information (identified by operation id):                                                        PLAN_TABLE_OUTPUT                                              1 - filter("YEAR"=2010 AND "QTR"=2)                                                13 rows selected.

Elapsed: 00:00:00.07
SQL> explain plan for select detail_value from test_bitmap_index where year=2010 and qtr=2;

Explained.

Elapsed: 00:00:00.01
SQL>  select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT                                           Plan hash value: 4148853163                                                  | Id  | Operation         | Name              | Rows  | Bytes | Cost (%CPU)| Time     |                                                           |   0 | SELECT STATEMENT  |                   |  1000 | 10000 |    15   (0)| 00:00:01 |                                                           |*  1 |  TABLE ACCESS FULL| TEST_BITMAP_INDEX |  1000 | 10000 |    15   (0)| 00:00:01 |                                                           Predicate Information (identified by operation id):                                                                                                                    PLAN_TABLE_OUTPUT                                                                                                          1 - filter("YEAR"=2010 AND "QTR"=2)                                                
13 rows selected.

Elapsed: 00:00:00.09

So, question is
a)why it's using index on count(*) and why not if I select individual columns?
b)Did I miss any other scenarios that I should be testing?



Tom Kyte
March 30, 2011 - 1:28 am UTC

a) because we can get the entire answer for the count(*) from the index - we never have to touch the table. That is why it uses it for count(*).

But when you ask for the data from the table - the other columns NOT IN the index, it very wisely, very intelligently, VERY CORRECTLY says "using the index would be dumb, I am getting much of the table - the very best way to do that would be to use really efficient multi-block IO - read the table and find the rows really really fast. Rather than going back and forth and back and forth and back and forth between an index and the table over and over and over and over and over.

http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:6749454952894#6760861174154

indexes are not all goodness, sometimes they are just a really utterly bad idea.



2) no idea, I have no idea what you are testing for?

Bitmap Index not used

A reader, March 30, 2011 - 8:16 am UTC

Thanks Tom.
Since we are using Bitmap Indexes [one for each Low cardinality column], how can we have all the selected columns in the index..I was not clear on that part.
So, It seems to me that Bitmap index is only good for
Count(*)[summaries/aggregates] and not for details, in a Data Warehouse. is my understanding Correct?

on number (2), I was wondering if I need to test the query by adding a Hint or something of that sort..but you can disregard it.

Thanks,
Tom Kyte
March 30, 2011 - 8:26 am UTC

Since we are using Bitmap Indexes [one for each Low cardinality column], how
can we have all the selected columns in the index..I was not clear on that
part


bitmaps are not for low cardinality data, they work well on that, but that is not what they are about.

if you want to get 1000 out of 4000 rows - the most efficient way to do that is almost certainly NOT via an index.... do you see that? if I asked you to find 1000 out of 4000 pages of a book and I gave you a keyword that you would have to

a) go to index
b) find a page
c) go to a 999 more times

or

just get all of the pages and flip through them


which would you rather do?



bitmap indexes are good for the same exact things a b*tree index is good for:

a) being used INSTEAD of the table (because they are smaller)
b) finding a FEW rows in a table



period - if you need to get say a million rows out of a table and you are using an index - YOU ARE DOING IT WRONG. You should have been using partitions to eliminate most of the data so a full scan is efficient for you.


bitmaps have an advantage over b*trees in that we can bitmap and/or MANY bitmaps together to get a few rows out of a table (or count them). but bitmaps are not 'magic', they work just like b*tree's when it comes to GETTING ROWS OUT OF A TABLE - you get a rowid and go back and forth and back and forth (which is slower and slower and slower as you get more and more and more rows)

index combine

Mohan K Gandhi, November 02, 2011 - 12:08 pm UTC

Hello Tom,

One of my query is performing very poor when it is using following plan:

--poor plan

0 INSERT STATEMENT - (Cost =1616 Card = Bytes =)
1 SORT UNIQUE - (Cost =1616 Card = 4 Bytes =256)
2 CONCATENATION - (Cost = Card = Bytes =)
3 NESTED LOOPS - (Cost =808 Card = 2 Bytes =128)
4 NESTED LOOPS - (Cost =806 Card = 2 Bytes =90)
5 INDEX RANGE SCAN bbb - (Cost =3 Card = 1 Bytes =21)
5 TABLE ACCESS BY INDEX ROWID ccc - (Cost =803 Card = 1 Bytes =24)
6 INDEX RANGE SCAN ccc_N13 - (Cost =801 Card = 2 Bytes =)
4 TABLE ACCESS BY INDEX ROWID ddd - (Cost =1 Card = 1 Bytes =19)
5 INDEX UNIQUE SCAN eee - (Cost =0 Card = 1 Bytes =)
3 NESTED LOOPS - (Cost =808 Card = 2 Bytes =128)
4 NESTED LOOPS - (Cost =806 Card = 2 Bytes =90)
5 INDEX RANGE SCAN bbb - (Cost =3 Card = 1 Bytes =21)
5 TABLE ACCESS BY INDEX ROWID ccc - (Cost =803 Card = 1 Bytes =24)
6 INDEX RANGE SCAN ccc_N13 - (Cost =801 Card = 2 Bytes =)
4 TABLE ACCESS BY INDEX ROWID ddd - (Cost =1 Card = 1 Bytes =19)
5 INDEX UNIQUE SCAN eee - (Cost =0 Card = 1 Bytes =)

But when plan changed to following its logical io/exec reduced to 1/4 to the original and query's performance is also very much improved

--------------------------------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | A-Rows | A-Time | Buffers | Reads | OMem | 1Mem | Used-Mem |
---------------------------------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 241 |00:00:11.16 | 61699 | 19855 | | | |
| 1 | SORT UNIQUE | | 1 | 3 | 241 |00:00:11.16 | 61699 | 19855 | 13312 | 13312 |12288 (0)|
| 2 | NESTED LOOPS | | 1 | 3 | 19773 |00:00:15.19 | 61699 | 19855 | | | |
| 3 | HASH JOIN | | 1 | 3 | 19773 |00:00:14.75 | 22151 | 19853 | 1397K| 1397K| 1650K (0)|
| 4 | TABLE ACCESS BY INDEX ROWID | aaa | 1 | 3 | 18061 |00:00:06.17 | 15779 | 13523 | | | |
| 5 | BITMAP CONVERSION TO ROWIDS | | 1 | | 18061 |00:00:00.14 | 40 | 35 | | | |
| 6 | BITMAP OR | | 1 | | 2 |00:00:00.12 | 40 | 35 | | | |
| 7 | BITMAP CONVERSION FROM ROWIDS| | 1 | | 1 |00:00:00.02 | 6 | 4 | | | |
| 8 | SORT ORDER BY | | 1 | | 1735 |00:00:00.02 | 6 | 4 | 106K| 106K|96256 (0)|
| 9 | INDEX RANGE SCAN | bbb | 1 | 1944K| 1735 |00:00:00.01 | 6 | 4 | | | |
| 10 | BITMAP CONVERSION FROM ROWIDS| | 1 | | 2 |00:00:00.10 | 34 | 31 | | | |
| 11 | SORT ORDER BY | | 1 | | 16326 |00:00:00.11 | 34 | 31 | 549K| 408K| 487K (0)|
| 12 | INDEX RANGE SCAN | bbb | 1 | 1944K| 16326 |00:00:00.03 | 34 | 31 | | | |
| 13 | TABLE ACCESS FULL | ccc | 1 | 199K| 275K|00:00:11.03 | 6372 | 6330 | | | |
| 14 | TABLE ACCESS BY INDEX ROWID | ddd | 19773 | 1 | 19773 |00:00:00.36 | 39548 | 2 | | | |
| 15 | INDEX UNIQUE SCAN | eee | 19773 | 1 | 19773 |00:00:00.18 | 19775 | 2 | | | |
---------------------------------------------------------------------------------------------------------------------------------------------------------------

I compared both plans but fail to see the reason. Is it index combine? If it is so then what are the places where we can use index combine?

Thanks

Mohan K Gandhi

index_combine

A reader, March 07, 2012 - 2:25 am UTC

Hi Tom,

My question is about index.

1. if i have index on table(c1) and table(c2), my understanding is for 'where c1... or c2...' will not use index
but my testing below do use index, why? under what circumstance it will be FTS

2. even use index, they have different behavior for '=' and '<>' condition, why?
'where id=3 or a=5' result in 'bitmap conversion', my assumption is that this only happens with 'index_combine' hint
'where id<3 or a>5' result in 'concatenation of index range scan'

create table test(id int, a int);
exec dbms_stats.set_table_stats(ownname=>user,tabname=>'TEST',numrows=>100000,numblks=>1000);
create index ind1 on test(id);
create index ind2 on test(a);
select * from test where id=3 or a=5;
Execution Plan
----------------------------------------------------------
Plan hash value: 2830690570

--------------------------------------------------------------------------------
---------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| T
ime |

--------------------------------------------------------------------------------
---------

| 0 | SELECT STATEMENT | | 1990 | 51740 | 203 (0)| 0
0:00:03 |

| 1 | TABLE ACCESS BY INDEX ROWID | TEST | 1990 | 51740 | 203 (0)| 0
0:00:03 |

| 2 | BITMAP CONVERSION TO ROWIDS | | | | |
|

| 3 | BITMAP OR | | | | |
|

| 4 | BITMAP CONVERSION FROM ROWIDS| | | | |
|

|* 5 | INDEX RANGE SCAN | IND1 | | | 1 (0)| 0
0:00:01 |

| 6 | BITMAP CONVERSION FROM ROWIDS| | | | |
|

|* 7 | INDEX RANGE SCAN | IND2 | | | 1 (0)| 0
0:00:01 |

--------------------------------------------------------------------------------
---------


Predicate Information (identified by operation id):
---------------------------------------------------

5 - access("ID"=3)
7 - access("A"=5)


Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
0 consistent gets
0 physical reads
0 redo size
329 bytes sent via SQL*Net to client
408 bytes received via SQL*Net from client
1 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
0 rows processed

select * from test where id<3 or a>5;
Execution Plan
----------------------------------------------------------
Plan hash value: 627178165

--------------------------------------------------------------------------------
-----

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time
|

--------------------------------------------------------------------------------
-----

| 0 | SELECT STATEMENT | | 9750 | 247K| 0 (0)| 00:00
:01 |

| 1 | CONCATENATION | | | | |
|

| 2 | TABLE ACCESS BY INDEX ROWID| TEST | 5000 | 126K| 0 (0)| 00:00
:01 |

|* 3 | INDEX RANGE SCAN | IND2 | 1 | | 0 (0)| 00:00
:01 |

|* 4 | TABLE ACCESS BY INDEX ROWID| TEST | 4750 | 120K| 0 (0)| 00:00
:01 |

|* 5 | INDEX RANGE SCAN | IND1 | 1 | | 0 (0)| 00:00
:01 |

--------------------------------------------------------------------------------
-----
Tom Kyte
March 07, 2012 - 6:51 pm UTC

in general it will not use an index, there is no rule stating "it cannot"

we can do bitmap conversions
we can do index joins

there are opportunities to use multiple b*tree indexes - they are rare, they are not common, but they've existed for a long long time.

Combined Bitmap Selectivity

Yo, September 06, 2019 - 4:19 pm UTC

You said -

if you want to get 1000 out of 4000 rows - the most efficient way to do that is almost certainly NOT via an index.... do you see that? if I asked you to find 1000 out of 4000 pages of a book and I gave you a keyword that you would have to

My Question -

But that should that stop us from creating a bitmap index on a high cardinality column? Specifically, we might have several high-cardinality bitmap indexes, but after combining them (BITMAP-AND) - it would result in a higher combined selectivity. Would that not be a net gain?
Chris Saxon
September 10, 2019 - 11:54 am UTC

Being able to combine bitmap indexes is a key reason to use them. And yes, this can still be useful if the individual columns return too many rows to deserve an index in their own right.

For example, if you have a table like this:

create table t as 
  select level id,
         mod ( level, 3 ) c1,
         mod ( level, 4 ) c2,
         mod ( level, 5 ) c3,
         mod ( level, 7 ) c4,
         lpad ( 'x', 100, 'x' ) stuff
  from   dual
  connect by level <= 1000;


Searching for a value in C4 returns ~14% of the rows, probably too much for an index to be useful.

But searching on all of the CX columns returns few rows. So BITMAP-ANDing of a bitmap index on each column enables the database to read the table few times:

create bitmap index b1 on t ( c1 );
create bitmap index b2 on t ( c2 );
create bitmap index b3 on t ( c3 );
create bitmap index b4 on t ( c4 );

set serveroutput off
select * from t
where  c1 = 0
and    c2 = 0
and    c3 = 0
and    c4 = 0;

select * 
from   table(dbms_xplan.display_cursor(null, null, 'BASIC LAST'));

----------------------------------------------------                       
| Id  | Operation                           | Name |                       
----------------------------------------------------                       
|   0 | SELECT STATEMENT                    |      |                       
|   1 |  TABLE ACCESS BY INDEX ROWID BATCHED| T    |                       
|   2 |   BITMAP CONVERSION TO ROWIDS       |      |                       
|   3 |    BITMAP AND                       |      |                       
|   4 |     BITMAP INDEX SINGLE VALUE       | B4   |                       
|   5 |     BITMAP INDEX SINGLE VALUE       | B3   |                       
|   6 |     BITMAP INDEX SINGLE VALUE       | B2   |                       
|   7 |     BITMAP INDEX SINGLE VALUE       | B1   |                       
----------------------------------------------------


This is particularly useful in data warehouse/ad hoc reporting environments where you can have queries on any combination of the columns. So making suitable multi-column b*tree indexes is... tricky.

More to Explore

Performance

Get all the information about database performance in the Database Performance guide.