Skip to Main Content

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Martin.

Asked: July 14, 2000 - 6:24 pm UTC

Last updated: May 28, 2012 - 2:27 am UTC

Version: 8.1.6.1

Viewed 10K+ times! This question is

You Asked

We had the problem that we were not allowed to specify NOT NULL
on a column but we had to search the NULLs efficiently very
often. They constitute a small sample of the complete data set.
The test case I prepared uses a function-based index on the
function nvl. Perhaps you can use it.

Martin


-- findnulls.sql demonstrates the use of function-based indexes
-- for finding nulls via an index.

-- Martin Haltmayer, 2000/07/15

-- The "nologging" is for speed only.

-- The plan.sql is a script just to display and to empty the plan_table.

-- Tested on 8.1.6.1 NT 4.0 SP 6 a


-- Please remember to grant query rewrite to the executing user, to set
-- query_rewrite_enabled = true and query_rewrite_integrity = trusted
-- in init.ora!


spool findnulls.sql.lst

set timing on

-- To make sure we can drop the table lock it.
lock table test in exclusive mode;
drop table test cascade constraints;

whenever sqlerror exit failure rollback

create table test pctfree 0 nologging as
select decode (mod (rownum, 100), 0, to_number (null), rownum) as n, a.object_name as c
from all_objects a, all_objects b
where rownum <= 1000000
/

describe test

create index test_i on test (n) pctfree 0 nologging;

analyze table test compute statistics;

explain plan for
select --+ all_rows
count (*) from test where n is null;

@plan.sql

select --+ all_rows
count (*) from test where n is null;

create index test_f on test (nvl (n, - 9999)) pctfree 0 nologging;

analyze table test compute statistics;

select num_rows from user_indexes where index_name = 'TEST_I';
select num_rows from user_indexes where index_name = 'TEST_F';

explain plan for
select --+ all_rows
count (*) from test where nvl (n, - 9999) = - 9999;

@plan.sql

select --+ all_rows
count (*) from test where nvl (n, - 9999) = - 9999;

whenever sqlerror continue

spool off


and Tom said...

Thanks Martin, one of the first times I've gotten advice instead of a question ;)

For more info on function based indexes, see
</code> http://asktom.oracle.com/~tkyte/article1/index.html <code>

In the above, you would pick a value (where Martin uses -9999) that cannot exist in your table naturally (else you will get a "false positive").

Bitmapped indexes would provide a similar functionality as they do index NULLS. For example (apache_log was analyzed previously):

scott@ORA816.US.ORACLE.COM> select count(*) from apache_log where host is null;

COUNT(*)
----------
10


Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1 Byt
1 0 SORT (AGGREGATE)
2 1 TABLE ACCESS (FULL) OF 'APACHE_LOG' (Cost=2 Card=



scott@ORA816.US.ORACLE.COM> create bitmap index apache_idx on apache_log(host);


Index created.

scott@ORA816.US.ORACLE.COM> select count(*) from apache_log where host is null;

COUNT(*)
----------
10


Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=1 Bytes
1 0 SORT (AGGREGATE)
2 1 BITMAP CONVERSION (COUNT)
3 2 BITMAP INDEX (SINGLE VALUE) OF 'APACHE_IDX'



Just be aware of concurrency issues with regards to bitmap indexes and concurrent updates of the indexed key value.

Rating

  (36 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Can you update this response for 9.0.1.4.0?

Matt, February 16, 2003 - 2:53 am UTC

For the belwo can you please make comments specific to 9.0.1.4.0?

I currently have a table with 9 columns, something like:

my_table
(
id NUMBER,
num NUMBER,
day DATE,
count NUMBER,
f_seq NUMBER,
t_seq NUMBER,
accept1_dt DATE,
accept2_dt DATE,
accept3_dt DATE,
)

The table is indexed on (id, num, day, count) as data is typically retrived from this table using these column values.

A new requirement is to retrieve data based on f_seq and t_seq. Something like:

select id, num, day, count
from my_table
where 1=1
and f_seq > :start_seq
and t_seq is null

Typical data in the table for f_seq and t_seq (where "..." indicates that id, num, day are the same in each row) are:

... count f_seq t_seq
... ===== ===== =====
1 999 1001
2 1001 8888
3 8888 9982
4 9982

My question is twofold:

1) Can I set up a functiona based index similar to above on the t_seq column so that I can easily pull out NULLs?
2) What is the preferred indexing approach:
- Add the f_seq column and NVL(t_seq,-9999) function to the existing index (and possibly adversely affect existing queries)
- Create a new index (id, num, day, count, f_seq, NVL(t_seq,-9999)) to cater for these new querying requirements. (and possibly adversely affect existing inserts / updates)
- Something else that I haven't thought of

Thanks and Regards,



Tom Kyte
February 16, 2003 - 11:13 am UTC

you don't need to.  An index on f_seq, t_seq would accomplish that.  

It is that ENTIRELY null entries are not made into a b*tree.  If you have a concatenated index (on more then one column) and AT LEAST one of the columns is not null -- that'll find it's way into the index.  

Consider:

ops$tkyte@ORA920> create table t ( f_seq int, t_seq int, x char(1) );

Table created.

ops$tkyte@ORA920>
ops$tkyte@ORA920> create index t_idx on t(f_seq,t_seq);

Index created.

ops$tkyte@ORA920>
ops$tkyte@ORA920> set autotrace on explain
ops$tkyte@ORA920> select f_seq, t_seq
  2    from t
  3   where f_seq > 0 and t_seq is null;

no rows selected


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE)



ops$tkyte@ORA920>
ops$tkyte@ORA920> select f_seq, t_seq, x
  2    from t
  3   where f_seq > 0 and t_seq is null;

no rows selected


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=CHOOSE
   1    0   TABLE ACCESS (BY INDEX ROWID) OF 'T'
   2    1     INDEX (RANGE SCAN) OF 'T_IDX' (NON-UNIQUE)



ops$tkyte@ORA920> set autotrace off

<b>see how the first query doesn't access the table -- the index is sufficient to answer the question -- hence it is getting f_seq and t_seq from the index -- the null entry is indexed in this case (since f_seq is not null)
</b>

 

Idea

Michael, February 18, 2003 - 3:30 am UTC

Hi!

To get NULL values indexed i used the following:

create table t as
select object_name, trunc(dbms_random.value(1, 1000)) testcol
from all_objects;

insert into t
select object_name, trunc(dbms_random.value(1, 1000)) testcol
from all_objects;

update t
set testcol = null
where object_name like 'ALL_OB%';
commit;

Now 12 rows (on my db) have an NULL in testcol.

analyze table t compute statistics
for table for all columns for all indexes;

select * from t where testcol is null;

... 12 rows ...

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=32 Card=12 Bytes=312)

1 0 TABLE ACCESS (FULL) OF 'T' (Cost=32 Card=12 Bytes=312)

5 recursive calls
0 db block gets
315 consistent gets
4 physical reads
0 redo size
712 bytes sent via SQL*Net to client
499 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
2 sorts (memory)
0 sorts (disk)
12 rows processed

Now with the index:

create index testidx on t(testcol, 1);
analyze table t compute statistics
for table for all columns for all indexes;

select * from t where testcol is null;

... Again 12 rows ...

0 SELECT STATEMENT Optimizer=CHOOSE (Cost=13 Card=12 Bytes=312)

1 0 TABLE ACCESS (BY INDEX ROWID) OF 'T' (Cost=13 Card=12 Bytes=312)

2 1 INDEX (RANGE SCAN) OF 'TESTIDX' (NON-UNIQUE) (Cost=2 Card=12)

0 recursive calls
0 db block gets
9 consistent gets
0 physical reads
0 redo size
712 bytes sent via SQL*Net to client
499 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
12 rows processed

Seems to work ...
Tom, are there any disadvantages with this solution?

Tom Kyte
February 18, 2003 - 7:37 am UTC

any time you have a concatenated index (index with more then one key in it) AND one of the columns in the index is NOT NULL (the constant "1" is not null in your example) -- then an index entry is made.

The thing is -- an index on an entirely NULL KEY is not made into a b*tree.  You just have an example where an entry is always made in the b*tree..

it works -- it is just "bigger".  Also -- it is a FUNCTION BASED INDEX so it'll not work in 8.0 and before:

ops$tkyte@ORA806> create index t_idx on t(x,1);
create index t_idx on t(x,1)
                          *
ERROR at line 1:
ORA-00904: invalid column name

and it requires the CBO (that index is invisible to the RBO) as is the bitmap though ;) 

your comment -- it is just "bigger"

A reader, April 22, 2003 - 12:16 pm UTC

Hi Tom, This is for 9.2.0...I liked the approach of including a constant when creating an index on columns that allow null values. But I also agree with your comment that the index will be bigger. Can you suggest what would be appropriate for the case below? Let's say I have
create table dev (devid number not null, devvalue varchar2(100), mtime date);
The table has 4.5 million rows - 2.5M are null and 2M are not null. I often would have to query on mtime as "where mtime is not null". Creating a simple index on mtime is doing a full table scan. As I undersatnd from this and your other postings its because null values don't make it to the index. If I create a composite index on (mtime,1) it will work but as you said it would be large because null entries will make it in the index. Can you please suggest a better way ( a function-based index ??) that would help me avoid a full table scan when I run this query: select devid, devvalue from dev where mtime is not null;
Thanks.

Tom Kyte
April 22, 2003 - 9:29 pm UTC

why do you want to avoid an index when 50% or more of the table would be returned?

a full scan IS APPROPRIATE and faster then an index here.

Continuation of my previous post

A reader, April 22, 2003 - 12:35 pm UTC

Tom, this is in continuation of my previous post...
I tried:

create index DUP_DEV_MTIME on DEV( decode(mtime, null, null, mtime));
analyze table dev compute statistics for columns mtime;

But its still doing a full table scan.What did I do wrong?

Tom Kyte
April 22, 2003 - 9:30 pm UTC


full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil


just keep saying that over and over and over

full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil
full scans are not evil




Pages 303-307 of your book say it all !!

A reader, April 24, 2003 - 10:13 am UTC

Tom, I wish I had read 'Why isn't my indexed being used' section of your book before asking stupid (make that semi-stupid :) questions. Pages 303-307 really say a lot, in very simple terms. Something really neat and USEFUL I learned was your trick about: "alter table .. disable table lock;" Thanks a lot!!

is (col,1) a fbi

bob, October 31, 2003 - 2:28 pm UTC

"For the optimizer to use function based indexes, the following session or system variables must be set:
QUERY_REWRITE_ENABLED=TRUE
QUERY_REWRITE_INTEGRITY=TRUSTED"

you mentioned that you cannot do this in 8.1.5 because it is an FBI, which wasn't available.

tom if index on t(col,1) was an FBI, why does a select count(*) from table where column is null still continue to use the index if QUERY_REWRITE_ENABLED=FALSE? (or is false not the correct way to turn it off)

i am using 9.2







Tom Kyte
November 01, 2003 - 12:30 pm UTC

it will do a fast full scan just to count rows. it doesn't even look at the data, it just knows that any index that is assured to return a NON NULL VALUE (as t(col,1) would -- it always returns 1) has an entry for each row.

I doesn't need query rewrite to make that leap, it is a "safe" operation.

Why?

A reader, December 06, 2004 - 10:35 am UTC

1. Why are entirely NULL entries not stored in a b*tree index? Short answer: just because! But I would like a long answer, if you dont mind!

2.

create index i on t(y);
create index j on t(f(x));
gather_table_stats using method_opt=>'for all indexed columns')

For the first index, if I use a predicate like where y=1234, the CBO will know exactly how many rows satisfy it because of the histograms.

How does this work for the FBI? Are histograms relevant (do they still apply) for columns having a FBI on them?

Thanks a lot

Tom Kyte
December 06, 2004 - 12:39 pm UTC

1) they aren't, there need be no other reason really, they just aren't. I cannot be more specific than that.

(one reason is unique indexes, ansi says NULL's are all different from eachother. if we stuff nulls in a unique index -- well, it wouldn't work anymore)

2) </code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:1547006324238 <code>


Alexander Kuznetsov, December 06, 2004 - 5:10 pm UTC

Very interesting, thanks!

Regarding the orginal problem and its solution, I think it would be nice to have a check constraint (n <> -9999), just to be on the safe side.

Also since Oracle does not store NULLs in regular indexes, that frequently causes issues during migration from SQL Server to Oracle. I think having an option (i.e. STORE NULLS) in CREATE INDEX statement could simplify the migration.

What do you think?

Tom Kyte
December 07, 2004 - 9:47 am UTC


I think that not having nulls in the index is the proverbial tip of the iceberg in a migration that from that database.


RE: NULLs in index

David, March 23, 2005 - 8:46 am UTC

Tom,

Further to the above query about using FBI to include NULLs in an index for later retrieval. How about if you only ever want to query where the column is NULL ?  Ideally, I only want the rows with NULL to be in the index to make it a very small index.  Eg. The column is "processed_date" and we only want to know about "unprocessed" records in a 10m+ row table where processed_date is NULL = ~1000 rows.

How about this method (using original example script above) ...

SQL> CREATE INDEX test_f2 ON TEST(NVL2(n, NULL, 0)) PCTFREE 0 NOLOGGING;

Index created

SQL> analyze table test compute statistics;

Table analyzed

SQL> select num_rows from user_indexes where index_name = 'TEST_I';

  NUM_ROWS
----------
    990000

SQL> select num_rows from user_indexes where index_name = 'TEST_F';

  NUM_ROWS
----------
   1000000

SQL> select num_rows from user_indexes where index_name = 'TEST_F2';

  NUM_ROWS
----------
     10000


Can you see anything wrong with this approach ?  



 

Tom Kyte
March 23, 2005 - 9:16 am UTC

I use this sort of example in my talks on function based indexes -- "selectively indexing" -- yes, this is fine.

I like to hide the function in a view:

create view v as select t.*, nvl2(n,null,0) f_of_n from t;
create index on t(nvl2(n,null,0));

lets me change the function/index without touching the application and ensures the applications get the function right.

RE: Selective Indexing

David, April 19, 2005 - 3:43 am UTC

Tom,

In this example, there is only 1 distinct value in the index, namely '0'. Does this make a difference to the optimiser? Eg. When the number of rows in the selective index are far smaller than the number of rows in the table. Is there a threshold value where the index will no longer be used? Just wondering.

Also, I'd like to know more about your selective indexing talks. Do you have any slides available for download ?

Many thanks,
David.

Tom Kyte
April 19, 2005 - 7:39 am UTC

see </code> https://www.oracle.com/technetwork/issue-archive/2014/14-may/o34asktom-2162246.html <code>selective indexing

Yes, the optimizer looks at the number of rows to be retrieved from the table and the clustering factor when considering whether to use an index or not. Even if all of the index values are 0, if they select a small amount of the table (selectivity is a "table" thing) it'll be used.

A reader, October 26, 2005 - 2:53 pm UTC

How to selective index multiple columns - composite index ?

Tom Kyte
October 27, 2005 - 4:42 am UTC

are you asking "how do I determine what the order of columns in a concatenated index should be"?

Varchar2(0) and null

Bhavesh Ghodasara, October 27, 2005 - 8:53 am UTC

Hi tom,
Why null is implemented in oracle??
I read that null's size in database is 0, so doesn't store anything so save memory,
but then why not varchar2(0),
arent both same??
test case :
SQL> create table t
  2  (a  varchar2(5));

Table created.

SQL> insert into t values('');

1 row created.

SQL> insert into t values(NULL);

1 row created.

SQL> select a,vsize(a)
  2  from t;

A      VSIZE(A)
----- ---------


so here both are 0.
why Null??
BHavesh.. 

Tom Kyte
October 27, 2005 - 12:36 pm UTC

Why null is implemented in oracle??

because the standard says we must?


A null takes zero or one bytes - not always zero.


both sizes are NOT zero in your example, they are NULL which is "I don't know"

A reader, October 27, 2005 - 4:11 pm UTC


</code> https://asktom.oracle.com/Misc/oramag/on-sorts-selecting-and-selectivity.html <code>

selective indexing


the link mentions only how to do selective index for 1 column. How does one do selective indexing(conditional indexing) only for some portion of a very big partitioned table, when the index has to contain multiple columns



Tom Kyte
October 28, 2005 - 1:43 am UTC

create index t_idx on t
( case when something then C1 end,
case when something then C2 end,
case when something then C3 end
)

not any different from "one column"

A reader, October 28, 2005 - 9:48 am UTC

create index eq_Trade_Fact_test_i1 on eq_trade_fact
(case when trade_Date > '01-Jan-05' then trade_date else null end,
case when trade_Date > '01-Jan-05' then prod_level_id else null end)

When I try the above to create a composite 'selective' index on partitioned table eq_trade_Fact I am getting the error
ora-01743:only pure functions can be indexed

EQ_TRADE_FACT is a big table of size 30 GB. It is range partitioned based on trade_date column for each year. The idea is to create a index on (trade_date,prod_level_id) only for the latest partition (year 2005), so as to limit the size of the index. Also most of the queries on eq_trade_fact access only the year 2005 partition

Is it possible to create a composite index of eq_trade_Fact(trade_date,prod_level_id) only for the 2005 partition ?



Tom Kyte
October 28, 2005 - 1:11 pm UTC

you are comparing a DATE to a STRING

stop stop stop STOP doing that.


your dates are NOT deterministic.

Using a format of 'dd-mon-yy' - that would be 1995 in the year 1999, that would be 2005 in the year 2000. Oracle is saying "no fair, you are comparing to DIFFERENT THINGS"

Use 4 character years
Use a DATE FORMAT
Use TO_DATE to convert the date into a string.

A reader, October 28, 2005 - 9:52 am UTC

alter session set nls_date_format='dd-Mon-yy';

create index eq_Trade_Fact_test_i1 on eq_trade_fact
(case when trade_Date > '01-Jan-05' then trade_date end,
case when trade_Date > '01-Jan-05' then prod_level_id end);

Still getting the error - ora 01743

Tom Kyte
October 28, 2005 - 1:15 pm UTC

yy is NOT deterministic. depends on the current DATE what year 05 means!!!!

al0, October 28, 2005 - 10:20 am UTC

Why not to include TO_DATE with proper format string in your case statement?

create index eq_Trade_Fact_test_i1 on eq_trade_fact
(case when trade_Date > to_date('01-Jan-05','DD-MON-RR') then trade_date end,
case when trade_Date > to_date('01-Jan-05','DD-MON-RR') then prod_level_id end);

Really, it is a must - your index shall not depend on current NLS settings.

BTW, of which tzpe is trade_date column?

Tom Kyte
October 28, 2005 - 1:16 pm UTC

RR is NOT determistic either!!


ops$tkyte@ORA9IR2> create table t ( x date );

Table created.

ops$tkyte@ORA9IR2> create index t_idx on t( case when x > to_date('01-jan-05','dd-mon-rr') then 1 end );
create index t_idx on t( case when x > to_date('01-jan-05','dd-mon-rr') then 1 end )
                                               *
ERROR at line 1:
ORA-01743: only pure functions can be indexed


ops$tkyte@ORA9IR2> create index t_idx on t( case when x > to_date('01-jan-2005','dd-mon-yyyy') then 1 end );

Index created.

 

A reader, October 28, 2005 - 3:42 pm UTC

Thanks, Tom. It works now

I had in fact tried explicit to_date as well, but was getting the same error as you have demonstrated

Tom Kyte
October 29, 2005 - 10:44 am UTC

it is all about the YYYY format

05 - totally ambigous, changes over time, NOT DETERMINISTIC. is that 05 2005, 1905, 2105 or 5???




A reader, October 28, 2005 - 3:47 pm UTC

And it works without to_date, if it is of the format dd-mon-yyyy

So I guess yyyy is pure

Tom Kyte
October 29, 2005 - 10:47 am UTC

YYYY is deterministic.

YY, RR - they are NOT, depends on what the date was when you call them what value is returned.

function based indexes need to use deterministic functions.

Nulls and indexes

David Piazza, March 15, 2007 - 6:17 pm UTC

What doesn't the two queries below, one using a FTS, and the other an index, return the same answer?

user> CREATE table t(c number(10),d date);

Table created.

user> insert into t values (1,null);

1 row created.

user> insert into t values (2,null);

1 row created.

user> insert into t values (3,null);

1 row created.

user> insert into t values (4,null);

1 row created.

user> COMMIT;

Commit complete.

user> set autotrace on explain
user> select c,d from t where d >= sysdate and c is not null;

no rows selected


Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE
1 0 TABLE ACCESS (FULL) OF 'T'



user> create index tidx on t(d,c);

Index created.

user> select c,d from t where d >= sysdate and c is not null;

C D
---------- ---------
1
2
3
4


Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE
1 0 INDEX (RANGE SCAN) OF 'TIDX' (NON-UNIQUE)

Tom Kyte
March 16, 2007 - 3:10 pm UTC

what version do you have there?

that is obviously wrong.

I cannot reproduce

Sorry for the repeating questions

David Piazza, March 16, 2007 - 2:09 pm UTC

I tried to submit a review three times and each time my browser came back with a page not found. I just realized it HAD posted even though I got an error message. Sorry about that... When I got home I posted it in a different note I felt was a little more relevant without realizing I had already posted it three times.

oracle version is still missing ??

A reader, March 16, 2007 - 4:33 pm UTC


david - what is the oracle version you were using ?

A reader, March 16, 2007 - 4:35 pm UTC


Versions

David Piazza, March 16, 2007 - 7:52 pm UTC

Oracle 9.2.0.6.0
SunOS 5.9
Tom Kyte
March 17, 2007 - 4:09 pm UTC

should not be so, please utilize support - it is wrong.

Update

David Piazza, March 20, 2007 - 1:26 pm UTC

As a side note, I was able to reproduce on one other Sun machine and two other Windows machines. I also had someone on the East Coast try it in another environment, and they got the same results on a Sun machine and a Windows machine...
Tom Kyte
March 20, 2007 - 2:14 pm UTC

my response stands....

I get the same result as David

Nick, March 20, 2007 - 3:17 pm UTC

Same version, 9.2.0.6, on Solaris 8.

Sounds like support is the way to go.

Resolution

David Piazza, April 02, 2007 - 3:20 pm UTC

Bug 4080972. Thanks Tom...

How to do?

Mark, May 18, 2007 - 11:11 am UTC

Hello,

SELECT COL1, COL2
FROM T1
WHERE col5 is not null;

CREATE INDEX I1_T1 ON T1 (DECODE (COL5,NULL,'NULL',COL5))

SELECT COL1, COL2
FROM T1
WHERE DECODE (COL5,NULL,'NULL',COL5) = COL5

It still does not use the index. How can I make it happen so that index will be used?


Tom Kyte
May 18, 2007 - 4:25 pm UTC

you just need a concatenated index where at least one of the columns is not null - if you don't have a not null column you want in there, just use "0"

ops$tkyte%ORA10GR2> create table t as select * from all_objects;

Table created.

ops$tkyte%ORA10GR2> exec dbms_stats.gather_table_stats( user, 'T' );

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> set autotrace traceonly explain
ops$tkyte%ORA10GR2> select * from t where status is null;

Execution Plan
----------------------------------------------------------
Plan hash value: 1601196873

--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |     1 |    93 |   155   (2)| 00:00:02 |
|*  1 |  TABLE ACCESS FULL| T    |     1 |    93 |   155   (2)| 00:00:02 |
--------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("STATUS" IS NULL)

ops$tkyte%ORA10GR2> create index t_idx on t(status,0);

Index created.

ops$tkyte%ORA10GR2> select * from t where status is null;

Execution Plan
----------------------------------------------------------
Plan hash value: 470836197

-------------------------------------------------------------------------------------
| Id  | Operation                   | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |       |     1 |    93 |     1   (0)| 00:00:01 |
|   1 |  TABLE ACCESS BY INDEX ROWID| T     |     1 |    93 |     1   (0)| 00:00:01 |
|*  2 |   INDEX RANGE SCAN          | T_IDX |     1 |       |     1   (0)| 00:00:01 |
-------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - access("STATUS" IS NULL)

ops$tkyte%ORA10GR2> set autotrace off

A Clarification

Muhammad Riaz Shahid, May 25, 2007 - 8:06 am UTC

Dear Tom,

Please consider the following test case:

SQL> explain plan for
2 SELECT count(*)
3 FROM job_headers jh
4 WHERE bol_number IS NULL;

Explained.

SQL> set autot off
SQL> @explain1

----------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |
----------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 10 |
| 1 | SORT AGGREGATE | | 1 | 10 |
|* 2 | TABLE ACCESS FULL| JOB_HEADERS | 22473 | 219K|
----------------------------------------------------------

(Removed Cost (%CPU) Column to make it readable).

PLAN_TABLE_OUTPUT
-----------------------------------------------------------

2 - filter("BOL_NUMBER" IS NULL AND "SHIPCOMP_CODE"='0001')


Statistics
----------------------------
2 recursive calls
0 db block gets
92109 consistent gets
.......

SQL> create index test_idx on job_headers(bol_number,1) compute statistics;

SQL> SELECT count(*) FROM job_headers jh
WHERE bol_number IS NULL

Its still doing an FTS. Lets try to force the usage of index:

explain plan for
SELECT /*+index(jh test_idx) */ count(*) FROM job_headers jh
WHERE bol_number IS NULL

Here is the nex ecution plan (Cost & Bytes columns removed for clarity).

-----------------------------------------------------------
| Id | Operation | Name | Rows
-----------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 |
| 1 | SORT AGGREGATE | | 1 |
|* 2 | TABLE ACCESS BY INDEX ROWID| JOB_HEADERS | 22473 |
|* 3 | INDEX RANGE SCAN | TEST_IDX | 22507 |
-----------------------------------------------------------

Statistics
----------------------------
2 recursive calls
0 db block gets
71265 consistent gets

So here comes my question:
Why the optimizer is not using my index when it will have less consistent gets in using that?

More Info
I am using Oracle 10g R2 on AIX box. The job_headers table is a VPD protected table.
Tom Kyte
May 26, 2007 - 11:59 am UTC

... The job_headers table is a VPD protected table. ...


so, show us what the real query WOULD BE after the VPD is done.

for example you run

select count(*) from t where c is null;

but after VPD it is really:

select count(*)
from (select * from t where YOUR_FUNCTION)
where c is null;


the optimizer, using IO and cpu costing, decided full scan with multi-block IO more efficient than single block IO

More Data

Muhammad Riaz Shahid, May 30, 2007 - 12:02 pm UTC

Tom,
As you can see in execution plan of first query, the VPD function will add "SHIPCOMP_CODE = '0001'" to the where clause. So the final query will become like:

SELECT count(*)
FROM job_headers
WHERE BOL_NUMBER IS NULL AND SHIPCOMP_CODE='0001'
Tom Kyte
May 30, 2007 - 4:01 pm UTC

so, that explains it - isn't is self evident? What I was trying to get across is - you are not tuning a simple select count(*), there is more at work there, shipcomp_code ISN"T in your index (and probably should be) and there are lots of rows to count - so it blew off the index because of the many table access by index rowids it would have to do.

Muhammad Riaz Shahid, May 31, 2007 - 10:08 am UTC

But as you can see, using index it is doing less work (Less LOIs + Good Response time) as compared to Full Table Scan. Isn't optimizer intelligent enough to Foresee this?

Thanks in Advance
Riaz
Tom Kyte
May 31, 2007 - 10:36 am UTC

I do not see any response times, so I cannot address that - however, the optimizer uses imperfect information in order to come up with a plan that is likely the best.

You have 35k + table access by index rowids
Versus, a simple full scan using multiblock IO

it is a wash

Selective Indexing

Ritesh Malhotra, August 24, 2007 - 1:59 pm UTC

Hi Tom,

After creating a FBI on a column having 'Y' & 'N' Values the count(*) from table has started giving me the count for the 'N' Values.

SQL> Create Index DWH_RNK_PULL_IDX on DWH_RNK_PULL(Case Is_Stored When 'N' Then 'N' END) tablespace PULLER_INDEX parallel 8;
Index Created.

SQL> Select INDEX_NAME,INDEX_TYPE,BLEVEL,LEAF_BLOCKS  from user_indexes where index_name = 'DWH_RNK_PULL_IDX';

INDEX_NAME                           INDEX_TYPE                          BLEVEL LEAF_BLOCKS  ---------------------------------              ------------------------------------------    ------------- -----------------------
DWH_RNK_PULL_IDX               FUNCTION-BASED NORMAL              0                      1

SQL> Select COLUMN_EXPRESSION from user_ind_expressions where index_name = 'DWH_RNK_PULL_IDX';

COLUMN_EXPRESSION
--------------------------------------------------------------------------------
CASE  WHEN "IS_STORED"='N' THEN 'N' END

puller@HSHK>set autotrace on
puller@HSHK>select count(*) from dwh_rnk_pull;

  COUNT(*)
----------
        21


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=ALL_ROWS (Cost=1 Card=1)
   1    0   SORT (AGGREGATE)
   2    1     INDEX (FULL SCAN) OF 'DWH_RNK_PULL_IDX' (INDEX) (Cost=1
          Card=6280840)

puller@HSHK>select /*+ full(a) */ count(*) from  dwh_rnk_pull a;

  COUNT(*)
----------
   6286025


Execution Plan
----------------------------------------------------------
   0      SELECT STATEMENT Optimizer=ALL_ROWS (Cost=48729 Card=1)
   1    0   SORT (AGGREGATE)
   2    1     TABLE ACCESS (FULL) OF 'DWH_RNK_PULL' (TABLE) (Cost=4872
          9 Card=6280840)

Why the Count(*) from table is getting results from Index Scan when we know they are just string 'N' Values.

Thanks
Ritesh

Tom Kyte
August 27, 2007 - 3:53 pm UTC

I would need a full test case to reproduce this with, this does not correspond with my view of reality.

You can create an empty table, set stats on it using dbms_stats in order to reproduce for us

Bug in 10gR2

Muhammad Riaz Shahid, January 16, 2008 - 8:40 am UTC

Hi All,
Just wanted to tell, when index of type (<Col>,1) is being used by your query and your query have condition like "ROWNUM=1", you may get incorrect results (well, no result at all!!!). I have already raised a TAR on metalink regarding this. The details can be found in below URL:

https://metalink.oracle.com/metalink/plsql/f?p=110:20:7877477013137298234::::P20_TARNUM,P20_CTRYCODE:6701767.992,

Regards,

null entry

A reader, March 21, 2010 - 2:12 pm UTC

Dear Tom,

What was design consideration so that Oracle don't indexed entirely null entries into Oracle B*tree index ? Thanks

Tom Kyte
March 22, 2010 - 9:08 am UTC

it is mostly because of the "uniqueness" rule of ansi SQL.

(null,null,....,null) is not equal, nor equal to (null,null,.....,null)

so, we cannot put (null,null,....null) into a unique index - and pretty much that ruled out non-unique for us as well.

What the heck is getting indexed?

A reader, May 27, 2012 - 10:35 pm UTC

Hi Tom,

I was reading one of your articles about the importance of meta datd and NOT NULL constraints.

In your example I saw this
create index t_idx on t(object_type,0);

I had not seen a concatenated index with a literal before and tried the following.
create table test_null_idx_tab as select * from dual;
insert into test_null_idx_tab values (NULL);

I then created the following index
 create index idx_nulls on test_null_idx_tab(dummy);

 select count(*) from test_null_idx_tab;

does a full table scan. Since entirely null vales are not indexed.
I then ran
 create index idx_nulls_2 on test_null_idx_tab(1);

followed by my select again.
 select count(*) from test_null_idx_tab;

and the operation was completed using INDEX (FULL SCAN)

I am trying to work out what is actually being indexed
by
 create index idx_nulls_2 on test_null_idx_tab(1);

I can see Oracle is actually creating a function based index on my behalf but would be grateful if you could explain a bit more about how this index is working please






Tom Kyte
May 28, 2012 - 2:27 am UTC

the expression "1" is being indexed

it wouldn't be confusing if you indexed:

create index t_idx on test_null_idx_tabl( lower(dummy) );

right? lower( dummy ) is just an expression.

1 is just an expression....

rpad( dummy, 20 ) is just an expression

0 is just an expression....


it is like a function based index on a very very simple function :)

Thankyou

A reader, May 28, 2012 - 5:51 am UTC


>it wouldn't be confusing if you indexed:
>create index t_idx on test_null_idx_tabl( lower(dummy) );
>right?

light bulb moment.. thanks Tom I was way over thinking this

Oracle 11g supporting index on Null values

gopal, January 18, 2015 - 8:23 am UTC

Hi Tom,

It looks like Oracle 11g supports indexing Null values as mentioned below.
create index
idx_mgr
on
emp
(mgs asc, 1)
;
Your thought on this please.

Regards;
Gopal

More to Explore

Performance

Get all the information about database performance in the Database Performance guide.