Skip to Main Content
  • Questions
  • to_date function preventing index use

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Chris.

Asked: March 05, 2005 - 6:11 pm UTC

Last updated: March 24, 2005 - 12:24 pm UTC

Version: 10.1.0.2

Viewed 10K+ times! This question is

You Asked

Tom,

I'm hoping you can help me with the following.

The background situation is that we're fixing some code to make use of bind variables and in this particular situation we don't know the actual variables until runtime, therefore I'm making use of a ref cursor and application context. Two of the inputs are dates so when writing to the context I 'to_char' the variable to preserve the time element and I've wrapped the sql_text referencing the context in a 'to_date'.

The problem I'm seeing is the use of the to_date function is somehow changing the plan. I'm managed to put together a simple test case to reproduce my findings (no need for dynamic sql in this case)

drop table t
/
create table t as select * from all_objects
/
create index t_idx on t(status,created) <<<<----- low leading edge cardinality to increase a chance of index skip scan
/
exec dbms_stats.gather_table_stats (user,'t',cascade=>true)

ALTER SESSION SET EVENTS '10046 trace name context forever, level 12'
/
DECLARE
l_date DATE := SYSDATE - 7;
l_char VARCHAR2(20) := TO_CHAR(SYSDATE - 7,'ddmmyyyyhh24miss');
l_cnt number;
BEGIN
SELECT max(object_id)
into l_cnt
FROM t WHERE created >= l_date;

SELECT max(object_id)
into l_cnt
FROM t WHERE created >= TO_DATE(l_char,'ddmmyyyyhh24miss');
END;
/
exit
/

The first select shows I'm comparing a date to a date but I would have thought the second is pretty much the same, date to a date. The following are the relevant sections from the tkprof report.

-- first select

SELECT MAX(OBJECT_ID)
FROM
T WHERE CREATED >= :B1


call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.01 0.00 0 0 0 0
Fetch 1 0.00 0.12 2 4 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.01 0.13 2 4 0 1

Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer mode: ALL_ROWS
Parsing user id: 64 (recursive depth: 1)

Rows Row Source Operation
------- ---------------------------------------------------
1 SORT AGGREGATE (cr=4 pr=2 pw=0 time=128323 us)
3 TABLE ACCESS BY INDEX ROWID T (cr=4 pr=2 pw=0 time=128282 us)
3 #BINDEX SKIP SCAN T_IDX (cr=3 pr=2 pw=0 time=128240 us)(object id 50417)#B


-- second select

SELECT MAX(OBJECT_ID)
FROM
T WHERE CREATED >= TO_DATE(:B1 ,'ddmmyyyyhh24miss')


call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.01 0.00 0 0 0 0
Fetch 1 0.08 0.36 359 659 0 1
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 3 0.09 0.36 359 659 0 1

Misses in library cache during parse: 1
Misses in library cache during execute: 1
Optimizer mode: ALL_ROWS
Parsing user id: 64 (recursive depth: 1)

Rows Row Source Operation
------- ---------------------------------------------------
1 SORT AGGREGATE (cr=659 pr=359 pw=0 time=361152 us)
3 #BTABLE ACCESS FULL T (cr=659 pr=359 pw=0 time=361100 us)#B


This is happening on 10.1.0.2 & 9.2.0.4

Hope you can shed some light on this.

Thanks in advance,
Chris

btw - Predictive discussion - Brilliant, simply brilliant...


and Tom said...

#b#b
#b#b

The problem is that bind variable peeking is not taking place with

column <op> function(:bind)

when we do

column <op> :bind

the optimizer will "peek" at the bind variable the first time we hard parse. When you apply a function to the bind, it no longer peeks (well, in 10.2 it will and there appear to be patches that would let 9ir2/10gr1 do it as well -- bug 3668224)


You can see this by a rather simple test case:

drop table t;
create table t as select 99 id, all_objects.* from all_objects;
update t set id = 1 where rownum = 1;
create index t_idx on t(id);
exec dbms_stats.gather_table_stats( user, 'T', method_opt => 'for all indexed columns', cascade=>true );

alter session set sql_trace=true;
declare
l_n number := 1;
cursor c1 is select * from t where id = l_n;
cursor c2 is select * from t where id = to_number(to_char(l_n));
begin
open c1;
open c2;
close c1;
close c2;
end;
/


Now, you would think that both queries should have the same plan -- but if bind variable peeking doesn't happen with the to_number(to_char(:n)), then won't and in fact, what I see is:

SELECT * FROM T WHERE ID = :B1

Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 88 (recursive depth: 1)

Rows Row Source Operation
------- ---------------------------------------------------
0 TABLE ACCESS BY INDEX ROWID T (cr=0 pr=0 pw=0 time=12 us)
0 INDEX RANGE SCAN T_IDX (cr=0 pr=0 pw=0 time=8 us)(object id 80760)
********************************************************************************
SELECT * FROM T WHERE ID = TO_NUMBER(TO_CHAR(:B1 ))

Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 88 (recursive depth: 1)

Rows Row Source Operation
------- ---------------------------------------------------
0 TABLE ACCESS FULL T (cr=0 pr=0 pw=0 time=3 us)


Now, this is changed in 10gr2 and can be applied to earier releases as well. the "workaround" for you would be to not use the function to_date in the query short of applying a patch.


(and the function is not 'preventing' the index, a hint could likely be used as well to workaround)

Rating

  (9 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Thanks

Chris, March 18, 2005 - 2:02 pm UTC

Most appreciated. We'll resort to using the hint until we can patch - even though it's not on the good hint list ;-)

Tom Kyte
March 18, 2005 - 2:38 pm UTC

(or use the DATE type...)

Ah but...

Chris, March 18, 2005 - 3:51 pm UTC

I'd like to use the date type in this type of situation, but i thought this wasn't possible when using application contexts.

Tom Kyte
March 18, 2005 - 8:07 pm UTC

ahh, well that whole "bind variable peeking" thing isn't going to happen with sys_context at all.

Jeff Beal, March 18, 2005 - 5:30 pm UTC

Thanks for responding to this, Tom. I ran into this problem several months ago, and wasn't able to find any definitive information on it. Now, I've run into it again. I had guessed from what I had been seeing that this bind parameter peeking was the culprit.

I do have an interesting wrinkle to this one.

My starting query is

select patient_visit.patient_visit_rsn, patients.birthdate
from patients, patient_visit
where patient_visit.patient_rsn = patients.patient_rsn
and (patients.birthdate >=
to_date('1977-12-19','yyyy-mm-dd')
and patients.birthdate <
to_date('1977-12-20','yyyy-mm-dd'))

This takes about six seconds to run

Using the literal DATE value, I get my answer back (345 rows) practically instantly.

On a whim, I created my own my_to_date function:

It looks like this:

CREATE FUNCTION "RADWARE"."MY_TO_DATE" (dateStr IN VARCHAR2,
formatStr IN VARCHAR2)
RETURN DATE IS
BEGIN
RETURN to_date(dateStr,formatStr);
END;

When the query uses this function instead of the builtin to_date() function, I get results back in the same amount of time as with the DATE literal.

How is there a difference between my function and the built-in Oracle function?

Tom Kyte
March 18, 2005 - 8:14 pm UTC

i'm not following you here. does the query take 6 seconds or run instantly?

Jeff Beal, March 18, 2005 - 9:17 pm UTC

The query as written takes six seconds.

If I replace every occurrence of the to_date function with the DATE 'yyyy-mm-dd' syntax, it returns instantly.

If I replace every occurrence of the to_date function with the my_to_date function, it returns instantly.

Tom Kyte
March 19, 2005 - 6:49 am UTC

I'm not understanding you. show both queries.

This statement:

If I replace every occurrence of the to_date function with the DATE 'yyyy-mm-dd'
syntax, it returns instantly.

does not compute.


compare the plans as well -- what are the plans at the end of the day.

follow-up to an OTN thread ...

Gabe, March 18, 2005 - 11:16 pm UTC

Jeff:

If you're still running with cursor_sharing=similar then better mention that to save Tom from having to diagnose it himself ... sorry to interject, but that setting is rather relevant.


Custom function behaves differently

Jeff Beal, March 21, 2005 - 11:08 am UTC

Sorry about the confusion, Tom. Let me try one more time to see if I can make myself clear. First, as Gabe pointed out, I am using CURSOR_SHARING=SIMILAR. Sorry I didn't mention that before.

Now, here are the queries that I'm working with, including execution time and execution plan (from TKPROF, not EXPLAIN PLAN):

Query 1:
select patient_visit.patient_visit_rsn, patients.birthdate
from patients, patient_visit
where patient_visit.patient_rsn = patients.patient_rsn
and (patients.birthdate >=
to_date('1977-12-19','yyyy-mm-dd')
and patients.birthdate <
to_date('1977-12-20','yyyy-mm-dd'));
Exec. Time: 7.766 seconds

Rows Row Source Operation
------- ---------------------------------------------------
345 FILTER
345 TABLE ACCESS BY INDEX ROWID PATIENT_VISIT
551 NESTED LOOPS
205 VIEW
205 HASH JOIN
205 INDEX RANGE SCAN IX_PATIENTS_BIRTHDATE (object id 108091)
5029424 INDEX FAST FULL SCAN PK_PATIENTS (object id 108081)
345 INDEX RANGE SCAN IX_FK_PATIENT_VISIT_1 (object id 97460)


Query 2:
select patient_visit.patient_visit_rsn, patients.birthdate
from patients, patient_visit
where patient_visit.patient_rsn = patients.patient_rsn
and (patients.birthdate >= DATE '1977-12-19'
and patients.birthdate < DATE '1977-12-20');
Exec. Time: 0.047 seconds

Rows Row Source Operation
------- ---------------------------------------------------
345 TABLE ACCESS BY INDEX ROWID PATIENT_VISIT
551 NESTED LOOPS
205 TABLE ACCESS BY INDEX ROWID PATIENTS
205 INDEX RANGE SCAN IX_PATIENTS_BIRTHDATE (object id 108091)
345 INDEX RANGE SCAN IX_FK_PATIENT_VISIT_1 (object id 97460)


Query 3 (using the my_to_date function that I included above):
select patient_visit.patient_visit_rsn, patients.birthdate
from patients, patient_visit
where patient_visit.patient_rsn = patients.patient_rsn
and (patients.birthdate >=
my_to_date('1977-12-19','yyyy-mm-dd')
and patients.birthdate <
my_to_date('1977-12-20','yyyy-mm-dd'));
Exec. Time: 0.047 seconds

Rows Row Source Operation
------- ---------------------------------------------------
345 TABLE ACCESS BY INDEX ROWID PATIENT_VISIT
551 NESTED LOOPS
205 TABLE ACCESS BY INDEX ROWID PATIENTS
205 INDEX RANGE SCAN IX_PATIENTS_BIRTHDATE (object id 108091)
345 INDEX RANGE SCAN IX_FK_PATIENT_VISIT_1 (object id 97460)


Obviously, the performance difference between Query 1 and Query 2 is caused by the FAST FULL SCAN on the PK_PATIENTS index. Based on your original answer in this article, I can understand how "bind peeking" affects the execution plans in these two queries. What I do *not* understand is how the optimizer "gets it right" for Query #3. How is there a difference between my function and the built-in Oracle function?

Tom Kyte
March 22, 2005 - 10:51 am UTC

do you know what cursor sharing similar does?

it binds...

where col <op> function(:bind)

no bind peeking...

where col <op> :bind

bind peeking - probably the same thing as above -- search for 10053 and check out the estimated cardinalities between the two -- it'll probably show "lots" and that would confirm this.

Custom function behaves differently

Jeff Beal, March 22, 2005 - 1:37 pm UTC

Yes, I do know what CURSOR_SHARING=SIMILAR does. Getting rid of it is somewhere on my to-do list ;) Thanks for your original answer that clearly explained what was going on with the difference between Query 1 and Query 2.

I set the 10053 event, and ran the queries again, and the trace file revealed the difference between Query 1 and Query 3:

No statistics type defined for function MY_TO_DATE
No default cost defined for function MY_TO_DATE

Now I understand enough of what's going on to get on with my work.

Thanks!

Tom Kyte
March 22, 2005 - 6:04 pm UTC

forget your function for a minute, between literal without function and literal with to_date() on it, just compare the estimated cardinalities from the row sources.

Join cardinalities

Jeff Beal, March 23, 2005 - 10:36 am UTC

I'm not too familiar with how to read 10053 trace files, so forgive me if I give you the wrong numbers. I found the 'Join Cardinality' numbers for the three queries. Query 1 and Query 3 had the same cardinality: 27477. Query 3's cardinality was much lower: 147.

Thanks again for your help

Tom Kyte
March 23, 2005 - 6:05 pm UTC

that just shows that bind variable peeking was the likely culprit (that is all the 10053 trace will do -- show us that this is the issue)

so, I would confirm this as an issue with bind variable peeking.

Now, are you using this user defined function? You can associate selectivity with them:
</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:1547006324238#28800949028084 <code>



Custom function behaves differently

Jeff Beal, March 24, 2005 - 12:24 pm UTC

We won't be using the user defined 'my_to_date' function in our application. It was just something that I had tried out while trying to figure out why Query 1 and Query 2 were behaving so differently. Thanks for the pointer to the artical on associating statistics with functions, though. That will probably come in handy down the road.

More to Explore

Performance

Get all the information about database performance in the Database Performance guide.