Skip to Main Content
  • Questions
  • Left join with filter condition is not working as supposed

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, }ilvinas.

Asked: October 25, 2016 - 7:21 am UTC

Last updated: October 28, 2016 - 1:52 pm UTC

Version: Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

Viewed 10K+ times! This question is

You Asked

Hello,

One of my colleagues asked a question like "when I insert query result to table, I found records that meet my criteria, but when I put the same filter on the same query without inserting rows to table, I did not get those records". After little investigation I managed to minimise problem to Dual table:

SELECT *
FROM Dual a
WHERE a.Dummy < 'Y'

X

SELECT *
FROM Dual a
LEFT JOIN Dual b ON b.Dummy = a.Dummy AND b.dummy > 'Y'
WHERE a.Dummy < 'Y'

no rows selected

Left join should not reduce rows returned by query, but it does.

Seems like query rewrite did not the right thing.
Ok lets assume that condidtion "b.dummy > 'Y'" did not touch table a and Oracle puts it to the where clause(but I think it is not right thing to do)
Let's rewrite query to clearly state that filter must be applied not after left join:

SELECT *
FROM Dual a
LEFT JOIN (SELECT b.Dummy FROM Dual b WHERE b.Dummy > 'Y') b ON b.Dummy = a.Dummy
WHERE a.Dummy < 'Y

no rows selected

SELECT *
FROM Dual a
LEFT JOIN (SELECT /*+ NO_MERGE*/ * FROM Dual b WHERE b.Dummy > 'Y') b ON b.Dummy = a.Dummy
WHERE a.Dummy < 'Y'

X

NO_QUERY_TRANSFORMATION hint does the same thing.
Hint should not change query result, but it changes.

I've checked Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

Works the same.
Is there some bugfix to the problem?


and Chris said...

Oracle is generating the transitive closure of predicates. i.e. it can see that:

a = b (+) and a = 10 
then b(+) = 10


See MOS note 68979.1 for more details.

This means that Oracle transforms a query like:

create table t (
  x int
);

insert into t values (1);

select * from t t1
left join t t2
on     t1.x = t2.x
and    t2.x > 2
where  t1.x < 2;


To:

select "T1"."X" "X","T2"."X" "X"
from "CHRIS"."T" "T1","CHRIS"."T" "T2"
where "T1"."X" <2
and "T1"."X"   ="T2"."X"(+)
and "T2"."X"(+)>2
and "T2"."X"(+)<2;


This gives a plan like:

---------------------------------------+-----------------------------------+
| Id  | Operation            | Name    | Rows  | Bytes | Cost  | Time      |
---------------------------------------+-----------------------------------+
| 0   | SELECT STATEMENT     |         |       |       |     1 |           |
| 1   |  FILTER              |         |       |       |       |           |
| 2   |   HASH JOIN OUTER    |         |     1 |    26 |     5 |  00:00:01 |
| 3   |    TABLE ACCESS FULL | T       |     1 |    13 |     2 |  00:00:01 |
| 4   |    TABLE ACCESS FULL | T       |     1 |    13 |     2 |  00:00:01 |
---------------------------------------+-----------------------------------+
Predicate Information:
----------------------
1 - filter(NULL IS NOT NULL)
2 - access("T1"."X"="T2"."X")
3 - filter("T1"."X"<2)


Note the "NULL IS NOT NULL" filter at the start. This is always false. The optimizer uses this to bypass the execution tree below it. Which is why there's no result!

So, yes this looks like a bug. I've raised it internally.

But!

Your query makes little sense! You have conflicting conditions on the outer joined table. Notice that the transformed query checks whether X is < 2 and > 2. This is impossible. So you'll never get any values from this. You may as well write:

select x, null from t
where  x < 2


Either that, or change your queries so you don't have conflicting conditions on the join column(s)!

So really you need to fix your SQL.

Rating

  (3 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

}ilvinas Vidmantas, October 28, 2016 - 7:32 am UTC


I've done that investigation and found myself "null is not null" in the execution plan.
I understand that condition is contradictory. I fully understand how and why this happened.
But it is OUTER JOIN not INNER JOIN. And in this case Oracle SHOULD obey what is written in the query.
In my real life case there is no reason to rewrite query.
It works well because in real working environment there is no last contradictory where clause.
That clause was written by developer only for testing purposes.
I think it is definitely a bug and very big and awfull bug.
It sholud be fixed as soon as possible.

Do you remember your presentation called "SQL magic" in Riga conference?
Let me be your follower ;)

Prepare for magic:

CREATE OR REPLACE VIEW v AS
WITH a AS(SELECT LEVEL - 10 a FROM Dual CONNECT BY LEVEL < 20),
b AS(SELECT LEVEL - 10 a FROM Dual CONNECT BY LEVEL < 20)
SELECT a.a, NVL(a.a*b.a, -1) s
FROM a
LEFT JOIN b ON b.a = a.a AND b.a > 0

Let shot starts!

1. "SIGN(a) = -1" IS THE same AS "a < 0". Don't you agree with me? Ok lets see:
SELECT * FROM v WHERE a < 0

no rows selected

SELECT * FROM v WHERE SIGN(a) = -1

a s
-- --
-4 -1
-3 -1
-7 -1
-9 -1
-1 -1
-6 -1
-2 -1
-8 -1
-5 -1

Wow! Isn't it nice?

2. The following query is too complex AND can be rewriten with more simple but semanticaly identical? Isn't it identical? Lets see what Oracle thinks about it:
SELECT * FROM v WHERE a < 0
UNION ALL
SELECT * FROM v WHERE a = 0

a s
-- --
0 -1

SELECT * FROM v WHERE a <= 0

a s
-- --
-4 -1
-3 -1
-7 -1
-9 -1
-1 -1
-6 -1
0 -1
-2 -1
-8 -1
-5 -1

Where all these records with values less than zero gone it first query? Magic?

3. Following query should return only records where a = 0 shouldn't it? Lets see what Oracle thinks about it:

SELECT * FROM v WHERE a <= 0
MINUS
SELECT * FROM v WHERE a < 0

a s
-- --
-9 -1
-8 -1
-7 -1
-6 -1
-5 -1
-4 -1
-3 -1
-2 -1
-1 -1
0 -1

I clearly stated: "MINUS every record that is less than zero". Seems that Oracle is not obedient one

4. Now lets do some ETL stuff. The following query states that there's no records with a less than zero

SELECT * FROM v WHERE a < 0

no rows selected

CREATE TABLE c0mp1ex_name AS SELECT * FROM v

SELECT * FROM c0mp1ex_name WHERE a < 0

a s
-- --
-4 -1
-3 -1
-7 -1
-9 -1
-1 -1
-6 -1
-2 -1
-8 -1
-5 -1

WTF!!!
I just created table using query that not returns any records below zero!
Where those records appeared from?
If it would be insert statement then I could suspect some trigger on table, but this was a CTAS!
Maybe some other person was so quick to insert those rocords for me, but how he cold know the name of table I just created???
Ha! I'm mentalist. I can read your mind and know table name you will tell me in advance!

Now I feel real magician like you was in Riga :)

Lets assume that view is written by some vendor of the system and I have not right to modify the view. I can just only query it.
I cannot believe that all this magic stuff is a feature and not a bug.

Chris Saxon
October 28, 2016 - 9:47 am UTC

I agreed with you! It's a bug.

Lets assume that view is written by some vendor of the system and I have not right to modify the view.

If you're on 12c you could use the SQL translation framework to intercept and rewrite the query! ;)

https://connormcdonald.wordpress.com/2016/10/20/translating-sql-a-migration-tool/

}ilvinas Vidmantas, October 28, 2016 - 10:41 am UTC

Sorry,

I thought that "looks like a bug... But!" means that it is not a bug.
I agree that such query is a little bit strange. But not completely witless.

In this case logic is as follows: you want to filter out table before outer joining it and filter out results of main query afterwards. It has some sense. Of course you can outer join it without filtering and after that in select clause write "case when a < 0 then null else Field_1 end". But if there are many fields it is tedious to write that "case ..." stuff for every column. When you filter table before outer joining is is enough to write field_1 to get the same result.

It's very strange that such a bug is still here. Can't believe that nobody complained about it.

Very nice stuff about Translating SQL. Never new about it. Thank you.

Show that "magic" stuff to your colleagues and do not tell how it works right away. They won't believe you at first. It should be quite funny :)

You can decorate a little bit this "magic" stuff and add to your next presentation of "SQL Magic"

Chris Saxon
October 28, 2016 - 1:52 pm UTC

Your second example makes more sense :)

It's still unusual though - I can't recall seeing real SQL like that. Which is probably why no-one else has complained about it.

I'm not going to make a habit of exposing bugs as "magic"! The SQL translation framework on the other hand... Now, that is magic :)

To ilvinas

Gh, October 29, 2016 - 5:54 am UTC

Does
SELECT a.a, NVL(a.a*b.a, -1) s
FROM a
LEFT JOIN b ON b.a = a.a AND b.a > 0

Equivalent to

SELECT a.a, NVL(a.a*b.a, -1) s
FROM a
LEFT JOIN b ON b.a = a.a
Where b.a > 0

So try both and feedback.


Also : NVL(a.a*b.a, -1) does always mean a <0?

More to Explore

Performance

Get all the information about database performance in the Database Performance guide.