Skip to Main Content
  • Questions
  • Scalar Functions Not Allowed in ORDER BY Clause of Compound Queries

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Ben.

Asked: March 02, 2023 - 9:22 pm UTC

Last updated: March 07, 2023 - 3:08 pm UTC

Version: Oracle Database 19c; 19.0.0.0.0

Viewed 10K+ times! This question is

You Asked

Why are scalar functions and case statements not allowed in the ORDER BY clause of a compound query? What Oracle SQL rules are violated by attempting this?

By "compound" query, I'm referring to a query composed of two (or more) distinct queries combined using the set operators UNION, UNION ALL, INTERSECT, and MINUS.

Scalar functions and case statements are typically allowed in the ORDER BY clause, but not with a compound query. The ORDER BY clause works with a compound query, but not when it contains a scalar function or a case statement. These two statements appear to be rules in SQL, but I haven't found any documentation that addresses this specific issue one way or the other.

I'll create two similar tables then UNION them together. The ORDER BY clause with a function or CASE statement works fine with both tables when run individually, but not when unioned together. It also works with the UNION, but not if the ORDER BY contains a function.


create table myschema.vip_customers
(
firstname  varchar2(20) not null,
lastname   varchar2(20) not null,
order_ct   number(5)
);


create table myschema.musical_customers
(
firstname  varchar2(20) not null,
lastname   varchar2(20) not null,
order_ct   number(5)
);


insert into myschema.vip_customers values ('George','Washington',89);
insert into myschema.vip_customers values ('John','Adams',97);
insert into myschema.vip_customers values ('Thomas','Jefferson',1);
insert into myschema.vip_customers values ('James','Madison',9);
insert into myschema.musical_customers values ('Wolfgang','Mozart',1756);
insert into myschema.musical_customers values ('Ludwig','Beethoven',1770);
insert into myschema.musical_customers values ('Johann','Bach',1685);
insert into myschema.musical_customers values ('Pyotr','Tchaikovsky',1840);
commit;


This query succeeds:
select firstname, lastname, order_ct
from myschema.vip_customers
union all
select firstname, lastname, order_ct
from myschema.musical_customers
order by firstname


This query fails:
select firstname, lastname, order_ct
from myschema.vip_customers
union all
select firstname, lastname, order_ct
from myschema.musical_customers
order by lower(firstname)


Error message:
ORA-01785: ORDER BY item must be the number of a SELECT-list expression

It appears that all scalar functions (decode, trunc, round, nvl, length, substr, etc.) in the ORDER BY clause result in this error. I initially found the issue while attempting to use a DECODE to make customized ordering on a column, but I'm using a LOWER function in this example in hopes of simplifying the scenario (obviously LOWER won't have an impact on the given dataset). As stated above, DECODE (and others) in the ORDER BY clause works fine on each table when run individually, but unioning the tables together causes it to fail. Attempting to reword the DECODE using a CASE statement results in the same error. However, ordering simply on the column itself (by name or number) works fine. Also, unioning the first table to itself (instead of using the 2nd table) also causes the error, so the issue appears to not be due to table structure mismatch, datatype mismatch, or an attempt to ORDER on a column that doesn't exist. I'm not sure what else to rule out and I still haven't found any reason why this shouldn't succeed.

Known workaround:
Wrapping the compound query with an outer query results in it running successfully:

select * from
(
select firstname, lastname, order_ct
from myschema.vip_customers
union all
select firstname, lastname, order_ct
from myschema.musical_customers
)
order by lower(firstname)


Thanks.
-Ben

and Chris said...

This is as documented:

If the select list preceding the set operator contains an expression, then you must provide a column alias for the expression in order to refer to it in the order_by_clause.

https://docs.oracle.com/en/database/oracle/oracle-database/21/sqlrf/The-UNION-ALL-INTERSECT-MINUS-Operators.html#GUID-B64FE747-586E-4513-945F-80CB197125EE

So to sort by lower ( column ), either you need to:

- Sort in an outer query as you've done
- Use the expression in each of the union branches, alias it, and sort by this alias

For example:

select lower(firstname) lower_fname, lastname, order_ct
from vip_customers
union all
select lower(firstname) lower_fname, lastname, order_ct
from musical_customers
order by lower_fname;

LOWER_FNAME    LASTNAME       ORDER_CT    
george         Washington              89 
james          Madison                  9 
johann         Bach                  1685 
john           Adams                   97 
ludwig         Beethoven             1770 
pyotr          Tchaikovsky           1840 
thomas         Jefferson                1 
wolfgang       Mozart                1756 

Rating

  (3 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

mathguy, March 04, 2023 - 2:07 am UTC

The two solutions (using a subquery for the set operation and an outer query for ordering, and - alternatively - including the sorting expression in the SELECT clause of each component query) do not produce the same output. The OP seems to hint strongly to a preference for the former, not the latter.

In the proposed form, the second solution may not include the entire projection required by the problem; it would probably make more sense to include both the "column" and the "function(column)" expression in SELECT, but either way the solution would produce different output from the first solution.

As an aside, it seems (to me) that the OP is asking why the specification was written that way. Why does Oracle SQL, in the case of a compound query, require that "ORDER BY item must be the number of a SELECT-list expression"? The error message is misleading to begin with, since a column name or alias will do, a number is not required (we can order by FIRSTNAME rather than by 1). But that is a tangent; why only order by SELECT-list expressions? There is no really good reason; the meaning is unambiguous, and the implementation would be trivial - do exactly what we do by hand. If I am not mistaken, the SQL Standard does, or did at some point in the past, require that all ORDER BY items be included in the SELECT list for all queries, of any kind; Oracle, quite wisely, does not require that in general, and there is no particular reason to require it for compound queries.
Chris Saxon
March 06, 2023 - 3:27 pm UTC

True, selecting the expression instead of the column does give different results.

I'm unclear why you say selecting the column and the expression would give different output. You can't sort by the component queries, only the overall result of the set operation.

why only order by SELECT-list expressions? There is no really good reason; the meaning is unambiguous

I disagree - this error applies specifically to set operations (UNION, INTERSECT, etc.).

If you sort by something NOT selected, what do you do if that column/expression is not in one of the subqueries?

For example, if you do:

alter table vip_customers
  add ( junk int );
  
select firstname, lastname, order_ct
from   vip_customers
union all
select firstname, lastname, order_ct
from   musical_customers
order by junk;


There's no JUNK column in MUSICAL_CUSTOMERS, so what would this resolve to?

This restriction exists in all the major RDBMSes to my knowledge.

mathguy, March 06, 2023 - 6:46 pm UTC

I say that adding the ordering expression to the SELECT clause of the component queries will produce a different result, because now the result will include the ordering expression as an additional column (not part of the business requirement).

For example if the UNION ALL will include a column PART_ID that is a string in the format G3, G10, etc. - always upper case G followed by a positive integer - we may want to order by TO_NUMBER(SUBSTR(PART_ID, 2)) - that is, numerically by the numeric part. The result should include PART_ID, sorted as G3, G10 etc. It should not also include another column with integers 3, 10 etc.

Of course, this can be fixed by adding an outer query - but then we end up with the solution OP already had.

As to the expressions to order by - what I meant is that any expression that would be valid in SELECT should also be allowed in ORDER BY, even if that expression is not, in fact, included in SELECT. In your example, JUNK would not be valid in SELECT (and it would be rejected there already, if it were included in the SELECT list).

I view this as somewhat analogous to ordering an aggregate query. If we sum salaries by JOB in the SCOTT.EMP table, we can't order by DEPTNO - that makes perfect sense; but we can order by any expression depending only on "legal" columns - those that would be valid in the SELECT list. For example we can order by LENGTH(JOB). In this case there is no requirement that all the ORDER BY expressions be columns included in SELECT, the only condition is that the ORDER BY expressions be valid in the context. Same could apply to set operations, but it does not.
Connor McDonald
March 07, 2023 - 1:49 am UTC

Excellent summary of the issue!

ALL but not DISTINCT

A reader, March 07, 2023 - 7:58 am UTC

In the case of LOWER(), another workaround is (johann Bach):
select firstname, lastname, order_ct
from vip_customers
union all
select firstname, lastname, order_ct
from musical_customers
order by firstname;

FIRSTNAME            LASTNAME               ORDER_CT
-------------------- -------------------- ----------
George               Washington                   89
James                Madison                       9
John                 Adams                        97
Ludwig               Beethoven                  1770
Pyotr                Tchaikovsky                1840
Thomas               Jefferson                     1
Wolfgang             Mozart                     1756
johann               Bach                       1685

select firstname, lastname, order_ct
from vip_customers
union all
select firstname collate binary_ci, lastname, order_ct
from musical_customers
/*OR: union all
select firstname collate binary_ci, lastname, order_ct
from musical_customers where 1=0*/
order by firstname;

FIRSTNAME            LASTNAME               ORDER_CT
-------------------- -------------------- ----------
George               Washington                   89
James                Madison                       9
johann               Bach                       1685
John                 Adams                        97
Ludwig               Beethoven                  1770
Pyotr                Tchaikovsky                1840
Thomas               Jefferson                     1
Wolfgang             Mozart                     1756

Chris Saxon
March 07, 2023 - 3:08 pm UTC

Neat suggestion

More to Explore

SQL

The Oracle documentation contains a complete SQL reference.