ROWNUM
A reader, April 23, 2006 - 4:21 pm UTC
This is the real awesome stuff!!!
Thanks Mr. James Bond
How is it possible to not sort entire set?
naresh, April 25, 2006 - 7:24 am UTC
Hello Tom,
You said:
<quote>
select *
from ( select * from emp order by sal desc )
where ROWNUM <= 5;
This version will sort EMP by salary descending, and then give us the first five records (top-five records) we encounter. As well see in the top-N discussion coming up shortly, Oracle doesnt really sort the entire result set; it is smarter than that, but conceptually, that is what takes place.
</quote>
How is it possible to not sort the entire result set in this case? Unless all rows are sorted, how will it be known that the top 5 are at indeed at the top?
Thanks,
Naresh.
April 25, 2006 - 10:12 am UTC
at some point, I'll have to say "buy the book" :)
<quote src=Effective Oracle by Design>
Top-N Query Processing with ROWNUM
In a top-N query, you are generally interested in taking some complex query, sorting it, and then retrieving just the first N rows (the top-N rows). ROWNUM has a top-N optimization (in Oracle9i Release 1 and up only) that facilitates this type of query. It can be used to avoid a massive sort of large sets. Well discuss how it does this conceptually, and then look at an example.
Suppose you have a query in this form:
Select
From
. Where
. Order by columns
Assume this query returns a lot of data: thousands, hundreds of thousands, or more rows. However, you are interested only in the top-N, say the top-10 or top-100. There are two ways to attack this:
· Have the client application open that query and fetch just the first N rows.
· Use that query as an inline view and use ROWNUM to limit the results, as in SELECT * FROM (your_query_here) WHERE ROWNUM <= N.
The second approach is by far superior to the first for two reasons. The lesser of the two reasons is that it requires less work by the client. The database will take care of limiting the result set. The most important reason is the special processing the database can do in order to give you just the top-N rows. Using the top-N query means you have given the database extra information. You have told it, Im only interested in getting N rows; Ill never consider the rest. Now, that doesnt sound too earth-shattering until you think about sortinghow sorts work and what the server would need to do. Lets walk through the two approaches with a sample query.
select * from t order by unindexed_column;
Now, assume T is a big table, with more than one million records, and each record is fat, say 100 or more bytes. Also assume that UNINDEXED_COLUMN is, as its name implies, a column that is not indexed. We are interested in getting just the first ten rows. Oracle would do the following:
1. Full-scan T.
2. Sort T by UNINDEXED_COLUMN. This is a full sort.
3. Presumably run out of sort area memory and need to swap temporary extents to disk.
4. Merge the temporary extents back to get the first ten records when they are requested.
5. Clean up (release) the temporary extents as we are finished with them.
Now, that is a lot of I/O. Oracle has most likely copied the entire table into TEMP and written it out, just to get the first ten rows.
Next, well look at what Oracle can do conceptually with a top-N query:
select *
from (Select * from t order by unindexed_column)
where ROWNUM < :N;
In this case, Oracle will take these steps:
1. Full-scan T, as before (we cannot avoid this step).
2. In an array of :N elements (presumably in memory this time), sort only :N rows.
The first N rows will populate this array of rows in sorted order. When the N+1 row is fetched, it will be compared to the last row in the array. If it would go into slot N+1 in the array, it gets thrown out. Otherwise, it is added, sorted, to this array and one of the existing rows is discarded. Our sort area holds N rows maximum, so instead of sorting one million rows, we sort N rows.
This seeming small detail of using an array concept and just sorting N rows can lead to huge gains in performance and resource usage. It takes a lot less RAM to sort ten rows than it does to sort one million rows (not to mention TEMP space usage!).
</quote>
Mirjana, April 25, 2006 - 10:16 am UTC
Thanks
Naresh, April 26, 2006 - 12:07 am UTC
Thanks for the response Tom.
How ROWNUM Works
Ravi Vedala, May 17, 2006 - 11:39 am UTC
The best explanation of ROWNUM ever I have come across.
Thanks Tom,
r-a-v-i
order of rows in the result of inline query
A reader, May 22, 2006 - 3:50 am UTC
Relationl theory says the ORDER of rows in a table (relation) is of no significance.
Can we rely on the ORDER of rows in the result set
of INLINE QUERY/VIEWS.?
May 22, 2006 - 7:53 am UTC
</code>
http://docs.oracle.com/docs/cd/B19306_01/server.102/b14200/pseudocolumns009.htm#SQLRF00255 <code>
Yes:
<quote>
If you embed the ORDER BY clause in a subquery and place the ROWNUM condition in the top-level query, then you can force the ROWNUM condition to be applied after the ordering of the rows. For example, the following query returns the employees with the 10 smallest employee numbers. This is sometimes referred to as top-N reporting:
SELECT * FROM
(SELECT * FROM employees ORDER BY employee_id)
WHERE ROWNUM < 11;
In the preceding example, the ROWNUM values are those of the top-level SELECT statement, so they are generated after the rows have already been ordered by employee_id in the subquery.
</quote>
Small Query
Anand, May 22, 2006 - 12:16 pm UTC
Tom,
Excellent explanation regarding ROWNUM.
I have one query though -
In the order of processing steps that has been mentioned in the original reply -
"
1. The FROM and WHERE clause go first.
2. ROWNUM is assigned and incremented to each output row from the FROM/WHERE.
3. SELECT is applied.
4. GROUP BY is applied.
5. HAVING is applied.
6. It is sorted.
"
Is step 3 (SELECT) applied before step 4 (GROUP BY).
How will the following query work then -
select deptno, Max(sal)
from emp
group by deptno;
Here the group by is applied first and then we select the columns...is this correct...or something else happens.
Regards,
Anand
May 22, 2006 - 4:05 pm UTC
umm, there be no rownum in your query???
so, skip step 2 in my list and you havethe list of how it'll conceptually be processed.
Thanks for a Great Explanation about rownum
pasko, May 22, 2006 - 4:03 pm UTC
Hi Tom,
Thanks for a Great Explanation!
I have a Question related to the rownum, but using the Analytic row_number().
For example:
SELECT * FROM
(SELECT e.*, row_number() over ( order by employee_id) rn FROM employees e )
WHERE rn < 11;
So, in the above case the rows will also be ordered correctly and there will be no need to specify an extra Order-By Clause, even if i put the above SQL in a View and allow users to select from that view without using an Order-By ;
will the Order By employee_id always be guaranteed?
Thanks in advance.
May 22, 2006 - 4:14 pm UTC
that would be the wrong way though.
ANYTIME you can use rownum
use it, there are special top-n optimizations just for it
USE ROW_NUMBER() only if rownum does not suffice.
in the above, you would get the first 10 after sorting by employee_id - but the order might not be assured after that, it would probably be OK, but....
Cannot EVER avoid this step?
Doug Clark, January 24, 2007 - 1:03 pm UTC
Hey Tom,
I've read and re-read this section of your book as quoted in several answers, and also in the web-accessible part of the September/October 2006 Oracle Magazine at:
https://asktom.oracle.com/Misc/oramag/on-rownum-and-limiting-results.html I have a question about the following part:
<quote>
Next, we'll look at what Oracle can do conceptually with a top-N query:
select *
from (Select * from t order by unindexed_column)
where ROWNUM < :N;
In this case, Oracle will take these steps:
1. Full-scan T, as before (we cannot avoid this step).
2. In an array of :N elements (presumably in memory this time), sort only :N rows.
</quote>
The part that I'm uncomfortable with is the claim that "we cannot avoid" the full-scan on T. Now, I understand how this would be the case in your simplified example which specifies that we're ordering by an unindexed column. However, you never seem to get back to the case of ordering by an indexed column.
Conceptually, if we're ordering by an indexed column, the database shouldn't have to do any sorting at all, and we should be able to avoid the full-table scan altogether. However, it doesn't appear that Oracle wants to do this, as it seems to insist on doing the full-table scan. Why is this?
Is there a way to force Oracle to use an index, and short-circuit this full-table scan?