Skip to Main Content

Breadcrumb

Warning

Before you submit your comment...

Remember - this is about feedback to the original question. AskTOM is about building a strong Oracle community, not stealing time away from others needing assistance by sneaking in a brand new question under the guise of "Comment".

If your comment requires a response that might include a query, please supply very simple CREATE TABLE and INSERT statements. Nice simple scripts so we can replicate your issue simply.

Remember to take a look at Legal Notices and Terms of Use before supplying a comment...

Don't forget, your comments are public. If you want to send a comment to just the AskTOM team, please use the feedback option

Comment

Highlight any SQL, PL/SQL, or fixed-width text and click the <code> button
 (will never be published or used to spam you)

Question and Answer

Tom Kyte

Thanks for the question, Kartik.

Asked: June 17, 2003 - 4:54 pm UTC

Last updated: August 04, 2011 - 7:21 pm UTC

Version: 8.1.7

Viewed 10K+ times! This question is

You Asked

I have a table that contains 2 million transaction records. A procedure reads these transaction records and creates an output record in the RESULT table. Usually for each record in the TRANSACTION table there is a record in the RESULT table. Even though all the SQL queries and the code has been optimized, to the best of our ability, the procedure take a very long time to run.

I thought of splitting the transaction table into four tables (T1,T2,T3,T4), each containing 500,000 records. There would be a main procedure (PROC_MAIN)that would spawn four instances of the procedure (P1, P2, P3 and P4) using DBMS_JOB package. This would create four RESULT tables (R1,R2,R3,R4).

1. In the main procedure (PROC_MAIN) how do I check if all the four jobs (P1,P2,P3,P4) have completed?
2. The Main procedure (PROC_MAIN) has to wait until all the four instances are complete because I need to consolidate the RESULT tables (R1,R2,R3,R4) into a single table and generate some reports. All this has to be automated without any manual intervention. Any suggestions?

and Tom said...

I use a technique like the following:

create or replace procedure simulation( p_procedure in varchar2, p_jobs in number, p_iters in number )
authid current_user
as
l_job number;
l_cnt number;
begin
for i in 1 .. p_jobs
loop
begin
execute immediate 'drop table t' || i;
exception
when others then null;
end;
execute immediate 'create table t' || i || ' ( x int )';
end loop;

for i in 1 .. p_jobs
loop
dbms_job.submit( l_job, p_procedure || '(JOB);' );
insert into job_parameters
( jobid, iterations, table_idx )
values ( l_job, p_iters, i );
end loop;

statspack.snap;
commit;
loop
dbms_lock.sleep(30);
select count(*) into l_cnt from job_parameters;
exit when (l_cnt = 0);
end loop;
statspack.snap;
end;
/


I have a parameter table which I populate with inputs to the "back ground" processes.

After I commit, the jobs start running -- I just "poll" the job parameter table till they are done...


Now, you don't need to split the table into 4 -- you can just use ROWID ranges. If you use this script I call "split"

-------------------------------------------------


set verify off
define TNAME=&1
define CHUNKS=&2


select grp,
dbms_rowid.rowid_create( 1, data_object_id, lo_fno, lo_block, 0 ) min_rid,
dbms_rowid.rowid_create( 1, data_object_id, hi_fno, hi_block, 10000 ) max_rid
from (
select distinct grp,
first_value(relative_fno)
over (partition by grp order by relative_fno, block_id
rows between unbounded preceding and unbounded following) lo_fno,
first_value(block_id )
over (partition by grp order by relative_fno, block_id
rows between unbounded preceding and unbounded following) lo_block,
last_value(relative_fno)
over (partition by grp order by relative_fno, block_id
rows between unbounded preceding and unbounded following) hi_fno,
last_value(block_id+blocks-1)
over (partition by grp order by relative_fno, block_id
rows between unbounded preceding and unbounded following) hi_block,
sum(blocks) over (partition by grp) sum_blocks
from (
select relative_fno,
block_id,
blocks,
trunc( (sum(blocks) over (order by relative_fno, block_id)-0.01) /
(sum(blocks) over ()/&CHUNKS) ) grp
from dba_extents
where segment_name = upper('&TNAME')
and owner = user order by block_id
)
),
(select data_object_id from user_objects where object_name = upper('&TNAME') )
/

-------------------------------- eof --------------------


and run it

SQL> @split T 4

it'll produce something like:

big_table@ORA920LAP> @split big_table 4

GRP MIN_RID MAX_RID
---------- ------------------ ------------------
0 AAAHchAAJAAAAAJAAA AAAHchAAJAAAA4ICcQ
1 AAAHchAAJAAAA4JAAA AAAHchAAJAAABwICcQ
2 AAAHchAAJAAABwJAAA AAAHchAAJAAACgICcQ
3 AAAHchAAJAAACgJAAA AAAHchAAJAAADgICcQ


those are 4 non-overlapping rowid ranges that complete "cover" the table -- so, you can query:


for x in ( select /*+ FIRST_ROWS */ * from t where rowid between X and Y )
loop

and just pass in a rowid pair for X and Y. that way, you can use the above query to generate 4 rowid ranges, insert them into the job parameter table and run 4 copies of your procedure. they'll each process about 1/4th of the table.

Rating

  (75 ratings)

Is this answer out of date? If it is, please let us know via a Comment