You Asked
I have a table that contains 2 million transaction records. A procedure reads these transaction records and creates an output record in the RESULT table. Usually for each record in the TRANSACTION table there is a record in the RESULT table. Even though all the SQL queries and the code has been optimized, to the best of our ability, the procedure take a very long time to run.
I thought of splitting the transaction table into four tables (T1,T2,T3,T4), each containing 500,000 records. There would be a main procedure (PROC_MAIN)that would spawn four instances of the procedure (P1, P2, P3 and P4) using DBMS_JOB package. This would create four RESULT tables (R1,R2,R3,R4).
1. In the main procedure (PROC_MAIN) how do I check if all the four jobs (P1,P2,P3,P4) have completed?
2. The Main procedure (PROC_MAIN) has to wait until all the four instances are complete because I need to consolidate the RESULT tables (R1,R2,R3,R4) into a single table and generate some reports. All this has to be automated without any manual intervention. Any suggestions?
and Tom said...
I use a technique like the following:
create or replace procedure simulation( p_procedure in varchar2, p_jobs in number, p_iters in number )
authid current_user
as
l_job number;
l_cnt number;
begin
for i in 1 .. p_jobs
loop
begin
execute immediate 'drop table t' || i;
exception
when others then null;
end;
execute immediate 'create table t' || i || ' ( x int )';
end loop;
for i in 1 .. p_jobs
loop
dbms_job.submit( l_job, p_procedure || '(JOB);' );
insert into job_parameters
( jobid, iterations, table_idx )
values ( l_job, p_iters, i );
end loop;
statspack.snap;
commit;
loop
dbms_lock.sleep(30);
select count(*) into l_cnt from job_parameters;
exit when (l_cnt = 0);
end loop;
statspack.snap;
end;
/
I have a parameter table which I populate with inputs to the "back ground" processes.
After I commit, the jobs start running -- I just "poll" the job parameter table till they are done...
Now, you don't need to split the table into 4 -- you can just use ROWID ranges. If you use this script I call "split"
-------------------------------------------------
set verify off
define TNAME=&1
define CHUNKS=&2
select grp,
dbms_rowid.rowid_create( 1, data_object_id, lo_fno, lo_block, 0 ) min_rid,
dbms_rowid.rowid_create( 1, data_object_id, hi_fno, hi_block, 10000 ) max_rid
from (
select distinct grp,
first_value(relative_fno)
over (partition by grp order by relative_fno, block_id
rows between unbounded preceding and unbounded following) lo_fno,
first_value(block_id )
over (partition by grp order by relative_fno, block_id
rows between unbounded preceding and unbounded following) lo_block,
last_value(relative_fno)
over (partition by grp order by relative_fno, block_id
rows between unbounded preceding and unbounded following) hi_fno,
last_value(block_id+blocks-1)
over (partition by grp order by relative_fno, block_id
rows between unbounded preceding and unbounded following) hi_block,
sum(blocks) over (partition by grp) sum_blocks
from (
select relative_fno,
block_id,
blocks,
trunc( (sum(blocks) over (order by relative_fno, block_id)-0.01) /
(sum(blocks) over ()/&CHUNKS) ) grp
from dba_extents
where segment_name = upper('&TNAME')
and owner = user order by block_id
)
),
(select data_object_id from user_objects where object_name = upper('&TNAME') )
/
-------------------------------- eof --------------------
and run it
SQL> @split T 4
it'll produce something like:
big_table@ORA920LAP> @split big_table 4
GRP MIN_RID MAX_RID
---------- ------------------ ------------------
0 AAAHchAAJAAAAAJAAA AAAHchAAJAAAA4ICcQ
1 AAAHchAAJAAAA4JAAA AAAHchAAJAAABwICcQ
2 AAAHchAAJAAABwJAAA AAAHchAAJAAACgICcQ
3 AAAHchAAJAAACgJAAA AAAHchAAJAAADgICcQ
those are 4 non-overlapping rowid ranges that complete "cover" the table -- so, you can query:
for x in ( select /*+ FIRST_ROWS */ * from t where rowid between X and Y )
loop
and just pass in a rowid pair for X and Y. that way, you can use the above query to generate 4 rowid ranges, insert them into the job parameter table and run 4 copies of your procedure. they'll each process about 1/4th of the table.
Rating
(75 ratings)
Is this answer out of date? If it is, please let us know via a Comment