Skip to Main Content

Breadcrumb

Question and Answer

Tom Kyte

Thanks for the question, Ron.

Asked: September 07, 2002 - 7:31 pm UTC

Last updated: October 11, 2013 - 4:29 pm UTC

Version: pc oracle 9.2.0

Viewed 10K+ times! This question is

You Asked

Tom:
I am using the merge command, using the this program with this message:
MERGE INTO newemp n1
USING external_table e1
ON ( n1.empno = e1.empno )
WHEN MATCHED THEN UPDATE
SET n1.empno = e1.empno,
n1.ename = e1.ename,
n1.job = e1.job,
n1.mgr = e1.mgr,
n1.hiredate = e1.hiredate,
n1.sal = e1.sal,
n1.comm = e1.comm,
n1.deptno = e1.deptno
WHEN NOT MATCHED THEN
INSERT ( empno, ename, job, mgr, hiredate, sal, comm, deptno )
values ( e1.empno,
e1.ename,
e1.job,
e1.mgr,
e1.hiredate,
e1.sal,
e1.comm,
e1.deptno )
/
SQL> desc newemp;
Name Null? Type
----------------------------------------- -------- --------------
EMPNO NUMBER(4)
ENAME VARCHAR2(10)
JOB VARCHAR2(9)
MGR NUMBER(4)
HIREDATE DATE
SAL NUMBER(7,2)
COMM NUMBER(7,2)
DEPTNO NUMBER(2)

SQL> @ c:\oracledir\programs\merge1.sql
ON ( n1.empno = e1.empno )
*
ERROR at line 3:
ORA-00904: "N1"."EMPNO": invalid identifier

Thank You
Ron


and Tom said...

It appears to be an undocumented restriction that you cannot UPDATE any column you are MERGING on -- this works:

ops$tkyte@ORA920.US.ORACLE.COM> MERGE INTO newemp n1
2 USING external_table e1
3 ON ( n1.empno = e1.empno )
4 WHEN MATCHED THEN UPDATE
5 SET -- n1.empno = e1.empno,
6 n1.ename = e1.ename,
7 n1.job = e1.job,
8 n1.mgr = e1.mgr,
9 n1.hiredate = e1.hiredate,
10 n1.sal = e1.sal,
11 n1.comm = e1.comm,
12 n1.deptno = e1.deptno
13 WHEN NOT MATCHED THEN
14 INSERT ( empno, ename, job, mgr, hiredate, sal, comm, deptno )
15 values ( e1.empno,
16 e1.ename,
17 e1.job,
18 e1.mgr,
19 e1.hiredate,
20 e1.sal,
21 e1.comm,
22 e1.deptno )
23 /

14 rows merged.


but if you add to the ON something like "and n1.sal > 55", you would get

ON ( n1.empno = e1.empno and n1.sal > 55 )
*
ERROR at line 3:
ORA-00904: "N1"."SAL": invalid identifier




Rating

  (296 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

What about identifying the inserted/updated records?

Nitin Sood, August 22, 2003 - 9:56 am UTC

Thanks... for the clarification. When I started using MERGE about 8 months ago, I ran into this exact problem... got the answer right here!

However, is there an easy way to identify which records have been inserted versus the updated records following a MERGE execution?

This might be required if after the MERGE into a table, the inserted records have to processed in one way and the updated ones in another.


Tom Kyte
August 22, 2003 - 10:21 am UTC

well, you could certainly FLAG THEM.


when matched update set .... status = 'U'
when not matched insert ( ...., status, ... ) values ( ...., 'I', .... )


Awww, man...

Dan Kefford, August 22, 2003 - 11:43 am UTC

Too bad MERGE doesn't support the RETURNING CLAUSE:

SQL> DECLARE
  2     updated_rows_tab    DBMS_SQL.number_table ;
  3     inserted_rows_tab   DBMS_SQL.number_table ;
  4  BEGIN
  5     MERGE INTO target_tab t1
  6     USING source_tab s1 ON (t1.key_col = s1.key_col)
  7        WHEN MATCHED THEN
  8           UPDATE SET t1.key_val = s1.key_val
  9           RETURNING s1.key_col BULK COLLECT INTO updated_rows_tab
 10        WHEN NOT MATCHED THEN
 11           INSERT (key_col,
 12                   key_val)
 13           VALUES (s1.key_col,
 14                   s1.key_val)
 15           RETURNING s1.key_col BULK COLLECT INTO inserted_rows_tab ;
 16     DBMS_OUTPUT.PUT_LINE(updated_rows_tab.COUNT || ' rows updated.') ;
 17     DBMS_OUTPUT.PUT_LINE(inserted_rows_tab.COUNT || ' rows inserted.') ;
 18  END ;
 19  /
         RETURNING s1.key_col BULK COLLECT INTO updated_rows_tab
                                           *
ERROR at line 9:
ORA-06550: line 9, column 10:
PL/SQL: ORA-00905: missing keyword
ORA-06550: line 5, column 4:
PL/SQL: SQL Statement ignored

Are there any plans to support this construction in the future?

 

Tom Kyte
August 22, 2003 - 7:42 pm UTC

not that I am aware of.

here is the solutions

Bijay, August 22, 2003 - 2:25 pm UTC

set echo on
set pause on
DROP Table t1;
drop table t2;
drop sequence shop_ssq;

create table t1
( pfcntr number(6),
brand varchar2(2),
customer_number number(6),
city varchar2(24),
zip1 number (6));

INSERT INTO T1 VALUES(1,'DD',1,'USA',1);
INSERT INTO T1 VALUES(2,'DD',1,'USA',1);
INSERT INTO T1 VALUES(3,'DD',1,'USA',1);
INSERT INTO T1( PFCNTR, BRAND, CUSTOMER_NUMBER,CITY) VALUES(4,'DD',1,'BBSR');
commit;
cl scr
REM select * from t1;

create sequence shop_ssq;
select shop_ssq.nextval from dual;

create table t2
( pfcntr number(6),
shop_sk number(6) primary key,
brand varchar2(2),
customer_number number(6),
city varchar2(24),
zip1 number(6),
status varchar2(1) );

merge into t2
using ( select pfcntr, brand, customer_number, city, zip1 from t1
MINUS
select pfcntr, brand, customer_number, city, zip1 from t2 ) t1
on ( t2.pfcntr = t1.pfcntr AND t2.brand = t1.brand)
when matched
then update
set t2.customer_number = t1.customer_number,
t2.city = t1.city,
t2.zip1 = t1.zip1,
t2.status = 'Y'
when not matched
then insert ( shop_sk, pfcntr, brand, customer_number, city, zip1, status )
values ( shop_ssq.nextval, t1.pfcntr, t1.brand, t1.customer_number, t1.city, t1.zip1, 'N' );


REM 2nd day after adding 2 rows and chaging one row.
cl scr
select * from t1 order by pfcntr;
select * from t2 order by pfcntr;



update t1 set city = 'Modified to INDIA' wherE pfcntr=1;

INSERT INTO T1 VALUES(5,'DD',1,'Newly Added',1);
INSERT INTO T1 VALUES(6,'DD',1,'Newly Added',1);
commit;

merge into t2
using ( select pfcntr, brand, customer_number, city, zip1 from t1
MINUS
select pfcntr, brand, customer_number, city, zip1 from t2 ) t1
on ( t2.pfcntr = t1.pfcntr AND t2.brand = t1.brand)
when matched
then update
set t2.customer_number = t1.customer_number,
t2.city = t1.city,
t2.zip1 = t1.zip1,
t2.status = 'Y'
when not matched
then insert ( shop_sk, pfcntr, brand, customer_number, city, zip1, status )
values ( shop_ssq.nextval, t1.pfcntr, t1.brand,t1.customer_number, t1.city, t1.zip1, 'N' );
commit;
select * from t1 order by pfcntr;
select * from t2 order by pfcntr;



unique constraint violation when using Merge

A reader, January 29, 2004 - 5:11 am UTC

I am using merge statment to update/insert from one table to another table. The data is coming from flat file,
intially i load it into staging tables as such -  using sqlloader. Then i use merge to upsert with prd table (final table). 
It so happened that two records were simillar in staging table and so the merge statment failed when trying to insert.

SQL> @/u01/MWP_DB_Mart/LoadRoot/util/CPM_Customer_Merge.sql
merge into tb_prd001_customer dst using tb_stg001_cpmcustomer src
*
ERROR at line 1:
ORA-00001: unique constraint (MWPDBA.CN_PRD001_PK) violated


So, the whole load for the day got failed. What should be done to avert this problem. I dont have any validation 
at staging table. and the prd table (final table) has a unique constraint.

merge into tb_prd001_customer dst using tb_stg001_cpmcustomer src on (dst.customer_num_dsc=src.customer_num_dsc)
when matched then
update set
    dst.customer_nam=src.customer_name_dsc,
    dst.customer_address1_dsc=src.customer_address1_dsc,
    dst.customer_address2_dsc=src.customer_address2_dsc,
    dst.customer_address3_dsc=src.customer_address3_dsc,
    dst.customer_address4_dsc=src.customer_address4_dsc
when not matched then
insert (
    dst.customer_num_dsc,
    dst.customer_nam,
    dst.customer_address1_dsc,
    dst.customer_address2_dsc,
    dst.customer_address3_dsc,
    dst.customer_address4_dsc)
values (
        src.customer_num_dsc,
        src.customer_name_dsc,
        src.customer_address1_dsc,
        src.customer_address2_dsc,
        src.customer_address3_dsc,
        src.customer_address4_dsc
    );
commit;        

Please help me to resolve this issue Thanks for your consideration... 

Tom Kyte
January 29, 2004 - 8:07 am UTC

similar??? seems they were "the same" as far as the customer number was concerned. You had duplicate data.


You could filter this in the loading query but -- which record is which -- which gets loaded -- it would be rather "arbitrary".

You could...

Peter, January 29, 2004 - 9:11 am UTC

From the above it looks like you have a clear need to validate your dataload data.

Lots of things you could do - but here is a suggestion. You will need to create constraints on your staging table based on your business rules, in your case customer is unique. You will need to create an exceptions table - this is documented by Oracle. In your case you will need to create a 'reject' table like your staging table (maybe, add a couple of columns for timestamp and reject reason.
To use this setup.
1) Disable constraints on the stage table.
2) Load data.
3) Enable constraints with an "exceptions into" clause.
4) Inspect the exceptions table and copy the rows that fail validation from the stage table to your reject table, if you have the columns defined add timestamp and reason
5) Delete the distinct rows that have failed validation from the stage table. If you have more than one rule it is possible that a row fails for multiple reasons and will appear several times in the exceptions table
6) publish your clean data
7) clear down the exceptions table.
8) investigate why you have rows in the reject table.

Apart from step 8 the whole lot can be automated. We do this with our DWH dataload - the odd duff record is sidelined until support staff can investigate / fix. But the 5 million-odd good records are published. In our case we add checks to count the rejects - too many and we have a problem that alerts the duty DW team.


thanks ....

A reader, January 30, 2004 - 1:06 am UTC


Thanks a lot Peter - you made me clear on how i should approach...

tom ...
... but -- which record is which -- which gets loaded -- it would be rather "arbitrary".

i could not understand that and as well. i did not understand how merge works in this case. will it not insert the first or second one (of the dups) and then take the next and update?...

Thanks a lot



Tom Kyte
January 30, 2004 - 8:14 am UTC

if you have duplicates -- and you wanted the merge to go forward somehow without failing -- tell me, which of the two "duplicate" keys should be used. that is what i meant.

Which record

Peter, January 30, 2004 - 4:42 am UTC

Ah..

As soon as I submitted my note I thought I should have added...
If you use the enable constraints exceptions into approach you will find that find for a duplicate key error *BOTH* rows will be exceptions table. So if you remove the error rows from STAGE neither row will be there. Of course you will be able to look at both rows in your reject table and decide which one (if any!) should be there. It is up to you what to do next.

Thanks ...

A reader, January 30, 2004 - 9:51 am UTC

Ok, i got it...Thanks for sharing your thoughts ..

Thanks tom ...
Thanks peter...

One more query

A reader, February 23, 2004 - 7:32 am UTC

Hi Tom,

In my insert clause, I want something like
Insert into table1 select table2.*, sysdate from table2.

I am getting ORA-00926 when trying to do this.
But if I just run
Insert into table1 select table2.*, sysdate from table2 from SQL prompt, it works fine.

Regards,
Su

Tom Kyte
February 23, 2004 - 7:54 am UTC

umm, so where are you running it when it does not work.

Getting ORA-00926 in Merge statement

A reader, February 23, 2004 - 8:12 am UTC

Hello Tom,

My apologies for incomplete question.

I am running following Merge statement at SQL prompt.

c019074@WDMDE01A> MERGE INTO CPT_UABADJE USING DELTA_UABADJE
2 ON (CPT_UABADJE.UABADJE_AR_TRANS = DELTA_UABADJE.UABADJE_AR_TRANS)
3 WHEN MATCHED THEN UPDATE
4 SET CPT_UABADJE.UABADJE_ACTIVITY_DATE = DELTA_UABADJE.UABADJE_ACTIVITY_DATE,
5 CPT_UABADJE.UABADJE_ADJM_CODE = DELTA_UABADJE.UABADJE_ADJM_CODE,
6 CPT_UABADJE.UABADJE_ADJR_CODE = DELTA_UABADJE.UABADJE_ADJR_CODE,
7 CPT_UABADJE.UABADJE_ALLOWANCE_CONSUMPTION = DELTA_UABADJE.UABADJE_ALLOWANCE_CONSUMPTION,
8 CPT_UABADJE.UABADJE_BALANCE = DELTA_UABADJE.UABADJE_BALANCE,
9 CPT_UABADJE.UABADJE_BHST_TRAN_NUM = DELTA_UABADJE.UABADJE_BHST_TRAN_NUM,
10 CPT_UABADJE.UABADJE_BILLED_CHG = DELTA_UABADJE.UABADJE_BILLED_CHG,
11 CPT_UABADJE.UABADJE_BUDGET_VARIANCE = DELTA_UABADJE.UABADJE_BUDGET_VARIANCE,
12 CPT_UABADJE.UABADJE_CHARGE = DELTA_UABADJE.UABADJE_CHARGE,
13 CPT_UABADJE.UABADJE_CONSUMPTION_ADJ = DELTA_UABADJE.UABADJE_CONSUMPTION_ADJ,
14 CPT_UABADJE.UABADJE_DATE = DELTA_UABADJE.UABADJE_DATE,
15 CPT_UABADJE.UABADJE_END_DATE = DELTA_UABADJE.UABADJE_END_DATE,
16 CPT_UABADJE.UABADJE_EXPIRED_IND = DELTA_UABADJE.UABADJE_EXPIRED_IND,
17 CPT_UABADJE.UABADJE_FEED_DOC_CODE = DELTA_UABADJE.UABADJE_FEED_DOC_CODE,
18 CPT_UABADJE.UABADJE_FEED_SEQ_NUM = DELTA_UABADJE.UABADJE_FEED_SEQ_NUM,
19 CPT_UABADJE.UABADJE_GL_IND = DELTA_UABADJE.UABADJE_GL_IND,
20 CPT_UABADJE.UABADJE_INSTAL_IND = DELTA_UABADJE.UABADJE_INSTAL_IND,
21 CPT_UABADJE.UABADJE_OA_IND = DELTA_UABADJE.UABADJE_OA_IND,
22 CPT_UABADJE.UABADJE_ORIGIN_AR_TRANS = DELTA_UABADJE.UABADJE_ORIGIN_AR_TRANS,
23 CPT_UABADJE.UABADJE_PPD_DUE_DATE = DELTA_UABADJE.UABADJE_PPD_DUE_DATE,
24 CPT_UABADJE.UABADJE_PRINTED_DATE = DELTA_UABADJE.UABADJE_PRINTED_DATE,
25 CPT_UABADJE.UABADJE_PRINTED_IND = DELTA_UABADJE.UABADJE_PRINTED_IND,
26 CPT_UABADJE.UABADJE_REFERENCE = DELTA_UABADJE.UABADJE_REFERENCE,
27 CPT_UABADJE.UABADJE_START_DATE = DELTA_UABADJE.UABADJE_START_DATE,
28 CPT_UABADJE.UABADJE_TRANSFEROR_AR_TRANS = DELTA_UABADJE.UABADJE_TRANSFEROR_AR_TRANS,
29 CPT_UABADJE.UABADJE_USER_ID = DELTA_UABADJE.UABADJE_USER_ID,
30 CPT_UABADJE.UABADJE_VAT_REVENUE_AMT = DELTA_UABADJE.UABADJE_VAT_REVENUE_AMT,
31 CPT_UABADJE.UABADJE_W_O_BALANCE = DELTA_UABADJE.UABADJE_W_O_BALANCE
32 WHEN NOT MATCHED THEN INSERT INTO CPT_UABADJE SELECT DELTA_UABADJE.*, (SELECT RUN_DATE FROM AC_
IMPLEMENTATION_RUN WHERE RUN_STATUS = 'C') FROM DELTA_UABADJE;
WHEN NOT MATCHED THEN INSERT INTO CPT_UABADJE SELECT DELTA_UABADJE.*, (SELECT RUN_DATE FROM AC_IMPLE
*
ERROR at line 32:
ORA-00926: missing VALUES keyword

If I run just 'INSERT INTO CPT_UABADJE SELECT DELTA_UABADJE.*, (SELECT RUN_DATE FROM AC_
IMPLEMENTATION_RUN WHERE RUN_STATUS = 'C') FROM DELTA_UABADJE;', it works perfect without any error.

Not able to understand why getting error when using same insert statement as part of merge.

Thanks.

Cheers,
Su


Tom Kyte
February 23, 2004 - 8:20 am UTC

merge does not support insert as select -- it is sort of "not sensible" even.

You use insert values in merge only, merge is like a row by row subroutine. It says "take the USING set and row by row see if it exists in the INTO set. If so, do the ROW update, if not do the ROW insert"

you use insert values.

Make sure to fully reference in the VALUES clause, too

Jim Nasby, February 25, 2004 - 5:15 pm UTC

1 MERGE INTO reference_backup b
2 USING (SELECT * FROM reference) o ON (o.reference_id = b.reference_id)
3 WHEN MATCHED THEN UPDATE SET b.REFERENCE_NAME = o.REFERENCE_NAME, b.URL = o.URL
4* WHEN NOT MATCHED THEN INSERT VALUES(REFERENCE_ID, REFERENCE_NAME, URL)
ops$decibel@DEV> /
MERGE INTO reference_backup b
*
ERROR at line 1:
ORA-00904: "B"."REFERENCE_ID": invalid identifier


ops$decibel@DEV> ed
Wrote file afiedt.buf

1 MERGE INTO reference_backup b
2 USING (SELECT * FROM reference) o ON (o.reference_id = b.reference_id)
3 WHEN MATCHED THEN UPDATE SET REFERENCE_NAME = o.REFERENCE_NAME, URL = o.URL
4* WHEN NOT MATCHED THEN INSERT VALUES(o.REFERENCE_ID, o.REFERENCE_NAME, o.URL)
ops$decibel@DEV> /

3 rows merged.

ops$decibel@DEV>


Help

Ketan Popat, March 10, 2004 - 7:15 am UTC

Hi tom,

I am using

merge into emp
using dual on ( empno = 7782 )
when matched then
update set SAL = SAL+123
when not matched then
insert ( EMPNO,ENAME,deptno)
values ('123','KETAN',10);

"when matched " part of code works normally
but "But not matched " parts do not even
employee no i.e. in above case 7782 does not exists
in table EMP

regards
Ketan

Tom Kyte
March 10, 2004 - 9:22 am UTC

what do you mean the "but not matched parts do not work?"


ops$tkyte@ORA9IR2> create table t ( x int, y int, z int );
 
Table created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> insert into t values ( 1, 1, 1 );
 
1 row created.
 
ops$tkyte@ORA9IR2> commit;
 
Commit complete.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> merge into t
  2  using dual on ( x = 55 )
  3  when matched then
  4  update set z = z+1
  5  when not matched then insert (x,y,z) values ( 2, 2, 2 );
 
1 row merged.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> select * from t;
 
         X          Y          Z
---------- ---------- ----------
         1          1          1
         2          2          2

<b>so there, x=55 did not exist, the one row was inserted..</b>
 
ops$tkyte@ORA9IR2> rollback;
 
Rollback complete.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> insert into t values (55, 1, 1 );
 
1 row created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> merge into t
  2  using dual on ( x = 55 )
  3  when matched then
  4  update set z = z+1
  5  when not matched then insert (x,y,z) values ( 2, 2, 2 );
 
1 row merged.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> select * from t;
 
         X          Y          Z
---------- ---------- ----------
         1          1          1
        55          1          2

<b>there x=55 did exist and got updated</b>


although i just tested in 9.0 instead of 9.2 and it did not update, perhaps that is it?  (if so, you'll want to file a tar with support for that)
 

OK

A reader, March 25, 2004 - 9:20 am UTC

Dear Tom,
Can a single table be *merged to itself*?Do you have any simple example for that?Please do reply.
Bye!


Tom Kyte
March 25, 2004 - 8:55 pm UTC

Yes you "can"

Why you "would" escapes me :)  doesn't really seem to make sense (eg: when would you not have a "match" ?)

but anyway, simple test shows "yes"

ops$tkyte@ORA9IR2> create table t ( x int, y int );
 
Table created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> insert into t values ( 1, 1 );
 
1 row created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> merge into t t1
  2  using t t2
  3  on ( t1.x = t2.x )
  4  when matched then update set t1.y = t2.y
  5  when not matched then insert  values ( t2.x, t2.y );
 
1 row merged.
 

Excellent

Nafeesah, March 26, 2004 - 3:57 pm UTC

You are the best, Tom!

ORA-00904 after analyze

Tony, April 02, 2004 - 12:27 pm UTC

Tom,
Thanks a lot for all your help to DBA community. My question is, when I analyze the schema I start getting ORA-00904 error for some queries,

ORA-00604: error occurred at recursive SQL level 1
ORA-00904: "tab"."column1": invalid identifier

I tried to use both dbms_stats and dbms_utility to analyze the schema and used different values for estimate_percent(like 20, 40 , dbms_stats.auto_sample_size etc) but still getting the error.
if i delete the stats the query just works fine.
please help!!!

Tom Kyte
April 02, 2004 - 1:47 pm UTC

Abstract : ORA-904 WHEN DBMS_STATS.GATHER_SCHEMA_STATS IS PERFORMED.
Rediscovery Information : ORA-904 was sometimes raised when dbms_stats.gather_schema_stats was performed against the table whose column name includes 1 or more multi-byte characters.

any chance that is your case?

ORA-00904

Tony, April 02, 2004 - 1:33 pm UTC

star_transformation_enabled init.ora parameter was set to true and setting this value to false resolved the issue, but I'm still wondering why is that?

Tom Kyte
April 02, 2004 - 3:01 pm UTC

you'd want to file a tar on that -- I remember something about that, you gathered stats, it started doing a star transformation and did it "wrong" leading to the issue.

ORA-00904: invalid identifier

Botmund, April 05, 2004 - 6:29 am UTC

Excellent! I have never used MERGE before and Metalink did not have any explanation, none I found at least. Well now I know the reason.

ORA-00904 when changing column order in MERGE SELECT?

Vlado, April 07, 2004 - 4:03 pm UTC

I run the following two MERGE statements. The first executes, the second fails. The only difference between the two MERGE statements is the column order in the SELECT.
I have asked several other people about this and everybody is perplexed, since this behaviour goes against a fundamental assumption in RDBMSs, namely that the column order in a SELECT is not relevant to the internal db processing mechanisms...
I have changed the column orders a few times, and it seems like that the hedge_type_id and class_type_id have to be the first two columns in the SELECT clause.
Why?

Connected to:
Oracle9i Enterprise Edition Release 9.2.0.3.0 - 64bit Production
With the Partitioning option
JServer Release 9.2.0.3.0 - Production

msapp@MSETTST> MERGE INTO COTOTALS.FTR_SCHED9_BIDS fsb
2 USING (
3 SELECT
4 ibe.hedge_type_id hedge_type_id,
5 ibe.class_type_id class_type_id,
6 ibe.marketname,
7 ibe.orgid,
8 mnm.market_name_mapping_id market_name_mapping_id,
9 NVL(ibe.bidtotal,0) bidtotal,
10 NVL(ibe.auctionround,0) auctionround
11 FROM eftr.i_bridge_eftrbids@dbl_mset_ftr ibe,
12 ftrarr.class_type ct,
13 ftrarr.hedge_type ht,
14 ftrarr.market_name_mapping mnm
15 WHERE ibe.class_type_id = ct.class_type_id AND
16 ibe.hedge_type_id = ht.hedge_type_id AND
17 mnm.market_name_mapping_id = 8 AND
18 ibe.marketname = mnm.market_name
19 ) eftr
20 ON (
21 fsb.fk_market_name_mapping_id = eftr.market_name_mapping_id AND
22 fsb.fk_hedge_type_id = eftr.hedge_type_id AND
23 fsb.fk_class_type_id = eftr.class_type_id AND
24 fsb.fk_org_id = eftr.orgid
25 )
26 WHEN MATCHED THEN UPDATE SET
27 auction_round = eftr.auctionround,
28 bidtotal = eftr.bidtotal
29 WHEN NOT MATCHED THEN INSERT (ftr_sched9_bids_id,
30 fk_market_name_mapping_id,
31 auction_round,
32 fk_hedge_type_id,
33 fk_class_type_id,
34 fk_org_id,
35 bidtotal
36 )
37 VALUES (
38 COTOTALS.FTR_SCHED9_Bids_id.NEXTVAL,
39 8,
40 eftr.auctionround,
41 eftr.hedge_type_id,
42 eftr.class_type_id,
43 eftr.orgid,
44 eftr.bidtotal
45 );

3 rows merged.

msapp@MSETTST>
Wrote file afiedt.buf

1 MERGE INTO COTOTALS.FTR_SCHED9_BIDS fsb
2 USING (
3 SELECT
4 ibe.marketname,
5 ibe.hedge_type_id hedge_type_id,
6 ibe.class_type_id class_type_id,
7 ibe.orgid,
8 mnm.market_name_mapping_id market_name_mapping_id,
9 NVL(ibe.bidtotal,0) bidtotal,
10 NVL(ibe.auctionround,0) auctionround
11 FROM eftr.i_bridge_eftrbids@dbl_mset_ftr ibe,
12 ftrarr.class_type ct,
13 ftrarr.hedge_type ht,
14 ftrarr.market_name_mapping mnm
15 WHERE ibe.class_type_id = ct.class_type_id AND
16 ibe.hedge_type_id = ht.hedge_type_id AND
17 mnm.market_name_mapping_id = 8 AND
18 ibe.marketname = mnm.market_name
19 ) eftr
20 ON (
21 fsb.fk_market_name_mapping_id = eftr.market_name_mapping_id AND
22 fsb.fk_hedge_type_id = eftr.hedge_type_id AND
23 fsb.fk_class_type_id = eftr.class_type_id AND
24 fsb.fk_org_id = eftr.orgid
25 )
26 WHEN MATCHED THEN UPDATE SET
27 auction_round = eftr.auctionround,
28 bidtotal = eftr.bidtotal
29 WHEN NOT MATCHED THEN INSERT (ftr_sched9_bids_id,
30 fk_market_name_mapping_id,
31 auction_round,
32 fk_hedge_type_id,
33 fk_class_type_id,
34 fk_org_id,
35 bidtotal
36 )
37 VALUES (
38 COTOTALS.FTR_SCHED9_Bids_id.NEXTVAL,
39 8,
40 eftr.auctionround,
41 eftr.hedge_type_id,
42 eftr.class_type_id,
43 eftr.orgid,
44 eftr.bidtotal
45* )
msapp@MSETTST> /
fsb.fk_class_type_id = eftr.class_type_id AND
*
ERROR at line 23:
ORA-00904: "EFTR"."CLASS_TYPE_ID": invalid identifier


msapp@MSETTST>

Tom Kyte
April 08, 2004 - 9:43 am UTC

if the only difference is in fact the order of the columns in the select -- then you have the perfect test case for support. Please file a tar on this.

Akash from India

Akash Birari, April 08, 2004 - 10:31 am UTC

Thanx TOM ..!!!
It is very helpful.

HI

A reader, May 06, 2004 - 5:14 pm UTC

If I want to just update using MERGE statement and ignore the INSERT part, can I do that?
In other words, "when not matched" is not required for me.

Thanks

Tom Kyte
May 07, 2004 - 7:00 am UTC

In 10g yes, in 9i no

Another question??

A reader, May 06, 2004 - 7:19 pm UTC

I have a situation to update a table "test" using MERGE utility on "oid" column using a second table "new". The test table has multiple values of oid for a given client.
I want to use merge utility and update only for a given client. I need to give a condition with in merge statement saying "where client = 1".

Is it possible?

Thanks in advance.

Tom Kyte
May 07, 2004 - 7:05 am UTC

if you want to update -- why would you not use update?


update ( select client_columns, new_columns...
from client, new
where client.id = 1
and client.oid = new.oid )
set client_columns = new_columns;



MERGE INTO ... clause : can we use variables within this clause?

Yogesh, May 20, 2004 - 11:05 am UTC

Hi Tom,

Thanks for running such a wonderful forum.

I am using MERGE INTO ... clause within a stored procedure. And it uses variables.

When executed, it gives an error 'ORA-03114: not connected to ORACLE'.

Here are the scripts:
I am using scott schema.
I have one new table and a procedure:
------
create table BONUSES
(
EMPNO NUMBER,
BONUS NUMBER
) ;

create or replace procedure y_test2 IS
v_sample_size number;
v_just_number number;
BEGIN
v_sample_size := 0 ;
v_just_number := 1 ;

MERGE INTO bonuses D
USING (SELECT empno, sal, deptno FROM emp
WHERE deptno = 80
and sal > v_sample_size /* if you remove this clause */
) S
ON (D.empno = S.empno)
WHEN MATCHED THEN UPDATE SET D.bonus = D.bonus + S.sal*.01
WHEN NOT MATCHED THEN INSERT (D.empno, D.bonus)
VALUES (S.empno, ( S.sal*0.1 * v_just_number /*if you remove this variable*/ ) );

COMMIT ;
END ;

--------

if you remove the variables from the MERGE clause, it will work fine.

I think there is some mistake on my side, but I am not able to find it.

Your thoughts are welcome, as always.

Thanks,
Yogesh

Tom Kyte
May 20, 2004 - 12:01 pm UTC

you'll need to file a tar with support on that one.

scott@ORA9IR2> exec y_test2;

PL/SQL procedure successfully completed.


might be a known problem with your release.

MERGE INTO ... clause : can we use variables within this clause? - oracle version

Yogesh, May 21, 2004 - 10:55 am UTC

It doesnt work on: (dev box)
Personal Oracle9i Release 9.2.0.1.0 - Production

And it works on: (qa box)
Oracle9i Enterprise Edition Release 9.2.0.4.0 - Production

Is there a 9.2.0.4.0 available in Personal Oracle9?

I tried to visit the download page, but I couldnt find the exact release. I think I will have to download and see which one is it.
Plus, my laptop has Windows 2000 OS (hmmmm!)

Thanks,
Yogesh

Tom Kyte
May 21, 2004 - 11:27 am UTC

you'd contact support for patch information (metalink)




A reader, June 01, 2004 - 4:16 pm UTC

How can the Merge functionality be implemented in 8.1.7?

Tom Kyte
June 01, 2004 - 4:56 pm UTC

takes two steps

update ( select table_merging_into_columns (eg: existing table),
table_containing_updates_columns
from table_merging_into, table_containing_updates
where ... )
set table_merging_into_columns = table_containing_updates_columns
/

insert into table_merging_into
select *
from table_containing_updates
where (primary_key) NOT IN ( select primary_key from table_merging_into);



more on unique constraint violation when using Merge January 29, 2004

DD, June 01, 2004 - 5:16 pm UTC

Hi Tom,
Had a query on an earlier post. "unique constraint violation when using Merge January 29, 2004"
Seems that if duplicate keys exist in the source table then merge would fail? What about this situation- the population of the source table is done online. But moving of the data to the final target is a batched affair. Suppose 2 records with the same key did sneak in, could one use merge? As to why such a case would happen, well, say one column contained incremental values and that had to be aggregated in the target table.
I apologise for what may be an obvious query, but I'm not able to try this myself, we're shifting to 9i later this year.

Tom Kyte
June 01, 2004 - 5:23 pm UTC

no, merge protects against such a thing

think about it - given that there is no such thing as "order" -- it is just a set, which row should be applied?????????

the problem would be that given the same set of data -- two different end results could/would occur just by changing the order of insertion of rows in a source table.

if they had to be aggregated -- you would merge into ( select ...., sum(....) group by key )

you would aggregate and then merge the aggregate into the other table.

Executing more than one statement WHEN MATCHED

External Table and Merge user, June 08, 2004 - 11:54 am UTC

Say I have a table TEST with columns
CODE varchar2(3),
DESCRIPTION varchar2(15),
START_DATE date,
END_DATE date.

I am using an external table that inserts/updates data in the table test.

Can I use MERGE to do

When matched, update end_date = SYSDATE
AND insert into test(code, description, start_date, end_date)
values(code, description, sydate, null)
when not matched, insert into test.....

Tom Kyte
June 08, 2004 - 1:19 pm UTC

nope.

Update, Insert and Delete?

A reader, June 14, 2004 - 5:38 pm UTC

I have a table I would like synced up with a external table (file).

merge
when matched then update ...
when not matched then insert ....
ELSE DELETE ...

In other words, the file is the master source, so after all the merging if my database table has any extra rows, get rid of them. How can I do this? Thanks


Tom Kyte
June 15, 2004 - 8:17 am UTC

you'd need a "delete where not in" afterwards.

perhaps more efficient to

a) truncate
b) insert /*+ append */ as select * from et;



MERGE Inserting NULLS!!

Andy, August 16, 2004 - 2:54 pm UTC

I have a base table and a "Merge" Table. I want to update the Base on certain key. If there's a match then update all fields , if there isn't any the Insert NOTHING. Now , to do this I wrote something like this :

merge into stg_test a
using ( select CENDID ,
BUISNESSUNITID ,
BRANDID ,
BUISNESSUNITCUSTOMERID ,
MAILOPTINFLAG ,
PHONEOPTINFLAG ,
EMAILOPTINFLAG ,
TITLE ,
LASTNAME ,
FIRSTNAME ,
MIDDLEINITIAL
from merg_test ) b
on ( a.BUISNESSUNITCUSTOMERID = b.BUISNESSUNITCUSTOMERID )
when matched then update set CENID = '66666666666'
when not matched then insert (CENDID) values (NULL)

But as expected, it inserted a row having columns values as NULL. But I don't want that. Is there a way we can say to MERGE to do nothing if it doesn't find and MATCHES. Can you suggest a way around ? We want to use the MERGE as its the MOST EFFICIENT(after taking statistics) way to go. Any suggestions ?

Thanks and Regards

Tom Kyte
August 16, 2004 - 7:52 pm UTC

in 10g, you have this option.. In 9i, you have to make sure the "using" set has only rows to merge.


merge into stg_test a
using ( select CENDID ,
BUISNESSUNITID ,
BRANDID ,
BUISNESSUNITCUSTOMERID ,
MAILOPTINFLAG ,
PHONEOPTINFLAG ,
EMAILOPTINFLAG ,
TITLE ,
LASTNAME ,
FIRSTNAME ,
MIDDLEINITIAL
from merg_test where BUISNESSUNITCUSTOMERID
in (select BUISNESSUNITCUSTOMERID from stg_test) b
on ( a.BUISNESSUNITCUSTOMERID = b.BUISNESSUNITCUSTOMERID )
when matched then update set CENID = '66666666666'
when not matched then insert (CENDID) values (NULL)


However, since this is simple table to table -- and you only want to update, just:

update ( select b.cenid
from merg_test a, stg_test b
where b.BUISNESSUNITCUSTOMERID = a.BUISNESSUNITCUSTOMERID )
set cenid = '66666666666';

much easier if you just want to merge updates -- just update the join!

But what about Error!!

Andy, August 17, 2004 - 12:29 pm UTC

Tom,
What about ORA-01779: cannot modify a column which maps to a non key-preserved table ERROR , if I do that ?

Regards

Tom Kyte
August 17, 2004 - 2:02 pm UTC

merge_test needs a unique or primary key on the join column.

that join column *must be* unique else the update (and merge) are ambigous -- so it cannot possible hurt to have the constraint in place to verify it.

Error Persists even though I have UNQUE Recs on Key Preserved Columns

Andy, August 17, 2004 - 2:47 pm UTC

CREATE TABLE STG_TEST
(
CENID VARCHAR2(22),
BUISNESSUNITID VARCHAR2(12),
BRANDID VARCHAR2(12),
BUISNESSUNITCUSTOMERID VARCHAR2(20),
MAILOPTINFLAG VARCHAR2(1))
)
/
CREATE TABLE MERG_TEST
(
CENID VARCHAR2(22),
BUISNESSUNITID VARCHAR2(12),
BRANDID VARCHAR2(12),
BUISNESSUNITCUSTOMERID VARCHAR2(20),
MAILOPTINFLAG VARCHAR2(1))
/
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'1', 'CCRG', 'ABCD', '171593', 'Y');
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'2', 'CCRG', 'ABCD', '1372', 'Y');
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'3', 'CCRG', 'ABCD', '1374', 'Y');
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'4', 'CCRG', 'ABCD', '1381', 'Y');
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'5', 'CCRG', 'ABCD', '1386', 'Y');
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'6', 'CCRG', 'ABCD', '1388', 'Y');
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7', 'CCRG', 'ABCD', '2444520', 'Y');
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'8', 'CCRG', 'ABCD', '2444536', 'Y');
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'9', 'CCRG', 'ABCD', '2444542', 'Y');
INSERT INTO MERG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'10', 'CCRG', 'ABCD', '2813344', 'Y');
COMMIT;
/
INSERT INTO STG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7777777777', 'CCRG', 'AVI', '171593', 'Y');
INSERT INTO STG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7777777777', 'CCRG', 'AVI', '1372', 'Y');
INSERT INTO STG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7777777777', 'CCRG', 'AVI', '1374', 'Y');
INSERT INTO STG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7777777777', 'CCRG', 'AVI', '1381', 'Y');
INSERT INTO STG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7777777777', 'CCRG', 'AVI', '1386', 'Y');
INSERT INTO STG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7777777777', 'CCRG', 'AVI', '1388', 'Y');
INSERT INTO STG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7777777777', 'CCRG', 'AVI', '2444520', 'Y');
INSERT INTO STG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7777777777', 'CCRG', 'AVI', '2444536', 'Y');
INSERT INTO STG_TEST ( CENID, BUISNESSUNITID, BRANDID, BUISNESSUNITCUSTOMERID,
MAILOPTINFLAG ) VALUES (
'7777777777', 'CCRG', 'AVI', '2444542', 'Y');
COMMIT;

The above is the script with repective insert statement's.

Ideally I would want the cendid & BRANDID from merg_test to be updated to stg ? Now if I write the query as you have said I get the same ORA error. Even if I write it as :

update ( select b.CENID
from merg_test a, stg_test b
where b.BUISNESSUNITCUSTOMERID = a.BUISNESSUNITCUSTOMERID )
set cenid = '66666666666';

Just to see if at all I could update, but got the same :
ORA-01779 error even if my Merg Table has got all UNIQUE BUISNESSUNITCUSTOMERID.

Your help is much appreciated.

Regards



Tom Kyte
August 17, 2004 - 3:31 pm UTC

I believe you've made a mistake somewhere, show me the entire example:

ops$tkyte@ORA9IR2> CREATE TABLE STG_TEST
  2  (
  3    CENID                   VARCHAR2(22),
  4    BUISNESSUNITID              VARCHAR2(12),
  5    BRANDID                     VARCHAR2(12),
  6    BUISNESSUNITCUSTOMERID      VARCHAR2(20),
  7    MAILOPTINFLAG               VARCHAR2(1))
  8  /
 
Table created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> CREATE TABLE MERG_TEST
  2  (
  3    CENID                   VARCHAR2(22),
  4    BUISNESSUNITID              VARCHAR2(12),
  5    BRANDID                     VARCHAR2(12),
  6    BUISNESSUNITCUSTOMERID      VARCHAR2(20) <b>UNIQUE,</b>
  7    MAILOPTINFLAG               VARCHAR2(1))
  8  /
 
Table created.
 
 
ops$tkyte@ORA9IR2> update ( select b.CENID
  2             from merg_test a, stg_test b
  3            where b.BUISNESSUNITCUSTOMERID = a.BUISNESSUNITCUSTOMERID )
  4  set cenid = '66666666666';
 
9 rows updated.
 
 

Sorry !!

arin_am@yahoo.com, August 17, 2004 - 3:55 pm UTC

My mistake I did not take you words "literally" (making the column UNIQUE). Now its updated. My related question was , if I needed to update values from columns of MERGE TABLE and not hard code what should be the way ? Please help.

Regards

Tom Kyte
August 17, 2004 - 4:35 pm UTC

update ( select table_to_update.columns,
table_to_update_from.columns
from ... )
set table_to_update.columns = table_to_update_from.columns;



Updating the old value with new value

Logan Palanisamy, August 17, 2004 - 4:23 pm UTC

Here you go:

SQL> update ( select b.brandid oldbrand, a.brandid newbrand
  2              from merg_test a, stg_test b
  3             where b.BUISNESSUNITCUSTOMERID = a.BUISNESSUNITCUSTOMERID )
  4  set oldbrand = newbrand; 

THANK YOU!!!!

Andy, August 18, 2004 - 9:49 am UTC

Thanks Tom and Logan for your help !!

Performance of Merge

Nags, August 18, 2004 - 11:37 am UTC

What would be the performance impact of using the MERGE. Would doing separate insert and update be faster or MERGE ?

We have a datawarehouse, where when required we do a complete refresh by truncating the tables and reloding. Would truncate and reload be faster or MERGE ?

We did do some testing, but would like to know your opinion to have a better understanding of the working of MERGE. Is it like MERGE works like a procedure ?

Tom Kyte
August 18, 2004 - 11:49 am UTC

truncate and reload would most likely be faster than merge if most all of the table is going to be modified. the reload could be done in direct path, without indexes -- rebuild indexes in parallel with nologging afterwards.


merge does an outer join between the two sets basically and updates matches as it hits them and inserts misses. it is not like a "procedure", it is like insert, update or delete.

Interpreting Trace Analyzer output for a Merge statement

Vlado Barun, September 01, 2004 - 5:39 pm UTC

Tom,

I have some questions in regards to interpreting the trace analyzer output for a Merge Statement. I have tried to find answers to these questions on your site and the Oracle docs but no luck (I might have missed it :(()

1. If you look at the top most line in the explain plan it states that 31.15 seconds elapsed to complete that step (and all the children steps). However, If you look at the total line above the explain plan (Call, Parse, Execute, Total) the elapsed time for the whole query is 136.77 seconds. How can I find out what the remaining 105.62 seconds were spend on?
2. In the explain plan, what does the "Buffer Sort" step do?
3. In the explain plan, does the "Sequence" step means that it took 8.74 seconds to retrieve the nextval values?
4. In the explain plan, what does the "View" step mean?
5. In the explain plan, what does the "Merge" step mean?

BTW, the merge, insert & update statement refer to views which are just simple queries against single tables (no joins) and with no where clause.

We are using 9.2.0.3.0.

Also, I understand that a way to tune this merge statement is to move it out of the loop, however, due to the complexity of the code that might not be possbile.

Thanks in advance.

MERGE INTO bus.net_POSITION np
USING (
SELECT :b10 AS BUS_ID
, :b9 AS ORG_ID
, (NVL(:b5, 0) * (:b6 / :b11))
+ (NVL(:b7, 0) * (:b8 / :b11)) AS Withdrawl_MW
, (NVL(:b1, 0) * (:b2 / :b11))
+ (NVL(:b3, 0) * (:b4 / :b11)) AS Injection_MW
FROM dual
) mg
ON (
hour_date = :b14
AND fk_settlement_type_id = :b13
AND fk_bus_id = mg.bus_id
AND fk_org_id = mg.org_id
AND dst_flag = :b12
)
WHEN MATCHED THEN
UPDATE /*+ INDEX (NP NDX_NET_POSITION2_T) */
SET
withdrawl_mw = Round(np.withdrawl_mw + mg.Withdrawl_MW,9),
injection_mw = Round(np.injection_mw + mg.Injection_MW,9)
WHEN NOT MATCHED THEN
INSERT
(
net_position_id
, hour_date
, fk_settlement_type_id
, dst_flag
, fk_bus_id
, fk_org_id
, withdrawl_mw
, injection_mw
)
VALUES
(
bus.net_position_id.NEXTVAL
, :b14
, :b13
, :b12
, mg.bus_id
, mg.org_id
, mg.Withdrawl_MW
, mg.Injection_MW
)

call count cpu elapsed disk query current rows misses
------- ----- ------ ------- ---- ------ ------- ------ ------
Parse 1 0.00 0.01 0 4 0 0 1
Execute 819 139.53 136.76 0 482209 285035 68386 0
------- ----- ------ ------- ---- ------ ------- ------ ------
total 820 139.53 136.77 0 482213 285035 68386 1

| Rows Row Source Operation
|------ ---------------------------------------------------
|136772 MERGE (cr=482209 pr=0 pw=0 time=31.15)
| 68386 .VIEW (cr=482197 pr=0 pw=0 time=22.32)
| 68386 ..SEQUENCE (cr=482197 pr=0 pw=0 time=20.11)
| 68386 ...MERGE JOIN OUTER (cr=478778 pr=0 pw=0 time=11.37)
| 68386 ....TABLE ACCESS FULL DUAL (cr=205158 pr=0 pw=0 time=4.02)
| 68386 ....BUFFER SORT (cr=273620 pr=0 pw=0 time=5.24)
| 68386 .....PARTITION RANGE SINGLE PARTITION: KEY KEY (cr=273620 pr=0 pw=0 time=3.64)
| 68386 ......TABLE ACCESS BY LOCAL INDEX ROWID NET_POSITION_DEC_2003_PLUS_T PARTITION: KEY KEY (cr=273620 pr=0 pw=0 time=3.11)
| 68386 .......INDEX UNIQUE SCAN NDX_NET_POSITION2_T PARTITION: KEY KEY (cr=205158 pr=0 pw=0 time=2.16)

Event Times Count Max. Total Blocks
waited on Waited Zero Time Wait Waited Accessed
--------------------------- --------- --------- --------- --------- ---------
log file switch completion. 2 0 0.05 0.06
latch free (156)........... 1 0 0.00 0.00
latch free (097)........... 2 0 0.00 0.00
--------------------------- --------- --------- --------- --------- ---------
total...................... 5 0 0.05 0.07 0

non-idle waits............. 5 0 0.05 0.07 0
idle waits................. 0 0 0.00 0.00


Tom Kyte
September 01, 2004 - 8:44 pm UTC

i don't use the trace analyzer -- can I see the original tkprof which should have unadulterated information/timings?



requested tkprof output

Vlado Barun, September 02, 2004 - 12:15 pm UTC

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 819 135.42 132.34 0 478790 278145 68386
Fetch 0 0.00 0.00 0 0 0 0
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 820 135.42 132.35 0 478790 278145 68386

Misses in library cache during parse: 1
Optimizer goal: CHOOSE
Parsing user id: 61 (recursive depth: 1)

Rows Row Source Operation
------- ---------------------------------------------------
136772 MERGE (cr=482209 r=0 w=0 time=31145823 us)
68386 VIEW (cr=482197 r=0 w=0 time=22320461 us)
68386 SEQUENCE (cr=482197 r=0 w=0 time=20108370 us)
68386 MERGE JOIN OUTER (cr=478778 r=0 w=0 time=11366231 us)
68386 TABLE ACCESS FULL DUAL (cr=205158 r=0 w=0 time=4019515 us)
68386 BUFFER SORT (cr=273620 r=0 w=0 time=5240891 us)
68386 PARTITION RANGE SINGLE PARTITION: KEY KEY (cr=273620 r=0 w=0 time=3636818 us)
68386 TABLE ACCESS BY LOCAL INDEX ROWID NET_POSITION_DEC_2003_PLUS_T PARTITION: KEY KEY (cr=273620 r=0 w=0 time=3109204 us)
68386 INDEX UNIQUE SCAN NDX_NET_POSITION2_T PARTITION: KEY KEY (cr=205158 r=0 w=0 time=2158108 us)(object id 18284)


Elapsed times include waiting on following events:
Event waited on Times Max. Wait Total Waited
---------------------------------------- Waited ---------- ------------
log file switch completion 2 0.05 0.06
latch free 3 0.00 0.00
********************************************************************************


Tom Kyte
September 02, 2004 - 1:37 pm UTC

ah, -- easy enough.

add them up. the steps are not rolled up.

you'll not get "exact matches", but it'll be alot closer (for example, it is technically impossible for cpu time to EXCEED elapsed time -- they are all using different clocks -- all have small errors, and if you have millions of small errors, they tend to look larger).


view them as something that could be "ratioed" -- what % of time was spent on which operation sort of numbers.

The reason they won't add up is:

start1 = gettimeofday
for i in 1 .. big_number
loop
start2 = gettimeofday
fast process
total2 = total2 + (getimeofday-start2)
end loop
total1 = (getimeofday-start1)



Lets say the clock measures in 1/1,000th of a second.

Say "fast process" takes about 45/10,000th of a second.

sometimes it'll measure as 4/1,0000 -- sometimes at 5/1,000 of a second. IT WILL NEVER MEASURE as 4.5/1,000 of a second.

total2 will have the potential for some errors. total1 will too -- but of a much much smaller degree.

Add in the fact that the elapsed time is wall clock (and has a granularity of N) and the cpu is system accounting information (and has a finer granularity) and that we are measuring different things at different levels -- and well, things can get off by a bit.



What about 8.1.7

Harendra, September 07, 2004 - 4:20 pm UTC

Hi there,
MERGE is cool.
I am working with oracle 8.1.7. So unfortunately I cannot use merge.
Is there any existing thread which has samples for the same logic(update if exists or else insert) in 8.1.7? Please point me to the discussion.

Thanks in advance



Tom Kyte
September 07, 2004 - 4:26 pm UTC

you can either:

update (the join of T and the table to "merge with")
set....

insert into t
select * from table_to_merge_with
where join_key not in (select join_key from t)

commit;


or write procedural code to read from table to merge with, try to update and if no rows updated -- insert.

re: tkprof output

Vlado Barun, September 07, 2004 - 5:11 pm UTC

I understand that I will not get "exact matches" etc.

However, taking a step back, I wonder why the plan execution steps are not rolled up? Is that true only for merge statements?

The fact that the steps are not rolled up makes it even more important that I understand each step in the plan so I can tune them properly. Can you please answer the following questions or point me to some documentation(I tried to find documentation explaining these particular plan steps, but no luck).
1. In the explain plan, what does the "Buffer Sort" step do?
2. In the explain plan, does the "Sequence" step means that it took 20.11 seconds to retrieve the nextval values?
3. In the explain plan, what does the "View" step mean?
4. In the explain plan, what does the "Merge" step mean?

Tom Kyte
September 08, 2004 - 7:48 am UTC

researching further, well, it is actually a hierarchy here. so you can use this to see "where the low hanging fruit is" and from that "where the lowest hanging fruit in that fruit" is.

meaning -- the subtrees sort of add up to their parent nodes, but the parent nodes will in general be larger because they contribute their own times and so on. but each introduces it's own timing errors.

The longer the run time, the more discrete steps "timed", the more the error becomes. I just did a quick test -- in the single digit second runtime range, the diff was under 3/10s of a second. Getting into the double digit runtime range (10-15 seconds), the difference was about 2-2.5 seconds. Getting into the 30 time range, the difference was in the 7 second range. when I took it up into the triple digits, the difference was 40-50 seconds (given my specific example, with my specific steps). So, the difference your are seeing is that "we are timing things at very different granularities -- there is the elapsed time of the statement versus the elapsed time of individiually times steps of the statement.

what was interesting was that the ratios (individual time= with respect to the overall time for a subtree) remained more or less constant. That is, this is useful to find the low hanging fruit step of a plan.

let me see the MERGE statement that generated this (which answers #4, you were merging - it is the statement type)

re: tkprof merge output

Vlado Barun, September 08, 2004 - 9:28 am UTC

Below is the merge statement that produced the tkprof output.

In regards to your explanation of the timing errors, I understand that, but what I'm surprised about is that the errors are significantly higher than for update/insert/select/delete statements, at least based on my experience in using traces...

Thank you for helping me work this out.

MERGE INTO bus.net_POSITION np
USING (
SELECT :b10 AS BUS_ID
, :b9 AS ORG_ID
, (NVL(:b5, 0) * (:b6 / :b11))
+ (NVL(:b7, 0) * (:b8 / :b11)) AS Withdrawl_MW
, (NVL(:b1, 0) * (:b2 / :b11))
+ (NVL(:b3, 0) * (:b4 / :b11)) AS Injection_MW
FROM dual
) mg
ON (
hour_date = :b14
AND fk_settlement_type_id = :b13
AND fk_bus_id = mg.bus_id
AND fk_org_id = mg.org_id
AND dst_flag = :b12
)
WHEN MATCHED THEN
UPDATE /*+ INDEX (NP NDX_NET_POSITION2_T) */
SET
withdrawl_mw = Round(np.withdrawl_mw + mg.Withdrawl_MW,9),
injection_mw = Round(np.injection_mw + mg.Injection_MW,9)
WHEN NOT MATCHED THEN
INSERT
(
net_position_id
, hour_date
, fk_settlement_type_id
, dst_flag
, fk_bus_id
, fk_org_id
, withdrawl_mw
, injection_mw
)
VALUES
(
bus.net_position_id.NEXTVAL
, :b14
, :b13
, :b12
, mg.bus_id
, mg.org_id
, mg.Withdrawl_MW
, mg.Injection_MW
)


Tom Kyte
September 08, 2004 - 10:28 am UTC

The BUFFER SORT operation can be used by optimizer when it thinks that temporarily storing the input row source and sorting it by a key column might be useful in eventually doing join; this would make the join efficient in terms of IO. This is no different from an ordinary sort in terms of overhead.

the MERGE step is just saying "hey, this be a merge" (as opposed to a select or insert...)


the VIEW is just a "we created a result set at this step" sort of process.

the sequence step does seem to be a special "going for a sequence" step there (if you remove the nextval, it goes away).


You might be able to dramatically reduce this:

68386 TABLE ACCESS FULL DUAL (cr=205158 r=0 w=0 time=4019515 us)

by using an IOT with one row you create yourself, or by using sys.x$dual


but all in all -- I would probably use the logic of:


update
if (sql%rowcount = 0)
then
insert
end if


UNLESS i was array executing this -- which you don't seem to be doing. If you feed single values in to this and execute -- update/insert might be more efficient.

re: tkprof merge output

Vlado Barun, September 08, 2004 - 4:31 pm UTC

Thank you for the explanations.
BTW, what do you mean by "array executing"?

Tom Kyte
September 08, 2004 - 7:09 pm UTC

array binding -- like you can fill up an array with say 100 values and:


exec sql update t set x = :x_array where y = :y_array;

and in one fell swoop that would execute that sql update 100 times.

In plsql, you would:

forall i in 1 .. array.count
delete from t where x = array(i);

for example -- one execute call -- but the server does it array.count times.

re: tkprof merge output

Vlado Barun, September 09, 2004 - 10:43 am UTC

Actually, that's what I'm doing, array binding.
What I don't understand is how that would make a difference in choosing whether to use the merge or update/insert approach, because you wrote
"...but all in all -- I would probably use the logic of:
update
if (sql%rowcount = 0)
then
insert
end if


UNLESS i was array executing this -- which you don't seem to be doing. If you
feed single values in to this and execute -- update/insert might be more
efficient. "

Can you explain?

Tom Kyte
September 09, 2004 - 12:11 pm UTC

i based that on 819 executes. true, you could have executed that 819 times with "50 array elements"


like this:

declare
type array is table of number;
l_data array := array(1,2,3,4,5);
begin
for j in 1 ... 5
loop
forall i in 1 .. l_data.count
merge into t using ( select l_data(i) y from dual ) d
on ( t.x = d.y )
when matched then update set z = 5
when not matched then insert (x,z) values (d.y,4);
end loop;
end;
/


that would show 5 executes, 25 rows modified.



re: tkprof merge output

Vlado Barun, September 09, 2004 - 12:45 pm UTC

But why would you use the update/insert approach assuming I don't use array binding?


Tom Kyte
September 09, 2004 - 12:49 pm UTC

68386 MERGE JOIN OUTER (cr=478778 r=0 w=0 time=11366231 us)
68386 TABLE ACCESS FULL DUAL (cr=205158 r=0 w=0 time=4019515 us)
68386 BUFFER SORT (cr=273620 r=0 w=0 time=5240891 us)

because we can nuke an outer join (needed by merge) and 1/4 million LIOs against dual and tons of "buffer sorts"...

the update would just:


68386 PARTITION RANGE SINGLE PARTITION: KEY KEY (cr=273620 r=0 w=0
time=3636818 us)
68386 TABLE ACCESS BY LOCAL INDEX ROWID NET_POSITION_DEC_2003_PLUS_T
PARTITION: KEY KEY (cr=273620 r=0 w=0 time=3109204 us)
68386 INDEX UNIQUE SCAN NDX_NET_POSITION2_T PARTITION: KEY KEY
(cr=205158 r=0 w=0 time=2158108 us)(object id 18284)




Merge does not work over db link

Thiyag, September 16, 2004 - 4:29 am UTC

Hi Tom, I have the following statement:

MERGE INTO emp@dblink n1
USING emp e1
ON ( n1.empno = e1.empno )
WHEN MATCHED THEN UPDATE
SET n1.ename = e1.ename,
n1.job = e1.job,
n1.mgr = e1.mgr,
n1.hiredate = e1.hiredate,
n1.sal = e1.sal,
n1.comm = e1.comm,
n1.deptno = e1.deptno
WHEN NOT MATCHED THEN
INSERT ( empno, ename, job, mgr, hiredate, sal, comm, deptno )
values ( e1.empno,
e1.ename,
e1.job,
e1.mgr,
e1.hiredate,
e1.sal,
e1.comm,
e1.deptno )
When I try out this statement it gives ORA-00904: "e1"."empno": invalid identifier. Can you please let me know what is the problem. I have Oracle 9.2.0.4.


Tom Kyte
September 16, 2004 - 8:14 am UTC

ops$tkyte@ORA9IR2> MERGE INTO emp@dblink n1
  2  USING emp e1
  3  ON ( n1.empno = e1.empno )
  4  WHEN MATCHED THEN UPDATE
  5      SET n1.ename    = e1.ename,
  6          n1.job      = e1.job,
  7          n1.mgr      = e1.mgr,
  8          n1.hiredate = e1.hiredate,
  9          n1.sal      = e1.sal,
 10          n1.comm     = e1.comm,
 11          n1.deptno   = e1.deptno
 12  WHEN NOT MATCHED THEN
 13  INSERT ( empno, ename, job, mgr, hiredate, sal, comm, deptno )
 14  values ( e1.empno,
 15           e1.ename,
 16           e1.job,
 17           e1.mgr,
 18           e1.hiredate,
 19           e1.sal,
 20           e1.comm,
 21           e1.deptno )
 22  /
 
14 rows merged.


hows about an exact cut and paste and a santity check in place to verify that emp has an empno column 
 

Correct Statement simulating the error

Thiyag, September 16, 2004 - 6:11 am UTC

Tom, Please ignore my previous query I posted. Here is the query which simulates the error:

MERGE INTO emp@dblink n1
USING (select ename, job, mgr, hiredate, sal, comm,
deptno, empno from emp) e1
ON ( n1.empno = e1.empno )
WHEN MATCHED THEN UPDATE
SET n1.ename = e1.ename,
n1.job = e1.job,
n1.mgr = e1.mgr,
n1.hiredate = e1.hiredate,
n1.sal = e1.sal,
n1.comm = e1.comm,
n1.deptno = e1.deptno
WHEN NOT MATCHED THEN
INSERT ( empno, ename, job, mgr, hiredate, sal, comm, deptno )
values ( e1.empno,
e1.ename,
e1.job,
e1.mgr,
e1.hiredate,
e1.sal,
e1.comm,
e1.deptno )
If the rearrage the "empno" column to be the first column in the USING statement it works fine. Can you share your insights why it throws error when the empno is the last column. Hence the problem is not due to presence of dblink

Tom Kyte
September 16, 2004 - 8:45 am UTC

confirmed, i filed a bug for that. thanks!

A reader, September 16, 2004 - 3:58 pm UTC

Tom
which is faster.

insert into Huge_table select * from stage

or
use merge

We need to do this on a 90 million table
in order to do a reorg

Thanks in advance

Tom Kyte
September 16, 2004 - 7:26 pm UTC

if you mean to insert, by all means, insert.

a reorge would be "alter table T move" wouldn't it?

if huge_table is empty, rename would apply even.

A reader, September 17, 2004 - 9:45 am UTC

We will be eliminating some rows while insert

Tom Kyte
September 17, 2004 - 10:17 am UTC

then by all means insert.

merge would not even be remotely applicable.

Merge a single row

Ofir Manor, October 10, 2004 - 12:31 pm UTC

Hi Tom, 
Good to have you back. Thanks for the detailed discussion.
I disagree with you about your preference to the structure of
 update
 if (sql%rowcount = 0)
 then 
     insert
 end if
when merging a single value.
First of all, there is a big difference between this structure and a merge command. As far as I understand, MERGE is an atomic Oracle operation, as opposed to this structure. If two sessions runs this piece of code, it may clash - both updates might return 0 rows, and then both sessions will try to insert the same row and one will fail on primary key constraint. This alone wins the day.
Second, if the code is ran outside of Oracle (not a pl/sql), it involves two network round trips, which might negate the performance advantage.

Anyway, there is a simple way to replace the MERGE JOIN OUTER step with NESTED LOOPS OUTER without hints - works for my on 9.2.0.5. I took it from Ketan Popat response earlier in this post.
CREATE TABLE qq_ofir (pk NUMBER PRIMARY KEY, n NUMBER, v VARCHAR2(30));
INSERT INTO qq_ofir VALUES (1, 11,'AAA');
INSERT INTO qq_ofir VALUES (2, 11,'BBB');                      
INSERT INTO qq_ofir VALUES (3, 33,'CCC');

SQL> MERGE INTO qq_ofir q 
  2    USING (SELECT 2 pk, 22 n, 'CCC' v FROM DUAL) v ON (q.pk = v.pk)
  3    WHEN matched     THEN UPDATE SET n = v.n
  4    WHEN NOT matched THEN INSERT (pk, n, v) VALUES (v.pk, v.n, v.v);

1 row merged.

Execution Plan
----------------------------------------------------------
   0      MERGE STATEMENT Optimizer=CHOOSE (Cost=4096 Card=8168 Bytes=465576)
   1    0   MERGE OF 'QQ_OFIR'
   2    1     VIEW
   3    2       MERGE JOIN (OUTER) (Cost=4096 Card=8168 Bytes=408400)
   4    3         TABLE ACCESS (FULL) OF 'DUAL' (Cost=11 Card=8168)
   5    3         BUFFER (SORT) (Cost=4085 Card=1 Bytes=50)
   6    5           TABLE ACCESS (BY INDEX ROWID) OF 'QQ_OFIR' (Cost=2 Card=1 Bytes=50)
   7    6             INDEX (UNIQUE SCAN) OF 'SYS_C008908' (UNIQUE)

Statistics
----------------------------------------------------------
          0  recursive calls
          1  db block gets
          5  consistent gets
          0  physical reads
        236  redo size
        622  bytes sent via SQL*Net to client
        691  bytes received via SQL*Net from client
          3  SQL*Net roundtrips to/from client
          2  sorts (memory)
          0  sorts (disk)
          1  rows processed

SQL> MERGE INTO qq_ofir q 
  2    USING dual ON (q.pk = 2)
  3    WHEN matched     THEN UPDATE SET n = 22
  4    WHEN NOT matched THEN INSERT (pk, n, v) VALUES (2,22, 'BBC');

1 row merged.

Execution Plan
----------------------------------------------------------
   0      MERGE STATEMENT Optimizer=CHOOSE (Cost=16347 Card=8168 Bytes=351224)
   1    0   MERGE OF 'QQ_OFIR'
   2    1     VIEW
   3    2       NESTED LOOPS (OUTER) (Cost=16347 Card=8168 Bytes=424736)
   4    3         TABLE ACCESS (FULL) OF 'DUAL' (Cost=11 Card=8168 Bytes=16336)
   5    3         VIEW (Cost=2 Card=1 Bytes=50)
   6    5           TABLE ACCESS (BY INDEX ROWID) OF 'QQ_OFIR' (Cost=2 Card=1 Bytes=50)
   7    6             INDEX (UNIQUE SCAN) OF 'SYS_C008908' (UNIQUE) (Cost=1 Card=1)


Statistics
----------------------------------------------------------
          0  recursive calls
          1  db block gets
          8  consistent gets
          0  physical reads
        236  redo size
        622  bytes sent via SQL*Net to client
        648  bytes received via SQL*Net from client
          3  SQL*Net roundtrips to/from client
          1  sorts (memory)
          0  sorts (disk)
          1  rows processed


Besides, I think that the second SQL is cleaner and easier to read, it is just straight forward (no join).
What do you think?
  Ofir 

Tom Kyte
October 10, 2004 - 6:29 pm UTC

that merge is "single statment" doesn't really impact the "This alone wins the day" thing.

concurrency control is a transactional thing -- under the covers, you know what merge does?

The statement:

<quote>
If two sessions runs this piece of code
</quote>

applies equally to "merge" as well as "update/insert".  Think about it! 

both merges will not find a row.
both merges will attemp to insert a new row.
only one of the merges will be successful!!!!!

The 'second' merge (if there is such a thing in a SMP machine even) cannot see the 'insert' of the first -- read consistency doesn't permit it.  Only one of the merges would 'win' (try it -- rather than just disagreeing with me -- 'show it to be better')..

consider:

ops$tkyte@ORA9IR2> drop table t;
 
Table dropped.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> create table t ( x int primary key, y int );
 
Table created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> merge into t
  2  using ( select 1 a, 2 b from dual ) D
  3  on ( t.x = d.a )
  4  when not matched then insert (x,y) values (a,b)
  5  when matched then update set y = b;
 
1 row merged.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> declare
  2          pragma autonomous_transaction;
  3  begin
  4          merge into t
  5          using ( select 1 a, 2 b from dual ) D
  6          on ( t.x = d.a )
  7          when not matched then insert (x,y) values (a,b)
  8          when matched then update set y = b;
  9  end;
 10  /
declare
*
ERROR at line 1:
ORA-00060: deadlock detected while waiting for resource
ORA-06512: at line 4

<b>in the real world -- that deadlock would have been "you are blocked", until the first merge commits (or rollsback) at which point the merge would either get ora-00001 (if you commit), or succeeds (if you rollback)</b>

The only way to avoid that would be:

insert
when dup val on index
  then update


but the performance implications of that are pretty bad if you get the dup val on index more than a couple of times....

or you could code:

update
if zero rows
then
    insert
    when dup val on index 
    then
        update


but merge -- merge will do exactly what you thought you would avoid -- definitely.




And from a performance perspective - I would still lean towards what I said:

ops$tkyte@ORA9IR2> create table t1 ( x int primary key, y int );
Table created.
 
ops$tkyte@ORA9IR2> insert into t1 select object_id, null from all_objects where rownum <= 5000;
5000 rows created.
 
ops$tkyte@ORA9IR2> commit;
Commit complete.
 
ops$tkyte@ORA9IR2> create table t2 ( x int primary key, y int );
Table created.
 
ops$tkyte@ORA9IR2> insert into t2 select * from t1 where rownum <= 5000;
5000 rows created.
 
ops$tkyte@ORA9IR2> commit;
Commit complete.
 
ops$tkyte@ORA9IR2> create or replace procedure p1
  2  as
  3  begin
  4      for C in (select x from t1)
  5      loop
  6          for i in 1 .. 2
  7          loop
  8              merge into t1
  9              using dual on ( t1.x = C.x )
 10              when matched then update set y = -C.x
 11              when not matched then insert (x,y) values ( c.x, -c.x );
 12
 13              c.x := -c.x;
 14          end loop;
 15      end loop;
 16  end;
 17  /
Procedure created.
 
ops$tkyte@ORA9IR2> create or replace procedure p2
  2  as
  3  begin
  4      for C in (select x from t2)
  5      loop
  6          for i in 1 .. 2
  7          loop
  8              update t2 set y = -c.x where x = c.x;
  9              if ( sql%rowcount = 0 )
 10              then
 11                  insert into t2 (x,y) values ( c.x, -c.x );
 12              end if;
 13
 14              c.x := -c.x;
 15          end loop;
 16      end loop;
 17  end;
 18  /
Procedure created.
 
ops$tkyte@ORA9IR2> exec runStats_pkg.rs_start;
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> exec p1
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> exec runStats_pkg.rs_middle;
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> exec p2
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA9IR2> exec runStats_pkg.rs_stop(500);
Run1 ran in 393 hsecs
Run2 ran in 205 hsecs
run 1 ran in 191.71% of the time

<b>the merge was slower when we are doing row/row processing here....</b>

 
Name                                  Run1        Run2        Diff
LATCH.simulator lru latch              679          20        -659
LATCH.simulator hash latch           5,193       1,458      -3,735
STAT...recursive calls              15,186      20,125       4,939
STAT...consistent gets - exami      15,155      20,112       4,957
STAT...shared hash latch upgra      10,009       5,050      -4,959
STAT...calls to kcmgas               5,033          53      -4,980
LATCH.shared pool                   10,222      15,211       4,989
STAT...execute count                10,021      15,018       4,997
STAT...free buffer requested         5,315         318      -4,997
LATCH.cache buffers lru chain        5,316         317      -4,999
STAT...switch current to new b       5,000           0      -5,000
STAT...table fetch by rowid          5,006           6      -5,000
LATCH.checkpoint queue latch         5,080          16      -5,064
LATCH.library cache                 20,328      30,310       9,982
LATCH.library cache pin             20,185      30,182       9,997
STAT...index fetch by key                6      10,006      10,000
STAT...table scans (short tabl      10,001           1     -10,000
STAT...table scan rows gotten       15,000       5,000     -10,000
STAT...table scan blocks gotte      15,006       5,006     -10,000
STAT...index scans kdiixs1          10,009           6     -10,003
STAT...calls to get snapshot s      33,101      18,096     -15,005
STAT...no work - consistent re      25,050      10,025     -15,025
STAT...buffer is not pinned co      25,036       5,036     -20,000
STAT...consistent gets              68,272      33,246     -35,026
STAT...session logical reads       113,770      78,734     -35,036
STAT...redo size                 5,384,376   5,446,584      62,208
LATCH.cache buffers chains         293,139     218,073     -75,066
 
Run1 latches total versus runs -- difference and pct
Run1        Run2        Diff       Pct
388,142     323,452     -64,690    120.00%
 
PL/SQL procedure successfully completed.


<b>the merge used more resources when we go row by row (slow by slow).</b>

so, if you have a single row to update existing or insert if not there yet -- i'll stick with update/if zero rows/insert.


even in 10g, when dual is not really dual the table (but fast dual, the "function"), and you use using (select 1 from dual) -- to avoid the real dual table and use the "fast dual", the update/if zero/insert logic still bests merge (albeit by a smaller margin).  

So, row by row (slow by slow) -- i'll keep with my approach for right now.
 

Merge without update

Kiran, October 10, 2004 - 7:03 pm UTC

Tom - I have a production table. Before we insert data into this production table, we have staging table with the same structure as the production table. After a person fully authorize the validity of the staging table data, we want to merge them to the production table. I have two questions
1) How do i merge the staging table to the production table? Assume that the staging table data and the production table data are disjoint - all new records going in. We are talking about atleast 2 million to 40 million records in the staging table - depends up on whether I have daily usage data or monthly usage data. I am looking for a fater insert into the production table. Please let me know the steps that I should do before I merge the data to the production table. I can turn off logging on the production table since I have the data in the staging table if something happened to the insert.
2) After I merge the data to the production table, and if the user decide to re-run the same process, then I have to delete the records for that particular period (each set of data is categoriezed by daily or monthly or with specific dates - so that the user can delete with a range of dates) and then do the merge operation. The problem here is to delete millions of records at once. One key thing is that the production table might have millions of records for other periods also which I dont want to touch.
I thought about using partitions and taking the partition offline and do whatever I want to do - works well with Oracle but unfortunately the product has to support for Oracle, DB2 and SQLserver and so looking for a soln which is SQL standard and works across all three DBs. If not please give me a soln which is not based on partitions.

Kiran

Tom Kyte
October 10, 2004 - 7:23 pm UTC

database independence.... yuck.

good luck with that -- you will write a program that behaves poorly on one database and really does horribly on the rest.


your best bet?

a) do everything in stored procedures
b) hire the best Oracle developer you can, the best DB2 programmer you can, the best SQLServer programer you can.

c) give each the specs and say "now, do this as best as you can in your database, you will write a transaction that gets these inputs and must provide these outputs (scalars and/or result sets)


Then, you'll

a) have database independence
b) with code that actually uses the database features of each
c) and runs correctly (getting a single set of code to work correctly on all three -- almost impossible -- i would actually say "impossible" if you include any modifications at all -- read only, maybe, read write with multiple users -- you won't do it)
d) and runs fast
e) and is developed tons and tons faster and cheaper then your current approach.




single row merge

Ofir Manor, October 11, 2004 - 7:45 am UTC

Hi Tom,
great response, you won me over. I thought I was on to something, and didn't question my intuition. My bad.
It is a bit wierd that the MERGE implementaion is much less efficient then update - if 0 then insert. It should be just a single index look in this case, with insert or update at the end of it. Too bad there is no direct syntax for a single row merge without using dual.
Anyway, thanks for the lesson.
Ofir

What is the "source" table doesn't have the row?

J Vogel, October 21, 2004 - 12:58 pm UTC

I am trying to support a "true" UPSERT...

DECLARE

ln_temp NUMBER := 0;

lrec CONTRACT_DETAILS%ROWTYPE := Null;

BEGIN

lrec.CONTRACT_ID := 1;
lrec.TYPE_PREFIX := 'CTRDETAIL';
lrec.TYPE_CODE := 'YEAR_CNTR';
lrec.TYPE_VALUE2 := '1';

BEGIN
MERGE INTO contract_details DEST
USING (
SELECT contract_id,type_prefix,type_code,type_value2
FROM contract_details
WHERE
CONTRACT_ID = 1 AND
TYPE_PREFIX = 'CTRDETAIL' AND
TYPE_CODE = 'YEAR_CNTR'
) SRC
ON ( DEST.CONTRACT_ID = SRC.CONTRACT_ID)
WHEN MATCHED
THEN UPDATE SET
DEST.TYPE_PREFIX = lrec.TYPE_PREFIX,
DEST.TYPE_CODE = lrec.TYPE_CODE,
DEST.TYPE_VALUE2 = lrec.TYPE_VALUE2
WHEN NOT MATCHED
THEN INSERT (
DEST.CONTRACT_ID,
DEST.TYPE_PREFIX,
DEST.TYPE_CODE,
DEST.TYPE_VALUE2
) VALUES (
lrec.CONTRACT_ID,
lrec.TYPE_PREFIX,
lrec.TYPE_CODE,
lrec.TYPE_VALUE2
);

EXCEPTION
WHEN NO_DATA_FOUND THEN
common_func.display_output('NO DATA');
END;

common_func.display_output('SQL%ROWCOUNT : '||TO_CHAR(SQL%ROWCOUNT));
SELECT COUNT(*) INTO ln_temp FROM CONTRACT_DETAILS WHERE CONTRACT_ID = 1;

common_func.display_output('Count : '||TO_CHAR(ln_temp));

COMMIT;

EXCEPTION
WHEN OTHERS THEN
common_func.display_output(SQLERRM);
ROLLBACK;
END;


If the

Tom Kyte
October 21, 2004 - 3:14 pm UTC

you will honestly have to define "true" in this context -- i've no clue what you might mean.

and you know, NO_DATA_FOUND applies to SELECT INTO only -- never ever insert, never ever update, never ever delete, never ever merge...

Followup to Merge into help from 3/10/2004 from ketan popat

colleen noonan, October 29, 2004 - 2:00 pm UTC

Hello Tom,  I followed the example you gave Ketan because I'm stuck with a similar issue.  My merge into command only partially works.  It works for the update but not the insert.

I followed your example statement for statement and our results were different.  

SQL> create table t ( x int, y int, z int );
Table created.

SQL> insert into t values ( 1, 1, 1 );
1 row created.

SQL> commit;
Commit complete.

SQL> merge into t
  2  using dual on ( x = 55 )
  3  when matched then
  4  update set z = z+1
  5  when not matched then insert (x,y,z) values ( 2, 2, 2 );

0 rows merged.

SQL> select * from t;

         X          Y          Z
---------- ---------- ----------
         1          1          1

According to your example and Merge into logic, a row
should always be affected (either merged or created).  Why
didn't Oracle insert another row in to table t?
I'm using Oracle 9.2.1
 

Tom Kyte
October 29, 2004 - 5:01 pm UTC

9.2.1 never existed -- it is not a version.

Merge -9i- understand

karine, December 15, 2004 - 9:16 am UTC

I try to use the merge command. I think that is work as follow, if the information is not found it is inserted the line else it is updated the line. But I have this result:
there are already 6 lines in the table and after the merge 6 new lines are added. 
It look like that it doesn't take into account the "and" in the "on" part of the merge.
So I think that I don't understand something but I don't know what, So what is wrong on my request. Can you please help me or give me the correct syntax.


This is the table and requests:

select * from bs_product_type;

C_PRD_TYPE_ID        C_PRD_TYPE_LBL
-------------------- ----------------------------------------
BS                   basesystem
CI                   Communication interface
GENERAL              General
GCCM                 GemConnect Campaign Manager
SAS                  Secure Applet Server
RCA                  Remote Card Administrator

6 rows selected.


This my request:

  1  merge into bs_product_type p1
  2  using bs_product_type p2 on (p2.c_prd_type_id=p1.c_prd_type_id
  3  and p2.c_prd_type_id='SAS')
  4  when matched then update set p1.c_prd_type_lbl=p2.C_PRD_TYPE_LBL
  5  when not matched then insert (p1.C_PRD_TYPE_ID,p1.C_PRD_TYPE_LBL) values
  6* ('SAS','Toto')
  7  ;

6 rows merged.

--> after the merge:
 select * from bs_product_type;

C_PRD_TYPE_ID        C_PRD_TYPE_LBL
-------------------- ----------------------------------------
BS                   basesystem
CI                   Communication interface
GENERAL              General
GCCM                 GemConnect Campaign Manager
SAS                  Secure Applet Server
RCA                  Remote Card Administrator
SAS                  Toto
SAS                  Toto
SAS                  Toto
SAS                  Toto
SAS                  Toto

11 rows selected.

Normally it uses the match case, I would like to have in final => one row match.

This is the request that normally uses the no match case:
 1   merge into bs_product_type p1
  2   using bs_product_type p2 on (p2.c_prd_type_id=p1.c_prd_type_id
  3   and p2.c_prd_type_id='TOTO')
  4   when matched then update set p1.c_prd_type_lbl=p2.C_PRD_TYPE_LBL
  5   when not matched then insert (p1.C_PRD_TYPE_ID,p1.C_PRD_TYPE_LBL) values
  6*  ('TOTO','Toto')

6 rows merged.

SQL> select * from bs_product_type;

C_PRD_TYPE_ID        C_PRD_TYPE_LBL
-------------------- ----------------------------------------
BS                   basesystem
CI                   Communication interface
GENERAL              General
GCCM                 GemConnect Campaign Manager
SAS                  Secure Applet Server
RCA                  Remote Card Administrator
TOTO                 Toto
TOTO                 Toto
TOTO                 Toto
TOTO                 Toto
TOTO                 Toto

C_PRD_TYPE_ID        C_PRD_TYPE_LBL
-------------------- ----------------------------------------
TOTO                 Toto

12 rows selected.


thanks a lot for your answer.
karine 

Tom Kyte
December 15, 2004 - 2:06 pm UTC

merge into bs_product_type p1
2 using bs_product_type p2 on (p2.c_prd_type_id=p1.c_prd_type_id
3 and p2.c_prd_type_id='SAS')
4 when matched then update set p1.c_prd_type_lbl=p2.C_PRD_TYPE_LBL
5 when not matched then insert (p1.C_PRD_TYPE_ID,p1.C_PRD_TYPE_LBL) values
6* ('SAS','Toto')

should probably be:

merge into bs_product_type p1
2 using (select * from bs_product_type where c_prd_type_id = 'SAS') p2 on (p2.c_prd_type_id=p1.c_prd_type_id )

4 when matched then update set p1.c_prd_type_lbl=p2.C_PRD_TYPE_LBL
5 when not matched then insert (p1.C_PRD_TYPE_ID,p1.C_PRD_TYPE_LBL) values
6* ('SAS','Toto')


if all you wanted to do was take the records from P2 such that the type_id was SAS and merge then in.

Else, EVERY row in P2 is going to be merged in using your join condition (which happened to have a predicate -- a filter in it -- as well as a join condition)

Merge in 10g

A reader, December 15, 2004 - 4:55 pm UTC

From one of your presentations...

"MERGE can be used as INSERT-only respective UPDATE-only operation
No outer join necessary (UPDATE-only)
Can use ANTI-JOIN (INSERT-only)
Conditional INSERT and UPDATE branches
Full flexibility for individual data filtering
Optional DELETE clause for UPDATE branch"

Can you please elaborate on each of the points above with an example? Thanks a lot

Tom Kyte
December 15, 2004 - 6:52 pm UTC

 
ops$tkyte@ORA10GR1> create table t ( x int not null, y int );
 
Table created.
 
ops$tkyte@ORA10GR1> insert into t values ( 1, 1 );
 
1 row created.
 
ops$tkyte@ORA10GR1> create table t2 ( x int not null, y int );
 
Table created.
 
ops$tkyte@ORA10GR1> insert into t2 values ( 2, 2 );
 
1 row created.
 
ops$tkyte@ORA10GR1> exec dbms_stats.set_table_stats( user,'T' ,numrows=>10000, numblks=>10000);
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA10GR1> exec dbms_stats.set_table_stats( user,'T2',numrows=>1000, numblks=>10000);
 
PL/SQL procedure successfully completed.
 
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> merge into t
  2  using t2 d
  3  on ( t.x = d.x )
  4  when not matched then insert (x,y) values ( d.x, d.y );
 
1 row merged.
<b>insert only, has same effect as:

insert into t select * from t2 where x not in ( select x from t );

(which leads me to question -- why? we didn't really need that one, but anyway....</b>
 
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> set autotrace on explain
ops$tkyte@ORA10GR1> merge into t
  2  using t2 d
  3  on ( t.x = d.x )
  4  when not matched then insert (x,y) values ( d.x, d.y )
  5  when matched then update set y = y+2;
 
1 row merged.

<b>that is 9i's only way... both insert and update</b>

 
Execution Plan
----------------------------------------------------------
   0      MERGE STATEMENT Optimizer=ALL_ROWS (Cost=4412 Card=10000 Bytes=650000)
   1    0   MERGE OF 'T'
   2    1     VIEW
   3    2       HASH JOIN (OUTER) (Cost=4412 Card=10000 Bytes=590000)
   4    3         TABLE ACCESS (FULL) OF 'T2' (TABLE) (Cost=2206 Card=1000 Bytes=26000)
   5    3         TABLE ACCESS (FULL) OF 'T' (TABLE) (Cost=2206 Card=10000 Bytes=330000)
 
 
 
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> merge into t
  2  using t2 d
  3  on ( t.x = d.x )
  4  when matched then update set y = y+2;
 
1 row merged.
 
<b>but now you can just do the update portion (no more "key preserved table" issues...</b>
 
Execution Plan
----------------------------------------------------------
   0      MERGE STATEMENT Optimizer=ALL_ROWS (Cost=4412 Card=10000 Bytes=260000)
   1    0   MERGE OF 'T'
   2    1     VIEW
   3    2       HASH JOIN (Cost=4412 Card=10000 Bytes=590000)
   4    3         TABLE ACCESS (FULL) OF 'T2' (TABLE) (Cost=2206 Card=1000 Bytes=26000)
   5    3         TABLE ACCESS (FULL) OF 'T' (TABLE) (Cost=2206 Card=10000 Bytes=330000)
 
 
 
ops$tkyte@ORA10GR1> set autotrace off
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> select * from t;
 
         X          Y
---------- ----------
         1          1
         2          6
 
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> merge into t
  2  using t2 d
  3  on ( t.x = d.x )
  4  when matched then update set y = y+2
  5                    delete where y > 6;
 
1 row merged.
 
ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> select * from t;
 
         X          Y
---------- ----------
         1          1
 
<b>that shows that it can delete after applying the update -- eg: data warehouse -- apply the incremental inventory counts -- when qty = 0, delete it)</b>

 

Merge -9i- understand (suite)

Karine, December 16, 2004 - 9:57 am UTC

In fact the request you give me doesn't give the expected result for the case where the line is not present in the table, there is no merge. It is working very well for the case where the line is present in the table.
I would like to use the update case if the line already exist and if not insert the line.

the type doesn't exist in table:
merge into bs_product_type p1
2 using (select * from bs_product_type where c_prd_type_id = 'TOTTO') p2 on
3 (p2.c_prd_type_id=p1.c_prd_type_id )
4 when matched then update set p1.c_prd_type_lbl=p2.C_PRD_TYPE_LBL
5 when not matched then insert (p1.C_PRD_TYPE_ID,p1.C_PRD_TYPE_LBL) values
6* ('TOTTO','Toto')
/
0 rows merged.


for the update case: the type already exist in table:
merge into bs_product_type p1
2 using (select * from bs_product_type where c_prd_type_id = 'SAS') p2 on
3 (p2.c_prd_type_id=p1.c_prd_type_id )
4 when matched then update set p1.c_prd_type_lbl=p2.C_PRD_TYPE_LBL
5 when not matched then insert (p1.C_PRD_TYPE_ID,p1.C_PRD_TYPE_LBL) values
6* ('SAS','Toto')

1 row merged.


thanks a lot.
Karine



Tom Kyte
December 16, 2004 - 10:08 am UTC

not following you.

..
for the case
where the line is not present in the table,
.....

not present in WHAT table?


the merge:

merge into bs_product_type p1
2 using (select * from bs_product_type where c_prd_type_id = 'TOTTO') p2 on
3 (p2.c_prd_type_id=p1.c_prd_type_id )
4 when matched then update set p1.c_prd_type_lbl=p2.C_PRD_TYPE_LBL
5 when not matched then insert (p1.C_PRD_TYPE_ID,p1.C_PRD_TYPE_LBL) values
6* ('TOTTO','Toto')
/

will take all of the rows from bs_product_type where c_prd_type_id = 'TOTTO'

and merge then (either updating the already existing row or inserting a new one)

IF NO rows exist in bs_product_type for that predicate, nothing will happen -- zero rows. (it is not that "nothing is matched", it is that there is NOTHING to match to!)


I don't think you have a "merge" condition here -- your set you are merging from (bs_product_type) doesn't have the rows you need in this case. Merge will "not make up" data.




Excellent. I struggled with this for quite a while.

Kimathi Njeru, December 27, 2004 - 6:45 pm UTC

Excellent. I struggled with this for quite a while.
I should have come here first!


Group function in Merge Statement

Ashish Somani, January 07, 2005 - 6:54 am UTC

Respected Tom,
Sir, can I use group fuction given in below example. If not then what should be other alternative to this apart from using a sequence. We have constraint we cannot use sequence.
MERGE INTO MERCH_DIM oldt
USING HRSP.STG_MERCH_DIM newt
ON (newt.org_cde = oldt.org_cde AND newt.logo_cde = oldt.logo_cde)
WHEN MATCHED THEN UPDATE
SET indus_cde = trim(newt.indus_cde)
,merch_mgmt_name = trim(newt.merch_mgmt_name)
,merch_sys_name = trim(translate(newt.merch_sys_name,',',' '))
,merch_pgm_sys_name = trim(newt.merch_pgm_sys_name)
,merch_stat_cde = trim(newt.merch_stat_cde)
,merch_rpt_name = trim(newt.merch_rpt_name)
,merch_pgm_rpt_name = trim(newt.merch_pgm_rpt_name)
,merch_own_svc_cde = trim(newt.merch_own_svc_cde)
,sale_regn_cde = trim(newt.sale_regn_cde)
,acct_exec_cde = trim(newt.acct_exec_cde)
,merch_tier_num = trim(newt.merch_tier_num)
,portf_cde = trim(newt.portf_cde)
,org_group_cde = trim(newt.org_group_cde)
,portf_desc = trim(newt.portf_desc)
,indus_desc = trim(newt.indus_desc)
,sale_regn_desc = trim(newt.sale_regn_desc)
,acct_exec_desc = trim(newt.acct_exec_desc)
WHEN NOT MATCHED THEN
INSERT (merch_key,org_cde,logo_cde,creat_dt,updt_dt)
VALUES (max(oldt.merch_key)+1,newt.org_code, newt.logo_cde,sysdate,sysdate);

Tom Kyte
January 07, 2005 - 9:21 am UTC

you'll have to run that select max query first -- but I don't see how you'll get it to insert more than one row really.


looks like a "two step" to me -- update rows that exist, insert rows that don't (eg: no merge)

Gotcha clarification?

Tom Starr, January 21, 2005 - 1:38 pm UTC

To paraphrase a previous response above in this thread: IF NO rows exist (for that predicate) nothing will happen - it is not that "nothing is matched", it is that there is NOTHING to match to!
In the case below, UPDATE works, but not INSERT.
Can you tell me please if this is for the above reason, and if one should select from dual in USING to insert literals?
MERGE INTO some_table A
USING (SELECT id, key, value
FROM some_table
WHERE key = 'test_key') B
ON (B.id = A.id)
WHEN MATCHED THEN UPDATE SET A.value = 'literal_value'
WHEN NOT MATCHED THEN INSERT (A.id, A.key, A.value)
VALUES (7, 'literal_key', 'literal_value');
(Yes I know id is a keyword - I inherited many such cases and haven't managed to fix them yet. Thanks in advance!!)

Tom Kyte
January 21, 2005 - 7:42 pm UTC

the insert "works" -- it is just that there is nothing to match to at all (the using subquery returns "zero rows")


So, is the number "7" magic? is id the KEY of some_table or is key ?

need more info, where do all of the literals come from and what do they mean.


(but, for a single row update, you'll find:

update ...
if (sql%rowcount = 0)
then
insert ....
end if;

to be more performant, less obscure and easier to maintain over time...)

merge with delete

sudipta, January 23, 2005 - 3:45 am UTC

MERGE USING Product_Changes S
INTO Products D ON (D.PROD_ID = S.PROD_ID)
WHEN MATCHED THEN
UPDATE SET D.PROD_LIST_PRICE =S.PROD_NEW_PRICE, D.PROD_STATUS = S.PROD_NEWSTATUS
DELETE WHERE (D.PROD_STATUS = "OBSOLETE")
WHEN NOT MATCHED THEN
INSERT (PROD_ID, PROD_LIST_PRICE, PROD_STATUS)
VALUES (S.PROD_ID, S.PROD_NEW_PRICE, S.PROD_NEW_STATUS);


is this feature supported only on 10g r1 and up releases

Tom Kyte
January 23, 2005 - 10:07 am UTC

yes. delete as an option to update in a merge is 10g ...

Gotcha clarification - thanks!!

Tom Starr, January 24, 2005 - 4:58 pm UTC

The intent was just update a "value" field if a hardwired key existed in a "key" field, otherwise increment the PK and insert the new key-value pair (the "7" was just a simplification).

Thanks for clarifying one cannot use merge to insert literal and computed values not returned by the using subquery.

Please help

Kumar, January 31, 2005 - 12:10 pm UTC

Dear Tom,
I tried to perform an insert with this merge statement.
But it is performing properly.

SQL> merge into emp e
  2  using (select * from dept)d
  3  on (d.deptno <> e.deptno)
  4  when matched then
  5   update set sal = sal
  6  when not matched then
  7   insert(empno,ename,sal,mgr,job,hiredate,comm,deptno)
  8           

values(2001,'Joe',3200,7566,'CLERK',SYSDATE-200,100,d.deptno)
  9  /

I expect that deptno -> 40 to be inserted along with other
column values.Is there any way to achieve this?? 

Tom Kyte
January 31, 2005 - 12:22 pm UTC

<quote>
But it is performing properly.
</quote>

that sounds good to me!


d.deptno <> e.deptno, hmm, think about it.

sounds like you actually want:

insert into emp (.... )
select .....
from dept
where deptno not in ( select deptno from emp );



OK

Kumar, February 01, 2005 - 11:53 am UTC

Dear Tom,
Thanks for your reply.That's what I want to do using Merge statement.Is there any way to do it using Merge??

Tom Kyte
February 01, 2005 - 2:18 pm UTC

why, that would be *wrong*

you want an insert statement.

Duplicate records in MERGE statement

Rakesh Bansal, February 21, 2005 - 8:12 am UTC

Dear Tom,
As per discussion thread MERGE statement throws error if source table have duplicate records. But I found inconsistent behaviour of MERGE statement. Can you clarify in this regard? Following are my obesrvation,

CASE 1
=======
SQL> select * from merge_src;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          NT
       107       1007 SMITH                          NT

SQL> select * from merge_dest;

no rows selected

SQL> MERGE INTO MERGE_DEST D
  2  USING MERGE_SRC S
  3  ON (D.CARD_NUM = S.CARD_NUM)
  4  WHEN MATCHED THEN UPDATE SET
  5  D.CUST_ID = S.CUST_ID,
  6  D.CUST_NAME = S.CUST_NAME,
  7  D.CUST_DEPT = S.CUST_DEPT
  8  WHEN NOT MATCHED THEN INSERT (D.CARD_NUM,D.CUST_ID, D.CUST_NAME, D.CUST_DEPT)
  9  VALUES (S.CARD_NUM,S.CUST_ID, S.CUST_NAME, S.CUST_DEPT);

2 rows merged.

SQL> select * from merge_dest;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          NT
       107       1007 SMITH                          NT

CASE 2
========

SQL> select * from merge_src;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          NT
       107       1007 SMITH                          NT

SQL> select * from merge_dest;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          NT

SQL> MERGE INTO MERGE_DEST D
  2  USING MERGE_SRC S
  3  ON (D.CARD_NUM = S.CARD_NUM)
  4  WHEN MATCHED THEN UPDATE SET
  5  D.CUST_ID = S.CUST_ID,
  6  D.CUST_NAME = S.CUST_NAME,
  7  D.CUST_DEPT = S.CUST_DEPT
  8  WHEN NOT MATCHED THEN INSERT (D.CARD_NUM,D.CUST_ID, D.CUST_NAME, D.CUST_DEPT)
  9  VALUES (S.CARD_NUM,S.CUST_ID, S.CUST_NAME, S.CUST_DEPT);

2 rows merged.

SQL> select * from merge_dest;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          NT

CASE 3
=======

SQL> select * from merge_src;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          NY
       107       1007 SMITH                          NT

SQL> select * from merge_dest;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          NT

SQL> MERGE INTO MERGE_DEST D
  2  USING MERGE_SRC S
  3  ON (D.CARD_NUM = S.CARD_NUM)
  4  WHEN MATCHED THEN UPDATE SET
  5  D.CUST_ID = S.CUST_ID,
  6  D.CUST_NAME = S.CUST_NAME,
  7  D.CUST_DEPT = S.CUST_DEPT
  8  WHEN NOT MATCHED THEN INSERT (D.CARD_NUM,D.CUST_ID, D.CUST_NAME, D.CUST_DEPT)
  9  VALUES (S.CARD_NUM,S.CUST_ID, S.CUST_NAME, S.CUST_DEPT);

2 rows merged.

SQL> select * from merge_dest;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          NY

CASE 4
========

SQL> select * from merge_src;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          PB
       107       1007 SMITH                          NT

SQL> select * from merge_dest;

  CARD_NUM    CUST_ID CUST_NAME                      CUST_DEPT
---------- ---------- ------------------------------ ---------------
       107       1007 SMITH                          NY

SQL> MERGE INTO MERGE_DEST D
  2  USING MERGE_SRC S
  3  ON (D.CARD_NUM = S.CARD_NUM)
  4  WHEN MATCHED THEN UPDATE SET
  5  D.CUST_ID = S.CUST_ID,
  6  D.CUST_NAME = S.CUST_NAME,
  7  D.CUST_DEPT = S.CUST_DEPT
  8  WHEN NOT MATCHED THEN INSERT (D.CARD_NUM,D.CUST_ID, D.CUST_NAME, D.CUST_DEPT)
  9  VALUES (S.CARD_NUM,S.CUST_ID, S.CUST_NAME, S.CUST_DEPT);
MERGE INTO MERGE_DEST D
*
ERROR at line 1:
ORA-30926: unable to get a stable set of rows in the source tables

Thanks,
Rakesh 

Tom Kyte
February 21, 2005 - 11:20 am UTC

(why don't I get create tables and insert into's like I give you guys when I give examples??  so frustrating to have to generate that stuff from describes and selects, especially when the page you used to add this clearly at the top has:

If your followup requires a response that might include a query, you had better supply very very simple create tables and insert statements. I cannot create a table and populate it for each and every question. The SMALLEST create table possible (no tablespaces, no schema names, just like I do in my examples for you))

but i simplified your test case down to this:

ops$tkyte@ORA9IR2> create table merge_src ( key int, data int );
 
Table created.
 
ops$tkyte@ORA9IR2> insert into merge_src values ( 1, 1 );
 
1 row created.
 
ops$tkyte@ORA9IR2> insert into merge_src values ( 1, 2 );
 
1 row created.
 
ops$tkyte@ORA9IR2> create table merge_dest ( key int, data int );
 
Table created.
 
ops$tkyte@ORA9IR2> insert into merge_dest values ( 1, 1 );
 
1 row created.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> select * from merge_src;
 
       KEY       DATA
---------- ----------
         1          1
         1          2
 
ops$tkyte@ORA9IR2> MERGE INTO MERGE_DEST D
  2  USING MERGE_SRC S
  3  ON (D.key = s.key)
  4  WHEN MATCHED THEN UPDATE SET data = s.data
  5  WHEN NOT MATCHED THEN INSERT (key,data) values (s.key,s.data);
USING MERGE_SRC S
      *
ERROR at line 2:
ORA-30926: unable to get a stable set of rows in the source tables
 
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> rollback;
 
Rollback complete.
 
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> update merge_src set data = decode(rownum,1,2,1);
 
2 rows updated.
 
ops$tkyte@ORA9IR2> commit;
 
Commit complete.
 
ops$tkyte@ORA9IR2> select * from merge_src;
 
       KEY       DATA
---------- ----------
         1          2
         1          1
 
ops$tkyte@ORA9IR2> MERGE INTO MERGE_DEST D
  2  USING MERGE_SRC S
  3  ON (D.key = s.key)
  4  WHEN MATCHED THEN UPDATE SET data = s.data
  5  WHEN NOT MATCHED THEN INSERT (key,data) values (s.key,s.data);
 
2 rows merged.


It appears that if the update would not really update the row (not change the values) it "skips" it conceptually -- so, if we try to update DATA in the dest table to itself first - it says "hey, nothing to do, skip" (conceptually).  Later it gets the new value and says "these are different, update"

In the first case above, the data was processed in such a way that it changed Data from 1 to 2 in dest and then tried to change 2 to 1 -- that failed.

when I reversed the order of the "data" in the src table, it tried to change 1 to 1, did nothing in effect and then changed 1 to 2, successfully (but erroneously)


It should have failed both times.  I'll file a bug (but it might not be a bug -- i'll leave that for them to decide.  I see an existing closed bug that leads me to believe that they will say "deterministic behavior is observed here even if one fails and the other succeeds -- the one that succeeds always gets the same answer and the other one just doesn't "work""

so the merge is deterministic in its processing -- just the inclusion of the duplicates may prevent you from running the statement... 

merge example

A reader, March 24, 2005 - 2:56 pm UTC

Hi Tom,

I have a merge stmt as follows

Merge into t1
using
(
select a.*,first_value(id) over (partition by col1 order by imagedate asc) first_id
from t1 a
where col1 in (select col1 from t1)
) t2
on (t1.id = t2.id)
when matched then -- code here
when not matched then -- code here

Now I want to change this to partition by more than 1 column. Is that possible? Something like


Merge into t1
using
(
select a.*,first_value(id) over (partition by col1 order by imagedate asc) first_id,
over (partition by col2 order by imagedate asc) .....
from t1 a
where col1 in (select col1 from t1)
) t2
on (t1.id = t2.id)
when matched then -- code here
when not matched then -- code here


Thanks.

Tom Kyte
March 24, 2005 - 4:06 pm UTC

sure, just be aware that it has to do as much work as it looks like it has to do (sort the set by col1 and then by col2 after processing by col1)

A reader, March 24, 2005 - 4:12 pm UTC

This is what I want to achieve. If two rows in t1 have same col1 and col2 I want to assign all those rows the first value of id.

So, if there are 2 rows in t1 like

id col1 col2 imagedate
1 22 'JOHN' 03-21-2005
2 22 'JOHN' 03-22-2005

Since col1 and col2 are the same I want to update id 2 to id 1 sorting by imagedate.


Thanks.


Tom Kyte
March 24, 2005 - 4:26 pm UTC

insufficient data.

what if you have

id col1 col2
1 a b
1 a b
1 d e

do you need to be merging by more than just ID?

A reader, March 24, 2005 - 5:14 pm UTC

id is a sequence so two rows will never have same id.

Thanks.

Tom Kyte
March 24, 2005 - 5:31 pm UTC

you can partition by col1, col2 -

first_value(id) over (partition by col1, col2 order by ...)

Thanks a lot

A reader, March 31, 2005 - 12:57 am UTC

Your article was of gr8 use...

thanks a lot.

oracle's merge approach

Marc Werner, April 06, 2005 - 11:26 am UTC

tom,

is it possible for a "merge into" to follow a nested loop strategy - say :
full scan of the table which get merged and index lookup on the table which is used solely (or maybe even joined with others) in the source SQL ??

thanks a lot
regards
marc

Tom Kyte
April 06, 2005 - 1:56 pm UTC

nope, not with a "when matched" "when not matched"

it wouldn't make sense, EVERY row in "big table" gets processed!

but in 10g, sure


new 1: explain plan for merge into small using big on ( small.object_id = big.object_id ) when matched then update set object_name = big.object_name

Explained.


PLAN_TABLE_OUTPUT
-------------------------------------------------------------------------------------------------------------------------
Plan hash value: 285406743

-----------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------
| 0 | MERGE STATEMENT | | 3000K| 97M| 203 (1)| 00:00:03 |
| 1 | MERGE | SMALL | | | | |
| 2 | VIEW | | | | | |
| 3 | NESTED LOOPS | | 3000K| 572M| 203 (1)| 00:00:03 |
| 4 | TABLE ACCESS FULL | SMALL | 200 | 20000 | 2 (0)| 00:00:01 |
| 5 | TABLE ACCESS BY INDEX ROWID| BIG | 15000 | 1464K| 1 (0)| 00:00:01 |
|* 6 | INDEX UNIQUE SCAN | BIG_PK | 1 | | 0 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

6 - access("SMALL"."OBJECT_ID"="BIG"."OBJECT_ID")

18 rows selected.


when you only to the "when matched part"

merge into big using small ...

Marc Werner, April 07, 2005 - 1:55 am UTC

but what if you got a scenario like this :
100 Mio rows fact table range partitioned etc.
5 Mio rows source - table (sql) to merge.
A nested loop the other way round ( e.g. using hash on the small table ) should run much faster than doing NL
by full scan of "small" and index lookup for every row in there on "big" !
its a bit like :
update big
set big.X = (select small.Y from small where small.bla = big.blaaaaa);

but my logic is too complex for that and has more than one column to update :
have a look ( dont get surprised - I am using NL your way forced by hints,
because last time optimizer decided for a path which took
24 hours to merge 4,7 Mio rows into 70 Mio , which is unacceptable I believe) :

MERGE /*+ use_nl(FAKT_LIQUID) index(FAKT_LIQUID IDX_FAKT_LIQUID_ZFDSID2) */ INTO FAKT_LIQUID
USING (
SELECT /*+ parallel 4 */ to_date( ZLEF.ZAUSDAT, 'YYYYMMDD') KALK_TATS_ZEDATM,
CASE WHEN ZLEF.zquell = '1' THEN to_date( ZLEF.zldate, 'YYYYMMDD')
ELSE to_date( decode( ZLEF.ZLDAT1, '00000000', '19000101', ZLEF.ZLDAT1 ) , 'YYYYMMDD')
END KALK_LEISTDATM_PART,
ZLEF.ZAUSDAT,
ZLEF.MANDT,
ZLEF.KUNNR,
ZLEF.ZFDSID,
ZNSS_VOR.ZINSSATZ
FROM lkl_zca_zlef ZLEF LEFT JOIN LKUP_ZINSEN ZNSS_VOR
ON ( ZNSS_VOR.KZ_ZINSSATZ = 1 AND ZNSS_VOR.AKTUELL = 'J')
) SRC
ON ( SRC.KALK_LEISTDATM_PART = FAKT_LIQUID.KALK_LEISTDATM_PART AND
'ZSDMAUT_FAKT' = FAKT_LIQUID.HERKUNFTTAB_ID AND
SRC.ZFDSID = FAKT_LIQUID.SAP_ZFDSID
)
WHEN MATCHED THEN
UPDATE
SET TS_AEND_FAKT_DSATZ = sysdate,
SAP_ZAUSDAT = SRC.ZAUSDAT,
FINANZGSART_SID = DECODE( KALK_ZAHLART, 'GU', FINANZGSART_SID,
CASE WHEN KALK_GEPL_AUSKEHR_WFRIST < SRC.KALK_TATS_ZEDATM THEN
1 ELSE 2
END),
KALK_TATS_ZEDATM = DECODE( KALK_ZAHLART, 'GU', KALK_TATS_ZEDATM, SRC.KALK_TATS_ZEDATM ),
BWGART_ZAHLEINGANG_SID = DECODE( KALK_ZAHLART, 'GU', BWGART_ZAHLEINGANG_SID,
CASE WHEN KALK_GEPL_ZEDATM = SRC.KALK_TATS_ZEDATM THEN 1
ELSE
CASE WHEN SRC.KALK_TATS_ZEDATM > KALK_GEPL_ZEDATM THEN 2
ELSE 3
END
END ),
KALK_TATS_AUSKEHR = DECODE( KALK_ZAHLART, 'GU', KALK_TATS_AUSKEHR,
tc_tools.get_dest_date( least( SRC.KALK_TATS_ZEDATM, KALK_GEPL_AUSKEHR_WFRIST ) , 1 ) ),
KALK_ZINSTAGE_VOR = DECODE( KALK_ZAHLART, 'GU', 0,
greatest( 0, SRC.KALK_TATS_ZEDATM - KALK_GEPL_AUSKEHR_WFRIST ) ),
KALK_ZINSEN_VOR = DECODE( KALK_ZAHLART, 'GU', 0,
SAP_ZPREIS * ZINSSATZ * greatest( 0, SRC.KALK_TATS_ZEDATM - KALK_GEPL_AUSKEHR_WFRIST) ),
BWGART_AUSKEHR_SID = DECODE( KALK_ZAHLART, 'GU', BWGART_AUSKEHR_SID,
CASE WHEN SRC.KALK_TATS_ZEDATM < KALK_GEPL_AUSKEHR_WFRIST THEN
CASE WHEN SRC.KALK_TATS_ZEDATM <= trunc( pSAPeffDate) THEN 3
ELSE 6
END
ELSE BWGART_AUSKEHR_SID
END ) ,
KALK_ZINSEN_BASISWERT_VOR = DECODE( KALK_ZAHLART, 'GU', 0,
CASE WHEN SRC.KALK_TATS_ZEDATM -
least( SRC.KALK_TATS_ZEDATM, KALK_GEPL_AUSKEHR_WFRIST ) <= 0 THEN 0
ELSE SAP_ZPREIS
END )
WHERE SAP_ZAUSDAT = '00000000'
;





Tom Kyte
April 07, 2005 - 9:18 am UTC

but in 9i a merge will always:

a) update EXISTING rows in small based on their match in BIG
b) take ALL ROWS IN BIG (all rows) that didn't have a match and attempt to insert them.

So all rows, every row, in BIG will be processed.


nested loops, when processing all rows in BIG, doesn't make sense, we need to process even the rows in BIG that have no matches.


Now, in 10g, when you can just say "update" in MERGE, it can in fact work with Nested Loops.

Merge with RETURNING Clause

Maverick, April 07, 2005 - 3:46 pm UTC

Tom, I have read in your post above that MERGE does not currently support RETURNING Clause. It was posted and answered in Aug 2003.
Do you know if there are any updates on this? Does it support now in 10g (or 9i )?

Thanks,


Tom Kyte
April 07, 2005 - 4:38 pm UTC

I see nothing in the wire diagram indicating support for returning in merge.

Merge with RETURNING Clause

Maverick, April 07, 2005 - 5:14 pm UTC

Tom, Is there any reason for Oracle not supporting this feature for MERGE?

I mean, do you see any problems using 'Returning' with MERGE Command?

Thanks for your help,


Tom Kyte
April 07, 2005 - 5:19 pm UTC

to get new features in -- you need to file them as enhancement requests in metalink.

I think two good ones would be:

support returning with insert as select and with merge.

Merge -Statement

NN, April 13, 2005 - 11:44 am UTC

Hi Tom,

Here is what I want to do:

MERGE INTO Table1 a
USING (SELECT oid
FROM table2 b,
table3 c
WHERE b.oid = c.oid
AND c.other_oid = <user_input>
)d
ON (condition1=true)
WHEN MATCHED THEN
If <condition2=true> then
UPDATE ...
end if;
WHEN NOT MATCHED THEN
INSERT ...


Update/Insert decision is based on <condition1>.
But, before updating I have to check for
<condition2>, how do I do it? I cannot include <condition2>
in the on clause, because it is should not decide whether insert should take place or not.

Thanks in advance!



Tom Kyte
April 13, 2005 - 11:56 am UTC

only thing I can think is to set the values back to themselves


when matched then
update set c1 = case when (condition2) then d.c1 else c1 end,
c2 = case when (condition2) then d.c2 else c2 end,
.....


setting a column to itself will not touch the indexes, it'll just update the table data so minimal work is done, but the update will be done.

the merge will either update or insert for every row int he "using"

NN

A reader, April 13, 2005 - 1:56 pm UTC

Thanks a lot Tom!

A reader, April 19, 2005 - 4:38 pm UTC

MERGE INTO TABLE_2 a
USING ( SELECT COL_1 FROM TABLE_1 WHERE COL_2=<user_input>
)b
ON a.COL_1 = b.COL_1
WHEN MATCHED THEN
UPDATE SET COL3=<val1>
COL4=<val2>
WHEN NOT MATHCED THEN
INSERT (COL3,COL4)VALUES(val1,val2);

---

TABLE_1 has a matching record; whereas TABLE_2
does not have matching record; even then
no record is inserted (all not_null columns are taken
care and there are no triggers).

Thanks for your help.

Tom Kyte
April 19, 2005 - 7:45 pm UTC

prove it to me. set up the create tables, put a row in table_1 and show me that it does not merge.

where are val1 and val2 coming from?

A reader, April 20, 2005 - 8:58 am UTC

Thanks for your response Tom!

val1,val2 are input parameters to the stored proc.

What will happen when given <user_input>
is not found in TABLE_1? Will it go to when not matched?

Tom Kyte
April 20, 2005 - 8:50 pm UTC

give example

OK

Siva, May 13, 2005 - 1:19 pm UTC

Hi Tom,
Sorry to ask this clarification in this thread.
What is the use of Mapping Table clause??


Tom Kyte
May 13, 2005 - 2:05 pm UTC

do you mean for creating bitmap indexes on index organized tables?

OK

Siva, May 14, 2005 - 3:12 am UTC

Hi Tom,
Thanks for your reply.
I am unaware of that clause.
How to use that??


OK

Kumar, May 14, 2005 - 10:38 am UTC

Hi Tom,
Can a column be merged to another column so that either of the columns cease to exist and data of that column pasted
to the other column??
Any possibility there??


Tom Kyte
May 14, 2005 - 1:34 pm UTC

not sure what you mean exactly. "so that either of the columns cease to exist"?


do you mean simply:

update t
set c1 = c1 || c2,
c2 = NULL;



Lazy Developer MERGE Statement Creator (for copying data between Test/Production)

Devil Dog, June 03, 2005 - 4:05 pm UTC

Hi Tom,

I frequently copy data between Test and Production environments and found the MERGE statement to be the best possible way to do it (when I want to preserve the target environment's already present data).

While MERGE works great - it takes a while to write. So I wrote a lazy little utility that I hope you and your readers might find useful for building a MERGE statement programmatically. Please note it is only useful for copying data between two tables which have the same column names (such as in the case of copying data from Production to Test)...

Here goes:

CREATE OR REPLACE TYPE varchar2_table_type AS TABLE OF VARCHAR2(255);
/

SHOW ERRORS


CREATE OR REPLACE FUNCTION create_merge_stmt (
p_srce_table_name user_tables.table_name%TYPE
, p_trgt_table_name user_tables.table_name%TYPE
, p_srce_db_link user_db_links.db_link%TYPE DEFAULT NULL
, p_join_columns varchar2_table_type DEFAULT NULL
)
RETURN CLOB AUTHID CURRENT_USER AS
-- Constants
oracr CONSTANT VARCHAR2 (1) := CHR (10);
-- Variables
v_db_link VARCHAR2 (31) DEFAULT NULL;
v_join_columns varchar2_table_type;
v_and_str VARCHAR2 (5) DEFAULT NULL;
v_update_comma_str VARCHAR2 (3) DEFAULT NULL;
v_insert_comma_str VARCHAR2 (3) DEFAULT NULL;
v_join_clause VARCHAR2 (2000);
v_insert_col_list VARCHAR2 (2000);
v_update_clause VARCHAR2 (2000);
v_insert_clause VARCHAR2 (2000);
v_sql_stmt VARCHAR2 (32767);
v_trgt_primary_key user_constraints.constraint_name%TYPE;
-- Custom Exceptions
no_pk_and_no_join_cols EXCEPTION;
BEGIN
IF p_srce_db_link IS NOT NULL THEN
v_db_link := '@' || p_srce_db_link;
END IF;

-- Proc Validity Checks
-- The tables must have primary key - otherwise the passed in parameter "p_join_columns" is used...
IF p_join_columns IS NULL THEN
-- Get the target primary key for later use...
BEGIN
SELECT constraint_name
INTO v_trgt_primary_key
FROM user_constraints
WHERE table_name = p_trgt_table_name
AND constraint_type = 'P';
EXCEPTION
WHEN NO_DATA_FOUND THEN
RAISE no_pk_and_no_join_cols;
END;

SELECT column_name
BULK COLLECT INTO v_join_columns
FROM user_cons_columns
WHERE table_name = p_trgt_table_name
AND constraint_name = v_trgt_primary_key
ORDER BY POSITION;
ELSE
v_join_columns := p_join_columns;
END IF;

-- Build the MERGE "ON" join clause string
v_and_str := NULL;

FOR i IN v_join_columns.FIRST .. v_join_columns.LAST LOOP
IF i > 1 THEN
v_and_str := ' AND ';
END IF;

v_join_clause := v_join_clause || v_and_str || 'srce.' || v_join_columns (i) || ' = trgt.' || v_join_columns (i) || oracr;
END LOOP;

-- Build the MERGE UPDATE AND INSERT clause strings...
v_update_comma_str := NULL;
v_insert_comma_str := NULL;

-- Get the non-key columns...
FOR rec IN (SELECT column_name
, primary_key_column
, ROW_NUMBER () OVER (PARTITION BY primary_key_column ORDER BY column_id ASC) AS update_col_seq_num
, column_id AS insert_col_seq_num
FROM (SELECT LOWER (column_name) AS column_name
, CASE
WHEN column_name IN (SELECT column_value
FROM TABLE (v_join_columns)) THEN 'Y'
ELSE 'N'
END AS primary_key_column
, column_id
FROM user_tab_columns utc
WHERE utc.table_name = p_trgt_table_name)
ORDER BY insert_col_seq_num ASC) LOOP
IF rec.primary_key_column = 'N' THEN
IF rec.update_col_seq_num > 1 THEN
v_update_comma_str := ', ';
END IF;

v_update_clause := v_update_clause || v_update_comma_str || 'trgt.' || rec.column_name || ' = srce.' || rec.column_name || oracr;
END IF;

IF rec.insert_col_seq_num > 1 THEN
v_insert_comma_str := ', ';
END IF;

v_insert_col_list := v_insert_col_list || v_insert_comma_str || rec.column_name;
v_insert_clause := v_insert_clause || v_insert_comma_str || 'srce.' || rec.column_name || oracr;
END LOOP;

-- Now build the statement...
v_sql_stmt :=
'MERGE INTO '
|| p_trgt_table_name
|| ' trgt
USING (
SELECT *
FROM '
|| p_srce_table_name
|| v_db_link
|| oracr
|| ' ) srce
ON ('
|| v_join_clause
|| ' )
WHEN MATCHED THEN UPDATE SET '
|| v_update_clause
|| 'WHEN NOT MATCHED THEN INSERT ('
|| v_insert_col_list
|| ' )
VALUES ('
|| v_insert_clause
|| ' )';
RETURN v_sql_stmt;
EXCEPTION
WHEN no_pk_and_no_join_cols THEN
raise_application_error (-20101, 'There is no primary key defined for the target table - and no join columns were passed.' || oracr || 'Please place a primary key on the table - or pass in the list of join columns for the MERGE statement.');
END create_merge_stmt;
/

SHOW ERRORS


Tom Kyte
June 03, 2005 - 5:30 pm UTC

I frequently write sql to write sql -- it is very handy indeed.

thanks

I owe YOU thanks, Tom!

Philip Moore, June 10, 2005 - 1:55 pm UTC

Dear Tom,

I just received my July/August 2005 issue of my favorite publication - Oracle Magazine. I must say I was THRILLED to see you mention my name (Philip from Cincinnati) in the Ask Tom section on page 71. I can't tell you how happy it made me that you mentioned me of all people.

I posted this little create_merge_stmt routine to provide a helpful utility for you Tom - because I can't tell you how much you, your site, and your Oracle Magazine article have helped me out of jams during my 4 years of working with the Oracle database (I'm a newbie by your standard - just since 8i).
I sometimes use the name Devil Dog - I am a former U.S. Marine - and that is a nickname we cherish believe it or not (derived from German "Teuful Huenden" which they branded the Marines after seeing their ferocity in WWI). But Devil Dog and me are one in the same.

At any rate - I won't waste more of your time - but I just wanted to say thanks for the true honor of referencing my suggestions on using GROUPING_ID. (Side note: my DBA Jon (as you guessed - I'm a Developer) - was thrilled to see his database schema in the test case I sent (dtmgt1)).

Thanks again sir for your recognition of us little guys :) - also for the GREAT service you provide to us Oracle lovers and users.

Sincerely,

Philip Moore - Hewlett Packard
(a.k.a. Devil Dog)


Tom Kyte
June 10, 2005 - 3:52 pm UTC

Excellent feedback -- thanks much!

I learn stuff here as well, and anytime someone teaches me something, I like to give credit where credit is due.


write sql

A reader, June 22, 2005 - 8:40 am UTC

"I frequently write sql to write sql" -
did you ever had to
write sql to write sql to write sql ?

Tom Kyte
June 22, 2005 - 4:33 pm UTC

no :)

that would be getting beyond obscure at that point!

write sql to write sql to write sql

A reader, June 27, 2005 - 2:56 pm UTC

I sometimes use

select
'select ''alter table ' || table_name || ' modify(' || column_name || ' varchar2('' || max(lengthb(' || column_name || ')) || ''))'' from ' || table_name || ''
from user_tab_columns
where data_type='VARCHAR2'



MERGE with no UPDATE

Jagannath Dalvi, July 05, 2005 - 11:31 am UTC

Hi Tom,

I would like to use MERGE command just for INSERTing new records, can I do that?

Something like:

MERGE INTO newemp n1
USING external_table e1
ON ( n1.empno = e1.empno )
WHEN MATCHED THEN
NULL ----- Do nothing....
WHEN NOT MATCHED THEN
INSERT ( empno, ename, job, mgr, hiredate, sal, comm, deptno )
values ( e1.empno,
e1.ename,
e1.job,
e1.mgr,
e1.hiredate,
e1.sal,
e1.comm,
e1.deptno )
/


Tom Kyte
July 05, 2005 - 11:39 am UTC

in 10g sure.

but -- you do not need merge for that, never had


insert into t select * from t2 where x not in ( select x from t );




Data volume and performance...

Jagannath Dalvi, July 05, 2005 - 11:52 am UTC

Thanks for the quick reply...

We have more than 100 million rows in source table and much more in target.... I think, IN command will be very slow in this case.

I am also trying to use BULK COLLECT but because memory leak, it dies after 25 million rows...

Any suggestion would be helpful.

Thanks.


Tom Kyte
July 05, 2005 - 12:15 pm UTC

guess what merge would do, in 10g when you can do just the insert part, it just does an "anti join" which is what NOT IN and this:

insert into t1
select t2.*
from t1, t2
where t1.key(+) = t2.key
and t1.key is null;

do. why in the world do you think merge would be better?!?

Thanks Tom!

Jagannath Dalvi, July 05, 2005 - 12:26 pm UTC




data volumn on Merge statement

J, July 07, 2005 - 3:21 pm UTC

Hi Tom,

I found it is consistently that when target table is big, Merge statement will not work well. Following is the explain plan from my merge statement. The target table DSS.LV has about 15mm records with size close to 3GB. When cost is low due to Nested Loop on target, I am keeping kicked off from system, due to ora-01555.

Is there soft limitation on Merge statement? what is the rule of thumb for target table size when using Merge? What can I do on this case? back to traditional way, do insert and update seperately?

Thanks for any suggestion.

I notice that my SYSTEM free space is low (23MB) out of 700MB. Will it matter? usually the size for SYSTEM tablespace is about 500-600MB. How can I identify the real problem here? Appreciate for the help as always.

MERGE STATEMENT, GOAL = CHOOSE 1700 1 1246
MERGE DSS LV
VIEW
NESTED LOOPS OUTER 1700 1 360
HASH JOIN 1699 1 204
TABLE ACCESS FULL DSP LV_KEY_TMP 17 16360 638040
PARTITION RANGE ITERATOR 683 684
TABLE ACCESS FULL DSP LV 1108 683 684 657312 108456480
PARTITION HASH ITERATOR KEY KEY
TABLE ACCESS BY LOCAL INDEX ROWID DSS LV 1 KEY KEY 1 156
INDEX UNIQUE SCAN DSS LV_PK KEY KEY 1


Tom Kyte
July 07, 2005 - 3:59 pm UTC

1555 means "your dba has insufficiently configured the system"

it better not be using system for anything.


3gb is pretty small, 15m records is pretty small, these are not HUGE numbers in any sense.

The 1555 happens when you need some UNDO for the read portion of your query but it doesn't exist, the dba has setup the rollback segments too small (or undo retention is not set high enough depending on version and whether you use automatic or manual undo).



followup on Merge

J, July 07, 2005 - 5:14 pm UTC

actually the query above couldn't continue due to high cost during run time. I removed the threshold for this program and it is still running. When as shown above, the cost from explain plan is 1700, the actual cost in v$sql_plan is as high as 437824. How does this number come out? How could I estimate the cost in advance like that?

Thanks!

Tom Kyte
July 07, 2005 - 5:26 pm UTC

explain plan shows you the cost.

Ravi Kumar, August 10, 2005 - 3:54 am UTC

This is the data from my EMP Table
RAVI-STDDEV>select empno,ename,deptno,sal from emp;

EMPNO ENAME DEPTNO SAL
---------- ---------- ---------- ----------
7369 SMITH 20 800
7499 ALLEN 30 1600
7521 WARD 30 1250
7566 JONES 20 2975
7654 MARTIN 30 1250
7698 BLAKE 30 2850
7782 CLARK 10 2450
7788 SCOTT 20 3000
7839 KING 10 5000
7844 TURNER 30 1500
7876 ADAMS 20 1100
7900 JAMES 30 950
7902 FORD 20 3000
7934 MILLER 10 1300

14 rows selected.

This is my DEPT_SAL Table
RAVI-STDDEV>desc dept_sal
Name Null? Type
----------------- -------- ------------
DEPT_ID NUMBER
SALARY NUMBER

RAVI-STDDEV>select * from dept_sal;

no rows selected

Now I want the total of the salary of each department should be stored in DEPT_SAL table and I will update that table time to time using the merge.
I issued following command..

RAVI-STDDEV>MERGE INTO DEPT_SAL D
2 USING EMP E
3 ON (D.DEPT_ID = E.DEPTNO)
4 WHEN MATCHED THEN UPDATE
5 SET SALARY = SALARY +SAL
6 WHEN NOT MATCHED THEN INSERT
7 (DEPT_ID,SALARY) VALUES(E.DEPTNO,E.SAL)
8 /

14 rows merged.

RAVI-STDDEV>SELECT * FROM DEPT_SAL;

DEPT_ID SALARY
---------- ----------
10 1300
10 5000
10 2450
30 950
30 1500
30 2850
30 1250
30 1250
30 1600
20 3000
20 1100
20 3000
20 2975
20 800

14 rows selected.

But I am expecting only 3 rows in this table (ie Department Number and their total).

Can you please let me know where I am wrong.

Thanks & Regards
Ravi Kumar



Tom Kyte
August 10, 2005 - 10:13 am UTC

you should be using a materialized view, it'll automate all of this. You cannot "from time to time" merge in the rows.

think about it, the first time you merged -- there were NO ROWS, they all were inserted.

The second time it would get really ugly.


Merge is not "a procedure", it is a SQL statement and hence works with sets, it is not like it inserts the first deptno = 10 and then updates IN THE SAME STATEMENT, there was no deptno 10 to begin with so there will never be anything to update. (and if it did hit the same row 2 times in the merge -- it would tend to raise an error "cannot get a stable set of rows" since the statement would NOT be deterministic anymore)

You would have to (with merge)

merge into dept_sal d
using ( select deptno, sum(sal) sal from emp ) E
on ( d.deptno = e.deptno )
when matched then update set salary = e.sal
when not matched then insert (....)


but what you are looking for is called a materialized view and can even support incremental changes if that makes sense.

See
</code> http://docs.oracle.com/docs/cd/B10501_01/server.920/a96520/mv.htm#721 <code>


Merging into a view

VA, August 13, 2005 - 6:46 pm UTC

Are there any restrictions with the MERGE command as far as merging into views is concerned?

merge into contact c
using (select * from tt where c001 in (select cust_no from contact)) t
on (t.c001=c.cust_no)
when matched then update set last_name=t.c002
when not matched then insert(cust_no,last_name) values (t.c001,t.c002)

This gives me a ORA-903 invalid table name

But when I write the same thing using procedural code like

begin
for rec in (select * from tt where c001 in (select cust_no from contact))
loop
update contact set last_name=rec.c002
where cust_no=rec.c001;
if (sql%rowcount=0) then
insert into contact(cust_no,last_name) values (rec.c001,rec.c002);
end if;
end loop;

it works fine.

What gives? Thanks

Tom Kyte
August 13, 2005 - 8:16 pm UTC

give example.


ops$tkyte@ORA9IR2> create table emp as select * from scott.emp;

Table created.

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> create table tt as select empno c001, ename c002 from scott.emp;

Table created.

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> create or replace view contact
  2  as
  3  select empno cust_no, ename last_name from emp;

View created.

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> merge into contact c
  2  using (select * from tt where c001 in (select cust_no from contact)) t
  3  on (t.c001=c.cust_no)
  4  when matched then update set last_name=t.c002
  5  when not matched then insert(cust_no,last_name) values (t.c001,t.c002)
  6  /

14 rows merged.
 

VA, August 13, 2005 - 8:24 pm UTC

I thought I did give an example. Yes, I can see that it works for simple views. But my CONTACT view is a view joining 4-5 tables. And like I showed, the procedural example works, so the view is updatable and all that, so why is the MERGE giving that strange error about invalid table name?

Tom Kyte
August 13, 2005 - 8:32 pm UTC

create table
create table
create table
create table

create view ...........

insert into table....
......

merge

resulting in error. Just like the example I gave. That is all, give us a test case to reproduce with. As small as possible

(I know that sometimes I sound like a broken record (if you know what a record is :) saying the same thing over and over and over -- but, if you just say "getting error from this, why" -- odds are the answer will be "I don't", so supplying a tiny test case does two things

a) avoids that back and forth

b) 999 times out of 1000 I find my own mistake making it and never need to ask someone "why"

)

VA, August 13, 2005 - 8:46 pm UTC

SQL> create table cust_test as select cust_id,cust_no,last_nm from cust where 1=2;

Table created.

SQL> create table cust_org_test as select * from cust_org where 1=2;

Table created.

SQL> create table custtaxphys_test as select * from custtaxphys where 1=2;

Table created.

SQL> create or replace view contact_test as select c.cust_no,c.last_nm last_name from cust_test c,cust_org_test co,custtaxphys_test ct
  2   where c.cust_id=co.cust_id(+) and c.cust_id=ct.cust_id(+);

View created.

SQL> create table ttt as select * from tt where 1=2;

Table created.

SQL> l
  1  merge into contact_test c
  2  using (select * from ttt where c001 in (select cust_no from contact)) t
  3  on (c.cust_no=t.c001)
  4  when matched then update set c.last_name=t.c002
  5* when not matched then insert (cust_no,last_name) values ('xyz','smith')
SQL> /
using (select * from ttt where c001 in (select cust_no from contact)) t
                            *
ERROR at line 2:
ORA-00903: invalid table name

SQL> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle9i Enterprise Edition Release 9.2.0.6.0 - 64bit Production
PL/SQL Release 9.2.0.6.0 - Production
CORE    9.2.0.6.0       Production
TNS for Solaris: Version 9.2.0.6.0 - Production
NLSRTL Version 9.2.0.6.0 - Production

 

VA, August 13, 2005 - 8:50 pm UTC

SQL>  get /tmp/f1
  1   merge into contact_test c
  2   using (select * from ttt where c001 in (select cust_no from <b>contact_test</b>)) t
  3   on (c.cust_no=t.c001)
  4   when matched then update set c.last_name=t.c002
  5*  when not matched then insert (cust_no,last_name) values ('xyz','smith')
  6  /
 using (select * from ttt where c001 in (select cust_no from contact_test)) t
                           *
ERROR at line 2:
ORA-00903: invalid table name

In Line 2, I was selecting from my old view, I fixed that but that didnt change anything.

The query in the USING is just to ensure that I never hit the WHEN NOT MATCHED part.

Thanks 

Tom Kyte
August 13, 2005 - 8:57 pm UTC

you do understand that I cannot run that, right???

Like I give you, you give me. A complete example.

An example I can actually *run*, on my machines, myself.

Something to reproduce with.

Like you see me provide time after time.

VA, August 13, 2005 - 10:56 pm UTC

You can run this on your machine

SQL> create table cust_test(cust_id int primary key,cust_no varchar2(100),last_name varchar2(100));

Table created.

SQL> create table cust_test_1(cust_id int primary key,foo varchar2(100));

Table created.

SQL> create table cust_test_2(cust_id int primary key,foo2 varchar2(100));

Table created.

SQL> create table tt(c001 varchar2(100),c002 varchar2(100));

Table created.

SQL> create or replace view contact_test as select c.cust_no,c.last_name from cust_test c,cust_test_1 c1,cust_test_2 c2
  2  where c.cust_id=c1.cust_id(+) and c.cust_id=c2.cust_id(+);

View created.

SQL> get /tmp/f1
  1   merge into contact_test c
  2   using (select * from tt where c001 in (select cust_no from contact_test)) t
  3   on (c.cust_no=t.c001)
  4   when matched then update set c.last_name=t.c002
  5*  when not matched then insert (cust_no,last_name) values ('xyz','smith')
  6  /
 using (select * from tt where c001 in (select cust_no from contact_test)) t
                   *
ERROR at line 2:
ORA-00903: invalid table name
 

Tom Kyte
August 14, 2005 - 9:30 am UTC

It is an "under documented" restriction that you cannot merge into a join view.

In your case, with the outer joins, the other tables are not updatable so the work around would be to merge into the base table itself. (it would also be a tad more efficient as well)

VA, August 14, 2005 - 10:03 am UTC

My original question was

"Are there any restrictions with the MERGE command as far as merging into views is concerned?"

I guess at that time you didnt think of the answer

"It is an "under documented" restriction that you cannot merge into a join view"

It doesnt seem to have anything to do with the outer join in my view. Even if I change those to inner joins, MERGE still complains.

I dont really understand this "restriction". If I rewrite the MERGE as procedural code (as I showed earlier), it works fine, so there is no "technical" reason behind the restriction, is there?

Thanks

Tom Kyte
August 14, 2005 - 11:39 am UTC

join view -- didn't say outer join.


It is not supported. I had never tried it.

You see, when asking questions and saying "this doesn't work", getting to the crux of the matter is infinitely faster when you say:

1) create this
2) create that
3) do this
4) observe what happens, is that right.


rather than

4) I observed this happening to some arbitrary structures, is that right.




VA, August 14, 2005 - 12:51 pm UTC

Well, my create view statement earlier did involve a join even though I had created the underlying tables using CTAS, but I see your point. You answer tons of questions everyday, the least we can do is make it easy for you to reproduce what we are asking. Point noted.

Anyway, since I was using the WHEN MATCHED part of MERGE only, I didnt really need to use MERGE, I built a dynamic UPDATE statement like

update target t set
(c1,c2,c3,...,cN) = (select c1,c2,c3,...,cN from source s where s.pk=t.pk)
where t.pk in (select s.pk from s);

Thanks for your help, as always.

Database job fails due to - ORA-00904: : invalid identifier

Mahesh Kumar, September 02, 2005 - 3:49 am UTC

Hi Tom

i found this error in alert log

ORA-12012: error on auto execute of job 188
ORA-00904: : invalid identifier
ORA-06512: at "SYS.DBMS_STATS", line 9375
ORA-06512: at "SYS.DBMS_STATS", line 9857
ORA-06512: at "SYS.DBMS_STATS", line 10041
ORA-06512: at "SYS.DBMS_STATS", line 10095
ORA-06512: at "SYS.DBMS_STATS", line 10072
ORA-06512: at line 1

############################
select job ,what from dba_jobs where job=188;

JOB WHAT
------- --------------------------------------
188 BEGIN DBMS_STATS.GATHER_SCHEMA_STATS(OWNNAME => 'UV_DICOM',CASCADE => TRUE); END;


Please help me to diagnose this problem.

Thanks

Tom Kyte
September 02, 2005 - 6:17 am UTC

you can either have your job turn on sql_trace then gather to see what it is failing on...

or work with support to diagnose the issue.

If you are curious, just grant yourself directly (NOT VIA A ROLE) alter sesssion, do the execute immediate to turn on trace, have the job run and see what sql is failing in the trace file.

Merge Question

BC, September 06, 2005 - 3:42 pm UTC

Tom,

I am attempting to merge data using the following.



CREATE TABLE SHIP
(TRACKING_NO VARCHAR2(50), SIGNATURE VARCHAR2(30), DELIVERY_DATE DATE )
/
CREATE TABLE APP_POD
(APP_POD_ID NUMBER(10,0), TRACKING_NO VARCHAR2(50), DELIVERY_DATE DATE, SIGNATURE VARCHAR2(30))
/
CREATE SEQUENCE APP_POD_SEQ
INCREMENT BY 1 START WITH 1 MINVALUE 1
MAXVALUE 999999999999999999999999999
NOCYCLE ORDER CACHE 20
/

INSERT INTO SHIP VALUES ('1Z4R0A510304878334',NULL,NULL)
/
INSERT INTO SHIP VALUES ('1Z4R0A510304878352',NULL,NULL)
/
INSERT INTO SHIP VALUES ('1Z4R0A510304878343',NULL,NULL)
/
INSERT INTO SHIP VALUES ('1Z4R0A510304878334','VENTURE',SYSDATE)
/
INSERT INTO SHIP VALUES ('1Z4R0A510304878352','PRIETO',SYSDATE)
/
INSERT INTO SHIP VALUES ('1Z4R0A510304878343','WHITLEY',SYSDATE)
/

MERGE INTO APP_POD A
USING ( SELECT *
FROM SHIP ) B
ON ( A.TRACKING_NO = B.TRACKING_NO)
WHEN MATCHED THEN
UPDATE
SET A.DELIVERY_DATE = B.DELIVERY_DATE,
A.SIGNATURE = B.SIGNATURE
WHEN NOT MATCHED THEN
INSERT ( A.APP_POD_ID,
A.TRACKING_NO,
A.DELIVERY_DATE,
A.SIGNATURE )
VALUES ( APP_POD_SEQ.NEXTVAL,
B.TRACKING_NO,
B.DELIVERY_DATE,
B.SIGNATURE );


SELECT *
FROM APP_POD;

APP_POD_ID TRACKING_NO DELIVERY_DATE SIGNATURE
---------- -------------------------------------------------- ---------------------- ------------------------------
1 1Z4XXXXXXXXXXXXX52 6-Sep-2005 15:09:48 PRIETO
2 1Z4XXXXXXXXXXXXX52
3 1Z4XXXXXXXXXXXXX34 6-Sep-2005 15:09:48 VENTURE
4 1Z4XXXXXXXXXXXXX34
5 1Z4XXXXXXXXXXXXX43 6-Sep-2005 15:09:48 WHITLEY
6 1Z4XXXXXXXXXXXXX43




How do I prevent the merge from creating duplicate records ? Your help as usual is highly appreciated.

Thanks

BC

Tom Kyte
September 06, 2005 - 9:06 pm UTC

I see no primary keys, hence there are no duplicates

You'll need to explain what you mean by "duplicates" I guess.

Merge Question

BC, September 07, 2005 - 10:34 am UTC

Tom,

The TRACKING_NO column in the APP_POD table should be and is unique on our systems, APP_POD_ID is the primary key for this table. I apologize that the example above does reflect it. The SHIP table may contain rows with duplicate TRACKING_NO. The above example will be a part of a stored procedure that will periodically refresh the APP_POD table with the most recent tracking information.

Thanks

BC

Tom Kyte
September 07, 2005 - 1:42 pm UTC

if you have unique keys, are you saying we are violation ??!?!

How does merge work?

VA, September 23, 2005 - 11:05 am UTC

Lets say I have a dest table with a pk. The source table does NOT have a unique constraint on the "pk" column, so there are multiple rows for a given pk value.

If I do

merge into dest d
using (select * from src) s
on (d.pk=s.pk)
when matched then update set d.col1=s.col1
when not matched then insert (pk,col1) values (s.pk,s.col1);

I get a ORA-0001 on the PK constraint on D.

Why would this be?

How does MERGE work? Does it do a for rec in (<using>) and loop over that and update, if sql%rowcount=0 then insert, for each row.

Or does it pre-determine the records to insert vs. update and separate them out. Then once the PK=1 record is inserted, the next time it sees it again, it would try to insert it again and get the ORA-0001?

Can you please clarify?

Thanks

Tom Kyte
September 23, 2005 - 8:14 pm UTC

merge works in general by doing an outer join (not in 10g and above where the when matched and when not matched are OPTIONAL) - and if the rows matched -- do an update, if not matched (outer join returned null) then insert

and if you try to update the same row twice with different values from the original source row, it'll fail (unable to get a stable set of rows to update)

Merge would never "insert row where x=5" and then "update where x=5" later - that would be totally non-deterministic.

read consistency for the READ part ( the outer join or just join ) and then current mode for the modifications.

How does merge work?

VA, September 23, 2005 - 11:14 am UTC

Here's an example

SQL>  create table target (i int primary key,j varchar2(20));

Table created.

SQL> create table source (i int,j varchar2(20));

Table created.

SQL> insert into source select 1,substr(object_name,1,20) from all_objects where rownum<=10;


10 rows created.

SQL> insert into target values (2,'Two');

1 row created.

SQL> l
  1  merge into target t using (select * from source) s
  2  on (s.i=t.i)
  3  when matched then update set t.j=s.j
  4* when not matched then insert (i,j) values (s.i,s.j)
SQL> /
merge into target t using (select * from source) s
*
ERROR at line 1:
ORA-00001: unique constraint (xxxx.SYS_C0079761) violated

Since I am merging on the PK, I should never get a ORA-0001, right? Or maybe MERGE doesnt behave like I think it does?

Thanks 

Merge

Bob B, September 23, 2005 - 8:34 pm UTC

I think you understand how it works, but I think the error message returned is not as accurate as it could be. It looks like the merge should've errored with "couldn't find a stable set of rows" error. The merge inserted a row and then tried to insert another row from the source with the same PK.

Tom Kyte
September 23, 2005 - 9:05 pm UTC

concurr in a way, but the unstable error is a special side effect of updating the same row twice, this is a case of "two new rows" - different and hard to discern from any other Unique constraint violation

think of a merge into EMP (primary key emp, join key EMPNO - but consider if we added a UNIQUE constraint on ENAME too....)

It isn't feasible to detect a "tried to insert the same emp twice" versus "you just tried to create the same named emp twice"

A reader, September 26, 2005 - 11:29 am UTC

Is it necesssary to have two different table to use merge?
The users insert a row from the front end gui, I want to insert it into the table if it doesnt exist, and if it exists I want to update. What should I show in -- on ( join ) -- in such a case

Tom Kyte
September 27, 2005 - 9:26 am UTC

merge into T
using ( select :a A, :b B, :c C, :d D from dual )
on (.....)
when matched ...
when not matched.....

But in this case, I think you'll find the logic of:

update t set ......
if ( sql%rowcount = 0 )
then
insert .....
end;

to be just as easy.

A reader, September 27, 2005 - 10:00 am UTC

update t set ......
if ( sql%rowcount = 0 )
then
insert .....
end;
-------------------------------

why not insert first and then update if the row already exists?



Tom Kyte
September 27, 2005 - 11:56 am UTC

In general it is more efficient to update and then try to insert.

Unless the row almost NEVER exists, you want to update then insert if not updated.


The insert would fail if the row exists.  In order to fail, it actually did the insert (generated redo, undo), then hits an error (likely duplicate key), then rollsback the row (undoes the changes to indexes, table) which generates yet more redo/undo, then queries the dictionary table cons$ to turn the constraint into a friendly name (runs more sql) and initiates the exception back to the client

then it updates.


Versus:

an update index unique scans by primary key, finds zero rows
then inserts.


Lets look at the two extremes - first run, all rows are inserted, second run, all rows are updated.

i'd rather have the penalty of the first run many times than the penalty implied by the second run sometimes!



ops$tkyte@ORA10GR1> drop table t1;

Table dropped.

ops$tkyte@ORA10GR1> drop table t2;

Table dropped.

ops$tkyte@ORA10GR1> drop table t3;

Table dropped.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> create table t1 ( x int primary key, y int, z int );

Table created.

ops$tkyte@ORA10GR1> create index t1_idx1 on t1(y);

Index created.

ops$tkyte@ORA10GR1> create index t1_idx2 on t1(z);

Index created.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> create table t2 ( x int primary key, y int, z int );

Table created.

ops$tkyte@ORA10GR1> create index t2_idx1 on t2(y);

Index created.

ops$tkyte@ORA10GR1> create index t2_idx2 on t2(z);

Index created.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> create table t3 as select object_id from all_objects ;

Table created.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> exec runstats_pkg.rs_start

PL/SQL procedure successfully completed.

ops$tkyte@ORA10GR1> begin
  2      for c in ( select * from t3 )
  3      loop
  4      begin
  5          insert into t1 values ( c.object_id, c.object_id, c.object_id );
  6      exception
  7          when dup_val_on_index
  8          then
  9                  update t1 set y = c.object_id, z = c.object_id where x = c.object_id;
 10      end;
 11      end loop;
 12  end;
 13  /

PL/SQL procedure successfully completed.

ops$tkyte@ORA10GR1> exec runstats_pkg.rs_middle

PL/SQL procedure successfully completed.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> begin
  2      for c in ( select * from t3 )
  3      loop
  4          update t2 set y = c.object_id, z = c.object_id where x = c.object_id;
  5          if ( sql%rowcount = 0 )
  6          then
  7                  insert into t2 values ( c.object_id, c.object_id, c.object_id );
  8          end if;
  9      end loop;
 10  end;
 11  /

PL/SQL procedure successfully completed.

ops$tkyte@ORA10GR1> exec runstats_pkg.rs_stop(50000)
Run1 ran in 1780 hsecs
Run2 ran in 2273 hsecs
run 1 ran in 78.31% of the time

Name                                  Run1        Run2        Diff
STAT...session pga memory                0      65,536      65,536
STAT...session logical reads       353,074     447,856      94,782
STAT...consistent gets               3,622      98,404      94,782
STAT...consistent gets from ca       3,622      98,404      94,782
LATCH.library cache pin             96,750     192,498      95,748
LATCH.library cache                 97,593     193,603      96,010
LATCH.cache buffers chains       1,455,133   1,599,054     143,921

Run1 latches total versus runs -- difference and pct
Run1        Run2        Diff       Pct
1,778,953   2,172,394     393,441     81.89%

PL/SQL procedure successfully completed.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> exec runstats_pkg.rs_start

PL/SQL procedure successfully completed.

ops$tkyte@ORA10GR1> begin
  2      for c in ( select * from t3 )
  3      loop
  4      begin
  5          insert into t1 values ( c.object_id, c.object_id, c.object_id );
  6      exception
  7          when dup_val_on_index
  8          then
  9                  update t1 set y = c.object_id, z = c.object_id where x = c.object_id;
 10      end;
 11      end loop;
 12  end;
 13  /

PL/SQL procedure successfully completed.

ops$tkyte@ORA10GR1> exec runstats_pkg.rs_middle

PL/SQL procedure successfully completed.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> begin
  2      for c in ( select * from t3 )
  3      loop
  4          update t2 set y = c.object_id, z = c.object_id where x = c.object_id;
  5          if ( sql%rowcount = 0 )
  6          then
  7                  insert into t2 values ( c.object_id, c.object_id, c.object_id );
  8          end if;
  9      end loop;
 10  end;
 11  /

PL/SQL procedure successfully completed.

ops$tkyte@ORA10GR1> exec runstats_pkg.rs_stop(50000)
Run1 ran in 7059 hsecs
Run2 ran in 727 hsecs
run 1 ran in 970.98% of the time

Name                                  Run1        Run2        Diff
STAT...session pga memory           65,536           0     -65,536
STAT...table fetch by rowid         95,416           0     -95,416
STAT...execute count               143,131      47,714     -95,417
STAT...redo entries                143,140      47,723     -95,417
LATCH.session allocation            95,462           0     -95,462
LATCH.SQL memory manager worka      97,028         138     -96,890
STAT...table scan blocks gotte     143,676         550    -143,126
STAT...calls to get snapshot s     190,842      47,713    -143,129
LATCH.In memory undo latch         143,140           0    -143,140
STAT...consistent gets - exami     238,546      47,715    -190,831
STAT...buffer is not pinned co     190,832           0    -190,832
STAT...no work - consistent re     239,092      48,257    -190,835
LATCH.library cache lock           191,164          32    -191,132
STAT...db block changes            287,357      96,016    -191,341
LATCH.shared pool                  286,722      47,773    -238,949
LATCH.library cache pin            477,616      95,472    -382,144
LATCH.row cache objects            429,585          12    -429,573
STAT...recursive calls             525,283      48,191    -477,092
STAT...consistent gets from ca     620,773      95,981    -524,792
STAT...consistent gets             620,773      95,981    -524,792
STAT...db block gets from cach     622,394      48,869    -573,525
STAT...db block gets               622,394      48,869    -573,525
LATCH.library cache                716,511      95,524    -620,987
STAT...session logical reads     1,243,167     144,850  -1,098,317
LATCH.cache buffers chains       2,780,018     387,769  -2,392,249
STAT...table scan rows gotten    3,461,115     360,085  -3,101,030
STAT...undo change vector size   7,789,004   4,544,836  -3,244,168
STAT...redo size                32,253,444  12,562,056 -19,691,388

Run1 latches total versus runs -- difference and pct
Run1        Run2        Diff       Pct
5,422,938     645,024  -4,777,914    840.73%

PL/SQL procedure successfully completed.
 

Fastest method to refresh Data..

Karmit, October 13, 2005 - 9:24 am UTC

Hi,

From the top of your mind what method would you recommend (or think "normally works best") - assuming performance is the criteria - to refresh a large table daily.

The table is having ~3 million records and the "refresh" would basically consist of 5-15% updates and 20-40% inserts.

The different methods I can of think are:

1. SET based DML (MINUS/ UNION etc)
2. MERGE
3. INSERT followed by UPDATE (using BULK method)

I'll be doing some tests.. but just wanted to know if there
is something which is know to work best.. but again, you might comment that if a certain way works best, Oracle wouldn't have so many options!. In that case, would like to know which cases are better suited for methods listed above (or any which I missed, apart from Streams - which we are not planning to implement currently).

Thanks,
Karmit

Tom Kyte
October 13, 2005 - 10:54 am UTC

depends on the number of indexes. But if a table has ~3mill record and you update 5/15% of them and insert 20/40% (??? 20/40% of what?) - the table would be constantly growing at a very fast pace?


are there deletes here as well?

Fastest method to refresh Data

Karmit, October 13, 2005 - 4:43 pm UTC

Hi,

1. Yes.. tables would be growing at a fast rate. These are
partitioned tables - one partition for a day and we
will be archiving data (removing partitions) at some
stage (5yrs+). Its a datawarehouse (some tables are
composite partitioned as well). Maybe I should have said
updates of existing records at any time... different
tables have different update/insert percentages.

2. Yes. there are indexes. Lets say on average 3 per table.

So in light of the above which method would suit which kind of tables best? some reasoning behind it will help me
in seeing some light(!) and I could then formulate/deduce test results accordingly (Its Oracle 10G).

Tom Kyte
October 13, 2005 - 9:01 pm UTC

well, then there insufficient data to say here -- 5/15% of updates - against a single partition? all partitions? how many partitions are we talking about?

the inserts -- all into a single partition? any of the partitions?

help me see the light first ;)

Fastest method to refresh data...

Karmit, October 14, 2005 - 6:05 am UTC

Hi Tom,

Ok! I have tried to summarize the nature of the tables below. The "Legend" is the format in which the character
of the tables and refresh has been listed.

Hope it helps you in helping me!! :-)

Thanks & Regards,
Karmit


Legend
======
(
#Records in Table
,Partitioned?
,Partition Scheme?
,%Updates and nature?
,%Inserts and nature?
)


Table Type 1
============
(
500 Million
,Yes
,List Partitioned. There are 50 Partitions.
,1-10%. The update can be across partitions as the loader can refresh any list type.
,1-15%. The insert can be across partitions as the loader can refresh any list type.
)


Table Type 2
============
(
2 Billion
,Yes
,Range-List Partitioned. There are 25 List Partitions and each one is then subpart per day. Data for 4 years.
,0%-5%. But only ever to 1 particular list subpartition for the day [refresh date].
,1-10%. Across all sub-partitions for a particular day [refresh date].
)


Table Type 3
============
(
20 Mil
,No
,N/a
,0-1%
,1-5%
)


Table Type 4
============
(
1 Mil
,No
,N/a
,0-1%
,1-5%
)



Table Type 5
============
(
20 Mil
,No
,N/a
,0%
,5-10%
)


--


Tom Kyte
October 14, 2005 - 8:14 am UTC

type 1 looks like a merge candidate. update existing, insert new rows in one statement.


type 2 - not sure what you mean by

Range-List Partitioned. There are 25 List Partitions and each one is then subpart per day. Data for 4 years.

you could have range partitions with 25 SUBpartitions that are lists, but it sounds like you are saying the opposite?

type 2 might be an update against the partitions in question (using the hardcoded partition key to eliminate all but the one set in question), followed by an insert over all partitions where the key isn't in the set of rows you just updated.



the rest are merge candidates due to the lack of partitioning

#2 is the only "special" one since you know you will update on

Fastest method to refresh..

Karmit, October 14, 2005 - 8:56 am UTC

Hi,
Yes.. you'r correct. I should have said its range
partitioned per day and within each day its sub-partitioned
for 25 lists.

Why is MERGE better for non-partitioned tables?

Thanks,
Karmit

Tom Kyte
October 14, 2005 - 9:11 am UTC

merge will do it in one pass and since you cannot "optimize" either of the update or insert against that single partition (a non-partitioned table could be considered a table with a single partition) like we might be able to with case 2 (the update only has to consider a few, not all partitions).....

My theory - when you are going to hit all partitions in both the update and insert - just merge, merge will hit all partitions one time.

when you can eliminate most of the partitions for the update, but not the insert - maybe we don't want to merge - but rather use a two step.

Fastest method of refresh..

karmit, October 14, 2005 - 9:00 am UTC

Hi Tom,

You commented "the rest are merge candidates due to the lack of partitioning".. yet for Table Type 1 - which is a partitioned table - the suggested method is MERGE?

Just trying to figure out the reasoning behind the
recommended approach for all types.

Thanks,
Karmit

Tom Kyte
October 14, 2005 - 9:13 am UTC

see above "my theory"....

if you can effectively use partition elimination on one of the two steps (update or insert) - maybe we want to two step it and ensure the update can use partition elimination to the best possible end.

true or false

karthick, October 14, 2005 - 11:30 am UTC

merg statement is some thing like

update ....

if sql%notfound
then
insert ....
end if;

true or false

Tom Kyte
October 14, 2005 - 5:22 pm UTC

something like that - yes.

but, it would be more like:


for x in ( select * from source )
loop
    if record exists
    then
        if record hasn't been modified by us yet
        then
              update it
        else
              fail
        end if
    else
        insert it
    end if
end loop




ops$tkyte@ORA10GR1> create table t1 ( x int, y int );

Table created.

ops$tkyte@ORA10GR1> create table t2 ( x int, y int );

Table created.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> insert into t1 values ( 1, 3 );

1 row created.

ops$tkyte@ORA10GR1> insert into t1 values ( 2, 1 );

1 row created.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> merge into t2 using t1
  2  on (t1.x = t2.x)
  3  when matched then update set y = t1.y
  4  when not matched then insert (x,y) values (t1.x,t1.y);

2 rows merged.

ops$tkyte@ORA10GR1> /

2 rows merged.

ops$tkyte@ORA10GR1> insert into t1 values ( 1, 1 );

1 row created.

ops$tkyte@ORA10GR1>
ops$tkyte@ORA10GR1> merge into t2 using t1
  2  on (t1.x = t2.x)
  3  when matched then update set y = t1.y
  4  when not matched then insert (x,y) values (t1.x,t1.y);
merge into t2 using t1
*
ERROR at line 1:
ORA-30926: unable to get a stable set of rows in the source tables
 

Fastest method of refresh.

Karmit, October 15, 2005 - 7:00 pm UTC

H Tom,
Assuming we go for the INSERT..UPDATE for the partitioned tables where it can perform "pruning" - what would you expect to perform better, SET operations or BULK operations?
Thanks,
Karmit.

Tom Kyte
October 16, 2005 - 8:03 am UTC

set

true or false

karthick, October 17, 2005 - 3:50 am UTC

merge..
on(a.x = b.x and a.y = b.y and a.z = b.z)...

for above statement to succeed the below statement must be true

select x,y,z,count(1) cnt from a group by x,y,z (here all the row should return only 1 as its count)

that is if there is a duplicate row exist in source table then the merge will tell us

"ORA-30926: unable to get a stable set of rows in the source tables"

but update ... sql%notfound insert.. will update both the rows

am i correct tom.


Tom Kyte
October 17, 2005 - 7:45 am UTC

the "using" rowsource's "join key" <b>should</b> be unique, yes.  However, there are times when it need not be - but you should definitely avoid it:

ops$tkyte@ORA9IR2> create table t1 ( x int, y int, z int, a int );

Table created.

ops$tkyte@ORA9IR2> create table t2 ( x int, y int, z int, a int );

Table created.

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> insert into t1 values ( 1, 1, 1, 0 );

1 row created.

ops$tkyte@ORA9IR2> insert into t2 values ( 1, 1, 1, 1 );

1 row created.

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> merge into t1 using t2
  2  on (t1.x = t2.x and t1.y = t2.y and t1.z = t2.z)
  3  when matched then update set a = t2.a
  4  when not matched then insert (x,y,z,a) values ( t2.x,t2.y,t2.z,t2.a );

1 row merged.

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> insert into t2 values ( 1, 1, 1, 1 );

1 row created.

ops$tkyte@ORA9IR2> merge into t1 using t2
  2  on (t1.x = t2.x and t1.y = t2.y and t1.z = t2.z)
  3  when matched then update set a = t2.a
  4  when not matched then insert (x,y,z,a) values ( t2.x,t2.y,t2.z,t2.a );

2 rows merged.


<b>Here, the second merge (update phase) worked because the values didn't actually change, didn't conflict....</b>

that should be avoided for logical reasons.


 

Very useful tip

Shreyas Desai, October 19, 2005 - 1:38 am UTC

Thanks for the tip Tom.. was very useful, as I was facing the same problem and it got solved due to your tip

merge

karthick, October 24, 2005 - 4:43 am UTC

so tom let me know why it works that way.... oracle did it that way for some purpose or the statement came out that way.... and what is the reason behind the invention of this statement.... what situation in which it should be used (as per the statement creator's mind)


When matched then updateÂ…
When not matched then insertÂ…

Is this the way the statement should be or we can use any dml statement we want i.e. a delete.

Tom Kyte
October 24, 2005 - 6:18 am UTC

merge was designed to replace procedural code that did this:

for x in ( select ... )
loop
update t ...
if ( sql%rowcount = 0 )
then
insert ...
end if;
end loop;


In 10g, you can use either of when matched and when not matched (don't need both) and the when matched may have an optional delete component as well.

is MERGE a *BLUK* operation?

darin, November 02, 2005 - 4:34 pm UTC

Tom,

Is merge a BULK type operation (i.e. avoids unnecessary context switches b/w PL/SQL & SQL), or more like single stepping through a cursor for loop, which performs a context switch per row.

I'm trying to decide b/w using a single merge statement, or 2 BULK cursor statements.

thannks in advance

Tom Kyte
November 03, 2005 - 6:49 am UTC

yes, merge is bulk

Please check its accuracy!

Vikas Khanna, November 15, 2005 - 8:31 pm UTC

Hi Tom,

We have some tables in the OLTP database, which on a timely basis we need to get the changed records to our warehouse system. I have written a procedure which will dynamically build a MERGE command for each table passed.

Please see to it if there are any issues with this.There are about 24 tables for which these MERGE statements will build up.

declare
String_merge VARCHAR2(500);
String_matched VARCHAR2(500);
String_not_matched VARCHAR2(500);
p_first_time int := 0;
pk_column_name VARCHAR2(4000);
String_inserted VARCHAR2(4000);
begin
for x in (Select table_name from user_tables where table_name = 'DEPT') loop
Select ucc.column_name into pk_column_name from user_constraints uc, user_cons_columns ucc
where
ucc.table_name = x.table_name and uc.constraint_name = ucc.constraint_name and uc.constraint_type = 'P';
for y in (Select column_name from user_tab_columns where table_name = x.table_name order by column_id) loop
if p_first_time = 0 then
String_merge := String_merge || 'MERGE INTO ' || x.table_name || ' DWH USING '|| x.table_name
|| '@db_link' || ' OLTP ON ' || '(' || 'DWH.' || pk_column_name || ' = ' || 'OLTP.' || pk_column_name || ')';
String_matched := ' WHEN MATCHED THEN UPDATE SET DWH.' || y.column_name || ' = ' || 'OLTP.' || y.column_name || ',';
p_first_time := 1;
String_not_matched := ' WHEN NOT MATCHED THEN INSERT (' || y.column_name || ',';
String_inserted := 'OLTP.' || y.column_name || ',';
else
String_matched := String_matched || 'DWH.' || y.column_name || ' = ' || 'OLTP.' || y.column_name || ',';
String_not_matched := String_not_matched || y.column_name || ',';
String_inserted := String_inserted || 'OLTP.' || y.column_name || ',';
end if;
end loop;
String_matched := Substr(String_matched,1,length(String_matched)-1);
String_not_matched := Substr(String_not_matched,1,length(String_not_matched)-1) || ') VALUES (' ;
String_inserted := Substr(String_inserted,1,length(String_inserted)-1) || ')';
end loop;
String_merge := String_merge || String_matched || String_not_matched || String_inserted;
dbms_output.put_line(String_merge);
end;
/


Tom Kyte
November 16, 2005 - 8:39 am UTC

sorry, I cannot debug your code for you - that is what peer review at work, code review at work, psuedo code, design docs, testing and QA are for....

Need your advice!

Vikas Khanna, November 16, 2005 - 5:55 pm UTC

Hi Tom,

I have tested the pull from production to Datawarehouse using the script and is working fine, with some modifications.

However I need your advice in this case presented below:

Production: Datawarehouse
Table T Table T
10:20 PM 10:20 PM
10:25 PM A row gets inserted but
not committed.
10:30 PM another row gets inserted
but committed.
Jobs runs after 10
mins to get the
incremental
changes. Get me all records whose updated_date > 10:20 PM as this is the MAX date.

Pull does happen which gets the DW one row, which is committed.

Now next time the job runs will have to get those records which have time > 10:30 PM. Thus that record will NEVER come to the datawarehouse.

Please suggest besides MAX date can we use something else to take care of the missing record.

How will SCN implementation work.

Regards,


Tom Kyte
November 16, 2005 - 6:28 pm UTC

</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:16998677475837 <code>

you can use v$transaction to find the time to use for the next refresh.

help

Kumar, November 17, 2005 - 7:25 am UTC

Hi Tom,
The insert part in merge is not working.

SQL> create table ttab1(x int,y int)
/
Table created.

SQL> insert into ttab1 values(4,22)
/
1 row created.

SQL> select * from ttab1
  2  /

         X          Y
---------- ----------
         4         22

1 row selected.

SQL> merge into ttab1
  2  using dual on (x=3)
  3  when matched then
  4   update set y = y+20
  5  when not matched then
  6   insert values(100,1001)
  7  /

0 rows merged.

SQL> select * from ttab1
  2  /

         X          Y
---------- ----------
         4         22

1 row selected.

why the insert is not working??
 

Tom Kyte
November 17, 2005 - 8:20 am UTC

eh? (x=3)???



Know aboout INSERTS and UPDATES

Vikas Khanna, November 17, 2005 - 4:11 pm UTC

Hi Tom,

Can we find out how many records have bee inserted and how many got updated when we are using the MERGE command

Thanks

Tom Kyte
November 18, 2005 - 10:02 am UTC

No, there is no such facility, just the number of rows affected is returned.

OK

A reader, November 18, 2005 - 12:31 am UTC

Hi Tom,
Thanks for your reply.
I thought When we use x=3 the insert will work since it is
not the matching condition.
How to make that Insert work??

Tom Kyte
November 18, 2005 - 10:33 am UTC

join the tables together? merge is about joining tables together, if you are not joining then you should just be using insert.

Merge Command

Antonio Diaz, December 02, 2005 - 12:22 pm UTC

Congratulation Tom.



A reader, December 02, 2005 - 2:38 pm UTC


overcoming MERGE determinism

Duke Ganote, December 02, 2005 - 2:43 pm UTC

Tom-- I get a flat file with a month of change data; call it SOURCE.

create table SOURCE as
select mod(level,5) primary_key -- an object identifier
, level AS row_number -- sequence of operations
, level AS payload -- object state information
from dual connect by level < 15
/

There's a TARGET table that will get the latest state of each object:

create table target ( primary_key number primary key,
payload number )
/

This fails:

MERGE INTO target T
USING ( select * from source order by row_number ) S
on ( T.primary_key = S.primary_key )
when matched then update
set t.payload = s.payload
when not matched then insert
( t.primary_key, t.payload )
values
( S.primary_key, S.payload )
/
*
ERROR at line 1:
ORA-00001: unique constraint (SYS_C006637) violated

The error threw me, but appears to be because "MERGE is a deterministic statement. That is, you cannot update the same row of the target table multiple times in the same MERGE statement." See
</code> http://download-west.oracle.com/docs/cd/B14117_01/server.101/b10759/statements_9016.htm#SQLRF01606 <code>

However, I can get the "latest state" of each object like so:

select *
from
(
select s1.*, max(r#) over (
partition by primary_key
) r#max
from (
select s.*,
row_number() over (
partition by primary_key
order by row_number) r#
from source s
) s1
) S2
where r# = r#max
/
PRIMARY_KEY ROW_NUMBER PAYLOAD R# R#MAX
----------- ---------- ---------- ---------- ----------
0 10 10 2 2
1 11 11 3 3
2 12 12 3 3
3 13 13 3 3
4 14 14 3 3

and use that to merge into TARGET:

MERGE INTO target T
USING (
select S2.* from (
select s1.*, max(r#) over (
partition by primary_key
) r#max
from (
select s0.*,
row_number() over (
partition by primary_key
order by row_number) r#
from source S0
) s1
) S2
where r# = r#max
) S
ON ( T.primary_key = S.primary_key )
WHEN MATCHED THEN UPDATE
set t.payload = s.payload
WHEN NOT MATCHED THEN INSERT
( T.primary_key, T.payload )
VALUES
( S.primary_key, S.primary_key )
/

but is there a better way?

Tom Kyte
December 03, 2005 - 10:12 am UTC

ops$tkyte@ORA10GR2> MERGE INTO target T
  2   USING ( select *
  3             from (select row_number()
                 over (partition by primary_key order by row_number desc) rn,
  5                          source.*
  6                     from source)
  7            where rn = 1 ) S
  8  on ( T.primary_key = S.primary_key )
  9  when matched then update
 10     set t.payload = s.payload
 11  when not matched then insert
 12   ( t.primary_key, t.payload )
 13   values
 14   ( S.primary_key, S.payload )
 15  /

5 rows merged.
 

Too many notes, Mozart...

Duke Ganote, December 03, 2005 - 8:46 am UTC

Maybe this? Then wrap in a MERGE...

select * from (
select s1.*
, max(s1.row_number) over
(partition by primary_key) r#max
from source s1
) where r#max = row_number
/

Tom Kyte
December 03, 2005 - 10:30 am UTC

that is another way to reduce the work - yes.

Ya'll are having too much fun with "Where in the world are you?"

A reader, December 05, 2005 - 10:10 am UTC


RE: Too much fun

Duke Ganote, December 05, 2005 - 2:56 pm UTC

Ask an open-ended question, and you deserve the answer you get I figure :)

RE: Too many notes...

Duke Ganote, December 05, 2005 - 3:34 pm UTC

I assure readers that my quote from Emperor Joseph II was directed at my original, too verbose query, not Tom's response. It's not obvious that I'd posted the "Too many notes, Mozart" review before Tom responded to my preceding post.

Tom can jump and edit at will, like the 3D visitor dropping in on Flatland.
</code> http://www.alcyone.com/max/lit/flatland/15.html <code>

Merge with oracle 10g

Madhulika Pendse, January 24, 2006 - 5:46 am UTC

Hi,
this was of help. but now i have one more query. In Merge command say i am using T1 and T2 which are identical. I want to update T1 as per T2. hence i use

Merge into T1
using T2
on (T1.empno=T2.empno and T1.dept=T2.dept)
when matched then
--- update some thing
when not matched
--- insert something

Now this is ok for newly inserted records. But say there are 10000 records in T1 and T2 which are matching out of which only 100 are actually updated in T2... hence we just need to update only those 100 records in T1 ( As per chanes in T2), where as Merge command will update entire 10000 rows which were matched.
And till this is commited, all updated records will occupy Rollback segment.
How can i avoid this.
Pls Help

Tom Kyte
January 24, 2006 - 8:15 am UTC

I don't know what you mean by "only 100 are actually updated in T2"

A reader, February 03, 2006 - 5:00 pm UTC

Tom,

By using merge command, can we get how many rows updated and inserted individually. This can be achieved by querrying the table again, but without that, can we get the counts? Please advise.

Tom Kyte
February 03, 2006 - 5:29 pm UTC

No you cannot.

Use MERGE to Refresh Data Warehouse

Su Baba, February 03, 2006 - 7:33 pm UTC

Hi Tom,

I'm trying to simulate a very very simple data warehouse extraction and load process. Please see the example below.

In the following example, I have a fact table, fact_tbl, and a dimension table, customer_dim. fact_tbl gets its values solely from the "customers" table. customers.total_purchase will become fact_tbl.revenue.

To refresh customer_dim table, I use MERGE command (see below).

* Can MERGE be used for updating/inserting into 2 tables at the same time?
In this case, can I use just one MERGE command to populate both
customer_dim and fact_tbl tables?

If not, what's the best way to refresh the fact table?

thanks

-- ----------------------------------------------------------------------------
-- OLTP
-- ----------------------------------------------------------------------------
CREATE TABLE city (
city_id INT PRIMARY KEY,
city VARCHAR2(15) NOT NULL
);

CREATE TABLE state (
state_id INT PRIMARY KEY,
state CHAR(2) NOT NULL
);

CREATE TABLE customers (
customer_id INT PRIMARY KEY,
name VARCHAR2(15),
city_id INT REFERENCES city (city_id),
state_id INT,
total_purchase INT
);

INSERT INTO city VALUES (1, 'San Jose');
INSERT INTO city VALUES (2, 'San Carlos');
INSERT INTO city VALUES (3, 'Boston');

INSERT INTO state VALUES (1, 'CA');
INSERT INTO state VALUES (2, 'MA');

INSERT INTO customers VALUES (1, 'John', 1, 1, 25000);
INSERT INTO customers VALUES (2, 'Mary', 3, 2, 30000);
INSERT INTO customers VALUES (3, 'Larry', 2, 1, 15000);
INSERT INTO customers VALUES (4, 'Michelle', 2, 1, 45000);
commit;

SELECT a.customer_id, a.name, b.city, c.state, a.total_purchase
FROM customers a, city b, state c
WHERe a.city_id = b.city_id AND
a.state_id = c.state_id;

CUSTOMER_ID NAME CITY_ID STATE_ID TOTAL_PURCHASE
----------- --------------- ---------- ---------- --------------
1 John 1 1 25000
2 Mary 3 2 30000
3 Larry 2 1 15000
4 Michelle 2 1 45000

-- ----------------------------------------------------------------------------
-- Data Warehouse
-- ----------------------------------------------------------------------------
DROP TABLE customer_dim;
DROP TABLE fact_tbl;

CREATE TABLE customer_dim (
customer_key INT PRIMARY KEY,
customer_id INT NOT NULL,
name VARCHAR2(15),
city VARCHAR2(15),
state CHAR(2)
);

CREATE TABLE fact_tbl (
customer_key INT PRIMARY KEY,
revenue INT
);

CREATE SEQUENCE customer_dim_s START WITH 100;

-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-- Refreshing the dimension table.
-- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
MERGE INTO customer_dim D
USING (
SELECT a.customer_id, a.name, b.city, c.state, a.total_purchase
FROM customers a, city b, state c
WHERe a.city_id = b.city_id AND
a.state_id = c.state_id
) X
ON (X.customer_id = D.customer_id)
WHEN MATCHED THEN
UPDATE SET D.name = X.name,
D.city = X.city,
D.state = X.state
WHEN NOT MATCHED THEN
INSERT (D.customer_key, D.customer_id, D.name, D.city, D.state)
VALUES (customer_dim_s.NEXTVAL, X.customer_id, X.name, X.city, X.state);

4 rows merged.


SELECT * FROM customer_dim;

CUSTOMER_KEY CUSTOMER_ID NAME CITY ST
------------ ----------- --------------- --------------- --
100 4 Michelle San Carlos CA
101 3 Larry San Carlos CA
102 1 John San Jose CA
103 2 Mary Boston MA



Tom Kyte
February 05, 2006 - 11:55 pm UTC

merge works on "a table or view".

It will modify one table.

If your source query is extremely expensive to evaluate and you would like to persist the results for both modifications, global temporary tables pop into mind.

Need to update only rows which are updated

Madhulika Pendse, February 06, 2006 - 7:32 am UTC

Look into this.
I have emp table and emp_bak table. I am updateing emp_bak table as per emp at interval of 2 hrs. I am using MERGE. e.g I have following records in emp at 10AM
empno, ename, salary 1)100,Jane,10000 2)200,jonhy,20000 3)300,dick,30000. i update this in emp_bak.Now at 12AM,only empno=100 data gets changed. rest empno=200 & 300 remains same. also empno=400 gets added. So in when_not_matched, condition for empno=400 will get catched and hence inserted in emp_bak. but in when_matched condition (since day only empno is PK), all the three records will get catched. Hence will get updated. but ideally i just want to update only empno=100 record.
Am i able to make the requirement clear. can anyone help me? This will have rollback seg full..and if i check for update of individual data in a single row, this will become extreemly slow in case of huge tables.

Direct Path Load for MERGE

Su Baba, February 13, 2006 - 9:03 pm UTC

Can /*+ APPEND */ hint be used with MERGE?

Tom Kyte
February 14, 2006 - 7:43 am UTC

yes, the insert component of a merge can be done in append mode.

"MERGE is a deterministic statement" - could you please explain?

Oraboy, February 17, 2006 - 11:40 am UTC

Hi Tom,
I thought Merge statement would work row-by-row. Is it a wrong assumption?

This is what I tried..

-- consider "data" as my source table
-- -----------------------------------
10g>select * from data;

C1 C2
---------- ----------
row 1 column 1
row 2 column 0
row 0 column 1
row 1 column 0
row 2 column 1
row 0 column 0
row 1 column 1
row 2 column 0
row 0 column 1

9 rows selected.

10g>select distinct c1,c2 from data;

C1 C2
---------- ----------
row 1 column 1
row 2 column 0
row 0 column 0
row 0 column 1
row 1 column 0
row 2 column 1

6 rows selected.

-- thats exactly what I want to upload (no duplicates)
--

-- "target" is my destination table
-- --------------------------------

10g>select * from target;

no rows selected

10g>merge into target t using data d
2 on (t.c1=d.c1 and t.c2=d.c2)
3 when not matched then
4 insert values(d.c1,d.c2)
5 /

9 rows merged.

10g>select * from target;

C1 C2
---------- ----------
row 0 column 1
row 0 column 1
row 2 column 0
row 2 column 0
row 1 column 0
row 1 column 1
row 1 column 1
row 2 column 1
row 0 column 0

9 rows selected.

-- I expected only the distinct rows ..
-- but it did upload all 9 rows from source
--


10g>merge into target t using data d
2 on (t.c1=d.c1 and t.c2=d.c2)
3 when not matched then
4 insert values(d.c1,d.c2)
5 /

0 rows merged.


-- but if I rerun, it checks for matches, and since all rows in source find a match , no rows are merged.

10g>

I checked the docs (</code> http://download-east.oracle.com/docs/cd/B19306_01/server.102/b14200/statements_9016.htm#SQLRF01606 <code>, there is this line which says "Merge is a deterministic statement".. is that what I am missing to understand? (and hence the result)

My thinking on "merge" was more like this
take a row from source table
loop
<<check for condition>>
if matched do match part;
if not matched do the corresponding part;
repeat the loop with next row from source until last row


version: 10gR2

Thanks in advance


Tom Kyte
February 17, 2006 - 2:56 pm UTC

merge is a set operation.

It takes SOURCE and TARGET and merges them - TARGET doesn't "change" during the merge, rows that were inserted at the beginning of the merge cannot be updated later (that - that would make it NON-DETERMINISTIC!!!! the order of rows would affect the outcome making the same merge produce different results just because the bits were aligned differntly on system 1 from system 2)

follow-up to prev question

oraboy, February 17, 2006 - 11:58 am UTC

As an additional note to my prev question-

I know I can do this by changing the merge statement into something like
merge into target
using (select distinct ... from source)
on <<condition>>
when...;

but in real situation , I have a flatfile with millions of rows in it and I am trying to extract and populate dimension table from that source. ( flat file has lots of other columns (attributes) related to master key)..
and I need to check only for match on one/more master key fields and perform
"if a match is not found
insert into dimension table key values and it related attributes"

(means sorting is not something I can use here..Instead I would use PL/SQL bulk fetch & insert )

Thanks again











"log errors" to help

Oraboy, February 17, 2006 - 12:18 pm UTC

This looks like a viable workaround..

10g>create unique index target_idx on target(c1,c2)
2 /

Index created.

10g>exec dbms_errlog.create_error_log('TARGET','TARGET_ERR');

PL/SQL procedure successfully completed.

10g>merge into target t using data d
2 on (t.c1=d.c1 and t.c2=d.c2)
3 when not matched then
4 insert values(d.c1,d.c2)
5 log errors into target_err ('badrows') REJECT LIMIT UNLIMITED
6 /

6 rows merged.

10g>ed
Wrote file afiedt.buf

1 select c1,c2,ora_err_mesg$ from target_err
2* where ora_err_tag$='badrows'
10g>/

C1 C2 ORA_ERR_MESG$
------- -------- ----------------------------------------
row 1 column 1 ORA-00001: unique constraint (SYSTEM.TAR
row 2 column 0 ORA-00001: unique constraint (SYSTEM.TAR
row 0 column 1 ORA-00001: unique constraint (SYSTEM.TAR

10g>

a) I just wanted to know your thoughts abt this approach. (whether it makes sense or not)

b) Since I dont care about LOG_ERRORS ..is there anyway I cant say "do nothing" if error is encountered
something like
insert into ..
log_errors null reject Limit unlimited; (do nothing)

Thanks for your time


Tom Kyte
February 17, 2006 - 2:59 pm UTC

why not just select out the data you actually want to insert?????


using (select from data DISTINCTLY <<<<=== )


eg:

select *
from (
select data.*, row_number() over (partition by c1, c2 order by something) rn
from data
)
where rn = 1;


use THAT as your source.

great

oraboy, February 17, 2006 - 3:23 pm UTC

super..exactly what I needed.

Analytics and YOU ,rock!

Fantastic!!

Oraboy, February 17, 2006 - 3:31 pm UTC

and for future readers who would come across this ..just thought of posting this follow-up statistic to prove two things
a)how effectiveness of Tom's solution.
b)the power of analytics.

the total number of rows in my source was 11+ million rows

PL/SQL approach (bulk limit 100) -
session started (v$session_longops showed estimated time 23 hours remaining after 2 minutes)

vs
Merge using Analtyical SQL
"
4792 rows merged.

Elapsed: 00:03:49.31
"

Thanks once again ,Tom

meger comparisons wrtcontention over dblink

dani, February 23, 2006 - 3:47 am UTC

Hi Tom,
I have two physicall different database servers. I want to replicate 1 table SAY TAB1 from DB1 to DB2.
The table is almost static in terms of rows i.e it has 700 rows always. But the DB1 is updating this table very very very heavily, all its columns. The update is so heavy that it is like 200 updates(200 commits means 200 trans) per sec.

I have written a procedure at the DB1 , which wakes up every 1 sec, and replicates all contents of tab1 to DB2 via dblink.
And the the tab1 at DB2 will never be updated but only read from users who are connected to db2.

There should not be any issues of dblink, becuase my connection at DB1 stay connected all the time(that means dblink will also stay alive across db2).

I have tried different approach and like your opinion on this please, so tell me which approach is better:

Approach1:
============
declare
begin
FOR C IN (SELECT * FROM SMDT_TEST ORDER BY SMDT_STK_CD, SMDT_REC_TYP)
LOOP
DELETE SMDT_TEST@dblink1
WHERE SMDT_STK_CD = C.SMDT_STK_CD AND SMDT_REC_TYP = C.SMDT_REC_TYP;
COMMIT;
INSERT INTO SMDT_TEST@MDF
(
SMDT_OFR_QTY_06,
SMDT_OFR_PRIC_06,
........
....) values (...);
COMMIT;
END LOOP;
END;
/
Result:
This one tool 2.07 secs elapsed time:
and trace fromntkprof:
OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 19 0.09 0.25 0 2 1 0
Execute 572 0.70 3.32 0 14 596 555
Fetch 399 0.01 0.07 0 44 0 387
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 990 0.81 3.65 0 60 597 942

Approach2:
============
DECLARE
BEGIN
FOR C IN (SELECT * FROM SMDT_TEST ORDER BY SMDT_STK_CD, SMDT_REC_TYP)
LOOP
UPDATE SMDT_TEST@MDF SET
SMDT_OFR_QTY_06 = C.SMDT_OFR_QTY_06,
SMDT_OFR_PRIC_06 = C.SMDT_OFR_PRIC_06,
SMDT_OFR_MQTY_06 = C.SMDT_OFR_MQTY_06,
SMDT_OFR_EINST_06 = C.SMDT_OFR_EINST_06,
..............
..........
commit;
end loop;
end;
/
This one took 1.06 secs and tkprof:
OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 2 0.09 0.21 0 0 0 0
Execute 274 0.50 1.61 0 0 272 273
Fetch 274 0.01 0.06 0 4 0 273
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 550 0.60 1.89 0 4 272 546

Approach3:
=========
begin
delete from SMDT_TEST@MDF;
insert /*+ append */ into SMDT_TEST@MDF nologging select * from smdt_test;
commit;
end;

This is even fater took takes 0.01 sec and tkprof:
OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 4 0.06 0.27 0 0 0 0
Execute 3 0.20 1.13 0 6 0 546
Fetch 3 0.03 0.08 0 6 0 273
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 10 0.29 1.49 0 12 0 819

Approach 4:
============
The best to me:
MERGE INTO smdt_test@mdf a
USING smdt_test c ON (a.SMDT_STK_CD = c.SMDT_STK_CD and a.SMDT_REC_TYP =c.SMDT_REC_TYP)
WHEN MATCHED THEN
UPDATE
set SMDT_OFR_QTY_06 = C.SMDT_OFR_QTY_06,
SMDT_OFR_PRIC_06 = C.SMDT_OFR_PRIC_06,
SMDT_OFR_MQTY_06 = C.SMDT_OFR_MQTY_06,
SMDT_OFR_EINST_06 = C.SMDT_OFR_EINST_06,
....
WHEN NOT MATCHED THEN
insert ... values ...
commit;
This one took 0.0 sec secs as in sqlplus i can not get elapsed less than this:
The tkprof:
OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS

call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 3 0.00 0.00 0 0 0 0
Execute 1 0.00 0.00 0 0 0 0
Fetch 2 0.00 0.00 0 5 0 273
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 6 0.00 0.00 0 5 0 273

Funy thing tkprof did not fill any elapse colum too for merge, is merge parsing included in tkprof???

Anyway questions:
1. Which option is best from the point of best performance and also no-contention.

As you know the tab1 at db1 will be heavily updated so I dont want any of the replication process to have snapshop too old errors? Same case at db2, even though tab1 there is not updated from within db2,but it is being updated from db link, so the users on db2 who will be selecting from this tab should not get also snapshop too old.
The tab1 at db2 for your info will be selected by 1000 concurr sessions is the select rate is very very higher.
both dbs are online apps.

Now you could have aksed why am not using streams or anything, well the reason is that my proc whic i wrote to replicate is a controlled one that is i can change the replication to occur 1 sec or 2 sec (not more 2) and if there is any problem at db1, the proc which i wrote can be stopped any time. This proc is not in db1(sorry for sounding liek that in the beginning) but its a power builder procedure which has connections to db1 for this.

Cheers







Tom Kyte
February 23, 2006 - 8:05 am UTC

insert /*+ append */ would be wrong - that table would only grow, it would never reuse space.

The one with two commits is horribly wrong, fatally wrong. Terribly wrong. Erase it and never do that. It should have one commit, outside of the loop at most.


You have proven once again that slow by slow processing is to be avoided, don't do that procedural loop.


The merge just took no measurable time to perform the operation in question, looks like it was "lucky" based on the wide elapse vs cpu time disparities in the other ones.






1) isn't it obvious?





(I teach these methods of measuring not so I can analyze them - but rather so that each of you can).

You have a method that

a) takes no measurable time
b) does very little measurable work
c) satisfies your requirement (as long as you NEVER delete rows, or update
the join key - merge is good)
d) does very little logical IO


if you get snapshot too old on such an itty bitty table, you have bigger fish to fry - it should not even be a concern.


I cannot imagine using power builder to run this, but whatever.


thanks a million

dani, February 23, 2006 - 8:57 am UTC

tom,
Great, i will go ahead with merge idea.

Just to clear the powerbuilder, it is not that data will
flow to powerbuilder and via lan.
Powerbuilder will merely exeute a stored proedure at the db1 every 1 or 2 secs.
Since power builder application is a monitoring one for ops,
we added an option of replication there, easy gui thing for ops.


Merge

A reader, February 28, 2006 - 2:44 pm UTC

I have two tables datcompiled_temp and xmtdc.
I need to select max(setdatetime) and compare its values with corresponding values in xmtdc for meterid, recordertype, datetime_.
if match is found just update record to sysdate otherwise add new record
I am using following statement. It is just adding new or dupllicate records ignoring matched option



merge into datcompiled_temp dc
using (select meter_id, recordertype_id, datetime_, afcorrected, method from xmtdc ) xm
on (dc.meterid=xm.meter_id and dc.datetime_=xm.datetime_ and dc.compiled=xm.afcorrected and dc.method=xm.method
and
dc.setdatetime=(select max(setdatetime) from datcompiled_temp
where meterid=8 and recordertype=1 and datetime_= xm.datetime_
))
when matched then update set dc.confirmdatetime=sysdate
when not matched then insert (meterid, recordertype, datetime_,compiled, method, setdatetime, confirmdatetime)
values(xm.meter_id, xm.recordertype_id, xm.datetime_,xm.afcorrected,xm.method,sysdate, sysdate)

Merge statement and Partition pruning

Anand, March 02, 2006 - 8:15 am UTC

Tom,

I have a question regarding Merge statement and Partition pruning (Oracle 9iR2).

Can the Target table in a Merge statement be partition pruned?
OR
Will the use of partition key on the Target table in the ON clause of Merge statement enable partition pruning?


I will try to explain my query with the following sample tables...

I have a partitioned source table as -
---------------------------------------------------------
create table src_part
(key1 varchar2(10)
,measure1 number
,part_key1 number
)
partition by range (part_key1)
(partition par_10 values less than (11)
,partition par_20 values less than (21)
,partition par_30 values less than (31)
,partition par_maxvalue values less than (MAXVALUE)
);


insert into src_part
select rownum, round(dbms_random.value(1,5)), round(dbms_random.value(1,35))
from dual
connect by level <= 100;

---------------------------------------------------------


I have partitioned target table as -
---------------------------------------------------------
create table tgt_part
(key1 varchar2(10)
,sum_measure1 number
,part_key1 number
)
partition by range (part_key1)
(partition par_10 values less than (11)
,partition par_20 values less than (21)
,partition par_30 values less than (31)
,partition par_maxvalue values less than (MAXVALUE)
);

insert into tgt_part
select rownum, round(dbms_random.value(1,5)), round(dbms_random.value(1,35))
from dual
connect by level <= 10;

---------------------------------------------------------


My merge statement is as follows -
---------------------------------------------------------
Merge into tgt_part tgt
using
(select key1, measure1, part_key1
from src_part
where part_key1 > :v_lower_boundary -- conditions for partition prune
and part_key1 <= :v_upper_boundary -- on source table
) src
ON
( tgt.key1 = src.key1
and tgt.part_key1 > :v_lower_boundary -- does this ensure partition pruning
and tgt.part_key1 <= :v_upper_boundary -- on the target table
)
when matched
then
update
set
tgt.sum_measure1 = src.measure1 + tgt.sum_measure1
--
when not matched
then
insert (key1 ,sum_measure1 ,part_key1 )
values (src.key1 ,src.measure1 ,src.part_key1 );
---------------------------------------------------------

In the above Merge statement, is partition pruning on the Target table taking place.



I executed the above statement for different values of lower and upper boundaries -

----------
variable v_lower_boundary number;
variable v_upper_boundary number;

exec :v_lower_boundary := 5;

PL/SQL procedure successfully completed.

exec :v_lower_boundary := 10;

PL/SQL procedure successfully completed.


explain plan for
merge....


select * from table(dbms_xplan.display);

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost | Pstart| Pstop |
--------------------------------------------------------------------------------------------
| 0 | MERGE STATEMENT | | 1 | 92 | 4 | | |
| 1 | MERGE | TGT_PART | | | | | |
| 2 | VIEW | | | | | | |
|* 3 | FILTER | | | | | | |
| 4 | NESTED LOOPS OUTER | | 1 | 73 | 4 | | |
| 5 | PARTITION RANGE ITERATOR| | | | | KEY | KEY |
|* 6 | TABLE ACCESS FULL | SRC_PART | 1 | 33 | 2 | KEY | KEY |

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------
| 7 | PARTITION RANGE ITERATOR| | | | | KEY | KEY |
|* 8 | TABLE ACCESS FULL | TGT_PART | 1 | 40 | 2 | KEY | KEY |
--------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

3 - filter(TO_NUMBER(:Z)>TO_NUMBER(:Z) AND TO_NUMBER(:Z)<TO_NUMBER(:Z))
6 - filter("SRC_PART"."PART_KEY1">TO_NUMBER(:Z) AND
"SRC_PART"."PART_KEY1"<=TO_NUMBER(:Z))
8 - filter("TGT"."KEY1"(+)="SRC_PART"."KEY1" AND "TGT"."PART_KEY1"(+)<=TO_NUMBER(:Z) AND

PLAN_TABLE_OUTPUT
------------------------------------------------------------------------------------------------
"TGT"."PART_KEY1"(+)>TO_NUMBER(:Z))

Note: cpu costing is off

25 rows selected.


As per the plan it appears that partition pruning is taking place.

My concern is if there is a FTS on the Target table (assuming Partition Pruning is not taking place),
then the performance of the Merge statement will keep on degrading as the volume of data in target
keeps on increasing (we will be getting approx 10 million records in the target table every day).


Related queries :

1) If the ON clause in the above merge statement is changed to following

ON
( tgt.key1 = src.key1
and tgt.part_key1 = src.part_key1
)

The explain plan for this also shows the same plan as above.

Query: How does the condition "tgt.part_key1 = src.part_key1" enable partition pruning (if it is taking place at all)


2) If I am able to write the ON clause of the Merge statement as follows, will it be more beneficial?

ON
( tgt.key1 = src.key1
and tgt.part_key1 = src.part_key1
and tgt.part_key1 > :v_lower_boundary -- <<===
and tgt.part_key1 <= :v_upper_boundary -- <<===
)



Thanks in advance...

Regards,
Anand

Tom Kyte
March 02, 2006 - 12:37 pm UTC

------------------------------------------------------------------------
| Id | Operation | Name |..t | Pstart| Pstop |
----------------------------------------------------..--------------------
| 0 | MERGE STATEMENT | |.. 5 | | |
| 1 | MERGE | TGT_PART |.. | | |
| 2 | VIEW | |.. | | |
|* 3 | FILTER | |.. | | |
|* 4 | HASH JOIN OUTER | |.. 5 | | |
| 5 | PARTITION RANGE ITERATOR| |.. | KEY | KEY |
|* 6 | TABLE ACCESS FULL | SRC_PART |.. 2 | KEY | KEY |
| 7 | PARTITION RANGE ITERATOR| |.. | KEY | KEY |
|* 8 | TABLE ACCESS FULL | TGT_PART |.. 2 | KEY | KEY |
----------------------------------------------------..--------------------

simply by using explain plan on the merge and

select * from table(dbms_xplan.display);


you can get the plan including partition pruning information - you can see that it'll partition prune due to the KEY KEY (it doesn't know at parse time what partition(s) it'll hit but it knows it might not hit them all.

Merge statement and Partition Pruning

Anand, March 03, 2006 - 4:49 am UTC

Tom,

Thanks a lot for the quick response.

However there are still a few things which are not quite clear. (I had already asked the following questions in the earlier review, but maybe the review was a bit too long :))

1) If the ON clause in the above merge statement is changed to following

ON
( tgt.key1 = src.key1
and tgt.part_key1 = src.part_key1 -- <== added
)

still the explain plan shows that partition pruning is taking place.

I always thought that a "filter" condition (e.g condition like "tgt.part_key1 > :b1 and tgt.part_key1 < :b2" enabled partition pruning.

However the condition "tgt.part_key1 = src.part_key1" is a "join" condition. So how does this enable pruning.

2) If I am able to write the ON clause of the Merge statement as follows, will it be more beneficial for the optimizer while generating the plan...

ON
( tgt.key1 = src.key1
and tgt.part_key1 = src.part_key1
and tgt.part_key1 > :v_lower_boundary -- <<===
and tgt.part_key1 <= :v_upper_boundary -- <<===
)

Thanks & Regards,
Anand

Tom Kyte
March 03, 2006 - 8:17 am UTC

when you join by partition key, it knows it can partition prune, but it doesn't know what it'll prune to until runtime (hence the key/key stuff).




Better way of MERGE

Hitesh Bajaj, April 09, 2006 - 5:54 am UTC

Is there a better way in merging the two tables together in sync, where in one table is accessed via db link and the date is compared across two tables to get the newly inserted & updated data gets merged.

Date : 04/04/2006 05:27:25 The total time spent to merge records (1299267 avg. 74.34 MB ) for table : EMPDUPL1 is 80.17 secs

Date : 06/04/2006 06:29:34 The total time spent to merge records (1 avg. 60 bytes ) for table : EMPDUPL1 is 18.96 secs

The profiling statistics in the IInd case where only one record gets merged doesn't look too great. It took nearly 19 secs to merge the tables for just once record.

MERGE INTO empdupl1 n1
USING empdupl@dblink e1
ON ( n1.empno = e1.empno )
WHEN MATCHED THEN UPDATE
SET n1.ename = e1.ename,
n1.job = e1.job,
n1.mgr = e1.mgr,
n1.hiredate = e1.hiredate,
n1.sal = e1.sal,
n1.comm = e1.comm,
n1.deptno = e1.deptno
WHERE e1.hiredate > (Select max(hiredate) from empdupl1)
WHEN NOT MATCHED THEN
INSERT ( empno, ename, job, mgr, hiredate, sal, comm, deptno )
values ( e1.empno,
e1.ename,
e1.job,
e1.mgr,
e1.hiredate,
e1.sal,
e1.comm,
e1.deptno )
WHERE e1.hiredate > (Select max(hiredate) from empdupl1)
/

Is that the whole table across db link is first brought back to the local database and then corresponding WHERE clause is evaluated.

Should this where clause be part of USING statement?

Ideas would be helpful!

Tom Kyte
April 09, 2006 - 8:58 am UTC

you may have modified only one record, but you might have pulled millions from empdupl@dblink.


What if you did this instead:

merge into empdupl n1
using (select * from empdupl@dblink
where hiredatedate > (select max(hiredate) from empdupl1) ) e1
on (...)
when matched <no where clause needed anymore>
when not matched <no where clause needed anymore>;




Can I use merge ProC ?

Yukun, April 10, 2006 - 5:23 am UTC

Hi Tom,

It seems merge method can do a good job, while, can i use merge in ProC?

Regards.

Tom Kyte
April 10, 2006 - 6:01 am UTC

by using dynamic sql so the pro*c pre-processor does not "see it"

ORA-30926

mal, May 31, 2006 - 1:57 pm UTC

function load_post_sales return number
is
curr_sndate date;
prev_sndate date;
tmp number;
row_num number;
start_date date := sysdate;
err_id number := 0;
errm varchar2(300):= null;

begin

if( jctl.trace('Load F_Post_Sales',start_date)!= 0)
then
errm := 'Cannot insert to job_log, initial';
raise stop_job;
end if;

if (jctl.get_sndates('Load F_Post_Sales',curr_sndate,prev_sndate) != 0)
then
tmp := jctl.trace('Load F_Post_Sales',start_date,sysdate,'Get Snapshot date failed');
errm := 'Cannot get Snapshot_Date';
raise stop_job;
end if;


/* for purchase_receipts */
merge into f_post_sales ps
using (select pr.vehicle_id,
pr.purchase_receipt_id,
pr.purchase_date,
pr.transport_preference_id,
tp.name transport_preference_name,
pr.transport_fee,
pr.pickup_location_id,
pr.delivery_location_id,
pr.payment_method_id,
pm.name payment_method_name,
pr.delivery_distance
from purchase_receipts pr,
transport_preferences tp,
payment_methods pm
where pr.transport_preference_id = tp.transport_preference_id(+)
and pr.payment_method_id = pm.payment_method_id(+)) aa
on (ps.vehicle_id = aa.vehicle_id)
when matched then update set
ps.pr_purchase_receipt_id = aa.purchase_receipt_id,
ps.pr_purchase_date = aa.purchase_date,
ps.pr_transport_preference_id = aa.transport_preference_id,
ps.pr_transport_preference_name = aa.transport_preference_name,
ps.pr_transport_fee = aa.transport_fee,
ps.pr_pickup_location_id = aa.pickup_location_id,
ps.pr_delivery_location_id = aa.delivery_location_id,
ps.pr_payment_method_id = aa.payment_method_id,
ps.pr_payment_method_name = aa.payment_method_name,
ps.pr_delivery_distance = aa.delivery_distance
when not matched then insert
(
ps.vehicle_id,
ps.pr_purchase_receipt_id,
ps.pr_purchase_date,
ps.pr_transport_preference_id,
ps.pr_transport_preference_name,
ps.pr_transport_fee,
ps.pr_pickup_location_id ,
ps.pr_delivery_location_id,
ps.pr_payment_method_id ,
ps.pr_payment_method_name,
ps.pr_delivery_distance
)
values
(
aa.vehicle_id,
aa.purchase_receipt_id,
aa.purchase_date,
aa.transport_preference_id,
aa.transport_preference_name,
aa.transport_fee,
aa.pickup_location_id ,
aa.delivery_location_id,
aa.payment_method_id ,
aa.payment_method_name,
aa.delivery_distance
);

commit;

/* for the additional transport */

merge into f_post_sales ps
using (select p.vehicle_id,
p.sale_id,
p.seller_organization_id,
p.buyer_organization_id,
p.transport_fee,
p.transport_preference_id,
tp.name transport_preference_name,
p.void,
p.delivery_distance,
p.delivery_location_id,
p.estimated_delivery_date,
p.login_id,
p.payment_due_date,
p.payment_method_id,
pm.name payment_method_name,
p.pickup_location_id,
p.purchase_date
from ps_transactions p,
payment_methods pm ,
transport_preferences tp
where p.purchase_type_id = 2
and p.transport_preference_id = tp.transport_preference_id(+)
and p.payment_method_id = pm.payment_method_id(+)) aa
on (ps.vehicle_id = aa.vehicle_id)
when matched then update set
ps.pst_sale_id = aa.sale_id,
ps.pst_seller_organization_id = aa.seller_organization_id,
ps.pst_buyer_organization_id = aa.buyer_organization_id,
ps.pst_transport_fee = aa.transport_fee,
ps.pst_transport_preference_id = aa.transport_preference_id,
ps.pst_transport_preference_name = aa.transport_preference_name,
ps.pst_void = aa.void,
ps.pst_delivery_distance = aa.delivery_distance,
ps.pst_delivery_location_id = aa.delivery_location_id,
ps.pst_estimated_delivery_date = aa.estimated_delivery_date,
ps.pst_login_id = aa.login_id,
ps.pst_payment_due_date = aa.payment_due_date,
ps.pst_payment_method_id = aa.payment_method_id,
ps.pst_payment_method_name = aa.payment_method_name,
ps.pst_pickup_location_id = aa.pickup_location_id,
ps.pst_purchase_date = aa.purchase_date
when not matched then insert
(
ps.vehicle_id,
ps.pst_sale_id,
ps.pst_seller_organization_id,
ps.pst_buyer_organization_id,
ps.pst_transport_fee,
ps.pst_transport_preference_id,
ps.pst_transport_preference_name,
ps.pst_void,
ps.pst_delivery_distance,
ps.pst_delivery_location_id,
ps.pst_estimated_delivery_date,
ps.pst_login_id,
ps.pst_payment_due_date,
ps.pst_payment_method_id,
ps.pst_payment_method_name,
ps.pst_pickup_location_id,
ps.pst_purchase_date
)
values
(
aa.vehicle_id,
aa.sale_id,
aa.seller_organization_id,
aa.buyer_organization_id,
aa.transport_fee,
aa.transport_preference_id,
aa.transport_preference_name,
aa.void,
aa.delivery_distance,
aa.delivery_location_id,
aa.estimated_delivery_date,
aa.login_id,
aa.payment_due_date,
aa.payment_method_id,
aa.payment_method_name,
aa.pickup_location_id,
aa.purchase_date
);

commit;

Hi Tom,
I have above statement and getting following error
ORA-30926: unable to get a stable set of rows in the source tables
can you please help me to solve this?


Tom Kyte
May 31, 2006 - 3:41 pm UTC

[tkyte@dellpe ~]$ oerr ora 30926
30926, 00000, "unable to get a stable set of rows in the source tables"
// *Cause:  A stable set of rows could not be got because of large dml
//          activity or a non-deterministic where clause.
// *Action: Remove any non-deterministic where clauses and reissue the dml.
[tkyte@dellpe ~]$


this usually happens when  you try to update the same row twice.

You need a very very very simple and small example to see this.


ops$tkyte@ORA10GR2> create table t1 ( x int, y int );

Table created.

ops$tkyte@ORA10GR2> create table t2 ( x int, y int );

Table created.

ops$tkyte@ORA10GR2>
ops$tkyte@ORA10GR2> insert into t1 values ( 1, null );

1 row created.

ops$tkyte@ORA10GR2> insert into t2 values ( 1, 1 );

1 row created.

ops$tkyte@ORA10GR2> insert into t2 values ( 1, 2 );

1 row created.

ops$tkyte@ORA10GR2>
ops$tkyte@ORA10GR2> merge into t1 using t2 on (t1.x = t2.x)
  2  when matched then update set y = t2.y;
merge into t1 using t2 on (t1.x = t2.x)
                    *
ERROR at line 1:
ORA-30926: unable to get a stable set of rows in the source tables


Should T2.Y be 1 or 2 after we are done?  Oracle is saying "I am not going to make that decision for you!!!"


So, I would assume that in  your really large example (which I did not really read through to be honest), one of your USING sets returns "more than a single record" for your ON join key and we don't know which record to use. 

example wrong

mal, May 31, 2006 - 5:21 pm UTC

SQL>  merge into t1 using t2 on (t1.x = t2.x)
  2    when matched then update set y = t2.y
  3  when not matched then insert(y) values (t2.y);
 merge into t1 using t2 on (t1.x = t2.x)
                     *
ERROR at line 1:
ORA-30926: unable to get a stable set of rows in the source tables

It was incomplete example :( 

Tom Kyte
June 01, 2006 - 9:25 am UTC

how so?

do you see above

a) create table t1
b) create table t2
c) inserts into t1
d) inserts into t2
e) merge showing the problem

now, tell me, how is that "incomplete"

It totally 100% demonstrates my point: The set you are "using" to merge into your table contains duplicates by the join key (t1 has one row, t2 has two rows - both rows have the same join key).

So, tell me please - what is missing or incomplete or wrong here?


Maybe you got confused by the really big block of incomplete test case above my concise, yet 100% complete - but small - but entirely there - test case??



Depends on Version

Greg W, June 01, 2006 - 10:51 am UTC

Mal didn't specify version, but your example works on 10g, but not on 9i.  As you had said in a earlier posting, 9i requires "when not matched then insert" part, but 10g doesn't.  Such as in 9.2.0.7:

 1   merge into t1 using t2 on (t1.x = t2.x)
  2* when matched then update set y = t2.y
SQL> /
when matched then update set y = t2.y
                                    *
ERROR at line 2:
ORA-00905: missing keyword

But:
SQL>  merge into t1 using t2 on (t1.x = t2.x)
  2  when matched then update set y = t2.y
  3  when not matched then insert(y) values (t2.y);
 merge into t1 using t2 on (t1.x = t2.x)
            *
ERROR at line 1:
ORA-30926: unable to get a stable set of rows in the source tables
will get the error.

Just an observation.


 

Tom Kyte
June 01, 2006 - 11:09 am UTC

what is very funny to me - Mal ran the example (see the sqlprompt).

Apparently

a) they have 10g
b) they ran the test case

sort of showing that it was complete and self contained. They did not just cut and paste MY output - they used their own.

"Merge " for just inserting and not updating

Mave, June 07, 2006 - 11:53 am UTC

Tom, in my scenario, I want to see if a record exists in table already and if exists then do nothing, else insert into this table. Can I use MERGE for this?

Sort of ..
Merge table a
using a.id=l_id --variable in procedure
When matched
--Do nothing
When not Matched
Insert..

Is it possible?
Oracle version is 9i

I know I can do a select and in no_data_found exception, I can insert a new record..but just playing around with MERGE and kind of wondering!!

Tom Kyte
June 07, 2006 - 3:20 pm UTC

that is just:

insert into t select * from t2 where (key...) not in (select key.... from t);



You could update column you are merging on.

Saibabu Devabhaktuni, June 16, 2006 - 2:45 am UTC

Let's say you want to merge "SOURCE" table into "TARGET" table with "ID" column as joining condition and you want to update "ID" column too.

merge into TARGET using
(select target.rowid rid, source.* from SOURCE, TARGET
where source.id = target.id(+)
) SOURCE
on (TARGET.rowid = SOURCE.rid)
when matched then
update set target.id = source.id, target.column2 = source.column2, ....
when not matched then
insert values (source.id, source.column2, ....)
/

Thanks,
</code> http://sai-oracle.blogspot.com/ <code>

Tom Kyte
June 16, 2006 - 6:58 pm UTC

you cannot, not with merge

Merge and Deadlocks

Alex, June 20, 2006 - 9:02 am UTC

Tom,

We have a real-time procedure (which is part of a package) that is called for an incoming event from a middleware messaging system. The procedure first merges one row into BTCH_PROG_S table, then it merges one row into BTCH_PROG_FCST_S table in that order only. This procedure is called over 50 times per second.

We are seeing deadlock errors generated by these two merge statements. The deadlock errors have been occurring about once a day. What is strange is that the deadlocks seem to be between the sessions doing MERGE's into two different tables. Do you know what the cause of these deadlocks may be?

Thanks in advance for your help,
Alex


Here is an exerpt from one of the trace files:

*** SESSION ID:(388.1362) 2006-06-15 04:05:05.031
Undo Segment 144 Onlined
*** 2006-06-16 05:11:12.101
DEADLOCK DETECTED
Current SQL statement for this session:
MERGE /*+ index(d) */ INTO BTCH_PROG_FCST_S D USING ...
----- PL/SQL Call Stack -----
object line object
handle number name
52cface38 858 package body DPMMGR.DPM_VRP_BPS_PKG
52cface38 1163 package body DPMMGR.DPM_VRP_BPS_PKG
52cface38 1463 package body DPMMGR.DPM_VRP_BPS_PKG
51e4f52a0 288 procedure DPMMGR.DPM_INS_WHSE_CTNR_EVNT_SP
4ffcd8ea0 1 anonymous block
The following deadlock is not an ORACLE error. It is a
deadlock due to user error in the design of an application
or from issuing incorrect ad-hoc SQL. The following
information may aid in determining the deadlock:
Deadlock graph:
---------Blocker(s)-------- ---------Waiter(s)---------
Resource Name process session holds waits process session holds waits
TX-00010000-00c74605 303 388 X 312 170 S
TX-0006002e-00855c6c 312 170 X 303 388 X
session 388: DID 0001-012F-00000006 session 170: DID 0001-0138-00000006
session 170: DID 0001-0138-00000006 session 388: DID 0001-012F-00000006
Rows waited on:
Session 170: obj - rowid = 00123358 - AAEjNYAAGAAACPPAAA
(dictionary objn - 1192792, file - 6, block - 9167, slot - 0)
Session 388: obj - rowid = 00124180 - AAEkGAAAGAAAFlEAAG
(dictionary objn - 1196416, file - 6, block - 22852, slot - 6)
Information on the OTHER waiting sessions:
Session 170:
pid=312 serial=54 audsid=2770316 user: 23/<none>
O/S info: user: egate, term: unknown, ospid: , machine: eaiux103
program: JDBC Thin Client
application name: JDBC Thin Client, hash value=0
Current SQL Statement:

MERGE /*+ index(d) */ INTO BTCH_PROG_S D USING ...



Merge using aggregates

Thiru, July 05, 2006 - 5:04 pm UTC

Tom,

Will merge be a good thing to use (if so, how)when dealing with aggregates in a multi-million records tables.

For instance:

create table source ( a varchar2(2),b number)
/
create table target ( a varchar2(2),b number,status varchar2(10))
/
insert into source values('ab',100);
insert into source values('bc',100);
insert into source values('cd',100);
insert into source values('ef',100);
insert into source values('fg',100);
insert into source values('gh',100);
insert into source values('hi',100);
insert into source values('ij',100);
insert into source values('jk',100);
insert into source values('kl',100);
insert into source values('mn',100);

insert into target (a,b) values('ab',50);
insert into target (a,b)values('ab',50);
insert into target (a,b) values('bc',50);
insert into target (a,b)values('bc',50);
insert into target (a,b)values('cd',50);
insert into target (a,b)values('cd',50);
insert into target (a,b)values('de',50);
insert into target (a,b)values('de',50);
insert into target (a,b)values('de',50);
insert into target (a,b) values('ef',50);
insert into target (a,b)values('ef',50);
insert into target (a,b)values('gh',50);
insert into target (a,b)values('hi',50);
insert into target (a,b)values('hi',50);
insert into target (a,b)values('ij',50);
insert into target (a,b)values('ij',50);
insert into target (a,b)values('jk',50);
insert into target (a,b)values('jk',50);
insert into target (a,b)values('jk',50);
commit;

I would want to update the target STATUS column to 'MCHED' if the sum(target.b) = source.b and target.a=source.a.
In all other cases, update the STATUS column to 'UNMCHED'. The number of records in both the tables are over 2mill.

Also, can this be done in a single update statment?

Tom Kyte
July 08, 2006 - 8:46 am UTC

10's, millions, billions - not really relevant.

You need to update EVERY SINGLE ROW here.
You need to aggregate TARGET up to the level of source (assume source.a is a primary key..)
And then join back.


ops$tkyte@ORA10GR2> select t.a, t.b, s.b,
  2         decode(t.b, s.b, 'MCHED', 'UNMCHED' ) tag
  3    from (select a, sum(b) b from target group by a) t, source s
  4   where t.a = s.a(+);

A           B          B TAG
-- ---------- ---------- -------
ab        100        100 MCHED
bc        100        100 MCHED
cd        100        100 MCHED
ef        100        100 MCHED
gh         50        100 UNMCHED
hi        100        100 MCHED
ij        100        100 MCHED
jk        150        100 UNMCHED
de        150            UNMCHED

9 rows selected.

ops$tkyte@ORA10GR2>
ops$tkyte@ORA10GR2> merge into target T
  2  using
  3  (
  4  select t.a, decode(t.b, s.b, 'MCHED', 'UNMCHED' ) tag
  5    from (select a, sum(b) b from target group by a) t, source s
  6   where t.a = s.a(+)
  7  ) S
  8  on (t.a = s.a)
  9  when matched then update set t.status = s.tag;

19 rows merged.



If the tables are really this skinny, might be faster as a create table as select, rather then an update.

ops$tkyte@ORA10GR2> create table new_target
  2  as
  3  select t.a, t.b,
  4         decode( sum(t.b) over (partition by t.a), s.b, 'MCHED', 'UNMCHED' ) status
  5    from target t, source s
  6   where t.a = s.a(+)
  7  /

Table created.
 

Follow up on Merge: Using Case for more flexibility

Thiru, July 08, 2006 - 12:12 pm UTC

Tom,

Please see this statement as an extension to what you gave:

merge into target T
using
(
select
T.A a,
s.a a1,
S.B b1,
(CASE
WHEN T.B = S.B
THEN 'MCHED'
WHEN (T.B > S.B OR T.B < S.B)
THEN 'NEWREC'
ELSE 'ERROR'
END ) tag
from (select a, sum(b) b from target group by a) t
FULL OUTER JOIN source s
on (t.a=s.a )
) SR
on (t.a = sR.a)
when matched then update set t.status = sR.tag
when not matched then
insert
(T.A,T.B,T.STATUS)
values
(sr.a1,sR.b1,'MCHED');

ab 50 MCHED
ab 50 MCHED
bc 50 MCHED
bc 50 MCHED
cd 50 MCHED
cd 50 MCHED
de 50 ERROR
de 50 ERROR
de 50 ERROR
ef 50 MCHED
ef 50 MCHED
gh 50 NEWREC
hi 50 MCHED
hi 50 MCHED
ij 50 MCHED
ij 50 MCHED
jk 50 NEWREC
jk 50 NEWREC
jk 50 NEWREC
kl 100 MCHED
mn 100 MCHED
fg 100 MCHED


This looks so far what is required. There is one additional step that I am trying to achieve.

Insert into target, records that are marked as NEWREC in the above statment by taking the corresponding values from source.
So in the example above, the target table should get
two records:

gh 100 MCHED ( the 100 is from souce for 'gh'
jk 100 MCHED ( the 100 is from souce for 'jk'

Is this possible as part of the above merge statement? Or should it be a separte insert like the one below.
insert into target t
select a,b,'MCHED'
from source s
where s.a in (select distinct a from target where status='NEWREC')

thanks Tom for the time.

Tom Kyte
July 08, 2006 - 9:06 pm UTC

well, this is "entirely different" from the original problem statement.

Not even a tiny bit different, radically different.

It would be "tricky" to do that - since gh DOES exist in target - so the "when not matched" would "not be true"

skip some rows update when the old value is expected

Charlie Zhu, July 10, 2006 - 5:42 pm UTC

For example, I have a books table.
table Books
(process_flag varchar2(1),
delete_flag varchar2(1));

1) only if :old.delete_flag='Y', then I update column process_flag and delete_flag both
UPDATE books
SET process_flag='Y', delete_flag='N'
WHERE process_flag = 'N' and delete_flag='Y';

2) if :old.delete_flag='N' already, I will not update this column.
UPDATE books
SET process_flag='Y'
WHERE process_flag = 'N' and delete_flag='N';

Can I do it in single UPDATE?
to save some redo and logical reads.

Thanks in advance.

Tom Kyte
July 11, 2006 - 7:39 pm UTC

you will either update 2 columns or 1 column - if you want to do in a single sql statement, we'll have to update both columns - there is no way to say "if this, then update these, else update those"

Merge on same table

sara, July 10, 2006 - 10:36 pm UTC

In the following procedure I am using same table in
the USING clause as the INSERT/UPDATE. But the insert is not working and the update is working.

create or replace procedure merge_proc(l_empno IN number,l_ename IN varchar2,l_sal in number,
l_deptno in number)
as
BEGIN
MERGE INTO emp et
USING ( SELECT * from emp1 where empno = l_empno
and ename = l_ename ) es
ON (et.empno = es.empno and et.ename = es.ename)
WHEN MATCHED THEN
UPDATE
SET et.sal = l_sal
, et.deptno = l_deptno
WHEN NOT MATCHED THEN
INSERT
( et.empno, et.ename, et.sal, et.deptno )
VALUES
( l_empno, l_ename, l_sal, l_deptno );

end;
/

Thanks for your help.

Tom Kyte
July 11, 2006 - 7:48 pm UTC

please define "not working" and provide a full example to work with (something we can all cut and paste and run ourselves)

MERGE

sara, July 12, 2006 - 10:09 am UTC

Example, in the following example, the record is
not there in the EMP table and though it is not
inserting this record.

BEGIN
MERGE INTO emp et
USING ( select * from emp
where empno = 1107
and ename = 'NEWNAME' ) es
ON (et.empno = es.empno and et.ename = es.ename)
WHEN MATCHED THEN
UPDATE
SET et.sal = 90000
, et.deptno = 10
WHEN NOT MATCHED THEN
INSERT
( empno, ename, sal, deptno )
VALUES
( 1107, 'NEWNAME', 90000, 10);
END;
/

Thanks


Tom Kyte
July 12, 2006 - 4:59 pm UTC

<b>and provide a full example to work with (something 
we can all cut and paste and run ourselves)  </b>!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


but in looking at the query, it cannot ever "NOT MATCH".  Think about it.

You have a table EMP you are merging INTO
You have a table EMP you are merging FROM

they are the same table.

If the record doesn't exist in emp to be merged INTO, how could the record exist in the table emp you are merging FROM???????????

If empno = 1107, 'NEWNAME' doesn't exist in EMP, the source "using" is EMPTY.  It won't "make stuff up".

I'm going to make the perfectly reasonal assumption that the primary key is EMPNO, not empno,ename - therefore:

ops$tkyte@ORA9IR2> create table emp ( empno number, ename varchar2(10), job varchar2(10), sal number, deptno number );

Table created.

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> variable p_empno number
ops$tkyte@ORA9IR2> variable p_sal number
ops$tkyte@ORA9IR2> variable p_deptno number
ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> select * from emp;

no rows selected

ops$tkyte@ORA9IR2> exec :p_empno := 1107; :p_sal := 123; :p_deptno := 10;

PL/SQL procedure successfully completed.

ops$tkyte@ORA9IR2> merge into emp et
  2  using ( select :p_empno empno from dual ) es
  3  on (et.empno = es.empno)
  4  when matched then update set sal = :p_sal, deptno = :p_deptno
  5  when not matched then insert(empno,ename,sal,deptno) values(:p_empno,'NEWNAME',:p_sal,:p_deptno);

1 row merged.

ops$tkyte@ORA9IR2> select * from emp;

     EMPNO ENAME      JOB               SAL     DEPTNO
---------- ---------- ---------- ---------- ----------
      1107 NEWNAME                      123         10

ops$tkyte@ORA9IR2>
ops$tkyte@ORA9IR2> exec :p_empno := 1107; :p_sal := 1234; :p_deptno := 100;

PL/SQL procedure successfully completed.

ops$tkyte@ORA9IR2> merge into emp et
  2  using ( select :p_empno empno from dual ) es
  3  on (et.empno = es.empno)
  4  when matched then update set sal = :p_sal, deptno = :p_deptno
  5  when not matched then insert(empno,ename,sal,deptno) values(:p_empno,'NEWNAME',:p_sal,:p_deptno);

1 row merged.

ops$tkyte@ORA9IR2> select * from emp;

     EMPNO ENAME      JOB               SAL     DEPTNO
---------- ---------- ---------- ---------- ----------
      1107 NEWNAME                     1234        100



That'll either

a) update the existing record with your sal, deptno
b) create a NEW record with 'NEWNAME' as the name and your sal, deptno 

awesome

A reader, July 12, 2006 - 5:58 pm UTC

An awesome explanation and an unbelivable solution.

Thanks a lot.

PL/SQL table in merge statement

Bala, July 16, 2006 - 7:17 am UTC

Hi tom,
Thanks for your valuable help for the oracle community.

Is it possible to use the PL/SQL tables in MERGE statement.
My requirement is I have records in the PL/SQL table.
If the record in PL/SQL table is present in the table, I have to update otherwise Insert.

I have to do this process for 1 million records. Which is the best way to do this? Delete and Insert or Update and Insert.

Please advise me.



Tom Kyte
July 16, 2006 - 9:49 am UTC

do you really have a plsql table with 1,000,000 records in it?

if you use a collection, sure - pretty easy.


ops$tkyte%ORA10GR2> create or replace type myScalarType
  2  as object
  3  ( x int,
  4    y date
  5  )
  6  /

Type created.

ops$tkyte%ORA10GR2> create or replace type myTableType
  2  as table of myScalarType
  3  /

Type created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> create table t ( x int primary key, y date );

Table created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> insert into t values ( 1, sysdate );

1 row created.

ops$tkyte%ORA10GR2> commit;

Commit complete.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> select * from t;

         X Y
---------- ---------
         1 16-JUL-06

ops$tkyte%ORA10GR2> declare
  2      l_data myTableType;
  3  begin
  4
  5      select myScalarType( rownum, sysdate-100+rownum )
  6        bulk collect into l_data
  7        from all_users
  8       where rownum <= 3;
  9
 10      merge into T
 11      using (select data.x, data.y
 12               from TABLE(cast( l_data as myTableType ) ) data ) data
 13      on (t.x = data.x)
 14      when matched then update set y = data.y
 15      when not matched then insert (x,y) values(data.x,data.y);
 16  end;
 17  /

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> select * from t;

         X Y
---------- ---------
         1 08-APR-06
         2 09-APR-06
         3 10-APR-06
 

Problem in using type in MERGE statement

Bala, July 17, 2006 - 3:47 am UTC

Hi Tom,
Thanks for your solution of using TYPE in MERGE.
I have tried by using TYPE in merge statement. But I am getting strange error.

Following is my merge statement. If I add a condition "a.dept_dt=b.dept_dt" in the ON condition ,I am getting "ORA-00904: : invalid identifier".
But I am using the same column name in insert statement.
I tried the same column name in update statement also, it is working fine. It is giving error only in ON clause.


MERGE INTO fe_currentdcp a
USING(SELECT * FROM TABLE(tabCurrentDCP)) b
ON (a.online_orgn=b.online_orgn AND
a.online_dstn=b.online_dstn AND
a.trip_orgn = b.trip_orgn AND
a.trip_dstn = b.trip_dstn AND
a.pos = b.pos AND
a.pax_type = b.pax_type AND
a.comp = b.comp AND
a.bkg_class = b.bkg_class AND
a.dept_dt =b.dept_dt)

WHEN MATCHED THEN UPDATE SET
A.current_bkg_qty=B.current_bkg_qty

WHEN NOT MATCHED THEN
INSERT
(run_id, online_orgn,
online_dstn, trip_orgn,
trip_dstn, pos,
dept_dt, comp,
bkg_class, pax_type,
sig_flag, flt_no_a,
flt_no_b, flt_no_c,
dcp, current_bkg_qty)
VALUES
(b.run_id, b.online_orgn,
b.online_dstn, b.trip_orgn,
b.trip_dstn, b.pos,
b.dept_dt, b.comp,
b.bkg_class, b.pax_type,
b.sig_flag, b.flt_no_a,
b.flt_no_b, b.flt_no_c,
b.dcp, b.current_bkg_qty);


Please let me know how to handle this.

Tom Kyte
July 17, 2006 - 1:26 pm UTC

without an ENTIRE EXAMPLE (eg: I haven't a CLUE how you defined *your* types)

No, I cannot.

SQL%ROWCOUNT

A reader, July 28, 2006 - 5:28 pm UTC

After a MERGE statement, the SQL%ROWCOUNT variable contains the number of rows returned by the USING part of the statement.

If it returns 100 rows and 30 were used by the WHEN MATCHED part and 70 were used by the WHEN NOT MATCHED part, is there a way to get at those 30 and 70 numbers?

Thanks

Tom Kyte
July 28, 2006 - 9:07 pm UTC

no, there is not.  

sql%rowcount isn't the count of the using part. depends on whether a row was updated twice and whether you do the entire when matched/when not matched part.

sql%rowcount is the number of rows modified and/or created.


ops$tkyte%ORA10GR2> create table t1 ( x int, y int );

Table created.

ops$tkyte%ORA10GR2> create table t2 ( x int, y int );

Table created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> insert into t1 values ( 1, null );

1 row created.

ops$tkyte%ORA10GR2> insert into t2 values ( 1, 1 );

1 row created.

ops$tkyte%ORA10GR2> insert into t2 values ( 2, 1 );

1 row created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> begin
  2          merge into t1
  3          using t2
  4          on (t1.x = t2.x)
  5          when matched then update set y = t2.y;
  6
  7          dbms_output.put_line( sql%rowcount );
  8  end;
  9  /
1

PL/SQL procedure successfully completed.

 

Merge with RETURNING Clause

Duke Ganote, August 08, 2006 - 12:01 pm UTC

Pending any future enhancement allowing a RETURNING clause for MERGE, do we have to overcome ORA-14551 errors by using autonomous functions like below? (Admittedly, this is just row-by-row) Or is there another work-around?

CREATE TABLE t
( surrogate_key NUMBER
, first_name VARCHAR2(20)
, last_name VARCHAR2(20)
, data_payload VARCHAR2(20)
, CONSTRAINT t_pk PRIMARY KEY
( surrogate_key )
, CONSTRAINT t_ak UNIQUE
(first_name, last_name)
)
/
CREATE SEQUENCE seq_surrogate_key
/
CREATE OR REPLACE FUNCTION t_merge
( p_first_name VARCHAR2
, p_last_name VARCHAR2
, p_payload VARCHAR2
) RETURN NUMBER IS
PRAGMA autonomous_transaction;
l_surrogate_key NUMBER;
BEGIN

UPDATE t
SET data_payload = p_payload
WHERE first_name = p_first_name
AND last_name = p_last_name
RETURNING surrogate_key INTO l_surrogate_key;

IF SQL % rowcount = 0 THEN
INSERT INTO t
(surrogate_key
, first_name, last_name, data_payload)
VALUES
(seq_surrogate_key.nextval
, p_first_name, p_last_name, p_payload)
RETURNING surrogate_key INTO l_surrogate_key;
END IF;

COMMIT;
RETURN l_surrogate_key;
END;
/



Tom Kyte
August 09, 2006 - 9:53 am UTC

I have no idea why you would use an autonomous transaction here.

I don't even see how surrogate_key is getting POPULATED here.

RETURNING clause -> SR 5675144.993 & autonomous function

Duke Ganote, August 09, 2006 - 10:44 am UTC

Following up on your response here
</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:5318183934935#38270316877648 <code>
I've created SR 5675144.993 because it wasn't clear if someone else had already done this.

I'm using an autonomous function, otherwise I get an ORA-14551.


Tom Kyte
August 09, 2006 - 11:16 am UTC

your example provided here is utterly incomplete.

If you provide a full one, I'll take a look.

I've a scary feeling though that you are totally killing transactional consistency with whatever you are doing.

autonomous vs ORA-14551

Duke Ganote, August 09, 2006 - 11:07 am UTC

Following up on
</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:5318183934935#69669478067871 <code>
Here's some data in t:

select * from t;

SURROGATE_KEY FIRST_NAME LAST_NAME DATA_PAYLOAD
------------- -------------------- -------------------- --------------------
41 Duke Ganote stuff
42 Duchess Ganote apple pie

and the result of the autonomous function:

select t_merge('Duke','Ganote','new stuff') from dual;

T_MERGE('DUKE','GANOTE','NEWSTUFF')
-----------------------------------
41

If I make the SELECT function part of the same transaction by commenting out the PRAGMA, like this:

CREATE OR REPLACE FUNCTION t_merge
( p_first_name VARCHAR2
, p_last_name VARCHAR2
, p_payload VARCHAR2
) RETURN NUMBER IS
-- PRAGMA autonomous_transaction;
l_surrogate_key NUMBER;
BEGIN
UPDATE t
SET data_payload = p_payload
WHERE first_name = p_first_name
AND last_name = p_last_name
RETURNING surrogate_key INTO l_surrogate_key;
IF SQL % rowcount = 0 THEN
INSERT INTO t
(surrogate_key
, first_name, last_name, data_payload)
VALUES
(seq_surrogate_key.nextval
, p_first_name, p_last_name, p_payload)
RETURNING surrogate_key INTO l_surrogate_key;
END IF;
COMMIT;
RETURN l_surrogate_key;
END;
/

Then I get ORA-14551:

select t_merge('Duke','Ganote','happy') from dual;
select t_merge('Duke','Ganote','happy') from dual
*
ERROR at line 1:
ORA-14551: cannot perform a DML operation inside a query
ORA-06512: at "DGANOTE.T_MERGE", line 9






Tom Kyte
August 09, 2006 - 11:16 am UTC

why would you SELECT THAT????


just execute the function - don't select functions from dual to run them.




Why SELECT THAT... just a demo...

Duke Ganote, August 09, 2006 - 12:28 pm UTC

> why would you SELECT THAT????

In practice I wouldn't.  The function T_MERGE would be called inside one of an existing row-by-row ETL procedure, so that I could capture the changed surrogate_key... more like this:

  1  DECLARE
  2    changed_surrogate_key NUMBER;
  3  BEGIN
  4    changed_surrogate_key := t_merge('Duke','Ganote','zippy');
  5    DBMS_OUTPUT.PUT_LINE( changed_surrogate_key );
  6* END;
SQL> /
41

PL/SQL procedure successfully completed.
 

Tom Kyte
August 09, 2006 - 4:07 pm UTC

show me - give me the use case, I cannot suggest a best practice method without one.

I don't know why you would call the function from SQL in an "existing row by row etl procedure"

ah...now I get it...

Duke Ganote, August 09, 2006 - 12:35 pm UTC

Perhaps you're implying that the ORA-14551 is a artifact of how I chose to TEST the function.  In other words, I wouldn't get the error if I called the de-autonomized function inside a block... let's try:

SQL> CREATE OR REPLACE FUNCTION t_merge
  2  ( p_first_name VARCHAR2
  3  , p_last_name VARCHAR2
  4  , p_payload VARCHAR2
  5  ) RETURN NUMBER IS
  6  --  PRAGMA autonomous_transaction;
  7    l_surrogate_key NUMBER;
  8  BEGIN
  9    UPDATE t
 10       SET data_payload = p_payload
 11     WHERE first_name = p_first_name
 12       AND last_name = p_last_name
 13    RETURNING surrogate_key INTO l_surrogate_key;
 14    IF SQL % rowcount = 0 THEN
 15      INSERT INTO t
 16      (surrogate_key
 17      ,   first_name,   last_name,   data_payload)
 18      VALUES
 19      (seq_surrogate_key.nextval
 20      ,   p_first_name,   p_last_name,   p_payload)
 21      RETURNING surrogate_key INTO l_surrogate_key;
 22    END IF;
 23    COMMIT;
 24    RETURN l_surrogate_key;
 25  END;
 26  /

Function created.

Elapsed: 00:00:00.94
SQL> DECLARE
  2    changed_surrogate_key NUMBER;
  3  BEGIN
  4    changed_surrogate_key := t_merge('Duke','Ganote','zippy');
  5    DBMS_OUTPUT.PUT_LINE( changed_surrogate_key );
  6  END;
  7  /
41

PL/SQL procedure successfully completed.

Thanks!
 

Tom Kyte
August 09, 2006 - 4:08 pm UTC

I wasn't implying it :)

I was stating it as fact...

Can it be possible thru MV ?

parag j patankar, August 11, 2006 - 5:32 am UTC

Hi,

I am having two tables e ( transaction table ), b ( balance table i.e. summary table)

create table b ( x char(1), y number);
create table e ( a char(1), b number);

insert into e
select substr(object_name, -1), object_id
from user_objects
where rownum < 50
/

table b is summary of records.

Now suppose I am having following data in table e

S 150488
T 181174
W 74574
X 36656
Y 76763

Now table b should have a summariise entry like

S 150488
T 181174
W 74574
X 36656
Y 76763

table e is getting change every day. So if next day e table having data
S 50
S 50
T 100
W 100
A 500

then table b should have following updated balance

S 150488
T 181174
W 74574
X 36656
Y 76763

and new entry of 'A'
A 500


In short basically, if table b not having details of table e it should insert data otherwise it should update data.

I know it can be done with "MREGE" command, But I think it can be also possible thru MV.

Can you show me how to write MV in this case ?

thanks & regards
pjp


Tom Kyte
August 11, 2006 - 10:52 am UTC

materialized views are all about data syncronization - making two things "the same"

this is not "the same", hence materialized views are entirely not appropriate.

your example is somewhat confusing, since you only see to be adding new values - not adding anything to existing.

If this was ADDITIVE, you would be executing something like:

merge into b
using (select a, sum(b) from e group by a) e
on (b.a = e.a)
when matched then update set b = b+e.b
when not matched then insert (a,b) values (e.a, e.b);

multi table merge

abz, August 18, 2006 - 6:54 am UTC

Like multi-table INSERT, is there anything like
multi-table MERGE in 9i or 10g or any plans in upcoming
versions.

There can be two types of multi-table MERGE.

1- UPDATE one table and INSERT INTO multiple tables in a single MERGE statement.

2- UPDATE multiple tables and INSERT INTO multiple tables
in a single MERGE statment.



Tom Kyte
August 18, 2006 - 8:28 am UTC

tell me how a multi-table merge would work.

I've never understood the request for this.

does not seem to make sense to me. give me a real world business case where this makes sense.

insert only merge

abz, August 18, 2006 - 7:20 am UTC

I want to insert records in a table from a subquery,
but only those records which are new and not already
present in the target table. Can I take
the advantage of WHEN MATCHED of MERGE?.

I mean can we ignore the WHEN MATCHED part and just
use the WHEN NOT MATCHED part?

or
Is there a possibility in the INSERT INTO statement
to use WHEN MATCHED or WHEN NOT MATCHED?


Tom Kyte
August 18, 2006 - 8:29 am UTC

if you just want to insert.....

why wouldn't you use INSERT??

insert into t
select * from stage MINUS select * from t;

if the "entire record" is to be compared or:

insert into t
select * from stage where (key,columns) NOT IN (select key,column from t);


but yes, you can use merge in 10g with just a "when not matched", but it becomes identical to the NOT IN query above really.

INSERT ONLY

abz, August 18, 2006 - 9:58 am UTC

But NOT IN is not good in every case, although with CBO, NOT IN can prove to be faster than other options but its
not fast in every case, consider the tables having
10 million records.


Tom Kyte
August 18, 2006 - 10:32 am UTC

not in is good with the CBO, as long as the columns in the subquery are not nullable (and if they include nulls, just skip them)

where (key,columns) not in (select key,columns from t where key is not null and columns is not null)

what about NOT IN wihtout key

abz, August 18, 2006 - 10:48 am UTC

if there is only COLUMNS, not KEY columns in the WHERE clause and the SELECT LIST of subquery, would NOT IN
be still faster?
e.g.
INSERT INTO t SELECT t2.c1 FROM t2 WHERE
(t2.non_key_columns) NOT IN (SELECT t3.non_key_coumns FROM t3)

Tom Kyte
August 18, 2006 - 12:31 pm UTC

(key,columns)
is a surrogate for
(the,things,you,decide,determine,uniqueness,for,you)


they are your KEY columns for all intents and purposes here, they are the column you would use in the "on" clause of the merge. They are your key columns used to determine "i am unique"



business case for multi table merge

abz, August 18, 2006 - 1:50 pm UTC

I wrote a multitable INSERT, it works fine, and because
oracle doesnt include features in its product until
they have a valid bussiness case, therefore
this is a valid business case.
Then I thought that only those records which are new
(new with repect to each target table in the multi table insert) should be inserted only, and the rest of records,
should be UPDATE in each target table.

Isnt it a real business case?




Tom Kyte
August 18, 2006 - 4:27 pm UTC

I don't see a "real world use" listed here yet, no.

do you have "an example". "A story".


A multi-table merge, I cannot fathom would it would "look like" or where you would use it. Show me.

Business Case for multi-table MERGE

Duke Ganote, August 24, 2006 - 10:52 am UTC

I'd consider using (were it available) a multi-table MERGE to load fact data while insuring dimensional integrity. Maybe something like this:

MERGE ALL
USING(SELECT dim1_cd, dim2_cd, current_qty
FROM external_table) b
INTO fact_table a
ON (some...conditions)
WHEN NOT MATCHED THEN
INSERT
(dim1_cd, dim2_cd,
current_qty)
VALUES
(b.dim1_cd, b.dim2_cd,
b.current_bkg_qty)
INTO dim1 d1
ON (b.dim1_cd = d1.dim1_cd)
WHEN NOT MATCHED THEN
INSERT INTO dim1 ( dim1_cd, dim1_description )
VALUES (b.dim1_cd, 'unknown' )
INTO dim2 d2
ON (b.dim1_cd = d2.dim2_cd)
WHEN NOT MATCHED THEN
INSERT INTO dim2 ( dim2_cd, dim2_description )
VALUES (b.dim2_cd, 'unknown' )
;


Tom Kyte
August 27, 2006 - 7:54 pm UTC

I don't get it.

why wouldn't the dimensions already possibly/PROBABLY be there???

Merge problem

Khurram Siddiqui, August 26, 2006 - 3:36 am UTC

hi, 

SQL> SELECT banner FROM v$version
  2  /

BANNER
----------------------------------------------------------------
Oracle9i Enterprise Edition Release 9.2.0.1.0 - Production
PL/SQL Release 9.2.0.1.0 - Production
CORE    9.2.0.1.0       Production
TNS for 32-bit Windows: Version 9.2.0.1.0 - Production
NLSRTL Version 9.2.0.1.0 - Production


Tom i run yours previous example the same but getting the same problem as Ketan raised,Please tell me whats wrong with it?


SQL> DROP TABLE t
  2  /

Table dropped.

SQL> CREATE TABLE t (x  INT,y  INT,z  INT)
  2  /

Table created.

SQL> INSERT INTO t VALUES (1,1,1)
  2  /

1 row created.

SQL> COMMIT;

Commit complete.

SQL> SELECT * FROM t
  2  /

         X          Y          Z
---------- ---------- ----------
         1          1          1

SQL> MERGE INTO t 
  2  USING dual ON (x=55)
  3    WHEN MATCHED THEN 
  4     UPDATE SET z=z+1
  5    WHEN NOT MATCHED THEN
  6     INSERT (x,y,z) VALUES (2,2,2)
  7  /

0 rows merged.

x=55 Doesnt exist it should be inserted but not at my end?

SQL> SELECT * FROM t
  2  /

         X          Y          Z
---------- ---------- ----------
         1          1          1

SQL> INSERT INTO t VALUES (55, 1, 1 )
  2  /

1 row created.

SQL> SELECT * FROM t
  2  /

         X          Y          Z
---------- ---------- ----------
         1          1          1
        55          1          1

SQL> MERGE INTO t 
  2  USING dual ON (x=55)
  3    WHEN MATCHED THEN 
  4     UPDATE SET z=z+1
  5    WHEN NOT MATCHED THEN
  6     INSERT (x,y,z) VALUES (2,2,2)
  7  /

1 row merged.

SQL> SELECT * FROM t
  2  /

         X          Y          Z
---------- ---------- ----------
         1          1          1
        55          1          2

Its k fine its updating.

Now i have second problem

SQL> TRUNCATE TABLE t;

Table truncated.


SQL> ALTER TABLE t DROP COLUMN y;

Table altered.

SQL> ALTER TABLE t DROP COLUMN z;

Table altered.

SQL> INSERT INTO t VALUES (1);

1 row created.


SQL> MERGE INTO t 
  2    USING dual ON (1=1)
  3     WHEN MATCHED THEN 
  4      UPDATE SET x=x+1
  5     WHEN NOT MATCHED THEN
  6      INSERT (x) VALUES (3)
  7  .
SQL> /

1 row merged.

SQL> SELECT * FROM t;

         X
----------
         2


SQL> MERGE INTO t 
  2    USING dual ON (1=2)
  3     WHEN MATCHED THEN 
  4      UPDATE SET x=x+1
  5     WHEN NOT MATCHED THEN
  6      INSERT (x) VALUES (3)
  7  .
SQL> /

0 rows merged.

SQL> SELECT * FROM t;

         X
----------
         2

why its not inserting ?
Is it version problem ?

Thanx tom


Khurram 

Tom Kyte
August 27, 2006 - 9:04 pm UTC

I would presume so, yes

ops$tkyte%ORA9IR2> DROP TABLE t
  2  /

Table dropped.

ops$tkyte%ORA9IR2> CREATE TABLE t (x  INT,y  INT,z  INT)
  2  /

Table created.

ops$tkyte%ORA9IR2> INSERT INTO t VALUES (1,1,1)
  2  /

1 row created.

ops$tkyte%ORA9IR2> COMMIT;

Commit complete.

ops$tkyte%ORA9IR2> SELECT * FROM t
  2  /

         X          Y          Z
---------- ---------- ----------
         1          1          1

ops$tkyte%ORA9IR2> MERGE INTO t
  2  USING dual ON (x=55)
  3    WHEN MATCHED THEN
  4     UPDATE SET z=z+1
  5    WHEN NOT MATCHED THEN
  6     INSERT (x,y,z) VALUES (2,2,2)
  7  /

1 row merged.

ops$tkyte%ORA9IR2> select * from t;

         X          Y          Z
---------- ---------- ----------
         1          1          1
         2          2          2

ops$tkyte%ORA9IR2> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle9i Enterprise Edition Release 9.2.0.7.0 - Production
PL/SQL Release 9.2.0.7.0 - Production
CORE    9.2.0.7.0       Production
TNS for Linux: Version 9.2.0.7.0 - Production
NLSRTL Version 9.2.0.7.0 - Production

 

Merge Problem

Khurram Siddiqui, August 28, 2006 - 1:17 am UTC

Thanx a lot Tom.

Khurram

Business Case for Multi-Table MERGE

Duke Ganote, August 30, 2006 - 9:19 am UTC

"why wouldn't the dimensions already possibly/PROBABLY be there?".

The dimensional records probably exist, but we cannot *absolutely* guarantee the referential integrity (you know how source systems are). So the options I see are:

1. Ensure the referential integrity of the dimensions using several MERGE statements to conditionally insert (if a dimensional record NOT MATCHED) 'dummy' dimension record(s) for each dimension, then INSERT the fact records.

2. Write a single multi-table MERGE that simultaneously does all the above: INSERT into the fact table, and conditionally INSERT (if a dimensional record NOT MATCHED) 'dummy' dimension record(s).


Tom Kyte
August 30, 2006 - 9:27 am UTC

so the inserts would hence FAIL and this statement would never actually be able to execute in real life.

RE: Business Case for Multi-table MERGE

Duke Ganote, August 30, 2006 - 12:31 pm UTC

Never execute? I'd expect this script to execute:

drop table external_tbl;
drop table dim1;
drop table dim2;
drop table fact_tbl;
create table external_tbl ( dim1_cd char(1), dim2_cd char(1), qty number );
insert into external_tbl values ( 'A', 'B', 1 );
insert into external_tbl values ( 'B', 'C', 2 );
insert into external_tbl values ( 'A', 'D', 3 );
create table dim1 ( dim1_cd char(1) primary key, dim1_description varchar2(20) );
insert into dim1 values ( 'A', 'At the Earth''s Core');
insert into dim1 values ( 'B', 'Barsoom');
create table dim2 ( dim2_cd char(1) primary key, dim2_description varchar2(20) );
insert into dim2 values ( 'B', 'Buck Rogers');
insert into dim2 values ( 'C', 'Carter, John');
create table fact_tbl ( dim1_cd char(1) references dim1, dim2_cd char(1) references dim2, qty number )
;
MERGE ALL
USING(SELECT dim1_cd, dim2_cd, current_qty
FROM external_tbl) b
INTO fact_tbl a
ON ( b.dim1_cd = a.dim1_cd
AND b.dim2_cd = a.dim2_cd)
WHEN NOT MATCHED THEN
INSERT
(dim1_cd, dim2_cd, current_qty)
VALUES
(b.dim1_cd, b.dim2_cd, b.qty)
WHEN MATCHED THEN
UPDATE current_qty
SET qty = b.qty
INTO dim1 d1
ON (b.dim1_cd = d1.dim1_cd)
WHEN NOT MATCHED THEN -- insert a dummy record
INSERT INTO dim1 ( dim1_cd, dim1_description )
VALUES (b.dim1_cd, 'unknown' )
INTO dim2 d2
ON (b.dim1_cd = d2.dim2_cd)
WHEN NOT MATCHED THEN -- insert a dummy record
INSERT INTO dim2 ( dim2_cd, dim2_description )
VALUES (b.dim2_cd, 'unknown' )
;

and the results to be:

select * from dim2;

D DIM2_DESCRIPTION
- --------------------
B Buck Rogers
C Carter, John
D unknown -- added during the MERGE to maintain ref integrity

select * from fact_tbl;

D D QTY
- - ----------
A B 1
B C 2
A D 3



Tom Kyte
August 30, 2006 - 5:49 pm UTC

Ok, I'm "sort of seeing it", but I'm not seeing how it would be any superior (IO wise) to multiple merges.


EACH INTO would have to be done sort of like a "union all" - so what is the benefit here?

RE: Business Case for multi-table MERGE

Duke Ganote, September 01, 2006 - 4:48 pm UTC

With multiple MERGEs, we're pulling the data from the source/external table each time. In this case, 3 times: once for the fact load, and once for each dimensional integrity check.

With the multi-table MERGE the source is 'tapped' only once, but it's a bigger load on the dimensional tables to get 'pinged' for every row in the fact table.

Offhand, I'm not sure of the trade off, but conceptually I just see this as an extension of the multi-table INSERT.

Tom Kyte
September 01, 2006 - 5:24 pm UTC

no, it could not be tapped once - you might "reference" it once but look at the physics involved there.

it would have to be merged into each one. one by one.

business case

abz, September 02, 2006 - 4:42 am UTC

But atlease it is more easy to write, more readable,
more manageable as a code.

Using Merge for 50 Mill records

Thiru, September 12, 2006 - 1:50 pm UTC

Tom,

If I may use this thread to get your opinion on the issue we are facing with the merge statement.

Here is the merge statement and other details:

merge into t
using
(select PS.LED,
PS.ACT,
PS.CSP,
(case
when ps.QTY=t1.total or ABS(PS.QTY-T1.TOTAL) <= 1
then 1
when (ps.QTY > t1.total + 1) or (t1.total > ps.QTY + 1)
then 2
end) tag
from ( select t.LED,
t.ACT,
t.CSP,
abs(nvl(sum(case when o_s='B'then e_qty end),0)
-nvl(sum(case when o_s='S'then e_qty end),0)) total
from t
where t.row_key between P_start and p_end
group by LED,ACT,CSP
) t1
JOIN PS
on (
t1.LED=ps.LED and
t1.ACT=ps.ACT and
t1.CSP=ps.CSP
) ) SR
on (t.row_key between P_start and p_end and T.LED=Sr.LED AND t.ACT=sr.ACT AND T.CSP=Sr.CSP)
when matched then
update set STATUS = SR.tag

Can you please comment/suggest based on the above merge statement and the undernoted observations:

No of records in table T : 50mill
No of records in table PS: 10mill
T.row_key is primary key
PS.led,PS.ACT,PS.CSP is the primary key on PS

a. When I tested the merge with a subset of table T (5 mill) and subset of PS (1 mill) and split the
data into 10 chunks ( that means this merge is being called simultaneously 10 times from 10 sessions
(the column row_key on the table T is used for chunking and is the primary key), the merge completes
in about 6 minutes.

b. I extrapolated this timing and estimated 10 times this time for around 60-80 minutes for the whole
50milll records.

c. But when the whole load is being run, this merge takes close to 12 hours.

d. SGA_TARGET=8gb

e. PGA_AGGREGATE_TARGET=500M

f. Data in each chunk does not overlap with the data in any other chunk on the table T.

Does this reveal anything abnormal?
Is it that Merge does not scale to the level I am expecting?
Is an update statement for this query be more efficient?
Any other suggestions please?

The explain plan when the load was running:

SQL Statement:


MERGE INTO T USING (SELECT PS.LED, PS.ACT, PS.CSP, (CASE WHEN PS.QTY=T1.TOTAL OR ABS(PS.QTY-T1.TOTAL) <= :B6 THEN :B7 WHEN (PS.QTY > T1.TOTAL + :B6 ) OR (T1.TOTAL > PS.QTY + :B6 ) THEN :B5 END) TAG FROM ( SELECT T.LED, T.ACT, T.CSP, ABS(NVL(SUM(CASE WHEN O_S=:B4 THEN E_QTY END),0) -NVL(SUM(CASE WHEN O_S=:B3 THEN E_QTY END),0)) TOTAL FROM T T WHERE T.ROW_KEY BETWEEN :B2 AND :B1 GROUP BY LED,ACT,CSP ) T1 JOIN PS PS ON ( T1.LED=PS.LED AND T1.ACT=PS.ACT AND T1.CSP=PS.CSP ) ) SR ON (T.ROW_KEY BETWEEN :B2 AND :B1 AND T.LED=SR.LED AND T.ACT=SR.ACT AND T.CSP=SR.CSP) WHEN MATCHED THEN UPDATE SET STATUS = SR.TAG

Optimizer Mode Used:
ALL_ROWS
Total Cost:
35,234
Execution Steps:


Step # Step Name

14 MERGE STATEMENT
13 MERGE
12 PT. VIEW
11 PT.T TABLE ACCESS [BY INDEX ROWID]
10 NESTED LOOPS
8 NESTED LOOPS
5 PT. VIEW
4 SORT [GROUP BY]
3 FILTER
2 PT.T TABLE ACCESS [BY INDEX ROWID]
1 PT.ROW_KEY_PK INDEX [RANGE SCAN]
7 PT.PS TABLE ACCESS [BY INDEX ROWID]
6 PT.PS_PK INDEX [UNIQUE SCAN]
9 PT.T_KEY_IDX INDEX [RANGE SCAN]



Tom Kyte
September 12, 2006 - 5:37 pm UTC

why aren't you just using pdml? parallel dml.

things that tend to spill into temp are NOT going to scale linearally with respect to time of course, your small example - probably entire ram based.

Thiru, September 12, 2006 - 5:53 pm UTC

That means if I am running the merge from 10 sessions split on the row_key, then each session would spawn mulitple sessions due to pdml.

a.Is that right?

b.In that case, will there be an issue with pga_agg_target that has to be allocated for all those parallel sessions?
c. Will I have to bump up the temp tablespace?

d. Would you advise doing the whole 50mill in one shot instead of splitting? I see "cache buffer chains" in the event of v$session.

Thanks.

Tom Kyte
September 13, 2006 - 7:04 am UTC

a) you would either use pdml or "do it yourself" parallel - which you have done. The problem with your "do it yourself" approach by some key value is that every 'parallel' session you have will likely be wanting to work on the same regions of disk (blocks) that the other sessions do, pdml will break the work up by regions of data, physically - not logically by some attribute value.

b) no more so than your "do it yourself" approach.

c) no more so than your "do it yourself" approach.

d) you have cbc issues likely due to contention for the same stuff.

Thiru, September 13, 2006 - 9:29 am UTC

Thanks so much. One more thing about pdml.

Is it required to alter the table to parallel and then issue the alter session enable parallel dml?

Does it make a difference if I do just the session change instead of table alter?

Thanks again for the time and the quick responses.

Tom Kyte
September 13, 2006 - 2:47 pm UTC

have you read through the warehousing guide? It lists the steps for doing pdml:

</code> http://docs.oracle.com/docs/cd/B19306_01/server.102/b14223/usingpe.htm#CACFJJGG <code>

MERGE assumes every row needs a sequence #?

Duke Ganote, September 21, 2006 - 2:50 pm UTC

SET UP:

drop table test_tbl;
drop table test_src;
drop sequence test_seq#;
create
table test_tbl ( surrogate number primary key
, business_key number unique not null
, payload number );
insert into test_tbl
select level, level, level
from dual connect by level < 4;
select * from test_tbl;
create
table test_src ( business_key number primary key
, payload number );
insert into test_src
select level*2, level*3
from dual connect by level < 3;
select * from test_src;
create sequence test_seq# start with 4;


TEST:


MERGE INTO test_tbl tgt
USING ( select * from test_src ) src
ON ( src.business_key = tgt.business_key )
WHEN MATCHED THEN
UPDATE SET tgt.payload = src.payload
WHEN NOT MATCHED THEN
INSERT ( surrogate, business_key, payload )
VALUES
( TEST_SEQ#.nextval, src.business_key, src.payload )
/

2 rows merged.

/

2 rows merged.

/

2 rows merged.

/

2 rows merged.

/

2 rows merged.

/

2 rows merged.

select test_seq#.currval from dual;

CURRVAL
----------
15

select * from test_tbl;

SURROGATE BUSINESS_KEY PAYLOAD
---------- ------------ ----------
1 1 1
2 2 3
3 3 3
5 4 6

Tom Kyte
September 22, 2006 - 2:41 am UTC

yes, please remember sequences are NOT gap free, sequences will NEVER BE gap free.

and if you burn 1,000,000,000 sequences per second, it would still be so many long years from now before you exhausted the number type.....

MERGE and seq# burn

Duke Ganote, September 22, 2006 - 8:43 am UTC

I'd been wondering if (since MERGE won't tell us) INSERTs and UPDATEs could be distinguished or counted by recording the currval of a sequence before and after a MERGE. The answer is: apparently not! (Gaps? As long as sequence numbers are monotonically increasing, I'm happy!)

Tom Kyte
September 22, 2006 - 3:17 pm UTC

NO, never, not ever, never ever, not ever - never.

Appreciation Note....

Soumendra Acharya, September 27, 2006 - 3:25 am UTC

I have visited this for the first time, and I found, its a forum where you would get every answer to your questions related to Oracle!!




merge with exception...

sara, October 03, 2006 - 4:35 pm UTC

I have a merge statement like
MERGE INTO abc et
USING ( SELECT * FROM abc@prod_item
WHERE date >= l_date) es
ON ( et.colone = es.colone )
WHEN MATCHED THEN
UPDATE
SET et.coltwo = es.coltwo
, et.colthree=es.colthree
when not matched then
insert
(et.colone, et.coltwo,...)
values(es.colone,es.coltwo...)

There are about 100000 rows in source table. This generates a unique constraint violation.
how do i show which row generated the violation
error.

Thanks

Tom Kyte
October 03, 2006 - 4:58 pm UTC

sigh....

version?

merge

sara, October 05, 2006 - 12:03 pm UTC

version is 9i

Thanks



Tom Kyte
October 05, 2006 - 1:22 pm UTC

it is not going to be practical in 9i, in 10gr2 with dml error logging it will be. sorry.

MERGE with exceptions?

Duke Ganote, October 05, 2006 - 2:01 pm UTC

Is Sara just asking for what rows in the source violate MERGE's deterministic requirement? Something like this?

select colone, count(*) from (
SELECT * FROM abc@prod_item
WHERE date >= l_date
) group by colone
having count(*) > 1

where her solution as discussed elsewhere above, e.g.
</code> http://asktom.oracle.com/pls/asktom/f?p=100:11:::::P11_QUESTION_ID:5318183934935#52657699523363 <code>

Tom Kyte
October 05, 2006 - 4:21 pm UTC

no, she is asking for rows that violate a unique constraint.




I think you can narrow it down even more

Jon Waterhouse, October 05, 2006 - 3:47 pm UTC

I think if you get two or more rows in the update query that match on the key value you will get a "cannot get a stable set of rows in source table" error (don't remember the number.

So I think you can restrict your query only to rows that do not match the key values into the target table.

Tom Kyte
October 05, 2006 - 4:31 pm UTC

but the unique constraint does not have to be on the key at all here.  If the key matched - there would be no unique constraint violation.

This is a unique key constraint, not a stable set of rows as far as I can tell...


ops$tkyte%ORA9IR2> create table t1 ( x int primary key, y int constraint yu unique, z int constraint zu unique );

Table created.

ops$tkyte%ORA9IR2> insert into t1 values ( 1, 1, 1 );

1 row created.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> create table t2 ( x int primary key, y int unique, z int unique );

Table created.

ops$tkyte%ORA9IR2> insert into t2 values ( 2, 1, 2 );

1 row created.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> merge into t1
  2  using  t2
  3  on ( t1.x = t2.x)
  4  when matched then update set y = t2.y, z = t2.z
  5  when not matched then insert values ( t2.x, t2.y, t2.z );
merge into t1
*
ERROR at line 1:
ORA-00001: unique constraint (OPS$TKYTE.YU) violated
 

Merging problems, a bit related to sara's problem but not exactly the same

Carlos Escobar, November 07, 2006 - 11:02 am UTC

Hi Tom, I have the following:

merge into catprod a using (select R.COD_CAT,R.COD_CAT_PADRE,R.DESCAT,R.CAT_NIVEL,R.CAT_TIPO,R.ID_CADENA
FROM CAR_CATPROD R
LEFT JOIN CATPROD C
ON R.COD_CAT = C.ID_CATPROD
AND R.ID_CADENA = C.ID_CADENA
WHERE C.ID_CATPADRE IS NULL) b
on ( b.cod_cat = a.id_catprod )
when matched then
update set a.ID_CATPADRE = b.COD_CAT_PADRE,a.DESCAT = b.DESCAT,
a.CAT_NIVEL = b.CAT_NIVEL,a.CAT_TIPO = b.CAT_TIPO
when not matched then
insert (a.ID_CATPROD, a.ID_CADENA, a.ID_CATPADRE, a.DESCAT, a.CAT_NIVEL, a.CAT_TIPO) values
(b.COD_CAT, b.ID_CADENA, b.COD_CAT_PADRE, b.DESCAT, b.CAT_NIVEL, b.CAT_TIPO);

and i'm getting a stable set problem..
Using Ora 10g R2

the tables' structures are as follow:


TABLE car_catprod
Name Null? Type
------------------- -------- -------------
COD_CAT NOT NULL VARCHAR2(10)
COD_CAT_PADRE VARCHAR2(10)
DESCAT VARCHAR2(30)
CAT_NIVEL NUMBER
CAT_TIPO CHAR(1)
IDCARGA NUMBER
ID_CADENA NOT NULL NUMBER


TABLE catprod
Name Null? Type PK/FK
------------- -------- ------------- ----------
ID_CATPROD NOT NULL VARCHAR2(20) PK
ID_CADENA NOT NULL NUMBER PK
ID_CATPADRE VARCHAR2(20) FK (Ref: ID_CATPROD)
ID_CADPADRE NUMBER FK (Ref: ID_CADENA)
DESCAT VARCHAR2(30)
CAT_NIVEL NUMBER
CAT_TIPO CHAR(1)
ESTADOACTIVO CHAR(1) <-(Not used for this case)

For the case, we will call CATPROD "A" and CAR_CATPROD "B", given that, we have some similities between rows:

A.ID_CATPROD = B.COD_CAT
A.ID_CATPADRE = B.COD_CAT_PADRE (Checked against A.ID_CATPROD)
A.ID_CADENA = B.ID_CADENA
A.ID_CADPADRE = B.ID_CADENA (Checked against A.ID_CADENA)
A.DESCAT = B.DESCAT
A.CAT_NIVEL = B.CAT_NIVEL
A.CAT_TIPO = B.CAT_TIPO

The complete structure of all tables is way too large to post so, i'm posting the most important for the case, the foreign keys in the second table do a self-reference to the table, it is a categories table which also contains Sub-Categories, both tables are similar in the logic, with the difference that the first table is used for mass-load of data with SQL*Loader so it doesn't have any constraints, the second one is the "production" table, which has the mentioned constraints.

Now my problem with this is that i can't directly mass-load data from the sql*loader into the production table, because not all the data in the incoming files will be healthy (to the constraints), and i don't know how to disable those so i can insert the data there.

I also thought of doing a couple of procedures and functions to solve the fact of the "unhealthy" data regarding the keys which i think that might have solved it, but it's way too slow, maybe i'm doing something wrong about them, so if i can have your attention about those, please reply me.

Thanks

Carlos

Processing order

Scott, March 02, 2007 - 12:48 am UTC

G'day Tom,

I'm just clarifying the deterministic nature of the merge process. In a review above, you changed the 'order' of rows in a table, by just flipping values of the attributes using decode(rownum,1,2,1)

I've always not relied on physical order of rows for anything, yet you demonstrated Oracle's erroneous processing of duplicates when the physical order is manipulated.

I've drawn up a test case to play with the logical
order by
of the source to see if it has any impact.
create table swmerge (a number, b number, primary key (a));
insert into swmerge values (1, 2);
insert into swmerge values (2, 2);

create table swtest(a number, b number, constraint x foreign key (b) references swtest(a), constraint xy primary key (a));

merge into swtest m using (
select a, b from swmerge order by a desc
)s
on (m.a = s.a)
when matched then update set  m.b = s.b
when not matched then insert (a, b) values (s.a, s.b)
;

Changing the sort order from desc to asc has no impact, both times the merge succeeded, where you'd think if where a=1 was inserted first, the foreign key relationship would fail (which i proved by inserting just that row). So this behaviour complies with your comments on merge operating on the data as a set.

So am I chasing my tail if I'm merging a larger amount of data with this sort of self foreign key relationship, dictating the order with a hierarchical query?

The merge is failing due to key constraint violation, but if I disable the constraint, merge, enable the constraint, the constraint is validated.

Is there a way to modify the merge around this?
Tom Kyte
March 04, 2007 - 12:51 pm UTC

short of using a deferrable constraint - no.

MERGE with a Sequence

Alistair, April 17, 2007 - 6:20 am UTC

Tom,

We are writing a new DSS system and getting some really nice performance with MERGE however it appears we have found a small problem . When you MERGE with a sequence number in the insert part it updates the sequence number even when doing an update so that if we have 6mill updates , it updates the sequence by 6 mill which includes the I/O involved.

Is there any way to turn this feature off


Tom Kyte
April 17, 2007 - 10:14 am UTC

alter sequence S cache 100000;

that'll fix any sort of performance issues with the sequence - and the sequence will not exhaust itself, not for trillions of years.

A Merge Gotcha

Jon, April 17, 2007 - 7:10 pm UTC

We've made extensive use of MERGE statements in some of our systems and have come across the following issue with MERGE against tables with high DML rates.

It looks like the MERGE statement updates all columns in the table, regardless of what's in the update list, even the join column which is usually the PK! So it requires TM locks on all child tables, even if they're not in the update list.

If any of those are un-indexed, then the MERGE statement requests a TM mode 4 on the child tables, blocking subsequent DML as well.

Our approach to indexing FKs has been to index only where necessary for query purposes, or if we are deleting from the parent table. We never update a PK.

A simple test shows what I mean:

-- Session 1
-- create tables
create table t1(x int primary key, y int);
create table t2(x int references t1);

insert into t1 values(1,1);
insert into t1 values(2,2);

commit;

-- initiate an uncommited transaction
insert into t2 values(1);

-- DO NOT COMMIT

>> New Session <<
-- Session 2
-- update a non-referenced column in the parent table
update t1
set y = -y
where x = 1;

-- Ok, that works fine. Rollback
rollback;

-- Now do the same thing, this time using a merge statement
merge into t1
using (select 1 x from dual) d
on (t1.x = d.x)
when matched then update set y = -y;

-- you'll be blocked... yet only updating y, which is not referenced by table t2

With high transaction rates, this gets pretty nasty. We've logged an SR for it, and it's currently with development. It looks like this behaviour is still the same thru to 11.1. As implemented it makes Merge unsuitable for high-volume OLTP use.

We've worked around the problem by indexing all the FKs - only required another 1800 indexes... :)

Regards,
Jon

merge in stored procedure

A reader, April 25, 2007 - 2:12 pm UTC

   
create table my_table ( id number, id2 number, 
  is_true   number,ts timestamp,
  p_code varchar2(10),
  user_code varchar2(10),
  price number(10,4) ,
  size2 number);

     create or replace PROCEDURE insert_proc (
        p_g_cursor    OUT      credit.cursortype,
        p_id           IN    number,
        p_id2        IN       number,
        p_true       IN       number,
        p_ts           IN       timestamp,
        p_p_code      IN       varchar2,
        p_user_code   IN       varchar2,
        p_price        IN      number,
        p_size         IN     number
     )
     AS
        v_existing_id   number;
        v1                    NUMBER;
        v2                    NUMBER;
        v3                    NUMBER;
     BEGIN
        BEGIN
           SELECT id
           INTO   v_existing_id
           FROM   my_table
           WHERE  id = p_id;
           
        EXCEPTION
           WHEN NO_DATA_FOUND
           THEN
              MERGE INTO my_table a
                 USING (SELECT p_id, p_true, p_id2
                        FROM   DUAL) q
                 ON (    a.id = p_id
                     AND a.is_true = p_true)
                 WHEN MATCHED THEN
                    UPDATE
                       SET id = p_id,
                           ts = p_ts,
                           user_code = p_user_code,
                           price = p_price,
                           size2 = p_size
                 WHEN NOT MATCHED THEN
                    INSERT (id, id2,
                            is_true, ts,
                            p_code, user_code,
                            price, size2)
                    VALUES (p_id, p_id2, p_true,
                            p_ts, p_p_code, p_user_code,
                            p_price, p_size);
        END;
        OPEN p_g_cursor FOR
           SELECT *
           FROM   my_table
           WHERE  1 = 0;
     END;
/




Caused by: java.sql.SQLException: ORA-00904: "P_TRUE": invalid identifier
ORA-06512: at "MYSCHEMA.A_UTIL", line 65
ORA-01403: no data found
ORA-06512: at line 1

 at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
 at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)
 at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)
 at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:743)
 at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:215)
 at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:954)
 at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1168)
 at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3316)


any idea ? how can i get this working ?

I am trying to write a procedure, where i get 5-6 in params, if the value exists in my table on a (UNIQUE not PK) key ==> update else insert !!
Tom Kyte
April 25, 2007 - 3:52 pm UTC

someone has a trigger in place - see that error stack, you are getting a java exception - from

ORA-06512: at "MYSCHEMA.A_UTIL", line 65

nothing to do with the plsql code, you have a broken DDL trigger in your system.


if your goal is as you state, I fail to see why you have procedural code at all and why there is not just a merge statement????

merge in stored procedure

A reader, April 25, 2007 - 2:43 pm UTC

please use the following code

create table my_table ( id number primary key, id2 number, is_true number not null,ts timestamp,p_code varchar2(10),user_code varchar2(10),price number(10,4) , size number)
   
    alter table my_table add constraint unq_id2_is_true
    unique (id2,is_true);
   
     create or replace PROCEDURE insert_proc (
        p_g_cursor    OUT      APP.cursortype,
        p_id           IN    number,
        p_id2        IN       number,
        p_true       IN       number,
        p_ts           IN       timestamp,
        p_p_code      IN       varchar2,
        p_user_code   IN       varchar2,
        p_price        IN      number,
        p_size         IN     number
     )
     AS
        v_existing_id   number;
        v1                    NUMBER;
        v2                    NUMBER;
        v3                    NUMBER;
     BEGIN
        BEGIN
           SELECT id
           INTO   v_existing_id
           FROM   my_table
           WHERE  id = p_id;
           
        EXCEPTION
           WHEN NO_DATA_FOUND
           THEN
              MERGE INTO my_table a
                 USING (SELECT p_id, p_true, p_id2
                        FROM   DUAL) q
                 ON (    a.id = p_id2
                     AND a.is_true = p_true)
                 WHEN MATCHED THEN
                    UPDATE
                       SET id = p_id,
                           ts = p_ts,
                           user_code = p_user_code,
                           price = p_price,
                           size2 = p_size
                 WHEN NOT MATCHED THEN
                    INSERT (id, id2,
                            is_true, ts,
                            p_code, user_code,
                            price, size2)
                    VALUES (p_id, p_id2, p_true,
                            p_ts, p_p_code, p_user_code,
                            p_price, p_size);
        END;
        OPEN p_g_cursor FOR
           SELECT *
           FROM   my_table
           WHERE  1 = 0;
     END;
  /

Tom Kyte
April 25, 2007 - 3:58 pm UTC

see above, you have a trigger...

Thanks

A reader, April 25, 2007 - 4:19 pm UTC

thanks for your reply but , that is not a trigger.

As I did not want to post the real variable names I modified the procedure, actually it is a package A_UTIL.insert_proc
in the schema called MYSCHEMA

sorry for that confusion. It really failing on



  ON (    a.id2 = p_id2
                     AND a.is_true = p_true)"


as it somehow can't recognize the in parameters...

 (SELECT p_id, p_true, p_id2 FROM   DUAL) q


Tom Kyte
April 25, 2007 - 4:31 pm UTC

it recognized them, it just doesn't know their names

USING (SELECT p_id, p_true, p_id2
FROM DUAL) q

is

USING (SELECT :b1, :b2, :b3
FROM DUAL) q

use an alias:

USING (SELECT p_id P_ID, p_true P_TRUE, p_id2 P_ID2
FROM DUAL) q

so they have names....


ops$tkyte%ORA10GR2> variable a refcursor
ops$tkyte%ORA10GR2> variable b refcursor
ops$tkyte%ORA10GR2> declare
  2          p_id number;
  3          p_true number;
  4  begin
  5      open :a for select p_id, p_true from dual;
  6      open :b for select p_id P_ID, p_true P_TRUE from dual;
  7  end;
  8  /

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> print a

       :B2        :B1
---------- ----------


ops$tkyte%ORA10GR2> print b

      P_ID     P_TRUE
---------- ----------


ops$tkyte%ORA10GR2>



this merge will 'work' (removed an update of A.ID, fixed the using clause) and if you ask me - should be the ONLY CODE, I don't know why you select and then merge, just merge:

 27                MERGE INTO my_table a
 28                   USING (SELECT p_id P_ID, p_true P_TRUE, p_id2 P_ID2
 29                          FROM   DUAL) q
 30                   ON (    a.id = p_id2
 31                       AND a.is_true = p_true)
 32                   WHEN MATCHED THEN
 33                      UPDATE
 34                         SET
 35                             ts = p_ts,
 36                             user_code = p_user_code,
 37                             price = p_price,
 38                             size2 = p_size
 39                   WHEN NOT MATCHED THEN
 40                      INSERT (id, id2,
 41                              is_true, ts,
 42                              p_code, user_code,
 43                              price, size2)
 44                      VALUES (p_id, p_id2, p_true,
 45                              p_ts, p_p_code, p_user_code,
 46                              p_price, p_size);

Thanks

A reader, April 25, 2007 - 4:37 pm UTC

Thanks tom.

I understand that I have select and than merge but I have reasons, the reason is the merge is not on the primary key it is on the unique key (which does not include primary key !!)

so there are other code snippets. you got a sharp eye man, I could not see the alias thing as fast as you saw it !! but once you said it.. I feel embarrassed for wasting your time on a silly question !!

reader

A reader, May 01, 2007 - 4:37 pm UTC

Is it posible to use "merge" statement to merge table t1 to table t2 , such that
1. if any column value of t1 is different from the corresponding column of table t2 update the row
2. if the row of t1 does not exist on t2 , then insert
3. otherwise skip the row


Tom Kyte
May 01, 2007 - 9:01 pm UTC

merge into t1
using (select * from t2 MINUS select * from t1) t2
on .......

reader

A reader, May 02, 2007 - 10:20 am UTC

"
merge into t1
using (select * from t2 MINUS select * from t1) t2
on .......
"

The columns in the "on ...." Must they be primary key. What is the significance of the "on ...." columns
Tom Kyte
May 02, 2007 - 5:17 pm UTC

it would have the columns to join T1 to T2

they need not be a primary key, no. They just need to be the correct way to join T1 to T2

Using merge in a procedure - Invalid Number Exception

Aps, June 02, 2007 - 4:04 am UTC

Hi Tom,
I was hoping to use MERGE for a particular requirement that seemed fairly simple. But ran into an issue and dont seem to understand the cause, not sure if I am missing something very basic. Procedure failes with Invalid Number ORA 01722 exception. I have a table as shown below,
CREATE TABLE SECURITY_TEST
(
  ID                 NUMBER                     NOT NULL,
  TYPE               VARCHAR2(30 CHAR)          NOT NULL,
  REMINDER_QUESTION  VARCHAR2(200 CHAR)         NOT NULL,
  REMINDER_ANSWER    VARCHAR2(200 CHAR)         NOT NULL
)

CREATE UNIQUE INDEX PK_SECURITY_TEST ON SECURITY_TEST
(ID)

Procedure is created as below
CREATE OR REPLACE PROCEDURE merge_test(p_propertyvalue VARCHAR2, p_userid NUMBER
) IS
 v_type VARCHAR2(10);
 BEGIN
   MERGE INTO SECURITY_TEST us
    USING (
       SELECT p_userid USERID,v_type SEC_TYPE
       FROM DUAL
       ) d
   ON (us.id = d.USERID AND us.type=d.SEC_TYPE)
   WHEN MATCHED THEN
     UPDATE SET reminder_question = p_propertyvalue
   WHEN NOT MATCHED THEN
     INSERT (id,  type,   reminder_question,   reminder_answer)
     VALUES(p_userid,   v_type,   p_propertyvalue, 'dummyans');
END merge_test;
/

Executing the proc as shown below
DECLARE 
  P_userid NUMBER(10);
  P_PROPERTYVALUE VARCHAR2(200);
BEGIN 
  P_userid := 123;
  P_PROPERTYVALUE := 'test';
  merge_test ( P_PROPERTYVALUE,p_userid );
END; 

This is failing consistently with an Invalid Number ORA 01722 exception But when I execute the MERGE standalone with constant values it works.
MERGE INTO SECURITY_TEST us
    USING (
       SELECT 10 USERID,'PWD' SEC_TYPE
       FROM DUAL
       ) d
   ON (us.id = d.USERID AND us.type=d.SEC_TYPE)
   WHEN MATCHED THEN
     UPDATE SET reminder_question = 'test'
   WHEN NOT MATCHED THEN
     INSERT (id,  type,   reminder_question,   reminder_answer)
     VALUES(10,   'PWD',   'test', 'dummyans');


Am I missing something very basic?
Tom Kyte
June 03, 2007 - 5:35 pm UTC

give us a full example, i tried to reproduce, had to change code just to get it to run, gave up after it worked...

ops$tkyte%ORA10GR2> CREATE OR REPLACE PROCEDURE merge_test(p_propertyvalue VARCHAR2, p_userid NUMBER) IS
  2   v_type VARCHAR2(10) := 'x';
  3   BEGIN
  4     MERGE INTO SECURITY_TEST us
  5      USING (
  6         SELECT p_userid USERID,v_type SEC_TYPE
  7         FROM DUAL
  8         ) d
  9     ON (us.id = d.USERID AND us.type=d.SEC_TYPE)
 10     WHEN MATCHED THEN
 11       UPDATE SET reminder_question = p_propertyvalue
 12     WHEN NOT MATCHED THEN
 13       INSERT (id,  type,   reminder_question,   reminder_answer)
 14       VALUES(p_userid,   v_type,   p_propertyvalue, 'dummyans');
 15  END merge_test;
 16  /

Procedure created.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> exec merge_test( 'x', 0 );

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> exec merge_test( 'x', 0 );

PL/SQL procedure successfully completed.

Merge Problem Invalid Number

Aps, June 02, 2007 - 4:08 am UTC

Hi Tom,

Please note I am setting

v_type := 'PWD' in the proc. Sorry missed writing that in my prev comment. So my proc actually looks like.

CREATE OR REPLACE PROCEDURE merge_test(p_propertyvalue VARCHAR2, p_userid NUMBER
) IS

 v_type VARCHAR2(10);
 
BEGIN
 
 v_type := 'PWD';
  
   MERGE INTO SECURITY_TEST us
    USING (
       SELECT p_userid USERID,v_type SEC_TYPE
       FROM DUAL
       ) d
   ON (us.id = d.USERID AND us.type=d.SEC_TYPE)
   WHEN MATCHED THEN
     UPDATE SET reminder_question = p_propertyvalue
   WHEN NOT MATCHED THEN
     INSERT (id,  type,   reminder_question,   reminder_answer)
     VALUES(p_userid,   v_type,   p_propertyvalue, 'dummyans');
END merge_test;
/


Tom Kyte
June 03, 2007 - 5:36 pm UTC

see above, need directions to fail.

Merge

Aps, June 04, 2007 - 6:24 am UTC

Hi Tom,

That was the full example I had provided. Both the approaches below are giving the same Invalid Number error. Kind of clueless what could be the reason.

DECLARE 
  P_userid NUMBER(10);
  P_PROPERTYVALUE VARCHAR2(200);

BEGIN 
  P_userid := 123;
  P_PROPERTYVALUE := 'test';

  merge_test ( P_PROPERTYVALUE,p_userid );
END; 

OR
exec merge_test('test',123);

Tom Kyte
June 05, 2007 - 8:26 am UTC

I cannot reproduce - do we perhaps need some data in the table or something.

take your example in a 'clean schema' and make it reproduce from start to finish, cut and paste the entire thing (inserts and all)

ops$tkyte%ORA9IR2> CREATE OR REPLACE PROCEDURE merge_test(p_propertyvalue VARCHAR2, p_userid NUMBER) IS
  2
  3   v_type VARCHAR2(10);
  4
  5  BEGIN
  6
  7      v_type := 'PWD';
  8
  9     MERGE INTO SECURITY_TEST us
 10      USING (
 11         SELECT p_userid USERID,v_type SEC_TYPE
 12         FROM DUAL
 13         ) d
 14     ON (us.id = d.USERID AND us.type=d.SEC_TYPE)
 15     WHEN MATCHED THEN
 16       UPDATE SET reminder_question = p_propertyvalue
 17     WHEN NOT MATCHED THEN
 18       INSERT (id,  type,   reminder_question,   reminder_answer)
 19       VALUES(p_userid,   v_type,   p_propertyvalue, 'dummyans');
 20  END merge_test;
 21  /

Procedure created.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> DECLARE
  2    P_userid NUMBER(10);
  3    P_PROPERTYVALUE VARCHAR2(200);
  4
  5  BEGIN
  6    P_userid := 123;
  7    P_PROPERTYVALUE := 'test';
  8
  9    merge_test ( P_PROPERTYVALUE,p_userid );
 10  END;
 11  /

PL/SQL procedure successfully completed.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> exec merge_test('test',123);

PL/SQL procedure successfully completed.


MERGE without the clause for INSERT

Rama Subramanian G, June 05, 2007 - 1:00 am UTC

Hi Tom,

Suppose that when I use merge, I need to only update when matching records are found. No action needs to be taken on non matching rows.

Is there any restriction that I necessarily handle the non matched rows as well ?

Warm Regards
Rama Subramanian G
Tom Kyte
June 06, 2007 - 12:49 pm UTC

depends....

entirely on the version. in 9i you had to supply both. in 10g, you do not.

To Rama Subramanian G

Michel CADOT, June 05, 2007 - 9:31 am UTC


So why not just use update in this case?

Regards
Michel

Tom Kyte
June 06, 2007 - 1:01 pm UTC

typically - to avoid the "non key preserved" issue...

MERGE Invalid Number

Aps, June 05, 2007 - 12:15 pm UTC

Hi Tom,

Started with a fresh schema on a fresh database machine. Still same problem. As system user created a new schema. And this isnt a local issue, facing it on different environments we are using, both on Unix and Windows servers, that's what seems confusing.

create user test_user identified by test_user
grant resource, connect to test_user


CREATE TABLE SECURITY_TEST
(
  ID                 NUMBER                     NOT NULL,
  TYPE               VARCHAR2(30 CHAR)          NOT NULL,
  REMINDER_QUESTION  VARCHAR2(200 CHAR)         NOT NULL,
  REMINDER_ANSWER    VARCHAR2(200 CHAR)         NOT NULL
)

CREATE UNIQUE INDEX PK_SECURITY_TEST ON SECURITY_TEST
(ID)

CREATE OR REPLACE PROCEDURE merge_test(p_propertyvalue VARCHAR2, p_userid NUMBER
) IS
 v_type VARCHAR2(10);
 BEGIN
   MERGE INTO SECURITY_TEST us
    USING (
       SELECT p_userid USERID,v_type SEC_TYPE
       FROM DUAL
       ) d
   ON (us.id = d.USERID AND us.type=d.SEC_TYPE)
   WHEN MATCHED THEN
     UPDATE SET reminder_question = p_propertyvalue
   WHEN NOT MATCHED THEN
     INSERT (id,  type,   reminder_question,   reminder_answer)
     VALUES(p_userid,   v_type,   p_propertyvalue, 'dummyans');
END merge_test;
/

DECLARE 
  P_userid NUMBER(10);
  P_PROPERTYVALUE VARCHAR2(200);
BEGIN 
  P_userid := 123;
  P_PROPERTYVALUE := 'test';
  merge_test ( P_PROPERTYVALUE,p_userid );
END;

Tom Kyte
June 06, 2007 - 1:24 pm UTC

I want YOU to cut and paste something like THIS (your code has the NULL constraint issue AGAIN! this is not the code you ran to test this!!!)

ops$tkyte%ORA10GR2> drop user test_user cascade;

User dropped.

ops$tkyte%ORA10GR2> create user test_user identified by test_user;

User created.

ops$tkyte%ORA10GR2> grant resource, connect to test_user;

Grant succeeded.

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> connect test_user/test_user
Connected.
test_user%ORA10GR2>
test_user%ORA10GR2> CREATE TABLE SECURITY_TEST
  2  (
  3    ID                 NUMBER                     NOT NULL,
  4    TYPE               VARCHAR2(30 CHAR)          NOT NULL,
  5    REMINDER_QUESTION  VARCHAR2(200 CHAR)         NOT NULL,
  6    REMINDER_ANSWER    VARCHAR2(200 CHAR)         NOT NULL
  7  )
  8  /

Table created.

test_user%ORA10GR2>
test_user%ORA10GR2> CREATE UNIQUE INDEX PK_SECURITY_TEST ON SECURITY_TEST
  2  (ID)
  3  /

Index created.

test_user%ORA10GR2>
test_user%ORA10GR2> CREATE OR REPLACE PROCEDURE merge_test(p_propertyvalue VARCHAR2, p_userid
  2  NUMBER
  3  ) IS
  4   v_type VARCHAR2(10) := 'PWD';
  5   BEGIN
  6     MERGE INTO SECURITY_TEST us
  7      USING (
  8         SELECT p_userid USERID,v_type SEC_TYPE
  9         FROM DUAL
 10         ) d
 11     ON (us.id = d.USERID AND us.type=d.SEC_TYPE)
 12     WHEN MATCHED THEN
 13       UPDATE SET reminder_question = p_propertyvalue
 14     WHEN NOT MATCHED THEN
 15       INSERT (id,  type,   reminder_question,   reminder_answer)
 16       VALUES(p_userid,   v_type,   p_propertyvalue, 'dummyans');
 17  END merge_test;
 18  /

Procedure created.

test_user%ORA10GR2>
test_user%ORA10GR2> DECLARE
  2    P_userid NUMBER(10);
  3    P_PROPERTYVALUE VARCHAR2(200);
  4  BEGIN
  5    P_userid := 123;
  6    P_PROPERTYVALUE := 'test';
  7    merge_test ( P_PROPERTYVALUE,p_userid );
  8  END;
  9  /

PL/SQL procedure successfully completed.



Not unwilling to help, but I sort of do need a working example.

IOT with Partition and MERGE

Eric, June 06, 2007 - 6:10 pm UTC

My boss has asked me to show an example of using an IOT with partitioning. To fill it, I'm trying the MERGE statement, so I wouldn't have to do a semi-join or anti-join. Took me a bit to get the syntax, but now I'm getting a PK constraint. I'm not sure which part is causing that. Can you point me in the direction to get this?

The reason to play date math is that in the tbl it is a TIMESTAMP and I'm trying to convert it to DATE format in this IOT.

Looking at the table, I see one row of data. I am expecting something like 18 million rows.


CREATE TABLE iot_test 
(
   when           DATE,             -- YYYYMMDDHH24
   pcode          VARCHAR2(20), 
   cpcode         VARCHAR2(20), 
   category       VARCHAR2(250),
   userid         VARCHAR2(1000),
   CONSTRAINT iot_test_pk PRIMARY KEY 
   (
      when,
      pcode,
      cpcode,
      category
   )
)
ORGANIZATION INDEX
COMPRESS 3
PCTFREE 10
LOGGING
INCLUDING userid
OVERFLOW
TABLESPACE rpt_large_idx 
PARTITION BY RANGE ( when )
(
   PARTITION p200601 VALUES LESS THAN ( TO_DATE ( '200602', 'YYYYMM' ) ),
   PARTITION p200602 VALUES LESS THAN ( TO_DATE ( '200603', 'YYYYMM' ) ),
   PARTITION p200603 VALUES LESS THAN ( TO_DATE ( '200604', 'YYYYMM' ) ),
...
   PARTITION p_max VALUES LESS THAN ( MAXVALUE )
 ); 

MERGE INTO iot_test T 
USING ( 
   SELECT 
      TO_DATE ( TO_CHAR  ( starttime, 'YYYYMMDDHH24' ), 'YYYYMMDDHH24' ) AS ts, 
      pcode AS pc, 
      cpcode AS cp, 
      NVL ( category, '-' ) AS cat, 
      userid AS id 
   FROM 
      tbl
   ) R 
ON ( R.ts   = T.when          AND 
     R.pc   = T.pcode         AND 
     R.cp   = T.cpcode        AND 
     R.cat  = T.category      AND 
     R.ts   BETWEEN TO_DATE ( '200606', 'YYYYMM' ) AND 
                    TO_DATE ( '200901', 'YYYYMM' ) 
) 
WHEN MATCHED THEN
   UPDATE SET 
      T.userid = R.id
WHEN NOT MATCHED THEN 
   INSERT ( 
      T.when, 
      T.pcode, 
      T.cpcode, 
      T.category, 
      T.userid ) 
   VALUES ( 
      R.ts, 
      R.pc, 
      R.cp, 
      R.cat, 
      R.id ); 

MERGE INTO iot_test T
*
ERROR at line 1:
ORA-00001: unique constraint (RPTMGR.IOT_TEST_PK) violated


Tom Kyte
June 06, 2007 - 9:34 pm UTC

we sort of don't have your data, so, it is really hard to comment.

ORA-00001 -- MERGE is deterministic

Duke Ganote, June 07, 2007 - 9:02 am UTC

We're waiting for the basic demonstration that the incoming data is unique for your primary key constraint,
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:5318183934935#52657699523363
that is, nothing is returned for:
select ts, pc, cp, cat
( 
   SELECT 
      TO_DATE ( TO_CHAR  ( starttime, 'YYYYMMDDHH24' ), 'YYYYMMDDHH24' ) AS ts, 
      pcode AS pc, 
      cpcode AS cp, 
      NVL ( category, '-' ) AS cat, 
      userid AS id 
   FROM 
      tbl
   ) R
group by ts, pc, cp, cat
having count(*) > 1

I'm not sure what this statement "Looking at the table, I see one row of data. I am expecting something like 18 million rows" implies.

predicate on source vs. predicate on router

Gabe, June 07, 2007 - 12:58 pm UTC

Eric,

Your PK violation could be attributed to few things related to the data we don't see.

But one thing which stands out is:
R.ts   BETWEEN TO_DATE ( '200606', 'YYYYMM' ) AND 
               TO_DATE ( '200901', 'YYYYMM' )

in the ON clause.

Say you have 2 "similar" rows in the source and target. If the date component is, let's say, '200605' then it won't match the ON condition and be treated as an INSERT ... and hence the PK violation. If your intention was to filter source rows to that time range then that predicate should've been in the USING clause.

MEREGE

LUAY ZUBAIDY, June 12, 2007 - 4:00 am UTC

HI ,,,,
WHEN I USING MERGE INSIDE PROCEDURE LIKE
MERGE INTO TABLE1 USING (SELECT COL FROM TAB2 WHERE COL1=VARIABLE)
VARIABLE PASSED TO PROCEDURE USING THIS MERGE STATMENT
THE FOLLLOWING ERROR WILL APPEAR:
ORA-00600: internal error code, arguments: [%s], [%s], [%s], [%s], [%s], [%s]
Tom Kyte
June 12, 2007 - 10:11 am UTC

might be all of the upper case letters...

seriously, ora-600, 7445, 3113 -> please utilize support.

conditional merge

prasanna, August 01, 2007 - 5:07 pm UTC

Is it possible to use merge just to update certain rows based on a condition while leaving the rest of the rows untouched during the update.
The merge should insert extra row sets that are not in the target though...
I am not able to get a way around it...

Here is the test case:

create table testit(a number primary key,c char(1));
insert into testit values(1,'A');
insert into testit values(2,'A');
insert into testit values(3,'C');
insert into testit values(4,'C');
insert into testit values(5,'F');
insert into testit values(6,'F');
commit;
create table testit1(a number primary key);
insert into testit1 select a from testit;
insert into testit values(7);
commit;

merge into testit t1
using
(
select a from testit1
) c_rec
on
(
t1.a = c_rec.a
and t1.a = 4
)
when matched then
update set c = 'B'
when not matched then
insert
(a,c)
values(c_rec.a,'P');

The above merge did not work...

As you can notice, the requirement is to just update row with testit.c = 4 (It is a business rule)
and not affecting any other row values, but also inserting the extra row from testit1 that does not have a matching row set in testit which is the target...

Any suggestions would be great...
Tom Kyte
August 05, 2007 - 10:12 am UTC

if by "not work" you meant - I got the following error - then you need to tell us what you would have happen?????

ops$tkyte%ORA9IR2> merge into testit t1
  2  using
  3  (
  4  select a from testit1
  5  ) c_rec
  6  on
  7  (
  8  t1.a = c_rec.a
  9  and t1.a = 4
 10  )
 11  when matched then
 12  update set c = 'B'
 13  when not matched then
 14  insert
 15  (a,c)
 16  values(c_rec.a,'P');
merge into testit t1
*
ERROR at line 1:
ORA-00001: unique constraint (OPS$TKYTE.SYS_C002807) violated


conditional merge

prasanna, August 02, 2007 - 10:56 am UTC

Is it possible to use merge just to update certain rows based on a condition while leaving the rest of the rows untouched during the update.
The merge should insert extra row sets that are not in the target though...
I am not able to get a way around it...

Here is the test case:

create table testit(a number primary key,c char(1));
insert into testit values(1,'A');
insert into testit values(2,'A');
insert into testit values(3,'C');
insert into testit values(4,'C');
insert into testit values(5,'F');
insert into testit values(6,'F');
commit;
create table testit1(a number primary key);
insert into testit1 select a from testit;
insert into testit values(7);
commit;

select * from testit;

A C
---------- -
1 X
2 X
3 X
4 X
5 X
6 X
7 Y
8 Y
9 N
10 Y
11 Y

A C
---------- -
99 P

select * from testit1;

A
----------
1
2
3
4
5
6
99

merge into testit t1
using
(
select a from testit1
) c_rec
on
(
t1.a = c_rec.a
and t1.a = 4
)
when matched then
update set c = 'B'
when not matched then
insert
(a,c)
values(c_rec.a,'P');

The above merge did not work...

As you can notice, the requirement is to just update row with testit.c = 4 (It is a business rule)
and not affecting any other row values, but also inserting the extra row from testit1 that does not have a matching row set in testit which is the target...

Any suggestions would be great...

conditional merge

Prasanna, August 06, 2007 - 6:43 pm UTC

Hi Tom,
Yes...I got the unique constraint error same as what you got...
Apologies for not mentioning that...

Now, can you answer the question of using merge to just update selective matching rows (based on some business rule) while still being able to insert extra rows that are in the source but not in target...

Please use the above test case that I had given...

Tom Kyte
August 07, 2007 - 11:34 am UTC

I did answer the question

however - what do you THINK would happen here??????

You have a duplicate key - ummm, what should happen???????????

Please give some "logic", it is obvious that the above statement MUST FAIL, you are duplicating your key.

Conditional merge

prasanna, August 08, 2007 - 4:42 pm UTC

Hi Tom,

I thought I had my requirements detailed well to begin with...!
Anyway, here I go again...

Test case:

create table testit(a number primary key,c char(1));
create table testit1(a number primary key);

INSERT INTO TESTIT ( A, C ) VALUES ( 1, 'X');
INSERT INTO TESTIT ( A, C ) VALUES ( 2, 'X');
INSERT INTO TESTIT ( A, C ) VALUES ( 3, 'X');
INSERT INTO TESTIT ( A, C ) VALUES ( 4, 'X');
INSERT INTO TESTIT ( A, C ) VALUES ( 5, 'X');
INSERT INTO TESTIT ( A, C ) VALUES ( 6, 'X');
INSERT INTO TESTIT ( A, C ) VALUES ( 7, 'Y');
INSERT INTO TESTIT ( A, C ) VALUES ( 8, 'Y');
INSERT INTO TESTIT ( A, C ) VALUES ( 9, 'N');
INSERT INTO TESTIT ( A, C ) VALUES ( 10, 'Y');
INSERT INTO TESTIT ( A, C ) VALUES ( 11, 'Y');
COMMIT;

INSERT INTO TESTIT1 ( A ) VALUES ( 1);
INSERT INTO TESTIT1 ( A ) VALUES ( 2);
INSERT INTO TESTIT1 ( A ) VALUES ( 3);
INSERT INTO TESTIT1 ( A ) VALUES ( 4);
INSERT INTO TESTIT1 ( A ) VALUES ( 5);
INSERT INTO TESTIT1 ( A ) VALUES ( 6);
INSERT INTO TESTIT1 ( A ) VALUES ( 99);
COMMIT;

select * from testit;

A C
---------- -
1 X
2 X
3 X
4 X
5 X
6 X
7 Y
8 Y
9 N
10 Y
11 Y

select * from testit1;

A
----------
1
2
3
4
5
6
99

Requirement:
I want to update/insert records into testit (target table) based on the records from testit1.
Lets assume the records to be updated have a business rule where only a particular record that satisfies the rule need to be updated and all other records with matching key(for updates) should not be disturbed...
However, any new record from testit1(source) that is not present currently on testit should still be inserted...

I tried to use merge this way:
merge into testit t1
using
(
select a from testit1
) c_rec
on
(
t1.a = c_rec.a
and t1.a = 4
)
when matched then
update set c = 'B'
when not matched then
insert
(a,c)
values(c_rec.a,'P');

In the above merge, as you can notice, only record with testit.a=4 needs to be updated from testit1(source) and all other records (even though they may match by key)
should not be updated/disturbed...
Also, if you notice the data, there is one extra record in testit1 (for a=99) that is not there in testit ...This should still be inserted ...

The above merge gives me a unique constraint...

Expected result:

select * from testit:

A C
---------- -
1 X
2 X
3 X
4 B
5 X
6 X
7 Y
8 Y
9 N
10 Y
11 Y
99 P

If I comment out the extra logic as you can notice in the merge below:

merge into testit t1
using
(
select a from testit1
) c_rec
on
(
t1.a = c_rec.a
--and t1.a = 4
)
when matched then
update set c = 'B'
when not matched then
insert
(a,c)
values(c_rec.a,'P');

The unique constraint error goes away ...
But, it does not give the desired result as the update is affecting all matching rows...

select * from testit;

A C
---------- -
1 B
2 B
3 B
4 B
5 B
6 B
7 Y
8 Y
9 N
10 Y
11 Y
99 P

This is the reason why I introdiced a filter "t1.a = 4" in the "ON" clause...to just update testit.a=4...


Tom Kyte
August 14, 2007 - 10:34 am UTC

so, ummm, don't insert the number 4, insert 99 - or whatever makes you use 99 (seems rather OBVIOUS - doesn't it? You are inserting rec.a, stop doing that, insert whatever new key you deem correct and proper - you are inserting duplicate data - don't do that)

conditional merge

prasanna, August 14, 2007 - 4:41 pm UTC

Hi Tom,
I have not understood your reply...
I obviously cannot hard code the values in the insert ...so, what should it be instead of c_rec.a!
I do know that 99 value needs to be inserted as it is not in the testit table and it is only present in the testit1 table which is the source...
It is only in the update that I am having problems of unique constraint when I add the additional filter:ttestit.a=4...
A little more clarity from your end would be much appreiciated...
I have given the complete test case remember...:)

Tom Kyte
August 20, 2007 - 12:45 pm UTC

I don't understand why you don't see why this obviously fails.

c_rec.a is already in the table (by definition)
c_rec.a cannot be inserted in there again (by definition)

so - what value should be inserted - what LOGIC do you have to supply a value for that attribute.

You have given a complete test case that shows the database functions perfectly. Anyone looking at your example would say "no kidding it fails, what do you expect to have happen, you are OBVIOUSLY trying to insert the same unique value"

I don't care how complete you "test is", it only demonstrates the obvious here - you need to tell us "what should be done upon that insert, with the duplicate value - what value DID YOU MEAN TO INSERT" (perhaps you meant to use a sequence or something else to give a new value to c_rec.a

wasteful updates ...

Gabe, August 20, 2007 - 4:59 pm UTC

Prasanna,

flip@FLOP> merge into testit t1
  2  using ( select a from testit1
  3        ) c_rec
  4  on ( t1.a = c_rec.a )
  5  when matched then
  6    update set t1.c = decode(t1.a,4,'B',t1.c)
  7  when not matched then
  8    insert (t1.a,t1.c) values(c_rec.a,'P')
  9  ;

7 rows merged.

flip@FLOP> select * from testit;

         A C
---------- -
         1 X
         2 X
         3 X
         4 B
         5 X
         6 X
         7 Y
         8 Y
         9 N
        10 Y
        11 Y
        99 P

12 rows selected.

flip@FLOP>

wasteful updates

Prasanna, August 27, 2007 - 11:25 am UTC

Attaboy...
Many thanks Gabe...and to Tom as well...
But, it would be tricky and cumborsome I guess if the condition needs to be applied to more than one value of testit.a!
But, the decode is neat...damn me I didnt think of it !
Thanks again...

MERGE statement RETURNS CLAUSE followup

Balaji Chellappa, January 25, 2008 - 9:48 am UTC

Hi Tom,
It looks like even 11g doesn't support RETURNS CLAUSE with Merge statement.
But I like the Merge statement since it combines both insert and update into one.
In case of a single row insert update scenario we will always have the primary key in a variable and it will be null for the insert case.
So we can Use Merge statement like this..

PROCEDURE SAVE_EMP(io_emp_id IN OUT NUMBER, i_ename VARCHAR2) AS
BEGIN
    
    MERGE INTO EMP USING (EMP_ID  = io_emp_id)
    WHEN MATCHED THEN UPDATE SET ENAME = i_ename
    WHEN NOT MATCHED THEN INSERT(ENAME) VALUES(i_ename); -- If EMP_ID is generated in Insert Trigger through Sequence
    -- or
    WHEN NOT MATCHED THEN INSERT(EMP_ID,ENAME) VALUES(EMP_ID_SEQ.nextval,i_ename); -- If there is no Insert Trigger
    
    IF io_emp_id IS NULL THEN -- New Emp Creation Request 
        io_emp_id := EMP_ID_SEQ.currval; -- Oracle 11g
        SELECT EMP_ID_SEQ.currval INTO io_emp_id FROM DUAL; -- Lower versions..
       
    END IF; 

END;

I have a general question for you.
Is there a keyword or hint I can use in Google that will take me to the Oracle documentation..
For example if I want to see the documentation for html tag "TABLE" in MSDN I just need to type in MSDN TABLE in Google.
If I type in ORACLE MERGE STATEMENT in Google I don't get the Oracle documentation link in first page..

PS: PLSQL Merge documentation says merge statement is not efficient for single insert/update. http://download.oracle.com/docs/cd/B28359_01/appdev.111/b28370/langelems.htm#LNPLS01329 Usage Notes
This statement is primarily useful in data warehousing situations where large amounts of data are commonly inserted and updated. If you only need to insert or update a single row, it is more efficient to do that with the regular PL/SQL techniques: try to update the row, and do an insert instead if the update affects zero rows; or try to insert the row, and do an update instead if the insert raises an exception because the table already contains that primary key.
Tom Kyte
January 25, 2008 - 11:05 am UTC

merge will in general best by a large margin the
insert
exception
   update


approach, by a large margin if there are any duplicates (catching and handling exceptions is expensive, causing the duplicate - is expensive)

merge will in general be comparable to

update
if sql%rowcount=0
then
   insert


but I would say that the update/insert approach is a bit flawed as the insert can block in a multi-user environment and hence it would really be:


update
if rowcount=0
then
    insert
    exception
    when dup val on index then update


If 99.999999% of the time either the initial update works
or 99.999999% of the time the initial insert works

they would be better than merge. When it gets to be more distributed between the two, merge comes into favor.




site:oracle.com search terms

might be appropriate on google - or get the firefox plugin

http://www.google.com/search?q=oracle+documentation+search+firefox

I usually use the oracle documentation search itself, it was specially designed to search the documentation set, the quality of the results are quite good.



ops$tkyte%ORA11GR1> create table stage as select object_id, object_name, owner, object_type from all_objects where rownum <= 20000;

Table created.

ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1> create table t1
  2  as
  3  select object_id, object_name, owner, object_type
  4    from all_objects
  5   where 1=0;

Table created.

ops$tkyte%ORA11GR1> alter table t1 add constraint t1_pk primary key(object_id);

Table altered.

ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1> create table t2
  2  as
  3  select object_id, object_name, owner, object_type
  4    from all_objects
  5   where 1=0;

Table created.

ops$tkyte%ORA11GR1> alter table t2 add constraint t2_pk primary key(object_id);

Table altered.

ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1> create or replace procedure do_merge
  2  ( p_object_id in number, p_object_name in varchar2, p_owner in varchar2, p_object_type in varchar2 )
  3  as
  4  begin
  5      merge into t1 t
  6      using
  7      (select p_object_id object_id, p_object_name object_name, p_owner owner, p_object_type object_type
  8         from dual) x
  9      on (t.object_id = x.object_id)
 10      when matched then update set object_name = x.object_name, owner = x.owner, object_type = x.object_type
 11      when not matched then insert( object_id, object_name, object_type, owner )  values (x.object_id, x.object_name, x.object_type, x.owner);
 12  end;
 13  /

Procedure created.

ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1> create or replace procedure do_insert_then_update
  2  ( p_object_id in number, p_object_name in varchar2, p_owner in varchar2, p_object_type in varchar2 )
  3  as
  4  begin
  5      insert into t2 (object_id, object_name, owner, object_type )
  6      values ( p_object_id, p_object_name, p_owner, p_object_type );
  7  exception
  8      when dup_val_on_index
  9          then update t2 set object_name = p_object_name, owner = p_owner, object_type = p_object_type
 10                where object_id = p_object_id;
 11  end;
 12  /

Procedure created.

ops$tkyte%ORA11GR1> create or replace procedure do_update_then_insert
  2  ( p_object_id in number, p_object_name in varchar2, p_owner in varchar2, p_object_type in varchar2 )
  3  as
  4  begin
  5      update t2 set object_name = p_object_name, owner = p_owner, object_type = p_object_type
  6       where object_id = p_object_id;
  7      if ( sql%rowcount = 0 )
  8      then
  9          insert into t2 (object_id, object_name, owner, object_type )
 10          values ( p_object_id, p_object_name, p_owner, p_object_type );
 11      end if;
 12  end;
 13  /

Procedure created.

ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1> exec runStats_pkg.rs_start;

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR1> begin
  2      for x in ( select object_id, object_name, owner, object_type from stage )
  3      loop
  4          do_merge( x.object_id, x.object_name, x.owner, x.object_type );
  5      end loop;
  6      for x in ( select object_id, object_name, owner, object_type from stage )
  7      loop
  8          do_merge( x.object_id, x.object_name, x.owner, x.object_type );
  9      end loop;
 10  end;
 11  /

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR1> exec runStats_pkg.rs_middle;

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR1> begin
  2      for x in ( select object_id, object_name, owner, object_type from stage )
  3      loop
  4          do_insert_then_update( x.object_id, x.object_name, x.owner, x.object_type );
  5      end loop;
  6      for x in ( select object_id, object_name, owner, object_type from stage )
  7      loop
  8          do_insert_then_update( x.object_id, x.object_name, x.owner, x.object_type );
  9      end loop;
 10  end;
 11  /

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR1> exec runStats_pkg.rs_stop(100000);
Run1 ran in 616 hsecs
Run2 ran in 2633 hsecs
run 1 ran in 23.4% of the time

Name                                  Run1        Run2        Diff
STAT...no work - consistent re      19,164     158,565     139,401
LATCH.row cache objects              1,117     181,051     179,934
STAT...consistent gets from ca       1,083     181,044     179,961
STAT...recursive calls              41,811     241,682     199,871
STAT...consistent gets             100,865     301,375     200,510
STAT...consistent gets from ca     100,865     301,375     200,510
STAT...db block gets                84,249     284,891     200,642
STAT...db block gets from cach      84,249     284,891     200,642
STAT...session logical reads       185,114     586,266     401,152
LATCH.cache buffers chains         502,790   1,177,894     675,104
STAT...physical read total byt   1,024,000           0  -1,024,000
STAT...physical read bytes       1,024,000           0  -1,024,000
STAT...undo change vector size   5,890,840   7,170,752   1,279,912
STAT...table scan rows gotten      114,577   2,146,252   2,031,675
STAT...redo size                17,164,588  26,558,072   9,393,484

Run1 latches total versus runs -- difference and pct
Run1        Run2        Diff       Pct
569,986   1,818,148   1,248,162     31.35%

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1> truncate table t1;

Table truncated.

ops$tkyte%ORA11GR1> truncate table t2;

Table truncated.

ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1>
ops$tkyte%ORA11GR1> exec runStats_pkg.rs_start;

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR1> begin
  2      for x in ( select object_id, object_name, owner, object_type from stage )
  3      loop
  4          do_merge( x.object_id, x.object_name, x.owner, x.object_type );
  5      end loop;
  6      for x in ( select object_id, object_name, owner, object_type from stage )
  7      loop
  8          do_merge( x.object_id, x.object_name, x.owner, x.object_type );
  9      end loop;
 10  end;
 11  /

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR1> exec runStats_pkg.rs_middle;

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR1> begin
  2      for x in ( select object_id, object_name, owner, object_type from stage )
  3      loop
  4          do_update_then_insert( x.object_id, x.object_name, x.owner, x.object_type );
  5      end loop;
  6      for x in ( select object_id, object_name, owner, object_type from stage )
  7      loop
  8          do_update_then_insert( x.object_id, x.object_name, x.owner, x.object_type );
  9      end loop;
 10  end;
 11  /

PL/SQL procedure successfully completed.

ops$tkyte%ORA11GR1> exec runStats_pkg.rs_stop(100000);
Run1 ran in 613 hsecs
Run2 ran in 471 hsecs
run 1 ran in 130.15% of the time

Name                                  Run1        Run2        Diff
STAT...redo size                17,164,320  17,582,956     418,636

Run1 latches total versus runs -- difference and pct
Run1        Run2        Diff       Pct
606,840     608,606       1,766     99.71%

PL/SQL procedure successfully completed.


DUAL usage in MERGE

Balaji Chellappa, January 25, 2008 - 1:21 pm UTC

Thanks Tom.
Even though MERGE statement expects a source/reference table (in our case Dual) it doesn't expect the source table should have the same structure as table getting merged (in our case t1).

So I usually use the parameters directly in the update as well as insert on a Merge statement.

merge into t1 t
using (select p_object_id object_id, p_object_name object_name, p_owner owner, p_object_type object_type
        from dual) x
   on (t.object_id = x.object_id)
 when matched then update set object_name = x.object_name, owner = x.owner, object_type = x.object_type
 when not matched then insert( object_id, object_name, object_type, owner ) 
                       values(x.object_id, x.object_name, x.object_type, x.owner);

can be written as
merge into t1 t
using dual
   on (t.object_id = p_object_id)
 when matched then update set object_name = p_object_name, owner = p_owner, object_type = p_object_type
 when not matched then insert( object_id, object_name, object_type, owner ) 
                       values(p_object_id, p_object_name, p_object_type, p_owner);


It just saves few extra key strokes.

Update followed by an inser

Otn, January 27, 2008 - 6:56 pm UTC

scenario 1:

begin

update...
.....;

If sql%rowcount =0 then then

inser ....
....;

end;

scenario 2:

begin

insert ....
....;

exception

when dup_val_on_index then

update....


END;


1. Which is more efficient peformance wise an update followed by an insert if the row is not found, or an insert followed by an in update if the row is found?

2.Merge statement in SQL is an implementation of the functionality i n scenario 1. Correct ?
Tom Kyte
January 29, 2008 - 2:24 am UTC

1) the only answer possible is

it depends. what if the data almost NEVER exists, doing an update (searching for the data) then an insert would take more effort in general than doing an insert (which almost always works in this example) and infrequently an update.

what if the data almost ALWAYS exists, doing an insert, failing - raising the exception, that is a lot of work and you do it over and over - then you do the update anyway. better to do the update and skip the attempt to insert.


what if you are somewhere in the middle? then - well - I think you see what I mean...


in general, I would "update, then insert" because doing the insert and having it fail is a lot of work typically.

2) merge does an update of existing data, insert of new, yes.

Re: update/insert or insert/update

Saad Ahmad, February 16, 2008 - 6:35 pm UTC

If merge is not to be used, then there is only one "right" answer that will not fail due to timing issues - and that is insert and then update. (Assuming table has a PK).

If you decide update/insert then you will need to put additional locks to make sure there are no timing issues.
Tom Kyte
February 17, 2008 - 7:47 am UTC

and if you actually need to user insert/update to avoid a timing issue....

you of course have the classic lost update problem on your hand - take the same inputs and run them against the same data twice and you end up with two different answers...

because one time, session 1 inserts the record, session 2 updates it
the next time you try, session 1 updates the record, session 2 inserts it


so one time you end up with session 2 being the session that leaves the record in the database, and one time you end up with session 1 being that session.



and with the update/insert - you need NO ADDITIONAL LOCKS (think about it... should be easy to see why)


if the update updates zero records
and then you try to insert....

only one of you will insert - the other will block and then fail.

Merge giving -30926 when source table does not have duplicate rows

Saad Ahmad, February 16, 2008 - 6:47 pm UTC

Hello Tom,

I read the fair warning but the issue I am seeing is not reproducable. It runs several times then fails with this error. I reconnect and it works again. My update portion sets an audit date to sysdate so it should always fire the update.

The merge statement is :
merge into prtmst d /* pk is prtnum, wh_id_tmpl, prt_client_id */
using (
select pm.*, wh.wh_id
from var_part_sync_work w /* NO PK but unique rows for prtnum, wh_id, prt_client_id; could this be an issue there there is no PK defined? */
join prtmst_mst pm /* same pk as prtmst */
on pm.prtnum = w.prtnum
and pm.wh_id_tmpl = w.wh_id
and pm.prt_client_id = w.prt_client_id

join
(
select wh_id from wh
union all
select '----' wh_id from dual
) wh on 1=1 /* this is deliberate to fan out the results */
where 1=1
) s
on (d.prtnum = s.prtnum
and d.prt_client_id = s.prt_client_id
and d.wh_id_tmpl = s.wh_id)
when matched then
update set <list and set the moddte = sysdate>
when not matched then
insert/values where set the moddte = sysdate

The above will work several times and then fail once. I will reconnect and it would work. Just to make sure that there are no duplicates, I ran the following SQL:

select d.prtnum, d.prt_client_id, d.wh_id_tmpl, count(*)
from
prtmst d
join var_part_sync_work w
join prtmst_mst pm
on pm.prtnum = w.prtnum
and pm.wh_id_tmpl = w.wh_id
and pm.prt_client_id = w.prt_client_id
join
(
select wh_id from wh
union all
select '----' wh_id from dual
) wh on 1=1
where 1=1
) s
on (d.prtnum = s.prtnum and d.prt_client_id = s.prt_client_id and d.wh_id_tmpl = s.wh_id)
where 1=1
group by d.prtnum, d.prt_client_id, d.wh_id_tmpl
having count(*) > 1


So the SQL ensures that there are really no duplicates on source and there aren't

Database is 10.2.0.2.0 on Linux

Basic question is, "Are there any other reasone for this error?" or "Are there any known bugs for merge?"

Thanks and I appologize for not providing the create table scripts

Saad

Tom Kyte
February 17, 2008 - 8:24 am UTC

... Thanks and I appologize for not providing the create table scripts ...

no problem, just means I don't actually have to read it - I can just go onto the next one :)



actually, look at this:

select pm.*, wh.wh_id
from var_part_sync_work w /* NO PK but unique rows for prtnum, wh_id,
prt_client_id; could this be an issue there there is no PK defined? */
join prtmst_mst pm /* same pk as prtmst */
on pm.prtnum = w.prtnum
and pm.wh_id_tmpl = w.wh_id
and pm.prt_client_id = w.prt_client_id

join
(
select wh_id from wh
union all
select '----' wh_id from dual
) wh on 1=1 /* this is deliberate to fan out the results */
where 1=1
) s

"fan out"?? you mean cartesian product - but what if wh has ---- already.

Your query to test the data - please look at it. Take the query you are MERGING INTO - and see if it doesn't return duplicates, I don't see why you would use a different query to "test with". Take the query you generate in your USING clause and validate IT.

ops$tkyte%ORA10GR2> select *
  2    from dept
  3  join
  4  (select '----' from dual
  5   union all
  6   select '----' from dual)
  7  on 1=1
  8  /

    DEPTNO DNAME          LOC           '---
---------- -------------- ------------- ----
        10 Accounting     NEW YORK      ----
        20 RESEARCH       DALLAS        ----
        30 SALES          CHICAGO       ----
        40 OPERATIONS     BOSTON        ----
        10 Accounting     NEW YORK      ----
        20 RESEARCH       DALLAS        ----
        30 SALES          CHICAGO       ----
        40 OPERATIONS     BOSTON        ----

8 rows selected.




You will need to provide SIMPLE SMALL creates if you even want me to begin to look at this.

Re: insert/update or update/insert

Saad Ahmad, February 17, 2008 - 1:50 pm UTC

Update/insert has a timing issue. See below:

Scenario: Row did not exist

Time Session 1 Result 1 Session 2 Result 2
1 update -1403 update -1403
2 insert ok insert wait on lock
3 commit ok error from insert

Now insert/update - row did not exist
Time Session 1 Result 1 Session 2 Result 2
1 insert ok insert wait on lock
2. commit ok dupl error
3 update ok
4. commit


Now insert/update - row exists
Time Session 1 Result 1 Session 2 Result 2
1 insert dupl insert dupl
2. update ok update wait
3. commit ok
4. commit


Tom Kyte
February 17, 2008 - 2:06 pm UTC

I'm seeing "below" and I see no timing issues that require a programmer to explicitly lock or think about this.

You wrote:

If you decide update/insert then you will need to put additional locks to make sure there are no timing issues.


what additional locks did you need to protect the data, to get the right thing to happen.


The insert/update suffers (as I stated) from the classic "lost update" problem (non-deterministic).

The update/insert actually gets a better answer - which is "hey, STOP, you are totally messing with this data in a bad way".

I'll agree there is a timing issue that requires additionaly locking - but it is the insert/update scenario (to avoid the dreaded lost update) NOT the update/insert situation where by the natural locking and serialization that'll happen prevents that tragic lost update.





merge data from 2 tables

Ann, March 21, 2008 - 1:10 pm UTC

Hi Tom,

Table ABC
A B C
------ ------- --
A1000 500 10
B1000 600 20
C1000 700 30
D1000 800 40
E1000 900 50


Table XYZ
X Y Z
-------- ------- -------
OTHERS 10000 150.00
OPS 20000 250.00
TIG 30000 350.00

Result Merge ABC and XYZ tables
A B C X Y Z
-------------- -- --------- ------ -------
A1000 500 10 OTHERS 10000 150.00
A1000 500 10 OPS 20000 250.00
A1000 500 10 TIG 30000 350.00
B1000 600 20 OTHERS 10000 150.00
B1000 600 20 OPS 20000 250.00
B1000 600 20 TIG 30000 350.00
C1000 700 30 OTHERS 10000 150.00
C1000 700 30 OPS 20000 250.00
C1000 700 30 TIG 30000 350.00
D1000 800 40 OTHERS 10000 150.00
D1000 800 40 OPS 20000 250.00
D1000 800 40 TIG 30000 350.00
E1000 900 50 OTHERS 10000 150.00
E1000 900 50 OPS 20000 250.00
E1000 900 50 TIG 30000 350.00


I need a select query to merge for each row in ABC with all rows in XYZ


Thanks a lot

Tom Kyte
March 24, 2008 - 10:40 am UTC

that is not a merge - merge would result in 3 rows or 5 rows (depending on whether you merged 3 into 5 - resulting in 5, or 5 into 3 resulting in 3)

you appear to want a cartesian join

select * from t1, t2;


just list the tables, every row in T1 will be joined to every row in T2.

Ana, March 24, 2008 - 11:17 pm UTC

Tom .. Thanks for your help.

Merging self to update sum value

Prem, April 03, 2008 - 5:30 am UTC

Hi Tom,
I have a following situation. Would be interested to know the

correct and efficient query for the same:

I need to update records of a table by summing the amounts of matching

rows (based on set of columns being the key) . After this process, I

should be able to uniquely identify the row that has this total sum.



create table tab1(num number, sumval number, status varchar2(10));

insert into tab1(num,sumval) values(1,10);
insert into tab1(num,sumval) values(2,2);
insert into tab1(num,sumval) values(1,12);
insert into tab1(num,sumval) values(1,8);
insert into tab1(num,sumval) values(2,6);
insert into tab1(num,sumval) values(5,5);
insert into tab1(num,sumval) values(7,10);

Say I have following rows:

NUM SUMVAL
1 10
2 2
1 12
1 8
2 6
5 5
7 10

I need to have sum of matching rows and uniquely identify one row

having the sum.

NUM SUMVAL STATUS
1 30 VALID (10+12+8)
2 8 VALID (2+6)
1 12
1 8
2 6
5 5 VALID
7 10 VALID

I need to do this from Pro*c on a table having millions of records.

Hence some sort of bulk operation would be desired. could one use

merge?

I have following query, but unable to identify the unique result of

summation process:

merge into tab1 t1
using (select num,sum(sumval) sumval
from (select tab1.*,
row_number() over(partition by num order by

num) rn
from tab1 )
where rn <> 1 group by num
) t2
on (t1.num = t2.num ) -- Is there a way for us to restrict

row_number() over partition by num = 1 here ??
when matched then update set t1.sumval = t1.sumval+t2.sumval,

status = 'valid'
when not matched then insert (num) values(null)

or somthing like:
-- this gives me error though
merge into (select tab1.*,
row_number() over(partition by num order by num) rn
from tab1) t1
using (select num,sum(sumval) sumval
from tab1 group by num
) t2
on (t1.num = t2.num and t1.rn=1 )
when matched then
update set t1.status = 'valid'
when not matched then
insert (num) values(null)

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.2.0 - Prod
PL/SQL Release 10.2.0.2.0 - Production
CORE 10.2.0.2.0 Production
TNS for Linux: Version 10.2.0.2.0 - Production
NLSRTL Version 10.2.0.2.0 - Production

merge

sanjay, April 07, 2008 - 8:31 am UTC

Hi Tom,
  
Oracle9i Enterprise Edition Release 9.2.0.1.0 - Production
With the Partitioning, OLAP and Oracle Data Mining options
JServer Release 9.2.0.1.0 - Production

SQL> select * from t;
         X          Y
---------- ----------
         1          2

SQL> merge into t
  2    using dual on ( x = 1 )
  3    when matched then
  4    update set y = y+1
  5    when not matched then insert (x,y) values ( 2, 2 );
1 row merged.

SQL> merge into t
  2    using dual on ( x = 5 )
  3    when matched then
  4    update set y = y+1
  5*   when not matched then insert (x,y) values ( 2, 2 )
SQL> /
0 rows merged.

pls help me where i am wrong?

Regards

Tom Kyte
April 07, 2008 - 9:34 am UTC

get current....


SQL> select * from t;
         X          Y
---------- ----------
         1          2

SQL> merge into t
  2    using dual on ( x = 1 )
  3    when matched then
  4    update set y = y+1
  5    when not matched then insert (x,y) values ( 2, 2 );
1 row merged.

SQL> merge into t
  2    using dual on ( x = 5 )
  3    when matched then
  4    update set y = y+1
  5*   when not matched then insert (x,y) values ( 2, 2 )
SQL> /

A reader, April 21, 2008 - 5:34 pm UTC

drop table test_merge_stg purge;

create table test_merge_stg
(id number
,qty number
,val varchar2(40)
,comments varchar2(40)
,src varchar2(10)
);

insert into test_merge_stg 
values
(1,100,'A','qty is 100','prod');

insert into test_merge_stg
values
(2,100,'B','qty is 100','prod');

insert into test_merge_stg 
values
(3,100,'C','qty is 100','prod');

insert into test_merge_stg
values
(4,100,'D','qty is 100','prod');


commit;

SQL> select * from test_merge_stg;

 ID        QTY VAL  COMMENTS     SRC
--- ---------- ---- ------------ ----------
  1        100 A    qty is 100   prod
  2        100 B    qty is 100   prod
  3        100 C    qty is 100   prod
  4        100 D    qty is 100   prod


drop table test_merge_sys purge;

create table test_merge_sys
(id number
,qty number
,val varchar2(4)
,comments varchar2(40)
,src varchar2(10)
,st_dt date
,end_dt date default null
);

insert into test_merge_sys 
(id,qty,val,comments,src,st_dt,end_dt)
select id,qty,val,comments,src,sysdate,null
from test_merge_stg;

commit;


SQL> select * from test_merge_sys;

 ID QTY  VAL    COMMENTS   SRC     ST_DT       END_DT
--- ---- ------ --------- -------- ----------  ----------- 
  1  100 A      qty is 100   prod  21-APR-2008
  2  100 B      qty is 100   prod  21-APR-2008
  3  100 C      qty is 100   prod  21-APR-2008
  4  100 D      qty is 100   prod  21-APR-2008


On the next day test_merge_stg will be truncated and loaded with new data set

truncate table test_merge_stg;

insert into test_merge_stg 
values
(1,200,'A','qty is 200','prod');

insert into test_merge_stg 
values
(2,300,'B','qty is 300','prod');

insert into test_merge_stg
values
(1,100,'C','qty is 100','newprod');

insert into test_merge_stg 
values
(2,100,'D','qty is 100','newprod');

commit;

select * from test_merge_stg;

 ID        QTY VAL  COMMENTS     SRC
--- ---------- ---- ------------ ----------
  1        200 A    qty is 200   prod
  2        300 B    qty is 300   prod
  1        100 C    qty is 100   newprod
  2        100 D    qty is 100   newprod


I need to load this to test_merge_sys table based on id and src. 
1)If there is a change in qty, comments columns, I need to update the end_dt of the existing record to sysdate
2)and insert a new record with end_dt=null. 
3)If the record is not present at all, I need to insert it

I am doing the below merge to do step 1 and 3


can you tell me how do the step 2?

merge into test_merge_sys sys
  using (select * from test_merge_stg) stg
  on (sys.id = stg.id and sys.src = stg.SRC)
  when matched then
   update set sys.END_DT=sysdate
  when not matched then
   insert (sys.ID,sys.QTY,sys.VAL,sys.COMMENTS,sys.SRC,sys.ST_DT,sys.END_DT)
   values(stg.ID,stg.QTY,stg.VAL,stg.COMMENTS,stg.SRC,sysdate,null);

4 rows merged.

commit;

SQL> select * from test_merge_sys;

ID  QTY  VAL  COMMENTS     SRC        ST_DT       END_DT
--- ---------- ---- ------------ ---------- ----------- 
1   100 A    qty is 100   prod       21-APR-2008 22-APR-2008
2   100 B    qty is 100   prod       21-APR-2008 22-APR-2008
1   200 A    qty is 200   prod       21-APR-2008
2   300 B    qty is 300   prod       21-APR-2008    
3   100 C    qty is 100   prod       21-APR-2008
4   100 D    qty is 100   prod       21-APR-2008
1   100 C    qty is 100   newprod    21-APR-2008
2   100 D    qty is 100   newprod    21-APR-2008




A reader, April 28, 2008 - 1:29 pm UTC

Can you please answer the above question?
Tom Kyte
April 28, 2008 - 1:42 pm UTC

you asked...

I need to load this to test_merge_sys table based on id and src.
1)If there is a change in qty, comments columns, I need to update the end_dt of the existing record
to sysdate
2)and insert a new record with end_dt=null.
3)If the record is not present at all, I need to insert it



well, merge does not do that.

merge either

a) updates a record
b) inserts it

merge does not do a) & b), merge does a) OR b)


so, to do 1, 2, 3 you will:

1) update a join of old and new (to set the end_dt).
2) insert all of the records.


vERY GOOD

AMAN, May 24, 2008 - 3:52 pm UTC

THANKS,ARTICLE WAS OF GRT HELP

A reader, June 25, 2008 - 12:30 am UTC

CREATE TABLE DEPT
(
DEPTNO NUMBER(2),
DNAME VARCHAR2(14 BYTE),
LOC VARCHAR2(13 BYTE)
);

INSERT INTO DEPT ( DEPTNO, DNAME, LOC ) VALUES (
10, 'ACCOUNTING', 'NEW YORK');
INSERT INTO DEPT ( DEPTNO, DNAME, LOC ) VALUES (
20, 'RESEARCH', 'DALLAS');
INSERT INTO DEPT ( DEPTNO, DNAME, LOC ) VALUES (
30, 'SALES', 'CHICAGO');
INSERT INTO DEPT ( DEPTNO, DNAME, LOC ) VALUES (
40, 'OPERATIONS', 'BOSTON');
COMMIT;

if dname already exists then update LOC in Dept table and if dname not exists then insert into Dept table. Can we achieve it through MERGE. How can it be done in optimized way.

Tom Kyte
June 25, 2008 - 8:30 am UTC

I only see one table here?? what are you updating "with"

A reader, June 27, 2008 - 5:49 am UTC

yes it is one table only. The requirement is, We get "DNAME" from front end and if this "DNAME" already exists in Dept table then update that record. IF this "DNAME" is not there then Isert it.
Tom Kyte
June 27, 2008 - 9:20 am UTC

easiest approach is therefore

update
if (sql%rowcount=0)
then insert


alternate approach is

merge into t
(select :x dname, :y c2, :z c3 from dual )
on ( t.dname = d.dname )
when matched then update....
when not matched then insert ....

Merge to two tables at a single time

Raja, August 08, 2008 - 10:27 am UTC

Hi Tom,

Is it possible to do a merge into two separate tables at a single time.

MERGE INTO T1,T2
USING
......

I was not able to find any syntax on this.

Oracle Version: 10g
Thanks
Tom Kyte
August 08, 2008 - 1:26 pm UTC

no.

merge into 2 tables at once...

Duke Ganote, August 09, 2008 - 1:05 pm UTC

Curious idea; a combination of MERGE and multi-table INSERT.
Tom Kyte
August 12, 2008 - 4:37 am UTC

I don't see how it would work at all.....

merge is like an upsert - if record exists - update it, if not insert it. I don't see how it would make sense with two tables at all.

If it it exists in either table - update it and if not then what? and merging into two tables just doesn't compute in my mind, I cannot fathom the real world case whereby this would be necessary or desirable.

merge into 2 tables; redux

Duke Ganote, August 11, 2008 - 2:40 pm UTC

No wonder it sounded vaguely familiar; discussed above in 2006:
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:5318183934935#70712435912934

ORA-00904: : invalid identifier

ravi, September 10, 2008 - 10:57 am UTC

I am a regular reader of Asktom and a great fan of you. Seriously !!!

Database Version : Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Prod

I tried my best to provide sample scripts, but I could not reproduce this issue with test scripts and test data.
Due to my company policies I am not allowed to provide the actual tables, so I am giving dummy table names and column names.

Please bear with me and I need your help.

I am aware of the undocumented limitation that the keys in the on matching clause should NOT be there in the update statement.

This is a little bit big sql query which aggregates data from 2 different transaction tables for a given date.

The query that works :

merge into destiantion_table t
using (select a,b,c, run_date from 
         (
    select a,b,c, run_date from txn_a
           union all
    select a,b,c, run_date from txn_b
         ) <b>where trunc(run_date) = trunc(sysdate-1)</b> ) e
on (t.a = e.a and t.b = e.b)
when matched then
update set t.c = e.c
when not matched then
insert(c)
values(e.c)


Since the the transaction tables are having 100's of millions of rows, I wanted to limit the rows in the union itself.
So, I modified the above query to :

merge into destiantion_table t
using (select a,b,c, run_date from 
         (
    select a,b,c, run_date from txn_a
     <b>where trunc(run_date) = trunc(sysdate-1)</b>
           union all
    select a,b,c, run_date from txn_b
     <b>where trunc(run_date) = trunc(sysdate-1)</b> 
         )  ) e
on (t.a = e.a and t.b = e.b)
when matched then
update set t.c = e.c
when not matched then
insert(c)
values(e.c)


I am getting the error message :

ORA-00904: "e"."c": invalid identifier

I have also tried to change the column order in all the selects but the error message remains same.

Once again, sorry I could not provide you with test scripts and test data to reproduce the issue.

Thanks in advance for your valuable time.


Tom Kyte
September 11, 2008 - 11:05 am UTC

... I am aware of the undocumented limitation that the keys in the on matching clause should NOT be there in the update statement. ....

hmmm...
http://docs.oracle.com/docs/cd/B19306_01/server.102/b14200/statements_9016.htm#sthref9539

it has always been documented.


unless and until you give me an EXAMPLE, I cannot possibly even begin to think about looking at your issue.

just take your existing schema and rename the tables to be t1, t2, t3, ....

and their columns c1, c2, c3, .....

that should be all you have to do - then provide the create tables and the failing statement.

to Ravi ...

A reader, September 11, 2008 - 4:55 pm UTC


Syntax as posted works.

Somewhere between your actual SQL and the slimmed-down like example you posted you probably "solved" your actual syntax problem.

Since the transaction tables are having 100's of millions of rows, I wanted to limit the rows in the union itself.

What you're trying to do is unlikely to improve performance ... as long as you change to "where run_date between <this> and <that>" and have indexes on run_date you should do better.

Inserting NULLs for A and B is kind of odd though.

Might consider benchmarking two separate merge statements ... if it is applicable in your case (seems it might) ... in general though, two merge statements based on the individual source sets would not be equivalent with the one merge on the union all.

Commit frequency in Merge Statement

Akshay, October 16, 2008 - 5:45 am UTC

All,
When we execute merge functions internally it sorts both the table on the common column used for merge, and it will fail giving error "Unable to extent tablespace temp".

In my scenario I got two tables parent table having 45 million records and 2nd table having 20 millions records, I have set auto ¿ extent on temp table to true ¿.. so it can grow max up to 180GB, the merge query failed after 10 hrs. I just need to update the 2nd table.

So according to me I got two possible solution

To remove merge function and use for all, which will be faster as compare to merge.
To include commit frequency in merge function.

Please any one can let me know how we can set commit frequency in merge function


Tom Kyte
October 17, 2008 - 8:48 pm UTC

... it sorts both the table on the
common column used for merge, ...

no, it might, it might not - it would typically use hashing - not sorting.
ops$tkyte%ORA10GR2> create table t1 ( x int , data char(2000) );

Table created.

ops$tkyte%ORA10GR2> create table t2 ( x int , data char(2000) );

Table created.

ops$tkyte%ORA10GR2> exec dbms_stats.set_table_stats( user, 'T1', numrows => 45000000, numblks => 45000000/100 );

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> exec dbms_stats.set_table_stats( user, 'T2', numrows => 20000000, numblks => 20000000/100 );

PL/SQL procedure successfully completed.

ops$tkyte%ORA10GR2> @at
ops$tkyte%ORA10GR2> column PLAN_TABLE_OUTPUT format a72 truncate
ops$tkyte%ORA10GR2> set autotrace traceonly explain
ops$tkyte%ORA10GR2> merge into t1
  2  using t2
  3  on (t1.x = t2.x)
  4  when matched then update set t1.data = t2.data
  5  /

0 rows merged.


Execution Plan
----------------------------------------------------------
Plan hash value: 2683531971

------------------------------------------------------------------------
| Id  | Operation            | Name | Rows  | Bytes |TempSpc| Cost (%CPU
------------------------------------------------------------------------
|   0 | MERGE STATEMENT      |      |    45M|    88G|       |   938K  (1
|   1 |  MERGE               | T1   |       |       |       |
|   2 |   VIEW               |      |       |       |       |
|*  3 |    HASH JOIN         |      |    45M|  8583M|  2136M|   938K  (1
|   4 |     TABLE ACCESS FULL| T2   |    20M|  1907M|       | 44436   (2
|   5 |     TABLE ACCESS FULL| T1   |    45M|  4291M|       | 99980   (2
------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   3 - access("T1"."X"="T2"."X")

ops$tkyte%ORA10GR2> set autotrace off


.... which will be faster as compare to
merge.
...

hah, why do you think that?

... To include commit frequency in merge function. ...

interesting, since commiting frequently will generate more redo, more undo and cause you to wait for log file syncs (eg: committing causes you to run LONGER and would have NO EFFECT on temp space usages.......)

And of course, MERGE is a sql statement, a sql statement is ATOMIC - it either

a) happens
b) does not happen

entirely, there is no concept of committing a sql statement halfway through (thank goodness)


check your PGA settings make sure they are sufficient for the size job you are considering here.

check to see if parallel dml makes sense to you (to break the merge up into many separate smaller merges that run concurrently)

merge gives ORA-00001 when PK is part of 'ON' conditions

Troy, November 03, 2008 - 3:04 pm UTC

Hi Tom,

Re. your response on "Merge a single row October 10, 2004 - 12pm US/Eastern".

I would think that it would be nicer and make more sense that MERGE decide what to do (either insert or update) after it obtains the proper lock. It makes more sense for the following merge statement to always succeed:

create table t1 (a number primary key, b number);
MERGE INTO t1 tgt
USING ( SELECT 1 ID FROM DUAL ) src
ON ( tgt.a = src.ID )
WHEN MATCHED THEN
UPDATE SET b = NULL
WHEN NOT MATCHED THEN
INSERT( a, b )
VALUES( src.ID, -1 );

It seems that it determines to do insert as soon as it does not see matching rows but waits for the lock to be released before it actually does the insert and fails with ORA-00001.

Thanks.
Tom Kyte
November 11, 2008 - 11:46 am UTC

only if the other session commits, if the other rollsback - it continues on.

This is entirely consistent with how insert itself works.

Re: merge gives ORA-00001 when PK is part of 'ON' conditions

Troy, November 03, 2008 - 3:07 pm UTC

I thought the purpose of MERGE was to replace the following and make code simpler and more efficient:
update...
if zero rows then
insert...
when dup val on index then
update...
Tom Kyte
November 11, 2008 - 11:48 am UTC

I've never seen code like that before - I've never actually seen anyone

update
if zero rows
then
    insert
    when dup val on index
    then
        update



and no, merge is not a way to replace that code, merge reads the table consistently and if a row is present and committed as of the time the merge starts - it will update - else merge cannot see it (that would be an inconsistent read) and attempts to insert it.


new feature comparing to 9i ?

martinka, November 10, 2008 - 6:29 am UTC

i found in 10g documentation this sentence

Use the MERGE statement to select rows from one or more sources for update or insertion into one or more tables

I would like to use MERGE statement into two tables ( if matched update table1, if not matched insert into table2), however i'm desperate, since i could not find any example on it. As cited above documentation clearly says it is possible.

Is this true, is it something what was not possible in 9i and in 10g could be done? If yes what is the syntax, since i can't figure it out.

Tom Kyte
November 11, 2008 - 3:40 pm UTC

http://docs.oracle.com/docs/cd/B19306_01/server.102/b14200/statements_9016.htm#SQLRF01606

what I see is:

"Use the MERGE statement to select rows from one or more sources for update or insertion into a table or view. You can specify conditions to determine whether to update or insert into the target table or view."

If you provide a reference to elsewhere (I don't know why people don't provide a reference by 'default'), I'll have them fix it.

You merge into A thing.



What you want to do is.

merge into table1 using source on (join)
when matched then update set ....;

insert into table1
select * from source
where (key) not in (select key from table1 where key is not null);


Re: merge gives ORA-00001 when PK is part of 'ON' conditions

Troy, November 11, 2008 - 1:39 pm UTC

Hi Tom,

I thought you recommended the following code in "Followup October 10, 2004 - 6pm US/Eastern". As a matter of fact, it works just fine. I am still trying to justify the extra 'update' at the beginning, though.

update
if zero rows
then
insert
when dup val on index
then
update

Now we are all clear that merge is entirely consistent with how insert itself works - as per the current design.

Let's take a step back. Talking about insert/merge into a table with a PK:
1. INSERT has no other choices but inserting successfully or failing with ORA-1. If tx1 inserts a new PK, tx2 inserting the same PK is doomed to fail with ORA-1 EITHER a) it does the insert straight away and commit after tx1 commits (how it works now), OR b) it does both the insert and commit after tx1 commits.
2. However, MERGE does have a 2nd choice - 'UPDATE'. If tx1 inserts a new PK, tx2 inserting the same PK does not have to do insert and to fail with ORA-1. Why don't we take that 2nd choice in this case? It needs to wait for tx1 to commit before finishing the merge anyway. It could just wait for tx1 to commit before determining to do insert or update. I am just wondering why merge HAS to work the same way as insert?

Thanks.
Tom Kyte
November 11, 2008 - 4:52 pm UTC

as for the first update:

... but the performance implications of that are pretty bad if you get the dup val on index more than a
couple of times....
....

that was why - and I didn't really recommend that, I recommended merge.

it would mean that merge would be seeing rows in the table as of different points in time.

That - to us - would be like a rip in the space time continuum. Unthinkable.

It is called "correct answers"

RE: rip in the space-time continuum

Duke Ganote, November 11, 2008 - 5:13 pm UTC

The closest to a "rip" is a "restart".
http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:11504247549852
But still consistent as of "some point in time".

Converting CLOB to XMLTYPE and MERGE

Reddy K S, November 24, 2008 - 3:43 am UTC

Tom,
I have source table column 'XMLCONTENTS' is a CLOB. Target table column 'XMLCONTENTS' is a XMLTYPE. I am trying to convert CLOB to XMLTYPE and merge (Oracle Version 9.2.x).

1st attempt:
merge into BH TGT using (SELECT bbx1, bby1, bbz1, guid, XMLTYPE(xmlcontents) XMLCONTENTS FROM BH_ ) SRC
on (SRC.GUID = TGT.GUID)
when matched then update set
TGT.BBX1 = src.BBX1
,TGT.BBY1 = src.BBY1
,TGT.BBZ1 = src.BBZ1
, TGT.XMLCONTENTS = SRC.XMLCONTENTS
when not matched then insert (
TGT.GUID
, TGT.BBX1
, TGT.BBY1
, TGT.BBZ1
, TGT.XMLCONTENTS )
VALUES (
SRC.GUID
, SRC.BBX1
, SRC.BBY1
, SRC.BBZ1
, SRC.XMLCONTENTS);
merge into BH TGT using (SELECT bbx1, bby1, bbz1, guid, XMLTYPE(xmlcontents) XMLCONTENTS
*
ERROR at line 1:
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at line 1

2nd attempt:
merge into BH TGT using BH_ SRC on (SRC.GUID = TGT.GUID) when matched then update set
TGT.BBX1 = src.BBX1
,TGT.BBY1 = src.BBY1
,TGT.BBZ1 = src.BBZ1
, TGT.XMLCONTENTS = XMLTYPE(SRC.XMLCONTENTS)
when not matched then insert (
TGT.GUID
, TGT.BBX1
, TGT.BBY1
, TGT.BBZ1
, TGT.XMLCONTENTS )
VALUES (
SRC.GUID
, SRC.BBX1
, SRC.BBY1
, SRC.BBZ1
, XMLTYPE(SRC.XMLCONTENTS) );
, XMLTYPE(SRC.XMLCONTENTS) )
*
ERROR at line 17:
ORA-06502: PL/SQL: numeric or value error
ORA-06512: at line 1

Is there any other way to convert CLOB to XMLTYPE and merge?

Thanks in advance
Tom Kyte
November 24, 2008 - 6:00 pm UTC

I ran this in 9i, 10g, 11g - all ok, so, you'll need to give me something soup to nuts that doesn't work

ops$tkyte%ORA9IR2> create table t1 ( x int, y xmltype, cnt number );

Table created.

ops$tkyte%ORA9IR2> create table t2 ( x int, y clob );

Table created.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> insert into t2 values ( 1, '<doc><a>hello</a><b>world</b></doc>' );

1 row created.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> select * from t1;

no rows selected

ops$tkyte%ORA9IR2> merge into t1
  2  using (select x, xmltype(y) y_xml from t2) t2
  3  on (t1.x = t2.x )
  4  when matched then update set t1.y = t2.y_xml, cnt = cnt+1
  5  when not matched then insert (x, y, cnt ) values ( t2.x, t2.y_xml, 0 )
  6  /

1 row merged.

ops$tkyte%ORA9IR2> select * from t1;

         X Y                      CNT
---------- --------------- ----------
         1 <doc><a>hello</          0
           a><b>world</b><
           /doc>


ops$tkyte%ORA9IR2> merge into t1
  2  using (select x, xmltype(y) y_xml from t2) t2
  3  on (t1.x = t2.x )
  4  when matched then update set t1.y = t2.y_xml, cnt = cnt+1
  5  when not matched then insert (x, y) values ( t2.x, t2.y_xml )
  6  /

1 row merged.

ops$tkyte%ORA9IR2> select * from t1;

         X Y                      CNT
---------- --------------- ----------
         1 <doc><a>hello</          1
           a><b>world</b><
           /doc>


I found the culprit

Reddy K.S., November 26, 2008 - 3:55 pm UTC

Thanks Tom,
I found the culprit. The error message was due to one record that has XMLCONTENTS null.

insert/delete delsert in 9i

Misha, January 06, 2009 - 5:34 pm UTC

Oracle Version: 9i

Is there a way to do delete/insert (delsert) in one statement in 9i...sort of like this merge statement (upsert).
Will there be any advantages of doing a "delsert"...instead of two steps:

delete statement where there is match;
insert new records;

This will be done during a batch situation (single session)...so there won't be multiple oracle sessions trying to modify same tables.


Thanks
Tom Kyte
January 07, 2009 - 9:17 am UTC

define "where there is a match"

draw out the example for us.

if it is

delete from T1 where key in ( select key from T2 );
insert into T1 select * from T2;



that would be
merge into T1 using T2 on (t1.key = t2.key)
when matched then update ....
when not matched then insert ....


but we'd need a definition of "where there is a match" - if it is by join key, then no delete is necessary, you just update existing records and insert any that do not yet exist.



delsert

Misha, January 07, 2009 - 10:04 am UTC

Yes...it is exactly this:

delete from T1 where key in ( select key from T2 );
insert into T1 select * from T2;

but the joining "key" is not unique.


Tom Kyte
January 07, 2009 - 11:30 am UTC

then you cannot merge, you have to delete and then insert - in all releases. You cannot get a deterministic result if key is not unique.

re: delsert

Stew Ashton, January 07, 2009 - 11:55 am UTC


Tom, when you say "join key" I believe you imply "unique join key"?

If the join key is not unique in T1, the two pieces won't do the same thing.
create table T1(key number, C2 varchar2(256));
create table T2 as select * from T1;
insert into T1 values(1, 'Twice in T1, once in T2');
insert into T1 values(1, 'Twice in T1, once in T2');
insert into T2 values(1, 'Twice in T1, once in T2 UPDATED')
/
commit
/
merge into T1
using T2
on (T1.key = T2.key)
when matched then update set C2 = T2.C2
when not matched then insert values(T2.key, T2.C2)
/
select * from T1
/
-- KEY                    C2
-- ---------------------- -------------------------------
-- 1                      Twice in T1, once in T2 UPDATED
-- 1                      Twice in T1, once in T2 UPDATED

-- 2 rows selected

ROLLBACK
/
delete from T1 where key in ( select key from T2 )
/
-- 2 rows deleted
insert into T1 select * from T2
/
-- 1 rows inserted
select * from T1

-- KEY                    C2
-- ---------------------- -------------------------------
-- 1                      Twice in T1, once in T2 UDPATED
--
-- 1 rows selected

If the join key is not unique in T2, the MERGE won't work.

drop table T1;
create table T1(key number, C2 varchar2(256));
drop table T2;
create table T2 as select * from T1;
insert into T1 values(1, 'Twice in T2, once in T1');
insert into T2 values(1, 'Twice in T2, once in T1 a');
insert into T2 values(1, 'Twice in T2, once in T1 b');
merge into T1
using T2
on (T1.key = T2.key)
when matched then update set C2 = T2.C2
when not matched then insert values(T2.key, T2.C2)
/
-- Error starting at line 1 in command:
-- merge into T1
-- using T2
-- on (T1.key = T2.key)
-- when matched then update set C2 = T2.C2
-- when not matched then insert values(T2.key, T2.C2)
-- Error report:
-- SQL Error: ORA-30926: unable to get a stable set of rows in the source tables
-- 30926. 00000 -  "unable to get a stable set of rows in the source tables"
-- *Cause:    A stable set of rows could not be got because of large dml
--            activity or a non-deterministic where clause.
-- *Action:   Remove any non-deterministic where clauses and reissue the dml.

delete from T1 where key in ( select key from T2 )
/
-- 1 rows deleted
insert into T1 select * from T2
/
-- 2 rows inserted

So the definition of "where there is a match" needs to take key uniqueness into account, right?

MY BAD

Stew Ashton, January 07, 2009 - 11:57 am UTC


Rats, the time it took me to format my post right and you handled the issue in two lines.

My apologies !

delsert

Misha, January 07, 2009 - 12:03 pm UTC

Thanks Tom..and Thanks Stew for your example..

Reader, February 19, 2009 - 4:22 pm UTC

Tom,
I am running a merge statement which updates around 25,000 records on Oracle 10g RAC server with two nodes. I am getting the following error. What is causing this error?

ORA-12801: error signaled in parallel query server P000, ORA-01438: value larger than specified precision allowed for this column
Tom Kyte
February 21, 2009 - 8:31 pm UTC

[tkyte@localhost ~]$ oerr ora 1438
01438, 00000, "value larger than specified precision allowed for this column"
// *Cause: When inserting or updating records, a numeric value was entered
//         that exceeded the precision defined for the column.
// *Action: Enter a value that complies with the numeric column's precision,
//          or use the MODIFY option with the ALTER TABLE command to expand
//          the precision.


ops$tkyte%ORA10GR2> create table t ( x number(1) );

Table created.

ops$tkyte%ORA10GR2> insert into t select rownum from all_users;
insert into t select rownum from all_users
                            *
ERROR at line 1:
ORA-01438: value larger than specified precision allowed for this column


you tried to put something into a field and the field wasn't big enough to hold it. I have more than 9 users, when we insert rownum=10, that fails, 10 cannot be held in a number(1) field, only -9..9 can be.

Finding number of updates and inserts

Manish N, April 21, 2009 - 6:44 pm UTC

Is there a way to find the number updated and inserted instead of number MERGED by a MERGE statement.

Thanks a lot for this service.
Tom Kyte
April 22, 2009 - 1:57 pm UTC

no, there is not.

Workaround to get merge counts

Naveen, April 23, 2009 - 10:28 am UTC

Hi,
There is a work around to get the merge counts for inserts and updates. Let me share it here. I got this after searching.
create or replace package merge_counts as
   function get_counts (v_ins_upd_in number default 0) return number;
   function ins_counts return number;
   function upd_counts (v_rowcount number) return number;
   procedure reset_counts;
end merge_counts;
/
create or replace package body merge_counts as
    g_ins_cnt number := 0;
    g_upd_cnt number := 0;

    function get_counts (v_ins_upd_in number default 0) return number is
    begin
       If v_ins_upd_in = 0 then       -- inserting
          g_ins_cnt := g_ins_cnt + 1;
       elsif  v_ins_upd_in = 1 then   -- updating
           g_upd_cnt := g_upd_cnt + 1;
       else
           null;
       end if;
       return 0;
    end get_counts;

    function ins_counts return number is
    begin
       return g_ins_cnt;
    end ins_counts;

    function upd_counts (v_rowcount number) return number is
    begin
       If v_rowcount > g_ins_cnt then
          return (v_rowcount - g_ins_cnt);
       else
          return 0;
       End if;
    end upd_counts;

    procedure reset_counts is
    begin
       g_ins_cnt := 0;
       g_upd_cnt := 0;
    end reset_counts;
end merge_counts;

--USAGE:

declare
   v_counter pls_integer;
begin   
   merge into sdvtrans_audit tgt
   using (
    --your query
   ) source
   on(
        source.id = tgt.id
   )
   when matched then
      update set
      --updatable columns
   when not matched then
      insert(
         id,
         column1,
         column2,
         .....
      )
      values(
         case merge_counts.get_counts(0) when 0 then source.id end
        ,source.column1
        ,source.column2
        .........
      )
      ;

      v_counter := sql%rowcount;

      --'Display statistics';
      dbms_output.put_line('# Rows Merged: '||to_char(v_counter));
      dbms_output.put_line('      Inserts: '||to_char(merge_counts.ins_counts));
      dbms_output.put_line('      Updates: '||to_char(merge_counts.upd_counts(v_counter)));

      merge_counts.reset_counts;
end;
/


Tom Kyte
April 27, 2009 - 10:51 am UTC

ugh, these "solutions" designed to make the database as slow as humanly possible.

Well, I guess as long as you don't need the numbers to be entirely accurate - sure. You seem to presume we'll call your function some predicable number of times - very bad assumption on your part, you have no idea how we'll process your stuff internally.


merge returns the number of rows merged, period, there is no safe, reliable (not to mention PERFORMANT) way to get anything else.

merge 9i

Naveen, April 27, 2009 - 3:55 pm UTC

Hi Tom,
Greetings!

I totally agree to what you said. It will hit performance and slows down the process. However its a workaround, I too doesn't suggest to use this logic unless its mandatory to get insert and update counts


Naveen

Tom Kyte
April 27, 2009 - 4:40 pm UTC

if it were mandatory and you want the right answer every time (in your lifetime as well), you have to two step it.

update and then insert


Cant I merge over DB link in 9i..

Dinker Joshi, December 07, 2009 - 4:57 pm UTC

I am doing this.

CREATE TABLE metric_last_extstat
(metrictypeid NUMBER,
studyid NUMBER,
last_end_dt DATE,
systemid NUMBER)
/
MERGE INTO metric_last_extstat a
USING (SELECT 300 metrictypeid, 17 studyid,
sysdate last_end_dt, 8 systemid
FROM DUAL) b
ON ( a.studyid = b.studyid
AND a.metrictypeid = b.metrictypeid
AND a.systemid = b.systemid)
WHEN MATCHED THEN
UPDATE
SET last_end_dt = b.last_end_dt
WHEN NOT MATCHED THEN
INSERT (metrictypeid, systemid, studyid, last_end_dt)
VALUES (b.metrictypeid, b.systemid, b.studyid, b.last_end_dt);
/

if this table is in local DB, it works all fine. But if I drop this table and create it in remote database (MHDV1 in my case), create a dblink and synonym in local DB for this table.

CREATE DATABASE LINK MHDV1.WORLD
CONNECT TO monitoruid IDENTIFIED BY temp1234
USING 'MHDV1'
/
CREATE SYNONYM metric_last_extstat
FOR metric_last_extstat@mhdv1.world
/

and issue same Mregs statement it will fail with fillowing error.

5:22:12 PM ORA-01008: not all variables bound
5:22:12 PM ORA-02063: preceding line from MHDV1

Local DB is 9i, MHDV1 is 10g. If I take 10g local DB, it works fine.
Can you please help me with this..
Tom Kyte
December 10, 2009 - 8:05 am UTC

I cannot reproduce, please contact support


ops$tkyte%ORA9IR2> CREATE TABLE metric_last_extstat
  2      (metrictypeid                   NUMBER,
  3      studyid                        NUMBER,
  4      last_end_dt                    DATE,
  5      systemid                       NUMBER)
  6  /

Table created.

ops$tkyte%ORA9IR2> MERGE INTO metric_last_extstat a
  2     USING (SELECT 300 metrictypeid, 17 studyid,
  3                   sysdate last_end_dt, 8 systemid
  4              FROM DUAL) b
  5     ON (    a.studyid = b.studyid
  6         AND a.metrictypeid = b.metrictypeid
  7         AND a.systemid = b.systemid)
  8     WHEN MATCHED THEN
  9        UPDATE
 10           SET last_end_dt = b.last_end_dt
 11     WHEN NOT MATCHED THEN
 12        INSERT (metrictypeid, systemid, studyid, last_end_dt)
 13        VALUES (b.metrictypeid, b.systemid, b.studyid, b.last_end_dt);

1 row merged.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> create database link ora10gr2 connect to ops$tkyte identified by foobar using 'ora10gr2';

Database link created.

ops$tkyte%ORA9IR2> select * from dual@ora10gr2;

D
-
X

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> create or replace synonym metric_last_extstat_remote for metric_last_extstat@ora10gr2;

Synonym created.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> MERGE INTO metric_last_extstat_REMOTE a
  2     USING (SELECT 300 metrictypeid, 17 studyid,
  3                   sysdate last_end_dt, 8 systemid
  4              FROM DUAL) b
  5     ON (    a.studyid = b.studyid
  6         AND a.metrictypeid = b.metrictypeid
  7         AND a.systemid = b.systemid)
  8     WHEN MATCHED THEN
  9        UPDATE
 10           SET last_end_dt = b.last_end_dt
 11     WHEN NOT MATCHED THEN
 12        INSERT (metrictypeid, systemid, studyid, last_end_dt)
 13        VALUES (b.metrictypeid, b.systemid, b.studyid, b.last_end_dt);

1 row merged.

ops$tkyte%ORA9IR2>
ops$tkyte%ORA9IR2> select * from v$version;

BANNER
----------------------------------------------------------------
Oracle9i Enterprise Edition Release 9.2.0.8.0 - Production
PL/SQL Release 9.2.0.8.0 - Production
CORE    9.2.0.8.0       Production
TNS for Linux: Version 9.2.0.8.0 - Production
NLSRTL Version 9.2.0.8.0 - Production

ops$tkyte%ORA9IR2> select * from v$version@ora10gr2;

BANNER
----------------------------------------------------------------
Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - Prod
PL/SQL Release 10.2.0.4.0 - Production
CORE    10.2.0.4.0      Production
TNS for Linux: Version 10.2.0.4.0 - Production
NLSRTL Version 10.2.0.4.0 - Production

Merge limitation, maximum number of columns is 1000.

Tania, April 15, 2010 - 8:06 am UTC

Hi Tom,

We are in an environment where we get source files from vendors that contain over 5000 columns. In order for us to load these files we have to create 5 destination tables because of the 1000 column limitation. The problem we are experiencing at the moment is to port data from a staging/temp table to the fact tables. The merge statement calculates the actual columns on stage + destination tables. Thus we are now limited to a 1000 columns combined. You can just imagine what type of environment this is creating. Now we have 10 staging tables with 100 columns each for one destination table that contain 900 columns. Obviously we want to use the merge statement because of the performance we get on it. We tried to split the merge to only do batches of columns but this did not work because the limitation is on the actual tables and not on the columns specified in the merge statement. Can you please provide us with some other options to use.

Tom Kyte
April 15, 2010 - 8:59 am UTC

I'd need a concrete example to work with to demonstrate the issue you are having.

Feel free to use ... when creating the example for us


create table t (
c1 int primary key,
c2 int,
....
c1000 );



for example. keep it small, but keep it complete enough to reproduce the issue you need to work around.

Merge Statement - Limitation of 1000 columns

Tania, April 16, 2010 - 1:55 am UTC

Example:

CREATE TABLE DEST.N3G_SERVICELEVEL_DLY (
ORIG_CELL_SEQ NUMBER NOT NULL,
DAY DATE NOT NULL,
CELL_SEQ NUMBER NOT NULL,
GRANULARITY_PERIOD NUMBER NOT NULL,
C1 NUMBER,
.....
C950 NUMBER)
PARTITION BY RANGE (DAY)
(
PARTITION Y2010W05 VALUES LESS THAN (TO_DATE(' 2010-02-01 00:00:00', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')) TABLESPACE DEST_DATA PCTFREE 1
)
PARALLEL (DEGREE 1)
/
CREATE INDEX DEST.N3G_SERVICELEVEL_DLY_PK ON DEST.N3G_SERVICELEVEL_DLY (ORIG_CELL_SEQ, DAY) PCTFREE 5 TABLESPACE DEST_IDX NOLOGGING;
ALTER TABLE DEST.N3G_SERVICELEVEL_DLY ADD CONSTRAINT N3G_SERVICELEVEL_DLY_PK PRIMARY KEY (ORIG_CELL_SEQ, DAY) USING INDEX PCTFREE 5 TABLESPACE DEST_IDX;

CREATE GLOBAL TEMPORARY TABLE N3G_SERVICELEVEL_DLY_TMP
ON COMMIT DELETE ROWS
AS
SELECT
ORIG_CELL_SEQ,
DAY,
CELL_SEQ,
GRANULARITY_PERIOD,
C1,
....
C950
FROM DEST.N3G_SERVICELEVEL_DLY

--Dump from source file.
INSERT INTO N3G_SERVICELEVEL_DLY_TMP VALUES (63,'2/15/2010',957,86400,1,....,950);

MERGE INTO DEST.N3G_SERVICELEVEL_DLY d
USING N3G_SERVICELEVEL_DLY_TMP s
ON (s.orig_cell_seq = d.orig_cell_seq AND s.day = d.day)
WHEN MATCHED THEN
UPDATE
SET d.cell_seq = s.cell_seq,
d.granularity_period = s.granularity_period,
d.c1 = s.c1)
WHEN NOT MATCHED THEN
INSERT (d.orig_cell_seq, d.day, d.cell_seq,
d.granularity_period, d.c1)
VALUES (s.orig_cell_seq, s.day, s.cell_seq,
s.granularity_period, s.c1)

Results in error: ORA-01792: maximum number of columns in a table or view is 1000.
Tom Kyte
April 16, 2010 - 10:39 am UTC

ops$tkyte%ORA11GR1> MERGE INTO N3G_SERVICELEVEL_DLY d
  2     USING (select orig_cell_seq, day, cell_seq, granularity_period, c1 from N3G_SERVICELEVEL_DLY_TMP ) s
  3     ON (s.orig_cell_seq = d.orig_cell_seq AND s.day = d.day)
  4     WHEN MATCHED THEN
  5        UPDATE
  6           SET d.cell_seq = s.cell_seq,
  7               d.granularity_period = s.granularity_period,
  8               d.c1 = s.c1
  9     WHEN NOT MATCHED THEN
 10        INSERT (d.orig_cell_seq, d.day, d.cell_seq,
 11                d.granularity_period, d.c1)
 12        VALUES (s.orig_cell_seq, s.day, s.cell_seq,
 13                s.granularity_period, s.c1)
 14  /


does that help?

Merge limitation on 1000 columns

Tania, April 19, 2010 - 4:10 am UTC

Hi Tom,

Thank you so much. That did help. We would still have to split the merge into small parts because of the nr columns in destination table. For a destination table of 960 columns we would have to do a merge 24 times (40 columns at a time). Would you recommend this or should we take another route? Think we would have to get a balance, maybe not to exceed 800 columns (thus merging 4 times).
Tom Kyte
April 19, 2010 - 8:57 am UTC

probably time to think about an update of existing records followed by an insert of new records instead.


update the join
insert the minus
commit;


Great Article!

Faisal, November 11, 2010 - 4:47 am UTC

Hi Tom,

I have been a long time reader (first time poster) of your "asktom.oracle.com" and it has helped me every single time :)

This time, I have a situation, it might have been answered already but I could not find it.

I am using 11g and trying to use MERGE to insert / update records into a table.

This table contains calculated asset values for each day, account. So INSERT is no problem, simply if NOT MATCHED then I can insert new set of 30k each day.

But there are updates on the rates and security prices every 15 min, I have to update the same set as well. There might be new accounts for the same date, that will get inserted, but there might be removals of accounts, how can I do a cleanup of those leftovers after INSERT/UPDATE merge has completed?

Is there no other way other than doing a DELETE all the day's transactions first then do an INSERT /*+ APPEND */ ? This eats up a lot of space and my production database has already consumed 10GB in 2 weeks of this. Since it runs every 15 minutes and I certainly cannot do a TRUNCATE and INSERT because I have to retain the previous days calculated values! Also this table is used by many online web services and interfaces to be queried by the public so truncate is not an option.

Thanks a lot.
Faisal.
Tom Kyte
November 11, 2010 - 2:43 pm UTC

... but there might be removals of accounts, how can I do a
cleanup of those leftovers after INSERT/UPDATE merge has completed? ...

you forgot to tell us how to identify these guys. merge can insert, update, delete

give a tiny example (create tables, inserts) and tell us what the logic for inserting, updating AND deleting would be.

MERGE

A reader, November 27, 2010 - 4:41 pm UTC

Hi Tom,
We are on 10g R1. The merge command is failing with the error - ORA-30926. On checking, I found that

30926, 00000, "unable to get a stable set of rows in the source tables"
// *Cause: A stable set of rows could not be got because of large dml
// activity or a non-deterministic where clause.
// *Action: Remove any non-deterministic where clauses and reissue the dml.

The statement - A stable set of rows could not be got because of large dml
does not help much, quite broad...
In typical env., data grows, with this stmnt., don't you think one day ... job is going to fail? Your suggestion will really appreciated.
Regards,

Tom Kyte
November 28, 2010 - 3:12 pm UTC

This "warning", I do not consider it an error, is because a row in "TARGET" has been modified by "SOURCE" more than once and changed its values more than once (e.g. the merge is not deterministic)

It doesn't matter the size of the tables (I can do it with a total of three rows in src and one in tgt)

ops$tkyte%ORA11GR2> create table src( x int, y int );

Table created.

ops$tkyte%ORA11GR2> create table tgt( x int primary key, y int );

Table created.

ops$tkyte%ORA11GR2> insert into tgt values ( 1, 1 );

1 row created.

ops$tkyte%ORA11GR2> insert into src values ( 1, 2 );

1 row created.

ops$tkyte%ORA11GR2> insert into src values ( 1, 3 );

1 row created.

ops$tkyte%ORA11GR2> 
ops$tkyte%ORA11GR2> 
ops$tkyte%ORA11GR2> merge into tgt using src on (src.x = tgt.x)
  2  when matched then update set tgt.y = src.y
  3  when not matched then insert(x,y) values (src.x,src.y);
merge into tgt using src on (src.x = tgt.x)
                     *
ERROR at line 1:
ORA-30926: unable to get a stable set of rows in the source tables



If you hit this, it means your logic is fundamentally flawed.

ORA Error Codes

Rajeshwaran, Jeyabal, December 29, 2010 - 7:59 am UTC

Tom:

Just before a couple of post you provided the below snippet.

[tkyte@localhost ~]$ oerr ora 1438
01438, 00000, "value larger than specified precision allowed for this column"
// *Cause: When inserting or updating records, a numeric value was entered
//         that exceeded the precision defined for the column.
// *Action: Enter a value that complies with the numeric column's precision,
//          or use the MODIFY option with the ALTER TABLE command to expand
//          the precision.


When I tried this, I am getting this error.

rajesh@10GR2>
rajesh@10GR2>
rajesh@10GR2>
rajesh@10GR2> $ oerr ora 1438
'oerr' is not recognized as an internal or external command,
operable program or batch file.

rajesh@10GR2>
rajesh@10GR2> oerr ora 1438
SP2-0734: unknown command beginning "oerr ora 1..." - rest of line ignored.
rajesh@10GR2>
rajesh@10GR2> host
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

D:\>$ oerr ora 1438
'$' is not recognized as an internal or external command,
operable program or batch file.

D:\>oerr ora 1438
'oerr' is not recognized as an internal or external command,
operable program or batch file.

D:\>


Is this $ oerr ora something a custom defined function? If yes can you please share that ?
Tom Kyte
December 30, 2010 - 1:17 pm UTC

you are running on an inferior operating system.

oerr is a unix only command - since scripting actually works on that platform :)

You'll find google to be useful

http://www.google.com/search?q=ora+1438

The first hit(s) are usually links to the in depth message text.

Commit inside merge

A Reader, January 14, 2011 - 4:15 am UTC

Can we have a commit statement when using merge in stored procedure, so that we could get some data to be committed at different intervals in case of huge amount of data.
Tom Kyte
January 14, 2011 - 12:06 pm UTC

Duplicate Records

Abhisek, March 23, 2011 - 7:16 am UTC

Hi Tom,

Can you please sugegst a way to load 10 huge tables (~5 million rows) from one table to another table considering there are duplicates in source table

Just for example:

<code>
create table t1(a number primary key, b number);

create table t2(a number, b number);


insert into t2 values(1,22);
insert into t2 values(2,33);
insert into t2 values(2,33);
insert into t2 values(3,44);

merge into t1
    using t2 ON (T1.A=T2.A)
    WHEN NOT MATCHED THEN
        INSERT(a,b) values(T2.A,T2.B);



This will throw an exception that it found a duplicate row. Now if I go by Cursors I can still load the data by discarding the duplicate row, even if it is slow. I think, since it is set operation so it rollbacks everything.

So could you please suggest.
</code>
Tom Kyte
March 23, 2011 - 8:41 am UTC

5 million rows, tiny. 5 billion rows - maybe huge. 5 million - almost certainly tiny (it depends on the rowsize of course, if your rows are 100mb each - then maybe it is huge, but 5,000,000 is pretty small these days)

insert into t1(a,b) select distinct a,b from t2;

would be one way.

insert with DML error logging would be another.

insert into t1(a,b) select a,b from t2 log errors;


NO CODE, no procedural code, should be written for this - if there is, you've done it wrong.


Thanks

A reader, March 23, 2011 - 11:05 am UTC

Hi Tom,

My 10 Tables are each for around 5 million records.. which is smaller by your expertise but huge for me :-(

Regarding the DML statement

insert into t1(a,b) select a,b from t2 log errors;

Do you mean
BEGIN
insert into t1(a,b) select a,b from t2;
EXCEPTION
WHEN DUP_VAL_ON_INDEX THEN
dbms_output.put_line('DUP_VAL_ON_INDEX exception.');
END;

But I found it will not insert any row and will rollback the changes if it encounters duplicates.

Is it method the same fast as the merge statement ones are?


Tom Kyte
March 23, 2011 - 11:16 am UTC

which is smaller by your expertise but huge for me :-(

not by expertise, but by experience.

And unless you are running on 15 year old hardware, it is small for you too.


No, I did not mean PROCEDURAL CODE. I meant to use dml error logging via the log errors clause. Available in the database since 10g.

http://asktom.oracle.com/Misc/how-cool-is-this.html

Thanks

A reader, March 23, 2011 - 11:35 am UTC

Thanks Tom

Yes that was meant to be your experience :)

But bad luck to me as the package dbms_errlog.create_error_log

is not installed so may be I have to take anyother way around.. But this was rally a cool feature I was missing.

Tom Kyte
March 23, 2011 - 1:20 pm UTC

just create the error logging table yourself.

you don't need that package.

Thanks

Abhisek, March 24, 2011 - 3:11 am UTC

Hi Tom,

I tried to create table manually and got an error.

create table oldtable(field1 number);

create table newtable(field1 number primary key);

insert into oldtable values(1);

insert into oldtable values(2);

insert into oldtable values(3);

insert into oldtable values(3);

commit;

select * from oldtable;



Now my Error logging table create script is:


create table error_log_dmlel_2
 (ora_err_number$ number, 
  ora_err_mesg$ varchar2(2000),
  ora_err_rowid$ rowid,   
  ora_err_optyp$ varchar2(2),
  ora_err_tag$ varchar2(2000) );


Whenever I am trying to do

insert into newtable select * from oldtable; --Normal Error duplicate

insert into newtable
select * from oldtable
log errors into error_log_dmlel_2
reject limit UNLIMITED;



I get the error as

Error at line 1
ORA-06550: line 7, column 5:
PL/SQL: ORA-00933: SQL command not properly ended
ORA-06550: line 5, column 5:
PL/SQL: SQL Statement ignored

Please suggest
Tom Kyte
March 24, 2011 - 8:16 am UTC

you are using 10g R2 or a above right?

Sorry

Abhisek, March 24, 2011 - 5:19 pm UTC

Hi Tom,

My friend had Oracle 10GR1.. So it works on 10G R2

Apologies for not checking the version.

But how do we handle it 10G r1 then? I think it should be with the distinct operation but my only concern with that is if an unique index is on emp_id and we have two emp_name in the first table( a case of bad data).. How do we handle in MERGE statement then?
Tom Kyte
March 25, 2011 - 10:46 am UTC

You would NOT use merge, you are trying to insert - use insert into t select distinct from t2;


Data

A reader, March 25, 2011 - 11:16 am UTC

Hi Tom,


I meant.. lets Say..

CREATE TABLE EMPLOYEE(emp_id number primary key, ename varchar2(20));

INSERT INTO EMP VALUES(1, 'ABC');

INSERT INTO EMP VALUES(2, 'XYZ');

INSERT INTO EMP VALUES(2, 'DEF');

It is a case of bad data in the file.

Now if I try to insert it will throw me an error message.. I can take the first row of the table through the Analytics : OVER Partition by clause

But do we have any other options in 10G R1? I hope i am clear

Tom Kyte
March 25, 2011 - 11:21 am UTC

answer me this first:

if you have that data, how do you know which record to load?????

Ok..Logic

A reader, March 25, 2011 - 1:45 pm UTC

Hi Tom,

I simply want to avoid procedural attempt to load data..

BEGIN
for i in (select emp_id, ename from employees)
LOOP
BEGIN
 insert into employees_2(emp_id, ename) values(i.emp_id, i.emp_name);
 EXCEPTION
  WHEN DUP_VAL_ON_INDEX THEN
     dbms_output.put_line('DUP_VAL_ON_INDEX exception.');
 END;

END LOOP;

END;
 


This will load data even if I get an error.. I want a workaround in a single statement operation.
Tom Kyte
March 25, 2011 - 4:53 pm UTC

but that is non-deterministic, you don't really care what row gets loaded - what is the point?

have you really thought this through? smells fishy to me...

Data

A reader, March 28, 2011 - 4:42 am UTC

Hi Tom,

I see your point. But if, the data for the table comes from a different data destination, lets say any external department and due to their wrong query structure it is very much possible that the data file, delivered from them may contain duplicate rows with some attribute different keeping the primary key same.

I agree that they should modify their query or we modify the table structure so that we capture the correct primary key.

But if in case the delivery of bad data happens because of their one row which was a duplicate by mistake at the end of of 50 million records, my whole transaction gets rolled back.

Hope I was able to make the things a little bit clear
Tom Kyte
March 29, 2011 - 3:03 am UTC

But if, the data for the table comes from a different data
destination, lets say any external department and due to their wrong query
structure it is very much possible that the data file, delivered from them may
contain duplicate rows with some attribute different keeping the primary key
same.


then you have what we call "garbage" and "garbage in = garbage out"

either

a) get to 10gr2 and above and use log errors

b)

insert into your_table
select ....
from (select c1, c2, c3, ... cn,
row_number() over (partition by your_unique_key order by 1) rn
from other_table_with_garbage_in_it)
where rn = 1;

and for the error report:


select ....
from (select c1, c2, c3, ... cn,
row_number() over (partition by your_unique_key order by 1) rn
from other_table_with_garbage_in_it)
where rn <> 1;

Agreed.. Thanks for support

Abhisek, March 29, 2011 - 7:34 am UTC

hi Tom,

I agree with your point. As I said, I had only this solution for 10G R1 to select the first row.

http://asktom.oracle.com/pls/apex/f?p=100:11:0::::P11_QUESTION_ID:5318183934935#3241203200346393007

I hope it works the same as our DML Log error functionality's end result would have looked like.. Insert the first row and if you get a duplicate then log error message and move to next record.

The same happens here to select the first row only. The only difference we had to do a little bit of extra processing.

Thanks for the support.
Tom Kyte
March 30, 2011 - 1:13 am UTC

you could use a multi-table insert as well - to put the rn=1 into the good table and rn <> 1 into the 'bad'

http://docs.oracle.com/docs/cd/B14117_01/server.101/b10736/transform.htm#sthref717

code is not working

Venkata, April 14, 2011 - 1:07 am UTC

Reviewer: Duke Ganote from the eastern hills of Anderson Township, Hamilton County, Ohio; part of greater Cincinnati, Ohio USA


Maybe this? Then wrap in a MERGE...

select * from (
select s1.*
, max(s1.row_number) over
(partition by primary_key) r#max
from source s1
) where r#max = row_number
/



Followup December 3, 2005 - 10am Central time zone:

that is another way to reduce the work - yes.


Hi,

this was a piece of code reviewed by you, Tom, i have copied and tried to execute the code as it is. i was getting ORA-00923 error. modified it in many ways, couldn't find what was the error, i feel there is some problem with "row_number" clause.


Tom Kyte
April 14, 2011 - 9:47 am UTC

you'll need to be more clear here. You are commenting on a followup to a followup of a followup.

Tell me what YOU are trying to do, giving the full example (table creates and all) and we can work from there.


here is the code for you to help me

Venkata, April 15, 2011 - 12:45 am UTC

create table t(a1 varchar2(10), a2 number(10));

insert into t(a1,a2) values(to_char(userenv('sessionid')),to_number(to_char(sysdate,'ss')) );
insert into t(a1,a2) values(to_char(userenv('sessionid')),to_number(to_char(sysdate,'ss')) );
insert into t(a1,a2) values(to_char(userenv('sessionid')),to_number(to_char(sysdate,'ss')) );
insert into t(a1,a2) values(to_char(userenv('sessionid')),to_number(to_char(sysdate,'ss')) );

commit;

create table t1 as select * from t where a2 < 34;

now t has 4 rows and t1 has only 1 row.

insert into t1(a1,a2) values('X',to_number(to_char(sysdate,'ss')))
insert into t1(a1,a2) values('Y',to_number(to_char(sysdate,'ss')))
insert into t1(a1,a2) values('Z',to_number(to_char(sysdate,'ss')))

commit;

now what i want is..

update t1 with values of t if t1.a1 = t.a1
else
insert into t1 new values of t


i am using merge(i am not showing the code for merge because i don't want to use merge and you will try to debug merge statement, which of no help for me) which is throwing a unique key constraint error. i want to know if there is another way other than using merge.
thanks for your time my dear Tom
Tom Kyte
April 15, 2011 - 10:04 am UTC

I do not see any way this could throw a unique constraint violation.

And further, I will be coding a merge because that is what your current logic DICTATES I should be doing.


the example you have is broken. I don't know how you can say "t1 has only 1 row" unless you just happened to have run this at 33 seconds past the minute and the clock ticked after the first insert, before the second.


Your update would be ambiguous unless T1.A1 is defined as unique - is it?

please answer to this

Venkata, April 18, 2011 - 2:04 am UTC

Hi Tom, hope you had a good weekend :).

i want to know alternatives for merge statement, now please don't dig into merge. am running short of time for my deliverable.
Tom Kyte
April 18, 2011 - 10:36 am UTC

there are none. A merge does something that cannot be done by any other single sql statement.

Unless you tell me what you are trying to do specifically, clearly and in detail - please stop asking such 'generic' questions. I cannot help you if I have no clue what you are trying to do.

So, either open up and be clear with us, or try somewhere else.

So far, everything you've shown me just leads to more questions - I've no clue what your goal is (and stating "my goal is to not use a merge" IS NOT A GOAL, it won't get you anywhere with anyone) and until we do - there is nothing to really discuss.

here it is

Venkata, May 03, 2011 - 7:32 am UTC

Hi Tom,

lets say, i have 2 tables

create table x as select 1 x,a.* from dual a;
/
alter table x add primary key(x);
/
create table y as select 2 y,a.* from dual a;
/
alter table y add primary key(y);
/
insert into x select 2,'Y' from dual;
/
insert into x select 3,'Z' from dual;
/
insert into y select 4,'A' from dual;
/
insert into y select 5,'B' from dual;
/

commit;

create procedure mergep is
begin
merge into x using (select * from y) r
on(x.x = r.y)
when matched then
update x set x.dummy = r.dummy
when not matched then
insert into x(x,dummy) values (r.y,r.dummy);
exception
when others then
dmbs_output.put_line(sqlerrm||' '||sqlcode);
end;
/

now if i am calling this procudue from 2 different sessions
i am getting unique key constraint error in the session where i have not issued a commit.

like this

session 1

exec mergep;

now merge is still running in session 1, now i will execute the procedure in session 2

session 2
exec mergep;

in session 1 i will issue a commit since the procedure has completed.now in session i am getting unique key voilated error.

hope i am able to convey the error i am facing.
thanks for your help.

Tom Kyte
May 04, 2011 - 12:34 pm UTC

exception 
when others then
dmbs_output.put_line(sqlerrm||' '||sqlcode);
end;


http://asktom.oracle.com/pls/asktom/asktom.search?p_string=%22i+hate+your+code%22


I hate your code even more since your example is fraught with errors.

If you use ";", you don't use "/"

If you type dmbs_output, you get a compile error.

If you type update x in a merge, you get an invalid statement.

If you type insert into x in a merge, you get an invalid statement




and when I fix all of that junk, I find....


I cannot reproduce - not in 10.2.0.4, not in 11.2.0.2



ops$tkyte%ORA10GR2> create table x as select 1 x,a.* from dual a;

Table created.

ops$tkyte%ORA10GR2> alter table x add primary key(x);

Table altered.

ops$tkyte%ORA10GR2> create table y as select 2 y,a.* from dual a;

Table created.

ops$tkyte%ORA10GR2> alter table y add primary key(y);

Table altered.

ops$tkyte%ORA10GR2> insert into x select 2,'Y' from dual;

1 row created.

ops$tkyte%ORA10GR2> insert into x select 3,'Z' from dual;

1 row created.

ops$tkyte%ORA10GR2> insert into y select 4,'A' from dual;

1 row created.

ops$tkyte%ORA10GR2> insert into y select 5,'B' from dual;

1 row created.

ops$tkyte%ORA10GR2> create or replace procedure mergep
  2  as
  3  begin
  4  merge into x using (select * from y) r
  5  on(x.x = r.y)
  6  when matched then
  7  update set x.dummy = r.dummy
  8  when not matched then
  9  insert (x,dummy) values (r.y,r.dummy);
 10  end;
 11  /

Procedure created.

ops$tkyte%ORA10GR2> exec mergep

PL/SQL procedure successfully completed.

<b>I did another call to mergep here in another session, it blocked</b>

ops$tkyte%ORA10GR2>
ops$tkyte%ORA10GR2> commit;

Commit complete.

<b>and it succeeded here</b>





thank you

Venkata, May 05, 2011 - 6:27 am UTC

i tried to convince you by giving some examples, but invain. so i typed all the code in this editor, so that i can get an answer for the problem i am facing. when others i used just to know the error code when i set a transaction to serializable and it fails. i too know its a very bad practise to use when others. i feel when others tells us what is the error that is thrown when not handled.

thanks for marking my code as "i hate....." :(
Tom Kyte
May 06, 2011 - 9:41 am UTC

why didn't you ACTUALLY TEST WHAT YOU TYPED IN????

It seems like that would be a no-brainer, something that you would "of course do". I don't understand why people post something they never ever tested?????? It just doesn't even begin to make sense.


i feel when others tells us what is the error that is thrown when not
handled.


it doesn't do that, it HIDES the error. If you just let the error propagate up - it would be BLINDINGLY obvious "there is an error, here is what it is and here is where it happened"


Remove it, run your code, and let it get an error. You'll see a lot more information - in a form that is just not ignorable.

forgot to add this

venkata, May 05, 2011 - 7:44 am UTC

tom,

instead of wrapping the merge statement inside a procedure, try executing the merge statement in two different sessions, i bet, you will get Unique key error. want to know how to avoid this error. i too wrapped the merge statement inside a procedure, but i didn't get any error. :)
Tom Kyte
May 06, 2011 - 9:54 am UTC

Why in the world, i mean really - WHY IN THE WORLD would you post a test case that doesn't even demonstrate that which you are trying to demonstrate? That you have a question about? Ugh, the second person in like 30 seconds that has done this to me today. Oh wait, it is you again - compounding the issue. Not only did your example not compile, not run, was fraught with errors - but it doesn't even DEMONSTRATE THE ISSUE YOU ARE TRYING TO DEMONSTRATE. Nice.


drop table x;
drop table y;

set echo on

create table x as select 1 x,a.* from dual a;
alter table x add primary key(x);
create table y as select 2 y,a.* from dual a;
alter table y add primary key(y);
insert into x select 2,'Y' from dual;
insert into x select 3,'Z' from dual;
insert into y select 4,'A' from dual;
insert into y select 5,'B' from dual;
commit;

create or replace procedure merge_demo
as
begin
merge into x using (select * from y) r
on(x.x = r.y)
when matched then
update set x.dummy = r.dummy
when not matched then
insert (x,dummy) values (r.y,r.dummy);
end;
/

merge into x using (select * from y) r
on(x.x = r.y)
when matched then
update set x.dummy = r.dummy
when not matched then
insert (x,dummy) values (r.y,r.dummy);

set echo off
prompt Run this merge in another session
prompt merge into x using (select * from y) r
prompt on(x.x = r.y)
prompt when matched then
prompt update set x.dummy = r.dummy
prompt when not matched then
prompt insert (x,dummy) values (r.y,r.dummy);;
prompt rollback;;
pause
set echo on
commit;



I ran that, no issue. Now, you may feel free to modify that in whatever fashion you see fit to reproduce the issue.

Just make sure that whatever you post - you actually TEST and run in your system first.

Update and merge

lalu, June 07, 2011 - 1:29 pm UTC

Hi,

I have a table as below

id f1 f2 f3 newid
1 a b c
1 a b c
1 a x y
2 a b c

I want the output as below:

id f1 f2 f3 newid
1 a b c 1
1 a b c 1
1 a x y 2
2 a b c 3

For duplicate rows in the table I want same newid number.
Else it will be generated using some sequence.

Thanks.

Tom Kyte
June 07, 2011 - 2:50 pm UTC

no creates
no inserts
no look

don't know if your problem can be solved, didn't really read it. Just saw that you wanted some SQL but didn't do the ground work to allow me to try it out.

Update and merge

lalu, June 07, 2011 - 3:11 pm UTC

Hi Tom,

I do not have any unique column in the table.
So I can't use merge statement.

I want to generate same(newid) value for each duplicate row.
And other rows will have different newid.

Thanks.
Tom Kyte
June 07, 2011 - 4:37 pm UTC

merge doesn't require a unique key at all.

again, I'll say it:

no create
no inserts
no look

please don't bother posting again unless you also supply an example to work with.

Oracle Certification

Charlie, June 23, 2011 - 4:16 pm UTC

Thanks for all your work on this site.
I'm preparing for the Oracle exams.

I've been looking for something solid to review objectives with. Seems to be a good idea to use the Ask Tom web site to review test objectives.

Lots of insight and practical stuff here.

Very many thanks,
Charlie

A reader, October 02, 2011 - 10:35 am UTC

hi...

my merge statement taking hours to complete please suggest how to tune it. actually the merge does an update only i guessed merge will be faster. also i did explain plan on the merge query. it says 5 secs but when i run it it takes hours. please suggest me some tuning

MERGE INTO CYCLETIME_STAGING CS USING
(SELECT CYCLE.PK_CLAIM_NUMBER ,
CYCLE.COMPANY_CODE ,
CYCLE.ASGMTSNT_DATETIME AS ASGMTSNT_DATETIME,
CURRENT_TIMESTAMP
FROM CYCLETIME_STAGING CYCLE,
CES_DW_STAGING_pro CES
WHERE CES.PK_CLAIM_NUMBER = trim(CYCLE.PK_CLAIM_NUMBER)
AND CES.COMPANY_CODE = trim(CYCLE.COMPANY_CODE)
AND CYCLE.LASTESTASGMTSNT_DATETIME > CES.ESTSTART_DATETIME
AND CYCLE.ASGMTSNT_DATETIME < CES.ESTSTART_DATETIME
AND CES.FILE_SOURCE_CODE = 'PW'
) TEMP ON ( TEMP.PK_CLAIM_NUMBER = CS.PK_CLAIM_NUMBER )
WHEN MATCHED THEN
UPDATE SET
CS.LASTESTASGMTSNT_DATETIME= TEMP.ASGMTSNT_DATETIME;


explain plan:

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------------------------
| 0 | MERGE STATEMENT | | 1 | 17 | 257 (4)| 00:00:05 |
| 1 | MERGE | CYCLETIME_STAGING | | | | |
| 2 | VIEW | | | | | |
| 3 | NESTED LOOPS | | | | | |
| 4 | NESTED LOOPS | | 1 | 140 | 257 (4)| 00:00:05 |
| 5 | MERGE JOIN CARTESIAN | | 1 | 93 | 255 (4)| 00:00:05 |
|* 6 | TABLE ACCESS FULL | CES_DW_STAGING_PRO | 1 | 30 | 2 (0)| 00:00:01 |
| 7 | BUFFER SORT | | 192K| 11M| 253 (4)| 00:00:05 |
| 8 | TABLE ACCESS FULL | CYCLETIME_STAGING | 192K| 11M| 253 (4)| 00:00:05 |
|* 9 | INDEX RANGE SCAN | PK_CYCLETIME_STAGING | 1 | | 1 (0)| 00:00:01 |
|* 10 | TABLE ACCESS BY INDEX ROWID| CYCLETIME_STAGING | 1 | 47 | 2 (0)| 00:00:01 |



Tom Kyte
October 03, 2011 - 9:50 am UTC

are the estimated cardinalities near correct - do you expect to find 192k rows in cycletime_staging. Do you expect to merge 1 row?


I'm suspecting this:


| 5 | MERGE JOIN CARTESIAN | | 1 | 93 |


I have a feeling the cardinality estimate is way off, and I can see why:

 WHERE CES.PK_CLAIM_NUMBER        = trim(CYCLE.PK_CLAIM_NUMBER)
AND CES.COMPANY_CODE               = trim(CYCLE.COMPANY_CODE)
AND CYCLE.LASTESTASGMTSNT_DATETIME > CES.ESTSTART_DATETIME
AND CYCLE.ASGMTSNT_DATETIME        < CES.ESTSTART_DATETIME
AND CES.FILE_SOURCE_CODE           = 'PW'



that does just about everything wrong as you can for an optimizer to try and come up with a realistic guess as to row counts.

So, tell me, are the estimated cardinalities anywhere near realistic. Do we have representative stats on CES. Do you really need those trim() calls (which throw off estimates.


I would look at that part of the merge and see how long just doing that sql is. It is the likely culprit

Rownum usage for key generation in MERGE

Achyuth, January 05, 2012 - 7:09 am UTC

merge INTO tgt t USING
(SELECT t.id1 id1,
t.id2 id2 ,
r.detail2 detail2
FROM lookup r ,
temp t
WHERE t.id2 =r.detail1(+)
) tmp ON ( t.id1=tmp.id1 )
WHEN matched THEN
UPDATE SET t.id2=tmp.id2, t.detail2=tmp.detail2 WHEN NOT matched THEN
INSERT
(key1,id1,id2,detail2
) VALUES
(NEWKEY,tmp.id1,tmp.id2,tmp.detail2
);
In the above query is it possible to use rownum to generate key(NEWKEY) for new combinations. What exactly needed is to have the NEWKEY as the {max(key1) from tgt table + rownum} ,without missing any keys.
For ex: if 1001 is the max key in tgt table, the NEWKEY for the next two new inserts into it should be 1002 & 1003.

Thanks for the help.
Tom Kyte
January 05, 2012 - 10:09 am UTC

No, it would not be possible.

The rownum would have to come from the TMP inline view - but then it would include the rows that get merged in as well.

you would need to two step this if you wanted to do that.

merge in the rows using when matched only.

then insert the rows that don't exist.


and remember, this only works if you have a single user.

Delete where not matched?

A reader, January 17, 2012 - 3:45 pm UTC

Say my target table has 10 rows and the USING dataset has 5 rows and all of them match the keys specified in the ON clause. All of them would meet the WHEN MATCHED clause and update the target table. This is fine but is there a way to *delete* the rows that don't match? Sort of like a DELETE WHEN NOT MATCHED. Thanks
Tom Kyte
January 17, 2012 - 3:53 pm UTC

no, the only way delete works in the merge is as a subclause of the when matched.

to remove 'missing' ones you would


delete from target where (key1,key2,,,) not in (select key1,key2,,,,, from source)

MERGE OUTPUT

Ravi B, March 27, 2012 - 1:10 pm UTC

Hi Tom,

Do we have something like this for Type 2 slow changing dimension implementation in oracle? This example is supposed to be from SQL Server i got from one of the db design manuals.

...MERGE can stream its output to a subsequent process. We’ll use this to do the final insert of the
changed Type 2 rows by INSERTing into the Customer_Master table using a SELECT from the
MERGE results. This sounds like a convoluted way around the problem, but it has the advantage of
only needing to find the Type 2 changed rows once, and then using them multiple times.
The code starts with the outer INSERT and SELECT clause to handle the changed row inserts at the
end of the MERGE statement. This has to come first because the MERGE is nested inside the
INSERT. The code includes several references to getdate; the code presumes the change was
effective yesterday (getdate()-1) which means the prior version would be expired the day before
(getdate()-2). Finally, following the code, there are comments that refer to the line numbers..

<code>

INSERT INTO Customer_Master
 SELECT Source_Cust_ID, First_Name, Last_Name, Eff_Date, End_Date, Current_Flag
 FROM
 ( MERGE Customer_Master CM
 USING Customer_Source CS
 ON (CM.Source_Cust_ID = CS.Source_Cust_ID)
 WHEN NOT MATCHED THEN
 INSERT VALUES (CS.Source_Cust_ID, CS.First_Name, CS.Last_Name,
  convert(char(10), getdate()-1, 101), '12/31/2199', 'y')
 WHEN MATCHED AND CM.Current_Flag = 'y'
  AND (CM.Last_Name <> CS.Last_Name ) THEN
  UPDATE SET CM.Current_Flag = 'n', CM.End_date = convert(char(10), getdate()-2, 101)
  OUTPUT $Action Action_Out, CS.Source_Cust_ID, CS.First_Name, CS.Last_Name,
         convert(char(10), getdate()-1, 101) Eff_Date, '12/31/2199' End_Date, 'y'Current_Flag
 ) AS MERGE_OUT
 WHERE MERGE_OUT.Action_Out = 'UPDATE';



Thanks!</code>

MERGE New/Insert/Update

Ravi B, May 25, 2012 - 11:43 am UTC

Hi Tom,

I have a table that we do complete refresh from source data every day. Now we have added a STATUS column to this table.Is there a way to flag this STATUS column with New/Update/Same using MERGE statement?

Thanks
Tom Kyte
May 25, 2012 - 1:17 pm UTC

No, the entire goal of a materialized view, the only goal, is to make the materialized view look exactly the same as a query against a table.


so, unless there is a query that you can run against the base table to determine "new, updated, same" since a certain point in time, there is no query against the materialized view you can use.

MERGE New/Insert/Update

Ravi B, May 25, 2012 - 2:28 pm UTC

Hi Tom,

This is not a materialized view. sorry i said "complete refresh" which might have implied materialized view.

I have a table A in DB1, table B in DB2. We truncate table A and insert data from table B daily over DB link.

Now, we plan to add STATUS column in table A. The values would be New/Updated/Unchanged or Same.

Could we use MERGE to accomplish this?
Tom Kyte
May 26, 2012 - 1:47 am UTC

You could, but, you would be taking a direct path - no buffer cache approach (what you are doing now - skips all undo and probably redo to if you want) and turning that into a conventional path operation. Lots of undo, lots of redo. It would probably take a lot long and use a lot more resources.


Is there anything in the base table that would tell you "new", "updated", "the same"

for example, if you maintained "created on", "updated on" columns - you'd be able to tell if a record was new, modified or the same since your last data pull...

MERGE New/Same/Update

Ravi B, May 27, 2012 - 3:41 pm UTC

Yes. We have "last_update_date" column in the base table based on which could find out status of this record. But, I might have to do row-by-row processing to flag each record as
New/Same/Updated. I was thinking if there could be a possibility of doing this in single shot as MERGE. Looks like it is not possible.
Tom Kyte
May 28, 2012 - 2:21 am UTC

why would you have to do "row by row" processing? I'm going to assume there is a flag in there that tells us when this last update date is reflecting "record was just created" versus updated. In fact, I'll assume there is a created time and a last update time


you would have a table with a single row "last_refreshed_time" showing the as of the last refresh.


and then

create or replace view vw
as
select ...., 
       case when x.last_refreshed_time <= y.created_on then 'NEW'
            when x.last_refreshed_time <= y.last_update_date then 'UPDATED'
            else 'SAME'
        end status
  from your_table Y, your_last_refreshed_time_table X



there is no processing necessary, this is entirely derived data and should not be stored anywhere. Just update this your_last_refreshed_time_table when you refresh and the data will take care of itself.


MERGE bug?

A reader, June 11, 2012 - 12:28 pm UTC

Merge requirements: (please see CREATE TABLE statements below) if y.id and x.id matched, set x.name to y.name. However, if y.name IS NULL, leave x.name intact.

This seems to work with the first MERGE statement. However, in the second MERGE statement, which has an additional "WHEN NOT MATCHED" component, x.name gets overwritten when y.name IS NULL. Is this a MERGE bug? How do I get around this problem? THanks.

SQL> SELECT * FROM v$version;

BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
CORE 11.2.0.3.0 Production
TNS for 64-bit Windows: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production

SQL> 
SQL> CREATE TABLE x (
  2     id   INTEGER,
  3     name VARCHAR2(20)
  4  );

Table created.

SQL> 
SQL> CREATE TABLE y (
  2     id   INTEGER,
  3     name VARCHAR2(20)
  4  );

Table created.

SQL> 
SQL> INSERT INTO x VALUES (1, 'John Doe');

1 row created.

SQL> INSERT INTO x VALUES (2, 'Laura Jones');

1 row created.

SQL> 
SQL> INSERT INTO y VALUES (1, 'John Doe');

1 row created.

SQL> INSERT INTO y VALUES (2, NULL);

1 row created.

SQL> 
SQL> COMMIT;

Commit complete.

SQL> 
SQL> MERGE INTO x
  2  USING y
  3  ON (
  4     x.id = y.id
  5  )
  6  WHEN MATCHED THEN
  7     UPDATE SET name = NVL(y.name, name)
  8  ;

2 rows merged.

SQL> 
SQL> SELECT * FROM x;

        ID NAME
---------- --------------------
         1 John Doe
         2 Laura Jones

SQL> 
SQL> rollback;

Rollback complete.

SQL> 
SQL> MERGE INTO x
  2  USING y
  3  ON (
  4     x.id = y.id
  5  )
  6  WHEN MATCHED THEN
  7     UPDATE SET name = NVL(y.name, name)
  8  WHEN NOT MATCHED THEN
  9     INSERT (id, name) VALUES (y.id, y.name)
 10  ;

2 rows merged.

SQL> 
SQL> SELECT * FROM x;

        ID NAME
---------- --------------------
         1 John Doe
         2

Tom Kyte
June 12, 2012 - 4:33 am UTC

workaround:

ops$tkyte%ORA11GR2> MERGE INTO x
  2  USING y
  3  ON (
  4     x.id = y.id
  5  )
  6  WHEN MATCHED THEN
  7     UPDATE SET name = NVL(y.name, x.name)
  8  WHEN NOT MATCHED THEN
  9     INSERT (id, name) VALUES (y.id, y.name )
 10  ;

2 rows merged.

ops$tkyte%ORA11GR2> SELECT * FROM x;

        ID NAME
---------- ------------------------------
         1 John Doe
         2 Laura Jones




do you have access to support to file the bug? if not, let me know

A reader, June 12, 2012 - 10:23 am UTC

Thanks Tom. Currently I don't have access to support.
Tom Kyte
June 12, 2012 - 11:37 am UTC

ok, i've put a note in my todo list to file this when I get back from traveling next week.

One of the two merges is a bug


Type 2 dimension loading using MERGE

Ravi B, September 14, 2012 - 12:30 pm UTC

Hi Tom,

I know oracle doesn't have RETURNING or INTO clause for MERGE statement as in SQLServer which makes the implementation much easier, but i found this article on how to do a Type 2 dimension loading using MERGE as a workaround for oracle.

I think this is brilliant implementation. But could I have your opinion on this approach or better way of doing this.

http://grouchgeek.blogspot.com/2010/02/oracle-merge-type-2-dimension.html

The thing i dint like is MINUS clause which tends to be expensive on very large data sets. But i am trying to benchmark with a simple join instead.
Tom Kyte
September 14, 2012 - 6:59 pm UTC

Minus is brutally efficient - given you have the resources (temp, memory, cpu). if you don't, nothing is going to be 'efficient'.

I'm not a huge fan of these "slowly changing dimensions".

old data is old data and should never ever change. (revisionist history in a data warehouse??)

new data might cause a row to be added to a dimension.

and that is about it.

update/insert or merge

A reader, June 26, 2013 - 9:45 am UTC

I have a table which has a unique natural key and a surrogate primary key (sequence). I want to insert into table if row doesn't exist or update if it does. After that, I want to return to the calling routine the surrogate PK that were affected by the operation. I also have to handle cases when multiple threads can be inserting duplicates into the same table;

I read your answer above using update/insert instead of MERGE and came up with the following;

update t
set    t.data = :data
where  t.nk1 = :b1
and    t.nk2 = :b2
and    t.nk3 = :b3
returning t.pk into :b4;

if  sql%rowcount = 0
then
    begin
      insert into t (pk, nk1, nk2, nk3, data)
        values (pk_seq.nextval, :b1, :b2, :b3, :data)
        returning pk into :b4;
    exception
      when dup_val_on_index then -- handle multiple threads inserting same row
        select pk
        into   :b4
        from   t
        where  t.nk1 = :b1
        and    t.nk2 = :b2
        and    t.nk3 = :b3;
    end;
end;
return :b4;

But do you think a MERGE like below is more appropriate ? 

begin
  begin
    merge into t
    using (select :b1 nk1, :b2 nk2, :b3 nk3
           from  dual
          ) d
    on   (    t.nk1 = d.nk1
          and t.nk2 = d.nk2
          and t.nk3 = d.nk3
         )
    when matched then
      update set t.data = :data
    when not matched then
      insert (pk, nk1, nk2, nk3, data)
        values (pk_seq.nextval, :b1, :b2, :b3, :data);
  exception
    when dup_val_on_index then null;
  end;

  select pk
  into   :b4
  from   t
  where  t.nk1 = :b1
  and    t.nk2 = :b2
  and    t.nk3 = :b3;
end;
/
return :b4;


Any suggestions?
Tom Kyte
July 01, 2013 - 6:56 pm UTC

i would benchmark the two approaches under what you consider to be realistic situations.

if the row is almost always going to be inserted, I would guess that option 1 would perform best. if the row is almost always going to be updated, I would lay odds on the merge.


all I can say is I'd rather have a single SQL statement doing this - no parallel worker threads, just a big CREATE TABLE AS SELECT to merge the two sets together into a new set - direct path load, no logging available, parallel already done, no locking issues, less code.

ORA-00904: : invalid identifier

A reader, July 10, 2013 - 5:42 pm UTC

Tom,

I got this error in run time when execute a merge statement with insert clause only in a package, and please notice that no object name (column etc) is reported. And I can't reproduce it consistently( if I run it again using the same data, it works). Can you think of anything that would cause this error? If it's an invalid identifier, would it not even compile? It's static sql statement.

Thanks

Error 904 when including join column in update

ARON, T E, August 14, 2013 - 5:17 pm UTC

This is not so much a question, but more an update on something originally reviewed by:

Reviewer: Vlado from Norristown, PA
April 7, 2004 - 4pm UTC

A 904 error still happens when:
- joining tables in the "using" clause
- AND including the join columns in the "update" clause
NOTE: the "joined column" is NOT the PK in my source table, but IS a PK in the joined table. We need a col in the joined table for a comparison; none of the columns in that table are used in the update/insert.

There seems to be a workaround: I found Valhalla when creating a view of the select in the "using."

In the original merge, there are several more columns, and I found no changing of the order that makes things work.

Oracle Support will be the happy recipient of this... unless someone points out that I'm doing something silly.

select * from v$version ;

Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for IBM/AIX RISC System/6000: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production

create table Stats
( ID number primary key
, IN_DATE date
) ;
insert into Stats values ( 1, trunc( sysdate ) ) ;
insert into Stats values ( 3, trunc( sysdate - 365 ) ) ;

create table Log
( ID number primary key
, STATS_ID number -- FK, not referenced
) ;
insert into Log values ( 101, 1 ) ;
insert into Log values ( 102, 1 ) ;
insert into Log values ( 103, 3 ) ;

create table Log_Log -- same same as Log
( ID number primary key
, STATS_ID number -- FK, not referenced
) ;

create or replace view Stat_Log
( ID
, STATS_ID
, IN_DATE
) as
( select l.ID, l.STATS_ID
, s.IN_DATE
from Log l
inner join Stats s
on l.STATS_ID = s.ID
) ;

-- ANSI
merge into Log_Log tgt
using
( select *
from Log l
inner join Stats s
on l.STATS_ID = s.ID
where s.IN_DATE <= '1 jan 2013'
) src
-- on ( l.ID = tgt.ID ) --<== this fails w/ same error
on ( src.ID = tgt.ID )
when matched then
update
set tgt.STATS_ID = src.STATS_ID
;
ERROR at line 10:
ORA-00904: "SRC"."ID": invalid identifier

-- kickin' it ol' school
merge into Log_Log tgt
using
( select *
from Log l
, Stats s
where s.IN_DATE <= '1 jan 2013'
and l.STATS_ID = s.ID
) src
on ( src.ID = tgt.ID )
when matched then
update
set tgt.STATS_ID = src.STATS_ID
; *
ERROR at line 10:
ORA-00904: "SRC"."ID": invalid identifier

-- now the view; we're expecting 1 row merged

select *
from Stat_Log l
where l.IN_DATE <= '1 jan 2013' ;

ID STATS_ID IN_DATE
---------- ---------- ---------
103 3 14-AUG-12

-- try using a view inside the "using"... SUCCESS!
merge into Log_Log tgt
using
( select *
from Stat_Log
where IN_DATE <= '1 jan 2013'
) src
on ( src.ID = tgt.ID )
when matched then
update
set tgt.STATS_ID = src.STATS_ID
when not matched then
insert ( ID, STATS_ID )
values ( src.ID, src.STATS_ID )

1 row merged.
Tom Kyte
August 14, 2013 - 5:32 pm UTC

don't use select *, remove the ambiguity and it works just fine (you didn't use select * in the view, that is substantially different)

ops$tkyte%ORA11GR2>  merge into Log_Log tgt
  2    using
  3       (  select l.id, l.stats_id
  4          from   Log l
  5          inner join Stats s
  6                   on l.STATS_ID = s.ID
  7          where   s.IN_DATE <= to_date( '1 jan 2013', 'dd mon yyyy' )
  8          ) src
  9    on    ( src.ID = tgt.ID )
 10    when matched then
 11       update
 12       set tgt.STATS_ID = src.STATS_ID
 13    when not matched then
 14       insert ( id, stats_id ) values ( src.id, src.stats_id )
 15  ;

1 row merged.





oh - and please - to_date() your dates with explicit formats!!!!!!!

merge and wasted sequence numbers

Alwin, August 16, 2013 - 12:15 pm UTC

Hello Tom,

I'm quoting a followup of this thread about wasted sequence numbers:
Followup September 22, 2006 - 2am UTC:

yes, please remember sequences are NOT gap free, sequences will NEVER BE gap free.

and if you burn 1,000,000,000 sequences per second, it would still be so many long years from now
before you exhausted the number type.....


In our case, it would take about 2 seconds (just kidding, but in reality there are only 3-4 months left) before we are in trouble as our application currently only supports 32 bit signed IDs. Migrating to 64bit is a mid to long-term planned project, so I need to find solutions for the current code. I'm not looking for a gap free solution, but simply less wasting of sequence values.

I have two questions for you:
1) why does MERGE calculate a sequence value for every merged row and not only for the inserted rows ? Shouldn't this be reported to oracle as a bug ?
2) what alternatives would you suggest ? In my list, I have already following options:
2a) changing the merge in favor of update and insert statements
2b) using a trigger before insert to initialize the ID with a sequence value if it's NULL and change the merge by removing the use of the sequence in favour of NULL. So only real inserts fire the trigger and generate sequence numbers.

Regards
-- Alwin


Tom Kyte
August 28, 2013 - 5:06 pm UTC

i have no problem "wasting" sequence numbers. they are never gap free, NEVER. a sequence will never generate gap free numbers.

I'm afraid that you are going to (just like everyone else on the planet) recognize that surrogate keys are getting into the billions - easily into the billions - as we have billions of rows in tables.

Your app will either have to treat the key as a string (host variable of a string type, use to_number(:x) when sending back to query - use to_char(id) when selecting it

or

use a big integer type.


merge by design will have a sequence number generated for each possible row.


DO NOT USE A TRIGGER. repeat: do not use a trigger. do anything else but do not use a trigger.


I would go for option three: use a type in my program that works with big numbers as big numbers are inevitable, they are here.

It would be a great time to move the sql from your client into plsql procedures - and not have to worry about things like this in the future...


there is no quick fix for something like this, no band-aids. sometimes we have to change code.


Other option if not merge

A reader, October 10, 2013 - 6:17 pm UTC

Hi Tom,

In my merge operation i would always likely to get ORA-30926 because my source table has keyrow updated many times.

It is a change data process. My source table contain all changed rows. which can have same pk row multiple times. It also has flag showing I--insert,U--update,D--delete. I thought of doing I and U in single operation using merge but it is not possible. due to reason i mentioned above.

let me know what would be the best option. how about

option 1
for i in ( select * from source table where flag in (I, U))

update destination table .....
if sql%rowcount 0 then
insert into destination table
---------------------------------------
Option 2
for i in ( select * from source table )
IF flag = 'I; then
Insert into destination table.....
ELSif flag ='U'

update destination table .....


Or if any other better option.. javascript:apex.submit('SUBMIT_REVIEW');
Tom Kyte
October 10, 2013 - 7:29 pm UTC

what is a keyrow?

if you have the same primary key in the source table multiple times, tell us - how the HECK do you process that?????


what if the record is updated five times. how do you know which one is the "right" one?

what if you have an update, delete and insert for the same primary key. how do you know if

a) the record was inserted then updated then deleted
versus
b) the record was deleted then inserts then updated
versus
c) the record was updated then deleted then inserted

and so on..........................


be very specific
be very precise
write what is call "a specification"

RE: Other option if not merge

Vikas Sharma, October 11, 2013 - 9:02 am UTC

Hi Tom,

Sorry for Not providing the complete info. it is as follows.

Current Version of Database: 11gr2
The source table has got 2 fields ( ch_seq_no - populated by oracle seq. other field it change datetime (date). The ch_seq_no will always unique as it comes from oracle seq. we will be processing on the basis of ch_seq_no (asc)

We shall be performing this operation on about 100 twice in a day and as per our analysis there could be max 20k rows (all table total) to process.

my_source_table ( destination_pk_col varchar2(30),
destination_col1
destination_col2
......
ch_seq_no Number -- from oracle seq always unique and
PK of source table
ch_datetime date, -- date and time is stored here
IUD_FLAG VARCHAR2(1) -- indicator for insert update or delete
)

Option 1
FOR i in (select * my_source_table order by ch_seq_no,ch_datetime)
IF iud_flag in ('I,'U') Then

update destination table using destination_pk_col
if sql%rowcount 0 then
insert into destination table

elsif IUD_flag ='D' THEN

DELETE from ......
---------------------------------------
Option 2
for i in ( select * from source table order by ch_seq_no,ch_datetime )
IF flag = 'I; then
Insert into destination table.....
ELSif flag ='U'

update destination table using..destination_pk_col
ELSIF iud_flag ='D'
delete

Please suggest if there is any other better way of doing the same.

Tom Kyte
October 11, 2013 - 1:42 pm UTC

delete from table where primary_key in (select primary_key from update_table where iud_flag in ( 'U', 'D' );

insert into table
select ....
  from (select table.*, 
               row_number() over 
                  (partition by primary_key order by ch_seq_no DESC) rn 
           from update_table table)
 where rn = 1
   and iud_flag in ('I','U')
/



get rid of any record modified (updated or deleted)

insert the last image of the record if it was last updated or inserted (but do nothing for records where the last change was a delete...)





of course you should just use a materialized view and be done with it. No code.


less code = less bugs.
more code = more bugs.


you are reinventing a wheel that was invented well over 20 years ago.

custom-made CDC

Kanellos, October 11, 2013 - 11:41 am UTC

Hi Vikas,

If I have got it right, this is some sort of custom-made Changed Data Capture implementation you are trying to do. Every 12 hrs or so (twice a day) you run a process to reproduce the -already captured and kept in a table- data changes that happened in those 100 tables that you monitor. If this is the case, I would suggest that you didn't reproduce one by one all the changes that have happened in your data during the last/current window (period of time between last time that you applied the changes and this time) but rather perform only the required ones (the last one maybe). Let's assume for instance that for one record 10 (or even better 10,000,000) updates and one delete have happened. Why would you perform all these 11 operations one by one when you actually only need to perform just one delete?
If I were you, I would group my data set per PK and maybe employ some analytic function(s) (like first_value, last_value) so that I would have only ONE action per primary key. Something like:

For i in ( SELECT pk_column, LAST_VALUE(iud_flag) OVER (PARTITION BY pk_column ORDER BY ch_seq_no RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) operation from my_source_table ) 

MERGE .... INTO destination_table







RE: Other option if not merge

Vikas Sharma, October 11, 2013 - 1:32 pm UTC

Thanks Kanellos,

Well Actually this is processing of change data capture (CDC). Whether it is ORACLE CDC or custom CDC the data is capture in this way, each and every update is recorded.

I cannot use the PK_last changed record may not be right. Think about a scenario below

pk_col col1 col2
1 0 9 << before update
1 1 9 << after first update
1 0 1 << second update...
1 0 4 << third update

All update are done in the same duration which needs to be merged in a go.

We cannot take the last action on one pk_col as multiple update operation can perform selected column update.

Thanks
Vikas
Tom Kyte
October 11, 2013 - 1:49 pm UTC

bear in mind that CDC is deprecated in current releases.

http://docs.oracle.com/cd/E11882_01/server.112/e23633/changes.htm#CJAECCIJ

http://docs.oracle.com/cd/E16655_01/server.121/e17642/deprecated.htm#sthref605


I suggest materialized views.



I don't know what you mean about not being able to use the last changed record? what about your scenario makes it so you cannot?

you just need the last updated record - 1,0,4 - that is the record you need. The other updates are not relevant.

RE: Other option if not merge

Vikas Sharma, October 11, 2013 - 4:14 pm UTC

You are correct Tom, i was wrong by saying "cannot take the last record of a PK" realized later. It is just the last record of that PK.

May be with this i think i can even do "MERGE"

I understand that materialized views with Refresh would be the best option. But the table/s i want to populate is on other database (destination) used for datawarehousing /BI reporting purpose. AND the source database other remote location which does not have a Dedicated IP. (It is not available by service provider) So Destination database (BI database) cannot access source database to refresh.

What we are doing is we are capturing the Changed data in a schema >> exp the dump and sending it BI database using FTP. The Dump is imported in a different schema and then records further processed to BI database.

The scenario you suggested might be the better option then.
-----------
delete from table where primary_key in (select primary_key from update_table where iud_flag in (
'U', 'D' );

insert into table
select ....
from (select table.*,
row_number() over
(partition by primary_key order by ch_seq_no DESC) rn
from update_table table)
where rn = 1
and iud_flag in ('I','U')

Vikas Sharma
Tom Kyte
October 11, 2013 - 4:29 pm UTC

just bear in mind that your approach will not work for much longer at all.

not having a dedicated IP shouldn't be a problem - DNS can deal with that.

Ankit, October 21, 2013 - 10:02 am UTC

Hi Tom

We have following MERGE statement in our code.
We upgraded from Oracle 9.2 to 11gR2 yesterday, and below statement is not working now.
Receiving error "SQL Error: ORA-00904: "CS"."TYP": invalid identifier" in the line marked with arrow.

Same statement is working perfectly fine in Oracle 9.2

MERGE INTO abc_cd_lkup cl
USING (SELECT DISTINCT 'ACCT_STATUS_CD' typ
,acct_status_cd cd
,acct_status_desc des
,source_system_cd src
FROM abc_account_dtl_stg
WHERE acct_status_cd IS NOT NULL
) cs

ON (cl.cd_lkup_type = cs.typ
AND cl.native_cd = cs.cd
AND cl.source_system_cd = cs.src)

WHEN MATCHED THEN
UPDATE
SET cl.native_cd_desc = cs.des
,cl.row_update_user_id = 'ABCADMIN'
,cl.row_update_tmst = SYSDATE

WHEN NOT MATCHED THEN
INSERT (code_lkup_id, cd_lkup_type, abc_cd_req_ind, source_system_cd
,native_cd, native_cd_desc, row_insert_user_id
,row_insert_tmst)
VALUES (abc_cd_lkup_code_lkup_id_s.NEXTVAL, cs.typ
, (SELECT DISTINCT abc_cd_req_ind
FROM abc_cd_lkup
WHERE cd_lkup_type = cs.typ), <=======
cs.src, cs.cd
,cs.des, 'ABCADMIN', SYSDATE);



Has there been any change in MERGE in 11g R2 or there is some other reason.

Please help.

Using MERGE to find changed data

Tony Keller, August 21, 2014 - 9:42 pm UTC

Hi Tom,
I read your columns, books and this site every chance I get. I have to say you rock!

I have the following code in package I wrote. It works, so debugging is not the issue. I have one table, A, that gets inserted and updated all the time.  I want to send new rows
to an interface, but updated rows only get a subset of the data. So table B contains table A as it appeared before the last change. The MERGE statement uses a MINUS to determine if any changes were made. If the USING statement returns nothing then nothing is updated or inserted. It works fast.

My question is, when performing the update part of the MERGE does it update all the columns or only the columns that contain data after the MINUS? I'm just curious.
<code>
CREATE TABLE A
(
  INTERNAL_ID        NUMBER(8),
  EXTERNAL_ID        VARCHAR2(9 CHAR),
  LAST_NAME          VARCHAR2(60 CHAR),
  FIRST_NAME         VARCHAR2(60 CHAR),
  MI                 VARCHAR2(60 CHAR),
  NETID              VARCHAR2(30 CHAR),
  EMAIL_ADDRESS      VARCHAR2(128 CHAR),
  TELEPHONE          VARCHAR2(30 CHAR),
  START_DATE         DATE,
  ACCRUAL            VARCHAR2(35 CHAR),
  PAY_CALC           VARCHAR2(35 CHAR),
  PAY_PREP           VARCHAR2(20 CHAR)
);
CREATE UNIQUE INDEX PK_A ON A
(INTERNAL_ID);

CREATE TABLE B
(
  INTERNAL_ID        NUMBER(8),
  EXTERNAL_ID        VARCHAR2(9 CHAR),
  LAST_NAME          VARCHAR2(60 CHAR),
  FIRST_NAME         VARCHAR2(60 CHAR),
  MI                 VARCHAR2(60 CHAR),
  NETID              VARCHAR2(30 CHAR),
  EMAIL_ADDRESS      VARCHAR2(128 CHAR),
  TELEPHONE          VARCHAR2(30 CHAR),
  START_DATE         DATE,
  ACCRUAL            VARCHAR2(35 CHAR),
  PAY_CALC           VARCHAR2(35 CHAR),
  PAY_PREP           VARCHAR2(20 CHAR),
  EFFECTIVE_DATE     DATE
);
CREATE UNIQUE INDEX PK_A ON A
(INTERNAL_ID);

DECLARE
CURSOR C_A IS
  SELECT *
    FROM A;
BEGIN
  FOR I_A IN C_A LOOP
    MERGE INTO B up USING
    (SELECT I_A.INTERNAL_ID,
            I_A.EXTERNAL_ID,
            I_A.LAST_NAME,
            I_A.FIRST_NAME,
            I_A.MI,
            I_A.NETID,
            I_A.EMAIL_ADDRESS,
            I_A.TELEPHONE,
            I_A.START_DATE,
            I_A.ACCRUAL,
            I_A.PAY_CALC,
            I_A.PAY_PREP
       FROM DUAL
     MINUS
     SELECT B.INTERNAL_ID,
            B.EXTERNAL_ID,
            B.LAST_NAME,
            B.FIRST_NAME,
            B.MI,
            B.NETID,
            B.EMAIL_ADDRESS,
            B.TELEPHONE,
            B.START_DATE,
            B.ACCRUAL,
            B.PAY_CALC,
            B.PAY_PREP
       FROM B
      WHERE B.INTERNAL_ID = I_A.INTERNAL_ID)
     ON (B.INTERNAL_ID = I_A.INTERNAL_ID)
       WHEN NOT MATCHED THEN
     INSERT (INTERNAL_ID,
            EXTERNAL_ID,
            LAST_NAME,
            FIRST_NAME,
            MI,
            NETID,
            EMAIL_ADDRESS,
            TELEPHONE,
            START_DATE,
            ACCRUAL,
            PAY_CALC,
            PAY_PREP,
            EFFECTIVE_DATE)
     VALUES (I_A.INTERNAL_ID,
            I_A.EXTERNAL_ID,
            I_A.LAST_NAME,
            I_A.FIRST_NAME,
            I_A.MI,
            I_A.NETID,
            I_A.EMAIL_ADDRESS,
            I_A.TELEPHONE,
            I_A.START_DATE,
            I_A.ACCRUAL,
            I_A.PAY_CALC,
            I_A.PAY_PREP,
            TRUNC(SYSDATE))
       WHEN MATCHED THEN
            UPDATE SET 
            B.INTERNAL_ID = I_A.EXTERNAL_ID,
            B.EXTERNAL_ID = I_A.LAST_NAME,
            B.LAST_NAME = I_A.FIRST_NAME,
            B.FIRST_NAME = I_A.FIRST_NAME,
            B.MI = I_A.MI,
            B.NETID = I_A.NETID,
            B.EMAIL_ADDRESS = I_A.EMAIL_ADDRESS,
            B.TELEPHONE = I_A.TELEPHONE,
            B.EFFECTIVE_DATE = TRUNC(SYSDATE);
  END LOOP;
  .
  .
  -- Code to send rows in B updated today to interface.
</code>