Skip to Main Content

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, james.

Asked: October 27, 2017 - 3:04 pm UTC

Last updated: January 10, 2018 - 12:42 am UTC

Version: 12.1.0.2.0

Viewed 1000+ times

You Asked

Hi Connor,

I asked this question on OTN:
We have some mviews with joins, and we found out sometimes an update on the base table will result in insert+delete on the mview, not delete+insert. Because insert comes first, we have to make the primary key or unique constraint deferrable. This is fine, but in case the mview is replicated by GG, on the target site it will delete by primary key, and after the refresh it deletes not only the old data but also the new one. If delete comes before insert then this will not be an issue. What is the reason behind this implementation?

Below is a test case to reproduce it:

jsu@JSU12P> select * from v$version;

BANNER CON_ID
-------------------------------------------------------------------------------- ----------
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production 0
PL/SQL Release 12.1.0.2.0 - Production 0
CORE 12.1.0.2.0 Production 0
TNS for 64-bit Windows: Version 12.1.0.2.0 - Production 0
NLSRTL Version 12.1.0.2.0 - Production 0

-- Create two tables for the test case

CREATE TABLE test_parent (
pid number not null constraint test_parent_pk primary key
,c1 number
,c2 number
)
/

CREATE MATERIALIZED VIEW LOG ON test_parent WITH ROWID, PRIMARY KEY, SEQUENCE INCLUDING NEW VALUES;

CREATE TABLE test_child (
cid number not null constraint test_CHILD_pk primary key
,c3 number
,c4 number
,pid number constraint test_child_fk references test_parent(pid)
)
/

CREATE MATERIALIZED VIEW LOG ON test_child WITH ROWID, PRIMARY KEY, SEQUENCE INCLUDING NEW VALUES;

INSERT INTO test_parent VALUES (1,1,1);
INSERT INTO test_child VALUES (1,1,1,1);
COMMIT;

------ Now create the mview

CREATE MATERIALIZED VIEW MV_TEST
REFRESH FAST ON DEMAND
WITH PRIMARY KEY
AS
SELECT p.ROWID AS P_ROWID
,c.rowid AS C_ROWID
,c.cid
,c.c3
,c.c4
,c.pid
,p.c1 + 123 as c1
from test_child c, test_parent p
WHERE c.pid = p.pid(+)
/

ALTER TABLE MV_TEST ADD CONSTRAINT MV_TEST_PK PRIMARY KEY(cid) DEFERRABLE INITIALLY DEFERRED;

select * from MV_TEST;


--------------- set up a trigger and audit table for observation of the refresh behavior

create sequence mv_audit_seq order;

create table mv_audit (seq number,dml varchar2(10),cid number);

create or replace trigger trg_mv_test
after insert or delete or update on MV_TEST for each row
declare
v_dml varchar2(10);
begin
v_dml := case when inserting then 'I'
when deleting then 'D'
when updating then 'U'
end;

insert into mv_audit values (mv_audit_seq.nextval,v_dml,:new.cid);

end;
/

------- run the below DML to reproduce this issue. An update on the parent table will trigger the unusual behavior in mview refresh

jsu@JSU12P> UPDATE test_parent SET C1=NVL(C1,0)+1;

1 row updated.

jsu@JSU12P> COMMIT;

Commit complete.

jsu@JSU12P> EXEC DBMS_MVIEW.REFRESH('MV_TEST','F');

PL/SQL procedure successfully completed.

jsu@JSU12P> select * from mv_audit order by seq;



SEQ DML CID
---------- ---------- ----------
1 U 1
2 I 1
3 D

------ as you can see, it first did an update, then insert, then delete
------ delete happened after insert
------ at this time, if you have GG replication, the source mv_test has one row, target mv_test has zero row because when it deleted the target table by primary key cid=1, the newly inserted row was also deleted


and Connor said...

I spoke to the Goldengate PM about this.

He said that mviews are supported with Goldengate, subject to the following:


Materialized views are supported by Extract in classic and integrated modes with the following limitations.

- Materialized views created WITH ROWID are not supported.
- The materialized view log can be created WITH ROWID.
- The source table must have a primary key.
- Truncates of materialized views are not supported. You can use a DELETE FROM statement.
- DML (but not DDL) from a full refresh of a materialized view is supported. If DDL support for this feature is required, open an Oracle GoldenGate support case.
- For Replicat the Create MV command must include the FOR UPDATE clause
- Either materialized views can be replicated or the underlying base table(s), but not both.


I think the first bullet point on the list is applicable here.

Rating

  (1 rating)

Is this answer out of date? If it is, please let us know via a Comment

Comments

DML generated order

James Su, January 09, 2018 - 4:29 pm UTC

hi Connor,

In the above script, my mview was created with primary key, not with rowid.

But the GG issue is just a side effect. What I'm interested in is, why the DML generated in the refresh procedure is in the order of insert+delete? It makes more sense to me to first delete the existing data, then insert new rows. If we can do it this way, then GG replication won't be a problem.
Connor McDonald
January 10, 2018 - 12:42 am UTC

Whilst the audit triggers suggest we did a simple insert then delete, in reality, there's a lot more complexity going on behind the scenes here. If you trace the refresh you'll see something like:

DELETE FROM "MCDONAC"."MV_TEST" "SNA$" 
WHERE
 ROWID IN (SELECT RID FROM (  SELECT "SNA$".ROWID RID,  ROW_NUMBER() OVER 
  (PARTITION BY "C_ROWID" ORDER BY RID$ NULLS LAST) R,  COUNT(*) OVER 
  (PARTITION BY "C_ROWID" ) T_CNT,  COUNT(RID$) OVER (PARTITION BY "C_ROWID" )
   IN_MVLOG_CNT  FROM "MCDONAC"."MV_TEST" "SNA$", (SELECT DISTINCT RID$ FROM 
  (SELECT  CHARTOROWID("MAS$"."M_ROW$$") RID$     FROM 
  "MCDONAC"."MLOG$_TEST_PARENT" "MAS$"   WHERE "MAS$".SNAPTIME$$ > :B_ST0 AND 
  NOT        ("MAS$".DMLTYPE$$ = 'U' AND         
  (sys.dbms_snapshot_utl.vector_compare(:B_CV0,                               
                  "MAS$".change_vector$$) = 0))) )  AS OF SNAPSHOT(:B_SCN) 
  MAS$ WHERE "SNA$"."C_ROWID" IN (SELECT "C_ROWID" FROM "MCDONAC"."MV_TEST" 
  "SNA$" WHERE "P_ROWID" IN (SELECT * FROM (SELECT  
  CHARTOROWID("MAS$"."M_ROW$$") RID$     FROM "MCDONAC"."MLOG$_TEST_PARENT" 
  "MAS$"   WHERE "MAS$".SNAPTIME$$ > :B_ST0 AND NOT        ("MAS$".DMLTYPE$$ =
   'U' AND         (sys.dbms_snapshot_utl.vector_compare(:B_CV0,              
                                   "MAS$".change_vector$$) = 0))) AS OF 
  SNAPSHOT(:B_SCN) MAS$)) AND "SNA$"."P_ROWID" = MAS$.RID$(+) ) "SNA2$" WHERE 
  T_CNT > 1  AND ((IN_MVLOG_CNT = T_CNT AND R > 1) OR      (IN_MVLOG_CNT < 
  T_CNT AND R <= IN_MVLOG_CNT))) 


UPDATE "MCDONAC"."MV_TEST" SNA$ SET "P_ROWID"=NULL,"C1"=NULL+123 
WHERE
 "P_ROWID" IN (SELECT /*+ NO_MERGE  HASH_SJ  */ * FROM (SELECT  
  CHARTOROWID("MAS$"."M_ROW$$") RID$     FROM "MCDONAC"."MLOG$_TEST_PARENT" 
  "MAS$"   WHERE "MAS$".SNAPTIME$$ > :B_ST0 AND NOT        ("MAS$".DMLTYPE$$ =
   'U' AND         (sys.dbms_snapshot_utl.vector_compare(:B_CV0,              
                                   "MAS$".change_vector$$) = 0))) AS OF 
  SNAPSHOT(:B_SCN) MAS$)

INSERT  /*+ NOAPPEND */ INTO "MCDONAC"."MV_TEST" SELECT /*+ NO_MERGE("JV$") 
  */ "JV$"."RID$","MAS$1".ROWID,"MAS$1"."CID","MAS$1"."C3","MAS$1"."C4",
  "MAS$1"."PID","JV$"."C1"+123 FROM ( SELECT "MAS$"."ROWID" "RID$"  ,  
  "MAS$".*  FROM "MCDONAC"."TEST_PARENT" "MAS$" WHERE ROWID IN (SELECT  /*+ 
  HASH_SJ */  CHARTOROWID("MAS$"."M_ROW$$") RID$     FROM 
  "MCDONAC"."MLOG$_TEST_PARENT" "MAS$"   WHERE "MAS$".SNAPTIME$$ > :B_ST0 AND 
  NOT        ("MAS$".DMLTYPE$$ = 'U' AND         
  (sys.dbms_snapshot_utl.vector_compare(:B_CV0,                               
                  "MAS$".change_vector$$) = 0))))  AS OF SNAPSHOT(:B_SCN) 
  "JV$", "TEST_CHILD" AS OF SNAPSHOT(:B_SCN)  "MAS$1" WHERE "MAS$1"."PID"=
  "JV$"."PID"

DELETE FROM "MCDONAC"."MV_TEST" "SNA$" 
WHERE
 ROWID IN (SELECT ROWID FROM (  SELECT  ROW_NUMBER() OVER (PARTITION BY 
  "C_ROWID" ORDER BY "P_ROWID" NULLS FIRST) R,  COUNT(*) OVER (PARTITION BY 
  "C_ROWID" ) T_CNT,  COUNT("P_ROWID") OVER (PARTITION BY "C_ROWID" ) 
  NONNULL_CNT  FROM "MCDONAC"."MV_TEST" "SNA2$" WHERE "C_ROWID" IN (SELECT 
  "MAS$1".ROWID  FROM ( SELECT "MAS$"."ROWID" "RID$"  ,  "MAS$".*  FROM 
  "MCDONAC"."TEST_PARENT" "MAS$" WHERE ROWID IN (SELECT  /*+ HASH_SJ */  
  CHARTOROWID("MAS$"."M_ROW$$") RID$     FROM "MCDONAC"."MLOG$_TEST_PARENT" 
  "MAS$"   WHERE "MAS$".SNAPTIME$$ > :B_ST0 AND NOT        ("MAS$".DMLTYPE$$ =
   'U' AND         (sys.dbms_snapshot_utl.vector_compare(:B_CV0,              
                                   "MAS$".change_vector$$) = 0))))  AS OF 
  SNAPSHOT(:B_SCN)  "JV$", "TEST_CHILD" AS OF SNAPSHOT(:B_SCN)  "MAS$1" WHERE 
  "MAS$1"."PID"="JV$"."PID") ) "SNA2$" WHERE T_CNT >= 1  AND ((NONNULL_CNT = 
  0 AND R >= 1) OR      (NONNULL_CNT > 0 AND R <= T_CNT - NONNULL_CNT))) 


but ultimately I would suspect this is due to the fact that we have to handle all potential permutations here. This test case is a simple update, but more complex scenarios could involve deleting a row, re-inserting it, then updating it, then deleting etc etc etc...all of which we have to get "just right".

More to Explore

Administration

Need more information on Administration? Check out the Administrators guide for the Oracle Database