Skip to Main Content

Breadcrumb

Question and Answer

Connor McDonald

Thanks for the question, krishna.

Asked: March 28, 2017 - 7:32 am UTC

Last updated: May 16, 2023 - 2:45 am UTC

Version: 11.2.0.4

Viewed 1000+ times

You Asked

Hi ,

As we all aware that to implement golden gate replication ,minimal supplemental logging should be enabled at source db end.

1)Why we need to enable supplemental logging as redo contains both undo and redo vector details.

2)What is the use by logging extra column details in the redo .

Please explain with example

and Connor said...

Consider a simple statement - we'll use a 1 row insert and we'll just assume its going to work and does not need to rollback)

insert into t values (1);

Now, in order to re-apply that command in a database *recovery* model (ie, a restore/recover or physical dataguard), we dont really need to know *any* of the specifics. All we need is (conceptually) in the redo log is:

block 1335 in file 17 was modified:
- the row directory changed
- the SCN changed
- bytes 1234-1247 now contains

Because during a recovery those instructions are sufficient to apply the exact same change to a physical replica.

But a Goldengate (GG) copy is NOT a physical replica - it could be a totally different database structure, it might be a totally different database *vendor*.

So we need *more* that just that information, and depending on how complete we want GG to be able to capture changes and apply them onwards dicates just how much more information we need to capture. Because GG has the job of turning those "instructions" back into real SQL statements to run against the target database.


Rating

  (8 ratings)

Is this answer out of date? If it is, please let us know via a Comment

Comments

A reader, March 28, 2017 - 11:37 am UTC

Excellent Explaination

Golden gate Oracle to MSSQL 2017

Roshan, April 14, 2019 - 7:27 am UTC

Hello,

Windows SQL server 2017 (target)

OS: Windows Server 2019

OGG 12.2.0.2.2



Oracle Database 12.1.0.2 (source)

RHEL 7.4



Hello Team,



I am doing a table replication from the source to a target SQL server.



TARGET MGR

PORT 7809

ACCESSRULE, PROG REPLICAT, IPADDR *, ALLOW



REPLICATION PARAMETER FILE

REPLICAT RINIBI

TARGETDB BIREPORT, USERID BIREPORT, PASSWORD BIREPORT

SOURCEDEFS ./dirdef/source_defs.def

DISCARDFILE ./dirrpt/RINIBI.dsc, PURGE

MAP BIREPORT.INSBILL_GROUPMTH, TARGET BIREPORT.INSBILL_GROUPMTH;



Source parameter file(extract)

EXTRACT EXBIMS

--- User login

USERIDALIAS oraggs

RMTHOST <sql SERVER IP>, MGRPORT 7809

RMTTASK REPLICAT, GROUP RINIBI

TABLE BIREPORT.INSBILL_GROUPMTH;



Please find error from ggserr.log



/data1/gg/dirtmp.

2019-04-13T20:21:09.644+0400  WARNING OGG-06439  Oracle GoldenGate Capture for Oracle, exbims.prm:  No unique key is defined for table INSBILL_GROUPMTH. All viable columns will be used to represent the key, but may not guarantee uniqueness. KEYCOLS may be used to define the key.

2019-04-13T20:21:09.644+0400  INFO    OGG-06509  Oracle GoldenGate Capture for Oracle, exbims.prm:  Using the following key columns for source table BIREPORT.INSBILL_GROUPMTH: TRANS_NUM_V, BILL_GROUP, TRANS_DATE, TRANS_AMOUNT, BILL_CYCLE_FULL_CODE_N, ARTICLE_CODE_V, ARTICLE_CATEGORY_V, DB_CR_V, TAX_OPTN_V, BASE_TRANS_AMT_N, DURATION_N, CURRENCY_CODE_V, GL_CODE_N, NUM_DURATION_UNITS_N, NUM_VOLUME_UNITS_N, TAX_PLAN_CODE_V, TAX_FORMULA_CODE_V, VOLUME_N, FOREIGN_CURRENCY_CODE_V, EXCHNG_RATE_N, ARTICLE_DISC_AMT_N, CROSS_DISC_AMT_N, DESC_V, SCHEME_REF_CODE_N, ACCOUNT_ID, ACCOUNT_LINK_CODE_N, SERVICE_CODE_V, TELNO, ACTIVATION_DATE, FROM_DATE_D, TO_DATE_D, STATUS_CODE_V, STATUS_CHANGE_DATE_D, CONTRACT_TYPE_V, SUB_SERVICE_CODE_V, SUBSCRIBER_TYPE_V, TAX_OPTN_SERV, TAX_PLAN_CODE_SERV, LEDGER_CODE_N, PACKAGE_CODE_V, TARIFF_CODE_V, DETAIL_DESCRIPTION_V, DISPLAY_DESCRIPTION_V, ACCOUNT_NAME_V, SUBSCRIBER_CODE_N, FIRST_NAME_V, LAST_NAME_V, SUBS_CATEGORY_CODE_V, SUBS_SUB_CATEGORY_CODE_V, CUSTOMER_ID, ACCTLINKMAST, DATEINS, DURATION, SUBS_CATEGORY_DESC_V, SUBS_SUB_CATEGORY_DESC_V, OFFERDESC, PACKDESC, INSTSTREET, INSTCITY, INSTLOCALITY, BILLSTREET, BILLCITY, BILLLOCALITY, FDP, CABINET, MDF, EXCHANGE, AREA, WAIVEDAMT, OUTSTANDAMT, PURCOUTSTNDAMT, PACKAGE_DESCRIPTION.

2019-04-13T20:21:09.644+0400  INFO    OGG-00993  Oracle GoldenGate Capture for Oracle, exbims.prm:  EXTRACT EXBIMS started.

2019-04-13T20:21:15.636+0400  WARNING OGG-01194  Oracle GoldenGate Capture for Oracle, exbims.prm:  EXTRACT task RINIBI abended : Executing statement for select operation Database error 6550 ([Oracle][ODBC][Ora]ORA-06550: line 1, column 9:

                              PLS-00103: Encountered the symbol "@" when expecting one of the following:

 

 

                                 begin function pragma procedure subtype type <an identifier>

                                 <a.

2019-04-13T20:21:15.637+0400  ERROR   OGG-01203  Oracle GoldenGate Capture for Oracle, exbims.prm:  EXTRACT abending.


2019-04-13T20:21:15.637+0400 ERROR OGG-01668 Oracle GoldenGate Capture for Oracle, exbims.prm: PROCESS ABENDING.
Connor McDonald
April 15, 2019 - 4:40 am UTC

A reader, February 21, 2022 - 6:56 am UTC

Hi,

We currently use Oracle Streams on our 11gR2 DBs.
Since, we will be upgrading to 19C so we need to user Goldengate going forward.

As you are aware, streams could be manipulated from within the DB but Goldengate uses a CLI since it resides out of the DB as it uses param & trail files.

We currently have bi-directional replication and are thinking of using GG Microservices Architecture using the command line.

Architecture:
DBServer1: Exadata DB1
DBServer1: GG installed

DBServer2: Exadata DB2
DBServer2: GG installed

Application_Server: Install GG Microservices Architecture and control GG on the DB servers using RESTful API

Is the above allowed (based on the docs it seems it is supported)?
Commands which can help us make calls from "Application_Server" and manipulate/query GG on the DB servers.

Thanks in advance as always.
Cheers!

Connor McDonald
February 28, 2022 - 6:46 am UTC

Yes. At the low level, its still "just" Goldengate, the microservices layer allows it to be managed via REST.

Oracle Ace Veera had a nice write up on the two

https://www.oracle-scn.com/oracle-goldengate-microservices-architecture/

BLOB replication in Goldengate

A reader, March 22, 2023 - 7:32 am UTC


Hi,

We are in the process of upgrading our DBs to 19c and use OGG MA for replication.

Few tables have LOB columns and we wanted to know the following:

1. When we append/update a CLOB column for example, is only the delta/changes are captured on source and applied to target?

2. Is there anyway we can check that?

Thank you.



Connor McDonald
April 04, 2023 - 6:25 am UTC

Goldengate is fundamentally about mining redo logs. Thus it is the content in the redo that matters. For a blob, this is going to be deltas to the lobsegment that underpins the blob.

But you have choices as well - Thanks to Veeratteshwaran Sridhar for this info:

Extract captures LOBs from the redo log. For UPDATE operations on a LOB document, only the changed portion of the LOB is logged. To force whole LOB documents to be written to the trail when only the changed portion is logged, use the TRANLOGOPTIONS parameter with the FETCHPARTIALLOB option in the Extract parameter file. When Extract receives partial LOB content from the logmining server, it fetches the full LOB image instead of processing the partial LOB. Use this option when replicating to a non-Oracle target or in other conditions where the full LOB image is required.

Thank you :)

A reader, April 04, 2023 - 7:06 am UTC


Thank you very much Connor and Veeratteshwaran.
Much appreciated.

Cheers!

Connor McDonald
April 06, 2023 - 7:00 am UTC

glad we could help

DML error capture

A reader, May 04, 2023 - 6:06 am UTC


Hello,

We want to capture the DML errors and for that enabled macros with replicat.

But although it works just fine it only captures the DML with bind variables and not the actual values.

Setting the reperror parameter to DISCARD risks filling up the discard file in case of too many errors.

Any suggestions to have the DML errors captures with actual values?

Thank you.

Connor McDonald
May 10, 2023 - 4:46 am UTC

Does the approach blogged by Ace Director Gavin help in this case?

https://gavinsoorma.com.au/knowledge-base/handling-goldengate-exceptions-and-errors-with-reperror/

GG Exception handler

A reader, May 10, 2023 - 6:00 am UTC

Thank you Connor.

yes, doing exactly that using a macro inside the replicat param file. 
But in the case of an DML error it captures the SQL as follows:

SQL <INSERT INTO "TOPAZ_GG_REPCHK"."TBGGWITHDATA" ("COLNUM","COLTXT","COLRANDNUM","NEWCHR") VALUES (:a0,:a1,:a2,:a3)>

Instead of the actual/literal values instead of the bind variables (if you recall streams apply error table had the complete SQL with actual values).

Hope it calrifies.

Thanks.

Connor McDonald
May 16, 2023 - 2:45 am UTC

Sorry, I should have been more precise. I meant are you using the MAP , TARGET , EXCEPTIONSONLY to capture failed rows.

Then your exception table should contain the failed rows.

GG exceptions macro details

A reader, May 16, 2023 - 6:13 am UTC

Thank you Connor.

  CREATE TABLE "GGREP_EXCEPTIONS"
   (    "REP_NAME" VARCHAR2(8),
        "TABLE_NAME" VARCHAR2(61),
        "ERRNO" NUMBER,
        "DBERRMSG" VARCHAR2(4000),
        "OPTYPE" VARCHAR2(20),
        "ERRTYPE" VARCHAR2(20),
        "LOGRBA" NUMBER,
        "LOGPOSITION" NUMBER,
        "COMMITTIMESTAMP" TIMESTAMP (6)
   );

I have the following in my replocat param file:

--Start of the Macro
MACRO #dml_exception_handler
BEGIN
, TARGET C##GGSUSER.GGREP_EXCEPTIONS
, colmap ( rep_name = 'GGREP'
, table_name = @GETENV ('GGHEADER', 'TABLENAME')
, errno = @GETENV ('LASTERR', 'DBERRNUM')
, dberrmsg = @GETENV ( 'LASTERR', 'DBERRMSG')
, optype = @GETENV ( 'LASTERR', 'OPTYPE')
, errtype = @GETENV ( 'LASTERR', 'ERRTYPE')
, logrba = @GETENV ( 'GGHEADER', 'LOGRBA')
, logposition = @GETENV ( 'GGHEADER', 'LOGPOSITION')
, committimestamp = @GETENV ( 'GGHEADER', 'COMMITTIMESTAMP') )
, INSERTALLRECORDS
, EXCEPTIONSONLY;
END;
--End of the Macro

DBOPTIONS SETTAG 00
DISCARDFILE GGREP.dsc, APPEND, MEGABYTES 500
DDLERROR DEFAULT DISCARD

DDL INCLUDE MAPPED &
EXCLUDE OBJNAME TGTDB.C##GGSUSER.GGREP_EXCEPTIONS
REPERROR (DEFAULT, EXCEPTION)

MAP SRCDB.SRCSCHEMA.TBGGWITHDATA, TARGET TGTDB.TGTSCHEMA.TBGGWITHDATA;
MAP SRCDB.SRCSCHEMA.TBGGWITHDATA #dml_exception_handler();
MAP SRCDB.SRCSCHEMA.TBGGNODATA, TARGET TGTDB.TGTSCHEMA.TBGGNODATA;
MAP SRCDB.SRCSCHEMA.TBGGNODATA #dml_exception_handler();
MAP SRCDB.SRCSCHEMA.TBNEWTAB, TARGET TGTDB.TGTSCHEMA.TBNEWTAB;
MAP SRCDB.SRCSCHEMA.TBNEWTAB #dml_exception_handler();
MAP SRCDB.SRCSCHEMA.TBBIGTAB, TARGET TGTDB.TGTSCHEMA.TBBIGTAB;
MAP SRCDB.SRCSCHEMA.TBBIGTAB #dml_exception_handler();

Probably you can shed light if I need to change something to capture the SQLs with the actual values for the failed DMLs.


More to Explore

Backup/Recovery

Check out the complete guide to all of the Backup & Recovery techniques in the Oracle Database.