Configuration of Bi-directional Oracle 11gR2 GoldenGate Replication

1. Overview of GoldenGate Installation

The following outlines the intended setup for this configuration Table 1. Server Information
Source source Oracle Enterprise Linux 6.1 /ggd/goldengate_aad/ggs PMGAAPOC_SRC/PMGAAPOC_SRC@ pmgdrgg 11.2.0.3 11.1.1.1 Destination target Oracle Enterprise Linux 6.1 /ggd/goldengate_aad/ggs PMGAAPOC_TGT/PMGAAPOC_TGT@ pmgdev 11.2.0.4 11.1.1.1

Hostname OS GoldenGate Home Oracle Instance Oracle Version GoldenGate version

Table 2. GoldenGate Processes Information

Source Extract DataPump Replicat Trails GoldenGate Admin User schemas extsrc extpmp REPSRC ./dirdat/se ./dirdat/tt ggs_aad pmgdrgg exttgt tgtpmp REPTGT ./dirdat/te ./dirdat/st ggs_aad pmgdev Destination

2. Supplemental logging & TRANDATA for Golden Gate:

2.1 Supplemental logging: One of the main reasons to enable supplemental logging is to generate logs for update statement. As insert and delete involves change to whole row (either addition or deletion of complete row) and all columns change in it whereas update can happen on a column of a row. Supplemental logging makes sure that enough information is captured during this update that can be used by any process (GG). SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA; --Verify that it is enabled. SQL> SELECT SUPPLEMENTAL_LOG_DATA_MIN FROM V$DATABASE; SUPPLEMENTAL_LOG_DATA_MIN -------------------------< YES 2.2 Trandata: By default, Oracle only logs changed columns for update operations. Normally,this means that primary key columns are not logged during an update operation.However, Replicat requires the primary key columns in order to apply the update on the target system. The ADD TRANDATA command in GGSCI is used to cause Oracle to log primary key columns for all updates For Golden gate replication minimum level of supplemental logging require is at primary key level to uniquely identify each row for replication purpose. To add TRANDATA through GoldenGate: Run the following in source as Oracle user to replicate from: SOURCE / SOURCE> select 'ADD TRANDATA pmgdrgg.'||tname from tab; Results running in GGSCI for all tables in schema: GGSCI> ADD TRANDATA pmgdrgg.PMG_CARRIER_INFO GGSCI> ADD TRANDATA pmgdrgg.PMG_AGGREGATOR_INGO ... GGSCI> Logging of supplemental redo log data is enabled for table pmgdrgg.PMG_CARRIER_INFO GGSCI> Logging of supplemental redo log data is enabled for table ADD TRANDATA pmgdrgg.PMG_AGGREGATOR_INGO

This has to be done for all tables in schema.

3. Configuration of GoldenGate Change Data Capture Processes for Source to Target Replication
The following section describes GoldenGate extract, replicat and datapump configuration for the Source to Target replication. 3.1.1 Add Checkpoint Table: GGSCI (source) 3> ADD CHECKPOINTTABLE ggs_add.checkpoint Successfully created checkpoint table ggs_add.checkpoint. 3.1.2 Creating parameter files You will be working with Oracle GoldenGate parameter files throughout the deployment process. To create and edit a parameter file, use the EDIT PARAMS command in GGSCI at any. EDIT PARAMS <name> 3.1.2.1 Manager process: The Manager process must be running on both the source as well as target systems before the Extract or Replicate process can be started and performs a number of functions including monitoring and starting other GoldenGate processes, managing the trail files and also reporting. The only mandatory parameter that we need to specify is the PORT which defines the port on the local system where the manager process is running. The default port is 7809 and we can either specify the default port or some other port provided the port is available and not restricted in any way. The USERID and PASSWORD parameter and required if you enable GoldenGate DDL support and this is the Oracle user account that we created for the Manager (and Extract/Replicat). The DYNAMICPORTLIST parameter to specify the ports that Manager can dynamically allocate for dynamic communications between the source and target systems, such as to the Collector and Replicat processes and to GGSCI sessions. You can specify an individual port, a range of ports, or both. To control how long Manager waits before attempting to reassign a port, use the DYNAMICPORTREASSIGNDELAY parameter.

Create the Manager at Source (steamer04) : a) Go to the goldengate home directory : cd $GG_HOME (/ggd/goldengate_aad/ggs) steamer04:/home/synggaad> cd $GG_HOME steamer04:/goldengate_aad/ggs> pwd /ggd/goldengate_aad/ggs b) Enter into the goldengate system command interface (GGSCI) by executing ggsci command steamer04:/goldengate_aad/ggs> ./ggsci Oracle GoldenGate Command Interpreter for Oracle Version 11.2.1.0.16 17781910 OGGCORE_11.2.1.0.0OGGBP_PLATFORMS_131205.0902 AIX 5L, ppc, 64bit (optimized), Oracle 11g on Dec 6 2013 11:11:29 Copyright (C) 1995, 2013, Oracle and/or its affiliates. All rights reserved. c) Create the manager parameter file GGSCI 2> EDIT

PARAMS MGR

d) Add the following script statements to the manager: PORT 9809 DYNAMICPORTREASSIGNDELAY 5 DYNAMICPORTLIST 9800-9900 USERID ggs_aad@pmgdrgg, PASSWORD ggs_aad
The manager can be stopped and started via the GSSCI commands MANAGER and STOP MANAGER . GGSCI (steamer04) 5> start mgr Manager started.

START

GGSCI (steamer04) 4> stop manager

Manager process is required by other GGS processes. Are you sure you want to stop it (y/n)? y Sending STOP request to MANAGER ... Request processed. Manager stopped. Information on the status of the Manager can be obtained via the GGSCI (steamer04) 6> info manager Manager is running (IP port steamer04.9809).

INFO MANAGER command

Create the Manager at Target (rhel-db03) : a) Go to the goldengate home directory : cd $GG_HOME ( /ggd/goldengate_aad/ggs) rhel-db03:/home/synggaad> cd $GG_HOME rhel-db03:/ggd/goldengate_aad/ggs> pwd /ggd/goldengate_aad/ggs b) Enter into the goldengate system command interface (GGSCI) by executing ggsci command rhel-db03:/ggd/goldengate_aad/ggs> ./ggsci Oracle GoldenGate Command Interpreter for Oracle Version 11.2.1.0.16 17781910 OGGCORE_11.2.1.0.0OGGBP_PLATFORMS_131205.0902_FBO Linux, x64, 64bit (optimized), Oracle 11g on Dec 5 2013 13:29:00 Copyright (C) 1995, 2013, Oracle and/or its affiliates. All rights reserved. c) Create the manager parameter file GGSCI 2> EDIT

PARAMS MGR

Add the following script statements to the manager: PORT 8809 DYNAMICPORTREASSIGNDELAY 5 DYNAMICPORTLIST 8800-8900 USERID ggs_aad@pmgdev, PASSWORD ggs_aad

d) Start the manager

GGSCI 2> START MGR Manager Started

3.1.2.2 Extract process: The Extract process runs on the source system and is the data capture mechanism of GoldenGate. It can be configured both for initial loading of the source data as well as to synchronize the changed data on the source with the target. This can be configured to also propagate any DDL changes on those databases where DDL change support is available.

To Create the Extract:

a) Create the extract parameter file GGSCI (steamer04) 8> EDIT PARAM EXTSRC b) Add the below statements EXTRACT extsrc EXTTRAIL ./dirdat/se USERID GGS_AAD@pmgdrgg PASSWORD ggs_aad TRANLOGOPTIONS DBLOGREADER RAWDEVICEOFFSET 0 TRANLOGOPTIONS EXCLUDEUSER ggs_aad TABLE PMGAAPOC_SRC.PMG_SHORT_CODE_INFO, GETBEFORECOLS (ON UPDATE ALL ); c) Add the Extract to the manager GGSCI (steamer04) 8> ADD extract extsrc, tranlog BEGIN NOW EXTRACT added. d) Create trail : create a trail note that this path pertains to the GoldenGate software location on the target system and this is where the trail files will be created having a prefix se which will be used by the Replicat process also running on the target system GGSCI (steamer04) 8> ADD exttrail ./dirdat/se, extract extsrc, megabytes 200 RMTTRAIL added.

e) Start the Extract GGSCI (steamer04) 8> START extract extsrc

3.1.2.4 Data pump process: Data Pumps are secondary extract mechanisms which exist in the source configuration. This is optional component and if Data Pump is not used then Extract sends data via TCP/IP to the remote trail on the target. When Data Pump is configured, the the Primary Extract process will write to the Local Trail and then this trail is read by the Data Pump and data is sent over the network to Remote Trails on the target system.

In the absence of Data Pump, the data that the Extract process extracts resides in memory alone and there is no storage of this data anywhere on the source system. In case of network of target failures, there could be cases where the primary extract process can abort or abend. Data Pump can also be useful in those cases where we are doing complex filtering and transformation of data as well as when we are consolidating data from many sources to a central target. Create Data Pump: a) Create the replicat parameter file GGSCI (steamer04) 8> EDIT PARAM EXTPMP b) Add the below statements RMTHOST 10.2.1.8, MGRPORT 8809, COMPRESS RMTTRAIL ./dirdat/tt PASSTHRU TABLE PMGAAPOC_SRC.PMG_SHORT_CODE_INFO; c) Add the data pump to manager GGSCI (steamer04) 8> ADD extract extpmp, exttrailsource, ./dirdat/se d) Add the remote trails GGSCI (steamer04) 8> ADD rmttrail ./dirdat/tt, extract extpmp, megabytes 200

e) Start the data pump

GGSCI (steamer04) 8> START extract extpmp 3.1.2.5 Replicat process: The Replicat process runs on the target system and reads transactional data changes as well as DDL changes and replicates then to the target database. Like the Extract process, the Replicat process can also be configured for Initial Load as well as Change Synchronization.

Create Replicat : a) Create the replicat parameter file at target system GGSCI (steamer04) 8> EDIT PARAM REPTGT b) Add the below statements REPLICAT REPTGT ASSUMETARGETDEFS DISCARDFILE ./dirrpt/tgtrep.dsc, PURGE USERID ggs_aad@pmgdev PASSWORD ggs_aad REPORTCOUNT EVERY 30 MINUTES, RATE REPORT AT 00:00 REPORTROLLOVER AT 00:01 EOFDELAY 120 MAP PMGAAPOC_SRC.PMG_SHORT_CODE_INFO, TARGET PMGAAPOC_TGT.PMG_SHORT_CODE_INFO, RESOLVECONFLICT (UPDATEROWEXISTS,(DEFAULT, USEMAX(UPDATEDAT))), COMPARECOLS ( ON UPDATE, KEYINCLUDING(UPDATEDAT)); c) Add the Replicat to manager GGSCI (steamer04) 9 > ADD replicat REPTGT, exttrail ./dirdat/tt, checkpointtable ggs_aad.checkpoint

d) Start replicate GGSCI (steamer04) 10 > START REPLICAT REPTGT

4. Configuration of GG Change Data Capture Processes for Target to Source Replication (Active Active)
So far we have configured the source to target replication means one way replication. By adding the below configuration at source and target, we can achieve the Active-Active replication i.e. two way replication. Here we are going to configure the one Extract and Data pump at target system (rhel-db03) and one Replicat process at source system (steamer04) . 4.1.1 Add Checkpoint Table

GGSCI (rhel-db03) 4> ADD CHECKPOINTTABLE ggs_add.checkpoint Successfully created checkpoint table ggs_add.checkpoint. 4.1.2 Creating parameter files 4.1.2.1 Manager process: We already created the manager process in the above section. 4.1.2.2 Extract process at target system: a) Create the extract parameter file GGSCI (rhel-db03) 4> EDIT PARAM EXTTGT b) Add the below statements EXTRACT exttgt EXTTRAIL ./dirdat/te USERID ggs_aad@pmgdev PASSWORD ggs_aad TRANLOGOPTIONS DBLOGREADER RAWDEVICEOFFSET 0 TRANLOGOPTIONS EXCLUDEUSER ggs_aad c) Add the extract process GGSCI (rhel-db03) 5> ADD extract exttgt, threads 2 ,tranlog BEGIN NOW d) Add the trail GGSCI (rhel-db03) 6> ADD exttrail ./dirdat/te, extract exttgt, megabytes 200

e) Start the extract GGSCI (rhel-db03) 7> START EXTRACT exttgt 4.1.2.3 Data pump process at target system:

a) Create the replicat parameter file GGSCI (rhel-db03) 7> EDIT PARAM tgtpmp b) Add the below statements RMTHOST 10.2.1.8, MGRPORT 8809, COMPRESS RMTTRAIL ./dirdat/st PASSTHRU TABLE PMGAAPOC_SRC.PMG_SHORT_CODE_INFO; c) Add the data pump to manager GGSCI (rhel-db03) 8> ADD extract tgtpmp, exttrailsource, ./dirdat/te d) Add the remote trails GGSCI (rhel-db03) 9> ADD rmttrail ./dirdat/st, extract tgtpmp, megabytes 200

e) Start the data pump GGSCI (rhel-db03) 10> START extract tgtpmp 4.1.2.4 Replicat process at Source system: a) Create the replicat parameter file at source system GGSCI (steamer04) 8> EDIT PARAM REPSRC b) Add the below statements REPLICAT REPSRC ASSUMETARGETDEFS

DISCARDFILE ./dirrpt/tgtrep.dsc, PURGE USERID ggs_aad@pmgdrgg PASSWORD ggs_aad REPORTCOUNT EVERY 30 MINUTES, RATE REPORT AT 00:00 REPORTROLLOVER AT 00:01 MAP PMGAAPOC_TGT.PMG_SHORT_CODE_INFO, TARGET PMGAAPOC_SRC.PMG_SHORT_CODE_INFO, RESOLVECONFLICT (UPDATEROWEXISTS, (DEFAULT, USEMAX(UPDATEDAT))), COMPARECOLS ( ON UPDATE, KEYINCLUDING(UPDATEDAT)); c) Add the Replicat to manager GGSCI (steamer04) 9 > ADD replicat REPSRC, exttrail ./dirdat/st, checkpointtable ggs_aad.checkpoint d) Start replicate GGSCI (steamer04) 10 > START REPLICAT REPSRC

5. Testing Active-Active Replication

Till now we have configured the Active-Active replication on PMG_SHORT_CODE_INGO table, now we are going to test the replication, Pre-requisites: Manager should be running in both source and target system Source: GGSCI (steamer04) 3> status mgr Manager is running (IP port steamer04.9809). Target : GGSCI (rhel-db03) 17> status mgr Manager is running (IP port rhel-db03.8809). Extract , Data pump and replicat s should be running in both systems Source : GGSCI (steamer04) 4> info all Program MANAGER EXTRACT EXTRACT REPLICAT Target : GGSCI (rhel-db03) 21> info all Program MANAGER EXTRACT EXTRACT Status Group Lag at Chkpt Time Since Chkpt Status Group Lag at Chkpt Time Since Chkpt

RUNNING RUNNING RUNNING RUNNING EXTPMP EXTSRC REPSRC 00:00:00 00:00:00 00:00:00 00:00:08 00:00:01 00:00:08

RUNNING RUNNING RUNNING EXTTGT TGTPMP 07:00:03 00:00:00 42:56:53 00:00:08

REPLICAT Replication Testing:

RUNNING

REPTGT

00:00:00

00:00:13

A) Source to Target Replication i) Insert & commit a row at source system steamer04:/home/synggaad> sqlplus PMGAAPOC_SRC/PMGAAPOC_SRC@pmgdrgg SQL*Plus: Release 11.2.0.3.0 Production on Wed Mar 12 06:50:26 2014 Copyright (c) 1982, 2011, Oracle. All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Partitioning, OLAP, Data Mining and Real Application Testing options SQL > Insert into pmg_short_code_info (SHORTCODEID,SHORTCODE,AGGREGATORID,CONTENT_PROVIDERID,CAMPAI GNID,RESELLERID,SHORTCODE_STATUS,STARTDATE,ENDDATE,SHORTCODE_D ESC,PURPOSE,TYPE,USECASE,TRAFFIC_PROJECTION,TEST_NUMBERS,CONTEN T_URL,INSERTEDBY,INSERTEDAT,UPDATEDBY,UPDATEDAT,ACTIVE,EXCLUSIVE_ RESELLER_CODE_ID,CARRIER,MSGTYPE,BIND_TYPE,IS_NEW_RECORD,STATUS _CHANGED,ISPREMIUM) values (SHORTCODEID.nextval,'44446',1001,1018,1205,304,2,to_timestamp('17-FEB-05','DDMON-RR HH.MI.SSXFF AM'),to_timestamp('30-JUN-15','DD-MON-RR HH.MI.SSXFF AM'),'Interactive with on-air DJ, alerts, song requests, QA, voting',null,'S','Text <keyword> to 97373 Text STOP, QUIT, or END to 97373',null,null,null,'sadmin',to_timestamp('17JAN-06','DD-MON-RR HH.MI.SSXFF AM'),'SC_ENDDATE_EXTENTION',sysdate,0,0,'sprint','SMS',1001,0,null,0); 1 row inserted SQL> commit;

Commit complete. ii) execute the select query on the target system rhel-db03:/home/synggaad> sqlplus PMGAAPOC_TGT/PMGAAPOC_TGT@pmgdev SQL*Plus: Release 11.2.0.4.0 Production on Wed Mar 12 06:53:49 2014 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SQL> select count(*) from pmg_short_code_info where shortcode=44446; COUNT(*) ---------1 Note: We have introduced the 120 seconds delay at replicat, so row will be replicated after 120 seconds. B) Target to Source Replication i) Insert & commit a row at target system SQL > Insert into pmg_short_code_info (SHORTCODEID,SHORTCODE,AGGREGATORID,CONTENT_PROVIDERID,CAMPAI GNID,RESELLERID,SHORTCODE_STATUS,STARTDATE,ENDDATE,SHORTCODE_D ESC,PURPOSE,TYPE,USECASE,TRAFFIC_PROJECTION,TEST_NUMBERS,CONTEN T_URL,INSERTEDBY,INSERTEDAT,UPDATEDBY,UPDATEDAT,ACTIVE,EXCLUSIVE_ RESELLER_CODE_ID,CARRIER,MSGTYPE,BIND_TYPE,IS_NEW_RECORD,STATUS _CHANGED,ISPREMIUM) values (SHORTCODEID.nextval,'44447',1001,1018,1205,304,2,to_timestamp('17-FEB-05','DDMON-RR HH.MI.SSXFF AM'),to_timestamp('30-JUN-15','DD-MON-RR HH.MI.SSXFF AM'),'Interactive with on-air DJ, alerts, song requests, QA, voting',null,'S','Text <keyword> to 97373 Text STOP, QUIT, or END to 97373',null,null,null,'sadmin',to_timestamp('17JAN-06','DD-MON-RR HH.MI.SSXFF AM'),'SC_ENDDATE_EXTENTION',sysdate,0,0,'sprint','SMS',1001,0,null,0); 1 row inserted SQL> commit; Commit complete. ii) execute the select query on the source system SQL> select count(*) from pmg_short_code_info where shortcode=44446; COUNT(*) ---------1 Note: in this case row should be replicated immediately.