Oracle GoldenGate Best Practice Extract ASM Connection Methods

4 January 2011
GoldenGate Center of Excellence

Table of Contents OVERVIEW .................................................................................................................................. 1 ORACLE ASM .............................................................................................................................. 2 ORACLE ASM: SAME AUTOMATED............................................................................................. 2 UNLOCKING ASM ...................................................................................................................... 2 USING ORACLE GOLDENGATE IN AN ORACLE ASM ENVIRONMENT ..................... 3 CAPTURING DATABASE TRANSACTIONS VIA ASM USER ACCESS.............................................. 3 ENABLING REMOTE ACCESS INTO ORACLE ASM ............................................................................ 3 USING A PASSWORD FILE ................................................................................................................. 3 CONFIGURING THE LISTENER........................................................................................................... 4 ADDING THE ASM INSTANCE TO THE TNSNAMES.ORA FILE ............................................................ 5 ADDITIONAL PARAMETERS FOR THE GOLDENGATE CAPTURE PROCESS ........................................ 5 BEQ PROTOCOL CONNECTIONS ................................................................................................... 6 ADDING THE BEQ ASM INSTANCE TO THE TNSNAMES.ORA FILE ................................................... 6 CAPTURING DATABASE TRANSACTIONS VIA THE DBLOGREADER API ................................ 7 BEQ PROTOCOL CONNECTIONS ................................................................................................... 8 ADDING THE BEQ ORCL INSTANCE TO THE TNSNAMES.ORA FILE ................................................. 8

Overview
This document discusses the GoldenGate configuration for Oracle Automatic Storage Management (ASM). Oracle ASM was introduced with Oracle Database 10g and is recommended by Oracle as a best practice storage solution for database files. Oracle ASM was developed to optimize database performance and high availability while minimizing storage maintenance. With its real-time, transactional change data capture, transformation and delivery capabilities, Oracle GoldenGate supports Oracle ASM on all platforms directly to enable high availability and real-time data integration solutions. This document provides information on how to configure Oracle GoldenGate Change Data Capture for ASM.

Disclaimer: The examples and scripts presented in this document are for demonstration purposes only and were created in a controlled laboratory environment. Before attempting to perform this procedure in a customer production situation we strongly recommend that you walkthrough the procedures and test, test, test!

-1-

Oracle ASM
Oracle ASM: SAME automated Oracle ASM provides out-of-the-box striping capabilities for Oracle database files, including data files, temporary tablespace files, redo log files, flash recovery area (i.e. archive log files, backup files), and the spfile. Optionally ASM can be used to implement mirroring. ASM will stripe the database storage across all available storage devices using a stripe size of 1 MB for most database files. As opposed to other software-based RAID solutions ASM will automatically maintain the striping (and mirroring provided by ASM) in case of changes to the storage configuration. ASM supports a wide variety of storage solutions, including those that do not normally support storage RAID configurations. ASM is a cluster file system so it can also be used as storage solution for an Oracle Real Application Clusters (RAC) database. ASM is a software-based solution that may be used to implement RAID-like site-level storage availability between disk arrays, which cannot be accomplished using storagebased RAID. ASM may be used to define so-called failure groups, and with ASM mirroring enabled, ASM will store every piece of data in different failure groups. With failure groups per disk array, the database could survive an entire storage array outage simply because all data would still be stored on a different disk array. Unlocking ASM ASM is not a general purpose file system; files in ASM file systems cannot be accessed like they can on a traditional file system. The value of ASM comes from automating the restriping across storage systems thus ensuring optimum performance. When configuration changes occur, maintaining file copies outside of ASM will require manual, cumbersome intervention. In addition, the weakest performing storage system will set the performance level of the overall system. Thus, keeping some of the files outside of the ASM system may slow down the performance. Oracle GoldenGate directly accesses redo and archive log files in Oracle ASM for transactional and heterogeneous replication to other environments to implement realtime data integration and high availability solutions.

-2-

Using Oracle GoldenGate in an Oracle ASM Environment

GoldenGate connects to the ASM instance through a database listener. For Oracle Database 10gR2 versions earlier than 10.2.0.5, and Oracle Database 11gR2 versions earlier than 11.2.0.2, GoldenGate accesses the database via the SYS ASM user over a SQLNET connection. For Oracle Database 10gR2 versions 10.2.0.5 and above, and Oracle Database 11gR2 versions 11.2.0.2 and above, the DBLOGREADER API may be enabled to access the database directly; providing improved data capture performance. For high volume Oracle 10gR2 and above databases having GoldenGate connect using the Oracle BEQ protocol, commonly referred to as bequeath, has shown data capture performance improvements over a traditional Listener. However, to use the BEQ protocol, the GoldenGate Capture process must run on the database server. Capturing Database Transactions via ASM User Access Enabling remote access into Oracle ASM Oracle ASM is managed by a set of processes called an Oracle ASM instance. In comparing an ASM instance to a database instance; one can see similar processes for an ASM instance, as for a database instance at the operating system level. Similarly, ASM can be implemented in a cluster configuration with multiple instances accessing a single ASM storage system (compare with a RAC database in which multiple database instances access a single database). An Oracle ASM instance can be accessed remotely through a database listener. Connections to an Oracle ASM instance may only be made as a user with the SYSDBA privilege. The default ASM user is SYS. A major difference with ASM, as compared to a database instance, is that users cannot be added to an ASM instance. However a different username with the SYSDBA privilege may be specified through the password file. Using a password file The use of a password file is mandatory to connect remotely to an instance using a SYSDBA privilege. The first step is to find out whether the ASM instance is configured to make use of a password file. Connect to the ASM instance (using SYS (or /) as SYSDBA) and run the following query: select value from v$parameter where name = remote_login_passwordfile; There are three possible outcomes for this query: 1. NONE a. The instance will not make use of a password file if there is one. 2. EXCLUSIVE: a. The password file is only used for this ASM instance. 3. SHARED:

-3-

a. The password file may be used by other ASM or database instances. If there is no password file, then the ASM instance will behave as if the setting for remote_login_passwordfile is NONE. If remote_login_passwordfile is set to NONE, set it to EXCLUSIVE or SHARED using an alter system command or by editing the init+ASM.ora file (followed by a restart of the ASM instance). Next make sure a password file was generated for the ASM instance, and if it wasnt, generate one. More information on how to configure and generate a password file is in the Oracle Database documentation. For GoldenGate to connect to the ASM instance with a username other than the SYS username and password a SHARED password file is required. Configuring the listener The ASM instance must be explicitly added to the listener configuration file to enable connections. For example, in order to be able to connect to the ASM instance5. In the SID descriptions add a SID for ASM as shown below: SID_LIST_LISTENER = (SID_LIST = (SID_DESC = (SID_NAME = PLSExtProc) (ORACLE_HOME = /app/oracle/ora102) (PROGRAM = extproc) ) (SID_DESC = (ORACLE_HOME = /app/oracle/ora102) (SID_NAME = +ASM) ) ) LISTENER = (DESCRIPTION_LIST = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP) (HOST = MVDW-LT01) (PORT = 1521) ) (ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC0)) ) ) The listener must be restarted to enable this change. As an option to adding the ASM instance to the regular listener configuration, a separate listener may be created for the ASM instance.

-4-

Adding the ASM instance to the tnsnames.ora file Finally in order to enable remote connections add the ASM instance to the tnsnames.ora file (or LDAP directory). For example: ASM = (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP) (HOST = MVDW-LT-01) (PORT = 1521) ) (CONNECT_DATA = (SERVER = DEDICATED) (SID = +ASM) ) ) Additional parameters for the GoldenGate Capture Process The GoldenGate Change Data Capture parameter file has to include one extra line to connect to the ASM instance. All other parameters, including the database connection, remain the same. Include the connection to the ASM instance in the beginning of the Capture parameter file (all on a single line):
TRANLOGOPTIONS ASMUSER <user>@<ASM_instance>, ASMPASSWORD <password>

Replace the portions with the angle brackets with the relevant settings for the ASM environment. The ASM password may be encrypted via the GGSCI command encrypt password <password to be encrypted>. Refer to the GoldenGate Administrator Guide for details on how to use encryption. With this parameter option enabled, the Change Data Capture (Extract) parameter file will look similar to this: EXTRACT EASM EXTTRAIL ./dirdat/ea USERID gguser, PASSWORD Oracle1 TRANLOGOPTIONS ASMUSER SYS@ASM, ASMPASSWORD Oracle1 TABLE amer.*; Items to consider when using this configuration: The Change Data Capture process uses a read buffer with a maximum size of 28,672 bytes. This may be problematic for high volume environments with large database transactions due to the limited memory buffer allocated for reading from the ASM storage. This connection method may also be used when Oracle GoldenGate Change Data Capture is running on a server remote from the database; however, data capture performance will be reduced significantly.

-5-

BEQ Protocol Connections BEQ protocol, or bequeath, connections are dedicated links into the database. Since our Change Data Capture process runs directly on the database server, using a bequeath connection in high volume environments will improve overall data capture rates because the overhead associated with the TCP/IP Listener connection is eliminated. Adding the BEQ ASM instance to the tnsnames.ora file To enable bequeath connections add the ASM instance to the tnsnames.ora file. For example: ASM_BEQ = (DESCRIPTION = (ADDRESS = (PROTOCOL = BEQ) (PROGRAM = /u01/app/oracle/product/11.2.0/asm/bin/oracle) (ARGV0 = oracle+ASM1) (ARGS = '(DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=BEQ)))') (ENVS = 'ORACLE_HOME=/u01/app/oracle/product/11.2.0/asm,ORACLE_SID=+ASM1') ) (CONNECT_DATA = (SERVICE_NAME = +ASM) (INSTANCE_NAME = +ASM1) ) ) The Change Data Capture configuration for the BEQ connection will look similar to this: EXTRACT EASM EXTTRAIL ./dirdat/ea USERID gguser, PASSWORD Oracle1 TRANLOGOPTIONS ASMUSER SYS@ASM_BEQ, ASMPASSWORD Oracle1 TABLE amer.*;

-6-

Capturing Database Transactions via the DBLOGREADER API For Oracle Database 10gR2 versions 10.2.0.5 and above, and Oracle Database 11gR2 versions 11.2.0.2 and above, with Oracle GoldenGate version 11 and above, the DBLOGREADER API may be enabled to access the redo and archive logs via the database server. It is very important that the Oracle Database patch levels be verified prior to enabling DBLOGREADER. Enabling DBLOGREADER on an incorrectly patched database will result in Change Data Capture failure and possible data loss. When capturing from an ASM instance via the ASMUSER option, the maximum amount of data read by Capture is 28,672 bytes. With a default read and write buffer size of 1,024,000 bytes DBLOGREADER provides the capability to capture large database transactions more efficiently. To enable the DBLOGREADER API, grant the GoldenGate Oracle Database user select any transaction permission, remove the ASMUSER login information from the Change Data Capture parameter file, and add the following:
TRANLOGOPTIONS DBLOGREADER

With this parameter option enabled, the Change Data Capture parameter file will look similar to this: EXTRACT EASM EXTTRAIL ./dirdat/ea USERID gguser, PASSWORD Oracle1 TRANLOGOPTIONS DBLOGREADER TABLE amer.*; DBLOGREADER provides options for tuning the change data capture process in high volume, large transaction database environments.
TRANLOGOPTIONS BUFSIZE <buffer size> Controls the size of the Change Data capture internal buffer that holds the results of each redo/archive read. The value of this buffer may be between 8,192 and 10,000,000 bytes; with a default value of 1,024,000. TRANLOGOPTIONS DBLOGREADERBUFSIZE <buffer size> Controls the size of the buffer used by DBLOGREADER to hold data captured from the ASM instance. The value of this buffer may be between 8,192 and 4,096,000 bytes; with a default value of 1,024,000 bytes. The value of BUFSIZE must be equal to, or greater than the DBLOGREADERBUFSIZE setting.

With these parameter options enabled, the Change Data Capture parameter file will look similar to this:

-7-

EXTRACT EASM EXTTRAIL ./dirdat/ea USERID gguser, PASSWORD Oracle1 TRANLOGOPTIONS BUFSIZE 2048000 TRANLOGOPTIONS DBLOGREADER, DBLOGREADERBUFSIZE 2048000 TABLE amer.*; BEQ Protocol Connections BEQ protocol, or bequeath, connections are dedicated links into the database. Since our Change Data Capture process runs directly on the database server, using a bequeath connection in high volume environments will improve overall data capture rates because the overhead associated with the TCP/IP Listener connection is eliminated. Adding the BEQ ORCL instance to the tnsnames.ora file To enable bequeath connections for DBLOGREADER add the ORCL instance to the tnsnames.ora file. For example: ORCL_BEQ = (DESCRIPTION = (ADDRESS = (PROTOCOL = BEQ) (PROGRAM=oracle) (ARGV0 = oracleORCL) (ARGS=(DESCRIPTION=(LOCAL=YES)(ADDRESS =(PROTOCOL=BEQ)))) ) (CONNECT_DATA = (SERVICE_NAME = ORCL)) ) The Change Data Capture configuration for the BEQ connection will look similar to this: EXTRACT EASM EXTTRAIL ./dirdat/ea USERID gguser@ORCL_BEQ, PASSWORD Oracle1 TRANLOGOPTIONS BUFSIZE 4096000 TRANLOGOPTIONS DBLOGREADER, DBLOGREADERBUFSIZE 4096000 TABLE amer.*;

-8-