IBM Power and Oracle Real Applications Testing SRao 16JUL12

Oracle Real Application Testing
Database Replay
Surya Rao
Service Line Architect, IBM Senior Certified IT Specialist
IBM GTS, Service Delivery, Data Management
July 15, 2012
In Collaboration with the IBM Oracle International Competency Center
Copyright IBM Corporation, 2012. All Rights Reserved.

All trademarks or registered trademarks mentioned herein are the property of their respective holders
Table of contents
Abstract..................................................................................................................................... 1
Prerequisites............................................................................................................................. 1
1.0 Executive Summary............................................................................................................ 1
2.0 Introduction ........................................................................................................................ 2
2.1 Licensing Considerations ............................................................................................................. 3
3.0 Birds eye view of Workload Capture & Replay................................................................ 4

4.0 PRE CAPTURE CHECKLIST............................................................................................... 6
4.1 Limitations & Restrictions ............................................................................................................. 6
4.2 Check List .................................................................................................................................... 6
5.0 Capturing a Workload ........................................................................................................ 9

5.1 Creating a Capture Directory ........................................................................................................ 9
5.2 Capture Directory and its contents................................................................................................ 9
5.3 Filtering the capture workload......................................................................................................10
5.4 Start workload capture ................................................................................................................11
5.5 Completing the Capture process .................................................................................................12
5.6 Workload Capture Files ...............................................................................................................14
5.7 Workload Capture Report............................................................................................................15
5.8 Exporting AWR data....................................................................................................................16
5.9 Useful Views ...............................................................................................................................17
6.0 PRE REPLAY CHECKLIST ............................................................................................... 18

6.1 Pre-process the captured workload ........................................................................18
6.2 Prepare the Test System .......................................................................................19
7.0 Replaying a Workload ...................................................................................................... 21

7.1 Workload Analyzer ......................................................................................................................21
7.2 Calibrating the workload before replay.........................................................................................22
7.3 Initialize the Workload Replay......................................................................................................23
7.4 Connection Maps ........................................................................................................................23
7.5 Prepare the Workload Replay and check status...........................................................................24
7.6 Starting the Workload Replay Clients (wrc)..................................................................................26
7.7 Starting the Workload Replay ......................................................................................................27
7.8 Changing Workload Replay execution status ...............................................................................28
7.9 Replay information extraction ......................................................................................................29
7.10 Extracting the Replay report for a specific Workload ..................................................................29
7.11 Sample Replay Report ..............................................................................................................30
7.12 Exporting AWR data..................................................................................................................31
7.13 Useful Views .............................................................................................................................33
8.0 Data Divergence on Replay Sample Simulation ............................................................. 33

8.1 A Test case for demonstrating Data Divergence ..........................................................................34
8.2 Replaying the workload to cause a Data Divergence ...................................................................38
8.3 Checking for Data Divergences ..................................................................................................41
Oracle Real Application testing

http://www.ibm.com/support/techdocs
Copyright 2012, IBM Corporation
8.4 Extract the Diverging SQL statement ...........................................................................................42
9.0 Using Enterprise Manager for Real Application Testing................................................ 43

Summary................................................................................................................................. 45
Operating system..............................................................................................................................45
Storage System Management ...........................................................................................................45
Resources............................................................................................................................... 46
About the author..................................................................................................................... 46
Appendix................................................................................................................................. 47
1.0 Common Errors and Resolutions .................................................................................................47
2.0
Patch level for RAT......................................................................................................47
Trademarks and special notices ........................................................................................... 48
Oracle Real Application testing

Abstract
This white paper describes Oracle Real Application Testing product functionality. This is a
tool that captures workload including timing, concurrency and dependency information and
is able to run this captured load into an alternate environment. Real application Testing is
two tools. It is SQL Performance Analyzer where you perform SQL unit testing to identify
SQL regression. Database replay is what is described in this document. The target audience
is installations running Oracle Database on IBM hardware and IBM field specialist that
support those environments.
Prerequisites
The reader of this document is expected to have a general and basic knowledge of the Oracle Database
and IBM servers.
1.0 Executive Summary

Change is an inevitable part of the IT industry and it is almost always
associated with risks. To mitigate risks in production systems due to planned
changes, there are various approaches prevalent in the industry the most
prominent approach pertains to load testing with third party tools on nonproduction environments. Majority of these conventional tools rely on the
Test Scripters understanding of the Production workload and generate their
workloads based on certain inputs and parameters specified. However, none
of these Load Testing tools available today are able to generate an exact
workload of production and run them on non-production databases and
hence open to risks
This results in issues being faced in production (Post the change activity)
even after successfully completing the conventional load testing. To
drastically mitigate this risk and address the shortcomings prevalent in
conventional Load Testing tools, Oracle introduced the Real Application
Testing offering. This is the solution in the industry which captures an actual
production workload including timing, concurrency and dependency
information and is able to run this captured load into an alternate
environment. With this load testing, we are able to know the exact impact of
a prospective change and hence avoid unknown risks.
Below are some of change examples where customer will benefit with Real
Application Testing tools.

Database upgrades
Database Patches
Storage changes
Network Changes


Hardware upgrades
OS upgrades
Database configuration changes
Scale up testing and consolidation
Some of Key features of Real Application Testing tools

Web based interface
Able to access impact of complex change on complex production
environment from months to days
Able to capture load from prior releases and replay them only on 11g
upwards
SQL Performance Analyzer will analyze change on SQL execution plan
and performance
The Oracle Real Application Testing option includes the following features:

Database Replay
SQL Performance Analyzer (SPA)
SQL Tuning Sets (STS). This is also part of the Tuning Pack
SPA and STS are not in the scope of this white paper.
2.0 Introduction
Oracle Real Application Testing is a separately licensed option with Oracle
Database Enterprise Edition 11g. The purpose of this product is to allow
extraction of production workloads and simulate running them on preproduction or test databases to replicate a current performance issue for
deeper analysis and resolution on non production environments.
The Database Replay feature of the Real Application Testing suite can be
used to test any substantial changes such as Database upgrades, Storage or
network changes, Hardware or OS upgrades, Database configuration
changes etc.
It can also be used in Change Management related baselines and regression
testing scenarios where the behavior of the system has to be observed
based on a fixed and unchanging workload to ensure that the results are
comparable across test runs.
Note that although this feature was introduced in Oracle Database 11g, it is
possible to capture workloads from 10.2.0.4 databases but it is possible to
replay them only on Oracle Database 11g and upwards. We can also capture
workloads from 9.2.0.8, 10.2.0.2 & 10.2.0.3 provided that the RAT capture
patch is installed. This could be an excellent avenue for testing pre and post
upgrade environments, an area that is not very easy to cover at the
moment, in the absence of Real Application Testing
2.1 Licensing Considerations
The Oracle Real Application Testing option includes the following features:
Database Replay
SQL Performance Analyzer (SPA)
SQL Tuning Sets (STS)
In order to use the features listed in this whitepaper, it is required to

purchase licenses for the Oracle Real Application Testing option, with one
exception: SQL Tuning Sets (STS) can be used if you have licensed either
the Tuning Pack or Oracle Real Application Testing. Oracle Real Application
Testing functionality can be accessed from the Enterprise Manager as well as
through the database server command-line APIs. The use of either interface
requires an Oracle Real Application Testing option license. Additionally, the
Oracle Real Application Testing license is required on both the capture and
replay systems for Database Replay and is charged by the total number of
CPUs on those systems. In production Oracle charges according to the
license model that is used. On Test you can either choose between CPU or
NUP (No. of Processors). Refer to this link for more information on the
various licensing options

3.0 Birds eye view of Workload Capture & Replay

Oracle strongly recommends the usage of SPA to address SQL regression
analysis before using the Database Replay option.
The steps involved in using Oracle Real Application Testing can be
summarized as follows:
1.
2.
3.
4.
5.
6.
7.
8.
Workload Capture (On Source/Production database)

Workload Pre-processing (On the Target/Test database)
Prepare / Initialize the Target DB for the workload replay
Calibrate the workload to estimate number of workload replay clients
needed
Initiate the workload replay clients (wrc clients)
Start the Workload replay
Generate the Replay report
Analyze the reports


4.0 PRE CAPTURE CHECKLIST

4.1 Limitations & Restrictions
1. For versions prior to 11g, i.e. 10.2.0.x, and 9.2.0.8, there are mandatory patches to
be applied before workloads can be captured. These patches (applied via opatch
utility) install the required Real Application Testing functionality in the Pre-11g
releases.
2. For a source 10g R2 database, the minimum version for which capture is possible is
10.2.0.2 for non-Windows operating systems and 10.2.0.3 for Windows operating
systems.
3. Note that the captured workloads can only be replayed on versions 11.1 and above.
4. The entire list of patches for each source (capture) and target (replay) version is
mentioned in the My Oracle Support article 560977.1
5. The following types of client requests are not captured in a workload as
per the Oracle Documentation
* Direct path load of data from external files using utilities such as
SQL*Loader
* Non-PL/SQL based Advanced Queuing (AQ)
* Flashback queries
* Oracle Call Interface (OCI) based object navigations
* Non SQL-based object access
* Distributed transactions (any distributed transactions that are captured
will be replayed as local transactions). Distributed transactions using two
phase commit are not supported and will not be replayed as even local
transactions.
4.2 Check List

1. Ensure Oracle Home binary is at the correct patch level for Real
Application Testing capture. (see appendix )
2. Real Application Testing option must be installed
3. Activate the Real Application Testing option.

a. For Oracle Database 11g it is enabled by default but package

DBMS_WORKLOAD_CAPTURE must be valid. If this package is
absent, it can be created by running dbmswrr.sql.
b. For Oracle Database release 10.2.0.4 and 10.2.0.5 execute
wrrenbl.sql to set PRE_11G_ENABLE_CAPTURE.
4. Setting up a Database Replay Administrator
The SYS or SYSTEM account is not required to be used for the capture and replay steps
described above. Any user that is granted the below privileges can be used to administer
Capture and Replay
CREATE USER <USERNAME> IDENTIFIED BY oracle;

GRANT CREATE SESSION, RESOURCE to <USERNAME>;
GRANT EXECUTE ON DBMS_WORKLOAD_CAPTURE TO <USERNAME>;
GRANT EXECUTE ON DBMS_WORKLOAD_REPLAY TO <USERNAME>;
GRANT CREATE SESSION TO <USERNAME>;
GRANT CREATE ANY DIRECTORY TO <USERNAME>;
GRANT SELECT_CATALOG_ROLE TO <USERNAME>;
GRANT BECOME USER TO <USERNAME>;
GRANT UNLIMITED TABLESPACE TO <USERNAME>
5. Save the spfile / database initialization parameters and environment
specifics in order to replicate in replay environment. This is particularly
applicable for Oracle Real Application Clusters (RAC) environments.
6. Capture scripts should be run as SYS
7. A Database Directory Object has to be created for capture file. In a
RAC environment must be on shared disk storage. Ensure enough
capacity is available.
8. Identify workload period processing window based on the target
technical goal of completing the replay as outlined in introduction
section of this document.
9. Plan to capture the logical/physical state of the database to clone in
the test environment, has to be time synchronized with the start of the
capture window to avoid data divergence results on the test
environment.. Other alternative methods are the use of datapump or
snapshot standby.
10.
May need to consider restarting the database or commencing
capture in dormant periods to ensure transaction activity is at a
consistent point . This will lessen the volume of divergence events on
the Test environment when replayed.
11.
The recommendation is to put the database into restricted mode
to allow the capture process to be started . Once capture is started
then database is automatically taken out of restricted mode.In a

production environment that would limit client access for a short

period of time. Can be bypassed but at the
expense of transaction capture integrity ( auto_unrestrict => FALSE )

5.0 Capturing a Workload

Capturing a workload is performed either using the Oracle supplied PL/SQL
APIs or using Enterprise Manager. In this document, we have predominantly
shown examples using the Oracle supplied PL/SQL APIs.
5.1 Creating a Capture Directory

SQL> create directory CAPTURE_DIR as 'c:\app\capture';
Directory created.
5.2 Capture Directory and its contents

This is the directory where the metadata related to the workload capture will
be placed. Initially the directory and its sub-directories will either be empty
or not even exist.
SQL> !ls -l /home/oracle/capture
total 0
drwxr-xr-x
drwxr-xr-x
2 oracle
3 oracle
oinstall
oinstall
256 Oct 07 00:41 cap

256 Oct 07 00:24 capfiles
If the capture processing is initiated with a previous workload's captured

files still existing in the directory, the below error would be encountered.
ORA-15505: cannot start workload capture because instance 1
encountered errors while accessing directory
Reason:
There is an existing file in the CAPTURE Directory. There should be
no files including no hidden files, before the
dbms_workload_capture.start_capture is executed. Delete all
content in the capture directory and retry

The schema that is used to run the workload can belong to any user who has
the necessary privileges to perform the required DML, DDL against the
objects referred to in the workload SQL sets.
5.3 Filtering the capture workload

Filtering can be done on the following attributes. The default action is to
include all transactions in the replay. To include or exclude specific
workloads, the following filters can be applied before the Capture or during
the Replay
1.
2.
3.
4.
5.
6.
Instance_Number
User
Module
Program
Action
Service
The steps for the Workload capture are as demonstrated below.

The recommendation is to filter out monitoring activities like those
performed by Enterprise Manager.
Adding a Filter is basically performed to restrict and exclude unwanted
workloads from being captured. By default everything is captured in the
absence of the filters
BEGIN
DBMS_WORKLOAD_CAPTURE.ADD_FILTER (
fname => 'user_scott',
fattribute => 'USER',
fvalue => 'SCOTT');
fname => 'instance_filter',
fattribute => 'INSTANCE_NUMBER',
fvalue => '1' );
END;
/

10
If a filter that was previously added needs to be removed before capturing

the workload, the following command can be used to delete it.
BEGIN
DBMS_WORKLOAD_CAPTURE.DELETE_FILTER (fname => user_scott);
END;
/
To check the filters defined and their statuses i.e NEW or USED, use the
below step
SQL> select * from dba_workload_filters;

TYPE
ID STATUS NAME
ATTRIBUTE VALUE
--------------------------------------------------CAPTURE 0 NEW
USER_SCOTT
USER
SCOTT
5.4 Start workload capture

BEGIN
DBMS_WORKLOAD_CAPTURE.START_CAPTURE (
name => 'PEAK_PROCESSING_2',
dir => 'CAPTURE_DIR',
default_action=>'EXCLUDE',
duration => null, --means capture has to be ended manually
auto_unrestrict => FALSE
--if the above is TRUE all instances started up in RESTRICT mode will be
made
-- unrestricted when the capture ends
);
END;
/
PL/SQL procedure successfully completed.

11
To check for the captures that have been completed on this database or to query
the ones that are currently in progress, follow the step provided below
set lines 200
col status format a15
col name format a15
col dbversion format a12
SELECT ID, NAME, DBID,DBNAME,DBVERSION,
PARALLEL,STATUS, to_char(START_TIME,'dd/mm/yyyy hh24:mi'),
to_char(END_TIME,'dd/mm/yyyy hh24:mi'),DURATION_SECS
FROM DBA_WORKLOAD_CAPTURES;
Once capture complete, status should be completed.

SQL> select id, name, status,start_scn, end_scn, dbtime_total, connects,
dir_path from dba_workload_captures;
ID NAME
STATUS
START_SCN
END_SCN DBTIME_TOTAL
CONNECTS DIR_PATH
---------- -------------------- -------------- ---------- ---------- -1 PEAK_PROCESSING
COMPLETED
625372
628334
986970
1
0 c:\app\capture
3 PEAK_PROCESSING_2
IN PROGRESS
631001
715205
5
3 c:\app\capture
For RAC, capture is executed on one node only For 10.2.0.4

onwards. In earlier releases the capture will have to be
initiated on all the nodes.
5.5 Completing the Capture process

12
BEGIN
DBMS_WORKLOAD_CAPTURE.FINISH_CAPTURE ();
END;
/
set lines 200

col name format a15

13
5.6 Workload Capture Files

The following files are created in the designated capture directory. These
files are release dependent and may vary for older releases
SQL> !ls -l /home/oracle/capture/cap
total 104
-rw-r--r-1 oracle
oinstall
-rw-r--r-1 oracle
oinstall
-rw-r--r-1 oracle
oinstall
-rw-r--r-1 oracle
oinstall
30038
11338
268
171
Oct
Oct
Oct
Oct
SQL> !ls -l /home/oracle/capture/capfiles/inst1

total 16
drwxr-xr-x
2 oracle
oinstall
4096 Oct
drwxr-xr-x
2 oracle
oinstall
256 Oct
drwxr-xr-x
2 oracle
oinstall
256 Oct
drwxr-xr-x
2 oracle
oinstall
256 Oct
drwxr-xr-x
2 oracle
oinstall
256 Oct
drwxr-xr-x
2 oracle
oinstall
256 Oct
drwxr-xr-x
2 oracle
oinstall
256 Oct
drwxr-xr-x
2 oracle
oinstall
256 Oct
drwxr-xr-x
2 oracle
oinstall
256 Oct
drwxr-xr-x
2 oracle
oinstall
256 Oct
-rw-r--r-1 oracle
oinstall
143 Oct
wcr_8x37xh0000000.rec
07
07
07
07
00:41
00:41
00:40
00:24
wcr_cr.html
wcr_cr.text
wcr_fcapture.wmd
wcr_scapture.wmd
07
07
07
07
07
07
07
07
07
07
07
00:40
00:24
00:24
00:24
00:24
00:24
00:24
00:24
00:24
00:24
00:24
aa
ab
ac
ad
ae
af
ag
ah
ai
aj
NOTE - Once a set of filters are defined and used by starting and finishing a
capture, these filters are marked as USED in dba_workload_filters and can't
be used again.
The same filter names can be redefined and will show up as NEW in the
dba_workload_filters view.
TYPE
ID STATUS NAME
ATTRIBUTE
VALUE
------------------------------------------------------------CAPTURE
1 USED
USER_SCOTT
USER
SCOTT
CAPTURE
1 USED
INSTANCE_FILTER
INSTANCE_NUMBER 1
CAPTURE
0 NEW
USER_SCOTT
USER
SCOTT
CAPTURE
0 NEW
INSTANCE_FILTER
INSTANCE_NUMBER 1

14
5.7 Workload Capture Report

The workload capture report is very similar to an AWR report detailing most
aspects of the workload as shown below
SQL> set pagesize 0 long 30000000 longchunksize 2000

SQL> select dbms_workload_capture.report (capture_id =>1,format=>'TEXT') from dual;
Database Capture Report For ORCL

DB Name
DB Id
Release
RAC Capture Name
Status
------------ ----------- ----------- --- -------------------------- --------ORCL
1286621985 11.2.0.1.0 NO PEAK_PROCESSING_2
COMPLETED
Start time: 11-Aug-11 15:24:53 (SCN = 631001)
End time: 11-Aug-11 15:48:47 (SCN = 633971)
Duration: 23 minutes 54 seconds
Capture size: 15.12 KB
Directory object: CAPTURE_DIR
Directory path: c:\app\capture
Directory shared in RAC: TRUE
Filters used: 2 INCLUSION filters
Statistic Name
Value
% Total
---------------------------------------- ------------- --------DB time (secs)
5.89
79.59
Average Active Sessions
0.00
User calls captured
18
5.90
User calls captured with Errors
0
Session logins
3
3.61
Transactions
41
22.53
------------------------------------------------------------Top Events Captured
DB: ORCL Snaps: 78
Avg Active
Event
Event Class
% Event
Sessions
----------------------------------- --------------- ---------- ---------db file scattered read
User I/O
13.64
0.00
CPU + Wait for CPU
CPU
9.09
0.00
WCR: capture file IO write
Other
4.55
0.00
control file sequential read
System I/O
4.55
0.00
-------------------------------------------------------------

15
Sid, Serial# % Activity Event

% Event
--------------- ---------- ------------------------------ ---------User
Program
# Samples Active
XIDs
-------------------- ------------------------------ ------------------ ------6,
1
22.73 control file parallel write
13.64
SYS
ORACLE.EXE (CKPT)
3/1,434 [ 0%]
0
control file sequential read
9.09
2/1,434 [ 0%]
0
5,
9.09 db file parallel write

ORACLE.EXE (DBW0)
9.09
2/1,434 [ 0%]
21,
280
4.55 WCR: capture file IO write

ORACLE.EXE (J001)
4.55
1/1,434 [ 0%]
24,
28
4.55 WCR: capture file IO write

ORACLE.EXE (M001)
4.55
1/1,434 [ 0%]
58,
SYS
0
SYS
0
SYS
0
SYS
0
4.55 control file sequential read

4.55
ORACLE.EXE (ARC1)
1/1,434 [ 0%]
------------------------------------------------------------Workload Filters
DB: ORCL Snaps: 78
# Filter Name
Type
Attribute
Value
--- ------------------------ ------- ------------ -------------------------1 INSTANCE_FILTER
INCLUDE INSTANCE_NUM 1
2 USER_SCOTT
INCLUDE USER
SCOTT
-------------------------------------------------------------
5.8 Exporting AWR data

AWR data can be exported during the time of the capture as well as during the replays.
Exporting the workload capture AWR data is is recommended since it is used for replay
performance comparison.
At the end of each capture or replay, the relevant AWR snapshots for the period during which
the capture or replay was executing will be automatically exported and included in the captured
or replayed data. The export log would look something like the below.

16
. . exported "SYS"."WRH$_SYSMETRIC_HISTORY" 13.46 KB 100 rows

. . exported "SYS"."WRH$_SQL_PLAN" 6.355 MB 26787 rows
. . exported "SYS"."WRH$_SYSMETRIC_SUMMARY" 32.19 KB 296 rows
. . exported "SYS"."WRH$_ENQUEUE_STAT" 18.02 KB 174 rows
. . exported "SYS"."WRH$_WAITCLASSMETRIC_HISTORY" 9.429 KB 0 rows
These exported AWR snapshots can then be imported into another system into a specified and
empty schema. The commands to export AWR reports during Capture and Replay are provided
below
EXEC DBMS_WORKLOAD_CAPTURE.EXPORT_AWR (capture_id => 12);
EXEC DBMS_WORKLOAD_REPLAY.EXPORT_AWR (replay_id => 13);
5.9 Useful Views

This section summarizes the views that you can display to monitor workload replay. You need DBA
privileges to access these views.
The DBA_WORKLOAD_CAPTURES view lists all the workload captures that have been captured in
the current database.
The DBA_WORKLOAD_FILTERS view lists all workload filters for workload captures defined in the
current database.
The DBA_WORKLOAD_REPLAYS view lists all the workload replays that have been replayed in the
current database.
The DBA_WORKLOAD_REPLAY_DIVERGENCE view enables you to view information about
diverged calls, such as the replay identifier, stream identifier, and call counter.
The DBA_WORKLOAD_REPLAY_FILTER_SET view lists all workload filters for workload replays
defined in the current database.
The DBA_WORKLOAD_CONNECTION_MAP view lists the connection mapping information for
workload replay.
The V$WORKLOAD_REPLAY_THREAD view lists information about all sessions from the replay
clients.
The following Oracle supplied PL/SQL API packages contain the subprograms to completely manage and administer the Capture and Replay
processes
package
DBMS_WORKLOAD_REPLAY package
DBMS_WORKLOAD_CAPTURE

17
6.0 PRE REPLAY CHECKLIST

6.1 Pre-process the captured workload
The preprocessing should always be done on a system which has the same
DB version as the source DB where the workload was captured. This step
converts the captured data into replay files and also creates the necessary
metadata. This step is required to be done only once (for each version of
the target system) after which it can repeatedly be replayed on a target
system.
Preprocess of captured work load is resource intensive hence this step
should be done on a test system separate from the Production system from
where the workload was captured. This system can be the same as the
replay system or a different one. For a RAC configuration, it is required to do
the pre-processing against any one node only.
Workload capture happen into 'c:\app\capture directory in previous section
while for pre-processing we are doing on 'c:\app\capture2 directory.
CREATE OR REPLACE DIRECTORY capture_dir AS 'c:\app\capture2';
Directory created.
BEGIN
DBMS_WORKLOAD_REPLAY.PROCESS_CAPTURE (capture_dir => 'CAPTURE_DIR');
END;
/
Pre-processed files are created on c:\app\capture2 directory. These preprocessed files will be used by the replay clients in constructing the workload
during the replay stage. The preprocessed files contain the metadata (i.e.
attributes) describing the workload

18
6.2 Prepare the Test System
The test system can be prepared as follows
Ensure Oracle Home binary on database replay hosts and replay client
hosts, is at the correct patch level for Real Application Testing replay.
(see appendix ).
Setting up a Database Replay Administrator
The SYS or SYSTEM account is not required to be used for the capture and replay steps
described above. Any user that is granted the below privileges can be used to administer
Capture and Replay
CREATE USER <USERNAME> IDENTIFIED BY oracle;

GRANT CREATE SESSION, RESOURCE to <USERNAME>;
GRANT EXECUTE ON DBMS_WORKLOAD_CAPTURE TO <USERNAME>;
GRANT EXECUTE ON DBMS_WORKLOAD_REPLAY TO <USERNAME>;
GRANT CREATE SESSION TO <USERNAME>;
GRANT CREATE ANY DIRECTORY TO <USERNAME>;
GRANT SELECT_CATALOG_ROLE TO <USERNAME>;
GRANT BECOME USER TO <USERNAME>;
GRANT UNLIMITED TABLESPACE TO <USERNAME>
Ensure key database initialization parameters are set correctly based
on target goal of replay. Check host(s) environment configuration is
set up correctly e.g RAC.
Restore the Production system and clone it to the test system
OR
Restore only selected and relevant schemas from Production to Test
In addition, the System time of the replay system must be close to the
Production system if the workload has time sensitive data
Note from the Oracle documentation:
It is recommended that the system time on the replay system host be
changed to a value that approximately matches the capture start time
just before replay is started. Otherwise, an invalid data set may result
when replaying time-sensitive workloads. For example, a captured
workload that contains SQL statements using the SYSDATE and
SYSTIMESTAMP functions may cause replay divergence when replayed
on a system that has a different system time. Resetting the system
time will also minimize job scheduling inconsistencies between capture
and replay. The Workload Analyzer will recommend if the captured
workload is sensitive to time and if system clock needs to be reset.

19
This is not mandatory, but recommended for workloads that are

sensitive to time(e.g., sysdate)
If the source system makes use of DB Links, these DB links have to be

created or modified on the target system. The same consideration
applies to external directories and external tables.
Ensure all SQLNET connectivity files do not point to the production
environment target host and distributed interfaces .
Ensure object code is checked and amended for all external interface
communication e.g url access , mail , ftp.
Execute Workload analyzer on preprocessed directory to validate
capture files before replay. Analyze the result and address the
recommendations.
For repeated testing i.e. to repeatedly replay the captured workload,
each time making changes to the code, DB configuration or other
variables, it is useful to create a Savepoint as described below. In
addition, if we find that the replay runs for long periods of time, we
could make use of the create restore point <savepoint name>
guarantee flashback database.
drop restore point <Savepoint Name>;
create restore point <Savepoint Name>;
Once a Savepoint has been created, the DB can be reverted back to

the Savepoint as follows and then the replay can be executed once
again.
See the steps described in the below box.

20
SQL> connect / as sysdba

Connected.
SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup mount
ORACLE instance started.
Total System Global Area 954155008 bytes
Fixed Size 1303916 bytes
Variable Size 692062868 bytes
Database Buffers 255852544 bytes
Redo Buffers 4935680 bytes
Database mounted.
SQL> flashback database to restore point <Savepoint_name>;
Flashback complete.
SQL> alter database open resetlogs;
Database altered.
After the DB has been reverted back to a clean slate as described above,
the workload can be replayed once again. This approach allows us to
make changes to the configuration, code or other variables and replay the
workload any number of times until we achieve a satisfactory
performance.
7.0 Replaying a Workload

7.1 Workload Analyzer
Refer to this Metalink article for full details. A snippet from the My Oracle
Support note states as follows. The workload analyzer can be downloaded
from the above article. It will be available in the 11.2.0.2 RDBMS and can
also be directly executed from EM 12c.

21
The Workload Analyzer is a Java program that analyzes a workload capture

directory and identifies parts of a captured workload that may not replay
accurately due to factors that include insufficient data, errors that occurred
during workload capture, or usage of features that are not supported by
Database Replay. The results of the workload analysis are saved to an HTML
report named wcr_cap_analysis.html located in the capture directory that is
being analyzed. If an error can be prevented, the workload analysis report
displays available preventive actions that can be implemented before replay.
If an error cannot be corrected, the workload analysis report provides a
description of the error so it can be accounted for during replay. Running
Workload Analyzer is the default option and is strongly recommended.
7.2 Calibrating the workload before replay

Calibrating the workload is done to understand the characteristics of the
captured workload such as number of concurrent sessions that were in play
and total number of sessions. The calibration feature also provides a
recommended estimate of the number of wrc clients per CPU that could be
used for the actual replay for best simulation of the workload. In addition, it
also reports on the memory needed per client process
wrc mode=calibrate replaydir=c:\app\capture2

Workload Replay Client: Release 11.2.0.1.0 - Production on Thu Aug 11 19:33:10.11
Copyright (c) 1982, 2009, Oracle and/or its affiliates.
All rights reserved.
Report for Workload in: c:\app\capture2

----------------------Recommendation:
Consider using at least 1 clients divided among 1 CPU(s)
You will need at least 3 MB of memory per client process.
If your
machine(s)
cannot match
that number, consider using more clients.
wrc
system
mode=replay
replaydir=c:\app\capture
Password:
Workload Characteristics:
Errors
- max in
concurrency:
file :
1 sessions
total
number
of
sessions:
3
ORA-15554: cannot start workload
replay client because the database server
isAssumptions:
not in PREPARE mode
-
1 client process per 50 concurrent sessions

4 client process per CPU
256 KB of memory cache per concurrent session
think time scale = 100
connect time scale = 100
synchronization = TRUE

22
7.3 Initialize the Workload Replay

This step loads the metadata contained in the pre-processed files into
internal tables. Basically, this step places the DB in a state of readiness for
the replay. Connections strings are loaded into the
DBA_WORKLOAD_CONNECTION_MAP view
BEGIN
DBMS_WORKLOAD_REPLAY.INITIALIZE_REPLAY (replay_name => 'PEAK_REPLAY',
replay_dir => 'CAPTURE_DIR');
END;
/
7.4 Connection Maps

When the workload is captured on the source system, all the connection
strings used to establish connections to the target DB are captured as part of
the workload. Needless to say, these connection strings will not be valid for
the replay system as it will be a different DB and even perhaps on a different
server.
Therefore, connections strings need to be remapped prior to the replay on
the target system. Connections can be remapped as follows. The
connection_id is obtained by querying the
DBA_WORKLOAD_CONNECTION_MAP view
exec dbms_workload_replay.remap_connection(<connection_id>,
'<replay_connection>');
e.g.
exec dbms_workload_replay.remap_connection(1,
'(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=test_host)(PORT=1521))(CON
NECT_DATA=(SERVICE_NAME=TEST_SERVICE)))');

23
For Oracle Real Application Clusters databases, you can map all connection
strings to a load balancing connection string. This is especially useful if the
number of nodes on the replay system is different from the capture system.
Alternatively, if you
want to direct workload to specific instances, you can use services or
explicitly specify the instance identifier in the remapped connection strings.
7.5 Prepare the Workload Replay and check status
BEGIN
DBMS_WORKLOAD_REPLAY.PREPARE_REPLAY (SYNCHRONIZATION=>TRUE,
--means transactions are replayed in SCN order
CONNECT_TIME_SCALE=>100,
--means same time between connections as in captured workload
THINK_TIME_SCALE=>100,
--means 1.5x the time between
subsequent calls
THINK_TIME_AUTO_CORRECT=>false,
SCALE_UP_MULTIPLIER=>1);
--means how many times to multiply the workload
by during replay
END;
/
Checking for replays in progress

COL NAME FORMAT A15
COL STATUS FORMAT A20
SET LINES 200
SELECT ID,NAME, STATUS FROM DBA_WORKLOAD_REPLAYS;

24
Workload replay filters can be added if required similar to the filters defined
during the capture process. If no filters are defined, the entire captured
workload is replayed.
The following replay options can be specified with the
DBMS_WORKLOAD_REPLAY.PREPARE_REPLAY procedure during the
Prepare stage
synchronization
Controls whether or not the commit order is

preserved during replay. I.e. the transactions
are committed in exactly the same order as
they were committed in the Capture system.
Alternatively
the
transactions
can
be
committed in any order as long as they do not
violate their transactional dependencies.
Synchronization can also be done at the
Object level.
connect_time_scale
Scales the time elapsed between the start of
the replay and the start of each session
Think_time_scale
Scales the time elapsed between two
successive user calls from the same session.
This parameter can be used to reduce or scale
up the think time to simulate different types
of variations to the same workload
Think_time_auto_correct Auto corrects the think time between calls
when user calls takes longer during the
replay than during the capture
The below query displays one row for every replay client session that is
connected to the database and executing the captured workload.
SQL> select * from v$workload_replay_thread;

no rows selected

25
7.6 Starting the Workload Replay Clients (wrc)
wrc system mode=replay replaydir=c:\app\capture2

Workload Replay Client: Release 11.2.0.1.0 - Production on Thu Aug 11
19:58:11 2011
reserved.
All rights
Password:***********
Wait for the replay to start (19:58:14)
COL LOGON_USER FORMAT A15

COL PROGRAM FORMAT A8
COL EVENT FORMAT A25
COL SID FORMAT 99
COL FILE FORMAT A5
COL CLOCK FORMAT 99
SET LINES 200
SELECT CLOCK, SID, LOGON_USER, PROGRAM, FILE_NAME,CLIENT_OS_USER,EVENT FRO
V$WORKLOAD_REPLAY_THREAD;
Note: Once a workload replay is started, new replay clients will not be able
to connect to the database. Only replay clients that were started before the
START_REPLAY procedure is executed will be used to replay the captured
workload.

26
7.7 Starting the Workload Replay

A few pre-checks to ensure that we have a blank slate to start with.
SQL> SHOW USER
USER is "SYS"
SQL> DESC SCOTT.PEAK_PROCESSING2
ERROR: ORA-04043: object SCOTT.PEAK_PROCESSING2 does not exist
Note The error is expected since the workload is expected to
create this object
Now start the workload replay, using the START_REPLAY procedure as

follows:
BEGIN
DBMS_WORKLOAD_REPLAY.START_REPLAY ();
END;
/
Then, we will see the below appearing in the output of the terminals where
the wrc.exe was started

27
7.8 Changing Workload Replay execution status

BEGIN
DBMS_WORKLOAD_REPLAY.PAUSE_REPLAY ();
END;
/
BEGIN
DBMS_WORKLOAD_REPLAY.RESUME_REPLAY ();
END;
/
Completion Messages
Replay started (20:02:43)
Replay finished (20:04:50)

28
7.9 Replay information extraction
SET SERVEROUTPUT ON
DECLARE
cap_id
NUMBER;
rep_id
NUMBER;
rep_rpt
CLOB;
BEGIN
cap_id := DBMS_WORKLOAD_REPLAY.GET_REPLAY_INFO(dir =>
'CAPTURE_DIR');
/* Get the latest replay for that capture */
SELECT max(id)
INTO
rep_id
FROM
dba_workload_replays
WHERE capture_id = cap_id;
dbms_output.put_line('Max REP_ID is :'||rep_id);
end;
/
Max REP_ID is :1
7.10 Extracting the Replay report for a specific Workload

set pagesize 0 long 30000000 longchunksize 2000
select DBMS_WORKLOAD_REPLAY.REPORT(replay_id => 1, format =>
'TEXT') from dual;

29
7.11 Sample Replay Report
DB Replay Report for PEAK_REPLAY

----------------------------------------------------------| DB Name | DB Id
| Release
| RAC | Replay Name | Replay
Status |
-----------------------------------------------------------|
ORCL1
| 1143348259 | 11.2.0.1.0 | NO | PEAK_REPLAY |
COMPLETED
|
----------------------------------------------------------Replay Information
-----------------------------------------------------------|
Information
| Replay
| Capture
-----------------------------------------------------------|
Name
| PEAK_REPLAY
| PEAK_PROCESSING_2
-----------------------------------------------------------|
Status
| COMPLETED
| COMPLETED
-----------------------------------------------------------|
Database Name
| ORCL1
| ORCL
-----------------------------------------------------------|
Database Version | 11.2.0.1.0
| 11.2.0.1.0
-----------------------------------------------------------|
Start Time
| 11-08-11 14:32:42 | 11-08-11 15:24:53
-----------------------------------------------------------| End
Time
| 11-08-11 14:32:55 | 11-08-11 15:48:47
-----------------------------------------------------------|
Duration
| 13 seconds
| 23 minutes 54 seconds |
-----------------------------------------------------------|
Directory Object | CAPTURE_DIR
| CAPTURE_DIR
-----------------------------------------------------------|
Directory Path
| c:\app\capture
| c:\app\capture
----------------------------------------------------------Replay Options
--------------------------------------------------------|
Option Name
| Value
|
--------------------------------------------------------| Synchronization
| SCN
|
--------------------------------------------------------| Connect Time
| 100%
|
--------------------------------------------------------| Think Time
| 150%
|
--------------------------------------------------------| Think Time Auto Correct | FALSE
|
30
--------------------------------------------------------| Number of WRC Clients

| 1 (1 Completed, 0 Running ) |
--------------------------------------------------------Replay Statistics
-----------------------------------------------------------|
Statistic
| Replay
| Capture
-----------------------------------------------------------| DB
Time
| 2.635 seconds | 5.889 seconds
-----------------------------------------------------------|
Average Active Sessions |
.2 |
0
-----------------------------------------------------------|
User calls
|
18 |
18
-----------------------------------------------------------|
Network Time
| 0.426 seconds | .
-----------------------------------------------------------|
Think Time
| 10.004 seconds | .
-----------------------------------------------------------
7.12 Exporting AWR data

AWR data can be exported during the time of the capture as well as during the replays. The
purpose of exporting the AWR data is primarily to compare
At the end of each capture or replay, the relevant AWR snapshots for the period during which
the capture or replay was executing can be exported and included in the captured or replayed
data. The export log would look something like the below.
To run the export, the command is as follows:
BEGIN
DBMS_WORKLOAD_REPLAY.EXPORT_AWR (capture_id => <CAPTURE ID>);
END;
/
.
.
.
.
.
.
.
.
.
.
exported
exported
exported
exported
exported
"SYS"."WRH$_SYSMETRIC_HISTORY" 13.46 KB 100 rows

"SYS"."WRH$_SQL_PLAN" 6.355 MB 26787 rows
"SYS"."WRH$_SYSMETRIC_SUMMARY" 32.19 KB 296 rows
"SYS"."WRH$_ENQUEUE_STAT" 18.02 KB 174 rows
"SYS"."WRH$_WAITCLASSMETRIC_HISTORY" 9.429 KB 0 rows

31
These exported AWR snapshots can then be imported into another system into a specified and
empty schema. The commands to export AWR reports during Capture and Replay are provided
below
EXEC DBMS_WORKLOAD_CAPTURE.EXPORT_AWR (capture_id => 12);
EXEC DBMS_WORKLOAD_REPLAY.EXPORT_AWR (replay_id => 13);

32
7.13 Useful Views

This section summarizes the views that you can display to monitor workload replay. You need DBA
privileges to access these views.
The DBA_WORKLOAD_CAPTURES view lists all the workload captures that have been captured in
the current database.
The DBA_WORKLOAD_FILTERS view lists all workload filters for workload captures defined in the
current database.
The DBA_WORKLOAD_REPLAYS view lists all the workload replays that have been replayed in the
current database.
The DBA_WORKLOAD_REPLAY_DIVERGENCE view enables you to view information about
diverged calls, such as the replay identifier, stream identifier, and call counter.
The DBA_WORKLOAD_REPLAY_FILTER_SET view lists all workload filters for workload replays
defined in the current database.
The DBA_WORKLOAD_CONNECTION_MAP view lists the connection mapping information for
workload replay.
The V$WORKLOAD_REPLAY_THREAD view lists information about all sessions from the replay
clients.
The following Oracle supplied PL/SQL API packages contain the subprograms to completely manage and administer the Capture and Replay
processes
package
DBMS_WORKLOAD_REPLAY package
DBMS_WORKLOAD_CAPTURE
8.0 Data Divergence on Replay Sample Simulation

When the captured workload is replayed on the target system it is highly
possible that some workload statements may fail due to DML, DDL or
privilege related errors. These errors are called divergences and the
effectiveness of the replay is measured by the % of the captured workload
that could successfully be replayed on the target system. This allows the
tester to make appropriate adjustments while reviewing the system
performance results.
Data Divergences could occur due to lack of privileges, synonyms on the
objects that are referred to in the workload, missing objects and/or
schemas, unique key violations, referential integrity violations, space related
issues etc.

33
8.1 A Test case for demonstrating Data Divergence
To understand a typical data divergence scenario, a sample test case has

been devised based on a unique key constraint violation. In this test case,
an employee table has been created with a unique key index and during
replay we add a duplicate row into the table and that causes a unique index
violation in the data divergence dictionary views.
Same db and schema is used for capture and reply process during test case.
Below are the high level steps outlined below
1) Create an employee table in the SCOTT schema.
2) Start the capture process and insert unique rows into the EMPLOYEE table
3) Once capture completes, remove inserted rows from employee table to start
with a clean slate for the replay
4) Before the replay process is started, insert one duplicate row into the
EMPLOYEE table. This gives rise to an unique key constraint violation which will
then be populated in the divergence views
5) Check divergence in the Divergence view for the details of the SQL and bind
variables that caused the divergence
The following section provides a step by step approach to simulating and

diagnosing a Data Divergence scenario during replay
Create a sample table which will be used for demonstrating the data
divergence scenario
create table employee (employeeid number, employee_name varchar2(40));
Remove all previous capture metadata if desired. This is not a mandatory

step.
select id from DBA_WORKLOAD_CAPTURES;
Add
begin
DBMS_WORKLOAD_CAPTURE.DELETE_CAPTURE_INFO(<Capture ID>);
aend;
filter on the SCOTT schema
/

34
Add the required filters to restrict the workload that is captured as

described below.
BEGIN
fname => 'user_scott',
fattribute => 'USER',
fvalue => 'SCOTT');
fname => 'instance_filter',
fattribute => 'INSTANCE_NUMBER',
fvalue => '1' );
END;
/
Check the current filters and their statuses
SQL> select * from dba_workload_filters;

TYPE
ID STATUS NAME
ATTRIBUTE VALUE
--------------------------------------------------CAPTURE 0 NEW
USER_SCOTT
USER
SCOTT

35
Create a capture directory (if non-existent) and start the capture process
SQL> create or replace directory CAPTURE_DIR as 'c:\app\capture';

Directory created.
SQL>
BEGIN
DBMS_WORKLOAD_CAPTURE.START_CAPTURE (
name => 'PEAK_PROCESSING_2',
dir => 'CAPTURE_DIR',
default_action=>'EXCLUDE',
duration => null, --means capture has to be ended manually
auto_unrestrict => FALSE
--if the above is TRUE all instances started up in RESTRICT mode will
be made
-- unrestricted when the capture ends
);
END;
/
Check the status of the capture process that has been started as
described in the steps below
set lines 200

col name format a15
ID NAME
DBID DBNAME
DBVERSION
PAR STATUS
TO_CHAR(START_TI TO_CHAR(END_TIME DURATION_SECS
---------- --------------- ---------- ---------- ------------ --- -------------- ---------------- ---------------- ------------57 PEAK_PROCESSING_2 1292646392 ORCL
11.2.0.1.0
NO IN
PROGRESS
30/10/2011 05:24

36
Insert data into employee table to generate work load in the SCOTT
schema.
insert
Insert
insert
insert
into
into
into
into
employee
employee
employee
employee
values
values
values
values
(1,'Akash');
(2,'Kehul');
(3,'Milan');
(4,'Ajay');
Complete the workload capture step

BEGIN
DBMS_WORKLOAD_CAPTURE.FINISH_CAPTURE ();
END;
/
Check the capture process status to verify it shows as COMPLETED
set lines 200
col name format a15
PARALLEL,STATUS,
to_char(START_TIME,'dd/mm/yyyy
hh24:mi'),
ID NAME
DBID DBNAME
DBVERSION
PAR STATUS
TO_CHAR(START_TI TO_CHAR(END_TIME DURATION_SECS
---------- --------------- ---------- ---------- ------------ --- -------------- ---------------- ---------------- ------------57 PEAK_PROCESSING_2 1292646392 ORCL
11.2.0.1.0
NO
COMPLETED
30/10/2011 05:24 30/10/2011 05:27
148
This concludes creation and capture of the workload which will be used to
simulate the Data Divergence in the subsequent steps
37
8.2 Replaying the workload to cause a Data Divergence

To simulate a data divergence, after the capture completes, we remove
inserted data from employee table and add a unique key index and one
dummy row which will get duplicated when the replay runs
delete from employee;
create unique index employee_empid on employee (employeeid);
insert into employee values (1,'Raja');
Pre-process the captured workload

BEGIN
DBMS_WORKLOAD_REPLAY.PROCESS_CAPTURE (capture_dir =>
'CAPTURE_DIR');
END;
/
Initialize and calibrate the workload before replay
BEGIN
DBMS_WORKLOAD_REPLAY.INITIALIZE_REPLAY (replay_name =>
'PEAK_REPLAY',
replay_dir => 'CAPTURE_DIR');
END;
/
$ wrc mode=calibrate replaydir=c:\app\capture

38
Prepare the Workload Replay
BEGIN
DBMS_WORKLOAD_REPLAY.PREPARE_REPLAY (SYNCHRONIZATION=>TRUE,
--means transactions are replayed in SCN order
CONNECT_TIME_SCALE=>100,
--means same time between connections as in captured workload
THINK_TIME_SCALE=>100,
--means 1.5x the time between
subsequent calls
THINK_TIME_AUTO_CORRECT=>false,
SCALE_UP_MULTIPLIER=>1);
--means how many times to multiply the workload by
during replay
END;
/
Check for the replay status as shown below.

COL NAME FORMAT A15
COL STATUS FORMAT A20
SET LINES 200
SELECT ID,NAME, STATUS FROM DBA_WORKLOAD_REPLAYS;
ID
---------16
1
13
17
34
12
43
21
NAME
--------------PEAK_REPLAY
PEAK_REPLAY
PEAK_REPLAY
PEAK_REPLAY
PEAK_REPLAY
PEAK_REPLAY
PEAK_REPLAY
PEAK_REPLAY
STATUS
-------------------COMPLETED
CANCELLED
CANCELLED
COMPLETED
COMPLETED
COMPLETED
PREPARE
COMPLETED
Start the replay clients as described in the steps below.

wrc system mode=replay replaydir=c:\app\capture
Workload Replay Client: Release 11.2.0.1.0 - Production on Thu Aug 11
19:58:11 2011

(c)
1982, 2009,
Copyright
Copyright 2012, IBM
Corporation
reserved.
Oracle and/or its affiliates.
39
Password:***********
All rights
Start the workload replay as shown below
BEGIN
DBMS_WORKLOAD_REPLAY.START_REPLAY ();
END;
/
Then, we will see the below appearing in the output of the terminals where
the wrc.exe was started
wrc system mode=replay replaydir=c:\app\capture
Workload Replay Client: Release 11.2.0.1.0 - Production on Sun Oct 30
11:00:18 2011
reserved.
All rights
Password:
Replay started (11:00:40)
To check what SQLs are being currently replayed by the wrc clients, the
following query can be executed.
SQL> select sid,sql_id from v$session where program like

'wrc%';
SID SQL_ID
---------- ------------13 7wn3wubg7gjds

40
8.3 Checking for Data Divergences

After the replay clients have completed, check for any Data Divergences
that might have occurred as depicted below.
SQL> set long 5000

SQL> select * from DBA_WORKLOAD_REPLAY_DIVERGENCE where replay_id=43;

41
8.4 Extract the Diverging SQL statement
To extract the details of the diverging SQL statement and its associated bind
variables, follow the steps described below. This can then be used to
ascertain if the exclusion of the diverging statements causes an impact to
the workload simulation. If it is important that the diverged statements must
be executed, then the data must be corrected on the target system prior to
re-running the workload replay
select DBMS_WORKLOAD_REPLAY.GET_DIVERGING_STATEMENT(replay_id => 43,

stream_id => stream_id, call_counter => 17)
from DBA_WORKLOAD_REPLAY_DIVERGENCE
where replay_id=43 and call_counter=17;
Diverging Statement Details
----------------------------
<replay_divergence_info>
<sql_id>786cwqgg8m5j7</sql_id>
<sql_text>
insert into employee values (1,'Akash')
</sql_text>
<full_sql_text>
insert into employee values (1,'Akash')
</full_sql_text>
<binds/>
</replay_divergence_info>
SQL>

42
9.0 Using Enterprise Manager for Real Application

Testing
Oracle Enterprise Manager is integrated with the APIs and can be used for performing
Database Replays as an alternative to the command prompt.
As shown below (for EM 11g), the Software and Support tab contains the links to the Real
Application Testing suite. Clicking on the Database Replay takes us to the next screen shown
further below.
The Database Replay screen contains the links for performing the tasks related to Capturing
Workloads, Preprocessing workloads and Replaying workloads.

43
In the preprocess workload screen, a list of available directory objects are displayed from which
the desired directory, containing the captured data, is to be chosen
The detailed steps for performing Capture and Replay using Enterprise Manager have
intentionally been excluded from this document as the steps and screens are self explanatory

44
Summary
This paper attempted to describe the best practices on deploying Oracle Real Application
Testing. This is a toolset/feature which captures an exact production workload including timing,
concurrency and dependency information and is able to run this captured workload into an
alternate environment. With this load testing, we were able to learn the impact of a prospective
change to a workload. The installation was done on an IBM POWER6 System.
Operating system
AIX 6.1 (6100-06-04-1112) is the operating system used in the tests described in this paper. This test was
pursued in PowerVM VIOS environment. VIOS version 2.2.0.10-FP-24 was used. Virtual SCSI disk
method was used to present the Storage LUNs to the AIX Logical partitions (LPAR).
Storage System Management

IBM System Storage Storwize V7000 provides a new embedded web browser based Graphical User
Interface (GUI) interface for configuring and managing all aspects of the storage sub system. It also helps
to troubleshoot and correct problems. IBM V7000 also provides Command Line Interface (CLI) for the
advanced users who are familiar of administering this storage server.

45
Resources
These Web sites provide useful references to supplement the information contained in this document:
IBM Power Systems Information Center
o http://publib16.boulder.ibm.com/pseries/index.htm
IBM AIX Information Center
o http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp
IBM Publications Center
http://www-05.ibm.com/e-business/linkweb/publications/servlet/pbi.wss?CTY=US
.
IBM Redbooks
o http://www.redbooks.ibm.com/
My Oracle Support articles & references (https://support.oracle.com/CSP/ui/flash.html)
(please note: an userid and password are required to access the above website)
o Real Application Testing Now Available for Earlier Releases [ID 560977.1]
o Real Application Testing (RAT) API Setup and Verification [ID 1083063.1]
o Using the workload capture and replay in 11G [ID 445116.1]
o Database Replay Diagnostic information [ID 1287620.1]
o Database Capture and replay common errors and reasons [ID 463263.1
Oracle Documentation
http://docs.oracle.com/cd/E11882_01/server.112/e16540/rat_intro.htm#RATUG101
About the author

The author, Surya Rao, is an IBM Senior Certified IT Specialist, an Oracle Certified Master and the
Service Line Architect for Data Management, IBM GTS, SO Delivery, India
Viren Parikh, is an Oracle Subject Matter Expert (SME) in Data Management, IBM GTS, SO Delivery
India, and has made significant technical contributions to this document

46
Appendix
1.0 Common Errors and Resolutions
Please refer to the article DATABASE CAPTURE AND REPLAY COMMON
ERRORS AND REASONS [ID 463263.1] for information on common issues
that could be faced
2.0 Patch level for RAT

Refer to the MOS Article 560977.1 as the list of patches is constantly
changing

47
Trademarks and special notices

Copyright. IBM Corporation 1994-2012. All rights reserved.
References in this document to IBM products or services do not imply that IBM intends to make them
available in every country.
IBM, the IBM logo are trademarks or registered trademarks of International Business Machines
Corporation in the United States, other countries, or both:
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other
countries, or both.
Microsoft, Windows, Windows Server, and the Windows logo are trademarks of Microsoft Corporation in
the United States, other countries, or both.
Intel and Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the
United States and other countries.
AMD and AMD Opteron are trademarks of Advanced Micro Devices, Inc.
Red Hat, the Red Hat "Shadow Man" logo, and all Red Hat-based trademarks and logos are trademarks
or registered trademarks of Red Hat, Inc., in the United States and other countries.
UNIX is a registered trademark of The Open Group in the United States and other countries.
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
The information provided in this document is distributed AS IS without any warranty, either express or
implied.
The information in this document may include technical inaccuracies or typographical errors.
All customer examples described are presented as illustrations of how those customers have used IBM
products and the results they may have achieved. Actual environmental costs and performance
characteristics may vary by customer.
Information concerning non-IBM products was obtained from a supplier of these products, published
announcement material, or other publicly available sources and does not constitute an endorsement of
such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly
available information, including vendor announcements and vendor worldwide homepages. IBM has not
tested these products and cannot confirm the accuracy of performance, capability, or any other claims
related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the
supplier of those products.
All statements regarding IBM future direction and intent are subject to change or withdrawal without
notice, and represent goals and objectives only. Contact your local IBM office or IBM authorized reseller
for the full text of the specific Statement of Direction.
Some information addresses anticipated future capabilities. Such information is not intended as a
definitive statement of a commitment to specific levels of performance, function or delivery schedules with

48
respect to any future products. Such commitments are only made in IBM product announcements. The
information is presented here to communicate IBM's current investment and development activities as a
good faith effort to help with our customers' future planning.
Performance is based on measurements and projections using standard IBM benchmarks in a controlled
environment. The actual throughput or performance that any user will experience will vary depending
upon considerations such as the amount of multiprogramming in the user's job stream, the I/O
configuration, the storage configuration, and the workload processed. Therefore, no assurance can be
given that an individual user will achieve throughput or performance improvements equivalent to the
ratios stated here.
Photographs shown are of engineering prototypes. Changes may be incorporated in production models.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in
any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part
of the materials for this IBM product and use of those Web sites is at your own risk.

49

IBM Power and Oracle Real Applications Testing SRao 16JUL12

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

IBM Power and Oracle Real Applications Testing SRao 16JUL12

Uploaded by

Copyright:

Available Formats

Oracle Real Application Testing

July 15, 2012

In Collaboration with the IBM Oracle International Competency Center

Copyright IBM Corporation, 2012. All Rights Reserved.

3.0 Birds eye view of Workload Capture & Replay................................................................ 4

5.0 Capturing a Workload ........................................................................................................ 9

6.0 PRE REPLAY CHECKLIST ............................................................................................... 18

7.0 Replaying a Workload ...................................................................................................... 21

8.0 Data Divergence on Replay Sample Simulation ............................................................. 33

Oracle Real Application testing

8.4 Extract the Diverging SQL statement ...........................................................................................42

9.0 Using Enterprise Manager for Real Application Testing................................................ 43

Trademarks and special notices ........................................................................................... 48

Oracle Real Application testing

1.0 Executive Summary

Oracle Real Application Testing

Some of Key features of Real Application Testing tools

2.1 Licensing Considerations

In order to use the features listed in this whitepaper, it is required to

Oracle Real Application Testing

3.0 Birds eye view of Workload Capture & Replay

Workload Capture (On Source/Production database)

Oracle Real Application Testing

Oracle Real Application Testing

4.0 PRE CAPTURE CHECKLIST

4.2 Check List

Oracle Real Application Testing

a. For Oracle Database 11g it is enabled by default but package

CREATE USER <USERNAME> IDENTIFIED BY oracle;

Oracle Real Application Testing

production environment that would limit client access for a short

Oracle Real Application Testing

5.0 Capturing a Workload

5.1 Creating a Capture Directory

5.2 Capture Directory and its contents

256 Oct 07 00:41 cap

If the capture processing is initiated with a previous workload's captured

Oracle Real Application Testing

5.3 Filtering the capture workload

The steps for the Workload capture are as demonstrated below.

Oracle Real Application Testing

If a filter that was previously added needs to be removed before capturing

SQL> select * from dba_workload_filters;

5.4 Start workload capture

PL/SQL procedure successfully completed.

Oracle Real Application Testing

Once capture complete, status should be completed.

For RAC, capture is executed on one node only For 10.2.0.4

Oracle Real Application Testing

set lines 200

Oracle Real Application Testing

5.6 Workload Capture Files

SQL> !ls -l /home/oracle/capture/capfiles/inst1

Oracle Real Application Testing

5.7 Workload Capture Report

SQL> set pagesize 0 long 30000000 longchunksize 2000

Database Capture Report For ORCL

Oracle Real Application Testing

Sid, Serial# % Activity Event

9.09 db file parallel write

4.55 WCR: capture file IO write

4.55 WCR: capture file IO write

4.55 control file sequential read

5.8 Exporting AWR data