<Insert Picture Here>

Oracle 11g Real Application Testing: Avoiding Performance Regressions with SQL Performance Analyzer
Khaled Yagoub, Pete Belknap, Benoit Dageville, Karl Dias Database Manageability Group

Presentation Outline
SQL Performance Analyzer Overview
Common Usage Scenarios Workflow Phases
<Insert Picture Here>

SQL Performance Analyzer Interface

Enterprise Manager Command Line

Q/A

SQL Performance Analyzer: Overview

SQL Workload (SQL text + binds)

Helps users predict the impact of system changes on SQL workload response time Builds different versions of SQL workload performance (i.e., SQL execution plans and statistics)

SQL plans + stats Pre-change

SQL plans + stats

Post-change

Compare SQL Performance

Analyzes performance differences

Offers fine-grained performance analysis on individual SQL

Analysis Report

Integrated with SQL tuning set, SQL plan baselines, and SQL tuning advisor to form an End-toend solution

Common Usage Scenarios

Database upgrades and patches Database parameter changes Schema changes Statistics gathering Implementation of tuning recommendations OS/hardware changes

SQL Performance Analyzer: Workflow

Production Test
Make Change
Steps (1) (2) (3) (4) (5)

Capture SQL (STS)

Transport STS

Execute SQL Pre-change

Execute SQL Compare Post-change Perf.

(6)

Reiterate
No

Done?

(7) Production Change / (7) Tuning Deployment Tuned System

Yes

Step 1: Capture SQL Workload

SQL Tuning Set (STS) used to store & manage SQL workload STS includes complete execution context

Cursor Cache

SQL Text, bind variables, parsing schema Execution plans and run-time statistics Number of executions

STS can be built from AWR, Cursor Cache, user workload and existing STS

Incremental Capture
SQL Tuning Set Production Database

Incremental capture used to populate STS from cursor cache over a time period
Negligible performance overhead (<2 %) Filtering and ranking capabilities filters out undesirable SQL

Step 2: Move SQL Workload to Test System

Production Test

Cursor Cache

SQL Tuning Set

Export/Import

SQL Tuning Set

Test system should resemble production (stats ) Copy SQL tuning set to staging table (pack) Transport staging table to test system (datapump, db link, etc.) Copy SQL tuning set from staging table (unpack)

Step 3: Execute SQL Before Making Change

SQL Tuning Set

Fetch Next SQL

Establishes SQL workload performance baseline before change

Each SQL test-executed once in isolation with production context

SQL Trial: SQL execution plans and run-time statistics captured

Test Execute

DDL skipped
Only query part of DML is executed Customizable global and per-SQL time limit to handle long running/run-away SQL Option to do Explain Plan only analysis Remote test-execute using database link (on 10.2 and up)

Execution Plan & Statistics

Save Result

SQL Performance Analyzer

Step 4: Execute SQL After Making Change

SQL Tuning Set
Fetch Next SQL

Test Execute

Execution Plan & Statistics

Save Result Completed

Manually implement the planned change Database upgrade, patches Optimizer statistics refresh Schema changes Database parameter changes Validate tuning actions, e.g., SQL Profile creation, Partitioning, etc. Hardware changes, e.g., Enabling direct IO, File System to ASM conversion, etc.
Re-execute SQL after change Gathers new SQL execution plans and run-time statistics (SQL Trial)

SQL Performance Analyzer

Step 5: Compare & Analyze Performance

Completed Completed

Compare performance using different metrics, e.g.,

Buffer Gets CPU Time Elapsed Time Optimizer Cost Weigh SQL using original number of executions

Compare SQL Performance

SPA Report shows change impact on both workload, individual SQL

Improved SQL Regressed SQL Unchanged SQL SQL with plan changes

Analysis Report

SQL Performance Analyzer

Fix regressed SQL using SQL Tuning Advisor or SQL Plan Baselines

SQL Performance Analyzer Report

3 2

1 4

SQL Performance Analyzer Report

Regressed SQL Statements

Step 6: Reiterate
Regression Remediation

Two recommended solutions to fix regressed SQL:

Create SQL Plan Baselines: Helps revert to previous stable plans

SQL Tuning Advisor: Helps explore better execution plans

Step 6: Reiterate
Report before using Plan Baselines

Step 6: Reiterate
Report after using Plan Baselines

Change Accepted
No Regressions after SPM fix

Step 7: Production Change & Tuning Deployment

Implement validated changes permanently on production Transport any tuning solution to production
Export/Import SQL Plan Baselines Export/Import SQL Profiles Make schema changes Gather new optimizer statistics Change parameter values

Enterprise Manager Interface

SQL Performance Analyzer Home Page

Enterprise Manager Interface

Guided Workflow

Enterprise Manager Interface

Parameter Change Workflow

Parameter Changes
SPA Report

Parameter Changes
SQL Tuning Advisor Recommendations

Parameter Changes
SPA Report After Profiles Implemented

Change Accepted

Command Line Interface

dbms_sqlpa PL/SQL Package
-- create an empty sql tuning set exec dbms_sqltune.create_sqlset(my_sts); -- incrementally capture sql from the sql cache exec dbms_sqltune.capture_cursor_cache_sqlset( sqlset_name => 'my_sts', time_limit => 24*60*60, repeat_interval => 5*60);

-- create an analysis task Var tname varchar2(30); exec :tname := dbms_sqlpa.create_analysis_task(sqlset_name => my_sts task_name => my_task);

Command Line Interface

dbms_sqlpa PL/SQL Package
-- before change execution exec dbms_sqlpa.execute_analysis_task(:tname, execution_type=>'test execute'); -- make your change here -- alter session set -- after change execution exec dbms_sqlpa.execute_analysis_task(:tname, execution_type=>'test execute'); -- compare the before and after execution using buffer_gets exec dbms_sqlpa.set_analysis_task_parameter(:tname, -

'comparison_metric', 'buffer_gets');
exec dbms_sqlpa.execute_analysis_task(task_name => :tname, execution_type=>'compare performance');

Command Line Interface

dbms_sqlpa PL/SQL Package
Projected Workload Change Impact: ------------------------------------------Overall Impact : 49.18% Improvement Impact : 61.48% Regression Impact : -12.3% SQL Statement Count ------------------------------------------SQL Category SQL Count Plan Change Count Overall 12 9 Improved 1 1 Regressed 3 3 Unchanged 8 5

SQL Statements Sorted by their Absolute Value of Change Impact on the Workload ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | | Impact on | Metric | Metric | Impact | % Workload | % Workload | Plan | | object_id | sql_id | Workload | Before | After | on SQL | Before | After | Change | --------------------------------------------------------------------------------------------------------| 26 | 8answpy3djcj5 | 61.48% | 67 | 7 | 89.55% | 68.65% | 14.11% | y | | 29 | gcfysssf6hykh | -4.1% | 3 | 7 | -133.33% | 3.07% | 14.11% | y | | 30 | 8yyjua0yhcc92 | -4.1% | 3 | 7 | -133.33% | 3.07% | 14.11% | y | | 32 | 0782mpfq27aq5 | -4.1% | 3 | 7 | -133.33% | 3.07% | 14.11% | y | ---------------------------------------------------------------------------------------------------------

SQL Performance Analyzer

Data Dictionary Views Modified views in Oracle Database 11g:
DBA{USER}_ADVISOR_TASKS:details about the advisor task DBA{USER}_ADVISOR_FINDINGS:analysis findings

New views in Oracle Database 11g:

DBA{USER}_ADVISOR_EXECUTIONS:metadata information for a task execution DBA{USER}_ADVISOR_SQLPLANS: SQL execution plans DBA{USER}_ADVISOR_SQLSTATS: SQL compilation and execution statistics

Summary
Enables identification of SQL performance changes (e.g., regressions) before end-users can be impacted Helps with any change that impacts SQL execution plans (e.g., DB upgrades, Optimizer statistics refresh, schema changes, etc.) Automates SQL performance tracking of hundreds of thousands of SQL statements impossible to do manually Prevents side effects to database content when executing SQL. Keeps history of all experiments performed on SQL workloads. Integrated with SQL Tuning Advisor and SQL Plan Baselines for regression remediation GUI provides predefined workflows for Database upgrades and parameter changes

SQL Performance Analyzer In-house Testing

SQL Performance Regression Bugs
Errors + performance degradations caused by code changes SQL tuning set with 500K SQL statements Nightly runs

SQL Plan Management

Failures in reproducing execution plans

Partitioning Advisor
Verify partition recommendations

SQL Tuning Advisor

SQL Profiles, index recommendations

SQL Repair Advisor

Real Application Testing: Tools of the Trade

SQL Performance Analyzer
What is it? Predicts SQL performance deviations before end-users can be impacted Assess impact of change on SQL response time Executes each SQL, stored in SQL Tuning Set, in isolation using production context and then compares before and after execution plans and run-time statistics Unit testing of SQL with the goal to identify the set of SQL statements with improved/regressed performance

Database Replay
Replays real database workload on test system Assess impact of change on workload throughput Captures workloads and replays it with production context including concurrency, synchronization & dependencies

What Purpose?

How it works?

When to use?

Comprehensive testing of all sub-systems of the database server using real production workload

SQL Performance Analyzer and Database Replay are complimentary