1

SQL Tuning for Smarties, Dummies and Everyone in Between Novices

Jagan AthreyaDirector, Database Manageability, Oracle

Arup Nanda

2

Senior Director, Database Architecture, Starwood Hotels and Resorts

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver anycontract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development release and timing of anyThe development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3

Outline

• SQL Tuning Challengesg g

• SQL Tuning Solutions – New Feature Overview

• Problem Root Causes and their Solutions

• Preventing SQL Problems

• Q & A

4

SQL Tuning ChallengesReal-world DBA and Development TeamsReal-world DBA and Development Teams

• DBA team– Mostly average, some superstars– Superstars take most of the burden – over-stretched

• Development staff– Mostly non-Oracle skills – Java, C++y ,– Usually considers the DB as a “black box”– Writing efficient queries, troubleshooting performance issues

is delegated to DBAsis delegated to DBAs

5

SQL Tuning ChallengesProduction PerformanceProduction Performance

• Situation:– Query from hell pops up– Brings the database to its knees

DBA is blamed for the failure– DBA is blamed for the failure

• Response– DBA: “Developer should be taking care of this.”– Developer: “Why is the DBA not aware of this problem?”– Manager: “DBA will review all queries and approve them.”

• Challenge• Challenge– What is the most efficient way to manage this process?

6

SQL Tuning ChallengesChange Causing ProblemsChange Causing Problems

• Situation– New SQL statements added as part of application patch

deployment– Database upgradesDatabase upgrades– Database patching

• Response– Users: “How will the application perform after the changes?”– DBA: “How do I ensure that our SLA remains intact after the

changes are rolled out?”

• Challenge– How to reduce business risk while absorbing new

technologies?

7

technologies?

SQL Tuning ChallengesOptimizer Statistics ManagementOptimizer Statistics Management

• Situation– Data in Production has evolved over time. Have the optimizer

statistics stayed current?

• Response• ResponseDBA:– Will statistics refresh break something?– What will happen if we don’t collect?– How often should I collect the statistics ?– What happens when you collect a new set?pp y

• Challenge– What is the recommended strategy for managing optimizer

statistics to ensure the best performance?

8

statistics to ensure the best performance?

SQL Tuning ChallengesBad Plans – Diagnosis and ResolutionBad Plans – Diagnosis and Resolution

• No time to find the root cause. How to prevent this from recurring?

• Bind variables: How do you prevent bad plans based on choice of bind variables?on choice of bind variables?

• How to diagnose a bad plan– 10053 trace, endless pouring over traces, p g– Wrongly constructed predicates

• How to fix a bad plan Hi t ? h f d ?– Hints? change of code?

– Baselines vs. SQL Profiles– Pick out a single SQL or a bunch from the shared pool

9

Outline

• SQL Tuning Challengesg g

• SQL Tuning Solutions – New Feature Overview

• Problem Root Causes and their Solutions

• Preventing SQL Problems

• Q & A

10

Real-Time SQL MonitoringLooking Inside SQL Execution

• Automatically monitors long running SQL

g

• Enabled out-of-the-box with no performance overhead

• Monitors each SQL execution

• Exposes monitoring statisticsp g– Global execution level– Plan operation level– Parallel Execution level

• Guides tuning efforts

11

New capabilities in SQL MonitoringNew in Oracle Database 11g Release 2 g

• PL/SQL monitoring including associated high load SQL monitored recursively• Exadata aware I/O performance monitoring and associated metric datap g• Capture rich metadata such as bind values, session details e.g. user,

program, client_id and error codes and error messages• Save as Active Report for rich interactive offline analysis

12

DEMO

13

Application TuningAutomatic SQL TuningAutomatic SQL Tuning

High-Load

Packaged Apps +SQL Profile

C t i bl A

Well-Tuned SQL

Customizable Apps + SQL Advice

Customizable Apps + Indexes & MVs + Partitions

Applications

Automatic Tuning Optimizer

• Automatic SQL Tuning• Identifies high-load SQL from AWR• Tunes SQL using SQL Profilesg• Implements greatly improved SQL plans (optional)

• Performance benefit of advice provided• SQL Profiling tunes execution plan without changing SQL text

14

• Enables transparent tuning for packaged applications

Automatic SQL TuningNew in Oracle Database 11g Release 2New in Oracle Database 11g Release 2

Gather Missing or Stale Statistics

C t SQL P filSQL ProfilingStatistics Analysis

Access Path AnalysisSQL Restructure Analysis

Create a SQL Profile

Add Missing Access Structures

Modify SQL Constructs

Adopt Alternati eAlternative Plan AnalysisParallel Query Analysis

Automatic Tuning

Administrator

Comprehensive

Adopt AlternativeExecution Plan

Create Parallel SQL Profile

SQL Tuning Advisor

• SQL Tuning Advisor • NEW: Identifies alternate execution plans using real-time and historical

Automatic Tuning Optimizer SQL Tuning

Recommendations

NEW: Identifies alternate execution plans using real time and historical performance data

• NEW: Recommends parallel profile if it will improve SQL performance significantly (2x or more)

15

SQL Tuning for DevelopersIntegration with Visual Studio

• Introduced in Oracle Developer Tools for Visual Studio Release 11.1.0.7.20• Oracle Performance Analyzer

– Tune running applications with the help of ADDM• Query Window

– Tune individual SQL statements with STA• Server Explorer

– Manage AWR snapshots and ADDM tasksManage AWR snapshots and ADDM tasks

16

Agenda

• SQL Tuning Challengesg g

• SQL Tuning Solutions – New Feature Overview

• Problem Root Causes and their Solutions

• Preventing SQL Problems

• Q & A

17

What makes SQL go bad?Root Causes of Poor SQL Performance1. Optimizer statistics issues

a. Stale/Missing statisticsb. Incomplete statisticsc. Improper optimizer configurationd. Upgraded database: new optimizerpg pe. Changing statisticsf. Rapidly changing data

2. Application IssuesMi i t ta. Missing access structures

b. Poorly written SQL statements3. Cursor sharing issues

a. Bind-sensitive SQL with bind peekingp gb. Literal usage

4. Resource and contention issuesa. Hardware resource crunchb C t ti ( l k t ti bl k d t t ti )b. Contention (row lock contention, block update contention)c. Data fragmentation

5. Parallelism issuesa. Not parallelized (no scaling to large data)

18

p ( g g )b. Improperly parallelized (partially parallelized, skews)

What makes SQL go bad?Root Causes of Poor SQL Performance

1. Optimizer statistics issuesa. Stale/Missing statisticsb. Incomplete statisticsc. Improper optimizer configurationd. Upgraded database: new optimizere. Changing statisticsg gf. Rapidly changing data

2. Application Issues3. Cursor sharing issues 4. Resource and contention issues5. Parallelism issues

19

Oracle Optimizer Statistics

Inaccurate statistics Suboptimal Plans

Optimizer Statistics

CBO

Optimizer Statistics• Table Statistics

CBO• Column Statistics

I d S i i• Index Statistics

• Partition Statistics• Partition Statistics

• System Statistics

20

System Statistics

Oracle Optimizer StatisticsPreventing SQL Regressions

Novice Mode

• Automatic Statistics Collection Job (stale or missing)g)• Out-of-the box, runs in maintenance window• Configuration can be changed (at table

level)G th t ti ti d di ti• Gathers statistics on user and dictionary objects

• Uses new collection algorithm with accuracy of compute and speed faster than

Nightlyaccuracy of compute and speed faster than sampling of 10%

• Incrementally maintains statistics for partitioned tables very efficientpartitioned tables – very efficient • Set DBMS_STATS.SET_GLOBAL_PREFS

21

Oracle Optimizer StatisticsPreventing SQL Regressions

Expert Mode

• Extended Statistics• Extended Optimizer Statistics provides a mechanism

to collect statistics on a group of related columns:• Function-Based Statistics• Multi-Column Statistics

• Full integration into existing statistics frameworkFull integration into existing statistics framework• Automatically maintained with column statisticsDBMS_STATS.CREATE_EXTENDED_STATS

• Pending StatisticsPending Statistics• Allows validation of statistics before publishing• Disabled by default• To enable, set table/schema PUBLISH setting to FALSE, gDBMS_STATS.SET_TABLE_PREFS('SH','CUSTOMERS','PUBLISH','false')

• To use for validationALTER SESSION SET optimizer_pending_statistics = TRUE;

22

• Publish after successful verification

What makes SQL go bad?Root Causes of Poor SQL Performance

1. Optimizer statistics issues2. Application Issues

a Missing access structuresa. Missing access structuresb. Poorly written SQL statements

3. Cursor sharing issues 4 Resource and contention issues4. Resource and contention issues5. Parallelism issues

23

Identify performance problems using ADDMAutomatic Database Diagnostic Monitor Novice

M d

• Provides database and cluster-wide performance

Mode

cluster-wide performance diagnostic

• Throughput centric - Focus on reducing time ‘DB time’g

• Identifies top SQL:• Shows SQL impact• Frequency of

occurrence• Pinpoints root cause:

– SQL stmts waiting for Row Lock waitsRow Lock waits

– SQL stmts not shared

24

Identify High Load SQL Using Top ActivityNovice M dPerformance Page

• Identify Top SQL by DB Time:• CPU

Mode

• I/O• Non-idle waits

• Different Levels of Analysisy• Historical analysis

• AWR data• Performance Page

Top Activity

g• Real-time analysis

• ASH data• More granular analysisg y• Enables identification of

transient problem SQL• Top Activity Page

25

• Tune using SQL Tuning Advisor

Advanced SQL TuningUniverse of Access Structures

Novice+Mode

• Indexes: B-tree indexes, B-tree cluster indexes, Hash cluster indexes, Global and local indexes, Reverse key indexes, Bitmap indexes, Function-based indexes, Domain indexes

• Materialized Views: Primary Key materialized views, Object materialized viewsObject materialized views ROWID materialized views Complex materialized views

• Partitioned Tables: Range partitioning, Hash partitioning,List partitioning, Composite partitioning,Interval Partitioning REF partitioningInterval Partitioning, REF partitioning,Virtual Column Based partitioning

B-tree index

26

SQL Access Advisor: Partition Advisor Novice+Mode

Indexes

MaterializedviewsSQL Access

AdvisorRepresentative

Workload

Materializedviews logs

Automatic Tuning Optimizer

Access Path

Partitionedobjects

Access Path Analysis

objects

27

SQL Access AdvisorAdvanced Options

Expert Mode

• Workload filtering• Limited vs. advanced mode• Tablespaces for access structuresTablespaces for access structures• Hypothetical workload tuning• Factoring in the cost of creation• Space limitations for indexes and MVs• Space limitations for indexes and MVs

28

1 O ti i t ti ti i

What makes SQL go bad?Root Causes of Poor SQL Performance

1. Optimizer statistics issues2. Application Issues3. Cursor sharing issues

a Literal usagea. Literal usageb. Bind-sensitive SQL with bind peeking

4. Resource and contention issues5. Parallelism issues5. Parallelism issues

29

What makes SQL go bad?a. Literal Usage Issue

Expert Mode

SELECT * FROM jobs WHERE min_salary > 12000;

SELECT * FROM jobs WHERE min_salary > 15000;

SELECT * FROM jobs WHERE min_salary > 10000;

SELECT * FROM …SELECT * FROM …SELECT * FROM

cursor_sharing ={exact, force, similar}SELECT FROM …

Sh i

Library Cache

Sharing Cursors is good!

30

Library Cache

What makes SQL go bad?b. Bind Peeking Issue

NMode

Processed_Flag

YY Full Table ScanYYY CBO10g FTS

99

Two different optimal plans for different bind values

Index Range Scan

NIRS

1

31

Problem: Binds will affect optimality in any subsequent uses of the stored plan

Fixing problems with Adaptive Cursor SharingAdaptive Cursor Sharing Expert

ModeSELECT * FROM emp WHERE wage := wage_value

Selectivity Ranges:

Mode

Selectivity Ranges:

1

2

20 25

Same Plan

322 24

Different Plan

4

30 35Same Plan,

Expand Interval

32

34 43Interval

Agenda

• SQL Tuning Challengesg g

• SQL Tuning Solutions – New Feature Overview

• Problem Root Causes and their Solutions

• Preventing SQL Problems

• Q & A

33

Preventing problems with SQL Plan Management

• Problem: changes in the environment cause plans to change

NL

GB

Parse

• Plan baseline is establishedNL

NL

Statement log

Plan history

• SQL statement is parsed again and a different plan is generated

HJ

GB

Plan baselineGB

NL

NL

• New plan is not executed but k d f ifi ti HJmarked for verification

34

SQL Plan ManagementMigration of Stored Outlines to Plan Baselines

Oracle Database 11g

GB 5. Migrate Stored Outlines into SPM

Plan History

GB

OH Schema

GB

No planregressions

HJ

HJ

into SPMHJ

HJ HJ

GB

HJ

4. Upgrade to 11g

Oracle Database 11g

1 B iOH Schema

HJ

GB

CREATE_STORED_OUTLINES=true1. Begin with

2. Run all SQL in the Application and auto

t St d O tli

Oracle Database 9 or 10g

HJ

CREATE_STORED_OUTLINES=false

create a Stored Outline for each one

3. After Store Outlines are

captured

35

captured

SQL Performance Analyzer (SPA) Validate statistics refresh with SPA

• Steps:1. Capture SQL workload in STS

using automatic cursor cache

Validating upgrade with SPASQL Workload

using automatic cursor cache capture capability

2. Execute SPA pre-change trial3. Refresh statistics using

PENDING i

SQL plans + stats SQL plans + stats

Pre change Trial Post change TrialPENDING option4. Execute SPA post-change trial5. Run SPA report comparing SQL

execution statistics Compare

Pre-change Trial Post-change Trial

execution statistics

• Before PUBLISHing stats:• Remediate individual few SQL Analysis Report

SQL Performance

Qfor plan regressions: SPM, STA

• Revert to old statistics if too many regressions observed

36

Conclusion Identify, Resolve, Prevent

PreventSPASPM

1. Production Performance2. Change Causing Problems3. Optimizer Statistics Management4 B d l Di i d R l ti

Resolve

4. Bad plans – Diagnosis and Resolution

ADDM, Top Activity, SQL MonitoringResolve

Tuning Advisor

Identify

Tuning Advisor,Access Advisor,

Auto Stat CollectionTop Activity,

ADDM, SQL Monitoring

37

38