sql tuning for expert dbas - rainfocus€¦ · konstantin kerekovski 58 work experience: american...

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

SQL Tuning for Expert DBAs

GP (Gongloor Prabhaker)Senior Director of Product ManagementOracle Management Cloud, Oracle Corporation

Konstantin Kerekovski, DBA, Raymond James Financial Inc.

Oct 03, 2017

Confidential – Oracle Internal/Restricted/Highly Restricted


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Confidential – Oracle Internal/Restricted/Highly Restricted 3


Program Agenda

SQL Tuning: Challenges and existing solutions

Proactive approach and tips

Reactive approach and tips

Customer Case study

1

2

3

4


Please visit us at Demo Grounds: Moscone West, SOA-052, Database Monitoring and Performance

Management with Oracle Enterprise Manager


Program Agenda




Customer Case study

1

2

3

4



SQL Tuning: Why is guess work involved?


Lack of Automated , E2E and scalable solution

Resource Contention Optimizer Related Application SpecificIssues

End-user Expertise


SQL Performance Issues: Broad categories


Op

tim

ize

r R

elat

ed • Stale/Missing statistics

• Incomplete statistics

• Improper optimizer configuration

• Upgraded Database: new optimizer

• Rapidly changing data

• Stale/Missing statistics

• Incomplete statistics

• Improper optimizer configuration

• Upgraded Database: new optimizer

• Rapidly changing data

Re

sou

rce

Rel

ated •Hardware

resource crunch

•Contention (row lock/ block contention)

•Not parallelized (no scaling to large data)

• Improperly parallelized (partially parallelized, skews)

•Hardware resource crunch

•Contention (row lock/ block contention)

•Not parallelized (no scaling to large data)

• Improperly parallelized (partially parallelized, skews)

Ap

plic

atio

n Is

sue

s • Missing access structures

• Poorly written SQL statements

• Bind-sensitive SQL with bind peeking (Cursor Sharing)

• Literal usage

• Missing access structures

• Poorly written SQL statements

• Bind-sensitive SQL with bind peeking (Cursor Sharing)

• Literal usage

Lack

of

SQL

& O

pti

miz

er

Exp

ert

ise • Limited

knowledge of SQL and Optimizer

• Tunes individual or few statements vs. entire workload

• Limited knowledge of SQL and Optimizer

• Tunes individual or few statements vs. entire workload

Lack

of

Au

tom

atio

n • Manual effort, time consuming

• Lack of scalability with high volume SQL (100K)

• Unintuitive techniques and point solutions

• Manual effort, time consuming

• Lack of scalability with high volume SQL (100K)

• Unintuitive techniques and point solutions


SQL Tuning: Existing solutions

• How DBAs tune SQL today?– Use SQL Trace to identify offending SQL solution

• Prohibitive on production

– Disable index access by modifying predicates (to_char, to_number)

– Change init.ora parameters (optimizer_..)

– Adding indexes• Most apps are already over-indexed

– Materialized Views

– Partitioning, etc.

• Existing solutions have severe limitations, are not scalable and mostly reactive in nature, use trial and error methodology



Program Agenda




Customer Case study

1

2

3

4


Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 12

Proactive ApproachBecome the “Proactive DBA” using Oracle’s rich tool set!

• A DBA has Oracle’s rich tool set available to identify problems and remediate problems before they happen in production.

• Proactive DBA should use the following tools:

– ADDM to detect systemic problems and remediate them by following ADDM recommendations*

– Automatic SQL Tuning advisor to tune high load SQL statements over time – it is available out of the box*

– SQL Access Advisor to make sure your application has optimal access structures (indexes and type, MVs, MV logs, etc.) and partitioning scheme, tune entire workload*

– SPA Quick in production environments to quickly assess impact of routine system changes (for e.g., SQL Profile validation, optimizer related changes, optimizer refresh statistics) without impacting production*

– SPA to validate system changes such as migration to Oracle Database Cloud, Exadata Cloud Machine, etc.* *OOW 2016: Eliminating Guesswork from SQL Tuning [CON6975]


SQL Performance Analyzer


Analysis Report

Compare SQLPerformance

Post-change TrialPre-change Trial

SQL Plans + Statistics SQL Plans + Statistics

– Helps users predict the impact of system changes on SQL workload

– Low overhead capture of SQL workload to SQL Tuning Set (STS) on production system

– Build different SQL trials (experiments) of SQL statements performance by test execution or explain plan

– Integrated with STS, SQL Plan Baselines, & SQL Tuning Advisor to form an end-to-end solution


DBCS Migration Use Case

• As a DBA, you have been tasked by management to migrate your 11.2 database to the latest Cloud DB release. At the same time, the requirement is to make sure the performance is same or better than before, how can I accomplish this?



Validate Cloud Migration with SPA

On-Premise Oracle Cloud

15

Trial 2: Test Execute or Explain Plan

Step 1: Capture representative workload to STSStep 2: Clone On-premise database to Cloud

Analysis Report

Step 3b: Generate SPA Report and fix regressions

Trial 1: Build (Convert) from STS

Step3a: Conduct SPA trials

SPA Task


SPA - Concurrent SQL Execution

• SPA trials are currently executed serially one statement at a time – When SQL Tuning Sets contain a large number of statements, the trial could take

significant time. For example, EBS suite may have 1 million unique SQL statements

– Under utilized server if using large system

• Concurrent SQL execution within a SPA trial reduces duration to complete the trial, therefore reduces testing time– User specifies degree of parallelism with SPA parameter TEST_EXECUTE_DOP

– For example, a SQL Tuning Set with 125,000 statements reduced testing from 12.5 hours to 2 hours with a degree of parallelism of 8

• Tip: Use SPA concurrent SQL execution in test environment for STS with a large number (thousands or higher) of SQL statements


New in

18.1


SPA – Result Set Validation

• Result set validation required for certain environments – e.g. government drug research trials, pharma industries, etc. An additional assurance that SQL produces correct results with new environment / software

• SPA report can be generated for before and after change trials comparing whether identical result sets were produced

• SPA result set validation enabled with SPA parameter COMPARE_RESULTSET (defaults to true)

• Tip: Run before and after trials with exact same dataset for result set validation

• If result sets differ with same input dataset – contact Oracle Support


New in

18.1


Program Agenda


Proactive Approach and tips

Reactive Approach and tips

Customer Case study

1

2

3

4



SQL Tuning Tips - Reactive Approach

• Proactive SQL Tuning should resolve most of your day-to-day issues, however sometimes reactive tuning is needed

• DBA should use the following tools for reactive SQL Tuning:– ASH Analytics for transient performance analysis and drilling down on SQL dimensions (by execution

plan operation, SQL_ID, PDB, Action, Module, etc.)

– SQL Monitor for complex run-time execution statistics on long running or Parallel executions such as which plan operation line that consumes resources such as CPU and I/O, overall PGA usage and Parallel distribution

– SQL Tuning Advisor getting comprehensive analysis and recommendations on problematic SQL statements such as profiles, statistics and new access structure



Tip: Use Real-Time SQL Monitoring For Detailed Execution Statistics On Long Running Or Parallel Executions

• Enabled out-of-the-box with no performance impact

• Automatically monitors SQL executions that:

– Consume more than 5 seconds of CPU or I/O time

– Are running parallel: PQ, PDML, PDDL

• Monitors each execution independently

• Exposes monitoring statistics at multiple levels

– Global execution level

– Plan operation level (Plan Tuning)

– Parallel Execution level (PX Tuning)

• Guides your tuning efforts

• SQL level metrics—CPU, I/O requests, throughput, PGA, temp space

• Graphical explain plan

• I/O statistics for each operation


Looking inside the SQL


Real-Time SQL Monitoring






• Can also be accessed via– ASH Analytics -> Select an SQL ID -> Select SQL Monitor from Detail section.



• Case Study 1 – Adaptive Plans

• Case Study 2 – Long Execution plan

• Case Study 3 – PL/SQL Monitoring

• Case Study 4 – Poor indexing

• Case Study 5 – Parallel Execution Downgrade


Use cases

Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal 29

Real-time SQL Monitoring with Adaptive PlansGenerate script for problem SQL with binds and execute(6 min)


SQL Monitoring Case Study 1


Case Study 1: Adaptive Plans

• How can I determine if a query has self tuned with the adaptive plan feature?

• Can I identify the values of my bind variables?


SQL Monitoring Use Cases


Case Study 1: Adaptive Plans


• Comprehensive view of execution plan

• Easy to pinpoint operations and objects that consume the most resources


SQL Monitoring Use CasesCase Study 2: Analyzing Large SQL Execution Plans



SELECT /*+ MONITOR */ avg(ss_quantity),avg(ss_ext_sales_price),avg(ss_ext_wholesale_cost),sum(ss_ext_wholesale_cost)from store_sales ,store ,customer_demographics ,household_demographics,customer_address,date_dimwhere s_store_sk = ss_store_skand ss_sold_date_sk = d_date_sk and d_year = 2001and (

(ss_hdemo_sk=hd_demo_sk and cd_demo_sk = ss_cdemo_sk and cd_marital_status = 'M’ and cd_education_status = '4 yr Degree‘ and ss_sales_price between 100.00 and 150.00 and hd_dep_count = 3 )

or(ss_hdemo_sk=hd_demo_sk and cd_demo_sk = ss_cdemo_sk and cd_marital_status = 'D’ and cd_education_status = 'Primary'

and ss_sales_price between 50.00 and 100.00 and hd_dep_count = 1 )or

(ss_hdemo_sk=hd_demo_sk and cd_demo_sk = ss_cdemo_sk and cd_marital_status = 'U’ and cd_education_status = 'Advanced Degree‘ and ss_sales_price between 150.00 and 200.00 and hd_dep_count = 1 )

)and(

(ss_addr_sk = ca_address_sk and ca_country = 'United States‘ and ca_state in ('KY', 'GA', 'NM')and ss_net_profit between 100 and 200 )

or(ss_addr_sk = ca_address_sk and ca_country = 'United States‘ and ca_state in ('MT', 'OR', 'IN')

and ss_net_profit between 150 and 300 )or

(ss_addr_sk = ca_address_sk and ca_country = 'United States‘ and ca_state in ('WI', 'MO', 'WV')and ss_net_profit between 50 and 250 ))

• Q: I have a complex query with a complex query plan, how can I identify which operations and objects consume the most resources?




• Solution: All activity is against STORE_SALES. Consider partitioning the table.




Real-Time PL/SQL Monitoring

• PL/SQL execution no longer a “black box”

– Answers questions like “why did my DBMS_STATS job take twice as long this time?”

• Shows global (PL/SQL) and SQL level statistics

• Each SQL called by PL/SQL recursively monitored

• Drill-down to slow SQL for diagnosing unexpected PL/SQL behavior




Case Study 3: Real-Time PL/SQL Monitoring

• Q: I have a PL/SQL procedure that takes a long time, can I identify which SQL statement that is my largest resource consumer?





Most elapsed time

Highest CPU usage• Q: I have a PL/SQL procedure that takes a long time, can I

identify which SQL statement that is my largest resource consumer?


• Initial Query

• Q: I have added many indexes to my database. I still have several queries with bad performance. Can I determine if my indexes are used efficiently?


SQL Monitoring Use CasesCase Study 4: Poor Indexing



• Q:I have added a large amount of indexes to my database. I still have several queries with bad performance. Can I identify that my indexes are used efficient?

• Initial Query • Tuned Query



• Q: I have added many indexes to my database. I still have several queries with bad performance. Can I determine if my indexes are used efficiently?

• Initial Query • Tuned Query • Tuned Query / Cached


SQL Monitoring Use CasesCase Study 5: Parallel Execution Downgrade

• Query requests parallel degree 4. Why is it downgraded to parallel degree 2?




select qksxareasons,indx

from x$qksxa_reason

where qksxareasons like '%DOP downgrade%’;

QKSXAREASONS INDX

------------------------------------------------------------ ----------

DOP downgrade due to adaptive DOP 351

DOP downgrade due to resource manager max DOP 352

DOP downgrade due to insufficient number of processes 353

DOP downgrade because slaves failed to join 354




• Solution: Consider changing the maximum DOP on Resource Consumer Group


SQL Tuning Advisor Exadata Enhancement

• SQL Tuning Advisor detects if SQL is executing on Exadata

• SQL Tuning Advisor privately gathers system statistics and does analysis with and without these statistics

• If a better execution plan is found with these system statistics, an Exadata-aware SQL Profile is recommended

• Can result in 10x or better performance improvement for SQLs which can benefit from Exadata hardware - e.g. cell smart scans


NEW IN

18.1


SQL Tuning Advisor Exadata Enhancement


NEW IN

18.1

Without Exadata aware SQL Profile – 1.9 minutes:

With-Exadata aware SQL Profile – 13 seconds:


Program Agenda


Proactive Approach and tips

Reactive Approach and tips

Customer Case study

1

2

3

4



Customer Case study


RAYMOND JAMES FINANCIALHTTPS://WWW.RAYMONDJAMES.COM/

57

KONSTANTIN KEREKOVSKI

58

Work Experience:

American Express – Senior Oracle DBA, 11g, 12c, RAC, GG, ADG

Raymond James – Senior Oracle DBA, 11g, 12c, Exadata, ODA, RAC, GG, ADG

LinkedIn: www.linkedin.com/in/konstantin-kerekovski-34740071

Speaker:

IOUG Collaborate 2017

SOUG (Suncoast Oracle User Group)

NYOUG (New York Oracle User Group)

THE PROBLEM

59

• Production Oracle Data Integrator (ODI) jobs are running slowly due to suboptimal execution plans on INSERT INTO XYZ SELECT…. queries.

• The INSERT queries cannot be readily altered due to declarative nature of ODI and internal Change Management process.

• The business is being impacted because multiple other ETL and batch jobs depend on the completion of ODI jobs.

PROBLEM IDENTIFICATION

60

The problem was clearly evident in the Top Activity page of OEM for the database in question.

THE SOLUTIONSQL TUNING ADVISOR

61

PLAN COMPARISON

62

IMPLEMENTATION

63

The implementation of the SQL Profiles and/or statistics gathering is as simple as following the guided instructions from the OEM SQL Tuning recommendations screen.

TOP ACTIVITY BEFORE TUNING

64

TOP ACTIVITY AFTER TUNING

THE IMPACT OF SQL TUNING ADVISOR

65

• The long running ODI job’s runtime improved dramatically.

• No development effort was necessary, thus saving money on labor costs on tuning efforts and regression testing efforts.

• The change was easily implementable and because SQL Profiles can easily be disabled, the change can be backed out if any issues arise.

• The change can be easily propagated throughout the entire SDLC due to the portable nature of SQL Profiles.

• Business processes which depend on ODI jobs to complete were no longer impacted and completed ahead of schedule.

66

What is it?

- Extra information to be used by the optimizer during parsing.

When do I use it?

- When you just want to help the optimizer with extra information without dictating a specific execution plan.

- When SQL Execution Plans are unstable and changing adversely due to issues such as inaccurate/stale Optimizer Statistics, bind variable peeking, etc.

TIPSHOULD I USE SQL BASELINES OR SQL

PROFILES ?SQL PROFILE

What is it?

- A set of stored, approved execution plans for a given SQL statement that are used instead of plans obtained during normal hard parsing.

When do I use it?

- During Database Upgrades for the purpose of avoiding performance regression due to new optimizer features ( E.G. Adaptive Plans in 12.1), missing statistics, etc.

- When you only want one specific execution plan to be used.

SQL BASELINE


SQL Tuning for Expert DBAs: Summary


• Become the proactive DBA, use:– ADDM to detect systemic problems and remediate

them by following ADDM recommendations

– Automatic SQL Tuning Advisor to tune high load SQL statements over time

– SQL Access Advisor to assure that your application has optimal access structures (indexes and type, MVs, MV logs, etc.). Tunes the entire workload

– SPA Quick Check in production environments to quickly assess impact of routine system changes (for e.g., optimizer refresh statistics)

– SPA to validate system changes such as migration to Oracle Database Cloud, Exadata Cloud Machine, etc.

• If you have to perform reactive SQL tuning, use:– ASH Analytics to identify your top resource

consumers

– SQL Monitor for detailed execution statistics on long running or parallel executions

– SQL Tuning Advisor to identify and remediate:

– Stale statistics

– Incorrect cardinality estimations

– Data correlations

sql tuning for expert dbas - rainfocus€¦ · konstantin kerekovski 58 work experience: american...

Documents