oraclesqlhighperformancetuning-110719182010-phpapp01

© 2011 Quest Software, Inc. ALL RIGHTS RESERVED

Oracle SQL High Performance Tuning

Guy HarrisonDirector, R&D Melbournewww.guyharrison.net [email protected]@guyharrison

http://www.guyharrison.net/

mailto:[email protected]

http://www.twitter.com/guyharrison

http://www.twitter.com/guyharrison

2

Introductions

3

Agenda• Philosophy and methodology• Optimizing the optimizer• Detecting errant SQLs• Changing the plan • Table lookups• Joins• Sorts• Group BY• Other topics

4

Prison guard analogy

• Keep the SQLs contained – Indexing and clustering – Optimizer configuration

• Detect break outs– Monitor and detect errant SQLs

• Re-capture the fugitives– Traditional SQL tuning– Outlines, baselines, indexing,

denormalization, hints

5

Optimizing the optimizer

6

Optimizer inputs

Object Statistics

Cardinality

Estimates

IO and CPU

Estimates

DB parameters

And config

Cost estimateSystem Statistics

Table and index

Structure

7

Optimizing the optimizer• Create necessary physical structures for optimal plans

– Indexes, partitions, clusters • Collect object statistics

– Histograms, extended statistics • Optimizer configuration parameters

– Memory_target, db_block_size, etc– Optimizer_index_caching, optimizer_index_cost_adj

• System statistics– DBMS_STATS.gather_system_stats

Histograms

Saudi

Arabia

South

Afri

ca

Turke

y

New Z

ealan

d

Denm

ark

Argen

tina

Singap

ore

Japa

n

Poland

China

Austra

liaBra

zil

Canad

aSpa

in

Franc

e

United

King

dom

Italy

Germ

any

United

Sta

tes

of A

mer

ica0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

18,000

20,000

Nu

mb

er o

f ro

ws

10

11g Extended Statistics

Select *

from people

Where gender=‘boy’

And name=‘Sue’

Boys Girls

People named SuePeople named Sue

Histogram limitations

Height balanced histograms don’t have

the granularity we might want or

expect.

Be realistic about histograms.....

• Histograms often fail to push cardinalities through multi-table SQLs

• Default histogram collections are not always optimal

13

Finding tunable SQL

14

Detecting break outs

• V$SQL & V$SQL_PLAN– Find SQLs with high resource costs

• EXPLAIN PLAN & DBMS_STAT– Determine the execution plan

• SQL Trace/Tkprof– Best drilldown at the session level

15

Mining V$SQL

16

DBMS_XPLANSQL> SELECT * FROM TABLE (DBMS_XPLAN.display_cursor ('at6ss8tmxm5xz', '0',

'TYPICAL -BYTES'));

PLAN_TABLE_OUTPUT

-----------------------------------------------------------------------------------------

SQL_ID at6ss8tmxm5xz, child number 0

-------------------------------------

SELECT department_name, last_name, job_title FROM hr.employees JOIN

hr.departments USING (department_id) JOIN hr.jobs USING (job_id)

ORDER BY department_name, job_title

Plan hash value: 3225241925

--------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Cost (%CPU)| Time |

--------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | | 26 (100)| |

| 1 | SORT ORDER BY | | 106 | 26 (8)| 00:00:01 |

| 2 | NESTED LOOPS | | 106 | 25 (4)| 00:00:01 |

| 3 | MERGE JOIN | | 107 | 24 (5)| 00:00:01 |

| 4 | TABLE ACCESS BY INDEX ROWID| EMPLOYEES | 107 | 20 (0)| 00:00:01 |

| 5 | INDEX FULL SCAN | EMP_JOB_IX | 107 | 12 (0)| 00:00:01 |

|* 6 | SORT JOIN | | 19 | 4 (25)| 00:00:01 |

| 7 | TABLE ACCESS FULL | JOBS | 19 | 3 (0)| 00:00:01 |

| 8 | TABLE ACCESS BY INDEX ROWID | DEPARTMENTS | 1 | 1 (0)| 00:00:01 |

|* 9 | INDEX UNIQUE SCAN | DEPT_ID_PK | 1 | 0 (0)| |

--------------------------------------------------------------------------------------

Predicate Information (identified by operation id):

---------------------------------------------------

6 - access("EMPLOYEES"."JOB_ID"="JOBS"."JOB_ID")

filter("EMPLOYEES"."JOB_ID"="JOBS"."JOB_ID")

9 - access("EMPLOYEES"."DEPARTMENT_ID"="DEPARTMENTS"."DEPARTMENT_ID")

17

SQL Trace & tkprof• Trace in current session: DBMS_SESSION

• In other session: DBMS_MONITOR– Session_trace_enable – specific session – Serv_mod_act_trace_enable – service, module or action name

• Analyze with tkprof– Or third party tools (Toad, Spotlight, others)

19

The best laid plans of Mice and Oracle....

20

Returning to captivity – changing the plan

• Options for improving the plan:– Indexing– Configuration changes (esp. Memory)– Stored outlines (stability)– SQL Tuning sets and profiles – 11g SQL Baselines (flexibility)– Hints and re-writes (last resort)

Use hints with extreme caution• Hints reduce optimizer flexibility and can lead to bad plans• Eg: USE_NL can force a nested loops join without an index• USE_NL_WITH_INDEX is safer..

22

Exploit baselines and plan management

23

SQL Baselines in SQL Optimizer

24

Indexing and single table lookups

25

Single table lookup

• Index or table scan?• Avoid accidental table scans • Optimize indexes

– best combination of concatenated indexes

• Optimize necessary table scans – Vertical/Horizontal partitioning– Compression– Parallel Query

0 10 20 30 40 50 60 70 80 90 100

1

10

100

1000

Full Scan no caching

Index sorted data, no caching

Index unsorted, cached data

Full Table scan, cached data

Pct of table accessed

Ela

sp

ed

Tim

e (

s)

Break even points for index vs table scan

None

last name

last+first name

last,first,BirthYear

last,first,birthyear,id

0 200 400 600 800 1000 1200 1400 1600

1459

63

6

4

3

Logical IO

SELECT cust_id

FROM sh.customers c

WHERE cust_first_name = 'Connor'

AND cust_last_name = 'Bishop'

AND cust_year_of_birth = 1976;

Concatenated Index Effectiveness

28

Bitmap indexes

29

Bitmap indexes

1 10 100 1000 10000 100000 10000000.01

0.1

1

10

100

Bitmap index B*-Tree index Full table scan

Distinct values in table

Ela

pse

d T

ime

(s)

30

Vertical partitioning

31

Joins

32

Optimizing joins

• Best join order – Eliminate rows as early as possible

• Join Type: – Nested loops

• Optimize the join index– Sort merge

• Avoid, esp. if memory scarce – Hash join

• Avoid multi-pass executions

Nested loops join

No Index

prod_id

prod_id,channel_id

time_id

prod_id,channel_id,cust_id,time_id,promo_id

0 100 200 300 400 500 600

546.55

48.36

23.43

3.14

2.2

Elapsed time (s)

Ind

exin

g

Sort-merge and hash join

1 10 100 10000

50

100

150

200

250

Hash Join Sort Merge Join

Workarea Memory (MB)

Ela

pse

d T

ime

(s)

In Memory

In Memory

Single pass disk sortMulti pass disk sort

Disk Sort

Bitmap join index

36

Bitmap join performance

SELECT SUM (amount_sold)

FROM customers JOIN sales s USING (cust_id) WHERE

cust_email='[email protected]';

Bitmap Join index

Bitmap index

Full table scan

0 2000 4000 6000 8000 10000 12000 14000

68

1,524

13,480

Logical IO

Acc

ess

Pat

h

37

Sorting

38

Sorting – what we expect

Table/Index IO CPU Time Temp Segment IO

PGA Memory available (MB)

Tim

e

Memory Sort

Single Pass

Disk Sort

Multi-pass

Disk Sort

39

Flash drive to the rescue?

0501001502002503000

500

1000

1500

2000

2500

3000

3500

4000

SAS based TTS SSD based TTS

Sort Area Size

Ela

pse

d t

ime

(s)

Single Pass

Disk Sort

Multi-pass

Disk Sort

0 200 400 600 800 1000 1200 1400 1600 1800 20000

50

100

150

200

250

300

350

400

450

Max process PGA Max workarea PGA

PGA_AGGREGATE_TARGET (MB)

Siz

e L

imit

(M

B)

Less memory than you may think....

41

Memory and sorting• Nothing matters as much as PGA to sort operations• Automatic work area management restricts you to a subset of

PGA• For big sorts, “opt out” of automatic work area management

ALTER SESSION SET workarea_size_policy = manual;

ALTER SESSION SET sort_area_size = 524288000;

42

Grouping

42

Hash Group by

• 11g introduced the Hash GROUP BY

• Using an ORDER BY can suppress the hash GROUP BY....

• Can override with USE_HASH_AGGREGATION

hint

GROUP BY SORT (10g)

GROUP BY HASH (11g)

0 1000 2000 3000 4000 5000 6000 7000 8000 9000

44

Analytic (windowing) functions

Partition by

Self Join

0 5000 10000 15000 20000 25000

CPU

temp IO

Table IO

Elapsed time (ms)

45

Pivot vs CASE

PIVOT

CASE

0 2000 4000 6000 8000 10000 12000 14000 16000 18000

CPU Time

IO Time

46

DML

47

DML tuning - indexes

1 (PK only)

2

3

4

5

6

7

0 2,000 4,000 6,000 8,000 10,000 12,000 14,000 16,000 18,000

1,191

6,671

8,691

10,719

12,727

14,285

16,316

Logical reads required

Nu

mb

er o

f in

dex

es

Multi-table insert

49

Two Inserts

Multi-table insert

0 1 2 3 4 5 6

Insert US

Insert EMEA

Insert both

Elapsed time (s)

Multi-table insert

INSERT + UPDATE

MERGE

0 1 2 3 4 5 6 7 8

3.89 3.71

3.32

Update

Insert

Merge

Elapsed Time (s)

Nested loops (indexed)

Hash (full scan)

0 1 2 3 4 5 6 7 8

4.23

7.96

Elapsed time (s)

Mer

ge

join

met

ho

d

Merge optimization

•The optimizer usually can’t determine

the overlap between Tables

•Forcing a Nested loops Merge outer

join may be significant

53

Other DML optimizations

• Array insert• Direct path• NOLOGGING• Commit frequency• NOWAIT and BATCH redo logging

54

Other topics

• Database logical design

• Clustering and exotic physical options

• PL/SQL

• Parallel SQL

• Application optimization (Arrays, bind variables)

55

Conclusion• Maximizing optimizer accuracy provides the best return on

investment• Sub-optimal optimizations are unfortunately inevitable• Key SQL tuning skills are therefore:

– Configuring Oracle to maximize optimizer effectiveness– Detection of sub-optimal SQLs– Techniques for coercing SQLs to acceptable performance

oraclesqlhighperformancetuning-110719182010-phpapp01

Documents