hitchhikers guide to free oracle tuning tools rost - free oracle tuning tools.pdfmoats: the mother...
TRANSCRIPT
hitchhikers guide to free Oracle tuning toolsa quick guide
Björn Rost
© 2015 Pythian Confidential2
• Consultant
– Oracle Database HA, Tuning, etc
– Linux&Solaris automation
• Oracle ACE Director
• (ex) president RAC SIG
• organizer OTN EMEA tour
© 2016 Pythian3
ABOUT ME
ABOUT PYTHIAN
Pythian’s 400+ IT
professionals help companies
adopt
and manage disruptive
technologies to better
compete
© 2016 Pythian. Confidential 4
© 2016 Pythian. Confidential 5
Systems currently
managed by Pythian
EXPERIENCED
Pythian experts
in 35 countries
GLOBAL
Millennia of experience
gathered and shared over
19 years
EXPERTS
11,800 2400
© 2016 Pythian. Confidential 6
TECHNICAL EXPERTISE
CLOUD
ADVANCED
ANALYTICS
DATABASES
BIG DATA
DEVOPS
Using the disruptive nature of cloud for accelerated, cost-effective growth
INFRASTRUCTURE
MANAGEMENT
Mining data for insights & business transformation using data science
Ensuring databases are reliable, secure, available and continuously optimized
Harnessing the transformative power of data on a massive scale
Providing critical velocity in software deployment by adopting DevOps practices
Transforming and managing the IT infrastructure that supports the business
WHY ORACLE?
7
• basic operations same
between DBs
• instrumentation makes the
difference
• allows to deeply investigate
and explain issues
© 2016 Pythian
(WRONG) TUNING CYCLE
© 2016 Pythian8
guess
change
hope
DON’T BE A STORMTROOPER DBA
© 2016 Pythian9
TUNING CYCLE
© 2016 Pythian10
understand
change
measure
TUNING IS SIMPLE, REALLY
11 © 2016 Pythian
LAST WEEK’S EXAMPLE
• increasing gap in standby db– change max_archive_processes parameter
– change some other parameters
– try rman recovery vs MRP
– increase parallelism
– reduce parallelism
– try more silver bullets…
– have SRE look into IO issues
– have SRE look at CPU issues
12 © 2016 Pythian
guess
change
hope
LAST WEEK’S EXAMPLE
• select event from v$session
where program like '%MRP%’
– PX deq credit: send blkd
– PX deq credit: need buffer
• search for that on MOS– 10.2 Parallel Standby Recovery Process Is Very Slow (Doc ID 420337.1)
• PARALLEL_EXECUTION_MESSAGE_SIZE=16k
13 © 2016 Pythian
understand
change
measure
TYPES
14
report real-time historic drill-down
© 2016 Pythian
ENTERPRISE MANAGER
• incredibly useful and thought through
• from vendor, perfectly integrated
• many common views and methods– coloured AAS view
– Top-SQL view
– Real-Time SQL monitor
– blocking session tree
– AWR Reports
– ADDM and other advisors
16 © 2016 Pythian
ENTERPRISE MANAGER
© 2015 Pythian Confidential17
SO WHY USE CLI TOOLS?
• not licensed for Diag&Tuning
– Standard Edition
– EE without Diag&Tuning (a cardinal sin)
• very low throughput or no GUI access
• EM not setup/installed
– 12c EM express can get the job done
• you simply want to look like a nerd
18 © 2016 Pythian
WHY OPEN SOURCE
• see and learn– you can actually look at the code you are running
• collaborate and contribute– want a new feature? implement yourself!
• free(*) means– no costs (duh)
– get up and running really fast
– don’t worry about audits
19 © 2016 Pythian
© 2015 Pythian Confidential20
RLSQLPLUS
• sqlplus on steroids
• edit sqlplus CLI
• history
– up/down arrow
– search with CTRL-r
• auto-completion
– against custom wordlist
21 © 2016 Pythian
RLSQLPLUS INSTALLATION STEP1
22
[root@kickpuppet ~]# yum install http://dl.fedoraproject.org/pub/epel/6/x86_64/rlwrap-0.42-1.el6.x86_64.rpm
================================================================================
Package Arch Version Repository Size
================================================================================
Installing:
rlwrap x86_64 0.41-1.el6 epel 92 k
Transaction Summary
================================================================================
Install 1 Package(s)
Installing : rlwrap-0.41-1.el6.x86_64 1/1
Verifying : rlwrap-0.41-1.el6.x86_64 1/1
Installed:
rlwrap.x86_64 0:0.41-1.el6
Complete!
© 2016 Pythian
RLSQLPLUS INSTALLATION STEP2
• optional: download or generate wordfile
– for autocompletion
23
$ curl -o wordfile.txt
http://blog.tanelpoder.com/files/scripts/setup/wordfile_11gR2.txt
http://blog.tanelpoder.com/files/scripts/setup/
© 2016 Pythian
RLSQLPLUS INSTALLATION STEP 3
• create aliases around sqlplus
– and rman
– and dgmgrl
• avoid overriding the original name
24
alias rlsqlplus='rlwrap -D2 -irc -b'\''"@(){}[],+=&^%#;|\'\'' -f ~/wordfile.txt
sqlplus'
alias rlrman='rlwrap -D2 -irc -b'\''"@(){}[],+=&^%#;|\'\'' rman'
© 2016 Pythian
SQLCL
• sqlplus “replacement” from SQLdeveloper team
– history and editor
– auto-completion
– auto-formatting
• ansiconsole for col widths
• json, xml, csv etc
– tons of good stuff
25 © 2016 Pythian
© 2015 Pythian Confidential26
AAS VIEW
• my favourite real-time DB dashboard
• one-glance DB activity overview
• colour-coded
– red -> blocking, app, concurrency
– blue -> IO, missing indexes, etc
– green -> CPU, executing too much?
– other -> weird, take a look
27 © 2016 Pythian
AAS VIEW
28 © 2016 Pythian
WEB-ASH
29
• simple and free
• reads v$session by default
• stores data in plain files
• lean javascript to display
http://datavirtualizer.com/web-ash-w-ash/
© 2016 Pythian
MOATS
• top-like output that refreshes
• excellent overview
• needs small installation on database
• samples v$-views while running
• improved forks
– MOATS 2.0
– SQL Dashboard
30 © 2016 Pythian
MOATS OUTPUT
31
MOATS: The Mother Of All Tuning Scripts v1.0 by Adrian Billington & Tanel Poder
http://www.oracle-developer.net & http://www.e2sn.com
+ INSTANCE SUMMARY ------------------------------------------------------------------------------------------+
| Instance: KICKPUPPET | Execs/s: 581.6 | sParse/s: 60.0 | LIOs/s: 4989.5 | Read MB/s: 0.7 |
| Cur Time: 19-Nov 21:11:09 | Calls/s: 164.2 | hParse/s: 0.0 | PhyRD/s: 86.7 | Write MB/s: 1.6 |
| History: 0h 9m 45s | Commits/s: 50.3 | ccHits/s: 578.8 | PhyWR/s: 112.7 | Redo MB/s: 0.2 |
+------------------------------------------------------------------------------------------------------------+
+ TOP SQL_ID (child#) -----+ TOP SESSIONS ---------+ + TOP WAITS -------------------------+ WAIT CLASS -+
| 29% | 7t0959msvyt5g (0) | 78,65 | | 29% | db file sequential read | User I/O |
| 14% | () | | | 29% | ON CPU | ON CPU |
| 14% | 147a57cxq3w5y (0) | 70 | | 14% | oracle thread bootstrap | Other |
| 14% | gh2g2tynpcpv1 (0) | 65 | | 14% | resmgr:cpu quantum | Scheduler |
| 14% | 4phvdvx32a3mf (0) | 73 | | 14% | db file parallel read | User I/O |
+--------------------------------------------------+ +--------------------------------------------------+
+ TOP SQL_ID ----+ PLAN_HASH_VALUE + SQL TEXT ---------------------------------------------------------------+
| 7t0959msvyt5g | 856749079 | SELECT ORDER_ID, ORDER_DATE, ORDER_MODE, CUSTOMER_ID, ORDER_STATUS, ORD |
| | | ER_TOTAL, SALES_REP_ID, PROMOTION_ID, WAREHOUSE_ID, DELIVERY_TYPE, COST |
+ ---------------------------------------------------------------------------------------------------------- +
| 147a57cxq3w5y | 0 | BEGIN :1 := orderentry.browseproducts(:2 ,:3 ,:4 ); END; |
+ ---------------------------------------------------------------------------------------------------------- +
| gh2g2tynpcpv1 | 0 | INSERT INTO CUSTOMERS ( CUSTOMER_ID , CUST_FIRST_NAME , CUST_LAST_NAME |
| | | , NLS_LANGUAGE , NLS_TERRITORY , CREDIT_LIMIT , CUST_EMAIL , ACCOUNT_MG |
+ ---------------------------------------------------------------------------------------------------------- +
| 4phvdvx32a3mf | 0 | begin prvt_ilm.stopjobs(-1,true,true,:1); end; |
+ ---------------------------------------------------------------------------------------------------------- +
© 2016 Pythian
MOATS 2.0 OUTPUT
32 © 2016 Pythian
SNAPPER
• realtime session monitor and drill-downs
• just one sqlscript to run– no installation needed
• runs for sample period– “snaps” v$session etc
– reports on diffs
• two modes– “ash” for v$session
– stats for v$sesstat
33
http://tech.e2sn.com/oracle-scripts-and-tools/session-snapper
© 2016 Pythian
SNAPPER ARGUMENTS
• snapper ash 5 1 all
– mode can be ash, stat or custom column list
– seconds to run/sample
– number of runs
– “all” or filter on v$session columns
• sid=42
• username=brost
34 © 2016 Pythian
SNAPPER DEFAULT MODE
35
SQL> @snapper ash 5 1 all
Sampling SID all with interval 5 seconds, taking 1 snapshots...
-- Session Snapper v4.15 - by Tanel Poder ( http://blog.tanelpoder.com )
-- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet! :)
----------------------------------------------------------------------------------------------------
Active% | INST | SQL_ID | SQL_CHILD | EVENT | WAIT_CLASS
----------------------------------------------------------------------------------------------------
10% | 1 | | 0 | db file async I/O submit | System I/O
10% | 1 | | | log file sync | Commit
7% | 1 | 5ckxyqfvu60pj | 0 | db file sequential read | User I/O
7% | 1 | | 0 | log file parallel write | System I/O
5% | 1 | | | ON CPU | ON CPU
5% | 1 | 3fw75k1snsddx | 0 | db file sequential read | User I/O
5% | 1 | | 0 | ON CPU | ON CPU
5% | 1 | c13sma6rkr27c | 0 | db file parallel read | User I/O
5% | 1 | gzhkw1qu6fwxm | 0 | ON CPU | ON CPU
5% | 1 | 0w2qpuc6u2zsp | 0 | ON CPU | ON CPU
-- End of ASH snap 1, end=2016-04-24 15:24:37, seconds=5, samples_taken=42
© 2016 Pythian
AAS IN SNAPPER
36
SQL> @snapper ash=wait_class 5 1 all
Sampling SID all with interval 5 seconds, taking 1 snapshots...
-- Session Snapper v4.11 - by Tanel Poder ( http://blog.tanelpoder.com )
-- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet!
-------------------------
Active% | WAIT_CLASS
-------------------------
44% | System I/O
19% | Commit
7% | Scheduler
7% | ON CPU
5% | User I/O
-- End of ASH snap 1, end=2014-11-19 16:23:06, seconds=5, samples_taken=43
© 2016 Pythian
SNAPPER STATS MODE
37
SQL> @snapper stats,gather=tw 15 1 sid=70
Sampling SID sid=70 with interval 15 seconds, taking 1 snapshots...
-- Session Snapper v4.11 BETA - by Tanel Poder ( http://blog.tanelpoder.com ) - Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet!
:)
---------------------------------------------------------------------------------------------------------------------------------------------------
SID, USERNAME , TYPE, STATISTIC , DELTA, HDELTA/SEC, %TIME, GRAPH , NUM_WAITS, WAITS/SEC, AVERAGES
---------------------------------------------------------------------------------------------------------------------------------------------------
70, SOE , TIME, repeated bind elapsed time , 300, 19.8us, .0%, [ ], , ,
70, SOE , TIME, parse time elapsed , 9272, 611.99us, .1%, [ ], , ,
70, SOE , TIME, PL/SQL execution elapsed time, 230999, 15.25ms, 1.5%, [# ], , ,
70, SOE , TIME, DB CPU , 569000, 37.56ms, 3.8%, [@ ], , ,
70, SOE , TIME, sql execute elapsed time , 1730975, 114.25ms, 11.4%, [## ], , ,
70, SOE , TIME, DB time , 2038543, 134.55ms, 13.5%, [## ], , , .88 % unaccounted
70, SOE , WAIT, log file sync , 235408, 15.54ms, 1.6%, [W ], 183, 12.08, 1.29ms average
70, SOE , WAIT, db file sequential read , 1123224, 74.14ms, 7.4%, [W ], 397, 26.2, 2.83ms average
70, SOE , WAIT, db file parallel read , 52763, 3.48ms, .3%, [ ], 3, .2, 17.59ms average
70, SOE , WAIT, latch: In memory undo latch , 499, 32.94us, .0%, [ ], 1, .07, 499us average
70, SOE , WAIT, resmgr:cpu quantum , 51794, 3.42ms, .3%, [ ], 3, .2, 17.26ms average
70, SOE , WAIT, SQL*Net message to client , 1458, 96.23us, .0%, [ ], 393, 25.94, 3.71us average
70, SOE , WAIT, SQL*Net message from client , 60177, 3.97ms, .4%, [ ], 393, 25.94, 153.12us average
70, SOE , WAIT, PL/SQL lock timer , 12918348, 852.66ms, 85.3%, [WWWWWWWWW ], 1328, 87.65, 9.73ms average
70, SOE , WAIT, events in waitclass Other , 960, 63.36us, .0%, [ ], 2, .13, 480us average
-- End of Stats snap 1, end=2014-11-20 08:04:39, seconds=15.2
© 2016 Pythian
TOP SQL
38
SQL> @snapper ash=sql_id+wait_class 15 1 user=soe
Sampling SID user=soe with interval 15 seconds, taking 1 snapshots...
-- Session Snapper v4.11 BETA - by Tanel Poder ( http://blog.tanelpoder.com )
-- Enjoy the Most Advanced Oracle Troubleshooting Script on the Planet! :)
-------------------------------------------
Active% | SQL_ID | WAIT_CLASS
-------------------------------------------
14% | | Commit
5% | 7hk2m2702ua0g | User I/O
4% | 5ckxyqfvu60pj | User I/O
3% | 7t0959msvyt5g | User I/O
3% | gh2g2tynpcpv1 | User I/O
2% | 0w2qpuc6u2zsp | Scheduler
2% | g81cbrq5yamf5 | User I/O
2% | 7ws837zynp1zv | User I/O
2% | f7rxuxzt64k87 | User I/O
2% | 1qf3b7a46jm3u | User I/O
-- End of ASH snap 1, end=2014-11-20 06:14:42, seconds=15, samples_taken=100
© 2016 Pythian
SQLDEVELOPER
39 © 2016 Pythian
SQLDEVELOPER
40 © 2016 Pythian
HISTORIC PERFORMANCE
• moats, snapper etc only sample when running
• EE (+diag/tuning) has ASH and AWR
41 © 2016 Pythian
AWR REPORTS -> STATSPACK
• good for a high-level overview
• AWR has a bit more “stuff” than statspack
• AWR is setup automatically (60min snaps)
– be mindful of averages
• but AWR needs EE and diag&tuning pack
42 © 2016 Pythian
STATSPACK IN 12C
• no problem in non-CDB
• with multitenant would make most sense in CDB
– but not really supported due to common user c#...
• there is a nasty and unsupported workaround
43
https://jonathanlewis.wordpress.com/2013/07/04/12c-statspack-hack/
© 2016 Pythian
ORASASH
• does pretty much what ASH does (but for free)
• sample and store information from v$-views
• mostly compatible with EE ASH
– ashmon
– ashmasters etc
• stores data from multiple sources in repo db
– minimal footprint in source, only view and dblink
44
http://pioro.github.io/orasash/
© 2016 Pythian
SQLT
• oracle supported scripts
• needs a schema an installation in db
– (this is annoying)
• will gather *everything* about a single SQL
– option to use diag&tuning
• output is one zipped html archive
45
Metalink Doc id 215187.1
© 2016 Pythian
SQLD360
• sql scope (just like SQLT)
• needs no installation
• one zipfile including html output
46
http://mauro-pagano.com/2015/02/16/sqld360-sql-diagnostics-collection-made-faster/
© 2016 Pythian
SQLD SCREENSHOTS
47 © 2016 Pythian
SQLD SCREENSHOTS
48 © 2016 Pythian
SQLD INCLUDES SQL
49
SELECT MIN(a.snap_id) snap_id,
TO_CHAR(a.begin_interval_time, 'YYYY-MM-DD HH24:MI') begin_time,
TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI') end_time,
NVL(TRUNC(SUM(b.elapsed_time_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) elapsed_time,
NVL(TRUNC(SUM(b.cpu_time_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) cpu_time,
NVL(TRUNC(SUM(b.iowait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) iowait,
NVL(TRUNC(SUM(b.clwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) clwait,
NVL(TRUNC(SUM(b.apwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) apwait,
NVL(TRUNC(SUM(b.ccwait_total/NVL(NULLIF(executions_total,0),1)/1e6),3),0) ccwait
FROM (SELECT snap_id, instance_number, elapsed_time_total, cpu_time_total,
iowait_total, clwait_total, apwait_total, ccwait_total, executions_total
FROM dba_hist_sqlstat
WHERE sql_id = 'gm9dkz4u1hwfv') b,
(SELECT snap_id, instance_number, begin_interval_time, end_interval_time
FROM dba_hist_snapshot
WHERE snap_id BETWEEN 0 AND 166142) a
WHERE a.snap_id = b.snap_id(+)
AND a.instance_number = b.instance_number(+)
AND 'Y' = 'Y'
AND a.instance_number = a.instance_number
GROUP BY
TO_CHAR(a.begin_interval_time, 'YYYY-MM-DD HH24:MI'),
TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI')
ORDER BY
TO_CHAR(a.end_interval_time, 'YYYY-MM-DD HH24:MI');
1850 rows selected.
© 2016 Pythian
EDB360
• database scope
• no installation needed!
– just run a sqlfile
• creates one zipfile with html for offline viewing
– also AWR etc
• T | D | N control for access packs
• convenient: graphs
50
http://www.enkitec.com/products/edb360
© 2016 Pythian
EDB SCREENSHOTS
51 © 2016 Pythian
TUNAS360
• best of sqld and snapper
• samples v$-views for period
– works without AWR/ASH license!
• produces output like sqld
52 © 2016 Pythian
TUNAS SCREENSHOTS
53 © 2016 Pythian
TUNAS SCREENSHOTS
54 © 2016 Pythian
GOOD OLD 10046 TRACE
• guaranteed to catch everything (unlike sampling)
– why not turn on instance-wide?
• overhead vs usefulness?
• no license needed at all
55 © 2016 Pythian
READING TRACEFILES
• sqldeveloper
• trcanlzr– needs installing but now also part of SQLT
• Kyle Hailey’s script– https://github.com/khailey/oracle_trace_parsing/
• Chris Antognini’s script– https://antognini.ch/downloads/top2/chapter03/tvdxtat
_40beta10_20140630.zip
56 © 2016 Pythian
REVIEW
• make sqlplus great again– rlsqlplus
– sqlcl
• real-time analysis– snapper
– moats
– web-ash
– sqldeveloper instance viewer
– TUNAs360
57
• historical analysis– Orasash / S-ASH
– statspack
– SQLT
– eDB360
– sqld360
• 10046 tracing– raw
– various analyzers
© 2016 Pythian