oow 2013: where did my cpu go
TRANSCRIPT
Where did my CPU go? Session ID# UGF9790
Karl Arao Enkitec
whoami
Karl Arao • Senior Technical Consultant @ Enkitec • Performance and Capacity Planning Enthusiast
7 years DBA experience
Oracle ACE, OCP-‐DBA, RHCE, OakTable Blog: karlarao.wordpress.com Wiki: karlarao.Kddlyspot.com TwiLer: @karlarao
www.enkitec.com 2
www.enkitec.com 3
200+
3
Booth #501 Moscone
South
Agenda
• The different CPU events
• CPU Monitoring/Capacity Planning on consolidated environments • HOWTO compare CPU speeds
• Cores vs Threads
www.enkitec.com 4
www.enkitec.com 5
12:27:15 SYS@DEMO1> show parameter cpu_count!!NAME TYPE VALUE!------------------------------------ ----------- --------!cpu_count integer 16!
www.enkitec.com 6
Socket0 Core0
CPU0 CPU8
Core1
CPU1 CPU9
Core2
CPU2 CPU10
Core3
CPU3 CPU11
Socket1 Core0
CPU4 CPU12
Core1
CPU5 CPU13
Core2
CPU6 CPU14
Core3
CPU7 CPU15
Exadata V2 => 2s8c16t
PART1: Different CPU events CPU
CPU Wait CPU Scheduler
www.enkitec.com 7
www.enkitec.com 8
AAS CPU
www.enkitec.com 9
CPU Wait
www.enkitec.com 10
www.enkitec.com 11
CPU Scheduler
www.enkitec.com 12
www.enkitec.com 13
www.enkitec.com 14
Pu^ng it all together
Instances Caged at 12 CPUs
SQL Applied to lock in good plan.
Problem: A single SQL Stmt. overwhelming CPU resources.
PART2: CPU monitoring and Capacity Planning
www.enkitec.com 15
OS Tools • The usual OperaKng System commands
– vmstat – top – mpstat –P ALL 1 5
• Cool tools – collectl –sC (hLp://collectl.sourceforge.net) – turbostat.c – dcli (Exadata)
• dcli -‐l oracle -‐g /home/oracle/dbs_group -‐-‐vmstat 2 • dcli -‐l oracle -‐g /home/oracle/dbs_group upKme
www.enkitec.com 16
www.enkitec.com 17
Load Map
www.enkitec.com 18
Performance Page – Historical View
AWR Toolkit • DIY performance data warehouse
www.enkitec.com 19
run_awr run_extract
Extract AWR data points as csv files Package all the csv files Customer site
FRESH_LOAD CHECK_LOAD DELTA_LOAD
Create new client “dimension” tables Check new data points Load new data points
DIY DW server
awr_topevents_(ClientNameX) awr_cpuwl_(ClientNameX) awr_iowl_(ClientNameX)
1
2
3 Tableau AnalyKcs
awr_topevents_(ClientNameY) awr_cpuwl_(ClientNameY) awr_iowl_(ClientNameY)
awr_topevents_(ClientNameZ) awr_cpuwl_(ClientNameZ) awr_iowl_(ClientNameZ)
www.enkitec.com 20
• Tableau auto creates a Kme dimension for the Kme column “MM/DD/YY HH24:MI:SS” of AWR csv output
www.enkitec.com 21
www.enkitec.com 22
• Summary and Underlying data
1-‐2AM
2-‐3AM
www.enkitec.com 23
CPU usage across half rack Exadata
www.enkitec.com 24
CPU usage per host
www.enkitec.com 25
CPU redistribuKon across nodes
PART3: compare CPU speeds
www.enkitec.com 26
Different methods:
• Published benchmarks – TPC-‐C – SPECint_rate2006
• Actual Benchmarking – cputoolkit – SLOB (lio test)
www.enkitec.com 27
TPC-‐C • TransacKon Processing Performance Council (TPC)
• Throughput => transacKons per minute (tpmC) • Price/Performance => USD / tpmC
www.enkitec.com 28
• CPU performance => tpmC / core • 1609186.39 / 16 = 100574
SPECint_rate2006 • Standard Performance EvaluaKon CorporaKon (SPEC)
• SPECint_rate2006 • Integer performance • All CPUs are used • Used by OEM12c ConsolidaKon Planner (SYSMAN.EMCT_SPEC_RATE_LIB)
• CPU performance => SPECint_rate2006/core • 702/16 = 43.875
www.enkitec.com 29
$ cat spec.txt | grep -i sun | grep -i x3-2 | sort -rnk1!44.0625, 16, 2, 8, 2, 632, 705, Oracle Corporation, Sun Blade X3-2B (Intel Xeon E5-2690 2.9GHz)!44.0625, 16, 2, 8, 2, 630, 705, Oracle Corporation, Sun Server X3-2L (Intel Xeon E5-2690 2.9GHz)!43.875, 16, 2, 8, 2, 628, 702, Oracle Corporation, Sun Server X3-2 (Intel Xeon E5-2690 2.9GHz)!!
2007 vs 2012
www.enkitec.com 30
tpmC/core, System, tpmC, Price/Perf, Total System Cost, Currency, Database Software, Server CPU Type, Total Server Cores, Cluster, Date Submitted
Result/# Cores, # Cores, # Chips, # Cores Per Chip, # Threads Per Core, Baseline, Result, Hardware Vendor, System
Actual Benchmarking • cputoolkit and SLOB (lio test) • LIOs/sec
www.enkitec.com 31
CPU1
CPU2
CPU3
CPU4
CPU5
CPU6
CPU7
CPU8
CPU1
CPU2
CPU3
CPU4
CPU5
CPU6
CPU7
CPU8
cputoolkit ./runcputoolkit-‐auto <start CPU> <end CPU> <db name> ./runcputoolkit-‐auto 1 2 dw
SLOB ./runit.sh <writers> <readers> while :; do ./runit.sh 0 2; done
Both at 25% CPU u[liza[on
V2 and X2 CPU perf comparison
www.enkitec.com 32
3.6M LIOs/sec
2.1M LIOs/sec
V2 -‐> X2 migraKon
www.enkitec.com 33
chip efficiency factor = (source LIOs/sec) / (desKnaKon LIOs/sec) = 2.1M / 3.6M = .5833 X2 CPU requirement = source host CPUs * uKlizaKon * chip efficiency factor = 16 * .46 = 7.36 * .5833 = 4.29 CPUs X2 CPU UKlizaKon = CPU requirement / CPU capacity = 4.29 / 24 = 17.8 %
V2 X2
PART4: Cores vs Threads
www.enkitec.com 34
Socket0 Core0 Core1 Core2 Core3
Socket0 Core0
CPU1 CPU5
Core1
CPU2 CPU6
Core2
CPU3 CPU7
Core3
CPU4 CPU8
www.enkitec.com 35
~30% depends on the workload
www.enkitec.com 36
cputoolkit SLOB
17% 21%
Intel HT Technology Technical User's Guide http://goo.gl/3Ec5Z
Pu^ng them all together
• Enkitec has developed a tool called Provisioning Worksheet that is mainly used for sizing and consolidaKon of databases.
– AccounKng Per database CPU usage not like OEM12c ConsolidaKon Planner which is just host level CPU uKlizaKon
– Consolidated database workload analysis across mulKple servers – CPU, memory (SGA/PGA & HugePages), storage capacity (storage & IOPS) planning – All the capacity planning math integrated with the tool (chip efficiency factor across mulKple servers) – Layout instances across nodes according to Preferred, Available, Failover – Create a baseline of the instances node layout and their current uKlizaKon – Simulate node failure and different what-‐if scenarios (add/delete node, increase/decrease capacity,
etc.) – VisualizaKon module to quickly see node level resource imbalance – Cluster and node level uKlizaKon – Recommended Hardware (equivalent compute nodes)
• See the tool here hLp://www.slideshare.net/karlarao/pape-‐rkaraoconsolidaKon-‐successstory
www.enkitec.com 37
Wrap up! • The different CPU events
o 1 AAS CPU = 1 CPU core o Oracle CPU may not correlate with Host CPU if you have a lot of
CPU acKvity outside of the database
• CPU Monitoring/Capacity Planning on consolidated environments o AWR analyKcs
• HOWTO compare CPU speeds o SPECint_rate2006, TPC-‐C, Actual benchmarking
• Cores vs Threads o Always have HT on o ~30% performance benefit awer core count
www.enkitec.com 38
Read the WhitePaper
Where did my CPU go? hLp://goo.gl/eq9Sn
www.enkitec.com 39
Resources • Whitepaper: Where did my CPU go? hLp://goo.gl/eq9Sn • cputoolkit -‐ hLp://karlarao.wordpress.com/scripts-‐resources/ • AWR Tableau and R toolkit VisualizaKon Examples -‐ hLp://goo.gl/xZHHY • AAS invesKgaKon -‐ hLp://goo.gl/5WaAg • Cores vs Threads -‐ hLp://goo.gl/1MLFf • Turbostat.c -‐ hLp://goo.gl/jDUKg • cpu_topology -‐ hLp://goo.gl/EUDG7 • CPU centric benchmark comparisons -‐ hLp://goo.gl/nR9Yy • SLOB -‐ hLp://goo.gl/yKa45 • Kyle Hailey -‐ hLp://dbopKmizer.com/2011/07/21/oracle-‐cpu-‐Kme/ • The mindmap of this presentaKon -‐ hLp://goo.gl/XeY0e
www.enkitec.com 40
– Network with 5,000+ database and
applicaKon professionals – 5 days of in-‐depth educaKon built by
users, for users – Complimentary Pre-‐Conference
Workshops for IOUG registrants
April 7 – 11, 2014 The Venetian Resort,
Las Vegas, NV
A]end for free! IOUG’s Call for Speakers is now open collaborate14.ioug.org/call-‐for-‐speakers
Communities
Maximum Availability Architecture with Oracle’s Larry Carpenter and Joe Meeks October 15: Milwaukee, WI October 16: Rochester, NY Performance and Internals with Craig Shallahamer November 6: Chicago, IL November 8: Atlanta, GA Get more informa[on at www.ioug.org/masterclass or visit the IOUG kiosk in Moscone West, 2nd floor
IOUG Master Class Series Returns Coming soon to a city near you!
One Day Interac[ve Training Events to meet your educa[onal needs
Win a Free Registration to COLLABORATE 14!
Stop by the IOUG kiosk in the User Group Pavilion in Moscone West, 2nd floor, to pick up a lucky poker chip and see if you are a winner of a free registration to
COLLABORATE 14 in Las Vegas!
Complimentary eBook Register Now
www.mhprofessional.com/dbsec
Use Code: db12c
THANK YOU