shapira oda perf_webinar_v2
DESCRIPTION
TRANSCRIPT
![Page 1: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/1.jpg)
An Insider’s Guide to ODA Performance
Prepared by: Alex Gorbachev, Pythian CTO & Gwen
Shapira
Presented by: Gwen Shapira, Senior Pythian Consultant
![Page 2: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/2.jpg)
Gwen ShapiraSenior Consultant, Pythian
Oracle Ace Director
Alex GorbachevCTO, Pythian
President, Oracle RAC SIG
![Page 3: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/3.jpg)
3
Why Companies Trust Pythian
Recognized Leader:• Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server
• Work with over 150 multinational companies such as Western Union, Fox Interactive Media, and MDS Inc. to help manage their complex IT deployments
Expertise:• One of the world’s largest concentrations of dedicated, full-time DBA expertise.
Global Reach & Scalability:• 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response
8
![Page 4: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/4.jpg)
4 © 2012 Pythian4
![Page 5: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/5.jpg)
5 © 2012 Pythian
Oracle Database Appliance
5
•Simple RAC-In-A-Box
•2 database servers + shared storage + interconnect
• Inexpensive
![Page 6: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/6.jpg)
We will talk about:
•Node Hardware•Interconnect•Storage•Benchmark results•Capacity planning tips
6
![Page 7: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/7.jpg)
What’s in a Server Node?
7
![Page 8: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/8.jpg)
8 © 2012 Pythian
ODA Front View
8
![Page 9: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/9.jpg)
9 © 2012 Pythian
ODA Rear View
9
![Page 10: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/10.jpg)
10 © 2012 Pythian
System Controller View
10
![Page 11: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/11.jpg)
11 © 2012 Pythian
System Controller View
11
![Page 12: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/12.jpg)
12 © 2012 Pythian
Server Node (SN) / System Controller (SC)
12
•Two X5675 - 3.06GHz, 6 core
•96G RAM
•Two SATA 7500 RPM, 500G disks
•Lots of network ports, both 1GbE and 10GbE
•Identical to X2-2 Exadata node
![Page 13: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/13.jpg)
13
Oracle Database Appliance Storage
13
•20 SAS 15000 RPM 600GB
•4 SAS SSD 73GB
•Each SN – 2 HBA
•Each SN – 2 Expanders
•Each Expander – 12 disks
•Each disk – 2 SAS ports
![Page 14: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/14.jpg)
14 © 2012 Pythian14
Only $50K
![Page 15: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/15.jpg)
Sound of a Single Node Scaling
15
![Page 16: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/16.jpg)
Cluster Interconnect
16
![Page 17: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/17.jpg)
Where’s the Interconnect?
[root@odaorcl1 ~]# /u01/app/11.2.0.3/grid/bin/oifcfg getif
eth0 192.168.16.0 global cluster_interconnect
eth1 192.168.17.0 global cluster_interconnect
bond0 172.20.31.0 global public
eth0 Link encap:Ethernet HWaddr 00:21:28:E7:C3:72
inet addr:192.168.16.24 Bcast:192.168.16.255
inet6 addr: fe80::221:28ff:fee7:c372/64
UP BROADCAST RUNNING MULTICAST MTU:9000
17
![Page 18: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/18.jpg)
[root@odaorcl1 ~]# ethtool eth0
Settings for eth0:
Supported ports: [ FIBRE ]
Supported link modes: 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: FIBRE
PHYAD: 0
Transceiver: external
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: d
Current message level: 0x00000001 (1)
Link detected: yes
18
![Page 19: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/19.jpg)
19 © 2012 Pythian
Interconnect Performance
Is 1GbE a problem?
•Dedicated 2 x 1 GbE Fibre links
•No switches
•IC latency ~ 0.5 ms.
•Like Exadata over IB
•Only 2 nodes
•Workload matters
19
![Page 20: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/20.jpg)
Throughput – 400 VUsers
20
![Page 21: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/21.jpg)
But Wait!
Event Waits Time(s) (ms) time Wait Class
------------------------------ ------------ ----------- ------ ------ DB CPU 6,459 29.9
buffer busy waits 123,162 3,725 30 17.3 Concurrenc
gc buffer busy release 8,871 3,383 381 15.7 Cluster
gc current block 2-way 3,282,774 1,969 1 9.1 Cluster
gc buffer busy acquire 11,073 1,364 123 6.3 Cluster
21
![Page 22: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/22.jpg)
But Wait!
Event Waits Time(s) (ms) time Wait Class
------------------------------ ------------ ----------- ------ ------ enq: US - contention 1,123,271 33,733 30 38.2 Other
enq: HW - contention 42,551 17,317 407 19.6 Configurat
buffer busy waits 156,152 11,550 74 13.1 Concurrenc
latch: row cache objects 798,648 6,181 8 7.0 Concurrenc
DB CPU 5,796 6.6
22
![Page 23: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/23.jpg)
I need that buffer.
I’m busy!
381 ms later:
Here’s the buffer!
Waiting
23
![Page 24: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/24.jpg)
Interconnect Again
Send ReceiveUsed By Mbytes/sec Mbytes/sec---------------- ----------- -----------Global Cache 48.94 43.04Parallel Query .00 .00DB Locks 4.99 5.23DB Streams .00 .00Other .00 .01
24
Instance Latency 500B MSG
Latency8K MGS
1 0.14 0.13
2 0.58 0.69
![Page 25: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/25.jpg)
Storage Performance-
REDO LOG
25
![Page 26: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/26.jpg)
26 © 2012 Pythian
No Storage Cache
26
Implications:
•Excessive IO will impact latency
•Online redo logs are on SSD
•Tune DBWR processes (MTTR target)
![Page 27: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/27.jpg)
SSD
•4x 73GB
•Dedicated to redo logs
•Reminder:
•0.025ms read
•0.250ms write (best case)
•Writes are not just writes
•Over-provisioning
27
![Page 28: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/28.jpg)
28 © 2012 Pythian28
![Page 29: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/29.jpg)
SSD for Redo
29
•Not a general recommendation
•Consistent low latency
•Works well for multiple databases
•Leftover space
![Page 30: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/30.jpg)
ODA: SSD Performance for LGWR
30
![Page 31: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/31.jpg)
More LGWR Performance
Saturating LGWR Test
• 3200 writes, 2 nodes, 0.2ms latency
• LGWR spent 70% of time on CPU
SwingBench Order Entry
• 4500 TPS
• Bottleneck was buffer busy contention
Big data load
• 100K size write, several ms latency
• Data warehouse load – bad fit for ODA
31
![Page 32: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/32.jpg)
Storage Performance-
DATA
32
![Page 33: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/33.jpg)
HDD Performance
We tested:
•HDD Scalability
•Effects of disk placement
•Backups!
33
![Page 34: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/34.jpg)
34 © 2012 Pythian34
ODA Small Random Reads - HDDs Scalability
![Page 35: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/35.jpg)
35 © 2012 Pythian35
ODA Write IO impact - Minimal
![Page 36: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/36.jpg)
ODA Write IO impact - Minimal
36
![Page 37: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/37.jpg)
37 © 2012 Pythian
ODA Small Random Reads: Data Placement
37
![Page 38: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/38.jpg)
38
Co-locating data onto outer 40% of a disk adds 50% more
IOPS
![Page 39: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/39.jpg)
39 © 2012 Pythian
ODA Sequential Reads Scalability (Single node)
39
I could reach 2.4 GBPS with 24 parallel
reads for a single stream
![Page 40: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/40.jpg)
40 © 2012 Pythian
RMAN Backup Performance (1)
40
Backup to FRA:• Optimal number of channels - 8• 42 GB of data in 1 min 45 seconds = 400 MBPS• 1.6 TB full backup in about 1 hour
![Page 41: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/41.jpg)
41 © 2012 Pythian
RMAN Backup Performance (2)
41
Backup to external location:• BACKUP VALIDATE with 8 channels• 42 GB of data in 45 seconds = 1 GBPS•Theoretical maximum wire speed for one link 10 GbE
• 4 TB database in 1 hour 15 minutes
![Page 42: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/42.jpg)
42 © 2012 Pythian
Configurations of note:
42
![Page 43: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/43.jpg)
Capacity Planning forMigration or Consolidation
43
![Page 44: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/44.jpg)
Choosing Consolidation Candidates
•Vendor limitations•SLAs•Dependencies•CPU utilization•Workload type
Big Question: Will it fit?
44
![Page 45: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/45.jpg)
Collect metrics
•CPU utilization•Memory usage – SGA + PGA•Storage requirements•Workload types•I/O requirements – IOPS, throughput•RAC – current interconnect load
45
![Page 46: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/46.jpg)
CPU
Build time-based model of utilization on existing servers:
46
Time S1 (8 core)
S2 (4 core)
S3 (32 core)
Total
00:00 50% 25% 10% 8*0.5+4*0.25+32*0.1 = 8.2
00:15 30% 50% 10% 7.6
00:30 100% 25% 10% 12.2We calculated 12.2 cores in use at peak time.
ODA’s 24 cores give plenty of spare capacity
You can get more accurate results by taking core speed into account. This is a rough model.
![Page 47: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/47.jpg)
Memory
•Easiest way: Sum memory on existing servers
•Actually: Sum SGA and PGA sizes, and leave 20-30% spare
Use advisors:•OEM gives graphs with SGA and PGA size recommendations.
47
![Page 48: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/48.jpg)
IO Capacity
•OLTP and DWH go in separate boxes•Each can be standby of the other•Consider throughput and latency requirements•According to our tests:• 12K redo IOPS at 0.5 ms latency
• Over 3000 data file IOPS at 15ms latency
• Almost 6000 if using outside only
• Can reach 2.4GBPS
48
![Page 49: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/49.jpg)
Disk Space•High redundancy – triple data usage•Can use external storage if needed•ZFS supports HCC•Take backups into account
49
![Page 50: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/50.jpg)
Testing
•Always test•Bad tests are still better than no tests•Replicating production load:• RAT
• “Brewing Benchmarks”
• Jmeter, Loadrunner, etc
•Especially test:• Migration strategy and times
• Non-RAC applications going to RAC
• Upgrades
50
![Page 51: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/51.jpg)
51 © 2012 Pythian
Oracle Database Appliance Requires 11.2.0.2
51
We will upgrade andmigrate your DB
to ODA for free
![Page 52: Shapira oda perf_webinar_v2](https://reader033.vdocuments.net/reader033/viewer/2022061223/54c658814a7959d9368b45d6/html5/thumbnails/52.jpg)
© 2012 Pythian52
http://www.pythian.com/news/
http://on.fb.me/pythianfacebook
@pythian
http://linkd.in/pythian
1-877-PYTHIAN [email protected]
To contact us…
To follow us…
Thank you and Q&A
@pythianjobs
Gwen Shapira – [email protected] Alex Gorbachev – [email protected]