real world performance benchmarking

32
1 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Upload: prasad-chandane

Post on 08-Jan-2017

360 views

Category:

Technology


3 download

TRANSCRIPT

Page 1: Real World Performance Benchmarking

1 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Page 2: Real World Performance Benchmarking

2 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Real World Performance Benchmarking Oracle Transportation Management/Global Trade Management August 2012

Page 3: Real World Performance Benchmarking

3 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Safe Harbor Statement

• The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Page 4: Real World Performance Benchmarking

4 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Agenda

• Internal Exa-data/Exa-logic Certification Testing • Customer Exa-data/Exa-logic POC • Other Performance Activities • Performance Benchmarking Advice

Page 5: Real World Performance Benchmarking

5 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Exalogic/Exadata Performance Update

• The OTM Exadata/Exalogic performance comparisons were captured as part of the OTM Certification testing for Exadata/Exalogic. These tests were not intended to be a true benchmark for directly comparing Exa/non-Exa hardware alternatives. In the future OTM may perform a true set of benchmark tests to capture this data.

Page 6: Real World Performance Benchmarking

6 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

OTM/GTM Exadata/Exalogic Certification • Certification Scenarios

• Basic Regression Test • Multi-stop Planning • Multi-leg Planning • “End-End” Orders through Shipments • FTI Initial ETL using ODI

• Baseline OTM 6.2.5 • Exadata/Exalogic OTM 6.2.6

Page 7: Real World Performance Benchmarking

7 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

OTM Certification Deployment Exalogic and Exadata

15X 5X

OTM WebTier

Web Server

OTM App Tier

App Server

Page 8: Real World Performance Benchmarking

8 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Exalogic

Page 9: Real World Performance Benchmarking

9 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Certification Scenario #1 • Certification of OTM version 6.2 • Test Description

• Multi-leg Planning • 5,000 Orders • Building into 20,000 Shipments

• OTM Settings • 120 Planning Threads • 16 GB Heap

Page 10: Real World Performance Benchmarking

10 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Certification Scenario #1 Hardware Comparison Database EXADATA

Hardw are Exadata X2-2 Half Rack (4 nodes, 48 physical cores) 2x Intel Xeon, 3 GHz processors (12 cores), 96GB 1.3Ghz RAM per node

Softw are OEL 5.5 Oracle Database 11gR2 Enterprise Edition Release 11.2.0.2

Storage

4x 512GB SSD for read cache (“Readzilla”, 2TB total) 4x 18GB SSD in disk shelf for w rite cache 20x 2TB 7200 rpm SAS HDD in disk shelf – RAID-1

Application Server

EXALOGIC

Hardw are Exalogic X2-2 (1 node, 12 physical cores) 2x Intel Xeon, 3GHz processors (12 cores), 96GB 1.3Ghz RAM per node

Softw are OEL 5.5 Oracle Weblogic Server 10.3.4

Database BASELINE

Hardw are Dell 2950 2 x 3.1 GHz processors (4 cores), 32GB RAM

Softw are OEL 5.5 Oracle Database 11gR2 Enterprise Edition Release 11.2.0.2.

Storage

Direct Attached Storage 8 arrays 800GB each RAID 10=800GB

Application Server

BASELINE

Hardw are Dell 2950 2 x 3.1 GHz processors (4 cores), 32GB RAM

Softw are OEL 5.5 Oracle Weblogic Server 10.3.4

Page 11: Real World Performance Benchmarking

11 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

RuntimeRuntime

Certification Scenario #1 Exalogic and Exadata for OTM 6.2

• Improved Engine Performance: – 4.7x Faster Bulk Plan Runtime*

• Higher Through-put – 8x Improvement in Shipment Persistence* – 7.5 ms per shipment

* Some changes made could benefit baseline scenario

OTM Shipment Persistence 8x Faster Performance

8X

OTM Bulk Plan

4.7x Faster Performance

4.7X

47minutes

10 minutes

Baseline

Exalogic and Exadata

60ms

7.5ms

Presenter
Presentation Notes
Standard hardware refers to Customer Platform Specs in slide” Hardware Comparison”
Page 12: Real World Performance Benchmarking

12 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Certification Scenario #2 • Test Description

• 5,000 inbound transmissions, generating workflow resulting in: • 5,000 Order Releases • 10,000 Order Movements • 10,000 Shipments • 10,000 Invoices • 5,000 Tender Offers • 25,000 Outbound Transmissions

• OTM Settings • 36 threads each for various dataqueue and worker thread groups • 16 GB Heap

Page 13: Real World Performance Benchmarking

13 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Certification Scenario #2 Hardware Comparison Database EXADATA

Hardw are Exadata X2-2 Half Rack (4 nodes, 48 physical cores) 2x Intel Xeon, 3 GHz processors (12 cores), 96GB 1.3Ghz RAM per node

Softw are OEL 5.5 Oracle Database 11gR2 Enterprise Edition Release 11.2.0.2

Storage

4x 512GB SSD for read cache (“Readzilla”, 2TB total) 4x 18GB SSD in disk shelf for w rite cache 20x 2TB 7200 rpm SAS HDD in disk shelf – RAID-1

Application Server

EXALOGIC

Hardw are Exalogic X2-2 (1 node, 12 physical cores) 2x Intel Xeon, 3GHz processors (12 cores), 96GB 1.3Ghz RAM per node

Softw are OEL 5.5 Oracle Weblogic Server 10.3.4

Database BASELINE

Hardw are HP DL360 2 x 2.8 GHz processors (6 cores), 32GB RAM

Softw are OEL 5.6 Oracle Database 11gR2 Enterprise Edition Release 11.2.0.2.

Storage

Direct Attached Storage 8 arrays 800GB each RAID 10=800GB

Application Server

BASELINE

Hardw are HP DL360 2 x 2.8 GHz processors (6 cores), 32GB RAM

Softw are OEL 5.6 Oracle Weblogic Server 10.3.4

Page 14: Real World Performance Benchmarking

14 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Runtime

Certification Scenario #2 Exalogic and Exadata for OTM 6.2

OTM Overall Processing Time 9x Faster Performance

9x

2 hr 35 m

16 minutes

• Improved Performance: – 9x reduction in processing time*

* Some changes made could benefit baseline scenario

Baseline

Exalogic and Exadata

Page 15: Real World Performance Benchmarking

15 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Certification Details • Worked closely with

• Java team to fix contentions observed in JRE • Database team to fix an obscure bug in SDP protocol • OEL and Hardware team to measure/tweak OS and hard disk

performance • Of course, OTM team to

• Make ExaLogic specific changes • Certifying OTM with newer releases of the stack (weblogic, jdbc,

OEL, Database)

Page 16: Real World Performance Benchmarking

16 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Future • Explore Advanced Exadata Capabilities

• SDP over InfiniBand – High Performance Network Connection, in lieu of TCP/IP

• Smart Scan – Indexing Feature to Enhance Full Table Scan Performance

• Exadata Hybrid Columnar Compression – Advanced Feature for High Performance Data Compression

• Smart Flash Cache – Flash Memory Cache for Reduced i/o

Page 17: Real World Performance Benchmarking

17 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Agenda

• Internal Exa-data/Exa-logic Certification Testing • Customer Exa-data/Exa-logic POC • Other Performance Activities • Performance Testing Advice

Page 18: Real World Performance Benchmarking

18 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Analyzed Process

• Bulk Plan Order Release : To Buy Side Shipments – Average planning of 6,000 orders in 20 minutes, resulting in 250 shipments

Page 19: Real World Performance Benchmarking

19 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

© 20102Oracle Corporation

• Hardware

• Software

Produto Server GPA Server PoC

Weblogic Server 10.3.1 10.3.1

OTM 6.1 6.1

Database Server 11gR2 11gR2

JVM Software 1.6 JRockit 1.6 JRockit

SO Software Oracle Linux 5.5 Oracle Linux 5.5

Server Server GPA Server PoC

Application Tier

Intel Linux - 2 x 2 Intel quad core 64 GB (cluster) - 1 x 16 cores 64 GB (fail over)

Exalogic Half Rack - 4 Compute Nodes HA setup - Used just 1 Compute Node*

* 1 Compute Node = 2 Intel 6 cores 96 GB

Database Tier Exadata Quarter Rack - 2 Db Nodes HA

Exadata Quarter Rack - 2 Db Nodes HA

Page 20: Real World Performance Benchmarking

20 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Results

Date Bulk Plan ID Orders Shipments Total Time

22/11/2011 - 08h54m 20111122-0001 3,512 156 5 minutes

21/11/2011 - 19h17m 20111121-0007 5,114 215 9 minutes

22/11/2011 - 11h55m 20111122-0004 10,028 392 22 minutes

22/11/2011 – 20h10m 20111122-0006 15,342 565 32 minutes

• Bulk Plan Order Release : To Buy Side Shipments

Reference from PROD

Orders 6,000

Shipments 250

Total Time 20 minutes

Page 21: Real World Performance Benchmarking

21 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Agenda

• Internal Exa-data/Exa-logic Certification Testing • Customer Exa-data/Exa-logic POC • Other Performance Activities • Performance Testing Advice

Page 22: Real World Performance Benchmarking

22 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Other Performance Activities

• UI Benchmark on Sun OEL – Goal: Prove linear scalability on web tier for UI processing – Result: Achieved linear scalability, response degraded after 300 auto

users, linearity returned by adding a second web server – Other benefits: identified and corrected some single threaded web

functions

• Order XML Benchmark on Sun OEL – Goal: Process XML integration through order creation in 1 sec per – Result: 16 orders / second (24 orders / second scaled) – Other benefits: Provided new data queue feature in 6.2

Page 23: Real World Performance Benchmarking

23 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Other Performance Activities (Continued)

• Client Integration prepare for Go-Live – Goal: Process 200,000+ i_transmissions per hour – Result: 230,000+ achieved (limited by sending system) – Other Benefits: Improved Object Locking, lessons learned re database

contention and ways to alleviate

Page 24: Real World Performance Benchmarking

24 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Agenda

• Internal Exa-data/Exa-logic Certification Testing • Customer Exa-data/Exa-logic POC • Other Performance Activities • Performance Testing Advice

Page 25: Real World Performance Benchmarking

25 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Performance Testing Advice

• Types of Performance Testing – Load Testing – 1x, 2x, – Stress Testing – Extreme Load, Testing for Breakage, not

Performance – Endurance Testing – Sustained Volume w/ Consistent

Performance – Spike Testing – Peak Volume (# users, transactions, etc.)

Page 26: Real World Performance Benchmarking

26 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Performance Testing Advice

• Follow a Methodical Process – Ramp up Tests Starting from a Point of Success – Increase Volume to Failure Point – Tune to Address Failure Point – Repeat

Page 27: Real World Performance Benchmarking

27 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Performance Testing Advice

• What to Monitor – CPU, Memory Utilization on all tiers – OTM Diagnostic Servlets

• Event Diag • Cache Diag

– Database AWR/ADDM Reports

• Note: What to Collect When Performance Problems Start to Exist in OTM? (Doc ID 887885.1)

Page 28: Real World Performance Benchmarking

28 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Performance Testing Advice

• Tuning Options – Planning Parameters – Data Cache Sizes – OTM Event Threads – JVM Memory – Number of CPUs – Database: Queries, Indexes, Init.ora Params – Servers (Web, App, DB RAC)

Page 29: Real World Performance Benchmarking

29 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Performance Testing Advice

• Common Mistakes – Failure to monitor all systems, including upstream and

downstream servers – Jumping to solutions, rather than analyzing data – Not leaving enough time to address issues – Using unrealistic test data – Throttling test w/ too much wait time

Page 30: Real World Performance Benchmarking

30 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Performance Testing Advice

• Sustaining Performance – Create a Baseline for Comparison – Compute DB Statistics Regularly – Purge/Archive Unnecessary Data – Institute a Change Control Process

Page 31: Real World Performance Benchmarking

31 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.

Page 32: Real World Performance Benchmarking

32 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.