the following is intended to outline our general …...2 the following is intended to outline our...

2

The following is intended to outline our general

product direction. It is intended for information

purposes only, and may not be incorporated into any

contract. It is not a commitment to deliver any

material, code, or functionality, and should not be

relied upon in making purchasing decisions.

The development, release, and timing of any

features or functionality described for Oracle’s

products remains at the sole discretion of Oracle.

<Insert Picture Here>

S317423: Deploying PeopleSoft Enterprise Applications on Exadata

Tips, Techniques and Best Practices

Oracle Development: Database and Applications MAA / Exadata

Richard Exley, Darryl Presley – Oracle MAA/Exadata

4

PepleSoft on Exadata

Introduction: MAA and Exadata

PeopleSoft on Exada

PeopleSoft MAA

HR Payroll Test Case

Test Case Findings

5

Introduction

6

IntroductionMAA/Exadata Team

Committed to delivering a complete hardware and

software solution for deploying Oracle applications on

Exadata

High Performance

Scalability

High Availability

Disaster Recovery

Current focus:

E-Business Suite

PeopleSoft

Siebel

7

Exadata Hardware Architecture

Database Grid

• 8 compute servers (1U)

• 64 Intel cores

Storage Grid

• 14 storage servers (2U)

• 112 Intel cores in storage

• 100 TB SAS disk, or336 TB SATA disk

• 5 TB PCI Flash

• Data mirrored across storage servers

Scaleable Grid of industry standard servers for Compute and

Storage

• Eliminates long-standing tradeoff between Scalability, Availability,

Cost

InfiniBand Network

• Redundant 40Gb/s switches

• Unified server & storage net

© 2010 Oracle Corporation

8

Keys to Speed and Cost Advantage


Exadata Hybrid Columnar

Compression

Exadata Intelligent Storage

Grid

Exadata Smart Flash Cache

9

Exadata is Smart Storage

Storage Server is smart storage, not a DB node

Storage remains an independent tier

Database Servers

Perform complex database processing such as

joins, aggregation, etc.

Exadata Storage Servers

Search tables and indexes filtering out data that is

not relevant to a query

Cells serve data to multiple databases enabling

OLTP and consolidation

Simplicity, and robustness of storage appliance


Compute and Memory Intensive Processing

Data IntensiveProcessing

10© 2010 Oracle Corporation

Exadata Hybrid Columnar CompressionHighest Capacity, Lowest Cost

Data is organized and compressed by column

Dramatically better compression

Speed Optimized Query Mode for Data Warehousing

10X compression typical

Runs faster because of Exadata offload!

Space Optimized Archival Mode for infrequently accessed data

15X to 50X compression typical

Qu

ery

Faster and Simpler

Backup, DR, Caching, Reorg, Clone

Benefits Multiply

11


Exadata Smart Flash CacheExtreme Performance OLTP

5X More I/Os than

1000 Disk Enterprise

Storage Array

• Exadata has 5 TB of flash

• 56 Flash PCI cards avoid disk

controller bottlenecks

• Intelligently manages flash

• Smart Flash Cache holds hot data

• Gives speed of flash, cost of disk

• Exadata flash cache achieves:

• Over 1 million IO/sec from SQL (8K)

• Sub-millisecond response times

• 50 GB/sec query throughput

12

Failure Protection

Redundant Hardware

Servers, Storage, Network

Database Level HA

Tolerate failures and changes

Documented and validated

Exadata MAA best practices

to achieve the highest

availability


Data GuardRAC

ASM

Flashback

Secure

Backup

Online

Redefinition

ERP

CRM

Warehouse

Data Mart

HR

13

PeopleSoft on

Exadata MAA

14

PeopleSoft on ExadataMaximum Availability Architecture (MAA)

MAA best practices for PeopleSoft

Planned and unplanned outages

Integrated

Validated

Documented

15


16


Full support for RAC

Failover through TAF and FAN

Application Server enhanced to support Fast Application

Notification (FAN) to expedite application failover as of

PeopleTools 8.50.09

Role base services

Single Client Access Name (SCAN) support for PeopleSoft

application servers utilizing Oracle 11g Release 2 Client

Fast transaction recovery with Flashback technology

17


Full support for Oracle Data Guard

Physical or logical

Recovery Point Objective (RPO) < 5 minutes, 0 if required

Data Guard Broker Fast Start Failover - validated

Full application stack startup at DR site (example in white

paper)

PeopleTools 8.51 support for Active Data Guard

Utilized to offload queries on standby database

Auto-Block Recovery transparent to PeopleSoft

Active Data Guard configured as “secondary” database in

PeopleSoft application server configuration

18


Planned outages:

Only minutes of downtime for RAC rolling database upgrades

Online database patching

Near zero downtime database upgrade

Supports Transient Logical Standby

19

Storage

Data GuardFully Active

Failover Replica

Recovery Manager &

Oracle Secure BackupLow Cost High Performance

Data Protection & Archival

Real Application Clusters

& ClusterwareFault Tolerant

Server Scale-Out

FlashbackCorrect Errors by

Moving Back in Time

Online RedefinitionRedefine Tables Online

DatabaseDatabase

Storage

Oracle Maximum Availability Architecture

Automatic Storage ManagementFault Tolerant

Storage Scale-Out

Integrated suite of best-of-breed HA technologies

- Each is scale-out, fully active, data centricBest Availability AND

Lowest Cost

Online UpgradeUpgrade Hardware

and Software Online

20


PeopleSoft Specific Failover Conditions for

application server clients

PeopleSoft Client Operation Behavior

Web client user is updating data and

submits or saves the updates during or

just after the database failure.

PeopleSoft resubmits the update

Web client user is paging through

queried data when the database failure

occurs.

Pages are rendered from pre-fetched

result-set.

Web client user is issuing a new query

or switching screens just after the

database failure.

Oracle reconnects and session resume

the query

21


PeopleSoft Specific Failover Conditions for batch

server processes

PeopleSoft Batch Operation Behavior

Process Scheduler The process scheduler reconnects to

the surviving node successfully

Application Engine (AE) job submitted

just BEFORE primary failure

AE jobs may fail and show as “No

Success” and require restart

Application Engine (AE) submitted

during or just AFTER primary instance

failure

AE job completes successfully

COBOL jobs just BEFORE primary

instance failure

Programs with only SELECTs failover

successfully otherwise, restart required.

Crystal and SQR reports Same as COBOL programs

PSQUERY, Tree Viewer, XMLP Viewer These components will all failover

successfully

22


All MAA best practices are supported for PeopleSoft

on Exadata

PeopleSoft MAA white paper at:

http://www.oracle.com/technetwork/database/features/availability/maa-

peoplesoft-bestpractices-134154.pdf

23

PeopleSoft

HR Payroll Test Case

24

PeopleSoft on ExadataHR Payroll Test Case

PeopleTools 8.49 (32bit client)

PeopleSoft HCM 9.0

PeopleSoft Payroll batch processing:

PaySheet

PayCalc

PayConfirm

500,000 Employees paid

128 Pay Groups

1 Company

Multiple run controls/run IDs

Single Check = “NO” (Consolidated pay checks)

PayCalc can run with multiple concurrent processes

25

PeopleSoft on ExadataHR Payroll Test Case – Application Profile

Single row processing - SQL SELECTs / Fetch,

INSERTs

“sweep” style - very few tables are reread

Business logic processing (COBOL) is CPU

intensive

High IO Rates

15K IOPs/second (Peek)

200MB reads/second (Peek)

Index bound

Very high network chattiness

12MB/second

37K messages round trips/second

26

PeopleSoft on ExadataHR Payroll Test Case - Lab Setup

Database Server

Quarter Rack Exadata V2 Database Machine

2 Compute nodes

3 Storage Cells

Oracle Database 11g Release 2

Oracle Enteprise Linux 5 Update 3

Hugepages must be configured for SGA

Process Scheduler

Co-located with the database, or

Separate machine

Oracle Enterprise Linux

27

PeopleSoft on ExadataTest Case Findings – note!

Results based on our project work and analysis - your

mileage may vary

System configuration

Application configuration

Business requirements

Results provided in relative terms so that effects can

be assessed for different machines

This was an MAA exercise, not a benchmark

28

PeopleSoft on ExadataHR Payroll Test Case – RAC Tuning

Motivation: Scale the workload

Exploit both compute nodes

Minimize RAC cluster contention

Use automatic RAC load balancing

Tuning Method: Table/Index Range Partitioning

PAYGROUP range for partition keys

9 Payroll tables were partitioned

Most indexes were locally partitioned

29


Partition Configuration

128 Partitions

1 PAYGROUP per partition

Reconfigured run controls to redistribute workload

No run control had overlapping paygroups

32 and 64 concurrent payroll processes (streams)

Workload redistribution was done for 64 concurrent

streams

Workload load balanced across both RAC instances

30


Results weighted average:

RAC cluster wait events are low, so higher throughput

and scaling can be achieved

Metric 128 Partitions

Cluster (%) 8.69

DB CPU (%) 57.0

IO (%) 35.0

31

PeopleSoft on Exadata HR Payroll Test Case – RAC Tuning

Results:

Overall elapse time was reduced by 15% when using

128 partitions with 32 concurrent payroll processes

More productive work being done

Wait Event (%) PaySheet PayCalc PayConfirm

DB CPU % 50.97 44.44 68.54

RAC Cluster % 13.15 8.54 8.30

IO % 36.88 47.48 24.02

32

PeopleSoft on Exadata HR Payroll Test Case – RAC Tuning

Doubling the workload processes from 32 to 64

reduced overall elapsed time by 20%

Workload was not evenly distributed

Some PAYGROUPs have fewer paid employees than others

Some process streams had more work than others

33

PeopleSoft on Exadata HR Payroll Test Case – RAC Tuning Conclusions

PeopleSoft Payroll can utilize RAC successfully

On Exadata, RAC gives you more CPU and IO

bandwidth

Each RAC node is connected to the storage cells using

InfiniBand yielding higher IO bandwidth and Cluster

interconnect

IOPs requests are distributed between the two RAC nodes

Payroll workload does not require affinity to a static

RAC instance, so scaling is easily achieved

34

PeopleSoft on ExadataHR Payroll Test Case – Flash Cache

Exadata Smart Flash Cache

Reduce Latency

0.7 ms v 7 to 8 ms for conventional IO

High access rates

> 1,000,000 IOPs in Full Rack database machine

No additional requirements to use Flash Cache for

PeopleSoft

Kicks in automatically

CELL_FLASH_CACHE KEEP segment storage attributes

for finer grain control

35

PeopleSoft on ExadataHR Payroll Test Case – Flash Cache Results

Results

With Flash Cache, overall performance improved by 15%

Average IO latency was reduced from 7ms to 3ms with 80%

hit ratio

Worked automatically – no manual intervention required

36

PeopleSoft on ExadataHR Payroll Test Case – Process Scheduler Placement

PeopleSoft Batch Processes have high number of

network round trips (due to single row processing)

Placement options:

Collocate with database

Separate application tier

On Exadata Database Machine

Can use either TNS or BEQ connection

Must consider CPU usage by the COBOL processes and

database RAC instance

37


Not recommended to collocate process scheduler on

Exadata compute node where CPU usage may be high

Node 1 16 cores, RAC instance only (Green line)

Node 2 16 cores, RAC instance + process scheduler running 32

payroll processes (Light blue line)

38


Process Scheduler placed on separate middle Tier

Initial tests were conducted on GigE 2-hops away from

Database Machine

Latency: 0.264ms

Results (PayConfirm)

43% client side

57% database

39


Process Scheduler placed on separate middle tier

Move to a mid tier one-hop away on GigE

Latency: 0.121ms

Results (PayConfirm)

27% client side

73% database

Of total elapse time

SQL Net message waits

reduction

76K sec to 36K sec

Aggregated across all

payroll processes

40


Waits in Seconds and Elapse Time

41


Consider placing the process scheduler on separate

middle tiers with minimal network latency

Configure multiple process schedulers on multiple

application servers

Configure queues to support 2 or more process schedulers

Allows for scale-out for driving the Database Machine

For HA, configure Master Scheduler

InfiniBand can be considered for lower latency and

wide bandwidth to share with other applications

4GB/sec and 0.064ms round trip latency

42


Other factors to considered for process scheduler

placement

Number and speed of CPU/cores

32 concurrent COBOL process consume 25 – 30% CPU of an

8 core server

43

Summary

44

Summary

Exadata is a fully integrated hardware and software

stack

Peoplesoft benefits from the performance and HA

capabilities of Exadata

PeopleSoft products with integrated MAA features

now leverage MAA solutions for data protection, high

availability and scalability

the following is intended to outline our general …...2 the following is intended to outline our...

Documents