market risk analysis and oracle exadata

49
Exadata V2 / Sun Oracle Database Machine and POC results for Market Risk Reporting

Upload: eddiesharkey

Post on 16-Nov-2014

125 views

Category:

Documents


2 download

TRANSCRIPT

Exadata V2 / Sun Oracle Database Machine and POC results for Market Risk Reporting

Presentation is split into 2 parts Slides 3 26 highlight Market Risk POC results, just completed for another Tier 1 Investment Bank I wanted to provide a very relevant real life example of Oracle Exatadatas performance capabilities Slides 27 - 49 provide an overview of Oracles Exadata / Database Machine

Market Risk Reporting & AnalysisPOC just completed for another Tier 1 Investment bank

Value Proposition and why Oracle We make near real-time / on-demand risk and reporting a reality This demonstrably improves risk management and governance This increases trading revenues, reduces trade losses and reduces operational costs We are not market risk specialists We do not have a pre-built market risk platform We do have game changing technology and experience of building risk platforms

Why are firms doing this now?Most investment banks are unable to view risk on-demand or in near real-time Especially by trade, by instrument, by book, by counter-party, across asset classes or bank wide Why? Trading systems and risk systems are typically siloed Historically, risk has been calculated overnight Recent market events driving need for intra-day risk capabilities More advanced risk calculations e.g. gamma risk for credit derivatives has taken too long to be viable so quants ran simpler risk models not any more! Oracle Sun Exadata now offers Complex Risk Reporting and management of large data volumes in Near Real Time. Oracle Coherence is still the solution for Real Time Data Access needs

Tier 1 Market Risk POC - Test Configuration HP Oracle Database Machine (V1 Hardware) Exadata V2 Software Oracle 11g Release 2 8 Oracle Database servers 64 Intel processor cores Oracle Enterprise Linux

14 Exadata Storage Servers 75.6TB raw storage

InfiniBand switches

Test Scenarios Tests based on actual Market Risk data. Tests carried out on risk data volumes from 3 million (current) to 80 Billion (future) rows. Load tests based on loading from 3 million (current) to 2 Billion (Future) rows four times a day (8B total/day). Query tests based on actual SQL generated from frontend reporting systems. Additional query tests run using Materialised Views and OLAP cubes for comparison. The tests were carried out to provide a comparison between an Oracle Exadata solution and an existing Oracle Data Warehouse solution

Test Results Loading: Current customer solution load takes 6 minutes (3M rows). Tested load on Exadata takes only 4 minutes to load 2 Billion rows

1000 times greater load capacity (Could be improved with tuning). Data loaded in a compressed form (about 7-1 compression) saving large amounts of storage space and improving performance. Queries: All query tests initially carried out against base tables (no indexes, materialised views or other tuning). The current system has reached its performance limit Exadata scaled to data volumes over 25,000 times greater than the current system (and still had lots of room for greater expansion).

Test Results Query Performance Enhancements: Current system uses over 250 Materialised Views for performance reasons. It currently takes 25 minutes to create these views (against 3 million rows). It takes only 15 minutes to create all the views in Exadata (against 2 Billion rows). One OLAP Cube can be used to replace over 250 Materialised Views (and can provide many more dimensions of analysis!) It took only 10 minutes to create the single OLAP cube. Queries from the Cube have sub-second response times.

Test Results A generic Materialised view (can be used in lots of different queries) reduced the query time for one of the sample queries from 1 Min 8 Seconds to 17 seconds and took only 20 seconds to create. A targeted Materialised View (specific to a number of queries including the example above) reduced the query time to subsecond and took less than 2 seconds to create (from the generic materialised view). A single OLAP cube took only 20 minutes to create, replaced all 250+ materialised views and provided many more dimensions of analysis than the MVs it replaced. The same query run against the OLAP cube was also sub-second

Sample Test Results One of the sample queries currently takes 10 Mins 18 Secs to run against the 3M rows of data. The tested Exadata solution took only 2 mins 3 Secs to run against 80B rows of data!

One of the sample queries currently takes 1 Mins 45 Secs to run against the 3M rows of data. The tested Exadata solution took only 47.5 Secs to run against 80B rows of data

Future Testing The Sun Oracle Database Machine is even faster (more than twice as fast). Faster Processors - Intel Nehalem at both the Database and Storage tiers Faster I/O Infiniband Quad Data Rate cards (21 GBytes/Second) Flash Storage (5TB) for data caching Automatic Storage Indexes

Market Risk Platform High Level Target Architecture for Tier 1 bankOracle ComponentsTrading Apps Risk Engines Counterparty Data Reporting Analytics ModellingMDM FX BI

Rates

Intra Day Data

Historical Data

Analytics Modelling

Credit

Risk Analytics powered by Oracle Business IntelligenceData Integration Ad-hoc Analysis Interactive Dashboards Essbase Analytics Reporting & Publishing Proactive Detection and Alerts Disconnected & Mobile Analytics MS Office & Outlook Integration

Common Enterprise Information ModelIntegrated Security, User Management, Personalization Multidimensional Calculation and Integration Engine Intelligent Request Generation and Optimized Data Access Services

OLTP & ODS Systems

Data Warehouse Data Mart

Essbase

SAP, Oracle PeopleSoft, Siebel, Custom Apps

Files Excel XML

Business Process

Daily Summary Example: Alerts

Alerts to show where immediate action is required

Daily Summary Example: Risk Measures

Dashboard display of measures and trends for the day

Daily Summary Example: Shortcuts

Shortcut links to reports

Daily Summary Example: Spatial

Data availability

Daily Summary Example: Summary

Daily Summary Example: Drill to Detail

Risk Reporting Example: Key Risks

Risk Reporting Example: RCSA Dashboard

Risk Reporting Example: CRO Dashboard

Risk Reporting Example: LOB Dashboard

Risk Reporting Example: Market Risk

Comprehensive Heterogeneous Data Integration PlatformBest-in-class Platform for Major Data Integration Requirements

Exadata V2 Sun Oracle Database Machine

Sun Oracle Database Machine Grid is the architecture of the future Highest performance, lowest cost, fault tolerant, scalable on demand

Oracle Database Server Grid 8 compute servers 64 Intel Cores 576 GB DRAM

Exadata Storage Server Grid 14 storage servers 100 TB raw SAS disk storage or 336 TB raw SATA disk storage

InfiniBand Network 40 Gb/sec unified server and storage network Fault Tolerant

5TB flash storage!

Sun Oracle Database Machine Extreme Performance

Oracle Database Server Grid Millions of transactions per minute Tens of millions of queries per minute Billions of rows per minute

Exadata Storage Server Grid 21 GB/sec disk bandwidth 50 GB/sec flash bandwidth 1 million I/Os per second

InfiniBand Network 880 Gb/sec aggregate throughput

Semiconductor Cache HierarchyMassive throughput and IOs through innovative Cache Hierarchy Database DRAM Cache 400GB raw capacity Up to 4TB compressed user data 100 GB/sec

Exadata Smart Flash Cache 5TB raw capacity Up to 50TB compressed user data 50 GB/sec raw scan 1 million IO/sec

Exadata disks 100TB or 336TB raw Up to 500TB compressed user data 21 GB/sec scan 50K IO/sec

Sun Oracle Database Machine Hardware Improvements Same architecture as Exadata V1 Database Machine Same number and type of Servers, CPUs, Disks

Faster

Latest Technologies80% Faster CPUs 100% Faster Networking 50% Faster Disk Throughput 200% Faster Memory Xeon 5500 Nehalem 40 Gb InfiniBand 6 Gb SAS Links DDR3 DRAM 600 GB SAS Disks 2 TB SATA Disks

New

Bigger

33% More SAS Disk Capacity 100% More SATA Disk Capacity 125% More Memory 100% More Ethernet Connectivity

72 GB per DB Node 4 Ethernet links per DB Node

Plus Flash Storage!

Scale Performance and Capacity

Scalable Scales to 8 rack database machine by just adding wires

Redundant and Fault Tolerant Failure of any component is tolerated Data is mirrored across storage servers

More with external InfiniBand switches Scales to hundreds of storage servers

Multi-petabyte databases

Drastically Simplified Deployments Database Machine eliminates the complexity of deploying database systems Months of configuration, troubleshooting, tuning

Database Machine is ready on day one Pre-built, tested, standard, supportable configuration

Runs existing applications unchanged

Extreme performance out of the boxMonths to Days

Best Machine for Data Warehousing

Best Data Warehouse Machine Massively parallel high volume hardware to quickly process vast amounts of data Exadata runs data intensive processing directly in storage

OLAP

Most complete analytic capabilities OLAP, Statistics, Spatial, Data Mining, Real-time transactional ETL, Efficient point queries

ETL

Powerful warehouse specific optimizations Flexible Partitioning, Bitmap Indexing, Join indexing, Materialized Views, Result Cache

Data MiningNew

Dramatic new warehousing capabilities

Exadata Database Processing in Storage Exadata storage servers implement data intensive processing in storage Row filtering based on where predicate Column filtering Join filtering Incremental backup filtering Storage Indexing Scans on encrypted data Data Mining model scoring

New

10x reduction in data sent to DB servers is common No application changes needed Processing is automatic and transparent Even if cell or disk fails during a query

Simple Query ExampleWhat were my sales yesterday? Optimizer Chooses Partitions and Indexes to Access

Exadata Storage Grid

Oracle Database GridSelect sum(sales) where Date=24-Sept

Scan compressed blocks in partitions/indexes Retrieve sales amounts for Sept 24

SUM

10 TB scanned 1 GB returned to servers

Exadata Hybrid Columnar Compression Data is grouped by column and then compressed Query Mode for data warehousing Optimized for speed

New

10X compression typical Scans improve proportionally

Archival Mode for infrequently accessed data Optimized to reduce space 15X compression is typical Up to 50X for some data

Flash60

Query Throughput

Query Throughput with Flash

Flash storage more than doubles scan throughput 50 GB/sec

50 40 30

GB/sec Uncompressed Data

Query Throughput

50

Flash21

Combined with Hybrid Columnar Compression Up to 50 TB of data fits in flash Queries on compressed data run up to 500 GB/sec

20

11.410 0

7.5HITACHIUSP V

10

Disk

TERADATA2550

NETEZZA SUN ORACLE

TwinFin 12 Database Machine

In-Memory Parallel QueriesQphH: 1 TB TPC-H1,166,976 1,018,321

New

One Sun Oracle Database Machine rack 400GB of DRAM usable for caching

Exadata Hybrid Columnar Compression enables 4TB data in DRAM Database release 11.2 introduces parallel query processing on DRAM cached data Harnesses DRAM capacity of entire database cluster for queries Technology for world record benchmarkExasol Oracle Faster than specialized inmemory warehouse databases ParAccel

315,842

DRAM has 100x more bandwidth than Disk

Source: Transaction Processing Council, as of 9/14/2009: Oracle on HP Bladesystem c-Class 128P RAC, 1,166,976 QphH@1000GB, $5.42/QphH@1000GB, available 12/1/09. Exasol on PRIMERGY RX300 S4, 1,018,321 QphH@1000GB, $1.18/QphH@1000GB, available 08/01/08. ParAccel on SunFire X4100 315,842 QphH@1000GB, $4.57 /QphH@1000GB, available 10/29/07.

Exadata Storage IndexTransparent I/O Elimination with No Overhead TableA B C D 1 3 5 5 8 3 Min B = 3 Max B =8

New

Index

Exadata Storage Indexes maintain summary information about table data in memory Store MIN and MAX values of columns Typically one index entry for every MB of disk

Min B = 1 Max B =5 Eliminates disk I/Os if MIN and MAX

can never match where clause of a query Completely automatic and transparent

Select * from Table where B