Slide 1

Oracle Exadata v2 Fast Track Hsn ensoy Global Maksimum Data & Information Tech Founder, VLDB Expert Global Maksimum Data&Information Technologies 1 Not x2

Slide 2

Agenda Why do we need Exadata v2 ? Exadata Hardware Exadata Software Better to show rather than talk. Conclusion Global Maksimum Data&Information Technologies 2

Slide 3

3 Who am I ? Data & Information expert on VLDB environments DWH Data Mining Inference Systems Data Archiving Solutions Niche Storage Technologies Recovery Strategies & Solutions HA Systems Oracle ACED on BI field Only one in Turkey Still the youngest one all over the community. DBA of the Year 2009 7th and still the youngest all over the world. Only one in Turkey Member of Oracle CAB for 12g DWH development Worldwide presenter of Oracle conferences and user group events Before completing the year HrOUG in two weeks later Optimized Analytical Processing Capabilities of 11g Release 2 Database Consolidation Best Practices ACED Session with Jose Seneganik Denes Kubiek UKOUG in December Optimized Analytical Processing Capabilities of 11g Release 2

Slide 4

Global Maksimum Data&Information Technologies 4 Global Maksimum Data & Information Technologies and Oracle Exadata v2 Only company in Turkey having IB interconnected RAC 11g implementation experience on Linux x86-64bit. Only company in Turkey having sufficient consultancy experience (more than 120 TB conventional system data) on Exadata v2 Physical & Architecture Design Migration Performance Optimization Backup & Recovery Architectures Design Trains customers, Oracle partners, and Oracle employees all over the Europe Strong joint relation with Oracle Platinum Partners, Oracle Development Team Head Office, and IB technology leaders. X X-Migrator service provider for high capacity customers.

Slide 5

Oracle Exadata v2 Dont think Exadata as yet another product sold by SALES guys. As a customer take it as an effortless solution for hardware software integration. As an engineer take it as an elegant solution of so-called unsolvable I/O problem for Oracle databases. Global Maksimum Data&Information Technologies 5

Slide 6

Who needs Exadata v2 ? Engineers To learn that The mechanic with a hammer thinks that all problems are nail Customers Shorter setup time Non-Exadata Customers More stable Oracle releases Oracle Easy to manage/standardize its code repository Global Maksimum Data&Information Technologies 6

Slide 7

Oracle Exadata v2 Hardware Best thing about Exadata is that it has nothing magical in it in terms of hardware. A few Sun Fire X4170 x86-64 bit servers. A few Sun Fire X4275 x86-64 bit servers. A few IB switches. Global Maksimum Data&Information Technologies 7

Slide 8

Exadata v2 X-Ray Global Maksimum Data&Information Technologies 8 42U Sun Datacenter 36-port Managed QDR IB Switched Exadata Storage Servers Sun Fire X4170 Oracle Database Server KVM IP Console Switch Rackmount KMM Keyboard with TFT monitor 48-port Gigabit Ethernet Switch

Slide 9

Interconnect Network Hardware IB Switches 3 x 36-port managed switches as opposed to Exadata v1 (2+1). 2 leaf 1 spine switches Spine switch is only available for Full Rack because it is for connecting multiple full racks side by side. A subnet manager running on one switch discovers the topology of the network. HCA Each node (RAC & Storage Cell) has a PCIe x8 40 Gbit HCA with two ports Active-Standby Intracard Bonding. Global Maksimum Data&Information Technologies 9

Slide 10

RAC Node Global Maksimum Data&Information Technologies 10 Sun Fire X4170 Server 2 socket Quad Core 2.53 GHz 2 Hyper-Threads So, CPU_COUNT=16 18 DDR3 DIMM Slots 72 GB@800 MHz (2x3x3x4 GB) 4 10/100/1000Base-T Ethernet ports NET0 : Management NET1 : Public Network NET2 : Public Network NET3 : - PCIe PES24T6G2 Switch x8

Slide 11

Storage Node Sun Fire X4275 Server 2 socket Quad Core 2.53 GHz 6 DDR3 DIMM Slots 24 GB@1066 MHz (2x3x1x4 GB) HDD Storage 12 x 3.5-inch 600 GB 15 K RPM SAS disks 12 x 3-5-inch 2 TB 7.2 K RPM SATA disks 4 Sun Flash Accelerator F20 PCIe Cards Global Maksimum Data&Information Technologies 11

Slide 12

Global Maksimum Data&Information Technologies 12 Soft Storage Node CELLSRV Multithreaded block server Buffer cache reads Smart scans Performs I/O Resource Management Gather operational statistics Communicates over iDB with the clients. MS OC4J application Provides functionalities for Cell management Cell administration Aler generation RS First process becoming live in storage cell. Work as a hang analyzer for CELLSRV and MS MS RS CELLSRV IORM i DB

Slide 13

600 GB 15K RPM SAS2 TB 7.2K RPM SATA Global Maksimum Data&Information Technologies 13 HDD Sequential Read Performance 204 MB/s 122 MB/s 144 MB/s 90 MB/s

Slide 14

600 GB 15K RPM SAS2 TB 7.2K RPM SATA Global Maksimum Data&Information Technologies 14 HDD Random Read Performance 79 IOPS @ 2KB 182 IOPS @ 2KB 380 IOPS @ 2KB 175 IOPS @ 2KB

Slide 15

Global Maksimum Data&Information Technologies 15 F20 PCIe Card Not a SATA/SAS SSD driver but a x8 PCIe device providing SATA/SAS interface. 4 Solid State Flash Disk Modules (FMod) each of 24 GB size 256 MB Cache ESM SuperCap Power Reserve (EnergyStorageModule) provides write-back operation mode. ESM should be enabled for optimal write performance Should be replaced in every two years. Can be monitored using various tools like ILOM Embedded SAS/SATA configuration will expose 16 ( 4 cards x 4 FMod ) Linux devices. /dev/sdn 4K sector boundary for Fmods Each FMod consists of several NAND modules best performance can be reached with multithreading (32+ thread/FMod etc)

Slide 16

Performance of F20 Global Maksimum Data&Information Technologies 16 F20 PCIe Card (4 FMod) SequentialRead: 1.1 GB/s Max Write: 567 MB/s (~145K IOPS @ 4K) Random @ 4KRead: 101K IOPSWritePeak: 88K IOPSAverage : 37K IOPS Random Write Performance Degeneration As the flash cache get full (sustained write) Wear Leveling SLC Update Mechanism : Delete + Write Garbage Collector write performance is degenerated due to Write Amplification. Thats why you are not advised to put real-time performance demanding files on flash cards Online Redo Logs

Slide 17

CapacityPerformance Global Maksimum Data&Information Technologies 17 Aggregate Capacity Quarter Rack Half Rack Full Rack Raw HDD SAS21 TB50 TB100 TB SATA72 TB168 TB336 TB Raw Flash1.1 TB2.6 TB5.3 TB User Data SAS6 TB14 TB28 TB SATA21 TB50 TB100 TB Quarter Rack Half Rack Full Rack HDD Throughput SAS4.5 GB/s10.5 GB/s21GB/s SATA2.5 GB/s6 GB/s12 GB/s Flash Throughput11 GB/s25 GB/s50 GB/s Flash IOPS225,000500,0001,000,000

Slide 18

Oracle Exadata v2 Software Exadata hardware is almost sufficient to beat any hardware configuration possible to work with Oracle Database. But why to stop there while it is possible to do more with Smart Scan Storage Indexes I/O Resource Manager EHCC Global Maksimum Data&Information Technologies 18

Slide 19

Soft Components of Exadata v2 Open Soft Pieces Oracle Enterprise Linux 5.3 Oracle defined set of RPMs Oracle OFED (bug fixed version) Oracle Exadata Storage Software Smart Scan Encrypted Data Data Mining Smart Flash Cache HCC Storage Index IO Resource Manager (IORM) Oracle Exadata Bundle Patches Common Soft Pieces Oracle RDBMS 11.2.0.1 1.Partitioning Pruning 2.Parallel Hash Join Bloom Filtering Pairwise/Semi-pairwise Join 3.Compression HCC DBFS Oracle Grid IS 11.2.0.1 ASM Clusterware Oracle Exadata Bundle Patches iDB Global Maksimum Data&Information Technologies 19

Slide 20

Smart Scan Smart Scan is initially formed to be column and row filtering based on projection and predicates. But this was just the seed idea. Today Smart Scan can also do Projection (column) filtering Predicate (row) filtering SELECT * FROM v$sqlfn_metadata WHERE offloadable = 'YES'; Preperation of bloom filters for join Smart Incremental backup Scan on encrypted data Smart File Creation RMAN Restore Tablespace Creation File Grow Scoring for Data Mining All data mining scoring functions are offloaded Global Maksimum Data&Information Technologies 20

Slide 21

Smart Scan OFF. Why ? CELL_OFFLOAD_PROCESSING = FALSE The table or partition is small. CBO doesnt choose to use direct path read. ROW_DEPENDENCY ENABLED or rowscn is fetched. Fetch rows in rowid order. CREATE INDEX... NOSORT LOB or LONG fetch Scan on flashback table Cell based decryption is disable. Tablespace is not completely on Exadata More than 255 columns are queried. Predicate evaluation on virtual column. For dirty blocks Global Maksimum Data&Information Technologies 21

Slide 22

Storage Index Smart Scan is about saving RAC node CPUs during I/O processing, but storage index is about saving the processors of Exadata storage cells. Anyhow if we figure out that T = E+W, decreasing E in any layer will decrease T. This means faster queries or more queries within the same period. Storage Index is not something first used in Exadata. It is borrowed from Netezza ZoneMap. Oracles SI is in memory It is about filtering out for a super set of actual result set. Global Maksimum Data&Information Technologies 22

Slide 23

First Execution Global Maksimum Data&Information Technologies 23 select A,B,C from T1 where B