“extending in-memory database processing to … · imc (in-memory columnar) data database server...

“ExtendingIN-MemoryDatabaseProcessingtoSharedFlash

GurmeetGoindiMasterProductManager

SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirecQon.ItisintendedforinformaQonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncQonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andQmingofanyfeaturesorfuncQonalitydescribedforOracle’sproductsremainsatthesolediscreQonofOracle.

ExadataDatabaseMachine

Performance,AvailabilityandSecurity

BestPlaVormforOracleDatabaseson-premisesandintheCloud

Enabledby:•  Single-vendoraccountability•  Exclusivefocusondatabases•  Deeph/wands/wintegraQon•  RevoluQonaryapproachtostorage

ExadataAchievesMemoryPerformancewithSharedFlash

•  ExadataX6delivers300GB/secflashbandwidthtoanyserver–  Approaches800GB/secaggregateDRAMbandwidthofDBservers

•  MustmovecomputetodatatoachievefullflashpotenQal–  Requiresowningfullstack,can’tbesolvedinstoragealone

•  Fundamentally,storagearrayscanshareflashcapacitybutnotflashperformance–  Evenwithnextgenscale-out,PCIenetworks,orNVMeoverfabric–  e.g.newEMCDSSDhas3-6QmeslowerthroughputthanExadataX6

•  Sharedstoragewithmemory-levelbandwidthisaparadigmchangeintheindustry–  GetnearDRAMthroughput,withthecapacityofsharedflash

ExadataDBServers

ExadataSmartStorage

InfiniBand

CPUPCIeNVMeFlash

QueryOffload

NVMePCI-eFlashDisruptstheStorageArrayModel

|OracleConfidenQal–HighlyRestricted 5

LatestPCIeFlash5.4GB/sec

SANLink=40Gb5GB/sec

Lessthan1Flashcard

LeadingAllFlashArray24GB/sec

Lessthan5Flashcard

Newimprovementsarecausing100XboTlenecksacrosssharedstoragestack

ArrayHeads

All-FlashStorageArrayIOPath:manysteps,eachaddslatencyandcreatesboTlenecks

SAS/SATA PCIeFlashChips

Switches

SAN/LAN

SSDCtrl

HostHBA

SAN/LAN

RedesigningScanOffloadforMemoryThroughput

• WithExadataFlashthroughputapproachingmemorythroughput,SQLbohleneckmovesfromI/OtoCPU

•  ExadatawillautomaQcallytransformtabledataintoIn-MemoryDBcolumnarformatsinExadataflashcache–  DualformatarchitectureextendedfromDRAMtoflash

•  Enablesfastvectorprocessingforstorageserverqueries–  SmartScanresultssenttoDBusingIn-MemoryColumnarformattoreduceDBCPUusage

•  UniquelyopQmizesnextgeneraQonflashasmemory

In-MemoryColumnarscans

In-FlashColumnarscans

12.8TBFlashStorageServer

In-MemoryColumnarFormatsinDRAM(pre12.2.1.1.0)

Super-FastScansfromMemory,butAllQueriesComplete

Upto1.5TBDRAM

DatabaseServer

DatanotinDRAM

12.8TBFlash

In-MemoryColumnarFormatsinFlashCache(12.2.1.1.0)

3-4xOverallAnaly`csPerformanceImprovement

Upto1.5TBDRAM

12.8TBFlashx3=38.4TB(ormore)IMC(In-MemoryColumnar)data

DatabaseServer

In-FlashColumnarscans

HybridColumnarCompressedData

StorageServer

SmartAnaly`cs:JoinandAggregaQonSmartScan

•  ExtendIn-MemoryAggregaQontechniqueintostorage(vectorjoinsandvectoraggregaQon)

•  FindSalespercountrySELECT /*+ VECTOR_TRANSFORM */ country_id, sum(amount_sold) amount_sold FROM customers, sales WHERE customers.cust_id = sales.cust_id GROUP BY customers.country_id ORDER BY customers.country_id;

•  Storagecellsscanningsalesfacttablereturntuples{country_id, sum_amount_sold }

•  JoinandAggregaQonoffloadedtothestorageserver

NEWINDB12.2

SmartAnaly`cs:MoreSmartScanEnhancements

•  SmartScanenhancementsforXMLandJSON– JSON_EXISTS,JSON_VALUE,JSON_QUERY,"ISJSON"and"ISNOTJSON"– XML:XMLExists,XMLCast(XMLQuery())

•  SignificantspeedupinJSONanalyQcworkloadsselect count(*) from pictures where json_value(photo, ‘$.tag’) like ‘%spain%’;

NEWINDB12.2

ColdorRawData

RealTimeDataAnalysis

Hot(FrequentlyAccessed)Data

CoolingData

DataTiering

DBIM&FlashCache

HCConExadataStorage

ZFSorBigDataSQL

SmartAnaly`cs:SmartWriteBurstsandTempIOinFlashCache

• Writethroughputoffourflashcardshasbecomegreaterthanthewritethroughputof12-disks

• Whendatabasewritethroughputexceedsthroughputofdisks,SmartFlashCacheintelligentlycacheswrites

• WhenquerieswritealotoftempIO,SmartFlashCacheintelligentlycachestempIO– WritestoflashfortempspillreduceselapsedQme–  ReadsfromflashfortempreduceselapsedQmefurther

•  SmartFlashCacheprioriQzesOLTPdataanddoesnotremovehotOLTPlinesfromthecache

•  Smartflashwearmanagementforlargewrites

•  SupportsDatabase11.2.0.4,12.1.0.2and12.2.0.1

WriteBurstsandTempIOinFlashCache

AcceleratesLargeJoinsandSortsandLargeDataLoads

ExadataCloud–YourWay

OraclePublicCloud

ExadataCloudService

ExadataCloudMachine

CloudatCustomer

SameProduct

SamePrice

OracleManaged

OracleOwned

ExadataCustomerCaseStudies

NTTdocomo:MoBills（MobileBillingSystem）

Benefits

BusinessObjec`ves

•  Real-TimeBillingPlaVormfor66millioncustomer

•  DramaQcallyimproveperformanceandavailability

•  Reducecostandcomplexity

Solu`on

•  OracleExadata:30racks•  OracleMAA(RAC/AcQve

DataGuard-Local&RemoteStandbydatabase)

FasterBillingProcessing

10Xspeedup

3millionSQL/sec

DataCenterCostSavings

90%SpaceReduc`on

ReducedIntroduc`onCost

25%"MoBills is a very important position as a mission-critical system to promote efforts toward the realization of “+d”. Oracle Exadata is running very stable as a expected performance. We will continue to use the "Oracle Exadata" and we would like to establish a further advantage for our business." - Shimamura, Manager, Information System Department, NTT docomo

ReducedOpera`onalCost

MaximumAvailability

Local&RemoteStandby

Pre-Exadata ExadataMAA30racks/Local&RemoteStandby/RMANbackup

Tokyo Osaka

DataGuard

LocalStandby5racks

RemoteStandby5racks

LocalStandby5racks

Ra`ng(primary)5nodeRAC*5racks

Billing(primary)5nodeRAC*5racks

•  Real-TimeBillingProcessing•  High-endSMPServer

+High-endStorage:350racks•  StorageMirrorBackup

•  StorageMirrorReplica`on•  Oracle9iDatabaseRelease2

Upgrade&Migra`on

DCMHoldings:SystemConsolida`onof3companies

Benefits

BusinessObjec`ves

•  $10billionSales,FasterM&A•  HighConsolidaQonraQoand

improveservicelevel•  ReduceoperaQonalcost

Solu`on

•  OracleDatabase12cMulQtenantonExadata

•  OracleFullStack(MiddlewareandServerproducts)

Realized the Database consolidation and integration due to the high performance provided by Oracle Exadata. And, Oracle Database 12c Multitenant Architecture also achieved high consolidation ratio while maintaining the independence of each group companies. Platinum Service could improve the service level, Oracle Full-stack products could provide One-Stop Support.

Pre-Exadata

OracleMul`tenantonExadata •  Consolida`onandIntegra`on

3groupcompanies(Homac,KahmaandDAIKI)ofsysteminfrastructure

•  ReplacedfromIBMpServers•  TeradataMigra`ontoExadata•  OracleDatabase12cMul`tenant•  Pla`numService•  Zerodown`meSystemMigra`onbyusingGoldenGate

Integra`on/Consolida`on

FasterBatchProcessingReducedIntroduc`onCost

2Xspeedup 40%Off

Standardiza`on

HighConsolida`onRa`oImprovedManageability

6DBsConsolida`on

Mul`tenantArchitecture

SimplifiedSupport

NonStopSupport

OracleFullStack

Sprint:CallDataRecord-DataWarehouseBenefits

BusinessObjec`ves

• Improveperformance

• Improvesustainability

• Improveavailabilityandmaintainability

Solu`on

• FullRack(Prod),HalfRack(Dev/Test);ZFS

• StorageExpansion• HalfRack(Prod)

“Wereducedthequeriesfrom30secondsdowntosub-secondresponse8me.Quickinforma8on,quickqueriesgiveCustomerCaretheabilitytodotheirjobbeDerandmeetthecustomer’sneeds.”-RichardEwald,SeniorTechnicalArchitect,DataWarehousing

•  ExadataX3-2HalfRack

Produc`on

•  ExadataX3-2FullRack•  HCC:950TBto150TB•  ZFSStorageAppliance(Backup)•  ExadataStorageExpansion•  ExadataX5-2HalfRack

Pre-Exadata

2012X3-2ZFS

Storage

MaximumAvailabilityNounplanneddown`me

FasterQueries>10x

15billiontransac`ons/day

StorageSavings6x

+removed150TBIndexes

OraclePla`numServices

DataCenterCostSavings

3:1Consolida`on

FasterReports24X

7daysto7hours

AutoServiceRequest

StorageExpansion

2014X4-2

SunFireE6900

SunM9000

•  4xSunFireE6900,1xM9000• MixedStorage• Mul`plebackupsystems•  90DayCDRDW1.15PB•  OracleDB11gR2

EMC/IBM/NetAppStorage

2015X5-2

Dev/Test

PulteGroup:MulQtenantConsolidaQonBenefits

BusinessObjec`ves

•  Scalability

•  Supportability

•  Sustainability

Solu`on

•  quarterrack&eighthrack

“Exadatadeliveredtremendousimprovementsinproduc8vity.Usersnolongerhavetowaitfordata.Datasharingisnowreal8me.”-BrianPawlik,ISManager,PulteHomes

FasterApplica`ons2x-15xFaster

LowerAdmin&SupportCosts40%Reduc`on

BusinessImpact

40%Produc`vityÛMonthlyClose2Days

Faster

CostSavings

40%CapEx

•  InforLawsonS3ERP;RebateTracking•  Consolidate35DBs:4CDBs,35PDBs•  Produc`on,LocalStandbyandQA•  Primarydatabases:>5TB

Pre-Exadata

EMC Storage

•  IBMP7•  EMCstoragearrays

IBM P7

ExadataQuarterRackProduc`on/Standby/TestDev/UAT

ExadataEighthRackDisasterRecovery

WAN@800milesAc`veDataGuard

ExadataAdvantagesIncreaseEveryYear

• SmartScan• InfiniBandScale-Out

• DatabaseAwareFlashCache• StorageIndexes• ColumnarCompression

• IOPrioriQes• DataMiningOffload• OffloadDecryptonScans

• In-MemoryFaultTolerance• Direct-to-wireProtocol• JSONandXMLoffload• InstantfailuredetecQon

• NetworkResourceManagement• MulQtenantAwareResourceMgmt• PrioriQzedFileRecovery

• UnifiedInfiniBand

• Scale-OutServers

• Scale-OutStorage• DBProcessorsinStorage

• PCIeNVMeFlash

• TieredDisk/Flash

• Soxware-in-Silicon

• 3DV-NANDFlash

• In-MemoryColumnarinFlash• SmartFusionBlockTransfer

• ExadataCloudService

Drama`callyBeTerPlawormforAllDatabaseWorkloads

• ExadataCloudMachine

“extending in-memory database processing to … · imc (in-memory columnar) data database server...

Documents

open-source columnar database technology beyond textbooks...

evaluating a modern in-memory columnar data …

columnar vortices v22 - vixra

scaling up concurrent main-memory column-store scans ... ·...

spacer length controlled oblique-columnar to lamello...

pyarrow documentation - read the docspyarrow documentation,...

teradata columnar

an approach for hybrid-memory scaling columnar in-memory

[]oracle® enterprise manager cloud control …analytic...

kriptanalisis algoritma transposisi columnar skripsi

in-memory columnar databases hyper - cs.helsinki.fi ·...

vertica mpp columnar dbms

!oracle!database!! in.memory opon - aioug · •...

columnar norway maple*

state of arkansas: lessons from three parallel sap hana ......

best practices in the use of columnar...

engineeringdatabasehardwareand sowaretogether’ ·...

memory scans based on alternative test

flotacion columnar

erck5/6 flat epithelial atypia columnar cell lesions: a...