“extending in-memory database processing to … · imc (in-memory columnar) data database server...
TRANSCRIPT
![Page 1: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/1.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
“ExtendingIN-MemoryDatabaseProcessingtoSharedFlash
GurmeetGoindiMasterProductManager
![Page 2: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/2.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirecQon.ItisintendedforinformaQonpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfuncQonality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andQmingofanyfeaturesorfuncQonalitydescribedforOracle’sproductsremainsatthesolediscreQonofOracle.
2
![Page 3: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/3.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
ExadataDatabaseMachine
3
Performance,AvailabilityandSecurity
BestPlaVormforOracleDatabaseson-premisesandintheCloud
Enabledby:• Single-vendoraccountability• Exclusivefocusondatabases• Deeph/wands/wintegraQon• RevoluQonaryapproachtostorage
![Page 4: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/4.jpg)
Copyright©2016Oracleand/oritsaffiliates.Allrightsreserved.
ExadataAchievesMemoryPerformancewithSharedFlash
• ExadataX6delivers300GB/secflashbandwidthtoanyserver– Approaches800GB/secaggregateDRAMbandwidthofDBservers
• MustmovecomputetodatatoachievefullflashpotenQal– Requiresowningfullstack,can’tbesolvedinstoragealone
• Fundamentally,storagearrayscanshareflashcapacitybutnotflashperformance– Evenwithnextgenscale-out,PCIenetworks,orNVMeoverfabric– e.g.newEMCDSSDhas3-6QmeslowerthroughputthanExadataX6
• Sharedstoragewithmemory-levelbandwidthisaparadigmchangeintheindustry– GetnearDRAMthroughput,withthecapacityofsharedflash
4
ExadataDBServers
ExadataSmartStorage
InfiniBand
CPUPCIeNVMeFlash
Chips
QueryOffload
![Page 5: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/5.jpg)
Copyright©2016Oracleand/oritsaffiliates.Allrightsreserved.
NVMePCI-eFlashDisruptstheStorageArrayModel
|OracleConfidenQal–HighlyRestricted 5
LatestPCIeFlash5.4GB/sec
SANLink=40Gb5GB/sec
Lessthan1Flashcard
LeadingAllFlashArray24GB/sec
Lessthan5Flashcard
Newimprovementsarecausing100XboTlenecksacrosssharedstoragestack
ArrayHeads
CPU
All-FlashStorageArrayIOPath:manysteps,eachaddslatencyandcreatesboTlenecks
SAS/SATA PCIeFlashChips
Switches
SAN/LAN
SSDCtrl
HostHBA
SAN/LAN
![Page 6: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/6.jpg)
Copyright©2016Oracleand/oritsaffiliates.Allrightsreserved.
RedesigningScanOffloadforMemoryThroughput
• WithExadataFlashthroughputapproachingmemorythroughput,SQLbohleneckmovesfromI/OtoCPU
• ExadatawillautomaQcallytransformtabledataintoIn-MemoryDBcolumnarformatsinExadataflashcache– DualformatarchitectureextendedfromDRAMtoflash
• Enablesfastvectorprocessingforstorageserverqueries– SmartScanresultssenttoDBusingIn-MemoryColumnarformattoreduceDBCPUusage
• UniquelyopQmizesnextgeneraQonflashasmemory
6
In-MemoryColumnarscans
In-FlashColumnarscans
![Page 7: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/7.jpg)
Copyright©2016Oracleand/oritsaffiliates.Allrightsreserved.
12.8TBFlashStorageServer
In-MemoryColumnarFormatsinDRAM(pre12.2.1.1.0)
7
Super-FastScansfromMemory,butAllQueriesComplete
In-MemoryColumnarscans
Upto1.5TBDRAM
SGA
IMC
DatabaseServer
DatanotinDRAM
![Page 8: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/8.jpg)
Copyright©2016Oracleand/oritsaffiliates.Allrightsreserved.
12.8TBFlash
In-MemoryColumnarFormatsinFlashCache(12.2.1.1.0)
8
3-4xOverallAnaly`csPerformanceImprovement
Upto1.5TBDRAM
SGA
IMC
12.8TBFlashx3=38.4TB(ormore)IMC(In-MemoryColumnar)data
DatabaseServer
In-MemoryColumnarscans
In-FlashColumnarscans
HybridColumnarCompressedData
StorageServer
![Page 9: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/9.jpg)
Copyright©2017Oracleand/oritsaffiliates.Allrightsreserved.
SmartAnaly`cs:JoinandAggregaQonSmartScan
• ExtendIn-MemoryAggregaQontechniqueintostorage(vectorjoinsandvectoraggregaQon)
• FindSalespercountrySELECT /*+ VECTOR_TRANSFORM */ country_id, sum(amount_sold) amount_sold FROM customers, sales WHERE customers.cust_id = sales.cust_id GROUP BY customers.country_id ORDER BY customers.country_id;
• Storagecellsscanningsalesfacttablereturntuples{country_id, sum_amount_sold }
• JoinandAggregaQonoffloadedtothestorageserver
9
NEWINDB12.2
![Page 10: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/10.jpg)
Copyright©2017Oracleand/oritsaffiliates.Allrightsreserved.
SmartAnaly`cs:MoreSmartScanEnhancements
• SmartScanenhancementsforXMLandJSON– JSON_EXISTS,JSON_VALUE,JSON_QUERY,"ISJSON"and"ISNOTJSON"– XML:XMLExists,XMLCast(XMLQuery())
• SignificantspeedupinJSONanalyQcworkloadsselect count(*) from pictures where json_value(photo, ‘$.tag’) like ‘%spain%’;
10
NEWINDB12.2
![Page 11: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/11.jpg)
Copyright©2017Oracleand/oritsaffiliates.Allrightsreserved. 11
ColdorRawData
RealTimeDataAnalysis
Hot(FrequentlyAccessed)Data
CoolingData
DataTiering
DBIM&FlashCache
HCConExadataStorage
ZFSorBigDataSQL
DBIM
![Page 12: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/12.jpg)
Copyright©2017Oracleand/oritsaffiliates.Allrightsreserved.
SmartAnaly`cs:SmartWriteBurstsandTempIOinFlashCache
• Writethroughputoffourflashcardshasbecomegreaterthanthewritethroughputof12-disks
• Whendatabasewritethroughputexceedsthroughputofdisks,SmartFlashCacheintelligentlycacheswrites
• WhenquerieswritealotoftempIO,SmartFlashCacheintelligentlycachestempIO– WritestoflashfortempspillreduceselapsedQme– ReadsfromflashfortempreduceselapsedQmefurther
• SmartFlashCacheprioriQzesOLTPdataanddoesnotremovehotOLTPlinesfromthecache
• Smartflashwearmanagementforlargewrites
• SupportsDatabase11.2.0.4,12.1.0.2and12.2.0.1
12
WriteBurstsandTempIOinFlashCache
AcceleratesLargeJoinsandSortsandLargeDataLoads
![Page 13: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/13.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
ExadataCloud–YourWay
13
OraclePublicCloud
ExadataCloudService
ExadataCloudMachine
CloudatCustomer
SameProduct
SamePrice
OracleManaged
OracleOwned
![Page 14: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/14.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
ExadataCustomerCaseStudies
14
![Page 15: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/15.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
NTTdocomo:MoBills(MobileBillingSystem)
15
Benefits
BusinessObjec`ves
• Real-TimeBillingPlaVormfor66millioncustomer
• DramaQcallyimproveperformanceandavailability
• Reducecostandcomplexity
Solu`on
• OracleExadata:30racks• OracleMAA(RAC/AcQve
DataGuard-Local&RemoteStandbydatabase)
FasterBillingProcessing
10Xspeedup
3millionSQL/sec
DataCenterCostSavings
90%SpaceReduc`on
ReducedIntroduc`onCost
25%"MoBills is a very important position as a mission-critical system to promote efforts toward the realization of “+d”. Oracle Exadata is running very stable as a expected performance. We will continue to use the "Oracle Exadata" and we would like to establish a further advantage for our business." - Shimamura, Manager, Information System Department, NTT docomo
ReducedOpera`onalCost
50%
MaximumAvailability
Local&RemoteStandby
Pre-Exadata ExadataMAA30racks/Local&RemoteStandby/RMANbackup
Tokyo Osaka
DataGuard
DataGuard
LocalStandby5racks
RemoteStandby5racks
RemoteStandby5racks
LocalStandby5racks
Ra`ng(primary)5nodeRAC*5racks
Billing(primary)5nodeRAC*5racks
• Real-TimeBillingProcessing• High-endSMPServer
+High-endStorage:350racks• StorageMirrorBackup
• StorageMirrorReplica`on• Oracle9iDatabaseRelease2
Upgrade&Migra`on
![Page 16: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/16.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
DCMHoldings:SystemConsolida`onof3companies
16
Benefits
BusinessObjec`ves
• $10billionSales,FasterM&A• HighConsolidaQonraQoand
improveservicelevel• ReduceoperaQonalcost
Solu`on
• OracleDatabase12cMulQtenantonExadata
• OracleFullStack(MiddlewareandServerproducts)
Realized the Database consolidation and integration due to the high performance provided by Oracle Exadata. And, Oracle Database 12c Multitenant Architecture also achieved high consolidation ratio while maintaining the independence of each group companies. Platinum Service could improve the service level, Oracle Full-stack products could provide One-Stop Support.
Pre-Exadata
OracleMul`tenantonExadata • Consolida`onandIntegra`on
3groupcompanies(Homac,KahmaandDAIKI)ofsysteminfrastructure
• ReplacedfromIBMpServers• TeradataMigra`ontoExadata• OracleDatabase12cMul`tenant• Pla`numService• Zerodown`meSystemMigra`onbyusingGoldenGate
Integra`on/Consolida`on
FasterBatchProcessingReducedIntroduc`onCost
2Xspeedup 40%Off
Standardiza`on
HighConsolida`onRa`oImprovedManageability
6DBsConsolida`on
Mul`tenantArchitecture
SimplifiedSupport
NonStopSupport
OracleFullStack
![Page 17: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/17.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
Sprint:CallDataRecord-DataWarehouseBenefits
BusinessObjec`ves
• Improveperformance
• Improvesustainability
• Improveavailabilityandmaintainability
Solu`on
• FullRack(Prod),HalfRack(Dev/Test);ZFS
• StorageExpansion• HalfRack(Prod)
“Wereducedthequeriesfrom30secondsdowntosub-secondresponse8me.Quickinforma8on,quickqueriesgiveCustomerCaretheabilitytodotheirjobbeDerandmeetthecustomer’sneeds.”-RichardEwald,SeniorTechnicalArchitect,DataWarehousing
• ExadataX3-2HalfRack
Produc`on
• ExadataX3-2FullRack• HCC:950TBto150TB• ZFSStorageAppliance(Backup)• ExadataStorageExpansion• ExadataX5-2HalfRack
Pre-Exadata
2012X3-2ZFS
Storage
MaximumAvailabilityNounplanneddown`me
FasterQueries>10x
15billiontransac`ons/day
StorageSavings6x
+removed150TBIndexes
OraclePla`numServices
DataCenterCostSavings
3:1Consolida`on
FasterReports24X
7daysto7hours
AutoServiceRequest
StorageExpansion
2014X4-2
SunFireE6900
SunM9000
• 4xSunFireE6900,1xM9000• MixedStorage• Mul`plebackupsystems• 90DayCDRDW1.15PB• OracleDB11gR2
EMC/IBM/NetAppStorage
2015X5-2
Dev/Test
17
![Page 18: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/18.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
PulteGroup:MulQtenantConsolidaQonBenefits
BusinessObjec`ves
• Scalability
• Supportability
• Sustainability
Solu`on
• quarterrack&eighthrack
“Exadatadeliveredtremendousimprovementsinproduc8vity.Usersnolongerhavetowaitfordata.Datasharingisnowreal8me.”-BrianPawlik,ISManager,PulteHomes
FasterApplica`ons2x-15xFaster
LowerAdmin&SupportCosts40%Reduc`on
BusinessImpact
40%Produc`vityÛMonthlyClose2Days
Faster
CostSavings
40%CapEx
• InforLawsonS3ERP;RebateTracking• Consolidate35DBs:4CDBs,35PDBs• Produc`on,LocalStandbyandQA• Primarydatabases:>5TB
Pre-Exadata
EMC Storage
• IBMP7• EMCstoragearrays
IBM P7
ExadataQuarterRackProduc`on/Standby/TestDev/UAT
ExadataEighthRackDisasterRecovery
WAN@800milesAc`veDataGuard
18
![Page 19: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/19.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.|
ExadataAdvantagesIncreaseEveryYear
19
• SmartScan• InfiniBandScale-Out
• DatabaseAwareFlashCache• StorageIndexes• ColumnarCompression
• IOPrioriQes• DataMiningOffload• OffloadDecryptonScans
• In-MemoryFaultTolerance• Direct-to-wireProtocol• JSONandXMLoffload• InstantfailuredetecQon
• NetworkResourceManagement• MulQtenantAwareResourceMgmt• PrioriQzedFileRecovery
• UnifiedInfiniBand
• Scale-OutServers
• Scale-OutStorage• DBProcessorsinStorage
• PCIeNVMeFlash
• TieredDisk/Flash
• Soxware-in-Silicon
• 3DV-NANDFlash
• In-MemoryColumnarinFlash• SmartFusionBlockTransfer
• ExadataCloudService
Drama`callyBeTerPlawormforAllDatabaseWorkloads
• ExadataCloudMachine
![Page 20: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/20.jpg)
Copyright©2017,Oracleand/oritsaffiliates.Allrightsreserved.| 20
![Page 21: “Extending IN-Memory Database Processing to … · IMC (In-Memory Columnar) data Database Server In-Memory Columnar scans In-Flash Columnar scans Hybrid Columnar Compressed Data](https://reader031.vdocuments.net/reader031/viewer/2022022613/5b9f2fa309d3f2083f8cb46e/html5/thumbnails/21.jpg)