oracle in-memory data processing...oracle in-memory data processing doug hood @scalabledbdoug...
TRANSCRIPT
Copyright©2016, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleIn-MemoryDataProcessing
DougHood@ScalableDBDougConsultingMemberofTechnicalStaffProductManagerforOracleTimesTen,OracleInMemoryandOracleNoSQLDec12,2018
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
OracleDatabaseIn-MemoryEnablingReal-TimeAnalytics
AndyRivenesDatabaseIn-MemoryProductManagement
Oracle Corporation
Email:[email protected]:@TheInMemoryGuy
Blog:blogs.oracle.com/in-memory
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.
3
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
WhatareOracle’sIn-MemoryDatabases
4
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
In-MemoryDatabaseTechnologyAcrossTiers
Application Application Application
Application Application Application
TimesTen In-MemoryDatabase• EmbeddableIn-MemoryDatabaseforApplicationTier• PrimaryUsecase: Latency-criticalcustomOLTP
qMicrosecondResponseTime• StandaloneDatabaseorasApplication-TierCacheforOracleDatabase
OracleDatabaseIn-Memory• Dual-FormatIn-MemoryDatabase• PrimaryUsecase: RealTimeAnalyticsonanysource
qBillionsofRows/Secanalyticdataaccess• Faster mixed-workloadenterpriseOLTP• Storage-Tiering:Combinesbestofmemory,flash,disk• Transparent:packagedappsrunwithnochanges
In-MemoryRowStore
In-MemoryColumnStore
ApplicationTier
DatabaseTier
5
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
WhatisDatabaseIn-Memory
6
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
RowFormatDatabasesvs.ColumnFormatDatabases
RowsStoredContiguously
§ Transactions runfasteronrowformat– Example:QueryorInsertasalesorder– Fastprocessingfewrows,manycolumns
ColumnsStoredContiguously
§ Analytics runfasteroncolumnformat– Example:Reportonsalestotalsbyregion– Fastaccessingfewcolumns,manyrows
SALES
SALES
7
UntilNowMustChooseOneFormatandSufferTradeoffs
QueryQuery
Que
ry
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
Breakthrough:DualFormatDatabase
• BOTH rowandcolumnformatsforsametable
• Simultaneouslyactiveandtransactionallyconsistent
• Analytics&reportingusenewin-memoryColumnformat
• OLTPusesprovenrowformat
8
BufferCacheNewIn-MemoryColumnStore
SALES SALESRow
FormatColumnFormat
SALES
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
Howeasyisittogetstarted
9
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
DatabaseIn-Memory:TrivialtoImplement
10
SimpletoConfigure
NocomplexsetupSetcolumnstoresize
DeclareIn-Memorytables
FullFunctionality
NoSQLrestrictions
100%Compatible
Noapplicationchanges
Nodatamigration
EasytoDeploy
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
Oracle In-Memory: Simple to Setup
1. Configure Memory Capacity• inmemory_size = XXX GB
2. Configure tables or partitions to be in memory• alter table | partition … inmemory;
3. Later drop analytic indexes to speed up OLTP
11
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
WhyAreAnalyticQueriesFasterInTheIn-MemoryColumnStore?
12
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
ScanningandfilteringdatamoreefficientlyDatabaseIn-MemoryTechnology
StorageIndexesCompression
Scan&filterdataincompressedformat
Accessonlythecolumnsyouneed
ColumnarFormat SIMDVectorProcessing
ProcessmultiplecolumnvaluesinasingleCPUinstruction
VectorRegister
Loadmultipleregionvalues
VectorCompareallvaluesan1cycle
CPUCA
CACA
CA
Min1Max3
Min4Max7
Min8Max12
Pruneoutanyunnecessarydatafromthecolumn
13
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
• Speedofmemory• ScanandFilteronlytheneededColumns
• VectorInstructions
OptimizerEnhancementsImprovesallaspectsofanalyticqueries
DataScansVe
ctorRegisterCPU
CA
CACA
CA
In-MemoryAggregation
•CreateIn-MemoryReportOutlinethatisPopulatedduringFastScan
•RunsReportsInstantly
Joins
•ConvertStarJoinsinto10XFasterColumnScans
•Searchlargetableforvaluesthatmatchsmalltable
HASHJOIN
TableA TableB
SALES
STATE=CA
14
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
DatabaseIn-MemoryandotherOracleDatabasefeatures
15
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
DatabaseIn-Memory:ScaleOut
16
Scale-Up
•Scale-UponlargeSMPs•NUMAOptimized
•Scale-OutAcrossServerstoGrowMemoryandCPUs
• In-MemoryQueriesParallelizedAcrossServers
Scale-Out CombinewithFlashandDisk
DISKPCIFLASHDRAM
ColdDataHottestData
ActiveData
•Easilyplacedataonmostcosteffectivetier
•SimultaneouslyAchieve:•Speed ofDRAM• I/OsofFlash•Cost ofDisk
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.| 17
RAC
ASM
RMAN
Data Guard & GoldenGate
• Pure In-Memory format does not change Oracle’s storage format, logging, backup, recovery, etc.
• All Oracle’s proven availability technologies work transparently
• Protection from all failures- Node, site, corruption, human error, etc.
DatabaseIn-Memory:IndustrialStrengthAvailability
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
ExtendIn-MemoryAnalyticsintoStorage
18
In-MemoryColumnarscans
In-FlashColumnarscans
• ExadataautomaticallytransformstabledataintoIn-MemoryDBcolumnarformatsinExadataFlashCache– Enablesfastvectorprocessingforstorageserverqueries
• AdditionalcompressionforOLTPcompressedoruncompressedtablesinflash– newin18.1
• Enablesdictionarylookupandavoidsprocessingunnecessaryrows
• SmartScanresultssentbacktodatabaseinIn-MemoryColumnarformat– ReducesDatabasenodeCPUutilization
• Uniquely optimizesnextgenerationFlashasmemory
Upto1.5TBDRAM perServer
Upto25.6TBFlash perServer
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
HowarecustomersusingDatabaseIn-Memory
19
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
AT&TWiFi– DataWarehouse• BusinessObjectsreports100X faster
• ETLprocessesimprovedby50% faster
• NochangestoSAPBusinessObjectsreports
Villeroy&Boch– SAPBW• SAPBWCOPAqueries30– 33X faster
• SAPTransactionlistqueries4– 4,800Xfaster
• Avoidedexpensive&riskyupgradetoS4/Hana
20
BOSCH – SAPCRM• Dropped allcustomindexes
• Analyticqueries2-20Xfaster,DML2-3X faster
• Nochangestoapplicationrequired
DieMobiliar– MixedWorkload• Analyticqueries50-200X faster
• Databasesizereduced considerably
• PhaseoutofNetezzaandmainframesystems
DatabaseIn-MemoryReferences
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
MankindPharma– MixedWorkload• Analyticalreports11x faster
• DroppingindexesimprovedOLTP• 90% reduction indatabasesize
Lufthansa – ReportingApplication• Analyticqueriesupto100x faster
• Improveddataingestperformance• Reductionindatabasesize
21
• LION – SAPERP• Analyticqueries4X faster
• Transactions2X faster• Analyticqueriesnowpossibleon100sofMillionsofPoint-of-SalesTransactions
ShanghaiCustoms– MixedWorkload• ProcessesClearance43x Faster
• ImprovesDeclaration-ServicesEfficiency• ReducedCosts
DatabaseIn-MemoryCustomers
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
Canweseeanexample
22
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
TemperatureTrendsDemo
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
WherecanIgetmoreinformation
24
Copyright©2018, Oracleand/oritsaffiliates.Allrightsreserved.|
JointheConversation
25
https://twitter.com/TheInMemoryGuyhttps://blogs.oracle.com/In-Memory/
WhitePapers(otn.com)• OracleDatabaseIn-MemoryWhitePaper• OracleDatabaseImplementationandUsageWhitePaper
• OracleDatabaseIn-MemoryAggregationPaper• WhentouseOracleDatabaseIn-Memory• OracleDatabaseIn-MemoryAdvisor
Videos• OracleDatabaseIn-MemoryYouTubeChannel• oracle.comPoweringtheReal-TimeEnterpriseoracle.com/us/corporate/events/dbim/index.htmlReal-TimeAnalyticsDemo
•AskTOMDBIMOfficeHoursSessions
AdditionalQuestions• In-Memoryblog:blogs.oracle.com/In-Memory• Myemail:[email protected]
https://www.facebook.com/OracleDatabasehttp://www.oracle.com/goto/dbim.html
AdditionalResources
https://twitter.com/db_inmemory