oil & gas big data use cases
TRANSCRIPT
1 ©HortonworksInc.2011– 2016.AllRightsReserved
PoweringDataDrivenTransformationsHortonworksinOil&Gas
2 ©HortonworksInc.2011– 2016.AllRightsReserved
DATAATRESTDATAINMOTION
ACTIONABLEINTELLIGENCE
ModernDataApplications
PERISHABLEINSIGHTS
HISTORICALINSIGHTS
INTERNETOF
ANYTHING
HortonworksDataFlow
HortonworksDataPlatform
HortonworksDeliversConnected DataPlatforms…
Capturingperishableinsightsfromdatainmotion
Ensuringrich,historicalinsightsondataatrest
3 ©HortonworksInc.2011– 2016.AllRightsReserved
FieldDataCapture OfficeorDatacenter
HortonworksIndustrialDataAnalyticsPlatform– InPractice
WITSML
Files/OtherUnstructuredData
Video
IoTGateways
PLC/RTU
SCADA, DCS,Historians
CentralHDPCluster
Hive
CentralHDFCluster
NiFi
Kafka
Storm
StreamingOptions
HBase Solr
YARN
HDFS
Location1
NiFi
Locationn
NiFi
DataCenter
DataIngestionFramework
Endusers
DATA IN MOT ION – HDF DATA AT REST – HDP
HDFEdge(MiNiFi+NiFi)§ Reliablecollection§ Smallfootprint§ Edgeprocessing
§ Dataprovenance§ Integrateswithcore
policies
HDFCore(NiFiwithStreaming)§ Processingatlargerscale§ Distributedstreamprocessing
HDP§ Securityanddatagovernance§ Monitoring,management,operations§ Applications§ Analytics
StructuredDataSets
4 ©HortonworksInc.2011– 2016.AllRightsReserved
Oil&GasIndustrySelectedUseCases
5 ©HortonworksInc.2011– 2016.AllRightsReserved
CyberSecurityAnalytics
G&GAnalytics
YieldAnalytics
DefectDetection
AssessPhysicalThreats
ChemicalsandRefinery
R&D
ProductDesign
M&A
CompetitiveAnalysis
Geo
Asset
SupplyChain
PipelineSurveillance
LeasedSites
BusinessUnit
LeakDetection
MfgProcess Customers
Basin
Records Mgmt
Legacy Offloads
Historical Archive
Device Data
Ingest
Rapid Reporting
Digital Protection
WellPlanning Real-TimeOperations
RiskModeling IoT
Pre-DrillAnalytics
RouteOptimization
ReduceQHSEIncidents
TradingSurveillance
EquipmentPredictions
EquipmentPredictions
EDW Archive
Dataas a Service
eDiscovery
Connected Well
Seismic Analytics
PublicDataCapture
6 ©HortonworksInc.2011– 2016.AllRightsReserved
Whatà StoringyearsworthoflogfilesfromalloftheirwellswasverycostlyinlegacyO&GsystemsandtheirEDWà Analystsdidn’thaveaccesstologdatastoredinlegacysystemsandcouldn’tusecommonanalyticapplicationstoperformdata
science,buildforecastmodels,predictiveuse-cases,etc.Ã Existingplatformsdidn’thavethefunctionalitytocombinelogdatawithdatafromothersystemslikeauctionhistory,
productiondata,seismicdata,geolocation&perforationdata.
Whyà Activearchivebroadensaccesstowelllogdatawhichisotherwiseonlyavailabletospecializedsoftwareà Servesasafoundationaldatasetforfutureusecaseswherelogdatacanbeeasilyjoinedaspartofotherwellandformation
analysisordatascienceà Accelerationofgeologicalandgeophysicalworkflowsandprocessautomation
Howà UsingHDPanalysiscannowstorealloftheirhistoricallogfilesatafractionofthecostinadditionjoiningthattowelldatafrom
othersystemsà Analystsnowhavealllogfilesinasinglelocationwiththeabilitytobuild“syntheticlogfiles”comprisedofdatafrom
multiplesourcesandperformanalyticsoncombineddatasetsfromsiloedapplications.
WhyHortonworks?
DataDiscoveryBlogpost:http://hortonworks.com/blog/big-data-on-a-budget-in-oil-gas/
LASActiveArchive&Analytics
7 ©HortonworksInc.2011– 2016.AllRightsReserved
FieldDataCapture OfficeorDatacenter
LASActiveArchive&Analytics– ReferenceArchitecture
WITSML
Files/OtherUnstructuredData
Video
IoTGateways
PLC/RTU
SCADA, DCS,Historians
CentralHDPCluster
Hive
CentralHDFCluster
NiFi
Kafka
Storm
StreamingOptions
HBase Solr
YARN
HDFS
Location1
NiFi
Locationn
NiFi
DataCenter
DataIngestionFramework
Endusers
DATA IN MOT ION – HDF DATA AT REST – HDP
HDFEdge(MiNiFi+NiFi)§ Reliablecollection§ Smallfootprint§ Edgeprocessing
§ Dataprovenance§ Integrateswithcore
policies
HDFCore(NiFiwithStreaming)§ Processingatlargerscale§ Distributedstreamprocessing
HDP§ Securityanddatagovernance§ Monitoring,management,operations§ Applications§ Analytics
StructuredDataSets
8 ©HortonworksInc.2011– 2016.AllRightsReserved
Blogpost:http://hortonworks.com/blog/industrial-iot-offshore-drilling-solution-delivered-less-90-days/
Real-timeDrilling&Production WhyHortonworks?
DataDiscovery
Whatà Streamcriticaltelemetryofdrilling,completionsandproductionprocessintobigdataplatformà Drillingtypicallydoneinremotepartsoftheworld,whereconnectivityisintermittent,latentandprovideminimalbandwidth.à Typicalremotemonitoringtechnologiesdonotperformwellintheseconditions,thereforelargevolumesofdataendup
strandedandoutofreachforanalystandsupportteams
Whyà Remotedrillingoperationsoperatereactivelyandsufferfromunnecessarydowntime,equipmentfailures,efficiencylosses,and
safetyrisks.à ProvidesIndustrialControlSystemdatasetsinformatcomprehensibletotraditionalanalyticaltechniquesà BSEE(thegoverningbodyforoffshoredrillingintheU.S.)proposednewregulationsthatrequireoffshoredrillerstomonitor
safetycriticalequipmentinreal-timeandarchivethedataatanoffshorefacility
Howà HortonworksandKepwaredeliveredanIndustryIIoT Solutioninlessthan90daysforacquiringandstoringtimeseries
measurementsandrelatedsafetyequipmentdatafromanoperateddrillshipusingKepwareIoTGateway,HDF&HDP.Ã KeydataconsumptionpatternsplannedincludeKPIdashboards,condition-basedmonitoringandmaintenance,event-based
surveillance,andtraditionalBIreporting;ensuringsafermoreefficientoffshoreoperations.
9 ©HortonworksInc.2011– 2016.AllRightsReserved
FieldDataCapture OfficeorDatacenter
Real-timeDrilling&Production– ReferenceArchitecture
WITSML
Files/OtherUnstructuredData
Video
IoTGateways
PLC/RTU
SCADA, DCS,Historians
CentralHDPCluster
Hive
CentralHDFCluster
NiFi
Kafka
Storm
StreamingOptions
HBase Solr
YARN
HDFS
Location1
NiFi
Locationn
NiFi
DataCenter
DataIngestionFramework
Endusers
DATA IN MOT ION – HDF DATA AT REST – HDP
HDFEdge(MiNiFi+NiFi)§ Reliablecollection§ Smallfootprint§ Edgeprocessing
§ Dataprovenance§ Integrateswithcore
policies
HDFCore(NiFiwithStreaming)§ Processingatlargerscale§ Distributedstreamprocessing
HDP§ Securityanddatagovernance§ Monitoring,management,operations§ Applications§ Analytics
StructuredDataSets
10 ©HortonworksInc.2011– 2016.AllRightsReserved
Whatà Modelbasedfailurepreventionbasedonequipmenttelemetryandassetpatternsà Everyformofartificialliftispronetofailurewhensubjectedtodifferenttypesofreservoirconditionsà Problemslikescale,lowstaticbottomholepressure,overloadandrodstringfailurearetheresultofmachinesencountering
gas,sand,H2SandCO2corrosionandothersubsurfaceconditions
Whyà Increasedproductionthroughfewerunplannedwellinterventionsà Decreasedunplanneddowntimeandcostforrecoveringfailedequipmentfromdownhole
Howà CollectstreamingdatausingHDF,storeallcurrentandhistoricalpatternsofdatausageincludingfailurehistoriesà NOTE:thiscanbedevelopedmanuallyorthroughpartnerlikeOspreyData
WhyHortonworks?
PredictiveAnalyticsProductionOptimization
11 ©HortonworksInc.2011– 2016.AllRightsReserved
FieldDataCapture OfficeorDatacenter
ProductionOptimization– ReferenceArchitecture
WITSML
Files/OtherUnstructuredData
Video
IoTGateways
PLC/RTU
SCADA, DCS,Historians
CentralHDPCluster
Hive
CentralHDFCluster
NiFi
Kafka
Storm
StreamingOptions
HBase Solr
YARN
HDFS
Location1
NiFi
Locationn
NiFi
DataCenter
DataIngestionFramework
Endusers
DATA IN MOT ION – HDF DATA AT REST – HDP
HDFEdge(MiNiFi+NiFi)§ Reliablecollection§ Smallfootprint§ Edgeprocessing
§ Dataprovenance§ Integrateswithcore
policies
HDFCore(NiFiwithStreaming)§ Processingatlargerscale§ Distributedstreamprocessing
HDP§ Securityanddatagovernance§ Monitoring,management,operations§ Applications§ Analytics
StructuredDataSets
12 ©HortonworksInc.2011– 2016.AllRightsReserved
SingleViewofanAsset WhyHortonworks?
SingleView/DataDiscovery
Whatà DatasourcesandapplicationsusedbyassetteamresidedinsiloswithlimitedvisibilityacrossOT&ITsystems.à OperationsteamhavingtorelyonITtogeneratereportsfromenterprisesystemsthatweren’ttimelyandrequiredmanualdata
integrationusingspreadsheetsà Extractdatafromrelationaldatastoresthatunderpinlineofbusinessapps(Wellview,OFMetc)
Whyà Provideasingleenvironmentforexploringcurrentandhistoricalconditions,KPIsandwelleconomicsà Usingthesolutiontheassetteamhasbeenabletoidentifypotentialwellfailures4X-5Xfasterthanbefore
Howà UsingHDPandacommonvisualizationapplication,the360ViewsolutionwasbuiltinthreemonthscombiningOTdatasources
likeSCADA,maintenancelogs,&unstructureddata(textfiles,emails)withERPdata,geospatial,andexternaldatafrommidstreampartners,andweatherdata.
à Thesolutionisfairlyself-servicethatcanbemodifiedbytheoperationsteamwithlimitedinvolvementfromIT.
13 ©HortonworksInc.2011– 2016.AllRightsReserved
FieldDataCapture OfficeorDatacenter
SingleViewofanAsset– ReferenceArchitecture
WITSML
Files/OtherUnstructuredData
Video
IoTGateways
PLC/RTU
SCADA, DCS,Historians
CentralHDPCluster
Hive
CentralHDFCluster
NiFi
Kafka
Storm
StreamingOptions
HBase Solr
YARN
HDFS
Location1
NiFi
Locationn
NiFi
DataCenter
DataIngestionFramework
Endusers
DATA IN MOT ION – HDF DATA AT REST – HDP
HDFEdge(MiNiFi+NiFi)§ Reliablecollection§ Smallfootprint§ Edgeprocessing
§ Dataprovenance§ Integrateswithcore
policies
HDFCore(NiFiwithStreaming)§ Processingatlargerscale§ Distributedstreamprocessing
HDP§ Securityanddatagovernance§ Monitoring,management,operations§ Applications§ Analytics
StructuredDataSets
14 ©HortonworksInc.2011– 2016.AllRightsReserved
AboutHortonworksCustomerMomentumà ~1400customers(asofOctober2016)à ~400customersaddedin2015à PubliclytradedonNASDAQ:HDP
TheLeaderinConnectedDataPlatformsà HortonworksDataFlowfordatainmotionà HortonworksDataPlatformfordataatrestà Poweringnewmoderndataapplications
PartnerforCustomerSuccessà Leaderinopen-sourcecommunity,focused
oninnovationtomeetenterpriseneedsà Unrivaledsupportsubscriptions
Foundedin2011
Original24Architects,Developers,OperatorsofHadoopfromYahoo!
1000+EM P LOY E E S
1500+ECO S Y S T EM PA R T N E R S