iot insights and outcomes - ai convention europe...pentaho streamlines the entire machine learning...

© Hitachi Vantara Corporation 2019. All Rights Reserved.© Hitachi Vantara Corporation 2019. All Rights Reserved.© Hitachi Vantara Corporation 2019. All Rights Reserved.© Hitachi Vantara Corporation 2019. All Rights Reserved.

Intelligent DataOps powered

by Pentaho

IoT Insights and

Outcomes

Pol HuyzentruytSenior Solutions Consultant, Hitachi Vantara Belux

October 2019

© Hitachi Vantara Corporation 2019. All Rights Reserved.© Hitachi Vantara Corporation 2019. All Rights Reserved.

Agenda

Introduction to Hitachi and Intelligent DataOps

Accelerate your journey to DataOps with Pentaho Data Pipelines

Use cases

Q & A


100+ years of experience

The Hitachi Story


Your Data. Your Innovation.Digital Unlocks the Potential of

Data to Drive Business Outcomes

Reduce thefts,

improve safety,

lower costs

Reduce Costs

Offer your

customer new

services in order to

enlarge your

business

New Revenue

Models

Increase asset

utilization, improve

employee

effectiveness

Increase

Efficiency


The Customer Challenge

Getting the right data, to the right place at the right time is challenging for most

organizations. Here’s why:

Data is more diverse,

than ever before

Data is more

distributed than ever

Data is siloed across

applications, clouds, and

internal tools

It’s difficult for

organizations to know

what data they have

and where it is

Discovering and preparing

the right data for analytics

can be slow and manual

Lack of lean and agile

methodologies across

the data value chain

It’s difficult to reduce

costs in the way data is

stored and managed

The process of

transforming data into

value is difficult to repeat

Governance and

compliance requirements

are growing


Data Journey

Leaky Pipe Analysis

Value of DataPotential Value

IN

Value of DataActual Value

OUT

Growth of data complexity, coupled with customer transition to hybrid multi-cloud,

creates major challenges in data integration, orchestration and analytics

Still Heavy IT Involvement

Only 40% claim to provide

self-service

Unstructured Data

< 1% of unstructured

data analyzed.

80% Rule

Data prep is still 80% of an

analytics project

Deploying ML models is Hard

66% do not have automated

process

Skills gap with Big Data tools

28k Jobs on LinkedIn with

“Spark”

Cloud Creating Data Siloes

Average 5 clouds per

organization


Intelligent DataOps – the concept

Acquire the Data Store the Data Share the Data

Developers

Data Scientist

Data Engineers

Data Analysts

Business Users

Right Data. Right Place. Right Time.

STREAMING DATA

UNSTRUCTURED DATA

STRUCTURED DATA Enrich the Data

Explore the Data

“Data Delayed is Data Denied”


Ingestion Profiling Enrichment Blending AI / Orchestration

Discovery

Edge-to-Cloud Data Pipelines

Administration SecurityLifecycle

ManagementScalability Lineage Monitoring Automation

Data Analyst/

Steward

Business

AnalystData

EngineerData Scientist /

Data Ops

Data

EngineerData Analyst/

Steward

Data Insights

Pentaho Data

Integration

Pentaho Business Analytics


Intelligent DataOps and Pentaho

Acquire the Data Store the Data Share the Data

Developers

Data Scientist

Data Engineers

Data Analysts

Business Users

Right Data. Right Place. Right Time.

STREAMING DATA

UNSTRUCTURED DATA

STRUCTURED DATA Enrich the Data

Explore the Data

Pentaho Data

Integration

Pentaho Business Analytics

“Data Delayed is Data Denied”

Big Data Analytics ArchitectureIntegrating The Analytics & IoT World

Location

Web

Social

Media

Network

EDW Data

Marts

Customer

Orders

Financials

PDIPDI

Pentaho

Pentaho

Hadoop

Cluster

NoSQL

Analytic

DB

PDI

Pentaho

On-Demand Integration & Blending

PDIPentaho

PDIPentaho

HDInsight

Machine Learning

Embeddable,

Governed Datasets

Curated

Data

Business

Apps

Self-Service

Analytics

Predictive

Models

Machine Learning Orchestration - Solution

Pentaho streamlines the entire machine learning workflow and enable teams of data

scientists, engineers and analysts to train, tune, test and deploy predictive models

1 1 2

3

4Customer

Provisioning

Billing

Location

Web

Social

Media

Network

Notebook IntegrationsData Prep


Intelligent DataOps – Pentaho Foundation

End-to-end Analytics Platform

Big Data Accelerator

IoT Enabler

Machine Learning Orchestrator


Solutions through 3 use cases

Enablement &

collaborationVolume Real time ML


Case study


Alarm and process collection

correlation and analytics

Heat exchange network

data collection, analytics, and data science

Robust Quality Estimator (RQE) system

data analytics and data science


Many data

silos with

LOB

disconnect

Data volumes

growing –

100,000 data

sets / day

Manual

processes

and lack of

data maturity


Improved

lead time

from 2 days

to 10 minutes

Delivered

data

science

enablement

Unified data

architecture


Case study


Started by UK Government 2015 to reduce

travel time between London, Leeds,

Newcastle and Edinburgh


Hitachi Class 800: 20+ UK Suppliers


Collect Data From Trains

Load Information into Data Lake

Perform Computations for

Predictive Maintenance

(Machine Learning)

Perform Computations

for Signal Analysis

(Engineering)

Perform Computations

for Operations (PCS –

Passenger Count

System)

3.6 Millions Data Points /s(Hundreds of Trillions in 2years

!!)3000+ Sensors

Provide multi-tenant visualisation

20+ hardware suppliers


Solutions

• Pentaho Big data Platform

• End to end solution

• No code data prep

• ML automation

• Governance of data

Benefits

• Train Presence rate is optimized

• Better service for customers

• 1million+ £ in savings on maintenance

Pentaho Data

Integration

Pentaho Business

Analytics


Case study


Fuel usage is a major cost saving opportunity

In many cases

fuel cost

accounts for

30% of revenue

in a shipping

company

Fuel prices are

steadily rising,

as are tariffs on

high emissions

Customers are

increasingly

conscious of

reducing

environmental

impact

OPTIMISING FUEL USAGE CAN SAVE YOU MONEY, IMPROVE YOUR OPERATIONS,

AND GROW YOUR CUSTOMER BASE


AI enabled support to allow a captain take decisions based on

facts to optimize fuel consumption

Route planning optimization is

dependent on varying a ship's

speed in different parts of a route.

Since the external conditions

(wind, current and depth) vary,

fuel consumption cannot be

maintained at a minimum, if a

constant speed and direction is

kept throughout the route.

Therefore we have to find the

speed distribution and optimal

direction that minimizes the total

fuel consumption within the

constraint of keeping the

scheduled arrival time.

DESTINATION

The background..

Captain, reduce speed by 1 knot and

change direction with 1 degree east.

That will save you yy kg fuel.

AI ASSISTANT TO IMPROVE FUEL CONSUMPTION

OK, assistant. We saved xx%

on the last journey we made so I

trust you.

A captain assisted by a

trained AI to make the best

possible decisions in order

to improve fuel efficiency.

Scaling of the best

knowledge and make it

available to all captains

through a highly trained AI

assistant

The vision..

START


High level Machine Learning approach for PoC.

Feature selection and parameter learning

Data extraction Feature selection Relevant featuresLearn the

relationshipValidate

▪ water depth

▪ weather changes

▪ number of engaged engines

▪ water current, direction and

speed

▪ winds and waves, direction and

speed

▪ speed relative to ground

▪ speed through water

▪ Aerodynamical hull design

▪ Hull cleaning interval

▪ trim of the vessel

▪ pitch angle

▪ load balancing

▪ propeller RPM, torque

▪ bearing or routes

▪ fuel consumption of past trips

▪ Propulsion power

▪ …

▪ …

Input (history)Neural network

or any other ML

algorithm

▪ weather changes

▪ water current, direction and

speed

▪ winds and waves, direction

and speed


▪ speed through water



▪ fuel consumption of past

trips


▪ Propulsion power

▪ …

▪ …

OutputInput

▪ weather changes

▪ water current, direction and speed

▪ winds and waves, direction and speed




▪ fuel consumption of past trips


▪ …

▪ 𝒇𝒇𝒖𝒆𝒍𝒄𝒐𝒔𝒕(𝑠𝑝𝑒𝑒𝑑, 𝑡𝑟𝑖𝑚, 𝑐𝑢𝑟𝑟𝑒𝑛𝑡,𝑤𝑖𝑛𝑑. . )

▪ 𝒗𝒔𝒑𝒆𝒆𝒅 𝑡𝑟𝑖𝑚, 𝑐𝑢𝑟𝑟𝑒𝑛𝑡,𝑤𝑖𝑛𝑑, 𝑏𝑒𝑎𝑟𝑖𝑛𝑔, …

▪ 𝒓𝒕𝒓𝒊𝒎 𝑠𝑝𝑒𝑒𝑑, 𝑐𝑢𝑟𝑟𝑒𝑛𝑡,𝑤𝑖𝑛𝑑, 𝑏𝑒𝑎𝑟𝑖𝑛𝑔 …▪ 𝒙𝒓𝒐𝒖𝒕𝒆(𝑠𝑝𝑒𝑒𝑑, 𝑡𝑟𝑖𝑚, 𝑐𝑢𝑟𝑟𝑒𝑛𝑡,𝑤𝑖𝑛𝑑, … )▪ …

▪ speed


▪ Trim

▪ estimated fuel

consumption

▪ …

▪ …

▪ …

Input (present)

Output

Learnt

functions

Neural network

or any other ML

algorithm


Fuel Optimization Dashboard

© Hitachi Vantara Corporation 2018. All Rights Reserved© Hitachi Vantara Corporation 2019. All Rights Reserved.

Thank You

Pol Huyzentruyt

Hitachi Vantara Belux

Senior Solutions Consultant

[email protected]

iot insights and outcomes - ai convention europe...pentaho streamlines the entire machine learning...

Documents