sap hana: predictive analytics in real-time · 2014-12-17 · 4 5 6? ? ? ? ? ? crm payment history...

SAP HANA: Predictive Analytics in

Real-time Dr. Markus Kohler, SAP Data Science solutions

December 02, 2014

Custo

mer

© 2014 SAP SE or an SAP affiliate company. All rights reserved. 2 Customer

Agenda

SAP Data Science solutions

Overview: Predictive Analytics

SAP HANA: Platform for Predictive

HANA Customer stories

SAP Predictive Analytics

SAP Data Science solutions


„Advanced Analytics“

Advanced Analytics as a Competitive Differentiator

Co

mp

eti

tive A

dv

an

tag

e

Analytical Maturity

Standard Reporting

Ad-hoc Analysis

Data Mining

Modelling

Optimization

SAP provides products and services that cover all stages of analytical

maturity


The SAP Data Science Team

SAP Data Science has the expertise to deliver advanced analytics

projects end-to-end

Big

Data

S

cie

nc

e

Visualization

SAP Business Objects

SAP UI5

SAP Visual Business

Mathematical modeling

Statistical analysis

Algorithm development

SAP HANA and Sybase

Data modeling

ETL processes

> 100 CUSTOMERS

> 50 EXPERTS


Data Science Services – a key pillar of SAP’s advanced

analytics portfolio

SAP’s tool chain comes to life with SAP Data Science predictive

business content

SAP Big Data

platform

SAP Business Objects

SAP UI5

Predictive

and

Optimization

Algorithms

http://imagedirectory.wdf.sap.corp:6085/ImageLibrary/ImageDetailsView?imageId=575da6e7-86fa-4bc3-b967-cdd22416f02b

http://imagedirectory.wdf.sap.corp:6085/ImageLibrary/ImageDetailsView?imageId=0d838ce6-abcc-44c7-b2e7-3c1cfcc2ee96

Overview What is advanced analytics?


Competing in today’s demanding

marketplace means going beyond “what happened”

Tom Davenport International Institute for Analytics

How and why

did it happen?

What is the risk if it

does/doesn’t happen?

How do you prevent /

ensure it happens again?

What

happened?

What is

happening now?

What will

happen?

OP

ER

AT

ION

S l H

R l F

INA

NC

E

|

IT

| S

AL

ES

l M

AR

KE

TIN

G

MANUFACTURING l RETAIL l HEALTHCARE l BANKING l UTILITIES l TELCO | PUBLIC SECTOR | FINANCIAL SERVICES


Applications for predictive analytics


What is predictive analytics? An example – target group for marketing campaign

Business req. Identify data source Consolidated data

Predictive Model Analyze Operationalize

attribute

att

rib

ute

1 2 3

4 5 6

? ? ?

?

? ?

CRM Payment

history

Service

request

age

Click

stream

Business partner Age Frequency of

recent service

requests

…….

000010333000 22 0

000011132343 40 3

000012223443 35 15


What is predictive analytics?

An example – target group for marketing campaign

Gender

Age Spending

power

Decision tree model

m f

Soccer

journal

<5

Daily

newspaper Girls journal

Fashion

magazine

>5 <15 >15

SAP HANA Platform for advanced analytics


SAP HANA In-Memory Database and Platform for Predictive Analytics

SAP HANA

Studio

SAP Industry &

LoB Applications &

Rapid-Deployment

Solutions (RDS)

Partner Tools &

Solutions

In-Memory Processing Engine

Application Function Library

R-Engine R Script

R-Machine

Text-Analysis

Text Search Spatial

Processing

Business

Function Library

Predictive

Analysis Library

Core Database

RulesEngine

Automated

Predictive Library

Data Types Connects to SAP HANA directly OR via Sybase IQ/Hadoop/ESP/Data Services

Location

Data Machine

Data

Time-Series

Data Transaction

Data

Unstructured

Data Real-time (Stream)

Data

Data

Connectors

SAP Business Suite HADOOP SAP Sybase IQ,

ESP, ASE

3rd Party Data

Source SAP Data Services

Application

Function

Modeler

SAP HANA

Studio

Java-Script Server for

HANA Native Application

13

SAP HANA Unified Analytics Platform

SAP HANA XS Engine

* APL is a possible future

innovation and may be subject to

changes.

*

SAP Lumira

Predictive Analysis

SAP InfiniteInsight

SAP HANA Studio

SAP HANA

Studio Web Browser Partner tools &

Solutions



SAP HANA

Studio

SAP Industry &

LoB Applications &

Rapid-Deployment

Solutions (RDS)

Partner Tools &

Solutions



R-Engine R Script

R-Machine

Text-Analysis

Text Search Spatial

Processing

Business

Function Library

Predictive

Analysis Library

Core Database

RulesEngine

Automated

Predictive Library


Location

Data Machine

Data

Time-Series

Data Transaction

Data

Unstructured


Data

Data

Connectors


ESP, ASE

3rd Party Data


Application

Function

Modeler

SAP HANA

Studio



14


SAP HANA XS Engine



changes.

*

SAP Lumira

Predictive Analysis

SAP InfiniteInsight

SAP HANA Studio

SAP HANA


Solutions


Hardware-Innovations

64bit address-space – 2TB

in current servers

100 GB/s Data throughput

Prices for RAM are

decreasing

Improvememt of

Price/Performance

Multi-Core Architecture (8 x 10 core CPU per Server)

Massive parallel scaling with

many nodes

RAM-Locality improved

Row and

Column Store

Compression

Partitioning

No aggregates

neccessary

Avoid expensive

database operations

Software-Innovations

In-Memory Computing

Rethink


Attribute Views

Analytical Views

Calculation Views

Analytical Privileges

Transportable design time artifacts

stored in the repository

Sc

rip

tin

g

Main procedural language of the SAP HANA database

Push data intensive operations into the database

In script-based Calculation views and procedures

Utilize Predictive Analysis Library (PAL)

and Business Function library (BFL)

HANA Studio AFM for graphical modeling

SQL

Script

Mo

de

lin

g

Views

Open-Source, statistical functions through

R Integration leveraging predefined functions

Utilized from within procedures written in R-Lang R

SAP HANA modeling and predictive fundamentals S

AP

HA

NA

Stu

dio



SAP HANA

Studio

SAP Industry &

LoB Applications &

Rapid-Deployment

Solutions (RDS)

Partner Tools &

Solutions



R-Engine R Script

R-Machine

Text-Analysis

Text Search Spatial

Processing

Business

Function Library

Predictive

Analysis Library

Core Database

RulesEngine

Automated

Predictive Library


Location

Data Machine

Data

Time-Series

Data Transaction

Data

Unstructured


Data

Data

Connectors


ESP, ASE

3rd Party Data


Application

Function

Modeler

SAP HANA

Studio



17


SAP HANA XS Engine



changes.

*

SAP Lumira

Predictive Analysis

SAP InfiniteInsight

SAP HANA Studio

SAP HANA


Solutions


SAP HANA In-Memory Predictive Analytics Predictive Analysis Library (PAL) - Algorithms Supported

Association

Analysis Apriori

Apriori Lite

FP-Growth *

Classification

Analysis CART *

C4.5 Decision Tree

Analysis

CHAID Decision Tree

Analysis

K Nearest Neighbour

Logistic Regression

Naïve Bayes

Support Vector Machine

Regression Multiple Linear Regression

Polynomial Regression

Exponential Regression

Bi-Variate Geometric

Regression

Bi-Variate Logarithmic

Regression

Outlier Detection Inter-Quartile Range Test

(Tukey’s Test)

Variance Test

Anomaly Detection

Statistic Functions

(Univariate) Mean, Median, Variance,

Standard Deviation

Kurtosis

Skewness

Link Prediction Common Neighbors

Jaccard’s Coefficient

Adamic/Adar

Katzβ

Data Preparation Sampling

Random Distribution

Sampling *

Binning

Scaling

Partitioning

Statistic Functions

(Multivariate) Covariance Matrix

Pearson Correlations Matrix

Chi-squared Tests:

- Test of Quality of Fit

- Test of Independence

F-test (variance equal test)

Other Weighted Scores Table

Substitute Missing Values

Cluster Analysis ABC Classification

DBSCAN

K-Means

K-Medoid Clustering *

Kohonen Self Organized Maps

Agglomerate Hierarchical

Affinity Propagation

Time Series

Analysis Single Exponential Smoothing

Double Exponential Smoothing

Triple Exponential Smoothing

Forecast Smoothing

ARIMA *

Probability

Distribution Distribution Fit *

Cumulative Distribution

Function *

Quantile Function *


SAP HANA Business Function Library I

• Annual Depreciation – Calculates annual depreciation according to three

common methods: Diminishing Balance Depreciation,Straight-line

Depreciation and Sum-of-year Depreciation. It allows variable length of

timescales for all assets/items.

• Cycles – Calculates the cumulative totals in one row based on the original

numbers in another row.

• Cumulate – Calculates seasonal factors from Fourier coefficients. It

combines sine and cosine waves to help you determine seasonality or other

cyclical business factors.

• Days – Returns the number of days in each period defined by each pair of

From and To dates.

• Days Outstanding – Calculates receipts or payments based on the level of

days outstanding.

• De-cumulate –Calculates the original series starting from the cumulated

totals.

• Delay – Calculates receivables or payables based on a delay between the

time of invoice and the time of payment.

• Delay Debt – Calculates cash receipts using actual sales. The closing debtor

balance for each period is calculated by referring to historic sales levels for a

specified number of days.

• Delay Stock – Calculates purchases required to meet future demand.

• Discounted Cash Flow – Converts a future stream of cash flow to constant

prices. It calculates the inflated value of today's money.

• Driver – Calculates the forecast for future periods using historical data and

as many drivers as needed. A driver drives cost, such as headcount, floor

space, units sold, and unit price.

• Feed - Calculates the closing balance and "feeds" it to the opening balance

of the next time period.

• Feed Overflow – Calculates the closing balance and feeds it to the opening

balance of the next time period.

• Forecast – Combines actual and forecast data to produce a rolling forecast.

Eliminates scripting of feeds.

• Forecast Agents – A specialized version of the Driver function focused on the

entities required to meet service levels. Used primarily for labor in areas like

call centers and mortgage processing based on interest rate.

• Forecast Driver – A specialized version of the Driver function that calculates

the forecast for future periods using historical data and one single driver.

• Forecast Dual Driver – Calculates the forecast for future periods using

historical data and two drivers. It also calculates the incremental effect of each

driver on the historical base figure.

• Forecast Mix – Mixes actual data prior to the switchover date with forecast

data on and after the switchover date.

• Forecast Sensitivity – Returns a calculation for the proportion of requests that

will be queued because there were no agents available when the request was

answered.

• Funds – Calculates the use of funds or the source of funds.

• Future – Calculates the closing balance of an account given the start balance

and the conditions under which the account runs.

• Grow – Grows a base figure by a specified percentage each period. It can be

compound or linear.

• Inflated Cash Flow – Calculates the amount of cash you must receive in a

future period to compensate for inflation.

• Internal Rate of Return (IRR) – Calculates the internal rate of return for a

series of cash flow on specified dates.

• Lag – Calculates a result in one row by lagging an input from another row by a

specified number of periods.

• Last – Looks back over the series of data of the input row and returns the most

recent non-zero value.

• Lease – Calculates a payment schedule for a lease, loan, mortgage, annuity or

savings account.

• Lease Variable – Allows an account to be scheduled along a time scale

representing the life of the loan.


SAP HANA Business Function Library II

• Linear Average – Calculates a linear average that applies a larger

weight to more recent periods. The weights applied decrease linearly as

time goes backward.

• Max Value – Returns the maximum value of a range.

• Minimum Value – Returns the minimum value of a specific range.

• Moving Average&Moving Sum – Calculates a moving average or

moving sum over specified periods. Key statistical component

• Moving Median – Takes the median value after sorting all input values

into an ascending sequence.

• Number of Periods – Calculates the number of periods over which the

account must run.

• Net Present Value – Calculates the sum of a series of future cash flow

values after discounting each to a present value based on the annual

rate input for the period in which it is being calculated.

• Outlook – The outlook is calculated by using actuals of past months

and plan figures of future months.

• Payment – Calculates the regular payment to an account for each

period.

• Present Value – Calculates opening value through the given target

closing balance and various parameters.

• Proportion – Allows you to input a start and end date, and then

calculates the proportion of the period length. Important for project

planning with performance to plan calculations

• Rate – Calculates the percentage interest rate per period for an

account, given its start balance, end balance, payment amount per

period and the number of periods.

• Repeat – It is used to repeat data from a single period or group of

periods through the time scale of the Dimension List.

• Rounding – Calculates the rounded values for a specified input item

according to a chosen rounding method.

• Seasonal Simple&Seasonal Complex – Performs seasonal

adjustments of time to determine seasonal patterns in data.

• Seasonal Simple – Performs seasonal adjustments of time to

determine seasonal patterns in data.

• Seasonal Simulation – Provides the building blocks to seasonal

simulation seasonal data using a variety of characteristics.

• Stock Flow – Works out the level of supply needed to meet target

forecasts for stock cover.

• Stock Flow Reverse – s Allows you to input stock cover and work out

what purchases were needed to meet the target stock levels.

• Stock Flow Batch – Let’s you use batch quantities in stock flow

calculations. Key for constraint based models or non-discrete

manufacturing units of measure.

• Time – Returns the information requested by the option you have input.

Eliminates scripting of alternative time dimensions

• Time Sum – Allows you to accumulate an expense over a specified

number of periods in advance or arrears.

• Transform – Helps users to build equations using angles and

trigonometry functions when Cycles does not provide the functionality

that they need.

• Volume Driver – Calculates the year-over-year percentage difference

for each volume driver.

• Year-Over-Year Difference – Calculates the year over year difference

between the current and previous time periods.

• Year to Date – Calculates year to date totals based on original data.

• Year-to-Date Statistical – Calculates the original numbers in one row


SAP HANA In-Memory Predictive Analytics

R Integration for SAP HANA

Enables the use of the R open source environment (> 3,500 packages) in the context of the HANA in-memory database

R integration enabled via high performing parallelized connection

R script is embedded within SAP HANA SQL Script

Combine the depth and power of in-memory analytics within SAP HANA with the

breadth of R to support a variety of advanced analytic and predictive scenarios


R is Hugely Popular – and Disruptive!

What one Data Mining software package do you use most frequently?

Rexer Analytics’ Annual Data Miner Surveys: 2008-2013 (n=700)



SAP HANA

Studio

SAP Industry &

LoB Applications &

Rapid-Deployment

Solutions (RDS)

Partner Tools &

Solutions



R-Engine R Script

R-Machine

Text-Analysis

Text Search Spatial

Processing

Business

Function Library

Predictive

Analysis Library

Core Database

RulesEngine

Automated

Predictive Library


Location

Data Machine

Data

Time-Series

Data Transaction

Data

Unstructured


Data

Data

Connectors


ESP, ASE

3rd Party Data


Application

Function

Modeler

SAP HANA

Studio



23


SAP HANA XS Engine



changes.

*

SAP Lumira

Predictive Analysis

SAP InfiniteInsight

SAP HANA Studio

SAP HANA


Solutions


SAP HANA Extended Application Services (HANA XS)

Übersicht

Was ist HANA XS?

Ein leichtgewichtiger Anwendungsserver (Web Server) als

Basis für native Anwendungsentwicklung eingebettet in SAP

HANA

Ziele

Anwendung mit einer HTTP-basierten Oberfläche

(Browser, Mobile)

Einfache Systemarchitektur, da Anwendungen direkt auf

SAP HANA laufen ohne weitere externe

Serverinfrastruktur

Enge Integration mit SAP HANA Datenbank für

bestmögliche Performance und voller Zugriff auf Engines

und Bibliotheken

Große Bandbreite an Anwendungen

– Leichtgewichtige Web-Apps

– Komplexe Geschäftsanwendungen

?

!


Web Application

XS Engine

HANA Database Engine

SAP HANA Anwendungsentwicklung

XS Engine Skills

Presentation Logic

Control Flow Logic

Calculation logic Data

Front-End-Technologies

HTTP/HTTPS

HTML5 / SAPUI5

Client-Side JavaScript

Control Flow Technologies

Odata

Server-Side JavaScript

XMLA

Data Processing Technologie

SQL / SQLScript

Calculation Engine


Great user experience with SAP UI5

• Tailor-made cockpits accessible via web browser

• High-performance calculation on SAP HANA

• No additional hardware required

• Native SAP HANA solutions

SAP HANA Real time platform – Customer stories


BigPoint Games

Gaming Industry -

Predictive Game Player

Behavior Analysis

5,000 Events

Per Second loaded onto SAP HANA

(not possible before)

10 – 30% Increase in revenue per

year

Product: -

Business Challenges/ Objectives Increase conversion rates from free paying player

Increase the average revenue per paying player

Decrease churn – keep paying players playing longer

Technical Challenges Leverage real-time data processing in SAP HANA and classification algorithms with

R integration for SAP HANA to deliver personalized context-relevant offers to players

Analyze vast amounts of historical and transactional data to forecast player behavior

patterns

Benefits Real-time insights

Per player profitability analysis and increased understanding of player

behavior

Increase data volume and processing capabilities to communicate

personalized messages to players

http://www.youtube.com/watch?v=McDaSXCs5ZQ

Interactive data analysis leading to

improved design

thinking and game

planning


eBay Professional Service

(Internet)

American multinational

internet consumer-to-

consumer corporation

Product: Early Signal Detection System Powered by Predictive Analytics on SAP® HANA

Business Challenges/ Objectives

Increase ability to separate signal from noise to identify key changes to the health of

eBay’s marketplace

Improve predictability and forecast confidence of eBay’s virtual economy

Increase insights into deviations and their causes

Technical Challenges

Detect critical signals from 100 PBs of data in eBay EDW

Highly manual process because one model does not fit all the metrics hence requires

analyst intervention

Benefits

Automated signal detection system powered by predictive analytics on SAP HANA

selects best model for metrics automatically; increases accuracy of forecasts

Reliable and scalable system provides real-time insights allowing data analysts to

focus on strategic tasks

Decision tree logic and flexibility to adjust scenarios allows eBay to adapt best model

for their data

http://www.youtube.com/watch?v=hS-0ZadT6so

Determine with 100%

Accuracy that a signal is

positive at 97% confidence

Automated Early

Signal Detection system powered by SAP HANA

//upload.wikimedia.org/wikipedia/commons/1/1b/EBay_logo.svg


The Globe and Mail gains market insights


Example project Predictive model for demand forecasting of news papers


The SAP Data Science Solution

The Approach:

High-performance solution based on SAP HANA

Cutting-edge statistical algorithms for demand forecasting, tailored specifically to

the business

The Results:

Improved forecasting accuracy

Significant reduction in returns

Maintenance of sales and supply level

Decrease in forecast computation time from hours to minutes

Reduction in manual effort due to fewer forecast alerts

* The results described here were realized in a pilot project


Forecasting architecture

SAP HANA as platform for both demand forecasting and flexible

reporting

SAP HANA Database

Frontend (HTML5 Web

Browser Interface)

Sales Data

SAP Data Services

R Server (demand forecast)

Reporting/

Business

Intelligence (Predictive

Analysis/Lumira)

Administration/

Direct Data

Manipulation

(SAP HANA Studio)

Destination Systems

SAP Data Services

Advanced Analytics @ SAP Client tools



SAP HANA

Studio

SAP Industry &

LoB Applications &

Rapid-Deployment

Solutions (RDS)

Partner Tools &

Solutions



R-Engine R Script

R-Machine

Text-Analysis

Text Search Spatial

Processing

Business

Function Library

Predictive

Analysis Library

Core Database

RulesEngine

Automated

Predictive Library


Location

Data Machine

Data

Time-Series

Data Transaction

Data

Unstructured


Data

Data

Connectors


ESP, ASE

3rd Party Data


Application

Function

Modeler

SAP HANA

Studio



35


SAP HANA XS Engine



changes.

*

SAP Lumira

Predictive Analysis

SAP InfiniteInsight

SAP HANA Studio

SAP HANA


Solutions


SAP Lumira, Predictive Analysis, Infinite Insight

Discover data by building

visualizations

Synthesize and transform data the way you want it

Acquire data from corporate

& personal sources

SAP Lumira (http://www.sap.com/freelumira) SAP Predictive Analysis

Predictive

algorithms

Predictive modelling: Intelligent algorithms to

uncover and leverage patterns

Explorer

Prepare your data

Modeler

Automated model building

InfiniteInsight

Factory

Improve your models

SAP Infinite

Insight


Advanced Analytics SAP solutions for the entire spectrum of users


High Level Roadmap


2015 - Next Generation User Interface for both Business

Analysts as well as Data Scientists


SAP Predictive Analysis Modes of operation

HANA

algorithms

csv BO

universe Other

database

R & PA native

algorithms

Data Sources

Calculation on client High-performance calculation

on HANA

Reads

source data Writes results

Controls HANA

algorithms,

deploys model

Reads results


SAP Predictive Analysis & Infinite Insight

© 2014 SAP SE or an SAP affiliate company. All rights reserved.

Thank you

Contact information:

Dr. Markus Kohler

Data Scientist

Data Science solutions

SAP Deutschland AG & Co. KG

Email: [email protected]

Phone: +49 151 62345200

mailto:[email protected]

sap hana: predictive analytics in real-time · 2014-12-17 · 4 5 6? ? ? ? ? ? crm payment history...

Documents