predictive analytics: big data lessons from big physics

@masteringsap #BAT15 @theeventfulgroup #BAT15@masteringsap #BAT15

Jake BoumaBritehouse

Predictive analysis: big data lessons from big physics

@masteringsap #BAT15 @theeventfulgroup #BAT15

Predictive analysis: big data lessons from big physics

2


What I’ll Cover• Big data and predictive analysis• “What is stuff made of”, a journey of scientific models• Big physics – lessons learned from the CERN LHC• Elusive data science• A tough benchmark: Weighing SAP’s offering up against big

science

3

Look out for lessons learned…

@masteringsap #BAT15 @theeventfulgroup #BAT15@theeventfulgroup #BAT15@masteringsap #BAT15

Let’s get started…From big data to predictive analysis

4


Big DATA

VOLUME

VARIETY

VELOCITY

variability

veracity

ANAL

YZE

STORE

FILTER

TRANS-FORM

EXTRACT/ REPLICATE

DECISION MAKING

value

VERY

BIG

CH

ALLE

NGE

??


Predictive Analytics – Crash Course

6

PREDICTIVEMODEL

ATTRIBUTESageheightgender…

TARGET VARIABLE

credit risk?behaviour

1. Choose a model

?

INPUTS TARGET

2. Train (fit) the model with data

Goal: Predict the target variable

…what does this look like?


Simplest example

m, c?

INPU

T

TARG

ET

INPUT

TARG

ET

x

x

x

xx

x x

x

x x

x xxx

x

x x

x

DATASET INPU

TTARGET

TRAIN

APPLY


Touch more complex

INPUT 1

TARG

ET

INPUT 2

β1, β2, ε?

INPU

T1

TARG

ET

INPU

T2

INPU

T1

TARGET

INPU

T2

x

xx

xx x

xx

x xxx

x

x

x

DATASET

x

xx

TRAIN

APPLY


Why stop there…

?

?

?

INPUT TARGET?

?

?

?

?

neural networks and beyond



10

PREDICTIVEMODEL


TARGET VARIABLE


1. Choose a model

?

2. Train (fit) the model with data

INPUTS TARGET

3. Apply the model…

Goal: Predict the target variablePREDICTIVE

MODEL



11


TARGET VARIABLE


DECISION MODEL

4. Analyze success

5. Refine model & repeat

PROFITABLE ACTION

PREDICTIVE MODEL

TARGET $ %


Predictive Analytics – Why?

12


PREDICTIVE MODEL

TARGET VARIABLE


Profit is made in BEATING RANDOM

TARGET DECISION MODEL

PROFITABLE ACTION


INCOMING TRANSACTION

A more detailed example

PREDICTIVE MODEL

Transaction ID Location At_restaurant Customer Residestime since last

transaction was_fraudulent7000939500 Seapoint FALSE Bob Bryanston 1 h TRUE7000939499 Bryanston TRUE Bob Bryanston 25 h FALSE

… 0000000002 Krugersdorp FALSE Larry Krugersdorp 3.1 h FALSE0000000001 Parktown TRUE Christine Greenside 1.6 h TRUE

Entity: Credit card transactionsGoal: Detect probable fraud and request stricter verification

DECISION MODEL

PROFITABLE ACTION

$ %

PREDICTIVE MODEL

PROBABILITY OF FRAUD


A more detailed example

INCOMING TRANSACTION

PREDICTIVE MODEL

PROBABILITY OF FRAUD

Transaction ID Location At_restaurant Customer Residestime since last

transaction was_fraudulent7000939500 Seapoint FALSE Bob Bryanston 1 h TRUE7000939499 Bryanston TRUE Bob Bryanston 25 h FALSE

… 0000000002 Krugersdorp FALSE Larry Krugersdorp 3.1 h FALSE0000000001 Parktown TRUE Christine Greenside 1.6 h TRUE

What about timescales?

DECISION MODEL

PROFITABLE ACTION

REAL TIME

DAYS

REAL TIME

DAYS $ %

@masteringsap #BAT15 @theeventfulgroup #BAT15@theeventfulgroup #BAT15@masteringsap #BAT15 15

Lessons in modelling from big scienceA search for a model


Lesson: Be OK with a vague goal

16

What is matter (stuff in general) made of?

?


Lesson: Start simple


Lesson: Improve your model with understanding

ATOM

NUCLEUS

QUARKS bound by GLUONS


EXPERIMENT

Lesson: Iterative development

19

THEORY

“PREDICTIVE MODEL”

Accelerators & collisions Searching for a better model


Theory takes the lead

20

EXPERIMENT THEORYThe

standard model

Accelerators & collisionsWe’ve

got work to do…

Searching for a better model


Lesson: Eventually you will have to go BIG

smaller scale

more energy

bigger acceleratorAccelerators & collisions


The Large Hadron Collider at CERN

27 km underground tunnel

1232 steering + 392 focusing superconducting magnets

Liquid He cooling to temperatures lower than space

Ultra-high vacuum 10x higher than the surface of the moon

Packets of particles colliding at 40 MHz


Visualizing the particles

150 million sensors


LHC big data from the source

40 MHz

COLLISION EVENTS

15 Pb/yr

TIER 0 on premise facility

Discard uninteresting data ASAP

reaction products

150M sensors


Sharing LHC big data


Invention the web as we know it?

TCP/IP WWW


The grid today

Decreasing volume

Increasing data preparednessIncreasing dem

andTIER

0

TIER1

TIER2

TIER3

CERN itself

13 Data Centers around the world

155 Universities and research institutions

Cluster Computers and local pc’s serving individuals


The hunt goes on

So what can we learn from this journey?


Maybe we’ve found the SCIENCE in data science

“PREDICTIVE MODEL”

Big science is comfortable with the V’s of big data

@masteringsap #BAT15 @theeventfulgroup #BAT15@theeventfulgroup #BAT15@masteringsap #BAT15

Back to businessSAP benchmarked against big science


Back to business

34

Understand the importance of data

exploration

Data extraction / replication

Serving the data to analysts and enabling

cluster execution

Instant models!?IT WORKS.

ESP, SLT SAP Infinite Insights


Big Points to Take Home• V for Very big challenge• Big science can provide much needed skills, technology and peace

of mind• Be very happy with SAP’s plans for the future• Predictive modelling isn’t so scary -- start small, but start planning

to buy InfiniteInsights

35

@masteringsap #BAT15 @theeventfulgroup #BAT15@masteringsap #BAT15 @theeventfulgroup #BAT15 36


[email protected]

za.linkedin.com/jaketbouma/

@jaketbouma

37

[email protected]

80%PREDICTIVE MODEL

Probably walking out of here with a HANA on his back


The end

38

…bonus slides for questions


Difficult origins of big DATA“It’s a lot easier to talk big data than to do big data”

Consumers are data-ready

Concept of big data is pushed hard in marketing

circles

PROGRESS

General Acceptance

Technical Solutions to big data

The change in approach is drastic

Acceptance is widespread without understanding the challenge

Enterprise customers are unconvinced with vendors’ offerings

Specificity Location Structure Analytics

Can science help us catch up?


Nuclear landscape

predictive analytics: big data lessons from big physics

Data & Analytics