reducing it service disruption through text analytics

#1 Agile Predictive Analytics Platform for Today’s Modern Analysts

©2016 RapidMiner, Inc. All rights reserved.

June, 23, 2016

Sebastian Land – Old World Computing

Reducing IT Service Disruption Through Text Analytics

©2016 RapidMiner, Inc. All rights reserved. - 2 -©2016 RapidMiner, Inc. All rights reserved. - 2 -

Today’s Agenda

• Brief Company Overview• Problem Introduction• State Of The Art Approach• To Boldly Go Beyond State Of The Art

©2016 RapidMiner, Inc. All rights reserved. - 3 -

Leader

2016, 2015 & 2014

Gartner Magic Quadrant for Advanced Analytics Platforms

Strong Performer

2015

Forrester Wave on Big Data Predictive Analytics

Innovation Winner

2015Wisdom of Crowds for

Advanced & Predictive Analytics, Big Data Analytics &

End-User Data Preparation

#1 Open-Source Platform

2015, 2014, 2013

Data Mining & Analytics Software Poll

RapidMiner is #1 OPEN SOURCE


RapidMiner ACCELERATES Time-to-Value

DATA PREP Speed & optimize ALL data

exploration, blending & cleansing tasks

OPERATIONALIZEEasily deploy & maintain

models and embed analytic results

MODEL & VALIDATERapidly prototype and

confidently validate predictive models

DATA PREP Speed & optimize ALL data

exploration, blending & cleansing tasks

CONNECT TO ANY DATA SOURCE, ANY

FORMAT, AT ANY SCALE

SUPPORT FOR ALL MAJOR BI, DATA VISUALIZATION &

BUSINESS APPLICATIONS

- 5 -©2016 RapidMiner, Inc. All rights reserved.

About Old World Computing

• Small expert company in the area of data science• Establishing data science in a company is a chicken-egg

problem: You need experience to setup a project in a way to make it

successful You need a project to get experience You need to know who to turn to for information

We help with ten years of field experience

- 6 -©2016 RapidMiner, Inc. All rights reserved.

Service Offering Strategic Consulting

– Establish Data Science in your company– Avoid expensive pitfalls

On-site Training– Establish in-house expertise– Learn from tested best-practices

Solution Development– Minimize time to deployment– Joint development for knowledge transfer


Challenge: Problem Description

• Huge and complex infrastructure• Single elements of infrastructure fail from time-to-time• Some failures may directly affect quality of service

– But usually only in combination with others• All elements generate log files revealing failures• We want to:

– Detect when quality of service is affected– Find the cause

• But analysis is not trivial


Challenge: Log Files12:13:30,026 INFO [com.rapidminer.Process] (QuartzScheduler_Worker-9) No filename given for result file, using12:13:30,026 INFO [com.rapidminer.Process] (QuartzScheduler_Worker-9) Loading initial data (starting at port12:13:30,027 INFO [com.rapidminer.Process] (QuartzScheduler_Worker-9) Process //_LOCAL/projects/End to End12:13:30,089 INFO [com.rapidminer.Process] (QuartzScheduler_Worker-9) Process //_LOCAL/projects/End to End12:13:31,649 INFO [com.rapidminer.Process] (QuartzScheduler_Worker-9) No filename given for result file, using12:13:31,649 INFO [com.rapidminer.Process] (QuartzScheduler_Worker-9) Loading initial data (starting at port12:13:31,650 INFO [com.rapidminer.Process] (QuartzScheduler_Worker-9) Process //_LOCAL/projects/End to End12:13:31,655 WARNING [com.rapidminer] (QuartzScheduler_Worker-9) Error occurred and will be neglected by Handle Exception: The 'Retrieve' operator in the process executed by 'Execute Apply Normality Models on Now' failed with: Cannot retrieve repository data from entry '../../results/Train Help Desk Normality Models by Source System/VUMS/Regression Model'. Reason: Entry '//_LOCAL/projects/End to End Baselining/results/Train Help Desk Normality Models by Source System/VUMS/Regression Model' does not exist..: com.rapidminer.operator.error.ProcessExecutionUserErrorError: The 'Retrieve' operator in the process executed

Server-001

Server-002

...

t


Current Approach: Use Severity Levels

• Logs contain severity levels• One can aggregate over time and

see whether numbers grow unexpectantly

• Several shortcomings:– Severity set by developer for a

single element– Unlikely that single element

affects QoS– Sheer number of failures that are

normal hide important events


Idea: Use Machine Learning

• Machine Learning can reveal most complex patterns in data

• BUT: Also machines need to learn from something


Idea: Use Machine Learning

• Machine Learning can reveal most complex patterns in data

• BUT: Also machines need to learn from something

• Hand-tagging logs– Not possible: We simply don‘t

know the dependencies

okay error


Solution: Use Customer Feedback

• Once the QoS is affected customer will complain about it

• The complaint will be logged with the time

• We can estimate how long it takes the customer to complain and to notice the error

• Error occurs in this time frame• But not in a control frame a week

ago!• Difference unique entries are the

cause!com

plai

ntpo

ssib

leer

rors

all o

kay








cause!

control candidate

com

plai

ntpo

ssib

leer

rors

all o

kay








cause!

©2016 RapidMiner, Inc. All rights reserved. - 15 -- 15 -©2016 RapidMiner, Inc. All rights reserved.

DEMO


Review Assumptions

1) We assumed that customers will always complain– Some might just not care

2) We assumed that each failure will be noticed– Down-time during the night is unlikely to be detected

3) Hence, comparison group probably contains some failures(And we didn’t use any machine learning, yet)


Iterative Process

• Concentration of failure related entries higher in candidates– Hence machine learning will find

some related pattern

poss

ible

erro

rs

trueerrors


Iterative Process


some related pattern

cand

idat

esco

ntro

l


Iterative Process


some related pattern• We score the entire entries

predictederrors


Iterative Process


some related pattern• We score the entire entries • And mark the ones with highest

confidence as new candidates

cand

idat

esco

ntro

l


Iterative Process



confidence as new candidates• Iterate until results are stable

predictederrors


Iterative Process




cand

idat

esco

ntro

l


Iterative Process





Result

unkown truth predicted truth

- 26 -©2016 RapidMiner, Inc. All rights reserved.©2016 RapidMiner, Inc. All rights reserved.

Thank you

Email:Phone:

Web:Twitter:

Q & [email protected]+49 234 794-77-479https://oldworldcomputing.com@stiefelolm

reducing it service disruption through text analytics

Technology