techconnex think beyond bi: machine learning

enabling modern enterprise

1

Think Beyond Business Intelligence for your organization – Machine Learning

Albert Hui, MBA, M.A.Sc., CSMData Economist

05/03/2023 2enabling modern enterprise

About Me

2

• Co Founder of Data Economist, a data consulting firm based in Toronto.

• 18 years in data management consulting, business Intelligence, data warehousing and big data.

• Master in Engineering in the area of Artificial Intelligence • Big Data Architect, Data Scientist• Conference Speaker at IOUG, TOUG Collaborate since 2011• Technical editor on Oracle 12c Book. • M.A.Sc., MBA, University of Toronto• Toronto based • Twitter: @dataeconomist• Father of two twin boys


Agenda3

Objective of this Session

What is machine learning?

BI vs ML

Models and Tools

Use Cases

What’s for the Organization?

Concluding Thoughts


Objective of this Talk4

Introduction of Machine Learning

5

History of Data &Machine Learning


The Meaning of.....6

Data: Granular and raw and viewed as the

lowest level of abstraction from which information and knowledge are derived.

Information: Extracting data in order to effectively

derive value and meaning and establishing a relevant context, often selecting from many possible contexts.


The Meaning of.....7

Intelligence: Individuals differ from one another in their ability to understand complex ideas, to adapt effectively to the environment, to learn from experience.

Wisdom:A deep understanding of people, things, events or situations, resulting in the ability to choose or act to consistently produce the optimum results with a minimum of time and energy.


8

Gandalf

Legolas


Build Knowledge thru Experience

9

10

What is Machine Learning?


Definition11

Computer systems that automatically learn or improve with experience (data).


What is machine learning?12

Machine Learning is not new


Common Machine Learning Applications13


Machine learning: two major types14

Supervised Supervised learning is tasked with learning a function from

labelled training data in order to predict the value of any valid input. Common examples of supervised learning include classifying e-mail messages as spam, labelling Web pages according to their genre, and recognizing handwriting. Many algorithms are used to create supervised learners, the most common being neural networks, Support Vector Machines (SVMs), and Naive Bayes classifiers.

Unsupervised Unsupervised learning, is tasked with making sense of data

without any examples of what is correct or incorrect. It is most commonly used for clustering similar input into logical groups. It also can be used to reduce the number of dimensions in a data set in order to focus on only the most useful attributes, or to detect trends. Common approaches to unsupervised learning include k-Means, hierarchical clustering, and self-organizing maps


Data mining vs machine learning15

Data Mining Focus on extracting patterns, unknown properties on the data. Marketing Surveillance Fraud Detection science discovery Discover items usually purchased together

Machine learning Focus on extracting prediction models, based on known

properties learned from the training data E-Mail spam classification News-topic discovery Building recommendations


16

“I skate to where the puckIs going to be,not where it has been”

-- Wayne Gretzky

17

Let’s try it ourselves


Quick Quiz18

In US, a 45year male, Around 150-180K income, Post Graduate Education, if he wants to buy a car. Which brand?


Quick Quiz19

In US, a 45year male, 3 children, 180K income, Graduate School Education, if he wants to buy a car. And he lives in Texas, then which brand?


Quick Quiz20

In US, a 45year male, 3 children, Graduate School Education, making $60K/year. If he wants to buy a car. Then which brand?

21

BI vs. ML


Remember this?22


23

Make a decision


BI Reports ….24


Key factors of Successful Enterprises25


26


Learning Enterprise27


28


Decide – Learn - Experience29

Make better Builds


How do we keep up changes?30

When the winds of change blow,some people build walls and others build windmills

-- Chinese proverb


31

32

Models & Tools


Main Models33

Classification Regression

Clustering


Machine Learning Components34


Machine Learning : Data35


Modelling Concept (Dimensionality Reduction)

SeasonsWeather

Sal

es

Project three dimensional space into two dimensional space

Principal Component Analysis


Modelling Concept (generalization) 37

x

y Is it a better model?

31

52

54

4

332

210 xxbxbxxbxbxbby

)min(0

ii

n

i

b

Objective Function

Regularization


Build the model38

Learn a model from a manually trained dataset Predict the class of an unseen object based on features


Build the model: iterative process39

There is no single answer/model to your questions. It is often based on trial and error.


Some Models Available40

Multiple Variable Regression Decision Tree/Random Forrest Logistic Regression Neural Network Fuzzy Logic Support Vector Machine Bayesian Network K-means KNN – Knearest Neigbors


Tools 41

42

Use Case #1

Ad Click-thru rate Prediction using Machine Learning


43


Clickstream – Case Demo44

An Asia based Hotspot Wi-Fi provider Revenue Model: Advertising

Advertisers place ads before users can connect to Wifi

Data Survey data: Users are required to fill a survey before

logging in. Click logs including Ad click-through

Data Size: 12GB+ compressed a day. 15M signed on and 6% click-thru a day.

Problem definition: click-through rate is too low

Demo


Preference Matching : Clustering 45

Matching

Millions of People Thousands of Ads


Logistic Regression46


Ad: n-dimensional vectors Each Ad is represented as a vector of

customers who have clicked the ads. Probability of Ad clicked based on

how close with individual Ad vectors.Vector for customers who clicked and viewed burger king

Vector for customers who clicked and viewed wholefood

Vector for an signed on customer


Improvement48

Click-thru rate increased from 6% to 13% first year and 19% next

year.

49

Use Case #2

A Major Canadian retailer


Data Science – First Question50

What are the customers saying?


Listen to Customers51

install.packages("twitteR")

rdmTweets <- userTimeline("rdatamining", n=100)

install.packages("ROAuth")install.packages("RCurl")install.packages("tm")install.packages("SnowballC")install.packages("wordcloud")twitCred <- OAuthFactory$new(consumerKey="rp4fHagxYdsdfoSMR3FiOg", consumerSecret="webKiy93XZZDLQcBJzxHw64regfgZvbivJIpLctYcPKY", requestURL="https://api.twitter.com/oauth/request_token", authURL="https://api.twitter.com/oauth/authorize", accessURL="https://api.twitter.com/oauth/access_token")

wordcloud(d$word, d$freq, min.freq=3)


NPL : Sentiment Analysis 52

weather

waiting

Olympics

05/03/2023 53enabling modern enterprise53

Words PopularityOlympics 0.9Winter 0.9Customer Service 0.85Computer 0.85Annoy 0.85Auto Oil 0.7horrible 0.7line 0.7wait 0.7Found 0.65traffic 0.6walmart 0.6good 0.6back 0.57Target 0.56Parking 0.56sick 0.55

Association AssociationGame thanks

Olympics weatherwait productwait slowAsile wait

Engine pricewait stand

standcustomer Service

time findline stand

target weatherphone find

Olympics familyworst goodfind parking

target findservice slow

dictCorpus <- myCorpus# stem words in a text document with the snowball stemmers, # which requires packages Snowball, RWeka, rJava, RWekajarsmyCorpus <- tm_map(myCorpus, stemDocument)# inspect the first three ``documents"inspect(myCorpus[1:3])

myCorpus <- tm_map(myCorpus, stemCompletion, dictionary=dictCorpus)

inspect(myCorpus[1:3])

myDtm <- TermDocumentMatrix(myCorpus, control = list(minWordLength = 1))

#inspect(myDtm[266:270,31:40])print(myDtm)

findFreqTerms(myDtm, lowfreq=2)

findAssocs(myDtm, 'Olympics', 0.10)findAssocs(myDtm, 'service', 0.10)

NPL : Sentiment Analysis


Improvements54

Improve the customer service line

Increase Customer Service Staff on Sat.

Reduce Wait Time


Data Science – What Impact Sales55

Product

Sales

Weather

StoreDemograp

hic

Product Review

Inventory

On-line Clickstrea

m

Competitors

Store Size &

Attributes

Product Price

Environmental Data

Environic Data

Site, ForumReview, twitter

Store Inventory

Site Clickstream

Marketing Analysis

DealerManagement

Promotion


Data Science – What Impact Sales56

Multivariate Linear Regression

57

Use Case # 3

Cash flow projectionfor a major bank


Deposit Saving : Asset Liability Modeling 58

Asset Allocation Optimization

1. Predicting cash balance for customer segments.

2. Optimizing lending and asset allocation.

3. Minimize liquidity risk.4. Enhance pricing strategy.5. Manage better customer

relationship.

Every 10K deposit

10d 20d 1m …... 6m 1y 2 y

Survival modelfor Cash flow projectionbased on customerprofile

Retirees 65+

Single Saver

40 with kids

Single spender


Use Case #3: Benefits59

Estimated 100+ Millions per year revenue opportunity

60

What’s for your organization?


Enhance your BI Reports with prediction capability

61

Provide predictive capability in your BI Reports.


Decision Architecture62

models

businessapplications

BI Reports

Feedback


Decision as a Service 63

Make betterBuilds


Concluding Thoughts:64

In this connected age, what is the most

disruptive?


Customers are actually more disruptive than technology


Decisions Made by IoT

To create the experience, decisions need to be made where the events happen.


Thank you!67

Albert Hui, MBA, MASc., P.Eng, CSM

Data Economist Inc.,Email: [email protected] me at Twitter: @dataeconomist

mailto:[email protected]

techconnex think beyond bi: machine learning

Data & Analytics