v imp analytics appli ed
TRANSCRIPT
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Eric Siegel, Ph.D.Eric Siegel, Ph.D.Prediction Impact, Inc.Prediction Impact, [email protected]@predictionimpact.com(415) 683 - 1146(415) 683 - 1146
Predictive Analytics Applied:
Marketing and Web
Predictive Analytics Applied:Predictive Analytics Applied:
Marketing and WebMarketing and Web
Brought to you by
Prediction Impact and
World Organization of
Webmasters (WOW)
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
1
Training Program OutlineTraining Program OutlineTraining Program Outline
1. Introduction
2. How Predictive Analytics Works
3. App: Customer Retention
4. App: Targeting Ads
5. Summary and take-aways
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Eric Siegel, Ph.D.Eric Siegel, Ph.D.Prediction Impact, Inc.Prediction Impact, [email protected]@predictionimpact.com(415) 683 - 1146(415) 683 - 1146
About the speaker: Eric Siegel, Ph.D.
– President of Prediction Impact, Inc.
– Training and services in predictive analytics
– Former computer science professor atColumbia University
– Before Prediction Impact, cofounded twosoftware companies
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Business intelligence technology thatproduces a predictive score for eachcustomer or prospect
Predictive Analytics:Predictive Analytics:Predictive Analytics:
34 12 27 79 42 5934 1234 271234 79271234 42 5979271234 1234
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Optimizing Business ProcessesOptimizing Business ProcessesOptimizing Business Processes
4
“At a time whencompanies in manyindustries offer similarproducts and usecomparable technology,high-performancebusiness processes areamong the last remainingpoints of differentiation.”
- Tom Davenport
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Learn from Organizational ExperienceLearn from Learn from Organizational ExperienceOrganizational Experience
5
Data is a core strategic asset –it encodes your business’ collective experience
It is imperative to:
Learn from your data.
Learn as much as possible about your customers.
Learn how to treat each customer individually.
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Predictive Analytics:Predictive Analytics:Predictive Analytics:
6
Your organization learns
from its collective experienceCollective experience: Sales records, customer profiles, …
Learn: Discover strategic insights and business intelligence
…and puts this knowledge to action.
Discover something that makes business decisions
automatically
Discover business rules
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Eric Siegel, Ph.D.Eric Siegel, Ph.D.Prediction Impact, Inc.Prediction Impact, [email protected]@predictionimpact.com(415) 683 - 1146(415) 683 - 1146
How Predictive Analytics WorksHow Predictive Analytics WorksHow Predictive Analytics WorksBuilding and Deploying Predictive ModelsBuilding and Deploying Predictive Models
Predictive Analytics for Business, Marketing and WebPredictive Analytics for Business, Marketing and WebPredictive Analytics for Business, Marketing and Web
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Wisdom Gained: A Predictive Model is BuiltWisdom Gained: A Predictive Model is BuiltWisdom Gained: A Predictive Model is Built
8
Predictivemodel
Customerbehavior
Customercontact logs
Customerprofiles
Modeling tool
•Performed by modeling software
•Usually offline
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Applying a Model to Score a CustomerApplying a Model to Score a CustomerApplying a Model to Score a Customer
9
Predictivemodel
Customerprofile
Customerbehavior
Predictedresponse
•Often performed bymodeling software
•Possibly online
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Deploy to Take Business ActionDeploy to Take Business ActionDeploy to Take Business Action
10
•Usually not performed bymodeling software
Predictedresponse
Businesslogic
• Mail asolicitation
• Suggest across-sell option
• Retain with apromotion
Business actions,such as:
•Possibly online
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Predictive models are created automatically from your data
So, they’re generated according to your:– Product– Prediction goal– Business model– Customer base
This means customer intelligence specialized for your business– Customized business rules– Unique, proprietary mailing lists– Insights only your organization could possibly gain
Fine-Tuned Models for Your BusinessFine-Tuned Models for Your BusinessFine-Tuned Models for Your Business
11
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Predictors: Building Blocks for ModelsPredictors:Predictors: Building Blocks for Models Building Blocks for Models
12
• recency – How recent was the last purchase?
• personal income – How much to spend?
Combine predictors for better rankings:
recency + personal income
2_recency + personal income
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Training Data Is a Flat TableTraining Data Is a Flat TableTraining Data Is a Flat Table
13
Numberpurchases:
Lastpurchase: Gender: Income:
Likelyto buy:
7 shoes M high gloves
3 gloves F medium hat
1 hat M high shoes
46 gloves F low piano
This is the required form for training data; one record per customer.
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Why Not Memorize The Training Examples?Why Not Memorize The Training Examples?Why Not Memorize The Training Examples?
14
To learn from history is to remember history.
So, we could just keep a lookup table and compare newcases to old ones.
Answer: There are too many possible rows of data.
That is, each customer is unique!
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Decision Tree for Cross SellDecision Tree for Cross Sell
15
Decision
Trees•Non-linear•Specialized•Precise
Bought shoes?
Female?
Hat Gloves Shoes Piano
High Income?
If they bought shoes and they’re not female then sell gloves.If they didn’t buy shoes and they’re high income then sell shoes.
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Response Modeling for Direct MarketingResponse Modeling for Response Modeling for Direct MarketingDirect Marketing
16
Lower campaign costs and increase response rates
• Make campaigns more targeted and more selective
• Identify segments five or more times as responsive
• Achieve 80% of responses with just 40% of mailing
• Increase campaign ROI & profitability
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Predictive Models Score Each CustomerPredictive Models Score Each CustomerPredictive Models Score Each Customer
17
Name:
• Each customer is scored with a response probability
• The customers are listed in order of prediction score
• The highest-scoring customers are targeted first
No !14%T. Mitchel674
Yes "22%G. Clooney256
No !31%D. Leong528
Yes "35%E. Siegel429
Response:Score:ID number:
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
18
-700,000
-600,000
-500,000
-400,000
-300,000
-200,000
-100,000
0
100,000
200,000
300,000
400,000
Percent of Customers Contacted
Pro
fit
0% 25% 50% 100%75%
Customers ranked with predictive analytics
Customers not ranked
Cam
paig
n P
rofi
t C
urv
eC
am
paig
n P
rofi
t C
urv
eC
am
paig
n P
rofi
t C
urv
e
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Lift ChartLift ChartLift Chart
19
9.3 30.6
16.4 43.5
20.3 51.1
32.4 70.1
44.0 80.2
0
20
40
60
80
100
0 20 40 60 80 100
% C
lass
% Population
0
20
40
60
80
100
• 50,000 customers 3 times as likely to buy
• 200,000 customers 2 times as likely to buy
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Example Business RuleExample Business RuleExample Business Rule
20
New customers who come to the websiteoff organic search results, buy more than$150 on their first transaction, are male,and have an email address that ends with“.net” are three times as likely to be
return customers.
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Eric Siegel, Ph.D.Eric Siegel, Ph.D.Prediction Impact, Inc.Prediction Impact, [email protected]@predictionimpact.com(415) 683 - 1146(415) 683 - 1146
App: Customer RetentionApp: App: Customer RetentionCustomer RetentionRetain Customers by Predicting Who Will DefectRetain Customers by Predicting Who Will Defect
Predictive Analytics for Business, Marketing and WebPredictive Analytics for Business, Marketing and WebPredictive Analytics for Business, Marketing and Web
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Growth = Acquisition - DefectionGrowth = Acquisition Growth = Acquisition -- Defection Defection
22
Customer
base
Acquisition:
New customersDefection:
Lost customers
Which is the best way to
increase growth?
1. Increase acquisition
2. Decrease defection
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Increase retention by 3%
and boost growth by 12%
Increase retention by 3%Increase retention by 3%
and boost growth by 12%and boost growth by 12%
23
3%
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Predictor VariablesPredictor VariablesPredictor Variables
24
• Age of membership• External acquisition source• U.S. state of residence• Willing to receive postal mailing (opt-in)• Num days since last failed login attempt• Num days between logins
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
At-Risk SegmentAt-Risk SegmentAt-Risk Segment
25
• SOURCE_COBRAND$ == chat
• SUBSCR_AGE <= 237.819
• LOGIN_DAYSSINCELAST_F > 1.85344
• LOGIN_DAYSSINCELAST_F <= 22.6578
Churn rate: 33.5% (vs. 20% average)
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Loyal SegmentLoyal SegmentLoyal Segment
26
• STATE is one of many, including ME, NE, NH, NS, QC, SK, WY
• SOURCE_COBRAND$ == chat
• SUBSCR_AGE > 237.819
• TIME_SINCE_JOINED <= 535.456
• LOGIN_DAYSSINCELAST_F > 99.6539
• LOGIN_DAYSSINCEFIRST_F > 112.003
• LOGIN_DAYSSINCELAST_S > 131.45
Churn rate: 6.5% (vs. 20% average)
Very loyal
Small segment
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
27Cost per mailing: $25, ave profit: $100, num subscribers: 100,000
Forecasted Profit of Retention Campaign
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Loyal Segment: Online RetailLoyal Segment: Online RetailLoyal Segment: Online Retail
28
• ITEM_QTY > 2.5
• TAX_SHIP <= 8.78
• 19% will return (versus 10% average)
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Eric Siegel, Ph.D.Eric Siegel, Ph.D.Prediction Impact, Inc.Prediction Impact, [email protected]@predictionimpact.com(415) 683 - 1146(415) 683 - 1146
App: Targeting AdsApp: App: Targeting AdsTargeting Ads
PredictivelyPredictively Modeling Which Promotion Modeling Which Promotion
Each Customer Will AcceptEach Customer Will Accept
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
3030
Target content– Landing page– Featured product– Color of product displayed– Promotion or
advertisement
Based on:– Customer behavior profile
• Browsers vs. hunters
– Time of day
– Geographical location
– Pages
– Where they came from
• Ad that clicked them through
• Site they came from
• Search query they were on
• Profile, when available
Learn More From AB Testing:
“Dynamic AB Selection”
Learn More From AB Testing:
“Dynamic AB Selection”
A
B
C
?
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Dynamic AB SelectionDynamic AB SelectionDynamic AB Selection
31
Predictivemodel for B
Mary’sprofile
Trials of B
Trials of AModeling tool
Modeling tool
Predictivemodel for ATrials of A
Trials of BTrials of BTrials of B
offline online
Predictive score:
What’s the chance Mary responds to A?
Predictive score:
What’s the chance Mary responds to B?
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Selecting Between 291 Sponsored PromotionsSelecting Between 291 Sponsored PromotionsSelecting Between 291 Sponsored Promotions
32
Client: A leading student grant and scholarship search service
Sponsors include:
– Universities
– Student loans
– Military recruitment
– Other misc.
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
The Business: Great Potential GainThe Business: Great Potential GainThe Business: Great Potential Gain
33
• High bounties, up to $25 per acceptance
• High acceptance rates, up to 5%, due togeneral relevancy of the promotions
• Big Data: Wide and Long– Rich user profiles volunteered for funds eligibility
– Over 50 million training cases: 01/05 - 09/05
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
34
Prior solicitation of this program
0.00%
1.00%
2.00%
3.00%
4.00%
5.00%
6.00%
7.00%
No
Yes
Opt_in_status
0.00%
1.00%
2.00%
3.00%
4.00%
5.00%
6.00%
7.00%
Y
N
Region of current address
0.00%
1.00%
2.00%
3.00%
4.00%
5.00%
6.00%
south
northeast
west
midwest
Sority or Fraternity
0.00%
1.00%
2.00%
3.00%
4.00%
5.00%
6.00%
7.00%
8.00%
9.00%
frat
none
soro
Already seen? N/Y Email opt-in Y/N
Region: S, NE, W, M-W Fraternity/None/Sorority
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Highly Responsive Segment for "Art Institute" AdHighly Responsive Segment for "Art Institute" AdHighly Responsive Segment for "Art Institute" Ad
35
Segment definition included:
• Still in high school
• Expected college graduation date in 2008 or earlier
• Certain military interest
• Never saw this ad before
Segment probability of accepting ad: 13.5%
Average overall probability of accepting ad: 2.7%
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Highly Responsive Segment for a "Navy" AdHighly Responsive Segment for a "Navy" AdHighly Responsive Segment for a "Navy" Ad
36
Segment definition included:• Has opted in for emails• Has not seen this ad before• Is in college• SAT verb-to-math ratio is not too low, nor too high• SAT written is over 480• ACT score is over 15• No high school name specified
Segment probability of accepting ad: 2.6%
Average overall probability of accepting ad: 1.6%
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Money-Making Model:
Improved Resulting Revenue
Money-Making Model:Money-Making Model:
Improved Resulting RevenueImproved Resulting Revenue
37
A-B test deployment to compare:A. Legacy system based on acceptance rates across users
B. Model-based ad selection
Result:
25% increased acceptance rate
3.6% increased revenue observed; 5% later reported by the client
This comes to almost $1 million per year in additional revenue,given the existing $1.5 million monthly revenue.
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Eric Siegel, Ph.D.Eric Siegel, Ph.D.Prediction Impact, Inc.Prediction Impact, [email protected]@predictionimpact.com(415) 683 - 1146(415) 683 - 1146
ConclusionsConclusionsConclusionsSummary and take-Summary and take-awaysaways
Predictive Analytics for Business, Marketing and WebPredictive Analytics for Business, Marketing and WebPredictive Analytics for Business, Marketing and Web
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Predictive Analytics InitiativesPredictive Analytics InitiativesPredictive Analytics Initiatives
39
• Per-customer predictions: Unlimitedrange of business objectives
• Management: An organizational processensures predictions are actionable, drivenby business needs; multiple roles andskills
• Deployment: Mitigate risk and trackperformance
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
Predictive Analytics TechnologyPredictive Analytics TechnologyPredictive Analytics Technology
40
• Data preparation: An intensivebottleneck, critical to success
• Modeling methods and tools: No onemethod that is always best; comparemultiple methods
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
41
Prediction Impact
Predictive Analytics and Data MiningServices:
• Defining analytical goals & sourcing data• Developing predictive models• Designing and architecting solutions for model deployment• "Quick hit" proof-of-concept pilot projects
Training programs:
• Public seminars: Two days, in San Francisco, Washington DC, and other locations• On-site training options: Flexible, specialized• Instructor: Eric Siegel, Ph.D., President, 15 years of data mining, experiencedconsultant, award-winning Columbia professor• Prior attendees: Boeing, Corporate Express, Compass Bank, Hewlett-Packard, Liberty
Mutual, Merck, MITRE, Monster.com, NASA, Qwest, SAS, U.S. Census Bureau, Yahoo!
© Copyright 2006. All Rights Reserved Prediction Impact, Inc.© Copyright 2008. All Rights Reserved
If you predict it, you own it.www.PredictionImpact.com
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
42
Prediction Impact
Predictive Analytics and Data Mining
© Copyright 2006. All Rights Reserved© Copyright 2008. All Rights Reserved
If you predict it, you own it.www.PredictionImpact.com
Applications:
• Response modeling for direct marketing• Product recommendations• Dynamic content, email and ad selection• Customer retention• Strategic segmentation• Security
– Fraud discovery– Intrusion detection– Risk mitigation– Malicious user behavior identification
• Cutting-edge research forgroundbreaking data mining initiatives
Prediction Impact, Inc.
Verticals:
• Online business: Social networks,entertainment, retail, dating, job hunting
• Telecommunications
• Financial organizations
• A fortune 100 technology company
• Non-profits
• High-tech startups
• Direct marketing, catalogue retail
Prediction Impact, Inc.© Copyright 2008. All Rights Reserved.
43
Prediction Impact
Predictive Analytics and Data Mining
© Copyright 2006. All Rights Reserved© Copyright 2008. All Rights Reserved
If you predict it, you own it.
Team of several senior consultants:
• Experts in predictive modeling forbusiness and marketing
• Relevant graduate-level degrees• Communication in business terms• Complementary analytical
specialties and client verticals• Published in research journals and
industrials
Extended network of many more:
• Closely collaborating partner firms• East coast coverage
Prediction Impact, Inc.
Eric Siegel, Ph.D., President
Prediction Impact, Inc.
San Francisco, California
(415) 683 - 1146
For our bi-annual newsletter, click “subscribe”:www.PredictionImpact.com
To receive notifications of training seminars:[email protected]