database marketing
DESCRIPTION
Database Marketing. Dr. Ron Rymon Marketing Communications Program IDC, Herzliya. Overview. Goal: describe the framework, and touch on the current trends and buzzwords Outline: Uses of the marketing database The Data Implementation technologies Analysis techniques Modeling techniques. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/1.jpg)
Database Marketing
Dr. Ron Rymon
Marketing Communications Program
IDC, Herzliya
![Page 2: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/2.jpg)
Overview
Goal: describe the framework, and touch on the current trends and buzzwords
Outline: Uses of the marketing database The Data Implementation technologies Analysis techniques Modeling techniques
![Page 3: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/3.jpg)
Uses of the Marketing Database
![Page 4: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/4.jpg)
The Marketing Database
• Comprehensive collection of interrelated data ...
• Arranged around each customer ...
• Allow timely and accurate retrieval ...
• Support analytical, predictive, operational needs ...
• Serving multiple applications …
![Page 5: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/5.jpg)
Active Database:An Integrated Business Resource
Marketing
Distribution
CustomerService
SalesResearch
Finance
Database
![Page 6: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/6.jpg)
Information is Power:Active databases drive the business…
• Identify your best customers– profitability analysis, clustering
• Develop new customers and cross-sell– similar to current, identify competitors’ customers
• Improve delivery of sales promotion– response modeling / targeting
• Personalize message– based on purchase patterns, volume
• Use as a research tool across the organization– customer, product, market research
![Page 7: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/7.jpg)
Key Building Blocks
• Data– a database is only as powerful as its data
• Implementation technologies– hardware, networking, warehouses
• Analysis techniques– RFM, LTV, OLAP, Segmentation, Visualization
• Modeling– Regressions, Artificial Intelligence, Data Mining
![Page 8: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/8.jpg)
The Data
A database is only as powerful as the data it houses.
![Page 9: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/9.jpg)
• Every database is a collection of records• Each record is a collection of fields• Here, one record per customer and/or prospect
– unique identifier– general customer/account information
• including demographic, psychographic, socio-economic
– our offers+communications to the customer– customer’s actions: response, purchase, payment
Customer-centric Database
![Page 10: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/10.jpg)
What Data To Hold
• Too often, data is collected based on availability, and not based on projected need
• Should accumulate internally– data that can be used to support current and future
strategies (mktg and otherwise, e.g., operations)
– …. data that may be valuable to other organizations
• Should source external data– unavailable internally
– too expensive to maintain/update
![Page 11: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/11.jpg)
What’s in YOUR database?
Data Requirements
Basic Demo Usage Payment
Targeting
Retention
Mar
keti
ng +
oth
erA
ppli
cati
ons
Usageupgrading
![Page 12: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/12.jpg)
Data Sources : Internal
• Operations / Sales– past usage/purchase, e.g., amount, variability
• Finance– payments, e.g., timeliness, amounts
• Customer service– e.g., inquiries, complaints
• Other data collection methods:– sales/orders, promotions/drawings, inquiries, surveys,
warranty cards, research panels...
![Page 13: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/13.jpg)
Data Sources: Distribution Channels
• Many companies use distributors, retailers
• Problem: lack of direct communications with end-customer, no “relationship”
• Part-solution 1: keep tabs on channel + aggregate statistics on customers
• More aggressive solution: special marketing programs to reach customers
![Page 14: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/14.jpg)
Data Sources : External Lists
• 50% of U.S. DMers sell their lists
• Use to enlarge universe : new names
– can buy segments by specific features (model)
• Enhance data : cross information
– U.S. census data
– Credit bureau
– Various marketers of related products
– List compilers / maintainers / sellers
John Smith 60 10.2
Eric Cohen 35 1.3
Jack Marshal 20 0
enhance data
enlargeuniverse
![Page 15: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/15.jpg)
Other Data Sources
• Mass-advertised offers– TV call-ins, direct response
• Joint offers with other merchants– take-one brochures in banks, restaurants– drawings
• Trade shows, happenings, community activity
• Referrals!
![Page 16: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/16.jpg)
Data Management
• Many sources:– conversions, transformations, cleaning, merge-purge
• Many “clients”– marketing, sales, product managers, operations
• Temporal issues– updates, audits, archives/deltas
• Quantities: huge databases– Storage, access, processing, communications
• Resolution, Enhancement
![Page 17: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/17.jpg)
Merge-Purge : Example
• Palmer, Robert and Mary, 123 Sun Avenue, Apt 7, Key West, FL 31250
• Dr. Robert C. Palmer, Custom Engineers LLC, 123 Sun Avenue, 7th Floor, Key West, FL 31250
• Rob Charles Palmer, CE Inc., 123 Sun Ave #7, Key West FL 31250
• Bob Palmer Jr., 123 Sun #7, Key West, FL 31252• Maria Palmer, 123 Sun Avenue, Suite 7, Key
West, FL 31250
![Page 18: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/18.jpg)
Other Issues
• Legal– Privacy Act– Anti-discrimination– Advertising Code– Telephone Consumer Protection Act
• Consumer groups– Right to be omitted (just write to DMA)– Environmental issues
• DMA invests in education:– Dmers: best practice– Customers: better image
![Page 19: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/19.jpg)
Implementation Technologies
![Page 20: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/20.jpg)
Computing Platforms
• Issues / Needs:– Information sources + integration
– Storage/access, maintenance, completeness, update
– Computation: process queries, algorithms
– Analyses and reports, feed to operations, customer/user interaction
• Trends:– Traditionally, all DMers used mainframes
– Today, some migration to mid-range (UNIX)
– PC-based computers gaining power (NT)
– Client/Server architectures
– Everything networked
![Page 21: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/21.jpg)
Applications
• Database is a foundational software
• Must support variety of applications:– transaction processing
– analyses
– on-line interaction
• Trends:– Relational databases
– Data warehouses
– Data redundancy/multiplicity
Database
Database Management
O.S.
![Page 22: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/22.jpg)
Relational Database
ID Cust Name Address …1234 John Brown 123 Main St ….
ID Date Cust Product Quant. Price98765 3.5.98 1234 A703 5 150.0098766 4.5.98 1234 A707 2 240.0098767 4.5.98 1235 A703 1 30.00
Transactions for John Brown
3.5.98 5 Levis Jeans $1504.5.98 2 CK Jacket $120
Purchases of Levis Jeans
3.5.98 John Brown 5 $1504.5.98 Jane Doe 1 $30
ID Product Supplier …..A703 Levis Jeans S7003 ….
Tables
Reports (SQL Queries)
![Page 23: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/23.jpg)
Data Warehouse
• Stores data for informational and analytical processing
– Separate from operations
– Subject-oriented
– Integrated
– Historical
Operational Data Warehouse
loans
credit card
savings customer
product
investments
![Page 24: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/24.jpg)
Example: Computer-by-Mail Inc.
House Files
TelemarketersClient --- Server
operations
Analyst
MktgExecutive
DataWarehouse
![Page 25: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/25.jpg)
Analysis Techniques
![Page 26: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/26.jpg)
Data Limitations
• Important: The data is a limited encoding of reality
• Many potholes:– Omission
– Errors, noise
– Representation
– Sampling bias
• Cannot be too careful !
![Page 27: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/27.jpg)
Exploratory data analysis :Single-variable
• Descriptive statistics– Mean, Median– Variance
• Histograms– Shows distribution
0%
5%
10%
15%
20%
25%
30%
35%
40%
0-20 20-30 30-40 40+
![Page 28: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/28.jpg)
Exploratory data analysis:Multi-variable
• Examine relationship between two or more variables– Cross tabs
– Correlation
– Scatter plots
– Clustered histograms
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
0-20 20-30 30-40 40+
BuyerNoBuy
![Page 29: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/29.jpg)
RFM Analysis
• RFM score– Recency: how close is the last purchase– Frequency: number of recent purchases– Monetary: dollars spent recently
• Example:– Recency: 10 pts if within 3 mos, 1 pt lower per additional
month, to 1– Frequency: 1 pt for each purchase within 12 months– Monetary: 1pt for each $100 in past year, to 10– Score=R*F*M, the higher the better
![Page 30: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/30.jpg)
Life-Time Value of Customers
• LTV Goal:– recognize each customer’s contribution
• Method:– calculate the “expected” net revenue
– discounted:• risk of attrition
• probability of sales
• rate of money
• Typically computed per 1000, if possible by segment
![Page 31: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/31.jpg)
OLAP Tools
• OLAP : On-Line Analytical Processing
• Goal: A database-driven system that provides– Fast
– Analysis• common business reports, statistics
– of Shared• same information available to many users
– Multi-dimensional• every piece of information is multiply categorized
– Information“The OLAP Report”
![Page 32: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/32.jpg)
OLAP Tools
• Data represented internally as multi-dimensional cube
– e.g., customer’s attributes, purchases, payments, etc.
• User chooses presenting two dimensions at a time
– e.g., show $-sales, by geographic region and income
• Heavy use of hierarchical variables, with drilling capabilities:
– time: year, quarter, month, week, day, hour
– product: hardware, printers, small printers, PX-1000
– dollars: by ranges 0-1000, 1000-5000, 5000-25000, etc.
• Analyses, highlights of interesting cases, etc.
![Page 33: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/33.jpg)
OLAP Tools: Example Screen
![Page 34: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/34.jpg)
Data Visualization Tools
• Many relationships are best communicated visually:– histograms, pie-charts, scatter plots, graphs
– use color/texture, shapes
– temporal animations
• Visualization software allows– single-variable over time
– one variable as a function of another
– interaction detection
– segmentation
![Page 35: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/35.jpg)
Modeling Techniques
![Page 36: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/36.jpg)
Modeling Behavior
• Target variable– a.k.a. dependent/modeled/explained variable
– typically, whether bought/responded or not
• Goal:– Use other variables in a model to classify/predict
– other variables: a.k.a. independent, observable, explaining
– model: formula, algorithm
• Success criterion: future performance
![Page 37: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/37.jpg)
Modeling and Validation Framework
• Data flow:– Historical data
– Modeling software
– Constructs model
– Tested on more historical data
– Repeat until satisfied
– Use model to predict
Training SetModel
Productionor Test Set
Output
![Page 38: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/38.jpg)
Critical Success Factors
• Choice of data– scope: same/similar period, audience, offer,
communication
– explaining variables: available, useful, well-represented
• Choice of modeling technique– appropriate for the goal
– powerful: good fitting power
• Careful and “pessimistic” testing and validation
![Page 39: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/39.jpg)
Validation
• NEVER test on same data set– avoid “memorizing” the data, overfitting
• Out-of-sample methods– separate training set and test set– cross-validation, a.k.a. jack-knifing– remember temporal aspect
• Evaluate the model’s robustness– estimate chance probability, bootstrapping
![Page 40: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/40.jpg)
Classification v. Prediction Systems
• Classification systems:– distinguish few types of customers, e.g., responded or not– technically, target variable is discrete/categorical– validation through “hit rate”
• Prediction systems– predict probability of purchase, or purchase dollars– technically, target variable is continuous– validation through “closeness” measures
![Page 41: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/41.jpg)
Linear Scoring Systems
• Use linear regression
• Coefficients evaluated using historical data
• Higher score interpreted as greater likelihood of responding
• Every coefficient measures “independent” contribution
• Classification variant: discriminant analysis– e.g., predict response if score is above 0.3
dcba AgesePastPurchaIncomeScore
![Page 42: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/42.jpg)
Logistic Regressions
• Logistic regression (logit)
• Target variable– historical data: 0 or 1
– future application: used as probability
• Independent variables: continuous or categorical• Probit: variation that relies on normal distribution
dcba
dcba
1
AgesePastPurchaIncome
AgesePastPurchaIncome
e
eyProbabilit
![Page 43: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/43.jpg)
Presenting and Evaluating Results
• Lift table
Top Scoring % Respond %Non-Respond5% 26.8% 4.9%
10% 41.2% 9.8%15% 52.4% 14.8%20% 62.6% 19.8%
50% 87.9% 49.7%75% 96.5% 74.8%
![Page 44: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/44.jpg)
Presenting and Evaluating Results
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
% Non-Responders
%R
es
po
nd
ers
• Lift curve (a.k.a. Receiver Operating Characteristic curve)
![Page 45: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/45.jpg)
Presenting and Evaluating Results
• Confusion Matrix (given a specific threshold)
• Accuracy=(Pr,r+Pn,n)/Total• Detection=Pr,r/(Pr,r+Pr,n)• Two error types: Pr,n and Pn,r
PredictedRespond
PredictedNon-Respond
Actual Respond Pr,r Pn,r
ActualNon-Respond
Pr,n Pn,n
![Page 46: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/46.jpg)
Break-even Analysis
• Issue: how much to mail?• Solution: find break-even point• e.g.:
• Caution: use held-out data!
%mail %respond Income Cost ofproduct
Cost ofmail
MarginalProfit
5% 25% $1,250 $750 $50 $45010% 21% $1,050 $630 $50 $37015% 16% $800 $480 $50 $27020% 11% $550 $330 $50 $17025% 6% $300 $180 $50 $7030% 3% $150 $90 $50 $1035% 2% $100 $60 $50 -$10
![Page 47: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/47.jpg)
Non-Linear Systems
• In regressions, a change in one independent variables always affects in same direction– e.g., if age affects positively, then the older the better, always
• One solution: transformations– e.g., if U-shaped relation, use quadratic form
• Or, use non-linear techniques:– Neural networks
– Decision trees
– Other: rule-based systems, genetic algorithms, Bayesian nets
![Page 48: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/48.jpg)
Neural Networks
• Motivated by biological nervous system
• Perceptron = a model of a neuron
WiXi
WiXi
e
eactivation
1
1x 2x 3x 4x 5x
1w5w
4w2w 3w
![Page 49: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/49.jpg)
Classical Neural Net
• Multi-layer network of perceptrons
• Proper weights are “discovered” from random– forward propagation of training set
– compare output to actual target variable
– back propagation of error to adapt weights
![Page 50: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/50.jpg)
50
Decision Trees
• Partition the data based on one attribute...
A B C D Resp.0 0 1 0 Buy0 1 1 1 Buy1 0 1 0 No1 1 0 0 Buy1 1 1 1 No
![Page 51: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/51.jpg)
51
A=0
A=1
Induction of Decision TreesInduction of Decision Trees
Recursively, partition each of the nodes
A B C D Resp.0 0 1 0 Buy0 1 1 1 Buy1 0 1 0 No1 1 0 0 Buy1 1 1 1 No0 0 1 0 Buy
0 1 1 1 Buy1 0 1 0 No1 1 0 0 Buy1 1 1 1 No
![Page 52: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/52.jpg)
52
0 0 1 0 Buy0 1 1 1 Buy
1 0 1 0 No1 1 1 1 No
A B C D Resp.0 0 1 0 Buy0 1 1 1 Buy1 0 1 0 No1 1 0 0 Buy1 1 1 1 No
Induction of Decision TreesInduction of Decision Trees
…until the node is homogeneous
1 1 0 0 Buy
1 0 1 0 No1 1 0 0 Buy1 1 1 1 No
A=0
A=1
C=0 C=1
A B C D Resp.0 0 1 0 Buy0 1 1 1 Buy1 0 1 0 No1 1 0 0 Buy1 1 1 1 No
![Page 53: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/53.jpg)
53
Classification
• Go down a matching path...
A=0
A=1
C=0 C=1
Buy No
(A=1,B=0,C=0,D=1)
Buy
![Page 54: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/54.jpg)
54
ClassificationClassification
Continue...
A=0
A=1
C=0 C=1
(A=1,B=0,C=0,D=1)
Buy
Buy No
![Page 55: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/55.jpg)
55
Classification
A=0
A=1
C=0 C=1
No
(A=1,B=0,C=0,D=1)
NoBuy
Buy
…until reaching a leaf Use the leaf’s probability
![Page 56: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/56.jpg)
56
Set of Rules, or Market Segments
A=0
A=1
C=0 C=1
A=0 => BuyA=1 and C=0 => BuyA=1 and C=1 => No
Buy
Buy
NoNo
• Each rule represents a market segment
![Page 57: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/57.jpg)
Which Modeling Technique?
• Decision trees (ChAID, CART, C4.5)– symbolic: model is interpretable as set of rules
– essentially is a segmentation
– useful when few “classes”, e.g., based on action (send/not, or few offer types)
• Regressions, neural nets– numeric: allows fine-tuning, e.g., for prediction or ranking
– model is hard to interpret and used as “black-box”
– useful when target is continuous
![Page 58: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/58.jpg)
Data Mining
• Knowledge Discovery in Databases (KDD)
• KDD is the process of identifying valid, novel, potentially useful, and ultimately understandable patterns in data
• Includes all steps of data preparation + management• Data mining step uses statistical techniques, decision
trees, neural nets, etc.
![Page 59: Database Marketing](https://reader035.vdocuments.net/reader035/viewer/2022062321/568130f3550346895d971815/html5/thumbnails/59.jpg)
Summary
• Customer data can be leveraged to better understand and manage current customers, and target new ones
• Data analysis and visualization– insights about our customers
– business economics
• Modeling– “mined” insights
– classify/predict behavior