lead scoring: how a stanford engineer and mit mba approaches it - ilya mirman - onshape - growth...

Lead Scoring:How a Stanford Engineer

and MIT MBA Approaches itIlya Mirman

VP of Marketing, Onshape

(Lead Scoring Tips & Tricks)Ilya Mirman

VP of Marketing, Onshape

1. Why score leads?2. Multi-factor scoring3. Implementation4. Assessing effectiveness

Agenda

Why score leads?> Some leads are much

better than others> Sales (probably) cannot

(and should not) work every lead

> Marketing should develop and identify the best leads for Sales to follow up on

Single factor scoring> Type of lead source

(or another factor)> Prioritize leads> Identify cut-off

(Hopefully, we’re all doing at least this)

Benefits of Using Multiple Factors

> Lots of “signals” regarding lead quality:●Lead source●Activity on site●SaaS app’s usage metrics●Demographics (user, company)●Community participation●Etc.

> Taken together, a better way to assess lead quality

Benefits of multiple factors

Example: Activity on site

Example: SaaS app usage

Example: Demographics

Example: Community Participation

Implementation

> Outsource to a 3rd party “black box” grading service●Some good options out there (but I wouldn’t start here)

> Use marketing automation software to define scoring criteria●Requires trial/error for tuning

> Develop a mathematical expression that can be easily implemented in CRM tools●The focus of this talk

Implementation options: Pros/Cons

> Could be as simple as a single factor●Though ideally, find two or more

> Need ~100-1,000 leads and 10-50 won deals● If too early to have enough won deals, could use “MQL” or some other

level of qualification (e.g., “Opportunity”)> Start looking for factors EARLY

●Test multiple hypotheses●Make sure your systems support collecting the data (potentially, record

value at time of purchase)

How early? How sophisticated?

> Multiple options (a blend of science and art)

> I like linear regression● Linear combination of factors● Some simplifying assumptions (e.g., factors

are independent)● May not need ALL factors (in fact, might not

be able to use all)

> Goal: figure out each factor’s weight to maximizes prediction of whether lead will BUY

What Type of Model?

Goal: figure out each factor’s weight

Putting it to practice - with Excel

Source Data

Step 1: Find factors that correlate w/purchase

Step 1: Find factors that correlate w/purchase

Step 2: Linear Regression to Establish

Regression ResultStatistical measure of how close the data are to the fitted regression line

t-stat: Measure of the variable’s relevance.

Examples:● Above 2 (or less

than -2) 95% ⇒confidence that it’s relevant

● Above 3.5 (or less than -3.5) 99.96% ⇒confidence that it’s relevant

May need to iterate:● Are all variables

significant?● Does the sign of

the coefficient make sense?

Predicted vs. ActualWhat the regression model predicts

Difference between “Predicted” and “Actual”

Actual(add together “Predicted” and “Residuals”)

Predicted vs. ActualLet’s create a column for “Cumulative Deals”

Now, let’s SORT by lead grade...> This data set:

●Over 30,000 leads●Less than 600 deals (2%)

> Note the high hit rate of deals for the high score leads (over 50%)

Results> Top 10% of leads

drive 70% of deals

> Top 20% of leads drive 85% of deals

Evaluating & Improving Your Models

Comparing Multiple Models

Pop Quiz: Which Model is Better?

1. Identify parameters of interest2. Collect data on LEADS and DEALS3. Identify factors that correlate w/purchase4. Create Excel spreadsheet5. Run linear regression6. Iterate on which parameters to include in model7. Tune model to add/remove factors8. Evaluate quality of model

(Rinse & repeat, at the right time)

Summary

imirman [at] onshape.com

(Thanks!)

QUESTIONS?