big data analytics: the math, the implementation and how it can be effectively used to reach...

UP NEXT… 10:00am

Big Data Analytics: The Math, the Implementation

and How it can be Effectively Used to Reach

Customers

BECK NADIR

Follow the action on Twitter using #AtE2014

Big Data Analytics: The math, the implementation, and how it is

used to reach customers By Beck Nadir

10/15/14 2!

3!

By the Time You Walk Out of Here, You…

Will not be afraid of statistics!

4!

By the Time You Walk Out of Here, You…

Will not be afraid of statistics! Want to predict behavior

5!

The Person Occupying Your Lunchtime! How am I here? -  Huge nerd from the start -  Web analytics personnel at Moz

6!

The Person Occupying Your Lunchtime! How am I here? -  Huge nerd from the start -  Web analytics personnel at Moz

Education -  B.S. in Nuclear Engineering from Purdue University -  MBA from University of Washington – Bothell

7!

Synopsis

Can every action in life be calculated?

8!

We Will Go Over… How do we track data, and why do we care?

9!

We Will Go Over…

What tools do we use to track and capture data?

10!

We Will Go Over…

Mathematics!

11!

We Will Go Over…

How do we make sense out of data?

12!

We Will Go Over…

Can we predict future customer behavior at Moz?

13!

We Will Go Over…

Customer value, and TAGFEE culture

14!

How We Track Data, and the Tools We Use

15!

How We Track Data, and the Tools We Use

16!

Mathematics

The all watching eye?

17!

Mathematics

What if I wanted to get personal…REALLY personal?

18!


“…When someone suddenly starts buying lots of scent-free soap and extra-big bag of cotton balls, in addition to hand sanitizers and washcloths, it signals they could be getting close to their delivery date”. (Target, 2012)

Mathematics

19!

“…because people who are going through a divorce are more likely to miss payments, your domestic troubles are of great interest to a company that thrives on risk management. Exactly how the credit industry does it (predict divorce) – through sophisticated data-mining techniques – is a closely guarded secret.” (Visa, 2010)

Mathematics


20!

Would we do this at Moz?

Mathematics

21!

Mathematics

Case Study:

22!

Mathematics

Case Study:

Is there a way to expect a certain salary?

23!

Mathematics

Case Study:


Does education matter?

24!

Mathematics

Case Study:



Advanced Degree?

25!

Mathematics

Case Study:



Advanced Degree?

Gender?

26!

Mathematics

What if…

Salary = 49708.65 + 424.78*yrs_exp + 1723.46*yrs_educ + 153.47*supervis + 1280.52*perform – 4372.53*female + 1239.95*MBA

27!

Mathematics

What if…

…we know what to expect at all times?

28!

Mathematics

All data we have, says something about the future.

29!

Mathematics

All data we have, says something about the future. It’s a question of probability, and independent variables.

30!

Mathematics

All data we have, says something about the future. It’s a question of probability, and independent variables. Bond, James Bond!

31!

Mathematics

http://www.youtube.com/watch?v=l5C7LMOWyYc

32!

Mathematics We always start with lots of data

33!

Mathematics We always start with lots of data

Nm = Number of males= 24. Nf = Number of females = 17.

Xm = Average male management salary = $68,609.

Xf = Average female management salary = $65,763.

Sm = Male salary standard deviation = $6,108. Sf = Female salary standard deviation = $6,084

34!

Mathematics We always start with lots of data The trick is making sense out of it

Nm = Number of males= 24. Nf = Number of females = 17.

Xm = Average male management salary = $68,609.

Xf = Average female management salary = $65,763.

Sm = Male salary standard deviation = $6,108. Sf = Female salary standard deviation = $6,084

35!

Mathematics

Overwhelming!

36!

Mathematics 3 Common Ways of Creating Predictive Analytics:

37!

Mathematics 3 Common Ways of Creating Predictive Analytics: a. Multi-Colinearity Analysis

§  Practice of finding and relating one variable (KPI) to another.

§  The less related two variables are to each other, the better for analysis.

38!

Mathematics 3 Common Ways of Creating Predictive Analytics: a. Multi-Colinearity Analysis

Compare/Contrast

39!


b. Linear/Multi-Linear Regression

§  Where statistics come together, to predict a future event. §  A series of variables, determines a single outcome.

40!


b. Linear/Multi-Linear Regression

Fit the Pieces

41!


c. Cluster Analysis

§  Practice of grouping data points in similar “clusters”. §  Practice of statistical distribution, and multi-objective

optimization.

42!


c. Cluster Analysis

Group the Knowledge

43!


44!


Compare/Contrast

45!

Mathematics

Fit the Pieces

3 Common Ways of Creating Predictive Analytics:

Compare/Contrast

46!

Mathematics

Group the Knowledge Fit the Pieces

3 Common Ways of Creating Predictive Analytics:

Compare/Contrast

47!

How Do We Make Sense Out of Data?

Does Gender and/or education effect salary?

48!



Case Study:

Harvard Review’s Equal Pay for Equal Work

49!

How Do We Make Sense Out of Data? Multi-Colinearity Analysis

50!


Compare/Contrast

51!


y = -9E-06x + 1.0371 R² = 0.02627

0

0.5

1

45000 50000 55000 60000 65000 70000 75000 80000 85000 90000

Gen

der

Salary ($)

Gender vs. Salary

52!


Years Exp.

Yrs. Educa.on

Supervisor Exp. Performance Female MBA Salary

Salary Last Job

How Much Asked For Ambi.on

Salary 0.58 0.55 0.58 0.59 0.52 0.57 1 0.85 0.9 0.75

53!


y = -9E-06x + 1.0371 R² = 0.02627

0

0.5

1

45000 55000 65000 75000 85000

Gen

der

Salary ($)

Gender vs. Salary

Years Exp.

Yrs. Educa.on

Supervisor Exp. Performance Female MBA Salary

Salary Last Job

How Much Asked For Ambi.on

Salary 0.58 0.55 0.58 0.59 0.52 0.57 1 0.85 0.9 0.75

54!

How Do We Make Sense Out of Data? Transition from Multi-Colinearity Analysis…

55!

How Do We Make Sense Out of Data? Transition from Multi-Colinearity Analysis…

Now that we see a correlation, does this mean causation?

56!

How Do We Make Sense Out of Data? Linear Regression

57!


Fit the Pieces

58!


Ask a basic question:

59!




60!



Null Hypothesis = Ho = Mm – Mf = 0

61!



Null Hypothesis = Ho = Mm – Mf = 0

Alternate Hypothesis = Ha = Mm – Mf > 0 (Men make higher salaries).

62!


Determine T_critical – The maximum threshold disproving Ho.

63!



df (Degrees of Freedom) = N – 2 = 41 – 2 = 39

64!


To decide: α = Alpha = Margin of error = 5% (95% certainty)

df (Degrees of Freedom) = N – 2 = 41 – 2 = 39


65!


66!


T_critical = 1.7

67!

How Do We Make Sense Out of Data? Linear Regression Calculate T_actual – difference between the two means

68!


T _ actual = (Xm− Xf )− (Mm−Mf )Sm2 (Nm−1)+ Sf 2 (Nf −1)

Nm+ Nf − 21Nm

+1Nf

69!


T_actual = 1.47

T _ actual = (Xm− Xf )− (Mm−Mf )Sm2 (Nm−1)+ Sf 2 (Nf −1)

Nm+ Nf − 21Nm

+1Nf

70!


If Tactual is < Tcritical, reject the Null Hypothesis.

71!



1.47 < 1.7

72!



1.47 < 1.7

In this case, evidence shows women in management make less than male counterparts.

Linear Regression

73!


Is there a margin of error?

Linear Regression

74!


Confidence Interval Test: (Xm− Xf )− (tα /2 *γ ) ≤ µ1 −µ2 ≤ (Xm− Xf )+ (tα /2 *γ )


Linear Regression

75!


Confidence Interval Test: (Xm− Xf )− (tα /2 *γ ) ≤ µ1 −µ2 ≤ (Xm− Xf )+ (tα /2 *γ )

Where:

γ =Sm2 (Nm−1)+ Sf 2 (Nf −1)

Nm+ Nf − 21Nm

+1Nf

Linear Regression


76!


γ = 1933 and Tα/2 = 2.0


Linear Regression

77!


γ = 1933 and Tα/2 = 2.0

Therefore: -$1,020 ≤ Mm – Mf ≤ $6,712

Linear Regression


78!



Linear Regression

79!


Men could be making anywhere between $1,020 less, or $6,712 more than women.

Linear Regression


80!


What if I wanted to know more…what else affects pay?

Linear Regression

81!


82!


What if I wanted to dig even more…do education and MBA affect pay?

83!


84!


What if I wanted to specify groups to target?

85!


What if I wanted to specify groups to target?

Don’t worry, we can use math for that too!

86!

How Do We Make Sense Out of Data? Transition from Linear Regression…

87!

How Do We Make Sense Out of Data? Transition from Linear Regression…

We have lots of equations and linear regressions. What do we do with them?

88!

How Do We Make Sense Out of Data? Cluster Analysis

89!


Group the Knowledge

Cluster Analysis

90!


What kind of story can I tell about these clusters?

Cluster Analysis

91!


Discriminant Variable Cluster 1 Gender 1.406 Age 20-‐35 Educa.on (Years) 16-‐20

HH Income $40,000 -‐ $65,000 No. of Children 0-‐2 Conserva.ve? 0.275 Liberal? 0.801 Fun Loving? 0.622 Cu\ng Edge? 0.717 Family Oriented? 0.087 Trendy? 0.645


Cluster Analysis

92!


Discriminant Variable Cluster 2 Gender 1.404 Age 36-‐45 Educa.on (Years) 16-‐20



Cluster Analysis

93!


Discriminant Variable Cluster 3 Gender 1.102 Age 46-‐60 Educa.on (Years) 12



Cluster Analysis

94!


Left brain…Meet the right brain!


Cluster Analysis

95!


Not all predictions will be correct.

96!


Not all predictions will be correct.

“The Denver Broncos defeated the Seattle Seahawks 31-28 in the official EA SPORTS prediction of Super Bowl XLVIII”.

97!

Can We Predict Future Behavior at Moz?

98!


How can we better help our customers?

99!


How can we better help our customers? Signs of churn?

100!


101!


102!


Matt Peters, Moz Data Scientist Alyson Murphy, Senior Data Analyst Nick Sayers, Dir. of Customer Success and Support

103!


Proactively engaging Free Trialers (via chat and e-mail).

104!



Increasing vesting rate by 6.63%!

105!



Increasing vesting rate by 6.63%!

What actions or activities should we encourage customers to do?

106!


Free Trials longer than 1 month vest at a lower rate

107!


Prior Pro members vest at nearly TWICE the rate as first time customers. We should streamline their re-entry process.

108!


Community members vest at a higher rate.

109!


Users are most engaged during the first few days of their trial.

110!


Usage of MA and OSE drops to less then a few clicks / user after 10 days.

111!


Most campaigns are created during the first two days.

112!


Setting up a campaign is essential to vesting rate.

113!


What else should we look at?

114!

Customer Value and TAGFEE Culture

115!


We should also make sure:

116!


We should also make sure: -  The customer experience is personalized.

117!


We should also make sure: -  The customer experience is personalized. -  Realize not everyone will want to be chatted!

118!


We should also make sure: -  The customer experience is personalized. -  Realize not everyone will want to be chatted!

-  Customers realize the full value of Moz.

119!


120!


Transparent

121!


Transparent Authentic

122!


Transparent Authentic Generous

123!


Transparent Authentic Generous Fun

124!


Transparent Authentic Generous Fun Empathetic

125!


Transparent Authentic Generous Fun Empathetic Exceptional

126!

Conclusion:

127!

Conclusion:

128!

Conclusion:

Compare/Contrast

129!

Conclusion:

Fit the Pieces Compare/Contrast

130!

Conclusion:

Group the Knowledge Fit the Pieces Compare/Contrast

131!

Conclusion: Subscribers are not just another customer!

132!

Conclusion: Subscribers are not just another customer! Our help team answers all questions, one by one!

133!

Conclusion: Subscribers are not just another customer! Our help team answers all questions, one by one! Founder, and CEO, interact with subscribers regularly

134!

Conclusion: Subscribers are not just another customer! Our help team answers all questions, one by one! Founder, and CEO, interact with subscribers regularly Moz Community!

135!

Conclusion: Subscribers are not just another customer! Our help team answers all questions, one by one! Founder, and CEO, interact with subscribers regularly Moz Community!

136!

Homework! Dig through your data!

137!

Homework! Dig through your data! Are there metrics you can relate to each other?

138!

Homework! Dig through your data! Are there metrics you can relate to each other? What factors make up revenue (or a key metric) in your businesses?...hypothesis test, fit them together!

139!

Homework! Dig through your data! Are there metrics you can relate to each other? What factors make up revenue (or a key metric) in your businesses?...hypothesis test, fit them together! Have you segmented your customers? What groups do they represent?

140!

Homework! Dig through your data! Are there metrics you can tie revenue to? What factors make up revenue in your businesses?...fit them together! Have you segmented your customers? What groups do they represent? Make mistakes!

Thanks for Watching! LinkedIn: Beck Nadir

Twitter: @annalesparrales 141!

142!

Calculator picture, page 6 http://pixabay.com/en/calculator-calculation-insurance-385506/ Caring picture, page 7 http://pixabay.com/en/care-feeling-female-couple-give-20185/ Ciarelli, Nicholas (2010, April 6). How Visa Predicts Divorce. Retrieved March 24, 2013, from: www.dailybeast.com.

http://www.thedailybeast.com/articles/2010/04/06/how-mastercard-predicts-divorce.html Denver Broncos prediction, page 95: http://www.easports.com/madden-nfl/news/2014/super-bowl-48-prediction Hill, Kashmir (2012, February 16). How Target Figured Out a Teen Girl was Pregnant Before Her Father Did. Retrieved March

25, 2013, from: www.forbes.com. http://www.forbes.com/sites/kashmirhill/2012/02/16/how-target-figured-out-a-teen-girl-was-pregnant-before-her-father-did/

Adobe Omniture Site Catalyst Example Provided by: https://help.optimizely.com/hc/en-us/articles/200039985-Integrating-Optimizely-with-Adobe-Analytics-Omniture-SiteCatalyst-

Retail Dashboard Example Provided by: http://www.dashboardinsight.com/dashboards/live-dashboards/financial-operations-dashboard-dundas.aspx

References

143!

Compare contrast picture, pages 37, 43, 44, 45, 49, 127, 128, 129 http://en.wikipedia.org/wiki/Apples_and_oranges Database picture, page 15 http://pixabay.com/en/database-data-storage-cylinder-149760/ Hadoop logo, page 13,125 http://commons.wikimedia.org/wiki/File:Apache_Hadoop_Elephant.jpg SQL Server: 13, 125 http://commons.wikimedia.org/wiki/File:Sql-server-ce-4-logo.png Tetris picture, page 39, 44, 45, 56, 128, 129 http://commons.wikimedia.org/wiki/File:Tetrominoes_IJLO_STZ_Worlds.svg Nerd picture, page 4, 5 http://pixabay.com/en/nerd-scientist-chemist-physicist-155841/ Tool picture, page 8 http://pixabay.com/en/tool-pliers-screwdriver-145375/ Math formula picture, page 9 http://pixabay.com/en/math-function-symbol-icon-27248/

References

144!

Question Mark Picture, page 10, 11 http://commons.wikimedia.org/wiki/File:Red_question_mark.png Multiple Question Marks, page 26-29 http://pixabay.com/fr/point-d-interrogation-questions-63979/ James Bond Scene, page 30 http://www.youtube.com/watch?v=l5C7LMOWyYc Overwhelmed Picture, page 34 http://pixabay.com/en/mimic-panic-scratch-woman-person-156928/ Group Picture, page 41, 45, 88, 129 http://pixabay.com/en/queue-communal-community-group-154925/ Crystal Ball Picture, page 101, 102, 103, 104 http://pixabay.com/en/crystal-ball-glass-globe-glass-ball-32381/ Dashboard Example Picture, page 14, 126 http://commons.wikimedia.org/wiki/File:Well_Organized_Dashboard_Example.jpg Omniture Logo Picture, page 14, 126 http://commons.wikimedia.org/wiki/File:Omniture.png

References

big data analytics: the math, the implementation and how it can be effectively used to reach...

Marketing