challenges of predicting user engagement
TRANSCRIPT
Challenges of Predicting User
EngagementZahra FerdowsiData Scientist @ Snapchat
User Engagement
Who: New and/or existing
users?
What time period?
When do you want to know?
What engagement
metric?
How utilize the outputs?
What: Churned and/or super
engaged?
Who?• New users
• Optimizing the registration and onboarding process
• What activities in the first hours/days help users the most to retain/ cause churn?
• How many friends?
• Existing users • Change in behavior -> what direction they are going
• What is the effect of certain experiences -> To optimize those processes
• Both could be focused on churn and/or super engaged but usually they have different set of features and outputs
• You know a lot about the users • Desktop/mobile, time of the day, Android/iOS, PC/Mac, OS version,
navigation/messaging speed, high/low penetration market, permissions
• Need to know user segmentation/personas• High intent to buy/window shopper, Creators/consumers, adopting fast, level of
engagement
More on Who?
• There is always trade off between accuracy and knowing ASAP
• Overcome the false positives by applying a solution that does not have a high negative effect on the false positives• Annoying engaged users with push notification?
• Using different solutions depend on user persona/tenure/focus
• Time is critical for churn users
When?
• Short-term / long-term metrics• Long-term metrics are harder to predict
• What if you have a sporadic purchase behavior? • Avg purchase once in a quarter: it would be harder to
predict in the next week
• Keep an eye on the seasonality trends• if a segment of users are coming only on the
weekends, then better to look at the metric over a week
What Time Frame?
• Which one is more important?• Reducing churn
• Increasing engagement
• It helps you to define the metrics• It does not necessarily mean different models
What?
• What metric you are looking at?• Conversion, Time spend, create content, active number of days a week
• Do we have to look at one metric for all the users?• Users with 10 friends vs. 1 friend in the first week?
• Users in 3 zone:
• Red zone: High probability to churn
• Yellow zone: Low engagement
• Green zone: High engagement
Another What?
• Have an engagement score • Daily run to know the engagement score for each user
• What is the segment that has the most change in last x days/months?
• Use engagement to predict other metrics such as customer value
• Optimizing the onboarding process: What activity to suggest users to do based on the stage they are in (Red / Yellow / Green Zone)
• Optimizing push notification• A/B test or not A/B test: testing on personas
How?
