using data to drive user experiences
DESCRIPTION
TRANSCRIPT
Jeff Patti [email protected] Ram Parthasarathy [email protected]
Using Data to Drive User
Experiences
Monday, February 25, 13
On the web, one size doesn't fit all.• Demographics • Behavior• Device (laptop vs tablet vs phone)
Algorithmic Personalization •More relevance, reduced noise to useful information ratio. (eg: your facebook feed)
Personalization
Monday, February 25, 13
Amazon Homepage (Jeff)
Monday, February 25, 13
Amazon Homepage (Ram)
Monday, February 25, 13
Not personal anymore!
Monday, February 25, 13
User Data is PreciousEvery two days, we create as much information as we did up to 2003.– Eric Schmidt, then-CEO of Google
Facebook ingests 500+terabytes of new data into their databases every day.
“If you aren’t taking advantage of big data, then you don’t have big data, you have just a pile of data,” - Jay Parikh, VP of infrastructure at Facebook
Monday, February 25, 13
... And Scary Too!
Monday, February 25, 13
Relevant Search Results - Good!
IP spoofed with a proxy
Monday, February 25, 13
Recos based on purchases - Good!
Monday, February 25, 13
Info based on user action - Good!
Monday, February 25, 13
Excessive Ad Retargeting - Bad!
Monday, February 25, 13
Identity Theft, Selling data - Ugly!
Monday, February 25, 13
We provide the data!
• Personal Info - Name, DOB, family, education• Interests and Hobbies• Physical places visited• Websites visited• Contacts on the phone
Monday, February 25, 13
They "trust me"Zuck: Yeah so if you ever need info about anyone at Harvard
Zuck: Just ask.
Zuck: I have over 4,000 emails, pictures, addresses, SNS
[Redacted Friend's Name]: What? How'd you manage that one?
Zuck: People just submitted it.
Zuck: I don't know why.
Zuck: They "trust me"
Zuck: Dumb f****.
Monday, February 25, 13
Personalized Results - Good/Bad?
Monday, February 25, 13
DuckDuckGo
Monday, February 25, 13
A/B Testing
A - original version of websiteB - modified version of website
Show each version to different segments of website visitors and track metrics measuring
performance of visitors
source: visualwebsiteoptimizwer
Monday, February 25, 13
Obama Campaign A/B Testinghttp://kylerush.net/blog/optimization-at-the-obama-campaign-ab-testing/
Both Obama campaigns made extensive use of A/B testing to optimize their websites
Monday, February 25, 13
Obama Campaign A/B Testing
Monday, February 25, 13
Imitation is the sincerest form of flattery
Monday, February 25, 13
Monday, February 25, 13
Monday, February 25, 13
Why A/B Testing
Schrödinger's cat - The cat is either alive or dead, not both alive and dead.
Monday, February 25, 13
Which Version Won?
3.95%
Monday, February 25, 13
Which Version Won?
5.78%
Monday, February 25, 13
Which Version Won?
Source: visualwebsiteoptimizer.com
Monday, February 25, 13
change anything, anywhere and at anytime on a website
How Monetate Works
Default Page Campaign Delivered Optimized Version
Monday, February 25, 13
a smarter testing approach
Your Default Page
ExpressTag fires on page load
Optimizedversion delivered
Fast (Asynchronous)
change
Campaign rules applied
Monday, February 25, 13
Fast and Easy UIA Sentence based interface
Monday, February 25, 13
20% 20%of peak ecommerce in holiday 2011 & 2012
Monday, February 25, 13
Monetate Stack• AWS - hundreds of machines spread across multiple
regions
• Sharded and Replicated MySQL
• Elastic Map Reduce using mrjob, Mahout, and Hive
• Python on the backend
• Google closure tools in the UI and front-end
• Git and Github for version control.
Monday, February 25, 13
Data Flows• Real time system for executing A/B tests
• Uses MySQL, a relational database
• Good for many small operations quickly
• Post hoc analysis of experiments
• Uses Elastic Map Reduce using mrjob, Mahout, and Hive
• Good for batch processing - large jobs with high latency
Monday, February 25, 13
Map/Reduce Example - Word Count
Monday, February 25, 13
Role of Computer Science• Design - User Interface Design and UX• Speed and performance optimization• Cloud computing - scale and security• Algorithm design and development• Building data models and prediction systems• High performance web engineering
Monday, February 25, 13
We are hiring!
Interns and Software Engineers
monetate.com/jobs
Monday, February 25, 13
Thank You!
Questions?
Monday, February 25, 13