Personalization and experimentation at Booking.com

Shalini Sukumar | May 27, 2017

Shalini SukumarProduct Owner Data Science, Booking.com

Responsible for using data to drive product decisions(across funnel) and create the best possible customer experience at Booking.com.

As an avid technology enthusiast, Shalini has worked in various technology roles in the past, which range from a summer internship at ebay, California, US, to being a developer for 4 years in the IT department at the Credit Suisse Investment Banking division, Singapore. Prior to joining Booking.com, I worked as a Data Product Owner at a Big Data start-up, Crayon Data, in Singapore.

www.linkedin.com/in/ShaliniSukumar

● Introduction to Booking.com

● Introduction to Personalization?

● Product Data

● The Booking.com Approach -AB

testing

Agenda

15,000+People worldwide dedicated to serving both our customers and partners.

1996Founded in Amsterdam by Geert-Jan Bruinsma. 2005

Became part of the Priceline Group (PCLN).

195+Offices all over the world so our partners can reach us easily.

The Booking Story.

1.6M+Active properties in

225+ countries.

Customers Partners

Connecting travellers with great stays.

Introduction to Personalisation

Let us start with some definitions…

✓ Personalisation is “a means of meeting the customer's needs more effectively and efficiently, making interactions faster and easier and, consequently, increasing customer satisfaction and the likelihood of repeat visits. It’s achieved when a system tailors an experience based on a consumer’s previous behaviours. For example, Amazon personalises its home page for each user based on that user’s previous searches, views and purchases.

✓ Customisation, according to Merriam-Webster, is a method of “changing something in order to fit the needs or requirements of a person, business, etc.”

So, What’s the difference exactly?

Less is more .

Lifecycle of the customer

Eng

agem

ent

Awareness

Explore

Commit

ExperienceRelive

Retention

Win-back

Engagement Lifecycle Phase I

Low specificity in the customer intent – Personalisation drives them

Specific Intent in User behavior – Personalisation is a loyalty/retention tool

Engagement Lifecycle Phase II

How does personalization work?

Recommendation Engines/Models

User Behavior

Data

Contextual Data

Product Data

Instances in Front End

User Type Personalization

To adapt the product to the user and his context

Recommendations to direct the user

Powerful pattern: “Users who did X also did Y”

Example from search on mobile:“Users who searched from your location booked in …”

Learning from the customer behavior in the past

Natural Language Processing from User Generated Content

Image recognition

The Booking ApproachData Driven Experimentation - “What gets measured gets managed.”

Can we just ask people?

2. People make bad decisions.90% of product decisions have an inconclusive or negative effect on a product’s primary metric

Source: VWO 2016.

90%

10%

Short answer – No !

1. Intent vs. Behaviour.

Consumers don't really know what they want or need until they are faced with the product

Hypothesis Testing.

90%

1. <IDEA> How a user is able to perform another search from the Search Results page isn’t immediately clear due to the collapsed Search Box.

2. <METRICS> Shallow: SearchesDeep: Conversion

3. <MAKE IT>

A/B testing

Base. Variant.

Which one performed better?

Base.

Which one performed better?

Variant.

● All of us are wrong most of the time

● We want what we do to be profitable

● We want customer sentiment to drive product development

● We want decisions to be based on fact

Why should you do A/B Testing?

Why shouldn’t you be testing?

● You don’t have enough traffic

● You don’t know what success looks like (no success metrics)

● You like being ‘the expert’

Key Takeaways.

● Less is more ● Product Data● Address personalisation● Test everything● Retest everything● Question everything ● Data is power. But with power comes a lot of responsibility.

Travel. Explore. Empower.

Thank you.