joseph essas, opentable // mining diner talk

Mining Diner Talk at OpenTable

Joseph Essas CTO Opentable @jessas

DataDrivenNYC June 2015

• Over 32,000 restaurants worldwide

• more than 760 million diners seated since 1998, representing more than $30 billion spent at partner restaurants

• Over 16 million diners seated every month

• OpenTable has seated over 190 million diners via a mobile device. Almost 50% of our reservations are made via a mobile device

• OpenTable currently has presence in US, Canada, Mexico, UK, Germany and Japan

• OpenTable has nearly 600 partners including Facebook, Google, TripAdvisor, Urbanspoon, Yahoo, and Zagat.

2

OpenTable the world’s leading provider of online restaurant reservations

At OpenTable we aim to power the best dining

experiences!

Ingredients of a magical experience

Understanding the diner Understanding the restaurant

Building up a profile of you as a diner from explicit and implicit signals - information you have provided, reviews you have written, places you have dined at etc.

What type of restaurant is it? What dishes are they known for? Is it good for a date night/ family friendly/ has amazing views etc. What’s trending?

Connecting the dots

5

KafkaUsers

backups

Query/Processing Layer

Spark SQL

Spark Streaming

Spark MLlib

JSON

Data Products

we have a wealth of data

30 million reviews

diner requests and

notes

menus

external

ratings, searches and transactions

images

8

Our reviews are rich and verified, and come in all shapes and sizes

Superb!

This really is a hidden gem and I'm not sure I want to share but I will. :) The owner, Claude, has been here for 47 years and is all about quality, taste, and not overcharging for what he loves. My husband and I don't often get into the city at night, but when we do this is THE place. The Grand Marnier Souffle' is the best I've had in my life - and I have a few years on the life meter. The custard is not over the top and the texture of the entire dessert is superb. This is the only family style French restaurant I'm aware of in SF. It also doesn't charge you an arm and a leg for their excellent quality and that also goes for the wine list. Soup, salad, choice of main (try the lamb shank) and choice of dessert - for around $42 w/o drinks.

Many restaurants have thousands of reviews.

9

We expect diner reviews to be broadly composed of a handful of broad themes

Food & Drinks Ambiance Service Value for

Money Special

occasions

This motivated diving into the reviews with topic modeling

Modeling Reviews with

Topics

11

We approached the problem from the point of view of summarizing each restaurant using its reviews

Analyze the corpus of reviews in a geographic region to learn topics

Classify topics into categories (food, ambiance, service etc…)

Map topics back to restaurants

For each restaurant and a topic, surface relevant reviews

12

We applied non-negative matrix factorization to learn topics …

• stopword removal • vectorization • TFIDF • NNMF

13

… and obtained a pretty clean set of topics

14

Topics fell nicely into categories

DrinksFood Ambiance

15

Topics fell nicely into categories

ServiceValue Occasions

Topics to

Insights

Our topics reveal the unique aspects of each restaurant without having to read the reviews …

Each review for a given restaurant has certain topic distribution

Combining them, we identify the top topics for that restaurant.

0

0.5

1

Topic 01 Topic 02 Topic 03 Topic 04 Topic 05

0

0.5

1


0

0.5

1


review 1

review 2

review N

...

0

0.5

1

Topic 01 Topic 02 Topic 03 Topic 04 Topic 05Rest

aura

nt

Looking at the topics and the top reviews associated with it , we know Espetus Churrascaria is not just about meat and steak, but has good salad as well! The service is top notch, its kid friendly, and people go for special occasions, …

19

… while Fog Harbor Fish House is not just about fish, crab and local seafood, but boasts an excellent view of the bay, specially during sunset!

20

Topics often reveal the unique aspects of each metro.

CHICAGO SAN FRANCISCO

NEW YORK

LONDON

21

Our topics even reveal nuances across continents!

US CITIES LONDON

22

Topics on certain special days reveal unique insights

• Ordered steak medium well and steak came to table rare. • I ordered mine medium rare and she ordered hers medium plus. When we

received our steaks, we both had medium rare. My wife asked for the steak to be put on a little longer as it was not to her specification … She ended up not even eating it.

• Somehow a $200 steak was the worst steak I've ever had. My wife had a different steak and she agreed...worst steak ever.

• The steak I ordered Rare arrived well done. I sent it back and the second steak came Medium Rare.

• Went for Valentines day and ordered Black Angus Rib Eye. Big mistake.The steak was ordered medium and came burnt on BOTH sides.

One Star Reviews Valentine’s Day Steak Phenomenon

23

• amazing quality steak cooked to perfection. • I had the best time here. The food was amazing and the service was

unbelievable. My steak was cooked perfectly. :) • We ordered the Valentines Day special steak and lobster and the T-bone

steak with mashed potatoes and asparagus, one of the best steaks ever and even the asparagus was cooked perfect. The service is probably the best we ever had here in Hawaii.

• The steak was PHENOMENAL! It melted in your mouth like butter and was cooked perfectly. I'm going to be having dreams about that steak. Yum!! Thank you

Five Star Reviews

Topics on certain special days reveal unique insights

Valentine’s Day Steak Phenomenon

24

A lot of our marketing, blog, and email campaigns originate from insights like these found in the review data.

Reviews to

dishes and attributes

26

We use dish tags to highlight popular food and drink items at restaurants

We started with a curated list of dish tags and synonyms, and implemented several smart string matching rules to disambiguate varying mentions of the same dish.

27

We leveraged food and drink related topics to expand our corpus of dishes and drinks

Most dishes are usually 1-grams (“tiramisu”) 2-grams (“pork cutlets”) or 3-grams (“lemon ricotta pancake”)

For each restaurant, we perform an N-gram analysis of the reviews within the scope of food topics and surface candidate dish tags

We were able to generate several thousands of dish tags using this methodology!

28

With dish tags and a bit of linear algebra we can easily detect which dish is trending!

• caramelized cauliflower • cauliflower and vegetable cigar • roasted cauliflower. • cauliflower soup (the cauliflower soup was a revelation of layered

favors) • cauliflower au gratin • eggplant, mushroom and kale paella with a side of the agrodolce

cauliflower • cauliflower risotto with shaved black truffles • white cloud cauliflower (was perfection!) • Manchurian Cauliflower appetizer • the crudo with cauliflower • cauliflower steak • the whole cauliflower • The truffle cauliflower gratin • Cauliflower flatbread • pureed cauliflower bucatini with cauliflower crab and cauliflower

bisque • grilled shishito peppers, cauliflower toast (like a super thin pizza--

delicious!) • Have you ever had cauliflower parmigiana? • cauliflower mac & cheese

The NYC cauliflower trend!

29

With dish tags and a bit of linear algebra we can easily detect which dish is trending!

Right now, artichokes are trending!

30

We also learn restaurant specific attributes from review text

We learn features using one vs. all Logistic Regression with L1 regularization via a mech turk curated labeled set.

For outdoor seating features include obvious ones such as ‘outdoor’, ‘patio’, as well as ‘raining’, ‘sunny’, ‘smoke’, etc. …

31

Attribute tags and topics can be used to enhance user profiles

Sentiment &

Snippets

33

Sentiments - we use ratings as labels for positive and negative sentiments

Ingredients of a stellar experience

34

Sentiments - we use ratings as labels for positive and negative sentiments

Ingredients of a terrible experience

35

We are using nlp+sentiments to surface relevant snippets around tags

36

The model knows that “to die for”, “crispy”, “moist” are actually indicative of positive sentiment when it comes to food!

• The lobster and avocado eggs Benedict are to die for. • We finished out meal with the their blackberry bread pudding which was so moist and

tasty. • The pork and chive dumplings were perfectly crispy and full of flavor. • I had the Leg of Lamb Tagine and it was "melt in-your-mouth" wonderful. • … we did our best with the scrumptious apple tart and creme brulee. • My husband's lamb porterhouse was a novelty and extremely tender. • We resisted ordering the bacon beignets but gave in and tried them and were glad we

did---Yumm! …

Mining diner requests

and notes

38

Looking through what diners have requested to the restaurant can not only lead to comic relief …

“Can you make it extra romantic and nice? A booth will be preferable. My girlfriend is a bit on the crazy side, and I would like to spend

Valentine’s Day without getting screamed at. PLEASE HELP ME SURVIVE VALENTINES DAY!”

“If at all possible, we’d like a waiter with a ponytail/longer hair and/or accent.”

“Draw a puppy on a piece of paper and leave it on the table!”

Valentine’s Day

39

… but also deeper insights

Valentine’s Day requests are becoming more specific!

CONFIDENTIAL

keep in touch @jessas

joseph essas, opentable // mining diner talk

Technology