A Speech Recognizer Application

Zina Saadi

Anila Chowdhury

College of Computer Science Tufts University

2

Introduction & Inspiration

System Flow & Design

Implementation

Challenges

Branding

Usability Testing

Marketing & Future Work

Questions/ Comments

3

A Speech recognizer application that can help callers to find a restaurant in New York City.

Functionalities: Find a restaurant by Name

Search for restaurants by Cuisine

Search for restaurants by Neighborhood

Top Zagat Rated List

Narrow down a list by

Food Ratings

Décor Ratings

Service Ratings

Price Ratings

4

Regular and non-regular diner people can use it

Access it anytime by a phone call

Select top Zagat rated restaurants

Find a restaurant by name- get address, phone number

Find by Cuisine or Neighborhood

Helps you to narrow down your results based on rating

Get connected to the restaurant

Anila and Zina conducted a survey and about 87% people said “yes” to using a system to find restaurants.

5

System Flow

Main Menu

Restaurant

Information

Search

Results

Narrow

Options

Top List

6

User Scenario

• Scenario 1System: What is the name your restaurant?

Caller: Heidelberg

System: Heidelberg is a German restaurant located at East 80s neighborhood.

• Scenario 2Caller: French

System: I have found 203 restaurants that matched your search criteria.

Caller: Narrow Down (selected); Price Rating (selected)

System: In the Price rating, I've got restaurants that have price range between 0 (on the low end) and 86 (on the high end.) What's the maximum Price you want to pay?

Caller: 16

System: I have found 16 restaurants that matched your search criteria.

7

Recording Prompts

• Voice Talents:

– Katarina (Italian)

– Maria (Spanish)

– Jennifer (US English)

– Julie (French)

– Melissa (US English)

– Anila (US English)

• Music (selected by Zina)

– Mozart

8

17%

16%

6%

4%4%4%3%

46%

Cuisines DistributionItalian

American

French Bistro

Japanese

French

Chinese

Eclectic and InternationalOthers

9

Steakhouses Mexican and Tex Mex Asian Kosher Noodle Shops Swiss

Mediterranean Brasserie Dim Sum Coffeehouses and Desserts Tapas Tunisian

Pizza Spanish VietnameseSouth American Bakeries Ukrainian

Indian Coffee Shops and Diners

Delis and Sandwich Shops Brazilian Caribbean Armenian

Thai Greek Vegetarian Jewish Health Food Australian

Hamburgers Southern and Soul Continental Korean Middle Eastern Burmese

Bar B Q German Scandinavian Polish English Cambodian

Cuban Argentinean Soups Portuguese Ethiopian Colombian

Turkish Moroccan Californian Afghan Indonesian Dominican

Nuevo Latino Belgian Hot Dogs Jamaican Israeli Eritrean

Southwestern Russian Malaysian Austrian Persian Filipino

Irish Peruvian Egyptian Puerto Rican Lebanese

10

"The artist is nothing without the gift, but the gift is nothing without work."

11

User Interface

(Phone-Call <=> Bevocal)

Flow-of-Control Management

(cgi, vxml forms)

Data Extraction Interface

Data Extraction

(XML Parsing)

Data Storage

Front-EndBack-End

Merging

Ruby

Functions

12

State Tables

• Grouped them by functionality

VoiceXML

• Mappings: Cuisines Background soundsLanguage Voices

Interface• Updated/Added Functions

Result: CGI Files

Result: generated VXML via CGI

Result: <restaurantDataInterface.rb>

13

Xml

• Unicode Conversion

• Used Python

Data

• Extraction => Normalization=>Segmentation

Interface

• Data Encapsulation & Normalization

• Methods/Functions Definitions

Result: Normalized .xml File

Result: Created .pstore File

Result: <restaurantDataInterface.rb>

14

Restaurants Names

• <PVAL>Cafe Español</PVAL>

=> <PVAL>Cafe Espanol</PVAL>

string = re.sub(u"\u00F1", u"n", string)

• <PVAL>Mamá Mexico</PVAL>

<PVAL>Mama Mexico</PVAL>

re.sub(u"(\u00E0|\u00E1|\u00E4)", u"a", string)

15

Description

• "ok, not olé." or "Provençal country home"

• Celebrating its 40th anniversary, André and Rita Jammet's Midtown French classic”

• re.sub(u"(\u00E8|\u00E9|\u00EA|\u00EB)", u"e", string)

• re.sub(u"\u00E7", u"c", string)

16

• <PVAL>14 Wall Street Restaurant, The</PVAL>

The 14 Wall Street Restaurant

• <PVAL>Cloister Cafe, The</PVAL>

=> The Cloister Cafe

17

Punctuation Removal

• <PVAL>Salute!</PVAL>

• <PVAL>Pao!</PVAL>

Number Conversion (using ‘lingusitics’ ruby gem)

• <PVAL>101</PVAL>

18

Usability Testing

• Target Users– all kinds of web & phone users (laggard to advanced)

• What we tested– Basic functionalities

– How well they understood the product

– How easy was it to learn

• Test Subjects– Friends and families

• Challenges– Ask minimum numbers of questions and collect the right information

– How clear and intuitive each tasks are.

19

Test Plan• Wizard of Oz

– Subject pretended to be the caller

– I pretended to be the System

• Pilot Testing

– Small prototype

– Smaller set of Restaurant Data

– Less functionalities

• Final Testing

– Full implementation of all proposed design

– Full dataset

– All functionalities included (narrow, top list)

– Check values and boundaries for input and prompts

– Recorded Prompts + Background music

20

Usability Testing Results

• This application has great potential for real life use -- I often find myself trying to locate restaurants when I'm outside, and lack of access to a computer makes things difficult.

– Farial Anam, Senior Associate, LECG, LLC

• It's really neat (and it works great too). Since I am French, I focused on the 'French' cuisine, and it properly told me 10 restaurants with the proper pronunciation ("native quality" French)- I picked one ("Bistro les Amis") by saying the name (with my native, South-West, French accent) and it recognized it.... Very neat.

– Eric Montagut, Senior Principal Engineer, Basis Tech

21

Marketing & Future Work

• Suggest new restaurants

• Encourage people to use zagat.com and rate more restaurants

• Larger database

• Add new cities & restaurants

• Other languages

• Add features (buffet, romantic, music etc)

22

Teachers of the course:

Blade Kotelly & Chris Roby

23

شكرا بزاف

متشّكرين

唔該

非常感謝

多謝

Merci

danke

ευχαριστώ πολυ

תודה आभारी हँु

धन्यवाद

köszi

terima kasih

grazie

どうも

مرسي

متشكرم

Dzięki

obrigadinho

спасибо

ཐུགས་རྗེ་ཆེ་། teşekkürler

Cảm ơn rất nhiều

дякую

تشكر