© 2002 – 2007 versay solutions, llc. all rights reserved. building fault tolerant voice user...
TRANSCRIPT
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Building Fault Tolerant Voice User Interfaces
SpeechTEK 2007
Tuesday, August 21Track B
“Getting the VUI Right when Recognition Goes Wrong”
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Versay Solutions, LLC
Daniel PadgettSenior Speech Technology Consultant
Jessica Peterson Hicks, Ph.D.Speech Technology Consultant
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Fault Tolerant Voice User Interfaces
• Error Prevention
• Error Resolution
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Directed Dialog
• Prompt Design
• Grammar Coverage
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Proof of Concept NL Routing Application
• Natural Language (NL) Main Menu– “How can I help you?”
• Main Menu Confirmation
• “Back Off” Menu to support Main Menu failures
• Disambiguation dialog states
• Form-filling dialog states
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Case Study – Back Off Main Menu
• Initial Prompt– Designed to handle exception cases from Main Menu
– Options mapped to core business practices
• Grammar– Optimally constrained static grammar
– 5 slots / semantic interpretations
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Case Study – Back Off Main Menu
23%
17%
0%
5%
10%
15%
20%
25%
Pre-Optimized Optimized
Back Off Main Menu: NO-MATCH Rate
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Case Study – Main Menu Confirmation
• Initial Prompt– Dynamically constructed
– Covered ~100 categories
“Okay. [You need to change your address.] Is that right?”
• Grammar– Optimally constrained static grammar
– 2 slots / semantic interpretations
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Case Study – Main Menu Confirmation
9%8%
0%
3%
5%
8%
10%
Pre-Optimized Optimized
Confirmation State: NO-MATCH Rate
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Robust Interpretation (RI)
• Statistical Language Models
• Interpretation Grammars
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
RI – Back Off Main Menu
• Statistical Language Model– Training set of ~2500 utterances
– Test set of ~1000 utterances
• Interpretation Grammar– 8 slots / semantic interpretations
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
RI – Back Off Main Menu
23%
17%
8%
0%
5%
10%
15%
20%
25%
Pre-Optimized Optimized RobustInterpretation
Back Off Main Menu: NO-MATCH Rate
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
RI – Main Menu Confirmation
• Statistical Language Model– Training set of ~3000 utterances
– Test set of ~1000 utterances
• Interpretation Grammar– 20 slots / semantic interpretations
– Y, N, N+RoutingCategory, RoutingCategory
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
RI – Main Menu Confirmation
9%8%
2%
0%
3%
5%
8%
10%
Pre-Optimized Optimized RobustInterpretation
Confirmation State: NO-MATCH Rate
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Error Prevention - Summary
• Robust Interpretation for two key states
• Considerable increases in coverage
• Significant decreases in no-match rates
• Minimal False-Accepts
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Fault Tolerant Voice User Interfaces
• Error Prevention
• Error Resolution
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Error Resolution
• Dialog State Error Strategy• Universal Error Strategy• Task-related Error Strategy
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Established Methods
• Dialog State Error Handling– Escalating error strategy
– No-input and no-match handlers
– Implicit DTMF
• Universal Error Handling– Increment for each no-match, no-input
– Transfer when maximum exceeded
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Dialog State Error Strategy
• Escalating error handling– Rapid reprompt– Explicit DTMF on penultimate error prompt– Transfer on final error
• Combination of no-match and no-input handlers
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Universal Error Strategy
• Increment / decrement counter• Score errors on distinct dimensions
– Number– Consecutivity
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Task-related Error Strategy
• Determine “sensitivity” level for each call path• Allow fewer errors on sensitive paths, more
errors on less sensitive paths– Report a Problem = highly sensitive– Monthly Payment = moderately sensitive– Hours of Operation / Location = less sensitive
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Customer Service Universals
• Acknowledge• Explain• Re-prompt
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Customer Service Universals – Case Study
2496
811
355
0
500
1000
1500
2000
2500
3000
NL Main Menu 1 NL Main Menu 2 Back Off Menu
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Summary
Fault Tolerance means striking the right balance between:
• Error Prevention• Error Resolution
© 2002 – 2007 Versay Solutions, LLC. All rights reserved.
Contact Us
www.versay.com
(888) 869-0121