June 2002 AQUAINT June 2002 Workshop
Just-in-Time Interactive Question Answering
Sanda Harabagiu: PI
Language Computer CorporationLanguage Computer Corporation
AQUAINT June 2002 Workshop - JITIQA
Just-in-Time Interactive Question Answering
People Sanda Harabagiu, PI Dan Moldovan Larry Daffner John Williams
AQUAINT June 2002 Workshop - JITIQA
Objective of Research Project
Address the interactive aspect of Q&A systems
design and implement a dialog shell that can be used with any Q&A system
Extension of work in factual Q&A with dialog capabilities already tested in call center systems
AQUAINT June 2002 Workshop - JITIQA
Novelty
Usage of just-in-time citations of other analyst’s similar Q&A sessions: Just-in-time citations are provided by Just-in-Time
Information Seeking Agents (JITISA) JITISA: intelligent agents that proactively search and
retrieve information that might be useful without requiring any action from the professional analyst Non-intrusive software agents that monitor the dialog
contexts generated when different agents interrogate Q&A systems
AQUAINT June 2002 Workshop - JITIQA
Tasks Proposed
AQUAINT June 2002 Workshop - JITIQA
Tasks Proposed
AQUAINT June 2002 Workshop - JITIQA
Building a Dialog Manager
Wizard-of-Oz Experiments Extract the features from the Wizard-of-Oz Data Implement the dialog manager with a
strategy that optimizes the features
AQUAINT June 2002 Workshop - JITIQA
Wizard of Oz Experiments
Year 1: Open-Curtain experiments Main focus: gauge the acceptability of the system-user
interaction. General assumption = no actually system exists,
develop a detailed conception of the system such that the responses to the user’s questions are known
Year 2: Study annoyance factors
AQUAINT June 2002 Workshop - JITIQA
Our assumption
Build a “transition network” of questions, answers and follow-up questions Completely unrestricted dialog on the user’s side
The user asks any questions Be able to refine the transition network by
allowing different turns
AQUAINT June 2002 Workshop - JITIQA
Open-Curtain Wizard-of-Oz experiment
Start with a complex question (which eventually does not return any acceptable answer)
User1
InformationSatisfaction
User2
DifferentPerspectives
User1
TemporalPerspectives
Q1
Q2
Q3
Q2
Q1
Q3
Q1
Q2 Q3 Qn
10AM 11AM 2PM
AQUAINT June 2002 Workshop - JITIQA
AQUAINT June 2002 Workshop - JITIQA
AQUAINT June 2002 Workshop - JITIQA
Semantic Representations for Interactive Q/A
Example: Why is there so much violence on the West Bank now ?
Processing: Syntactic Parse- binary dependencies
SemanticRepresentation
CAUSEviolence
West Bank
now
AQUAINT June 2002 Workshop - JITIQA
Novel Representation
Frame-like template:Event: template
slots:event: violenceentity: agent ?? patient ?? location: West BankMotivation: the answertime: now May 2, 2002
The same template can be generated by another reformulation of the original question:
“What are the causes of the current violence in the West Bank ?”
information comingfrom the answer
now
AQUAINT June 2002 Workshop - JITIQA
AQUAINT June 2002 Workshop - JITIQA
Ideal Case
Create a complex “transition network” containing all possible follow-up questions and all possible dialog paths
Q2
Q1
Q3
AQUAINT June 2002 Workshop - JITIQA
AQUAINT June 2002 Workshop - JITIQA
Template River Flow
Question: What are the causes of the current violence in the West Bank?
STATEEVENT = Template Name Event: violence Entity: Israel Location: West Bank Time: now
Answer: Large parts of the West Bank and Gaza remain under Israeli military occupation.
New slot: Motivation: Israeli occupation of West Bank Filler is a paraphrase of the answer
Template (Question)
AQUAINT June 2002 Workshop - JITIQA
Follow-up Question
Possibilities:
1) -same event
-a new slot
2) Relation-slot from Template
-other entity/event
-…
Possible Motivator: Elaboration
AQUAINT June 2002 Workshop - JITIQA
Color Code
1) Same slot, Same filler yellow blue
2) Same slot, Different filler red
River of colors
AQUAINT June 2002 Workshop - JITIQA
AQUAINT June 2002 Workshop - JITIQA
How are templates generated?
2-Phase Process First, templates are proposed by a linguist
novice, having a clear goal. (Larry) Second, they are refined by John and Sanda.
Phase I: Larry develops an algorithm of identifying fillers for each template.Sources: Question Parse, Semantic Form
Answer Parse, Semantic Form
AQUAINT June 2002 Workshop - JITIQA
Phase II
Generate template slots automatically (John Williams)Sources: Question Parse, Semantic Form
Answer Parse, Semantic FormDialog annotations, NE RecognizerDialog motivators and Context
modeling
AQUAINT June 2002 Workshop - JITIQA
Template Generation
Larry has created several types of templates:
STATE
INVOLVEMENT CONTRAST
EVENT RELATION ATTITUDE
ATTRIBUTES-definition questions
MOTIVATION-causality questions
AQUAINT June 2002 Workshop - JITIQA
Example
RELATION Templates:Generated by:
What is Ariel Sharon’s connection to the Palestinians?How do Israel’s Likud and Labor parties differ on peace with Palestine? ATTITUDE Templates:What is Ariel Sharon’s position on Palestine?Why is Jerusalem important to Palestinians?What were Ehud Barak’s views on peace with the Palestinians?Why did the Lebanese Phalange hate the Palestinians?How do the Hashemites view the Palestinian Authority?
AQUAINT June 2002 Workshop - JITIQA
Research Challenges
How many template kinds are sufficient? How should they be organized? What is the relationship between template
kinds, question classes and dialog context?
AQUAINT June 2002 Workshop - JITIQA
Novelties
1) Open-Domain Questions
2) No pre-defined task
-Task needs to be “guessed” as the dialog evolves
3) A general template algebra needs to be implemented to:
a) Deal with templates created in an ad-hoc manner
b) Help define the template motivators
c) New set of motivators (e.g. elaboration, tangent)
AQUAINT June 2002 Workshop - JITIQA
Phase III and IV
Phase III: Use two analysts to conduct a dialog starting with the same first question and the same general goal. Study the completeness and efficiency of the dialog motivators as well as the possible overlap.
Phase IV: Allow mixed initiative.
AQUAINT June 2002 Workshop - JITIQA
Enhance the Q/A Accuracy
Definition questions Causality questions Causality-Effect/Involvement questions Attitude questions
AQUAINT June 2002 Workshop - JITIQA
Thank You !Thank You !