Download - Dialogue Manager Senior Companion
![Page 1: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/1.jpg)
Dialogue ManagerSenior Companion
University of Sheffield
March 2008
![Page 2: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/2.jpg)
review
![Page 3: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/3.jpg)
Senior Companion Dialogue Manager 2-tier planning where
High-level plans built using the Cognitive Model Topic specific plans built using Dialogue Action Forms
DAFS Dialogue Action Forms Augmented Transition networks Represent domain knowledge Modularizable to encapsulate sub-topics within a conversation
domain
Back channel response while working on more complete system utterances (hm, uh-huh)
![Page 4: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/4.jpg)
DialogueManager
Natural Language
Understanding
Dialoguehistory
Ontology Photoapp
Generation
NLU output
of user utterance
Semantic content of
system utterance
+
emotion tags
CognitiveModel
Dialogue Manager Architecture
![Page 5: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/5.jpg)
DialogueManager
•Dialogue Acts•Named Entities
•Person•Family Relations
•Sister•Mother•grandmother
•Location•Celebration event
•Birthday•Anniversary•Wedding•Christmas
•Pronoun references•Semantic categories for
•User input objects (nouns)•User input actions(verbs)
•Original user input string•Modalities for user input
NLP output•Emotion Tags•Planning•System Model
Cognitive Model
•Hierarchy of Semantic tags for Photos•User instances
Ontology
•Record of User and systemutterances
•Photo/s id•Photo tags•Photo attributes
Photo App
DialogueHistory
DM input closeup
![Page 6: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/6.jpg)
DialogueManager
•Number of output Utterancesfor each:•Semantic Representation
•objects•actions
•Dialogue Act•Emotional content•Output modality
•Expected user response category
Cognitive Model
Fission/Generation
Speech Analysis
Dialogue History
OntologyPhoto App
DM output closeup
![Page 7: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/7.jpg)
Dialogue Action Forms
GUI editor for creating DAFsComposed of nodes and arcs containing
tests and actionsDAFs pre-stacked, but can be overidden
by matching indexing terms (semantic classes, significant words)Essential for mixed-initiative conversation
![Page 8: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/8.jpg)
SC Dialogue Manager Stack
goodbye
greeting
Run Greeting DAFPop greeting DAFPush photo DAF
goodbye
photo
System start
goodbye
location
event
people
date
special memory
goodbye
location
event
date
special memory
Run photo DAFRun people DAFPop people DAF
People DAF
![Page 9: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/9.jpg)
DAFs (1)
![Page 10: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/10.jpg)
DAFs(2)
![Page 11: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/11.jpg)
SC Output (1) Natural Language Generation
Currently choosing from a selection of pre-defined surface forms. A Generation module to be developed (CU)
To be shared with the Health and Fitness Companion To include input from the Cognitive model for emotion and
conversational style (based on user profile) Will contain lists of alternative question realisations derived from
corpora
TTS Loquendo TTS
Avatar
AAA, CrazyTalk, Nabaztag (2-D)
![Page 12: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/12.jpg)
more detail
![Page 13: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/13.jpg)
Current Dialogue Manager Architecture
InputQueue
Input Manager
NLU Photo Watcher
DAF EngineDAFStack
OutputQueue
Output Manager NLG
![Page 14: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/14.jpg)
Output of ASRIn the format of Galaxy Communicator frame.For example:{ c output
:hypothesises ( {c hypothesis …} {c hypothesis …} … ) }
{c hypothesis:ROName “Language Model Name”:Likelyhood -159.035736:Confidence 0.758829:String “nice one”:NumOfWords 2:WordsInfo: ( {c word …} {c word …} ) }
{c word:String “nice”:AcousticScore 3.791455:Confidence 0.750377:StartPoint 0:EndPoint 42:Language: “en-gb”}
![Page 15: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/15.jpg)
NLU
InputA textual string. For example: “Kate is the mother of Marry”.
Output1. GATE Annotation Set.A snippet output for the sentence: “Kate is the mother of Mary.”[AnnotationImpl: id=0; type=Token; features={category=NNP, kind=word, orth=upperInitial, length=4, string=Kate}; start=NodeImpl: id=0; offset=0; end=NodeImpl: id=1; offset=4, AnnotationImpl: id=1; type=SpaceToken; features={kind=space, length=1, string= }; start=NodeImpl: id=1; offset=4;end=NodeImpl: id=2; offset=5, AnnotationImpl: id=2; type=Token; features={category=VBZ, kind=word, orth=lowercase, length=2, string=is}; start=NodeImpl: id=2; offset=5; end=NodeImpl: id=3; offset=7, AnnotationImpl: id=3; type=SpaceToken; features={kind=space, length=1, string= }; start=NodeImpl: id=3; offset=7; end=NodeImpl: id=4; offset=8……]
![Page 16: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/16.jpg)
NLU
2. A populated ontology in RDF format
A snippet output for the sentence: “Kate is the mother of Mary.”……………………… <owl:FunctionalProperty rdf:ID="hasFather"> <rdf:type rdf:resource="http://www.w3.org/2002/07/owl#ObjectProperty"/> <rdfs:range rdf:resource="#Man"/> <rdfs:domain rdf:resource="#Person"/> <rdfs:subPropertyOf rdf:resource="#hasParent"/> </owl:FunctionalProperty> <Person rdf:ID="someone2"> <hasMother> <Person rdf:ID="someone1"> <hasName rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >Kate</hasName> <hasSex rdf:resource="#female"/> </Person> </hasMother> <hasName rdf:datatype="http://www.w3.org/2001/XMLSchema#string" >someone2</hasName> </Person></rdf:RDF>
![Page 17: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/17.jpg)
DAFs
DAFs are either triggered byBeing on the top of the stack Through index term matching
Each DAF has an associated set of terms associated with it (semantic categories)
![Page 18: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/18.jpg)
Dialogue Manager Strategy
Very simple for now, discusses variety of things in and about the photos People - their names, ages and relationship to the
user Location of the photo and when it was taken and
for what purpose
DM strategy at present connects all photos that contain the same person and discusses the similarities and differences.
![Page 19: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/19.jpg)
DAFs
Created using a GUI editor
Set of nodes and arcsA node represents a dialogue stateEach arc contains a test and an associated
action
![Page 20: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/20.jpg)
DAF Editor
Each node represents a dialogue state
Each arc contains a test with an associated action
![Page 21: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/21.jpg)
The main knowledge base will be an ontology which will hold all the world knowledge. There is another ontology which holds the current session knowledge. We chose ontologies to represent the knowledge because in the same file we have a structure, instances and logical rules.
The tests invoke rules predefined in the ontology. If the rule evaluates to true, then the relative action is invoked, otherwise not. Like this, a person creating the rule needs not know anything on programming.
The rules will look like ...
Person(?p) ^ hasAge(?p,?age) ^ greaterThan(?age,200) -> TooOld(?p)
They can be written and tested in Protege using a graphical interface.
DAF tests invoke protégé Rules
![Page 22: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/22.jpg)
SWRL http://protege.cim3.net/cgi-bin/wiki.pl?SWRLLanguageFAQ
SWRL is an acronym for Semantic Web Rule Language. It is intended to be the rule language of the Semantic Web. SWRL is based on the OWL Web Ontology Language. It allows users to write rules to reason about OWL individuals and to infer new knowledge about those individuals.
•SWRL rule contains an antecedent part, which is referred to as the body, and a consequent part, which is referred to as the head.
•Both the body and head consist of positive conjunctions of atoms
•SWRL rules are written in terms of OWL classes, properties and individuals
•SWRL supports Open World Reasoning
•One of the most powerful features of SWRL is its ability to support user-defined
• built-ins. A built-in is a predicate that takes one or more arguments and evaluates
• to true if the arguments satisfy the predicate.
・ Person(?p) ^ hasAge(?p, ?age) ^ swrlb:greaterThan(?age, 17) -> Adult(?p)
![Page 23: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/23.jpg)
Not all user input can be structured
If the NLU module cannot fit data into a structured form of the type we are expecting then the NLU module outputs an Annotation Set which has coordinates and properties for
those coordinates.
![Page 24: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/24.jpg)
Dialogue Manager uses the Knowledge Base to
Access individual pieces of KnowledgeStatic system knowledgeDynamic user instances
Make inferences about the knowledge *relies on the knowledge being structured
![Page 25: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/25.jpg)
NLG
InputThe input of NLG is a logical representation which has three properties: domain, type and Value (optional). To generate a sentence of requesting some general information of a photo,following representation will be sent to NLG:
Domain: PHOTOType: REQUEST_GENERALValue: EMPTY
![Page 26: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/26.jpg)
NLG
OutputA predefined utterance from a template file. Below is a snippet of the template file.(The $ will be replaced by the value specified in the logical representation.)
………………<Template domain="PHOTO">
<Sentences type="REQUEST_AGE"><Sentence>How old is $?</Sentence><Sentence>What is $'s age?</Sentence><Sentence>What is the age of $?</Sentence><Sentence>$ is how old?</Sentence></Sentences>
<Sentences type="REQUEST_RELATION"><Sentence>How do you know $?</Sentence><Sentence>What is your relationship with $?</Sentence></Sentences>……………..
![Page 27: Dialogue Manager Senior Companion](https://reader035.vdocuments.net/reader035/viewer/2022062301/56815215550346895dc0555d/html5/thumbnails/27.jpg)
additions to the DM (near future)
Add emotions to the SC system
Incorporate a cognitive model component
Machine Learning for improving the dialogue strategy