gui: specifying complete user interaction soft computing laboratory yonsei university october 25,...
TRANSCRIPT
![Page 1: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/1.jpg)
gUI: Specifying Complete User gUI: Specifying Complete User InteractionInteraction
Soft computing LaboratoryYonsei University
October 25, 2004
![Page 2: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/2.jpg)
2
ContentsContents• Computer human user interfaces – Synthesis• Value of empathy in synthesis• Computer human user interfaces – Analysis• gUI research objectives• The role of the anthropomorphic interface metaphor• gUI data specification issues• Implementation issues• Results of the prototype system• Conclusion
![Page 3: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/3.jpg)
3
Computer Human User Interfaces – SynthesisComputer Human User Interfaces – Synthesis
• Humane CHIs (Computer Human Interfaces)– Embodied character agents (ECAs), Embodied
conversational agents, virtual conversational characters, virtual humans, talking heads
• Basic criteria for a standard markup language– Completeness, simplicity, consistency, intuitiveness,
abstraction, usability, standardization, evaluation
• VHML– An open specification available at the VHML web site– Two applications
MetaFace, an ECAThe Mentor systems, a dialogue management system
![Page 4: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/4.jpg)
4
Value of Empathy in SynthesisValue of Empathy in Synthesis• ECA with emotions or a personality: Increase the feeling of
familiarity
• Application areas with common anthropomorphic metaphors– Universities– E-commerce– Web guides– Traveling– Interactive games– Knowledge interactive companions
![Page 5: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/5.jpg)
5
Computer Human User Interfaces – AnalysisComputer Human User Interfaces – Analysis
• “Aware”: The user does not know how to perform the task
• Gestalt user interfaces (gUIs)– “gestalt”: An organized whole in which each part affects
every other part– The mind organizes events and situations as a pattern “big
picture”– The entities
Similarity, proximity, closure, continuity, membership
– Input: A combined “stimulus” of text, button clicks, and analyzed facial, vocal, and emotional gestures
– Recognition: Translate the stimulus into the intent
![Page 6: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/6.jpg)
6
Value of Empathy in AnalysisValue of Empathy in Analysis• “more complete” encapsulation of human interaction
– The empathic ability of an interface
– The extra functionality acquired
– An intuitive interface
• Overlooked empathic state of the virtual character– Attend: Considering the thoughts and feelings of the user
– Engage: Aligning actions, thoughts, and feelings with those of the user
– Value: Expressing the value of the user’s interaction
– Encourage: Expression encouragement for further interaction
– Parting: Suspending dialogue while user performs other tasks
– Available: Allowing the user to interrupt for interaction
gUIs
![Page 7: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/7.jpg)
7
gUI Research ObjectivesgUI Research Objectives• Formal specification, development, implementation,
evaluation, and standardization of a markup language is necessary to provide a stable, consistent base for both industrial use and future research into multi-modal Human-Computer interaction
• MPEG-4: Solutions to facial and body animation at the low level
• Require a unified standard architecture plus language to control/record the higher level human ECA interaction
![Page 8: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/8.jpg)
8
The Role of the Anthropomorphic Interface The Role of the Anthropomorphic Interface Metaphor (1)Metaphor (1)
• Anthropomorphic metaphor– To specify complete user interaction with an ECA– To use the analysis and synthesis of emotion and gesture
• Desktop metaphor– The most predominant metaphor used in computer systems– The metaphor’s likeness to our perception of the world
• “illusions”– Reduce cognitive overhead when users interact with an
anthropomorphic interface– gUI applications become an extension of real life
Waste paper basket on the Macintosh desktopThe humanity of talking heads
![Page 9: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/9.jpg)
9
The Role of the Anthropomorphic Interface The Role of the Anthropomorphic Interface Metaphor (2)Metaphor (2)
• Anthropomorphic web applications– Bryan and Gerchman
Browsing: The leisurely navigation along an arbitrary path Searching: Goal oriented finding specific information
– A knowledgable ECA (MetaFace) Knows that you are getting frustrated in web searching Offers assistance as a friend
• Transcription– Representation, transformation, and storage of analyzed emotions
and gestures
• Real-time interaction and data representation– Efficient storage and communication of the data– Intelligent television application: handle real-time emotions and
gestures to control the function of the TV
![Page 10: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/10.jpg)
10
gUI Data Specification Issues (1)gUI Data Specification Issues (1)• “Affect recognition”
– Proper gestalt understanding of the user requirements– “Non-verbal” behavior and “Non-verbal” communications
web site (http://www3.usal.es/~nonverbal/paper.htm)
• Temporal window– Differentiate short mixed emotions such as surprise before
anger or fear– Detect more lengthy emotions such as gradual happiness as
“understanding comes to someone”
• Specification of the gUI input data format– At the lowest level: Efficient binary format– At the higher levels: a more structured format aiding in
abstraction and cognitive understanding
![Page 11: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/11.jpg)
11
gUI Data Specification Issues (2)gUI Data Specification Issues (2)
![Page 12: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/12.jpg)
12
gUI Data Specification Issues (3)gUI Data Specification Issues (3)• Dynamic context transitions
• Time-stamping of emotion and gestures: Synchronizing the resulting XML document and providing context
• MPEG-4 face and body animation– Efficient communication standard– Emotion and gesture representation for real-time
applications
• Some problems in VHML data representation– Lack of a consistent set of emotions, and the ability to add
more– Lack of ability to show change in different sets of emotions
or just a single emotion– Lack of additional attributes for emotion besides intensity
![Page 13: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/13.jpg)
13
gUI Data Specification Issues (4)gUI Data Specification Issues (4)• A more realistic and extensible semantic structure
• Gesture input
![Page 14: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/14.jpg)
14
gUI Data Specification Issues (5)gUI Data Specification Issues (5)• Issues on specifying gestures
– A huge diversity of gestures for the human body to try and identify a base set for all analysis and synthesis applications
– Own associated information on each gesture: No standard parameters for all gestures
– Culturally dependent gestures
• A less naïve format
![Page 15: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/15.jpg)
15
Implementation Issues (1)Implementation Issues (1)• MetaFace framework
– An example ECA used to test the underlying theories– Based on MPEG-4 and VHML– The ability to vocalize and visually display emotion and
gesture– Dialogue management tool language (DMTL)
• Evaluate tag: Inference the context
![Page 16: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/16.jpg)
16
Implementation Issues (2)Implementation Issues (2)
![Page 17: GUI: Specifying Complete User Interaction Soft computing Laboratory Yonsei University October 25, 2004](https://reader035.vdocuments.net/reader035/viewer/2022062717/56649e425503460f94b34c86/html5/thumbnails/17.jpg)
17
ConclusionConclusion• Challenges
– The overlap of text input, gesture, and emotion as one coherent stimulus
– Standardization of emotion and gesture representation
• Complete user interaction with gestalt user interface– Use anthropomorphic metaphors– Reduce the conceptual load– Mentor and MetaFace research project