control of avatar gestures

4 November 2000 Bridging the Gap Workshop 1

Control of avatar

gestures

Francesca Barrientosfbarr@cs.berkeley.edu

Computer Science DivisionUC Berkeley

4 November 2000 Bridging the Gap Workshop 2

Communicating via desktop VR

Communicate verbally using speech Communicate visually using avatars

Body poseBody movement

4 November 2000 Bridging the Gap Workshop 3

Project goals

Build system for controlling avatar gestures

Simple input device such as a pen and tablet

Understand expression using avatar body

How do we express ourselves using avatar bodies?How do we evaluate a user’s sense of expressiveness?

4 November 2000 Bridging the Gap Workshop 4

Control problems

Limited input and complex outputControl interface divides user’s attention

4 November 2000 Bridging the Gap Workshop 5

Nonverbal behaviorsDisplays

GestureFacial expressionGaze

Posture

FunctionsAugment/replace speechRegulate conversationIndicate emotion

CognitionAwarenessConscious control

4 November 2000 Bridging the Gap Workshop 6

Other approaches

Expression selection

Automaticanimation

Naturallanguageinference

Nivi waves to Harry.)

4 November 2000 Bridging the Gap Workshop 7

Gesticulation

Co-occurs with speech

Free form and made up in the moment

Meaning derived from combination of words and gestures

4 November 2000 Bridging the Gap Workshop 8

Demo of Doodler

4 November 2000 Bridging the Gap Workshop 9

Doodler: things that worked

Within a small range, tracking is intuitiveCan produce free form gesturesMovement seems expressiveControl is transparentNo hunting for controls

4 November 2000 Bridging the Gap Workshop 10

Doodler: limitations

Hard to form specific shapesNeed a way to switch between different mappingsHard to do gestures which involve different body parts seriallyLack of avatar proprioception

4 November 2000 Bridging the Gap Workshop 11

Speech independent behaviors

Emblematic gestureReplace wordsSymbolicArbitrary

Affective expressionEmotionState of being

4 November 2000 Bridging the Gap Workshop 12

Demo with writing

Using alphabet

4 November 2000 Bridging the Gap Workshop 13

Writing as control

Discrete symbols+

Continuously valued variables

Intentional and unconscious

4 November 2000 Bridging the Gap Workshop 14

Writing as control

Elements of discrete and continuous control

Letter selects a discrete choiceWay it is written selects values for

Specify multiple variablesExplicit control?Implicit control?Don’t have to look

4 November 2000 Bridging the Gap Workshop 15

Summary

SimpleContinuousImplicitFlowsAbstractUnwittingTransparent

ComplexDiscreteExplicitConsciousIntentionalSymbolic

4 November 2000 Bridging the Gap Workshop 16

Future work

Design and record gesture vocabularyMap variations in writing to variations in gesture animationFigure out how to analyze expressive aspects of control

4 November 2000 Bridging the Gap Workshop 17

Facing the Gap

How to think about expressionComputer sciencePsychologyHuman-computer interactionPuppetryNew culture

How to study it

4 November 2000 Bridging the Gap Workshop 18

Explore design space

4 November 2000 Bridging the Gap Workshop 19

Two control schemes

Continuous control with free form motionDiscrete control with more complex motion and multiple parameters

4 November 2000 Bridging the Gap Workshop 20

Other solutions continued

Analysis of textComicChat uses keywords, acronyms, punctuation, etc.

Semi-autonomous behaviorsBodyChat by Vilhjálmsson

Simple kinematic controlsSliders and similar widgets (e.g.. Slater)

Full body motion capture

4 November 2000 Bridging the Gap Workshop 21

Other solutions for nonverbal communication

Discrete choices (menus) of expressions

Usually affective (happy, sad, angry…)

Usually facialUsually used with chat environmentsExamples:

Emotion wheel in ComicChatPalaceGesture/Mimic panel in Vlnet