1 layered avatar behavior model and script language for the object interaction jae-kyung kim...

1

Layered Avatar Behavior Model and Script Language for the Object

Interaction

Jae-Kyung Kim

2007-08-10

[email protected]

2

Contents

Part I : Ph.D Degree Dissertation– Introduction

– Related Works

– Research Goals

– Layered Avatar Behavior Model and Script Language• Avatar-Object Interaction Model

• Layered Script Language Approach

– Empirical Evaluations

– Conclusion

Part II : Current Research Interests– Avatar Augmented Annotation Interface for e-

Learning

3

Introduction

Recent studies on human character(Avatar) animation are mostly focused on realistic motion representation– ex) Facial Animation, synchronization between lips

movement and speech text, hair animation, so on.

Creating human character motion by motion capture technique– Very useful for creating human motion, and low-

level animation

Many researches about avatar motion control mainly care about realistic motion– Our main goal is to provide simple user interface by

controlling avatar motion at high-level concept

4

Low-level Animation– Classic and basic control technique– Rotate & move every parts of avatar body – Depending on animator’s skill, any desired motion can

be created in detail manner– However, it is very hard work for unskilled user to

control and create avatar motion– 3DS Max, Maya, Motion capture

Related Works(1/2)

5

Related Works(2/2)

Avatar Script Languages– Represents avatar body and facial motion in

language format (eg. XML)– Control synchronization and intensity of behaviors

through parameters– Most scripts use pre-defined general behaviors– Difficulty in creating script due to complicated

parameters and synchronization– Some scripts stick to specific format for an engine,

possibly causing low compatibility– AML, CML, XSTEP, TVML, etc.

6

Research Goals & Motivation

1. Create animation scenario script with less effort and time in a specific application domain• Pre-defined general behavior can’t support all

application domains

• More abstract representation is required for authoring script

2. Explicitly separate user interface and animation engine for extensibility and compatibility• Existing scripts support user interface and animation

engine level at the same time

• Standard format and structure is needed

7

Layered Avatar Behavior Model and Script

Overall structure

Task-LevelBehavior Script


Avatar-Object Interaction ModelAvatar-Object

Interaction Model

High-levelMotion ScriptHigh-level

Motion Script

Primitive Motion Script


1st Translator(Task-to-High)1st Translator(Task-to-High)

GeometricInformationGeometricInformation

2nd Translator(High-to-Primitive)

2nd Translator(High-to-Primitive) Primitive

Motion ScriptPrimitive

Motion Script

AvatarLow-levelAnimation

Data


Data

ObjectLow-levelAnimation

Data


Data


Data


Data


Data


Data

Application Application

Context MenuInterface(Display)


UserUser

Domain InterfaceLayer

Motion SequenceLayer

Recording

Translating

Translating

ApplicationLayer

8

Avatar-Object Interaction Model(1/2)

Basic Object Structure– Proposed model has multiple objects below the

ObjectList element– An object is composed of three elements

• Context : object state, access privilege, controlpoint and domain type

• ExecutableBehaviors : all executable behaviors for user interface menu

• MotionList : motion sequence to complete a behavior

ExecutableBehaviors

Object+

ObjectList

Context MotionList

Basic XML DTD sturcture

Avatar-Object Interaction Model(2/2)

9

Avatar-ObjectInteraction Model

Context

ExecutableBehaviors

MotionList

Domain

AcessGroup

ControlPoint

State

UserBehaviorsSelection

TaskPlanning

Selectable Behaviors

Behaviors List

Motion List

Select a Behavior

Generating Motions

…

10

Context Element(1/5)

State Element– Describes internal object states by pair of variable and

value attributes– Variable has affective motion and current state

methods• Affective motion defines post-state value, and

corresponding motion which changes the state• Current state function is defined in XSLT script that

processes state element to make boolean result output of current state value

– Behavior menu and motion sequence are decided according to changes of state variables value

11

Affective MotionPostStateclosed

State FunctionsIsClosed

Variable : DoorState

Event Motion

Result State

V V

V V

V

DoorObject

Interaction StateChanges

IF – (1) Event Motion = Affective Motion– (2) State Function = TRUE

THEN– (1) Make up of Variable list which satisfy conditions– (2) Set each variable Value to Post_State– (3) Output boolean result by State FunctionValue

closed

MotionNameclose‘close’

‘True’


12


ControlPoint Element– Spatial reference points around a object where avatar

stands and interact with it– Depending on the behaviors and object states, a

object may have various points– A point is consisted of position, direction and contact

elements– The elements has coarse references to represent

spatial location at high-level• Human readable• Independent from physical geometric object

models– Used for behavior sub-tasking and motion sequencing

13

Context Element(4/5)– Position

• Relative 5 basic and 4 composed positions for each objects• Standard axis is –z for front on zy-plane

– Direction• Object relative 4 basic and composed directions to determine avatar’s

orientation• Rotation axis is y and zero degree for forward direction

– Contact• designate a specific part of the object when the avatar makes contact

with the object • The corners of the bounding box of 3D objects and the center point of

the surfaceElement Reference Values

PositionFront, Behind, Left, Right, Center

Left&Front, Right&Front, Left&Behind, Right&Behind

DirectionBackward, Forward, Left, Right

Left&Forward, Right&forward, Left&Backward, Right&backward

Contact

Front, Behind, Left, Right, Top, Bottom, Center

Left&Front&Center, Left&Front&Top, Left&Front&Bottom,

Right&Front&Center ….

Position Directionx

z

yContact

14

Access Group Element– Object behavior can be accessed by several user types– Each user type has different purpose and usage– Each group has different access to behavior interface

DomainType Element– Not only internal state of object, but also external domain

state can effect behavior interaction.– The same object can behave as a different semantic

object• e.g. A car object in traffic simulation domain and

assembly training domain– DomainType element defines name of domain for

checking current domain type.


15

Layered Script Language

Overall structure



Avatar-Object Interaction ModelAvatar-Object

Interaction Model

High-levelMotion ScriptHigh-level

Motion Script



1st Translator(Task-to-High)1st Translator(Task-to-High)

GeometricInformationGeometricInformation

2nd Translator(High-to-Primitive)

2nd Translator(High-to-Primitive) Primitive

Motion ScriptPrimitive

Motion Script


Data


Data


Data


Data


Data


Data


Data


Data

Application Application



UserUser

Domain InterfaceLayer

Motion SequenceLayer

Recording

Translating

Translating

ApplicationLayer

16

Task-level Behavior Script(1/2)

Task-level Behavior Script– Presents object and avatar behaviors using XML DTD– Records and saves user action in temporal sequence– Extensible representation for behavior set

• Objects in a domain makeup the usable behavior set

• Cf) existing script which has pre-defined behavior DTD– Independent from rendering environments

• Doesn’t present appearance of avatar model, positional information of virtual objects, so on.

17

Task-level Behavior Script(2/2)

Definition of Task-level Avatar-Object Behavior Script– Template-based Avatar-Object Behavior Representation

• Domain behavior set is flexibly defined by object instances

• Task-level Behavior = (Object, Executable-Behavior, Narration)

Object Behavior NarrationTask-level Behavior

Script =

Door Enter

Text Highlight

Computer

Prev.pageScreen Point

Computer

Next page

Hello I’m...

DomainObject Pool

User Selection User Text Typing

18

High-level Motion Script(1/2)

High-level Motion Script– Bridges between task-level script and primitive motion

script

– Independent from both interface domain and application domain

– Parameterized motion support

• Synchronization, speed, target, repeat, intensity, decay, etc

• Abstract values rather than physical

19

High-level Motion Script(2/2)

Structure of high-level script DTD


Primitive motion– Supported by rendering engine or motion library

– Represented in physical parameter valuesex) Geometric values such as coordinates of avatar and

objects, radian value of avatar direction, etc

– Parameters are transmitted to animation engine to play animation scenario on screen

21

Task-level Behavior Script Translator– Translates task-level behavior script into high-level

motion script– Consisted of motion sequences to perform task-level

behaviors• Motion planning by task planner and generate the

sequences ex) <get target=“box”/>

1. <highLevelMotion name=“walk” …> 2. <highLevelMotion name=“bend_body”…> 3. <highLevelMotion name=“grab”…>

– Uses logical application domain knowledge– According to formal translation model, script of each

domains is converted to a high-level motion script

Script Translators(1/4)

22


Formal Task Translation Model– Identification of target

• find location of target object(Lo)

– Locomotive motion (Ml)

• identification of present avatar location (La)

• spatial distance between La and Lo

• generate Lm, if La ≠ Lo

– Manipulative motion (Mm)

• generate Mm for Lo

– Verbal information (Vi)

• generate verbal speech for avatar behavior if available• Speed and intensity parameters

• parameterize Mm and Ml for speed, intensity and duration

23


– Procedure modules

SpatialInformationGenerator

SpatialInformationGenerator

Temporal InformationGenerator

Temporal InformationGenerator

LocomotionGenerator

LocomotionGenerator

IntensityController

IntensityController

Task-levelBehavior

Task-levelScript

Translator

Ex) task-level behavior script<task name="bye" target="students">Let's do it next time!</task>

High-levelMotion

Ex) Generated high-level motion<highLevelMotion name="wave" type="gesture"> <spatialParam> <direction type="object"> <objectName>students</objectName> </direction> </spatialParam> <temporalParam> <speed>normal</speed> <repeat>default</repeat> <duration>default</duration> </temporalParam> <degreeParam> <intensity>normal</intensity> <decay>default</decay> <priority>normal</priority> </degreeParam> <verbalParam> <speech>Let's do it next time!</speech> </verbalParam> </highLevelMotion>

24

High-level Motion Script Translator– Major module : Object geometric information analyzer

• Calculates location, size, and direction of virtual objects

• Current status of avatar position and posture– Translates abstract parameters of high-level script to

physical values


Object GeometryAnalyzer

Object GeometryAnalyzer

Virtual Space

Object Geometry

High-level MotionScript High-level

MotionTrasnlator

High-levelMotion

Trasnlator

Primitive MotionScript

<primitiveMotion name="right_hand_point"> <target> <coord x="-200" y="-1" z="95"/> </target> <destination> <from x="-100" y="9" z="0"/> <to x="-100" y="9" z="0"/> </destination> <direction>-143</direction> <speed>15</speed> <intensity>15</intensity> <repeat>1</repeat> <duration>40</duration> <speech>this is …</speech></primitiveMotion>

25

System Implementation(1/2)

Creating a scenario script

26

System Implementation(2/2)

Running a scenario script in different environments– Different physical properties of virtual objects

– Applying same task-level script to other applications

OpenGL Application 2D Web Application(MSAgent)

Empirical Evaluation(1/3)

23 Users were asked to do the 20 tasks pre-defined in video material and text handouts

Features– Time to finish each test– Ratio of achieved tasks

Groups and Systems– G1 : Novice users(12)– G2 : Expert users(11)– S1 : Proposed method– S2 : Proposed method w/o context element– S3 : Alice system(comparatively lower behavior

representation) Analysis

– One-way ANOVA (analysis of variance)


Results

0

10

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30

Task

Ach

ieve

d Rat

e(%

)

Scripting Time

S3

S2

S1

0

10

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25

Scripting TimeTask

Ach

ieve

d R

ate

(%)

S3S2S1

Group 1(Novice)

Group 2(Expert)


Lesson Learned– Using abstract task-level behavior, both a novice and

expert user achieved high accuracy and fast scripting performance.

• Expert users made the better result in Alice system than novice users

– Expert users showed a tradeoff in Alice system• Those of who spent much time made almost same task

achievement ratio to the proposed system• However, the proposed system could save much time

– Both group answered that context behavior menu helped in creating scenario

• Novice user prefers abstract behavior, while some expert user tends to prefer primitive motion for detail manipulation

30

Conclusion

Conclusion– The proposed method consisted of 3 layered script

languages and object interaction model– Avatar-object task-level behavior, abstracts from low-

level concept to make script easier– By separating interface and application, the script can

be reused in various environments– Novice user can create scenario with less effort and

prefer the proposed method

Current Research Interests :Avatar Augmented Annotation Interface for e-Learning

Introduction– E-Learning, web-based teaching (WBT), and distance

learning• learn in their own spaces and times

– Interactions between students and teachers provide more educational effect

• Without interactions, students become bored with online coursework

– Interaction techniques • Video and animation clips • Animated Pedagogical Agent• Annotations(Digital Ink)

31

Current Interests :Avatar Augmented Annotation Interface for e-Learning

Animated Pedagogical Agent– The main advantage of classroom instruction over

distance education• Persona Effect• Gestures performed in live instruction

– Many systems have been studied• CTA, MASH, Wizlow, PPP, Steve, and etc

– Creating these avatar-augmented presentations• Usually laborious and time consuming• Easy and intuitive authoring interface technique is

needed

32

Current Interests :Avatar Augmented Annotation Interface for e-Learning

Annotation (digital inking)– an important feature of a digital lecture or

presentation system– Annotation or Digital inking can capture the

knowledge between student and teacher and easily share and reuse material

– AnnotatED, OATS, Xlibris, Intelligent Pen, etc

33


Avatar augmented annotation– Enhances learning effects by integrating these two

functionalities : avatar and annotation– To author animated lecture contents– To mark up the presentation using freehand

annotations • Reduce costs of controlling avatar animation• Author and user created narrative scenarios

34

IntegratingFramework

FreehandAnnotationInterface

AvatarAnimationScenario

Author/User

End-User

Overall structure of the proposed system

ContextModel

Free-Hand

Gesture

Menu

Animation

Model

ScenarioScript

Original Web

Document

AnimatedContent

MotionLibrary

AnnotationModel

Author’sInterface

User’sInterfac

e



Main components– Annotation model

• Recognize free-hand annotation types• Menu-based annotation types

– Context model• Structure information of web documents

– Annotated position in Xlink/Xpath– HTML/XML tag information of annotated area

– Animation model• Animation list• Animation decision rules

– Assign animation to annotation, according to each annotation types defined in annotation and context models

36

Summary– Still at the early proposal stage

• Working on defining the models and implementing interfaces

• Continue it as an individual research during post-doc period

– Considering co-work with other related systems• Applying pen-based interface to OATS or

AnnotatED• Agent interface for semantic navigation or

recommendation system

37


1 layered avatar behavior model and script language for the object interaction jae-kyung kim...

Documents

avatar motion control

animation scenario script

human motion

facial motion

desired motion

lowlevel animation classic

facial animation

hair animation