the entertainment technology center presents: an extensible platform for interactive, entertaining...

29
The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will Bosley, Dave Culyba, Sabrina Haskell, Andrew Hosmer, TJ Jackson, Shane Liesegang, Seema Patel, Christine Skarulis, Peter Stepniewicz, Jim Valenti, Salim Zayat, and Brenda Harger Carnegie Mellon University August 1st, 2005

Post on 19-Dec-2015

218 views

Category:

Documents


2 download

TRANSCRIPT

Page 1: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

The Entertainment Technology Center Presents:

An Extensible Platform for Interactive, Entertaining Social Experiences with an

Animatronic Character

Will Bosley, Dave Culyba, Sabrina Haskell, Andrew Hosmer, TJ Jackson, Shane Liesegang, Seema Patel, Christine Skarulis, Peter Stepniewicz, Jim

Valenti, Salim Zayat, and Brenda Harger

Carnegie Mellon University

August 1st, 2005

Page 2: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Motivation• Goals• Related Work• Hardware

Robot Kiosk

• Software Control Software Content Authoring Tools

• Rapidly Developed Content• Future Work

OUTLINE

Page 3: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• What is the Interbots Initiative?• Recent Explosion in Consumer Robotics Industry• Popularity of AIBO and Robosapien demonstrate

that Entertainment Robotics is a viable Industry Unlike traditional theme park animatronics, novelty is in

interactivity and unique experience

MOTIVATION

Page 4: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

1. Leverage the success of household entertainment robotics to introduce interactivity and uniqueness to higher end custom animatronics

Develop a fully interactive, autonomous animatronic character that is both believable and entertaining

2. Give non-technologists the ability to author content for interactive animatronic experiences

Develop an extensible platform that allows artists to rapidly design and create complete, entertaining social experiences with animatronic characters

GOALS

Page 5: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Within academia, most similar to that of Breazeal et al Animatronic robot Leonardo centers around human-robot

collaboration and learning• Our robot, Quasi, is influenced by behavioral, emotional, and

environmental motivators, but does not replicate artificial intelligence

Interactive Robot Theatre featuring an anemone-like creature• Work differs in depth of interactivity

• Within industry, most similar to Disney Animatronics Lucky the dinosaur

• In many ways, we attempt to bridge academia and industry

RELATED WORK

Page 6: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• In 14 weeks a team of 6 interdisciplinary graduate students created: A believable animatronic character named Quasi A suite of software tools that allow for the development

for interactive social experiences with Quasi

• The platform allows for experiences to be developed quickly and easily. No programming or technical knowledge is required.

THE IBI PLATFORM

Page 7: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

Quasi the Robot

HARDWARE

Quasi’s Kiosk

Page 8: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

FABRICATION

Page 9: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

QUASI’S HARDWARE

Page 10: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

KIOSK HARDWARE

Page 11: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Control Software Character State Control

System (CSCS) Real-time Show Control

System (RSCS) cVision, Inter-Process

Communication (IPC), iButton Reader, MIDI Reader, Babble

• Content Authoring Tools Behavior Authoring Tool (BAT) Maya Plug-In Macromedia Director & Flash

THE SOFTWARE

Page 12: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Character State Control System (CSCS) Backend run-time decision mechanism for a character Depends on set of system inputs and network of related

states and transition rules (essentially a FSA) This behavioral model is stored in a database format Inputs are split into internal “factors” and external “facts”

Facts represent knowledge acquired from the character’s environment

Direct (e.g. distance reading from IR sensor) Indirect (interpreted, e.g. “someone’s standing in front of me”)

Factors represent character’s internal environment Emotional State and Motivations (happiness, aggression, loneliness,

cleanliness, attention)

CONTROL SOFTWARE

Page 13: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

A character’s behavioral database is (at the high level) a system of “superstates”

Original Quasi personality included Idle, Sleep, Greeting, Farewell, and GiveCandy superstates

Each superstate (except the default Idles) has a set of entry requirements

Superstate is entered when facts and factors meet entry requirements

Superstates also have “stay requirements” Superstates are not connected, when stay requirements

aren’t met CSCS finds all superstates that match entry requirements and randomly picks one, weighted by priority

CONTROL SOFTWARE

Page 14: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

Within each superstate is a network of substates

Traditional FSA, substates are connected by transitons

Upon entry to a substate: Adjustments are made to

internal factors Actions take place (e.g.

animations, lighting changes, sound, candy dispensing, video display, etc.)

After adjustments, all exit transitions considered. If none, CSCS exits current superstate.

CONTROL SOFTWARE

Page 15: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Real-Time Show Control System (RSCS) Actual software interface to hardware Receives message from CSCS specifying which

animation file to load, which DMX channels to play them on, and any sound files that need to be broadcast

Sends DMX data to servos, and any other DMX controlled hardware

Has ability to linearly blend animations together, and seamlessly transition from one animation to another

CONTROL SOFTWARE

Page 16: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• cVision Face tracking tool; implementation of Intel’s OpenCV face

tracking library Inter-Process Communication (IPC)

Message passing protocol that utilizes Inter-process Communication Library developed in 1994 at CMU by Reid Simmons

MIDI Reader Listens to input signals from sensors, converts them to IPC

messages and transmits them to CSCS iButton Reader

Allows Quasi to uniquely identify people through log-in Babble

Implementation of open source Sphinx voice recognition package developed at Carnegie Mellon. Enchanced to pass messages over IPC.

CONTROL SOFTWARE

Page 17: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Goal: Create content authoring tools that are so intuitive and

easy to use that no prior programming or technical skills are required

Accomplished By: Leveraging power of popular mass-market software that

artists are already familiar with Creating custom, graphical authoring tools

CONTENT AUTHORING TOOLS

Page 18: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Behavior Authoring Tool One of the most powerful components of IBI Platform Used prior to run-time Graphical application for composing networks of

substates and superstates Written in Macromedia Flash, interfaces to database

through SQL commands Provides visualization of each superstate, making it easy

to understand complete picture of a character’s personality

CONTENT AUTHORING TOOLS

Page 19: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

CONTENT AUTHORING TOOLSSuperstate Panel

Page 20: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Maya Plug-In Industry standard application for 3D modeling and animation Natural choice for opening up animation of animatronic

characters to 3D animatiors Plug-in is a MEL script that converts keyframes to DMX values

for appropriate DMX channels; values are stored in a file that is read in by RSCS and sent to servos

Not limited to joint rotations– can control anything in robot’s environment that can talk to DMX and is keyable to some attribute in Maya: lights, colors, etc.

Also created Animation Editor Tool (AET) For those unfamiliar with Maya Consists of a series of sliders for controlling joints and

lights

CONTENT AUTHORING TOOLS

Page 21: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

CONTENT AUTHORING TOOLSAnimation Editor Tool

Page 22: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Usasbility of Content Authoring Tools put to the test in Fall 2004 Building Virtual Worlds course at Carnegie Mellon Two week project cycles Interdisciplinary teams of 4-5 students (1 programmer, 1

texture painter, 1 3D modeler, 1 designer, and possibly 1 audio artist)

None of the students had prior experience with IBI Platform

Tools met expectations admirably– all teams were able to rapidly develop complete, entertaining experiences

RAPIDLY DEVELOPED CONTENT

Page 23: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

RAPIDLY DEVELOPED CONTENT Quasi’s Interactive Dreams

Addressed problem of how to keep Quasi engaging while allowing his motors to “rest”

Quasi’s Photo Kiosk Allows guests to have their

picture taken with Quasi E-mails picture to guest Adds picture to album Displays Quasi’s life story

Page 24: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Quasi-oke Finale of Fall 2004 BVW show, a spot typically reserved for the best piece of

the semester

RAPIDLY DEVELOPED CONTENT

Page 25: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

New portable cable driven robot

FUTURE WORK

Page 26: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

Guided Performance Interface

FUTURE WORK

Page 27: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Virtual Show Control System Sends DMX Signals to 3D model in Panda3D instead of

RSCS

• Character Authoring Tool Expands BAT to allow for plug and play hardware

FUTURE WORK

Page 28: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• Brenda Harger• Todd Bowers• Zac Pavlov

ACKNOWLEDGEMENTS• Dan Kanitz• The ETC• Art Institute of Pittsburgh

Page 29: The Entertainment Technology Center Presents: An Extensible Platform for Interactive, Entertaining Social Experiences with an Animatronic Character Will

• For more information please visit: www.interbots.org www.etc.cmu.edu

QUESTIONS?