the entertainment technology center presents: an extensible platform for interactive, entertaining...
Post on 19-Dec-2015
218 views
TRANSCRIPT
The Entertainment Technology Center Presents:
An Extensible Platform for Interactive, Entertaining Social Experiences with an
Animatronic Character
Will Bosley, Dave Culyba, Sabrina Haskell, Andrew Hosmer, TJ Jackson, Shane Liesegang, Seema Patel, Christine Skarulis, Peter Stepniewicz, Jim
Valenti, Salim Zayat, and Brenda Harger
Carnegie Mellon University
August 1st, 2005
• Motivation• Goals• Related Work• Hardware
Robot Kiosk
• Software Control Software Content Authoring Tools
• Rapidly Developed Content• Future Work
OUTLINE
• What is the Interbots Initiative?• Recent Explosion in Consumer Robotics Industry• Popularity of AIBO and Robosapien demonstrate
that Entertainment Robotics is a viable Industry Unlike traditional theme park animatronics, novelty is in
interactivity and unique experience
MOTIVATION
1. Leverage the success of household entertainment robotics to introduce interactivity and uniqueness to higher end custom animatronics
Develop a fully interactive, autonomous animatronic character that is both believable and entertaining
2. Give non-technologists the ability to author content for interactive animatronic experiences
Develop an extensible platform that allows artists to rapidly design and create complete, entertaining social experiences with animatronic characters
GOALS
• Within academia, most similar to that of Breazeal et al Animatronic robot Leonardo centers around human-robot
collaboration and learning• Our robot, Quasi, is influenced by behavioral, emotional, and
environmental motivators, but does not replicate artificial intelligence
Interactive Robot Theatre featuring an anemone-like creature• Work differs in depth of interactivity
• Within industry, most similar to Disney Animatronics Lucky the dinosaur
• In many ways, we attempt to bridge academia and industry
RELATED WORK
• In 14 weeks a team of 6 interdisciplinary graduate students created: A believable animatronic character named Quasi A suite of software tools that allow for the development
for interactive social experiences with Quasi
• The platform allows for experiences to be developed quickly and easily. No programming or technical knowledge is required.
THE IBI PLATFORM
Quasi the Robot
HARDWARE
Quasi’s Kiosk
FABRICATION
QUASI’S HARDWARE
KIOSK HARDWARE
• Control Software Character State Control
System (CSCS) Real-time Show Control
System (RSCS) cVision, Inter-Process
Communication (IPC), iButton Reader, MIDI Reader, Babble
• Content Authoring Tools Behavior Authoring Tool (BAT) Maya Plug-In Macromedia Director & Flash
THE SOFTWARE
• Character State Control System (CSCS) Backend run-time decision mechanism for a character Depends on set of system inputs and network of related
states and transition rules (essentially a FSA) This behavioral model is stored in a database format Inputs are split into internal “factors” and external “facts”
Facts represent knowledge acquired from the character’s environment
Direct (e.g. distance reading from IR sensor) Indirect (interpreted, e.g. “someone’s standing in front of me”)
Factors represent character’s internal environment Emotional State and Motivations (happiness, aggression, loneliness,
cleanliness, attention)
CONTROL SOFTWARE
A character’s behavioral database is (at the high level) a system of “superstates”
Original Quasi personality included Idle, Sleep, Greeting, Farewell, and GiveCandy superstates
Each superstate (except the default Idles) has a set of entry requirements
Superstate is entered when facts and factors meet entry requirements
Superstates also have “stay requirements” Superstates are not connected, when stay requirements
aren’t met CSCS finds all superstates that match entry requirements and randomly picks one, weighted by priority
CONTROL SOFTWARE
Within each superstate is a network of substates
Traditional FSA, substates are connected by transitons
Upon entry to a substate: Adjustments are made to
internal factors Actions take place (e.g.
animations, lighting changes, sound, candy dispensing, video display, etc.)
After adjustments, all exit transitions considered. If none, CSCS exits current superstate.
CONTROL SOFTWARE
• Real-Time Show Control System (RSCS) Actual software interface to hardware Receives message from CSCS specifying which
animation file to load, which DMX channels to play them on, and any sound files that need to be broadcast
Sends DMX data to servos, and any other DMX controlled hardware
Has ability to linearly blend animations together, and seamlessly transition from one animation to another
CONTROL SOFTWARE
• cVision Face tracking tool; implementation of Intel’s OpenCV face
tracking library Inter-Process Communication (IPC)
Message passing protocol that utilizes Inter-process Communication Library developed in 1994 at CMU by Reid Simmons
MIDI Reader Listens to input signals from sensors, converts them to IPC
messages and transmits them to CSCS iButton Reader
Allows Quasi to uniquely identify people through log-in Babble
Implementation of open source Sphinx voice recognition package developed at Carnegie Mellon. Enchanced to pass messages over IPC.
CONTROL SOFTWARE
• Goal: Create content authoring tools that are so intuitive and
easy to use that no prior programming or technical skills are required
Accomplished By: Leveraging power of popular mass-market software that
artists are already familiar with Creating custom, graphical authoring tools
CONTENT AUTHORING TOOLS
• Behavior Authoring Tool One of the most powerful components of IBI Platform Used prior to run-time Graphical application for composing networks of
substates and superstates Written in Macromedia Flash, interfaces to database
through SQL commands Provides visualization of each superstate, making it easy
to understand complete picture of a character’s personality
CONTENT AUTHORING TOOLS
CONTENT AUTHORING TOOLSSuperstate Panel
• Maya Plug-In Industry standard application for 3D modeling and animation Natural choice for opening up animation of animatronic
characters to 3D animatiors Plug-in is a MEL script that converts keyframes to DMX values
for appropriate DMX channels; values are stored in a file that is read in by RSCS and sent to servos
Not limited to joint rotations– can control anything in robot’s environment that can talk to DMX and is keyable to some attribute in Maya: lights, colors, etc.
Also created Animation Editor Tool (AET) For those unfamiliar with Maya Consists of a series of sliders for controlling joints and
lights
CONTENT AUTHORING TOOLS
CONTENT AUTHORING TOOLSAnimation Editor Tool
• Usasbility of Content Authoring Tools put to the test in Fall 2004 Building Virtual Worlds course at Carnegie Mellon Two week project cycles Interdisciplinary teams of 4-5 students (1 programmer, 1
texture painter, 1 3D modeler, 1 designer, and possibly 1 audio artist)
None of the students had prior experience with IBI Platform
Tools met expectations admirably– all teams were able to rapidly develop complete, entertaining experiences
RAPIDLY DEVELOPED CONTENT
RAPIDLY DEVELOPED CONTENT Quasi’s Interactive Dreams
Addressed problem of how to keep Quasi engaging while allowing his motors to “rest”
Quasi’s Photo Kiosk Allows guests to have their
picture taken with Quasi E-mails picture to guest Adds picture to album Displays Quasi’s life story
• Quasi-oke Finale of Fall 2004 BVW show, a spot typically reserved for the best piece of
the semester
RAPIDLY DEVELOPED CONTENT
New portable cable driven robot
FUTURE WORK
Guided Performance Interface
FUTURE WORK
• Virtual Show Control System Sends DMX Signals to 3D model in Panda3D instead of
RSCS
• Character Authoring Tool Expands BAT to allow for plug and play hardware
FUTURE WORK
• Brenda Harger• Todd Bowers• Zac Pavlov
ACKNOWLEDGEMENTS• Dan Kanitz• The ETC• Art Institute of Pittsburgh
• For more information please visit: www.interbots.org www.etc.cmu.edu
QUESTIONS?