new interface design for human experience and expression
DESCRIPTION
New Interface Design for Human Experience and Expression. Sidney Fels Human Communication Technologies Laboratory Dept. of Electrical and Computer Engineering & Media and Graphics Interdisciplinary Centre University of British Columbia Vancouver, BC, Canada [email protected]. - PowerPoint PPT PresentationTRANSCRIPT
New Interface Design for Human Experience and
Expression
Sidney FelsHuman Communication Technologies Laboratory
Dept. of Electrical and Computer Engineering&
Media and Graphics Interdisciplinary Centre
University of British ColumbiaVancouver, BC, Canada
Human Communication Technologies Lab
• HCT lab started in 1998– Focus on human experience
• Intimacy, Embodiment• Aesthetics of relationships• Communication
– Interdisciplinary by practice and training– 8000ft2 of facilities, housed within ICICS complex
• MAGIC started in 1991– Focus on Media technologies and graphics
• Research• Industrial• Cultural• Education
– 2000ft2 of facilities housed in the Forest Sciences building
hct.ece.ubc.ca
Research Directions
• Three main areas of research– New interface design– Art, Music and Expression– Modeling, Tools and Graphics
Example Projects at HCT• New Interfaces
– Interaction with Large Display Surfaces• T. Tang
– Cubee: a 3D display• Ian Stavness & Florian Vogt
– Affective communication interface• 2 Hearts musical system (G. McCaig)• Interactive Yoga System (xxxx)
Example Projects at HCT• Art, Music and Technology
– Sound• Tooka• Tongue’n Groove• D’Groove
– Interactive Art• Iamascope• Swimming Across the Pacific• Plesiophone• 1,001,001 Faces• Sound Room & Sound Weave• 1 Kingsway: Flow
– Performance• Waking Dream• Forklift Ballet• Laika Space Program
Example Projects at HCT
• Modeling, Tools, Graphics– Biomechanical Modeling for
Articulatory Speech Synthesis
• large team of people
– Parallel Distributed Camera Array & Local Positioning System
• large team of people
New Interfaces, Art & Technology
Iamascope• Interactive multimedia
artwork• Participants are put inside
a large kaleidoscope• Participants movements
also map to music
Iamascope System
KaleidoscopicImage
ImageProcessing
Vision-to-music subsytem
ActiveVideo
RegionVideo
Camera
speakers
VideoImage
Kaleidscope subsytem
TextureMapping
TextureMemory Music
Production
MusicSynthesizer
170” Video Projector
Iamascope Video
Overview
Swimming Across the Pacific
• Swimming Across the Atlantic– Queen ElizabethII– Misheff, 1982– Southampton to New York
• Contemporary vision– Swimming in an airplane– Los Angeles to Tokyo
• Created swimming apparatus– Exhibition– Audience participation– Team of 8 people
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aSorenson Video 3 decompressorare needed to see this picture.
Tooka: A Two Person Flute
• Create musical instrument that needs two people to play
• Explore human intimacy
How Tooka Works
VolumePitch
Capture/SustainOctave
Pitch Bend
Vibrato
Cubee: 3D display
Hardware
Cubee Scenes
• Static object glued to box– e.g. teapot display case
• Static scene fixed in world– e.g. volumetric cut-plane
• Dynamic objects inside box– e.g. ball rolling through 3D maze
Glove-TalkII• Translates hand gestures to speech
– like a musical instrument
• mapping partially learned• speech is
– intelligible– expressive– slower than normal
Spectrum of Gesture-to-Speech Mappings
ArtificialVocalTract
PhonemeGenerator
FingerSpelling
SyllableGenerator
WordGenerator
Von
Kem
pel
en (
1790
)
Bel
l & B
ell (
1880
)
Du
dle
y et
al.
(193
9)
Fel
s &
Hin
ton
(19
94)
Kra
mer
& L
eife
r (1
989)
Fel
s &
Hin
ton
(19
90)
10-30 100 130 200 500
approximate time/gesture for connected speech
(msec)
Glove-TalkII Vocabulary• Loosely based on articulatory model of speech• Vowels
– open configuration of hand represents open vocal tract– X,Y position determines vowel sounds (like tongue)
• Consonants– constrictions in hand represent constriction in vocal tract
• Stop consonants produced with ContactGlove• Volume controlled with foot pedal (air pressure)• Pitch controlled by Z position of hand (vocal cord
tension)
SpeechOutput
Glove-TalkII System
Foot Pedal
x,y,z, roll, pitch, yaw(60 Hz)
10 flex angles4 abduction angles
thumb and pinkie rotationwrist pitch and yaw
(100Hz)
Right Hand Data
ContactSwitches
Preprocessor
Fixed PitchMapping
V/C DecisionNetwork
VowelNetwork
ConsonantNetwork
Fixed StopMapping
Synthesizer
CombiningFunction X
• 3 neural nets• Output: Parallel Formant Speech Synthesizer
– ALF, F1, A1, F2, A2, F3, A3, AHF, V, F0– 100 Hz, 6 bit quantization [0, 63]
Glove-TalkII VIDEO
Alphabet
Numbers
Sam I Am
Vocal Sounds
Intimacy and Embodiment
• Want interfaces that feel “good” to use
• Humans and machines intimately linked– degree of intimacy supported may
determine success• Types of relationships:
– human to human– human to machine
Intimacy• Intimacy is a measure of match between the
behaviour of an object and the control of that object.– extension of “control intimacy” from electronic
musical instruments analysis (Moore, 1997)
• High intimacy implies:– object feels like an extension of self– satisfaction derives from interacting with object– emotional expression flows
• requires cognitive effort to prevent
• Relationship dynamics– Aesthetics of interaction
Intimacy: Embodiment vs. Disembodiment
Case 1: Object disembodied from Self
Case 2: Self embodies Object
Case 3: Self disembodied from Object
Case 4: Object embodies Self
self object
objectself
objectself
object self
response
control
reflection
belonging
Aestheticsof Interaction
Directions
• How to achieve intimate experiences?– Two user-centric techniques:
• mirrors– Eliza (Weizenbaum, 1966), Iamascope (Fels &
Mase, 1999), Wooden Mirror (Rozin, 1999), bots
• masks
– many other complex techniques
Summary: HCT Activities• Modeling, Graphics and Tools
– ArtiSynth for articulatory speech synthesis– OpenVl for providing image processing infrastructure
• New Interface Design– approach to design using intimacy
• Art, Music and Technology– explore expressive directions of technology