mit general exams. grounding interfaces: shifting the body boundaries, 2008
DESCRIPTION
TRANSCRIPT
GROUNDING INTERFACESShifting the body boundaries
Oral Exam by Cati Vaucelle14th of November 2008
Committee:
Professor Hiroshi IshiiProfessor Edith AckermannProfessor Joe Paradiso
GROUNDING INTERFACES: Shifting the body boundaries �
INTRODUCTION
GROUNDING INTERFACES: Shifting the body boundaries �
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
GroundingTaking something that’s ephemeral and pulling it down into reality and foundation.
Grounding InterfacesPhysical interfaces that actively construct a relationship between the digital and the physical body of an individual.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Definitions
By Hannah HoneypieLiving (2008).
GROUNDING INTERFACES: Shifting the body boundaries �
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
The dolls become the body, and then becomes the video. The doll is a gesture object.
Grounding via body and visual perspectives.
Physical body Digital body
Video of the dollThe body holds the dolls
Picture This! (2007).
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
ExamplePicture This! a grounding object
GROUNDING INTERFACES: Shifting the body boundaries �
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
The child’s anthropomorphism makes the objects become part of the child’s physical body.
Combining the gestures with the objects grounds the child in his physical world while playing with a digital body. The physical body is then directly connected to the digital body via the doll (a gesture object).
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
ExamplePicture This! a gesture object
GROUNDING INTERFACES: Shifting the body boundaries �
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Semiotic squareGrounding Interfaces
GROUNDING INTERFACES: Shifting the body boundaries �
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Tangible interaction techniquesMain area with Dr. Hiroshi Ishii
Psychology, cultural artifacts and semioticsContextual area with Dr. Edith Ackermann
Technologies for body sensing and expression Technical area with Dr. Joe Paradiso
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Committee
GROUNDING INTERFACES: Shifting the body boundaries �
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Interaction techniques are key to pushing the way people engage with digital information.
Psychology is key to an interaction that is defined by the nature of human attention, cognition, and perceptual abilities.
Sensing technology is the key to translating a user’s environment as part of new applications.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Committee
GROUNDING INTERFACES: Shifting the body boundaries �
MOTIVATION
GROUNDING INTERFACES: Shifting the body boundaries 10
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Is it really so sad and dangerous to be fed up with seeing with your eyes, breathing with your lungs, swallowing with your mouth, talking with your tongue, thinking with your brain…
Why not walk on your head, sing with your sinuses, see through your skin….
Where psychoanalysis says, stop, find your self again, we should say instead, let’s go further still! Find your body without organs. Find out how to make it. It’s a question of life and death!
–Gilles Deleuze and Felix Guattari, A Thousand Plateaus: Capi-talism and Schizophrenia.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Inspiration
GROUNDING INTERFACES: Shifting the body boundaries 11
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Imagine when one can say:
“the technology delivered him from all his automatic reactions and restored him to his true freedom.”
Is it really so sad and dangerous to be fed up with seeing with your eyes, breathing with your lungs, swallowing with your mouth, talking with your tongue, thinking with your brain…
Why not walk on your head, sing with your sinuses, see through your skin….
Where psychoanalysis says, stop, find your self again, we should say instead, let’s go further still! Find your body without organs. Find out how to make it. It’s a question of life and death!
–Gilles Deleuze and Felix Guattari, A Thousand Plateaus: Capitalism and Schizophrenia.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Inspiration
GROUNDING INTERFACES: Shifting the body boundaries 1�
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
We have already partially discovered a body without organs by creating:
[] A digital copy of oneself.[] A copy that will coexist online starting from birth.[] A copy that will perpetuate its existence after death, living in a digital reality.
This copy will be our digital body, a body without organs.
The Human Speechome Project by Deb Roy. Record every minute of a baby’s life, 2006.
Socioscope by Alex Pentland, 2006.Track location and conversation patterns.
MyLifeBits by Gordon Bell, 2003.Capture everything and going paperless.
The goal of Lifelogging: to record and archive all information in one’s life.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Digital BodyBeyond Picture This!
GROUNDING INTERFACES: Shifting the body boundaries 1�
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
[] Google indexes our name and we can be found at any time!
[] Even after our death, we remain on the Google search engine!
[] If Google sentences us to an electronic death, we cease to exist online and we cease to be found!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Digital BodyLifecycle
GROUNDING INTERFACES: Shifting the body boundaries 1�
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
50 years agoProgressive learning under the family’s supervision.
NowVery fast “learning” curve. Children believe that one can be a champion of Karate without moving a finger. However there is almost no common point between physical and digital Karate.
Despite a learning curve, there is no limitation from the physical body.
ReferenceGauthier, J. M. and Moukalou, R. (2007) De la guerre des boutons a Harry Potter: Un siecle d’evolution de l’espace-temps des adolescents, pp 5-16 and pp153-173.
A champion of Karate!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Digital BodyLearning
GROUNDING INTERFACES: Shifting the body boundaries 1�
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
50 years agoProgressive learning under the family’s supervision.
NowVery fast “learning” curve. Children believe that one can be a champion of Karate without moving a finger. However there is almost no common point between physical and digital Karate.
Despite a learning curve, there is no limitation from the physical body.
You will never be denied the opportunity of being a digital Karate champion because you are physically weak!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Digital BodyLearning
ReferenceGauthier, J. M. and Moukalou, R. (2007) De la guerre des boutons a Harry Potter: Un siecle d’evolution de l’espace-temps des adolescents, pp 5-16 and pp153-173.
A champion of Karate!
GROUNDING INTERFACES: Shifting the body boundaries 1�
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
[] No need to interact through body language!
[] It might avoid miscommunication!
[] Now we coommunicate via our digital body, on social networks controlling the level of engagement and disengagement.
[] Construct your digital body *free of risk*!
With the computer, the relationship between dis-tance and proximity, direct communication, corporeal versus indirect and me-diation is transformed. (Gauthier and Moukalou, 2007).
Twins in tree by Rodney Smith, 1999.In face to face communication, distance between people can be an obstacle.
ReferenceHall, E. T (1973) The Silent Language. A Doubleday Anchor Book.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Digital BodyCommunication
GROUNDING INTERFACES: Shifting the body boundaries 1�
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
[] Children with atypical interests find other children with the same hobbies.
[] By connecting to someone else, one can relate and create bonds, can project onto someone else’s life and empathize.
[] Connecting with a digital body might sharpen our perception through sharing perspectives.
Moving Pictures, 2004.Children capture media to share with children who live in a different city.
ReferenceHarel, I. and Papert, S. (1991) Constructionism, Ablex Publishing Corporation.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Digital BodyConnection
GROUNDING INTERFACES: Shifting the body boundaries 1�
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Walking through the Internet requires:
[] being open to novelty[] being alert to distinction[] being sensitive to different contexts[] being aware of multiple perspectives
Scratch, Resnick et al. (2008).
ReferenceLanger, E. J. (1998) The Power of Mindful Learning. Published by Da Capo Press, 1998
The Internet teaches children to adapt themselves to new situations rather than providing them with conditional learning -- fitting themselves to an external norm (Langer, 1998).
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Digital BodyMindful Learning
GROUNDING INTERFACES: Shifting the body boundaries 1�
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
[] A digital body motivates us in exercising!
[] Joysticks that vibrate are ironic: they remind us of our body! As if the joystick said: Remember! You had a body!
[] This is not entirely grounding: this is the difference between a reminder that your hands exist and giving someone tools, asking them to craft a piece of furniture.
My virtual model!Free weight-loss calculator on a virtual model, Glamour magazine, 2007.
Wii Fit, 2008.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Digital BodyMotivating Feedback
GROUNDING INTERFACES: Shifting the body boundaries �0
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Delicate Boundaries by Chris Sugrue.Using human touch to dissolve the barrier of the computer screen.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Things get lost with de-materialization.
Where are the senses?
The digital world gives us power at the same time that it disconnects us from our physical body.
We connect to the world using our senses.
To have the benefits of technology without loosing our connections to the body, technology needs to bridge to our senses, grounding us in our surroundings.
Only being grounded in experience can give birth to new ways of living and new dreams.
GROUNDING INTERFACES: Shifting the body boundaries �1
OPPORTUNITIES
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
OpportunitiesPercepts - Concepts - Conducts
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Observation [] Current displays restrict vision to one depth of field, supporting visual fatigue and myopia.[] Stereoscopic vision rehabilitates the effects of looking at one depth of field.
MechanismTrain stereoscopy as new means of environment interaction.
OpportunitySee all surfaces in 3D with your own eyes!
Tri-Vision, a stereo camera, (1946-49).
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
PerceptsVision
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
ObservationSmell is key to past experience and is unexplored.
Mechanism [] Collect senses of smell to create and recall stories. [] Associate memories of smells, places and objects.
OpportunitySmell the grass by reading a book and remember an afternoon tea party!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
PerceptsSmell
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
ObservationThe spectrum of high frequencies that one can hear drops off with age.
MechanismEncode information at frequencies discernible by some populations and not others.
OpportunityHear secret messages that no one else can hear!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
PerceptsHearing
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
ObservationTouch interfaces between the physical space and the nervous system.
Mechanism [] Injectable electrodes at the peripheral limb where the leg has been amputated. [] Electrodes stimulate the nervous system, actuated by a prosthetic
OpportunityFeel the sand from a foot that doesn’t exist!
ReferenceAaron R., Herr H., Ciombor D., Hochberg L., Donoghue J., Briant C., Morgan J., Ehrlich M.(2006). Horizons in Prosthesis Development for the Restoration of Limb Function, Journal of the American Academy of Orthopaedic Surgeons. 14 (10): 198-204.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
PerceptsTouch
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
ObservationMemory aids could both allow us to focus on more important matters, or destroy our ability to recall.
MechanismDevelop a taxonomy of cognitive tasks to determine when cognitive aids liberate higher cognitive functions.
OpportunityThe computer does low level tasks while you create bridges between disciplines!
Externalizing body functions with technology might be liberating (Serres, M.).
People use pen and paper to simplify cognitive tasks! You could just lean on the structure of the environment to remember!
Putting brain, body and world together in epistemic action! (Andy Clark).
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
ConceptsMemory
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Observation [] Humans often outperform technology in pattern recognition. [] Technology outperforms humans in raw calculation.
MechanismAssist a player in becoming a better calculator, and defer to the person what she is uniquely expert in.
OpportunityBecome a superhuman computer!
A Go competition. MoGo (2008). MoGo beat Myungwan Kim who is an 8-dan master on a 19 x 19 board (most previous wins by computers were done on smaller amateur level 9 x 9 boards). Kin gave the computer a nine-stone handicap.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
ConceptsIntelligence as calculation
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Observation [] A majority of westerners think that as we get old, we don’t remember as well. [] Langer showed that the degree of memory loss is dependent on how we value aging!
MechanismStructure societal change in the conception of value in aging.
OpportunityRevisit every stereotype you can think of!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
ConceptsIntelligence as adaptation
GROUNDING INTERFACES: Shifting the body boundaries �0
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Observation [] Some role playing games have more than 10.9 million monthly subscribers! [] In RPG we are equipped with sonic hearing, super vision, abnormal strength!
MechanismUse virtual environments to ask people to rediscover what they can do with their senses in the real world.
OpportunityBe an everyday super hero!
Borderland by Lucien Alma and Laurent Hart, 2001.
“Borderland” is based on video game duels such as “Tekken” or “Mortal Kombat”. They use ordinary people as the characters: old ladies, tramps, workmen to fight each other against the backdrop of the suburbs of Paris that have become a desolated no-man’s land.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
ConceptsIdentity
GROUNDING INTERFACES: Shifting the body boundaries �1
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
ObservationUngrounded, “magical” interaction, romanticized via video games and TV, deny the power of real-life cause and effect.
MechanismCombine children’s physical limits with digital action and reward to reinstate the relationship between cause and effect.
OpportunityFeel your own body limits when you play video games!
50 years ago children had a direct understanding and relationship to cause an effect of a physical action.
In Harry Potter they use a “magic wand” to confront others!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
ConductsFeedback
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Observation50 years ago, kids with ADHD were given means to catalyze their energy (Golse). Now quick fix: pharmaceutical treatment.
MechanismKids with ADHD can focus if they vary the target of their attention in their own minds (Langer).
OpportunityPlay hopscotch in a museum and recall all the details of a Botticelli painting!
You are getting better day after day by Broadhong Design, 2006.
Designers envisioned the physical representation of the healing process as part of a pharmaceutical treatment.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
ConductsPerspective
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
ObservationWith social networks, we build “instant” friends. We are perpetually connected but out of touch!
MechanismSocial networks could better support realistic relationships by monitoring and reflecting activity between users.
OpportunityLive on Mars and have millions of real earth friends!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
ConductsFriendship
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
The link between the dematerialized digital body and sensory opportunities is:
Grounding
Creating a shift in perspectives, from a new discovery of the environment to playing with our senses.
Don’t just passively deliver information or stimulate the senses, actively construct a relationship between digital and physical body in the individual!
Squeeze Me! (2006) This vest simulates therapeutic holding to “ground” patients with sensory issues.
Grounding the body: the Me ++
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Things get lost with de-materialization.
Where are the senses?
The digital world gives us power at the same time that it disconnects us from our physical body.
We connect to the world using our senses.
To have the benefits of technology without loosing our connections to the body, technology needs to bridge to our senses, grounding us in our surroundings.
Only being grounded in experience can give birth to new ways of living and new dreams.
GROUNDING INTERFACES: Shifting the body boundaries ��
VISION
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
[] Between being nostalgic of the past and being a technophile.
[] The ones who innovate create new bridges between the past and the future.
[] Each time one transcends, one builds a hybrid with a nostalgic vision from the past/present and from the real/virtual.
Wallmaster & Home transporter by Antal Lakner, 1999.
Everyday activities are transformed into indoor sport activi-ties!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Where do I stand?
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
There are four consistent dimensions that comprise my work:
Stance in the world.Perspective.Time.Space and proxemics.
All these dimensions merge to ground the body in the physical environment while interacting with a digital body
STANCE IN THE WORLDI design mechanisms of interaction to enable users to position themselves in their environment.
PERSPECTIVEI design sensing technology to translate the undiscovered environment, and create a shift in perspective.
TIMEI aim to synchronize users with their internal motion and with life. Users take time and go through an iterative process of discovering / understanding / reflecting.
SPACE AND PROXEMICSI design systems for users to understand spatially their environment, where they are situated in relation to someone else and where they are located in regards to objects
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Dimensions in my work
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
A child projects his personal experiences while playing with toys. His stories reflect his stance in the world.
He wants to make a movie from the point of view of his Naruto action figure!
STANCE IN THE WORLDI design mechanisms of interaction to enable users to position themselves in their environment.
Lego people can even embed a scene drawn by Edward Hopper, Nighthawks, from 1942!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Picture This!The Secret Lives of Objects
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
The dolls become the body - a point of new perpective.
What if the toy had a visual perspective?
What if any objects could share their perspectives with a child?
Wouldn’t this open a new world to the child?
PERSPECTIVEI design sensing technology to translate the undiscovered environment, and create a shift in perspective.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Picture This!The dolls become the body
GROUNDING INTERFACES: Shifting the body boundaries �0
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Children methodically progress …
[] iterate back and forth to discover new visual angles
[] slowly appropriate the “new eye”
[] expand their visual sense
[] synchronize their senses
TIMEI aim to synchronize users with their internal motion and with life. Users take time and go through an iterative process of discovering / understanding / reflecting.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Picture This!Iterative process via visual feedback
GROUNDING INTERFACES: Shifting the body boundaries �1
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Children discover spatially their environment: [] where their body is situated in relation to someone else [] where they are located in regards to objects.
Picture This! is a gesture objects interface at the extension of natural play.
SPACE AND PROXEMICSI design systems for users to understand spatially their environment, where they are situated in relation to someone else and where they are located in regards to objects
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Picture This!Spatial understanding
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Semiotic squareGesture Object Interfaces
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Parkinson’s disease, a neurodegenerative disorder with primary symptoms occurring due to a loss of motor timing mechanisms in the brain.
What if patient with Parkinson’s disease could communicate to objects their need to be synchronized?
Would they be able to walk with accuratetiming and not fall?
STANCE IN THE WORLDI design mechanisms of interaction to enable users to position themselves in their environment.
Sir William Richard Gowers, 1886.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Hap-bitTalking to Objects
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
It has been shown that the delivery of an external auditory metronome, in home-based rehabilitation programs, significantly increases gait rehabilitation of patients.
Would it work with a haptic stimulus? Could it be customized?
Patients could alternate amplitude of pulse stimuli to alternate their body perspectives.
PERSPECTIVEI design sensing technology to translate the undiscovered environment, and create a shift in perspective.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Hap-bitAlternating body perspectives
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Patients can customize features of the stimulus presentation such as frequency and amplitude by pulse compressing the garment.
The customized haptic rhythm might allow the patient to resynchronize her body, providing her with a temporal grounding reference.
Hap-bit is a membrane knitted into the fabric to deliver a rhythmic stimulus.
TIMEI aim to synchronize users with their internal motion and with life. Users take time and go through an iterative process of discovering / understanding / reflecting.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Hap-bitTime: a grounding reference
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Our goal is to provide an interface for patients to modulate their neurological function as it relates to Parkinson’s motor timing.
We would like to design garments that support customizable intervention outside of the physical therapy laboratory.
SPACE AND PROXEMICSI design systems for users to understand spatially their environment, where they are situated in relation to someone else and where they are located in regards to objects
Neurologists investigated intrasubjective relationships among organs, the nervous system, and sensory experience through neural prosthetics (Cartwright & Goldfarb, 2006).
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Hap-bitThe haptic becomes part of the skin
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
The dolls become the body, and then becomes the video. The doll is a gesture object.
Grounding via body and visual perspectives. The child’s anthropomorphism makes the objects become part of the child’s body.
The body is then directly connected to the digital body (video of the dolls).
Physical body Digital body
Video of the dollThe body holds the dolls
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
SynthesisGrounding with gesture objects
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Physical bodyDigital body
Digital actuated clothing The body will wear theenveloping fabric
Hap-bit skecth, 2008
This interface grounds the patient in her physical world while being held by a digital body, the digital actuation of youself.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
SynthesisGrounding with a temporal reference
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Connect your body to an Avatar and feel your body limits as part of that connection!
My World of Warcraft Avatar, now level 70!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
SynthesisMore applications to come!
GROUNDING INTERFACES: Shifting the body boundaries �0
SENSING TECHNOLOGIES
GROUNDING INTERFACES: Shifting the body boundaries �1
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
How to exchange information between the digital and the physical body?
Let’s look at the gesture communication in Picture This!
ReferenceVaucelle, C. and Ishii, H. 2008. Picture this!: film assembly using toy gestures. In Proceedings of the 10th international Conference on Ubiquitous Computing. UbiComp ‘08, vol. 344. ACM, New York, NY, 350-359.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Exchange information
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Rehearsing mode: The live video feed comes from the camera attached to the doll and is continuously displayed on a screen in front of the child
Recording mode: If a doll wants to be in the video, the doll needs to move. If it wants to be recorded it has to move three times quickly and the doll’s conversation partner will start the recording.
Playback mode: To play back the movie, the two dolls have to be moved in synchrony, in essence, jumping horizontally together.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Gesture communicationModes to control Picture This!
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
[] In Picture This! we use a Piezo vibration sensor (PVDF). We distinguish between vertical and horizontal motions while our sensor is a single axis accelerometer --detects mechanical stress.
[] We detect small variations of the off-axis motion with the on-axis accelerometer and are able to categorize strong motion in one axis and weak motion in the orthogonal axis. Piezo Sensor
MiniSense 100 is a vibration sensor loaded by a mass to offer high sensitivity at low frequencies.
Impacts containing high frequency components will excite the resonance frequency (1oohz).
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Piezo vibration sensorTo distinguish between motions
In the piezo vibration sensor, crystal structures are stressed which generate voltage that can be converted to acceleration.
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Why use a Piezo film?A Piezo vibration sensor is low-cost, easy to use and it offers a nice signal, but it is sensitive to temperature variations.
Why not use optical flow calculation? Because the doll wears a camera, we could take advantages of the camera as input to detect when it is moving ...
The optical flow vector of a moving object in a video sequence.
The camera is on the doll.To be recorded the doll hasto shake! “film me!”
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Hardware versus softwarePiezo sensor vs optical flow calculation
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Why not use optical flow calculation?
[] we avoid the problem of line of sight from the optical flow calculation.
[] with a Piezo sensor, we can select the strong variations in a short period of time (not the medium variations that occur during play).
There is a problem with the optical flow method: the child can easily occlude the camera because she shakes the doll that will not record, thus does not have a direct feedback for motion detection occlusion.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Hardware versus softwareAvoiding the problem of line of sight
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
We can generalize Picture This! technology to new platforms
Using a software-based Picture This! can democratize film making with toys.
ExampleMum and Dad’s cherished objects become the platforms of a child’s perspective taking, i.e. imagining Picture This! on cell phones.
A Piezo vibration sensor can be a constraint, requiring the purchase of a peripheral for the toys.
We can be hopeful, lots of camera phones now have an integrated accelerometer!
Children already use stuffed animals cell phone holder!
Picture This! could easily be integrated within these character looking holders!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Generalizing Picture This!Picture This! on new platforms
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
Let’s look at other hardware methods to detect gestures with toys and wearable:
[] Tilt mechanical switch: ON/OFF.
[] Capacitive sensing: measuring the distance between an object and its reference point. Change in capacitance related to acceleration.
[] Piezo resistive effect: Resistance changes with acceleration.
[] Hall effect: Motion converted to an electrical signal detecting change in magnetic fields.
[] Heat transfer: Location of heated mass tracked during acceleration by sensing temperature.
[] Magnetoresistive: Material resistivity changes in presence of magnetic field
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Hardware methodsTo detect gestures
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
How to exchange information between the digital and physical body?
[] Sensing technology initiated with the physical body and output to the digital body:
e.g. gesture, touch, pressure.
[] Sensing technology from the digital body mapped onto the physical body:
e.g. actuation could be identified as “tiring”, “annoying” or “relaxing”. Actuation can be based on vibration; temperature: Peltier junction; pressure: pneumatics.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Mapping dataWhat kind of actuation to select?
GROUNDING INTERFACES: Shifting the body boundaries ��
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
How to find the right mapping between the digital and physical body?
[] User study: testing the mechanism of actuation.
[] Measuring the relationship between stimulus and sensation (psychophysics).
[] Allowing user customization of the technology: give users control and study it!
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Mapping informationMethods for mapping data
GROUNDING INTERFACES: Shifting the body boundaries �0
CONCLUSION
GROUNDING INTERFACES: Shifting the body boundaries �1
Oral ExamCati Vaucelle Tangible MediaMIT Media Lab14-11-2008
I design grounding interfaces that translate the undiscovered environment:
[] to create a shift in perspective[] to ground the user in her physical body
I actively construct a relationship between the digital and physical body of the individual.
I do not deliver information passively or stimulate the senses.
I create a shift in body boundaries from the digital to the physical body.
Part I - Motivation[] Inspiration[] Beyond Picture This![] Lifecycle[] Learning[] Communication[] Connection[] Mindful Learning[] Motivating Feedback
Part II - Oppportunities[] Percepts Vision . Smell . Hearing . Touch[] ConceptsMemory . Intelligence . Identity[] ConductsFeedback . Perspective . Friendship
Part III - Vision[] Where do I stand?[] Dimensions in my work[] Picture This![] Hap-bit[] Synthesis
Part IV - Sensing Technologies[] Exchange information[] Gesture communication[] Piezo vibration sensor[] Harware vs software[] Generalizing Picture This![] Other methods[] Mapping
Introduction Conclusion
Conclusion
GROUNDING INTERFACES: Shifting the body boundaries ��