reachmedia - massachusetts institute of...
TRANSCRIPT
![Page 1: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/1.jpg)
ReachMedia
on-the-move interaction with everyday objects
Assaf Feldman, Emmanuel Muguia Tapia, Sajid Sadi, Pattie Maesand Chris Schmandt
The Media LaboratoryMassachusetts Institute of Technology
![Page 2: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/2.jpg)
![Page 3: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/3.jpg)
Question
Can we merge the virtual world with the physical
world?
![Page 4: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/4.jpg)
Not enough
pixels
Too many clicks
An Interface Problem
![Page 5: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/5.jpg)
Requirements
• Natural and seamless interaction
• Hands and eyes free
• Socially acceptable
![Page 6: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/6.jpg)
IO
Input – RFID,Gestures,Keypad.
Output –audio, text
The smart phone as a general interface
![Page 7: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/7.jpg)
Touch based interaction with objects
![Page 8: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/8.jpg)
Gestures
Slight flick for menu navigation
Select
Next Previous
- Socially acceptable.
![Page 9: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/9.jpg)
Demo
![Page 10: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/10.jpg)
RFID
Multi modal Multi modal interface via interface via accessories accessories
MITe receiver
• Objects ID• Gestural input:
-Next-Previous-Select
RFID RFID Augmented Augmented
physical physical objectsobjects
Audio output:•Earcons •Voice menus
Interaction Interaction mediation via mediation via Mobile deviceMobile device
Object ID, user input
Audio & XML Content
Context Context ManagerManager
![Page 11: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/11.jpg)
Hardware - Sensors
Accelerometers (House_n MITEs
–Cheap
–Small
–Low power
• RFID– Cheap– Great Context
![Page 12: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/12.jpg)
• 25 features - Time domain feature extraction
– Cross correlation (3)
– Power (3)
– Turning points (18)
∑−
=
=1
0
21)(n
ii
an
aP
ji AA
jiji
AAAA
σσρ
),cov(),( =
Signal processing
![Page 13: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/13.jpg)
Classification
• Online (Real Time) classification
• Good accuracy using Naïve Bayes and HMMs
• Preliminary results ~98% accuracy: Previous Next Select Noise
Previous 0.97101 0.014493 0 0.014493Next 0 1 0 0
Select 0 0 1 0Noise 0.017241 0 0 0.98276
Trut
h
![Page 14: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/14.jpg)
The big picture
• Integrate digital world (information & services) and physical world (physical objects/environment)
• Make interfaces more responsive and proactive (objects & environment monitor user and (proactively) present information & services relevant to user’s current needs/interests)
![Page 15: ReachMedia - Massachusetts Institute of Technologyalumni.media.mit.edu/~emunguia/pdf/reachmedia.pdf · 2016-01-04 · ReachMedia on-the-move interaction with everyday objects Assaf](https://reader033.vdocuments.net/reader033/viewer/2022050520/5fa3c9682d57a245245a256e/html5/thumbnails/15.jpg)
Ambient Intelligence
3rd floor