brain, body, and bytes chi 2010 workshop presentations
Post on 18-Oct-2014
5.493 views
DESCRIPTION
Brain and Body Interfaces (BBI) were discussed in this workshop. Its goal was to provide a platform for creating synergies between two related and emerging HCI disciplines (PC and BCI). Find out more at the workshop website: http://www.eecs.tufts.edu/~agirou01/workshop/TRANSCRIPT
![Page 1: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/1.jpg)
![Page 2: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/2.jpg)
BBB Workshop Overview
Audrey Girouard, Erin Treacy Solovey,Regan Mandryk, Desney Tan,
Lennart Nacke, Robert J.K. Jacob
![Page 3: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/3.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BBB Workshop: Bringing Together BCI& Physiological Computing
3
![Page 4: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/4.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Goals
1. Provide a platform for creating synergies between two related and emerging HCI disciplines (PC and BCI).
2. Kick-start BBB research in HCI by identifying key research questions and application areas
3. Enhancing HCI methodologies by adding these new methods and techniques to the toolbox of HCI researchers
4
![Page 5: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/5.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
About Us
5
![Page 6: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/6.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
About you
• 25 Participants
• 9 Brain papers• 19 Body papers
• X EEG papers• Y fNIRS papers• Z EMG papers• A GSR• B Eye tracking
6
32%
68%
Brain Body
![Page 7: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/7.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
7
<on your tables>
![Page 8: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/8.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What terms/acronyms should we use to describe our work?
Psychophysiological computing seems a bit long… BBB?BBI?
Other suggestions?
8
![Page 9: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/9.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Today’s Schedule
9
09:00-10:30 Introduction of organizers & participants, CHI madness style
10:30-11:00 Break
11:00-12:30 Group Discussion (assign groups + topic)
12:30-14:00 Lunch
14:00-15:30 Small group discussion (1h of discussion + 30 minutes of reporting)
15:30-16:00 Break
16:00-17:00 Report from small groups, recap, group discussion – barriers to publication in HCI
![Page 10: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/10.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Staying connected
10
Website http://www.eecs.tufts.edu/~agirou01/workshop/
Twitter http://twitter.com/bbbCHI2010
Twitter Hashtag
#bbbCHI2010
Facebook http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and-Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786
Flickr http://www.flickr.com/groups/bbbchi/
Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological-user-interaction
![Page 11: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/11.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
And now… 2 Minute Introductions
• About your research:– Application or domain space? (e.g. video games)
– Signal you are measuring? (e.g. fNIRS)
– Device/hardware you are using? (e.g. ISS, Inc)
• What would you like to get out of workshop?– What problems are you facing?– What do you see in common with others here?– What device would you like to know more about?
11
![Page 12: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/12.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
List of Presentations1. Adriane Randolph2. Krzysztof Z. Gajos3. Saraswathi Bellur4. Danny Plass-Oude Bos5. Kai Kuikkaniemi6. Mitchel Benovoy7. Yee Chieh Chew8. Martha Crosby9. Jan Kallenbach10. Eva Oliveira11. Lucia Filgueiras12. Marc Grootjen13. Melody Moore Jackson14. Johann Schrammel15. Daniel Sjölie
12
16. Katarzyna Wac17. Feng Tian18. Veronica Zammitto19. Johanna Octavia20. Bob Wray21. Marjolein van der Zwaag
![Page 13: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/13.jpg)
Brain-Computer Interfaces as Child’s Play: Using Mental Training to Improve
Motor Skills in Children
Adriane B. Randolph, Ph.D.Ashley Ingraham
Kennesaw State University
![Page 14: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/14.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline• Research Position
– Examine ties between sensorimotor area and physical ability to uncover a training tool to aid children’s motor skill development
• Pilot Study– 3 able-bodied adults (2 Male, 1 Female)– Acquire baseline Wii tennis score– Conduct mental training with BCI2000 mu training protocol for 5 days– Retest Wii tennis score for improvements– Mixed results
• Future Work– Extend study with 20 able-bodied adults (currently in progress)– Analyze link between training and baseline ability– Test with elementary-aged children
14
Participant Session One Session Five
AZ 98 60
CM 70 64
MW 103 118
![Page 15: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/15.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Screening for MuAshley prepares participant to use mental imagery of hand and foot movements to determine his best channels for use in mu BCI training before he plays a match of Wii Tennis.
15
![Page 16: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/16.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Adriane B. Randolph, Ph.D.Kennesaw State University BrainLab
(770) 423-6083 [email protected]
http://coles.kennesaw.edu/brainlab/
16
![Page 17: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/17.jpg)
Understanding How to Design ComplexBrain-Controlled Applications
Krzysztof Z. Gajos
![Page 18: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/18.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?• Basic operations
18
![Page 19: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/19.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?
19
![Page 20: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/20.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?
20
![Page 21: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/21.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?• Basic operations
21
![Page 22: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/22.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?• Interaction between primary task and BCI
performance
22
![Page 23: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/23.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
How to Design Brain-Controlled Applications?• Error prevention, recovery, and actions
with side effects
23
![Page 24: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/24.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Krzysztof [email protected]://www.eecs.harvard.edu/~kgajos/
24
![Page 25: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/25.jpg)
Psychophysiological Responsesto Media Interfaces
S. Shyam SundarSaraswathi Bellur
Media Effects Research Labhttp://www.psu.edu/dept/medialab
Pennsylvania State University
![Page 26: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/26.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Biopac MP 35 & MP 150 ACQknowledge software
26
Skin Conductance (EDA) Heart Rate (ECG) Facial EMG Brain Wave (EEG) Eye Movement (EOG)
Application AreasInterface presentationInformation Processing Usability Testing & Evaluation User Experience
![Page 27: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/27.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 27
Physiological Measures: Affect, Arousal & Attention
1. EEG: Bipolar recordings Occipital Alpha Blocking
2. Skin Conductance (EDA) Tonic arousal levels
3. ECG: Heart Rate recording
OrientingResponse viaBPM change
4. Facial EMG:Corrugator &Zygomaticrecordings
![Page 28: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/28.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Theoretical Frameworks• The Media Equation: Mediated Life = Real Life (Reeves & Nass,
1996). • Structural aspects of media technologies: Automatic or
Controlled processing? Voluntary or Involuntary responses?• Specific cues triggered via interactions with four types of
media affordances, MAIN model (Sundar, 2008)
28
M: MODALITY
I: INTERACTIVITY
A: AGENCY
N: NAVIGABILITY
Animation
Customization
Contingency
PerceptualBandwidth
InformationScent
UGC & Sourceness
Heuristics
![Page 29: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/29.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Saras Bellur ([email protected])S. Shyam Sundar ([email protected])
29
Media Effects Research Labhttp://www.psu.edu/dept/medialab
Pennsylvania State University
![Page 30: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/30.jpg)
BCI for Games
Danny Plass-Oude BosBoris Reuderink, Bram van de Laar, Hayrettin Gürkök, Christian Mühl,
Mannes Poel, Anton Nijholt, Dirk HeylenHMI, University of Twente
![Page 31: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/31.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Target Group: the general population
31
![Page 32: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/32.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BCI: issues
32
![Page 33: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/33.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BCI: new information, new features
33
![Page 34: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/34.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Acceptance: Usability & Realistic Settings
34
![Page 35: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/35.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Subjects Users
35
![Page 36: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/36.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
[email protected]://hmi.ewi.utwente.nl/~oudebos
36
![Page 37: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/37.jpg)
Designing Biofeedback for Games and Playful Applications
Kai KuikkaniemiHelsinki Institute for Information
Technology
![Page 38: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/38.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Emoshooter
38
![Page 39: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/39.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Emoshigeru - cinema gamesReal-timePublic displayBiosignal adaptationMassive co-locatedShooter / StrategyPart of Shigeru-games
localwlan
Public displayConnected to game
server
![Page 40: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/40.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Presemo – presentation engine
![Page 41: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/41.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Emolisten – biosignal sonification
![Page 42: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/42.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Research statement
• Social Games and Playful Applications, which Utilize Biosignals
• Grey Area between Adaptation and Biofeedback– Implicit and Explicit Biofeedback
• Biosignal Synchrony (Inter-Personal)
42
![Page 43: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/43.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Our team and projects
Professor Marko TurpeinenProfessor Timo SaariIlkka Kosunen, Toni Laitinen, Petri Lievonen, Pauli Ojala
Fun of Gaming (EU FP6) (2006-2009)Enactive Social Media and Games (Aalto) (2009)Emokeitai (Tekes – National Innovation Agency) (2009-2011) Thanks:CKIR / Niklas Ravaja
43
![Page 44: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/44.jpg)
Psychophysiological signal analysis and classification
Mitchel BenovoyCentre for Intelligent Machine, McGill University
![Page 45: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/45.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Signals
• Hardware
• Application domains
45
![Page 46: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/46.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Signals
46
0.5 1 1.5 2 2.5
x 104
-0.5
0
0.5
1
1.5
Sample
∆C
ond.
( µS
)
1000 2000 3000 4000 5000 6000 7000 8000 9000 10000323436384042
Ref
l.
1000 2000 3000 4000 5000 6000 7000 8000 9000 100005
6
7
Am
p.
1000 2000 3000 4000 5000 6000 7000 8000 9000 1000090
95
100
105
HR
(bpm
)
1000 2000 3000 4000 5000 6000 7000 8000 9000 100001.21.41.61.8
22.2
Exp
.
1000 2000 3000 4000 5000 6000 7000 8000 9000 100000.20.40.60.8
1A
mp.
1000 2000 3000 4000 5000 6000 7000 8000 9000 1000015
20
25
30
RR
(bpm
)
Galvanic Skin Response Phalange Temperature
Electromyogram
Blood Volume Pulse – HR & Amp. Respiration – Rate & Amp.
![Page 47: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/47.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Hardware• Thought Technology ProCom Infinity
47
Images via www.thoughttechnology.com
![Page 48: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/48.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Application Domains• Affective Computing
– Interaction evaluation (UI, devices, latency)– Music & emotion research– Psychoneurology
48
![Page 49: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/49.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
49
![Page 50: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/50.jpg)
BCI and Creativity
Yee Chieh (Denise) ChewGeorgia Institute of Technology
![Page 51: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/51.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• fNIR and Creative Expression• EEG and Music
51
![Page 52: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/52.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Character Templates for fNIRCreated with a mouse, these are templates that subjects traced using fNIR
52
![Page 53: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/53.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
53
![Page 54: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/54.jpg)
Physiological Measures Used for Identification of Cognitive States and
Continuous AuthenticationCurtis S. Ikehara & Martha E. Crosby
University of Hawaii at Manoa
![Page 55: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/55.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Overview of our Research Using Physiological sensors– Assessing Cognitive States– Continuous Authentication
• Possible Applications
55
![Page 56: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/56.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 5656
Multi-modal Physiological Sensors System
MTF(Moving Target Fractions) HUMAN
Custom Electronics(Analog to Digital Converter)
Eye Tracker(Gaze Position, Fixation Number, Fixation
Duration, Repeat Fixations, Search Patterns, Pupil Size, Blink Rate, Blink Duration)
Pressure Mouse(Pressures on Mouse Button and Body)
Laptop(Data Collection, Analysis & Display) EDA / TSA Finger Sensors
(Electrodermal and Temperature Sensor Activity)
Oximeter(Pulse, Blood Oxygen)
![Page 57: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/57.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 20105757
A Composite of Over 20 Mouse Clicks of Three People (Click Signature)
Subject 5
Subject 4
Subject 7
US Patent # 7,245,218
![Page 58: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/58.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Possible Applications
• The student’s confidence can be assessed by evaluating the distortions of the mouse click signature.
• High Value Vehicle Control– Pressure and other physiological sensors can be
attached to a steering (e.g. aircraft yoke or steering wheel) .
– Sensors can be used to indicate that:• The authorized pilot or driver is in control.• The cognitive state of the pilot is consistent with the
vehicle’s activity.
58
![Page 59: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/59.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
University of Hawaii at Manoa1680 East-West Road - POST 317
Honolulu, Hawaii 96822 USA
59
Martha E. [email protected]
Curtis S. [email protected]
![Page 60: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/60.jpg)
Measuring Interaction Experiences:Integration of Multiple
Psychophysiological Methods
Jan KallenbachAalto UniversityHelsinki, Finland
![Page 61: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/61.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
OutlineUser Experience
=Choice & Interaction
+Cost-Benefit Evaluations
Measuring Interaction Experiences?
Software Framework61
![Page 62: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/62.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Client Application for UX ResearchSampling frequency: max 1000HzPsychophysiological Measures: Gaze, Pupil Size, EDA, EMG, EEG, …Behavioral Measures: Keyboard, Mouse, Slider, …System Measures: Window Events
62
MultimodalMeasurements
![Page 63: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/63.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Jan KallenbachResearch Scholar
Aalto UniversitySchool of Science & TechnologyDepartment of Media TechnologyP.O. Box 15500FIN - 00076 Aalto
63
![Page 64: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/64.jpg)
Towards enhanced video access and recommendation through emotions
Eva OliveiraNuno Ribeiro
Teresa ChambelIPCA | LASIGE
![Page 65: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/65.jpg)
65Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
We Want
• To access to videos based on emotions– user actual emotional (psychophysiological) state– video emotional content– director’s perspective
• To recommend videos based on emotions– create a set of rules from every affective
classification perspective (user and video, director)
![Page 66: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/66.jpg)
66Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Affective video recommendation system requirements ()• Every video must be classified affectively from the user
and from the content perspective.• Every user has an emotional profile constituted by all his
implicit (physiological) classification of every classified movie.
• User physiological signals can change the way we watch the videos
![Page 67: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/67.jpg)
67Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
User Classification Issues
• Automatic Classification– Users emotional state
• emotional model– Classification techniques
• Result output• Accuracy• Lab contexts
![Page 68: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/68.jpg)
68Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Current State
• Capturing users phychophysiological signals (BIOPAC System) while watching movies
• Next step: several classification techniques to convert signal result into emotional categories
This is my exciting picture
![Page 69: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/69.jpg)
69Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
• Eva Oliveira :
![Page 70: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/70.jpg)
Your users can’t talk!How can you tell they liked
your system? Lucia Filgueiras and Claudia Tambascia
Escola PolitecnicaUniversidade de São Paulo
![Page 71: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/71.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
What we want to do
• Develop products for cerebral palsy users• Perform participatory design• Understand non-verbal communication
71
![Page 72: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/72.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Direct measures and usability evaluation
Variabilidade da frequência cardíaca OP13
2,6
8,3
5,2
2,6 2,63,9
4,6 5,0
11,7
2,3
9,1
3,82,9
6,6
4,0 4,4
2,4
14,1
5,3 5,64,1
7,3
3,3
8,5
5,9
0,0
2,0
4,0
6,0
8,0
10,0
12,0
14,0
16,0
18,0
09:35
09:40
09:45
09:50
09:55
10:00
10:05
10:10
10:15
10:20
10:25
10:30
10:35
10:40
10:45
10:50
10:55
11:00
11:05
11:10
11:15
11:20
11:25
11:30
11:35
HORA
Índi
ce d
o B
alan
ço S
impá
tico?
Par
assi
mpá
tico
LF nu / HF nu LSC Média
Alarme de temperarura alta:
10:15
Operador analisa sistema após vários alarmes: temperatura alta (10:29);
nível alto (10:32) e segurança (10:32; 10:37; 10:43)
Ação do operador para correção
Ação do operador para correção
Eye tracking in iTV apps evaluation
NASA TLX, SWAT, Heart Rate Variability
Heart Rate Variability of an operator in a unstable unit situation
Comparison of eye tracker findings to conventional usability tests results
HRV and workload measures in control room operability evaluation
![Page 73: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/73.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Our questions:• How personalized are body expressions for cerebral palsy patients?
• Are there general body expressions that can mean the same for everyone?
• Is it possible to determine a generic way of measuring these expressions?
• Is there any other liaison besides gestures and facial expressions , like an unconscious connection of limbic systems that can be responsible for the mutual sensitive communication?
• Is there a reference body signal that can be used as a reliable parameter to evaluate body expressions?
• How can we establish a correlation between body expressions and patient´s opinion of approval or disapproval of a design solution?
73
![Page 74: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/74.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you!Contact us:
Lucia V. L. FilgueirasEscola Politecnica, Universidade de São [email protected]
Claudia A.TambasciaFundação [email protected]
http://lts-i.pcs.usp.br
74
![Page 75: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/75.jpg)
Human Computer Symbiosis by Mutual Understanding
Marc GrootjenEagleScience-TNO
![Page 76: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/76.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Symbiosis & mutual understanding• Information models• Interfaces
76
![Page 77: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/77.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 77
Tasks (goals)
Disturbance
computerMutual
understanding
Symbiotic entity
Maximum task performance
human
Human-computer symbiosis with mutual understanding as key issue for maximum task performance
![Page 78: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/78.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010 78
![Page 79: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/79.jpg)
Domesticating Brain-Computer Interfaces: Systems
for the HomeMelody Moore Jackson
BrainLabSchool of Interactive Computing
Georgia Tech
![Page 80: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/80.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Extended Home-based BCI study
• fNIR-based BCI for communication• 32 People with ALS (so far) over 1 year• 78% (25) could operate the device on
initial visit with accuracy > 70%• Accuracy to 100%, average 76.5%• Issues:
– Motivation– Data Collection– Remote Support
80
![Page 81: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/81.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
The Kokoro GatariBrain Computer Interface based on Functional Near Infrared imaging
81
![Page 82: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/82.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Melody Moore [email protected]
82
![Page 83: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/83.jpg)
Get up, move on! Using Electromyography to Explore
the Relationship of Experience and Motion
Johann SchrammelCentre for Usability Research and
Engineering
![Page 84: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/84.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Compare experiences and emotions in conditions with and without movements with Electromyography(EMG) and other methods
84
![Page 85: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/85.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Can EMG successfully be used to estimate a user‘s emotional experience in motion contexts (artefacts, feasibility, etc.)?
85
![Page 86: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/86.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Is a more embodied way of controlling interaction actually amplifying experience?
86
![Page 87: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/87.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Results
• EMG is suitable for motion contexts.• Both EMG and self-report data indicate
that natural body movement intensifies emotions.
• It’s crucial that body movements are in accordance with activity.
87
![Page 88: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/88.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
88
![Page 89: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/89.jpg)
Reality-Based Brain-Computer Interaction
Daniel SjölieUmeå University, Umeå, Sweden
![Page 90: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/90.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Reality-Based Brain-Computer Interaction• Complex, reality-based, interaction with the
ability to adapt to the user in real-time– Using brain measurements to evaluate
• Direct brain-computer interaction– The user can be unaware
• We’re using a combination of virtual reality (VR) and functional MRI (fMRI)
90
![Page 91: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/91.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
HCI and cognition
• An improved understanding of the brain in this interactive and complex context is needed
• Build on theories of cognition as a foundation for the development of advanced HCI– As interaction moves towards being reality-based
we move closer to “interacting with reality”, and thus to general cognition
• Cognitive neuroscience and computational models of the brain are important tools
91
![Page 92: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/92.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Working with fMRI and Virtual RealityWorking with fMRI requires specialist competence and forces us to deal with powerful magnetic fields and restricted mobility, but the ability to measure the whole brain is an invaluable research tool.
92
![Page 93: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/93.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Daniel Sjölie, [email protected]
93
![Page 94: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/94.jpg)
emoBAN: Improving Quality of Life
via Psychophysiological Mobile Computing
Katarzyna Wac, Anind K. DeyCarnegie Mellon University, HCII
![Page 95: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/95.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
emoBAN idea
• “what always speaks silently is the body” (Brown, 1990)
– measure QoL via psychophysiology
95
![Page 96: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/96.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
emoBAN: Usable Useful Functional
• Body Area Network– sensors
• vital signs, e.g., ECG, temp, resp• context, e.g., location, time, activity
– actuators• feedback via, e.g., AV, tactile, light
96
![Page 97: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/97.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank youDr Katarzyna WacCarnegie Mellon UniversityHuman-Computer Interaction [email protected]/~wac
97
![Page 98: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/98.jpg)
Leveraging Psychophysical Data in Monitoring and Analyzing the States of Badminton Players
Feng Tian, Wencan Luo, Jing Dai, Reza Naima, Yongmin Cheng, HonganWang, Guozhong Dai
Institute of Software, Chinese Academy of SciencesPhysical Education Collage, Beijing Sport University
Berkeley Institute of Design, UC. BerkeleyChina Badminton Team, General Administration of Sport of China
4/20/2010
![Page 99: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/99.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Traces and physical actions - Camera, accelerometers
Psychophysical data- Berkeley TricorderECG, EMG, Pulse Oximetry, BioImpedance Respiration
![Page 100: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/100.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
![Page 101: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/101.jpg)
Exploring Quantitative Methods for Evaluating Sports Games
Veronica ZammittoSimon Fraser University
Magy Seif El-NasrSimon Fraser University
Paul NewtonElectronic Arts Canada
![Page 102: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/102.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Current Study: NBA Live 10
• Eye tracking– Eye Tech TM3
• Psycho-physiological data– ProComp5 Infiniti– EMG (corrugator supercilii and zygomatic major),
EKG (heart rate), GSR and BVP• Gameplay Telemetry Data
– event-based RPC telemetry collection system• Triangulation
102
![Page 103: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/103.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Data Triangulation - NBA Live 10Game user experience
103
What was the user looking at?
Where user’s attention was?
What was the emotional response?
Was positive? Was negative?
Event, when
![Page 104: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/104.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Data Triangulation - NBA Live 10Game user experience
104
![Page 105: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/105.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Veronica [email protected]
Magy Seif [email protected]
Paul [email protected]
105
![Page 106: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/106.jpg)
Exploring Psycho-physiological Measures for the Design and Behavior of Socially-Aware
Avatars in Ubicomp Environments
Johanna Renny OctaviaHasselt University, Belgium
![Page 107: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/107.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Avatars in Ubicomp Environment User’s representation Mediator between system and user (companion) To design a socially-aware avatar that adapts its behavior according to user’s emotion Any incorrect actions of the avatar can lead to loss of identification of the user
107
![Page 108: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/108.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Our informal experiment ProComp Infiniti device, measured GSR, EMG and BVP Higher physiological values when playing against a friend How different social relationships between players influence those values?
108
![Page 109: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/109.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Expertise Centre for Digital Media (EDM)Web: www.edm.uhasselt.beEmail: [email protected]
109
![Page 110: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/110.jpg)
Tailoring Practice for Individualized Learning via
Cognitive StateBob Wray
Soar Technology, Inc.
![Page 111: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/111.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Why I need Psychophysiological Computing• Practice environments are becoming
common-place• Adaptive tailoring of practice is important for
individualized experience/learning• Need more than traditional adaptations (e.g.,
scaffolding)– Adapting to engage learner / maintain interest – Adaptation to reducing “gaming” in environment– Adaptation to elicit learner affect/arousal (“real”)
111
![Page 112: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/112.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Research TestbedNarrative, conversational, inter-cultural training application
112
![Page 113: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/113.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
Bob WraySoar [email protected], 919.967.5079
Brian MagerkoGeorgia Institute of [email protected]
113
![Page 114: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/114.jpg)
Guidelines to biosignaldriven HCI
Marjolein van der Zwaag
Philips Research Europe, University of Groningen
![Page 115: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/115.jpg)
Guidelines to biosignaldriven HCI
Marjolein van der ZwaagEgon L. van den Broek
Joris H. Janssen
![Page 116: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/116.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Outline
• Signal(s) measured: psycho-physiological • Application domain: Affective computing• Device used: Nexus 10, mind media
• Guidelines for processing biosignals– Triangulation– Identification of users– Biosignal characteristics – Temporal construction
116
![Page 117: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/117.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Unobtrusive affective computingMusic direct your mood concept
117
![Page 118: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/118.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Thank you
118
![Page 119: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/119.jpg)
![Page 120: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/120.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What is brain/body computing good for?
How can we think systematically about this space?
120
![Page 121: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/121.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What are the major themes of HCI that can be explored using brain/body interaction?
121
![Page 122: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/122.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What specific brain & body sensing modalities are interesting and how can we use them?
What are most innovative technologies in this space?
How do these compare with existing practices (NASA-TLX, Likert-style questionnaires, etc)?
122
![Page 123: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/123.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
How can BCI and PC be defined within an HCI context?
How is a HCI context different from neuropsychology, biomedicine or biomedical
engineering?
How does HCI contribute to psychophysiological computing (or more
specifically neural engineering, bio-engineering, etc)?
123
![Page 124: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/124.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
If we could measure anything on brain and body, what would we want to measure and
why?
What cognitive states and emotions have been investigated with sensors?
What methodologies are best for what signals?
124
![Page 125: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/125.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
How can accurate physiological, psychological and neurophysiological
measurements improve HCI?
125
![Page 126: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/126.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
With what software techniques and hardware technologies could these signals
be optimally processed?
How can software account for the combination of user actions taken to accomplish a task versus actions that
would surprise the user, yet optimize their experience?
126
![Page 127: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/127.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Workshop Discussion Questions
What are the issues that must be considered when using brain/body computing for
direct control, passive sensing, adaptation, etc?
127
![Page 128: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/128.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Groups
• How to go from raw data to something else– Emotions: TABLE 1– Data Fusion: TABLE 4
• How to use these things in interactive interfaces: TABLE 3
128
![Page 129: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/129.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BBB Workshop: Bringing Together BCI& Physiological Computing
129
![Page 130: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/130.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
Staying connected
130
Website http://www.eecs.tufts.edu/~agirou01/workshop/
Twitter http://twitter.com/bbbCHI2010
Twitter Hashtag
#bbbCHI2010
Facebook http://www.facebook.com/home.php#/pages/Atlanta-GA/Brain-Body-and-Bytes-Psychophysiological-User-Interaction-at-CHI-2010/136581171786
Flickr http://www.flickr.com/groups/bbbchi/
Grouplist http://groups.google.com/group/brain-body-interfaces/
Slide Share http://www.slideshare.net/group/brain-body-and-bytes-psychophysiological-user-interaction
![Page 131: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/131.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
BBB at CHI2010Brains and Brawn session: Tuesday 11:30-1pm
Making Muscle-Computer Interfaces More PracticalT. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, James A. LandayUniversity of Washington, USAMicrosoft Corporation, USA
A Novel Brain-Computer Interface Using a Multi-Touch SurfaceBeste F. Yuksel, Michael Donnerer, James Tompkin, Anthony Steed.University College London, UK
The Influence of Implicit and Explicit Biofeedback in First-Person Shooter GamesKai Kuikkaniemi, Toni Laitinen, Marko Turpeinen, Timo Saari, Ilkka Kosunen, Niklas Ravaja, Helsinki Institute for Information Technology, FinlandCenter for Knowledge and Innovation Research, Finland
Effects of Interactivity and 3D-motion on Mental Rotation Brain Activity in an Immersive Virtual Environment
Daniel Sjölie, Kenneth Bodin, Eva Elgh, Johan Eriksson, Lars-Erik Janlert, Lars NybergUmeå University, Sweden
131
![Page 132: Brain, Body, and Bytes CHI 2010 Workshop Presentations](https://reader033.vdocuments.net/reader033/viewer/2022051322/5442ea60b1af9f350a8b47c5/html5/thumbnails/132.jpg)
Brain, Body and Bytes: Psychophysiological User Interaction Workshop at CHI 2010
THANK YOU
• Keep in touch!
132