computational physiology lab department of computer science university of houston houston, tx 77004...

Computational Physiology LabDepartment of Computer Science

University of HoustonHouston, TX 77004

Eustressed or Distressed?Combining Physiology withObservation in User Studies

Avinash WesleyDr. Peggy Lindner (Co-Advisor)

Dr. Ioannis Pavlidis (Advisor)

2/13

Stress Signs

• Peripheral Physiological Measurement of Stress– Adrenergic response

• Elevates heart rate, respiration rate, and blood pressure – Cholinergic response

• Activates sweat glands on fingers and the perinasal area

• Introduction• Methods• Results and Discussion• Acknowledgements

- Stress Mechanism- Motivation- Background

3/13

Physiology and Observation

• Perspiratory response are – sympathetic in nature – Non-specific to positive or negative arousal

• Introduction• Methods• Results and Discussion• Acknowledgements

Distress Eustress

- Stress Mechanism- Motivation- Background

4/13

Emotions vs. Performance

• Introduction• Methods• Results and Discussion• Acknowledgements

- Stress Mechanism- Motivation- Background

Arousal

Performance

LOW MEDIUM HIGH

HIGH

Sleep Disorganization

AnxietyAlertness

Optimal

• An important goal in user studies: Study the role of emotions on human performance

• Emotions can be quantified via physiological response

• Physiological responses can be disambiguated via observation

5/13

Perspiration Signal and Observation

• Physiology– Perspiration extraction method in the Thermal Imagery [1]

• Observational Annotation– Traditional done in the Visual Imagery

• Manual

• Introduction• Methods• Results and Discussion• Acknowledgements

[1] D. Shastri, A. Merla, P. Tsiamyrtzis, and I. Pavlidis. Imaging facial signs of neurophysiological responses. IEEE Transactions on Biomedical Engineering, 56(2):477–484, 2009.

Courtesy of Science channel

- Stress Mechanism- Motivation- Background

6/13

Region Tracking

• Seven anatomical regions tracked over time by a dynamic template update tracker [2]

• Introduction• Methods• Results and Discussion• Acknowledgements

[2] Y. Zhou, P. Tsiamyrtzis, and I. Pavlidis. Tissue tracking in thermo-physiological imagery through spatio-temporal smoothing. Proc. of the 12th Int. Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2009),5762:1092–1099, 2009.

- Facial Expression Recognition- Field Study

7/13

Pattern Classification

• Feature Vector

• Classifier– Classify five action units (AU1+2, 4, 9, 10, and 12)– Multilayer Perceptron– 10-fold Cross Validation

• Introduction• Methods• Results and Discussion• Acknowledgements

AU 1+2 Inner + Outer Eyebrow Raise

d(x,5): Euclidean distance between ROI-x and 5, (x 5)

- Facial Expression Recognition- Field Study

8/13

Surgical Training

• Surgeon Pool (n=17)– Novices – Experienced

• Tasks– Running string (Task-1)– Pattern cut (Task-2)– Intracorporeal suture (Task-3)

• Dataset: 977 Thermal Clips

• Introduction• Methods• Results and Discussion• Acknowledgements

- Facial Expression Recognition- Field Study

9/13

Validation Results

• Using Thermal Imagery– 244 Facial Expressions – Ground Truth via Visual annotation– Method Accuracy 81.55%

• Introduction• Methods• Results and Discussion• Acknowledgements

* Use of visual images instead of thermal images for display purpose only

* Confusion matrix

- Quantitative Analysis- Qualitative Analysis- Conclusions

10/13

Results From The Field Study

• Distress is inversely related to experience

– EN (Perinasal perspiratory signal on portions of negative emotions)

• Introduction• Methods• Results and Discussion• Acknowledgements

- Quantitative Analysis- Qualitative Analysis- Conclusions

Novice Experienced

11/13

Example Visualizations

Eustress Distress

• Introduction• Methods• Results and Discussion• Acknowledgements

- Quantitative Analysis- Qualitative Analysis- Conclusions

12/13

Conclusions

• The proposed method is – Comprehensive (quantitative and qualitative) – Economical (single imaging modality with no labor)

• Conducted a study design that is applicable to a broad class of Human Machine Interaction

• Future Work– Expand the facial expression set– Apply the method to more field studies

• Detection of pain onset-offset

• Introduction• Methods• Results and Discussion• Acknowledgements

- Quantitative Analysis- Qualitative Analysis- Conclusions

13/13

• Support provided by NSF award # IIS-0812526

• Introduction• Methods• Results and Discussion• Acknowledgements