je computer besturen met je ogen en je hersenen jaap ham 2018-06-19آ  je computer besturen met je...

Je computer besturen met je ogen en je hersenen

Jaap Ham

19-06-18 1 MAMEM presenta

Introduc

Associate editor of the Interna

Jaap Ham j.r.c.ham@tue.nl Research Group: Human Technology Interac

19-06-18 5

MAMEM: Mul

MAMEM Visie

19-06-18 6

Patient & Clinician Academia & Research Industry & Society

Brain electrical signals

Eye movements

Perspiration & Heart Rate

WP2 WP3 WP4 WP5 WP6 WP7

WP1 (Project Management) & WP8 (Dissemination, Exploitation & Fostering Innovation)

and others …

Author

and others …

Share

vvvP and others …

Manage

Persuasive Design

+ Ψ =

MAMEM Visie

19-06-18 7

Patient & Clinician Academia & Research Industry & Society

Brain electrical signals

Eye movements

Perspiration & Heart Rate

WP2 WP3 WP4 WP5 WP6 WP7

WP1 (Project Management) & WP8 (Dissemination, Exploitation & Fostering Innovation)

and others …

Author

and others …

Share

vvvP and others …

Manage

Persuasive Design

+ Ψ =

MAMEM Visie

19-06-18 8

Patient & Clinician Academia & Research Industry & Society

Brain electrical signals

Eye movements

Perspiration & Heart Rate

WP2 WP3 WP4 WP5 WP6 WP7

WP1 (Project Management) & WP8 (Dissemination, Exploitation & Fostering Innovation)

and others …

Author

and others …

Share

vvvP and others …

Manage

Persuasive Design

+ Ψ =

Consor

Van Eindhoven tot Tel Aviv

19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 10

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

19-06-18 14

GazeTheWeb browser

19-06-18 15

Devices for signal capturing & recording

19-06-18 16

Eye-tracking •  Heavy-weight (SMI REDn Scien

•  C/C++ interfaces for LabSteamingLayer built upon the SDKs of each device

MAMEM Review, Luxembourg, September 16th, 2016 17

Eye tracker setup/calibra

System architecture & implementa

Translate eye-gaze signals into controls

19-06-18 19

eyeGUI

GazeTheWeb

•  Adoptable layout defined by XML files

•  Visual elements for gaze-based interac

Translate EEG signals into controls

19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 20

Features Classifier Time (s) Accuracy (%)

PWelch

Linear SVM 5.98 86.43

RBF SVM 5.95 87.14

MLR 6.05 87.86

LDA 5.78 70.71

PYAR

Linear SVM 5.72 87.14

RBF SVM 5.78 87.86

MLR 5.6 88.57

LDA 5.65 79.29

DWT

Linear SVM 5.86 89.29

RBF SVM 5.86 90.71

MLR 5.66 88.57

LDA 5.66 87.86

CSP

Linear SVM 6.99 83.57

RBF SVM 7.09 83.57

MLR 5.69 85

LDA 5.81 85.71

CSP with FB

Linear SVM 5.63 84.29

RBF SVM 7.08 84.29

MLR 5.69 85.71

LDA 6.36 89.29

Subject ID Op]mal Default

S001 99.99 98.55

S002 99.13 87.82

S003 47.82 34.78

S004 97.82 77.17

S005 26.08 30.43

S006 83.47 86.08

S007 98.26 60

S008 42.02 31.88

S009 99.99 99.99

S010 96.52 92.17

S011 98.26 98.26

Mean Acc. 80.85 72.47

SSVEPs SMRs

ErrPs

•  Reliable setup of EEG sensor infrastructure

•  In-house experimental protocols

•  Replicate SoTA results

•  Focus in on-line and low-cost BCIs

WP3

How to combine ET & EEG

19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 21

•  Cope with the Midas Touch problem in eye-tracking interfaces using the Senso- Motoric-Rhythms (EEG)

•  Facilitate error-correc

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

Wie zijn de gebruikers?

19-06-18 23

Example of req.Descrip]onRelevant Dimension

Compa

Task 1: User Profiles & Personas

Ø Spinal cord injury disorder Ø Neuromuscular disorders Ø Parkinson’s Disorder

19-06-18 24

Model users using Personas

19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 25

•  Based on requirements

•  User needs and computer habits

•  Medical condi

Usage scenarios

19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 26

•  Basic tasks

•  Intended for training and familiarizing with the system

•  Sophis

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S]muleren tot gebruik

•  Veldtest 1: Gebruikers in het lab •  MAMEM aanpassen

–  Sociale inclusie –  S

Stimuleren

19-06-18 WP5 - User modelling and interface design, Luxemburg, September 2016 28

1. User acceptance and

training engagement

2. Keep using to increase social

participation

How to influence the acceptance and use of MAMEM technology for the different target groups?

19-06-18 29

Persuasive Design

19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 30

•  Interven

Games om te leren

31

Games bedenken

19-06-18 32

b) Ac

Zoek de tovenaar

19-06-18 34

Zoek de tovenaar

19-06-18 35

Nog meer beinvloeding: Scores

19-06-18 36

Playorm Communica

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

Alle deelnemers werden beter

19-06-18 39

SCI, PD, NMD werden beter

19-06-18 40

Able-bodied warden beter

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 41

In de games, maar ook in 4 web-taken

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 42

Steeds sneller typen

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 43

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

Usage in non-controlled environments

Laboratory Environment

Heavy-weight infrastructure

Training processes

Phase I Home

Environment

Light-weight infrastructure

Monitor on- line ac

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

Sociale inclusie

19-06-18 47

•  Methodology synthesizing digital indicators with social inclusion areas

•  Par

19-06-18 48

Computer monitored ac

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

MAMEM Dashboard / Homepage

19-06-18 50

S

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

19-06-18 53

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 54

19-06-18 55

19-06-18 56

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 57

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 58

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 59

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 60

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 61

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 62

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 63

19-06-18 WP5 - User modelling and interface design, Florence, June 2017 64

MAMEM Dashboard / Homepage

19-06-18 65

•  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

Je computer besturen met je ogen en je hersenen

Jaap Ham

19-06-18 67 MAMEM presenta