je computer besturen met je ogen en je hersenen jaap ham 2018-06-19آ  je computer besturen met je...

Download Je computer besturen met je ogen en je hersenen Jaap Ham 2018-06-19آ  Je computer besturen met je ogen

Post on 24-May-2020

1 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Je computer besturen met je ogen en je hersenen

    Jaap Ham

    19-06-18 1 MAMEM presenta

  • Introduc

  • Associate editor of the Interna

  • Jaap Ham j.r.c.ham@tue.nl Research Group: Human Technology Interac

  • 19-06-18 5

    MAMEM: Mul

  • MAMEM Visie

    19-06-18 6

    Patient & Clinician Academia & Research Industry & Society

    Brain electrical signals

    Eye movements

    Perspiration & Heart Rate

    WP2 WP3 WP4 WP5 WP6 WP7

    WP1 (Project Management) & WP8 (Dissemination, Exploitation & Fostering Innovation)

    and others …

    Author

    and others …

    Share

    vvvP and others …

    Manage

    Persuasive Design

    + Ψ =

  • MAMEM Visie

    19-06-18 7

    Patient & Clinician Academia & Research Industry & Society

    Brain electrical signals

    Eye movements

    Perspiration & Heart Rate

    WP2 WP3 WP4 WP5 WP6 WP7

    WP1 (Project Management) & WP8 (Dissemination, Exploitation & Fostering Innovation)

    and others …

    Author

    and others …

    Share

    vvvP and others …

    Manage

    Persuasive Design

    + Ψ =

  • MAMEM Visie

    19-06-18 8

    Patient & Clinician Academia & Research Industry & Society

    Brain electrical signals

    Eye movements

    Perspiration & Heart Rate

    WP2 WP3 WP4 WP5 WP6 WP7

    WP1 (Project Management) & WP8 (Dissemination, Exploitation & Fostering Innovation)

    and others …

    Author

    and others …

    Share

    vvvP and others …

    Manage

    Persuasive Design

    + Ψ =

  • Consor

  • Van Eindhoven tot Tel Aviv

    19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 10

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

  • 19-06-18 14

  • GazeTheWeb browser

    19-06-18 15

  • Devices for signal capturing & recording

    19-06-18 16

    Eye-tracking •  Heavy-weight (SMI REDn Scien

  • •  C/C++ interfaces for LabSteamingLayer built upon the SDKs of each device

    MAMEM Review, Luxembourg, September 16th, 2016 17

    Eye tracker setup/calibra

  • System architecture & implementa

  • Translate eye-gaze signals into controls

    19-06-18 19

    eyeGUI

    GazeTheWeb

    •  Adoptable layout defined by XML files

    •  Visual elements for gaze-based interac

  • Translate EEG signals into controls

    19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 20

    Features Classifier Time (s) Accuracy (%)

    PWelch

    Linear SVM 5.98 86.43

    RBF SVM 5.95 87.14

    MLR 6.05 87.86

    LDA 5.78 70.71

    PYAR

    Linear SVM 5.72 87.14

    RBF SVM 5.78 87.86

    MLR 5.6 88.57

    LDA 5.65 79.29

    DWT

    Linear SVM 5.86 89.29

    RBF SVM 5.86 90.71

    MLR 5.66 88.57

    LDA 5.66 87.86

    CSP

    Linear SVM 6.99 83.57

    RBF SVM 7.09 83.57

    MLR 5.69 85

    LDA 5.81 85.71

    CSP with FB

    Linear SVM 5.63 84.29

    RBF SVM 7.08 84.29

    MLR 5.69 85.71

    LDA 6.36 89.29

    Subject ID Op]mal Default

    S001 99.99 98.55

    S002 99.13 87.82

    S003 47.82 34.78

    S004 97.82 77.17

    S005 26.08 30.43

    S006 83.47 86.08

    S007 98.26 60

    S008 42.02 31.88

    S009 99.99 99.99

    S010 96.52 92.17

    S011 98.26 98.26

    Mean Acc. 80.85 72.47

    SSVEPs SMRs

    ErrPs

    •  Reliable setup of EEG sensor infrastructure

    •  In-house experimental protocols

    •  Replicate SoTA results

    •  Focus in on-line and low-cost BCIs

    WP3

  • How to combine ET & EEG

    19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 21

    •  Cope with the Midas Touch problem in eye-tracking interfaces using the Senso- Motoric-Rhythms (EEG)

    •  Facilitate error-correc

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

  • Wie zijn de gebruikers?

    19-06-18 23

    Example of req.Descrip]onRelevant Dimension

    Compa

  • Task 1: User Profiles & Personas

    Ø Spinal cord injury disorder Ø Neuromuscular disorders Ø Parkinson’s Disorder

    19-06-18 24

  • Model users using Personas

    19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 25

    •  Based on requirements

    •  User needs and computer habits

    •  Medical condi

  • Usage scenarios

    19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 26

    •  Basic tasks

    •  Intended for training and familiarizing with the system

    •  Sophis

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S]muleren tot gebruik

    •  Veldtest 1: Gebruikers in het lab •  MAMEM aanpassen

    –  Sociale inclusie –  S

  • Stimuleren

    19-06-18 WP5 - User modelling and interface design, Luxemburg, September 2016 28

    1. User acceptance and

    training engagement

    2. Keep using to increase social

    participation

  • How to influence the acceptance and use of MAMEM technology for the different target groups?

    19-06-18 29

  • Persuasive Design

    19-06-18 MAMEM Review, Luxembourg, September 16th, 2016 30

    •  Interven

  • Games om te leren

    31

  • Games bedenken

    19-06-18 32

  • b) Ac

  • Zoek de tovenaar

    19-06-18 34

  • Zoek de tovenaar

    19-06-18 35

  • Nog meer beinvloeding: Scores

    19-06-18 36

  • Playorm Communica

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

  • Alle deelnemers werden beter

    19-06-18 39

  • SCI, PD, NMD werden beter

    19-06-18 40

  • Able-bodied warden beter

    19-06-18 WP5 - User modelling and interface design, Florence, June 2017 41

  • In de games, maar ook in 4 web-taken

    19-06-18 WP5 - User modelling and interface design, Florence, June 2017 42

  • Steeds sneller typen

    19-06-18 WP5 - User modelling and interface design, Florence, June 2017 43

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

  • Usage in non-controlled environments

    Laboratory Environment

    Heavy-weight infrastructure

    Training processes

    Phase I Home

    Environment

    Light-weight infrastructure

    Monitor on- line ac

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

  • Sociale inclusie

    19-06-18 47

    •  Methodology synthesizing digital indicators with social inclusion areas

    •  Par

  • 19-06-18 48

    Computer monitored ac

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

  • MAMEM Dashboard / Homepage

    19-06-18 50

  • S

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

  • 19-06-18 53

  • 19-06-18 WP5 - User modelling and interface design, Florence, June 2017 54

  • 19-06-18 55

  • 19-06-18 56

  • 19-06-18 WP5 - User modelling and interface design, Florence, June 2017 57

  • 19-06-18 WP5 - User modelling and interface design, Florence, June 2017 58

  • 19-06-18 WP5 - User modelling and interface design, Florence, June 2017 59

  • 19-06-18 WP5 - User modelling and interface design, Florence, June 2017 60

  • 19-06-18 WP5 - User modelling and interface design, Florence, June 2017 61

  • 19-06-18 WP5 - User modelling and interface design, Florence, June 2017 62

  • 19-06-18 WP5 - User modelling and interface design, Florence, June 2017 63

  • 19-06-18 WP5 - User modelling and interface design, Florence, June 2017 64

  • MAMEM Dashboard / Homepage

    19-06-18 65

  • •  MAMEM bouwen –  Technisch –  Gebruikers in beeld brengen –  S

  • Je computer besturen met je ogen en je hersenen

    Jaap Ham

    19-06-18 67 MAMEM presenta

Recommended

View more >