je computer besturen met je ogen en je hersenen jaap ham · 2018-06-19 · je computer besturen met...

Jecomputerbesturenmetjeogenenjehersenen

JaapHam

19-06-18 1MAMEMpresenta<ve–Duchenne,Luxembourg,September16th,2016

Introduc<ons

•  JaapHam

•  [email protected]

•  SocialPsychology•  Persuasivetechnology•  SocialRobo<cs

2

“Youshouldnothaveanotherdrink!”

AssociateeditoroftheInterna<onalJournalonSocialRobo<csProgramChairPersuasive2018,April18-19,Waterloo,Canada

[email protected]:HumanTechnologyInterac<onEindhovenUniversityofTechnologyPhDstudents:•  SofiaFountoukidou:Personaliza<onofPersuasiveTechnology

•  AimiGhazali:PersuasiveSocialRobots•  HanneSpelt:PsychophysiologyandPT•  CaixiaLiu:FacialExpressionIden<fica<on

19-06-18 5

MAMEM:Mul<mediaAuthoringandManagementusingyourEyesandMind(EU--H2020)www.mamem.eu

MAMEMVisie

19-06-18 6

Patient&Clinician Academia&Research Industry&Society

Brainelectricalsignals

Eyemovements

Perspiration&HeartRate

WP2 WP3 WP4 WP5 WP6 WP7

WP1(ProjectManagement)&WP8(Dissemination,Exploitation&FosteringInnovation)

andothers…

Author

andothers…

Share

vvvPandothers…

Manage

PersuasiveDesign

+Ψ =

MAMEMVisie

19-06-18 7



Eyemovements




andothers…

Author

andothers…

Share

vvvPandothers…

Manage

PersuasiveDesign

+Ψ =

MAMEMVisie

19-06-18 8



Eyemovements




andothers…

Author

andothers…

Share

vvvPandothers…

Manage

PersuasiveDesign

+Ψ =

Consor<um

Eye-tracking&Integra<on

EEGandsensorsop<miza<on

EEGAnalysis&interac<onparadigms

Eye-gazeanalysis&novelinterfaces

19-06-18 9

Psychology&persuasivedesign

Spinalcordinjury,trials&evalua<on

Neuro-musculardisorders,trials&

evalua<on

Parkinsondisease,trials&evalua<on

VanEindhoventotTelAviv

19-06-18 MAMEMReview,Luxembourg,September16th,2016 10

•  MAMEMbouwen–  Technisch–  Gebruikersinbeeldbrengen–  S<mulerentotgebruik

•  Veldtest1:Gebruikersinhetlab•  MAMEMaanpassen

–  Socialeinclusie–  S<mulerentotsocialeinclusie

•  Veldtest2:Bijgebruikersthuis

19-06-18 14

GazeTheWebbrowser

19-06-18 15

Devicesforsignalcapturing&recording

19-06-18 16

Eye-tracking•  Heavy-weight(SMIREDnScien<fic)

•  Light-weight(MyGazeAssis<ve-2eyetracker

EEGrecording•  Heavy-weight(EBNBePlusLTMEEG

amplifier)

•  Light-weight(EPOCEEGheadset)

19-06-18 16

GalvanicSkinResponse

HeartRate Shimmer3GSR+

MAMEMReview,Luxembourg,September16th,2016

WP2

•  C/C++interfacesforLabSteamingLayerbuiltupontheSDKsofeachdevice

MAMEMReview,Luxembourg,September16th,2016 17

Eyetrackersetup/calibra<on GSRsensorsetup(nocalibra<onrequired)

Headsetsetup+Ohmetercheck

Somwareforconnec<ngtoMiddleware

WP2

Systemarchitecture&implementa<on

18

•  Abstractdescrip<on

•  Requirementforflexibility&extensibility

•  Serveasamiddlewarefornovelmul<-modalinterfaces

•  Fourarchitecturallayers

•  Modulesineachlayer

•  Input/outputinterfacesforeachmodule

•  Relyingonexis<ngframeworks

WP4

Translateeye-gazesignalsintocontrols

19-06-18 19

eyeGUI

GazeTheWeb

•  AdoptablelayoutdefinedbyXMLfiles

•  Visualelementsforgaze-basedinterac<on

•  Ready-madeinterac<oncomponentsthatcanbeeasilyre-used

•  Browser-basedframeworkofferingalltypicalopera<onsthrougheyes

•  ReliesontheeyeGUIlibrary

•  Serveascontainerformanydifferentapplica<ons

WP3

TranslateEEGsignalsintocontrols


Features Classifier Time(s) Accuracy(%)

PWelch

LinearSVM 5.98 86.43

RBFSVM 5.95 87.14

MLR 6.05 87.86

LDA 5.78 70.71

PYAR


RBFSVM 5.78 87.86

MLR 5.6 88.57

LDA 5.65 79.29

DWT


RBFSVM 5.86 90.71

MLR 5.66 88.57

LDA 5.66 87.86

CSP


RBFSVM 7.09 83.57

MLR 5.69 85

LDA 5.81 85.71

CSPwithFB


RBFSVM 7.08 84.29

MLR 5.69 85.71

LDA 6.36 89.29

SubjectID Op]mal Default

S001 99.99 98.55

S002 99.13 87.82

S003 47.82 34.78

S004 97.82 77.17

S005 26.08 30.43

S006 83.47 86.08

S007 98.26 60

S008 42.02 31.88

S009 99.99 99.99

S010 96.52 92.17

S011 98.26 98.26

MeanAcc. 80.85 72.47

SSVEPs SMRs

ErrPs

•  ReliablesetupofEEGsensorinfrastructure

•  In-houseexperimentalprotocols

•  ReplicateSoTAresults

•  Focusinon-lineandlow-costBCIs

WP3

HowtocombineET&EEG


•  CopewiththeMidasTouchproblemineye-trackinginterfacesusingtheSenso-Motoric-Rhythms(EEG)

•  Facilitateerror-correc<onineye-trackingintefacesbyperformingundousingError-relatedpoten<als(EEG)

WP3

Wiezijndegebruikers?

19-06-18 23

Exampleofreq.Descrip]onRelevantDimension

Compa<blewithdifferentageorcogni<veprofiles

personalvariables,e.g.,age,individualworkspaces

Personaliza<on/adapta<on

Enableusingacomputerinavarietyofdifferentbodyposi<ons

Unifiedrequiredac<onsfromthesystem

Performance

Trainingphaseshouldprovide‘fun’fortheuser

Persuasivedesign(seeWP5)Emo<ons/Mo<va<on

ShouldworkwithMicrosomwindowsinterfaceswithothersomwareandhardware

Interoperability

Todis<nguishbetweenaninten<onalanduninten<onalmovement

easeofopera<onanduserexperience

Usability

Tremorgeneratedartefacts(4-6Hz)anddyskinesiageneratedartefactsinPDpa<entsshouldbetakenintoaccount

Specifictoeyemovementsandbrainwaves(i.e.,reclinicalcohorts)

Physiology

•  Literatuur,focusgroupsenvragenlijsten•  Qualita<eveandsta<s<scheanalyses•  Usersbehoemesengewoontes•  Requirementsop5dimensies

WP6

Task 1: User Profiles & Personas

Ø Spinal cord injury disorder Ø Neuromuscular disorders Ø Parkinson’s Disorder

19-06-18 24

ModelusersusingPersonas


•  Basedonrequirements

•  Userneedsandcomputerhabits

•  Medicalcondi<onandpsychologicalstatus

•  Makeuserprofiles•  Buildpersonasforeachcohort

•  Demographic•  Medicalinfo•  Computeruserinfo•  Goals&avtudes

WP5

Usagescenarios


•  Basictasks

•  Intendedfortrainingandfamiliarizingwiththesystem

•  Sophis<catedtasks

•  Intendedforbecomingmoresociallyac<ve

WP6

•  MAMEMbouwen–  Technisch–  Gebruikersinbeeldbrengen–  S]mulerentotgebruik




Stimuleren

19-06-18 WP5-Usermodellingandinterfacedesign,Luxemburg,September2016 28

1. User acceptance and

training engagement

2. Keep using to increase social

participation

HowtoinfluencetheacceptanceanduseofMAMEMtechnologyforthedifferenttargetgroups?

19-06-18 29

PersuasiveDesign


•  Interven<onMappingFramework

•  Sixstages

•  Determinestheinterven<ons

WP5

•  Changeobjec<ve

•  Determinant

•  Persuasivestrategy

Changeobjec]ve

Determinant Persuasivestrategies(examples)

Chooserealis<cgoalsandappropriatetasks

Effortexpectancy:Beliefthatgoalsyouchoosearedo-able,appropriateandselectedforyou.

Sugges]on:letthesystemmakesugges<onsandletusersmakechoiceguidedbywhatthesystemprovidesasgoodop<onTailoring:makesurethatofferedop<onsarerelevanttotheuserprofile

Gamesomteleren

31

Gamesbedenken

19-06-18 32

b)Ac<oncardupdates

•  Removeanytaskexplana<ons;•  Provideinstruc<onsinameaningfulway

19-06-18 33

Zoekdetovenaar

19-06-18 34

Zoekdetovenaar

19-06-18 35

Nogmeerbeinvloeding:Scores

19-06-18 36

PlayormCommunica<on

19-06-18 37

Alledeelnemerswerdenbeter

19-06-18 39

SCI,PD,NMDwerdenbeter

19-06-18 40

Able-bodiedwardenbeter

19-06-18 WP5-Usermodellingandinterfacedesign,Florence,June2017 41

Indegames,maarookin4web-taken


Steedssnellertypen


Usageinnon-controlledenvironments

LaboratoryEnvironment

Heavy-weightinfrastructure

Trainingprocesses

PhaseI Home

Environment

Light-weightinfrastructure

Monitoron-lineac<vity

PhaseII


WP6

Socialeinclusie

19-06-18 47

•  Methodologysynthesizingdigitalindicatorswithsocialinclusionareas

•  Par<cipa<onandsocial

capital

•  Empowermentandwellbeing

•  Educa<onandemployment

WP7

19-06-18 48

Computermonitoredac<vitypreandpostMAMEMuseQues<onnairefilledinbysamplewithdisabilityQues<onnairefilledinbyrespec<vecaretakersQues<onnairebenchmarkedagainsthealthysampleAudiodiariestocap<vateusagemomentumandinsightsaboutMAMEMop<miza<on

Toolsforquan<fyingsocialintegra<onWP7



–  Socialeinclusie–  S]mulerentotsocialeinclusie


MAMEMDashboard/Homepage

19-06-18 50

S<mulereninGazeTheWeb

19-06-18 51

19-06-18 53

19-06-18 55

19-06-18 56

MAMEMDashboard/Homepage

19-06-18 65

Jecomputerbesturenmetjeogenenjehersenen

JaapHam

19-06-18 67MAMEMpresenta<ve–Duchenne,Luxembourg,September16th,2016

je computer besturen met je ogen en je hersenen jaap ham · 2018-06-19 · je computer besturen met...

Documents