engineering design centre project updates –august 2014

Engineering Design Centre

Project Updates –August 2014


Aim of the study

•To compare –performance and preferences of users for different input modalities

–a standard computer mouse and HOTAS Joystick with different eye-gaze, head and hand movement tracking based pointing systems


Vision parameters


Cognitive parametersTrail Making Test Digit Symbol Test


Motor parameters

Range of Motion of wrist (Palm facing down)

Measuring Radial Deviation

Measuring Ulnar Deviation


Participants

Participants Age Sex Nationalities GS ROMW DST TMT VA CB

1 26 F Indian 24 65 64 37.4 1 N

2 23 F British 28 100 68 27.3 1 N

3 53 M British 52 68 42 22.8 0 N

4 34 M Indian 39 65 44 38.9 0 N

5 30 F Polish 27 70 56 40.4 0 N

6 46 M British 55 78 63 38 0 N

7 28 M South African 48 60 52 23.8 0 N

8 23 M British 47 95 78 18.44 0 N

9 19 M British 27 110 55 21.53 1 N

10 30 F Italian 30 106 64 16.8 3 N

11 22 M American 52 75 74 23.58 0 N

12 30 M British 30 80 68 21.5 0 N

13 26 M Spanish 54 95 78 17.09 0 N


Task


Design

• Modalities1. Eye Tracking with Z-Axis selection (ZET)2. Eye Tracking with Voice-based selection (VoiceET)3. Adapted Eye Tracking (AdaptedET)4. Multimodal Eye Tracking (MmET)5. Head Tracking with Z-Axis selection (ZHT)6. Head Tracking with Voice-based selection (VoiceHT)7. Adaptive Head Tracking (AdaptedHT)8. Hand Tracking with Z-Axis selection (ZGS)9. Hand Tracking with Voice-based selection (VoiceGS)10. Adaptive Hand Tracking (AdaptedGS)11. HOTAS Joystick12. Mouse

• Targets– Size

• 45, 55, 65, 75 pixels

– Distances• 80, 160, 240, 325 pixels [ 1 pixel ≈ 0.25 mm ]


Results

•All participants can undertake trials in all conditions

•We measured–Selection time–Cursor trace

• #Wrong Selections• Main movement + Homing Time• Extra Distance Travelled over Target Axis Length

–Pupil diameter–TLX scores–BRS scores


Selection TimesComparing Selection Times

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

0 0.5 1 1.5 2 2.5 3 3.5

ID

Se

lec

tio

n T

ime

(in

ms

ec

)

AdaptedET

HOTAS Joystick

VoiceET

ZET

Mouse

MmET

AdaptedHT

ZHT

VoiceHT

AdaptedHand

ZHand

VHand

Linear (AdaptedET)

Linear (HOTAS Joystick)

Linear (VoiceET)

Linear (ZET)

Linear (Mouse)

Linear (MmET)

Linear (AdaptedHT)

Linear (ZHT)

Linear (VoiceHT)

Linear (AdaptedHand)

Linear (ZHand)

Linear (VHand)


Fitts’ Law


Selection Times - ANOVA

Source df F Sig.Eta

Squared

DEVICE 4.02 58.49 0.00 0.71

Error(DEVICE) 96.36

WIDTH 2.20 66.25 0.00 0.73

Error(WIDTH) 52.70

DIST 3.00 9.65 0.00 0.29

Error(DIST) 72.00

DEVICE * WIDTH 33.00 3.79 0.00 0.14

Error(DEVICE*WIDTH) 792.00

DEVICE * DIST 33.00 1.41 0.07 0.06

Error(DEVICE*DIST) 792.00

WIDTH * DIST 5.30 1.60 0.16 0.06

Error(WIDTH*DIST) 127.15

DEVICE * WIDTH * DIST 99.00 1.22 0.07 0.05

Error(DEVICE*WIDTH*DIST) 2376.00

Device × Width × Distance


Selection Times - ANOVAPoint × Select × Width × Distance

Source df F Sig.Eta

Squared

POINT 1.32 70.77 0.00 0.75

Error(POINT) 31.58

SELECT 2.00 91.16 0.00 0.79

Error(SELECT) 48.00

WIDTH 3.00 28.61 0.00 0.54

Error(WIDTH) 72.00

DIST 2.07 8.38 0.00 0.26

Error(DIST) 49.56

POINT * WIDTH 3.45 6.16 0.00 0.20

Error(POINT*WIDTH) 82.77

SELECT * WIDTH 3.69 4.12 0.01 0.15

Error(SELECT*WIDTH) 88.56


Interaction Diagrams - Pointing

Selection times for Pointing Modalities

0

2000

4000

6000

8000

10000

12000

45 55 65 75

Target Width (in pixel)

Sel

ecti

on

Tim

es (

in m

sec)

EyeGaze

Head

Hand

Mouse

HOTAS


Interaction Diagrams

Selection Times for Selection Mechanisms

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

45 55 65 75

Target Width (in pixel)

Sel

ecti

on

Tim

es (

in m

sec)

Adapted

ZAxis

Voice


Analysing Trajectory

Source

Target

Reached Target

Click: End of Task


Main Movement TimeAverage Movement Time

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

Adapt

edET

ZET

Voice

ET

Mm

ET

Adapt

edHT

ZHT

Voice

HT

Adapt

edG

SZGS

Voice

GS

HOTA

S

Mou

se

Input Modality

Tim

e (i

n m

sec)


Time Spent near Target

Hom

ing

Tim

e =

Sel

ectio

n T

ime

– C

urso

r re

ache

d T

arge

t

Average Time Spent Near Target

0

2000

4000

6000

8000

10000

12000

14000

Adapt

edET

ZET

Voice

ET

Mm

ET

Adapt

edHT

ZHT

Voice

HT

Adapt

edG

SZGS

Voice

GS

HOTA

S

Mou

se

Input Modality

Tim

e (i

n m

sec)


Selection Times – Device ComparisonComparing Selection Times

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

0 0.5 1 1.5 2 2.5 3 3.5

ID

Se

lec

tio

n T

ime

(in

ms

ec

)

AdaptedET

HOTAS Joystick

VoiceET

ZET

Mouse

MmET

AdaptedHT

ZHT

VoiceHT

AdaptedHand

ZHand

VHand

Linear (AdaptedET)


Linear (VoiceET)

Linear (ZET)

Linear (Mouse)

Linear (MmET)

Linear (AdaptedHT)

Linear (ZHT)

Linear (VoiceHT)

Linear (AdaptedHand)

Linear (ZHand)

Linear (VHand)


Extra Distance Travelled

Ext

ra D

ista

nce

= T

otal

Dis

tanc

e –

Tar

get A

xis

Leng

th

Average Extra Distance Travelled

0

500

1000

1500

2000

2500

3000

Adapt

edET

ZET

Voice

ET

Mm

ET

Adapt

edHT

ZHT

Voice

HT

Adapt

edG

SZGS

Voice

GS

HOTA

S

Mou

se

Input Modality

Dis

tan

ce (

in p

ixel

)


Wrong Selections

Wrong Selection

0

2

4

6

8

10

12

14

Adapt

edET

HOTA

S Joys

tick

VoiceET

ZET

Mou

se

Mm

ET

Adapt

edHT

ZHTVHT

Adapt

edGS

ZGSVGS

Input Modalities

%W

ron

g S

elec

tio

ns


Cognitive Load

Comparing Average Cognitive Load

0

10

20

30

40

50

60

70

80

Adapt

edET

ZET

VoiceET

Mm

ET

Adapt

edHT

ZHTVHT

Adapt

edGS

ZGSVGS

HOTA

S Joys

tick

Mou

se

Input Modalities

TL

X S

core

s


Spare Mental Capacity

Comparing Average Cognitive Load

0

1

2

3

4

5

6

7

8

Input Modalities

BR

S S

core

s


Synopsis

• Pointing–Mouse << Head Movement << Hand Movement << Eye Gaze << HOTAS Joystick

• Selection–Adaptive << Z-Axis << MmET << Voice

• Cognitive Load–Mouse << AdaptedHT << …<< Joystick << VoiceET–Not enough spare mental capacity for VoiceGS, VoiceET and HOTAS Joystick


Pupil Data Analysis


Maximum Pupil Diameter

Comparing Pupil Diameters

4

4.5

5

5.5

6

6.5

7

0.5 1 1.5 2 2.5 3 3.5

ID

Pup

il D

iam

eter

(in

mm

)

AdaptedET

HOTAS Joystick

VoiceET

ZET

Mouse

MmET

AdaptedHT

ZHT

VHT


Uncertainty Principle


Gabor Limit

https://www.google.co.uk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=_e21A-8-17NxYM&tbnid=U-oTp2x59vjbzM:&ved=0CAUQjRw&url=https%3A%2F%2Fwww.ceremade.dauphine.fr%2F~peyre%2Fnumerical-tour%2Ftours%2Faudio_1_processing%2F&ei=sK7WU9SLB-qr0QXRnIGACg&bvm=bv.71778758,d.ZGU&psig=AFQjCNFh0kSckM9ul9RvEaPC5bS8ZxLgxQ&ust=1406664711430027


Pupil as a wave signal

Pupil Diameter

0

1

2

3

4

5

6

0 50 100 150 200 250 300 350 400

Pupil Diameter

0

1

2

3

4

5

6

7 8 9 10 11 12 13 14 15


Existing work & Patent

•Sudden change in pupil diameter•Methods

–Wavelet Transform–Linear Discriminate Analysis–Neural Network

•Applications–Driving simulation–Aviation –Map reading–ET as passive not active


Power Spectrum

Participants ρP1 0.94

P2 0.93

P3 0.67

P4 0.24

P5 0.73

P6 0.64

P7 0.19

P8 0.51

P9 0.72

P10 0.29

P11 0.76

P12 -0.87

P13 1

Average 0.83


Comparing Correlations with TLX Scores


Power Spectrum vs TLX ScoresAverage TLX Scores vs Maximum Pupil Power

0

100

200

300

400

500

600

700

800

0 10 20 30 40 50 60

TLX Scores

Max

imu

m P

up

il P

ow

er

TLX Scores vs Maximum Pupil Power

0

100

200

300

400

500

600

700

800

900

1000

0 10 20 30 40 50 60 70 80 90

TLX Scores

Max

imu

m P

up

il P

ow

er

P1

P2

P3

P4

P5

P6

P7

P8

P9

P10

P11

P12

P13

Linear (P1)

Linear (P2)

Linear (P3)

Linear (P4)

Linear (P5)

Linear (P6)

Linear (P7)

Linear (P8)

Linear (P9)

Linear (P10)

Linear (P11)

Linear (P12)

Linear (P13)


Power Spectrum vs Selection TimesScatter Plot between Pupil Power and Selection Times

0

10000

20000

30000

40000

50000

60000

0 500 1000 1500 2000 2500 3000

Max Power of Pupil Signal

Sel

ecti

on

Tim

es (

in m

sec)

Relating Maximum Pupil Power to Selection Times for different Modalities

0

10000

20000

30000

40000

50000

60000

0 500 1000 1500 2000 2500 3000 3500

Maximum Pupil Power

Sel

ecti

on

Tim

es (

in m

sec)

AdaptedET

HOTAS Joystick

VoiceET

ZET

Mouse

MmET

AdaptedHT

ZHT

VHT

Linear (AdaptedET)


Linear (VoiceET)

Linear (ZET)

Linear (Mouse)

Linear (MmET)

Linear (AdaptedHT)

Linear (ZHT)

Linear (VHT)


Power Spectrum for different IDs

Comparing Pupil Powers

0

100

200

300

400

500

600

700

800

900

0 0.5 1 1.5 2 2.5 3 3.5

ID

Pu

pil

Po

we

r

AdaptedET

HOTAS Joystick

VoiceET

ZET

Mouse

MmET

AdaptedHT

ZHT

VHT

Linear (AdaptedET)


Linear (VoiceET)

Linear (ZET)

Linear (Mouse)

Linear (MmET)

Linear (AdaptedHT)

Linear (ZHT)

Linear (VHT)


Power Spectrum vs Selection TimesComparing Pupil Powers

0

100

200

300

400

500

600

700

800

900

0 0.5 1 1.5 2 2.5 3 3.5

ID

Pu

pil

Po

we

r

AdaptedET

HOTAS Joystick

VoiceET

ZET

Mouse

MmET

AdaptedHT

ZHT

VHT

Linear (AdaptedET)


Linear (VoiceET)

Linear (ZET)

Linear (Mouse)

Linear (MmET)

Linear (AdaptedHT)

Linear (ZHT)

Linear (VHT)

Comparing Selection Times

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

0 0.5 1 1.5 2 2.5 3 3.5

ID

Se

lec

tio

n T

ime

(in

ms

ec

)

AdaptedET

HOTAS Joystick

VoiceET

ZET

Mouse

MmET

AdaptedHT

ZHT

VoiceHT

Linear (AdaptedET)


Linear (VoiceET)

Linear (ZET)

Linear (Mouse)

Linear (MmET)

Linear (AdaptedHT)

Linear (ZHT)

Linear (VoiceHT)


Comparing Effect Sizes

Comparing Effect Sizes

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

DEVICE WIDTH DIST DEVICE *WIDTH

DEVICE *DIST

WIDTH *DIST

DEVICE *WIDTH *

DIST

Factors

Eff

ect

Siz

e

Selection Time

Max Pupil Power

Max Pupil Diameter

( η

² )


Advantages over Previous work

•FFT over FWT –No need to choose basis wavelets

•Application agnostic evaluation•Works with low frequency (and cheaper) eye gaze tracker

•Validated while ET is used in both active and passive modes

•However distance to target can also affect pupil diameter which may not indicate a change in cognitive load

engineering design centre project updates –august 2014

Documents

engineering design centre

engineering design centre

device width distance

end of task slide

ulnar deviation slide

n 223fbritish281006827

n 646mbritish557863380n

n 823mbritish47957818