visual feedback in the control of reaching movements david knill and jeff saunders

89
Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Upload: zion-waterman

Post on 29-Mar-2015

218 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Visual feedback in the control of reaching movements

David Knill

and

Jeff Saunders

Page 2: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Two types of motor control

• Ballistic

• Feedback control

Page 3: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Motor planning

Physicalplant

Target state

Initial system state

Motorcommands New System

states

Ballistic control

Page 4: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Feedback control

Motor planning

Physicalplant

Target state

Initial System state

Motorcommands New System

states

Sensorysystem

Page 5: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Baseball examples

• Ballistic control– Hitting– Throwing

• Feedback control– Running to catch a ball

Page 6: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 7: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 8: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 9: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 10: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 11: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 12: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 13: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 14: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 15: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 16: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 17: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 18: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 19: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 20: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 21: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 22: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 23: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 24: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 25: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 26: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Target

Page 27: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 100 200 300 400 500 600 700 8000

100

200

300

400

500

600

700

Time (msecs.)

X position (mm)

EOG signal

Finger position

Eye-hand coordination

Page 28: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 100 200 300 400 500 600 700 8000

100

200

300

400

500

600

700

Time (msecs.)

X position (mm)

EOG signal

Finger position

Movement start

Page 29: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 100 200 300 400 500 600 700 8000

100

200

300

400

500

600

700

800

900

1000

Time (msecs.)

Speed (mm/sec)

Speed profile for pointing movement

Page 30: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Motor planning

Physicalplant

Target state

Initial system state

Motorcommands New System

states

Ballistic control

Page 31: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Questions

• Does the visuo-motor system use visual information about target on-line to update motor program?

Page 32: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Feedback control

Motor planning

Physicalplant

Target state

Initial System state

Motorcommands New System

states

Sensorysystem

Page 33: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Questions

• Does the visuo-motor system use visual information about target on-line to update motor program?

• Does the visuo-motor system use continuous feedback from the hand during a movement to control the movement

Page 34: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Questions

• Does the visuo-motor system use visual information about target on-line to update motor program?

• Does the visuo-motor system use continuous feedback from the hand during a movement to control the movement– What visual information is used?

Page 35: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Question

• Does the visuo-motor system use visual information about target on-line to update motor program?

Page 36: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Question

• Does the visuo-motor system use visual information about target on-line to update motor program?

• Yes - for detectable target motion (e.g. catching a moving object)

Page 37: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Question

• Does the visuo-motor system use visual information about target on-line to update motor program?

• Yes - for detectable target motion (e.g. catching a moving object)

• ?? - for imperceptible changes in target position

Page 38: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Experiment

• Perturb position of target during a saccade (imperceptible change)

• Does motor system correct for change in target position?

Page 39: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

infrared markerson finger

tabletop aligned to virtual targets

monitor

mirror

Page 40: Visual feedback in the control of reaching movements David Knill and Jeff Saunders
Page 41: Visual feedback in the control of reaching movements David Knill and Jeff Saunders
Page 42: Visual feedback in the control of reaching movements David Knill and Jeff Saunders
Page 43: Visual feedback in the control of reaching movements David Knill and Jeff Saunders
Page 44: Visual feedback in the control of reaching movements David Knill and Jeff Saunders
Page 45: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400-20

-15

-10

-5

0

5

10

15

X (mm)

Y (mm)

Perturbed trials

Unperturbedtrials

Page 46: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350

-100

-50

0

50

100

X (mm)

Y (mm)

Page 47: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Results

• Automatically correct for imperceptible target perturbations.

• Correct for perturbations – Perpendicular to movement– In direction of movement

• Reaction time = 150 ms

• Smooth corrections

Page 48: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Question

• Does the visuo-motor system use continuous feedback from the hand during a movement to control the movement?

Page 49: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Hypotheses

• Classic model – Ballistic control during “fast” phase of motion– Feedback control during end, “slow” phase of

motion

• Continuous model– Feedback control throughout movement

Page 50: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Arguments against continuous feedback

• Visuo-motor delay (~100 ms) is too large for effective control during fast phase.

• Removing vision of hand early in motion does not affect end-point error.

• Corrections to target perturbations are just as strong with or w/o vision of hand.

Page 51: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Experiment

• Imperceptibly perturb the position of the hand during a movement and measure motor response.

• Add perturbations early and late in pointing movement.

• Measure reaction time to perturbations.

Page 52: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

infrared markerson finger

tabletop aligned to virtual targets

monitor

mirror

Page 53: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

(c)

(b)

target

unseenhand

virtual fingertip

(a)

Page 54: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Reaction time predictions

End-phasefeedback

Continuousfeedback

Lateperturbation

Earlyperturbation

Page 55: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

X(cm)

0 5 10 15 20 25 30

2

0

-2

X(cm)

0 5 10 15 20 25 30

2

0

-2

X(cm)

0 5 10 15 20 25 30

2

0

-2X(cm)

0 5 10 15 20 25 30

2

0

-2

X(cm)

0 5 10 15 20 25 30

2

0

-2X(cm)

0 5 10 15 20 25 30

2

0

-2

Fast reaches (~450ms)

Fast reaches (~450ms) Slow reaches (~600ms)

Slow reaches (~600ms)

Early perturbation

Mid-reach perturbation

Early perturbation

Mid-reach perturbation

Early perturbation

Mid-reach perturbation

Early perturbation

Mid-reach perturbation

X(cm)

0 5 10 15 20 25 30

2

0

-2

X(cm)

0 5 10 15 20 25 30

2

0

-2

Page 56: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

-5 0 5 10 15 20 25 30-8

-6

-4

-2

0

2

4

6

X position (cm)

Y position (cm)

Sample finger paths

Page 57: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Autoregressive model

• Baseline (unperturbed) trajectories

• Perturbed trials

y(t) =w8y(t−8)+w7y(t−7)+L +w1y(t−1)

y(t) =w8y(t−8)+w7y(t−7)+L +w1y(t−1)+wP (t)ΔY

Page 58: Visual feedback in the control of reaching movements David Knill and Jeff Saunders
Page 59: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0

0 100 200 300 400 500time after perturbation (ms)

0

0 100 200 300 400 500time after perturbation (ms)

0 100 200 300 400 500time after perturbation (ms)

0

0 100 200 300 400 500time after perturbation (ms)

0

0 100 200 300 400 500time after perturbation (ms)

0

0 100 200 300 400 500time after perturbation (ms)

Fast reaches (~450ms)

Fast reaches (~450ms) Slow reaches (~600ms)

Slow reaches (~600ms)

Early perturbation

Mid-reach perturbation

Early perturbation

Mid-reach perturbation

Early perturbation

Mid-reach perturbation

Early perturbation

Mid-reach perturbation

0

0 100 200 300 400 500time after perturbation (ms)

0

0 100 200 300 400 500time after perturbation (ms)

0

Page 60: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Early Mid-reach0

20

40

60

80

100

120

140

160

180

200

Fast reaches

Early Mid-reach

Slow reaches

N = 6 N = 6

Page 61: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 100 200 300 400 500 600 700 800 900 10000

5

10

15

20

25

30

Time (ms)

X position (cm)

Subject 1: Trajectories for early perturbed trials

Positive perturbations

Negative perturbations

Page 62: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 100 200 300 400 500 600 700 800 900 100020

21

22

23

24

25

26

27

28

29

30

Time (ms)

X position (cm)

Subject 1: Trajectories for early perturbed trials

Positive perturbations

Negative perturbations

Page 63: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 100 200 300 400 500 600 700 800 900 10000

5

10

15

20

25

30

Time (ms)

X position (cm)

Subject 1: Trajectories for late perturbed trials

Positive perturbations

Negative perturbations

Page 64: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 100 200 300 400 500 600 700 800 900 100020

21

22

23

24

25

26

27

28

29

30

Time (ms)

X position (cm)

Subject 1: Trajectories for late perturbed trials

Positive perturbations

Negative perturbations

Page 65: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 100 200 300 400 500 600 700 800 900 10000

5

10

15

20

25

30

Time (ms)

X position (cm)

Subject 2: Trajectories for early perturbed trials

Positive perturbations

Negative perturbations

Page 66: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 100 200 300 400 500 600 700 800 900 10000

5

10

15

20

25

30

Time (ms)

X position (cm)

Subject 2: Trajectories for late perturbed trials

Positive perturbations

Negative perturbations

Page 67: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450 500-3

-2

-1

0

1

2

3x 10-3

Time (ms)

Perturbation weight

Perturbation weight function for in-line perturbations

Page 68: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450 500-4

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1x 10-3

Time (ms)

Perturbation weight

Perturbation weight function for perpendicular perturbations

Page 69: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Conclusions

• Visuomotor system uses directional error signal for feedback control?

• Position / speed error in direction of movement is not effective feedback signal?

• Why?– Position along path blurred by motion– Insensitivity to acceleration along direction of

motion

Page 70: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Question

• What visual information about hand does visuomotor system use– Position error?– Motion error?– Position and motion?

Page 71: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

(c)

(b)

target

unseenhand

virtual fingertip

(a)

Page 72: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

(c)

(b)

target

(a)

Page 73: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-12

-10

-8

-6

-4

-2

0

2

4x 10-4

Time (msecs.)

Perturbation weight

Page 74: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-4

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1x 10-3

Time (msecs.)

Perturbation weight

Page 75: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5x 10-3

Time (msecs.)

Perturbation weight

Page 76: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-2.5

-2

-1.5

-1

-0.5

0

0.5x 10-3

Time (msecs.)

Perturbation weight

Page 77: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-4

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1x 10-3

Time (msecs.)

Perturbation weight

Page 78: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

0.5

1x 10-3

Time (msecs.)

Perturbation weight

Page 79: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

Conclusions

• Visuomotor system uses continuous visual feedback to control reaching movements.

• Feedback signals include positional error.

• Feedback signals include motion error.

• System is approximately linear.

Page 80: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-6

-4

-2

0

2

4

6

8x 10-3

Time (msec.)

Perturbation weight

Page 81: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-0.06

-0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

Time (msec.)

Cumulative perturbation weight

Page 82: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-16

-14

-12

-10

-8

-6

-4

-2

0

2

4x 10-3

Time (msec.)

Perturbation weight

Page 83: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-0.35

-0.3

-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

Time (msec.)

Cumulative perturbation weight

Page 84: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-12

-10

-8

-6

-4

-2

0

2

4

6x 10-3

Time (msec.)

Perturbation weight

Page 85: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-0.2

-0.15

-0.1

-0.05

0

0.05

Time (msec.)

Cumulative perturbation weight

Page 86: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-12

-10

-8

-6

-4

-2

0

2

4

6x 10-3

Time (msec.)

Perturbation weight

Page 87: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-0.25

-0.2

-0.15

-0.1

-0.05

0

0.05

0.1

Time (msec.)

Cumulative perturbation weight

Page 88: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-10

-8

-6

-4

-2

0

2

4

6

8x 10-3

Time (msec.)

Perturbation weight

Page 89: Visual feedback in the control of reaching movements David Knill and Jeff Saunders

0 50 100 150 200 250 300 350 400 450-0.02

-0.015

-0.01

-0.005

0

0.005

0.01

Time (msec.)

Perturbation weight