predictive adaptation to dynamic environments and ... · impedance adaptation the impedance and...

Koji ItoRitsumeikan University, Kyoto, Japan

Predictive Adaptation to Dynamic Environments and Application to Motor Rehabilitation

University of Naples Federico II: March 11, 2013

2

1. Introduction- Redundancy among Brain-Body-Environment

2. Experiments of arm reaching movements -Adaptation to dynamic environments-

3. EEG – FES system for stroke patients 4. Conclusion

Contents

3

Basic controlmechanism

Neural mechanisms for motor control

Cerebellum

Motor learning & adaptation

Fine tuning

Motor program

Selection

Basal ganglia

Somatosensory / visual feedback

Somatosensory / visual feedback

Brainstem

Spinal cord

MusculoskeletalSystem

Cerebral Cortex

4

External Dynamics

Internal Dynamics

Dynamic interaction among Brain-Body-Environment

BodyMotorOutput

SensoryInput

Environment

Brain

Internal dynamics has much redundancy.

It is required to adjust the internal dynamics before beginning voluntary movements.

How to reduce redundancy ofsensorimotor mapping?

5

Controller

Identification of external dynamics

Feedback Controller

Gc

yd (t)Goal

Sensory feedback

+

FeedforwardController

GF

+ y (t)

MotoroutputMotor

commandu(t)

BodyGp

Environment

Identification

External Dynamics

6Motor control

Internal Model

Motor Command

Sensory Feedback

Brain

Body & Environments

1) Simulation of body-environment dynamics2) Prediction of sensory feedback

7

Internal Dynamics

Controller

)(ˆ ty

FeedforwardController

+

Prediction

+- y (t)

u(t)Body

Environment

Arm Impedance

External Dynamics

Internal ModelInternal ModelEfference copyu(t)

Sensory feedbacksTime Delay

)( tt Δy

Conceptual control model of human movements

Targetyd(t) Feedback

Controller+

Time Delay)( tt Δy

Sensory feedbacks

Feedforward & Feedback

8

Experiments of Arm Reaching Movements -Adaptation to dynamic environments-

9Adaptation to dynamic environments

Arm Reaching MotionUnknown environments

A B C

Computational Model of Adaptation to EnvironmentsModeling of motor adaptation and learning

Neuro-physiological Approach- Brain imaging during reaching motions- Development of new equipments

10Velocity-dependent force field (VF)

yx

yx

FF

y

x

13181813

32B

(by Shadmehr, 1994)

11Hand trajectories and hand force

Initial phase After learning After-effect

Hand trajectories Hand force

After learning

12Hand trajectories and EMG during reaching motion

1 st time

72 nd time

Initial position

(c)Triceps Time(sec)

EMG (average of 24 subjects）

EMG

(b)Biceps Time(sec)

ModifiedFirst 8 timesLast 8 times

Free motion

(a) Hand trajectories

Target (45ﾟ)

Force field

(by Shadmehr, 1999)

13Unstable (divergent) force fields (DF)

yx

kk

00

F

0dx

FF

y

x β>0

x [m]

y[m

]

0.1

0.2

0- 0.1 0.1

(E. Burdet et al, Nature, 2001)

14Hand trajectories and forces in DF

Hand trajectories Hand force

15Muscle activities (EMG) during reaching motions

* : P<0.05, ** : P<0.01, *** : P<0.001

NF: Free motionDF: Unstable force fieldAE: After-effects

Posteriordeltoid

Triceps long head

Tricepslateral head

Pectoralis major

Bicepsbrachii

Brachio radialis

(Franklin, Exp Brain Res., 2003)

The subject plans the muscle activities before beginning the reaching motion.

EMG under After-effects is similar to the unstable force field.

msec msec msec

16

Question?Can we program the internal modelcontrol and impedance control in afeedforward manner?

1) We plans Internal Model Control before beginningthe reaching movements under the dynamicenvironment.

2) We presets Impedance Control before beginningthe reaching movements.

17Switched force fields

(a) SF1 (VF→DF)

x(0, 0)

(0, 0.2)

y

VF

DF

(b) SF2 (DF→VF)

DF

VF

(0, 0)

(0, 0.2)

y

x

VF→DF （SF1）

Hand forceBefore learning

After learning

DF→VF （SF2）

Hand forceBefore learning

After learning

Impedance Adaptation

The impedance and internal-modelcontrols are programmed in afeedforward manner in adaptation to thecontexts of dynamic environments.

Program Adaptation0.2 m

Internal Model Adaptation

Internal Model Adaptation

Impedance Adaptation

18

19

Experimental results suggests that the predictive adaptationto the environment dynamics is composed of three levels:

Three adaptation levels of motor control

[1] Impedance adaptation→Parameter adaptation

[2] Internal model adaptation→State dynamics adaptation

[3] Program adaptation→Context adaptation

20

Motor Controller

Lateralpart

Intermediatepart & vermis

CerebellumInternal model adaptation

Programadaptation

Limb/Body

Muscleviscoelasticity

Spinalreflex system

+ +

Environment

External dynamics

Cerebral cortex

Prefrontalcortex

Supplementarymotor area

Premotorcortex

Parietalcortex

Motorcortex

Basalganglia

Brain stem-Spinal system

Impedance adaptation

Motor adaptation mechanisms

Electroencephalogram (EEG) –Functional Electrical Stimulation (FES) System

for Stroke Patients

22Motor/Neuro rehabilitation

FESPhysical therapy Practice for walking

It is not strictly to recover the motor performance as it was, but reconstruct optimal motor performance in the new situation

→ Re-optimization

It is essential to give the opportunity for motivated and intensivepractice and exercise in a stimulating environment.

23Motor intention and sensory feedback for motor rehabilitation

Visual & Auditory feedbacks

Prefrontalmotor cortex


Parietalcortex

Parietalcortex



Primarymotorcortex

Primarymotorcortex

Primarysomatosensory

cortex


cortex

Basal ganglia&

Cerebellum

Basal ganglia&

CerebellumMotor Intention Sensory feedbacks

Proprioceptive feedbacksMotor command

24Reconstruction of motor function

EEG-BCI EEG-FES EMG-FES Leg powered wheelchair

Target Spinal injury (paraplegia)

Spinal injury / stroke (paraplegia)

Stroke(hemiplegia)

Stroke(hemiplegia)

Object Transmission of intention

Transmission of motor intention

Reconstruction of motor function

Reconstruction of motor function

How to

Detect the motor intention from EEG at the motor imagery of arm or leg

Regain movements based on the motor intention from EEG at the motor imagery of arm or leg

Activate the muscles by FES based on the motor intention from EMG

Realize the somato-sensory feedback bypedaling the wheelchair

Advantage

Voluntary EMG is not required

Learning effects of voluntary somato-sensory feedbacks

Learning effects of somatosensory feedbacks & enhancement of motivation

Weak points

Not motor learning Difficult to detect motor intention from EEG

Voluntary EMG activity is required.

Effect on gait disorder is not clear.

Proposed EEG-FES system for motor rehabilitation

Cerebral cortex

Motor cortex

Somatosensoryarea

Premotor Cortex &

Supplementarymotor area

Parietalcortex

Basal ganglia


Environment

Sensory feedback

Motor command

EEG FES

－

+

Cerebellum

Proprioceptivefeedback

Musculo-Skeletal system

Muscle ActuationMotor Intention

25

26Event Related Desynchronization(ERD) The characteristics of ERD are as follows.

Decreasing of electric potential in specific frequency band (alpha (8-15Hz), beta (15-35Hz) band)

Observed during motion and motor imagery Activation of cortico-thalamus loop?

ERD detection. Band pass filtered (25-30Hz) Full wave rectified and

50 times sum averaged

(Lopes da Silva, 1991)

0 2 4 6 8 100

1

2

3

[μV

]

[sec]

Motor Imagery

ERD

ERS10-20 methods

Foot motor area (Cz)

27

Basic Experiments

28Experimental setup

Multi telemeter system

Computer screenEEG

FES

Cue presented (3 seconds)

Blank screen(3 seconds)……

EEG measurement during FES activation on both quadriceps. 17 healthy subjects participated in the experiment. EEG from 7 electrodes (around Cz area) were measured. One task consisted of 50 trials of FES.

Foot motor area (Cz)

Motorimagery

29Frequency analysis

500 msec

Elec

tric

pote

ntia

l[μ

V]

Time [sec]3

Pow

er

spec

trum

24 26 2822 frequency [Hz]20

2 Hz

Ensemble average

The data for each 3 seconds is divided into 500 msec data. After converting to frequency space, the ensemble average is

calculated. To quantify the amount of ERD, r2 value was calculated for the

frequency band for 24 – 26 Hz.

30Power spectrum before and after motor imagery training

No motor imagery

Motor imagery

0 10 20 30 40 50Frequency [Hz]

10

10

10

Pow

er sp

ectru

m

1

2

3

(a) Before training (b) After 3 days training

0 10 20 30 40 50Frequency [Hz]

10

10

10

Pow

er sp

ectru

m

1

2

3

No motor imagery

Motor imagery

31r2 value

frequency [Hz]24 26

Absence of FES

Presence of FES

x group

y group

x

y

yx ,

Pow

er sp

ectru

m r2 value is used to calculate the amount of ERD (This method is

popular in BCI researches). r2 value uses the within- and between- variance with two groups. Larger r2 value mean the clearer detection of ERD.

yx

yx

nnyxb

byx

bny

nx

yxr

22

22

22

2

)()(

),(

32Training effects of r2 value

Before training 1st day 2nd day 3rd day

0.3

0.2

0.1

0

-0.1

r2va

lue

* * p<0.05

33

Training results of a stroke patient

34Experiment set-up

FES

Telemeter(sender)

OptotrackPC

Telemeter(reciver)

main PC

Optotrack camera

Subject：Brain stem infarction. 30 months after stroke. Paralyzed on the left side of the body (Foot-pat test：0/6)

35Experiment

5 min. 5 min.30 min.

36Training by EEG-FES system

Intend to move left

foot

FES

outp

ut [m

A]

10

0

FES

500 msec

ERD

Sensory feedback

37Before training

38After training

39Leg movements

Paralyzed side Normal side

RightLeft

Ankle joint

Left Right

Knee joint

40

Statistical significance of ankle joint movements

(a) Paralyzed side (b) Normal side

41

70

60

50

40

Ang

le [d

eg]

0 2 4 6 8 10Time [sec]

70

60

50

40

Ang

le [d

eg]

0 2 4 6 8 10Time [sec]

Bef

ore

train

ing

Ankle joint movements


70

60

50

40

Ang

le [d

eg]

0 2 4 6 8 10Time [sec]

70

60

50

40

Ang

le [d

eg]

0 2 4 6 8 10Time [sec]

Afte

r tra

inin

g

42

0 2 4 6 8 10Time [sec]

80

60

40

20

Elec

tric

pote

ntia

l [μV

]

0

100

0 2 4 6 8 10Time [sec]

80

60

40

20

Elec

tric

pote

ntia

l [μV

]

0

100B

efor

e tra

inin

g

0 2 4 6 8 10Time [sec]

80

60

40

20

Elec

tric

pote

ntia

l [μV

]

0

100

0 2 4 6 8 10Time [sec]

80

60

40

20El

ectri

c po

tent

ial [μV

]

0

100

Afte

r tra

inin

g

Muscle activity levels


43Statistical significance of muscle activities

(a) Paralyzed side (b) Normal side

44Brain image

Brain stem

45Conclusion

EEG FES

EEG-FESSystem

Sensory feedbacks

Motor Intention

New non-invasive brain activity measurement

Sensory FeedbacksRobotics

Haptic devicesVariable impedance

46Motor intention and sensory feedback for motor rehabilitation

Visual & Auditory feedbacks



Parietalcortex

Parietalcortex



Primarymotorcortex


cortex

Basal ganglia&

Cerebellum

Basal ganglia&

CerebellumMotor Intention Sensory feedbacks

Proprioceptive feedbacksMotor command

47

Mitsuru Takahashi (Terumo corporation)

Kotaro Takeda (ATR Computational Neuroscience Laboratories)

Rieko Osu (ATR Computational Neuroscience Laboratories)

Kotaro Otaka (Keio University, Medical School Hospital)

Takashi Hanakawa (National Center of Neurology and Psychiatry)

Toshiyuki Kondo (Tokyo University of Agriculture and Technology)

Joint researches with

M. Takahashi, K. Takeda, Y. Otaka, R. Osu, T. Hanakawa, M. Gouko and K. Ito: Event related desynchronization-modulated functional electrical stimulation system for stroke rehabilitation: A feasibility study,Jr. of NeuroEngineering and Rehabilitation, 9:56 (16 Aug 2012)

Journal paper

48

Grazie mille per attenzione!Thanks a lot!

どうもありがとう(Domo Arigato)!

predictive adaptation to dynamic environments and ... · impedance adaptation the impedance and...

Documents