a functional electrical stimulation (fes...

A FUNCTIONAL ELECTRICAL STIMULATION

(FES) CONTROL SYSTEM FOR UPPER LIMB

REHABILITATION

MINGXU SUN

Thesis submitted to the School of Computing, Science and Engineering in

partial fulfilment of the requirements for the degree of

Doctor of Philosophy

School of Computing, Science and Engineering

University of Salford

Salford, UK, 04/2014

i

Contents

Contents .......................................................................................................................... i

List of Tables ............................................................................................................... vii

List of Figures ............................................................................................................ viii

Acknowledgements ...................................................................................................... xv

Abbreviations .............................................................................................................. xvi

Abstract ...................................................................................................................... xvii

Chapter 1 – Introduction ................................................................................................ 1

1.1 Research objectives .............................................................................................. 3

1.2 Overview of the thesis ......................................................................................... 3

Chapter 2 - Literature review ......................................................................................... 6

2.1 Upper limb rehabilitation following stroke.......................................................... 6

2.1.1 Stroke and the consequences for the upper limb .......................................... 6

2.1.2 Outcome measures of arm usage in everyday life ........................................ 7

2.1.3 Rehabilitation of the upper limb after stroke ................................................ 7

2.1.3.1 Electromyographic (EMG) biofeedback ................................................ 7

2.1.3.2 Constraint-induced movement therapy (CIMT) .................................... 8

2.1.3.3 Robotic-assisted therapy ........................................................................ 9

2.1.3.4 Mental practice with motor imagery .................................................... 10

2.2 Electrical stimulation ......................................................................................... 12

ii

2.2.1 Introduction ................................................................................................. 12

2.2.2 Excitable tissues .......................................................................................... 12

2.2.2.1 Structure of a nerve and motor unit ..................................................... 12

2.2.2.2 Electrochemical balance in a nerve cell ............................................... 14

2.2.2.3 Action potentials and their propagation ............................................... 14

2.2.2.4 Muscle fiber contraction ...................................................................... 16

2.2.3 Electrical stimulation of excitable tissues ................................................... 17

2.2.3.1 Stimulation parameters ........................................................................ 17

2.2.3.2 Stimulus pulse amplitude and duration ................................................ 18

2.2.3.3 Stimulus pulse frequency ..................................................................... 20

2.2.3.4 Simulation waveforms ......................................................................... 21

2.2.3.5 Factors affecting the spatial distribution of stimulating current .......... 22

2.2.3.6 Ramp time for stimulation ................................................................... 22

2.2.4 Functional electrical stimulation ................................................................. 23

2.2.5 FES applications for the upper extremity ................................................... 24

2.2.5.1 ES systems that are not under voluntary control, or are push-button

controlled ......................................................................................................... 25

2.2.5.2 FES systems that are controlled via voluntary effort from the user .... 26

2.2.5.3 Summary .............................................................................................. 31

2.3 FES control ........................................................................................................ 35

2.3.1 Introduction ................................................................................................. 35

2.3.2 Sensors for control of FES systems ............................................................ 35

iii

2.3.3 Continuous closed-loop control of FES ...................................................... 36

2.3.4 Open-loop control of FES ........................................................................... 42

2.3.5 Conclusion .................................................................................................. 44

2.4 Finite state machine (FSM) control ................................................................... 46

2.4.1 FSM principle ............................................................................................. 46

2.4.2 Finite state control....................................................................................... 49

2.5 Motion tracking and inertial sensors .................................................................. 56

2.5.1 Introduction ................................................................................................. 56

2.5.1.1 Motion tracking using only accelerometers ......................................... 57

2.5.1.2 Motion tracking using only gyroscopes ............................................... 58

2.5.1.3 Using sensor fusion with inertial and magnetic sensors ...................... 59

2.5.2 Deriving angle from accelerometers ........................................................... 59

2.5.2.1 Using single axis accelerometers ......................................................... 59

2.5.2.2 Using dual axis accelerometers ............................................................ 62

2.5.2.3 Using two accelerometers separated by a rod ...................................... 63

2.5.3 Conclusion .................................................................................................. 63

2.6 Conclusions ........................................................................................................ 64

Chapter 3 – Angle tracking for state transitions .......................................................... 66

3.1 Introduction ........................................................................................................ 66

3.2 First uncalibrated angle tracking method ........................................................... 67

3.2.1 Mathematical procedure.............................................................................. 69

iv

3.2.2 Testing and discussion of results ................................................................ 73

3.3 Second uncalibrated angle tracking method ...................................................... 76



3.4 Calibrated angle tracking method ...................................................................... 80


3.4.2 Initial testing ............................................................................................... 82

3.5 Auto-calibration of Xsens axes .......................................................................... 83



3.6 Conclusions ........................................................................................................ 93

Chapter 4 – Robust angle triggering algorithms .......................................................... 95

4.1 Introduction ........................................................................................................ 95

4.2 Algorithms to improve robustness of angle triggering ...................................... 96

4.2.1 Using the change in angle since entering a state......................................... 96

4.2.2 Ignoring readings where the acceleration vector is significant ................... 98

4.2.3 Requiring a given number of consecutive or non-consecutive valid readings

.............................................................................................................................. 99

4.3 Experimental protocol ...................................................................................... 103

4.3.1 Subject....................................................................................................... 103

4.3.2 Instrumentation ......................................................................................... 104

4.3.3 Experimental procedure ............................................................................ 105

v

4.3.4 Data processing ......................................................................................... 106

4.4 Results .............................................................................................................. 108

4.4.1 Change in angle since entering a state vs. absolute angle......................... 110

4.2.2 Ignoring readings where the acceleration vector is significant ................. 115


............................................................................................................................ 118

4.5 Conclusions ...................................................................................................... 123

Chapter 5 – Finite state machine controller for upper limb FES ............................... 126

5.1 Introduction ...................................................................................................... 126

5.2 Functionality of the FSM controller ................................................................ 126

5.2.1 Movement phases and stimulation control ............................................... 129

5.2.2 Transitions................................................................................................. 136

5.2.3 Dealing with early transitions ................................................................... 137

5.3 Implementation of the FSM controller ............................................................ 139

5.4 Testing the FSM controller .............................................................................. 149

5.4.1 Initialisation prior to testing ...................................................................... 149

5.4.2 Test results for the “open a door” task ...................................................... 150

5.5 Graphical user interface for setup of FSM controller ...................................... 154

5.5.1 Breaking the setup process into logical stages .......................................... 154

5.5.2 Stage 1 – Create, modify and select tasks ................................................. 156

5.5.3 Stage 2 – Don electrodes and sensors, and initial channel setup .............. 158

vi

5.5.4 Stage 3 – Set up stimulation parameters for each phase and capture manual

transitions data ................................................................................................... 160

5.5.5 Stage 4 – Setting up transition conditions................................................. 161

5.5.6 Stage 5 – Practising functional tasks ........................................................ 162

5.6. Conclusions ..................................................................................................... 163

Chapter 6 – Discussion .............................................................................................. 164

6.1 Limitations with existing FES systems ............................................................ 164

6.2 Angle tracking methods and methods to improve robustness ......................... 165

6.3 FSM and Setup GUI......................................................................................... 167

6.4 Comprehensive testing of the FES control system for upper limb rehabilitation

................................................................................................................................ 169

Chapter 7 – Conclusions ............................................................................................ 171

7.1 Summary of the research ................................................................................. 171

7.2 Achievements ................................................................................................... 174

7.3 Future work ...................................................................................................... 175

Appendix A ................................................................................................................ 177

A.1 Ethics statement .............................................................................................. 177

References .................................................................................................................. 179

vii

List of Tables

Table 2.1: Functional properties of existing FES systems ........................................... 33

Table 2.2: A typical description of attributes for the example FSM ........................... 47

Table 2.3: Attributes of a FSM for hand grasp and release ......................................... 51

Table 3.1: Average errors in test 1 and test 2 compared with the protractor angles of

30°, 60°, 90°, 150° and 180° ........................................................................................ 75

Table 3.2: Average errors in test compared with the protractor angles of 30°, 60°, 90°,

150° and 180° ............................................................................................................... 79

Table 3.3: Scaling of original accelerometer data ....................................................... 88

Table 4.1: Profile of subject ....................................................................................... 103

Table 4.2: Number of early triggers when using the data from 10 repeated trials for

the “Brush coins into other hand” task and from 8 repeated trials for the “Drink from a

cup” task..................................................................................................................... 122

Table 4.3: Mean delays in triggering for the three methods over all trials ................ 123

Table 5.1: Stimulation details for the FSM shown in Figure 5.2, including the outputs

in each phase .............................................................................................................. 133

Table 5.2: Transition rules for the example task “open a door” ................................ 137

Table 5.3: Transition conditions for leaving each phase ........................................... 146

Table 5.4: Stimulation settings for each channel that do not vary with phase ........... 147

Table 5.5: Stimulation settings for each channel that differ between phases ............ 147

Table 5.6: Stimulation parameters for each channel and each phase ........................ 150

viii

List of Figures

Figure 2.1: The basic structure of a nerve cell and the communication between nerve

cells .............................................................................................................................. 12

Figure 2.2: Structure and composition of muscles....................................................... 13

Figure 2.3: The resting potential of a nerve fiber. ....................................................... 14

Figure 2.4: (a) Membrane potential response of a nerve cell over time to a small

depolarizing stimulation; (b) Response to a stronger depolarizing stimulation; and (c)

Response to a depolarizing stimulation over the threshold of excitation .................... 15

Figure 2.5: (a) The action potential of peak voltage level of 40 mV causes current

flow to the negative area of the axon. The flow of current acts as the depolarizing

stimulation over the threshold of excitation; and (b) when the threshold of excitation

is achieved, another action potential will be elicited, and this process leads to the

propagation of the action potential along the axon ...................................................... 16

Figure 2.6: Three main parameters for FES................................................................. 17

Figure 2.7: Muscle force against current amplitude .................................................... 18

Figure 2.8: The relationship between pulse duration and current amplitude and

stimulation response..................................................................................................... 19

Figure 2.9: Common stimulation waveforms of different shapes................................ 21

Figure 2.10: Stimulation ramp times. .......................................................................... 23

Figure 2.11: State machine control of Saxena, Nikolic et al.’s EMG triggered system

...................................................................................................................................... 27

Figure 2.12: State machine control of Lauer, Peckham et al.’s EEG triggered system

...................................................................................................................................... 29

ix

Figure 2.13: A glove with sensors detects the degree of unaffected hand opening, and

stimulation of the finger and thumb extensor muscles will produce proportional

opening of affected hand .............................................................................................. 30

Figure 2.14: The Bionic Glove .................................................................................... 31

Figure 2.15: A model of a generic closed-loop FES control system ........................... 37

Figure 2.16: The four-layered neural network control system in the Kurosawa, Futami

et al.’s feedback error learning controller .................................................................... 38

Figure 2.17: The Hammerstein muscle model ............................................................. 39

Figure 2.18: Example of a closed-loop EMG control system ...................................... 40

Figure 2.19: Overview of experimental setup.............................................................. 41

Figure 2.20: The NeuroControl Freehand system........................................................ 43

Figure 2.21: Two types of output – Moore output and Mealy output .......................... 46

Figure 2.22: A typical example of a FSM.................................................................... 47

Figure 2.23: An example of a FSM which has two possible next states to enter ........ 48

Figure 2.24: An example FSM showing four different output actions ........................ 49

Figure 2.25: FSM for hand grasp and release .............................................................. 52

Figure 2.26: Screenshot of the CST GUI showing a FSM design for a drinking task 53

Figure 2.27: Example of a FSM for drinking from a glass task .................................. 54

Figure 2.28: FSM used for control of hand grasp ........................................................ 54

Figure 2.29: The activation of states controlled by MES signals ................................ 55

Figure 2.30: Example of a FSM for a task following a state sequence of Hold, Hand

close, Hold, Hand open ................................................................................................ 56

Figure 2.31: Angle calculation based on measurement on the sensitive axis and

gravity .......................................................................................................................... 61

Figure 2.32: Angle calculation based on measurements on the x and z axes .............. 62

x

Figure 3.1: The gravity vectors and in the global frame, before (left)

and after (right) the Xsens sensor has rotated, and their components ......................... 68

Figure 3.2: The gravity vectors and in the Xsens frame separated by the

rotation angle . .......................................................................................................... 69

Figure 3.3: The angle ranges where the cross product and the dot product are applied

...................................................................................................................................... 72

Figure 3.4: The protractor system used for testing ...................................................... 73

Figure 3.5: Calculated angle in degrees in test 1. The horizontal lines indicate ,

, , and .............................................................................................. 74


, , and .............................................................................................. 74

Figure 3.7: Angle between (Xsens x-axis) and (vertical) .......................... 77

Figure 3.8: Calculated angle in degrees. The horizontal lines indicate 0 , 30 , 60 ,

90 , 150 and 180 ..................................................................................................... 79

Figure 3.9: Cosine of the angle between the rotation planes ( ) ........................... 82

Figure 3.10: Corresponding magnitude of Xsens angle change ( ) ........................... 83

Figure 3.11: Corrected gravity vector and its components .......................................... 85

Figure 3.12: Accelerometer data-set 1 ......................................................................... 89




Figure 4.1: Setup of state machine for test purposes ................................................... 96

Figure 4.2: Change in angle since entering state ..................... 97

Figure 4.3: Change in angle since entering a state....................................................... 97

Figure 4.4: State number and timing of state transitions ............................................. 97

xi

Figure 4.5: Magnitude of measured g-vector ............................................................... 99

Figure 4.6: Acceleration flag value .............................................................................. 99

Figure 4.7: Transition condition must be satisfied for six consecutive readings ....... 100

Figure 4.8: Six consecutive valid readings that satisfy the g-tolerance trigger the

transition. The readings marked by the red circles are either not satisfying the

transition condition (angle below threshold) or failing to meet the g-tolerance

requirement. ............................................................................................................... 100

Figure 4.9: Six non-consecutive valid readings trigger the transition. The readings

marked by the red circles are either not satisfying the transition condition (angle

below threshold) or failing to meet the g-tolerance requirement. .............................. 101

Figure 4.10: Acceleration flag value .......................................................................... 102

Figure 4.11: Six consecutive valid readings to trigger a transition. Bad readings are

rejected ....................................................................................................................... 102

Figure 4.12: Count of consecutive valid points ......................................................... 103

Figure 4.13: Xsens inertial sensing units with clusters of four reflective markers

attached ...................................................................................................................... 104

Figure 4.14: Experimental setup showing Xsens units with reflective marker clusters

.................................................................................................................................... 105

Figure 4.15: FSM controller for the “Brush coins into other hand” task. The

transitions between states were manually triggered by the therapist ......................... 106

Figure 4.16: FSM controller for the “Drink from a cup” task. The transitions between

states were manually triggered by the therapist ......................................................... 106

Figure 4.17: Vicon graphic showing marker clusters on the upper arm and forearm

respectively ................................................................................................................ 107

xii

Figure 4.18: Example data for upper arm angle from the vertical, obtained from both

Vicon and Xsens systems, for the “Brush coins into other hand” task ...................... 109

Figure 4.19: Example data for forearm angle from the vertical, obtained from both

Vicon and Xsens systems, for the “Brush coins into other hand” task ...................... 110

Figure 4.20: Artificial misalignment of the Xsens unit relative to the Vicon gold

standard ...................................................................................................................... 111

Figure 4.21: Example data for upper arm “absolute angle from the vertical” during the

“Brush coins into other hand” task, obtained from the Vicon gold standard and the

artificially misaligned Xsens data .............................................................................. 112

Figure 4.22: Example data for upper arm “change in angle since entering a state”

during the “Brush coins into other hand” task, obtained from the Vicon gold standard

and the artificially misaligned Xsens data ................................................................. 113

Figure 4.23: Error in “absolute angle from the vertical” for the example trial of the

“Brush coins into other hand” task ............................................................................ 113

Figure 4.24: Error in “change in angle since entering a state” for the example trial of

the “Brush coins into other hand” task ...................................................................... 114

Figure 4.25: Mean error for both arm segments for the “Brush coins into other hand”

task (n=10) and the “Drink from a cup” task (n=8). These are the means of n trial-

means and the error bars indicate standard deviation over the n trials ...................... 115

Figure 4.26: Error in forearm “absolute angle from the vertical” for the example trial

of the “Drink from a cup” task before removal of bad readings ................................ 116


of the “Drink from a cup” task after removal of bad readings using a g-tolerance of

................................................................................................................. 116

xiii


of the “Drink from a cup” task after removal of bad readings using a g-tolerance

of ............................................................................................................. 117

Figure 4.29: Mean maximum error for both arm segments for the “Brush coins into

other hand” task (n=10) and the “Drink from a cup” task (n=8). These are the means

of n trial-maximums and the error bars indicate standard deviation over the n trials 118


during the “Brush coins into other hand” task. The suitable angle threshold for the

“Brush coins into other hand” task is 25°. The transition timing determined by

therapist is the 3.45 second, and the corresponding angle threshold is 32° (1 good

reading over the angle threshold to trigger). .............................................................. 120

Figure 4.31: Choosing a suitable angle threshold for the “Brush coins into other hand”

task by gradually decreasing the angle threshold in 1° steps ..................................... 120

Figure 4.32: Choosing a suitable angle threshold for the “Drink from a cup” task by

gradually decreasing the angle threshold in 1° steps ................................................. 121

Figure 4.33: The measured accelerometer vector ( ) is the sum of the true

acceleration vector ( ) and gravity ( ). Although is significant in

comparison to , the g-tolerance is still satisfied......................................... 124

Figure 5.1: General form of the FSM controller ........................................................ 127

Figure 5.2: Example FSM controller for “open a door” ............................................ 128

Figure 5.3: An example of stimulation target changing with phase .......................... 129

Figure 5.4: Jumping up to a threshold (before ramping up) and jumping down from a

threshold (after ramping down) ................................................................................. 130

Figure 5.5: Stimulation profile for Anterior Deltoid and Triceps during all phases of

the example task ......................................................................................................... 135

xiv

Figure 5.6: Stimulation profile for Forearm Extensors during all phases of the

example task............................................................................................................... 135

Figure 5.7: Stimulation profile for Posterior Deltoid during all phases of the example

task ............................................................................................................................. 135

Figure 5.8: Stimulation profile for Forearm Flexors during all phases of the example

task ............................................................................................................................. 135

Figure 5.9: An early transition (transition (i+1)) where target (i+1) and target (i+2) are

different. Ramp rate (i+2) is based on the specified ramp time and the difference

between target (i+1) and target (i+2), irrespective of the fact that target (i+1) was not

reached ....................................................................................................................... 139

Figure 5.10: An early transition (transition (i+1)) where target (i+1) and target (i+2)

are the same. Ramp rate (i+2) is inherited from phase (i+1) ..................................... 139

Figure 5.11: Real-time inputs and outputs for the FES control system ..................... 141

Figure 5.12: Upper limb FES control system flow chart, including FSM controller 142

Figure 5.13: A complete upper limb FES control system (including FSM controller)

implemented using Simulink ..................................................................................... 144

Figure 5.14: Acceleration profiles (x in blue, y in green, z in red). The dashed lines

indicate transitions between phases ........................................................................... 151

Figure 5.15: “Change in angle since entering the phase” during the “Open a door task”

.................................................................................................................................... 152

Figure 5.16: Phase number as output by the FSM controller .................................... 153

Figure 5.17: Stimulation pulse width outputs ............................................................ 153

Figure 5.18: Flow chart of GUI based flexible FSM setup ........................................ 156

Figure 5.19: Flow chart showing the process followed in stage 1 ............................. 158

xv

Acknowledgements

The work described in this thesis would not have been possible without the kind help

and support of the people around me over the last four years.

First, I would like to express my deepest appreciation to my supervisor Professor

David Howard and my co-supervisor Professor Laurence Kenney for their support,

help and advice. I would not have completed my research without them!

I would also like to thank Christine Smith, Karen Waring, Helen Luckie and Anmin

Liu for their support to support for the for the experimental work with volunteers.

I would particularly like to thank those patients and healthy subjects who kindly

participated in my research.

I am most grateful to my parents for their love and encouragement! I really would not

have completed my research without their love.

xvi

Abbreviations

ABS Absolute value

ADL Activities of daily living

ARAT Action Research Arm Test

CIMT Constraint-induced movement therapy

CMFT Central Manchester Foundation Trust

EEG Electroencephalogram

EMG Electromyographic

ENG Electroneurogram

FES Functional electrical stimulation

FMA Fugyl-Meyer assessment

FSM Finite State Machine

GUI Graphical user interface

MAL Motor Activity Log test

PA Pulse amplitude

PW Pulse width

pps Pulses per second

RCTs Randomized controlled trials

SCI Spinal cord injury

SRFT Salford Royal Foundation Trust

xvii

Abstract

Functional electrical stimulation (FES) is the controlled use of electrical pulses to

produce contraction of muscles in such a way as to support functional movement. FES

is now widely used to aid walking in stroke patients and research into using FES to

support other tasks is growing. However, in the more complex applications, it is very

challenging to achieve satisfactory levels of FES control.

The overall aim of the author’s PhD thesis is to develop improved techniques for real-

time Finite State Machine (FSM) control of upper limb FES, using multiple

accelerometers for tracking upper limb movement and triggering state transitions.

Specific achievements include: 1) Development of new methods for using

accelerometers to capture body segment angle during performance of an upper limb

task and use of that data to trigger state transitions (angle triggering); 2) Development

of new methods to improve the robustness of angle triggering; 3) Development of a

flexible finite state-machine controller for control of upper limb FES in real time; 4)

In collaboration with a clinical PhD student, implementation of a graphical user

interface (GUI) that allows clinical users (e.g. physiotherapists) to set up FSM

controllers for FES-assisted upper limb functional tasks.

Three alternative methods that use 3-axis accelerometer data to track body segment

angle with respect to gravity have been reported. The first uncalibrated method

calculates the change in angle during a rotation using the gravity vectors before and

after the rotation. The second uncalibrated method calculates the angle between the

accelerometer x-axis and the gravity vector. The third calibrated method uses a

calibration rotation to define the measurement plane and the positive rotation

direction. This method then calculates the component of rotation that is in the same

plane as the calibration rotation. All three methods use an algorithm that switches

between using sine and cosine, depending on the measured angle, which overcomes

the poor sensitivity problem seen in previous methods.

xviii

A number of methods can be included in the transition triggering algorithm to

improve robustness and hence the usability of the system. The aim of such methods is

to reduce the number of incorrect transition timings caused by signal noise, jerky arm

movements and other negative effects, which lead to poor control of FES during

reaching tasks. Those methods are: 1) Using the change in angle since entering a state

rather than absolute angle; 2) Ignoring readings where the acceleration vector is

significant in comparison to the gravity vector (i.e. the magnitude of the measured

vector is significantly different from 9.81); and 3) Requiring a given number of

consecutive or non-consecutive valid readings before triggering a transition. These

have been implemented with the second uncalibrated angle tracking method and

incorporated into a flexible FSM controller.

The flexible FSM controller and the associated setup software are also presented in

this thesis, for control of electrical stimulation to support upper limb functional task

practice. In order to achieve varied functional task practice across a range of patients,

the user should be able to set up a variety of different state machines, corresponding

to different functional tasks, tailored to the individual patient. The goal of the work is

to design a FSM controller and produce an interface that clinicians (even potentially

patients) can use to design and set up their own task and patient-specific FSMs.

The software has been implemented in the Matlab-Simulink environment, using the

Hasomed RehaStim stimulator and Xsens MTx inertial sensors. The full system has

been tested with stroke patients practicing a range of tasks in the laboratory

environment, demonstrating the potential for further exploitation of the work.

1

Chapter 1 – Introduction

Stroke is one of the top three causes of death in developed countries (Lindley, 2008;

WHO, 2003). Around 50% of survivors may lose some control of the arm and/or hand

(Broeks, Lankhorst, Rumping, & Prevo, 1999; Heller et al., 1987; Parker, Wade, &

Hewer, 1986; Timmermans et al., 2009). Such people typically suffer from a

reduction in muscle strength and coordination (Burgar et al., 2011; Harris & Eng,

2010). Other problems for the upper limb include a reduced ability to extend the

shoulder, elbow and hand joints (Ruth N. Barker, Brauer, & Carson, 2008;

Timmermans et al., 2009).

A wide range of rehabilitation interventions have been developed which aim to help

restore upper limb motor function after stroke. Interventions include constraint-

induced movement therapy, electromyographic (EMG) biofeedback, robotically

assisted therapy, mental practice with motor imagery, and functional electrical

stimulation (FES) therapy. FES is a controlled electrical stimulation for producing

contraction of muscles, and such technology is now widely used in helping restore

motor function for stroke patients (Lynch & Popovic, 2008). However, achieving

satisfactory levels of FES control is very challenging because of the nonlinear

(Ferrarin, Palazzo, Riener, & Quintern, 2001; Lynch & Popovic, 2008) and time-

varying (Lynch & Popovic, 2008) response of muscles to stimulation. In addition,

perturbations from muscle spasticity and other central nervous system feedback loops

introduce often unpredictable challenges to the controller (Lynch & Popovic, 2008).

To address these problems, designers have developed closed-loop, open-loop and

state machine controllers. Closed-loop controllers usually employ command, feed-

forward and feedback signals together with error detection and correction processes

for control of FES. The feedback signals are typically force (Lawrence et al., 2008),

position (Chadwick et al., 2011), and joint angle (Kurosawa, Futami, Watanabe, &

Hoshimiya, 2005). Command and feed-forward signals can include EMG (Hara,

2008), electroneurogram (ENG) (Inmann & Haugland, 2004) and

electroencephalogram (EEG) (Sinkjaer, Haugland, Inmann, Hansen, & Nielsen, 2003)

2

signals. Open-loop controllers are simple and typical systems usually consist of a

stimulator and a command source, which can be a switch (Tomović, Popović, & Stein,

1995) used by the patient or therapist (acting as the system controller) or a specific

muscle activity (Braz, Russold, & Davis, 2009; Francisco et al., 1998). A finite state

machine (FSM) controller is defined as a list of states, and the triggering conditions

for leaving each state (for a detailed definition, see section 2.4). The FSM have been

shown to be an effective and intuitive approach for the control of FES (Tresadern,

Thies, Kenny, Howard, & Goulermas, 2008). FSM control using sensors on the body

for transitions between states (movement phases) is a potentially good compromise

between open-loop control and true closed-loop control, but is currently limited by a

lack of flexibility. In other words, unless a control engineer is also involved, clinical

users are unable to set up FSM controllers to deliver different functional tasks for

different patients with different impairments. To overcome this problem, a flexible

FSM controller is required that can be set up to deliver upper limb FES for many

different functional tasks. At the same time, a quick and easy set-up tool is essential to

guide clinical users, with little or no software expertise, through the setup of new

FSM controllers for different upper limb rehabilitation tasks. This requires them to

define the number of states, the state transition conditions (angle thresholds, timeouts,

combinational logic etc.), and stimulation parameters for each state (thresholds,

targets, ramps etc).

A related problem is that of capturing upper limb motion with small, low cost, body

worn sensors and using this information to robustly trigger state transitions. Modern

accelerometers are low cost (Barbour & Schmidt, 2001; Verplaetse, 1996; Zheng,

Black, & Harris, 2005), low power consumption (Cuesta-Vargas, Galán-Mercant, &

Williams, 2010; Zheng et al., 2005), small in size (Barbour & Schmidt, 2001; Cuesta-

Vargas et al., 2010; Zheng et al., 2005), light weight (Barbour & Schmidt, 2001;

Cuesta-Vargas et al., 2010), and provide accurate and reliable outcomes (Cuesta-

Vargas et al., 2010). These advantages make accelerometers a promising solution for

use in the FSM control of upper limb FES.

However, the measurement of body segment angle using accelerometers suffers from

two main problems. Firstly, the existing methods for processing the accelerometer

signals, to obtain angle from the vertical, all suffer from very poor sensitivity when a

3

sensitive axis approaches the vertical. Secondly, the methods reported rely on the true

acceleration being negligible and are, therefore, only suitable for measuring angle

under static or low acceleration conditions. Furthermore, there are significant

challenges to overcome to make the triggering of state transitions, based on

accelerometer derived angles, more robust because of movement variability, sensor

misalignment, the true acceleration component, signal noise etc.

1.1 Research objectives

The overall aim of the author’s PhD study is to develop improved techniques for real-

time FSM control of upper limb FES, using multiple accelerometers for tracking

upper limb movement and triggering state transitions. Specific objectives include:

1) Creation of a flexible FSM controller that can deliver real-time control of upper

limb FES for many different functional tasks.

2) Creation of a user-friendly graphical user interface (GUI) for guiding therapists,

with little or no software expertise, through the setup of FSM controllers for upper

limb rehabilitation tasks. A GUI is defined as a type of interface that allows users to

interact with software via graphical icons. Users should be able to define: number of

states, state transition conditions (angle thresholds, timeouts, combinational logic etc.),

and stimulation parameters for each state (thresholds, targets, ramps etc);

3) Investigation of alternative methods for using accelerometers to capture body

segment angle during FES upper limb rehabilitation tasks;

4) Investigation of algorithms to improve the robustness of angle triggering for state

transitions.

1.2 Overview of the thesis

Chapter 2 is a comprehensive literature review covering: general background on

upper limb rehabilitation following stroke, principles of electrical stimulation,

available methods for the control of FES, FSM control of FES, and the use of inertial

sensors for motion tracking.

4

Chapter 3 describes the author’s work on angle measurement using accelerometers.

The methods presented include two uncalibrated angle tracking algorithms and a

calibrated angle tracking algorithm. The two uncalibrated angle tracking methods are

used for converting acceleration measurements into the sensor’s angle (or change of

angle) from the vertical. The term ‘uncalibrated’ refers to the fact that the subject is

not required to make any calibration movements after the sensors have been donned.

The first uncalibrated angle tracking method calculates the change in angle from the

vertical by calculating the angle between the gravity vectors before and after the

rotation, both expressed in the sensor’s coordinate frame. The second uncalibrated

angle tracking method calculates the angle of the sensor’s x-axis from the vertical and

rejects any rotation about the x-axis, which can be advantageous if the wish is to

avoid triggering a transition as a result of pronation-supination of the forearm. The

calibrated angle tracking method gives both the magnitude and the sign of the angle

change in a given plane that is defined by a calibration movement. Finally, a real-time

auto-calibration algorithm has been developed that updates the gains applied to the

three sensor signals (x, y and z components of acceleration) to compensate for

calibration errors. The auto-calibration method is designed to operate in parallel with

any of the angle tracking methods mentioned above.

Chapter 4 presents robust angle triggering algorithms, which are based on ignoring

bad sensor readings resulting from signal noise, jerky arm movements and other

negative effects. The aim is to avoid incorrect FSM transition timings and hence poor

control of FES during reaching tasks. The following methods have been implemented

and tested:

- Using the change in angle since entering a state rather than absolute angle,

which reduces the effects of sensor misalignments and movement variability;

- Ignoring readings where the acceleration vector is significant in comparison to

the gravity vector (i.e. the magnitude of the measured vector is significantly

different from )

- Requiring a given number of consecutive or non-consecutive valid readings

before triggering a transition.

5

Chapter 5 presents a flexible FSM controller for the real-time control of FES during

upper limb rehabilitation. A GUI guides clinical users through the process of setting

up new FSM controllers for different upper limb rehabilitation tasks for different

patients with different impairments. Users define the number of states, state transition

conditions (angle thresholds, timeouts, combinational logic etc.), and stimulation

parameters for each state (thresholds, targets, ramps etc).

6

Chapter 2 - Literature review

2.1 Upper limb rehabilitation following stroke

2.1.1 Stroke and the consequences for the upper limb

A stroke is a type of brain injury caused by disruption of blood to a specific part of the

brain (Caplan, 2005). Stroke can be divided into two main groups, haemorrhagic and

ischemic. Haemorrhagic stroke is a stroke in which damage to the brain is caused by a

blood vessel bursting and there are several subtypes, depending on the location of the

bleed. Ischaemic strokes are caused by a blockage in a blood vessel interrupting blood

supply to the brain and can also be classified according to the affected region of the

brain (Caplan, 2005).

Stroke is one of the top three causes of death in developed countries (Lindley, 2008;

WHO, 2003). Each year in England, about 110,000 people have a stroke. In addition

approximately 20,000 suffer a transient ischemic attack, a short term disruption of

blood supply to the brain in which the symptoms last less than 24 hours (Easton et al.,

2009; Lecouturier et al., 2010; NAO, 2005). Approximately 20 - 30% of people die

within a month of having a stroke, and at least 300,000 people are living with

moderate to severe disabilities as a consequence of stroke in England (Lecouturier et

al., 2010; NAO, 2005). Patients who survive a stroke often suffer impairments,

including poor control of the limbs, loss of swallowing function, disturbance of vision,

as well as reduced or heightened sensation. In addition, often severe psychological

impacts occur as a result of stroke (M Fisher, 2008).

Around 50% of survivors may lose some control of the arm and/or hand (Broeks et al.,

1999; Heller et al., 1987; Parker et al., 1986; Timmermans et al., 2009). Such people

typically suffer from a reduction in muscle strength, maximum voluntary force, and

coordination (Burgar et al., 2011; Harris & Eng, 2010). Other problems include a

reduced ability to extend the shoulder, elbow and hand joints (Ruth N. Barker et al.,

7

2008; Timmermans et al., 2009), shoulder pain (In Sook Lee et al., 2009) and

reduction in sensations of touch and heat (Nykänen, 2010). In the absence of

treatment, such arm and hand impairments often remain, or become more severe with

time. Additionally, shoulder pain is another common complication after stroke and

this may negatively impact on both rehabilitation and daily living activities (In Sook

Lee et al., 2009). In addition, upper limb impairments may affect balance and walking

(Shumway-Cook & Woollacott, 2001).

2.1.2 Outcome measures of arm usage in everyday life

In this section, the common measures of to what extent people after stroke use their

arm in everyday life are addressed. One of the widely used methods is the Motor

Activity Log (MAL) test, a questionnaire-based test (Hammer & Lindmark, 2010).

The MAL asks patients to recall which of a list of common tasks the subjects perform

in their daily lives. The items in the questionnaire include the actual activities of

everyday life, such as picking up a glass, brushing teeth etc (Uswatte, Taub, Morris,

Vignolo, & McCulloch, 2005). The score is based on the number of tasks performed,

as well as the extent to which the affected arm is used to perform each task.

2.1.3 Rehabilitation of the upper limb after stroke

A wide range of rehabilitation interventions have been developed which aim to help

restore upper limb motor function after stroke. Interventions include constraint-

induced movement therapy, EMG biofeedback, robotics, mental practice with motor

imagery, and FES therapy.

2.1.3.1 Electromyographic (EMG) biofeedback

EMG biofeedback is the use of visual or auditory feedback to the patient on activity

levels in paretic muscles. The EMG signals are derived from the electrical potentials

in the motor units and usually measured at the skin surface (Langhorne, Coupar, &

Pollock, 2009). By providing the patient with feedback on their muscle activity, it is

8

believed that patients may be able to retrain their brain or spinal cord to increase the

ability to activate these weak muscles. (Robinson & Snyder-Mackler, 2007).

EMG biofeedback therapy has some evidence of effectiveness in upper limb

rehabilitation (Armagan, Tascioglu, & Oner, 2003; Crow, Lincoln, Nouri, & Weerdt,

1989; Inglis, Donald, Monga, Sproule, & Young, 1984). A systematic review

(Woodford & Price, 2007) of EMG biofeedback therapy identified 13 trials (involving

a total of 269 subjects) and concluded that there was only a small amount of evidence

to suggest that EMG biofeedback therapy has a positive effect on arm functions. The

overall results were limited because of the small number of trials.

However, no evidence was found that EMG biofeedback therapy has significant

benefits in improving hand function (Langhorne et al., 2009). In addition, according

to Woodford and Price’s systematic review, EMG biofeedback therapy has limited

effect on improving wrist joint range of motion.

2.1.3.2 Constraint-induced movement therapy (CIMT)

Typically, stroke patients begin to use their intact upper limb for functional tasks

during the early stage of recovery from stroke, as use of the affected limb is difficult

(Kunkel et al., 1999). This may lead to impaired motor recovery in the following

months and years (Grotta et al., 2004). Constraint-induced movement therapy (CIMT)

is a rehabilitation technique that involves the intensive use of affected upper limb

through restraint of the unaffected upper limb (Kunkel et al., 1999; Suputtitada,

Suwanwela, & Tumvitee, 2004; E Taub et al., 1993). Highly intensive practice of

functional movements and exercises using the impaired limb is believed to be an

effective approach to accelerating the restoration of arm and hand function (Bruce,

2005; French et al., 2009; French et al., 2010; Prange, Jannink, Groothuis-Oudshoorn,

Hermens, & Ijzerman, 2006).

A number of large randomised, controlled studies have shown positive effects of

CIMT on the rehabilitation of upper limb function (Atteya, 2004; Lorie, Carolyn,

Melinda, & Amit, 2008; Page, Sisto, & Johnston, 2002; Page, Sisto, Levine, Johnston,

9

& Hughes, 2001; Wolf et al., 2006; Wolf et al., 2008). For example Wolf, Winstein et

al. (2008) investigated 106 three to nine months stroke patients, who received CIMT

therapy for up to 5 hours per day, 5 days per week over 2 weeks. The results show

that CIMT therapy may lead to a significant improvement in upper limb strength and

function, and that the effects are retained for at least 2 years. Thus, CIMT treatment is

believed to have long-term benefits to the rehabilitation of upper limb function. A

recent systematic review (Shi, Tian, Yang, & Zhao, 2011), which included 13

randomized controlled trials (RCTs), concluded that CIMT treatment has significant

benefits to the rehabilitation of the impaired upper limb . Importantly, no negative

medical side effects of CIMT have been found (Edward Taub, Uswatte, & Pidikiti,

1999).

However, CIMT is limited in its applicability. For example, it is not suitable for

widespread use in the acute phase of stroke. A study in acute stroke patients showed

that out of 187 potential CMIT participants, 30 patients had no movement in the hand

and another 55 patients had other reasons for exclusion, such as aphasia (Grotta et al.,

2004). In addition, CMIT is costly, as it is demanding of therapists time and the

resources of a rehabilitation unit (Grotta et al., 2004; Langhorne et al., 2009).

2.1.3.3 Robotic-assisted therapy

Robotic-assisted therapy is the use of mechanical devices to provide or support

highly- intensive, and controllable repetitive upper limb practice (Marc Fisher, 2009),

(Prange et al., 2006). Thus, robotic therapy offers a potential solution to the problem

of needing a therapist to be present throughout therapy sessions (Volpe, Ferraro,

Krebs, & Hogan, 2002). Additionally, robotic-assisted therapy appears to be more

acceptable to the patients compared with traditional exercise therapy (Kwakkel,

Kollen, & Krebs, 2008). Recent work has investigated the motivational benefits of

structuring robotic therapy sessions as games, thereby encouraging patients to

exercise for longer than would otherwise be the case (Kwakkel et al., 2008). Also,

robotic-assisted therapy allows for more control over the therapy to the arm during

treatment (Kwakkel et al., 2008). Robotic-assisted therapy typically allows both for

control of the environment (e.g. you can make the robot stiff about particular axes)

10

and monitoring of performance (Masiero, Celia, Rosati, & Armani, 2007) (e.g.

recording of movement, forces, speed and direction of residual movements etc)

(Masiero et al., 2007).

Kwakkel, Kollen et al. (2008) reviewed 10 studies, of robotic-assisted therapy. The

results show that robotic therapy has a significant positive effect on restoration of

upper limb function. Another review (Mehrholz, Platz, Kugler, & Pohl, 2009) has also

demonstrated that robotic-assisted therapy had a positive effect on arm motor function

and arm motor strength. Moreover, a large RCT (Lo et al., 2010) that involved 127

patients with upper limb impairment 6 months or more after stroke has concluded that,

for long-term stroke patients with upper limb impairment, robotic-assisted therapy

seems to offer potential benefits to the rehabilitation of motor functions for those

patients who receive over 36 weeks treatment.

However, robotic therapy provides limited improvement in activities of daily living

(ADL) (Kwakkel et al., 2008; Mehrholz et al., 2009). Also, according to a recent

review paper (Langhorne et al., 2009) no significant improvements in hand function

were observed.

2.1.3.4 Mental practice with motor imagery

Mental practice with motor imagery is a neuro-rehabilitation technique that involves

repeated imagining of particular motor tasks, with the aim of improving or stabilizing

the actual motor tasks. Motor imagery of a physical task activates very similar parts of

the brain to those that are activated when the physical task is performed(Jacksona,

Lafleur, Malouin, Richards, & Doyon, 2003). e (Berntson & Cacioppo, 2009). By

creating an imagined movement of the paretic limb using a part of the brain that is

undamaged by stroke, the motor imagery technique aims to strengthen alternative

neuron connections in the brain, to allow a different (undamaged) part of the brain to

take over the role of the damaged part (Marshall et al., 2000) and enhance the spared

area adjacent to the damaged part of the brain (Nudo, Wise, SiFuentes, & Milliken,

1996).

11

Mental practice with motor imagery is inexpensive (Dickstein & Deutsch, 2007) and

easy to use (Page, Szaflarski, Eliassen, Pan, & Cramer, 2009) therapy that may

contribute to the rehabilitation of motor function. Braun, S., A. Beurskens, et al. (2006)

reviewed the literature and found that mental practice, as an additional therapy, has

some benefits in improving motor performance. Another more recent review,

concluded that mental practice with motor imagery technique has potential positive

effects on recovery of arm functions (Langhorne et al., 2009). It is suggested that

mental practice therapy is more suitable for patients who were receiving physical

therapy or occupational therapy at the same time (Braun, Beurskens, Borm, Schack, &

Wade, 2006; Dickstein & Deutsch, 2007; Page, Levine, & Leonard, 2007).

However, the largest study to date, Letswaart, Johnston et al. examined 121 stroke

patients with a residual upper limb weakness (on average < 3 months post-stroke),

and found no improvement in outcome measures. Thus, there is no clear evidence that

mental practice with motor imagery technique improves motor functions and benefits

stroke rehabilitation in isolation (Letswaart et al., 2011).

12

2.2 Electrical stimulation

2.2.1 Introduction

Excitable muscle or nerve tissues provide the basis for FES as a rehabilitation tool for

patients following a stroke. In this section the reader is introduced to the basic theory

of excitable tissues and principles of activating excitable tissues using ES, including

the effects of stimulation amplitude, duration, frequency on physiological responses.

Finally, some examples of FES applications for the restoration of upper extremity

function are described.

2.2.2 Excitable tissues

2.2.2.1 Structure of a nerve and motor unit

Figure 2.1: The basic structure of a nerve cell and the communication between nerve

cells (Hegner, Acello et al. 2009). A nerve cell typically contains a cell body and

extensions called axon and dendrites. The nerve impluses enter the nerve cell via the

dendrites, are transmitted through the cell body and axon until they reach the synapse.

The nerve cell is the basic unit of the communication network in the human body. A

nerve cell typically contains a cell body and fiber processes of axon and dendrites (see

figure 2.1). The axon is the longest process and transmits the nerve impulses along its

Cell body

Dendrites

Axon

Dendrites

Cell body

Synapse

Chemical

neurotransmitters

13

length, while the dendrites are responsible for interfacing with other nerve cells

(Hegner, Acello, & Caldwell, 2009).

Figure 2.2: Structure and composition of muscles (Sheriff 2004)

Muscle responds to neural signals to produce force. The sarcomere is the basic unit

for production of force, and contains thick myosin and thin action filaments (Baker,

Wederich, Mcneal, Newsam, & Waters, 2000). Myofilaments comprise many

repeating interdigitated sarcomere. A group of myofilaments, organized in a pattern of

thick and thin contractile proteins, is termed a myofibril. The largest unit in the

muscle, the myofiber consists of a number of myofibrils. Specifically, the number of

myofibrils is determined by the size and function of the parent muscle. Myofibers are

enclosed by the muscle membrane and are elongated cylinders in shape, and the

myofibers comprise the muscle fasciculus (see figure 2.2) (Baker et al., 2000; Sheriff,

2004).

Sarcomere

Myosin Filaments

Myofilaments

Myofibril (made-up of

myofilaments)

Myofiber (made-

up of Myofibrils)

Muscle fasciculus

(made-up of myofibers)

14

2.2.2.2 Electrochemical balance in a nerve cell (sodium pump etc)

The nerve cell is rich in potassium ions and poor in sodium ions, compared with the

cell exterior, due to the structural characteristics of the membrane and the metabolic

membrane pumps (see figure 2.3). In the steady state resting potential, the membrane

is more permeable to potassium ions than sodium ions. The active transport (wide

arrow) moves more sodium ions and potassium ions than the passive diffusion

(narrow arrow). In addition, more sodium ions are transferred to the cell exterior

relative to potassium ions to the cell interior through the sodium-potassium pump.

Thus, the cell has a slightly negative charge, which is called the resting potential

(Baker et al., 2000).

Figure 2.3: The resting potential of a nerve fiber (Baker, Wederich et al. 2000). At the

steady state resting potential, the membrane is more permeable to potassium ions than

sodium ions.

2.2.2.3 Action potentials and their propagation

The nerve action potentials1 are the message units and responsible for transmission of

information within the nervous system (Baker et al., 2000).

1 Action potentials are short-lasting events and usually occur in excitable cells, such as neurons. In

neurons, action potentials are the mechanism by which cell to cell communication takes place. In

muscle cells, they initiate contraction.

Nerve fiber

Nerve membrane

Na+

Na+

K+

K+

+ + + + + + + + +

- - - - - - - - - - -

+ + + + + + + + +

- - - - - - - - - - -

15

When a positive electric field is applied to the neuron in its resting potential, there is

an outflow of current (ions) across the membrane, which if sufficiently long lasting

and of sufficient amplitude leads to the depolarization2

of the neuron. If the

depolarizing stimulation exceeds a specific level, which is usually called the threshold

of excitation, it will result in a massive depolarization of the membrane and explosive

opening of sodium channels. The sodium ions will flow across the membrane of the

neuron until reaching the sodium equilibrium point, which leads to the closure of

sodium channels. After this process, potassium ions start flowing out from the interior

of the nerve fiber and when the membrane returns to its resting potential, the

potassium channels will also close. The whole process is completed within one msec

for a nerve and one to five msec for a muscle (Baker et al., 2000). The response of

rapid depolarization followed by repolarization in response to any stimulation over

the threshold of excitation is the same (achieving a peak voltage of +30 mV), and

called the action potential (see figure 2.4).

Figure 2.4: (a) Membrane potential response of a nerve cell over time to a small

depolarizing stimulation; (b) Response to a stronger depolarizing stimulation; and (c)

Response to a depolarizing stimulation over the threshold of excitation (Kalat 2008)

2 In biology terms, depolarization is a potential difference change between the inside and outside of

membrane. A large enough depolarization (triggering the threshold for excitation) will lead to action

potentials in cells.

(a) (b)

(c)

16

The propagation of action potentials is described as the transfer of an action potential

from one point of the axon to its adjacent area. The action potential at the initial point

of the axon will act as the stimulation for production of other action potentials (see

figure 2.5). A basic character of propagation is the positive dependence of speed of

propagation on axon diameter. In another words, the greater diameter of axon, the

greater the speed of propagation along the axon (Kalat, 2008).

Figure 2.5: (a) The action potential of peak voltage level of 40 mV causes current

flow to the negative area of the axon. The flow of current acts as the depolarizing

stimulation over the threshold of excitation; and (b) when the threshold of excitation

is achieved, another action potential will be elicited, and this process leads to the

propagation of the action potential along the axon (Bullock, Boyle, & Wang, 2001).

When an action potential is propagated and reaches the axon terminals, a chemical

transmitter will be released by the presynaptic terminal. The chemical transmitter will

then bridge the synaptic gap between the nerve and the muscle fiber membrane, and

approach the postsynaptic muscle neuron. This process will lead to production of an

action potential in the muscle (Kalat, 2008).

2.2.2.4 Muscle fiber contraction

If adequate levels of chemical transmitters approach the postsynaptic terminals on the

muscle fiber, this will result in the depolarization of the muscle membrane and

generation of action potentials. As the basic unit of production of force, the sarcomere

starts storing calcium ions, released from the propagation process, the myosin and

actin filaments start interacting i.e. sliding across one another. When the muscle

membrane returns back to its resting potential, the myosin and actin filaments stop

+ + + +

+ 40

mV

- - - -

- 60

mV

+ + + +

+ 40

mV

- - - -

- 80

mV

- - - -

- 60

mV

Current flow Current flow

(a) (b)

17

interacting, and the sarcomere also returns to the initial length (Smeltzer, Bare,

Hinkle, & Cheever, 2009). Such a process will lead to the twitch and relaxation of

myofibrils, the basic mechanism underlying the contraction and relaxation of muscles.

2.2.3 Electrical stimulation of excitable tissues

As described above, a disturbance of the electrical field surrounding a nerve cell can

elicit action potentials. Such a disturbance can be achieved using a pair of electrodes,

comprising two ‘poles’ (an anode ‘+’ and a cathode ‘-’), (usually) located on the

surface of specific areas of the body to form a circuit. When electric current passes

within the circuit, positive ions are repelled from the tissue close to the anode and

those positive ions are driven by the Lorentz Force3 to flow towards the cathode.

Once the potential difference between inside and outside the membrane decreases

(which happens at tissues near a cathode) below the critical threshold for excitation,

an action potential will be elicited. The action potential propagates in both directions

from the point of electrical stimulation, leading to muscle contractions.

2.2.3.1 Stimulation parameters

Stimulation current pulses are typically characterized by three parameters, which are

pulse frequency, amplitude and duration (see figure 2.6), and all three stimulation

parameters have an effect on muscle contraction.

Figure 2.6: Three main parameters for FES. The pulse amplitude is the peak current

during the pulse and the pulse duration is the period of time a pulse remains on.

Increasing the pulse amplitude or pulse duration may increase the strength of muscle

3 Lorentz Force is the force on a point charge, which generated by the interaction between point

charge and electromagnetic fields.

Pulse amplitude

Pulse duration

Period =

1/Pulse

frequency

Time (µs)

Pulse amplitude (mA)

18

contractions by recruiting more nerve fibers. The pulse frequency is the number of

pulses per second. At a sufficiently high frequency, the contractions cannot be

individually distinguished (Lynch & Popovic, 2008).

2.2.3.2 Stimulus pulse amplitude and duration

An action potential in a nerve is only elicited once a stimulus pulse exceeds a

particular threshold in either current amplitude or pulse duration. The current

amplitude is described as the peak current value in the phase of a pulse and the pulse

duration is the period of time an electrical pulse remains on.

Increasing amplitude or duration of stimulations may increase the strength of muscle

contractions by recruiting more nerve fibers. A schematic muscle recruitment curve

(see figure 2.7) shows the typical, non-linear response to stimulation; at a point just

after initial motor response, a small change in stimulation amplitude can cause a large

increase in muscle force; further increasing of stimulation levels results in no increase

in muscle force.

Figure 2.7: Muscle force against current amplitude. No muscle motor units have been

recruited until the threshold of excitation for muscle excitation has been exceeded. A

further increase in current amplitude will lead to a large increase in muscle force, until

saturation is reached (Data collected from quadriceps femoris by using constant

120

100

80

60

40

20

20 40 60 80 100 120 140

Pulse amplitude (mA)

Torque (ft-lbs)

19

current surface stimulator at frequency of 35 pulses per second (pps) and pulse

duration of 300 µs. 1 ft-lbs = 1.3558 Nm) (Baker et al., 2000).

As nerve fibers vary in their diameter and proximity to the stimulation, those nerve

fibers that are largest (P. Hunter Peckham & Knutson, 2005) and closest to electrode

(Baker et al., 2000) will be stimulated at the lowest level of electrical stimulation

(intensity and duration of stimulation just above the critical threshold). Increasing the

current of stimulation or pulse width will excite the closer, smaller diameter nerve

fibers and larger diameter nerve fibers that are further from the electrode.

Figure 2.8: The relationship between pulse duration and current amplitude and

stimulation response. The lower curve represents the current amplitude and pulse

duration required to achieve the minimum neural excitation. The upper curve

represents the current amplitude and pulse duration for a near maximal muscle

contraction (Data collected from wrist extensors by using constant current surface

stimulator at frequency of 35 pps) (Baker et al., 2000)

As the curves in figure 2.8 show, if pulse duration is set to a constant value of 200 µs,

then the maximal recruitment of motor units can be achieved by increasing current

amplitude from about 15 to 40 mA. Increasing pulse duration to 400 µs will only

Pulse duration (µs)

Current amplitude (mA)

100 200 300 400 500 600 700

140

120

100

80

60

40

20

Threshold of excitation

Excitation level for a near maximal

muscle contraction

20

result in a small reduction in the current amplitude required for maximum motor

response and level of threshold of excitation. Further increasing pulse duration will

not significantly affect the amplitude required for either threshold or maximal

stimulation. Importantly, a decrease of pulse duration below 200 µs will lead to a

huge increase of current amplitude required for both maximum motor response and

the level of threshold for excitation. Similarly, if the current amplitude has been set to

a constant value, such as 40 mA, the maximum motor response can be achieved by

increasing pulse duration from about 40 to 200 µs. In the example shown in figure 2.8,

a decrease of current amplitude below 20 mA, would result in no motor response,

irrespective of the pulse amplitude. So, in summary, the parameters of pulse

amplitude and duration are the two main factors that affect the triggering of action

potentials.

Pulse durations of 200 – 400 µs are preferable for clinically used electrical stimulators,

which can provide relatively comfortable experiences for patients and sensitive

control for muscle contraction (Baker et al., 2000; Chapman & Fratianni, 2008).

2.2.3.3 Stimulus pulse frequency

The pulse frequency is described as the number of pulses per second. To achieve

smooth contraction of muscle, the stimulation pulses must be repeated above a certain

frequency, known as the fusion frequency. If the frequency is too low, the muscle will

return to resting state after stimulation, and the muscle will respond in a series of

twitches. As the frequency of stimulation increases and reaches a sufficiently high

frequency (fusion frequency), the muscle response becomes both stronger and the

summating contractions cannot be individually distinguished, which is recognized as

the tetany state. However, significant further increases in frequency will increase the

muscle fatigue rate. Typically, due to the problems of fatigue associated with

increasing frequency, engineers design FES systems that adjust the strength of muscle

contraction through changing the pulse amplitude and/or duration (Bullock et al.,

2001; Strojnik & Peckham, 2000), keeping stimulation frequency at a constant (low)

value. Typically, clinically used peripheral nerve stimulators usually work on a

frequency between 25 – 50 pps (Baker et al., 2000).

21

2.2.3.4 Simulation waveforms

Two stimulation waveforms, monophasic and biphasic, have been clinically used for

excitation of nerves. The monophasic waveform consists of repeating unidirectional

pulses and thus, only allows ion flow in one direction (example see figure 2.9 a). The

biphasic waveform consists of repeating bidirectional pulses, which allow ion flow in

both directions. Typically, the biphasic waveform has a cathodic phase and usually

followed by an anodic phase. If the cathodic phase and anodic phase are equal to each

other (equal charge flow in both direction), then the biphasic waveform is balanced

(example see Figure 2.9 c). The secondary pulse (usually the anodic phase) lets the

ions, which flow into the tissues near cathode in the first pulse, to flow out of the

electrode-tissue interface and probably can reverse the potential damage to tissues

(Mortimer, 1981). If the cathodic phase and anodic phase are not balanced, potential

muscle tissue damage can occur (see Figure 2.9 a). The mechanism of tissue damages

is the electrochemical irritation due to Ph imbalance.

The following figure gives examples of different stimulation waveforms of

monophasic and biphasic rectangular pulses.

Figure 2.9: Common stimulation waveforms of different shapes. (a) Monophasic

stimulation waveform provides rectangular pulses. It is not safe at, or above a net DC

current density of ⁄ . (b) Imbalanced biphasic stimulation waveform, which

(a)

(b)

(c)

22

may cause damage to muscle tissues if a net DC current density goes

above ⁄ . (c) Balanced biphasic stimulation waveform allows ion flow in

both directions, thereby avoiding potential damage due to electrochemical and other

processes. (Baker, Wederich et al. 2000; Scheiner, Mortimer et al. 1990).

2.2.3.5 Factors affecting the spatial distribution of stimulating current

Whether or not the applied stimulation reaches beyond the level required to initiate

action potential at a particular point in the tissue is also affected by electrode location.

If two electrodes (anode and cathode electrodes) are close to each other, more current

will choose to pass through surface tissues due to the shorter pathways through the

superficial layers, and hence deeper tissues will be less likely to be stimulated. If

deeper tissues are the targets for excitation during stimulation, the two electrodes

should be placed farther from each other to increase the current passing through

deeper tissues. The reason of this phenomenon is, when placing the two electrodes

farther apart, both surface tissues resistance and deeper tissues resistance will increase,

but the surface tissues resistance will increase faster than deeper tissues resistance.

The electrode size (contact area with the skin) will influence the current density,

which is another factor which may affect the neural or muscle excitation. The current

density is defined as the amount of charged ions moving through a specific area of

tissue. Thus, low current density may lead to inadequate depolarization of the

membrane and the stimulation will not exceed the threshold of excitation. The current

density will increase with a reduction in electrode size for a given stimulation current.

One good example of using the theory above is the use of a small cathode placed over

the tissue to be excited in combination with a large anode. In this case, the current

density will increase at the target tissue area (near the cathode) and diffuse the current

at the irrelevant area at the anode.

2.2.3.6 Ramp time for stimulation

When stimulation is used for a functional purpose, it is common to avoid abrupt

changes in stimulation intensity. A rapid increase in stimulation intensity is usually

uncomfortable and can lead to a rapid stretch of a muscle, which in turn can lead to an

23

unwanted reflex response in the antagonist muscle (muscle that opposes the action of

the muscle being stimulated). Similarly, an abrupt termination of stimulation can also

be problematic in some applications. For example, in FES for drop foot, ramping off

stimulation to the tibialis anterior muscle may improve stability during load

acceptance (Gerard M. Lyons, Sinkjær, Burridge, & Wilcox, 2002).

The gradual change in stimulation intensity with time is termed a ramp. Ramps are

usually specified in terms of the time during which stimulation changes from one

target level to another (see figure 2.10). The parameter that is changed in the ‘ramp’

can be intensity of pulse or duration of pulse. Most manufactures use increasing or

decreasing intensity of pulse width or amplitude to produce ramps.

The following figure gives an example of implementation of ramp times to

stimulations.

Figure 2.10: Stimulation ramp times. In a typical ramped stimulation pattern, the

stimulation intensity ramps up or down linearly (Baker, Wederich et al. 2000)

2.2.4 Functional electrical stimulation

FES is a controlled electrical stimulation for producing contraction of muscles. FES

(Baker et al., 2000), was first introduced during the 1960s. FES is now widely used in

helping restore motor function for stroke patients (Lynch & Popovic, 2008). FES is

used as a neural prosthesis, whose purpose is to substitute for the missing or corrupted

neural signals during the performance of a functional task. Typical applications

include cycling (Donaldson, Perking, Fitzwater, Wood, & Middleton, 2000), rowing

(Davoodi & Andrews, 2003, 2004; Davoodi, Andrews, Wheeler, & Lederer, 2002;

Pons, Vaughan, & Jaros, 1989), standing (Jaeger, 1986; Matjacic & Bajd, 1998) and

standing-up (Kagayaa et al., 1995; Kern et al., 1999), as well as enhancement of gait

Ramp up time

Ramp down time

Time (Sec)

Stimulation

24

pattern (Bogataj, Gros, Kljajić, Aćimović, & Maležič, 1995; Granat, Keating, Smith,

Delargy, & Andrews, 1992; Régine et al., 2000).

There are three types of electrode used in FES systems: surface electrodes

(Pfurtscheller, Müllera, Pfurtschellerc, Gernerd, & Ruppd, 2003), percutaneous

electrodes (electrodes implanted under the skin connected via wires crossing through

the skin) and fully implantable electrodes (Keith et al., 1989; Mulcahey, Betz, Smith,

Weiss, & Davis, 1997).

The first attempt at using surface electrodes applied to the upper limb to assist with

the control of the hand was reported in 1963 (C. Long, 1963; Vodovnik, Long,

Reswick, Lippay, & Starbuck, 1965). Subsequent developments have built on this

approach (Mangold, Keller, Curt, & Dietz, 2005; Pfurtscheller, Müller-Putz,

Pfurtscheller, & Rupp, 2005; Pfurtscheller et al., 2003; Mirjana B. Popovic, Popovic,

Sinkjær, Stefanovic, & Schwirtlich, 2002; Prochazka, Gauthier, Wieler, & Kenwell,

1997; Thrasher, Zivanovic, McIlroy, & Popovic, 2008), as described in detail below.

2.2.5 FES applications for the upper extremity

Current studies of FES therapy demonstrate its potential value for rehabilitation of the

upper extremity function after stroke. Studies have shown that FES may be an

efficacious intervention in rehabilitation of reaching and grasping function (Jayme S.

Knutson, Hisel, Harley, & Chae, 2009; Thrasher et al., 2008), elbow extension

(Hughesa et al., 2010; Thrasher et al., 2008), shoulder motion (Kameyama, Handa,

Hoshimiya, & Sakurai, 1999), and stabilization of wrist joints (Mulcahey et al., 1997).

However, as described in detail in section 3, the current FES control systems are

crude. Most clinical FES applications use open-loop control in which the stimulation

parameters are adjusted during the tasks by the control system without feedback. The

clinically available open-loop controlled FES systems also cannot provide precise

adaptation to the input in response to the users’ performance nor generate minimum

levels of stimulation to continually challenge the patient. Both of these are seen as

valuable functions in promotion of motor relearning (Hughes et al., 2010).

25

In the following section, electrical stimulation systems that have been used for upper

limb rehabilitation are described. The section begins with an overview of the systems

reported in the literature, including those that deliver pre-programmed stimulation

profiles (i.e. with no input from the user), systems that rely on “unnatural” triggering,

such as button activated ES, and finally, systems in which the onset, termination and

sometimes magnitude of stimulation is controlled by the user. In each section, the

different systems are described and the clinical evidence reviewed.

2.2.5.1 ES systems that are not under voluntary control, or are push-button controlled

Stimulators that are not voluntarily controlled are pre-programmed by the clinician or

user to repeat a fixed duty cycle, or cyclical electrical stimulation. Cyclical electrical

stimulation result in the repetitive contraction of muscles. In some cases pre-defined

stimulation profiles can be started or stopped via a push button. Commercial systems

that can deliver this kind of stimulation profile include the Bioness Handmaster,

Odstock Medical Microstim 2 and Odstock 4 Channel Stimulator Kit.

The Odstock Medical Microstim 2 is two-channel stimulator that can deliver cyclical

(exercise) stimulation to weak or paralysed muscles (Mann, Burridge, Malone, &

Strike, 2005). Another Odstock 4 Channel Stimulator Kit is available on the market

that allows the user to setup exercise stimulation profiles which require more than 2

channels of stimulation.

The BioNess Handmaster is one of the most widely used and well studied ES systems

for the upper limb of patients with spinal cord lesions or stroke (Alon & McBride,

2003; Ijzerman et al., 1996). The NESS Handmaster includes a spiral carbon fibre

splint with surface electrodes attached on the splint for stimulation of finger and

thumb muscles. The controller of NESS Handmaster is attached to the orthosis by a

cable, which provides 5 different selections (3 exercise modes and 2 function modes)

for the users (Snoek, Ijzerman, Groen, Stoffers, & Zilvold, 2000). The exercise modes

provide repetitive stimulation of targeted finger and thumb extensor and flexor and

enable the users to take repetitive exercises. The functional modes include key grip

26

and release mode and palmar grasp and release mode. The first triggering of system

activates finger flexors and thumb extensors in key grip mode, or finger and thumb

extensors in palmar grasp mode. After an adjustable preset duration, stimulation of the

appropriate flexors is sequentially activated and enables the users to maintain of either

key grip or palmar grasp. The second triggering of system enables the users to release

the hand in key grip and release mode or palmar grasp and release mode. Triggering

of the system in either exercise modes or functional modes can be done by pressing a

push pad on the controller, or by activating a remote trigger that is incorporated into

the orthosis.

Studies of cyclical stimulation have shown positive effects on motor recovery after

stroke (Handa & Hoshimiya, 1987; Kralj, Acimovic, & Stanic, 1993). The use of

cyclical stimulations can reduce spasticity of muscles, increase the strength of

muscles and movement of joints, and correct contractures of muscles (Powell,

Pandyan, Granat, Cameron, & Stott, 1999). However, such systems have shown less

effect on the restoration of upper limb motor functions compared with therapy using

ES triggered by voluntary movement (e.g. using position information or EMG signals

as the inputs to the control of ES) (Kroon, IJzerman, Chae, Lankhorst, & Zilvold,

2005).

2.2.5.2 FES systems that are controlled via voluntary effort from the user

Of central interest to this thesis are the systems that are controlled via voluntary effort

from the user. These include systems controlled by EMG, EEG and voluntary

movement of the upper limb itself. These systems are described and reviewed below:

a) EMG triggered systems

EMG triggered ES is one of the techniques that allows voluntary control from the

users. In general, EMG triggered systems start stimulation if the voluntary EMG

signals reach a specific threshold and terminate stimulation once it drops below this

threshold. Another EMG triggered system that controls grasping has been developed

(Saxena, Nikolic, & Popovic, 1995). The system records the EMG signals from wrist

27

extensors, and once the voluntary contractions of wrist extensors exceed a specific

threshold (next-state function), the stimulation of finger and thumb flexors will turn

on.

Figure 2.11: State machine control of Saxena, Nikolic et al.’s EMG triggered system

(Saxena et al., 1995). The system detects voluntary EMG signals from wrist extensors,

and stimulation of the finger and thumb extensors will produce hand opening when

the EMG signals exceed a specific threshold.

Another example of a system using EMG signals as control inputs is the STIWELL

med4 (Rakos, Hanh, Uher, & Edenhofer, 2007). This system provides the user with

two EMG channels for measuring muscle activation and up to four stimulation

channels. Exceeding a threshold will activate stimulation to the target muscle groups.

The system can provide pre-programmed functional tasks (e.g. hand to mouth).

EMG triggered systems that employ a proportional control strategy allow the control

of electrical stimulation intensity, to be proportional to the magnitude of voluntary

EMG signals. Muraoka has developed an EMG controlled system, which is called the

integrated volitional control electrical stimulator (IVES), for the elicitation of wrist

and fingers extension (Muraoka, 2001; Yamaguchi et al., 2011). The IVES activates

stimulation of muscles at a specific intensity level, and makes the intensity level

proportional to the voluntary EMG signals. In another word, the level of users’ wrist

and fingers extension will be in proportion to the amplitude of voluntary EMG signals

recorded from a target muscle.

The EMG triggered systems are believed to improve arm/hand motor functions during

stroke recovery (Bolton, Cauraugh, & Hausenblas, 2004). However, the limitations to

Hand close

(State 1)

Hand open

(State 2)

EMG signals are exceeding

a specific threshold

EMG signals are going

below a specific threshold

28

the EMG triggered system are: 1) a minimum voluntary muscle contraction is

required (Bolton et al., 2004; Saxena et al., 1995); 2) reliable surface EMG data is

limited in people with paretic upper limbs, especially in dynamic conditions. The

interpretation of surface EMG data becomes significantly complex in dynamic

conditions. The factors such as force output, muscle fiber length and relative position

of surface EMG electrodes and source occurring during dynamic tasks will all affect

the changes of surface EMG signals (Gazzoni, 2010; Yamaguchi et al., 2011).

b) EEG triggered systems

EEG triggered systems also allow voluntary control by the user (Lauer, Peckham, &

Kilgore, 1999; Pfurtscheller et al., 2005; Scherberger, 2009). The EEG signals

typically record the brain activity by using the surface scalp electrodes at several sites

over specific brain regions (Lauer et al., 1999; Sinkjaer et al., 2003). Lauer, Peckham

et al. developed an EEG based control system that allows control of hand opening and

closing. The system records EEG signals and converts such signals into the command

signals. Two thresholds (a high threshold and a low threshold) have been preset. If the

EEG signals go above the high threshold, this generates the command signals for

activating stimulation of the muscles, which are responsible for hand closure. When

the EEG signals go below the high threshold, command signals for stop closing are

generated. Such stimulation stops and the hand stops closing. In order to activate the

stimulation of muscles responsible for hand opening, the EEG signals need to go

below the low threshold, which generates command signals to go from hand closed to

hand open.

29

Figure 2.12: State machine control of Lauer, Peckham et al.’s EEG triggered system

(Lauer et al., 1999). The system records EEG signals and converts these into

command signals, which are used to control hand opening and closing.

Most of the current EEG triggered systems use non-invasive EEG electrodes. The

non-invasive EEG electrodes can only provide indirect neural signals and only have a

limited information transfer rate capacity (Scherberger, 2009). In addition, the current

EEG triggered systems, require subjects to complete a great amount of training before

using EEG triggered systems (Scherberger, 2009).

c) Motion-triggered systems

The current motion triggered systems typically use shoulder motion (P. Hunter

Peckham et al., 2001; P. Hunter Peckham & Knutson, 2005), wrist motion (Prochazka

et al., 1997; Prochazka, Wieler, Kenwell, & Gauthier, 1996), contra lateral hand

motion (Jayme S. Knutson et al., 2009) or head motion (P. H. Peckham, Mortimer, &

Marsolais, 1980) as the control source to the FES system. The typical examples of

motion triggered systems are described as follows.

Hand close

(state 2)

Hand open

(state 1)

EEG signals are going below a high

threshold

EEG signals are exceeding a high

threshold

Hand stop

closing

(state 3)

EEG signals are going below a

low threshold

30

Jayme S. Knutson et al., (2007) developed a new contralaterally controlled functional

electrical stimulation (CCFES) treatment which aims to restore finger and thumb

extension (see figure 2.13). The system employs a proportional control motion

triggered system, with the degree of paretic hand opening proportional to the

voluntary opening of the contralateral unimpaired hand (Jayme S. Knutson, Harley,

Hisel, & Chae, 2007; Jayme S. Knutson et al., 2009). The authors claim the CCFES

system requires no residual hand movement of patients and less occupational therapist

time due to self-administration of CCFES use.

Figure 2.13: A glove with sensors detects the degree of unaffected hand opening, and

stimulation of the finger and thumb extensor muscles will produce proportional

opening of affected hand (Jayme S. Knutson et al., 2009)

Another example was the Bionic Glove, first developed in 1989 (Prochazka et al.,

1997; Prochazka et al., 1996). The Bionic Glove FES system is designed for

producing functions of hand grasping and opening for C6/7 spinal cord injury (SCI)

patients (Dejan Popovic et al., 1999). The typical Bionic Glove usually has a wrist

position sensor, which is used for detecting voluntary wrist movement. Such signals

collected from wrist position sensors are used as the reference to the control of FES.

Voluntary wrist flexion to a specific preset angle activates stimulation and result in

hand opening. Conversely, wrist extension to another preset angle activates

stimulation of specific muscles and produces a pinch grip (see figure 2.14). However,

due to the requirement of voluntary wrist movement, the Bionic Glove is only suitable

for patients with C6/7 SCI. The two main effects of daily use of the Bionic Glove

Affected arm Unaffected arm

31

were (1) increasing of grasp force; and (2) increasing of the range of movements of

finger joints (Dejan Popovic et al., 1999).

Figure 2.14: The Bionic Glove. (a) The adhesive electrodes are placed on the muscles

that will be activated. (b) The wrist position sensor is used for detect the voluntary

movement of wrist. (c) When the wrist flexes to a preset angle, the stimulation is

activated to the muscles for production of hand opening. (d) When wrist extends to

another preset angle, the stimulation is activated to the muscles for production of

pinch grip (Prochazka et al., 1997)

Motion triggered systems allow voluntary control of stimulation to the target muscles,

via voluntary movement of the upper limb itself. Many motion triggered systems are

effective for restoration of upper limb functions (Jayme S. Knutson et al., 2009; P.

Hunter Peckham et al., 2001; P. Hunter Peckham & Knutson, 2005; Prochazka et al.,

1997).

2.2.5.3 Summary

In summary, most of the current motion triggered systems are pre-programmed for a

limited range of specific tasks (see Table 2.1). For example, as discussed above, the

Bionic Glove FES system (Dejan Popovic et al., 1999) can only stimulate wrist and

(a)

(b)

(c)

(d)

32

finger muscles, using wrist extension/flexion as the control signal, and hence is

limited to the restoration of grasp/release (see Figure 2.14). In order to adapt the

system to use different control inputs or vary the control profile, an engineer would be

required.

Current FES systems for the upper limb rehabilitation are also inflexible in terms of

the number and location of muscles to be stimulated. For example, the objective of

most commercial FES systems seems to be to restore wrist and hand function only

(Hara, 2008; Lynch & Popovic, 2008; P. Hunter Peckham & Knutson, 2005).

Relatively little attention has been paid to the development of flexible systems, which

allow the user to stimulate a set of muscles specific to both task and patient-specific

impairment patterns (Tresadern et al., 2008). To enable the reader to gain an overview

of the systems discussed in the literature review, the following table summarises their

functional properties.

Developers

(year)

Max

stimulation

channels

Restricted

stimulation

to

particular

body

anatomy?

Voluntary

triggered

via body-

worn

sensor?

If so, is the

sensor

configuration

flexible?

Programmable

by therapists

for specific

functional

task

NeuroControl

Freehand

system

8 Yes Yes No No

H200

Wireless

Hand

Rehabilitation

System

5 Yes No / No

Microstim 2 2 No / / No

Odstock 4

Channel

Stimulator

Kit

4 No / / No

STIWell

Med4

4 No Yes No Yes

NEC

FESMate,

Early 80’s

30 Yes No / No

Bionic Glove,

(1989)

3 Yes Yes No No

33

Saxena et al.

(1995)

2 Yes Yes No No

S. E Crook

and P. H

Chappell

(1998)

8 Yes Yes No No

Muraoka

(2001)

1 Yes Yes No No

Pfurtscheller

et al. (2003)

/ Yes Yes No No

J. S Knutson,

Hoyen,

Kilgore, and

Peckham

(2004)

No

/ Yes No No

Pfurtscheller

et al. (2005)

4

Yes Yes No No

Jayme S.

Knutson et al.

(2007)

3 Yes Yes No No

Tresadern et

al. (2008)

2

No Yes No Yes

Chadwick et

al. (2011)

/ Yes Yes No No

Table 2.1: Functional properties of existing FES systems

Referring to table 2.1, most FES applications are limited by their hardware design to

stimulate particular parts of the upper limb, specifically the wrist/hand (S. E Crook &

P. H Chappell, 1998; Hermann et al., 2010; Hobby, Taylor, & Esnouf, 2001; J. S

Knutson et al., 2004; Jayme S. Knutson et al., 2007; Pfurtscheller et al., 2003; Saxena

et al., 1995). Microstim2 and Odstock 4 channel stimulator kit (Odstock Medical Ltd,

Salisbury, UK) provides flexible stimulation channels and they are not restricted by

design to stimulation of the particular body anatomy. However, their systems are not

under voluntary control and only provide exercise stimulation profiles to repeat a

fixed duty cycle. Arguably the most flexible systems are the STIWell Med 4 and the

CST developed by Tresadern et al., (2008). However, the CST set-up interface was

not sufficiently user-friendly to be exploited clinically, the sensor configuration was

fixed (e.g. see figure 2.26) and the exploitation of accelerometer signals for triggering

was relatively crude. The STIWell Med 4 provides a user-friendly interface, but is

limited by a fixed sensor configuration (EMG only).

34

This PhD reports on the design of a real-time control system that allows for accurate

and flexible control of stimulation via a number of inputs, including therapist-

specified voluntary movements of the patient. To achieve this goal, a flexible FSM

controller is required that can be set up to deliver appropriately sequenced FES to

specified muscles at levels appropriate to the patient and task requirements. At the

same time, a set-up tool is essential to guide clinical users, with little or no software

expertise, through the setup of FSM controllers. The system should be able to provide

sufficient stimulation channels and flexibility over the set of muscles to be stimulated

to cover the range of upper limb impairments commonly seen after stroke and a range

of functional tasks of everyday life. The system should also allow the user to trigger

stimulation via a user-configurable sensor set.

35

2.3 FES control

2.3.1 Introduction

Achieving satisfactory levels of FES control is very challenging. The nonlinear

(Ferrarin et al., 2001; Lynch & Popovic, 2008) and time-varying (Lynch & Popovic,

2008) response of muscles to stimulation are the two major problems in FES control.

For example, muscle fatigue, which is an unwanted side effect of prolonged

stimulation, affects the capacity of muscle to generate force (Ferrarin et al., 2001). In

addition, perturbations from muscle spasticity and other central nervous system

feedback loops introduce often unpredictable challenges to the controller (Lynch &

Popovic, 2008). To address these problems, designers have developed closed-loop,

open-loop and state machine controllers. Following a brief section on body worn

sensors used in FES systems, the merits and limitations of the closed-loop and open-

loop control approaches are described below. State machine control is described in

section 4.

2.3.2 Sensors for control of FES systems

Sensors can be either used as the command sources or the feedback sources for the

control of stimulation (P. Hunter Peckham & Knutson, 2005). Common sensors that

are employed by FES systems include force sensors (Lawrence et al., 2008), position

sensors (Popovic, Stojanovic et al. 1999), and EEG and EMG sensors (Pfurtscheller,

Müllera et al. 2003; Gert, R et al. 2005).

Force sensors (S. E. Crook & P. H. Chappell, 1998), position sensors or acceleration

sensors (Tresadern et al., 2008) are usually employed in FES systems to produce

command signals (e.g. start/stop stimulation of muscles). However, in more recent

FES control systems, force (Lawrence et al., 2008), position, and angle sensors have

been used to provide feedback signals . Additionally, EMG (Hara, Ogawa, Tsujiuchi,

& Muraoka, 2008), ENG, and EEG signals, as common biopotentials signals, which

are recorded from muscles, nerves and the brain or individual cells (Sinkjaer et al.,

2003), are also used in feedback control systems.

36

2.3.3 Continuous closed-loop control of FES

Closed-loop FES control system usually employ feedback signals, error detection and

correction processes, and a model of the system to decide the output of stimulations to

the target muscle groups for achieving a desired movement or force (Lan, Crago, &

Chizeck, 1991). The feedback signals are typically force (Lawrence et al., 2008),

position (Chadwick et al., 2011), joint angle (Kurosawa et al., 2005), EMG (Hara,

2008), ENG (Inmann & Haugland, 2004) and EEG (Sinkjaer et al., 2003) signals. If

properly designed closed loop control may lead to the improvement of functional

output in the presence of perturbations or fatigue of muscle (P. Hunter Peckham &

Knutson, 2005). For example, Lemay, Crago et al. (1993) investigated one open-loop

controller and two closed-loop controllers used for the restoration of hand grasp

function and found that the closed-loop controller required lower stimulation levels

and provided better regulation of grasp output (subjects performance when grasping

the instrumented object and maintaining a certain grasp opening) in the presence of a

disturbance (Lemay, Crago, Katorgi, & Chapman, 1993). In addition, less interaction

from the users of the closed-loop FES control systems is required, as modelling errors,

perturbations or fatigue of muscle are compensated automatically (Lynch & Popovic,

2008).

The following figure shows a generic closed-loop FES control system (shown in

figure 2.15). This generic closed-loop FES system is used to regulate joint angle by

manipulating the amount of stimulation delivered to the flexor and extensor muscle

groups (Lynch & Popovic, 2008).

37

Figure 2.15: A model of a generic closed-loop FES control system. This generic

closed-loop FES system is used to regulate joint angle by manipulating the amount of

stimulation delivered to the flexor and extensor muscle groups (Lynch & Popovic,

2008)

Kurosawa, Futami et al. (2005) have developed a feedback error learning controller,

which includes a feedforward and a feedback controller. Their controller (see Figure

2.16) employs the desired joint angle and its first and second derivatives (angular

velocity and angular acceleration) at times t to t+5 (discrete time interval of 50 ms) as

inputs. The feedforward controller (a four-layered neutral network) learns from

outputs (measured joint angle and stimulation currents) of the feedback controller

while control of limbs, and thus, can imitate the response of the joint angle to

electrical stimulation. The feedforward controller outputs are the stimulation currents

to the target muscle groups. Other stimulation currents which aim to cancel out the

difference between the desired joint angle and actual angle output from the feedback

controller. In this system, the feedback controller can compensate for load (e.g.

control of wrist angle with a 250 g cup in hand) and other disturbance (Kurosawa et

al., 2005).

38

Figure 2.16: The four-layered neural network control system in the Kurosawa, Futami

et al.’s feedback error learning controller; is the desired joint angle, and are

the stimulation current delivered to the target muscle groups (Kurosawa et al., 2005)

Muscle models can be considered in two categories (Lynch & Popovic, 2008): the

physiological models and the empirical or black box models. Physiological model

tend to be accurate, complex and differ between subjects, and attempt to model the

physiological structure and behaviour of muscle. The second type of model does not

reflect the structure of muscles, and just attempts to reproduce the input-output

behaviour of the real muscle (Hunt, Munih, Donaldson, & Barr, 1998; Lynch &

Popovic, 2008).

One of the common empirical muscle models is the Hammerstein model (Hunt et al.,

1998). The Hammerstein model, as shown in figure 2.17, contains a static recruitment

nonlinearity (the recruitment curve describes the proportion of stimulation-recruited

muscle fibers as a function of describing static gain relation between stimulation

activation level and output torque while muscle length keeps constant) and the linear

dynamic model (linear discrete-time transfer function that describes the contraction

dynamics of the muscle in response to the stimulation). In the diagram, the ( ) is the

stimulation pulse width (constant amplitude), ( ) is the disturbance signal, and

( ) is the muscle moment. is the delay operator, is a discrete input-output

time-delay that is greater than or equal to 1, and are polynomials in the delay

operator. However, the Hammerstein model is not accurate in representing of

39

behaviour of stimulated isometric muscles (Hunt et al., 1998; Le, Markovsky,

Freeman, & Rogers, 2009).

Figure 2.17: The Hammerstein muscle model (Hunt et al., 1998)

By contrast to the lower limb, for which there are a number of closed-loop controllers,

including a few used in the clinical environment (Ferrarin et al., 2001), there are very

few FES applications for the upper extremities and those that have been reported are

not widely used in the clinical environment (Chae, Sheffler, & Knutson, 2008). One

possible reason is that the upper limb has kinematic redundancy, allowing for multiple

joint trajectories for any given end point trajectory (Jeng-Feng Yang, Scholz, &

Latash, 2007). Further, the upper limb is used for a much wider range of tasks than

the lower limb, where the objectives of FES are prevention of footdrop during the

swing phase of gait, restoration of standing and transfer (seated to standing), and

restoration of walking (P. Hunter Peckham & Knutson, 2005). Thus, control of FES to

the upper limb may meet more challenges than the control for lower limb. Some

examples of closed-loop FES systems are given in the following.

Hara has described a power-assisted FES system that comprises a surface electrode

for recording EMG signals and an electrical stimulator (Hara, 2008). The EMG

signals from the target muscles are recorded and delivered to a controller (Hara, 2008).

The stimulator activates the same muscles in proportion to the EMG signals. As the

power-assisted FES system employs closed-loop control, after proper setting of

parameters of EMG sensitivity and electrical stimulation, further adjustment is not

required (Hara, 2008).

40

Figure 2.18: Example of a closed-loop EMG control system (Hara, 2008). The EMG

signals are recorded from the target muscles and used as feedback signals.

Chadwick, Blana et al. have developed a closed-loop FES system for the restoration

of arm and hand functions, based on cortical neuron data (Chadwick et al., 2011). The

system records the cortical neuron signals and is able to control a stimulated dynamic

arm in real-time. The details of the system are shown in figure 2.19. The system

includes a dynamic arm model which has two degree of freedom and includes six

muscles. The controller is used to calculate and control activations of muscles to

produce desired movement trajectories of the virtual arm according to the decoded

movement command and feedback position of the virtual arm. Additionally, the

controller can correct for arm trajectory errors, which may be caused by perturbations

or the rapid change of muscle properties, such as muscle fatigue (Chadwick et al.,

2011).

41

Figure 2.19: Overview of experimental setup (Chadwick et al., 2011). The system

detects the cortical neuron signals and is able to control a simulated dynamic arm. The

position of the virtual arm is recorded and used as the feedback.

The neural data are collected from the area of the motor cortex that controls arm

movements and delivered to the decoder. Based on these inputs movement velocity

commands (velocity of planar movement of virtual arm) are decoded and delivered to

the FES controller, which is used to produce coordinated stimulations of multiple

muscles to achieve the desired movements. The current position signals from virtual

sensors on the arm are fed back to the FES controller. The dynamic movement of the

virtual arm, which is the simulation of muscle activations effects, is displayed to the

user via a monitor.

Inmann and Haugland (2004) compared open-loop control and closed-loop control of

a hand grasp neuroprosthesis. The closed-loop control system is controlled by three

push buttons (button on the wheelchair control unit controls system on/off, other two

buttons control system for selection of grasp patterns and increase/decrease the

stimulation intensity) and ENG feedback signals, which are recorded by natural

sensors in the skin of the index finger, and offer functions of open hand, lateral grasp

and palmar grasp. When the stimulation intensity is below a specific threshold, the

hand will only produce a negligible grasp force or open. Otherwise, the system will

Spike sorting

Decoder

FES Controller Dynamic Arm

Model

Cortical neuron signals

Command of real-time

movements

Feedback of positions of virtual arm

Resulting joint

angles

Virtual display of

dynamic arm

movements

Muscle

activations

Arm area of

motor cortex

42

enter automatic command control state that allow automatic control of stimulation

intensity based on the feedback ENG signals during the test. The user (tetraplegic

patient) of the closed-loop control system incorporating ENG feedback signals was

asked to attempt an eating task, using their neuroprosthesis. Also, open-loop control

increased the stimulation intensity and locked at the maximum level, and thus

accelerated the onset of muscle fatigue. The task completion time did not differ

significantly between open-loop control and closed-loop control (Inmann & Haugland,

2004).

Despite the potential advantages of feedback control, most FES applications still use

open-loop control (Kostov, Andrews, Popovic, Stein, & Armstrong, 1995; Prochazka,

1993; Schmidt & Wrisberg, 2008). The FES system with feedback control remains

largely a research system (Frankel et al., 2011; Prochazka, 1993). In addition, in

feedback control systems, both complexity of coordination and unpredicted

interaction with external objects are difficult to compensate for (Cole & Sedgwick,

1992; Teasdale, Forget, Bard, Paillard, & Lamarre, 1993).

2.3.4 Open-loop control of FES

The simplest method of controlling FES systems is open-loop control. Typical open-

loop control systems consist of a stimulator and a command source. The command

source is typically a switch (Tomović et al., 1995) used by the patient or therapist

(acting as the system controller) or a specific muscle activity (Braz et al., 2009;

Francisco et al., 1998). Open-loop control systems can be discrete (equivalent to a two

state finite state machine), in which the users trigger or terminate stimulation via a

(usually) manually operated switch. Alternatively, open-loop control systems can

provide continuous control, in which the level of a continuous signal from a body-

worn sensor sets the amplitude of stimulation. Both systems use the patient as the

controller, and inputs are determined based on proprioceptive feedback, vision, and

experience (Prochazka, 1993).

An upper limb FES system based on open-loop control, the NESS H200 hand

restoration system (NESS H200, Bioness Inc., Valencia, CA), is now available for

43

clinical use (Kanchiku, Lynskey, Protas, Abbas, & Jung, 2008). The system

incorporates a wrist extension orthosis and a controller. Users are required to

manually control the system via a push-button switch (Hermann et al., 2010), such as

selection of operation mode, turn on/off stimulation, adjustment of stimulation

intensity and etc.

The Freehand system is another open-loop neuroprosthesis, which was commercially

available. It was aimed at restoring hand grasp and release for patients with C5 or C6

tetraplegia (P. Hunter Peckham et al., 2001; P. Hunter Peckham & Knutson, 2005).

The Freehand system provides two kinds of hand grasps, a lateral grasp and a palmar

grasp. The users can select the hand grasps and turn on/off the system through a

switch located on the shoulder controller (also known as position detector). The

shoulder controller incorporates both a switch and a potentiometer. The magnitude of

stimulation to muscles that control hand opening is proportional to the extent of

retraction of the shoulder and stimulation to finger flexor muscles is proportional to

the extent of shoulder motion in the opposite direction. The system allows locking the

hand in a fixed position by a quick elevation or depression movement of the shoulder

(see figure 2.20).

Figure 2.20: The NeuroControl Freehand system (Hobby et al., 2001). The Freehand

system can provide both lateral grasp and palmar grasp, and users can select the hand

grasps and turn on/off the system through a switch located on the shoulder controller.

44

The stimulation of the muscles that control hand opening/ grasp is proportional to the

extent of shoulder motion.

Another open-loop neuromuscular applications is the Bionic Glove (Prochazka et al.,

1997; Prochazka et al., 1996). A wrist position sensor was implemented with the

Bionic Glove and used to detect the voluntary flexion of the wrist. If the angle of

wrist flexion exceeds a preset threshold, stimulation of muscles that open the hand

will be activated. Similarly, voluntary extending the wrist to another threshold angle

will activate stimulation of muscles that produce grasp (Prochazka et al., 1997). Thus,

the movement of wrist directly controls the onset/termination of the stimulation of

specific muscles that open/close the hand.

In summary, the open-loop control systems of FES demonstrate many problems.

Open-loop control requires a great deal of information about the system’s properties

in order to produce acceptable movements. However, in open-loop systems, the

controller is the person being stimulated, or a clinician. Therefore, any perturbation

(e.g. muscle length, contraction velocity, fatigues of muscle) may be difficult to

compensate for (Patrick E. Crago, Lan, Veltink, Abbas, & Kantor, 1996). This

phenomenon may also lead to over stimulation of corresponding muscle (Comer,

1995). There is a tendency for fatigue to rapidly become an issue in open-loop

systems, due to the difficulty in appropriately controlling stimulation levels to

particular muscles based entirely on observations of the limb state (Hoffer et al.,

1996). Some factors, such as the nonlinear and time-varying response of stimulation

of muscles, muscle fatigue, electrode drift, length-tension properties of the muscles

are hard to address in the open-loop system (Haugland, Lickel, Haase, & Sinkjaer,

1999). Also, many open-loop control systems required full attention from the users

(users need to continuously or repeatedly operate the FES system) (Lynch & Popovic,

2008).

2.3.5 Conclusion

Current FES control systems for the upper limb remain limited in the functionality

they can restore. Closed-loop control systems are slow compared with open-loop

45

systems, and provide accuracy of control of muscles at the expense of time (Frankel et

al., 2011; Prochazka, 1993). Also, most of the current models of muscles for closed-

loop control, despite their complexity, are not comprehensive enough (Cole &

Sedgwick, 1992; Lynch & Popovic, 2008; Teasdale et al., 1993), thus unpredicted

interaction with external objects and fatigue is hard to compensate for and will affect

the performance of the system (Haugland et al., 1999). Secondly, the users of open-

loop control systems must set the parameters of FES (e.g. stimulation intensity) based

visual feedback and experience (Prochazka, 1993) making continuous open-loop FES

control difficult (Haugland et al., 1999; Lynch & Popovic, 2008).

The finite state machine technique gives an alternative way of controlling FES. The

finite state machine can be implemented as an open-loop control system (e.g. with

pre-set timing transitioning between states) and as a closed-loop control system (e.g.

based on values from body-worn sensors). The finite state machine control using

sensors on the body for transitioning between states is a potentially good compromise

between patient-controlled open-loop systems and true closed-loop controllers and is

introduced in the next section (2.4 Finite State Machine (FSM) Control).

46

2.4 Finite state machine (FSM) control

2.4.1 FSM principle

A finite state machine is usually described in terms of a sequential circuit with several

internal states that the system may enter (Comer, 1995). Typically, a finite state

machine is composed of five entities: symbolic states, input signals, output signals,

next-state functions and output functions (Chu, 2006). In a finite state machine, each

state represents a possible situation (Ferdinand, Ruedi, Thomas, & Wolstenholme,

2006), and when in a particular state, the system must be able to stay in that state and

remain for a finite period of time, even if the input signals have changed (Comer,

1995). The transition between states will occur only if the next-state function has been

achieved. Therefore, the next-state function is also called a state transition condition.

The next-state function is based on the current state and input signals. The output

function depends either on the current state alone (defined as a Moore output), or a

combination of the input signals and the current state (defined as a Mealy output)

(Chu, 2006). (see figure 2.21).

Figure 2.21: Two types of output – Moore output and Mealy output

A simple example is introduced, in order to give a clear understanding of state

machines (see figure 2.22). The following finite state machine has i states, and state 1

is usually described as the initial state. In state 1, if the next-state function 1 has been

satisfied, transition 1 will be triggered and state moves to state 2. Similarly, in state 2,

if the next-state function 2 has been satisfied, state machine will move to state 3. The

state will move on to the next, which depends on the current state and next-state

function. As shown in the following state machine, once it moves to the state i, the

state machine will move back to state 1, if next-state function i has satisfied. Table 2.2

Current

state

Mealy

output

Moore

output

Input signals

47

gives more information on the example. The output of each state, in this example,

only depends on the state it is in and therefore can be classified as a Moore output.

Figure 2.22: A typical example of a FSM

State Next-state function Transition Output (Action)

State 1 Next-state function 1 & state 1 Transition 1

(move to state 2)

Action(s) for

state 1

State 2 Next-state function 2 & state 2 Transition 2

(move to next state)

Action(s) for

state 2

… … .. ..

State i Next-state function i &

state i

Transition i

(move to state 1)

Action(s) for

state i

Table 2.2: A typical description of attributes for the example FSM

In general state machines, the number of next states can be greater than 1 and the next

state to which the state machine will go, depends on which next-state function has

been satisfied (see figure 2.23). Specifically, in the case shown in figure 2.23, if next-

state function i has been satisfied, the state machine will move to next state 1, the state

machine will move to next state 2 if next-state function i+1 has been satisfied. Strictly,

only one next-state function can be satisfied. In this example, the current state, which

is state i, is the same for both next state 1 and next state 2. Thus, the transition

conditions of state i to next state 1 or next state 2 only depends on the next-state

function, but not on the current state.

Transition 1

Transition 2

Transition i

Transition i-1

… … …

State 1

State i

State 2

Next-state

function 1

Next-state

function 2

Next-state

function i-1

Next-state function i

48

Figure 2.23: An example of a FSM which has two possible next states to enter

Outputs (actions) of state machines are classified under four categories, which depend

on the conditions and the timing. The four types of outputs (actions) are entry action,

exit action, input action and transition action (Ferdinand et al., 2006). The entry action

is the output which happens when a state machine enters a state. The exit action is the

output which happens when a state machine leaves a state. The input action is the

output which happens when input signals satisfy specific conditions. For each state,

there can be several input actions. The transition action is the output which happens

when the state moves (see figure 2.24).

Next

state 1

Transition i+1

Next

state 2

Transition i

State i

Next-state

function i

Next-state

function i+1

49

Figure 2.24: An example FSM showing four different output actions

2.4.2 Finite state control

Finite state control of FES typically includes several states, their associated actions

and next-state functions associated with those states. The actions associated with a

specific state (e.g. starting and stopping stimulation, ramp times and delays) are pre-

defined (Postans & Granat, 2005). The transitions to the next state are governed by

the current state and the value of artificial signals (e.g. switches), motion sensors or

biosignals, such as EMG.

The finite state method has been shown to be an effective and intuitive approach for

controlling of FES for improving the performance of gait (Kojovic, Djuric-Jovicic,

Dosen, Popovic, & c, 2009; Sweeney, Lyons, & Veltink, 2000) and standing (Braz et

al., 2009). Below, the finite state machine applications for upper limb FES control is

described.

Input actions

for state i

Input action

for state i+1

: Output

Transition i

Next-state

function i State i

State

i+1 State

i+2

Transition i+1

Next-state

function i+1

Entry action Exit action

Transition action Transition action

Input action

for state i+2

Input signals

50

Crook and Chappell designed a closed-loop system for grasp and release, which is

implemented with FSM control. The system includes 5 grip force sensors, mounted on

the finger tips, and a wrist sensor that controls the starting and stopping of the hand

grasp. The information from 5 grip force sensors is sent to the controller, and used for

the control of grasp. The wrist force sensor is responsible to provide the start signal

(user presses the wrist force sensor) and end signal (user presses the wrist force sensor

again) to the FSM controller. The FSM includes 7 states (see figure 2.25). The state of

hand relaxed is the initial state and in this state, all stimulations are turned off. When

the subject wishes to open to attempt to grasp an object, he needs to press the wrist

force sensor. Once the wrist sensor detects the starting signal from the wrist sensor,

the state moves to the second state, “preset extension”. In this state stimulation of

finger and thumb flexors are activated and ramp up until reaching their preset

maximum value, which leads to the opening of the hand. Then the state machine

moves to the next state, “wait for contact”. Stimulation to each of the muscles is

maintained and the five force sensors on the tips of the fingers and thumb are

monitored. In this state, the subject starts to grasp the target object (finger tips start

contacting the object). If the contact force exceeds a certain level, the state machine

transitions to ‘grip initiated’ and the extension stimulation decreases and flexion

stimulation increases. This allows the subject to grasp tightly. This continues until the

flexion pulsewidth exceeds half the maximum extension pulsewidth at which point

the next transition occurs to “grasp maintained”. In this state, stimulation adjusts for

the errors, which are caused by the delay between information from the sensor and

stimulation adjustment. If the subject wishes to release the object, the wrist sensor

needs to be pressed again, and FSM enters the last state, ‘Preset ramp down of flexion

and preset ramp up of extension’. Flexion stimulation is ramped down and extension

stimulation is ramped up at the same time when the wrist sensor receives an end

signal. When the flexion stimulation drops to zero and extension stimulation is

ramped up to a certain level, then all the stimulation is turned off. Afterwards, the

state will move back to the initial state (see table 2.3). The timing of change over from

flexion to extension takes place is important. A too fast change over will lead to a risk

of releasing the object when the user is not ready for adjusting themselves (S. E.

Crook & P. H. Chappell, 1998).

51

State Next-state function Transition Output (Action)

Hand relaxed Wrist sensor detects

starting signal

Transition 1 All the stimulation off

Pre-set

extension

Stimulation of extension

is ramped up

Transition 2 Stimulation turns on

(finger extensors and

thumb electrode) and

increases up to a pre-set

level

Wait for

contact

Touch sensors sense the

contact force exceeds a

certain level

Transition 3 Stimulation remains

maximum

Grasp

initialted

Flexion pulsewidth

exceeds half the

maximum extension

pulsewidth

Transition 4 Extension stimulation

goes to zero and the

flexion stimulation

increases

Grasp

maintained

Wrist sensor detects end

signal

Transition 5 Stimulation adjust

Preset ramp

down of

flexion and

preset ramp up

of extension

All the stimulation is

turned off

Transition 6 Flexion stimulation is

ramped down and

extension stimulation is

ramped up, then all the

stimulation is turned off

Table 2.3: Attributes of a FSM for hand grasp and release (S. E. Crook & P. H.

Chappell, 1998)

52

Figure 2.25: FSM for hand grasp and release (S. E. Crook & P. H. Chappell, 1998)

In many cases, including the example above, the user does not have the capacity to

define finite state machine topolgies and next-state functions. A group from Salford

University developed a clinical set-up tool (CST), which was implemented with a

finite state controller that allows the user to easily define and set up new state

machines (Tresadern et al., 2008). The CST allows the users to create their own FSM

by defining the attributes of the FSM (e.g. number of states, next-state functions,

selections from a range of sensor inputs, and actions). In the CST, the available next-

state functions include trigger conditions of greater than or less than. The variables

that may be used in state transition conditions include acceleration and time. The

actions are whether stimulation is on or off during each state.

Hand relaxed

(State 1)

Preset

extension

(State 2)

Wait for

contact

(State 3)

Grasp

initiated

(State 4)

Wrist sensor pressed

by the user

Stimulation of flexion is

ramp up to maximum

Touch sensor sensed contact

force exceeds a certain level

Flexion pulsewidth exceeds

half the maximum extension

pulsewidth

Wrist sensor detects

end

All stimulation is

turned off

Preset ramp down of

flexion and preset

ramp up of extension

(State 6)

Grasp

maintained

(State 5)

53

Figure 2.26: Screenshot of the CST GUI showing a FSM design for a drinking task

(Tresadern et al., 2008)

An example of a FSM, created using the CST, for a drinking task for a subject with

the ability to close the hand, but weakness in muscles that open the hand is shown in

figure 2.27. In the initial state, named the neutral state, the patient keeps the arm in a

neutral position (example FSM takes no action). When the patient attempts to move

the arm towards to the glass, the state moves to the next (next-state function is

satisfied by the x/y acceleration value taken from a forearm-located accelerometer

exceeding a specific user-defined threshold). In the second state, the action is

stimulation to the wrist and finger extensors is ramped up, leading to the opening of

the hand. After a specific period of time (triggered by a user-specified timeout

function), the state moves again. The stimulation is ramped down to zero as the action,

which leads to the patient being able to close the hand and the glass is then lifted to

the mouth. When the arm is then lowered, the x/y acceleration values trigger the

transition of the state at a specific user-defined threshold, and state moves again. In

this state, the glass is replaced on the table, and stimulation remains off. The next-

state function for the transition to the last state is the x/y acceleration value exceeding

a specific threshold. If the next-state function is satisfied, the state moves and

stimulation is activated again as the action of last state. The hand opens again to

54

release the glass. After another short period of time (timeout function triggers the

transition), the state goes back to the initial state and all simulations are turned off.

Figure 2.27: Example of a FSM for drinking from a glass task

Knutson, Hoyen et al. (2004) developed a FSM, which used myoelectric signals

(MES) from wrist flexor and extensor muscles as inputs to modulate the stimulation

sent to the target muscles in each state. Their FSM has four states (see figure 2.28):

Hand open, Hand close, Hold and Grasp pattern change. Different regions of the MES

space correspond to these states (see figure 2.29). No electrical stimulation was

implemented in this study.

Figure 2.28: FSM used for control of hand grasp

Grasp pattern

change

Hand close

Hold

Hand open

x/y acceleration

exceeding a

specific threshold

x/y acceleration exceeding a

specific threshold

x/y acceleration is going below

a specific threshold

Timeout function

Timeout function

Neutral Open Lift

Replace Release

55

Hold

Hand close

Hand open

Grasp pattern

change

Wrist extensor MES

Wrist flexor MES

Figure 2.29: The activation of states controlled by MES signals (J. S Knutson et al.,

2004)

Knutson, Hoyen et al.’ have tested their FSM for a task following a state sequence of

Hold, Hand close, Hold, Hand open (see figure 2.30). At the beginning, the FSM stays

at the state of Hold (wrist extensor and flexor MESs remain at a low level) and no

command signal will be applied (command signal and stimulation level will remain

same at its most recent value). If the wrist extension is increasing until simultaneous

wrist extensor MES moves to the region of state of Hand close, the state moves to

Hand close, and the FSM will generate a command signal that controls the stimulation

sent to the target muscle for hand close as the action of this state. In the next, both

Wrist extension and flexion decrease and simultaneous Wrist extensor and Wrist

flexor MESs move back to the region of Hold, it will activate the state of Hold again.

The command signal will remain the same at its value of leaving state of Hand close,

which means the stimulation sent to the target muscle for hand close will not change.

When the wrist flexion increase and simultaneous Wrist flexor MES moves to the

region of Hand open, it will activate the state of Hand open. The action for the state of

Hand open is generation of a command signal that controls of stimulation sent to the

target muscle for hand open.

56

Figure 2.30: Example of a FSM for a task following a state sequence of Hold, Hand

close, Hold, Hand open

2.5 Motion tracking and inertial sensors

2.5.1 Introduction

Various motion tracking systems have been used to measure the movements of

patients who have suffered a stroke, or have some other disability in motor function

(Zhou & Hu, 2008). Optical motion tracking systems are widely used in human

movement research and can be either marker based or marker-free (Haché, 2010;

Zhou & Hu, 2008). The marker based systems, such as Vicon or Optotrack, provide

high accuracy position information (Roetenberg, 2006; Zhou & Hu, 2008). However,

the major limitations of these systems are their very high costs (Roetenberg, 2006;

Zhou & Hu, 2008), the intensive post-processing required (Zhou & Hu, 2008), and the

need for a specialized laboratory with fixed equipment (Haché, 2010; Roetenberg,

2006). Although marker-free optical tracking systems can be lower cost (Zhou & Hu,

2008), they still require a number of dedicated cameras (Haché, 2010; Zhou & Hu,

2008) and need to be used under well controlled lighting conditions (Haché, 2010).

Both marker based and marker-free optical tracking systems require time and

Wrist extension and simultaneous Wrist extensor MES

moves to the region of Hand close

Wrist flexion and simultaneous Wrist flexor MES moves to

the region of Hand open

Hand close Hold

Hand open

Wrist extension and flexion decrease and both simultaneous

Wrist extensor MES and simultaneous Wrist flexor MES

move to the region of Hold

57

expertise for set-up and camera calibration. All of these limitations currently prevent

their use in everyday life.

Inertial sensors are commercially available and have been used for human motion

tracking outside the laboratory (Haché, 2010). They are low cost (Barbour & Schmidt,

2001; Verplaetse, 1996; Zheng et al., 2005), low power consumption (Cuesta-Vargas

et al., 2010; Zheng et al., 2005), small in size (Barbour & Schmidt, 2001; Cuesta-

Vargas et al., 2010; Zheng et al., 2005), and light weight (Barbour & Schmidt, 2001;

Cuesta-Vargas et al., 2010). In addition, inertial sensors can provide accurate and

reliable outcomes for motion tracking during functional tasks (Cuesta-Vargas et al.,

2010) and are, therefore, an attractive option for measuring the upper limb movements

of stroke patients during therapy.

2.5.1.1 Motion tracking using only accelerometers

Accelerometers are now a particularly attractive option because 3-axis devices are

available at low cost, in very small packages (e.g. 5×5×1mm chips), and with very

low power consumption. Hence, they can be easily used as body worn devices for

measuring and assessing human movements in clinics, research laboratories (Bonato,

2005; Mathie, Coster, Lovell, & Celler, 2004) and even in free-living environments

(Mathie et al., 2004; Che-Chang Yang & Hsu, 2010). Accelerometers can also be

used to continuously record data for very long time periods, over weeks and even

months (Godfrey, Conway, Meagher, & ÓLaighin, 2008).

Examples of accelerometers being used to measure human movement include gait

analysis (Morris, 1973; Villanueva, Trujillo, Fennon, Cardie, & Hedz, 2002), posture

and trunk movement (G.M. Lyons, Culhane, Hilton, Grace, & Lyons, 2005; Morris,

1973), physical activities (Mathie et al., 2004) and upper limb movement (Wong,

Wong, & Lo, 2007; Zhou, Stone, Hu, & Harris, 2008). Examples specific to the upper

limb include: measurement of intensity and duration of wrist movement for patients

with either Alzheimer’s or Parkinson’s disease (Someren, 1997); measurement of

forearm cyclic movements for the assessment of bradykinesia in patients with

58

Parkinson’s disease (Veltink, Engberink, Hilten, Dunnewold, & Jacobi, 1997); and

use in an FES controller to trigger state transitions (Mann, Taylor, & Lane, 2011).

When accelerometer signals are integrated, to estimate velocity or position, this leads

to a significant drift over time as a result of the integration of sensor noise and offset

(Zhou & Hu, 2008). However, the measurement of angle (also referred to as tilt)

between the sensor axes and the vertical does not suffer drift because it relies on the

gravity component of the measurement, not on integration of the signal (H.J. Luinge,

Veltink, & Baten, 1999).

In summary, accelerometers offer promise as sensors for use in the control of FES for

upper limb rehabilitation, as 1) modern accelerometers are low cost, low power, small

in size and provide accurate and reliable data; and 2) integration-related drift can be

avoided. Thus, it was decided to focus the work described in Chapter 3 on the use of

accelerometers alone to measure body segment angle from vertical.

2.5.1.2 Motion tracking using only gyroscopes

When the angular velocity output is integrated to obtain angle, body worn gyroscopes

typically suffer drift of a few degrees per second (H. J. Luinge, 2002; Roetenberg,

2006; Zhou & Hu, 2007). This limits their use for long-term accurate measurement.

Nevertheless, they have been used for human movement measurement. Tsuruoka,

Ochi et al. (1999) introduced a method for the assessment of walking stability of

hemi-paresis patients using three gyroscopes attached to the head, the trunk at the

height of shoulder, and the pelvis respectively. Aminian, Najaf et al. (2002) used

gyroscopes placed on the thigh and shank for gait analysis including the estimation of

velocity and stride length during long periods of walking.

The disadvantages of gyroscopes include: their increased sensitivity to temperature

and shock compared with accelerometers; when measuring change in angle, the drift

caused by the integration of sensor noise and offset; and their higher cost and power

consumption as compared with accelerometers.

59

2.5.1.3 Using sensor fusion with inertial and magnetic sensors

When using inertial sensors, including accelerometers and gyroscopes, drift is a

particular problem when the signals are being integrated to obtain position and

orientation (Roetenberg, Slycke, & Veltink, 2007; Sabatini, 2011). For example, the

integrated outputs of microelectromechanical gyroscopes are only accurate for short

periods of no more than several seconds (Roetenberg, 2006; Zhou & Hu, 2007).

The application of sensor fusion can overcome some of these limitations (Wong et al.,

2007) and can be an effective way to reduce drift in human motion tracking systems

(Zhou & Hu, 2007). For example, in orientation measurement, accelerometers and

magnetic sensors have been used to correct for drift errors. When using gyroscopes to

measure angles in the vertical plane, integration related drift can be corrected by using

a Kalman filter and accelerometers to measure the gravity vector (Luinge, Veltink et

al. 1999, Zheng, Black et al. 2005). When using gyroscopes to measure angles in the

horizontal plane, drift errors can be corrected for by using magnetic sensors that are

sensitive to the earth’s magnetic field (Roetenberg, 2006; Zhou & Hu, 2007).

2.5.2 Deriving angle from accelerometers

A major focus of this thesis is the use of accelerometers to measure body segment

angle from the vertical during upper limb tasks. Therefore, a systematic literature

search was performed to identify existing techniques for angle calculation using only

accelerometer outputs. Analysis of the literature identified three general approaches,

which were then assessed in terms of their advantages and drawbacks.

2.5.2.1 Using single axis accelerometers

The first group of methods are based on using just one accelerometer signal, either

from a single axis device or from a multi-axis device where the separate signals are

not being used together. Furthermore, they all assume that the true acceleration is

negligible and, therefore, the accelerometer is simply measuring the projection of

gravity onto its sensitive axis.

60

In one approach, following calibration over a range of angles, an interpolation

algorithm can be used to derive the angle of the sensitive axis from the vertical (P. E.

Crago et al., 1998; Dikkenberg et al., 2002; Graham, 2008). It should be noted that the

calibration curve is significantly non-linear and, therefore, if reasonable accuracy is

required, calibration must be undertaken at many different angles.

Alternatively, a trigonometric approach can be adopted which reduces the number of

points required for calibration. Referring to Figure 2.31, the angle between the

measured vector and gravity can be obtained by using ( ⁄ )

(Baek & Yun, 2010; Bakhshi, Mahoor, & Davidson, 2011; Bourke, Torrent, Parra,

Catala, & Nelson, 2011; El-Khatib, Guillon, & Dômont, 1998; Fabera, Changb,

Kingmac, & Dennerleina, 2013; Juan, Chen, & Shen, 2013; Karantonis, Narayanan,

Mathie, Lovell, & Celler, 2006; Kengo, Morio, Takagi, & Kajitani, 2013; Latt et al.,

2007; Myong-Woo Lee, Khan, Kim, Cho, & Kim, 2010; Lugade, Fortune, Morrow, &

Kaufman, 2014; Song, Jang, & Park, 2009). Similarly, the angle can be

obtained by using ( ⁄ ) (Caroselli, Bagalà, & Cappello,

2013; Constandinou & Georgiou, 2008; Ha, Park, Choi, & Kim, 2013; Husak, 2002;

Kemp, Janssen, & Kamp, 1998; Remme et al., 2009; Shizuka et al., 2009; Skotte,

Korshøj, Kristiansen, Hanisch, & Holtermann, 2014; Williams, 2004; Zhang, Qiao,

Song, & Wang, 2012). In practise, the denominators of the arcsin and arcos

arguments should be obtained by calibration as the maximum reading on the sensitive

axis may not be exactly 9.81. The angle derived by using the arcsin function yields

a value in the range (Shizuka et al., 2009; Skotte et al., 2014). Using the arcos

function yields values in the range to . Note that these two ranges correspond

to the same range of actual arm movement.

61

Figure 2.31: Angle calculation based on measurement on the sensitive axis and

gravity

Regardless of which of these three techniques is adopted for processing the

accelerometer signal ( ), they all suffer from the same drawbacks, which are

as follows. When approaches either 9.81 or -9.81, the sensitivity approaches

zero which means the signal to noise ratio is very poor and, therefore, the angle error

band increases (Baek & Yun, 2010; Kengo et al., 2013). This corresponds to the

regions around the zero slope points on the sine and cosine curves. Hence, it has been

suggested that these methods should not be used to measure angles within of

these zero slope points (Kengo et al., 2013). Maximum errors over were observed

when using ( ⁄ ), when approaches either 9.81 or -

9.81 (Baek & Yun, 2010; Kengo et al., 2013). No reports on errors for

( ⁄ ) have been found in literature. However, it is self-evident

that the angle derived by using the arcsin function suffers similar problems.

Finally, as mentioned above, using an accelerometer to measure angle from the

vertical is based on the assumption that the true acceleration is negligible and,

therefore, the accelerometer is simply measuring the projection of gravity onto its

sensitive axis. Therefore they are only suitable for measuring angle under static or low

acceleration conditions.

𝑔

𝑎𝑠𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑒

𝜃

𝛽

62

2.5.2.2 Using dual axis accelerometers

Referring to figure 2.32, when using a dual axis accelerometer (or two axes of a 3-

axis device), the signals from both of the sensitive axes can be used to calculate the

angle from the vertical as follows ( ⁄ ) (Cech, Dlouhy, Cizek, Vicha,

& Rozman, 2009; Coulter, Dall, Rochester, Hasler, & Granat, 2011; Gebert, Snyder,

Lopez, Siddiqi, & Evers, 2003; Grzeda & Fichtinger, 2010; Łuczak, 2007; Miura,

Watanabe, Akasaka, & Suzuki, 2011; Nevins, Durdle, & Raso, 2002; Pallejà,

Tresanchez, Teixidó, & Palacin., 2010; Qilong, Ruihe, Feng, Leilei, & Laiju, 2013;

Rodriguez-Donate, Morales-Velazquez, Osornio-Rios, Herrera-Ruiz, & Romero-

Troncoso, 2010; Vinande, Axelrad, & Akos, 2010; Watanabe, Murakami, & Handa,

2013).

Figure 2.32: Angle calculation based on measurements on the x and z axes

This method suffers from decreasing sensitivity and, hence, increasing angle errors as

approaches (Grzeda & Fichtinger, 2010) and extreme sensitivity near

(Pallejà et al., 2010; Rodriguez-Donate et al., 2010). Maximum errors of near have

been observed for this method (Pallejà et al., 2010). It is self-evident that this method

will not work for measured angle equal to . Furthermore, this method also relies

on the true acceleration being negligible and is, therefore, only suitable for measuring

angle under static or low acceleration conditions.

𝑔

𝑔𝑥

𝜃

𝑔𝑧

𝑔𝑦

63

2.5.2.3 Using two accelerometers separated by a rod

Some researchers have investigated the use of a rigid rod with an accelerometer

mounted on each end to derive angle from the vertical, but without being restricted to

low acceleration conditions (Djurić-Jovičić, D, Jovičić, & Popović, 2011; Milica D.

Djurić-Jovičić, Jovičić, Popović, & Djordjević, 2012; Kusuhara et al., 2012;

Willemsen, Frigo, & Boom, 1991).

For example, Kusuhara, Jikuya et al., 2012, used this approach to measure knee-joint

angle in the vertical plane. However, their technique is only suitable for applications

where the position of the knee is fixed. Indeed their interest was to capture knee angle

during a seated stretch reflex test.

Djurić-Jovičić, D and colleagues used a similar approach to capture angular motions

of leg and foot segments but without the need for any fixed joint centres (Kusuhara et

al., 2012). In other words, their interest was to capture the motion of the lower limb

segments during gait. However, they only derive the angular velocity and angular

acceleration of the segment from the two accelerometer outputs. Therefore they still

have to integrate to obtain angle from the vertical and, hence, have not avoided the

associated drift problems (i.e. their system is effectively a crude gyroscope).

2.5.3 Conclusion

Angle measurement using a single accelerometer (1, 2 or 3-axis) suffers from two

main problems. Firstly, the existing methods for processing the accelerometer signals

to obtain angle from the vertical all suffer from very poor sensitivity when a sensitive

axis approaches the vertical. Thus, a method that overcomes this problem is required

in order to exploit the accelerometer-derived angle data irrespective of the limb (and

hence sensor) orientation. None of the published work appears to have used the

approach reported in Chapter 3 to overcome this problem. Secondly, the methods

reported rely on the true acceleration being negligible and are, therefore, only suitable

for measuring angle under static or low acceleration conditions. As it is reasonable to

assume that in the application that is the focus for the thesis, there will be occasional

64

cases in which true acceleration is significant, algorithms to identify and sensibly

address these cases are also required (see Chapter 4).

Methods that use two accelerometers separated by a rigid rod have been used to

overcome the limitation of measuring angle only under low acceleration conditions.

However, this only works for rotation around a fixed point (e.g. the knee) or to obtain

angular velocity which must then be integrated.

2.6 Conclusions

This chapter introduced and discussed current rehabilitation interventions for

restoring upper limb motor function after stroke. The interventions included EMG

biofeedback, CIMT, robotics, mental practice with motor imagery, and FES therapy.

However, each of these interventions has significant limitations. A shared limitation is

that there is insufficient evidence of general effectiveness in a routine clinical setting

(Langhorne et al., 2009). Although further and better designed clinical trials in the

area are undoubtedly required, it is clear that improvements to the technological

interventions are also required.

One of the most commonly used therapy for improving motor function is FES(Lynch

& Popovic, 2008). However, current FES control systems for the upper limb remain

limited. As discussed in the chapter, the FSM technique is a promising approach to

control of FES as an alternative to continuous control methods. FSM controllers

typically use time and/or signals from body-worn sensors as inputs to rules governing

transitions between states and represent a potentially good compromise between

simple open-loop and complex closed-loop control. However, as the specific structure

of a state machine and the set of rules (and associated sensor data) governing

transitions are likely to be both task and subject-specific, current FSM control

approaches are not sufficiently flexible for general purpose use.

One promising body worn sensor for use in FSM control is the accelerometer, which

offers the potential to give information about the orientation and motion of limb

segments. Accelerometers can be used in most environments (Bonato, 2005; Mathie

65

et al., 2004) (Mathie et al., 2004; Che-Chang Yang & Hsu, 2010) are low cost, very

small (e.g. 5×5×1mm chips), and require very low power. However, due to sensor

noise and offset problems, integration leads to drift in estimation of position and

orientation. Nevertheless, the problems associated with integration can be avoided if

orientation can be calculated directly (H.J. Luinge et al., 1999).

There are a number of published techniques to estimate orientation directly from

accelerometer signals, which cluster into three general approaches. Previous

approaches to angle measurement using a single accelerometer (1, 2 or 3-axis) suffer

from two main problems. Firstly, the calculated angle from the vertical demonstrates

very poor sensitivity when a sensitive axis approaches the vertical. Secondly, the

methods reported rely on the true acceleration being negligible and are, therefore,

only suitable for measuring angle under static or low acceleration conditions. Methods

that use two accelerometers separated by a rigid rod have also been explored.

However, using two separated accelerometers to directly estimate orientation only

works for rotations around a fixed point; the alternative approach requires the

calculation of angular velocity which must then be integrated (hence introducing drift).

The primary aim of this thesis is to develop improved techniques for setting up and

implementing real-time FSM control for upper limb FES; inputs to the controller

should include segment angle derived from body worn accelerometers. The following

chapters describe the work to achieve this goal. Chapters three and four describe

novel methods for measuring angle from the vertical using 3 axis accelerometer data,

and report on new approaches to improve the robustness of angle-based state

transition rules. Chapter five reports on the implementation of the FSM controller and

an associated graphical user interface for guiding therapists in the design of FSMs

which are specific to the task and the patient’s impairment profile. The final chapter

highlights the novel work and identifies potential ways forward.

66

Chapter 3 – Angle tracking for state transitions

3.1 Introduction

In this chapter, methods are investigated for using an accelerometer to track body

segment angle and, hence, to act as a trigger for moving to the next state in an FES

state-machine controller (see section 2.4.1). An Xsens inertial sensing unit (MTx,

Xsens technologies B.V., Netherlands) has been used for test purposes.

The Xsens accelerometer provides the x, y and z components of the vector sum of the

acceleration and gravity vectors. For motions where the accelerations are low, this is

approximately equal to the gravity vector. Some of this raw sensor information may

be used directly to trigger state transitions. However, if the acceleration data can be

successfully converted into the angle that the Xsens (and arm segment it is attached to)

rotates through, then this angle can also be used for triggering state transitions. As

will become apparent, using body segment angle is less affected by placement errors

when donning the Xsens sensors and is also more meaningful anatomically.

In this chapter, firstly two uncalibrated angle tracking methods are presented that

convert acceleration data into the Xsens angle (or change of angle) from the vertical.

It should be emphasised that the gravity vector alone cannot provide any information

about rotation in the horizontal plane. In this context, “uncalibrated” refers to the fact

that the subject is not required to make any calibration movements after the Xsens

sensors have been donned.

The first uncalibrated angle tracking method calculates the change in the angle of the

Xsens from the vertical. It does this by calculating the angle between the gravity

vectors before and after the rotation, both expressed in the Xsens coordinate frame.

Note that, because there is no defined reference orientation (zero angle orientation),

67

this method does not give an absolute angle from the vertical, only the change in the

Xsens angle from the vertical during a rotation.

The second uncalibrated angle tracking method calculates the angle of the Xsens x-

axis from the vertical. Because there is now a reference orientation (i.e. the angle is

zero when the x-axis is vertical), this method provides the Xsens angle from vertical.

However, any rotation about the x-axis is not detected which can be advantageous if,

for example, the wish is to avoid triggering as a result of pronation-supination of the

forearm.

The two uncalibrated angle tracking methods can only provide positive angle values.

Thus, for example, it is not possible to distinguish between 30° of flexion and 30° of

extension when the start angle is zero (vertical). Therefore, a calibrated angle tracking

method has been developed which gives both the magnitude and the sign of the angle

change. The disadvantage is that calibration movements must be undertaken after

donning the Xsens sensor to define both the required plane of rotation and the positive

rotation direction.

Finally, an auto-calibration algorithm has been developed that updates the gains

applied to the three Xsens signals (x, y and z components), in real time, to compensate

for calibration errors. For example, if the y-axis axis is reading half of what it should

and the other axes are accurately calibrated, then the algorithm will home in on gains

of 1, 2 and 1 for the x, y and z axes respectively. The auto-calibration method can

operate in parallel with any of the angle tracking methods mentioned above.

3.2 First uncalibrated angle tracking method

The first uncalibrated angle tracking method aims to convert the accelerometer data (x,

y and z signals) into the change in Xsens angle from the vertical during a rotation.

Referring to Figures 3.1 and 3.2, as the Xsens rotates, the gravity vector is stationary

and vertical in the global coordinate frame (not shown), but appears to rotate in the

Xsens frame (shown) in the opposite direction to the Xsens itself. The gravity vector

at the start of the rotation is and the gravity vector after the rotation is complete

68

is . It is clear that the angle between these two vectors ( ) when expressed in the

Xsens frame (Figure 3.2), is the same as the angle through which the Xsens has

rotated.

Figure 3.1: The gravity vectors and in the global frame, before (left) and

after (right) the Xsens sensor has rotated, and their components

O

X axis (𝑔 𝑥)

Y axis 𝑔 𝑦

Z axis (𝑔 𝑧)

𝑔𝑓𝑖𝑟𝑠𝑡

(𝑔 𝑥)

X axis

Z axis (𝑔 𝑧)

O

Y axis 𝑔 𝑦


𝑔𝑛𝑜𝑤

𝜃

69

Figure 3.2: The gravity vectors and in the Xsens frame separated by the

rotation angle .

3.2.1 Mathematical procedure

The gravity vectors at the start ( or ) and at the end ( or ) of an Xsens

rotation are as follows

(3.1)

| | √

(3.2)

(3.3)

| | √

(3.4)

Where the x, y and z components are collected from the Xsens. Note that these can

only be considered to represent the gravity vector if any translational acceleration

during the rotation is small in comparison to 9.81.

a) Calculating using the dot product

The definition of the dot product gives the following

Y axis

Z axis

X axis


𝑔𝑛𝑜𝑤

70

| | | | (3.5)

Therefore, cos can be expressed as follows

| || | (3.6)

And thus

√

√

(3.7)

An alternative is to use the standard acceleration due to gravity (9.80665 ms-2

) in the

denominator as follows

(3.8)

b) Calculating using the cross product

The definition of the cross product gives the following

| | | | [ (

) ] (3.9)

Where is a unit vector that is perpendicular to both and .

Therefore, sin can be expressed as follows

| |

| || | (3.10)

71

And thus

√

( )

√

√

(3.11)

An alternative is to use the standard acceleration due to gravity (9.80665 ms-2

) in the

denominator as follows

√

( )

(3.12)

c) Angle calculation algorithm

As Figure 3.3 shows, when sin or cos approach 1, their sensitivity to changes in

approaches zero (i.e. their derivatives approach zero). This leads to very small

changes in sin or cos and, hence, a poor signal to noise ratio and correspondingly

large errors in the calculated as a result of signal noise. Therefore, to maximize

accuracy, the cross product ( sin ) is used for and for ,

and the dot product ( cos ) is used for .

72

Figure 3.3: The angle ranges where the cross product and the dot product are applied

It is also clear from Figure 3.3 that, in the range 0° to +180°, any one value of sin

corresponds to two values of (compare points A ( ) and B ( ) ).

Therefore, the value of cos is used to decide whether the result of ( )

should be in the range or in the range .

Therefore, combining the principles described above, the following logical rules can

be used to calculate from sin (cross product) and cos (dot product).

If cos 0.707106781 angle range is from 0° to 45°

( )

Elseif cos 0.707106781 angle range is from 135° to 180°

( )

Else (angle range is from 45° to 135°)

( )

End

-1.5

-1

-0.5

0

0.5

1

1.5

-180° -135° -90° -45° 0° 45° 90° 135° 180°

cosine curve sine curve

c A B

𝜋 𝜃 𝜃

𝐴𝑛𝑔𝑙𝑒(𝜃)

𝑠𝑖𝑛𝜃 𝑐𝑜𝑠𝜃 𝑠𝑖𝑛𝜃 𝑐𝑜𝑠𝜃 𝑠𝑖𝑛𝜃

73

3.2.2 Testing and discussion of results

The following test was designed to evaluate the accuracy of the first uncalibrated

angle tracking method. Both the test results and a discussion of the results are

presented.

a) Method

A protractor attached to a flat board was employed (see figure 3.4). The Xsens sensor

was attached to the blade of the protractor, close to the blade pivot to reduce any

errors caused by the protractor blade bending. Before starting the test, the board was

placed on a table and adjusted to ensure it was horizontal using a spirit level. During

the test, the protractor was moved from to , with short pauses at , ,

and . The test was repeated twice.

Figure 3.4: The protractor system used for testing

Vertical & Xsens gravity vecoter

before the rotation (X axis)

Xsens sensor

Protractor system

X axis

X axis

Vertical

Vertical

Horizontal

Protractor

Direction of Xsens sensor

rotates

74

b) Results

Figures 3.5 and 3.6 show the results from two tests where the Xsens sensor was

rotated from to with short pauses at , , and .


, , and (the protractor settings)


, , and (the protractor settings)

Frame number (Time × 100Hz)

Angle

(deg

ree)


Angle

(deg

ree)

75

In this context the errors are the differences between the calculated angles (based on

Xsens accelerometer data) and the protractor angles, which were as follows:

Protractor angles Average errors in test 1 Average errors in test 2

30 0.315 0.138

60 0.584 0.413

90 0.79 0.619

150 0.688 0.802

180 4.217 4.389

Table 3.1: Average errors in test 1 and test 2 compared with the protractor angles of

30°, 60°, 90°, 150° and 180°

c) Discussion of results

In both tests, the errors are small (less than 1°) for all angles except 180 where the

errors are 217.4 and 389.4 in tests 1 and 2 respectively (The errors are not

necessarily getting bigger up to 150). This problem is probably a result of the fact that

this method only returns positive values between 0⁰ and 180⁰. When measuring a

nominal angle from the vertical of 0⁰, the actual angle could be +δ⁰ or –δ⁰ where δ is

the deviation from the nominal angle. In both cases, this method would return a

positive value (i.e. +δ⁰). Similarly, when measuring a nominal angle of 180⁰, the

actual angle could be (180–δ)⁰ or (–180+δ)⁰. In both cases, this method would return

a positive value (i.e. 180–δ). Therefore a 180⁰ change in angle (rotation) from -2⁰ to

+178⁰ would be interpreted as a rotation from +2⁰ to +178⁰ leading to a 4⁰ error. This

is indicative of the fact that this method should not be used for rotations close to 180⁰,

not that it is inherently inaccurate.

A limitation of this method is that it only gives positive results. Regardless of the

direction of the change in Xsens angle from the vertical, the calculated angle is

always positive. The cross product based calculation involves taking the modulus of

the cross product and therefore loses any directional information. The dot product

method gives , which is symmetrical about and hence cannot distinguish

between positive and negative angles.

76

A big advantage of this method is that it does not rely on a particular placement of the

Xsens sensor relative to the anatomy. The downside of this is that the method cannot

distinguish between different anatomical rotations. For example, with the forearm

horizontal, a calculated change in forearm angle from the vertical could be a result of

either forearm pronation-supination or movements at the elbow or shoulder.

Furthermore, this also means that the method can only provide the change in angle

from the vertical, not the absolute angle, because there is no defined reference

orientation (zero angle orientation).

3.3 Second uncalibrated angle tracking method

As discussed in the previous section, the first uncalibrated angle tracking method

calculates the change in angle from the vertical and does not distinguish between

different anatomical rotations and there is no defined reference orientation (zero angle

orientation). The second uncalibrated angle tracking method calculates the angle of

the Xsens x-axis from the vertical. Because there is now a reference orientation (i.e.

the angle is zero when the x-axis is vertical), this method provides the absolute Xsens

angle from vertical. Furthermore, any rotation about the x-axis is not detected which

can be advantageous if, for example, the wish is to avoid triggering as a result of

pronation-supination of the forearm. To do this, the long axis of the Xsens (the x-axis)

would be simply aligned with the long axis of the forearm.


Referring to Figure 3.7, the method calculates the angle between and , where

is the unit vector representing the Xsens x-axis, and is the current gravity

vector which is always vertical. Therefore, is the angle between the Xsens x-axis

(the long axis of the Xsens box) and vertical.

77

Figure 3.7: Angle between (Xsens x-axis) and (vertical)

By adapting the equations used in the previous uncalibrated angle tracking method,

replacing with , the following expressions for ( ) and ( ) can be

obtained

( )

| || |

[ ] [ ]

|[ ]||[ ]|

√|

|

(3.13)

( ) | |

| || |

|[ ] [ ]|

|[ ]||[ ]| (3.14)

Where the cross product is given by

|[ ] [ ]|

√

( )

√( )

Therefore

( ) | |

| || |

|[ ] [ ]|

|[ ]||[ ]|

√( )

√|

|

(3.15)

𝑔𝑛𝑜𝑤

X axis

Y axis

Z axis

Forearm

Xsens 𝛽

��

78

The angle calculation algorithm that uses these expressions is identical to that of the

first uncalibrated angle tracking method (see section 3.2.1c).


The following test was designed to evaluate the accuracy of the second uncalibrated

angle tracking method. Both the test results and a discussion of the results are

presented.

a) Method

Again, a protractor attached to a flat board was employed (see Figure 3.4). The Xsens

sensor was attached to the blade of the protractor, close to the blade pivot to reduce

any errors caused by the protractor blade bending. Before starting the test, the board

was placed on a table and adjusted to ensure it was horizontal using a spirit level.

During the test, the protractor was moved from 0 (Xsens x-axis and vertical in same

direction) to 180 (Xsens x-axis and vertical in opposite directions), with short pauses

at 30 ,60 ,90 150and .

b) Results

Figure 3.8 shows the results from the test where the Xsens sensor was rotated from 0

to 180 , with short pauses at 30 ,60 ,90 150and .

79

Figure 3.8: Calculated angle in degrees. The horizontal lines indicate 0 , 30 , 60 ,

90 , 150 and 180 (the protractor settings)

In this context the errors are the differences between the calculated angles (based on

Xsens accelerometer data) and the protractor angles, which were as follows:

Protractor angles Average errors in test

0 2.5

30 1.3

60 0.5

90 0.3

150 0.2

180 2.3

Table 3.2: Average errors in test compared with the protractor angles of 30°, 60°, 90°,

150° and 180°

c) Discussion of results

The errors are small (less than 1.3°) for all angles except 0° and 180 where the errors

are 2.5° and 2.3° respectively. Again, this problem is probably a result of the fact that

≈ °

≈ °

≈

≈

≈

≈

Time (Sec)

Angle

(deg

ree)

80

this method only returns positive values between 0⁰ and 180⁰ (see the discussion on

page 72). In this case, because an absolute angle is returned (not a change in angle),

the problem manifests itself at angles of both 0⁰ and 180⁰. This is indicative of the

fact that this method should not be used for angles from the vertical which are close to

0⁰ or 180⁰, not that it is inherently inaccurate.

As is the case for the first uncalibrated method, a limitation of this method is that it

only gives positive results. However, it does have the advantage of providing the

absolute Xsens angle from vertical. Furthermore, any rotation about the x-axis is not

detected which can be advantageous if, for example, the wish is to avoid triggering as

a result of pronation-supination of the forearm. To do this, the long axis of the Xsens

(the x-axis) would be simply aligned with the long axis of the forearm.

3.4 Calibrated angle tracking method

The two uncalibrated angle tracking methods cannot indicate the direction (sign) of

the change in angle (or of the absolute angle) from vertical. This is because the dot

product gives , which is symmetrical about , and the cross product based

calculation involves taking the modulus of the cross product and therefore loses any

directional information ( is always positive). Furthermore, neither method can

define a particular plane, relative to the anatomy, in which the measured rotation

should take place. Therefore, a calibrated angle tracking method has been developed

which overcomes these problems.


a) Calibration stage

This method requires a calibration rotation to establish the desired plane of rotation

relative to the Xsens sensor and, hence, relative to the body segment it is attached to.

Two gravity vectors are captured during the calibration rotation, separated by at least

81

30° to avoid noise having a large effect on the direction of their cross product vector.

Their cross product is defined as follows

1 2 1 2ˆsin .g g g g n (3.16)

Noting that sin2121

gggg this leads to

1 2

1 2

ˆcalib

g gn

g g

(3.17)

This calibration vector ( calibn ) is normal to the desired plane of rotation and therefore

defines that plane.

b) Procedure during state-machine operation

During normal state-machine operation (after calibration), the calibration vector can

be used to obtain the component of the rotation in the desired plane and to determine

its direction (sign). The magnitude of the Xsens angle change from vertical is

calculated in the same way as for the first uncalibrated angle tracking method. In

addition, a vector perpendicular to the rotation plane, , is calculated in the same way

as calibn above, which defines the plane of rotation during state-machine operation.

Then, to calculate the component of rotation that is in the same plane as the

calibration rotation, the dot product is applied as follows

calibnn ˆˆcos (3.18)

cosproj (3.19)

where: is the magnitude of the Xsens angle change (between 0° and 180°)

is the angle between the calibration and new rotation planes;

82

proj is the component of the new rotation that is in the same plane as the

calibration rotation (i.e. projected onto calibn ).

Note that proj will have a sign associated with it, which indicates whether it is in the

same direction as the calibration rotation (+ve proj ) or in the other direction (-ve

proj ).

3.4.2 Initial testing

The first stage of testing focused on establishing whether the calculation of is

effective as this is fundamental to this approach. Initially this was done using the

same protractor system that was used in the testing of the first uncalibrated angle

tracking method (Figure 3.4), and without changing the plane of rotation after

calibration. In other words, the Xsens sensor remained attached to the protractor blade

in the same relative orientation during both calibration and subsequent angle tracking.

Therefore, the two vectors, and 𝑐𝑎𝑙 𝑏, were parallel and should have been 1.

The results are shown in Figures 3.9 and 3.10

Figure 3.9: Cosine of the angle between the rotation planes ( )

Frame 190 Frame 530


Cosi

ne

funct

ion v

alue

83

Figure 3.10: Corresponding magnitude of Xsens angle change ( )

Referring to Figures 3.9 and 3.10, it is apparent that there are problems when

and In these cases, large errors occur in , which should be 1 or

very close to 1 for all angles. The problem is believed to do with the fact that as the

two gravity vectors approach being parallel ( and ), the direction of

their cross product ( ) becomes very sensitive to small errors in the gravity vectors.

For these reasons, further testing has been abandoned for the time being.

3.5 Auto-calibration of Xsens axes

An auto-calibration method has been developed that updates the gains applied to the

three Xsens signals (x, y and z components), in real time, to compensate for

calibration errors. For example, if the y-axis axis is reading half of what it should and

the other axes are accurately calibrated, then the algorithm will home in on gains of 1,

2 and 1 for the x, y and z axes respectively. The auto-calibration method can operate

in parallel with any of the angle tracking methods mentioned above.

150°

30°

Frame 190 Frame 530


Angle

in r

adia

n

84


Referring to Figure 3.11, the magnitude of the corrected gravity vector can be

expressed as follows

|

| √( )

( ) (3.20)

Where , and are the auto-calibration gains for each Xsens axis

, and are the Xsens accelerometer readings before applying the gains

The three gains are defined as follows

𝑎

𝑎 (3.21)

𝑎

𝑎 (3.22)

𝑎

𝑎 (3.23)

If the Xsens is perfectly calibrated, then there is no need for any corrections and the

auto-calibration gains are all equal to 1.

85

Figure 3.11: Corrected gravity vector and its components

Under ideal conditions, assuming no noise and also that the accelerometer readings

are a result of gravity alone (acceleration is negligible), the correct gains can be

obtained by capturing three independent sets of accelerometer readings (representing

three different gravity vectors). Then the gains are calculated by solving the following

three equations relating the three unknown gains:

| | √( )

( ) (3.24)

| | √( )

( ) (3.25)

| | √( )

( ) (3.26)

Where ( ), ( ) and ( ) are the three sets of

accelerometer readings.

These lead to

|𝑔𝑚|

O

Y axis 𝑔𝑦 𝑘𝑦𝑎𝑦𝑗

X axis 𝑔𝑥 𝑘𝑥𝑎𝑥𝑖

Z axis

𝑔𝑧 𝑘𝑧𝑎𝑧𝑘

86

| |

( )

( )

(3.27)

| |

( )

( )

(3.28)

| |

( )

( )

(3.29)

Where,

Eliminating in the above equations leads to

| |

| |

(

) (

)

| |

| |

(

) (

)

Then solving for X and gives

(| |

| |

)(

) (

| |

| |

)(

)

(

)(

) (

)(

)

| | [ 𝑎

𝑎 𝑎

𝑎 𝑎

𝑎 𝑎

𝑎 𝑎

]

𝑎 𝑎

𝑎 𝑎

𝑎 (𝑎

𝑎 𝑎

𝑎 )𝑎

𝑎 𝑎

𝑎 𝑎

𝑎

(| |

| |

)(

) (

| |

| |

)(

)

(

)(

) (

)(

)

| | [ 𝑎

𝑎 𝑎

𝑎 𝑎

𝑎 𝑎

𝑎 𝑎

]

𝑎 𝑎

𝑎 𝑎

𝑎 (𝑎

𝑎 𝑎

𝑎 )𝑎

𝑎 𝑎

𝑎 𝑎

𝑎

87

And also

| |

𝑎

(| |

| |

)(

) (

| |

| |

)(

)

(

)(

) (

)(

)

𝑎

𝑎

(| |

| |

)(

) (

| |

| |

)(

)

(

)(

) (

)(

)

𝑎

𝑎

| | 𝑎

𝑎 𝑎

𝑎 𝑎

𝑎 𝑎

𝑎 𝑎

𝑎 𝑎

𝑎 𝑎

𝑎 (𝑎

𝑎 𝑎

𝑎 )𝑎

𝑎 𝑎

𝑎 𝑎

𝑎

These equations can be simplified by letting

| |

𝑎 𝑎

𝑎 𝑎

𝑎 (𝑎

𝑎 𝑎

𝑎 )𝑎

𝑎 𝑎

𝑎 𝑎

𝑎

This leads to

[

] (3.30)

[(

) (

)

(

) ] (3.31)

[(

) (

)

(

) ] (3.32)

Thus, the gains for each axis of the Xsens are

√ [

] (3.33)

√ [(

) (

)

(

) ] (3.34)

√ [(

) (

)

(

) ] (3.35)

88


a) Method

A random set of accelerometer data was collected by slowly moving and rotating the

Xsens sensor. This data was then scaled to create 4 data-sets, the original set and three

sets representing different miscalibrations of the Xsens as follows:

x-axis

scaling factor

y-axis

scaling factor

z-axis

scaling factor

Data-set 1 1 1 1

Data-set 2 0.5 1 1

Data-set 3 1 0.7 1

Data-set 4 1 1 0.2

Table 3.3: Scaling of original accelerometer data

The three different vectors used to test the auto-calibration of Xsens gain on three

different axes were the 100th ( ), 1600

th ( ) and 3200

th

( ) samples from the scaled accelerometer data.

b) Results for accelerometer data-set 1

Figure 3.12 shows accelerometer data-set 1 (the original accelerometer data with no

scaling). The three vertical lines indicate the 100th ( ) , 1600

th

( ) and 3200th

( ) samples from the original

accelerometer data.

89

Figure 3.12: Accelerometer data-set 1 (X axis acceleration data in blue, Y axis

acceleration data in green and Z axis acceleration data in red)

The gains determined using the auto-calibration algorithm were as follows:

The gains were all very close to 1, which was to be expected as none of the data was

scaled. The x and y-axes of the Xsens accelerometer were well calibrated (error less

than 1%), whereas the z-axis was less well calibrated (error of 3.3%).


Acc

eler

atio

n (

m/s

2)

90

c) Results for accelerometer data-set 2

Figure 3.13 shows accelerometer data-set 2 (scaling factors of 0.5, 1, 1). The three

vertical lines indicate the 100th ( ), 1600

th ( ) and 3200

th





The y and z-axis gains were the same as those calculated for data-set 1, which was to

be expected as this data was not scaled. The x-axis gain was 2 times that for data-set 1,

which was also to be expected as this data was scaled by 0.5.


Acc

eler

atio

n (

m/s

2)

91

d) Results for accelerometer data-set 3

Figure 3.14 shows accelerometer data-set 3 (scaling factors of 1, 0.7, 1). The three


th ( ) and 3200

th





The x and z-axis gains were the same as those calculated for data-set 1, which was to

be expected as this data was not scaled. The y-axis gain was

times that for data-set

1, which was also to be expected as this data was scaled by 0.7.


Acc

eler

atio

n (

m/s

2)

92

e) Results for accelerometer data-set 4

Figure 3.15 shows accelerometer data-set 4 (scaling factors of 1, 1, 0.2). The three


th ( ) and 3200

th





The x and y-axis gains were the same as those calculated for data-set 1, which was to

be expected as this data was not scaled. The z-axis gain was 5 times that for data-set 1,

which was also to be expected as this data was scaled by 0.2.


Acc

eler

atio

n (

m/s

2)

93

f) Discussion of results

The auto-calibration gains calculated from the original unscaled accelerometer data

seem reasonable (all close to 1, indicating that the Xsens accelerometer was

reasonably well calibrated). Furthermore, the algorithm deals accurately with the

scaled data, calculating auto-calibration gains that are exactly equal to 𝑎𝑙 𝑎

𝑐𝑎𝑙 𝑎𝑐 .

Further testing is required to determine:

How accuracy is affected when the three samples are close together (the

gravity vectors are more similar);

Whether the algorithm works accurately when multiple axes are scaled;

How the algorithm can be incorporated into a real-time state-machine

controller with angle tracking.

3.6 Conclusions

Two uncalibrated angle tracking methods and a calibrated angle tracking method have

been developed, which use as inputs 3 axis accelerometer signals. The methods can be

used to track body segment angle which can be used as input(s) to transition rules

governing moving between states in an FES state-machine controller. The methods

have been demonstrated using an Xsens inertial measurement unit and tested using a

protractor system (see Figure 3.4).

Using the first two uncalibrated angle tracking methods that convert accelerometer

data into angle (or change of angle) from the vertical requires no calibration after the

accelerometer have been donned. For the first uncalibrated angle tracking method the

errors were found to be generally small (less than 1°) except when approaching 180°

(see Table 3.1). The second uncalibrated angle tracking method showed errors that

were below 1.3° except for when the angle was at either extreme value (0° and 180°

(see Table 3.2)). Compared to the first uncalibrated angle tracking method, the second

uncalibrated angle tracking method is insensitive to rotation about the x-axis. This

feature could be advantageous in FSM control of upper limb FES if, for example, the

94

wish is to avoid triggering as a result of pronation-supination of the forearm. A shared

limitation for both the first and second uncalibrated angle tracking methods is that

they can only provide positive angle values.

The calibrated angle tracking method can provide both the magnitude and the sign of

the angle change. The disadvantage is that calibration movements must be undertaken

after donning the Xsens sensor to define both the required plane of rotation and the

positive rotation direction. Significant errors were observed when Xsens angle change

from vertical is less than or greater than (see Figures 3.9 and 3.10).

Finally, an auto-calibration algorithm has been developed that can update the gains

applied to the three Xsens signals (x, y and z components), in real time, to compensate

for calibration errors (see Figures 3.12-3.15). The algorithm is expected to operate in

parallel with any of the angle tracking methods described in this chapter.

95

Chapter 4 – Robust angle triggering algorithms

4.1 Introduction

When one of the angle tracking algorithms described previously (referring to section

3.3) is incorporated into a FSM controller for the purpose of triggering state

transitions, a number of methods can be included in the transition triggering algorithm

to improve robustness and hence the usability of the system. The aim of such methods

is to reduce the number of incorrect transition timings caused by signal noise, jerky

arm movements and other negative effects, which lead to poor control of FES during

reaching tasks. This is most likely to cause the reaching task to fail when early

triggering occurs as the change in arm-segment angle may be insufficient to allow the

next movement phase to commence successfully. For example, if a transition between

‘forearm lift’ and ‘reach forward’ phases occurs too early, then the forearm may not

have lifted far enough to clear an obstacle such as a table.

The following methods have been implemented:

Using the change in angle since entering a state rather than absolute angle;

Ignoring readings where the acceleration vector is significant in comparison to


different from 9.81);

Requiring a given number of consecutive or non-consecutive valid readings


These have been implemented with the second uncalibrated angle tracking method

and incorporated into a state-machine controller (Figure 4.1) for demonstration

purposes.

96

Figure 4.1: Setup of state machine for test purposes

In the following sections, the details of the methods listed above are described, the

experimental test protocol is described, the results are discussed, and then conclusions

drawn.

4.2 Algorithms to improve robustness of angle triggering

4.2.1 Using the change in angle since entering a state

Because of kinematic variability from one execution of an upper limb task to the next,

particularly with stroke impaired subjects, it is likely that using fixed angles from the

vertical as triggers for state transitions may not be ideal. Furthermore, the acceleration

sensor may not be perfectly aligned with the long axis of its body segment, and also

its alignment may change during a therapy session as a result of the soft attachment

used (e.g. an elastic bandage slipping). Therefore, it is hypothesised that using the

change in angle since entering a state is likely to be more robust.

The second uncalibrated angle tracking algorithm calculates the absolute angle of the

Xsens x-axis from the vertical. Therefore, referring to Figure 4.2, the angle when first

entering a state ( 𝑎 ) must be captured, so that the change since entering the state

( 𝑐 𝑎 ) can be monitored. Then the trigger for leaving that state could

be, for example, when 𝑐 or 𝑐 , where is the trigger angle.

State 1 State 2 State 3

State 4

𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑

𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑

97

Figure 4.2: Change in angle since entering state 𝑐 𝑎

Figure 4.3 shows the change in angle 𝑐 as the state machine controller in Figure 4.1

moves from one state to the next. When a state transition occurs, a new value of 𝑎

is captured and the change in angle returns to zero. Figure 4.4 shows how the state

changes as transitions (angle triggers) occur.

Figure 4.3: Change in angle since entering a state

Figure 4.4: State number and timing of state transitions

Ang

le (

degre

e)

Time (sec)

6.5 s

4.6 s

19.9 s

17.95 s

15.8 s

11.3 s

8.4 s 32.65 s

Sta

te n

um

ber

Time (sec)

𝑥𝑛𝑜𝑤

𝜃𝑐

𝜃𝑠𝑡𝑎𝑟𝑡

𝜃𝑛𝑜𝑤

𝑥𝑠𝑡𝑎𝑟𝑡

𝑔𝑛𝑜𝑤 (Vertical)

98

4.2.2 Ignoring readings where the acceleration vector is significant

In this method, accelerometer readings are ignored if the acceleration vector is

significant in comparison to the gravity vector (i.e. if the magnitude of the measured

vector is significantly different from 9.81). This is done because the measured vector

is the sum of the acceleration vector and the gravity vector and, therefore, it can only

be relied on for calculating angle if the acceleration is small in comparison to gravity.

This also has the effect of ignoring readings when signal noise has resulted in the

vector magnitude being significantly different from 9.81.

Specifically, the following logic is applied:

If (9.81 – tolerance <| |<9.81+tolerance) then

flag = 0 (acceleration is ok)

Else

flag = 1 (acceleration too high)

End

If (flag = 1) then the accelerometer reading is ignored for state transition purposes

Figures 4.5 and 4.6 show the magnitude of the measured vector and the corresponding

value of the flag that indicates whether the reading is valid (flag=0) or invalid

(flag=1). The best tolerance has still to be determined but for demonstration purposes

a g-tolerance of has been used.

99

Figure 4.5: Magnitude of measured g-vector. The solid red lines are the lower limit

(9.31) and upper limit (10.31).

Figure 4.6: Acceleration flag value (Flag=0 means within g-tolerance)


Some form of signal smoothing or averaging is often used to reduce the effects of

signal noise and, in this case, jerky arm movements. In the case of state transition

triggering, a simple way to achieve this is to require that the trigger condition is

satisfied for a specified number of consecutive readings, which is equivalent to

satisfying the condition for a specified time. Figure 4.7 illustrates this approach where

the transition condition is that the angle must exceed the threshold for the specified

number of consecutive readings.

Accele

ratio

n (

m/s

2)

Time (sec)

Fla

g v

alu

e (

0 o

r 1)

Time (sec)

100

Figure 4.7: Transition condition must be satisfied for six consecutive readings

By combining this approach with the rejection of readings that don’t satisfy the g-

tolerance requirement, it is likely that improved noise rejection can be achieved.

Figure 4.8 illustrates the method’s operation when readings are also rejected because

they don’t satisfy the g-tolerance requirement (Flag=1).

Figure 4.8: Six consecutive valid readings that satisfy the g-tolerance trigger the

transition. The readings marked by the red circles are either not satisfying the

0.25 s (6 good samples)

Transition timing with

one valid reading

Timing with 6

consecutive valid

readings

Threshold

Angle

Time


one valid reading

Timing with 6

consecutive valid

readings

0.25 s (6 good samples)

Flag = 1

Below threshold

Threshold

Time

Angle

101

transition condition (angle below threshold) or failing to meet the g-tolerance

requirement.

Finally, an alternative approach is to accept a specified number of non-consecutive

valid readings as illustrated in Figure 4.9. This may be of benefit if requiring

consecutive valid readings turns out to be too conservative, possibly introducing

unacceptable delays in triggering. The readings marked by the red circles are either

not satisfying the transition condition (angle below threshold) or failing to meet the g-

tolerance requirement.

Figure 4.9: Six non-consecutive valid readings trigger the transition. The readings

marked by the red circles are either not satisfying the transition condition (angle

below threshold) or failing to meet the g-tolerance requirement.

Figures 4.10, 4.11 and 4.12 show some example test results for a trigger algorithm

that requires 6 consecutive valid readings and uses g-tolerance rejection of bad

readings.

Flag = 1


one valid reading

Timing with 6 non-

consecutive valid

readings

0.35 s (6 good samples,

2 bad samples)

Below threshold

Time

Angle

102

Figure 4.10: Acceleration flag value (flag=0 means within the range g ± g-tolerance)

Figure 4.11: Six consecutive valid readings to trigger a transition. Bad readings are

rejected

Transition

Good point Bad point

𝑇ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑

Time (sec)

Ang

le (

degre

es)

-11°

-14°

-17°

-20°

Fla

g v

alu

e (

0 o

r 1)

Time (sec)

Point that indicates the flag value

103

Figure 4.12: Count of consecutive valid points

4.3 Experimental protocol

4.3.1 Subject

One stroke patient participated in this study (details in Table 4.1). Only one subject

was used because the experiments were only intended to provide an initial indication

of the performance of the robust angle triggering algorithms.

Subject Gender Age Hemiplegic side Dominant side Years

since onset

1 M 81 Left Right 3 years

Table 4.1: Profile of subject

Num

ber

of co

nsecutive v

alid

po

ints

Time (sec)

Point that indicates the number of consecutive points

104

4.3.2 Instrumentation

Figure 4.13: Xsens inertial sensing units with clusters of four reflective markers

attached

Two Xsens inertial sensing units, each with a cluster of four reflective markers on

their upper corners (see Figure 4.13), were attached to the upper arm and forearm of

the subject’s affected arm using self-adherent bandage. The Xsens units were

approximately aligned with the long axes of the body segments (see Figure 4.14). A

Vicon motion analysis system (Vicon Motion Systems Ltd, Los Angeles, USA)

employing ten cameras was used to capture the positions of the reflective markers on

each Xsens unit at a sampling frequency of 100 Hz. Only acceleration data was

captured from the Xsens units, at a sampling frequency of 20 Hz, using a separate

laptop; which also ran the FSM controller that produced the necessary stimulation

profiles via a Hazomed 8-channel stimulator (Hasomed GmbH, Magdeburg,

Germany), and also ran the GUI used to set up the FSM controller. A pulse signal

from one of the Xsens analog output channels was fed to an analogue input channel in

the Vicon system to provide synchronization between the Xsens and Vicon systems.

105

Figure 4.14: Experimental setup showing Xsens units with reflective marker clusters

4.3.3 Experimental procedure

The subject was asked to practise two functional tasks, which were “Brush coins into

the other hand” and “Drink from a cup”. For both tasks, the subject sat at a table with

their affected hand comfortably placed on the table at the starting position. The

therapist gave instructions to the subject to guide them in achieving the required

movements during each FSM state (movement phase) and manually triggered the

transitions between the states via the laptop keyboard. Ten trials of each task were

recorded using both Vicon and Xsens systems to capture motion data. Descriptions of

the two functional tasks are as follows:

1) “Brush coins into the other hand”

The subject was required to reach for some coins and brush them back into his other

hand (see Figure 4.15 for the associated FSM). The position of the coins was not too

far from the subject so that he could achieve the task with only FES assistance. The

affected hand and the coins were placed at the same starting positions before each

repetition of the task.

106

Figure 4.15: FSM controller for the “Brush coins into other hand” task. The

transitions between states were manually triggered by the therapist

2) “Drink from a cup”

The subject was required to reach for a cup, grasp it, lift the cup to the mouth, replace

the cup and release it (see Figure 4.16 for the associated FSM). The position of the

cup was not too far away from the subject so that he could achieve the task with only

FES assistance. The affected hand and cup were placed at the same starting positions

before each repetition of the task.

Figure 4.16: FSM controller for the “Drink from a cup” task. The transitions between

states were manually triggered by the therapist

4.3.4 Data processing

The absolute angles of the two Xsens x-axes from the vertical were recorded directly

by the real-time FSM controller as it incorporated the second uncalibrated angle

tracking method (see section 3.3).

Vicon position data (coordinates) for the reflective markers attached on the Xsens

units were exported using Visual 3D software (C-Motion, Inc., Rockville, MD, USA).

A bespoke MATLAB program was used to process the marker coordinates. The

Neutral Reach for coins Brush back

those coins

Button Button

Button

Button

Neutral Reach for cup Grasp and lift

the cup

Replace the cup Release the cup

Button

Button Button Button

107

Vicon marker data were down-sampled to provide data at 20Hz and synchronized

with the Xsens data.

Figure 4.17: Vicon graphic showing marker clusters on the upper arm and forearm

respectively

Referring to Figure 4.17, the x-axes of the Xsens units ( and ) were

approximately aligned with the long axes of the upper arm and forearm respectively.

A cluster of four markers ( ) were used for tracking the movement

of A1 (the upper arm unit), while the other four markers ( ) were

used for tracking the movement of A2 (the forearm unit). Each cluster of markers was

used to derive a vector that corresponds to the associated Xsens x-axis. Hence, both

the Xsens and Vicon derived x-axis vectors were approximately aligned with the long

axes of their body segments and more exactly aligned with each other. The Vicon

derived vectors are as follows:

A1

𝑼𝑨𝟏

𝑿 𝑼𝒑𝒑𝒆𝒓𝒂𝒓𝒎

𝑿 𝑭𝒐𝒓𝒆𝒂𝒓𝒎

A2

𝑼𝑨𝟑

𝑼𝑨𝟐

𝑼𝑨𝟒

𝑳𝑨𝟏

𝑳𝑨𝟑 𝑳𝑨𝟐

𝑳𝑨𝟒

108

𝑎

(

)

(

)

[

]

(4.1)

𝑎

(

)

(

)

[

]

(4.2)

Where ( ) are the position coordinates of markers on the upper

arm

( ) are the position coordinates of markers on the forearm

and indicates the marker number

where is the number of Vicon frames captured during a

reaching task

Thus, the absolute angle of each body segment from the vertical can be calculated by

using the Vicon derived vectors 𝑎 and 𝑎 and the second uncalibrated

angle tracking method. These Vicon derived angles were treated as the gold standard

for comparison with the Xsens (accelerometer) derived angles.

4.4 Results

Figure 4.18 shows the absolute angle of the upper arm from the vertical during the

“Brush coins into the other hand” task, obtained from both Vicon and Xsens systems.

Figure 4.19 shows the same information for the forearm. Considering only the low

frequency content, the angles from the vertical obtained from the two systems are a

109

close match to each other. However, the Xsens derived angles are subject to more

high frequency noise. In this context, it should be noted that there was no additional

filter applied to either Vicon data or Xsens acceleration signals. The slight differences

in the low frequency content are probably a result of small misalignments between the

Xsens and Vicon derived x-axis vectors.

Figure 4.18: Example data for upper arm angle from the vertical, obtained from both

Vicon and Xsens systems, for the “Brush coins into other hand” task

0

10

20

30

40

50

60

Ab

solu

te a

ngl

e fr

om

ver

tica

l (d

egre

e)

Time (sec)

Vicon

Xsens

110

Figure 4.19: Example data for forearm angle from the vertical, obtained from both

Vicon and Xsens systems, for the “Brush coins into other hand” task

4.4.1 Change in angle since entering a state vs. absolute angle

One of the reasons for using “change in angle since entering a state” was to

compensate for misalignment between the sensor x-axis and the long axis of the body

segment. However, in this experimental protocol, the Vicon derived x-axis was used

to calculate the gold standard angles rather than the true long axis of the associated

body segment. Because only very small misalignments were expected between the

Xsens and Vicon derived x-axes, an artificial misalignment of 10° has been

introduced to mimic a more realistic scenario where the sensor has only been

approximately aligned with the long axis of the body segment.

0

20

40

60

80

100

120

Ab

solu

te a

ngl

e fr

om

ver

tica

l (d

egre

e)

Time (sec)

Vicon

Xsens

111

Figure 4.20: Artificial misalignment of the Xsens unit relative to the Vicon gold

standard: (a) The Xsens unit and Vicon gold standard are aligned; (b) the artificial

Xsens and Vicon gold standard are misaligned

Referring to Figure 4.20, the Xsens acceleration data was modified to mimic a

misalignment of about the unit’s z-axis by applying a rotation matrix as

follows

[

] [

] [

] (4.3)

Note that a fixed misalignment error (10° in this case) does not lead to a simple

constant offset in the angle data because of the nonlinear nature of both the rotation

operation and the second uncalibrated angle tracking method. For example, referring

to Figure 4.21, a 10° misalignment of the upper arm Xsens unit relative to the Vicon

gold standard leads to varying angle errors as the reaching task progresses.

Figures 4.21 and 4.22 show the “absolute angle from the vertical” and the “change in

angle since entering a state” for the same trial of the “Brush coins into the other hand”

task. It is quite clear that the errors between the Xsens and Vicon gold standard are

X axis

Y axis

Z axis

Rotate about z axis by

X axis

Y axis

Z axis

(a) (b)

112

reduced by using “change in angle since entering a state”. To illustrate this more

explicitly, Figures 4.23 and 4.24 show the errors for the two cases where these are

defined as follows:

𝑎𝑏 𝑙 | – |

𝑐 𝑎 | ℎ – ℎ |

Figure 4.21: Example data for upper arm “absolute angle from the vertical” during the

“Brush coins into other hand” task, obtained from the Vicon gold standard and the

artificially misaligned Xsens data

0

10

20

30

40

50

60

70

Ab

solu

te a

ngl

e f

rom

ve

rtic

al (

de

gre

es)

Time (Sec)

abs Vicon

abs xsens

113


during the “Brush coins into other hand” task, obtained from the Vicon gold standard

and the artificially misaligned Xsens data

Figure 4.23: Error in “absolute angle from the vertical” for the example trial of the

“Brush coins into other hand” task

-40

-30

-20

-10

0

10

20

30

40

50

Ab

solu

te a

ngl

e fr

om

ver

tica

l (d

egre

es)

Time (sec)

change vicon

change xsens

0

2

4

6

8

10

12

14

16

Erro

r Ab

solu

te (

de

gre

e)

Time (sec)

114

Figure 4.24: Error in “change in angle since entering a state” for the example trial of

the “Brush coins into other hand” task

Figure 4.25 shows the mean errors and their standard deviations for both body

segments over 10 trials of the “Brush coins into other hand” task and 8 trials of the

“Drink from a cup” task. This demonstrates that using “change in angle since entering

a state” provides a significant advantage in terms of being robust to sensor

misalignment. It is also self-evident that it increases robustness to kinematic

variability by resetting the “change in angle” to zero at the start of each state

(movement phase).

0

2

4

6

8

10

12

14

16

Erro

r ch

ange

(d

egre

e)

TIme (sec)

115

Figure 4.25: Mean error for both arm segments for the “Brush coins into other hand”

task (n=10) and the “Drink from a cup” task (n=8). These are the means of n trial-

means and the error bars indicate standard deviation over the n trials

4.2.2 Ignoring readings where the acceleration vector is significant

The measured vector obtained from a 3-axis accelerometer (in this case an Xsens unit)

is the sum of the acceleration vector and the gravity vector and, hence, it can only be

relied on for calculating angle if the acceleration is small in comparison to gravity.

Therefore, the measured vectors obtained from the Xsens units are ignored if the

magnitude of the vector falls outside the range g g-tolerance. This also has the

effect of ignoring readings when signal noise has resulted in the vector magnitude

being significantly different from 9.81. In this test, different settings for the g-

tolerance of , and have been used.

Figure 4.26 shows the error between the absolute angle obtained from the Vicon and

Xsens systems before removal of bad readings, whereas Figures 4.27 and 4.28 show

the same information after removal of bad readings using g-tolerances of

and respectively. In this case ‘error’ is defined as |

|. It can be seen that, in Figures 4.26 and 4.27, some unwanted spikes

have been removed (marked by the red circles) because they caused the g-tolerance to

be violated.

0

1

2

3

4

5

6

7

8

9

10

Brush coins intoother hand- Upper

arm

Brush coins intoother hand-

Forearm

Drink from a cup -Upper arm

Drink from a cup -Forearm

An

gle

(deg

ree)

Absolute angle

Change in angle

116


of the “Drink from a cup” task before removal of bad readings


of the “Drink from a cup” task after removal of bad readings using a g-tolerance of

02468

1012141618

0.0

5

1.0

5

2.0

5

3.0

5

4.0

5

5.0

5

6.0

5

7.0

5

8.0

5

9.0

5

10

.05

11

.05

12

.05

13

.05

14

.05

15

.05

16

.05

17

.05

18

.05

19

.05

20

.05

21

.05

22

.05

Erro

r (d

egre

e)

Time (sec)

0

2

4

6

8

10

12

14

16

18

0.0

5

1.0

5

2.0

5

3.0

5

4.0

5

5.0

5

6.0

5

7.0

5

8.0

5

9.0

5

10

.05

11

.05

12

.05

13

.05

14

.05

15

.05

16

.05

17

.05

18

.05

19

.05

20

.05

21

.05

22

.05

Erro

r (d

egr

ee

)

Time (sec)

117


of the “Drink from a cup” task after removal of bad readings using a g-tolerance

of

Figure 4.29 shows that, after removal of bad readings, the maximum errors have

reduced for both tasks. The maximum error decreases with smaller g-tolerances

because more unwanted spikes are removed. This demonstrates that “ignoring

readings where the true acceleration is significant in comparison to gravity” can

remove some unwanted spikes and thereby improve the robustness of angle triggering.

However, referring to Figures 4.26-4.28, it is clear that not all spikes are removed.

This is discussed further in the conclusions to this chapter (section 4.5).

0

2

4

6

8

10

12

14

16

18

0.0

5

1.0

5

2.0

5

3.0

5

4.0

5

5.0

5

6.0

5

7.0

5

8.0

5

9.0

5

10

.05

11

.05

12

.05

13

.05

14

.05

15

.05

16

.05

17

.05

18

.05

19

.05

20

.05

21

.05

22

.05

Erro

r (d

eree

)

Time (Sec)

118

Figure 4.29: Mean maximum error for both arm segments for the “Brush coins into

other hand” task (n=10) and the “Drink from a cup” task (n=8). These are the means

of n trial-maximums and the error bars indicate standard deviation over the n trials


Some form of signal smoothing or averaging is required to reduce the effects of signal

noise and, in this case, jerky arm movements. In the case of state transition triggering,

a simple way to achieve this is to require that the trigger condition is satisfied for a

specified number of consecutive or non-consecutive valid readings. In this section a

comparison is made between requiring 1, 2, 4, and 6 consecutive valid readings, and

also 2, 4, and 6 non-consecutive valid readings to trigger a transition. For this purpose

one transition from each task was selected as follows:

“Brush coins into other hand” – transition between ‘reach for coins’ and

‘brush back coins’ – triggered by upper arm angle.

“Drink from a cup” – transition between ‘grasp and lift cup’ and ‘replace cup’

– triggered by forearm angle.

0

2

4

6

8

10

12

14

16

18

Brush coins into otherhand- Upper arm

Brush coins into otherhand- Forearm

Drink from a cup-Upper arm

Drink from a cup-forearm

Max error before removal of bad readings Max error (g-tolerance = 0.5 m/s^2)

Max error (g-tolerance = 0.3 m/s^2) Max error (g-tolerance = 0.2 m/s^2)

119

Before comparisons could be made between the alternative methods, angle thresholds

had to be established that ensured that the transitions would be triggered in the

majority of cases when using the angle data collected in the repeated trials. In other

words, for an increasing angle, a threshold is required that is sufficiently low to ensure

that the transition will usually occur. This was achieved by gradually decreasing the

angle threshold from the value that corresponded to the therapist’s manual trigger.

This was done because the therapist always waited until the patient appeared to have

reached as far as they could, which meant that the corresponding angle threshold was

invariably too high (see Figure 4.30). This does not mean that the therapist was wrong,

simply that they applied a more sophisticated transition condition (i.e. “wait until the

patient has reached as far as they can”), which cannot be represented by a simple

angle threshold. Referring to Figures 4.31 and 4.32, using three randomly selected

training trials and starting from the therapist defined angle threshold (i.e. the mean of

the manual trigger angles in the 3 trials), the threshold was reduced in 1° steps until

the transition always occurred. In this process, only triggers requiring 6 consecutive

and 6 non-consecutive valid readings were applied because this ensures that the

transitions will also occur when fewer valid readings are required. This led to suitable

angle thresholds of 25° for the “Brush coins into other hand” task and 35° for the

“Drink from a cup” task.

120


during the “Brush coins into other hand” task. The suitable angle threshold for the

“Brush coins into other hand” task is 25°. The transition timing determined by

therapist is the 3.45 second, and the corresponding angle threshold is 32° (1 good

reading over the angle threshold to trigger).

Figure 4.31: Choosing a suitable angle threshold for the “Brush coins into other hand”

task by gradually decreasing the angle threshold in 1° steps

-45-40-35-30-25-20-15-10

-505

101520253035

0.05 0.55 1.05 1.55 2.05 2.55 3.05 3.55 4.05 4.55 5.05 5.55 6.05 6.55 7.05

An

gle

(deg

ree)

Time (sec)

angle threshold

change xsens

0

2

4

6

8

10

12

25° 26° 27° 28° 29° 30° 31° 32° 33°

Nu

mb

er

of

succ

ess

ful t

rigg

ers

Angle (degree)

6 non-consecutive

6 consecutive

121

Figure 4.32: Choosing a suitable angle threshold for the “Drink from a cup” task by

gradually decreasing the angle threshold in 1° steps

Having found suitable threshold angles, the alternative methods were compared using

two measures: one to assess the risk of early triggering; and one to assess the delay in

triggering. Early triggering is the most likely cause of task failure because the change

in arm-segment angle may be insufficient to allow the next movement phase to

commence successfully. Here early triggering is defined to have occurred when the

Xsens angle is being used to trigger the transition and, for an increasing angle, the

“true trigger angle” is less than the “angle threshold”, where the “true trigger angle”

is obtained from the Vicon system.

0

2

4

6

8

10

35° 36° 37° 38° 39° 40° 41° 42°

Nu

mb

er

of

succ

ess

ful t

rigg

ers

Angle (degree)

6 non-consecutive

6 consecutive

122

Table 4.2 shows the number of early triggers that occurred when the data from all of

the trials were used as input to the alternative methods.

Brush coins into other

hand

(10 trials)

Drink from a cup

(8 trials)

1 good reading to trigger 1 1

2 non-consecutive readings to

trigger

1 1

2 consecutive readings to trigger 1 1


trigger

0 1



trigger

0 0


Table 4.2: Number of early triggers when using the data from 10 repeated trials for

the “Brush coins into other hand” task and from 8 repeated trials for the “Drink from a

cup” task

Although using 6 consecutive or 6 non-consecutive points reduces the risk of early

triggers, this introduces delays in triggering, which are shown in Table 4.3 for the

same set of trials that were used to quantify early triggers. As would be expected, this

shows that triggering is most delayed when six consecutive valid readings are

required to trigger a transition, and least delayed when one valid reading is required.

Brush coins into other hand Drink from a cup

Angle delay

(degree)

Time delay

(sec)

Angle delay

(degree)

Time delay

timing (sec)

1 good reading to

trigger 1.38 0.045 1.14 0.025

2 non-consecutive

readings to trigger 1.84 0.170 0.85 0.144

123

2 consecutive


4 non-consecutive


4 consecutive


6 non-consecutive


6 consecutive


Table 4.3: Mean delays in triggering for the three methods over all trials. Delay is

quantified in terms of both angle (true trigger angle – threshold angle) and time

(trigger time – time when trigger should have occurred)

4.5 Conclusions

Three methods have been demonstrated for improving the robustness of angle

triggering and, hence, the usability of the FES control system; which were as follows:

Using the change in angle since entering a state rather than absolute angle;

Ignoring readings where the acceleration vector is significant in comparison to


different from 9.81);



These were implemented with the second uncalibrated angle tracking method and

incorporated into a state-machine controller (Figure 4.1) for demonstration purposes.

Using the “change in angle since entering a state” as a trigger for state transitions

(rather than absolute angle from the vertical) provides a significant advantage in terms

of being robust to sensor misalignment (Figure 4.25). It is also self-evident that it

increases robustness to kinematic variability by resetting the “change in angle” to zero

at the start of each state (movement phase).

124

Ignoring readings where “the true acceleration is significant in comparison to gravity”

can remove some unwanted spikes and thereby improve the robustness of angle

triggering (Figure 4.29). However, referring to Figures 4.26-4.28, it is clear that not

all spikes are removed. This is because only those spikes that alter the magnitude of

the measured vector (acceleration+gravity) are interpreted as bad readings. Referring

to Figure 4.33, it can be seen that a significant acceleration in addition to gravity (or

the equivalent noise from some other source) may not always mean that the

magnitude of the measured vector falls outside the g-tolerance.

Figure 4.33: The measured accelerometer vector ( ) is the sum of the true

acceleration vector ( 𝑎𝑐𝑐) and gravity ( 𝑎 ). Although 𝑎𝑐𝑐 is significant in

comparison to 𝑎 , the g-tolerance is still satisfied.

Although requiring a given number of consecutive or non-consecutive valid readings

before triggering a transition reduces the risk of early triggers (Table 4.2), this

introduces delays in triggering (Table 4.3). As would be expected, this shows that

triggering is most delayed when six consecutive valid readings are required to trigger

a transition, and least delayed when one valid reading is required.

The main limitation of this study is that a comprehensive search for the best

combination of the methods described above is still required, including the number of

valid readings required and the g-tolerance range. The alternatives should be

compared in terms of their robustness (i.e. avoidance of inappropriate state-transitions)

𝑔𝑔𝑟𝑎𝑣𝑖𝑡𝑦

𝑔𝑛𝑜𝑤

𝑔𝑎𝑐𝑐

125

and the time delay they introduce. Another related limitation is the need to establish

the most appropriate measure of overall robustness.

126

Chapter 5 – Finite state machine controller for upper limb FES

5.1 Introduction

In this chapter, a flexible FSM controller and the associated setup software is

presented, for control of electrical stimulation to support upper limb functional task

practice. In order to achieve varied functional task practice across a range of patients,

the user should be able to set up a variety of different state machines, corresponding

to different functional tasks, tailored to the individual patient. The goal of the work is

to design a FSM controller and produce an interface that clinicians (even potentially

patients) can use to design and set up their own task and patient-specific FSMs.

The following sections cover the functionality of the flexible FSM controller,

implementation and testing of the controller, and finally the design of a GUI for

controller setup.

5.2 Functionality of the FSM controller

A FSM controller is usually composed of a set of states, input signals, output

functions, and state transition conditions (Chu, 2006; Sweeney et al., 2000). In this

particular case, each “state” corresponds to one movement phase and the state’s

“output functions” implement the ramping of muscle stimulations towards their

respective targets (note the target may be zero) and then holding them at those targets.

The set of possible “input signals” for the FSM controller are button status, clock

time and angle data for different body segments (e.g. upper arm, forearm) via

accelerometer units attached to them. The “state transition conditions” implement the

conditions for exiting each movement phase.

Figure 5.1 illustrates the general form of the flexible FSM controller, consisting of

sequential movement phases (solid rectangles) and the corresponding transitions

between each phase (solid arrows). The first phase in the FSM is termed the neutral

127

phase, which is always associated with no muscle stimulation. The total number of

phases for the FSM controller is flexible and defined by the user (minimum 2 phases),

depending on the chosen task. The FSM returns to the neutral phase every time on

exiting the last phase. Thus, a functional task will always begin and end in the neutral

phase. The dashed arrows represent exceptional transitions (i.e. emergency stop or

default timeout), allowing return to the neutral phase from any phase. The exceptional

transitions have a higher priority than the normal transitions between successive

movement phases.

Figure 5.1: General form of the FSM controller. The labelled rectangles represent

sequential states and the solid arrows represent normal state transitions. The FSM

comprises a minimum of two states (states 1 and 2). The number of states (n) is

defined by the user (see section 5.5). The dashed arrows from any state to state 1

represent exceptional transitions (emergency stop or default timeout)

The state transition conditions determine the timing for transition between phases, and

are described in terms of the input signals and the current state. The number of states,

state transition conditions (angle triggers, timeouts, combinational logic etc.), and

stimulation parameters for each state (stimulation thresholds, targets, ramps etc) are

defined using the GUI (see section 5.5).

To illustrate the way in which the FSM controller can be set up for a specific FES task,

an example (“open a door”) is discussed below. Referring to figure 5.2, this FSM has

five movement phases; “neutral”, “reach for door”, “grasp handle”, “open door”

and “release door”. Each movement phase output function contains a set of muscles

to be stimulated and their associated stimulation parameters. For example, in phase 5,

to release the door handle, stimulation is applied to the Forearm extensor muscles (see

State 1 State 2 State 3 State 4

State n State 5

128

5.2.1 for details). Transitions between phases are instantaneous events that occur on

satisfaction of the transition condition. In the example, the transition between phase 4

(open door) and phase 5 (release door) will be triggered either by the angle of the

lower arm decreasing by 45° (since entering that phase – see 5.2.2 for details) or the

time period in phase 4 exceeding 5 seconds. This example is further expanded upon

later in the chapter and is used to illustrate the implementation of the various elements

of the FSM controller.

Figure 5.2: Example FSM controller for “open a door”. The text below each transition

arrow is the transition condition for exiting that phase. The muscles

stimulated in each phase are listed in the corresponding box and are

defined below:

Anterior deltoid and Triceps – (AD & Tr)

Forearm extensors – FE

Posterior deltoid – PD

Forearm flexors – FF

Transition condition

for exiting phase:

Timeout – 4 sec


for exiting phase:

Angle change – Upper

arm – increase by 53°


for exiting phase:

Button press


for exiting phase:

Timeout – 4 sec


for exiting phase:

Angle change – Lower

arm – decrease by 45°

OR

Timeout – 5 sec

Neutral phase

(Phase 1)

Reach for door

(Phase 2)

(AD & Tr) + FE

Grasp handle

(Phase 3)

(AD & Tr) + FF

Open door

(Phase 4)

PD + FF

Release door

(Phase 5)

FE

129

5.2.1 Movement phases and stimulation control

In each phase the associated set of muscles are stimulated to achieve the required

movement. The stimulation targets are the stimulation levels that produce sufficient

muscle force to achieve the expected movement in a phase. As the force required

from a particular muscle will vary across the task, stimulation targets for a particular

muscle are likely to vary with phase (see Figure 5.3). If muscles are not already at the

required stimulation target, they are ramped up or down to reach that target (which

can be zero). To achieve different movements in different phases, some muscles will

continue to be stimulated but their stimulation target will be changed (see Figure 5.3),

whereas stimulation of other muscles will be started or stopped (see Figure 5.4). Like

the stimulation targets, the ramp rates may also be changed to achieve different

movements in different phases.

Figure 5.3: An example of stimulation target changing with phase

The FSM controller also allows for stimulation to jump to a pre-defined threshold

before ramping up. Similarly, when stimulation is stopped, stimulation can jump

down to zero after ramping down to a threshold (see Figure 5.4). In this

implementation, sensory threshold is used (i.e. the lowest pulse width, at predefined

pulse amplitude, needed to elicit a sensory response). Stimulation below the threshold

will not lead to any movement or sensation. Each muscle will have its own

stimulation threshold that does not change with phase (see figure 5.4). In cases where

the target is lower than the stimulation threshold, the threshold is treated as the target.

Pulse width (µs)

Time (s)

Target (i)

Ramp time

Phase (i) Phase (i+1)

Target (i+1)

130

Figure 5.4: Jumping up to a threshold (before ramping up) and jumping down from a

threshold (after ramping down)

Ramp time is another user-defined FES parameter describing the time period over

which simulation ramps from its previous target to its new target. The ramp rate is

determined from ramp time and two consecutive nodes in the stimulation profile (i.e.

either threshold and target or two consecutive targets, see figure 5.3 and 5.4).

Obviously, for a given difference in stimulation levels a smaller ramp time means a

higher ramp rate.

The implementation of ramps depends on the frequency of the FSM. In this case, the

decision was made to use 20Hz and therefore, the minimum time step is 0.05 sec. The

reason for adopting 20Hz was to avoid the user noticing any latency. The following

rules are used to calculate ramp rates from the stimulation parameters set by the

therapist (ramp times, thresholds and targets). This is done in the setup GUI described

in section 5.5 and the calculated ramp rates are then passed to the FSM controller.

1. Zero any targets that are below the thresholds

If any targets are less than or equal to the stimulation threshold for their channel,

they are set to zero.

Pulse width (µs)

Time (s)

Threshold (j)

Target (i+1)

for muscle (j)

Ramp time Ramp time

Phase (i) Phase (i+1) Phase (i+2)

Threshold (k)

Target (i) for

muscle (k) Target (i+2)

for muscle (k)

131

( ℎ ℎ ) ℎ

2. Phase 2

Note that stimulation must be zero in Phase 1 (Neutral) before moving to Phase 2.

Ramping up from threshold

( ℎ ℎ ) ℎ

( ℎ ℎℎ

⁄ )

Note that 20Hz is the state-machine frequency, not the stimulation frequency.

3. Phase 3 to the last phase

ℎ

Inherit ramp

( ( ) ( ) ) ( ) ℎ

ℎ ℎ

Ramping down to threshold

( ( ) ℎ ℎ ) ( ( )

ℎ ℎ )

( ( ) ℎ ℎ

⁄ )

Ramping up from threshold

( ( ) ℎ ℎ ) ( ( )

ℎ ℎ )

( ( ) ℎ ℎ

⁄ )

Ramping between targets

( ( ) ℎ ℎ ) ( ( )

ℎ ℎ )

132

( ( ) ( )

⁄ )

4. Phase 1 (Neutral)

Inherit ramp

( ) ( )

ℎ ℎ

Ramping down to threshold

( ( ℎ ) ℎ ℎ )

( ( ℎ ) ℎ ℎ

⁄ )

5. Apply upper and lower limits to ramp rates

( ) ℎ

( ) ℎ

The logic above is designed to avoid rapid (and hence potentially painful) changes in

stimulation levels. So ramp rates are inherited from the previous phase when the

target has not changed ( ) or when the user

appears to have mistakenly entered a very short ramp time ( ). In

the first case, the user may enter a ramp time of zero because consecutive targets are

the same. In both cases, it is assumed that a very rapid ramp was not intended.

Inherited ramps ensure that the system will still work properly when there is an early

transition to the next phase before stimulation has reached the target for the current

phase (this is discussed in more detail in section 5.2.3).

Although the descriptions above have assumed pulse width is the variable used to

modulate stimulation level (and pulse amplitude is fixed), the implementation allows

133

pulse amplitude to be modulated (and hence pulse width fixed). The “fixed pulse

parameter” flag indicates which pulse parameter is fixed as follows:

1. If “fixed pulse parameter” flag = 1, then the pulse amplitude will be treated as

the fixed pulse parameter.

2. If “fixed pulse parameter” flag = 2, then the pulse width will be treated as the

fixed pulse parameter.

The Hasomed stimulator generates pulses with width that can be varied from 20 µsec

to 500 µsec in 1 µsec steps (i.e. 480 steps), while the pulse amplitude can be varied

from 0 mA to 126 mA in 2 mA steps (i.e. 63 steps). It is clear that the pulse width

resolution is far better than that of pulse amplitude and, therefore, the amplitude was

always treated as the fixed pulse parameter and the value was set at 30 mA.

Table 5.1 gives detailed information of the muscles stimulated and their stimulation

changes in each phase for the “open a door” example (referring back to Figure 5.2).

Phase Output

Neutral No stimulation is applied

Reach for door Both (AD & Tr) and FE ramp from threshold to target and are then

held at their respective targets.

Grasp handle (AD & Tr) ramps to a new target. FF ramps from threshold to

target. Both channels are held at their targets. FE turns off by

ramping to threshold and then jumping down to zero.

Open door FF ramps to a new target. PD ramps from threshold to target. Both

channels are held at their targets. (AD & Tr) turns off by ramping

to threshold and then jumping down to zero.

Release door FE ramps from threshold to target and is held at its target. Both PD

and FF turn off by ramping to threshold and then jumping down to

zero.

Table 5.1: Stimulation details for the FSM shown in Figure 5.2, including the outputs

in each phase

134

Figures 5.5 to 5.8 show the stimulation profiles for each muscle over all movement

phases for the example task of “open a door” (see Figure 5.2). In each phase, each

muscle is ramped to its target level for that phase.

135

Figure 5.5: Stimulation profile for Anterior Deltoid and Triceps during all phases of

the example task

Figure 5.6: Stimulation profile for Forearm Extensors during all phases of the

example task

Figure 5.7: Stimulation profile for Posterior Deltoid during all phases of the example

task

Figure 5.8: Stimulation profile for Forearm Flexors during all phases of the example

task

Stim target

in phase 2

Stim target in phase

4 & 5 <= threshold

Stim

target in

phase 3

Phase 1 Phase 2 Phase 3 Phase 4 Phase 5 Phase 1

Stim target in

phase 1 is always 0

(<= threshold)

Stim target

in phase 2

Stim target in phase 3

<= threshold

Stim target

in phase 5

Stim target in

phase 1 is always

0 (<= threshold)


Stim target

in phase 4

Stim target in phase 5 <= threshold


Stim target

in phase 3

Stim target in phase 5 <= threshold

Stim target

in phase 4


136

To illustrate the implementation described above, one example stimulation profile

(Figure 5.5 – Anterior Deltoid and Triceps) will be described in detail. In phase 1

(neutral), there is no stimulation. During phase 2 (reach for door), stimulation to

Anterior Deltoid and Triceps jumps up from zero to threshold before ramping to the

stimulation target. The aim in this phase is to assist the movement of the affected arm

to reach forward. Stimulation is held at the target level until the transition condition is

true. In the next phase (grasp handle), the target changes, so the stimulation is ramped

down to the new target and held at the new level. Stimulation during phase 3 is

designed to assist the affected arm to maintain its configuration (i.e. extended to the

door handle). In phase 4 (open door), stimulation ramps down to threshold, and

immediately jumps down from threshold to zero. During Phase 5 it remains at zero, as

there is no need to stimulate Anterior Deltoid and Triceps during the last two phases.

On exiting phase 5, the FSM returns to the neutral phase.

5.2.2 Transitions

Transitions between phases depend on input signals and the transition conditions for

leaving the current phase. The FSM controller, as implemented, can take signals from

up to four accelerometers for tracking the movements of the upper limb (i.e. hand,

lower arm, upper arm and torso). In this case, the accelerometer provides the x, y and

z components of the measured vector (acceleration+gravity) in the accelerometer

reference frame. The acceleration data are streamed into the FSM controller in real

time during a functional task. The second uncalibrated angle tracking method (section

3.3), incorporated into the FSM controller, takes as its input the three signals from a

given accelerometer and outputs the absolute angle of that accelerometer’s x-axis

from the vertical, which can be used to trigger a transition (see Chapter 4). Apart from

segment angle, transition conditions can also use button press and timeout functions.

To extend the flexibility of the system, logical operators (N/A, AND or OR) can be

used to combine a maximum of two Boolean conditions (condition A and condition B)

to create a transition rule. Using N/A as the logical operator means that only one

condition needs to be specified (always condition A).

137

Table 5.2 lists all of the Boolean conditions for the transitions between phases for the

“open a door” example task described in Figure 5.2. For this task, two accelerometers

are employed for tracking the movements of the upper arm and lower arm in order to

trigger transitions between phase 2 and 3, and phase 4 and 5. Other transition

conditions for triggering transitions include button pressing and timeout. In this

example, the logical operator OR is used for the transition between phase 4 and 5,

with the transition occurring either if the angle of the upper arm decreases by 45° or if

the FSM remains in phase 4 for over 5 seconds. Note that a transition between phases

not only depends on transition conditions, but also on the current phase. For example,

pressing the space bar will only trigger the transition into phase 2 when the state

machine is in phase 1.

Transition Logical operator Condition A Condition B

Transition between phase 1

and 2

N/A Button pressing

(space bar)

Disabled


and 3

N/A Upper arm has

increased by 53°

Disabled


and 4

N/A Wait for 4 sec Disabled


and 5

OR Upper arm has

decreased by 45°

Wait for 5 sec


and 1

N/A Wait for 4 sec Disabled

Table 5.2: Transition rules for the example task “open a door” (Figure 5.2)

5.2.3 Dealing with early transitions

An “early transition” is defined as being a transition between movement phases that is

triggered before the stimulation reaches its target. For example, referring to Figure 5.9,

a FSM controller has been set up such that one stimulation channel should ramp

towards target (i+1) for phase (i+1) with a ramp rate (i+1), and target (i+2) for phase

(i+2) with a ramp rate (i+2). If transition (i+1) occurs before the stimulation reaches

target (i+1), then this is an early transition. Early transitions are likely to occur during

upper limb rehabilitation tasks because, for example, patients may improve with

138

practise so that they achieve a required arm movement before the stimulation target is

reached. Ideally this should lead to adjustments in stimulation targets and/or transition

conditions, but these may not occur until a therapist is available.

When early transition occurs, the stimulation targets and ramps will be changed to

those associated with the new phase. There are two different situations for early

transitions, which depend on the previous and new targets. They are:

1. Previous target and new target are different (see Figure 5.9).

2. Previous target and new target are the same (see Figure 5.10).

When the targets associated with the previous phase and the new phase are different,

the controller should simply use the targets and ramp rates associated with the new

phase. This is the case because the therapist will have explicitly defined these using

the setup GUI described in section 5.5.

However, when the previous target and the new target are the same for a particular

channel, the therapist will not have explicitly defined a ramp rate because they will

not be expecting the stimulation to ramp at all. In fact, if they do specify a ramp time,

it is likely to be zero implying an immediate step change in stimulation. Therefore, for

that channel, the new phase should inherit the ramp rate associated with the previous

phase (see Figure 5.10); so that the stimulation continues to ramp at the same rate

until the target is reached.

The rules for the calculation of ramp rates (including inheriting ramp rates) have

already been presented in section 5.2.1.

139

Figure 5.9: An early transition (transition (i+1)) where target (i+1) and target (i+2) are

different. Ramp rate (i+2) is based on the specified ramp time and the difference

between target (i+1) and target (i+2), irrespective of the fact that target (i+1) was not

reached

Figure 5.10: An early transition (transition (i+1)) where target (i+1) and target (i+2)

are the same. Ramp rate (i+2) is inherited from phase (i+1)

5.3 Implementation of the FSM controller

Matlab/Simulink was used to implement the real-time FSM controller under the

Windows XP Professional platform. Simulink allows on-line data acquisition, data

Pulse width (µs)

Time (s)

Target (i)


Target (i+2)

Phase (i+2)

Transition (i) Transition (i+1)

Target (i+1)

Ramp rate (i+1)

Ramp rate (i+2)

Pulse width (µs)

Time (s)

Target (i)


Target

(i+1)/ (i+2)

Phase (i+2)

Transition (i) Transition (i+1)

Ramp rate (i+1)

Ramp rate (i+2)

140

processing and control of stimulation parameters in real-time. Figure 5.11 shows an

overview of the inputs and outputs of the FES control system. The real-time inputs to

the FSM controller in Simulink include three axis accelerations, button pressing

signals, and clock time for timeouts. The real-time outputs are stimulation pulse width

(µsec), pulse amplitude (mA) and the waveform. Note that the waveform is fixed and

pre-set in the Simulink model, and clinicians have no authority to change this. The

Simulink system runs at 20Hz and implements angle tracking, robust angle triggering,

the FSM controller, and safety checking.

141

Figure 5.11: Real-time inputs and outputs for the FES control system

Referring to figure 5.12, the absolute angles of the Xsens units’ x-axes from the

vertical, calculated using the angle tracking method described in section 3.3, are

streamed into the FSM controller. The other real-time inputs to the FSM controller are

the transition button status (‘move phase’ button on GUI or ‘space bar’ on keyboard),

emergency button status (‘stop’ button on GUI or ‘Enter’ on keyboard), and clock

time for timeouts.

The FSM controller includes: state transition control (sections 5.2.2 and 5.2.3);

methods to improve the robustness of angle triggering (chapter 4); and stimulation

output control. Stimulation output control simply involves stepping each channel

towards its current target at the associated ramp rate (or stepping up to /down from the

threshold for that channel).

Button pressing:

(i) transition (space bar)

(ii) emergency stop (enter) Xsens motion trackers

Hazomed stimulator

Angle tracking method,

angle triggering algorithm,

FSM controller and safety

check implementation in

Simulink

Clock

142

The real-time outputs from the FSM controller (pulse widths and pulse amplitudes)

are streamed into the safety block, which sits between the controller and the Hasomed

stimulator. The purpose of the safety block is to avoid pain due to inappropriate

stimulation levels or rates. The safety block limits pulse width, pulse amplitude, and

total charge in a single pulse, as well as maximum step size for ramping. If the

demanded step size exceeds the pre-defined maximum step size, then it is limited to

the maximum step size. If any other limits are exceeded, then the safety block stops

stimulation. Safety checking is applied to every stimulation channel.

Figure 5.12: Upper limb FES control system flow chart, including FSM controller

Data acquisition

Acceleration data

input Clock time

Angle tracking

Buttons (transition

& emergency stop)

Outputs to Hasomed

stimulator

FSM controller

Safety check

143

The Simulink implementation of the FES control system is shown in figure 5.13. It

includes seven main blocks: Xsens communication; acceleration to angle conversion;

FSM; stimulation control; safety block; Hazomed stimulator interface; and real-time

synchronisation block. Other blocks have secondary roles such as displaying data

(scopes). The explanations of each block are given in the following subsections. Apart

from the Hasomed stimulator interface and real-time synchronisation (Hasomed

GmbH, Magdeburg, Germany), the other five main blocks were implemented by the

author using embedded Matlab functions.

144

Figure 5.13: A complete upper limb FES control system (including FSM controller) implemented using Simulink

145

1. Xsens communication

The Xsens communication block collects real-time data from accelerometers and

passes the acceleration data to the acceleration to angle conversion block. It accesses

accelerometers every 0.05 sec, the same frequency as the entire Simulink system, to

record real-time acceleration data. The real-time outputs of this block are the

acceleration data from different Xsens inertial sensing units, streamed into the next

main block. Each Xsens unit provides three signals (x, y and z components).

2. Acceleration to angle conversion block

The second uncalibrated angle tracking method (section 3.3) is incorporated into the

acceleration to angle conversion block. Thus, this block calculates the absolute angles

of the Xsens units’ x-axes (usually aligned with the long-axes of the body segments)

from vertical in real time from the raw acceleration data. In addition, in this block, a

method of ignoring readings where the true acceleration is significant compared to

gravity is implemented (see section 4.2.2), which sets the acceleration flags that are

used to indicate whether the readings are valid or not. An acceleration flag is true (an

invalid reading) if the acceleration vector’s magnitude is significantly different from

9.81 m/s2.

Thus the real-time inputs to this block are the x, y and z accelerometer signals from

each Xsens unit and the real-time outputs are the acceleration flags and absolute

angles.

3. FSM block

This is the main block that includes the generic structure of the flexible FSM (see

Figures 5.1), which can be set up to achieve different functional task (for example

Figure 5.2). It decides the appropriate times for transitions between phases. Thus, this

block requires data defining the transition conditions (see Table 5.3), which is

obtained from the therapist via the GUI described in section 5.5.

146

Parameter Type Number

Logical operator (AND, OR, N/A) Integer (flag) 1 per phase

Condition A (angle change, button or timeout) Integer (flag) 1 per phase

Condition B (angle change, button or timeout) Integer (flag) 1 per phase

Angle change direction (increase or decrease) Integer (flag) As needed

Body segment used for angle change (hand,

forearm, upper arm)

Integer (flag) As needed

Angle change Float/Integer As needed

Timeout period Float/Integer As needed

Notes:

Condition B is not required if the logical operator is N/A.

The last 4 items are only needed where that condition (angle change or

timeout) has been specified for A or B. e.g. the last item is needed if a timeout

has been specified.

Table 5.3: Transition conditions for leaving each phase

Apart from the FSM structure and corresponding transition conditions, the following

methods for improving robustness of angle triggering are also implemented in this

block. For more details refer to sections 4.2.1 and 4.2.3.

Using change in angle since entering a state, rather than absolute angle.


for triggering a transition (a simple form of signal noise filtering).

Note that in this block, the real-time inputs are absolute angles and acceleration flags

from the previous block, emergency stop button status, clock time, and transition

button status. The emergency stop button has the highest priority and forces the FSM

to go back to phase 1 (neutral).

The transition button status is set by the clinician (by clicking the GUI ‘move’ button

or pressing the space bar) and is continuously monitored while the FES control system

is running. The clock time is provided by a digital clock block from the Simulink

library. The digital clock block outputs simulation time for the Simulink model. The

real-time output of this block is the current movement phase (state) number for the

FSM.

147

4. Stimulation control block

This block controls pulse width (PW) and pulse amplitude (PA) during each phase

and for each muscle, based on pre-defined ramp rates and thresholds. The

RehaStimTM

stimulator is used to generate stimulation pulses. The “fixed pulse

parameter” flag indicates which pulse parameter remains constant (PA), the other

parameter (PW) being used to create the varying stimulation profiles. The default

value for the fixed pulse parameter (PA) is 30 mA. Tables 5.4 and 5.5 list the various

stimulation parameters, which are obtained from the user via the GUI described in

section 5.5.


Fixed pulse parameter flag (pulse amplitude) Integer (flag) 1 per channel

Fixed pulse parameter value Float/Integer 1 per channel

Stimulation threshold (pulse width ) Float/Integer 1 per channel

Table 5.4: Stimulation settings for each channel that do not vary with phase


Stimulation target (pulse width) Float/Integer 1 per phase for each

channel

Ramp rate (pulse width) Float/Integer 1 per phase for each

channel

Table 5.5: Stimulation settings for each channel that differ between phases

The real-time inputs to this block are the phase number and emergency stop button

status. The emergency stop button has the highest priority and forces pulse width to

ramp down to zero for all channels. The real-time outputs are pulse widths and pulse

amplitudes of all channels.

5. Safety block

The safety block is responsible for checking the pulse widths and pulse amplitudes

that will be sent to the Hasomed communication block, and will cut stimulation off if

any value exceeds previously defined limits. For all stimulation channels, there are

three hard limits (pulse amplitude, pulse width and total charge) and also a soft limit

for total charge. Exceeding any of them will cause stimulation to ramp down at a

148

default rate. The term ‘hard’ indicates that the limits are pre-set by the programmer in

the safety block, and clinicians have no authority to access them. The term ‘soft’

means that clinicians can access and change these limits. The soft limit for each

channel is given by:

Thus, once the max stimulation for comfort for a channel has been set up by a

clinician, the soft limit will be updated and passed to the safety block. Note that, the

maximum stimulation for comfort was intended to be the stimulation level at which

functional movement is achieved and the patient reports it becoming uncomfortable.

However, in practise, this invariably prevented adequate stimulation from being

applied. Therefore, it was multiplied by a factor of 1.25, which was determined by

trial and error. There were no reports of pain from the patients when applying the

factor of 1.25 to provide a soft limit, when the system was tested in local hospitals.

Exceeding the limit on step size for the ramp will not cut off the stimulation. However,

the safety block limits the max step size to 6 µsec/step (120 µsec/second). Thus, if the

auto-calculated step size for a ramp exceeds this value, the safety block will

automatically reduce it to the max step size.

The real-time inputs are the pulse widths and pulse amplitudes for all channels, and

the outputs are also the pulse widths and pulse amplitudes, subject to the limits

described above.

6. Stimulator interface (Hasomed communication block)

The Hasomed communication block is responsible for accessing the RehaStim

stimulator through a serial port USB. It adopts the ScienceMode protocol to directly

control the 8-channel RehaStimTM

stimulator. This block is implemented using a

Matlab s-function written in C++ and Simulink masks. It was created by Hasomed

GmbH. The real-time inputs for this block are the safe pulse widths and pulse

amplitudes for all channels. The pulse waveform for all stimulation channels is

‘singlet stimulation pulse mode’ (a single stimulation pulse is repeatedly generated

and sent out on a specific channel with the desired pulse amplitude, pulse width and

149

frequency). The pulse waveform is fixed and pre-set in the Simulink model. Note that

the stimulation frequency does not have to be the same as the FSM frequency (20Hz);

the latter being selected to avoid users noticing any latency.

7. Real-time synchronisation block

The real-time synchronisation block is an s-function block that ensures a Simulink

execution frequency of ~20 Hz. It does this by synchronizing the Simulink FES

control system with the computer’s real-time clock.

5.4 Testing the FSM controller

The FSM controller was tested using the “open a door” task (see Figure 5.2). The

outputs monitored included:

Accelerometer signals from Xsens units on the upper arm and forearm;

Change in angle from the vertical since entering a state (movement phase);

Phase number;

Pulse width for each muscle.

5.4.1 Initialisation prior to testing

Before running the FSM controller, the Xsens Motion Tracking software needs to be

installed (Xsens technologies B.V., Netherlands, version 2.8.1), which provides a

solution for directly accessing the Xsens MTx communications hub from Matlab.

After installation, Matlab can communicate with the Xsens MTx hub through the

serial port and collect real-time acceleration data from the Xsens inertial sensing units

that are connected to the MTx hub. The Xsens system was set up to sample the real-

time accelerometer signals at a frequency of 100 Hz even though the FSM controller

only attempts to upload data at 20 Hz, which is thought to be sufficiently high to

prevent users noticing any latency. This was done to avoid the FSM controller

missing or double reading any Xsens data.

150

The parameters which define the example task (see Figure 5.2) were set up using the

GUI described in section 5.5. The number of phases, the muscles involved in each

phase, and the transition conditions are all shown in figure 5.2. The stimulation

parameters are given in Table 5.6. The “stimulation threshold” and “maximum

stimulation for comfort” were set to their default values, which are 0 µs for 360 µs

respectively.

Phase 1 Phase 2 Phase 3 Phase 4 Phase 5

Channel 1 0 108 108 0 0

Channel 2 0 54 0 0 72

Channel 3 0 0 72 72 0

Channel 4 0 0 0 90 0

(a) Stimulation targets (µsec) for each channel and each phase.

Phase 1 Phase 2 Phase 3 Phase 4 Phase 5

Channel 1 1 1 1 1 1

Channel 2 1 1 1 1 1

Channel 3 1 1 1 1 1

Channel 4 1 1 1 1 1

(b) Ramp time (sec) for each channel and each phase.

Table 5.6: Stimulation parameters for each channel and each phase

5.4.2 Test results for the “open a door” task

Data was collected from a healthy subject undertaking the “open a door” task. The

data was captured under real-time conditions and the dashed lines in the following

figures indicate the transitions between the phases. To enable angle-triggering, two

Xsens units were located on the upper arm and the forearm respectively. The

corresponding raw accelerometer data are shown in Figure 5.14 for one repetition of

the task.

151

(a) Upper arm

(b) Forearm

Figure 5.14: Acceleration profiles (x in blue, y in green, z in red). The dashed lines

indicate transitions between phases

Using the second uncalibrated angle tracking method, the acceleration data from the

two Xsens units were transformed into “change in angle since entering the phase” of



Acceleration (m/s2)

Acceleration (m/s2)

152

the upper arm and lower arm respectively (see Figure 5.15). The change in angle

returns to zero after each transition between phases (see values at dashed lines in

figure 5.15).

Figure 5.15: “Change in angle since entering the phase” during the “Open a door

task”. Upper arm in blue and lower arm in green. The dashed lines indicate transitions

between phases. The horizontal solid lines are the angle thresholds for transition 2

(increase by 53°) and transition 4 (decrease by 45°)

Figure 5.16 shows the phase number increasing, from 1 to 5, as the reaching task

progresses as shown in Figure 5.2. The phase numbers were output in real time.


Angle (degree)

153

Figure 5.16: Phase number as output by the FSM controller

Figure 5.17 shows the stimulation pulse width outputs to the Hasomed stimulator for

channels 1-4. On entering a new phase, the stimulation pulse widths ramp towards the

new targets at rates based on 1 second ramp times (Table 5.6).

Figure 5.17: Stimulation pulse width outputs (Anterior Deltoid & Triceps in dark blue;

Forearm Extensors in green; Posterior Deltoid in red; Forearm Flexors in light blue)


Phase number

Time (sec)

Stimulation pulse width (µsec)

154

5.5 Graphical user interface for setup of FSM controller

A GUI has been developed to allow users to set up an FSM controller and the

associated stimulation parameters for upper limb functional tasks. The design of the

GUI’s appearance and the specification of its functionality have been undertaken by

the research team as a whole, including Christine Smith who is studying for a PhD on

the usability aspects of setting up FES assisted upper limb functional tasks. The

author’s main role has been the software implementation of the GUI. Clinicians will

set up the flexible FSM controller for different patients and different functional tasks

by using the GUI, thus it needs to be sufficiently flexible to support the setup of a

range of different FSMs (i.e. different number of phases, different muscles stimulated

during each phase, different transition conditions etc.).

Current studies of less complex approaches to FES therapy demonstrate its value for

improving upper limb motor recovery after stroke (Langhorne et al., 2009; Popović,

Sinkjærc, & Popović, 2009). However, for more complex functional tasks, the GUI

should allow users to set up FSMs involving multiple muscles and multiple movement

phases. However, in many cases, clinicians have no programming skills to create a

new or modify an existing FSM controller. Therefore, the setup process should be

straightforward and user-friendly.

5.5.1 Breaking the setup process into logical stages

The GUI concept is to break the setup of a FSM for a particular upper limb functional

task into the four logical stages shown in Figure 5.18. To explain why the four stages

were chosen, the “open a door” example shown in Figure 5.2 will be used.

In Stage 1, the clinician would usually select a task from a pre-defined library of

standard reaching tasks. However, if this were a new task not in the library, then the

user would define the number and sequence of phases, and then enter the muscles

associated with each phase (e.g. forearm extensors in phase 5). In this case, there are

five phases but this may differ between patients depending on their impairment.

155

In Stage 2, the user sets up each of the stimulation channels for the previously

specified muscles, including donning stimulation electrodes, assigning channels to the

corresponding muscles, testing muscle response, and adjusting channel specific

settings (sensory threshold and max stimulation for comfort). The acceleration sensors

are also donned and assigned in stage 2 (e.g. Xsens sensing unit 1 – upper arm).

In Stage 3, the stimulation parameters for each movement phase are set up (i.e.

stimulation targets and ramp times for each channel in each phase). These stimulation

parameters can be finely tuned by manually moving through the phases of (using the

space bar to transition) until acceptably smooth task execution is achieved. During

good task executions, the change in angle since entering the phase (for each sensor)

and the time spent in each phase is logged to inform Stage 4.

In Stage 4, the user sets up the automatic transition conditions for moving between

movement phases. These include a logical operator (N/A, OR, AND) so that the

transition condition can take one of the following three forms: A; (A OR B); (A AND

B). The logical conditions (A and/or B) are one of the following: button press; timeout;

angle change.

After going through these four setup stages, the therapist can enter stage 5 (the

therapy session manager) and allow the patient to repeat the functional task. The setup

GUI stages are described in more detail in the following sections.

156

Figure 5.18: Flow chart of GUI based flexible FSM setup

5.5.2 Stage 1 – Create, modify and select tasks

During stage 1, users are guided to create, modify and import functional tasks for

either existing or new patients, as follows:

Create a data structure to store functional tasks for new patients. Note that any

modification to the tasks during stages 1-4 will update the data structure

associated with that patient.

Create a new functional task for either existing or new patients, includes

defining the number of phases, adding or removing muscles to be stimulated

during each phase.

Modify an existing functional task for either existing or new patients,

including changing the number of phases, adding or removing any phase of the

FSM, and adding or removing any muscles to be stimulated during each phase.

Import a functional task from a standard hand-arm task library or from other

existing patients.

Stage 1: Create, modify and select

tasks

Stage 5: Practising functional

tasks

Stage 2: Don electrodes and

sensors, and initial channel setup

Stage 3: Set up stimulation

parameters for each phase and

capture manual transitions data

Stage 4: Set up transition

conditions

157

In stage 1, the user can create a data structure for a new patient or load a data structure

for an existing patient. The data structure stores the FSM controllers for each

functional task and associated stimulation parameters. A functional task created under

a different data structure can easily be transferred to another. Every data structure is

given a name, which can be either the patient’s ID number or name. When a data

structure has been created or loaded for a patient, the user can then modify it.

Users define the phase number and task name for the FSM, give a name to each phase,

add or remove any muscles to be stimulated during each phase for creation of a new

functional task for a patient. Drop-down menus are used to define a phase number and

the muscles to be stimulated in each phase. A tick box for each phase allows

modification to each phase that has been ticked simultaneously, which is believed to

save time. For example, ticking phase 2 and phase 5 in the “open a door” task allows

adding of forearm extensors to them at the same time.

Modification of any functional task is carried out in a very similar way to creating a

new task. Users can change the number of phases and task name for the FSM, change

phase names, add or remove phases, and add or remove muscles in each phase.

Importing a functional task from either a standard hand-arm task library or from

existing patients may be helpful to accelerate setup. Import of a functional task is

implemented by a pop-up panel which shows the standard hand-arm task library and

data structures for other patients. The standard hand-arm task library, implemented in

the software, provides standard functional tasks with pre-defined numbers of phases

and associated muscles.

158

A flow chart of the stage 1 process is shown in figure 5.19.

Figure 5.19: Flow chart showing the process followed in stage 1

5.5.3 Stage 2 – Don electrodes and sensors, and initial channel setup

Stage 2 guides the user through the donning and initial setup of the hardware, as

follows:

Setup of stimulator channels, which includes donning stimulation electrodes,

assigning channels to the corresponding muscles, testing for better placement

of each electrode for muscle stimulation, and adjusting channel specific

settings (sensory threshold and max stimulation for comfort).

Edit task (add/remove a

phase, muscles to be

stimulated during each phase,

etc)

Create new patient Load an existing patient

Import a task from

task library or other

patients

Create a new task

(define number of

phases, muscles to be

stimulated during each

phase, etc)

Select tasks for today’s

practice

Need to add

another task?

No

Yes

159

Setup of acceleration motion sensors, which includes donning sensors and

assigning sensors to the corresponding upper limb segments.

a) Donning and initial setup of stimulator channels

A maximum of 8 stimulation channels can be assigned. The stimulation channels are

manually assigned, via a list box, to the set of muscles which will be stimulated in the

task.

The other stimulation parameters that are defined for each muscle are pulse amplitude

(the fixed pulse parameter), stimulation threshold (minimum stimulation level that can

be felt) and maximum comfortable stimulation. Stimulation can lead to pain

(Hendricks, IJzerman, de Kroon, in 't Groen, & Zilvold, 2001; Ring & Rosenthal,

2005), and this may be lead to the patient withdrawing from the treatment (Chae et al.,

1998). Thus, the maximum comfortable stimulation for each stimulated muscle is

multiplied by a factor of 1.25 to provide a soft safety limit (see safety block

description in section 5.3). The stimulation threshold and maximum comfortable

stimulation for each muscle are found by ramping up stimulation (pulse width) via

manually adjusting a slider from no stimulation to maximum stimulation (level at

which functional movement is achieved and the patient reports it becoming

uncomfortable).

b) Donning and initial setup of sensors

After setting up the channels, the Xsens unit(s) are placed on the appropriate limb

segment (e.g. upper arm). The GUI displays a plot of the Xsens unit data in real time

to guide the user during initial setup.

In a process similar to the setup of stimulator channels, the user assigns the Xsens

units to different body segments to match the actual location of units on the patient.

Another list box shows the final assignments of the Xsens units.

160

5.5.4 Stage 3 – Set up stimulation parameters for each phase and capture manual

transitions data

In stage 3, the user is guided through the setting up of stimulation parameters for each

muscle in each phase and, for good repeats of the task, sensor and time data is

recorded to characterise the manually triggered transitions. The process is as follows:

Setup of stimulation parameters (stimulation target and ramp time) for each

muscle in each phase, allowing a task to be achieved by manually triggering

transitions using a button press.

Defining a successfully achieved functional task as a good trial. The user will

decide whether the task was achieved successfully.

For those good trials, manually triggered transition data will be captured,

specifically change in angle of each instrumented body segment since entering

the phase and time spent in each phase.

There is no stimulation in the neutral phase. In each subsequent phase, the user is

required to manually adjust pulse width using a slider from zero to a stimulation target,

defined by the clinician observing the relevant limb motion. After the stimulation

targets for each muscle have been found, and maintaining stimulation at these levels,

the patient should be able to achieve the required arm movement for that phase. In

addition, the ramp time for each muscle is also defined at this stage by simply typing a

number in the ramp time edit box for each phase. Ramp time defines either ramp up to

target or ramp down to zero in a phase. The above processes are repeated for each

phase until stimulation parameters for all muscles in each phase have been set up

successfully.

Towards the end of this stage, patients are expected to achieve some good FES

assisted trials. Transitions are achieved by the clinician manually triggering the

transition between phases using a button press. Successful FES-assisted trials are

labelled by the clinician. At the end of the stage, the captured transition data from the

set of successful trials are averaged and passed as “suggested values” to stage 4 (see

below). The suggested values are calculated by averaging any corresponding values

from good trials as follows:

161

∑ 𝑎𝑙

∑ 𝑎𝑙

Where, N is the total number of good trials

is the suggested value for the time period to be used as a

timeout trigger for leaving a phase

is the suggested value for change in angle when using the

angle tracking method for triggering a transition

For example, in the “open a door” task (see Figure 5.2), for three separate good trials,

the transition data for leaving the phase “reach for door” are assumed to be upper arm

increased by 53°, 50° and 56°, while forearm increased by 13°, 9° and 11°. Therefore,

the average values of upper arm increased by 53° and forearm increased by 11° will

be passed to stage 4 as a guide for the user when setting up the angle trigger for

leaving phase 2.

5.5.5 Stage 4 – Setting up transition conditions

Stage 4 guides the user through the setup of transition conditions for automatically

leaving each phase as follows:

Based on the average angle change and duration of each movement phase

(calculated in Stage 3), it provides suggested values for use in phase transition

conditions (for angle thresholds and timeout periods).

These values are used to set up phase transition conditions for leaving each

phase.

The user can enter instructions (displayed or voice) to guide patients to

achieve the movements associated with each phase.

The transition between phases can be triggered by a button press, a timeout, a body

segment angle trigger, or some logical combination of these. Thus, the first thing the

user needs to decide is how to combine Boolean conditions using a logical operator

(AND, OR or N/A), which is selected using a drop-down menu. Therefore, transition

162

conditions can take one of the following three forms: A; (A OR B); (A AND B). If

N/A is selected then only one condition will be used (only A).

The individual Boolean conditions (A and B) are selected using drop-down menus,

which provide the following options: ‘increase angle by’, ‘decrease angle by’,

‘timeout’ and ‘button’. When an angle trigger is selected, another drop-down menu is

used to select the corresponding body segment, which can be hand, forearm, upper

arm or torso. An angle threshold edit box offers a suggested value from the average of

the good trials captured in Stage 3 (e.g. forearm increase by 15° to leave a phase). The

user can either accept or change the suggested angle threshold using the edit box. If a

‘timeout’ has been selected, an edit box offers a suggested value from the average of

the good trials captured in Stage 3 (e.g. 3 second time out). The user can either accept

or change the suggested timeout using the edit box. If a ‘button’ press has been

selected, no other information is required.

Finally, the user inputs patient instructions for each phase via an edit box for a

displayed instruction and a microphone for a voice instruction. These instructions are

used to guide patients to achieve the movements associated with each phase. For the

“open a door” example task (see Figure 5.2), an instruction to guide the patient in

phase 2 might be ‘open hand and reach for the door handle’.

5.5.6 Stage 5 – Practising functional tasks

Once the previous four setup stages have been completed, Stage 5 guides the patient

while they are practising functional tasks during a therapy session. At present this is

only a basic implementation which includes:

A simple control panel that includes two buttons, one to start/stop practice,

and one for button triggered transitions.

A display to provide instructions for guiding the patient during task practice.

A speaker to provide audio instructions for guiding the patient during task

practice.

163

5.6. Conclusions

A flexible FSM controller and an associated setup GUI have been developed that

allow clinical users (e.g. physiotherapists) to set up different FES assisted upper limb

functional tasks and the corresponding FSM controllers. The aim was to provide a

tool that allows clinicians to set up a variety of different FES assisted tasks for

different patients with different levels of impairment.

The FSM controller has been tested using the “open a door” example task. The results

provided in section 5.4 illustrate the functionality of the controller and also

demonstrate its successful implementation.

The usability of the system is heavily dependent on the GUI that has been developed

for the setup of different FES assisted tasks and the corresponding FSM controllers.

The GUI should not require specialist knowledge of control engineering and should

be easy to use; so that the assistance of a clinical engineer is not required.

Comprehensive user testing has been undertaken by another member of the research

team (Christine Smith) who is a physiotherapist with many years’ experience of

stroke rehabilitation.

164

Chapter 6 – Discussion

Around 50% of survivors from stroke lose some control of the arm and/or hand. The

disabilities for the upper limb has the remarkable negatively impact in the ADL (R. N.

Barker & Brauer, 2005). Typical upper limb impairments include a reduction in

muscle strength and the ability to extend the shoulder, elbow and hand joints, as well

as impaired coordination between limb segments. There are a range of different

interventions aimed at promoting upper limb recovery following stroke, including

robotic devices and FES. Current studies of FES therapy demonstrate its potential

value in rehabilitation of reaching and grasping function, elbow extension, shoulder

motion, and stabilization of wrist joints. However, as discussed in section 2.2.5, there

are limitations with the existing technologies.

6.1 Limitations with existing FES systems

There is good evidence supporting intensive, repetitive, task-focused (Alon, Levitt, &

McCarthy, 2007; Langhorne et al., 2009; Winstein et al., 2004), voluntary-initiated

(Jayme S. Knutson et al., 2009; P. Hunter Peckham & Knutson, 2005) FES-supported

practice for upper limb functional recovery. However, the ability to deliver this type

of therapy in clinical practice is limited by available tools (Hara, 2008; Lynch &

Popovic, 2008; P. Hunter Peckham & Knutson, 2005) (see section 2.2.5). The number

of commercially available FES systems for the upper limb is limited and most systems

providing a limited number of stimulation channels, with some systems restricted by

design to stimulation of the particular body anatomy (Alon & McBride, 2003; Hobby

et al., 2001; Dejan Popovic et al., 1999). Relatively little attention has been paid to the

development of flexible systems, which allow the user to set up their own tasks

(Rakos et al., 2007; Tresadern et al., 2008) (see table 2.1 and section 2.2.5.3).

As discussed in section 2.2.5, at the start of the PhD it was identified that a system

that allowed therapists to quickly and easily set up robust FES controllers that were

task and patient specific would address the limitations discussed above. Specifically,

165

the aim was to produce a system for guiding therapists through the setup of FES

controllers for patient-specific upper limb functional tasks, without involving a

clinical engineer. To achieve this required 1) sufficient stimulation channels and

flexibility over the set of muscles to be stimulated; 2) a robust movement sensing

configuration; and 3) a flexible FSM controller for delivering upper limb FES that is

specific to the impairment profile of the patient and task requirements.

6.2 Angle tracking methods and methods to improve robustness

In this thesis, a FSM controller has been presented that can take signals from

accelerometers tracking the movements of up to four different body segments (hand,

forearm, upper arm and torso). Accelerometer-based movement sensing methods were

used in this system because 1) accelerometers are available at low cost, in very small

packages, and with low power consumption, 2) angle relative to gravity can be

derived directly from accelerometers, therefore avoiding integration-related drift.

These properties make the sensors potentially suitable for long term use in clinical

settings (see section 2.5.1.1). In order to fully exploit the output of accelerometers

methods for estimating angle relative to gravity were needed. In chapter 2, these

methods were reviewed. Three methods were identified from the literature (see

section 2.5.2): 1) using single axis accelerometers; 2) using dual axis accelerometers;

and 3) using two accelerometers separated by a rigid link. As discussed in section

2.5.2, angle measurement using a single accelerometer (1, 2 or 3-axis) suffers from

two main problems. Firstly, the existing methods for processing the accelerometer

signals to obtain angle from the vertical all suffer from very poor sensitivity when a

sensitive axis approaches the vertical. Secondly, the methods reported rely on the true

acceleration being negligible and are, therefore, only suitable for measuring angle

under static or low acceleration conditions. Methods that use two accelerometers

separated by a rigid rod (see section 2.5.2.3) have been used to overcome the

limitation of measuring angle only under low acceleration conditions. However, this

only works for rotation around a fixed point (e.g. the knee) or to obtain angular

velocity which must then be integrated.

Three different methods were investigated for using an accelerometer to track body

segment angle and, hence, to act as a trigger for moving to the next state in an upper

166

limb FES FSM controller. The first uncalibrated method calculates the change in

angle during a rotation using the gravity vectors before and after the rotation (see

section 3.2). The second uncalibrated method (see section 3.3) calculates the angle

between the accelerometer x-axis and the gravity vector. The third calibrated method

uses a calibration rotation to define the measurement plane and the positive rotation

direction (see section 3.4). This method then calculates the component of rotation that

is in the same plane as the calibration rotation. All three methods use an algorithm

that switches between using sine and cosine, depending on the measured angle, which

overcomes the poor sensitivity problem seen in previous methods.

The second uncalibrated angle tracking method has been incorporated into a flexible

FSM controller and tested. The reason for using the second method was that, unlike

the first uncalibrated angle tracking method, this method provides a reference

orientation (i.e. the angle is zero when the x-axis is vertical). In addition, this method

is insensitive to rotation about the x-axis. This property can be usefully exploited if,

for example, the accelerometer is aligned with the x-axis oriented along the forearm’s

long axis, thereby making the output insensitive to pronation-supination. The

approach was found to be accurate, with errors below 1.3° except when the angle was

near 0° or 180°. The third, calibrated method, was shown to provide the sign of the

angle change, but significant errors were observed when the measured angle was <30°

or >150°. Referring to the techniques from the literatures (see section 2.5.2),

maximum errors of over 10° were observed for the angle derived using the arccos

function when the sensitive axis approaches vertical. (Baek & Yun, 2010; Kengo et

al., 2013). It is also self-evident that the angle derived by using the arcsin function

suffers from similar limitations. When using a dual axis accelerometer, angle can be

derived by using the arctan function. Although small errors (less than ) have been

observed for this method (Pallejà et al., 2010) it suffers from extreme sensitivity near

(Pallejà et al., 2010; Rodriguez-Donate et al., 2010).

Like other methods reported in section 2.5.2.1 (Baek & Yun, 2010; Bakhshi et al.,

2011; Bourke et al., 2011; Caroselli et al., 2013; Fabera et al., 2013; Ha et al., 2013;

Juan et al., 2013; Kengo et al., 2013; Myong-Woo Lee et al., 2010; Lugade et al.,

2014; Zhang et al., 2012) and 2.5.2.2 (Coulter et al., 2011; Grzeda & Fichtinger, 2010;

Miura et al., 2011; Pallejà et al., 2010; Qilong et al., 2013; Rodriguez-Donate et al.,

167

2010; Vinande et al., 2010; Watanabe et al., 2013), using an accelerometer to measure

angle from the vertical relies on the true acceleration being negligible and are,

therefore, only suitable for measuring angle under static or low acceleration

conditions. Furthermore, there are significant challenges to overcome to make the

triggering of state transitions, based on accelerometer derived angles, more robust

because of movement variability, sensor misalignment, the true acceleration

component, signal noise etc. Therefore, a number of methods that aim to improve the

robustness of angle triggering, and hence the usability of the system, have been

implemented based on the second uncalibrated angle tracking method. The method of

ignoring readings where “the true acceleration is significant in comparison to gravity”

provides guarantee of only measurements under static or low acceleration conditions

will be used for angle triggering. This method has been shown to remove some

unwanted spikes in the signal and thereby improve the robustness of angle triggering

(Figure 4.29). Other methods implemented include:

Using the change in angle since entering a state, rather than absolute angle;




(rather than absolute angle from the vertical) was found to provide improved

robustness to sensor misalignment (Figure 4.25). It is also self-evident that it increases

robustness to kinematic variability by resetting the “change in angle” to zero at the

start of each state (movement phase). However, while requiring a given number of

consecutive or non-consecutive valid readings before triggering a transition reduces

the risk of early triggers (Table 4.2), it also introduces delays in triggering (Table 4.3).

6.3 FSM and Setup GUI

As discussed in Chapter 2 (sections 2.3 and 2.4), current FES systems for the upper

limb are inflexible and insufficiently automated to support patient-specific and task-

specific FES-supported practice. This is because most FES systems are pre-

168

programmed to support stimulation of particular muscle groups, with limited numbers

of channels (see table 2.1 and section 2.2.5.3).

This PhD thesis has presented a real-time control system that can be setup to support

sequencing of stimulation to user-specified muscle groups, based on user-specified

transition rules, inputs to which can include signals from body-worn accelerometers.

A flexible FSM controller (see section 5.2 and 5.3) and an associated setup GUI (see

section 5.5) have been developed that allow clinical users (e.g. physiotherapists) to set

up different FES-assisted upper limb functional tasks and the corresponding FSM

controllers. The aim was to provide a tool that allows clinicians to set up a variety of

different FES assisted tasks for different patients with different levels of impairment.

Specifically, the clinicians are allowed to set up the number of phases, the state

transition conditions (angle thresholds, timeouts, combinational logic etc.), the

different muscles stimulated during each phase, and stimulation parameters for each

state (thresholds, targets, ramps etc) for the flexible FSM controller by using the setup

GUI. The flexible FSM controller has been demonstrated using the “open a door”

example task (see Figure 5.2) and the test results (see Figures 5.15-5.17) demonstrated

its successful implementation.

The GUI, designed to support setup of a range of tasks for patients with different

levels of impairment should not require specialist knowledge of control engineering

and should be easy to use, so that the assistance of a clinical engineer is not required.

The usability aspects of the GUI design and system performance evaluation in clinical

settings have been developed and evaluated by Christine Smith, a physiotherapist with

many years’ experience of stroke rehabilitation. The protocol and full results are

presented in Smith C PhD thesis, however, as a final demonstration of the

performance of the system, the results are briefly summarised below.

169

6.4 Comprehensive testing of the FES control system for upper limb

rehabilitation

The clinical evaluation part of Smith’s study consisted of two parts. During the first

part of testing (pre-deployment testing), six chronic stroke subjects were recruited to

test the system functionality. In these experiments, system set up was performed by a

member of the Salford research team, in a University laboratory. In the second part of

the evaluation to demonstrate the usability of the final system, Christine Smith

conducted a study (Smith, in preparation) in two early stroke rehabilitation settings.

The study involved therapists using the system to set up FES-supported tasks with a

range of patients Two physiotherapists and one therapy assistant from Salford Royal

Foundation Trust (SRFT) and two physiotherapists from Central Manchester

Foundation Trust (CMFT) received a half day’s training to use the system described

in this thesis. The therapists used the system to set up and practice FES-supported

functional tasks with acute stroke patients, spanning a range of different impairment

levels, four patients in SRFT and two patients in CMFT.

In total, seven different tasks, tailored to suit the impairment levels of the particular

patients were used across the two studies. The functional tasks either created from

scratch or modified for each stroke patients via using the setup GUI by the therapists.

On average 2.75 different functional tasks were used for each recruited stroke patient.

To demonstrate the performance of the system, completion rate was calculated.

Completion rate is defined as follows:

Apart from ‘opening door’ task, the completion rates were greater than 70% for all

tasks. The ‘opening door’ task had the lowest completion rate of 66.7%, although this

number should be treated with caution as the total number of repetitions was small

(only 3 attempts for this task, 2 of which were successful), and only one patient used

this task. 11 out of 12 stroke patients were asked to try the “Sweeping coins” task, a

task with only three phases believed to be easy task for the first use of the system. The

completion rate for this task was 70.3%. The ‘pushing up from chair’ (tried by 4

170

patients) and ‘picking up tray’ (tried by 5 patients) tasks achieved relatively high

completion rates, 85.2% and 84.8% respectively. The ‘answering phone’ task is

believed as the most difficult, as it requires good coordination. However, the

completion rate was 76.5% for this task in part due to therapist assistance to the

patients.

The completion rates are discussed in more detail in Smith C PhD thesis (Smith, in

preparation). In brief, the lower than expected task completion rates are partly due to

the approach used to identify angle thresholds. As discussed in section 4.2.3, initial

angle thresholds had to be established that ensured that the transitions would be

triggered in the majority of cases when using the angle data collected in the repeated

trials. This was achieved by gradually decreasing the angle threshold from the value

that corresponded to the therapist’s manual trigger. Therefore, during the first few

attempts to a task, the angle trigger was most likely to fail until an angle threshold that

is sufficiently low to ensure that the transition will usually occur was found. This

process will negatively affect the completion rate for a task in practice sessions with

small numbers of repeats, as was the case in most of the examples above.

Despite these limitations, the final testing clearly demonstrated that the system could

be set up by therapists with a range of patients, practicing a range of practical tasks.

Further details are available in Smith C PhD thesis (Smith, in preparation).

171

Chapter 7 – Conclusions

7.1 Summary of the research

The overall aim of the author’s PhD was to develop improved techniques for real-time

FSM control of upper limb FES, using multiple accelerometers for tracking upper

limb movement and triggering state transitions. This included the development of:

Alternative methods for using accelerometers to capture body segment angle

during FES upper limb rehabilitation tasks;

Algorithms to improve the robustness of angle triggering for state transitions.

A flexible FSM controller for delivering upper limb FES.

A GUI for guiding therapists through the setup of FSM controllers for upper

limb functional tasks.

Summary of Chapter 3

Three methods have been developed for angle measurement using accelerometers:

two uncalibrated angle tracking algorithms and a calibrated angle tracking algorithm.

The two uncalibrated angle tracking methods are used for converting acceleration

measurements into the sensor’s angle (or change of angle) from the vertical. The term

‘uncalibrated’ refers to the fact that the subject is not required to make any calibration

movements after the sensors have been donned.

The first uncalibrated angle tracking method calculates the change in angle from the

vertical by calculating the angle between the gravity vectors before and after the

rotation, both expressed in the sensor’s coordinate frame. This has the advantage of

not requiring careful alignment of the sensor with the anatomy. However, because

there is no particular alignment with the anatomy, it cannot distinguish between

different components of the change in angle, for example between rotation of the

172

long-axis of the forearm and pronation-supination. The second uncalibrated angle

tracking method calculates the angle of the sensor’s x-axis from the vertical and

rejects any rotation about the x-axis, which can be advantageous if the wish is to

avoid triggering a transition as a result of pronation-supination of the forearm.

However this requires the sensor’s x-axis to be aligned with the long axis of the

segment.

Both of the uncalibrated methods were accurate to with approximately 1° over most

of the measurement range, with the exceptions that the errors increased when close to

0° or 180° (but were still less than 5°). This was assumed to occur because the two

vectors (e.g. and ) used in the calculation are close to being parallel and,

therefore, any rotation out of the calibration plane produces a corresponding change in

the angle between the two vectors. In contrast, when the two vectors are perpendicular,

a rotation out of the calibration plane doesn’t alter the angle between the two vectors.

Therefore this effect will be ameliorated as the two vectors move away from being

parallel.

Unlike the previous uncalibrated methods, the calibrated angle tracking method gives

both the magnitude and the sign of the angle change in a given plane that is defined by

a calibration movement. However, large errors occurred when and

In these cases, large errors occur in , which should be 1 or very close to 1

for all angles. The problem is believed to do with the fact that as the two gravity

vectors approach being parallel ( and ), the direction of their cross

product ( ) becomes very sensitive to small errors in the gravity vectors. For these

reasons, further testing has been abandoned for the time being.

Finally, a real-time auto-calibration algorithm has been developed that updates the

gains applied to the three sensor signals (x, y and z components of acceleration) to

compensate for calibration errors. The auto-calibration method is designed to operate

in parallel with any of the angle tracking methods mentioned above. However, further

development and testing are required to confirm that it works well when all three axes

are initially out of calibration, and to test whether it can operate in real-time.

173


Robust angle triggering algorithms have been developed, which are based on ignoring

bad sensor readings resulting from signal noise, jerky arm movements and other

negative effects. The aim was to avoid incorrect FSM transition timings and hence

poor control of FES during reaching tasks. The following methods were implemented

and tested:

- Using the change in angle since entering a state rather than absolute angle,

which reduces the effects of sensor misalignments and movement variability;

- Ignoring readings where the acceleration vector is significant in comparison to


different from )

- Requiring a given number of consecutive or non-consecutive valid readings


These were implemented with the second uncalibrated angle tracking method and

incorporated into a state-machine controller for demonstration purposes.


(rather than absolute angle from the vertical) provides a significant advantage in terms

of being robust to sensor misalignment. Ignoring readings where “the true

acceleration is significant in comparison to gravity” removes some unwanted spikes

and thereby improve the robustness of angle triggering. Requiring a given number of

consecutive or non-consecutive valid readings before triggering a transition reduces

the risk of early triggers. However, this introduces delays in triggering.


A flexible FSM controller has been developed for the real-time control of FES during

upper limb rehabilitation. In addition, a Graphical User Interface (GUI) has been

developed which guides clinical users through the process of setting up new FSM

controllers for different upper limb rehabilitation tasks for different patients with

different impairments. The FSM controller has been tested using the “open a door”

174

example task. The results illustrate the functionality of the controller and also

demonstrate its successful implementation.

The usability of the system is heavily dependent on the GUI that has been developed

for the setup of different FES assisted tasks and the corresponding FSM controllers.

The GUI should not require specialist knowledge of control engineering and should

be easy to use; so that the assistance of a clinical engineer is not required.

Comprehensive user testing has been undertaken by another member of the research

team (Christine Smith) who is a physiotherapist with many years’ experience of

stroke rehabilitation.

7.2 Achievements

The main achievements of the work reported in this thesis can be summarised as

follows:

1) A flexible real-time FSM controller that can be set up for different upper limb

functional tasks for different patients with different impairments.

2) A user-friendly GUI for guiding therapists, with little or no software expertise,

through the setup of FSM controllers for different upper limb functional tasks.

3) Alternative methods for using accelerometers to track body segment angle from

the vertical.

4) A simple algorithm for overcoming the sensitivity problem that affects other

accelerometer-based angle measurement methods.

5) Methods for improving the robustness of angle triggering for state transitions.

175

7.3 Future work

Angle tracking and triggering

Although the methods developed to date have been quite successful, there is

substantial scope for improvement. In particular, problems that warrant further

investigation include: a) improved methods for aligning sensors with the anatomy

and/or calibration algorithms to compensate for misalignment; b) methods for

estimating joint angles from accelerometer data; and c) self-tuning approaches to

angle triggering that adapt to the patient’s changing performance as the therapy

session progresses.

The first two objectives are related in the sense that both require additional

information beyond that needed for simply measuring the segment angles from the

vertical. This could be obtained by using a redundant set of accelerometers (i.e. more

than one per body segment) and by making use of our knowledge of the kinematic

constraints imposed by the joints of the arm. The research challenge would then be to

understand how to use this additional information to solve the first two problems.

A self-tuning algorithm could provide a much more natural approach to angle

triggering that more closely mimics what a therapist would do. In other words, the

aim would be to trigger after it is clear that the patient has achieved as much as they

can without stimulation, but not before. This might make use of angular velocity

information (or an equivalent) as well as angle from the vertical.

However, because deriving angular velocity from the accelerometer data would

involve numerical differentiation, any noise would be amplified. Therefore, the key to

achieving this goal might be to make use of additional methods to improve the

robustness of triggering and appropriate filtering to remove the noise resulting from

numerical differentiation.

176

FES controller and setup GUI

Future improvements to the FSM controller could include improvements to the

implementation of both the stimulation profiles and the state transitions. In the first

case, the option of including delays before stimulation ramping begins could be

included to provide more flexibility and, hence, better coordination between

stimulation channels in each movement phase. Also proportional control of

stimulation could be included; in other words, allowing the stimulation profile to be a

function of some other variable rather than time (e.g. an EMG signal). With regard to

state transitions, a greater range of Boolean conditions could be provided; for example,

the requirement for an accelerometer to “remain still for a given time” before a

transition is triggered.

Another improvement would be to make the current FSM controller as generic as

possible. This would involve allowing branching/multiple paths to provide more

flexibility and, hence, more intelligent control of a task. For example, for a reaching,

grasp and release task, the patient may be able to decide whether they want to grasp or

release an object. The FSM controller for this task could have two alternative

branches when leaving the ‘reach forward’ phase (grasp an object/ release an object).

Future improvements to the setup GUI could include: a) better ergonomic design; b) a

library of pre-defined upper limb tasks, perhaps grouped according to level of

impairment; and c) a session manager for use during therapy after setup has been

completed. The first two would involve iterative development and user testing,

probably driven by the physiotherapists in the research team. A session manager

would require development of sensor based feedback of performance for both patients

and therapists. For example, in a session manager, trunk lean could be continuously

monitored during reaching tasks. The real-time biofeedback could be provided to

warn the patient when they are using trunk lean to acquire the object rather than

reaching with their arm. In this way the patient could be supported in achieving their

therapy goals (improved arm functions).

177

Appendix A

A.1 Ethics statement

179

References

Alon, Gad, Levitt, Alan F., & McCarthy, Patricia A. (2007). Functional electrical

stimulation enhancement of upper extremity functional recovery during stroke

rehabilitation: a pilot study. Neurorehabilitation and Neural Repair, 21(3),

207-215.

Alon, Gad, & McBride, Keith. (2003). Persons with C5 or C6 tetraplegia achieve

selected functional gains using a neuroprosthesis. Archives of Physical

Medicine and Rehabilitation, 84(1), 119-124.

Armagan, Onur, Tascioglu, Funda, & Oner, Cengiz. (2003). Electromyographic

biofeedback in the treatment of the hemiplegic hand: a placebo-controlled

study. American Journal of Physical Medicine & Rehabilitation, 82(11), 856-

861.

Atteya, AA. (2004). Effects of modified constraint induced therapy on upper limb

function in subacute stroke patients. Neurosciences, 9(1), 24-29.

Baek, Jonghun, & Yun, Byoung-Ju. (2010). Posture monitoring system for context

awareness in mobile computing. Instrumentation and Measurement, 59(6),

1589 - 1599

Baker, Lucinda L., Wederich, Cynthia L., Mcneal, Donald R., Newsam, Craig, &

Waters, Robert L. (2000). Neuromuscular electrical stimulation: A practical

guide (4th edition ed.): Los Amigos Research & Education Institute, Inc.

Bakhshi, S, Mahoor, M. H, & Davidson, B. S. (2011). Development of a body joint

angle measurement system using IMU sensors. Paper presented at the

Engineering in Medicine and Biology Society,EMBC, 2011 Annual

International Conference of the IEEE Boston, MA.

Barbour, Neil, & Schmidt, George. (2001). Inertial sensor technology trends. Sensors

Journal, 1(4), 332-339.

Barker, R. N., & Brauer, S. G. (2005). Upper limb recovery after stroke: the stroke

survivors' perspective. Disability & Rehabilitation 27(20), 1213-1223.

Barker, Ruth N., Brauer, Sandra G., & Carson, Richard G. (2008). Training of

reaching in stroke survivors with severe and chronic upper limb paresis using

a novel nonrobotic device: A randomized clinical trial. Stroke, 39, 1800-1807.

Berntson, Gary G., & Cacioppo, John T. (2009). Handbook of neuroscience for the

behavioral sciences (1st ed. Vol. 1): John Wiley and Sons.

180

Bogataj, Uroš, Gros, Nuša, Kljajić, Miroljub, Aćimović, Ruža, & Maležič, Matija.

(1995). The rehabilitation of gait in patients with hemiplegia: A comparison

between conventional therapy and multichannel functional electrical

stimulation therapy. Physical Therapy, 75(6), 490-502.

Bolton, David A.E., Cauraugh, James H., & Hausenblas, Heather A. (2004).

Electromyogram-triggered neuromuscular stimulation and stroke motor

recovery of arm/hand functions: A meta-analysis. Journal of the Neurological

Sciences 223(2), 121–127.

Bonato, Paolo. (2005). Advances in wearable technology and applications in physical

medicine and rehabilitation. Journal of NeuroEngineering and Rehabilitation,

2(1), 2.

Bourke, A. K, Torrent, M, Parra, X, Catala, A, & Nelson, J. (2011). Fall algorithm

development using kinematic parameters measured from simulated falls

performed in a quasi-realistic environment using accelerometry. Paper

presented at the Engineering in Medicine and Biology Society,EMBC, 2011

Annual International Conference of the IEEE Boston, MA.

Braun, Susy M., Beurskens, Anna J., Borm, Paul J., Schack, Thomas, & Wade,

Derick T. (2006). The effects of mental practice in stroke rehabilitation: A

systematic review. Archives of Physical Medicine and Rehabilitation, 87(6),

842-852.

Braz, Gustavo P., Russold, Michael, & Davis, Glen M. (2009). Functional electrical

stimulation control of standing and stepping after spinal cord injury: A review

of technical characteristics. Neuromodulation, 12(3), 180-190.

Broeks, J. G., Lankhorst, G. J., Rumping, K., & Prevo, A. J. H. (1999). The long-term

outcome of arm function after stroke: results of a follow-up study. Disability

& Rehabilitation, 21(8), 357-364.

Bruce, H, Dobkin. (2005). Rehabilitation after stroke. The New England Journal of

Medicine, 352(16), 1677-1684.

Bullock, John, Boyle, Joseph, & Wang, Michael B. (2001). Physiology (4th ed.):

Lippincott Williams & Wilkins.

Burgar, Charles G ., Lum, Peter S., Scremin, A. M. Erika, Garber, Susan L., Loos, H.

F. Machiel Van der, Kenney, Deborah, & Shor, Peggy. (2011). Robot-assisted

upper-limb therapy in acute rehabilitation setting following stroke:

Department of Veterans Affairs multisite clinical trial. Journal of

Rehabilitation Research and Development, 48(4), 445-458.

C. Long, II. (1963). An electrophysiologic splint for the hand. Archives of Physical

Medicine and Rehabilitation, 44, 499-503.

Caplan, Louis R. (2005). Stroke. New York: Demos Medical Publishing.

181

Caroselli, Alessio, Bagalà, Fabio, & Cappello, Angelo. (2013). Quasi-real time

estimation of angular kinematics using single-axis accelerometers. Sensors,

13(1), 918-937.

Cech, P, Dlouhy, J, Cizek, M, Vicha, I, & Rozman, J. (2009). Software for automatic

head position monitoring. Paper presented at the Radioelektronika, 2009. 19th

International Conference Bratislava

Chadwick, E K, Blana, D, Simeral, J D, Lambrecht, J, Kim, S P, Cornwell, A S, . . .

Kirsch, R F. (2011). Continuous neuronal ensemble control of simulated arm

reaching by a human with tetraplegia. Journal of Neural Engineering, 8(3).

Chae, John, Bethoux, Francois, Bohinc, Theresa, Dobos, Loreen, Davis, Tina, &

Friedl, Amy. (1998). Neuromuscular stimulation for upper extremity motor

and functional recovery in acute hemiplegia. Stroke, 29, 975-979.

Chae, John, Sheffler, Lynne, & Knutson, Jayme. (2008). Neuromuscular electrical

stimulation for motor restoration in hemiplegia. Stroke Rehabilitation, 15(5),

412-415.

Chapman, Bethany, & Fratianni, Mary. (2008). Kaplan national physical therapy

exam (1st ed.): Kaplan Publishing.

Chu, Pong P. (2006). RTL hardware design using VHDL: coding for efficiency,

portability, and scalability: John Wiley and Sons.

Cole, J D, & Sedgwick, E M. (1992). The perceptions of force and of movement in a

man without large myelinated sensory afferents below the neck. The Journal

of Physiology, 449, 503-515.

Comer, David J. (1995). Digital logic and state machine design (3rd ed.): Saunder

College Publishing.

Constandinou, T. G, & Georgiou, J. (2008). Micropower arcsine circuit for tilt

processing. Electronics Letters, 44(23), 1336-1338.

Coulter, Elaine H, Dall, Philippa M., Rochester, Lynn, Hasler, Jon P., & Granat,

Malcolm H. (2011). Development and validation of a physical activity monitor

for use on a wheelchair. Spinal Cord 49(3), 445-450.

Crago, P. E., Memberg, W. D., Usey, M. K., Keith, M. W., Kirsch, R. F., Chapman, G.

J., . . . Perreault, E. J. (1998). An elbow extension neuroprosthesis for

individuals with tetraplegia. Rehabilitation Engineering, 6(1), 1-6.

Crago, Patrick E., Lan, Ning, Veltink, Peter H., Abbas, James J., & Kantor, Carole.

(1996). New control strategies for neuroprosthetic systems. Journal of

Rehabilitation Research and Development, 33(2), 158-172.

182

Crook, S. E, & Chappell, P. H. (1998). A portable system for closed loop control of

the paralysed hand using functional electrical stimulation. Medical

Engineering & Physics, 20(1), 70-76.

Crook, S. E., & Chappell, P. H. (1998). A portable system for closed loop control of

the paralysed hand using functional electrical stimulation. Medical

Engineering and Physics, 20(1), 70-76.

Crow, J.L., Lincoln, N. B., Nouri, F. M., & Weerdt, W. De. (1989). The effectiveness

of EMG biofeedback in the treatment of arm function after stroke.

International Disability Studies 11(4), 155-160.

Cuesta-Vargas, Antonio I, Galán-Mercant, Alejandro, & Williams, Jonathan M.

(2010). The use of inertial sensors system for human motion analysis. Physical

Therapy Reviews, 15(6), 462-473.

Davoodi, Rahman, & Andrews, Brian J. (2003). Switching curve control of functional

electrical stimulation assisted rowing exercise in paraplegia. Medical and

Biological Engineering and Computing 41(2), 183-189.

Davoodi, Rahman, & Andrews, Brian J. (2004). Fuzzy logic control of FES rowing

exercise in paraplegia. IEEE Transactions on Biomedical Engineering, , 51(3),

541-543.

Davoodi, Rahman, Andrews, Brian J, Wheeler, Garry D, & Lederer, Robert. (2002).

Development of an indoor rowing machine with manual FES controller for

total body exercise in paraplegia IEEE Transactions on Neural Systems and

Rehabilitation Engineering, 10(3), 197-203.

Dickstein, Ruth, & Deutsch, Judith E. (2007). Motor imagery in physical therapist

practice. Journal of American Physical Therapy Association, 87(7), 942-958.

Dikkenberg, N. van den, Meijer, O. G., Slikke, R. M. A. van der, Lummel, R. C. van,

Dieën, J. H. van, Pijls, B., . . . Wuisman, P. I. J. M. (2002). Measuring

functional abilities of patients with knee problems: rationale and construction

of the DynaPort knee test. Knee Surgery, Sports Traumatology, 10(4), 204-212.

Djurić-Jovičić, D, Milica, Jovičić, Nenad S., & Popović, Dejan B. (2011). Kinematics

of gait: New method for angle estimation based on accelerometers. Sensors,

11(11), 10571-10585.

Djurić-Jovičić, Milica D., Jovičić, Nenad S., Popović, Dejan B., & Djordjević,

Antonije R. (2012). Nonlinear optimization for drift removal in estimation of

gait kinematics based on accelerometers. Journal of Biomechanics, 45(16),

2849-2854.

Donaldson, N, Perking, TA, Fitzwater, R, Wood, DE, & Middleton, F. (2000). FES

cycling may promote recovery of leg function after incomplete spinal cord

injury. Spinal Cord, 38(11), 680-682.

183

Easton, J. Donald, Saver, Jeffrey L., Albers, Gregory W., Alberts, Mark J., Chaturvedi,

Seemant, Feldmann, Edward, . . . Sacco, Ralph L. (2009). Definition and

evaluation of transient ischemic attack. Stroke, 40(6), 2276-2293

El-Khatib, A., Guillon, F., & Dômont, A. (1998). Vertical vibration transmission

through the lumbar spine of the seated subject—first results. Journal of Sound

and Vibration, 215(4), 763-773.

Fabera, Gert S., Changb, Chien-Chi, Kingmac, Idsart, & Dennerleina, Jack T. (2013).

Lifting style and participant's sex do not affect optimal inertial sensor location

for ambulatory assessment of trunk inclination. Journal of Biomechanics,

46(5), 1027–1030.

Ferdinand, Wagner, Ruedi, Schmuki, Thomas, Wagner, & Wolstenholme, Perer.

(2006). Modeling software with finite state machines: A practical approach.

Boca Roton, New York: Auerbach Publications.

Ferrarin, M., Palazzo, F., Riener, R., & Quintern, J. (2001). Model-based control of

FES-induced single joint movements. Neural Systems and Rehabilitation

Engineering, 9(3), 245-257.

Fisher, M. (2008). Stroke: Investigation and management: Elsevier Health Sciences.

Fisher, Marc. (2009). Handbook of clinical neurology: Stroke. investigation and

management (Vol. 94): Elsevier Health Sciences.

Francisco, Gerard, Chae, John, Chawla, Harmeen, Kirshblum, Steven, Zorowitz,

Richard, Lewis, Gerald, & Pang, Schone. (1998). Electromyogram-triggered

neuromuscular stimulation for improving the arm function of acute stroke

survivors: A randomized pilot study. Archives of Physical Medicine and

Rehabilitation, 79(5), 570-575.

Frankel, M.A, Dowden, B.R, Mathews, V.J, Normann, R.A, Clark, G.A, & Meek, S.G.

(2011). Multiple-input single-output closed-loop isometric force control using

asynchronous intrafascicular multi-electrode stimulation. Neural Systems and

Rehabilitation Engineering, 19(3), 325 - 332.

French, Beverley, Thomas, Lois, H, Leathley, Michael, J, Sutton, Christopher, J ,

McAdam, Joanna , Forster, Anne , . . . Watkins, Caroline, L (2009). Repetitive

task training for improving functional ability after stroke. Stroke, 40(4), 98-99.

French, Beverley, Thomas, Lois, Leathley, Michael, J, Sutton, Christopher , McAdam,

Joanna, Forster, Anne, . . . Watkins, Caroline (2010). Does repetitive task

training improve functional activity after stroke? A Cochrane systematic

review and meta-analysis. Journal of Rehabilitation Medicine, 42(1), 9-15.

Gazzoni, Marco. (2010). Multichannel surface electromyography in ergonomics:

potentialities and limits. Human Factors and Ergonomics in Manufacturing and

Service Industries, 20(4), 255-271.

184

Gebert, Glenn, Snyder, Deryl, Lopez, Juan, Siddiqi, Naveed, & Evers, Johnny. (2003).

Optical flow angular rate determination. Paper presented at the Image

Processing, Proceedings. 2003 International Conference

Godfrey, A., Conway, R., Meagher, D., & ÓLaighin, G. (2008). Direct measurement

of human movement by accelerometry. Medical Engineering & Physics,

30(10), 1364-1386.

Graham, Ryan Bevan. (2008). Effectiveness of an on-body lifting aid at reducing low

back physical demands during an automotive assembly task: Assessment of

EMG response and user acceptability. (PhD), Queen's University, Kingston,

Ontario, Canada.

Granat, M., Keating, J. F., Smith, A. C. B., Delargy, M., & Andrews, B. J. (1992).

The use of functional electrical stimulation to assist gait in patients with

incomplete spinal cord injury. Disability & Rehabilitation, 14(2), 93-97.

Grotta, James C., Noser, Elizabeth A., Ro, Tony, Boake, Corwin, Levin, Harvey,

Aronowski, Jarek, & Schallert, Timothy. (2004). Constraint-induced

movement therapy. Stroke, 35(11), 2699-2701.

Grzeda, Victor, & Fichtinger, Gabor. (2010). C-arm rotation encoding with

accelerometers. International Journal of Computer Assisted Radiology and

Surgery, 5(4), 385-391.

Ha, Dae Woong, Park, Hyo Seon, Choi, Se Woon, & Kim, Yousok. (2013). A

wireless MEMS-based inclinometer sensor node for structural health

monitoring. Sensors, 13(2), 16090-16104.

Haché, Gaëtanne. (2010). Development of a wearable mobility monitoring system.

(PhD), University of Ottawa.

Hammer, Ann M., & Lindmark, Birgitta. (2010). Responsiveness and validity of the

Motor Activity Log in patients during the subacute phase after stroke.

Disability and Rehabilitation, 32(14), 1184–1193.

Handa, Yasunobu, & Hoshimiya, Nozomu. (1987). Functional electrical stimulation

for the control of the upper extremities. Medical Progress Through

Technology, 51-63.

Hara, Yukihiro. (2008). Neurorehabilitation with New Functional Electrical

Stimulation for Hemiparetic Upper Extremity in Stroke Patients. Journal of

Nippon Medical School, 75(1), 4-14.

Hara, Yukihiro, Ogawa, Shinji, Tsujiuchi, Kazuhito, & Muraoka, Yoshihiro. (2008).

A home-based rehabilitation program for the hemiplegic upper extremity by

power-assisted functional electrical stimulation. Disability and Rehabilitation,

30(4), 296 – 304.

185

Harris, Jocelyn E., & Eng, Janice J. (2010). Strength training improves upper-limb

function in individuals with stroke: A meta-analysis. Stroke, 41, 136-140.

Haugland, M, Lickel, A, Haase, J, & Sinkjaer, Thomas. (1999). Control of FES thumb

force using slip information obtained from the cutaneous electroneurogram in

quadriplegic man. Rehabilitation Engineering, 7(2), 215-227.

Hegner, Barbara R., Acello, Barbara, & Caldwell, Esther. (2009). Nursing assistant: A

nursing process approach - basics (1st ed.): Cengage Learning.

Heller, A, Wade, D T, Wood, V A, Sunderland, A, Hewer, R L, & Ward, E. (1987).

Arm function after stroke: Measurement and recovery over the first three

months. Journal of Neurology, Neurosurgery, and Psychiatry, 50(6), 714-719.

Hendricks, H.T, IJzerman, M.J, de Kroon, J.R, in 't Groen, F.A.C.G, & Zilvold, G.

(2001). Functional electrical stimulation by means of the ‘Ness Handmaster

Orthosis’ in chronic stroke patients: an exploratory study. Clinical


Hermann, Valerie Hill, Herzog, Mandy, Jordan, Rachel, Hofherr, Maura, Levine,

Peter, & Page, Stephen J. (2010). Telerehabilitation and electrical stimulation:

an occupation-based, client-centered stroke intervention. The American

Journal of Occupational Therapy, 64(1), 73-81.

Hobby, J., Taylor, P. N., & Esnouf, J. (2001). Restoration of Tetraplegic Hand

Function by Use of the Neurocontrol Freehand System. Journal of Hand

Surgery, 26(5), 459-464.

Hoffer, J . Andy, Stein, Richard B ., Haugland, Morten K., Sinkjaer, Thomas, Durfee,

William K., Schwartz, Andrew B ., . . . Kantor, Carole. (1996). Neural signals

for command control and feedback in functional neuromuscular stimulation :

A review Journal of Rehabilitation Research and Development, 33(2), 145-

157.

Hughes, A. M, Freeman, C, Burridge, J, Chappell, P, Lewin, P, & Rogers, E. (2010).

Shoulder and elbow muscle activity during fully supported trajectory tracking

in people who have had a stroke. Journal of Electromyography and

Kinesiology, 20(3), 465-476.

Hughesa, A.M., Freemanb, C.T., Burridgea, J.H., Chappellb, P.H., Lewinb, P.L., &

Rogersb, E. (2010). Shoulder and elbow muscle activity during fully supported

trajectory tracking in people who have had a stroke. Journal of

Electromyography and Kinesiology, 20(3), 465-476.

Hunt, K. J, Munih, M. De, Donaldson, N. N, & Barr, F. M. D. (1998). Investigation of

the hammerstein hypothesis in the modeling of electrically stimulated muscle.

Biomedical Engineering, 45(8), 998–1009.

Husak, Miroslav. (2002). Model of tilt sensor system. Paper presented at the

Electronics, Circuits and Systems, 2002. 9th International Conference on.

186

Ijzerman, M. J., Stoffers, T. S., Groen, F. A. C. G. In 't, Klatte, M. A. P., Snoek, G. J.,

Vorsteveld, J. H. C., . . . Hermens, H. J. (1996). The NESS Handmaster

orthosis: Restoration of hand function in C5 and stroke patients by means of

electrical stimulation. Journal of Rehabilitation Sciences, 9(3), 86-89

Inglis, J., Donald, MW., Monga, TN., Sproule, M., & Young, MJ. (1984).

Electromyographic biofeedback and physical therapy of the hemiplegic upper

limb. Archives of Physical Medicine and Rehabilitation, 65(12), 755-759.

Inmann, Andreas, & Haugland, Morten. (2004). Implementation of natural sensory

feedback in a portable control system for a hand grasp neuroprosthesis.

Medical Engineering & Physics, 26(6), 449–458.

Jacksona, Philip L, Lafleur, Martin F, Malouin, Francine, Richards, Carol L, & Doyon,

Julien. (2003). Functional cerebral reorganization following motor sequence

learning through mental practice with motor imagery. Neuroimage, 20(2),

1171-1180.

Jaeger, R.J. (1986). Design and simulation of closed-loop electrical stimulation

orthoses for restoration of quiet standing in paraplegia. Journal of

Biomechanics, 19(10), 825-835.

Juan, Cheng, Chen, Xiang, & Shen, Minfen. (2013). A framework for daily activity

monitoring and fall detection based on surface electromyography and

accelerometer signals. Biomedical and Health Informatics, 17(1), 38 - 45.

Kagayaa, Hitoshi, Shimadaa, Yoichi, Ebataa, Kunio, Satoa, Mineyoshi, Satoa, Kozo,

Yukawab, Toshihiro, & Obinata, Goro. (1995). Restoration and analysis of

standing-up in complete paraplegia utilizing functional electrical stimulation

Archives of Physical Medicine and Rehabilitation, 76(9), 876-881.

Kalat, James W. (2008). Biological psychology (10th ed.): Cengage Learning.

Kameyama, Junichi., Handa, Yasunobu., Hoshimiya, Nozomu., & Sakurai, Minoru.

(1999). Restoration of shoulder movement in quadriplegic and hemiplegic

patients by functional electrical stimulation using percutaneous multiple

electrodes. Tohoku Journal of Experimental Medicine, 187(4), 329-338.

Kanchiku, Tsukasa, Lynskey, James V, Protas, Danielle, Abbas, James. J., & Jung,

Ranu. (2008). Neuromuscular electrical stimulation induced forelimb

movement in a rodent model. Journal of Neurosci Methods, 167(2), 317-326.

Karantonis, Dean M, Narayanan, Michael R., Mathie, Merryn, Lovell, Nigel H., &

Celler, Branko G. (2006). Implementation of a real-time human movement

classifier using a triaxial accelerometer for ambulatory monitoring.

Information Technology in Biomedicine, IEEE Transactions on, 10(1), 156-

167.

187

Keith, Michael W., Peckham, P. Hunter, Thrope, Geoffrey B., Stroh, Kathy C., Smith,

Brian, Buckett, James R., . . . Jatich, James W. (1989). Implantable functional

neuromuscular stimulation in the tetraplegic hand. The Journal of Hand

Surgery, 14(3), 524-530.

Kemp, Bob, Janssen, Ad JMW, & Kamp, Bob van der. (1998). Body position can be

monitored in 3D using miniature accelerometers and earth-magnetic field

sensors. Electroencephalography and Clinical

Neurophysiology/Electromyography and Motor Control 109(6), 484-488.

Kengo, Ohnishi, Morio, Toshiyuki, Takagi, Tomoo, & Kajitani, Isamu. (2013).

Multimodal sensor controlled three degree of freedom transradial prosthesis.

Paper presented at the Rehabilitation Robotics (ICORR), IEEE International

Conference

Kern, H., Hofer, C., Strohhofer, M., Mayr, W., Richter, W., & Stöhr, H. (1999).

Standing up with denervated muscles in humans using functional electrical

stimulation. Artificial Organs, 23(5), 447-452.

Knutson, J. S, Hoyen, H. A, Kilgore, K. L, & Peckham, P. H. (2004). Simulated

neuroprosthesis state activation and hand-position control using myoelectric

signals from wrist muscles. Journal of Rehabilitation Research and

Development, 41(3B), 461-472.

Knutson, Jayme S., Harley, Mary Y., Hisel, Terri Z., & Chae, John. (2007).

Improving hand function in stroke survivors: a pilot study of contralaterally

controlled functional electric stimulation in chronic hemiplegia. Archives of

Physical Medicine and Rehabilitation, 88(4), 513-520.

Knutson, Jayme S., Hisel, Terri Z., Harley, Mary Y., & Chae, John. (2009). A novel

functional electrical stimulation treatment for recovery of hand function in

hemiplegia: 12-week pilot study. Neurorehabilitation and Neural Repair,

23(1), 17-25.

Kojovic, Jovana, Djuric-Jovicic, Milica, Dosen, Strahinja, Popovic, Mirjana B., & c,

Dejan B. Popovi. (2009). Sensor-driven four-channel stimulation of paretic leg:

Functional electrical walking therapy. Journal of Neuroscience Methods,

181(1), 100-105.

Kostov, Aleksandar, Andrews, Brian J, Popovic, Dejan B, Stein, Richard B, &

Armstrong, William W. (1995). Machine learning in control of functional

electrical stimulation for locomotion Biomedical Engineering, 42(6), 541-551.

Kralj, A., Acimovic, R, & Stanic, U. (1993). Enhancement of hemiplegic patient

rehabilitation by means of functional electrical stimulation. Prosthetics and

Orthotics International, 17(2), 107-114.

Kroon, Joke R. de, IJzerman, Maarten J., Chae, John, Lankhorst, Gustaaf J., & Zilvold,

Gerrit. (2005). Relation between stimulation characteristics and clinical

188

outcome in studies using electrical stimulation to improve motor control of the

upper extremity in stroke. Journal of Rehabilitation Medicine, 37(2), 65-74.

Kunkel, Annett, Kopp, Bruno, Miiller, Gudrun, Villringer, Kersten, Villringer, Arno,

Taub, Edward, & Flor, Herta. (1999). Constraint-induced movement therapy

for motor recovery in chronic stroke patients. Archives of Physical Medicine

and Rehabilitation, 80(6), 624-628.

Kurosawa, Kenji, Futami, Ryoko, Watanabe, Takashi, & Hoshimiya, Nozomu. (2005).

Joint angle control by FES using a feedback error learning controller. Neural

Systems and Rehabilitation Engineering, 13(3), 359-371.

Kusuhara, Toshimasa, Jikuya, Kazuaki, Nakamura, Takao, Michinishi, Hiroyuki,

Yamamoto, Yoshitake, & Okamoto, Takuji. (2012). Detection of knee-joint

motions using two linear accelerometers for pendulum test. Electronics and

Communications in Japan, 95(7), 41-48.

Kwakkel, Gert, Kollen, Boudewijn J., & Krebs, Hermano I. (2008). Effects of Robot-

assisted therapy on upper limb recovery after stroke: A systematic review.

Neurorehabilitation and Neural Repair, 22(2), 111-121.

Lan, Ning, Crago, Patrick E., & Chizeck, Howard J. (1991). Feedback control

methods for task regulation by electrical stimulation of muscles. Biomedical

Engineering, 38(12), 1213-1223.

Langhorne, Peter, Coupar, Fiona, & Pollock, Alex. (2009). Motor recovery after

stroke: A systematic review. The Lancet Neurology, 8(8), 741-754.

Latt, W.T, Tan, U.-X, Veluvolu, K.C, Lin, J.K.D, Shee, C.Y, & Ang, W.T. (2007).

System to assess accuracy of micromanipulation. Paper presented at the

Engineering in Medicine and Biology Society, 2007, 29th Annual

International Conference of the IEEE Lyon.

Lauer, Richard T., Peckham, Hunter, & Kilgore, Kevin L. (1999). EEG-based control

of a hand grasp neuroprosthesis. NeuroReport, 10, 1767-1771

Lawrence, Marc, Gross, Gion-Pitschen, Lang, Martin, Kuhn, Andreas, Keller, Thierry,

& Morari, Manfred. (2008). Assessment of finger forces and wrist torques for

functional grasp using new multichannel textile neuroprostheses. Artificial

Organs, 32(8), 634-638.

Le, Fengmin, Markovsky, Ivan, Freeman, Christopher, & Rogers, Eric (2009).

Identification of electrically stimulated muscle after stroke. Paper presented at

the European Control Conference 2009, Budapest, Hungary.

Lecouturier, Jan, Murtagh, Madeleine J, Thomson, Richard G, Ford, Gary A, White,

Martin, Eccles, Martin, & Rodgers, Helen. (2010). Response to symptoms of

stroke in the UK: A systematic review. BMC Health Services Research,

10(157).

189

Lee, In Sook, Shin, Yong Beom, Moon, Tae-Yong, Jeong, Yeon Joo, Song, Jong

Woon, & Kim, Dong Hyun. (2009). Sonography of patients with hemiplegic

shoulder pain after stroke: Correlation with motor recovery stage. American

Journal of Roentgenology, 192(2).

Lee, Myong-Woo, Khan, A.M, Kim, Ji-Hwan, Cho, Young-Sun, & Kim, Tae-Seong.

(2010). A single tri-axial accelerometer-based real-time personal life log

system capable of activity classification and exercise information generation.

Paper presented at the Engineering in Medicine and Biology Society (EMBC),

2010 Annual International Conference of the IEEE Buenos Aires

Lemay, Michel A., Crago, Patrick E., Katorgi, Maher, & Chapman, Gregg J. (1993).

Automated tuning of a closed-Loop hand grasp neuroprosthesis. Biomedical

Engineering, 47(7), 675-685.

Letswaart, Magdalena, Johnston, Marie, Dijkerman, H. Chris, Joice, Sara, Scott, Clare

L., MacWalter, Ronald S., & Hamilton, Steven J. C. (2011). Mental practice

with motor imagery in stroke recovery: randomized controlled trial of efficacy.

Brain, 134(5), 1373-1386.

Lindley, R. I. (2008). Stroke (1st ed.): Oxford University Press.

Lo, Albert C., Guarino, Peter D., Richards, Lorie G., Haselkorn, Jodie K., Wittenberg,

George F., Federman, Daniel G., . . . Peduzzi, Peter. (2010). Robot-assisted

therapy for long-term upper-limb impairment after stroke. The New England

Journal of Medicine, 362(19), 1772-1783.

Lorie, Richards, Carolyn, Hanson, Melinda, Wellborn, & Amit, Sethi. (2008). Driving

motor recovery after stroke. Topics in Stroke Rehabilitation, 15(5), 397-411.

Łuczak, S. (2007). Recent advances in mechatronics (R. Jabłoński, M. Turkowski &

R. Szewczyk Eds.): Springer Berlin Heidelberg.

Lugade, Vipul, Fortune, Emma, Morrow, Melissa, & Kaufman, Kenton. (2014).

Validity of using tri-axial accelerometers to measure human movement—Part

I: Posture and movement detection. Medical Engineering & Physics, 36(2),

169-176.

Luinge, H. J. (2002). Inertial sensing of human movement. (PhD), Twente University,

The Netherlands.

Luinge, H.J., Veltink, P.H., & Baten, C.T.M. (1999). Estimating orientation with

gyroscopes and accelerometers. Technology and Health Care 7(6), 455-459.

Lynch, Cheryl L., & Popovic, Milos R. (2008). Functional electrical stimulation.

IEEE Control Systems magazine, 28(2), 40-50.

Lyons, G.M., Culhane, K.M., Hilton, D., Grace, P.A., & Lyons, D. (2005). A

description of an accelerometer-based mobility monitoring technique. Medical

Engineering & Physics, 27(6), 497-504.

190

Lyons, Gerard M., Sinkjær, Thomas, Burridge, Jane H., & Wilcox, David J. (2002). A

review of portable FES-based neural orthoses for the correction of drop foot.

Neural Systems and Rehabilitation Engineering, 10(4), 260-279.

Mangold, S, Keller, T, Curt, A, & Dietz, V. (2005). Transcutaneous functional

electrical stimulation for grasping in subjects with cervical spinal cord injury.

Spinal Cord, 43(1), 1-13.

Mann, Geraldine, Burridge, Jane H, Malone, L. J, & Strike, P. W. (2005). A Pilot

Study to Investigate the Effects of Electrical Stimulation on Recovery of Hand

Function and Sensation in Subacute Stroke Patients. Neuromodulation, 8(3),

193-202.

Mann, Geraldine, Taylor, Paul, & Lane, Rod. (2011). Accelerometer-triggered

electrical stimulation for reach and grasp in chronic stroke patients: A pilot

study. Neurorehabilitation and Neural Repair 25(8), 774-780.

Marshall, Randolph S., Perera, Gerard M., Lazar, Ronald M., Krakauer, John W.,

Constantine, Robert C., & DeLaPaz, Robert L. (2000). Evolution of cortical

activation during recovery from corticospinal tract infarction. Stroke, 31(3),

656-661.

Masiero, Stefano, Celia, Andrea, Rosati, Giulio, & Armani, Mario. (2007). Robotic-

assisted rehabilitation of the upper limb after acute stroke. Archives of

Physical Medicine and Rehabilitation, 88(2), 142-149.

Mathie, Merryn J, Coster, Adelle C F, Lovell, Nigel H, & Celler, Branko G. (2004).

Accelerometry: Providing an integrated, practical method for long-term,

ambulatory monitoring of human movement. Physiological Measurement,

25(2), R1-20.

Matjacic, Zlatko, & Bajd, Tadej. (1998). Arm-free paraplegic standing—Part II:

Experimental results. IEEE Transactions on Rehabilitation Engineering, 6(2),

139-150.

Mehrholz, Jan, Platz, Thomas, Kugler, Joachim, & Pohl, Marcus. (2009).

Electromechanical and robot-assisted arm training for improving arm function

and activities of daily living after stroke. Stroke, 40(5), 392-393.

Miura, N, Watanabe, Ta, Akasaka, K, & Suzuki, T. (2011). A clinical trial of a

prototype of wireless surface fes rehabilitation system in foot drop correction.

Paper presented at the Engineering in Medicine and Biology Society, 2011

Annual International Conference of the IEEE Boston, MA

Morris, J. R. W. (1973). Accelerometry—A technique for the measurement of human

body movements. Journal of Biomechanics, 6(6), 729-736.

Mortimer, JT. (1981). Motor prostheses: An American Physiological Society Book.

191

Mulcahey, M. J., Betz, Randal R., Smith, Brian T., Weiss, Albert A., & Davis, Sheryl

E. (1997). Implanted functional electrical stimulation hand system in

adolescents with spinal injuries: An evaluation. Archives of Physical Medicine

and Rehabilitation, 78(6), 597-607.

Muraoka, Yoshihiro. (2001). Development of an EMG recording device from

stimulation electrodes for functional electrical stimulation. Frontiers of

Medical & Biological Engineering, 11(4), 323-333.

NAO. (2005). Reducing brain damage: faster access to better stroke care. London:

Stationery Office: National Audit Office.

Nevins, Ralph Jay, Durdle, N. G., & Raso, V. J. (2002). A posture monitoring system

using accelerometers. Electrical and Computer Engineering, 2, 1087-1092.

Nudo, Randolph J., Wise, Birute M., SiFuentes, Frank, & Milliken, Garrett W. (1996).

Neural substrates for the effects of rehabilitative training on motor recovery

after ischemic infarct. Science, 272(5269), 1791-1794.

Nykänen, Kati. (2010). The effectiveness of robot-aided upper limb therapy in stroke

rehabilitation: A systematic review of randomized controlled studies.

ni ersit of sk l .

Page, Stephen J., Levine, Peter, & Leonard, Anthony. (2007). Mental practice in

chronic stroke results of a randomized, placebo-controlled trial. Stroke, 38,

1993-1997.

Page, Stephen J., Sisto, SueAnn, & Johnston, Mark V. (2002). Modified constraint-

induced therapy after subacute stroke: a preliminary study.


Page, Stephen J., Sisto, SueAnn, Levine, Peter, Johnston, Mark V., & Hughes, Mary.

(2001). Modified constraint induced therapy: A randomized feasibility and

efficacy study Rehabilitation Research and Development, 38(5), 583-590.

Page, Stephen J., Szaflarski, Jerzy P., Eliassen, James C., Pan, Hai, & Cramer, Steven

C. (2009). Cortical plasticity following motor skill learning during mental

practice in stroke. Neurorehabilitation and Neural Repair, 23(4), 382-388.

Pallejà, Tomàs, Tresanchez, Marcel, Teixidó, Mercè, & Palacin., Jordi. (2010).

Bioinspired electronic white cane implementation based on a LIDAR, a tri-

axial accelerometer and a tactile belt. Sensors, 10(12), 11322-11339.

Parker, V. M., Wade, D. T., & Hewer, R. Langton. (1986). Loss of arm function after

stroke: measurement, frequency, and recovery. International Rehabilitation

Medicine, 8(2), 69-73.

Peckham, P. H., Mortimer, J. T., & Marsolais, E. B. (1980). Controlled prehension

and release in the C5 quadriplegic elicited by functional electrical stimulation

192

of the paralyzed forearm musculature Annuals of Biomedical Engineering,

8(4-6), 369-388.

Peckham, P. Hunter, Keith, Michael W., Kilgore, Kevin L., MS, Julie H. Grill,

Wuolle, Kathy S., Thrope, Geoffrey B., . . . Wiegner, Allen. (2001). Efficacy

of an implanted neuroprosthesis for restoring hand grasp in tetraplegia: A

multicenter study. Archives of Physical Medicine and Rehabilitation, 82(10),

1380-1388.

Peckham, P. Hunter, & Knutson, Jayme S. (2005). Functional electrical stimulation

for neuromuscular applications. Annual Review of Biomedical Engineering,

7(1), 327-360. doi: 10.1146/annurev.bioeng.6.040803.140103

Pfurtscheller, Gert, Müller-Putz, Gernot R., Pfurtscheller, Jörg, & Rupp, Rüdiger.

(2005). EEG-based asynchronous BCI controls functional electrical

stimulation in a tetraplegic patient. EURASIP Journal on Applied Signal

Processing 2005(19), 3087-3174.

Pfurtscheller, Gert, Müllera, Gernot R., Pfurtschellerc, Jörg, Gernerd, Hans Jürgen, &

Ruppd, Rüdiger. (2003). ‘Thought’ – control of functional electrical

stimulation to restore hand grasp in a patient with tetraplegia Neuroscience

Letters, 351(1), 33-36.

Pons, D. J., Vaughan, C. L., & Jaros, G. G. (1989). Cycling device powered by the

electrically stimulated muscles of paraplegics. Medical and Biological

Engineering and Computing, 27(1), 1-7.

Popović, Dejan B, Sinkjærc, Thomas, & Popović, Mirjana B. (2009). Electrical

stimulation as a means for achieving recovery of function in stroke patients.


Popovic, Dejan, Stojanovic, Aleksandar, Pjanovic, Andjelka, Radosavljevic,

Slobodanka, Popovic, Mirjana, Jovic, Stevan, & Vulovic, Dragan. (1999).

Clinical evaluation of the bionic glove. Archives of Physical Medicine and

Rehabilitation, 80(3), 299-304

Popovic, Mirjana B., Popovic, Dejan B., Sinkjær, Thomas, Stefanovic, Aleksandra, &

Schwirtlich, Laszlo. (2002). Restitution of reaching and grasping promoted by

functional electrical therapy. Artificial Organs, 26(3), 271-275.

Postans, Neil J, & Granat, Malcolm H. (2005). Effect of functional electrical

stimulation, applied during walking, on gait in spastic cerebral palsy.

Developmental Medicine & Child Neurology, 47(1), 46-52.

Powell, Joanna, Pandyan, A. David, Granat, Malcolm, Cameron, Margaret, & Stott,

David J. (1999). Electrical stimulation of wrist extensors in poststroke

hemiplegia Stroke, 30, 1384-1389.

Prange, Gerdienke B, Jannink, Michiel J. A, Groothuis-Oudshoorn, Catharina G. M,

Hermens, Hermie J, & Ijzerman, Maarten J. (2006). Systematic review of the

193

effect of robot-aided therapy on recovery of the hemiparetic arm after stroke.

The Journal of Rehabilitation Research and Development, 43(2), 171. doi:

10.1682/jrrd.2005.04.0076

Prochazka, Arthur. (1993). Comparison of natural and artificial control of movement.

Rehabilitation Engineering, 1(1), 7-17.

Prochazka, Arthur, Gauthier, Michel, Wieler, Marguerite, & Kenwell, Zoltan. (1997).

The bionic glove: an electrical stimulator garment that provides controlled

grasp and hand opening in quadriplegia. Archives of Physical Medicine and


Prochazka, Arthur, Wieler, Marguerite, Kenwell, Zoltan R, & Gauthier, Michel J. A.

(1996). U. S. Patent.

Qilong, Xue, Ruihe, Wang, Feng, Sun, Leilei, Huang, & Laiju, Han. (2013).

Continuous measurement-while-drilling utilizing strap-down multi-model

surveying system. Instrumentation and Measurement, 63(3), 650 - 657

Rakos, M, Hanh, A, Uher, E, & Edenhofer, M. (2007). EMG triggered rehabilitation

of complex movements–biofeedback/stimulation system STIWELL med4. Paper

presented at the 12th Annual Conference of the International FES Society,

Philadelphia, PA USA.

Régine, Brissot, Philippe, Gallien, Marie-Pierre, Le Bot, Anne, Beaubras, Dominique,

Laisné, Jocelyne, Beillot, & Josette, Dassonville. (2000). Clinical experience

with functional electrical stimulation-assisted gait with Parastep in spinal cord-

injured patients. Spine, 25(4), 501-508.

Remme, Espen W, Hoff, Lars, Halvorsen, Per Steinar, Nærum, Edvard, Skulstad,

Helge, Fleischer, Lars A., . . . Fosse, Erik. (2009). Validation of cardiac

accelerometer sensor measurements. Physiological Measurement, 30(12),

1429.

Ring, Haim, & Rosenthal, Nechama. (2005). Controlled study of neuroprosthetic

functional electrical stimulation in sub-acute post-stroke rehabilitation.

Journal of Rehabilitation Medicine, 37(1), 32-36.

Robinson, Andrew J., & Snyder-Mackler, Lynn. (2007). Clinical electrophysiology:

Electrotherapy and electrophysiologic testing (3rd ed.): Lippincott Williams &

Wilkins.

Rodriguez-Donate, Carlos, Morales-Velazquez, Luis, Osornio-Rios, Roque Alfredo,

Herrera-Ruiz, Gilberto, & Romero-Troncoso, Rene de Jesus. (2010). FPGA-

based fused smart sensor for dynamic and vibration parameter extraction in

industrial robot links. Sensors, 10(4), 4114-4129.

Roetenberg, Daniel. (2006). Inertial and magnetic sensing of human motion. (PhD),

University of Twente.

194

Roetenberg, Daniel, Slycke, Per J., & Veltink, Peter H. (2007). Ambulatory position

and orientation tracking fusing magnetic and inertial sensing. IEEE

Transactions on Biomedical Engineering, 54(5), 883-890.

Sabatini, Angelo Maria. (2011). Estimating three-dimensional orientation of human

body parts by inertial/magnetic sensing. Sensors, 11(2), 1489-1525.

Saxena, Shalini, Nikolic, Slavica, & Popovic, Dejan. (1995). An EMG-controlled

grasping system for tetraplegics. Journal of Rehabilitation Research &

Development, 32(1), 17.

Scherberger, Hansjörg. (2009). Neural control of motor prostheses. Current Opinion

in Neurobiology, 19(6), 629–633.

Schmidt, Richard A., & Wrisberg, Craig A. (2008). Motor learning and performance:

A situation-based learning approach (4 ed.): Human Kinetics.

Sheriff, DS. (2004). Medical biochemistry (1st ed.): Jaypee Brothers.

Shi, Yue X., Tian, Jin H., Yang, Ke H., & Zhao, Yue. (2011). Modified constraint-

induced movement therapy versus traditional rehabilitation in patients with

upper-extremity dysfunction after stroke: A systematic review and meta-

analysis. Archives of Physical Medicine and Rehabilitation, 92(6), 972-982.

Shizuka, Kawatsu, Katsufumi, Sato, Yuuki, Watanabe, Susumu, Hyodo, P, Breves

Jason, K, Fox Bradley, . . . Nobuyuki, Miyazaki. (2009). A new method to

calibrate attachment angles of data loggers in swimming sharks. EURASIP

Journal on Advances in Signal Processing

Shumway-Cook, Anne, & Woollacott, Marjorie Hines. (2001). Motor control -

Theory and practical applications (2nd ed.). Maryland, Pennsylvania USA:

Lippincott Williams & Wilkins.

Sinkjaer, Thomas, Haugland, Morten, Inmann, Andreas, Hansen, Morten, & Nielsen,

Kim D. (2003). Biopotentials as command and feedback signals in functional

electrical stimulation systems. Medical Engineering and Physics 25(1), 29-40.

Skotte, Jørgen, Korshøj, Mette, Kristiansen, Jesper, Hanisch, Christiana, &

Holtermann, Andreas. (2014). Detection of physical activity types using

triaxial accelerometers. Journal of Physical Activity & Health, 11(1), 76-84.

Smeltzer, Suzanne C., Bare, Brenda G, Hinkle, Janice L., & Cheever, Kerry H. (2009).

Brunner and Suddarth's te t ook of medical-surgical nursing

Volume 1 of Brunner & Suddarth's Textbook of Medical-surgical Nursing (12 ed. Vol.

1): Lippincott Williams & Wilkins.

Smith, C. (in preparation). The design and usability evaluation of an FES Graphical

User Interface, for use by therapists with stroke patients in a clinical

environment. PhD thesis. (PhD), University of Salford.

195

Snoek, G. J., Ijzerman, M. J., Groen, F. A. C. G. in 't, Stoffers, T. S., & Zilvold, G.

(2000). Use of the NESS handmaster to restore handfunction in tetraplegia:

Clinical experiences in ten patients. Spinal Cord, 38(4), 244-249.

Someren, Eus J. W. Van. (1997). Actigraphic monitoring of movement and rest-

activity rhythms in aging, Alzheimer's disease, and Parkinson's disease. IEEE

Transactions on Rehabilitation Engineering, 5(4), 394-398.

Song, Sa-kwang, Jang, Jaewon, & Park, Soo-Jun (2009). Dynamic activity

classification based on automatic adaptation of postural orientation Paper

presented at the Engineering in Medicine and Biology Society, 2009. Annual

International Conference of the IEEE Minneapolis, MN

Strojnik, Primoz, & Peckham, P. Hunter (Eds.). (2000). Implantable stimulators for

neuromuscular control (2 ed. Vol. 1): CRC Press LLC and Springer-Verlag

GmbH & CO.

Suputtitada, Areerat, Suwanwela, Nijasri C, & Tumvitee, Suwita. (2004).

Effectiveness of constraint-induced movement therapy in chronic stroke

patients. Journal of the Medical Association of Thailand, 87(12), 482-490.

Sweeney, P. C., Lyons, G. M., & Veltink, P. H. (2000). Finite state control of

functional electrical stimulation for the rehabilitation of gait Medical and

Biological Engineering and Computing, 38(2), 121-126.

Taub, E, Miller, NE, Novack, TA, Cook, EW 3rd, Fleming, WC, Nepomuceno,

CS, . . . Crago, JE. (1993). Technique to improve chronic motor deficit after

stroke. Archives of Physical Medicine and Rehabilitation 74(4), 347-354.

Taub, Edward , Uswatte, Gitendra, & Pidikiti, Rama. (1999). Constraint-induced

movement therapy: A new family of techniques with broad application to

physical rehabilitation--A clinical review. Journal of Rehabilitation Research

and Development, 36(3), 237-251.

Teasdale, N., Forget, R., Bard, C., Paillard, J., & Lamarre, M. FleuryY. (1993). The

role of proprioceptive information for the production of isometric forces and

for handwriting tasks. Acta Psychologica, 82(1-3), 179-191.

Thrasher, T. Adam, Zivanovic, Vera, McIlroy, William, & Popovic, Milos R. (2008).

Rehabilitation of reaching and grasping function in severe hemiplegic patients

using functional electrical stimulation therapy. Neurorehabilitation and

Neural Repair, 22(6), 706-714.

Timmermans, Annick A. A, Seelen, Henk A. M., Willmann, Richard D, Bakx,

Wilbert, de Ruyter, Boris, Lanfermann, Gerd, & Kingma, Herman. (2009).

Arm and hand skills: Training preferences after stroke. Disability &


Tomović, Rajko, Popović, Dejan, & Stein, Richard B. (1995). Nonanalytical methods

for motor control: World Scientific.

196

Tresadern, Philip A., Thies, Sibylle B., Kenny, Laurence P. J., Howard, David, &

Goulermas, John Y. (2008). Rapid prototyping for functional electrical

stimulation control. Pervasive Computing, IEEE, 7(2), 62-69.

Uswatte, Gitendra, Taub, Edward, Morris, David, Vignolo, Mary, & McCulloch,

Karen. (2005). Reliability and validity of the upper-extremity Motor Activity

Log-14 for measuring real-world arm use. Stroke, 36(11), 2493-2496.

Veltink, Peter H., Engberink, Erwin G. Qlde, Hilten, Bob J. van, Dunnewold, Rob, &

Jacobi, Cathrien. (1997). Towards a new method for kinematic quantification

of bradykinesia in patients with Parkinson's disease using triaxial

accelerometry. Paper presented at the Engineering in Medicine and Biology

Society, IEEE 17th Annual Conference.

Verplaetse, Christopher. (1996). Inertial proproceptive devices: Self-motion-sensing

toys and tools. IBM Systems Journal, 35(3.4), 639-650.

Villanueva, D., Trujillo, A., Fennon, E., Cardie, E., & Hedz, Pablo-Ro. (2002).

Method for monitoring acceleration of the trunk during gait. Paper presented

at the 24th Annual Conference and the Annual Fall Meeting of the Biomedical

Engineering Society EMBS/BMES Conference.

Vinande, E, Axelrad, P, & Akos, D. (2010). Mounting-angle estimation for personal

navigation devices. Vehicular Technology, 59(3), 1129 - 1138

Vodovnik, L, Long, C, Reswick, J. B, Lippay, A, & Starbuck, D. (1965). Myo-electric

control of paralyzed muscles. IEEE Transactions on Biomedical Engineering,

12, 169-172.

Volpe, Bruce, T, Ferraro, Mark , Krebs, Hermano I , & Hogan, Neville (2002).

Robotics in the rehabilitation treatment of patients with stroke. Current

Atherosclerosis Reports 4(4), 270-276.

Watanabe, T, Murakami, T, & Handa, Y. (2013). Preliminary tests of a prototype FES

control system for cycling wheelchair rehabilitation. Paper presented at the

Rehabilitation Robotics (ICORR), 2013 IEEE International Conference on

Seattle, WA

WHO. (2003). World health report 2003: World Health Organization.

Willemsen, Antoon Th M, Frigo, Carlo, & Boom, Herman BK. (1991). Lower

extremity angle measurement with accelerometers—error and sensitivity

analysis. Biomedical Engineering, 38(12), 1186-1193.

Williams, Albert J. (2004). A solid state tilt meter for current meter attitude

determination. Paper presented at the Ocean'04. MTTS/IEEE Techno-

Ocean'04 Kobe.

197

Winstein, Carolee J, Rose, Dorian K, Tan, Sylvia M, Lewthwaite, Rebecca, Chui,

Helena C, & Azen, Stanley P. (2004). A randomized controlled comparison of

upper-extremity rehabilitation strategies in acute stroke: a pilot study of

immediate and long-term outcomes. Archives of Physical Medicine and


Wolf, Steven L, Winstein, Carolee J, Miller, J Philip, Taub, Edward, Uswatte,

Gitendra, Morris, David, . . . Nichols-Larsen, Deboral. (2006). Effect of

constraint-induced movement therapy on upper extremity function 3 to 9

months after stroke: the EXCITE randomized clinical trial. American Medical

Association, 296(17), 2095-2104.

Wolf, Steven L, Winstein, Carolee J, Miller, J Phillip, Thompson, Paul A, Taub,

Edward, Uswatte, Gitendra, . . . Clark, Patricia C. (2008). Retention of upper

limb function in stroke survivors who have received constraint-induced

movement therapy: the EXCITE randomised trial. The Lancet Neurology, 7(1),

33-40.

Wong, Wai Yin, Wong, Man Sang, & Lo, Kam Ho. (2007). Clinical applications of

sensors for human posture and movement analysis: A review. Prosthetics and

Orthotics International, 31(1), 62-75.

Woodford, Henry J, & Price, Christopher IM. (2007). EMG biofeedback for the

recovery of motor function after stroke. Cochrane Database of Systematic

Reviews Online (2).

Yamaguchi, Tomofumi, Tanabe, Shigeo, Muraoka, Yoshihiro, Imai, Syuji, Masakado,

Yoshihisa, Hase, Kimitaka, . . . Liu, Meigen. (2011). Effects of integrated

volitional control electrical stimulation (IVES) on upper extremity function in

chronic stroke. The Keio Journal of Medicine, 60(3), 90-95.

Yang, Che-Chang, & Hsu, Yeh-Liang. (2010). A review of accelerometry-based

wearable motion detectors for physical activity monitoring Sensors, 10(8),

7772-7788.

Yang, Jeng-Feng, Scholz, John. P., & Latash, Mark L. (2007). The role of kinematic

redundancy in adaptation of reaching. Experimental Brain Research 176(1),

54-69.

Zhang, Jun, Qiao, Guifang, Song, Guangming, & Wang, Aimin. (2012). Design and

implementation of a remote control system for a bio-inspired jumping robot.

International Journal of Advanced Robotic Systems, 9, 1-11.

Zheng, H, Black, N. D, & Harris, N. D. (2005). Position-sensing technologies for

movement analysis in stroke rehabilitation. Medical and Biological

Engineering and Computing, 43(4), 413-420.

Zhou, Huiyu, & Hu, Huosheng. (2007). Inertial sensors for motion detection of human

upper limbs. Sensor Review, 27(2), 151-158.

198

Zhou, Huiyu, & Hu, Huosheng. (2008). Human motion tracking for rehabilitation—A

survey. Biomedical Signal Processing and Control, 3(1), 1-18.

Zhou, Huiyu, Stone, Thomas, Hu, Huosheng, & Harris, Nigel. (2008). Use of multiple

wearable inertial sensors in upper limb motion tracking. Medical Engineering

& Physics, 30(1), 123-133.

a functional electrical stimulation (fes...

Documents