INSTITUTE OF PHYSICS PUBLISHING JOURNAL OF NEURAL ENGINEERING

J. Neural Eng. 2 (2005) S235–S249 doi:10.1088/1741-2560/2/3/S07

An optimal state estimation model ofsensory integration in human posturalbalanceArthur D Kuo

Department of Mechanical Engineering, University of Michigan, 2350 Hayward St., Ann Arbor,MI 48109-2125, USA

E-mail: artkuo@umich.edu

Received 4 March 2005Accepted for publication 13 July 2005Published 31 August 2005Online at stacks.iop.org/JNE/2/S235

AbstractWe propose a model for human postural balance, combining state feedback control withoptimal state estimation. State estimation uses an internal model of body and sensor dynamicsto process sensor information and determine body orientation. Three sensory modalities aremodeled: joint proprioception, vestibular organs in the inner ear, and vision. These are matedwith a two degree-of-freedom model of body dynamics in the sagittal plane. Linear quadraticoptimal control is used to design state feedback and estimation gains. Nine free parametersdefine the control objective and the signal-to-noise ratios of the sensors. The model predictsstatistical properties of human sway in terms of covariance of ankle and hip motion. Thesepredictions are compared with normal human responses to alterations in sensory conditions.With a single parameter set, the model successfully reproduces the general nature of posturalmotion as a function of sensory environment. Parameter variations reveal that the model ishighly robust under normal sensory conditions, but not when two or more sensors areinaccurate. This behavior is similar to that of normal human subjects. We propose thatage-related sensory changes may be modeled with decreased signal-to-noise ratios, andcompare the model’s behavior with degraded sensors against experimental measurements fromolder adults. We also examine removal of the model’s vestibular sense, which leads toinstability similar to that observed in bilateral vestibular loss subjects. The model may beuseful for predicting which sensors are most critical for balance, and how much they candeteriorate before posture becomes unstable.

(Some figures in this article are in colour only in the electronic version)

1. Introduction

The human central nervous system (CNS) regulates posturalbalance of a complex, multi-segment body with many actuatorsand a wide array of sensors. Each of these sensors provideslocalized and imperfect information regarding the motion ofone or more body segments, subject to the dynamics of thesensors themselves. These data are then conveyed to the CNSby an array of afferents, each of which provides only a fractionof the total sensory information. The CNS continuouslygenerates motor commands to compensate for the unstable

dynamics of the body. From a control systems standpoint,there are two challenging issues in posture. The first is the useof feedback to control an unstable system. The second is theprocessing of noisy sensory data to provide the informationnecessary to perform this control.

Upright standing places the body in an unstableconfiguration that requires active feedback stabilization (seefigure 1(a)). The minimal set of information necessary tostabilize the body appears to be the full dynamical state,that is, data equivalent to the positions and velocities of allbody segments. Inertial coupling between body segments

1741-2560/05/030235+15$30.00 © 2005 IOP Publishing Ltd Printed in the UK S235

A D Kuo

BodyDynamics

CNS Control

CNS Control CNS Control

Sensor Dynamics

FeedbackControl

K

SensorDynamics

state x

x

y

u

y-

u

y

x

sensoryoutput

y

stateestimate

x

estimatorcorrection

sensoryprediction

error

efference copy

move-ment

motorcmd

u

SensoryProcessing

(a) General Feedback Control Model (b) Sensor Model

(c) Direct Feedback Model (d) State Estimator Model

Weights

+ + + +

Ankle proprioception (ap)

Hip proprioception (hp)

Semicircular canals (sc)

Otoliths (ot)

Visual translation (vt)

Visual rotation (vr)

+

^

FeedbackControl

K

Internal Model:Body and Sensor

Dynamics

EstimatorGain

L

y^

Figure 1. Schematics of central nervous system (CNS) control. (a) General feedback model produces motor commands u that drive thebody dynamics, producing motion described by body state x. Sensory dynamics translate the state into sensory outputs y, feeding back tothe CNS. Feedback control K is modeled as state feedback, preceded by sensory processing if necessary. (b) Sensors for posture are of threetypes: proprioceptors, vestibular sensors and vision. Major proprioceptors include ankle and hip muscle spindles; vestibular sensors includesemicircular canals and otoliths; vision includes sensation of translation and rotation of the head. Each of these sensors has dynamics thattemporally filter the state x. (c) Direct feedback is a simple model of feedback control where sensory outputs y are fed directly into a matrixof gains to produce a motor command u that is essentially a weighted sum of the components of y. (d) State estimation is a different methodof sensory processing that uses an internal model of body and sensor dynamics and efference copy to produce the state estimate x that entersthe feedback control gain matrix. The internal model also predicts the sensory output y, and the error of this prediction is used todynamically correct x. State estimation can temporally process information from multiple sensors, each with distinct dynamics, so thatdisparate and noisy data can be integrated to yield an optimal estimate.

appears to make purely local feedback—controlling each jointwith information only from that same joint—an unsuitablestabilization strategy (Camana et al 1977). Stability istheoretically improved by coupling the feedback, that is,making the motor commands for a single joint dependent onfeedback from throughout the body (He et al 1991, Kuo 1995).Indeed, human postural behaviors appear to use heterogenicfeedback, both during quiet standing (Speers et al 2002) and inresponse to large perturbations (Barin 1989, Park et al 2004).This behavior is equivalent to full state feedback, where eachmotor command is dependent on the positions and velocitiesof many-body segments, with closed-loop stability determinedby body dynamics and a matrix of feedback gains.

The most simplistic implementation of this control is tofeed sensory data (figure 1(b)) directly into a matrix of gainswith no prior processing. The large variety and number ofsensors is certainly sufficient to provide state information

(He et al 1991) needed for such direct feedback control(figure 1(c)). Where multiple sensors provide redundant data,the gain matrix effectively performs a weighted sum thatwill average out errors from noisy measurements. However,direct feedback does not adjust for sensory dynamics. Forexample, the semicircular canals act as high-pass filters ofangular head velocity, yet posture control does not appearto be similarly filtered. Direct feedback is also not robustto sensory conflict. If one sensor conflicts with all others, itwill nonetheless contribute to a compensatory motor commandin direct proportion to its relative weight or gain within thefeedback matrix. If sensory data were applied directly infeedback, posture control would be expected to exhibit thedynamics of the sensors, and to be highly sensitive to erroneoussensory inputs.

These drawbacks could be addressed by processing orfiltering data as part of sensory integration. Such a processing

S236

An optimal state estimation model of sensory integration

system would receive sensory data as input, and produce thebody state as output. An obvious method of removing sensorydynamics would be to invert each sensory transfer function.However, an inverse transfer function is not necessarily stable,for example the inverse of the semicircular canal’s (high-passfilter) transfer function is unstable. The preferred meansof processing sensory data is the Kalman filter, or optimalstate estimator (figure 1(d)). It uses an internal, forwardmodel of body and sensory dynamics to predict sensoryoutput. The error in this prediction is then fed back througha set of estimator gains to correct the state estimate, whichis in turn used for feedback control. A state estimator isa near-inverse of sensory dynamics that is both stable andcausal. ‘Optimal’ refers to the design of estimator gains tominimize the mean-square estimation error, dependent on thepresence of noise acting on the system. State estimators areused in modern control systems such as aircraft avionics andnavigation systems, and have long been proposed as modelsfor CNS sensory processing (Borah et al 1988, He et al 1991,Kuo 1995, Merfeld et al 1993, Wiener 1948).

We propose a computational model for posture controlthat combines state feedback and estimation. (Portions ofthis work were previously described in abbreviated form,Kuo (1997)). The state estimator processes information frommultiple physiological sensors to provide the state informationrequired for feedback control. Our model is stochastic,with noise acting as disturbances to the body and errors insensory transduction. These noise characteristics are noteasily measurable, and therefore serve as free parameters inthe model. We examine the sensitivity of predicted posturebehaviors to variations in these free parameters, and comparethe stabilizing behaviors to human subjects from two agegroups and one sensory loss group. Section 2 provides a briefbackground on models for human body and sensor dynamics,human perception, state feedback control and experimentalmethods. Section 3 describes the model, which integratesmany of these components and is used to make predictionsof observable behavior. Section 4 presents a comparison oftheoretical predictions and experimental results along witha sensitivity analysis. Section 5 discusses biological andrehabilitative implications. Finally, section 6 summarizes thefindings.

2. Background

Human posture is controlled with feedback from a varietyof sensory organs. Three groups of sensors are thought tohave primary responsibility in posture control: somatosensors,vestibular organs and vision (Horak and Macpherson 1995).The somatosensors are responsible for proprioception—thesensing of joint motion and limb position. Proprioception isdominated by muscle spindles, which are embedded withinmuscle and sense a combination of muscle length change andthe rate of change. There are two types of vestibular organs.Semicircular canals are fluid-filled canals in the inner earwhich detect angular velocity of the head by sensing viscousmotion of the fluid; otoliths are crystal-like masses mounted onhair cells, and act as linear accelerometers. Vision is similarly

sensitive to self-motion of the head. In the case of posture, therelevant signals are processed by motion detection circuitrynot only in the retina but also in the visual cortex, so that thevestibular nuclei receive signals proportional to rotational andtranslational motion of the visual field (Young 1981).

These sensory signals are fed back to a series ofhierarchical feedback loops to generate stabilizing motorcommands. The conduit for all movement signals is the spinalcord (Horak and Macpherson 1995), which also produces thelowest level of neural feedback (see figure 1(a)). This feedbackis in the form of local reflexes, in which stretch signals from amuscle are relayed to the spinal cord, passed across one or a fewintermediate neural connections, and then fed directly back tothe muscle, commanding a compensatory contraction. Thisshort feedback loop has fast latency, in the range of 30–60 ms.However, the speed of such a loop comes with a disadvantage:local reflexes are the least integrative of postural responses,and are limited to relatively simple behaviors (Nashner 1977).

The second and most important level of feedback forbalance control involves signals traveling up to the mid-brain (see figure 1(a)). The brainstem serves as a relayand integration center, receiving and sending vast numbers ofsensory and motor command signals. The mid-level feedbackloop involves longer conduction paths and greater numbers ofneural synapses, and consequently has a longer latency thanspinal reflexes (often 90 ms and greater). But the convergenceof many signals and complexity of connections allows the brainstem to generate much more complex movements, mostly ofan automatic nature. The brainstem modulates the behavior oflower level reflexes, and is itself modulated by the next higherand final level. The cerebral cortex and related structuresgenerate highly complex movements, mostly of a voluntarynature, with longer latencies than the two lower feedbackloops (see figure 1). The longer latencies suggest that thecerebral cortex has a modulatory, rather than direct role inposture control. Brainstem lesions cause movement disordersaffecting control of balance, indicating a more direct role forthe mid-brain level of feedback (Nashner 1977).

Sensory integration can be assessed experimentally withclinical tests. A number of clinical tests exist for evaluatingbalance in patients suffering from instability (Baloh et al1995). While some tests examine individual componentssuch as a single sensory modality, others consider integrative,systems-level performance. The integrative tests involveapplying perturbations to a movable support surface andrecording the postural response, or recording motion orforces under the feet during quiet standing. One particularlyintegrative clinical test is the sensory organization test (SOT),also known as dynamic posturography (Mirka and Black1990, Peterka and Black 1990), which uses a support surfacewhich can be rotated about an axis approximately aligned tothe ankles, and a visual surround which provides a visualfield which can also be rotated about the same axis (seefigure 2). In response to fore-aft body motion, the supportsurface and visual surround may be servo-driven (sway-referenced) to match the motion of the ankles. Sway-referencing of the support surface has the effect of renderingankle proprioception inaccurate. Visual sway-referencing

S237

A D Kuo

= 1: Normal

= 2: Eyes closed

= 3: Vision sway-ref

= 4: Platformsway-ref

= 5: Eyes closed

Platform sway-ref

= 6: Vision sway-ref

Platform sway-ref

Figure 2. Experimental apparatus for testing balance using dynamicposturography, or sensory organization test (SOT). Subject standson a platform which can rotate about an axis collinear with theankles. A visual surround can also rotate about this axis. Subjectscan stand with platform and visual surround kept fixed, or witheither servo-controlled to match lower body motion (using ankleangle as a reference). These are referred to as visionsway-referenced and platform sway-referenced conditions, wherevisual or ankle proprioceptive feedback is rendered inaccurate.Combinations of these conditions (also with eye closure) test abilityto integrate sensory data under demanding situations. The sensorycondition number is referred to with the variable κ = 1, 2, . . . , 6.

diminishes accuracy of visual sensing. Combinationsof normal and sway-referenced vision and eyes closed(summarized in table 1), along with a fixed and sway-referenced support surface, are employed to test humanresponse to altered sensory conditions. Subjects with absentvestibular function are unable to balance when platform sway-referencing is combined with either eyes closed or vision sway-referencing. However, the SOT appears to be less sensitive indiagnosing more subtle balance deficits, and test results offerlimited insight to the cause of instability (Mirka and Black1990). It is possible that models of balance control mayaid interpretation of results. The present analysis models amodified version of the standard SOT (Speers et al 1999),in which ankle and hip joint kinematics are sensed directlyrather than inferred from force plate measurements, as is theconvention.

3. Model of state estimation and control of balance

The proposed model is based on several distinct componentswhich are assembled into a single state space system. It makesuse of a simplified model of human body dynamics, alongwith linear models of three sensory modalities: proprioception,vestibular organs and vision. The system combining thesemodels is then controlled using state feedback, designed withlinear quadratic regulator optimal control. A parametrizationscheme makes it possible to describe the objective functionwith only two parameters and yet describe a wide range of

postural behaviors. The state information for the resultingfeedback is provided by a state estimator, whose behavioris determined by parameters describing sensor precision andother noise characteristics. These estimation parameters aredifficult to determine independently, but can be constrainedthrough observation of experimental behaviors. The overallsystem can then be altered to model the effects of visual andplatform sway-referencing.

3.1. Body dynamics model

Human body dynamics are modeled as a two-segment invertedpendulum. Segment lengths and inertial properties are chosento approximate those of a typical human (Kuo and Zajac1993a). The two degrees of freedom correspond to motionabout the ankles and hips in the sagittal plane (no side-to-side motion, and relatively small knee motion). Thestate-space representation will use states based on segmentcoordinates φ1 and φ2, defined as the angle of the lowerleg and trunk measured counter-clockwise with respect tothe vertical, respectively. This model assumes that lower-level circuits translate commands coding desired movementinto motor signals to the muscles. The command inputsmay therefore be expressed in terms of two-dimensional jointmotion, rather than the higher-dimensional motor signals.However, constraints on muscle force are still incorporatedindirectly by calculating the set of feasible accelerationsof a model including fourteen muscle groups (Kuo andZajac 1993a, 1993b), and then performing a reduced-orderapproximation to that set (Kuo 1995). The result is a simplersystem that still models the mass and inertial coupling of thebody. Its state equations are in standard form,

xB = ABxB + BBucontrol (1)

where the state is defined as xB�= [φ1 φ2 φ1 φ2]T, and the

subscript ‘B’ denotes the body. The system matrices are

AB�=

[0 I 2×2

G 0

], BB

�=[

0H

],

where G and H are matrices summarizing the inertial properties(see the appendix for numerical values), I 2×2 is the 2 × 2identity matrix, and ucontrol corresponds to two directions ofmotion, scaled to equal command effort.

3.2. Sensor models

The following sensors are to be modeled: ankle and hipproprioception, visual rotation and translation, and vestibularrotation and translation. For each type of sensor, a simplelinear model is employed, presented in transfer function formfor simplicity. For interfacing with body dynamics, however,each transfer function is realized in a state-space form,

xs = Asxs + Bsus (2a)

ys = Csxs + Dsus (2b)

where ys is the sensory output, us the stimulus input, xs thesensor states, and ‘s’ is replaced by a specific subscript for eachsensor. System matrices {As , Bs , Cs , Ds} denote a state-space

S238

An optimal state estimation model of sensory integration

realization equivalent to that sensor’s transfer function. Thefollowing two-letter subscripts will be used to refer to eachsensory modality: ‘ap’ for ankle proprioception, ‘hp’ for hipproprioception, ‘sc’ for semi-circular canals, ‘ot’ for otoliths,‘vr’ for visual rotation, and ‘vt’ for visual translation.

Muscle spindles are modeled as simple lead-lag filterswhich provide a signal proportional to length and velocitywithin a limited bandwidth. The transfer function from lengthchange to spindle output is (Agarwal and Gottlieb 1984)

Yspindle

Xlength= Tsps + 1

αTsps + 1, (3)

where α and Tsp are lead-lag constants (see the appendix fornumerical values). Two sets of muscle spindles are included, atthe ankles and at the hips. The muscle moment arm about thesejoints is assumed to be nearly constant around the operatingpoint, adding a proportionality factor which is normalizedto one. A state-space realization of equation (3) for eachproprioceptor is given by the (1 × 1) matrices

Aap�= Ahp

�= −1

αTsp, Bap

�= Bhp�= 1

αTsp,

Cap�= Chp

�= α − 1

α, Dap

�= Dhp�= 1

α,

with the inputs to the sensors set to the ankle and hip jointangles,

uap = φ1, uhp = (φ2 − φ1). (4)

Each realization may be verified to be equivalent to the transferfunction of equation (3), through the transformation

Ys

Us

= Cs(sI − As)−1Bs + Ds (5)

where I is the identity matrix.Semicircular canals have an overdamped response to

angular acceleration, which is detected by hair cells whosefiring rate is roughly proportional to angular velocity.Fernandez and Goldberg (1971) proposed a linear model withtransfer function from head angular acceleration to neuralfiring rate. We modified their model to include a limitation onbandwidth, both to constraint high-frequency response and toensure the overall system’s causality:

Ysc

Xangacc= kscs (s + ωs1)

(s + ωs2) (s + ωs3), (6)

where ksc, ωs1, ωs2 and ωs3 are constant parameters (see theappendix for numerical values of constants). The canals arearranged in pairs oriented in mutually orthogonal directions,none of which are aligned with head rotation in the sagittalplane. However, because vestibular processing has beendemonstrated to perform coordinate transformations (Merfeldet al 1993), a single pair of canals is incorporated in the presentmodel, with state-space matrices {Asc, Bsc, Csc, Dsc} and input

usc = φ2. (7)

The otoliths respond to translational acceleration bydetecting deflection of hair cell mounted with massive crystals.Young and Meiry (1968) found a linear transfer function

Yot

Xlinacc= kot (s + ωo1)

(s + ωo2), (8)

where kot, ωo1 and ωo2 are constant parameters (see theappendix for numerical values of constants). Though otolithsare found in two locations, the utricle and saccule, on eachside of the head, a single pair is modeled here, detecting onlyfore-aft linear translation of the head, which is in turn fixedrigidly to the trunk. The corresponding state-space matricesare {Aot, Bot, Cot, Dot} with input

uot = ct1φ1 + ct2φ2 (9)

where cTt

�= [ct1 ct2] transforms the angular accelerations intolinear translation of the head (see the appendix for numericalvalues of constants).

The eyes respond to both visual rotation and translation(Robinson 1981). Many of the cues contributing to self-motion come from peripheral vision, and much of the signalprocessing necessary to detect motion is performed in theretinal circuitry. Further processing is performed by ganglioncells and the visual cortex, with a large number of connectionscontributing to significant phase lag. A simplified modelfor visual signal processing includes detection of velocityof sagittal plane rotation and of fore-aft translation, with abandwidth limitation:

Yv

Xv= 1

Tvs + 1. (10)

Tv = 0.1 s is based on visual processing lag (Robinson 1981).The phase lag for this transfer function is likely more of morebehavioral significance than the magnitude. The velocity inputfor rotation and translation is head (fixed to trunk) angularvelocity and head fore–aft translational velocity, with resultingstate space matrices {Avr, Bvr, Cvr, Dvr} and {Avt, Bvt, Cvt, Dvt}and inputs

uvr = φ2, uvt = ct1φ1 + ct2φ2. (11)

The individual component models are assembled into acomprehensive system, describing the combined dynamicsof the body and sensors. Body and sensor dynamics(equations (1), (4), (5), (7), (9), (11) and (12)) togetherform the system matrices ABS, BBS, CBS and DBS, wherethe subscript ‘BS’ refers to the body and sensors, and the state

vector is defined as xTBS

�= [xT

B xTap xT

hp xTsc xT

ot xTvr xT

vt

].

This system is subject to process noise w, and the sensors tomeasurement noise v, both modeled as zero-mean Gaussianwhite-noise processes. Process noise encompasses unmodeleddynamics, random external perturbations to equilibrium, andinternal effects such as random fluctuations in muscle force,all acting at relative levels similar to that of ucontrol. Becausethe body dynamics act as a dominant low-pass filter, it isunnecessary to impose a bandwidth limitation on the noiseinputs. Measurement noise describes the signal-to-noise ratiosof sensory signals, which depend on such effects as precisionof sensors, loss of fidelity due to synapses and transmissiondistance, and general information capacity in the sensorysystem. Together, these additions result in state equations

xBS = ABSxBS + BBSucontrol + BBSw (12a)

ys = CBSxBS + DBSucontrol + v. (12b)

The noise characteristics of w and v constitute the unknownparameters of the model.

S239

A D Kuo

3.3. State feedback control

State feedback gains for stabilizing the body are determined bya parametrized objective function for linear quadratic optimalcontrol (Kuo 1995). Parametrization is helpful for varyingthe shape of the state objective, in accordance with changesin behavior such as scaling of control with perturbationmagnitude (Park et al 2004). The state feedback gains forposture may generally differ from those produced by optimalcontrol, but linear quadratic control provides a means to userelatively few parameters to describe a large number of gains,and it guarantees a large degree of robustness to time delays(Kuo 1995). The quadratic objective function is

J =∫ ∞

0

(xT

BQxB + uTcontrolRucontrol

)dt . (13)

The control weighting matrix R is set to the identity matrix,because the entries of ucontrol are proportional to actual neuralcommand effort. Weighting of the state objective relative tocontrol effort is set by a single parameter, σ . The shape ofthe ankle versus hip trajectory is set by another parameter, µ

(described in detail in Kuo (1995)). The two parameters defineQ with the equation

Q = σ 2 µQcm + (1 − µ)Qup

λmax(µQcm + (1 − µ)Qup)(14)

where the function λmax (A) refers to the maximum eigenvalueof a matrix A. Parameter µ sets the joint trajectory shapeby varying the proportion of two objectives Qup and Qcm.Qup weights joint angles equally, reflecting an unbiaseddesire to maintain upright equilibrium. Qcm is a penaltyobjective modeling the relative undesirability of allowingthe body center of mass to approach the limits of the baseof support (see the appendix for numerical values). Therelative proportion of this penalty increases with perturbationsize, resulting in a continuum of behaviors approximatingthose seen experimentally (Kuo 1995). Equation (13) canbe minimized with feedback control, determined by solvingthe algebraic Riccati equation (Bryson 2002),

ATBS + SAB − SBBR−1BT

BS + Q = 0 (15)

KB = R−1BTBS. (16)

These equations are solved for KB, the matrix describingthe gains of the states to the control, and S, an intermediatequantity (sometimes referred to as the ‘cost-to-go matrix’).This feedback is applied to the entire system (12) using theaugmented matrix

KBS = [KB 0] (17)

with the feedback control given by

ucontrol = −KBSxBS. (18)

3.4. State estimator model

CNS sensory processing is modeled by an optimal stateestimator, which estimates the full state xBS of the body

and sensors for use in the feedback control. The estimateis produced by an internal model of the body and sensorydynamics (figure 1(d)). This model can produce both a stateestimate xBS and a prediction ys of sensory output. Processnoise and other disturbances can cause the actual state todeviate from this estimate. This deviation can be correctedwith feedback of the sensory prediction error ys − ys , withestimator gain matrix L determining the correction’s effect oneach state:

˙xBS = ABSxBS + BBSucontrol + L (ys − ys) (19a)

ys = CBSxBS + DBSucontrol. (19b)

The principal problem in state estimation is the design ofthe estimator gains L that govern the correction of estimatexBS based on sensory feedback y. There are many criteriaby which L may be designed, with the most importantrequirement being the stabilization of the estimation errorxBS − xBS. We employ optimal state estimation, whichminimizes the mean-square error for a particular set of processand sensor noise characteristics. One advantage of optimalstate estimation is that a relatively large number of entriesin L may effectively be parametrized by a smaller numberof noise power spectral densities. Another advantage isthat minimization of mean-square error could potentiallybe the outcome of an adaptation process mediated by thecerebellum (Paulin 1989). Optimization might therefore bea physiological explanation for how humans might learn toperform state estimation by iteratively adjusting feedbackgains to minimize prediction error. In the postural controlmodel, estimator gains L are computed using the algebraicRiccati equation:

XATBS + ABSX − XCT

BSV−1CBSX + W = 0 (20)

L = XCBSV−1, (21)

where W and V are diagonal matrices containing the powerspectral densities for w and v, respectively, and X is thesteady-state estimator error covariance. These equations aresolved for X and L. With both w and v assumed to be whiteand independent with zero mean, power spectral densitiesare sufficient to fully describe the process and sensor noise.Process noise is assumed to be of equal magnitude for eachinput, so that W is set by a single parameter W1:

W�=

[W1

W1

]. (22)

V is parametrized by a set of noise-to-signal power ratiosπap, πhp, . . . (Borah et al 1988). These ratios describe powerspectral densities relative to the output covariance Y of the statefeedback-stabilized system driven by process noise alone,

Y = CBSXfbCBS, (23)

where Xfb is the state covariance, found by solving the equation

(ABS − BBSKBS)Xfb + Xfb(ABS − BBSKBS)T+ BBSWBT

BS = 0.

(24)

S240

An optimal state estimation model of sensory integration

The sensor noise power spectral densities are then defined as

V�=

πapY11

πhpY22

πscY33

πotY44

πvrY55

πvtY66

.

(25)

The state feedback and estimator are combined into alinear quadratic Gaussian (LQG) system. The parameters thatdetermine the behavior of this system are

πT �= [σ µ W1 πap πhp πsc πot πvr πvt

],

so that two parameters describe the control feedback, andthe remaining seven parameters describe process and sensornoise. The stochastic nature of this system also means thatthe closed-loop system is not designed to predict explicit jointtrajectories. Rather, the model is used to predict the covarianceof joint motion, XLQG about upright equilibrium. Estimatorstate covariance X is found by solving the equation

(ABS − BBSKBS)X + X(ABS − BBSKBS)T + LV LT = 0 (26)

and XLQG is the sum of estimator state and error covariances,

XLQG = X + Xe. (27)

Of these covariances, only those corresponding to xbody can bemeasured experimentally, and the preferred coordinate systemfor comparisons is

xθ ≡[θshk

θhip

]= cθxBS =

[−1 1 0 · · · 00 1 0 · · · 0

]xBS (28)

where θshk and θhip are the shank and hip angles, respectively(Kuo et al 1998). The shank angle is defined as the angleof the shank relative to the support surface, and hip angle asthe angle of the trunk with respect to the leg (see figure 3),both increasing in the extension direction. The correspondingcovariance is

XLQGθ = cTθ XLQGcθ . (29)

3.5. Models of sensory conditions

Altered sensory conditions imposed by the sensoryorganization test affect the behavior of the system significantly,and are modeled by making changes to the nominal stateequations. These changes may also be accompanied bychanges in the state estimator gains, depending on the subject’sawareness of those changes. We assume that when subjectsconsciously close their eyes, they are aware that visualinformation is no longer available, and may make adjustmentsto how the remaining sensory inputs are weighted. We modelthis as a new set of state estimator gains L. However, inthe sway-referencing conditions, we assume that subjectsare unaware that sensory information has been renderedinaccurate, and use the nominal set of state estimator gains. Wetherefore model the inaccuracy induced by sway-referencingwith appropriate changes to the state equations, but with nochanges to the estimator gains L.

The eyes closed condition is modeled by removing thevisual states xvr and xvt, and their associated outputs. This is

0.15

0.15

θshk

θhip

Figure 3. Covariance ellipse summarizes amount of sway aboutankles and hips during upright standing. Motion of the body ismeasured in terms of angle of the shank with respect to supportsurface normal, θshk, and motion of the trunk with respect to theshank, θhip. A typical trace of θhip versus θshk is shown during quietstanding, under normal sensory conditions. The covariance matrixdescribes the variances in the hip and shank individually, along withthe coupling between the two joints. A single ellipse, representingone standard deviation of sway, summarizes the entire covariance.

accomplished by the equivalent action of setting the noise-to-signal parameters πvr and πvt to infinity, and then re-computingthe state estimation gains accordingly. Closing the eyes is theonly condition in which the subject is explicitly aware of achange in sensory conditions, and so the estimator gains arerecalculated taking this into account.

Visual sway-referencing takes place without the subject’sawareness, with the assumption that the CNS does not alterits control behavior under this condition. However, theeyes receive inaccurate motion cues, modeled as separateeffects on visual rotation and translation. Ankle-driven sway-referencing only partially affects visual rotation cues, becausethe hips induce significant motions of the upper body thatare not countered by the sway-referencing servo controller.In contrast, visual translation cues are affected to a largerdegree because translation of the head is dominated more by(inaccurate) ankle motion than by hip motion (see figure 2).The visual translation sensory input is therefore modeled ashaving zero gain, whereas the visual rotation input is modeledas feedback of hip motion alone, with the ankle-drivenmotion of the visual surround removed. For this condition,equations (11) are replaced by

uvr = φ2 − φ1, uvt = 0. (30)

Lack of subject awareness is modeled by retaining the originalstate estimator for the intact system, so that the weightingsof visual rotation and translation are unchanged. The stateestimate therefore depends on the inaccurate information tothe same degree as if the visual inputs were accurate.

Platform sway-referencing also takes place without thesubject’s awareness, and is modeled by a feedback controlperformed on the support surface under the feet. Motion ofthe support surface, φp, not only acts as an inaccurate reference

S241

A D Kuo

for ankle proprioception, but also acts as a small disturbanceto the body. The platform dynamics are modeled as a servo-driven second-order system,

φp = −kp(φp − φ1) − bpφp, (31)

with gains kp = 400 s−2 and bp = 32 s−1 chosen to make theplatform much faster than the body dynamics. These dynamicsare augmented to the regular state equations (12), and thedisturbance effect is computed by deriving the body dynamicsincluding an extra degree of freedom for the platform. Underplatform sway-referencing, equation (1) is altered to includethis disturbance:

xB = ABxB + BBucontrol + Bpφp (32)

where Bp describes the effect of platform acceleration onthe body, derived from the dynamical equations of motion.As with visual sway-referencing, platform sway-referencingonly affects the body dynamics, feeding back inaccurateinformation to an estimator that was designed for the original,intact system.

These sensory conditions make the overall behavior ofthe closed-loop system a function of not only the nominalparameters π but also the sensory condition number κ .We denote this function XLQG θ (κ; π), specifically excludingindependent anthropometric and sensory parameters thatremain relatively fixed. There are nine degrees of freedomin the free parameters π , of which two—the state feedbackparameters σ and µ—are determined by separate perturbationexperiments. The remaining parameters in π describe noisecharacteristics and are therefore determined from experimentalobservations of stochastic upright balancing.

4. Comparisons with experimental data

The model was tested by comparing theoretical joint anglecovariances under altered sensory conditions against thosemeasured experimentally. Some unknown parameters ofvector π were necessarily calibrated using these data; themodel was therefore not purely predictive. However, thelarge number of degrees of freedom in the possible observablebehaviors, compared to the smaller number of free parameters,means that data could constrain the model to a considerableextent. The most meaningful tests were to determine whetherthe model successfully predicted trends in behavior as afunction of sensory condition, and how sensitive this samemodel was to variations in parameter values.

4.1. Comparison with healthy young adult subjects

Experimental data were recorded from sensory organizationtests performed on eight subjects, in order to computeempirical joint angle covariances using a protocol describedpreviously (Kuo et al 1998). Subjects were young (age < 45years), healthy normal adults. One trial each of conditions1 and 2, and three trials of conditions 3 through 6, wereperformed. Recorded data included shank and hip kinematics(θshk and θhip, see figure 3) and ground reaction forces, allsampled at 50 Hz. Trials lasted 21 s, with 1 s at the beginningdiscarded to reduce transients. Data were band-pass filtered

using a third-order digital Butterworth filter, with high-passcutoff of 0.5 Hz and low-pass cutoff at 3.5 Hz. The high-passcutoff was used to filter out very slow drift, so that covarianceswere computed on at least 10 cycles of the lowest remainingfrequency components. The low-pass cutoff was employedto filter out noise. Sway-referencing commands were basedon feedback of the shank angle. For simplicity, samplecovariances were displayed for only θshk and θhip, forming asubset of the predicted covariance matrix without the angularvelocities. Mean covariances across all subjects were used forXexp θ (κ). Details of the experimental procedure are given inKuo et al (1998). A sample trace of θshk and θhip, along withthe covariance ellipse, is shown in figure 3. The horizontalextent of the ellipse corresponds to θshk variance, the (1, 1)element of Xexp θ (κ), and the vertical extent to θhip variance,the (2, 2) element of Xexp θ (κ). Covariance of θshk and θhip, theoff-diagonal element, is distinguished by a tilt to the ellipsewith respect to vertical.

The feedback control system parameters were the moststraightforward to determine. The state objective parameterµ determines the relative contribution of hip motion toposture control, with a value near 1 for responding to largedisturbances (Kuo 1995). In quiet standing, there is relativelylittle hip motion, with µ approaching zero. Because thesensory organization test performs no explicit perturbationsand subjects perform quiet standing in all conditions, we setµ = 0. Process noise covariance W1 was set to 0.08 and σ

was set to 2.5 in order to match the magnitudes of covariancefor κ = 1. These parameters were then kept constant for allcomparisons.

The remaining parameters were associated with sensornoise-to-signal ratios. Manipulation of the sensor parametersresulted in a relatively narrow range of behaviors for somesensory conditions despite extreme variations in parameters.Indeed, such robustness enabled the determination of σ , µ andW1 prior to setting the other parameter values. Conditionsκ = 5 and κ = 6 were relatively the most sensitive to theseremaining parameters, and were used as the basis for findingthe following values, which produced a relatively good matchto experimental data:

πap = 0.05, πhp = 0.01,

πsc = πot = πvr = πvt = 0.001.(33)

These parameter values, along with the sway-referencingmodels, together predict a single set of sway covariancesfor all six SOT conditions (see figure 4). State estimationmakes the system stable for all conditions despite removalor inaccuracy of some sensory information. For example,the model treats the eyes closed (κ = 2) condition as areduction in sensory channels that results in slightly greatersway. Interestingly, the visual sway-referencing (κ = 3)condition has less effect on sway, because the model allowssome useful motion cues to be detected in that condition.If visual motion could completely be removed, the modelwould predict a larger effect than for eyes closed. The modelis also relatively unaffected by platform sway-referencing(κ = 4), indicating that sufficient information is available fromother sensors to permit precise stabilization. However, when

S242

An optimal state estimation model of sensory integration

UNSTABLE UNSTABLE UNSTABLE

alteredcontrol

shk

hip1

4

2

5

3

6

0.1

Young AdultsDirect Feedback Model

(b) Direct Feedback Model of Normal Posture

shk

hip1

4

2

5

3

6

0.1

Young AdultsState Estimator Model

(a) State Estimator Model of Normal Posture

Figure 4. Comparison of experimental results from normal young adults (N = 8) against (a) state estimator and (b) direct feedback models,for six sensory conditions. Model covariance ellipses were computed for a consistent environment of process noise and measurement noise.The eyes closed condition (κ = 2) was modeled by removal of vision, and the vision sway-referenced condition (κ = 3) by reducing thefeedback from visual translation and rotation, as if the visual surround were driven by ankle motion. Platform sway-referencing (κ = 4–6)was modeled by applying a servo command to the support surface, also driven by ankle motion. (a) State estimator model produces changesin covariance similar to normals as a function of sensory condition κ . Both model and experiment show small increases in sway inconditions where only one type of sensory feedback is affected (κ = 1–4), with the most sway when two types are affected (κ = 5, 6). Allparameter values are kept fixed when computing model covariances for different sensory conditions. In the eyes closed condition (κ = 2),human subjects sway less than might be expected from the loss of sensory information. This behavior could potentially be explained by achange in control strategy, modeled by increasing parameter σ (denoted ‘altered control’, ellipse with dotted line). The resulting covarianceis smaller and more similar to that observed experimentally. (b) Direct feedback model can also produce covariances similar to thoseobserved experimentally, for three of the sensory conditions (κ = 1–3). However, for platform sway-referenced conditions (κ = 4–6) thedependence on inaccurate ankle feedback causes the direct feedback model to be unstable. In contrast, both normal humans and the stateestimator model are remarkably robust to inaccurate sensory information.

inaccurate ankle proprioception is combined with eyes closed(κ = 5) or visual sway-referencing (κ = 6) conditions, themodel predicts a much larger increase in sway. This is due

to the fact that accurate information is received only throughhip proprioception and the vestibular organs. Despite thisreduction of sensory information, the model is able to robustly

S243

A D Kuo

maintain stability, with the only penalty being an increase insway.

These model’s sway covariance ellipses compare wellwith those measured in healthy young adults (see figure 4(a)).The state estimator model produces behaviors that arequalitatively similar to how humans compensate for differentsensory conditions. The model agrees with experimental datashowing that covariance for κ = 3, visual sway-referencing,is similar to that for κ = 1. The data also show a relativelysmall increase in hip sway under platform sway-referencing(κ = 4), as compared to normal support surface (κ = 1). Thelargest disagreement between model and data was for eyesclosed (κ = 2), where the model predicted approximately50% higher hip sway than seen experimentally. This may bedue to the fact that the model employs the same state feedbackgains KB regardless of sensory condition. Human subjectsmay, in contrast, employ higher gain when their eyes areclosed. Indeed, this same behavior could easily be produced byincreasing control parameter σ for this condition (see ‘alteredcontrol’ in figure 4(a)).

4.2. Comparison with direct feedback model

An alternative to state estimation is for each sensory channel tobe filtered independently, and for all such signals to be directlyapplied to feedback control without passing through an internalmodel (figure 1(c)). Direct feedback can include pathways foreach sensory signal to contribute to control of each joint ormuscle, through a matrix of feedback gains. However, eachsensor’s feedback is taken literally, rather than corroboratedagainst an internal representation of the body’s state. One ofthe arguments for state estimation is that such corroborationcan make the posture control system more robust to inaccuratesensory information.

We tested the effect of altered sensory conditions on amodel of direct feedback (see figure 4(b)). We used the samefeedback control system as for the state estimation model, butassumed that sensory dynamics are inverted perfectly to yielddirect measurements of the states. For ankle proprioception,the muscle spindle dynamics were assumed to be filteredto yield shank angle and velocity. These were then feddirectly into the control gains K, without need for a stateestimator. Under ideal conditions, the resulting system hasidentical performance to the state estimator-based control.However, under altered sensory conditions, direct feedbackand estimator-based feedback predict very different behaviors.

Sway-referencing of the platform support surface (κ =4–6) was found to immediately destabilize the direct feedbacksystem. With no feedback from other sensors to indicateankle motion, the system cannot function with inaccurate ankleproprioception. In fact, the system becomes stable even fora 50% reduction in ankle gain by platform sway-referencing.This is because direct feedback takes each sensory channelliterally as an indicator of joint or limb position, so thatplatform sway-referencing has an effect equivalent to reducingthe feedback gains associated with ankle position and velocity.

This high sensitivity to inaccurate sensing is in starkcontrast to state estimation. To an internal representation of

body and limb dynamics, ankle proprioception is only oneof many channels contributing to an estimate of the overallorientation of the body and limbs. Even a 100% reductionin ankle information results in a much smaller reduction inthe internal model’s estimate of ankle state, and thereforedoes not stabilize estimator-based feedback. Normal humansubjects exhibit a similar degree of robustness to alteredsensory conditions (see figure 4(b)). Even though both modelsuse the same control gains K, the direct feedback model lacksthe robustness of human subjects or the state estimator model.

4.3. Parameter sensitivities

We also considered the effect of parameter variations onmodel behavior, by perturbing each sensory noise parameterand then recalculating the state estimation gains and SOTsway covariances (see figure 5). Each sensory parametervariation was perturbed by changing the noise-to-signal ratio.Expressed in units of decibels (dB) of signal-to-noise ratio,−10 log10 πsensor, a perturbation of +10 dB in signal to noisewas produced by decreasing the noise-to-signal ratio by afactor of 10. The model predicts that even extreme changesin an individual sensor’s precision can have very small effecton normal sway (κ = 1), because the state estimator is ableto integrate useful information from the other sensors. Thisremains the case even with eyes closed (κ = 2), with mostparameter variations resulting in sway changes of only a fewper cent. The most sensitive parameter under this conditionwas otolith signal-to-noise ratio, where a 10 dB decreaseproduced a 21% increase in hip sway. This indicates thatthe model places especially greater reliance on the otolithswhen visual information is lacking. Visual sway-referencingalone (κ = 3) is also predicted to have low sensitivitiesto parameter changes, but with one exception: the systembecomes unstable when ankle proprioception signal to noiseis reduced. The model compensates for poorer proprioceptionby placing greater weight on vision, which in this case isinaccurate. This is a recurring theme, in which the modelplaces lower weight on poorer sensors, and greater weight onthe most precise sensors, with the effects of these reweightingsmanifested most significantly when sensory conditions arealtered.

The model exhibits a larger range of parametersensitivities for conditions where the platform support issway-referenced. Many parameters changes have little effect,producing sway changes of a few per cent. With normalvision (κ = 4), the only sensitivities with more than 20%change in sway were for ankle and hip proprioception, andvisual translation. The ankle’s sensitivity is such that adecrease in signal-to-noise ratio actually results in decreasedsway, because the estimator reduces weight on the inaccurateinformation. This also produces the corollary effect ofincreased difficulty balancing on the sway-referenced platformwith more precise ankle proprioception. The hip and visualtranslation sensitivities are of the opposite nature, wheredecreases in signal-to-noise result in increases in sway.When these two modalities are less effective, the modelplaces relatively greater reliance on ankle feedback. Such

S244

An optimal state estimation model of sensory integration

shk

hip1

4

2

5

3

6

0.25

(d) State Estimator Model Covariances

12

34

56

- - - - - -

+ + + + + +

ap hp sc ot vr vt1

23

45 6

0.0

1.0

2.0

3.0

4.0

5.0

Rel

ativ

e va

lue

- - - - - -

+ + + + + +

ap hp sc ot vr vt1

23

45

6- - - - - -

+ + + + + +

ap hp sc ot vr vt

Rel

ativ

e va

lue

Rel

ativ

e va

lue

Parameter Parameter Parameter

(a) Ankle variance (b) Ankle-hip covariance (c) Hip variance

0.0

1.0

2.0

3.0

4.0

5.0

6.0

0.0

1.0

2.0

3.0

4.0

5.011.1 6.4

13.613.0

9.7

Figure 5. Parameter sensitivity study of state estimator model, showing changes in sway covariance as a function of perturbation parameterand SOT condition (κ). Each sensor was individually perturbed with a 10 dB decrease (denoted by ‘–’) or increase (‘+’) in signal-to-noiseratio. The state estimator model was used to optimally reweight sensors for each variation, and then predict the sway covariance underidentical applied process noise. Each bar shows the resulting change in the covariance matrix: (a) ankle variance, (b) ankle–hip covariancemagnitude, and (c) hip variance, relative to the nominal values of figure 4. All covariances are also shown as (d) ellipses as a function ofsensory condition. The state estimator model exhibits high robustness, remaining stable for nearly all parameter values, although withgreater sway under the most challenging sensory conditions. The model was highly insensitive to sensor precision in normal (κ = 1) andvision sway-referenced (κ = 3) sensory conditions. For conditions with no vision (κ = 2) or platform sway-referenced (κ = 5), there werelarger insensitivities to select sensors, although sway remained small for most parameter variations. In conditions combining platformsway-referencing with eyes closed (κ = 5) or vision sway-referenced (κ = 6), most variations in sensor parameters resulted in greatlyincreased amounts of sway. Parameter variations caused instability in only one case, κ = 3 when ankle proprioception signal-to-noise ratiois reduced (ap -); no covariance is shown for this case.

reweightings become more important when vision is removed(κ = 5), where a 10 dB decrease in hip proprioception causesa greater than five-fold increase in sway.

There are even greater sensitivities when visual andankle proprioceptive sensors are both sway-referenced. Ofparticularly note is an instability when ankle signal to noiseis decreased (κ = 6), due to the profound lack of accuratefeedback. Better stability, but still high sensitivities, arealso predicted for hip proprioception and visual translationand rotation, where decreases in signal-to-noise ratios causevery substantial increases in sway. Again, this is due to theestimator’s compensatory strategy of placing greater weight

on the remaining sensors, which do not necessarily receiveaccurate information.

These highest sensitivities are, however, very much in theminority. Of the 216 total parameter variations (figure 5),only 40 (19%) cause more than a 20% increase or decrease insway. For the majority of cases, there is very little sensitivityto ten-fold changes in sensor precision. This relatively smallrange of possible behaviors demonstrates constraints posed bythe model; parameters cannot be used to attain an arbitrarilyprecise post hoc fit to experimental data. It also suggestswhy normal conditions of eyes open or eyes closed (κ =1 or 2) might be poor indicators of sensorimotor function

S245

A D Kuo

shk

hip1

4

2

5

3

6

0.25

Older Adult Subjects (N = 15)

Figure 6. Experimental covariance data from healthy older adults (N = 15). Older adults showed very small increases in sway for the sameconditions (κ = 1 and 3), and intermediate amounts of variation under the other two conditions (κ = 2 and 4) where only one sensorychannel is missing or inaccurate. Many subjects also swayed a great deal in the last two conditions (κ = 5 and 6). The state estimator modelalso exhibited similar sensitivities in response to changes in sensor precision (signal-to-noise ratios, see figure 5(b)). Sensory and posturecontrol degradation with age may potentially be modeled by decreases in signal-to-noise ratios in one or more sensors.

(Peterka and Black 1990), because a state estimator cansuccessfully integrate information from the remaining sensorychannels.

4.4. Comparison with older adult subjects

Variations in signal precision might serve as a simple modelfor sensory degradation during aging. Sensors degrade inmany different ways with age, and by differing amounts perindividual (e.g., Alexander 1994). The parameter study aboveonly performed perturbations to a single sensor at a time,whereas the effect of aging is likely to affect multiple sensorssimultaneously. However, the general sensitivity of the model,for a variety of parameter perturbations, might exhibit a similartrend of behaviors as observed in a variety of older adults.To make this comparison, we examined the parameter studycovariance ellipses, along with those measured experimentallyin a group of healthy older adults (aged 60–69 years, Speerset al 2002).

A visual inspection of covariances (figure 6) revealssimilar trends in behavior as a function of sensory condition.The model is least sensitive to parameter changes in theconditions with normal support surface (κ = 1–3), as are theolder adults. Of these three conditions, eyes closed (κ = 2)produces the most sway in both model and experiment. Inthe platform sway-referenced conditions (κ = 4–6), the modelexhibited larger sensitivities, especially with eyes closed orinaccurate vision. The human subjects generally swayedfar more than the model, due to the model variations beingrestricted to single sensors. The sway observed in the modelcould be increased by changes of greater than 10 dB inindividual sensors, or by simultaneous changes in multiple

sensors. However, even with individual sensor variations,the model appears to exhibit broadly similar sensitivities toinaccurate or missing sensory information.

4.5. Comparison with bilateral vestibular loss subjects

An additional calculation was done to model response tobilateral vestibular loss, in which pairs of both canal andotolith sensors are completely dysfunctional. We modeled thisdysfunction by decreasing the canal and otolith signal-to-noiseratios by 80 dB, effectively removing any useful informationfrom these sensors. Recalculating the state estimation gainsand the resulting covariances, the model predicts remarkablysmall increases in sway for four SOT conditions (κ = 1–4).As long as either vision or proprioception remains accurate,the model remains stable, with the eyes closed (κ = 2)producing more sway than the others. This appears to bedue to the model’s greater dependence on otolith sensors witheyes closed, which was manifested (in the normal parametersensitivity discussion above) as an increase in sway with10 dB decrease in otolith signal-to-noise (see also figure 7).Extrapolating this same change to an 80 dB decrease, the resultis still greater amounts of sway. However, the remaining twoconditions (κ = 5, 6) were affected to a much larger degree,with the model becoming unstable. In the absence of vestibularfeedback, with inaccurate ankle proprioception and inaccurateor missing vision, there is simply insufficient data for thesystem to be stable.

We compared this model against sway covariancesmeasured from three subjects with bilateral vestibular loss.These subjects were measured under the same protocol asfor the older adults (Speers et al 2002). They were able

S246

An optimal state estimation model of sensory integration

shk

hip1

4

2

5

3

6

0.1

Vestibular Loss SubjectsState Estimator Model

State Estimator Model of Vestibular Loss

UNSTABLE UNSTABLEUNSTABLE UNSTABLE

Figure 7. Comparison of sway covariances predicted by state estimator model with results from subjects with bilateral vestibular loss(N = 3). Vestibular loss was modeled by removal of semicircular canal and otolith feedback. Despite this change, the model was stable forthe first four SOT conditions, producing substantial increases in sway only in the eyes closed (κ = 2) and platform sway-referencing (κ = 4)conditions. However, the model became unstable for the remaining conditions, where platform sway-referencing was combined with novision (κ = 5) or inaccurate vision (κ = 6). These results compared well with larger populations of human bilateral loss subjects, for whomthe same two conditions consistently produce instability (Mirka and Black 1990).

to balance well under four SOT conditions (κ = 1–4), withmodest increases in sway. However, as is typical of vestibularloss patients (Mirka and Black 1990), they were unstable underthe conditions with altered feedback from vision and ankleproprioception (κ = 5, 6). The model agreed well with theseresults, both in terms of the amount of sway in the first fourSOT conditions, and the instability in the last two conditions.The model displays a similar degree of robustness as vestibularloss patients to loss of one or two types of feedback, anda similar degree of instability with loss of three types offeedback.

4.6. Summary of comparisons

It is concluded that the model is able to reproduce much ofthe behavior seen experimentally in humans. It predicts highrobustness to sensor degradation under normal conditions, andonly slightly lower robustness when two or more sensors arerendered inaccurate. It also predicts other general changesin sway, in response to alterations in sensor information, thatare consistent with experimental evidence for older adults andbilateral vestibular patients. These results are conservativelyproduced with only a single set of control parameters across allconditions, rather than by exploiting all of the model’s degreesof freedom. It is likely that greater flexibility in parameterscould improve predictions further. For example, the modelresponded to closure of the eyes with a larger increase insway than exhibited by human subjects. But decreased swaycould potentially be explained compensating for lack of visionby increasing the feedback control gains, thereby ‘stiffening’the system. As discussed below, these behavioral predictionshave certain implications regarding the organization of sensoryprocessing in the CNS.

5. Biological implications

The proposed state estimator model for sensory integrationhas implications regarding the neural substrate for movementcontrol, and for further experimentation of movementdisorders. A neural implementation of state estimationwould be expected to exhibit a convergence of multisensoryinformation, as well as temporal filtering. Convergenceis, however, not uniquely required by state estimation,because even direct feedback requires convergence of signalsfrom many sensors. Human posture control appears tobe heterogenic, with feedback from throughout the bodycontributing to the torque at any single joint (Barin 1989,Camana et al 1977, Speers et al 2002). This feedbackproperty is due to the coupled dynamics of the joints.The same coupling implies that state estimation will alsoexhibit convergence. These two types of convergence havedistinct but subtle differences. For state feedback, each jointtorque will be correlated with multiple joint states, providedsensory information is accurate. The convergence in stateestimation is most easily demonstrated through inaccuratesensory information, as was the case with platform sway-referencing above. These behavioral differences do not,however, lead to useful anatomical predictions. Sensoryconvergence is a common property of the spinal cord andhigher levels of the CNS, and the state estimator model does notnecessarily localize any of this convergence for the purposesof an internal model. Nor need state estimation be functionallyconfined to postural control alone. Feedback control of othermovements such as locomotion may also require knowledgeof state information, and estimation circuitry could be used formultiple purposes.

S247

A D Kuo

The requirement for temporal filtering implies that theCNS may process sensory information through a set ofdynamics that have similar time scales to the body andsensors. Neuronal dynamics are typically on the order ofmilliseconds, whereas the body has time constants on theorder of seconds. A CNS implementation of state estimationmust therefore assemble neurons into networks that operateon far longer time scales than a single neuron’s. Suchslow dynamics have long been observed in central patterngenerators for locomotion, and in the velocity storage systemfor visual–vestibular interactions (Robinson 1981). It istherefore plausible that similar dynamics could perform stateestimation. Indeed, we have proposed state estimation as aninterpretation of the central pattern generator (Kuo 2002).

The internal model for state estimation also implies theneed for a mechanism for adaptation. Body and sensordynamics change throughout life, and an internal modelwould have to be continually updated with these changes.Such adaptation would require feedback, but in an outerloop operating at much slower time scales than posturecontrol. Physiological evidence implicates the spinal cord andbrainstem in feedback control of postural and other automaticmotor tasks, and the cerebellum in learning outside of thefeedback loop. The cerebellum receives a wide array ofsensory data, as well as efference copy, through the mossyfiber network (Ghez 1991). It also receives error signals whichappear to code differences between expected and actual input.The cerebellum does not necessarily participate in feedbackcontrol directly, but it does modify lower-level behaviors withmechanisms such as long-term depression. These could beinterpreted as a substrate for adaptive state estimation (Paulin1989). However, the present state of knowledge is insufficientto speculate beyond such an interpretation.

Aside from these neurophysiological implications, themodel may also have value as a diagnostic tool. While thesensory organization test is a reliable means of diagnosingvestibular disorders, it is less sensitive for detecting systems-level balance dysfunction in subjects suffering from otherdisorders, such as partial loss of proprioception or other senses.The proposed model could be used to predict the level ofperformance to be expected given certain degrees of sensoryloss, and to predict the critical points at which such loss couldresult in serious instability. For example, the elderly may sufferfrom loss of balance due to deterioration of multiple sensors.Used in conjunction with the model, the sensory organizationtest could therefore potentially be used to detect deteriorationat early stages and to specify appropriate rehabilitation orintervention procedures prior to loss of mobility.

6. Conclusion

A combined state estimator and state feedback controllerreproduces and predicts the statistical behavior of quietstanding under altered sensory conditions. This model impliesthat the CNS forms its feedback control with an awarenessof the dynamics of the system and the costs associated withcertain movements. It also appears to use the equivalent ofan internal model to process sensory information and form an

estimate of the body state, which is then used in feedback.Such processing optimally integrates multiple sensory signalsand minimizes the effects of noisy disturbances and imperfectsensors. As verified by experiment, the model predicts thatloss of any single sensory modality has little effect on balancecontrol. However, postural sway increases significantly whentwo of the three main modalities are disrupted. The ability topredict deterioration in performance as a function of sensoryfunction may improve the utility of clinical balance tests andhelp diagnose contributors to balance dysfunction.

Acknowledgments

This work was supported in part by NIH grants DC6466and DC02312, and the Claude Pepper Older AmericansIndependence Center at the University of Michigan.

Appendix. Details of estimator model

Numerical values are given for model parameters below. Thebody dynamics are defined in terms of matrices

G�=

[gT

1- - -gT

2

]=

[26.64 −13.70- - - - - - - - - - - - -−46.61 44.04

],

H�=

[hT

1- - -hT

2

]=

[0.048 −0.132- - - - - - - - - - - - - -−0.084 0.3540

],

the rows of which are to be referred to below as gT1 , gT

2 , hT1 and

hT2 . We also define a matrix F, which will be used to convert

between joint angles (θshk and θhip) and segment angles (φ1

and φ2):

F�=

[f T

1- - -f T

2

]=

[1 0- - - - - -−1 1

].

Sensor parameters describe the dynamics of musclespindles and vestibular organs. The muscle spindle parametersinclude the lead time constant Tsp = 0.4 s, and lag parameterα = 0.15 (Agarwal and Gottlieb 1984). Semicircular canalsare parametrized by ksc = 0.574, ωs1 = 100 rad s−1, ωs2 =0.1 rad s−1, and ωs3 = 0.033 rad s−1 (Fernandez and Goldberg1971). Similarly, the otoliths are parametrized by kot = 90,ωo1 = 0.1 and ωo2 = 0.2 rad s−1 (Young and Meiry 1968).

Several other parameters couple the body and sensordynamics. Motion of the head is determined by ankle and

hip motions, with cTt

�= [ct1 ct2] = [−0.835 m −0.735 m]transforming angular joint accelerations into translationalacceleration. The platform and body are also dynamicallycoupled, and the transformation from platform accelerationto joint motion described by Bp = [−0.13 0.026]T, derivedfrom the body equations of motion (Park et al 2004).

S248

An optimal state estimation model of sensory integration

The overall system matrices are composed from theindividual body and sensory matrices:

ABS =

0 I 2×2

G 0Bapf

T1 Aap

BhpfT2 Ahp

BscgT2 Asc

BotcTt gT

1 Aot

BvrfT2 Avr

BvtcTt Avt

,

BBS =

0H

00

BschT2

BotcTt hT

1

00

,

CBS =

0 0 Cap

0 0 Chp

DscgT2 Csc

DotcTt gT

1 Cot

0 0 Cvr

0 0 Cvt

,

DBS =

00

DschT2

DotcTt hT

1

00

,

where I 2×2 denotes the 2 × 2 identity matrix.The control objective uses the following matrices

Qcm =[

0.57 0.170.17 0.05

], Qup =

[1.45 −1.18

−1.18 0.96

],

where only the upper-left (2 × 2) entries are shown, the restbeing zero.

References

Agarwal G C and Gottlieb G L 1984 Mathematical modeling andsimulation of the postural control loop: Part II. Crit. Rev.Biomed. Eng. 11 113–54

Alexander N B 1994 Postural control in older adults J. Am. Geriatr.Soc. 42 93–108

Baloh R W, Spain S, Socotch T M, Jacobson K M and Bell T 1995Posturography and balance problems in older people J. Am.Geriatr. Soc. 43 638–44

Barin K 1989 Evaluation of a generalized model of human posturaldynamics and control in the sagittal plane Biol. Cybern. 6137–50

Borah J, Young L R and Curry R E 1988 Optimal estimatormodel for human spatial orientation Representation of

Three-Dimensional Space in the Vestibular, Oculomotor, andVisual Systems vol 545, ed B Cohen and V Henn (New York:The New York Academy of Sciences) pp 51–73

Bryson A E 2002 Applied Linear Optimal Control (Cambridge, UK:Cambridge University Press) p 362

Camana P C, Hemami H and Stockwell C W 1977 Determination offeedback gains for human posture control without physicalintervention J. Cybern. 7 199–225

Fernandez C and Goldberg J M 1971 Physiology of peripheralneurons innervating semicircular canals of the squirrelmonkey: II. Response to sinusoidal stimulation anddynamics of peripheral vestibular system J. Neurophysiol. 34661–75

Ghez C 1991 The cerebellum Principles of Neural Science 3rd edn,ed E R Kandel, J H Schwartz and T M Jessel (New York:Elsevier) chapter 41, pp 626–46

He J, Levine W S and Loeb G E 1991 Feedback gains for correctingsmall perturbations to standing posture IEEE Trans. Autom.Control 36 322–32

Horak F B and MacPherson J M 1995 Postural orientation andequilibrium Handbook of Physiology (New York: OxfordUniversity Press)

Kuo A D 1995 An optimal control model for analyzing humanpostural balance IEEE Trans. Biomed. Eng. 42 87–101

Kuo A D 2002 The relative roles of feedforward and feedback in thecontrol of rhythmic movements Motor Control 6 129–45

Kuo A D, Speers R A, Peterka R J and Horak F B 1998 Effect ofaltered sensory conditions on multivariate descriptors of humanpostural sway Exp. Brain Res. 122 185–95

Kuo A D and Zajac F E 1993a A biomechanical analysis of musclestrength as a limiting factor in standing posture J. Biomech. 26(Suppl. 1) 137–50

Kuo A D and Zajac F E 1993b Human standing posture: multi-jointmovement strategies based on biomechanical constraints Prog.Brain Res. 97 349–58

Merfeld D M, Young L R, Oman C M and Shelhamer M J 1993A multidimensional model of the effect of gravity on thespatial orientation of the monkey J. Vestib. Res. 3 141–61

Mirka A and Black F O 1990 Clinical application of dynamicposturography for evaluating sensory integration and vestibulardysfunction Neurol. Clin. 8 351–9

Nashner L M 1977 Fixed patterns of rapid postural responses amongleg muscles during stance Exp. Brain Res. 30 13–24

Park S, Horak F B and Kuo A D 2004 Postural feedback responsesscale with biomechanical constraints in human standing Exp.Brain Res. 154 417–27

Paulin M G 1989 A Kalman filter theory of the cerebellum DynamicInteractions in Neural Networks: Models and Dataed M A Arbib and S-I Amari (New York: Springer)

Peterka R J and Black F O 1990 Age-related changes in humanposture control: sensory organization tests J. Vestib. Res. 173–85

Robinson D A 1981 Control of eye movements Handbook ofPhysiology: The Nervous System vol II (Bethesda, MD:American Physiological Society) chapter 28, pp 1275–320

Speers R A, Kuo A D and Horak F B 2002 Contributions of alteredsensation and feedback responses to changes in coordination ofpostural control due to aging Gait Posture 16 20–30

Speers R A, Shepard N T and Kuo A D 1999 EquiTest modificationwith shank and hip angle measurements: differences with ageamong normal subjects J. Vestib. Res. 9 435–44

Wiener N 1948 Cybernetics (Cambridge, MA: MIT Press)Young L R 1981 Visual and vestibular influences in human

self-motion perception The Vestibular System: Function andMorphology ed T Gualatieratti (New York: Springer)pp 393–424

Young L R and Meiry J L 1968 A revised dynamic otolith modelAerosp. Med. 39 606–8

S249