Computer Methods and Programs in Biomedicine 23 (1986) 147-154 147 Elsevier

CPB 00768

Finalist

The multi-state Kalman Filter in medical monitoring

K e r r y G o r d o n

Department of Mathematics and Unit of Medical Information Technolog& Department of Ohstetrics and Qvnaecologv, Unit,ersity of Nottingham, Nottingham, United Kingdom

In order to gain the best advantage from a computer database the way in which the information is displayed is vitally important. On-line statistical techniques could prove to be a great bonus to medical monitoring but have been limited by the methodology available. The Kalman Filter is one of the most powerful methods for time series analysis, and we have now shown it to be useful in a variety of settings, including the detection of kidney transplant rejection, where detection in some patients precedes that of experienced clinicians.

Computer graphics Missing data Kalman Filter Rhythms

1. Introduction

When moni tor ing patients in critical situations fast and easy access to high-quality informat ion on a computer database is an obvious advantage. A complex medical database, however, is able to store very large quantities of data, and so the useful information contained within can be well hidden. It is necessary, therefore, to select and then present data in an easily comprehensible format, so that the impor tant aspects of the infor- mat ion are clearly visible.

When the moni tor ing situation involves the col- lection of numerical data over a period of time, the form of presentat ion of results is particularly important , and the use of computer graphics, col- our and sound may prove a great benefit, both in deciphering the messages inherent in the data, and in alerting the user to current or potential prob- lems.

Along with a sensible and comprehensible pre- sentation of numerical data there is a need to quantify some of the patterns one might see, rather than rely upon totally subjective assessments. This will involve the use of statistical methodology and,

for sequences of t ime-dependent numerical results, some form of time series analysis.

In the medical monitor ing situation, there are various additional features which are inherent to the problem. The very implication of 'moni to r ing ' is that one must have an on-line, rather than a retrospective, technique. One must also, because of the variability of a clinical populat ion, be able to adjust (or ' t une ' ) the analysis according to each individual being monitored. In many clinical set- tings the collection of data is not regular, i.e. there will not always be the equally spaced recordings in time that most available statistical methods de- pend upon. There are several reasons for irregu- larly spaced data, including the closure of labora- tories (e.g. at weekends), the fact that the patient 's health may improve (or deteriorate) with a subse- quently lower (or greater) frequency of monitor- ing, and periods with equipment failure that may create r andom gaps in the data set. It is necessary, therefore, to accept that missing data is inevitable and to take this into account when developing a statistical technique for these situations. Also, many time series collected in a medical setting exhibit predictable rhythmic or episodic be-

0169-2607/86/$03.50 © 1986 Elsevier Science Publishers B.V. (Biomedical Division)

148

haviour, and this pattern can create confusion if it is not taken into account. Most of the statistical techniques available have not considered this pre- dictable cause of variability.

The statistical algorithms of Kalman (the so- called Kalman Filter [1]), which have their roots in flight navigation [2], tracking [3] and control en- gineering [4], were developed and extended in this project. The method can now be applied to a variety of medical monitoring situations, though particular attention was given to developing the methods for the monitoring of patients who have undergone renal transplantation, in order to de- tect the onset of kidney allograft rejection.

2. Methods of data presentation

For the case where sequences of numerical data are available (time series) the form of presentation of these results is particularly important. The eye is not very good, in general, at picking out the important messages from information in the form of a list of numbers. A much better way of pre- senting this information is in the form of a graph; patterns being more easily recognised by the eye when displayed in pictorial form (see Fig. 1).

It was quickly established, as soon as satisfac- tory computer programmes were installed, that the

Time Measurement

1 229 .~ 2 244 e 3 226 E 4 2 3 8 5 221 6 244 7 232 8 255 9 244

10 266 11 274 12 271 13 283 14 282 15 298 16 2 9 0

T ime

Fig. 1. Numbers vs. graphics. For most people, patterns are more readily seen in graphical form.

availability of graphics offered a substantial be- nefit to those involved in the care of patients undergoing intensive treatment. The problem of clarifying and understanding the current state of a patient's health (or of the state of an illness) is, however, not always solved with the drawing of a graph. Certainly the collection of precise patient data on computer will have helped, as will the presentation of this data in graphical form (with the use of colour rendering a surprisingly benefi- cial effect [5]), but any allusions to the interpre- tation of any visible patterns are, at this stage, purely subjective. Subjective assessments for each patient under care result in an erratic, non-stan- dardised modus operandi; the number of errors of judgement may be greater than is acceptable even if less than with the traditional paper record. This is especially possible when the patterns which may or may not be present in the data are masked by a considerable amount of noise, whether this is due to biological variability, measurement error or some other cause.

If there is to be efficient and consistent patient care those providing it need a way of quantifying the messages inherent in data that is collected. Conclusions drawn could then be concordant both within a specific patient and across a wide range of patients, including those patients who may present in the future. There is, in other words, a need to apply statistical methodology, and, in particular, time series analysis.

3. Statistical methods

There are several ways of applying statistics to sequences of numerical data, though we are par- ticularly interested in monitoring techniques. On- line monitoring methods have been developed mostly for quality-control environments [e.g. 6].

3.1. The Cusum method

One of the simplest is the cumulative sum (Cusum) technique [7]. The Cusum is certainly simple but the effect it can have in clarifying a change in pattern is often startling.

The Cusum involves summing the deviations of the measured responses from a known target value,

and then plotting these sums as an alternative to, or as well as, the raw data.

3.1.1. Method Let the measurement made at time t be y( t ) . Then the Cusum statistic at time n, C(n), will be given by:

c(,) : (y(t) - T) t = l

where T is the target value. If the patient is stable, the responses y ( t ) wob-

ble randomly around the target value in both directions, and the deviations from that level tend to cancel themselves out. Changes in patient status, either improvement or deterioration, result in successive deviations from the target, and so there is a sustained incremental increase/decrease in C(n), making the change more easily noticeable (see Fig. 2). This technique is commonly adopted in laboratories, but it can also be used to great effect in patient care, though it is not often ap- plied in this setting.

3.1.2. Problems with the Cusum method There are certain problems which arise and, when they do, these make the interpretation of the Cusum technique rather cumbersome. For exam- ple, one of the most common features of medical time series is an irregularity of data collection. For any of a number of reasons, as discussed above, the frequency of sampling is unlikely to remain constant throughout a monitoring period. It is unfortunate that the Cusum technique was not designed to handle missing or unequally spaced data, as this was not a problem common to most of the quality-control situations where it was first used.

The spread of individual variation amongst dif- ferent patients is another common feature in the medical context. Anybody involved in medical research learns quickly that any 'homogeneous ' group of patients is unlikely to prove to be such in the reality of a clinical setting (as opposed to a controlled research study). In research studies, the problem has been addressed and reduced to manageable proportions by selecting homoge-

149

A Raw data

2

- 4 10 20 30 40 50 60

B Cusum

2

-4 j 10 20 30 40 50 60

Fig. 2. There was a sudden change in the mean level of this (simulated) series on day 30, though it is difficult to see this in the raw data plot (A). The changepoint becomes obvious when a Cusum chart is plotted (B).

neous groups and by discarding the atypical pa- tients. In practice all patients need clinical care, and so a method was needed which will be able to take this into account, and adjust itself to the differing variabilities and levels found in the com- mon heterogeneous accumulation of patients.

3.2. The Kalman Filter

There is a statistical method which can manage the sort of problems discussed above, now that recent modifications (made by this author) have been completed. It is based on the algorithms initially presented by Kalman [1], known as the Kalman Filter. This method was developed for entirely different reasons (that of estimation and prediction in a tracking context), but more recent work by Harrison and Stevens [8] has shown how useful it can be in a monitoring situation, though theirs was in the field of economics. At this point it is helpful to re-emphasise the central rationale of Harrison and Stevens paper [8] and to indicate some recent extensions to these ideas derived by this author [9]. For illustration we will consider, initially, the simple case of monitoring a single variable which, perhaps after suitable transforma- tion, tends to follow straight-line trends.

150

3.2.1. The linear growth model Let the measurement at time t, denoted by y( t ) , be thought of as a recording of the true level of the series, u(t), along with some random measure- ment perturbation, e(t). Then one formulation of the straight-line model, appropriate for on-line analysis, is the linear growth model given by the measurement equation:

y ( t ) = u(t) + e( t ) ( la)

and the system equations:

u ( t ) = u ( t - 1 ) + b ( t ) ( lb)

b(t) = b(t - ]) (]c)

where b(t) is the incremental growth at time t. So for a perfect straight line (assume e( t )= 0

for all t) we have a constant increment added to the previous level (see Fig. 3). This formulation as it stands assumes equally spaced measurements.

3.2.2. The dynamic linear model The Harrison and Stevens proposal [8] was to allow changes in the straight-line pattern (e.g. a change in slope) by attaching perturbations (du(t) and db(t)) to the system equations, and then incorporating what they call a multi-state struc- ture,-This is a model which is allowed to be either stable or in the process of changing in a particular way. The resulting formulation, termed a dynamic linear model (DLM), is thus:

y ( t ) = u(t) + e( t ) (2a)

-6

U(4)

U ~ b(3 )=b(1

. . . . . . _1

u(1 )~'~: . . . . . J

T i m e

Fig. 3. Linear growth model. When there are no errors in- volved, the increment remains constant and the resulting pat- tern is linear.

u(t) = u ( t - 1) + b(t) + du(t)

b ( t ) = b ( t - 1 ) + db(t)

(2b)

(2c)

3.2.3. Unequally spaced data This idea can, we have demonstrated, be extended to sequences of unequally spaced measurements [9].

If y(k) is the k th observation, measured at time T(k), the unequally spaced equivalent of Eq. 2 is given by:

y ( k ) = u ( k ) + e ( k ) (3a)

u(k) = u ( k - 1 ) + b ( k ) +du(k ) (3b)

b(k ) = w ( k ) . b ( k - 1 ) + d b ( k ) (3c)

where w(k) = r ( k ) / r ( k - 1), r(k) = T(k) - T(k -1) = time interval between ( k - 1 ) t h and kth measurements,

r(k)

du(k) = ~_, du(T(k) - n + 1), and n = l

, (k -D-1 db(kl=w(k)- E

n = l

n . d b ( T ( k - 2) + n + 1)

r(k)

+ E n . d b ( T ( k ) - n + 11.

3.2.4. The multi-state structure These algorithms can be further developed by considering the possibility of a set of possible ' states'.

Referring to Eq. 3, if e(k) were non-zero only for measurement k, this would affect only y(k), through Eq. 3a, and not future values of y, hence resulting in a single transient observation.

If du(k) were non-zero only for measurement k, this would affect u(k + 1), through Eq. 3b, and so on, resulting in a step change in the measure- ments y, beginning at k.

If db(k) were non-zero only for measurement k, this would affect b(k + 1), through Eq. 3c, and also u(k + 1), through Eq. 3b, and so on. Thus a

single non-zero db(k) would result in a slope change in the measurements y, beginning at k (see Fig. 4).

These three types of change, along with the stable state, form the 4-state structure attached to the linear growth model. Initial probabilities are then affixed to each of these states and, using a Bayesian prior-to-posterior analysis, involving the Kalman Filter, these probabilities are sequentially updated upon the receipt of each new measure- ment. So, for instance, at any timepoint, T(k), one could give the probability that, given all the information so far, the k th measurement was the beginning of a new trend (i.e. a slope change), and so on. With simple Bayesian manipulation, one- step-back probabilities, e.g. the probability that the (k - 1)th measurement was the beginning of a

X

X

X

X y

X

X •

J ] I I I I ~ I I I I I I~

A Steady s ta te t B Transient t

Y

X

X X

X •

X

I I I I I I r

Level change t

J

Y

X

X •

x X

x

I I I I I I

D Slope change t

Fig. 4. The multi-state structure. When a new data point, e , becomes available there are at least four possibilities. (A) The continuation of a previous trend (steady state). (B) A single abberant observation (transient). (C) A sudden change in over- all level. (D) A sudden change in slope. With (B), (C) or (D), a further data point is required for reasonable discrimination.

151

new trend, given all the information up to and including the k th measurement; two-step-back probabilities (and so on) can be derived. These are important discriminatory measures since, as il- lustrated by Fig. 4, it is difficult to say when the current data point is out of line which type of change has occurred until at least one further measurement has been received. This difficulty is increased when the time series is very 'noisy' .

3.2.5. Modelling noise In respect of the noise in the series, the on-line variance updating procedure proposed by Smith and West [10] was adopted with great success. This tunes the analysis to individual patients, and avoids incorporating the fallacies inherent in defi- ning a 'normal ' amount of noise, which would have to be based on the experience of a group of patients thought to be similar.

A great deal of attention was given, as the method evolved, to other ways of modelling in order to reduce the effect of biological and mea- surement variations. On the measurement side, for instance, the modelling of rounding errors (e.g. to the nearest whole unit) and of timing errors (e.g. to the nearest half an hour) were incorporated sensibly into the model. On the biological side there may be within 'day ' rhythms, and these can have a substantial effect on the global (day-to-day) model, and so they too have been incorporated (see Fig. 6B).

For the case where there may be reasons to suggest that the errors are correlated, autoregres- sive models have been integrated into the overall theoretical framework (see Fig. 6A).

4. Results

The usefulness of the Kalman Filter in a practical setting is illustrated by the problem of detecting kidney rejection following transplantation. Series of serum creatinine concentrations from the in- dividual patients are traditionally considered by inspection of columns of sequential results.

It was found by Knapp et al. [11] that after some simple physiologically based transforma- tions, notably reciprocation and correction for

152

Plasma creatinine concent rat ion C # m o l / l )

Probability system not in steady state

Probability of slope change at previous point

800

400

200

100

A

- - o O O O 0 O O 0 OOO O_. go

A A l DT DT

o O

O •

RT l l l l l l i l l l l l l J l l l l l l l l l l l l l

B

OO o o

O0 0 0 0 • o00O RT

m i l l I I I I I I t I ~ 1 J I I t I I I I I I I I J I

0.5

o , , , II , ,

_-

8 10 12 14 16 1820 2 2 2 4 2 6 2 8 3 0 32 34 32 3 4 3 6 3 8 4 0 4 2 4 4 4 6 4 8 5 0 5 2 5 4 5 6 5 8

Time after transplantation (days)

Fig. 5. Detection of positive slope change using a multi-state Kalman Filter. (A) Low-noise series. Positive slope change was signalled on day 28. Note dialysis treatment on days 11 and 15 which are detected as instabilities but not confirmed as slope changes. (B) High-noise series. Positive slope change was signalled on day 51. Treatment finally commenced one week later. Note that the slope changes in (A) and (B) both occurred within the 'normal ' range. (DT = dialysis therapy; RT = anti-rejection therapy).

patient weight, sequences of serum creatinines could, if graphed, be seen to be straight line trends, and that on rejection of the kidney the direction of the trend suddenly switched (i.e. a sudden change in slope). The linear growth model described above is well suited to this situation, particularly as treatment by dialysis (common for renal patients) was found (quite reasonably) to cause a sudden shift in the level of the creatinine series. The multi-state Kalman Filter is one of the few methods available which cannot only detect such changes in pattern, but can also discriminate between them. The Cusum method does not pro- vide this degree of sophistication.

Fig. 5 shows how the method detected and signalled positive slope changes, i.e. the sudden switches from improvement to deterioration that indicate a possible rejection episode. The height of the probability histogram provides a quantifiable measure by which the confidence in the signal can be calibrated. More importantly, the timing of the event is also pinpointed.

In an initial set of 28 patients, in which there were 32 rejection episodes, the Kalman Filter method signalled a change in kidney function on average one day earlier than the clinician, and on several occasions many days, or even a whole week, earlier [12].

5. Conclusions

The use of the multi-state Kalman Filter has been shown to be a powerful tool in the detection of kidney rejection. Certain points have to be borne in mind.

The initial presentation of information is, in the author's view, an important ingredient in the acceptance of medical databases as a medical ad- vance, and so the clarity of presentation of 're- sults' after statistical analysis will be equally criti- cal. If the results are going to be heeded they need to be presented unambiguously. For instance, in the kidney examples, the transformed data was

A

( i ) t

y[ ~,-vd%-"

( i i i ) t

yT Q Cii) t

t

j-wQ . • ' , i , ,~,~x

( iv) t

%x ( i ) t

® %;'b Cii) t

x x x

b v

Ciii) t ( iv) t

Fig. 6. Rhythm monitoring. (A) A 4-state representation of a first order autoregressive process: (i) steady state, (ii) transient, (iii) level change and (iv) impulse (note that there are two forms of impulse depending upon the sign of the autoregressive parameter). (B) A 4-state representation of a sinusoidal model: (i) steady state, (ii) transient, (iii) level change, (iv) amplitude change.

plotted on an inverted scale because, even though a reciprocal transformation had been applied, clinicians were used to seeing deterioration as 'going up'. 'User-acceptance' is much more likely if the presentation is 'user-sympathetic'.

153

6o 1 5.0 4.0 3.0 2.0

1.0

o.o ~ 6 . o lO.O I 5.0

9.0 4.0 8.0 3.0

7.0 2.0 6.0 I .0

4.0 P< 3.0

2.0 Y axis x 10 I . ~ [ ' ] 2 5 - ' - - X axis x 102 0.25

Z axis x 10 -1

Fig. 7. On-line estimation of the phase of a sine wave for 100 days (y axis) of non-sinusoidal (simulated) data. A probability distribution (z axis) is assigned the full 360 ° (along x axis) and this distribution is updated on-line. The multimodal outcome suggests a natural uncertainty as to the true whereabouts of the phase, suggesting that the sinusoidal model is a poor fit to the data.

There are many ways of presenting Kalman Filter signals, and the use of flashing lights and computer sound is currently under investigation. The programmes, including certain database packages, were rewritten so as to make them avail- able on small, inexpensive micro-computers, such as the BBC-B and Commodore machines. *

The Kalman Filter methodology has now been extended, by the author, into other areas of medical and scientific monitoring. Its potential has been demonstrated in a variety of situations, including rhythm monitoring (see Fig. 6) and the monitoring of several time series concurrently. In these in-

Note: Further recent developments by this author now make these techniques available on small, low-cost microcom- puters, such as the Commodore Vic 20, as well as on the larger main-frame systems upon which they were initially dependent, and on the mini-systems upon which the early 'on-line' monitoring was conducted. The practical appli- cation of these methods is now well within the financial limitations of any individual unit and, indeed, the pro- grammes could be afforded by most of the individual pa- tients themselves (as a home monitoring device).

154

stances there is still a need to find the best form of display for the results, but the use of three-dimen- sional computer graphics may have potential ad- vantages (see Fig. 7).

Further development of the Kalman Filter the- ory, and programmes, is being carried out, and in the fields of foetal monitoring and psychological testing suitable procedures have now been devel- oped, and these await clinical evaluation. It seems reasonable to assume that the general philosophy of the methodology is portable to a host of other contexts, both medical and in other fields of activ- ity.

Acknowledgements

I would like to thank Dr. M.S. Knapp of the Department of Obstetrics and Gynaecology, Uni- versity Hospital, Nottingham, U.K. (previously of the Renal Unit, City Hospital, Nottingham, U.K.) and Prof. A.F.M. Smith of the Department of Mathematics, University of Nottingham, U.K. for their advice and encouragement. I would also like to thank the Nottingham and Notts. Kidney Fund and the University of Nottingham, National West- minster Bank Research Fund for providing the financial support for much of this research.

References

[1] R.E. Kalman, A new approach to linear filtering and prediction problems, J. Basic Eng. 82 (1960) 35.

[2] S.F. Schmidt, State space techniques applied to the design of a space navigation system, JACC Conf. Paper (1962).

t 3] G.L. Smith, S.F. Schmidt and L.A. McGee, Application of statistical filter theory to the optimal estimation of position and velocity on board a circumlunar vehicle, NASA TR-R-135 (1962).

[4] L.A. Zadeh and C.A. Desoer, Linear System Theory (Mc- Graw-Hill, New York, 1963).

[5] W.J. Perkins and R.J. Green, Three-dimensional recon- struction of biological section, J. Biomed. Eng. 4 (1982) 37-43.

[6] A.F. Bissell, Cusum techniques for quality control, AppL Stat. 18 (1969) 1-25.

[7] M.S. Knapp, A.F.M. Smith, I.M. Trimble, R. Pownall and K. Gordon, Mathematical and statistical aids to evaluate data from renal patients, Kidney Int. 24 (1983) 474-486.

[8] P.J. Harrison and C. Stevens, Bayesian forecasting, J. R. Stat. Soc. B 38(3) (1976) 205-247.

[9] A.F.M. Smith, M. West, K. Gordon, M.S. Knapp and I.M. Trimble, Monitoring kidney transplant patients, Statistician 32 (1983) 46-54.

[10] A.F.M. Smith and M. West, Monitoring renal transplants: an application of the multi-process Kalman filter, Biomet- rics 39(4) (1983) 867-878.

[11] M.S. Knapp, R.W. Blamey, J.R. Cove-Smith and M. Heath, Monitoring the function of renal transplants, Lancet 2 (1977) 1183.

[12] I.M. Trimble, M. West, M.S. Knapp, R. Pownall and. A.F.M. Smith, Detection of renal allograft rejection by computer, Brit. Med. J. 286 (1983) 1695-1699.