physic ad 117

7/27/2019 Physic Ad 117

1/16

Physica D 117 (1998) 283298

Comparisons of new nonlinear modeling techniques

with applications to infant respiration

Michael Small , Kevin JuddDepartment of Mathematics, University of Western Australia, Nedlands, WA 6907, Australia

Received 1 August 1997; received in revised form 23 October 1997; accepted 2 December 1997

Communicated by A.M. Albano

Abstract

This paper concerns the application of new nonlinear time-series modeling methods to recordings of infant respiratorypatterns. The techniques used combine the concept of minimum description length modeling with radial basis models. Our

first application of the methods produced results that were not entirely satisfactory, particularly with respect to accurately

modeling long term quantitative and qualitative features of respiration patterns. This paperdescribes a number of modifications

of the original methods and makes a comparison of the improvements the various modifications gave. The modifications made

were increasing the class of basis function, broadening the range of possible embedding strategies, improving the optimization

of the likelihood of the model parameters and calculating a closer approximation to description length. The criteria used in

the comparisons were description length, root-mean-square prediction error, model size, free-run behavior and amplitude size

and variation.

We use surrogate data analysis to confirm the hypothesis that the recorded data are consistent with the nonlinear models we

construct, and not consistent with simpler models. We also investigate the free-run dynamics of the model systems to see if the

models exhibit features consistent with physiological characteristics observed independently in the respiratory recording. This

involved modeling breathing patterns prior to a sigh and onset of a phenomenon called periodic breathing; and comparing

the period of cyclic amplitude modulation of the free-run dynamics of the models and the period of the subsequent periodic

breathing observed in respiration recording. The periods were consistent in six out of seven recordings. Copyright 1998Elsevier Science B.V.

Keywords: Radial basis modeling; Description length; Surrogate analysis; Infant respiratory patterns; Periodic breathing

1. Introduction

This paper describes an attempt to accurately model

the respiratory patterns of human infants using new

nonlinear modeling techniques. We have identified

periodic fluctuation in regular breathing pattern of

sleeping infants using linear modeling techniques

Corresponding author. Tel.: +618 9380 3348; fax: +6189380 1028; e-mail: [email protected].

[3]. An accurate, reliable and replicable method of

building nonlinear models may further aid the iden-

tification of such subtle periodicities and give some

insight into the mechanisms generating them. Just as

a differential equation model of a system can lead to

greater understanding, so too can numerical, nonlinear

models. In this paper it is our aim to produce accu-

rate models of nonlinear systems (infant respiration)

from a scalar time series measurement of that system

(abdominal volume).

0167-2789/98/$19.00 Copyright 1998 Elsevier Science B.V. All rights reserved

PII S 0 1 6 7 - 2 7 8 9 ( 9 7 ) 0 0 3 1 1 - 4

7/27/2019 Physic Ad 117

2/16

284 M. Small, K. Judd/ Physica D 117 (1998) 283298

Initially we used a radial basis modeling algorithm

described by Judd and Mees [1] to model recordings

of the abdominal movements of sleeping infants. Al-

though these radial basis models give accurate short-

term predictions, they were not entirely satisfactory in

the sense that simulations of the models failed to ex-

hibit some characteristics of the original signals. Aftersome alteration of the model building algorithm, much

better results were obtained; simulations of the models

exhibit signals that are nearly indistinguishable from

the original signals.

In this paper we first describe the time series we

will model and briefly review the nonlinear modeling

methods of Judd and Mees [1]. We identify some fail-

ings of simulations of models produced by this algo-

rithm; suggest modifications that may overcome these

problems; and finally demonstrate the improved re-

sults we have obtained.

1.1. Data collection

For this study we collected measurements pro-

portional to the cross-sectional area of the abdomen

of infants during natural sleep (using standard non-

invasive inductive plethysmography techniques). Such

measurements are a gauge of lung volume.

Nineteen healthy infants were studied at one, two,

four and six months of age, in the sleep labora-

tory at Princess Margaret Hospital. The study was

approved by the Princess Margaret Hospital ethics

committee.The unfiltered analog signal from an inductance

plethysmograph was passed through a DC ampli-

fier and 12 bit analog to digital converter (sampling

at 50 Hz). The digital data were recorded in ASCII

format and were then transferred to Unix work-

stations at the University of Western Australia for

analysis.

The only practical limitation on the length of time

for which data could be collected is the period when

the infant remains asleep and still. The cross-sectional

area of the lung varies with the position of the infant.

However, in this study we are interested only in the

variation due to the breathing and so we have been

careful to avoid artifact due to changes in position or

band slippage. We have made observations of up to

2 h that are free from significant movement artifacts,

although typically observations are in the range 5

30 min.

1.2. Pseudo-linear radial basis modeling

We have previously used these data to estimate the

correlation dimension of the respiratory patterns of

sleeping infants [2], and to identify cyclic amplitude

modulation (CAM) in respiration during quiet sleep

[3]. Both these studies concluded that linear model-

ing techniques were unable to model the dynamics

of human respiration. 1 Furthermore, by comparing

the correlation dimension estimates for the data and

surrogates we were able to demonstrate that simula-

tions from radial basis models produced dimension

estimates that closely resembled that of the data [2].This implies that nonlinear models are more accurately

modeling the data than are linear models. However,

these nonlinear models appeared to have difficulty

with some data sets, most notably those with sub-

stantial noise contamination and data exhibiting non-

stationarity. In this paper we attempt to improve the

modeling techniques.

2. Modeling respiration

In this section we introduce the data set that we

will attempt to model. We use correlation dimension

estimation and false nearest neighbor techniques to

determine a suitable embedding dimension and exam-

ine three alternative criteria for embedding lag to de-

duce an appropriate value. We then apply the nonlinear

modeling technique described by Judd and Mees [1] to

this data set and examine the weaknesses of the result.

1 By calculating correlation dimension dc(0) for data embed-

ded in ,3 4 and 5 as a test statistic surrogate analysis of 27

recordings of infant respiration from 10 infants concluded thatthe data were inconsistent with each of the linear hypothesesaddressed by Theiler et al. [4]. The details of these calculationsare beyond the scope of this discussion, see [2,5].

7/27/2019 Physic Ad 117

3/16

M. Small, K. Judd / Physica D 117 (1998) 283298 285

Fig. 1. Data. The data we use in our calculations. The solid line represents the data set from which we build our radial basis models.The horizontal axis is time elapsed from the start of data collection and the vertical axis is the output from the analog to digitalconvertor (proportional to cross-sectional area measured by inductance plethysmography). Note the sigh (at about 300 s) and the

onset of periodic breathing following this.

Fig. 2. Periodic breathing. An example of a short episode of periodic breathing after a sigh (at 580 s on the second panel). Smaller

sighs are also present at about 275 and 470 s on the first panel. The horizontal axis is time elapsed from the start of data collectionand the vertical axis is the output from the analog to digital convertor (proportional to cross-sectional area measured by inductanceplethysmography).

2.1. Data

For much of the following sections we illustrate

the calculation and comparison using just one record-

ing, selected because it is a typical representation

of a range of important dynamical features. The data

set we use (see Fig. 1) is from a section of approx-

imately 10 min of respiration of a two-month-old

female in quiet (stages 3 and 4) sleep. These data

exhibit a physiological phenomenon of great interest

to respiratory specialists known as periodic breath-

ing [6,7]. Periodic breathing is simply extreme CAM

the minimum amplitude decreases to zero. Fig. 2

shows an example of periodic breathing. In all other

respects these data are typical of many of our record-

ings. The section which we examine first is from a

period of quiet sleep preceding the onset of periodic

breathing.

7/27/2019 Physic Ad 117

4/16


Fig. 3. False nearest neighbors. False nearest neighbor calculation for the data illustrated in Fig. 1 (1600 points sampled at 12 .5Hz)embedded with a time delay embedding,

=5 (Rtol

=15).

2.2. Embedding

Using a time delay embedding strategy and appeal-

ing to Takens [8] embedding theorem we produce

from our scalar time series

y1, y2, y3, . . . ,

a d-dimensional vector time series. Consider the

embedding

yt zt = (yt, yt2, . . . , ytd) t > d.To perform this transformation one must first identify

the embedding lag and the embedding dimension d.

We describe the selection of suitable values of these

parameters in the following paragraphs.

Any value of is theoretically acceptable, but the

shape of the embedded time series will depend criti-

cally on the choice of and it is wise to select a value

of which separates the data as much as possible.

General studies in nonlinear time series [9] suggest the

mutual information criterion [10], the autocorrelation

function [11] or one of several other criteria to choose

. Our experience and numerical experiments sug-

gest that selecting a lag approximately equal to one

quarter of the quasi-period of the time series pro-

duces comparable results to the autocorrelation func-

tion but is more expedient. Note that the first zero

of the autocorrelation function will be approximately

the same as one quarter of the quasi-period if the

data are almost periodic. Numerical experiments with

these data have shown that either of these methods

produce superior results to the mutual information

criterion.

Suitable bounds on d can be deduced by using afalse nearest neighbor analysis [12]. Numerical ex-

periments indicate that four dimensions are sufficient

to remove false nearest neighbors from the data, see

Fig. 3. Furthermore, it is at approximately this em-

bedding dimension that the correlation dimension es-

timates appear to plateau. Takens sufficient condition

on successful recreation of the attractor by embedding

requires d > 2dc + 1 where dc is the correlation di-mension of the attractor. For our data with 3 < dc 4(see [2]), this would suggest that d > 8 is necessary.

However, embedding in this dimension offers no im-

provement to the modeling process and our false near-

est neighbor calculations indicate that a much smaller

value of d is sufficient.

7/27/2019 Physic Ad 117

5/16


Fig. 4. Initial modeling results. Free-run prediction and noise-driven simulation of a radial basis model. The plot on the left is a

free-run prediction with no noise, on the right is a simulation driven by Gaussian noise at 10% of the root-mean-square prediction error

(

ti=1

2i

/

N). The horizontal axis is yt for t = 1, . . . , 500, the vertical axis is the output from the analog to digital convertor(proportional to cross-sectional area measured by inductance plethysmography). From 30 trials, 27 of them exhibited fixed points.

2.3. Modeling

We attempt to build the best model of the form

yt+1 = f (zt) + t,where t is the model prediction error and f :

d is of the form

f (zt) = 0 +n

i=1i ytli

+m

j=1j+n+1

zt cjrj

, (1)

where rj and j are scalar constants, 1 li < li+1 d are integers and cj are arbitrary points in

d.

The integer parameters n and m are selected to min-imize the description length [10] as described in [1].

Here () represents the class of radial basis functionfrom which the model will be built. We choose to use

Gaussian basis functions because they appear to be

capable of modeling a wide variety of phenomena.

The data set consists of 20 000 points sampled at

50 Hz. This is oversampled for our purposes and we

thin the data set to one in four points and truncate it

to a length of 1600 (see Fig. 1). We set d = 4 andchoose = 5.

Trials with the modeling algorithm as described in

[1] produced some problems with the model simula-

tions (see Fig. 4). None of the simulations look like

the data. When periodic orbits are evident they are still

unlike the data; the waveform is symmetric, whereas

the data have a definite asymmetry. Moreover the free-

run models often exhibit stable fixed points. This is

extremely undesirable as it is evidently not an accu-

rate representation of the dynamics of respiration

breathing does not tend to a fixed point, usually.

The remainder of this paper shall be concerned with

addressing these problems. These problems are the

result of three main deficiencies in the initial mod-

eling algorithm: (i) it over fits the data; (ii) it does

not produce appropriate simulations; and (iii) models

are not consistent or reproducible. We will attempt to

improve upon these problems whilst considering the

many competing criteria for a good model.

3. Improvements

Before we can attempt to improve our modeling

procedure we must be clear on what we mean by im-

provement. There are several criteria that might be

imposed to achieve a good model.

Modeling criteria measure quantities such as the

number of parameters in the model, its prediction error

and description length. It is desirable to have a model

with few parameters, a small description length and a

small root mean square prediction error.

Algorithmic criteria are concerned with optimizing

the modeling algorithm, to ensure that it searches the

broadest possible range of basis functions as efficiently

7/27/2019 Physic Ad 117

6/16


as possible. Unfortunately a larger search space comes

at the expense of more computation.

Qualitative criteria consider properties of the dy-

namics of models; for example, the behavior observed

in the simulations of the model. In modeling breath-

ing, for example, we expect something like stable

periodic (or quasi-periodic) solutions; divergence orstable fixed points seem unlikely. Furthermore we ex-

pect the shape of the periodic solution to closely match

the shape of the data and to occupy the same region

of phase space.

Modeling results should also be reproducible and

representative. It does not seem unreasonable to ex-

pect consistent, repeatable results from a modeling

algorithm, both qualitatively and quantitatively. Re-

producibility can be examined by repeatedly modeling

a single data set. Furthermore, the model should be

representative in that when making many simulations

of the model, we ought to obtain time series of whichthe original data are representative. Representativity

can be measured with the assistance of surrogate tests

using a statistic such as the correlation dimension es-

timates or cyclic amplitude modulation.

In the following sections we consider improvements

of the basic modeling procedure by: (i) broadening the

class of basis functions; (ii) using a more targeted se-

lection algorithm; (iii) making more accurate estimates

of description length; (iv) local optimization of nonlin-

ear parameters; (v) using reduced linear modeling to

determine embedding strategies; and (vi) simplifying

the embedding strategies using a form of sensitivityanalysis.

3.1. Basis functions

In this section we introduce a broader class of ba-

sis functions. This will produce an algorithm that is

capable of modeling a wider range of phenomena.

First we expand the embedding strategy so that

instead of radial (spherical) basis functions we in-

troduce cylindrical basis functions. Detailed argu-

ments about the advantages of these basis functions aredescribed elsewhere [13]. Generalize the functional

form (1) to

f (zt) = 0 +n

i=1i ytli

+m

j=1j+n+1

Pj(zt cj)rj

, (2)

where li , rj, j, cj, n, m are as described previously

and Pj d: dj (dj < d) are projections ontoarbitrary subsets of coordinate axes.

The functions Pj can be thought of as a local em-

bedding strategy. Each basis function has a different

projection Pj and so each Pj(zt cj) is dependenton a different set of coordinate axes.

We actually generalize the choice of embed-

ding strategy further by selecting the best lags

from the set {0, 1, 2, . . ., d}, not only subsets of{0, , 2 , . . . , d }. It seems that by allowing the se-lection of different embedding strategies in different

parts of phase space the model gives better free-run

behavior. This indicates that, naturally enough, the

optimal embedding strategy is not uniform over phase

space. Selecting from this larger set of embedding

lags is equivalent to embedding with a time lag of 1 in

d. However, the modeling algorithm rarely selects

more than a d-dimensional local embedding. There-

fore, these improved results are not contrary to our

previous estimates of optimal embedding dimension.

They do allow for an embedding in more than 2 dc +1dimensions (satisfying Takens sufficient condition) if

necessary. As noted earlier the choice of embedding

lag is largely arbitrary.Furthermore, to increase the curvature of the basis

functions we replace the choice of

(x) = exp

x2

2

by

(x,) = exp

(1 ) x

,

where 1 < < R is the curvature 2 and (1 )/ is acorrection factor so that

(x,) dx = 1. Hence,

maintaining consistent notation

2 To prevent large values of the second derivative of f it isnecessary to provide an upper bound R on .

7/27/2019 Physic Ad 117

7/16


(x,) =

2(1 )

x/2

,

and the basis functions become functions of the form

2(1 j)

j

Pj(zt cj)rj

j/2

,

where

(x) = exp x2

2.

Broadening the class of basis functions has in-

creased the complexity of the search algorithm.

Hopefully, it will also have broadened the search

space sufficiently to encompass functions which can

more accurately model the data. To overcome this

increased search space we consider a more efficient

search algorithm.

3.2. Directed basis selection

The method of Judd and Mees [1] involves ran-

domly generating a large set of basis functions

{(z cj/rj)}j = {j}j and evaluating them ateach point of the embedded time series z to give the

matrix V = [1|2| |M]. Following an iterativescheme they repeatedly select columns from this ma-

trix (and the corresponding candidate basis function)

to add to the optimal model.

Instead, we select a new set of candidate basis func-

tions {j}j (and a new matrix V) at each expansion ofthe optimal model. We then identify the column k ofV

that best fits the residuals (orthogonal to the previously

selected basis functions) and select the corresponding

basis function k . All the other candidate basis func-

tions {j}j=1,...,M;j=k are ignored and forgotten at thenext iteration. Because a new set of basis functions

is selected at each expansion, all the candidate basis

functions are much more appropriately placed. 3

The improvement in modeling achieved by this will

require a greater computation time. Furthermore, the

selection of basis functions that more closely fit the

3 Basis functions are selected according to either a uniformdistribution or the probability distribution induced by the mag-nitude of the modeling prediction error.

data may, possibly, increase the number of basis func-

tions allowed by the description length criterion. To

alleviate this problem we introduce a harsher more

precise version of description length.

3.3. Description length

The minimum description length criterion, sug-

gested by Rissanen [10], is used by Judd and Mees

[1] to prevent over fitting. However, the original im-

plementation of minimum description length used by

Judd and Mees only provides a description length

penalty for the coefficient j of each of the radial ba-

sis functions (and linear terms). Each basis function

also has a radius rj and coordinates cj which must

also be specified to some precision, and hence should

also be included in the description length calculation.

In [1] j is j truncated to some finite precision j,

then the description length is expressed as

L(z, ) = L(z|) + L(), (3)where

L(z|) = ln P (z|)is the description length of the model prediction errors

(the negative log likelihood of the errors) and

L() m+n+1

j=1ln

j

is the description length of the truncated parameters,

is an inconsequential constant.

We generalize Eq. (3) and include the finite preci-

sions ofrj and cj. Let represent the vector of all the

model parameters (j, cj, and rj) and the truncation

of those parameters to precision . Then

L(z, ) = L(z|) + L(), (4)where

L() (d+2)m+n+1

j=1

ln

j.

Now the problem becomes one of choosing to

minimize (4). By assuming that is not far from the

7/27/2019 Physic Ad 117

8/16


maximum likelihood solution (see Section 3.4), one

can deduce that

L(z, ) L(z|) + 12 TQ

+ k ln k

j=1ln j, (5)

where k = ((d+ 2)m + n + 1). Minimizing (5) gives(as in [1])

(Q)j = 1/j,

where Q = DL(z|) is the second derivative ofthe negative log likelihood with respect to all the

parameters.

Although algebraically complicated, this expression

can be solved relatively efficiently by numerical meth-

ods. However, by assuming that the precision of the

radii and the position of the basis function must be

approximately the same, 4 one can circumvent a great

deal of the computational difficulty, and simply cal-

culate the precision of j assuming the same values

for the corresponding precisions of the coordinates cj.

Much of the computational complexity of calculat-

ing description length could be avoided by utilizing

the Schwarz criterion [14]. Indeed, from experience

it appears that the Schwarz criterion gives compara-

ble size models. However, Schwarz criterion does not

take into account the relative accuracy of different ba-

sis functions an important feature of minimum de-

scription length.

3.4. Local optimization of nonlinear model

parameters

Once the best (according to sensitivity analysis) ba-

sis function has been selected we improve on its place-

ment by attempting to maximize the likelihood

P (z|, 2)

= 1(2 2)N/2

exp(y V )T(y V )

22,

4 Since a slight change in radii will affect the evaluation of abasis function over phase space in the same way as an equalsmall change in the position of the basis function.

where y V = is the model prediction error, and2 is the standard deviation of the (assumed to be)

Gaussian error. By setting 2 =ti=1 2i /N and tak-ing logarithms one gets that

ln P (z

|)

=N

2 +ln2

N

N/2

+ ln

ti=1

2i

N/2. (6)

To maximize the likelihood we optimize Eq. (6) by

differentiating ln (t

i=1 2i )

N/2 with respect to rj, cj,

and j. This calculation is algebraically messy, but

computationally straightforward provided a good op-

timization package is used.

3.5. Reduced linear modeling for embeddingselection

Allowing different embedding strategies from such

a wide class (due to the expansion of the class of basis

functions in Section 3.1) increases the computational

complexity of the modeling process. However, to cir-

cumvent this we note that for Gaussian basis functions

the first-order Taylor Series expansion gives

Pj(zt cj)rj

= d

i=1 pi (zt cj)2rj

d

i=1

|pi (zt cj)|rj

, (7)

where pi :d is the coordinate projection onto

the ith coordinate. We then build a minimum descrip-

tion length model of the residual of the form (7). From

this we deduce that the basis functions selected are

a good indication of an appropriate embedding strat-

egy. Although this method is approximate it is hoped

that this will provide a useful and efficient innovation

within the modeling algorithm.

7/27/2019 Physic Ad 117

9/16


Fig. 5. Improved modeling results. Free-run prediction and noise-driven simulation of a radial basis model. The plot on the left is afree-run prediction with no noise, on the right is a simulation driven by Gaussian noise at 10% of the root-mean-square prediction

error (

ti=1

2i

/

N). The horizontal axis is yt for t = 0, . . . , 500, the vertical axis is the output from the analog to digitalconvertor (proportional to cross-sectional area measured by inductance plethysmography).

3.6. Simplifying embedding strategies

Our final, very rudimentary alteration is designed

to account for some of the approximation required in

the reduced linear modeling of the embedding strate-gies. Given an embedding strategy suggested by the

method of Section 3.5, we generate additional can-

didate basis functions by using embedding strategies

whose coordinates are subsets of the coordinates of the

embedding strategy suggested by the linear modeling

methods.

4. Results

After implementing the alterations described in the

preceding section, we again apply our methods tothe same data set. This section describes the results

of these calculations and examines some of the im-

provements in the final model. We also examine the

individual effect of each modification and the effec-

tiveness of this modeling procedure in seven different

data sets (from six infants). Because of its physiologi-

cal significance, all the data sets selected for this anal-

ysis exhibit CAM suggestive of periodic breathing. We

compare dimension estimates for the original data sets

and simulations from the models. Finally, we apply

a linear modeling technique discussed elsewhere [3]

to detect CAM within the respiratory traces of sleep-

ing human infants, and present some results. That is,

we compare the CAM present in the data following a

sigh to that present in the models built from the data

preceding the sigh.

4.1. Improved modeling

Fig. 5 shows a section of free-run prediction, and

noisy simulation for a representative model. Using

an interactive three-dimensional viewer (see Fig. 6),

it is possible to determine that these models also have

many more common structural characteristics than

those created in Section 2.3. The size, placement,

shape and local embedding dimensions of the basis

functions of the models have many similarities.

Importantly, all of these models have a similar free-

run behavior. The free-run predictions are as large (in

amplitude) as the data; this was a substantial problem

with the original modeling procedure. Moreover, thefree-run behavior with noise appears more realistic

and the shape of the simulations mimics very closely

that of the data. Fig. 7 shows a short segment of a

simulation, along with the data. Note the similarities

in the shape of the prediction and the data. Finally,

the simulations exhibit a measurable cyclic amplitude

modulation which we use in Section 4.3.2 to infer

the presence of cyclic amplitude modulation in the

original time series.

4.2. Effect of individual alterations

Table 1 lists some characteristics of models built

from the data in Fig. 1 using various methods. The

7/27/2019 Physic Ad 117

10/16


Fig. 6. Cylindrical basis model. A pictorial representation of the interactive three-dimensional viewer we used. The axes range from

1.715415 to 3.079051, the same range of values as the data. The point (1.7, 1.7, 1.7) is in the front center, foreground.The cylinders, prisms and sphere represent the placement (cj) and size (rj) of different basis functions with different embeddingstrategies. the X, Y, and Z coordinates shown correspond to yt, yt5, and yt15, respectively. The coloring of the basis functionsrepresents the value of the coefficients (j).

different modeling strategies are: (A) the initial

method (described in Section 2.3); (B) extended ba-

sis functions and embedding strategies (Section 3.1);

(C) directed basis selection (Section 3.2); (D)

more accurate approximation to description length

(Section 3.3); (E) local optimization of nonlin-

ear model parameters (Section 3.4); (F) reduced

linear modeling to select embedding strategies

(Section 3.5); and (G) simplifying embedding strate-

gies (Section 3.6). These alterations to the algorithm

were progressively added in various combinations

and characteristics of the observed models measured.

The initial procedure (A) produced very bad free-

run predictions; 27 out of 30 trials produced simula-

tions with fixed points. Extending the class of basis

functions and adding cylindrical basis functions (B)

vastly improved this (only eight out of 30 simula-

tions did not have periodic (or quasi-periodic) orbits).

Most of the periodic orbits in these simulations were

smaller than the data (did not occupy the same part

of phase space) and one divergent simulation was ob-

served (hence the large standard deviation in Table 1).

This approach decreased the prediction error without

affecting either the model size or description length

(clearly, the required precision of the parameters was

greater).

Directed basis selection (C) greatly increased the

size of the model and decreased error whilst improv-

ing free-run behavior not only in amplitude but also

shape. The increase in computational time could al-

most entirely be due to the greater model size. Improv-

ing the description length calculation (D) decreased

the model size whilst, predictably increasing predic-

tion error. This also caused a surprising increase in

7/27/2019 Physic Ad 117

11/16


Fig. 7. Short-term behavior. Comparison of simulation and data. The solid line is the data, the dotdashed is a free-run prediction,

the dashed is a simulation driven by noise (20% of

ti=1 2i /N). The initial conditions for the artificial simulations are identical

and are taken from the data. The vertical axis is the output from the analog to digital convertor (proportional to cross-sectional area

measured by inductance plethysmography).

Table 1

Algorithmic performance: Comparison of the modeling algorithm with various improvements

Modeling method Nonlinear parameters RMS error MDL Free-run amplitude CPU time (s)ti=1

2i

/

N

A 12.52.4 0.1350.016 1086157 0.000.91 283103A + B 12.52.4 0.1130.011 1090155 1.2231.90 292113A + B + C 24.54.3 0.1040.015 1123198 1.581.04 567167A + B + D 10.72.3 0.1220.016 975191 0.3424.91 744504A + B + C + D 14.53.5 0.1230.018 909210 1.501.09 16261203A + B + C + D + E 9.52.9 0.1410.012 735131 1.591.31 20471253A + B + C + D + F 13.73.6 0.1260.009 87081 1.3117.48 43481959A + B + C + D + E + F 11.03.1 0.1170.013 990119 1.1717.94 58422973A + B + C + D + E + F + G 11.43.2 0.1170.011 980110 1.871.00 57532360

The seven different modeling procedures are the initial routine described by Judd and Mees, and six alterations described in

Section 3. Modeling methods are: (A) the initial method; (B) extended basis functions and embedding strategies; (C) directed basisselection; (D) exact description length; (E) local optimization of nonlinear model parameters; (F) reduced linear modeling to select

embedding strategies; and (G) simplifying embedding strategies. Results are from 30 attempts at modeling the data described inSection 2.1 and Fig. 1. The numbers quoted are (mean value)(standard deviation). Calculations were performed on a SiliconGraphics Indy running at 133MHz with 16 Mbytes of RAM. CPU time is measured in seconds using MATLABs cputime

command.

calculation time an indication of the computational

difficulty solving (Q)j = 1/j, where Q is thesecond derivative with respect to all the model para-

meters (or at least and r). Because there is a harsh

penalty these models are far less likely to be over

fitting the data. Combining the improved description

7/27/2019 Physic Ad 117

12/16


Fig. 8. Periodic breathing. An example of periodic behavior in one of our data sets. The solid region was used to build a nonlinear

radial basis model. Note that periodic breathing begins immediately after the sigh. The vertical axis is the output from the analog todigital convertor (proportional to cross-sectional area measured by inductance plethysmography).

length calculation and directed basis selection pro-

duced models comparable in both size and fitting error

to before either alteration was implemented (A + B).However, free-run behavior had an amplitude closer

to the mean amplitude of the data and exhibited anasymmetric waveform similar to the data.

Addition of the nonlinear optimization (E) and local

linear modeling (F) routines caused the greatest in-

crease to computational time. Individually these meth-

ods did not offer any considerable improvement to

the other model characteristics. However, many of the

statistics indicate a decrease in the variation between

trials. Combined, these modifications gave a slight im-

provement in prediction error and description length

whilst making the model smaller. They produced more

realistic simulations although the amplitude was

smaller than that of the data.Finally, the simple procedure of checking that sim-

pler embedding strategies would not produce better

(or equally good) results (G) caused a substantial im-

provement. This is perhaps due in part to the previous

optimization and local linear methods, particularly the

approximate nature of the local linear modeling. Re-

moving the coordinates helped produce some appre-

ciable improvement in suitability of the embedding

strategies suggested by the approximate local linear

methods. The local linear methods often produce a

high-dimensional local embedding (many significant

coordinates), eliminating some of these will usually

only slightly increase the prediction error. This sim-

ple addition increases the amplitude to a realistic level

(approximately 1.9 whilst the mean breath size for the

data is about 2.3 5) whilst decreasing the proportion

of fixed point and divergent trajectories to the lowest

level (8 and 0 of the 30 models, respectively) with-

out appreciably changing the description length, pre-diction error, or model size whilst decreasing slightly

the calculation time (and variance in calculation time).

Furthermore, these models have far more structural

similarities (in the size and placement of basis func-

tions) than the previous models have, indicating that

these models are far more consistent.

The remainder of this section is devoted to some

applications of these modeling methods and tests of

their representability.

4.3. Modeling results

From over 200 recordings of 19 infants, we identi-

fied seven data sets from six infants for a more careful

analysis. All seven of these data sets include a sigh

followed by a period of breathing exhibiting cyclic

amplitude modulation (CAM) [3]. Our present discus-

sion examines the analysis of these data sets.

In this section we examine the free-run behavior

of data sets created from seven models of seven data

sets from six sleeping infants. We compare the corre-

lation dimension of the data and simulations from mo-

dels. Following this we compare the period of CAM

5 Note, however, that the data are slightly nonstationary whilstthe model is not.

7/27/2019 Physic Ad 117

13/16


Fig. 9. Surrogate calculations. Comparison of dimension estimates for data and surrogates. The three figures on the left are dimension

estimates (for embedding dimension from 3 to 5, shown from top to bottom) for a model of Bs2t8. The left three plots aresimilar results for a model of Ms1t6. All surrogates are simulation driven by Gaussian noise with a standard deviation of half theroot-mean-square one step prediction error. Each picture contains one dimension estimate for the data (solid line), and 30 surrogates

(dotted). The two data sets used in these calculations are shown in Figs. 1 and 8, respectively.

detected in the free-run predictions from the models to

that visually evident after a sigh. Fig. 8 illustrates oneof the data sets used in our analysis. This is the only

set of data to exhibit periodic breathing, the others

merely exhibited strong amplitude modulation after

the sigh for 2560s ( 1530 breaths). Neverthelessthe change that the respiratory system undergoes after

a large sigh is of great interest to respiratory physio-

logists. We examine the system before and after a sigh

to determine evident physiological similarities in the

mechanics of breathing.

For each of our seven data sets, we identify the lo-

cation of the sigh, and extract data sets of 1501 points

spanning 120 s preceding the sigh. From these data

sets the respiratory rate of each recording was estab-

lished and the period of respiration deduced. Each data

set was embedded in 4 with a lag equivalent to the

integer closest to one quarter of the period. We thenapplied our modeling algorithm.

4.3.1. Surrogate analysis

To determine exactly how similar data and model

simulations are, we employ an obvious generalization

of the surrogate data analysis used by Theiler et al. [4].

The principle of surrogate data is the following. One

first assumes that the data come from some specific

class of dynamical system, possibly fitting a paramet-

ric model to the data. One then generates surrogate

data from this hypothetical system (producing simu-

lations from the parametric model) and calculates var-

ious statistics of the surrogate and original data. The

surrogate data will give the expected distribution of

7/27/2019 Physic Ad 117

14/16


Table 2

Periodic behavior: Comparison of CAM after apnea (apparent to visual inspection), the second set of results, and CAM detected in

the models limit cycle, the first set of results

Subject Sex Age Model CAM in free-run CAM after sigh

(months) size (breaths) (s) (breaths) (s)

A(As4t2) Male 6 8 (7) 56 a 14 a 5 25Bb(Bs2t8) Female 2 7 (6) 6 9 6 9Bb(Bs3t1) 6 (5) 5 10 5 10

G(Gs2t4) Female 2 4 (3) 5 11 5 9H(Hs1t2) Male 1 5 (3) 89 a 11 a 9 13

M(Ms1t6) Female 1 6 (4) None None 5 14.5R(Rs2t4) Male 2 8 (6) 9 18 8 16

Data sets Ms1t6 and Bs2t8 exhibited periodic breathing. For each data set marked cyclic amplitude modulation (CAM) occurred

after a sigh and was measured by inspection. Radial basis models were built on a section of quiet sleep preceding the sigh, noisefree limit cycles exhibited periodicities that were measured in both time and breaths from the simulation.

a Not strictly periodic but rather exhibited a chaotic behavior. Model size is m + n(m), see Eq. (2).

statistic values and one can check that the original

have atypical value. If the original data have atypical

statistics, then we reject the hypothesis that the sys-

tem that generated the original data is of the assumedclass. One always begins with simple assumptions and

progresses to more sophisticated models if the data

are inconsistent with the surrogate data. Theilers [4]

original paper frames surrogate analysis in the con-

text of noise-driven linear systems; a recent paper [17]

presents surrogate analysis in a more general context,

similar to the above.

In our case, however, we are not interested in

determining what type of system generated the data

at least not at present (we have considered this else-

where [2]). A simpler null hypothesis (for example

[15,16]) consistent with the data does not concern ushere. What is of greater interest to us is determin-

ing if the models really do behave like the data. By

calculating models and generating free-run predic-

tions from those models, we are in fact generating

surrogate data. The similarity of the value of various

statistics applied to data and surrogate can be used to

gauge the accuracy of the model. Fig. 9 shows calcu-

lations of correlation dimension estimates (following

the methods of Judd [18,19]) for data and surrogate.

Our calculations indicate a very close agreement be-

tween the correlation dimension of the data and that of

the simulations. In six of the seven data sets the corre-

lation dimension estimate dc(0) for the data is within

two standard deviations of the mean value of dc(0)

estimated from the ensemble of surrogates for all val-

ues of0 for which both converged. In the remaining

data set the value of correlation dimension differed by

more than two standard deviations only at the small-est values of 0 (the finest detail in the data). In all

calculations, dc(0) for the data is within three stan-

dard deviations of the mean value of dc(0) estimated

from the ensemble of surrogates. With respect to cor-

relation dimension, our models are producing results

virtually indistinguishable from the data.

4.3.2. Application

Previously [3], we have used a form of reduced

autoregressive modeling (RARM) to detect CAM in

the regular breathing of infants during quiet sleep. We

apply nonlinear modeling methods here with two aimsin mind: to demonstrate the accuracy of our modeling

methods; and to further demonstrate that CAM evident

during periodic breathing and in response to apnea or

sigh is also present during quiet, regular breathing.

We have built nonlinear models following the meth-

ods outlined in this paper of the regular respiration

of six sleeping infants immediately preceding seven

sighs and the consequential onset of periodic or CAM

respiration. For each of these models we produce sim-

ulations both driven by Gaussian noise, and without

noise. The noiseless simulations approach a stable pe-

riodic (or chaotic, quasi-periodic) orbit which may ex-

hibit slight CAM. Table 2 summarizes the results of

these calculations.

7/27/2019 Physic Ad 117

15/16


In all but one data set, CAM was present in the free-

run prediction of the nonlinear model. The absence of

CAM in one model may either indicate a lack of mea-

surable CAM in the data or a poor model (the data

are illustrated in Fig. 8). All other data sets produced

nonlinear models that exhibited CAM, the period of

which matched that observed after a sigh during visu-ally apparent CAM.

5. Conclusion

We have successfully modified and applied pseudo-

linear modeling techniques suggested by Judd and

Mees [1] to respiratory data from human infants. We

found that the initial modeling procedure had some

difficulties capturing all the anticipated features of res-

piratory motion (it was not periodic). Some alteration

to the algorithm and a considerable increase to compu-tational time provided results which display dynamics

very similar to those observed during respiration of

infants in quiet sleep (not only did the models exhibit

a periodic limit cycle, but its shape was very similar

to the data).

Correlation dimension and the methods of surro-

gate data demonstrated that the models did indeed pro-

duce simulations with qualitative dynamical features

indistinguishable from the data. Short term free-run

predictions appeared to behave similarly to the data;

and, most significantly, we were able to deduce the

presence of CAM in sections of quiet sleep precedingsighs by observing this behavior in free-run predic-

tions of models built from that data. This confirms our

earlier observations from linear models of tidal vol-

ume [3] and the observation of a (greater than) two-

dimensional attractor in reconstructions from data [2].

Based on the results of Section 4, we are able to

deduce that some of the alterations (specifically ex-

tending the class of basis functions, and directed basis

selection) improved short-term prediction. Other alter-

ations reduced the size of the model (accurate approx-

imation to description length) and improved free-run

dynamics (extending the class of basis function, local

optimization and linear modeling methods to predict

embedding strategies). A combination of these meth-

ods is required to produce an accurate model of the

dynamics.

We conclude that the modeling methods presented

here and in [1] are capable of accurate modeling

breathing dynamics (along with a wide variety of other

phenomena, see for example [20]). Furthermore, we

have presented some evidence that the CAM presentduring periods of periodic breathing (when tonic

drive is reduced) is also present, but more difficult to

observe, during eupnea (normal respiration).

Acknowledgements

We wish to thank Madeleine Lowe and Stephen

Stick of Princess Margaret Hospital for Children for

supplying the infant respiratory data, and for physio-

logical guidance.

References

[1] K. Judd, A. Mees, On selecting models for nonlinear timeseries, Physica D 82 (1995) 426444.

[2] M. Small, K. Judd, M. Lowe, S. Stick, Is breathingin infants chaotic? Dimension estimates for respiratory

patterns during quiet sleep, J. Appl. Physiol., to appear.[3] M. Small, K. Judd, S. Stick, Linear modelling techniques

detect periodic respiratory behaviour in infants duringregular breathing in quiet sleep, Am. J. Resp. Crit. CareMed., 153 (1996) A79.

[4] J. Theiler, S. Eubank, A. Longtin, B. Galdrikian, J.D.Farmer, Testing for nonlinearity in time series: The method

of surrogate data, Physica D 58 (1992) 7794.[5] M. Small, K. Judd, Detecting nonlinearity in experimental

data, Int. J. Bifurc. & Chaos 8 (1998), in press.[6] D. H. Kelly, D.C. Shannon, Periodic breathing in infants

with near-miss sudden infant death syndrome, Pediatrics63 (1979) 355360.

[7] W.B. Mendelson, Human Sleep: Research and ClinicalCare, Plenum Medical Book Company, 1987.

[8] F. Takens, Detecting strange attractors in turbulence,Lecture Notes in Mathematics, vol. 898, 1981, pp. 366381.

[9] H.D.I. Abarbanel, R. Brown, J.J. Sidorowich, L.S.Tsimring, The analysis of observed chaotic data in physical

systems, Rev. Mod. Phys. 65 (4) (1993) 13311392.[10] J. Rissanen, Stochastic Complexity in Statistical Inquiry,

World Scientific, Singapore, 1989.

[11] M.B. Priestly, Non-linear and Non-Stationary Time SeriesAnalysis, Academic Press, New York, 1989.

[12] M.B. Kennel, R. Brown, H.D.I. Abarbanel, Determining

embedding dimension for phase-space reconstruction using

7/27/2019 Physic Ad 117

16/16


a geometric construction, Phys. Rev. A 45 (1992) 34033411.

[13] K. Judd, A. Mees, Embedding as a modeling problem,Physica D, to appear.

[14] G. Schwarz, Estimating the dimension of a model, Ann.Stat. 6 (1978) 461464.

[15] J. Theiler, On the evidence for low-dimensional chaos in an

epileptic electroencephalogram, Phys. Lett. A 196 (1995)

335341.[16] J. Theiler, P. Rapp, Re-examination of the evidence

for low-dimensional, nonlinear structure in the humanelectroencephalogram, Electroencephalogr. Clin. Neuro-

physiol. 98 (1996) 213222.

[17] J. Theiler, D. Prichard, Constrained-realization Monte-Carlo method for hypothesis testing, Physica D 94 (1996)221235.

[18] K. Judd, An improved estimator of dimension and somecomments on providing confidence intervals, Physica D

56 (1992) 216228.[19] K. Judd, Estimating dimension from small samples,

Physica D 71 (1994) 421429.

[20] K. Judd, A. Mees, Modeling chaotic motions of a stringfrom experimental data, Physica D 92 (1996) 221236.

physic ad 117

Documents