Wavelet methods

Peter F. Craigmile

http://www.stat.osu.edu/~pfc/

Summer School on Extreme Value Modeling and Water Resources

Universite Lyon 1, France. 13-24 Jun 2016

Thank you to The Camille Jordan Institute, the MultiRisk LEFE project,

and the Labex Milyon. My research is partially supported by the US National Science

Foundation under grants NSF-DMS-1407604 and NSF-SES-1424481.

1

Introduction (again)

• We use spectral analysis to view a time series in terms of a sum of sinusoids at different

frequencies.

– Unless we do something “sneaky”, we lose all information about time.

• A wavelet transformation uses wavelets (“small waves”) to decompose a time series

into:

1. averages over a specified (typically long) time scale, and

2. changes of averages over a range of shorter time scales.

• It is a time-scale (some say frequency) decomposition.

• Thus wavelet methods are a natural way to explore and model non-stationarity.

2

Some useful references

• For time series: Vidakovic [1998] and Percival and Walden [2000].

• For gridded spatial processes (images): Gonzalez and Woods [2001], Mondal and Percival

[2012], Geilhufe et al. [2013]

• Berliner et al. [1999], Wikle et al. [2001], and Craigmile and Guttorp [2011] use hierarchical

Bayesian space-time wavelet-based models.

• Examples from the analysis of longitudinal data: Brown et al. [2001], Morris et al. [2003],

Vannucci et al. [2003], Morris and Carroll [2006], Morris et al. [2006], Ramirez and Vidakovic

[2007].

3

The discrete wavelet transform (DWT)

• Suppose {Xt} is a time series, observed from time 0 to N − 1 (slight change of notation).

Let X = (X0 . . . XN−1)T .

(assume N divisible by 2J , for some J).

• Most simply, the DWT coefficients are

W = WX,

where W = N ×N is an orthonormal DWT matrix.

• In practice use pyramid algorithm [Mallat, 1989] to calculate W .

4

Defining the wavelet and scaling filter

• We define two filters:

1. {hl} is a wavelet (mother) filter of even length L,

i.e.,∑

l hl = 0,∑

l hlhl+2k = 1[k=0].

2. {gl = (−1)lhL−1−l} is the scaling (father) filter.

• There are many choices of wavelet filter we can use in practice.

– Each filter has different properties.

5

Daubechies wavelet filters

• A popular choice is of filters is theDaubechies class [Daubechies, 1992, Sect. 6.2], which

includes the extremal phase (D) and least asymmetric (LA) sets of wavelet filters.

– This class of orthogonal filters are characterized by the maximum number of vanishing

moments.

– The wavelet filter {hl}, of even width L, passes high frequencies but filters out low ones.

– It has L/2 vanishing moments.

6

Daubechies wavelet filters, cont.

−1.0

0.0

1.0

Wave

let filter ●

●

−1.0

0.0

1.0

Haar

Scalin

g filt

er ● ●

0.0 0.2 0.4

0.0

0.5

1.0

1.5

2.0

f

Wave

let S

q. G

ain

−1.0

0.0

1.0

Wave

let filter

●●

●

●

−1.0

0.0

1.0

D4

Scalin

g filt

er

●

●

●

●

0.0 0.2 0.4

0.0

0.5

1.0

1.5

2.0

f

Wave

let S

q. G

ain

−1.0

0.0

1.0

Wave

let filter

● ● ●●

●

●

●

●

−1.0

0.0

1.0

D8

Scalin

g filt

er

●

●●

●●

● ● ●

0.0 0.2 0.4

0.0

0.5

1.0

1.5

2.0

f

Wave

let S

q. G

ain

−1.0

0.0

1.0

Wave

let filter

● ●●

●

●

●

●●

−1.0

0.0

1.0

LA8

Scalin

g filt

er

● ●

●

●

●

●● ●

0.0 0.2 0.4

0.0

0.5

1.0

1.5

2.0

f

Wave

let S

q. G

ain

7

The pyramid algorithm

• Let V0,t = Xt for each t.

• For each j = 1, . . . , J calculate

Wj,t =

L−1∑

l=0

hl Vj−1,2t+1−l mod Nj;

Vj,t =

L−1∑

l=0

gl Vj−1,2t+1−l mod Nj,

for Nj = N/2j and t = 0, . . . , Nj − 1.

8

Partitioning the DWT coefficients

• Decomposition of W :

W = (W 1,W 2, . . . ,W J ,V J).

• W j are the Nj level j wavelet coefficients:

– associated with changes in averages on scale 2j−1;

– relates to times spaced 2j units

– is associated with the frequency interval

λj = [−1/2j,−1/2j+1) ∪ (1/2j+1, 1/2j]

(this follows by examining Fourier transforms of the wavelet and scaling filters).

• V J are the NJ scaling coefficients:

– associated with averages on scale 2J ;

– relates to times spaced 2J units apart.

9

Example: Malindi coral

1800 1850 1900 1950 2000

4.1

4.3

4.5

4.7

Year

Neg.

change in 1

8O

0 5 10 15 20

−0.2

0.2

0.6

1.0

Lag

AC

F

5 10 15 20

−0.2

0.2

0.6

1.0

Lag

Part

ial A

CF

• 194 year record of the δ18O oxygen isotope measured from a 4m high coral colony growing

at a depth of 6m (low tide) in Malindi, Kenya [Cole et al., 2000].

• A decrease in the oxygen value corresponds to an increase in the sea surface tem-

perature (SST) (roughly -0.24 per mil concentration corresponds to 1oC).

10

DWT plot of the Malindi coral series

1800 1850 1900 1950 2000

time

XW

1W

2W

3W

4V

4

11

Decomposing trend using wavelets

• Can also decompose W as

W = W s +W b +W nb.

W s : scaling coefficients and zeros elsewhere.

W b : wavelet coefficients affected by boundaries.

W nb : wavelet coefficients not affected by boundaries.

• Since X = WTW

X = WT (W s +W b) +WT (W nb)

= µ + ǫ,

where

µ : an estimate of trend;

ǫ : “tapered” estimate of error (missing some of the dependence).

12

A trend model with FD process errors

• Suppose that a model for the Malindi oxygen isotope series is

Xt = µt + ǫt,

where µt is a polynomial of degree K, and ǫt is a mean zero stationary Gaussian error

process.

• Suppose {ǫt} is a long memory process – the fractionally differenced (FD) process with

parameters d and σ2.

13

Estimating the model using a wavelet decomposition

• We use a filter of length L/2 ≥ max{K + 1, ⌊d + 1/2⌋}.

• Since the wavelet filter acts as a differencing filter:

– W nb does not contain the trend component;

– E(W nb) = 0;

– Can useW nb to estimate the FD process parameters (e.g., via least squares or maximum

likelihood).

• Can employ approximate schemes to estimate the process parameters [e.g., McCoy and

Walden, 1996, Craigmile et al., 2005, Moulines et al., 2008].

14

Returning to the Malindi example

• Using an approximate ML estimate:

d = 0.359 (95% CI of [0.143,0.597])

σ = 0.0667.

• Strong evidence of long range dependence.

– But cannot discriminate between a stationary (d < 1/2) or nonstationary FD process

(d ≥ 1/2).

15

Estimating the trend

1800 1850 1900 1950 2000

4.1

4.2

4.3

4.4

4.5

4.6

4.7

Year

Neg.

change in 1

8O

• Evidence that the oxygen concentration is above average for the latter years. (Monte Carlo

test for trend P-value < 0.001.)

• Can extend to nonpolynomial {µt} using wavelet shrinkage methods [e.g., Donoho and

Johnstone, 1994, Percival and Walden, 2000].

16

Limitations of the DWT

• Not shift invariant [e.g., Percival and Mofjeld, 1997].

• We lose time resolution by downsampling (the use of the ’2t’ in the filtering operation).

• Instead, we can use themaximum overlap discrete wavelet transform (MODWT).

• Let gl = gl/√2 and hl = hl/

√2. Then the pyramid algorithm that yields the MODWT

scaling and wavelet coefficient on level j is given by:

Vj,t =∑

l

gl Vj−1,t−l mod N ;

Wj,t =∑

l

hl Vj−1,t−l mod N ,

respectively.

• This transformation is shift invariant. On each level j, the wavelet or scaling coefficient

can be assigned to a specific time point (not t but a circular shift of t).

17

MODWT plot of the Malindi coral series

1800 1850 1900 1950 2000

time

XW

1W

2W

3W

4V

4

18

Decomposing the variance using wavelets

• Suppose we perform the MODWT (ignoring circularity) on the entire process, {Xt}, ana-

lyzing to all possible scales.

• Let Hj(·) denote the transfer function (Fourier transform) of the wavelet filter that would

yield the level j wavelet coefficients directly from {Xt}.

• Then the wavelet variance decomposes the variance of Xt:

var(Xt) =

∫ 1/2

−1/2

SX(f )df =

∫ 1/2

−1/2

∞∑

j=1

|Hj(f )|2SX(f )df

=

∞∑

j=1

∫ 1/2

−1/2

|Hj(f )|2SX(f )df

=

∞∑

j=1

var(Wj,t).

• This is a time-scale (frequency) decomposition of the variance since each wavelet coefficient

is associated with a certain time point and scale, λj.

19

Decomposing the variance continued

• Given data, simple to estimate var(Wj,t) using averages of the squared wavelet coefficients.

• Properties of these estimators for stationary (and some nonstationary) processes are well

known [Percival, 1995, Serroukh et al., 2000]

• Can also extend to gappy time series [Mondal and Percival, 2010] and to images [Mondal

and Percival, 2012, Geilhufe et al., 2013].

• This extends to the decomposition of

cov(Xt, Xt+h) and cov(Xt, Yt+h)

(and associated correlations) for certain processes {Xt} and {Yt} [Serroukh and Walden,

2000a,b, Whitcher et al., 2000].

• Interesting spatio-temporal extensions!

20

Wavelet-based analysis of variance for the daily Swedish temperatures

(See the last set of lectures!)

0 2 4 6 8 10

−2

02

4

log(ν

j2)

6: Sundsvalls FlygplatsDaily average temperature

log(τj)

0 2 4 6 8

−2

02

4

log(ν

j2)

10: Films Kyrkby ADaily average temperature

log(τj)

0 2 4 6 8 10

−2

02

4

log(ν

j2)

16: Kilsbergen−SuttarbodaDaily average temperature

log(τj)

0 2 4 6 8 10

−7

−5

−3

−1

log(ν

j2)

log(τj)

6: Sundsvalls FlygplatsDeseasonalized and standardized temp.

0 2 4 6 8

−7

−5

−3

−1

log(ν

j2)

log(τj)

10: Films Kyrkby ADeseasonalized and standardized temp.

0 2 4 6 8 10−

7−

5−

3−

1

log(ν

j2)

log(τj)

16: Kilsbergen−SuttarbodaDeseasonalized and standardized temp

21

Wavelet-based analysis of variance for the daily Swedish temperatures

5 10 15

0.1

00

.20

0.3

00

.40

Location ID

LR

D e

stim

ate

s

(a)

12 14 16 18

56

58

60

62

64

Longitude

La

titu

de

.202

.254

.166

.238 .2410

.2013 .3114

.1915 .2316

.191

.173

.165

.227

.2412

.1717

.209.2311

(b)

22

References

L. M. Berliner, C. K. Wikle, and R. F. Milliff. Multiresolution wavelet analyses in hierarchical Bayesian turbulence models. In P. Muller and B. Vidakovic, editors,

Bayesian Inference in Wavelet-based Models, pages 341–359, New York, 1999. Springer.

P. J. Brown, T. Fearn, and M. Vannucci. Bayesian wavelet regression on curves with application to a spectroscopic calibration problem. Journal of the American

Statistical Association, 96:398–408, 2001.

J. E. Cole, R. B. Dunbar, T. R. McClanahan, and N. A. Muthiga. Tropical pacific forcing of decadal SST variability in the western Indian ocean over the past

two centuries. Science, 287:617–619, 2000.

P. F. Craigmile and P. Guttorp. Space-time modelling of trends in temperature series. Journal of Time Series Analysis, 32:378–395, 2011.

P. F. Craigmile, D. B. Percival, and P. Guttorp. Wavelet based estimation for polynomial contaminated fractionally differenced processes. IEEE Transactions on

Signal Processing, 53:3151–3161, 2005.

I. Daubechies. Ten Lectures on Wavelets. CBMS-NSF Series in Applied Mathematics. SIAM, Philadelphia, 1992.

D. L. Donoho and I. M. Johnstone. Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81(3):425–455, 1994.

M. Geilhufe, D. B. Percival, and H. L. Stern. Two-dimensional wavelet variance estimation with application to sea ice SAR images. Computers and Geosciences,

54(C):351–360, Apr. 2013.

R. C. Gonzalez and R. E. Woods. Digital Image Processing (Second Edition). Prentice Hall, Upper Saddle River, New Jersey, 2001.

S. Mallat. A theory for multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11:

674–693, 1989.

E. J. McCoy and A. T. Walden. Wavelet Analysis and Synthesis of Stationary Long-Memory Processes. Journal of Computational and Graphical Statistics, 5:

26–56, Mar. 1996.

D. Mondal and D. Percival. Wavelet variance analysis for gappy time series. Annals of the Institute of Statistical Mathematics, 62:943–966, 2010.

D. Mondal and D. B. Percival. Wavelet variance analysis for random fields on a regular lattice. Image Processing, IEEE Transactions on, 21(2):537–549, 2012.

J. S. Morris and R. J. Carroll. Wavelet-based functional mixed models. Journal of the Royal Statistical Society, Series B: Statistical Methodology, 68:179–199,

2006.

J. S. Morris, M. Vannucci, P. J. Brown, and R. J. Carroll. Wavelet-based nonparametric modeling of hierarchical functions in colon carcinogenesis. Journal of

the American Statistical Association, 98:573–583, 2003.

J. S. Morris, C. Arroyo, B. Coull, L. M. Ryan, R. Herrick, and S. L. Gortmaker. Using wavelet-based functional mixed models to characterize population

23

heterogeneity in accelerometer profiles: A case study. Journal of the American Statistical Association, 101:1352–1364, 2006.

E. Moulines, F. Roueff, and M. S. Taqqu. A wavelet whittle estimator of the memory parameter of a nonstationary Gaussian time series. The Annals of Statistics,

36:1925–1956, 2008.

D. Percival and A. Walden. Wavelet Methods for Time Series Analysis. Cambridge University Press, Cambridge, England, 2000.

D. B. Percival. On Estimation of the Wavelet Variance. Biometrika, 3:619–631, 1995.

D. B. Percival and H. Mofjeld. Analysis of Subtidal Coastal Sea Level Fluctuations Using Wavelets. JASA, 92(439):868–880, 1997.

P. Ramirez and B. Vidakovic. Wavelet-based 2D multifractal spectrum with applications in analysis of digital mammography images. Technical report, Department

of Biomedical Engineering, Georgia Tech., 2007.

A. Serroukh and A. T. Walden. Wavelet scale analysis of bivariate time series I: Motivation and estimation. Journal of Statistical Planning and Inference, 13:

1–36, 2000a.

A. Serroukh and A. T. Walden. Wavelet scale analysis of bivariate time series II: Statistical properties for linear processes. Journal of Statistical Planning and

Inference, 13:37–56, 2000b.

A. Serroukh, A. T. Walden, and D. B. Percival. Statistical properties and uses of the wavelet variance estimator for the scale analysis of time series. JASA, 95:

184–196, 2000.

M. Vannucci, P. Brown, and T. Fearn. A decision theoretical approach to wavelet regression on curves with a high number of regressors. Journal of Statistical

Planning and Inference, 112:195–212, 2003.

B. Vidakovic. Statistical Modeling by Wavelets. Wiley, New York, NY, 1998.

B. Whitcher, P. Guttorp, and D. B. Percival. Wavelet analysis of covariance with application to atmospheric time series. Journal of Geophysical Research, 105

(D11):14941–14962, 2000.

C. K. Wikle, R. F. Milliff, D. Nychka, and L. M. Berliner. Spatiotemporal hierarchical Bayesian modeling tropical ocean surface winds. Journal of the American

Statistical Association, 96:382–397, 2001.

24