1 the venzke et al. * optimal detection analysis jeff knight * venzke, s., m. r. allen, r. t. sutton...
TRANSCRIPT
1
The Venzke et al.* Optimal Detection Analysis
Jeff Knight
*Venzke, S., M. R. Allen, R. T. Sutton and D. P. Rowell, The Atmospheric Response over the North Atlantic to Decadal
Changes in Sea Surface Temperature, J. Clim, 12, 2562-2584, 1999.
2
Ensembles
Initial-value ensembles of climate model calculations allow estimation of the climate response to forcings, as multiple realisations of the chaotic noise inherent in the climate system are available. The independence of these noise components in each simulation allows it to be ‘averaged out’ by the formation of an ensemble mean.
3
Ensemble means
For data X from calculation k:
Xk = XM + XNk,
where XM is the ensemble mean and XNk
is the noise in k.
For large enough ensemble size n, the forced signal in X is
XF = E(XM) XM.
4
EOF analysisA common way to analyse the forced model response is principal component analysis (PCA, or EOF analysis) of the ensemble mean. PCA aims to find the covarying spatial patterns that have maximum variance in the data. In essence we can treat the mean XM as a set of time
series for each point,
XM(i, j) ; i = 1, l ; j = 1, m
i.e. the arrangement of points irrelevant.
5
SVD analysis
In general, an l×m matrix can be decomposed
Xl,m = El,m Sm,m PTl,m,
where the columns of E are the eof patterns of X, S is a diagonal matrix of the variance coefficients (square eigenvalues) and P contains the associated principal component time series. A Singular Value Decomposition.
6
The trouble with this…
EOF analysis of means of realistic size climate ensembles will only give accurate estimates of the EOFs of XF if the noise is not
spatially coherent. But this is not the case. Even worse, it is likely that the patterns of noise are in fact quite similar to the signal (e.g. NAO). So the ensemble mean EOFs are heavily biased away from the forced response towards the noise.
7
The way forward
Ensemble averaging throws out a lot of the information about the spatial characteristics of the noise. The aim of the Venzke technique is to estimate the properties of the noise so better account of its influence can be made in estimating the forced EOFs.
8
Technique
We can identify the noise in each experiment:
XNk = Xk - XM,
Rewrite the noise XN as the concatenation of
each run XNk:
XN = {XN1(i, j), XN2(i, j), … , XNn(i, j)},
i = 1, l ; j = 1, m,
making a matrix of size lnm.
9
Noise EOFs
Next, compute the EOFs of the noise by SVD:
XN = ENSNPNT,
so the columns of EN are the EOF patterns of the
noise. These patterns can be used to define a new BASIS for the ensemble mean, i.e. we seek to express the ensemble mean in terms of these EOF patterns. We choose the best defined patterns to do this.
10
Prewhitening Function
If we define a matrix function,
F = n½EN() (SN
())-1,
and its (pseudo-) inverse,
F(-1) = n-½EN() SN
(),
we notice that multiplying by F projects onto the noise EOFs and scales the result by the inverse singular values, or the variance.
11
Prewhitening
So the transform
XM = FTXM
converts the ensemble mean into a set of
time series of projections of the mean on the noise EOFs, such that each EOF has the same weight in the result. In other words, we have transformed the data into a basis made of these EOFs.
12
EOFs of transformed mean
As we now have a mean in which the signals lie amongst unit-variance noise EOFs, we can isolate them by further EOF analysis:
XM = ESPT,
so that E’ contains the EOFs under this new basis. But beware! The weighting at each point in the EOF relates to the strength of each noise pattern, not the spatial weight.
13
Back to physical space
To get back to a physical basis from the noise EOF basis, we need to invert the transform based on noise EOFs,
Ê = F(-1)E,
which also rescales, giving us back an l× matrix i.e. the truncated number of optimal EOF patterns of l points each. Projection on the mean gives the principal components.
14
To summarise
Concatenate the noise estimates for each realisation, XNk
Derive joint noise EOF patterns, EN
Project these patterns onto the ensemble mean, scaling the EOFs by their variance
Find the signal by deriving EOFs in this basis Transform these EOFs back to physical space
15
An Example
Courtesy of Rowan Sutton
We create a synthetic signal on a 20×20 grid (l = 400) by taking a simple harmonic pattern and modulating through m = 50 times with a sine of period 20.
16
Signal Pattern
17
Noise Patterns
Generate random noise by associating a random time series with each 2D harmonic pattern Hi,j, for i,j = 1,5, and
weighting the variance by e-(i+j).
Repeat this for n = 6 ensemble members.
18
Noise Patterns
19
Synthetic Data
20
Ensemble Mean EOFs
21
Noise EOFs
22
EOFs in the noise basis
23
Optimised 1st EOF
24
Conclusions
Venzke et al. find that Principal Component Analysis of the mean of realistic ensembles is heavily contaminated by the noise and so provides a poor estimate of the EOFs of the forced response.
By using information about the spatial characteristics of the noise in the ensemble, these estimates can be improved.