robust scale and autocovariance estimation

Robust scale and autocovariance estimation

Garth Tarr, Samuel Muller and Neville Weber

School of Mathematics and Statistics

THE UNIVERSITY OF SYDNEY

EMS 2013

Scale Autocovariance Conclusion

Outline

Robust Scale Estimator, Pn

Covariance

Autocovariance

Long RangeDependence

Short RangeDependence

Inverse CovarianceMatrix Estimation

Covariance Matrix


Outline

The robust scale estimator Pn

Autocovariance estimation using Pn

Conclusion and key references


History of scale efficiencies at the Gaussian (n = 20)R

elat

ive

Effi

cien

cy

Year

SD

1894

1.00

1931 1974

0.38

1993

0.67

2012

0.85



elat

ive

Effi

cien

cy

Year

SD

1894

1.00

IQR

1931

MAD

1974

0.38

1993

0.67

2012

0.85



elat

ive

Effi

cien

cy

Year

SD

1894

1.00

IQR

1931

MAD

1974

0.38

Qn

1993

0.67

2012

0.85



elat

ive

Effi

cien

cy

Year

SD

1894

1.00

IQR

1931

MAD

1974

0.38

Qn

1993

0.67

Pn

2012

0.85


Pairwise mean scale estimator: Pn

• Consider the U -statistic, based on the pairwise mean kernel,

Un(X) :=

(n

2

)−1∑i<j

Xi +Xj

2.

• Let H(t) = P ((Xi +Xj)/2 ≤ t) be the cdf of the kernelswith corresponding empirical distribution function,

Hn(t) :=

(n

2

)−1∑i<j

I{Xi +Xj

2≤ t}, for t ∈ R.

Definition (Interquartile range of pairwise means)

Pn = c[H−1n (0.75)−H−1

n (0.25)],

where c ≈ 1.048 is a correction factor to ensure Pn is consistentfor the standard deviation when the underlying observations areGaussian.


Influence curve

• Hampel (1974) defines the influence curve for a functional Tat distribution F as

IC(x;T, F ) = limε↓0

T ((1− ε)F + εδx)− T (F )

ε

where δx has all its mass at x.• Serfling (1984) outlines the IC for GL-statistics.

Influence curve for Pn (Tarr, Muller and Weber, 2012)

Assuming that F has derivative f > 0 on [F−1(ε), F−1(1− ε)] forall ε > 0,

IC(x;P , F ) = c

[0.75− F (2H−1

F (0.75)− x)∫f(2H−1

F (0.75)− x)f(x) dx

−0.25− F (2H−1

F (0.25)− x)∫f(2H−1

F (0.25)− x)f(x) dx

].


Influence curves when F = Φ

−4 −2 0 2 4

−1

01

2

x

IC(x

;T,Φ

)

Pn



−4 −2 0 2 4

−1

01

2

x

IC(x

;T,Φ

)

PnSD



−4 −2 0 2 4

−1

01

2

x

IC(x

;T,Φ

)

Pn

MAD

SD



−4 −2 0 2 4

−1

01

2

x

IC(x

;T,Φ

)

Pn

MAD

Qn

SD


Asymptotic variance and relative efficiency

• Tarr, Muller and Weber (2012) show that when the underlyingobservations are independent, Pn is asymptotically normalwith variance, V , given by the expected square of theinfluence function.

• When the underlying data are independent Gaussian,

V (Pn,Φ) =

∫IC(x;P ,Φ)2 d Φ(x) = 0.579.

• This equates to an asymptotic efficiency of 0.86 as comparedwith 0.82 for Qn and 0.37 for the MAD.

! But how does it compare at heavier tailed distributions?


Asymptotic relative efficiency when f = tν for ν ∈ [1, 10]

1 2 3 54 76 98 10

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Degrees of freedom

Asy

mp

toti

cre

lati

veeffi

cien

cy

Pn



1 2 3 54 76 98 10

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Degrees of freedom

Asy

mp

toti

cre

lati

veeffi

cien

cy

Pn

MAD



1 2 3 54 76 98 10

0.4

0.5

0.6

0.7

0.8

0.9

1.0

Degrees of freedom

Asy

mp

toti

cre

lati

veeffi

cien

cy

Pn

MAD

Qn


Asymptotic distribution under short range dependence

Assumption 1: Gaussian SRD

Let {Xi}i≥1 be a stationary mean-zero Gaussian process withautocovariances γ(h) = E(X1Xh+1) satisfying

∑h≥1 |γ(h)| <∞.

Result

Under Assumption 1, it can be shown that,

√n(Pn − σ) =

cΦ√n

n∑i=1

IC(Xi, P ,Φ) + op(1),

hence applying a limit theorem from Arcones (1994) we have√n(Pn − σ)

D−→ N (0, σ2),

where σ =√γ(0) and

σ2 = E(IC2(X1, P ,Φ)

)+ 2

∑k≥1

E (IC(X1, P ,Φ)IC(Xk+1, P ,Φ)) .


Asymptotic distribution under long range dependence

Assumption 2: Gaussian LRD

Let {Xi}i≥1 be a stationary mean-zero Gaussian process withautocovariance sequence γ(h) = E(X1Xh+1) satisfyingγ(h) = h−DL(h), 0 < D < 1, where L is slowly varying at infinity.

Result

Under Assumption 2,

1. If D > 1/2,√n(Pn − σ)

D−→ N (0, σ2),

σ2 = E IC2(X1, P ,Φ) + 2∑k≥1

E IC(X1, P ,Φ)IC(Xk+1, P ,Φ).

!Note that this is the same as the SRD result, however theproof is somewhat more involved.



Assumption 2: Gaussian LRD

Let {Xi}i≥1 be a stationary mean-zero Gaussian process withautocovariance sequence γ(h) = E(X1Xh+1) satisfyingγ(h) = h−DL(h), 0 < D < 1, where L is slowly varying at infinity.

Conjecture

Under Assumption 2,

2. If D < 1/2,

k(D)nDL(n)−1(Pn − σ)D−→ σ

2

(Z2,D(1)− Z2

1,D(1)),

where k(D) = B((1−D)/2, D) and B denotes the betafunction Z1,D is the standard fBm process and Z2,D is theRosenblatt process.

!This is consistent with results for other scale estimators,e.g. SD and Qn (Levy-Leduc et. al., 2011) and the IQR...


Interquartile range result

Consider equivalent result for the interquartile range,

Tn(x) = cΦ

(F−1n (3/4)− F−1

n (1/4)).

Result

Under Assumption 2 with D < 1/2, Tn satisfies the following limittheorem as n→∞:

k(D)nDL(n)−1(Tn − σ)D−→ σ

2

(Z2,D(1)− Z2

1,D(1)).

• In establishing this result we used Wu’s (2005) Bahadurrepresentation for sample quantiles under LRD:

F−1n (p)− r =

p− Fn(r)

φ(r)+X2n

2

φ′(r)

φ(r)+O(nh(D)L1(n)).

!It appears to be non-trivial to extend this result to the pairwisemean empirical distribution function.


Empirical densities ARFIMA(0, 0.1, 0); D = 0.8; n = 5000

0

10

20

30

40

0.95 1.00 1.05

Scale estimate

Den

sity

Pn

87%SD Relative Efficiency

Qn 83%IQR 39%MAD 39%



0

10

20

30

40

0.95 1.00 1.05

Scale estimate

Den

sity

Pn 87%SD Relative Efficiency




0

10

20

30

40

0.95 1.00 1.05

Scale estimate

Den

sity


Qn 83%

IQR 39%MAD 39%



0

10

20

30

40

0.95 1.00 1.05

Scale estimate

Den

sity


Qn 83%IQR 39%

MAD 39%



0

10

20

30

40

0.95 1.00 1.05

Scale estimate

Den

sity





0

2

4

1.2 1.4 1.6 1.8 2.0

Scale estimate

Den

sity

Pn





0

2

4

1.2 1.4 1.6 1.8 2.0

Scale estimate

Den

sity





0

2

4

1.2 1.4 1.6 1.8 2.0

Scale estimate

Den

sity


Qn 94%

IQR 82%MAD 83%



0

2

4

1.2 1.4 1.6 1.8 2.0

Scale estimate

Den

sity


Qn 94%IQR 82%

MAD 83%



0

2

4

1.2 1.4 1.6 1.8 2.0

Scale estimate

Den

sity




Empirical densities with 2% contamination at 5

0

2

4

1.2 1.4 1.6 1.8 2.0

Scale estimate

Den

sity

Pn

SD

QnIQRMAD



0

2

4

1.2 1.4 1.6 1.8 2.0

Scale estimate

Den

sity

Pn

SD

QnIQRMAD


Outline





From scale to autocovariance

• Gnanadesikan and Kettenring (1972) highlighted an identitythat relates scale and covariance.

• In the autocovariance setting,

γ(h) = cov(X1, Xh+1) =1

4[var(X1+Xh+1)− var(X1−Xh+1)] .

• For a series of n observations, Xn = {Xt}1≤t≤n,

γP (h) =1

4

[P 2n−h(X1:n−h+Xh+1:n)− P 2

n−h(X1:n−h−Xh+1:n)].

where X1:n−h are the first n− h observations in Xn andXh+1:n are the last n− h observations.

• The same technique can be used to turn other scaleestimators into covariance and correlation estimators, e.g. Maand Genton (2000) study γQ based on Qn.


Robustness properties

• Following Genton and Ma (1999), we have shown thatinfluence curve, IC((x, y), γP ,Φ), and therefore the grosserror sensitivity of γP can be derived from the IC of Pn.

x−4 −2 0 2 4

y−4

−20

24

IC(x,y)

−3−2−10123

IC((x,y),Pn,Φ)

! IC is bounded and hence the gross error sensitivity is finite.


Breakdown value

• Ma and Genton (2000) show that the breakdown value forautocovariance estimators is (roughly) half that of thecorresponding covariance estimator.

• Consider n = 13

, autocovariance at lag h = 2

• Working with X1:11 ±X3:13

• 4 contaminated observations, denoted by

• Leaves only 3 uncontaminated pairs, denoted by

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11X9 X10 X11 X12 X13

h = 2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X1 X2

h = 2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X11 X12 X13

! γP (h) has roughly half the 13.4% breakdown value of Pn.


Breakdown value


• Consider n = 13, autocovariance at lag h = 2




X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11X9 X10 X11

X12 X13

h = 2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X1 X2

h = 2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X11 X12 X13



Breakdown value






X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11

X9 X10 X11

X12 X13

h = 2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X1 X2

h = 2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X11 X12 X13



Breakdown value






X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11

X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11X9 X10 X11 X12 X13

h = 2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13

X1 X2

h = 2

X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13X11 X12 X13



Asymptotic distribution under short range dependence

Result

If {Xi}i≥1 satisfy Assumption 1 (SRD) then the sequences{Xi +Xi+h}i≥1 and {Xi −Xi+h}i≥1 also satisfy Assumption 1,hence we can use the previous results and apply the functionaldelta method to show,

√n(γP (h)− γ(h))

D−→ N (0, σ2(h)),

where

σ2(h) = E[IC2((X1, X1+h), γP ,Φ)

]+ 2

∑k≥1

E [IC((X1, X1+h), γP ,Φ) IC((Xk+1, Xk+1+h), γP ,Φ)] .



Result

Under Assumption 2 (LRD),

1. D > 1/2 (same limiting distribution as SRD):

√n(γP (h)− γ(h))

D−→ N (0, σ2(h)).

2. D < 1/2 (based on conjectured result):

k(D)nD

L(n)(γP (h)−γ(h))

D−→ γ(0) + γ(h)

2(Z2,D(1)− Z2

1,D(1)),

where k(D) = Beta((1−D)/2, D), Z1,D is the standard fBmprocess, Z2,D is the Rosenblatt process and

L(n) = 2L(n)+L(n+h)(1+h/n)−D+L(n−h)(1−h/n)−D.


Empirical densities γ•(1) ARFIMA(0, 0.1, 0); D = 0.8; n = 5000

0

10

20

0.95 1.00 1.05 1.10First order autocovariance estimate

Den

sity

Pn

87%SD Relative Eff.




0

10

20


Den

sity

Pn 87%SD Relative Eff.




0

10

20


Den

sity


Qn 83%

IQR 39%MAD 39%



0

10

20


Den

sity


Qn 83%IQR 39%

MAD 39%



0

10

20


Den

sity





0.0

0.5

1.0

1.5

1.5 2.0 2.5 3.0 3.5 4.0 4.5First order autocovariance estimate

Den

sity

Pn





0.0

0.5

1.0

1.5


Den

sity





0.0

0.5

1.0

1.5


Den

sity


Qn 94%

IQR 82%MAD 83%



0.0

0.5

1.0

1.5


Den

sity


Qn 94%IQR 82%

MAD 83%



0.0

0.5

1.0

1.5


Den

sity




Empirical densities γ•(1) with 2% contamination at 5

0.0

0.5

1.0

1.5


Den

sity

Pn

SD

QnIQRMAD



0.0

0.5

1.0

1.5


Den

sity

Pn

SD

QnIQRMAD


Outline





Summary

1. Aim

• Efficient and robust scale and autocovariance estimation.

2. Method

• Pn scale estimator transformed using the GK identity.

3. Results

• 86% asymptotic efficiency at the Gaussian and highasymptotic efficiency at heavier tailed distributions.

• Robustness properties transfer from Pn to γP .

• In certain LRD settings robust estimators have very highefficiencies relative to the standard deviation.


References

Genton, M. and Ma, Y. (1999).

Robustness properties of dispersion estimators.

Statistics & Probability Letters, 44(4):343–350.

Gnanadesikan, R. and Kettenring J. R. (1972).

Robust estimates, residuals and outlier detection with multiresponse data

Biometrics, 28(1):81–124.

Hampel, F. (1974).

The influence curve and its role in robust estimation.

Journal of the American Statistical Association, 69(346):383–393.

Levy-Leduc, C., Boistard, H., Moulines, E., Taqqu, M.S. and Reisen, V.A.(2011).

Robust estimation of the scale and of the autocovariance function ofGaussian short- and long-range dependent processes

Journal of Time Series Analysis, 32(2):135–156.


References

Ma, Y. and Genton, M. (2000).

Highly robust estimation of the autocovariance function

Journal of Time Series Analysis, 21(6):663–684.

Serfling, R. J. (1984).

Generalized L-, M -, and R-statistics.

The Annals of Statistics, 12(1):76–86.

Tarr, G., Muller, S. and Weber, N.C., (2012).

A robust scale estimator based on pairwise means.

Journal of Nonparametric Statistics, 24(1):187–199.

Wu, W.B., (2005).

On the Bahadur representation of sample quantiles for dependentsequences.

Annals of Statistics, 33(4):1934–1963.

robust scale and autocovariance estimation

Documents