the profound impact of negative power law noise on the estimation of causal behavior

The Profound Impact of Negative The Profound Impact of Negative Power Law Noise on the Estimation Power Law Noise on the Estimation

of Causal Behaviorof Causal BehaviorVictor S. ReinhardtVictor S. Reinhardt

Raytheon Space and Airborne SystemsRaytheon Space and Airborne SystemsEl Segundo, CA, USAEl Segundo, CA, USA

2009 Joint Meeting of the European Frequency and 2009 Joint Meeting of the European Frequency and Time Forum and the IEEE International Frequency Time Forum and the IEEE International Frequency

Control Symposium, Besançon, FranceControl Symposium, Besançon, France

EFTF-IFCS 2009 -- V. Reinhardt

Introduction

• Well-known that Well-known that correlated or systematic correlated or systematic noisenoiseCannot be properly separated from Cannot be properly separated from true causal true causal

behaviorbehavior imbedded in noisy data imbedded in noisy dataBy By any any fitting or estimation techniquefitting or estimation technique i.e., Least Squares Fit (LSQF) or Kalman filteri.e., Least Squares Fit (LSQF) or Kalman filter

• But the profound implications of this But the profound implications of this In dealing with In dealing with highly correlated highly correlated negative negative

power law (neg-p) noise power law (neg-p) noise Have not been fully appreciatedHave not been fully appreciatedNeg-p Neg-p PSD PSD LLpp(f) (f) |f| |f|

pp for for p < 0p < 0• This paper will investigate these implicationsThis paper will investigate these implications


Introduction

• Paper will 1Paper will 1stst show that such neg-p correlations show that such neg-p correlations Lead such fitting techniques Lead such fitting techniques To generate To generate anomalousanomalous fit solutions fit solutionsAnd And faultyfaulty estimates of the estimates of the true fit errorstrue fit errors

• Then it will explore profound consequences of Then it will explore profound consequences of this in a variety of areasthis in a variety of areas


x(t)

t

x(tn)

Over Data Observation Interval T = Nts (tn=nts)

Model We Will Use for Estimating Causal Behavior in Noisy Data

• x(t) = Any data variable representing a measurable output from an instrument

•N Samples of Data x(tn) Can also be fn of (t) but x(t,(t)) x(t) for most of paper


Model We Will Use for Estimating Causal Behavior in Noisy Data• For example For example x(t) = xx(t) = xoscosc(t) – x(t) – xoscosc(t-(t-)) is is

observableobservable output of 2-way ranging Rx output of 2-way ranging Rx• Where Where xxoscosc(t)(t) is the is the unobservableunobservable true time true time

error of the reference oscillatorerror of the reference oscillator


x(t)

t

x(t)



• x(t) is sum of (true) causal behavior xc(t)• plus (true) noise xr(t)

(True) Causal Behavior xc(t)

(True) Noise xr(t) = xc(t) + xr(t)


x(t)

t

x(t)



• Fitting process generates xa,M(t) an M-Parameter estimate of xc(t)

(True) Causal Behavior xc(t)

xa,M(t) = M-Parameter Estimate of xc(t)

(True) Noise xr(t) = xc(t) + xr(t)


x(tn)xc(t) xa,M(tn) Data interval T

x(t)

t

xw,M(tn) = xa,M(tn) – xc(tn)xw,M(t) only detailed measure of the true accuracy

The Accuracy of the Causal Estimate is xw,M(tn)

• Point Variance of xw,M(tn) (Kalman) (E xr = 0) ) w,M(tn)2 = E xw,M(tn)2 E = Ensemble average

• Weighted Average Variance over T (LSQF) w,M

2 = n n w,M(tn)2 n = data weighting


A Major Thesis of This Paper is

• Cannot substitute other error measures for Cannot substitute other error measures for w,Mw,M(t(tnn))22 or or w,Mw,M

22 Just because they diverge due to neg-p noiseJust because they diverge due to neg-p noise

• And that such divergences are in fact indicators And that such divergences are in fact indicators of a real inaccuracy problem with the fit solutionof a real inaccuracy problem with the fit solution


x(t)

tx(tn)xc(t) xa,M(tn) Data interval T

xw,M

But the Accuracy xw,M is Not an Observable Error Measure

j,M(tn)2 = E xj,M(tn)2 j,M2 = n n j,M(tn)2

Only true observable is xj,M(tn) = x(tn) – xa,M(tn)the Data Precision


x(t)


xw,M

But the Accuracy xw,M is Not an Observable Error Measure

xj,M(tn)

2wj,M(tn)

Estimates accuracy based on theoretical d(tn) & d

From Data Precision can form Fit Precisionwj,M(tn)2 = d(tn) j,M(tn)2 wj,M

2 = d j,M2


But the Accuracy xw,M is Not an Observable Error Measure • Can showCan show for uniform LSQF & uncorrelated xuniform LSQF & uncorrelated xrr(t(tnn)) do = M/(N – M)

• Will show Will show dodo & equiv & equiv dodo(t(tnn)) can generate can generatevery misleading accuracy estimatesvery misleading accuracy estimateswhen neg-p noise is presentwhen neg-p noise is present


x(t)


Mth order -Measures over Interval

,2,2(t(tnn))22 Allan variance of time error Allan variance of time error,3,3(t(tnn))22 Hadamard variance of time error Hadamard variance of time error

• Cannot use -measures for accuracy just because accuracy diverges

()x(tn) ()x(tn+)2()x(tn)

Mth Order -Measures: ()Mx(tn) ,M(tn)2 = E [()M(tn)]2 ,M

2 = n n ,M(tn)2

Can be shown to be measures of stability and precision under specific fitting conditions


Neg-p Noise xp(t)

• xxpp(t)(t) generally represented as generally represented as wide-sense wide-sense stationary (WSS)stationary (WSS) random process random process

• But But xxpp(t)(t) is inherently is inherentlynon-stationary (NS)non-stationary (NS)

• NSNS picture: picture: xxpp(t)(t) starts at starts atfinite tfinite t

• WSS picture: WSS picture: xxpp(t)(t) must muststart start at t = -at t = - to maintain to maintaintime invariancetime invariance

• So WSS So WSS xxpp(t) (t) for all t for all t since neg-p noise since neg-p noise grows without bound as time from start grows without bound as time from start

tx p(t)

NS Picture ofNeg-p Noisexp(t) = 0 [t<0]

0


Neg-p Noise xp(t)

• Only NS Only NS covariance or correlation function(E xp = 0) is finite

Difference time betweenDifference time betweencovariant argumentscovariant arguments

ttgg Time from start of process Time from start of process• Rp(tg,) finite for finite tg & • Rp(tg,) as tg for all

• Because of this theBecause of this theWSS covariance RWSS covariance Rpp(() ) is infinite for all is infinite for all

/2)(tx/2)(tx),(tR gpgpgp τττ E

t

x p(t)

tg

Neg-p Noisexp(t) = 0 [t<0]

),(tRLim)(R gptpg

ττ


Neg-p Noise xp(t)

• In NS f-domain are 3 major spectral functions In NS f-domain are 3 major spectral functions Wigner-Ville Function Loève SpectrumLoève Spectrum

Ambiguity functionAmbiguity function

• Can define Can define LLpp(f)(f)withoutwithout using using RRpp(())as limit of as limit of WWpp(t(tgg,f),f)

pgptp |f|f),(tWLim(f)L

g

),(tR f),(tW gpf,gp ττF f),(tWf),(fL gpt,fgp ggF

FourierTransf f,τF),(tR),(fA gpt,fgp gg

ττ F


xp(t) Not the Same as xr(t)Because of System Filtering Hs(f)• xxrr(t) = h(t) = hss(t) (t) x xpp(t) (t) X Xrr(f) = H(f) = Hss(f) X(f) Xpp(f)(f)• Important later because Important later because HHss(f)(f) can have can have HP HP

filteringfiltering as well as LP filtering properties as well as LP filtering properties• Typical HP filtering HTypical HP filtering Hss(f) for time or phase error(f) for time or phase error

Tx Rx

~xTx(t) x

v

d = x-v

Delay Mismatch

Hs(f) f 2 |f|<<1/d

(2nd Order) PLL

Hs(f) f 4 |f|<<1/Loop BW

Tx Rx

~xTx(t) ~

xRx(t)PLL

vx

In Asynchronous TimingSystems

In Both Synchronous& AsynchronousTiming Systems


• What is happening here is that What is happening here is that the neg-p noise ensemble memberthe neg-p noise ensemble memberis is mimickingmimicking the signature of the signature ofthe causal behavior over Tthe causal behavior over T

• So part of the noise So part of the noise cannot cannot be separated from be separated from the causal behavior the causal behavior

f -2 Noise

f 0 Noise

Kalman Filter

f 0 Noise

LSQF

The Effect of Neg-p Noise on LSQF & Kalman Estimation

long term error-1.xls

Correlated Noise Model

f -2 Noise

WhiteNoise Model

f -3 Noise

xa,M ± wj,M

–x --xc


• Variables that are Variables that are linearlylinearlydependentdependent over T over T cannotcannot be beseparated by separated by anyany solution process solution processSolution matrix has aSolution matrix has a

zero determinantzero determinant• Adding Adding correlated xcorrelated xrr(t) models(t) models when when xxrr(t) and (t) and

xxcc(t)(t) are correlated with each other are correlated with each otherwill just generate will just generate ill-formedill-formed equations equations

f -2 Noise

f 0 Noise

Kalman Filter

f 0 Noise

LSQF

The Effect of Neg-p Noise on LSQF & Kalman Estimation

long term error-1.xls

Correlated Noise Model

f -2 Noise

WhiteNoise Model

f -3 Noise

xa,M ± wj,M

–x --xc


E x(t)

x(1)(t)

x(3)(t)x(2)(t)

Ergodicity & Proper Behavior of a Practical Realization of a Fit• Fitting theory based on Fitting theory based on

ensemble means ensemble means EE• But practical realizations must But practical realizations must

use use <..><..>TT finite time meanfinite time mean over over single ensemble member single ensemble member

• Noise must be Noise must be ergodic-like over ergodic-like over TT for practical realization to for practical realization to work as expected work as expected <..><..>TT E E Strict ergodicity Strict ergodicity <..><..>T T = = EE

T<x(n)(t)>T

Ergodic- like<..>T E ..


Ergodicity & Proper Behavior of a Practical Realization of a Fit• Strong connection betweenStrong connection between

correlation timecorrelation time cc of noise of noise& ergodic-like behavior& ergodic-like behaviorA white process isA white process is

locally ergodic locally ergodic <..><..>T T 0 0 = = EEBut a correlated processBut a correlated process

is only intermediate ergodicis only intermediate ergodic<..> <..> EE

• So must have So must have T >> T >> cc for a practical realization to for a practical realization to work as expected when a noise process is work as expected when a noise process is correlatedcorrelated

E x(t) <x(t)>T

<x(n)(t)>T

x(1)(t)

x(3)(t)x(2)(t) x(n)(t)

c T

E

T >> T >> cc


Connection Between T, c & Anomalous Fitting Behavior

• But for neg-p noise can show that But for neg-p noise can show that cc = = Unless HP filter HUnless HP filter Hss(f) suppresses neg-p pole(f) suppresses neg-p pole

• Neg-p noise will generate anomalous fit Neg-p noise will generate anomalous fit behavior for behavior for all Tall T

• Unless HUnless Hss(f) makes (f) makes cc finite for x finite for xrr(t) (t)

T

T/c=2000 T/c=200 T/c=20 T/c=2 Correlated Noise with

Finite c

xa,M ± wj,M

–x --xc


Calculating d for Neg-p Noisewj,M

2 = dj,M2 Estimate of w,M

2 • Theoretically Theoretically d = w,M

2/j,M2

• Have shown previously that Have shown previously that

KK,M,M(f) = spectral kernel representing fit(f) = spectral kernel representing fitModel error adds extra term to Model error adds extra term to ,M,M

22

Model error occurs when xModel error occurs when xa,Ma,M(t) is not complex (t) is not complex enough to track xenough to track xcc(t) over T(t) over T

• Can use above to calculate Can use above to calculate dd when when p-orderp-orderof noise is knownof noise is known

(f)L(f)K|(f)H|dfσ pM,2

s2

M, ςς

= w or j


Calculating d for Neg-p Noisewj,M

2 = dj,M2 Estimate of w,M

2 • For xFor xa,Ma,M(t) = (M-1)(t) = (M-1)thth

order polynomialorder polynomialKKj,Mj,M(f) (f) 2M 2Mthth order order

highpass filterhighpass filterKKw,Mw,M(f) (f) 2M 2Mthth order order

lowpass filterlowpass filter• Can use above KCan use above K,M,M(f)(f)

properties toproperties tographicallygraphicallyexplain the behaviorexplain the behaviorof of dd for neg-p noise for neg-p noise

-2 -1 0 1 2-150

-100

-50

0

1Log10(fT)

M=5

f

10

M=4

f 8

M=3

f 6M=2

f 4M=1 f 2

1-M

0m

mmMa, ta(t)x

N=1000dB

Kj,M(f) (Uniform LSQF)

fT = M/(2T)


Graphing w,M2 & j,M

2

• Can showCan show

for white noisefor white noise(uniform LSQF) unless(uniform LSQF) unlessan ideal Nyquistan ideal NyquistLP HLP Hss(f) is used(f) is usedWell-known effect of over-sampling for BWWell-known effect of over-sampling for BW

2Mj,

2Mw,dpM,

2s

2M, σσj)orw((f)L(f)K|(f)H|dfσ / ρςςς

j,M2

fT fh0w,M

2

LL 00(f)(f)

KKw,Mw,M(f)(f) KKj,Mj,M(f)(f)

White NoiseWhite Noise

KK,M,M(f) (f) Sharp cut-offs at f Sharp cut-offs at fTT HHss(f) (f) LP cutoff f LP cutoff fhh

d = fT/(fh-fT) M/(N-M)


Graphing w,M2 & j,M

2

• For neg-p noiseFor neg-p noise w,M

2 & d While While w,M

2 is finiteFor all neg-p (when HFor all neg-p (when Hss(f) has no HP properties) (f) has no HP properties)

• Will later show this is an indication of a Will later show this is an indication of a realreal problem with the fit accuracy by using the NS problem with the fit accuracy by using the NS picture of neg-p noisepicture of neg-p noise

fT fh0

Neg-p NoiseNeg-p Noise

LL pp(f)(f)

2Mj,

2Mw,dpM,

2s

2M, σσj)orw((f)L(f)K|(f)H|dfσ / ρςςς


Effect of HP filtering Hs(f) for Neg-p Noise

• When HWhen Hss(f) suppresses(f) suppressesneg-p pole at f = 0 neg-p pole at f = 0 dd finite finite

• When this is true When this is true xxrr(t)(t) has has finite finite cc is is WSSWSS & is & is intermediate ergodicintermediate ergodic

• Practical fit behavior OK for Practical fit behavior OK for ffTT << f << fll T >> T >> cc 1/(2f 1/(2fll))

d < do for fT << fld > do for fT >> fl

2j,M

fT fhf dB fl

fT >> fl

2w,M

<

LL pp(f) d

B(f)

dB

Hs(f) suppresses Lp(f) pole

fl fhfT

fT << fl

f dB

LL pp(f) d

B(f)

dB

HHss(f) (f) HP knee HP knee ffll & LP knee f & LP knee fhh


Explaining Physical Reality of Infinite WSS w,M

2 with NS Picture

• xxrr(t(t00)) excluded from excluded from xxj,Mj,M(t(tnn)) because because xxrr(t(t00)) is is treated as part of causal behavior by fittreated as part of causal behavior by fit

w,Mw,M2 2 as as tt00

with no indication from with no indication from j,Mj,M22

• Thus the WSS Thus the WSS w,Mw,M2 2 infinity indicates that the infinity indicates that the

physical physical w,Mw,M22 will truly become very large when will truly become very large when

tt00 >> T (the usual situation) >> T (the usual situation)• For example a PLL will cycle slip when For example a PLL will cycle slip when w,Mw,M

2 2 for the calculated loop phase errorfor the calculated loop phase error

x = xr (xc = 0) xj,M LSQF with xa,1 = a0

Startof

Noise

Ensemble Members

t0

Startof

Data

T

xr(t0)


x = xr (xc = 0) xj,M LSQF with xa,1 = a0

Startof

Noise

Ensemble Members

t0

Startof

Data

T

xr(t0)

A Simulation Misrepresents Physical Situation when t0 not >> c

• Because steady state not reachedBecause steady state not reachedNote in above that Note in above that w,Mw,M j,Mj,M when when tt00 = 0 = 0 while while w,Mw,M >> >> j,Mj,M in physical situation when in physical situation when tt00 >> T >> T

• Thus all neg-p noise simulations are Thus all neg-p noise simulations are inherentlyinherently misleading unless Hmisleading unless Hss(f) makes (f) makes cc finite finite

T

Startof

Noise& Data

t0 = 0


,M(t) as Stability and Precision Measures When xa,M(t) = (M-1)th Order Polynomial

• Removing “causal” behavior from data biases Removing “causal” behavior from data biases ,M,M(t) as a (t) as a true random instability measuretrue random instability measure,M,M(t) generated from x(t) generated from xrr(t) with no fit not the same as (t) with no fit not the same as

,M,M(t) generated from x(t) with removal of x(t) generated from x(t) with removal of xa,Ma,M(t) (t) Fit removes noise correlated with causal behaviorFit removes noise correlated with causal behavior Reduces true noise contribution for Reduces true noise contribution for T T

j,M = ,M(t0)for “Unbiased” n

xj,M(tm)

xa,M(t,a)Unweighted LSQF

over all t0…tM

x(tm) Stability over = T/M = T/M Precision

x(tM)xj,M(tM)

xa,M(t)Passes thru

t0…tM-1

x(t0)º

Extrapolation Interpolation

j,M(t0) ,M(t0)

xj,M(tm)

xk,M(t,a) Passes thru

t0…tM but not tm

x(tm)º


Secondary Accuracy after Cal-ibration Can Sidestep Divergence• x(t) x(t) x(t, x(t,(t)) (t)) (t) = other independent(t) = other independent

variables variables • Cal x(t) periodically at tCal x(t) periodically at tnn’’

against primary standardagainst primary standardwhen when (t(tnn’) = ’) = ooTo generate cal function xTo generate cal function xa,M’a,M’

(k)(k)(t) (t) • Now use secondary (calibrated) dataNow use secondary (calibrated) data

xx(k)(k)(t) = x(t) – x(t) = x(t) – xa,M’a,M’(k)(k)(t) (t)

To determine To determine sensitivity coefficients sensitivity coefficients & sidestep x& sidestep xrr(t) divergence for(t) divergence for-coefficient determination -coefficient determination

• But divergence still present for determining true But divergence still present for determining true causal behavior free of noisecausal behavior free of noise

Cal Cal

x(k)(t) after Cal

Start of Noisex(k)(t) = x(t) – xa,M’

(k)(t)

(tn’) = o


Conclusions & Further Implications

• Cannot Cannot separate separate neg-p neg-p noise from true causal noise from true causal behavior for behavior for any Tany T Exception for HP H Exception for HP Hss(f) (f) Fits generateFits generate anomalous anomalous results for results for any Tany T

unless sufficiently HP filtered by Hunless sufficiently HP filtered by Hss(f)(f)• Unbiased or pure neg-p behavior Unbiased or pure neg-p behavior unobservableunobservable

in data containing causal behaviorin data containing causal behaviorx(t) with xx(t) with xa,Ma,M(t) removed (t) removed not not equivalent to xequivalent to xrr(t)(t)Can explain Can explain downward biasesdownward biases in Allan & in Allan &

Hadamard variances when Hadamard variances when T T



• Noise whitening is an Noise whitening is an improper improper procedure to use procedure to use when neg-p noise presentwhen neg-p noise presentFits to Fits to correlated noisecorrelated noise as well as as well as xxcc(t)(t)

• Must have Must have T >> T >> cc for fitting technique to behave for fitting technique to behave as theoretically expectedas theoretically expectedProblematic for neg-p noise Problematic for neg-p noise unless HP H unless HP Hss(f)(f)



• Also effects Also effects M-corner hatM-corner hat & & cross-correlation cross-correlation techniquestechniquesAssume Assume statistical independencestatistical independence implies implies

<..><..>TT cross-correlations cross-correlations 0 0 as as NN-1/2-1/2

• When there is When there is no HP filtering Hno HP filtering Hss(f)(f) expect such expect such anomalies anomalies for all T & all neg-p ordersfor all T & all neg-p orders

• For cross-correlation with For cross-correlation with 11stst order PLL H order PLL Hss(f)(f)Expect anomalies for fExpect anomalies for f -3 -3 noise that grow as noise that grow as

slowly as log(T)slowly as log(T)• The above needs further study The above needs further study

http://www.ttcla.org/vsreinhardt/http://www.ttcla.org/vsreinhardt/

the profound impact of negative power law noise on the estimation of causal behavior

Documents