understanding the law of propagation of uncertainties · uncertainty expression relative...

http://www.emceoc.org

Understanding the Law of Propagation

of UncertaintiesEmma Woolliams

4 April 2017

http://www.emceoc.org/

The Law of Propagation of

Uncertainties

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

Things you might already “know”

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

Has a sensitivity coefficient

Adding in quadrature (% or units)

Averages reduce by 1 n

This term is to do

with correlation

Building on that starting point

• The Law of Propagation of Uncertainties

• How many readings you should average

• What correlation is

Has a sensitivity coefficient

Adding in quadrature (% or units)

Averages reduce by

This term is to do with correlation

1 n

2c

2

2

1

1

1 1

2 ,

n

iii

n n

i ji ji j i

u y

fu x

x

f fu x x

x x

At the end of this module, you should

understand


• Why the sensitivity coefficient is central to the law

• How to determine the sensitivity coefficients

• What ‘adding in quadrature’ really means


• What the uncertainty associated with an average is

• How you know you have enough readings

• When you’ve taken too many readings


• What the uncertainties are for averaging partially correlated data

• How to estimate covariance from numerical and experimental data

• How to deal with not knowing what the correlation is

THE LAW OF PROPAGATION

OF UNCERTAINTIES

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x


understand













Sensitivity Coefficients

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

How sensitive is

my result to

this?

How sensitive is

my result to

this?

Do an

experiment

to find out!

Lamp power supply:

What is the sensitivity of the irradiance to the lamp current?

0.985

0.99

0.995

1

1.005

1.01

1.015

250 500 750 1000 1250 1500 1750 2000 2250 2500

Sig

nal ra

tio

wavelength / nm

Ratio 27.21 A / 27.19 A

Measured ratio0.2 A10 %

PTFE Sphere phase transition

6 °C

1.5 %

How sensitive is

my result to

this?

Calculate it from

the

measurement

equation

Analytical sensitivity coefficients

ixi

fc

x

1 2 3, , ,y f x x x

The local slope:

Translating a change in one

parameter to a change in

the other

//upload.wikimedia.org/wikipedia/commons/a/aa/Derivative_GIF.gif

//upload.wikimedia.org/wikipedia/commons/a/aa/Derivative_GIF.gif

When it works well

S

light

1V

V

S light darkV V V

S

dark

1V

V

Very simple case

When it works well

2FEL 0 45 cal

s 2use

E dL

d

2

s FEL 0 45 cal s

3

use use use

22

L E d L

d d d

Inverse square law leads to a 2

When it doesn’t work well

• Because you can’t write a relationship

• Because it’s too difficult to differentiate

• Because it’s a program not an equation

How sensitive is

my result to

this?

Model

it!

Radiance through double aperture

system

2 2

1 2-d 2

2 2 2 2 2 2 2 2

1 2 1 2 1 2

2

4

r rg

r r x r r x r r

1A g L

By differentiating …

22 2 2 2 2

1 2 1 2

2 2 2

1 2

4r r d r r

r r d

2 2

1 2-d 2

2 2 2 2 2 2 2 2

1 2 1 2 1 2

2

4

r rg

r r x r r x r r

2 2 1

1 22g r r

2 2 2

1 2 1 2

1

2 2 2

1 2 2 1

2

2 2

1 2

4 21

4 21

4

r r r rg

r

r r r rg

r

r r dg

d

Or by “modelling”…

Sensitivity to atmospheric model choice

Tuz Gölü CEOS comparison

2010

Difference between continental

and desert model lead up to 3%

difference in 𝜌 values

Monte Carlo simulation


How sensitive is

my result to

this?

Experimentally

Mathematically

Numerically

ixi

fc

x

Vary in the lab and see

what happens

Vary in the

model and

see what

happens

All are acceptable

Adding in Quadrature

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

Why quadrature?

Uncertainty Error

Describes the spread Difference to the

(unknowable) true

value

It’s very unlikely

that all will be

maximum in

same direction!

Combining

distributions

leads to the

quadrature rule

Uncertainty expression

Relative uncertainty:

5 mW m-2 nm-1 ± 0.2 %

i.e. uncertainty expressed as a percentage

Absolute uncertainty:

5 mW m-2 nm-1 ± 0.01 mW m-2 nm-1 i.e.

uncertainty expressed in the native

measurement units

Adding in quadrature for absolute

models

y BA C 1B

y y

CA

y

2 2 2 2Au y u u uB C

Adding in quadrature for absolute

models

y BA

1A

y

22 2 21Au u By u

1B

y

2 2 2Au y u Bu

What behaves like an absolute model?

Distance = Measured + aperture thickness

Signal = Light signal – Dark signal

Uncertainty expression

Relative uncertainty:

5 mW m-2 nm-1 ± 0.2 %

i.e. uncertainty expressed as a percentage

Absolute uncertainty:

5 mW m-2 nm-1 ± 0.01 mW m-2 nm-1 i.e.

uncertainty expressed in the native

measurement units

Adding in quadrature for relative

modelsy BA

y y

A AB

2 2 2

u y u

Ay

B

B

A u

y y

B BA

2 2

2 2 2y yu y BA

BAu u


modelsy BA

1

BA A

y y

2 2 2

u y u

Ay

B

B

A u

2

y A y

BB B

2 2

2 2 2Ay y

BB

u uA

y u


models

2y A B

2

1y y

BA A

2 2 2

2u y u A B

BA

u

y

3

2 2y A y

B BB

2 2

2 2 22A

y yB

By u u

Au

What behaves like a relative model?

• Most radiometric equations!

• Anything where uncertainties are in %

Gain = Radiance / Signal

2FEL 0 :45 cal

s 2use

E dL

d

s s

FEL FEL

L L

E E

s s

0 :45 0 :45

L L

s s

use use

2L L

d d

2 2 22

s 0 :45 useFEL

s FEL 0 :45 use

2u L u u du E

L E d


understand













HOW MANY READINGS

SHOULD BE AVERAGED?


understand









• What the ‘other term’ – the correlation bit means





Applying the Law of Propagation of

Uncertainties to an average

Averaging three readings

1 2 3M

3

E E EE

M M M

1 2 3

1

3

E E E

E E E

2 2 2

2 2 2 2

M 1 2 3

1 1 1

3 3 3u E u E u E u E

1 2 3 iu E u E u E u E

2

2 2

M

13

3iu E u E

2 2

M 2

3

3iu E u E

2 2

M

1

3iu E u E

2

2

M3

iu Eu E

M3

iu Eu E



Averaging n readings

M 1

n

iiE E n

M 1

i

E

E n

2 2

M 1

n

iiu E u E n

2

2 2

M

1iu E n u E

n

2 2

M

1iu E u E

n

2

2

M

iu Eu E

n

M

iu Eu E

n

Average light – average dark

s light, dark,

1 1

1 1N M

i j

i i

V V VN M

dark,

dark,

1, 1, ,M

jV

j

Vc j

V M

light dark

2 2

2 V V

V

u uu

N M

light, dark,

2 2

2 2 2

1 1

1 1i j

N M

V V V

i j

u u uN M

light,

light,

1, 1, , N

iV

i

Vc i

V N

Taking enough readings



Averaging three readings

1 2 3M

3

E E EE

M M M

1 2 3

1

3

E E E

E E E

2 2 2

2 2 2 2

M 1 2 3

1 1 1

3 3 3u E u E u E u E


2

2 2

M

13

3iu E u E

2 2

M 2

3

3iu E u E

2 2

M

1

3iu E u E

2

2

M3

iu Eu E

M3

iu Eu E

What is the uncertainty associated

with each reading?


Uncertainty

Describes the spread

Standard

deviation?

The dangers of taking a standard deviation

of a small number of readings

Reading numberValu

es a

nd c

um

ula

tive S

t d

ev

The dangers of taking a standard deviation

from a small number of readings

What you should do

Use more readings!

• Take more measurements today

M

iu Eu E

n

Better estimate of

uncertainty

More to average

What you should do

Use more readings!

• Have a facility commissioning phase where you

take more measurements

M

iu Eu E

n

Better estimate of

uncertainty

Today’s!

From standard deviation of 10: iu E

Today two readings, 2n

What you should do

Use more readings!

• Compare data from different days

Even if there is a drift:

e.g Use standard deviation of the

difference

between two readings

What you should do

Use more readings!

• Compare data from different wavelengths

(smooth out the bumps)

What you should do

Increase the estimate

• Increase standard uncertainty

Will be in revised GUM

2

light2

light,mean

1

3

sNu

N N

What you should do

Use more readings!

• Take more measurements today

• Have a facility commissioning phase

• Compare data from different days

• Compare data from different wavelengths

Increase the estimate

• Increased standard uncertainty

KNOWING WHEN TO STOP!

Use more readings!

But only when it’s worth it!

The metre: 1791 - 1799

Pierre-

Françoise-André

Méchain

Jean-Baptiste-

Joseph

Delambre

The Measure of all Things - Ken Alder

The uncertainties don’t keep dropping

forever

ADEV

Line Fit

Lower Bound

Upper Bound

AlaVar 5.2

Allan STD DEV

Produced by AlaVar 5.2

time / s 100 1000 10000

0.0001

0.001

The Allan Deviation – software

ADEV

Lower Bound

Upper Bound

AlaVar 5.2

Allan STD DEV


time / s 100 1000

1E-5

0.0001

www.alamath.com


understand













CORRELATION


understand













The Law of Propagation of

Uncertainties

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

Truei iE E S R

Correlation: Type A and Type B

methods

Correlation occurs when there is something in common between the

different quantities, or between the measured values being combined.

Type A: From the data Type B: From knowledge

This is where the

correlation comes

from!

Systematic

Effects!


methods


different quantities, or between the measured values being combined.

Type A: From the data

1

1,

1

ni i

i X Y

X X Y Yr x y

n s s

, ,u x y u x u y r x y

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

Correlation? http://www.tylervigen.com/

Spurious correlations

http://www.tylervigen.com/

Truei iE E S R


methods


different parameters, or between the measurements being combined.

Type B: From knowledge

This is where the

correlation comes

from!

Systematic

Effects!

Calculate covariance

Remove covariance

Averaging partially correlated data

Truei iE E S R 1 2 3M

3

E E EE

True 1 2 3M

3 3

3 3 3

E R R RSE

2

2 2

M3

iu Ru E u S

Averaging Partially correlated data

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

Truei iE E S R

2 2

i iu E u S u R

2, ;i ju E E u S i j


2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

1 2 1 3 1 4

2 3 2 4

3 4

, , , , , ,

, , , ,

, ,

u E E u E E u E E

u E E u E E

u E E


Truei iE E S R 1 2 3

M3

E E EE

M M M

1 2 3

1

3

E E E

E E E

2, ;i ju E E u S i j

2 2

i iu E u S u R

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

2 2

2 2 2

M

1 13 3

3 3iu E u S u R

1 2 2 3

1 3 21 12

3 3u S

2

213 2

3u S

2 2

2 2 2

M

1 13 6 3

3 3iu E u S u R

2

2 2

M3

iu Ru E u S

Systematic and random effects: Lamp

measured 5 times (continued)

Systematic effects Random effects

Reference calibration Noise

Alignment Lamp current fluctuation

Lamp current setting

Temperature sensitivities

,S u S ,i iR u R

Truei iE E S R

2 2 2

i iu E u S u R

2

2 2

M

iu Ru E u S

n


2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

1 2 1 3 1 4

2 3 2 4

3 4

, , , , , ,

, , , ,

, ,

u E E u E E u E E

u E E u E E

u E E

Covariance Matrix

21 1 2 1

22 1 2 2

21 2

, ,

, ,

, ,

1 2

1

2

n

E n

n n n

u E u E E u E E

u E E u E u E E

u E E u E E un E

n

U

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

1 2 1 3 1 4

2 3 2 4

3 4

, , , , , ,

, , , ,

, ,

u E E u E E u E E

u E E u E E

u E E

Covariance Matrix Version

1 2y

n

f f f

x x x

C

21 1 2 1

22 1 2 2

21 2

, ,

, ,

, ,

1 2

1

2

n

E n

n n n

u E u E E u E E

u E E u E u E E

u E E u E E un E

n

U

Covariance matrix

with absolute

variance (squared

uncertainty) and

covariance

2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

Covariance matrix

21 1 2 1

22 1 2 2

21 2

, ,

, ,

, ,

1 2

1

2

n

E n

n n n

u E u E E u E E

u E E u E u E E

u E E u E E un E

n

U

T 1 1i i i iE E S R s r

2 2 2 2 2

,2 2

( )

( ).

i i i

E ij

i j

E u S u R u s u r i jU

E E u S u s i j

Negative sensitivity coefficients

y BA 1A

y

1

B

y

22 2 21Au u By u

2 2 2Au y u Bu

E.g. Signal = Light - Dark

Negative sensitivity coefficients

y BA 1A

y

1

B

y

2 2 2 2 1 1 ,Bu y u AA u u B

E.g. Signal = Light - Dark

2 2 2 2 ,u y u u AB BA u

Correlation

reduces

uncertainty

Positive sensitivity coefficients

y BA 1A

y

1

B

y

2 2 2 2 1 1 ,BAu y u u u A B

E.g. Distance = measured + thickness

2 2 2 2 ,u y u u AB BA u

Correlation

increases

uncertainty

But I don’t know what the covariance is!

• Think of the worst-case scenario

Does correlation increase or decrease the

uncertainty?

Treat as “all random” and “all systematic”


understand













CONCLUSIONS AND

SUMMARY


understand














2 1

2 2c

1 1 1

2 ,

n n n

i i ji i ji i j i

f f fu y u x u x x

x x x

How sensitive is

my result to

this?

Experimentally

Mathematically

Numerically

Taking enough readings, but not too

many

M

iu Eu E

n

Better estimate of

uncertainty

More to average

ADEV

Line Fit

Lower Bound

Upper Bound

AlaVar 5.2

Allan STD DEV


time / s 100 1000 10000

0.0001

0.001

Allan Deviation

Correlation

Truei iE E S R


different parameters, or between the measurements being combined.

Type B: From knowledge

This is where the

correlation comes

from!

Systematic

Effects!

Calculate covariance

Remove covariance

understanding the law of propagation of uncertainties · uncertainty expression relative...

Documents