Experimental Uncertainties:A Practical Guide

• What you should already know well• What you need to know, and use, in this lab

More details available in handout ‘Introduction to Experimental Error’ in your folders.

• In what follows I will use convention:– Error = deviation of measurement from true value– Uncertainty = measure of likely error

Why are Uncertainties Important?

• Uncertainties absolutely central to the scientific method.

• Uncertainty on a measurement at least as important as measurement itself!

• Example 1:

“The observed frequency of the emission line was 8956 GHz. The expectation from quantum mechanics was 8900 GHz”

• Nobel Prize?

Why are Uncertainties Important?

• Example 2:

“The observed frequency of the emission line was 8956 ± 10 GHz. The expectation from

quantum mechanics was 8900 GHz”

• Example 3:

“The observed frequency of the emission line was 8956 ± 10 GHz. The expectation from

quantum mechanics was 8900 GHz ± 50 GHz”

Types of Uncertainty

• Statistical Uncertainties:– Quantify random errors in measurements between

repeated experiments– Mean of measurements from large number of

experiments gives correct value for measured quantity

– Measurements often approximately gaussian-distributed

• Systematic Uncertainties:– Quantify systematic shift in measurements away

from ‘true’ value– Mean of measurements is also shifted ‘bias’

Examples• Statistical Errors:

– Measurements gaussian-distributed

– No systematic error (bias)– Quantify uncertainty in

measurement with standard deviation (see later)

– In case of gaussian-distributed measurements std. dev. = in formula

– Probability interpretation (gaussian case only): 68% of measurements will lie within ± 1 of mean.

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

-3 -2 -1 -0 1 2 3

2

2

2 2exp

2

1

xx

True Value

Examples

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

-3 -2 -1 -0 1 2 3

2

2

2 2exp

2

1

xx

True Value• Statistical + Systematic Errors:– Measurements still gaussian-

distributed– Measurements biased– Still quantify statistical

uncertainty in measurement with standard deviation

– Probability interpretation (gaussian case only): 68% of measurements will lie within ± 1 of mean.

– Need to quantify systematic error (uncertainty) separately tricky!

Systematic Errors• How to quantify uncertainty?• What is the ‘true’ systematic

error in any given measurement?– If we knew that we could correct

for it (by addition / subtraction)

• What is the probability distribution of the systematic error?– Often assume gaussian

distributed and quantify with syst.

– Best practice: propagate and quote separately

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

-3 -2 -1 -0 1 2 3

True Value

Calculating Statistical Uncertainty

• Mean and standard deviation of set of independent measurements (unknown errors, assumed uniform):

• Standard deviation estimates the likely error of any one measurement

• Uncertainty in the mean is what is quoted:

ii

ii

xxN

xxN

x

22

0

1

1

;1

.)1(

12/1

2

iix xx

NNN

Propagating Uncertainties

• Functions of one variable (general formula):

• Specific cases:

Xx

fF

d

d

xx

x

xxxx

xn

x

xxnxx

x

x

x

xxxx

n

nnn

1ln

cossin

or

2or 2

1

2

22

f = Apply equation Simplify

Propagating Uncertainties

• Functions of >1 variable (general formula):

• Specific cases:

.22

2

yy

fx

x

ff

222 yxf 222 yxf

22222 yxxyf

222

y

y

x

x

f

f

yx 2

4

2

2

22 y

y

x

y

xf

222

y

y

x

x

f

f

xy

yx

Combining Uncertainties• What about if have two or more

measurements of the same quantity, with different uncertainties?

• Obtain combined mean and uncertainty with:

• Remember we are using the uncertainty in the mean here:

ii

iiix

x2

2

1

i i

22

11

Ni

Fitting

• Often we make measurements of several quantities, from which we wish to 1. determine whether the measured values follow a

pattern

2. derive a measurement of one or more parameters describing that pattern (or model)

• This can be done using curve-fitting• E.g. EXCEL function linest.• Performs linear least-squares fit

Method of Least Squares• This involves taking

measurements yi and comparing with the equivalent fitted value yi

f

• Linest then varies the model parameters and hence yi

f until the following quantity is minimised:

• Linest will return the fitted parameter values (=mean) and their uncertainties (in the mean)

-1.5

-1

-0.5

0

0.5

1

0.0026 0.0028 0.003 0.0032 0.0034 0.0036 0.0038

ln e

ta [c

Ps]

1/T [1/K]

N

i

fii yy

1

2

In this example the model is a straight lineyi

f = mx+c. The model parameters are m and c

In the second year lab nevernever use the equations returned by ‘Add Trendline’ or linest to estimate your parameters!!!

Weighted Fitting• Those still awake will have noticed the least

square method does not depend on the uncertainties (error bars) on each point.

• Q: Where do the uncertainties in the parameters come from?– A: From the scatter in the measured means about the

fitted curve

• Equivalent to:

• Assumes errors on points all the same• What about if they’re not?

i

i xxN

22

1

1

Weighted Fitting• To take non-uniform uncertainties (error bars) on

points into account must use e.g. chi-squared fit.• Similar to least-squares but minimises:

• Enables you to propagate uncertainties all the way to the fitted parameters and hence your final measurement (e.g. derived from gradient).

• This is what is used by chisquare.xls (download from Second Year web-page) this is what we expect you to use in this lab!

N

i i

fii yy

1

2

2

General GuidelinesAlways:• Calculate uncertainties on measurements and plot

them as error bars on your graphs• Use chisquare.xls when curve fitting to calculate

uncertainties on parameters (e.g. gradient).• Propagate uncertainties correctly through derived

quantities• Quote uncertainties on all measured numerical

values• Quote means and uncertainties to a level of precision

consistent with the uncertainty, e.g: 3.77±0.08 kg, not 3.77547574568±0.08564846795768 kg.

• Quote units on all numerical values

General GuidelinesAlways:• Think about the meaning of your results

– A mean which differs from an expected value by more than 1-2 multiples of the uncertainty is, if the latter is correct, either suffering from a hidden systematic error (bias), or is due to new physics (maybe you’ve just won the Nobel Prize!)

Never:• Ignore your possible sources of error: do not just

say that any discrepancy is due to error (these should be accounted for in your uncertainty)

• Quote means to too few significant figures, e.g.: 3.77±0.08 kg not 4±0.08 kg