intro to uncertainty analysis

Introduction toIntroduction toUncertaintyUncertainty AnalysisAnalysis

Howard Castrup, Ph.D.Suzanne Castrup, MSME

Integrated Sciences GroupBakersfield, CA 93306

www.isgmax.com

Introduction to Uncertainty Analysis 2 Integrated Sciences Group

Introduction to Uncertainty Analysis

Topic OutlineBasic ConceptsDirect MeasurementsUncertainty SidekickRecap



Basic ConceptsFundamental Measurement ModelError DistributionUncertainty DefinitionVariance and UncertaintyVariance Addition RuleCorrelation CoefficientsUncertainty Sidekick DistributionsType A and Type B Estimates


Basic ConceptsFundamental Measurement ModelThe difference between a measured value and the “true” value is the measurement error

measured true xx x ε= +

The true value is a fixed quantity

The measurement error is a variable


Basic Concepts

Error Distribution

A relationship between the value of a measurement error and its probability of occurrence

εx0

f(εx)


Basic Concepts

Uncertainty DefinitionMeasurement Uncertainty quantifies the “spread” of the measurement error distribution

0

SmallUncertainty

LargeUncertainty

0


Basic Concepts

Uncertainty Definition (cont.)The spread of an error distribution is the distribution standard deviationThe standard deviation is the square root of the distribution variance

2var( ) var( )measured x xx ε ε= =


Basic Concepts

Variance and Uncertainty

Measurement Uncertainty

var( ) var( )var( )

measured true x

x

x x εε

= +=

var( )xx xεσ σ ε= =

x xx xu uε εσ σ= = =

Distribution Variance

Distribution Standard Deviation


Basic Concepts

Variance Addition RuleSuppose we have a variable zcomposed of variables x and y

z ax by= +

The variance of z is given by

2 2

var( ) var( )var( ) var( ) 2 cov( , )

z ax bya x b y ab x y

= +

= + +


Basic Concepts

Variance Addition Rule (cont.)Variances and Covariance

2

2

var( ) var( )

var( ) var( )

cov( , ) cov( , )

x

y

x

y

x y

x u

y u

x y

ε

ε

ε

ε

ε ε

= =

= =

=

The Variance Addition Rule2 2var( ) var( ) var( ) 2 cov( , )x y x xz a b abε ε ε ε= + +


Basic Concepts

Correlation CoefficientsCovariances can be expressed in terms of Correlation Coefficients

,

cov( , )x y

x y

x y

u uε εε ε

ε ερ =

The Variance Addition Rule Becomes2 2 2 2var( ) 2

x y x y x yax by a u b u ab u uε ε ε ε ε ερ+ = + +

Correlation Coefficients range from –1 to +1


Basic Concepts

Uncertainty Sidekick DistributionsNormalUniformStudent’s t


Uncertainty Sidekick Distributions

The Normal Distribution

( )2 2/ 21( )2

uf eu

ε εε µ

ε

επ

− −=

f(ε)

ε0- a a

f(ε)

ε0- a a



CommentsThe “workhorse” of statistics and probability.

Usually assumed to be the underlying distribution for errors.

Most uncertainty analysis tools are based on the assumption that measurement errors are normally distributed, regardless of the distributions used to estimate the uncertainties themselves.



Uncertainty EstimatesType A Estimates: Compute mean and standard deviation from a sample. Nearly always assume a normal distribution

1 12

aupε

−=

+⎛ ⎞Φ ⎜ ⎟⎝ ⎠

( )2

1

11

n

ii

u x xnε

=

= −− ∑

Type B Estimates: Work with error containment limits ± a and containment probability p



The Uniform Distribution

1 ,( ) 2

0, otherwise ,

a af a

εε

⎧ − ≤ ≤⎪= ⎨⎪⎩

0

f (ε)

ε−a a0

f (ε)

ε−a a

3auε =



ApplicabilityApplicable to

Digital Resolution ErrorQuantization ErrorRF Phase Angle

Criteria for useNeed minimum bounding limitsUniform probability within the limits100% containment probability

Very limited applicability



CommentsArguments for Use

Lack of Knowledge – “Use in case you know nothing about the error other than its bounding limits.”Easy Out – Divide bounding limits by root 3

Equivalent to assuming a normal distribution with 91.67% in-tolerance probability. GUM, Sec 4.3.7: “When a component of uncertainty is determined in this manner contributes significantly to the uncertainty of a measurement result, it is prudent to obtain additional data for its further evaluation.”



Student’s t Distribution

2 ( 1) / 2

12( ) (1 / )

2

f x x ν

ν

ννπν

− +

+⎛ ⎞Γ⎜ ⎟⎝ ⎠= +

⎛ ⎞Γ⎜ ⎟⎝ ⎠

f (ε)

ε0

ν = Degrees of Freedom


Student’s t Distribution

ApplicabilityComputation of Confidence Limits for Normally Distributed Errors with Known Degrees of Freedom (ν )Statistical testing of hypotheses

Equivalence of laboratories (MAPs)Significance of curve fit linear slopeetc.


Basic Concepts

Type A and Type B EstimatesType A

Estimate the standard deviation from a sample of data

2 2

1

1 ( )1

n

x ii

u x xn =

= −− ∑

Type BEstimate heuristically from Error Limits and a Containment Probability (Confidence Level)Choose the appropriate error distribution



Direct MeasurementsDefinitionError SourcesThe Error ModelCombined UncertaintyError Source CorrelationsDegrees of FreedomError Source Uncertainties


Direct Measurements

DefinitionThe value of an attribute is measured directly by comparison with a measurement reference (device)


Direct Measurements

Error SourcesParameter BiasRandom Error (Repeatability)Resolution ErrorOperator Bias (Reproducibility)Environmental Factors


Direct Measurements

The Error ModelFor a Direct Measurement, themeasurement error is the sum of the error sources:

, , , , ,x x bias x ran x res x op x envε ε ε ε ε ε= + + + + + ⋅⋅⋅


Direct Measurements

Combined Uncertainty

The Uncertainty in εx

var( )x xu ε=

By the Variance Addition Rule

, , , , ,

, , , , , , , ,

, , , , ,

2 2 2 2 2

, ,

var( ) var( )

2 2x bias x ran x res x op x env

x bias x ran x bias x ran x bias x res x bias x res

x x bias x ran x res x op x env

u u u u u

u u u uε ε ε ε ε

ε ε ε ε ε ε ε ε

ε ε ε ε ε ε

ρ ρ

= + + + + + ⋅⋅⋅

= + + + + + ⋅⋅⋅

+ + + ⋅⋅⋅


Direct Measurements

Error Source UncertaintiesError Source Variances:

, ,

, ,

,

2 2, ,

2 2, ,

2,

var( ) var( )

var( ) var( )

var( )

x bias x res

x ran x env

x op

x bias x res

x ran x env

x op

u u

u u

u

ε ε

ε ε

ε

ε ε

ε ε

ε

= =

= =

=

Error Source Correlations:Nearly always zero for direct measurementsMay be present in making environmental correctionsMay appear in the analysis of operator bias


Direct Measurements

Degrees of FreedomThe amount of information used in obtaining an uncertainty estimateDetermined for each error source

,x biasενE.g., bias degrees of freedom =Estimated for the combined uncertainty

,

,

4

4 , , , ,x

x

x i

x ii

ui bias ran res

uε

εε

ε

ν

ν

= = ⋅⋅⋅

∑



Uncertainty SidekickInteractive Tool for Estimating Uncertainty in Measurement

Uncertainty Estimated for Direct MeasurementsUncertainty Estimated from Technical Data and User KnowledgeData and Technical Knowledge Entered in Special FormatsStatistics and other Math Performed in BackgroundBuilt-in Interface to Measurement Units DatabaseIncludes Bayesian Analysis of Measurement ResultsReport Preview, Export and PrintingAnalysis File Save / Open

Files can be opened in Sidekick, Sidekick Pro and UncertaintyAnalyzer


Uncertainty SidekickUncertainty Sidekick

AnalysisAnalysisSetupSetup

What’s BeingWhat’s BeingMeasured?Measured?

Who’sWho’sMeasuring It?Measuring It?

Where’s itWhere’s itBeingBeing

Measured?Measured?

MeasurementConfiguration

SubjectParameterMeasurement

Area and Units

What’sWhat’sMeasuring It?Measuring It?

Nominal Value

MeasurementReference

Operator

Environment


Uncertainty Sidekick

ExampleHP 973A Digital Multimeter Calibration

Accuracy: ± 0.1% of readingResolution: 1 mV

Measurement Reference: Fluke 732B DC Voltage Reference

Accuracy: ± 0.1 ppmLinear Stability: 2.0 ppm / year


Uncertainty Sidekick Example

Error SourcesBias in the Fluke 732B ReferenceError in Stability of the Fluke 732BRepeatability ErrorHP732A DMM Resolution Error



Analysis ProcedureSetup the AnalysisDefine the Subject Parameter (DMM)

Bias UncertaintyResolutionRepeatability

Define the Measurement ReferenceBias Uncertainty



Setup the AnalysisMeasurement Configuration

Subject Parameter Measures the Value of the Measurement ReferencePassive Reference Configuration

Measurement Area: DC VoltageNominal Value: 10 VTolerance Units: mV



The Subject ParameterHP 973A DMM10 V DC Nominal Value

Output by the Fluke 732BSpecs:

Accuracy: ± 0.1% of readingResolution: 1 mV

In-Tolerance Probability: 90%Measurement Sample



The Subject Parameter (cont.)Measurement Sample Reading Voltage

1 10.001

2 10.003

3 9.999

4 10.003

5 10.001

6 10.000

7 9.999

8 10.001

9 10.001

10 10.002

11 9.998

12 9.999

13 10.002

14 9.998

15 10.001



The Subject Parameter (cont.)Parameter Resolution

1 mVDigital Display



The Reference ParameterFluke 732B Voltage Reference10 V DC Nominal ValueSpecs:

Accuracy: ± 0.1 ppm of readingStability: ± 2 ppm / year

In-Tolerance Probability: 99%Measured by the Subject Parameter



The Analysis ResultsMeasured Mean Value

10.0005 V0.5 mV Above NominalWithin ± 10 mV Tolerance

Total Standard Uncertainty: 0.402 mVDegrees of Freedom: 15

Bayesian Analysis:0.5333 mV Estimated Bias6.1074 mV Bias Uncertainty100% In-Tolerance Probability



RecapUncertainty = Error Standard DeviationError Sources

bias, random, resolution, operator, environment, etc.

Direct MeasurementsThe value of an attribute is measured directly by comparison with a measurement reference (device)

CorrelationsDependence of error sources on one another

Type A AnalysisMean, Standard Deviation and Degrees of Freedom estimated from Measured Values

Type B AnalysisStandard Deviation computed from Error Limits and Containment ProbabilityEstimate Degrees of FreedomSelect Appropriate Distribution

intro to uncertainty analysis

Documents