l1 - error analysis
TRANSCRIPT
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 1/32
ESO 208A/ 218Computational Methods in Engineering
Summer Semester 2014-15 I
L1 – Error Analysis
Dr. Anubha Goel
[email protected]; x 7027
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 2/32
Mathematical model
A mathematical model can be broadly defined
as a formulation or equation that expresses
the essential features of a physical system or
process in mathematical terms.
5/20/2015 2L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 3/32
Numerical Methods
• Numerical methods are scientific in the sense that they
represent systematic techniques for solving mathematical
problems.
IMPORTANT – They are approximations and hence involve errors
• Out of several methods available choice of numerical
methods for solving a problem reduces to one of cost andaccuracy.
5/20/2015 L1 - Error Analysis 3
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 4/32
Accuracy and Precision
Definitions
• Accuracy refers to how
closely a computed or
measured value agrees with
the true value.
• Precision refers to how
closely individual computed
or measured values agreewith each other.
5/20/2015 L1 - Error Analysis 4
An example from marksmanship illustrating the
concepts of accuracy and precision. (a) Inaccurate
and imprecise; (b) accurate and imprecise; (c)
inaccurate and precise; (d) accurate and precise.
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 5/32
Lecture contents
1. Numerical Methods and Numerical AnalysisWhen they are methods of choice, objective
3. Approximation4. Concept of convergence
Cauchy sequence
4. Error Type- Model and Data Error
- Round Off and Truncation Error – Taylor, Mc Laurin series5. Error Analysis
– Forward, Backward
– Error propagation in a function
– Condition number of a problem – Cp
– Norms of a vector
– Round off error, Machine precision unit (u)
– Condition number of an algorithm – Ca
– Floating Point error
– Total Relative error
5/20/2015 5L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 6/32
Significant Figures
Basic rules for significant digits:-
1) All nonzero digits are significant.
2) All zeroes between significantdigits are significant.
3) All zeroes which are both(a) to the right of the decimal point
and,
(b) to the right of all non-zero
significant digits are themselvessignificant.
5/20/2015 L1 - Error Analysis 6
Example:
Number Significant
Digit
1.234 4
0.001 1
0.100 3
0.00100 3100 1
100.0 4
1.23 3
1.230 4
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 7/32
1. Concept of Numerical Methods and
Numerical Analysis
• Numerical method is an APPROXIMATION
→ Error exists
•
Define Numerical Method – The branch of mathematics that deals with transformation
of the mathematical problems to numerical calculations
• Need for Computation/ Numerical Methods in Engineering – Handling large datasets, repeated calculations,
– Examine result of variation in parameters
5/20/2015 7L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 8/32
The solutions often quick and is easily adapted for parametric
sensitivity studies.
• Objective
To obtain a reasonably accurate solution with optimum use of
resources
5/20/2015 8L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 9/32
Numerical Analysis
• Mathematical analysis conducted to determine whether thecomputational scheme converges or not is called numericalanalysis.
• Involves
– convergence analysis and – Error Analysis
5/20/2015 L1 - Error Analysis 9
• Problem - among the options available to solve a
problem, what is a numerical method?
• Answer - The computational scheme that converges to
the true solution can be termed as a numerical
method for computing the square root of a number.
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 10/32
Situations when numerical methods would be
the method of choice
1. integration of a function for which either the integral cannot
be expressed as analytical expressions or it is too
cumbersome and time consuming to evaluate,
2. solution of differential equations for complicated geometry,
and/or boundary conditions,
3. Large systems of equations, repeated solution of the same
system under changing conditions, etc.
5/20/2015 10L1 - Error Analysis
Need to know how the numerical methods work on
computers to trouble shoot or design new algorithms
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 11/32
Concept of convergence – Cauchy sequence
• Function CONVERGES → Limit exists
• Convergence
– monotonically decreasing sequence,
–
lower bound, – Cauchy sequence
5/20/2015 11L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 12/32
Error
Error = True value – approximation
3 basic kinds1. Data and Model Error
2. Round off/ Chop off
3. Truncation
5/20/2015 12L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 13/32
• Model and Data Error
Description Related to
Model error due to assumptions made
in arriving at an algorithm
describing a process
Precision
refers to how closely individual
computed or measured values
agree with each other.
Data error due to approximation of
values such as value of g
as 9.81 (may actually be
9.8088)
accuracy
refers to how closely a
computed or measured value
agrees with the true value.
5/20/2015 13L1 - Error Analysis
Ex. Data error : in case of function y = 5 ekx
Different values of k can lead to error in values of y.
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 14/32
• Truncation and Round off error
Error = True value – approximation
Two types – truncation and round off
Error Type Description
Truncation
Result when approximations are used to
represent exact mathematical procedures
Round-off Result when numbers having limited significant
figures are used to represent exact numbers.
E.g. value of ‘g’ is used as 9.81 (may actuallybe 9.8088)
5/20/2015 14L1 - Error Analysis
Question: Chopping vs. Round off ?
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 15/32
Note on value of ‘g’
• Although the precise strength of Earth's gravity variesdepending on location, the nominal "average" value atthe Earth's surface, known as standard gravity is, bydefinition, 9.80665 m/s2
• Parameters affecting the apparent or actual strength ofEarth's gravity include latitude, altitude, and the localtopography and geology.
• Gravity decreases with altitude, since greater altitudemeans greater distance from the Earth's centre
5/20/2015 15L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 16/32
What causes Truncation error (TE)?
This error is due to approximation of an infinite series by afinite number of terms or in other words, due to
truncation of the series after some points. Error
encountered in this way is termed as truncation error.
• e.g. Taylor Series
• Mc Laurin series (Taylor series with a = 0)
5/20/2015 16L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 17/32
Truncation Error in Series
• Taylor Series
• Mc Laurin series (Taylor series with a = 0)
For more details on these series – revise MTH 101, Thomas and Finney
5/20/2015 L1 - Error Analysis 17
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 18/32
Round Off error, Machine precision unit u
• Concept of significant figures and link to round off error
• The concept of significant figures has two important
implications for our study of numerical methods:
1. numerical methods yield approximate results.
• We must, therefore, develop criteria to specify how confident
we are in our approximate result.
•
One way to do this is in terms of significant figures.
5/20/2015 18L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 19/32
For example, we might decide that our approximation is
acceptable if it is correct to four significant figures.
2. All computing machines have finite maximum length for
storing a number, they are always rounded-off after a few
significant digits following decimal.
• This brings us to Floating Point representation. Lets revise.
5/20/2015 19L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 20/32
Floating Point representation of a number
• Fractional quantities are typically represented in computers
using floating-point form.
• In this approach, the number is expressed as a fractional part,
called a mantissa or significant. and an integer part, called an
exponent or characteristic, as in
m . be
• where m = the mantissa, b = the base of the number system
being used, and e = the exponent.
• b=2 – binary, 10 – decimal, 16 - hexadecimal
5/20/2015 20L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 21/32
• For instance, the number 156.78 could be represented as
0.15678 x 103 in a floating point base-I0 system.
Mantissa normalization
• Mantissa is usually normalized if it has leading zero digits.• The consequence of normalization is that the absolute value
of m is limited. That is,
I/b < = m < 1
• where b = the base.
5/20/2015 21L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 22/32
Example
• for a base-l0 system, m would range between 0. I and I,
• for a base-2 system, between 0.5 and I.
•
Important: Normalization introduces a source of error – why? because the mantissa holds only a finite number of
significant figures.
– Thus, a round-off error is introduced.
Self Study:• Derivation of machine precision unit (u)
• Calculation of Backward error
5/20/2015 22L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 23/32
Error Analysis
CONTENT
• Forward, Backward Error Analysis
• Error propagation in a function
•
Condition number of a problem – Cp• Condition number of an algorithm – Ca
• Total Error
5/20/2015 23L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 24/32
Forward, Backward Error Analysis
Given error at the beginning (∆x), determine how much error
will be caused in final result (∆y).
– More relevant Data Error
Backward error analysis:
• Given an error incurred, determine what it is equivalent to
in parameter values
• OR process of estimating perturbation (∆) required in the
input variable (x) to explain the final result
5/20/2015 24L1 - Error Analysis
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 25/32
Backward Error Analysis
• In short: you know – final error (∆y)
• You predict – what change/adjustment is needed to
cause/ remove the final error
• Advantage of Backward error analysis over Forward Error
we do not need to know the true value of the result, which
was needed in a forward error analysis
5/20/2015 L1 - Error Analysis 25
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 26/32
Error propagation in a function
Question: How the error in input data is translated to output
(even when no other error type such as truncation/ round off)
is encountered?
Answer:
• We know that Error in y is a function of data itself.
• Hence, objective is estimating the error introduced in the final
computation of the functional value as a result of these errors
in the variables (due to errors discussed above – data, model,
truncation, round off)
26
Error Bounds:In general, error is estimated using - Linear forward analysis
• Retain only the first order approximation term
Question: Why only first order?
Answer - Generally value ∆x is small and higher order analysis makes iteven smaller
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 27/32
True, and Relative Error• True error (e) = True value - Approximate value
• True Relative error
Why do we calculate Relative error?
Answer:
• To give a better idea of size of error vs. value concerned.
• Sometimes the error may be large, but only say 1% of True value.
Example:
TV = 1000, Calculated 950.
• Absolute error = 50, Relative error = 50/1000 = 0.05)
• Note: It is always better to use Relative Error than True Error
When TV is not known and needs to be approximated by a sequence:
5/20/2015 L1 - Error Analysis 27
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 28/32
Error Estimates for Iterative Methods
Approximate Error
Approximate error (e) = Current approximation - Previous
approximation
• Approximate relative error
5/20/2015 L1 - Error Analysis 28
Tutorial 1 problem: Ex. 3.2 Chapra Ed. 6
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 29/32
Condition number of a problem (Cp)
• Definition: ratio of the relative change in the function f(x), to
the relative change in the variable x for a small perturbation
dx,
• Ill conditioned vs. well conditioned
• derive values of Cp for each
• How can each type of problem be handled p
– USE higher precision
•
– i.e. reduce truncation error (include higher order terms)• Use different algorithm
5/20/2015 L1 - Error Analysis 29
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 30/32
Condition number of an algorithm (Ca)
• Definition: A measure of the change required in the input data toexplain the errors in the output result is termed the conditionnumber of the algorithm (C4).
How can this be obtained?
• Answer: This is done through a backward error analysis, whichdetermines the change necessary in the input variable (or data) toexplain the final error in the output (or result) obtained througherrors introduced at various stages of calculation.
• Importance/ Usefulness: among several algorithms available for aproblem, use the one with lowest value of Ca.
• Actual error in a computation –
• combines Cp and Ca
5/20/2015 L1 - Error Analysis 30
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 31/32
Total Numerical Error
5/20/2015 L1 - Error Analysis 31
A graphical depiction of the trade-off between round-off and truncation error that
sometimes comes into play in the course of a numerical method. The point of diminishing
returns is shown, where round-off error begins to negate the benefits of step-size
reduction.
7/23/2019 L1 - Error Analysis
http://slidepdf.com/reader/full/l1-error-analysis 32/32
5/20/2015 L1 - Error Analysis 32
Error Analysis of Numerical Differentiation