topic 8 linear systems - teaching.csse.uwa.edu.au

CITS2401 Computer Analysis & Visualisation

SCHOOL OF COMPUTER SCIENCE AND SOFTWARE ENGINEERING

FACULTY OF ENGINEERING, COMPUTING AND MATHEMATICS

Topic 8 Linear Systems

Material from MATLAB for Engineers, Moore, Chapters 10,13 Additional material by Peter Kovesi and Wei Liu

The University of Western Australia

Systems of Linear Equations

3x +2y −z = 10−x +3y +2z = 5x −y −z = −1

•  In the last lecture we saw how Matlab could solve systems of linear equations

A =3 2 −1−1 3 21 −1 −1

"

#

$$$

%

&

'''

A \ B =−25−6

"

#

$$$

%

&

'''

B =105−1

"

#

$$$

%

&

'''

•  Using left division we are able find values for x, y and z that satisfy these equalities.


Applications of Linear Systems ◊  We will look at two engineering applications of solving systems of linear

equations. ◊  The first uses a system of linear equations to represent an electronic circuit

and solve for unknown values. ◊  The second considers the problem of mixing fluids with different

concentrations.


Kirchhoff’s Circuit Laws ◊  Kirchhoff’s first circuit law states that sum of the currents entering a node is

equal to the sum of the currents leaving a node. ◊  Kirchhoff’s second circuit law states that the sum of all voltage drops around

a loop is equal to 0

•  In this system, from the first law we can see that I3= I1+I2

•  Using the second law, and Ohm’s law (V = IR or voltage = current x resistance), we have that 10I1+40I3 = 10


Solving the System ◊  To work out the current across the three resistors we apply Kirchhoff’s and

Ohm’s laws to build a system of linear equations

I1 +I 2 −I 3 = 010I1 +40I 3 = 10

20I 2 +40I 3 = 20

A =1 1 −110 0 400 20 40

"

#

$$$

%

&

'''

B =01020

!

"

###

$

%

&&&


Solution: ◊  State the problem: Find the current across the three resistors in the

system. ◊  Describe input and output: two batteries (Voltage), three resistors (Ohms),

and 3 currents (I1, I2 and I3, in amps) as output. ◊  Develop a simple example: using Kirchhoff’s and Ohm’s laws we can build

simple scenarios. For example with no voltage in the system we would have no current.

◊  Build a Matlab solution: the Matlab solution on the next slide allows for the user to set the voltages of the batteries and the resistors.

◊  Test the solution: for the stated system we get ◊  I1 = -0.14 amps ◊  I2 = 0.42 amps ◊  I3 = 0.29 amps

The negative value for I1 indicates that the current is going in the other direction.


Generalizations •  The same principal can be

applied to solve much larger circuit systems

•  If the system is underspecified (for example two equations but three unknowns) Matlab will set some of the variables to 0 to make the system fully specified.

•  If the system is over specified (eg three equations and two unknowns) Matlab will produce an answer, using the least squares method to resolve any anomalies.


Fluid mixing •  Suppose we had several different fluids with different concentrations of

components. •  For example, suppose we had three barrels of fuel, one with 90% petrol

and 5% ethanol, one with 80% petrol and 15% ethanol, and one with 95% petrol and 4% ethanol.

•  How can we produce a mixture which has a given concentration, say 85% petrol and 10% ethanol?


Solution: ◊  State the problem:

◊  Describe input and output:

◊  Develop a simple example:

◊  Build a Matlab solution: ◊  Test the solution:


ETHANOL : 0.05x1 +0.15x2 +0.04x3 = 0.10PETROL : 0.90x1 +0.80x2 +0.95x3 = 0.85Other : 0.05x1 +0.05x2 +0.01x3 = 0.05

0.05 0.15 0.040.90 0.80 0.950.05 0.05 0.01

!

"

###

$

%

&&&

x1

x2

x3

!

"

####

$

%

&&&&

=0.100.850.05

!

"

###

$

%

&&&

A= 0.05 0.15 0.040.90 0.80 0.950.05 0.05 0.01

!

"

###

$

%

&&&, X =

x1

x2

x3

!

"

####

$

%

&&&&

, b =0.100.850.05

!

"

###

$

%

&&&


output


Symbolic Differentiation ◊  If you have studied calculus, you will be familiar with symbolic

differentiation.

◊  Matlab is able to symbolically differentiate equations

◊  First you need to create some symbolic variables.

◊  Use either

! !x=sym(‘x’)!

or

! !syms x!

◊  using the diff function:

diff('(x^2+exp(2^(sin(x))))')

ans =

2*x + 2^sin(x)*exp(2^sin(x))*log(2)*cos(x)

◊  First you


Symbolic Differentiation ◊  Matlab is able to symbolically differentiate equations using the

diff function:

diff('(x^2+exp(2^(sin(x))))')

ans =

2*x + 2^sin(x)*exp(2^sin(x))*log(2)*cos(x)


Numerical differentiation: diff Function

◊  Numerical differentiation also uses the diff function

◊  The diff function is easy to understand, even if you haven’t taken Calculus

◊  It just calculates the difference between the points in an array


-1 0 1 2 3 4 5 60

2

4

6

8

10

12

14

16Sample Data

x-axis

y-ax

is

slope =y2 − y1x2 − x1





The derivative of a data set can be approximated by finding the slope of a straight line connecting each data point


The slope is an approximation of the derivative – in this case based on data measurements


0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

-5

-4.5

-4

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0

Rate of Change

time, hour

Rat

e of

tem

pera

ture

cha

nge,

deg

rees

/hou

r

The calculated slopes are discontinuous, if they are based on data. The appearance of this graph was adjusted using the interactive plotting tools.


Approximating Derivatives when you know the function ◊  If we know how y

changes with x, we could create a set of ordered pairs for any number of x values. The more values of x and y, the better the approximation of the slope

-2 -1 0 1 20

1

2

3

4y=x2

y-ax

is(a)

-2 -1 0 1 2-4

-2

0

2

4Slope of y=x2

(b)

-2 -1 0 1 20

1

2

3

4y=x2

y-ax

is

(c)-2 -1 0 1 2

-4

-2

0

2

4Slope of y=x2

(d)


-2 -1 0 1 20

1

2

3

4y=x2

y-ax

isx-axis

(e)

-2 -1 0 1 2-4

-2

0

2

4Slope of y=x2

x-axis(f)

The slope of a function is approximated more accurately, when more points are used to model the function

If we knew the function, why wouldn’t we just use the symbolic method?


Forward, Backward and Central Difference Techniques ◊  What if you want to approximate the derivative at a point,

instead of over a range?

◊  One approach is to use the slope between adjacent points

1

1

i i

i i

y ydydx x x

+

+

−⎛ ⎞ =⎜ ⎟ −⎝ ⎠

Implement in MATLAB with dydx = diff(y)./diff(x)

Forward Difference


Let’s approximate the derivative of sin(x)

◊  We know from basic calculus that the derivative of sin(x) is cos(x)

◊  How accurate an estimate can we get by using a difference calculation?

sin( )

cos( )

y xdy xdx

=

=


Notice that the size of dydx_analytical and dydx_approx are different


NaN is a placeholder NaN: Not a Number


Backward Differences

◊  Instead of approximating the derivative by forecasting forward, we can use the point before the current point in the approximation.

1

1

i i

i i

y ydydx x x

−

−

−⎛ ⎞ =⎜ ⎟ −⎝ ⎠


Add the placeholder, NaN, at the beginning of the array


Central Difference

◊  The absolute error associated with the forward difference and backward difference is very similar

◊  We can get closer using a central difference – where we use the point before and the point after the point of interest

1 1

1 1

i i

i i

y ydydx x x

+ −

+ −

−=

−


gradient function ◊  One downside of this approach is that it won’t work for either

the first point or the last point.

◊  Matlab includes a function called gradient that uses

◊  Forward difference for the first point

◊  Backward difference for the last point

◊  Central difference for the intermediate points

>> gradient(y,x)


Error in numerical differentiation ◊  The definition of the derivative of a function f(x) is the limit as

h->0 of

◊  This equation directly suggests how you would evaluate the derivative of a function numerically. Pick a small value of h and evaluate your function.

◊  There are two sources of error,

•  the discrete approximation introduced by the equation above for non-zero values of h, and

•  numerical roundoff error.

hxfhxfxf

h

)()(lim)(0

−+=ʹ′

→


The discrete approximation error ◊  Using the Taylor series expansion we have

◊  This implies

error from non-zero h

◊  The error is a function of the magnitudes of 2nd and higher order derivatives

...)(61)(

21)()()( 32 +ʹ′ʹ′ʹ′+ʹ′ʹ′+ʹ′+=+ xfhxfhxfhxfhxf

f (x + h)− f (x)h

= "f + 12h ""f +...


Numerical roundoff error

◊  There is likely to be roundoff error in representing h.

◊  For example: assume you are at x = 1000.0 and you have chosen h = 0.000001

◊  Neither x = 1000.0 nor x + h = 1000.000001 may be a number that has an exact representation in binary

◊  The effective error in h will be the error in evaluating the difference between 1000.0 and 1000.000001. This will be of the order EPS*x

EPS (epsilon) is the smallest difference that may be represented by the machine.


Numerical roundoff error

>> help eps!

EPS Spacing of floating point numbers.

D = EPS(X), is the positive distance from ABS(X) to the next larger in magnitude floating point number of the same precision as X.

>> eps(1)!

ans = 2.2204e-16!

>> eps(45)!

ans = 7.1054e-15!


Roundoff error ◊  Under single precision EPS is ~ , ◊  under double precision EPS is ~

◊  Thus the error in evaluating the difference between 1000.0 and 1000.000001 (~EPS*x) will be of the order for single

precision and for double precision.

◊  The fractional error in h will be for single precision !!! ◊  under double precision it will be , ok. ◊  Thus the errors you get are a function of the magnitude of x

(where on the function you are evaluating the derivative), and of the magnitude of h.

710−

26

4

101010

=−

−

1610−

76

13

101010 −

−

−

=

410−1310−


Integration ◊  Differentiation involves taking differences between

function values, integration involves addition. Numerical integration is also known as quadrature.

◊  We have a function f(x) that is discretely sampled at locations x0, x1, x2, ... to produce values f0, f1, f2,... In the simplest case the samples are a constant step size of h apart


Numerical Integration

◊  MATLAB handles numerical integration with two different quadrature functions

◊  quad

◊  quadl


0 2 4 60

5

10

15

An integral can be approximated by the area under a curve

y-ax

is

x-axis0 2 4 6

0

5

10

15A Trapazoid rule approximation

y-ax

is

x-axis

The area under a curve can be approximated using the trapezoid rule.

An integral is often thought of as the area under a curve


Trapazoid rule

These areas are equal

A= xi+1 − xi( )(yi+1 + yi )i=1

n−1

∑ / 2

xi+1

yiyi+1

xi width x average height


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

x-axis

y-ax

is

Evaluation of Data using the Trapazoid Rule

datacalculated midpoints

The integral of a function can be estimated using the trapezoid rule.

In Matlab the function: trapz(x,y) will use the trapezoid rule to approximate an integral.!


The trapezoid rule

◊  The building blocks for integration algorithms are simple formulas that integrate a function over a small number of intervals.

◊  The simplest algorithm is the trapezoid rule, this integrates a function over a single interval. It is exact for functions that are polynomials up to degree one (ie straight lines)

∫ +=

2

1

)21

21()( 21

x

xffhdxxf


Extend the trapezoid rule ◊  To integrate a function you repeatedly apply the trapezoid rule to

successive intervals of your function, from x1 to x2, x2 to x3, x3 to x4, etc

◊  If you apply the trapezoid rule N-1 times over N points you can combine the results to produce the formula

(Obviously very easy to implement in MATLAB)

◊  The trapezoid rule is a 2 point formula that is exact for polynomials up to degree 1. Formulas involving more points provide solutions that are exact for sampled polynomials of higher orders.

∫ +++++= −

nx

x nn fffffhdxxf1

)21...

21()( 1321


Simpson's rule: - a 3 point formula ◊  This rule has the nice property that it is exact for

sampled functions up to degree 3 due to a cancellation of error terms resulting from the left-right symmetry of the formula. ◊  (There are 4, 5, and greater point formulas but they

are generally not worth the bother) ◊  If you apply Simpson's rule to successive non-

overlapping pairs of intervals you get the formula

∫ ++=3

1

)31

34

31()( 321

x

xfffhdxxf

f (x)dx = h(13f1 +

43f2 +

23f3 +

43f4 +...+

43fn−1 +

13fn )x1

xn∫


How many intervals?

◊  The general strategy is to start with the integral evaluated using a coarse number of subdivisions, and then repeat the process with finer and finer subdivisions. The issue is when to stop.

◊  Assume the integral evaluated at some subdivision level will be Q, and the integral evaluated at the previous subdivision will be Qlast, if the difference between Q and Qlast is less than EPS*Q then we are at the limits of machine precision and we know there is no point in going any further.


Matlab’s function for numerical integration ◊  Efficient integration algorithms re-use the calculations made at

courser subdivision levels to generate the successive refined values.

◊  MATLAB has a function 'quad' that performs numerical integration

◊  Q = quad('F',A,B) approximates the integral of F(X) from A to B to within a relative error of 1e-3 using an adaptive recursive Simpson's rule. 'F' is a string containing the name of the function.

>> Q = quad('sin',0,2*pi)

Q = 0


Quadrature functions

◊  quad uses adaptive Simpson quadrature ◊  quadl uses adaptive Lobatto quadrature ◊  Both functions require the user to enter a function in the

first field. •  called out explicitly as a character string

•  defined in an M-file •  anonymous function.

◊  The last two fields in the function define the limits of integration


Quadrature Functions


Monte Carlo Integration ◊  There can be times when numerical integration can be

problematic, especially if one is evaluating double or triple integrations, or the function one is working with is very expensive to evaluate.

◊  The solution may be to use a Monte Carlo algorithm.

◊  Monte Carlo algorithms always give an answer, but the answer is not always correct. The probability of correctness increases as the time the algorithm runs increases.

◊  Simple example - Estimate the value of pi using a random number generator.


Estimating Pi

◊  How do we estimate this? ◊  Generate random points in the square No of points within r of the centre ------------------------------------------ Total number of points is our estimate of circle_area/square_area

squareofareacircleofarea

Thereforersquareofarearcircleofarea

____4

,4__

__2

2

=

=

=

π

π


Example: !

>> xs = rand(1000,1);!

>> ys = rand(1000,1);!

>> zs = xs.^2+ys.^2<1;!

>> err = pi/4 - sum(zs)/1000!

!

err =!

!

-0.014601836602552!


The volume of the intersection of a torus (donut) with a box

◊  Three simultaneous conditions

(the torus, centred on the origin

with major radius of 4

and minor radius of 2)

x >= 1 } - the two sides of the box

y >= -3 } that intersect the torus

1)3)(( 2222 ≤−++ yxz


Advantages of using Monte Carlo ◊  This is an example of how a relatively simply defined problem can be very

difficult to express in an analytical form. The integral that has to be evaluated is very awkward.

◊  But the implementation of a Monte Carlo algorithm is very simple. Generate points in 3-space and count the number of points that satisfy the three constraints above.

◊  The MATLAB function 'rand' generates uniformly distributed random numbers. To generate random 3D points just extract groups of 3 numbers from 'rand'.

◊  This kind of algorithm is an example of a solution technique that you cannot conceive of doing working with pencil and paper - They are solutions that arise from having a computing machine.

topic 8 linear systems - teaching.csse.uwa.edu.au

Documents