MA2213 Lecture 3

Approximation

Piecewise Linear Interpolation p. 147

],[,)( 2112

12 xxx

xx

xxx

1x

],[,)( 3223

23 xxx

xx

xxx

],[,)( 2112

21 xxx

xx

xxx

],[,0)( 213 xxxx

3x

3

2x1

2

1

1

11x

1x

2x

2x

3x

3x

],[,0)( 321 xxxx

],[,)( 3223

32 xxx

xx

xxx

can use Nodal Basis Functions

IntroductionProblem : Find / evaluate a function P that

(i) P belongs to a specified set S of functions, and (ii) P best approximates a function f among the

functions in the set S

Approximates = match, fit, resemble

If S were the set of ALL functions the choice P = f solves the problem – “you can’t get any closer to somewhere than by being there”. If S is not the set of all functions then S must be defined carefully

Furthermore, we must define the approximation criteria used to compare two approximations

Set of FunctionsIn practice S is closed under sums and multiplication

by numbers – this means that it is a vector space

Example: Bases for S = { Polynomials of degree < n } :

Furthermore, S is usually finite dimensional

},...,:)()({ 11RccxbcxPS nj

n

j j

and then S admits a basis })(),...,({ 1 xbxb n

Monomial Basis

Lagrange’s Basis

Newton’s Basis (used with divided differences)

)(),...,(1 xLxL n

12 ,...,,,1 nxxx

)()(),...,)((),(,1 11211 nxxxxxxxxxx

For distinct nodes nxx 1

For possibly repeated nodes nxx 1

Approximation CriteriaLeast Squares p. 178-187

http://en.wikipedia.org/wiki/Carl_Friedrich_Gauss

observed that measurement errors usuallyhave “Gaussian Statistics” and he invented an optimum method to deal with such errors.

In many engineering and scientific problems, data is acquired from measurements

Minimax or Best Approximation p. 159-165Arises in optimal design, game theory as wellas in mathematics of uniform convergence

Least Squares Criteria

Least Squares (over an finite set)

m

k kk xPy1

2))((minimize

Least Squares (over an interval)

b

adxxPx 2))()(f(minimize

Least Squares Over a Finite Set p. 319-333

If

m

k kk xPy1

2))((then we minimize

)()(1

xbcxP j

n

j j

21 1

)(

m

k

n

j kjjk xbcy

by choosing coefficients

ncc ,...,1to satisfy

nixbcyc

m

k

n

j kjjki

,...,1,0)(2

1 1

Least Squares Over a Finite Set

Remark: these are n equations in n variables

Since

ncc ,...,1the coefficients

m

k

n

j kjjki

xbcyc 1 1

2)(

nixbxbcy ki

n

j kjjk

m

k,...,1),()(2

11

satisfy the equations

niyxbcxbxbm

k kkij

n

j

m

k kjki ,...,1,)()()(11 1

Least Squares Equations

Construct matrices

niyxbcxbxbm

k kkij

n

j

m

k kjki ,...,1,)()()(11 1

)()(

)()(

1

111

mnm

n

xbxb

xbxb

B

nc

c

c 1

my

y

y 1

The least squares equations are yBBcB TT The interpolation equations yBc hold if and only if mkyxP kk ,...,1)( , Question When do these equations have solutions ?

Least Squares ExamplesFor

mxxxmxbn 211 ,1,1)(,1

1

1

B 1cc

my

y

y 1

The least squares equations are

m

yyc m

11

so

and the constant function

myyy 21

myymc 11

1111 1)( ccxbc is the least squares approximation or data fit for the data points ),(),...,,(),,( 2211 mm yxyxyx

by a constant function.

Least Squares ExamplesFor

mxxxmxxbxbn 2121 ,2,)(,1)(,2

mx

x

x

B

1

1

1

2

1

2

1

c

cc

my

y

y 1

The least squares equations are

myyy 21

mxxxS 211

The solution gives the least squares data fit

2

1

2

1

21

1

T

T

c

c

SS

Sm22

2212 mxxxS

myyyT 211

mm yxyxyxT 22112

xcc 21 by a polynomial of degree 1

Least Squares MATLAB CODEfunction c = lspf(n,m,x,y)% function c = lspf(n,m,x,y)%% Wayne Lawton 28 August 2007% Least Squares Polynomial Fit% Inputs : n = deg poly + 1, m = # data points, % data arrays x and y of size n x 1% Output : array c of poly coefficients%for i = 1:m for j = 1:n B(i,j) = x(i)^(j-1); endendc = (B'*B)\(B'*y);

Least Squares MATLAB CODE

Least Squares MATLAB CODE

sum of squares error for constant least squares fit

sum of squares error for ‘linear’ least squares fit

1.3250

13.469Question : Why did the ssq error decrease ?

Least Squares Algebraic FormulaThe sum of squares error

)()(.Sq Sum yBcyBc T can be computed, by substituting the value,

yBBBc TT 1)( to obtain

)(.Sq Sum BcyyT

Least Squares MATLAB CODE

ssq error for quadratic ls fit 76.463Question : Why did the error decrease so little ?

Marge, Where are the Least Squares ?

Least Squares Over an IntervalIf

b

adxxPx 2))()(f(then we minimize

)()(1

xbcxP j

n

j j

b

a

n

j jj dxxbcx2

)()(f1

by choosing coefficients

ncc ,...,1to satisfy

nidxxbcxc

b

a

n

j jji

,...,1,02

)()(f1

Least Squares Over an IntervalThese are also n equations in n variables

Since

nidxxbxbcxb

a i

n

j jj ,...,1,)()()(f21

the coefficients

ncc ,...,1satisfy the equations

b

a

n

j jji

dxxbcxc

2)()(f

1

b

a ij

n

j

b

a ji nidxxbxcdxxbxb ,...,1,)()(f)()(1

The ‘ interpolation equation ‘ on the interval is

],[),(f)()(1

baxxxbcxPn

j jj

Least Squares Over an Interval p. 181For S = polynomials of degree

nn xxbxxbxb )(,...,)(,1)( 121

over the interval

n

the equations are

1

0

1

0

11

11,,1,)(f)()(f

1nidxxxdxxbx

ji

c ii

n

j

j

The matrix of coefficients for these equationsis called the Hilbert matrix. It is a well-known example of an ill conditioned matrix and is discussed in Example 6.5.5 on pages 300-301

]1,0[

Legendre Polynomials p. 181-183defined by n

n

n

nn xdx

d

nxP 1

2!

1)( 2 ,1)(0 xP

xxxPxxPxxP 233

25

3212

23

21 )(,)(,)(

satisfy 1

1else ,0)()( dxxPxPji ji

)12/(2)()(1

1 jdxxPxP jj

they are examples of Orthogonal Polynomials

and are useful method to solve the least squared approximation by polynomials p. 183-185

Vandermonde Matrix

1

122

111

1

122

111

1

1

1

det)(),(

1

1

1

det

n

n

n

n

nnn

n

n

xx

xx

xx

xPxP

xx

xx

xx

n

i

i

jjin xxxP

2

1

1

)()(

1

11111 )(),...,()(0)()(n

kknn xxxxxPxPxP

211

222

211

1

1

1

det

nnn

n

n

xx

xx

xx

then use induction on n ),...,( 11 nxx

first expand det by last row to obtain

and

to obtain

Gramm Matrix

nnn

n

gg

gg

G

1

111

Theorem 1. If

dxxbxbgb

a jiij )()(

nibaCbi ,...,1]),,([

is nonsingular.

0],...,[ 1 TnvvvProof

are linearly

independent, then the Gramm matrix defined by

00)(2

1

GvdxxbvvGvb

a

n

j jjT

01

n

j jjbvh

nbb ,...,1lin. ind. and

Derive this equation

Semipositive Definite and Positive DefiniteDefinitions A real n x n matrix is (semi) positivedefinite if for every nonzero vector

Theorem 2. If is both positive semidefinite and symmetric and 0vPvT

nRv

satisfies

0)( vPvT

nRvthen .0vPProof Assume that

)()()( xuvPxuvxF Tu

nRuconstruct the functionand observe that since is positive semidefinite

0)0(),(0)0( dxdF

uuuRxxFF

.0vPvT For every

Since ,PPT

P

P

P

02)0( PvuTdxdFu therefore

nRu .0vP0PvuT for every hence

Gramm MatrixCorollary (of Thm 2). If the Gramm matrix for a

.0v

nibaCbi ,...,1]),,([ is nonsingular then this set of functions is linearly independent.

nTn Rvvv ],...,[ 1

isProof. Assume that the Gramm matrix

0)(2

1

dxxbvvGvb

a

n

j jjT

.01

n

j jjbv

Gnonsingular and that

set of functions

satisfies

It suffices to show that

We observe that

Furthermore, since satisfies the hypothesis ofGtheorem 2, it follows that 0Gv and the proof

is complete.

Questions for Thought and Discussion

Question 1. Is the matrix

2,00 RvMvMvvT Is it true that

semi positive definite, positive definite,symmetric ?

10

01M

Question 2. Give and example of a positive semidefinite matrix that is not positive definite.

Question 3. Find a positive semidefinite and a nonzero vectorMnonsingular matrix

such that

v.0MvvT

Question 4. Give a detailed derivation for all the assertions in the proofs of Theorems 1 and 2.

http://en.wikipedia.org/wiki/Image:Langrange_portrait.jpg

Joseph Louis Lagrange

Jan 1736 – April 1813Giuseppe Lodovico Lagrangia was an Italian-French mathematician and astronomer who made important contributionsto all fields of analysis and number theory and to classical and celestial mechanics as arguably the greatest mathematician of the 18th century. Before the age of 20 he was professor of geometry at the royal artillery school at Turin.

http://en.wikipedia.org/wiki/Isaac_Newton

Isaac Newton

January 1643 – March

1727)was an English physicist, mathematician, astronomer, natural philosopher, and alchemist, regarded by many as the greatest figure in the history of science. His treatisePhilosophiae Naturalis Principia Mathematica, published in 1687, described universal gravitation and the three laws of motion, laying the groundwork for classical mechanics.

http://en.wikipedia.org/wiki/Carl_Friedrich_Gauss

Carl Friedrich Gauss

April 1777 – February 1855 was a German mathematician and scientist of profound genius who contributed significantly to many fields, including numbertheory, analysis, differential geometry, geodesy, magnetism,astronomy and optics. 23, heard about the problem After three months of intense work, he predicted a position for Ceres in Dec 1801 …influential treatment of the method of least squares …minimize the impact of measurement error …under the assumption of normally distributed errors …when asked how he had been able to predict the trajectory of Ceres with such accuracy he replied, "I used logarithms." The questioner then wanted to know how he had been able to look up so many numbers from the tables so quickly. "Look them up?" Gauss responded. "Who needs to look them up? I just calculate them in my head!"

Homework Due Tutorial 2

Question 2. Use Newton’s divided differencemethod to find a polynomial of degree that satisfies

Question 1. Do Problem 8 (a) on page 133.

Question 3. Do Problem 1 on page 330. Please show all details in your computation.

)(2 xP 221)4(,0)1()1( 2

'22 PPP

Question 4. Compute the Gramm Matrix for the three piecewise linear nodal basis functions in the first vufoil if the three nodes are -1, 0, 1 and the interval of integration is [-1,1]

Homework Example

dxxxg )()( 2

1

`1 112

Question 4. Compute the Gramm Matrix for the three piecewise linear nodal basis functions in the first vufoil if the three nodes are -1, 0, 1 and the interval of integration is [-1,1]Solution The Gram matrix will be a 3 x 3 matrix, here is one of its entries

dxxdxxx )1()0()1()(1

`0

0

`1

6

10

0

12

213

31 xx

Questions for Thought and Discussion

Answer 1. The matrix

01

1but 0

1

1

MvMvvv T

is symmetric, however it is not semi positivedefinite and therefore not positive definite.

10

01M

Answer 3. The matrix is semi positive

10

21M

definite and nonsingular and if

1

1v

then 01

1]11[

MvvT and clearly

0Mv