3079

INTERPOLATION AND APPROXIMATION

THESIS

Presented to the Graduate Council of the

North Texas State University in Partial

Fulfillment of the Requirements

For the Degree of

MASTER OF SCIENCE

By

Ram Lal, B.S., M.S.

Denton, Texas

May, 1977

Lal, Ram, Interpolation and Approximation, Master of

Science (Mathematics), May, 1977, 53 pp., bibliography,

6 titles.

In this paper, there are three chapters. The first

chapter discusses interpolation. Here a theorem about the

uniqueness of the solution to the general interpolation

problem is proven. Then the problem of how to represent

this unique solution is discussed. Finally, the error

involved in the interpolation and the convergence of the

interpolation process is developed.

In the second chapter a theorem about the uniform

approximation to continuous functions is proven. Then the

best approximation and the least squares approximation (a

special case of best approximation) is discussed.

In the third chapter orthogonal polynomials as dis-

cussed as well as bounded linear functionals in Hilbert

spaces, interpolation and approximation and approximation

in Hilbert space.

TABLE OF CONTENTS

Chapter Page

I. INTERPOLATION ... .......... 1

II. APPROXIMATION.... ........... 25

III. HILBERT SPACE. . ........ . . . . . . 41

BIBLIOGRAPHY..-. .--.-...... .. ..... ... 53

iii

CHAPTER I

INTERPOLATION

In this chapter we will start with the interpolation

problem and will describe some interpolation processes,

the error involved in the interpolation and finally the

convergence of interpolation processes.

Definition 1.1. Let X be a given linear space. The

set of linear functionals defined on X forms a linear space

called the algebraic conjugate space of X and is denoted

by X*.

Theorem 1.1. If X has dimension n then X* has dimension

n also.

Proof. Let X have dimension n. Let x1,...,Xn be a

basis for X. Now, if x EX, then it can be uniquely expressed

as

x = a x + a2x + + ax11 2 2 n nTherefore, for a functional L EX* we have

L(x) = a L(x ) + a2L(x2) + ... + aL(x )1 . 2 ) +n nFor any x E X, let L.(x) = a ; i = 1,2,....,n. L. are func-

tional defined on X.

We will show L are independent. Suppose the L. are

not independent. Then for some . / 0, we have

1 1 1 + 2L2++ n Ln n= 0.

1

2

Therefore,

SL )+L 2L2( j) + . i.. L.(x.)+ (1.1)

+n L (x.) =0(x.) = 0.

We know L.(X.) = 0 when i / j and L.(x.) = 1 when i = j.

Thus from (1.1) we get

0+ 0 + .. + . 1+0 + ... + 0 = 03

or . = 0, a contradiction. Therefore, the L. are inde-

pendent.

We next show that every n+1 functionals are dependent.

Let L , .0.., Ln, Ln+1 E X*. Consider the n+1 n-tuples

(L 1),L(x2),. ..,Ln(x))'

i = 1,2,.. .,n+1. Since Rn (or Cn) is of dimension n,

therefore these n-tuples are dependent. Hence we can find

YlIY2.'''''n+1 not all zero such that

n+1y. i(L(X1 ),L.(x2),...,L1 n)) = 0 = (0,0,...,0).

i=l1

Therefore,

(yL1 + Y2 L2 + --- + Yn+y1 Ln+1 i)(x) = 0 ,

for i = 1,. ..,n. By taking linear combinations, we get

(yL1+ y2 L 2 + + yn+1 L 1 ) (x) = 0 for x E X.

Therefore, L 1 ,L2,...,LA+1 are dependent. Hence X* is of

domension n.

Q.E.D.

Now we define the general interpolation problem. Let

X be a linear space of dimension n and let L1 ,L2 ,...,Ln

be n given linear functionals defined on X. For given

numbers w1 ,w2,' .'w *n, can we find an element of X, say x,

3

such that

L.(x) = w i = 1,2,..

The answer is "yes" if the L. are independent in X*.

Lemma 1.2. Let X have dimension n. If x1 ,...x

are independent in X and Li,.*.Ln are independent in X*

then the determinant ILL(x.) I 0. Conversely, if either

x1,*...xn or L ,...,Ln are independent and IL (x)I 0,

then the other set is also independent.

Proof. Let x1 ,x2 .. 'xn be independent in X and

LL2,...,OJLnbe independent in X*. Suppose that ILi(x)H 0.

Then also IL.(x.) I = 0. The systemJ31

a L (x1 ) + a2L2 (x) + --- + anLn1(x1)=0

a L (xn) + a2L2(Xn) + --- + a L (x ) =011n 22n n n n

has a non-trivial solution a1 ,...,an. Then

(aL+ a2L2 + --- + anL )(x) 0,

i = 1,2,...,n. Since xl*.*xn forms a basis for X,

(alL + a2 L 2 +--- + anLn)(x) = 0 for x E X

and hence a1 1 + a 2L2 +...+ anLn = 0. This is a contra-

diction. Therefore ILi(x )I 0. Conversely, let

x 1 ,..,xn be independent and JL.(x.) I 0. Suppose on the

contrary L1 ,..,Ln are dependent in X*. Then there exist

a 1 ,...,an (not all zero) such that

a 1L1+ ... + a Ln = 0 or

(aL + +...+ anLn)(x) = 0 for all x E X.

4

Now x1,...,xnEX and hence

(a L +. + an L )(x.) = 01 1 n n 1

i = 1,...,n or

a1 L1(x1 ) + ... + anLn (x 1) =0

a1L (x,)+ ... +anL (x) =0.u1n n n n

Since IL. (x.) I 0, the homogeneous system should have only

the trivial solution, i.e., all a. are zero. But this is

a contradiction. Hence L1 ,. ..,Ln are independent in X*.

Q.E.D.

Theorem 1.3. Let a linear space X have dimension n and

let LJ,L2 ,...,Ln be n elements of X*. The interpolation

problem Li(x) = wi, i = 1,2,...,n possesses a solution for

arbitrary values w1 ,w2 , ... ,wn if and only if the Li's are

independent in X*. The solution will be unique.

Proof. (i) Suppose the interpolation problem L.(x) =w.1

possesses a unique solution. We have to show the L. are1

independent. Let x ,x2' 'X'n be a basis for the linear

space X. Let x EX be the solution to the interpolation

problem Li(x) = wi, i = 1,2,...,n. Since x EX, x can be

written as a linear combination of x1 ,x2 ,..'n. Let

x = a1x + a2X 2 + +...+axnxn

Then L.(a1x1 + a2x2 + ... +a x =i., i = 1,2,...,n or

alL.(x1 )+ a2 L.(x2 )+ ... +aa L. (x ) = w.T ss L(x wpo2 s ni n 1

The system L.(x = w. possesses a non-trivial solution if

5

IL (X) 0. Since xix2 ' *''Xn are independent and

IL. (x.) I 0, then by the Lemma 1.2, the L. are independent.1 3 1

(ii) Suppose the L. are independent. We have to show

that the interpolation problem L.(x) = w., i = 1,2,...,n

possesses a unique solution; i.e., we have to show

JL .(x) I / 0. We know xi,x2 '.. .IXn are independent and

also the L. are independent. Therefore by Lemma 1.2,

Q.E.D.

We state without proof the following theorem.

Theorem 1.4. Let z0,z,.. *.,zn be distinct real (or

complex) numbers and

1 ~ n1 z 0...z 0

1 z.z n

V(z0,z1 ,...,)zn) (1.2)

1 z ... z nn n

Then the recursion formulan-A

V(Z 0 Z,,...,zn) =v(z0 ,z1 ,... *zn-1) .R (zn-z) (1.3)i=l

holds and hence the Vandermonde determinant,

nv(zO,z.,...,zn) . T(z.-z.). (1.4)

Now we give some examples of systems possessing the inter-

polation property.

Example 1.1. Let there be given n+l distinct (real or

complex) points z0 ,z1 ,. .,zn and n+l (real or complex)

6

values w0 ,w1 ,. . .,wn. There exists a unique polynomial

pn (n for which pn(zi) = w , i = 0,l,...,n. Here

L (pn n 1n(z - 1- - ,=,,...,n and 1,z,...,zn is a

basis for Yn(=X). To show there exists a unique poly-n

nomial pn(z) EP , we know by Theorem 1.3 and Lemma 1.2

that we have to show JL.(z3) I 0. Now,

n

n1 z 0 ... zn

L (z )I = . =V(zO,z,...,z = II (z.-z.).

n

Since z0 , z,. ..,Zn are distinct,

n

IL.(z) = II (z.-z.) 0.j >i

Hence, there exists a unique polynomial pn EP for which

Pn(z i) = w, i 0,1,...,n.

Example 1.2. Taylor Interpolation

Let X = Yn, L0 (f) = f(z 0),...,Ln(f) = f(n)(z 0) Let

fn)) n n .-f(z) =Z2 a.z2 . We have Li( Z a.z )(ZO0'

j=0 3 j=0 2

i = 0,1,...,n. Then

a0L0 (1) + a1 LO(z) + + anL0 (z n

a + a1z + + a zn0 10 nO =

a L (1) + aL (z) + ... + a L (zn) = n!aO n l n n n n

and

7

1 21z 0 Z 0 ''' Zn

0n -0 1 2z0 .. nz0

IL (z) I = 0 0 2...n(n-)zn-2 2!...n!0.

0. 0 0... n!

Since ILi(z 3 ) I 0 and zi are independent, by Lemma 1.2

the L. are independent. Hence by Theorem 1.3 the inter-

polation problem

Li(f) = fUO (z0), i = ,1...,n

possesses a unique solution.

Example 1.3. Full Hermite Interpolation

Let X = .'N To avoid indexing we use the functional

information employed without using the symbol L as follows:

(me)0' ' (zn)

(m)f(z ),f'(z ),...,f (nn n (n

z. Z-,1N = m0 + ... +m +n).

If we can show the homogeneous problem has only the zero

solution, then the nonhomogeneous problem posseses a unique

solution.

If p E.N and satisfiesN (m0)

p(z0) = 0, p'(z0 ) =0,...,p (z0) = 0

(ml)p(z1 ) = 0, pt (z1) =0,... ,p (z1) = 0

0 1 (m n)p(zn) = 0, p'(zn) = 0,...,p n(zn) = 0

8

(mn)then p must vanish identically. Let p (zn) / 0 for the

time being. By the factorization theorem,

p(z) =A(z)(z-zo mO+1 (z-z )Ml+1... z-zn-1Mn-l+1 (z-z n~p) z)1= n

The degree of p(z) is N = m0 +m + .. + mn + n. Therefore,(in) 0 1' nM +1A is constant and p n(z) = A. mn,(z-zO)m0+ ..(z-z) )n-i)n n.zn- 0 n. -

Since the z are distinct, p(mn) (zn) = 0 which implies A = 0.

Thus p = 0. Hence the homogeneous interpolation problem

possesses only the zero solution.

Definition 1.2. A linear combination of 1, cos x,

cox 2x,...,cos nx, sin x, sin 2x,...,sin nx is known as a

trigonometric polynomical of degree n. The corresponding

linear space will be designated by Y . It has dimensionn2n+l.

Example 1.4. Let X = 6, L (f) f L( =

f(x),..,L2nf(x 2n) where -1 x0 <X1 <.'' <x 2 n <iT'

n nLet f(x) = a0 + Za cos jx+ 2Z a2j+1sin jx,

0 =1 4j j=

Lk k), k = 0,1,...,2n.

Let 11 cos x, sin x, cos 2x,, sin 2x,,

G = ILk ) I =

..cos nx0 sin nx0cos x sinx cos 2x sin 2x

...cos nx1 sin nx1

cos x 2n sin x 2n cos 2x2n sin 2x2n

...cos nx 2n sin nx2n

9

Multiply the 3rd, 5 ,..th columns by-i (iota) and add

them, respectively, to the 2nd 4th, columns. Then

ix. 2ix. nix.G = 1 e 3 sin x.e 3 sin 2x. ... e 3sin nx.j.

33 13

Multiply the 3 rd, 5th columns by -2i and to them add

the 2nd 4 th columns, respectively. Hence

ix. -ix. 2ix. -2ix. nix. -nix.(-2i)nG = 11 e 3 e e 3e ... e ie 31.

Interchange the columns,

(-l)n(n+l)(-2i)nG = le.nix. 1 enix

Mul tiply the j throw by enix , j = 0,1,. ..,2n to obtain

ni(x0+.. n+x2n)( n(n+l) ne (-1) (-2i) G

ix. 2nix

which is the Vandermonde. Henc

eni(x0+. x2n) (-1)

e,

n(n+l)n

2n ix. ixk= II (e - e k

j>kix. ixk

Since the x, are distinct, e 3 eik, for j / k. There-

fore, ILk.(xj) I = G / 0. We know the x. are distinct; there-

fore by Lemma 1.2 the Lk are independent and then by Theorem

1.3, the interpolation problem

Lk(f) f(xk) k = 0,1,...,2n,

possesses a unique solution.

We now proceed with how to represent a unique solution

guaranteed by Theorem 1.3. We will be dealing with the

10

Lagrange Formula and the Newton Formula. Let z 0 ',1,...,zn

be distinct points and define

n (z-z.)

lk(z) = 1 = ,,i=0 Tzk-z )i k

It is obvious that

1 k (z.= .=0Oif k j

lk(Z kif1 if k = j

For given values w0 ,w1 ,. .W ,wthe polynomial

Pn(Z) = J wklk(z) (1.6)k=0

is in andn

pn(z.) = w i = 0,l,...,n.

Formula (1.6) is called the Lagrange Interpolation Formula.

If we define w(z) = (z-zo)...(z-zn), then

1k(z) = wzz) (1.7)k ~(z-zk) wt( zk)

and (1.6) can be written as

n

pn (z) = w w(z(1.8)k=0 k (z-zk)w (zk)

If w = f(z.) for some function f, then

n WZpn(z) = f(zk) w(z) (1.9)

k=0 (~kw(k)

and pn (zk) = f(zk) k = 0,1,...,n.

Definition 1.3. We shall designate the unique poly-

nomial of class P that coincides with f at z0,.,.,zn byn0

Pn (f;z).

Suppose that q E -n . Then q is uniquely determined

by the n+1 value s q-(zi) , i =0 ,1,. . .,n. Hence we mus t have

11

Pn (q;z) E q(z). (1.10)

Note: nE 1k(Z) 1

k=0and (..

E (zk-z)lk(z) E 0, j = 1,2,..k=O

We know 1j(zk) = 6jk j,k = 0,1,...,n.

Definition 1.4. Let L1 ,...Ln be a set of functionals

and f1 ,'. fn be a set of polynomials such that L.(f.) = 6..

i,j = 1,...,n. Then the polynomials f. and the functionals

L. are said to be biorthonormal.

Let LO(f) = f(z 0), ... ,Ln(f) = f(zn). Obviously,

L.(l.) = 6... Therefore, the L. and the 1. are biorthonormal.1 3 13 1 1

We will state without proof the following theorem which says

that for a given set of independent functionals, we can

always find a related biorthonormal set of polynomials.

Theorem 1.5. Let X be a linear space of dimension n.

Let L1 ,L2 ,...Ln be n independent functionals in X*. Then

there are determined uniquely n independent elements of X,

say xx *a''' *, such that L(x6*) = .. For any x E X,

we haven

x = 2 L.(x)x.*.i=1

For every choice of wl,w2 ,...,wn the element

nx = 2w.x.*

i=l11

is the unique solution of the interpolation problem

L (x) = wi, i = 1,2,...,n.

12

The Lagrange formula has one drawback. If we want to

pass from a space of dimension n to a space of one higher

dimension, we must determine an entirely new set of ele-

ments y1*,y2 '*'' y 1 that are not related in a simple

fashion to the old set x*,x2..'n,x*. Newton's Represen-12 n

tation of the solution gets around this difficulty.

Let Z0 ,Z1 5,0...,zn be n+l distinct points and form

the n+l independent Newton polynomials 1,z-z0 ,(z-z0 )

(z-z ),...,(z-z )(z-zl)...(z-zn-1). For given values

w0 ,w 1,...,wn there is a unique member p of P for which1 n n

p(z.) = w i = 0,1,...,n (1.12)

and is given by

p(z) = a0 + a1(z-z0) + a2 (z-z0(z-z1 )

+ ... + an(z- z0 )(z-z 1 )...(z-zn- 1). (1.13)

If we solve (1.12) for ai's, we will get

a0 = w 0

w1- w 0 _ w 0 1a1 z -z -z -z + z -z (1.14)

10 0 1 1 0

1 w2 -w0 w1 -w 0

(z2 -z1 ) z2z0 Z1 Z0

w 0+w1

z0- 1 0)(z-z2) (zl-z0)(zl-z2 )

+ ()w 2

(z 2- z0)(z2-z 1)

13

Definition 1.5. The constant a. is called the dividedJ

difference of the jth order of w0 ,w1 ,...,w with respect

to z0 ,z1, ,...,z . It is designated by

a = [wol i' '...' ' j ]. (1.15)

On comparing the coefficient of zn in (1.8) and (1.13) we

getn

an = Ew0,...,wn kO E k (1.16)k=0 w'(zk)

i.e.,

n wan= w k

k=0 Z

and this gives the set defined in (1.14). Now, if we take

w to be some functionf at zy, i.e., f(z) = w i = 0,1,...,n

thenn

pn(f;z) = Z [f(zO),f(zI),...,f(zk)Jk=0

(z-z 0) ... (z-zk-1) (1.17)

and is called finite Newton series for f(z). If we define

LO(f) = f(z0)

f(z0) f(z1 )

L1 (f) %z-z1 + (z -z0 )

k f(z.)L (f)= ZLk .= k

j=0 (z.-z.)i=0 J

ifj

nthen, pnP(f; z) =2; Lk=f0 wk(z).

k=0

14

Now,

w.(z) EP if 0 j n -1,

and it follows that

Pn(wj(z);z) = w (z)

hencen

w.(z) = ZLk(w.(z))Wk(z). (1.19)J k=0

By putting j = 0,1,2,..., in (1.19), we get

Lk(w(z)) = k.

Therefore, the functionals L, L ,. . Ln,.. and the poly-

nomials w0 (z),w1 (z),...,wn(z),..., are biorthonormal.

In the Newton representation, we add an additional point

and increase the degree of the interpolating polynomial with

no extra work as in the case of Lagrange representation.

We say that the Newton representation has a permanence

property. We can achieve this type of biorthonormality and

permanence by the following theorem.

Theorem 1.6. Biorthonormality Theorem or Generalized

Newton Representation.

Let X be a linear space of infinite (countable) di-

mension. Let x1 ,x2 ,..., be a sequence of elements of X

such that for each n, x1,x 2''''n are independent. Suppose

further that L1,L2,..., is a sequence of linear functional

in X* such that for each n, the n xn determinant

ILi (xi)Ajn =0 0.Le ae i,j=0 o

Then, there are determined uniquely two triangular systems

15

of constants a..,b.. with a.. 0 (all i) such that if13 13 11

L* = allL1

L *= a L +a L2 21 1 22 2

L= a31L1 + a32 L2 +a 33 L3

.IL.* = Ea..L .

j=113 3

and

X.* =Xi1 1

x* b2 1x +x 2

x 3* b31x 1 +b 2x +x 3

i-1x.*=E b..x. + x.

1 3j=1 13 3 1

then we have L.*(x1) =S.. ij = 1,2,....1 3 13

Proof. We use mathematical induction. When n = 1, we

want to have L1*(x*) = 1. We get a 1L1(x1) = 1

a =f0.11 -L _____

Therefore, it is true for n = 1. Let it be true for n. We

will show that it is true for n+l. That will imply that it

is true for any n.

Now, since it is true for n, we have the following

constants.

a 11

a 2 1 a 22

anl an2 0 ann b bni n2 'n,n-1

wi th a1 1 *a 22...an/ 0 and such that

L *(x *= 6. i,j = 1,2,...,n.

We have to show that we can obtain first

bn+1, 1, bb+1,I 2, .. ,1n+l, nand from a knowledge of these values can then obtain

an].+ l11,0 . . .,a n+l, n+l

with an+ln+l / 0 and such that

.L *(x *= 6 i, j = 1,2,.. .,n,n+l.

The conditions not contained in (1.20) are

S n+ = 0

L*x1*)(

i = 1,2,...,n and

= 0 i = 1,2,...,n,

1.

Fro L.(x*) = 0, i =12From Li (xn+l) = i , ,...,n, we get

n+,l1*(x ) + ... + n+1,Ln 1 n 1 n+l

(1.22)

b L n(xi) + ... + b L *(x ) = -L *(xn+1,nn n n n+l

iLY*(xj) = ZaikLk(x) ,and using the fact Y

k=1

implies IY 1 =

(L *(x.))

IAI-B (where A,B,Y are matrices) and

i= aikLk j) we will eventually get the

k=1

following:

16

1

b 21

1

(1.20)

(1.21)

We know = A- B

17

L1l(xl) ... Li*(xn) a1 1 0 ... 0 L )...L n)

= a2 1 a2 2 ... 0

Ln ).Ln n) n 1)..n n

anl an 2 ... a

Hence, JL.*(x.) " = a1-a2 2 .. annIL. (x.) K._1 0.13 'i,3=1 1022 nfl 1 3 i1j-l

Now, L .* are independent and IL4*(x .) J1 . / 0. Thereforei 1 3 1,j=1

by Theorem 1.3 the system in (1.22) possesses a unique solu-

tion. Thus we know the b's and hence can determine the

xn+.Consider,1+1

L* (x*) = 0 i =l,2,...,nn+1

and

L* (x*) = 1.n+1 n

an+,iL 1 (x ) + .. + an+in+iLn+i (x = 0

n~ (x*)1 an+ln+l n+l~x

an L (x *) +...+ a L (x ) = 0 (1.23)n+1,1 I n n+1,n+1 n+1

an+ I, 1 n+l n+ln+1Ln+1 n+l

This system has a unique solution if

L (x J I'=, 0 or IL,.(x.i,'0.

Using the same arguments as above, we will eventually get

L *x-) .*. .L 1)L ..L (x+1)1 b21.bn+,1.1 Ln11 (x1 1(ixn+i) 1 - 9

0 1 ...b

L- .L* n+,2

n+l 1* ...Ln xL((xn+l) n+l 1)..(L+1n+1 .0 0...

i.e., 1 IL x Ijl Li-xj)ijl 1 0.

18

From (1.23), we have

an+l,n+1

0

1

L 3...n+l1 1

L (x* ) ...L (x* )1 n+1 n+l n+l

L (x * ).L (x* )

1 n+1 n+1 n+1

IL 3 x i 1

IL. (x.)jIn+11 3i,j=1

Since IL.(x.) ij=l 10and IL. (x.)ij= 0, then

an+1,n+l / 0. Hence it is true for n+1. Therefore, it is

true for any n.Q.E.D.

Definition 1.6. Let there be given a sequence of

values yOy 1 . . . . . . The difference of adjacent values is

designated by

Ayk Yk+l -Yk k = 0,1,...,

and is called the first order forward difference. Similarly,

A 2 AAkL ) =Ayk+1[ - Ayk

(yk+2 ~-Yk+l - (k+l Yk= Yk+2- 2 yk+l +yk

L (x* )o...L(x* )1n+1 n n+1

19

In general,

Yn1k = "( yk n 2k + 1 nyk

and we define Ayk = k'

We state without prdof the following theorems.

nTheorem 1.7. For n>-O , Anyk = n-r=nyk+r,=(n

r=0

n!

r!(n-r7!'

Theorem 1.8. In the case of interpolation at evenly

spaced points a,a+h,...,a+nh with values Y 0 'y ' ''in, then

the coefficients of interpolation polynomial are given by

kyo

ak -k k = 0,1,. .. ,n.k 1h

Theorem 1.9. Let p(z) be the unique polynomial of n

that takes on the values y0 'y1 '''.''n at the n+1 points

a,a+h,...,a+nh. Then

Ay0pn(z) = y0 + h (z-a) +

Any0+ -(z-a)(z-a-h)...(z-a-(n-l)h). (1.24)

n !h

If pn(f;z) interpolates to f at a,a+h,...,a+nh, then

pn(f;z) = f(a) hf a)(z-a) +n n

Anf(a)+ - (z-a)...(z-a-(n-1)h) (1.25)

where Af(a) = f(a+h) - f(a)

A2 f(a) = f(a+2h) - 2f(a+h) + f(a), etc.

Formula (1.24) and (1.25) are known as the Newton forward

difference formulas.

20

Note: If n = o, then the series on the right hand side

of (1.25) is called the Newton series for f.

Until now, we have been discussing the interpolation

process. Once the interpolation process has been carried

out, we would like to know how good are the approximations

that result. Remainder theory deals with this question and

we start with the Cauchy Remainder for Polynomial Inter-

polation.

Definition 1.7. The class of functions continuous on

[a,b] will be designated by C[a,b]. It is a linear space.

We state without proof the following theorem.

Theorem 1.10. Let f EC[a,b] and suppose that f(n+l)

exists at each point of (a,b). If as;x0 <XI '< ' <xn !b,

then

(x-x0)xx)f(x) - p (f;x) =- (n+l1) n.. (x-xQ f(n+l)(M

where min(x,x0 ... x '') < z<max(xx0 x) .The point

depends upon x,x0 '..'Ixn and f.

Corollary 1.11. Let Rn (f;x)=f(X) - Pn(fx), If

f ECn+l[a,bJ, we have

max jf(n+1) Ix-x0 1. ~x-xI1.. Ix-xnIIn fxl a !g t :g b (t)(+)

Definition 1.8. Let f be defined on [a,b] and assume

that at each point x0 E [a,bJ there is a power series ex-

pression of f valid in some interval such that

f(x) =a0 +a 1 (x-x0) + a2(x-xO 0) 2 + ...

21

Then f is said to be analytic on the inteval [a,b]. We

write f EA[a,bJ.

Definition 1.9. Let R be a region of the complex plane

and let f be a single valued function of the complex vari-

able z defined in R. If z0 ER, f is said to be analytic at

z0 (or regular at z0) if it has a representation of the form

C0

f(z)a= an(z-z) nn=0 n 0

A function is analytic (or regular) in R, if it is analytic

at each point of R. We write A(R) for the class of such

functions. A(R) is a linear space.

Theorem 1.12. Let R be a simply connected region and

let f EA(R) . Let z0 lie in R and suppose that C is a

simple, closed,rectifiable curve which goes around z0 in

the positive sense. Then

f(n) (z) n! jC f(z) dz.0 2ri C(z-z 0 n+1

We now prove a corollary to Theorem 1.10.

Corollary 1.13. Let f EA(R) where R is a region that

contains [a,b]. Let C be a closed curve that contains [a,bJ

in its interior and let L(C) be the length of C,

MC a zCf(z) I, 6 be the minimum distance from C to [a,b].

L(C)MThen, jRn(f X)If Pn(fX)IgLCn+ 01

(1.26)Proof. By Theorem 1.10,f(n+l)

JR (f*x) I=1f(WX>Pn(f;x) I jI M 10 1X-X0 ..- lx-xnJin n (1)!n (1.27)

where min(x, x 0 ' ' ''n ) < ( <max(x, x 0 ' ''n) By Theorem 1.12,

22

f(n+l) (n+1) f (z) dz2i c (z-)n+2

|f(n+1) )(n+l) !IS jf(z) dz, E E[a,b]2 IF c z-EjIf+ 2

maxCf(z) 2 L(n+)1

McL(C)n+2 -z(n+ 1).

From (1.27), we have

jf(x) -p (f;x)j M 2ML(C)2 n+1) 0 n I'n 2,F6n+2 T(n+1)!

Therefore,

M L(C)2f(x) - pn ( n+2 0 I nl.

Q.E.D.

Definition 1.10. (Triangular Interpolation Schemes)

Let there be given a triangular sequence of real or

complex points.

z 00

z 1 0 z11

T: z20 z21 z22

Suppose that. a function f has been defined on a region

containing the points of T, and let pn (f;z) be that element

of Y for whichn

pn(f;zni) = f(zni) i = 0,1,...,n; n = 0,1

In other words, pn (f;z) interpolates f at the points

of the (n+l)st row of T. The numbers in the rows of T may

23

or may not be distinct. This is an interpolation scheme of

great generality.

If we drop the row subscript, we get a scheme

z0

z 0 z

S: z0 z z 2

For a scheme of this type, we have

npn(f;z) = Z [f(zo),f(zl),...,f(zk)](z-zo)...(z-zk-l)'

nk=0

Note: The existence of lim pn (f;z) is identical with the

convergence of the interpolation series: symbolically,

f(z) ~ Z [f(zO), ... ,f(zk)](z-zO) ... (z-zk-1 'k=O

Definition 1.11. Let S be a set and (pnJ be a sequence

of functions defined on S. Let f be a function defined on

S. Now, either (a) if x ES and F->0, then there exists a

positive integer N such that if n>N, then

Jpn(x) - f (x) I < C.pn

or (b) if lim pn(x) f(x) for each x E S, then the sequence

{pn I is said to be convergent to f over S.

Definition 1.12. A sequence [pnI of functions defined

on S is said to be uniformly convergent to a function f over

S, if for each eF>0 there exists a positive integer N such

that if x E S and n >N, then

Ipn(x) - f(x) I <

24

Theorem 1.14. Let R, S, T be bounded and connected

subsets of IR. Let Rc Sc T and S is a proper subset of T.

Let A = maximum distance from any point in R to boundary of S.

6 = minimum distance from any point in R to boundary of T.

and assume that <1. Let the points of a triangular system

lie in R and let f be analytic in T. Then, pn (f;x) con-

verges to f uniformly in S.

Proof. Let

00

x1 0 x 1 1

X2 0 x 2 1 x 2 2

be a triangular system in RcT. Let C = T. Therefore,

L(T) = length of the interval T. Let MT = maxlf(x)I. LetxET

We can find a positive integer N such that

MTL(T) (A) N+l

2r6 6

Let xES and n>N. By Corollary 1.13,

MTL(T)f (x) - Pn (f2Xn I ':' n

2 6n+2

MTL(T) A n+l TL(T) N+l

Therefore, jf(x) - pnp(f;x) I <c. Hence pn(f;x) converges to

f uniformly in S. Q.E.D.

CHAPTER II

APPROXIMATION

In this chapter, we will start with uniform approxi-

mation and will discuss the best approximation and finally

the least squares approximation (a special case of the best

approximation).

The uniform approximation problem is as follows:

Let f be a continuous function defined on [a,bl; then for

a given s >0 can we find a polynomial p such that

f(x) - p(x)I<c, asx s b?

The answer to this question is yes and is assured by the

following theorem.

Theorem 2.1. (Weierstrass Approximation Theorem.)

Let f EC[a,bj. Given e>0, we can find a polynomial p for

which

If(x) -p(x)<, a:gx sb. (2.1)

Definition 2.1. Let f be defined on [0,1]. The nth

(n -1) Bernstein polynomial for f is given by

Bn kf(-)( )x (l n-k. (2.2)k=0

Note that Bn(f;0) = f(0), Bn(f;l) = f(l).

We state without proof the following:

Theorem 2.2. (Bernstein). Let f be bounded on [0,1].

25

26

Then

lim Bn(f x) f(x)

at any point at which f is continuous. If fECC[0,1], then

B n(f;x) converges uniformly to f in [0,13.

An immediate consequence of the Theorem 2.2 is the

following corollary.

Corollary 2.3. If f ECIO,l], then given an E >0 we

can find an integer N>0 such that for n>N we have

f(x) - Bn (f;x) I <6, 0 :!5 x :! 1.

By using the' transformation y awe can easily

transfer the result of Corollary 2.3 from [0,1] to [a,b]

and hence we will get Theorem 2.1.

By this method, we not only prove the existence of

polynomials of uniform approximation, but we get a simple

explicit representation for them.

In contrast to other modes of approximations, Bernstein

polynomials yield smooth approximants. If the approximated

function is differentiable, not only do we have Bn(f;x) -* f(x)

but

Bnp)(f;x) + f(P)(x) p = 1,2....

We state without proof the following.

Theorem 2.4. Let fECP[0,l]. Then lim B(P)(f;x) = f Ex)n~

uniformly on [0,13.

Note: The Bernstein approximants mimic the behavior of the

function to a remarkable degree. But there is a disadvantage

27

that the convergence of the Bernstein polynomial is slower

than other methods. This fact seems to have precluded any

numerical application of Bernstein polynomial from having

been made. Perhaps they will find application when the

properties of the approximant in the large are more impor-

tant than the closeness of the approximation.

Now we consider the following problem. If fE[a,b] and

x~x 2''4'9xnE[a,bJ and e>O, then we can find a polynomial such that

If(x) -p(x) I <., xE[a,b] and p(x ) = f(xi), i = 1,2,...,n?

The answer to this problem is yes, and we prove the

following theorem.

Theorem 2.5. Let f EC[a,bJ and xj,x 2',''xn be n

distinct points in [a,bJ. Then there is a polynomial p

which uniformly approximates f on [a,b] and satisfies the

auxiliary conditions

p(x1) = f(x ), i = l,2,...,n.

Proof. Recall

lk(x) w(xk (x-xk wEk)n n

where w(x) = k=l k Let kM = xEa,b k I. Let

c >0. Then -' = >0. There is a polynomial p, such that1+M1

f(x) - p,(x) j < ', x E [a,b-].

nLet q(x) = Z (f(xk) Pl(xk))lk(x). Therefore, q is the

k=l

unique element of "W-6 withn -1

q(xk) = f(xk) - pl(xk), k =1,2,...,,n.

28

Now,

nmx q() m XaxxE[a ,b] k If(xk) pl(Xk) xE[a b] Ilk(x) I

n<l'-kEmax llk(x) = -M.

k=l E k

Set p = pl+ q.

p(xk) = pl(Xk) + kq(x) = fk), k =,2,

and also

If(x) -p (x) I ! gf(x) -p1(x) I + q(x) I

< F_ + F_ 'M =_ '(1+1,4) = C

i.e., lf(x)-p(x)I <C.

Therefore, f is uniformly approximated by p on [a,b] and

f(xi) = p(xi) i = 1,2,...,n. Q.E.D.

Theorem 2.6. Every continuous function can be approxi-

mated uniformly on [a,b] by continuous piecewise linear

functions.

Proof. Let f be continuous on [a,bI. Therefore, f

is uniformly continuous on [a,b]. Let . >0. There is a

6 >0 such that if x,y E [a,b] and Jx-y 1 <6 ', then

If(x) - f(y) I <c.

Pick 6 >0Oso that 6 <7 and for some positive integer

n, n 6 = b - a. Divide [a,b] into n intervals so that the

i th interval I. = Uc i,c ], where c =,a+ iS i = 0,l,...,n.

Since f is continuous on [a,bI], it i continuous on

I c[a,b]. Hence f is bounded on Ii. Thus there exist m

and M. such that mi = f(t.) f(x) s f(hi) = M for xEI ,

where t,h EI .

29

Consider on the ijth interval the following linear

functionM. -M.

L(f;I. ;x) - '6 (x-c. 1 ) + m..

L(f;I.;x) is piecewise linear on [a,b]. Clearly,1

m. L(f;I.;x) M. for x EI., i = 1,2,,...,n. Let x E [a,b].1 1 1 1

Then for some j

-e <m.i- f(x) L(f;I.;x) - f(x) M. - f(x) <6.

Thus,

L(f;I;x) - f(x) I <E.

Hence, every continuous function can be approximated uni-

formly on [a,bj by continuous piecewise linear functions.

Q.E.D.

Now if we try to approximate f with some polynomials

of fixed degree, we have to define the closeness of approxi-

mation. Frequently we measure the closeness of approximation

over the interval by taking the maximum deviation between

the function and its approximant.

Let D be the criterion for closeness of approximation

and f be a function. If p E JP such that the closeness ofn

p to f is not exceeded by any other element of P n, then p

is known as a best approximation to f (under D) . If we

change the criterion of closeness of approximation, the best

approximation will change.

Note:: lxil will designate the norm of x.

Let X be a normed linear space and x1 ,...,xn be n

linearly independent elements of X. Let y be an element of

30

X. We define the closeness of two elements as the norm of

their difference. We say that the linear combination

alx1+ ... + ann X'of'x'xn is a best approximate to y if

we have

y - (a1x + .+..+anxjfj| |y-(b1 x1 + .+bn) (2.3)

for every choice of constants b ,..,b . The element

y - (a1x + ... + anxn) is called. the error and

y - (alx1 + ... + anxn)| the norm of the error.

The best approximation problem is: can we find

al,. ..,an such that the norm of the error is minimum?

We state without proof the following theorem.

Theorem 2.7. Given y and n linearly independent ele-

ments xi, .... ,xn. The problem of finding

min|y- (a1x1 + . .. + anxn)|I has a solution,where y anda.

x ,...,xn EX.

We state without proof some corollaries to Theorem 2.7.

Corollary 2.8. Let f ECa,b] and n be a fixed integer.

The problem of finding min max if(x)-(a0+a x+. . .+an)a. a5xgb

has a solution.

Corollary 2.9. Let f EC[a,bj and n be a fixed integer.

Let p :1. The problem of finding min a bIf(x)(a+ax+.. .+axn)IPdx. al1

has a solution. Such a solution yields a best approximation

to f in the sense of least pth powers. We need only assume

that f ELP[a,b].

31

Corollary 2.10. Let x0 'xl'''.'Xk be k+l distinct points

k n 2and k 2 n. The problem of finding mi n E(f(x.) - (a +. . .+a x

1 =

possesses a solution. (This is the common problem of least

squares data fitting by polynomials.)

Definition 2.2. A norm on a linear space X is called

strictly convex if |jx|Is r, |y|fJ r imply Jx+yjj<2r unless x=y.

We state without proof the following.

Theorem 2.11. In a normed linear space X with a

strictly convex norm, the problem of best approximation

(posed in Theorem 2.7) has a unique solution.

Theorem 2.12. (Tonelli) Let S be a closed and bounded

set in the complex plane that contains more than (n+l) points.

Let f be continuous on S and set

M = min maxlf(z) - p(z) . (2.4)pE.P zES

n

Let pn (z) be any polynomial that realizes this extreme

value and set r(z) = f(z) - pn(z). Then,

(a). The number of distinct points of S at which ir(z) I

takes on its maximum value is greater than n+l.

(b). There is a unique solution to the problem (2.4).

Remark. If S contains n+l points, then M.= 0 and (b)

holds but not (a). If S contains fewer than n+l points,

then M = 0 and the solution is not unique.

Let fEC[a,b. We know by Theorem 2.12 that the pro-

blem of finding min max Jf(x) - p(x) I has a unique solution.pE nagxsb

32

Designate the solution by pn(x) and set

En (f) a x b If(x) Pn(X)

We state the following theorem without proof.

Theorem 2.13. Let f EC[a,bJ and p En with

max jf(x) - p(x) I = 6 . Then p is a best uniform approxi-arx!bmant to f of degree n and 6 = En(f) iff there are at least

n+2 points a!x1<..,<x n+2 b such that f(x ) -p(x) = *

i = 1,2,...,n+2 in an alternating fashion.

Until now, we have been dealing with the best approxi-

mation by linear functions. Now, we will study the best

approximation by non-linear functions. The situation here

is more complicated than in the case of linear functions.

We will show a case of best approximation by non-linear

functions.

Theorem 2.14. Let fECLa,b] and let m and n be fixed

integers 20. The problem of finding

n n-la0xn+a1x +...+an

mmn max Jf (x) -0 1a.,b . axeb b0 xm+b xm-1+. ..+bnl

i'j 1 mhas a solution.

Proof. There is some redundancy in the coefficients

of the rational function. We can adjust them so that

b2 +2 2b0 + b2 + + bm = 1. As b's vary, we shall certainly

obtain some polynomials that do not vanish in [a,b].

If we seta xn +.+a

A= inf max If(x)- 0na.,b. axgb bOxm+...+bm

33

then 0 L\<oo. By the definition of L, we can find a se-

quence of rational functions

R Ak(X)k(X)T=- x

k Bkwhere

Akn. (k) n-iAk(x) Z a.i=O

and

Bk(x) = $ b~(k)m-j.

Bk j=O

so that if

= a bIf(x) - Rk(x) I (2.5)

then

lim A= .k+oo

The coefficients b k) are bounded due to the normalizing

condition. From (2.5)

- fK f(x) -Rk(x) <k'

Hence,

Rk(x)I k + max f(x)I M (2.6)

for some constant M. Therefore,

Ak(x) I MiBk (x) (2.7)

Since the b k) are bounded, the polynomial Bk

are bounded on La,b] and therefore Ak(x) are bounded. Now,

if a family of polynomials of bounded degrees are bounded,

then its coefficients are bounded. Therefore a k) are1

bounded.

34

(k)(k (k()Consider the points Pk = (a0 k .. n. ,ank),bk),.,bk

in the space Rm+n+2. They lie in a bounded portion of that

space. Hence, we may select a subsequence Pk that converges

to a point

P'= (as,.. .,a',bb,.. .,b').

Consider the rational functions corresponding to this

subsequence, and reindex the subsequence so that we have

lim a. k)= a! i = 0,1,...,nk-*oo11

(2.8)

lim bk) = b' j = 0,1...m

Form, n n-lafxn+a x +...+a '

R'(x) = 0 M-1 n (2.9)bxm+bxm+...+b'0 1m

If we can show that

max If(x) - R'(x)I = a(2.10)asxsb

then R' will be a best approximant and this will complete

the proof.

R', being rational, can have at most a finite number of

asymptotes. Let D(x) be the denomiator of R'(x). Select

an x E [a,b] such that D(x) 0. At such a point we must have

lim Rk(x) = R'(x).k+oo

R'(x) = f(x) + Rk(x) - f(x) + R'(x) - Rk(x)

IR'(x) I < If(x) + If (x) - Rk(x) I +.IR'(x) - Rk(x)

max If(x)I+ 'k + -k'anxsb

35

Let y = sup . As k +' -o,6k+ 0. Therefore,k

IR'(x) 1:5 max If(x)1+ p. (2.11)

The bound holds uniformly for any x E [a,bJ for which

D(x) 0. This implies that R'(x) cannot have any asymp-

totes on [a,b], because if it did, there would be values of

x in a neighborhood of the asymptotic point where the bound

would be exceeded.

Let x be any point of :[a,b].

(i) Suppose D(x) / 0. Then for k = 1,2,...,

If(x) - R'(x) 1 If(x) - Rk(x) 1+ IRk(x) - R'I(x) I

'k+ '

as k-+o, k + 0 and Ak + A. Thus

If(x) - R'(x) I sgA. (2.12)

(ii) Suppose D(x) = 0. Then, we may find a sequence of

points of [a,b],xI ,2..., such that x. + x, D(x.) 0. Then

by (2.12), we have

If(xi) -R'(x )I i = 1,2,...

By continuity, we get

f(x) -RI(x)1 A. (2.13)

Therefore, from (2.12) and (2.13) we have

f(x) .-R'(x)I A for x E Ea.b]

or

max If(x) - R'I(x) sAas x b

By the definition of A, max if(x) - R'(x) A. Henceasxsb

max If(x) -R'(x)I = A.asxgb Q.E.D.

36

J. C. Mason (Ref. Approximation Theory by A. Talbot)

has used the term "near-best approximation" to f by an

absolute distance or by a relative distance. He uses this

terminology to deal with those approximants which are not

best but are close to best. He uses two types of approxi-

mation methods in L1 , L2 , L norms. The two methods are

defined by the unique interpolation criteria and the unique

series expansion criteria. He has demonstrated that the

resulting approximations of explicit functions are either

best approximations or "near-best approximations" in the

relative norm.

We now turn to the most commonly used approximation

process. The least square approximation is the best approx-

imation in an inner product space.

We state without proof the following theorem.

Theorem 2.15. Let x1 ,x2 ,..., be a finite or infinite

sequence of elements such that any finite number of elements

xl,x2,.'xk are linearly independent. Then, we can find

constants

a 11

21 22

a31 a32 a33

such that the elements

x*=a x1 111X* a x + a x

2 211 +22 2x*= a3x + a3x + a x.3 31 1 32 2 33 3

37

are orthonormal

(x*,x* i,j = 1,2....

This result is very useful for orthonormalization.

Definition 2.3. Let x*,x*..., be a finite or infinite11' 2'

sequence of orthonormal elements. Let y be an arbitrary

element. The series E (y,xn )x* is the Fourier seriesn=l

for y. (If the sequence is finite we use finite sum). The

constants (y,x*) are known as the Fourier coefficients of y.

We write00

y ~E (y,x*)x* . (2.14)n=l n n

Note that the' symbol is used to indicate that the right-

hand side is associated in a formal way with the left-hand

side.

Note: If X is an inner product space, then |x|J = (x,x)

normalizes X.

If xx 2'''''In' 0 are orthogonal, but not

necessarily normalized, then

x* = k k =1,2,0...,Xk = xk1

are orthonormal so that (2.14) becomes

00 x xk (ylxk)y Z xk k___k=1 7'fIxk1 xk 11 k=l(xkxk) xk'

We state without proof the following theorem.

Theorem 2.16. Let x1 ,x2 '.'xn be independent and let

x * be the xi's orthonormalized. If w = a x1 +.. .+an n, then

38

nW =1 (w,X *)X *.

k=l

The least squares problem can be formulated in terms of

Nfinding min||y- Z a xll in an appropriate inner product

ai 1=lspace.

Truncated Fourier expansions have the following minimum

property. This helps in solving the least squares problem.

Theorem 2.17. Let x*,x be an orthonormal system1' 2

and let y be arbitrary. Then,

N N|fy - Z (y,x *)x *|l1:|fy - Z a xi* 11

i=1 i=l

for any selection of constants a1 ,a2 ,...,aN'

Proof. ConsiderN N N

|y - Za.X.*|11.i==i=

N N N= (yy) - a.(x.* ,y) - Ei.(y;x.*)+ Z a.Ta.(x*,x.*)

i= V i=1 i, j=1 13i

N N N= (yy) - Za.(x.*,y)- -d (y,x*) + L 1a. 2

i=l1 i=l1 i=lN N

+ E (x.*,y)(y,x.*) - 2 (x.*,y)(y,x.*)i=l i=l

N N

= (y,y) - iZlI1(yx*) 12+ Z (a-(y,xp)(dT- (x*,y))

= (y,y)- N1(yx*Z)1 2 + a -(y,x*)I 2.

Since the 'first two terms of the last line are indepen-

dent of the a's, it is clear that the minimum of

N 2fly- Z a x /| is achieved when and only when

i=1

39

a.= (y,x-*) i = 1,2,...,N

i.e., when the a's are the Fourier coefficients of y.

Q.E.D.

Corollary 2.18. Let x1 ,...,XN be an independent set of

elements. The problem of finding that linear combinationN

of x1 ,...,xN which minimizes ||y - Z a xi|1 is solved byi=l

NZ (y,x.* )X.*

i=l 1 Nminfy aNxyy 2Corollary 2.19. minly - Z ax. i=|y||1-l {yx ) I .

a. i=1i=1

Corollary _2.20. if x1',2* is a sequence of ortho-

normal elements, then00

(y ,x.* 1) 2 : l 12

i=1I

and

lim (y,x.*) = 0.i-+>

Corollary 2.21. Let x1,x2 '' '.'xn be independent. Let

x*,x2 9x' be the xk's orthonormalized according to the

triangular scheme of Theorem 2.15. Then, for all selections

of constants a1 ,...,an-1, we have

x,*Iln I1a n 11sa 1x1 + a2x2 + .. + an-Xn- + XnJ'nn

Theorem 2.22. Let xV,x2 '.. ' 'Xn be independent elements

and let x*,x2..'.,'x' * be the x.'s orthonormalized. Then, for1l 22 n 1

any element y,

n((y - (y,xk*)xk*),x.*) 0.

k=l k

The proof follows on the next page.

40

Proof.n n

(y - (y,xk*k)xk*'kk(Y'.)k=l k=1 k

= (y x*)- (y,x.*) = 0.

Q.E.D.

Corollary 2.23. Let y be arbitrary and x1 ,x2 '''',x'n

be linearly independent elements of X. Let

a x + a2x +.. + ax1 l 2 2 n n

be best approximation to y. Then the error,

y - (a1x 1 + a2 +.. + ann)

is orthogonal to xj..

CHAPTER III

HILBERT SPACE

In this chapter we will discuss orthogonal polynomials,

bounded linear functionals in Hilbert spaces and interpol-

ation and approximation in Hilbert space.

We start with the general properties of real ortho-

gonal polynomials.

We state without proof the following:

Theorem 3.1. Real orthonormal polynomials satisfy a

three term recurrence relationship.

P (anx + b -)p cnp

n = 2,3,.. . (3.1)

The following form is particularly convenient for machine

computation

P-1 = 0

PO = 1

(3.2)pn (x) = n n n(xpP)Pi*(x) - (p p 2p xpn+n)- (n'n n'n)Pnl-I

p*(x) = pn(x/(p )2 n = 0,1,2,...n Vn )~.n n

n = 0,1,2,...

Theorem 3.2. Let p*(x) = kn n + snxn-l + ... be ortho-n n nnormal polynomials. Then the coefficients in (3.1) are given

by

41

Th

are rea

Th

must co

Pr

42

a k Snb =a ( n _sn-1

n-l n n-ik k kkn-2 k n kn-2

cn an kn k2 n = 1,2,... (3.3)n-i

eorem 3.3. The zeros of real orthogonal polynomials

L, simple, and are located in the interior of [a,b].

eorem 3.4. Let fEC[a,b]; then the Fourier segmentn23,(f,p*)p*,(x)k=0 n n

incide with f in at least n+l points of (a,b).n

oof. Let Rn(x) = f(x) - Z (f,pj)p*(x). Now,n k=0

n(R (X) P*~(x)) =(f (X) Zff- P( *()

n k ~3=033kn

= (f,p*) - (f,p*)(p*,p*)j=0 3 3

= (f,p) - (f,p*) = 0.

Hence, (Rn,p) = 0 for k = 0,1,...,n. In particular,

(Rn(x),p*) = 0 = (Rn(x),l). Hence Rn(x) must vanish some-

where in (a,b). Suppose now that it vanishes at

a <x<... <X<b and at no other points of (a,b). Then,

R n(x) is of constant and alternating sign in the segments

(a x ),a~2 '''jb)

and this is true of the polynomial (x-xl) ...(x-x ). Thus,

the product Rn (X)(x-x...~(x-x) has constant sign in (a,b)

and (Rn (x)(x-x1)...(x-xj),) = (Rn (x),(x-x1)...(x-x))

cannot vanish. But by orthogonality it must vanish for

j : n. Hence j >n and the theorem follows.Q.E.D.

Now we turn to Hilbert space.

43

Definition 3.1. A complete inner product space will be

called a Hilbert space, H, if the following additional re-

quirements are fulfilled: (a) H is infinite dimensional;

that is, given any integer n, we can find n independent

elements, (b) There is a closed (or complete) sequence of

elements in H.

or

A complete inner product space will be called a Hilbert

space, if there is a complete orthonormal infinite sequence

of elements in H.

We state some examples of Hilbert space.

Example 3.1. The set of all infinite sequences [a.}1

for which

i=l

augmented by the usual definitions for addition and scalar

products and by

00 7(a,b) = Z a.~, a = [a.1J, b = [bi

i=l

as the definition of an inner product, constitutes a

Hilbert space.

It is called the sequential Hilbert space and is desig-

nated by 12

Example 3.2. L2 [a,bJ with inner product

(f,g) = a b f(x) g xdx,

is a Hilbert space.

Now we discuss bounded linear functionals in normed

linear spaces and Hilbert spaces.

44

Definition 3.2. Let L be a linear functional defined

over the elements of a normed linear space X. L is said to

be bounded if there exists a constant M such that

IL(x) I M|jxlI, for all x E X. (3.4)

If no such constant exists., the functional is called un-

bounded.

We state without proof the following.

Theorem 3.5. A linear functional L defined on a normed

linear space X is bounded if and only if it is continuous.

Definition 3.3. Let L be a bounded linear functional

defined on a normed linear space X. Then JILII is defined as

the minimum value M for which (3.4) holds. We have, ob-

viously

IL(x) I s ||L ||x|l, x E X (3.5)

and for every E >0, we can find an x0 E X for which

IL(x0) I > (|ILI|- c)|jx ff. (3.6)

An alternate formula for ||L|1 is given by

IL| =1 -sup jL(x) IxEX (3.7)

We state without proof the following.

Theorem 3.6. The set of all bounded linear functionals

defined over a normed linear space X is a linear space. The

quantity iL|| = up IL(x)I makes this linear space into a

normed linear space.

Theorem 3.7. If L is a bounded linear functional over

a complete inner product space X, then there exists a unique

45

element x0 EX such that L(x) = (x,x0), x E X, where x0 is

known as the representer of the linear functional L.

Corollary 3.8. Let L be a bounded linear functional

over a complete inner product space. Let x0 be its repre-

senter. Then

J|L 11 = |x 11

and JL(x0) I = |IL|I l|x0 11.

The representer of a functional has a simple formula

in terms of a complete orthonormal sequence.

Theorem 3.9. Let H be a Hilbert space and x*,x ,...,

be a complete orthonormal sequence of elements. If L is a

bounded linear functional on H, then L(x) = (x,y) where y

has the Fourier expression00

y ~ L(xj)x*. (3.8)k=1

Moreover,00

L(x) = Z (x,x*)L(x*) x E H (3.9k=1

and00

IL||2 Z L(x*)12(3.10)kk=1

Proof. Let y be the representer of L. Then

y ~ Z(y,xj)x* = Z (x,y)x = L(x*)x.k=1 k=1 k=1

We knowCO

(x,y) = (x,xj) (x,y)k=1

and00 00

L(x) = (x,y) = Z (x,x*) (xZ,y) = 2 (x,x)L(xf).k=1 k=1

46

2 2Also, |IL|| = I21

By Parseval's Identity, we have00 CO

|Y11 2 Z2I(yx*) 12 = Z J1L(x) 12k=1 k=1

orC0

|y||2 - L(x*) 12.k=l1

Q.E.D.

We state without proof the following.

Theorem 3.10. Let H be a Hilbert space and tx*J a com-n

plete orthonormal sequence. Let L be a linear functional

defined on H and suppose that for all x EH we have

00

L(x) = (x,xj)L(xj).k=l

Then L is bounded on H and

JIL||2 - Z IL(x*) 12

k= k

Theorem 3.11. Let H be a Hilbert space. Let H* be

its normed conjugate space. Then H* can be made into a

Hilbert space in such a way that H and H* are essentially

the same. More precisely, we can find a one to one corre-

spondence (++) between H and H* such that

(a) x1*-+Lx2 +-+L2 implies a1x1+a2 2+a1L1+a2L2'

(b) x++L implies JJxJ| = JLIJ.

(c) An inner product can be introduced in H* by writing

(L1 ,L2) = (xl'x2 ) where x*-+L1, x2++L2

(d) The norm arising from this inner product coincides

with the original norm in H* (i.e., JJLJ = suHlx).

(3.11)I

47

Proof. Let {x*J be a complete orthonormal system in

H. Let L EH*. By Theorem 3.7, we have L(y) = (y,w) for

a unique w EH and for all y EH. The quantities (w,xt) are

the Fourier coefficients of w and hence,

00

11 12 = 1 (w ,x ) 12 < co.

The quantities (x ,w) = w-,x0) satisfy the same inequality

SI(x*,w) 1 ci 1 '<, and hence by Theorem 8.9.1(F) and Theorem

8.9.2 (Refer Interpolation and Approximation by P. J. Davis,)

they are the Fourier coefficients of a unique element of H

which will be designated by W. Note ||W| = |lwil.

Make the correspondence L--++w. This correspondence is

one-to-one between the whole of H and the whole of H*, for

each L E H* determines w E H and each w determines a w.

If L1 - W1 and L 2 + w2 then L1 L2 implies w1 w21for

we can find an x E H such that L1 (x) L2 (x). Therefore,

0 / L (x) - L2(x) = (x,w1) - (x,w2 ). Hence w1 I w2. Now

w / w2 . For otherwise,

(w ,x*) = (w,xtY) i = 1,2....

Then,

(w1 ,'*) = ~ x*) i = 1,2,...,

implying w1 = w2 .

Conversely, let v EH. Consider V as above and define

L by means of L(x) = (x,-v). By the above, the element v

corresponds to L. But v = v. Thus v corresponds to some

L in H*.

48

(a) Let xi++L , x2 ++÷L 2 . Then,

L (x) = (x,x1 ), L2(x) = (x'X 2 )

so that

(a1L1+a2L2)(x) = aL(x)+a2L2(x)

= (x,a x, 2 2

Now

(aX1+ a2 2 1 1+a2 2'

Hence

(a1 L1+a2L2)(x) = (x,a x1+a2x2)

and therefore,

alL+ a2L2 -+ax +a2 2.

(b) If w++L then L(x) = (x,w); thus ILl = ||W| = jw|j.

(c) The inner product properties of (L1,L2) follow

from those i-n H.

(L1 +L2 ,L3) (x 1 +x 2 'X 3 ) = (xl'x 3 )+(x 2 ,x 3 )

- (L1,L3) + (L2 ,L3)'

(L1 ,L2) ' 2) =x 2,x1 2 = 2'L9

(aLPL2)= (ax ,x2) (xlx2) = a(L 1,L2).

(L1 ,L1) = (x1,x1) = 1fx 1112 > and = 0 if and only if x1 = 0,

hence if and only if L = 0.

(d) (LL)1 = (x,x)- = |1[x11 =11lx1,

since L(y) = (y,x), y EH, iLl = . Hence fLit = (LL) .

Thus, H* is an inner product space. To show complete-

ness we need to prove that iLmLn li , m,n >N implies the

existence of an L with iL-Ln|l+0*.

49

Let L -+--xn. Then flxn-xmII = L -L Lmfs efor m,n> N6.

Thus [xnI is a cauchy sequence in H. Hence there is an x

such that x-xnll+ 0. If x+-+L, then ||L-LnI1 = lxxn il*0.

Finally, there is a complete orthonormal sequence in H*,

for if {xj} is complete and orthonormal in H and x*++L*,k n n

then L*j} is complete and orthonormal in H*. H* is there-n

fore a Hilbert space.

Q.E.D.

Note: In virtue of (3.11), the spaces H and H* are known

as isomorphic and isometric.

We state without proof the following.

Theorem 3.12. Let H be a Hilbert space and {xfJ be

an orthonormal sequence that is not complete. Then we can

find a sequence of elements [y*} (finite or infinite) such

that {x*J and fyg} together form a complete orthonormal set.

Now we will deal with interpolation and approximation

in Hilbert space.

Definition 3.4. Let x 1 ,x 2 '.*''Xn be n independent

elements of a linear space. The set of all linear com-

binationsn

X0 + a x

i=l

is known as a linear variety of dimension n.

Definition 3.5. Let x,,x 2''''Xn be n independent

elements of an inner product space and let c,c2 ,. ..,cn be

n given constants. The set of elements y that simultaneously

satisfy the n equations

50

(y,x.) = C., i =

is known as a hyperplane of codimension n.

Note: In a linear space of finite dimension n, the concepts

of linear variety and hyperplane are equivalent.We state without proof the following.Theorem 3.13. Let X be an inner product space. Let

X1 ,X2 ''.'''Xn be n independent elements. Then given any set

of n constants cc 2 , . . . , cn, we can find an element y such

that

(y,x ) = cl, i = 1,2,...,n.

Theorem 3.14. Let [xjj be an infinite sequence of

independent elements of a Hilbert space H and let constants

{c } be given. A necessary and sufficient condition that

there exists an element y EH such that

(y,x.) = c., i = 1,..., (3.12)

is that00

ZIak 2 <00 (3.13)k=l

where a1 = c / gXj),

(x1,x1) ... (xxl)

an -0(3.14)

g(xl1,-..,x n- 1)g8 l'''''n (x xn ) ... (XnX -)

n>l 0i... dn

If there is a solution, it is unique if and only if [x) is

complete (or closed) in H.

Let X be an inner product space and xl, ... xn be n

independent elements. Let S be the linear subspace spanned

51

by the x's. Let x* be the orthonormalized xi. Take an11

element z EX which is not orthogonal to all the x*'s, and

let V consist of all elements y ES for which (y,z) = 1.

Now, can we find a y EV such that ||y|| = minimum?} (3.15)

Theorem 3.15. The unique solution to the problem (3.15)

of finding a y such that ||yf| = minimum is given by

n ny = (z,xt)x* I(z,x*) 12. (3.16)

i=l =

The minimal distance is given by

Y12 n 1(3.17)

E J(z,x*) 2

i=l

Proof. Let y ES. Therefore, y can be expressed as

linear combination of x's.1

y = a x* + + ax*'11 n n

We know (y, z) = 1 where y E S and z E X and

a(x*,z) + ... + an(x*,z) = (y,z) = 1.1 1 n n

n (z,x*)

Set Z I(z,x*)12 =s 0, and write a = s + b where

i=l

the b. are now to. be determined. Now,1

n

1 = (y,z) = 1 + E b.(x*,z),i=l1

so thatnZ b.(x*,z) = 0.i=l 1 1

But

52

n n (z,x*) (x*,z)||y||2 2 I' 1 + b i)( + 1)

sn 1in n 2= + 2 b.(x*, z) +- Z B.(z,x*) + ZJb. 12

S2 s si=ls i l

1 n 2- + Ebi.i=1

The selection leading to the minimum |IIy| 2 is uniquely given

1by b. -0 and the minimum value is -. Now,

1s

n n (z,x*) n n 2y= a.x = - x*1 = (z,x*)x*1I(z,x*)1.

Q.E.D.

We state without proof the following.

Theorem 3.16. Let H be a Hilbert space. Let

x*,x*, .. ,x* be n independent elements of H. Then,1' 2' n

nw = 2 a.xt

i=l1

solves the problem

min||w|!w

subject to (w,xt) = a , i = 1,2,...,n. Moreover,

2 n 1min||w||2= E Ja2w i1l

BIBLIOGRAPHY

Books

Boas, Mary L., Mathematical Methods in the Physical Sciences,New York, John Wiley and Sons, Inc., 1966.

Cheney, E. W., Introduction to Approximation Theory, NewYork, McGraw-Hill BookCompany, Inc., 1966.

Churchill, Ruel V., Introduction to Complex Variables andApplications, New York,7McGraw-Hill Book Company,Inc., 1948.

Davis, Philip J., Interpolation and Approximation, NewYork, Blaisdell Publishing Company, 1963.

Articles

Mason, J. C., "Orthogonal Polynomial Approximation Methodsin Numerical Analysis," Approximation Theory, (byA. Talbot), New York, Academic Press, Inc., 1970.

Pruess, Steven, "The Approximation of Linear Functionalsand h2 -Extrapolation, SIAM Review, XVII (1975) .

53