orthogonal polynomials, quadratures & sparse-grid methods ... · orthogonal polynomials,...

A Tutorial on Sparse Grid Integration, May 2010

Orthogonal Polynomials, Quadratures& Sparse-Grid Methods for Probability

Integrals

Dr. Abebe GeletuMay, 2010

Technische Universität Ilmenau,Institut für Automatisierungs- und SystemtechnikFachgebiet Simulation und Optimale Prozesse

1/31


Topics

Interpolatory Quadrature Rules

Orthogonal Polynomials

Embedded Quadrature Rules

Quadrature Rules and Probability Integrals

Full-Grid Tensor Product Multidimensional Integration

Sparse-Grid Tensor Product Multidimensional Integration

Part - I: Quadrature Rules;

Part - II: Sparse-Grid Integration Methods

2/31


1. Interpolatory Quadrature Rules

For an integrable function f : ℝ → ℝ to compute the integral

I[f ] =∫ b

af (x)�(x)dx ,

where

- [a,b] ⊂ ℝ is a bounde or unbounde interval;

- � : ℝ → ℝ is a nonnegative weight function.

Sometimes the weight function need to be adjusted toconform to the the properties of the integrand f , e.g.oscillatory behaviors, etc.; and the interval of integrationΩ1.

3/31


1.2. Some integrals with standard weight functionsand intervals of integration

(a) I1[f ] =∫ 1

−1f (x)dx ⇒ �(x) = 1; Ω1 = [−1,1]

(b) I2[f ] =∫ 1

−1f (x)(1 − x)�(1 + x)�dx

⇒ �(x) = (1 − x)�(1 + x)� , �, � > −1;

Ω1 = [−1,1]

(c) I3[f ] =∫ 1

−1f (x)

(

1 − x2)− 1

2 dx

⇒ �(x) =(

1 − x2)− 1

2; Ω1 = [−1,1]

4/31


(d) I4[f ] =∫ +∞

0f (x)x�e−xdx

⇒ �(x) = x�e−x , � > −1; Ω1 = [0,+∞)

(e) I5[f ] =∫ +∞

−∞f (x)e−x2

dx

⇒ �(x) = e−x2; Ω1 = [0,+∞)

Note:

transform other types of integrals to the standard ones.

Example:

∫ 1

−1sin(x)e−x2

dx −→∫ 1

−1

(

sin(x)e−x2√

1 − x2)

︸︷︷︸

=f (x)

(

1 − x2)− 1

2 dx

5/31


1.3. Interpolatory Quadrature Formulas

To construct an N-point quadrature formula (rule):

Q1N [f ] :=

N∑

k=1

wk f (xk ),

where the integration nodes X = {x1, x2, . . . , xN} and weights{w1,w2, . . .wN} are constructed based on Ω1 = [a,b] and theweight function �.

Question:How to determine the 2N unknowns x1, . . . , xN andw1, . . . ,wN?

Requirement: the weights w1,w2, . . . ,wN should benon-negative to avoid numerical cancelations.

6/31


1.4. Efficient interpolatory quadrature rules

1 Gauss qudrature rules and their extensions2 Curtis-Clenshaw quadrature rules

7/31


1.5. Polynomial Exactness

One measure of quality for a quadrature rule is its polynomialexactness.

The N-point quadrature Q1N [⋅] is said to be exact for a

polynomial pm of degree m if

I[pm] =

∫ b

apm(x)�(x)dx = Q1

N [pm] =N∑

k=1

wkpm(xk )

the degree of exactness or degree of accuracy d of anN-point quadrature rule is equal to the maximum degreepolynomial for which Q1

N is exact.

8/31


1.6.Gauss Quadrature Rules and OrthogonalPolynomials

An N-point Gauss quadrature rule is constructed toachieve the largest possible polynomial exactness.

Theorem (see Davis & Rabinowitz [1])

An N-point quadrature formula Q1N has degree of exactness

d = 2N − 1 if the integration nodes x1, x2, . . . , xN are zeros ofthe N−th degree orthogonal polynomial pN(x) w.r.t. � andΩ1 = [a,b].

Hence, there is a polynomial pm with degree m > 2N − 1,such that I[pm] ∕= Q1

N [pm].

Example:a) A 3-point Gauss quadrature rule Q1

3 has apolynomial exactness d = 2 × 3 − 1 = 5

9/31


Orthogonal Polynomials

Two polynomials pn and pm are orthogonal w.r.t. � andΩ1 = [a,b] if

< pn,pm >:=

∫ b

apn(x)pm(x)�(x)dx = 0.

In Theorem 1, the N−th degree orthogonal polynomial pN

is orthogonal to all polynomials pm degree m ≤ N − 1.

Examples:

(a) p0 = 1, p1(x) = x are orthogonal w.r.t.�(x) = 1 and Ω1 = [−1,1].

(b) H1 = 2x , H2(x) = 4x2 − 2 are orthogonal w.r.t.�(x) = e−x2

and Ω1 = (−∞,+∞).

10/31


Characterization of Orthogonal Polynomials

Theorem (Recurrence relationship, see Gatushi [2])

Suppose that p0 = 1,p1, . . . are orthogonal polynomials withdeg(pn) = n and leading coefficient equal to 1. For everyinteger n

pn+1(x) = (x − an)pn(x)− bnpn−1(x),n = 1,2 . . . (★)

where

an =

∫ ba xp2

n(x)�(x)dx∫ b

a p2n(x)�(x)dx

,n = 1, . . . ;

bn =

∫ ba xp2

n(x)�(x)dx∫ b

a p2n−1(x)�(x)dx

,n = 1, . . .

11/31


(Continued...) Characterization of OrthogonalPolynomials

Theorem (Continued...Recurrence relation)

and

p1(x) = (x − a0)p0(x) = x − a0,

a0 =

∫ ba xp2

0(x)�(x)dx∫ b

a p20(x)�(x)dx

=

∫ ba x�(x)dx∫ b

a �(x)dx.

Remark:

Given a nonnegative weight function � and a set Ω, youcan construct your own set of orthogonal polynomials ifyou can efficiently compute the recurrence coefficientsa0,an,bn,n = 1,2, . . ..

12/31


1.7. Algorithms to determine the recurrencecoefficients

Algorithms for the computation of the recurrencecoefficients are commonly known as Steleje’sProcedures.

Currently an efficient and stable algorithm for thecomputation of the coefficients a0,an,bn,n = 1,2, . . . isgiven by Gander & Karp [5].

Software:

FORTRAN: ORTHOPOL by Gatushi [4]

C++ implmentation of ORTHOPOL Fernandes & Atchley[4].

Various Matlab codes by Gatushi [1]

13/31


Some known sets of orthogonal polynomials

Jacobi polynomials orthogonal w.r.t.�(x) = (1 − x)�(1 + x)� , �, � > −1 and Ω1 = [−1,1]:

p(�,�)n (x) =

12n

n∑

k=0

(n + �

k

)(n + �n − k

)

(x − 1)n−k (1 + x)k ,

n = 0,1,2, . . .

Lagendre polynomials are the Jacobi polynomials for� = � = 0; i.e. �(x) = 1:

Ln(x) =12n

n∑

k=0

(nk

)(n

n − k

)

(x − 1)n−k (1 + x)k ,

n = 0,1,2, . . .

14/31


Hermite Polynomials w.r.t. � = e−x2and Ω1 = (−∞,+∞)

Hn(x) = n![n/2]∑

k=1

(−1)k

k!(n − 2k)!(2x)n−2k ,n = 0,1,2, . . .

The first 6 terms of Hermite polynomials:

H0(x) = 1, H1(x) = 2x , H2(x) = 4x2 − 2

H3(x) = 8x3 − 12x , H4(x) = 16x4 − 48x2 + 12,

H5(x) = 32x5 − 160x3 + 120x

H6(x) = 64x6 − 480x4 + 720x2 − 120,etc.

15/31


Chebychev Polynomials orthogonal w.r.t.�(x) = (1 − x2)−

12 and Ω1 = [−1,1]

C0(x) =1√�

T0(x),Cn(x) =

√

2�

Tn(x),n = 1,2, . . .

where Tn(x) = cos(n acrcos(x)), known as Chebychevpolynomials of first kind. Few terms are given by

T0(x) = 1, T1(x) = x , T2(x) = 2x2 − 1,

T3(x) = 4x3 − 3x , T4(x) = 8x4 − 8x2 + 1

T5(x) = 16x5− = 8x4 − 20x3 + 5x ,

T6(x) = 32x6 − 48x4 + 18x2 − 1

Corresponding to each set of orthogonal polynomials wehave the quadrature rules: Gauss-Jacobi,Gauss-Legendre, Gauss-Hermite, Gauss-Chebychev, etc.

16/31


1.8. Computation of quadrature nodes and weights

Theorem (see Gatushi [2, 3])

The quadrature nodes x1, . . . , xN and weights w1,w2, . . . ,wN

can be obtained from the spectral factorization

Jn = V⊤ΛV ; Λ = diag(�1, �2, . . . , �N), VV⊤ = IN ;

of the symmetric matrix tridiagonal Jacobi matrix

Jn =

⎡

⎢⎢⎢⎢⎢⎢⎣

a0√

b1√b1 a1

√b2

√b2

. . . . . .

. . . aN−2√

bN − 1√bN − 1 aN−1

⎤

⎥⎥⎥⎥⎥⎥⎦

,

17/31


Theorem (continued...)

where ak ,bk , k = 1,2, . . . ,N − 1 are known coefficients fromthe recurrence relation (★). In particular

xk = �k , k = 1,2, . . . ,N;

wk =(

e⊤1 Vek

)2, k = 1,2, . . . ,N.

Observe that: all the weights wk obtained above arenonnegative.

Exercise:Write a Matlab code not longer than 10 lines to compute x ′

k sand w ′

ks.

18/31


1.9. Gauss Quadrature Rules with Preassigned nodes

In some application integration nodes need to be prefixed(pre-given), eg. due to boundary conditions, constraints,etc.

Such nodes are usually needed in collocation methods inthe solution of ODEs and PDEs.

Objective: To construct an N−point quadrature rule Q1N with

x1, . . . , xm,m ≤ N, are preassigned (fixed) nodes in [a,b]:

determine the remaining N − m nodes ;

correspoindg weights w1,w2, . . . ,wN

so that Q1N has the maximum possible degree of exactness d .

19/31


Classical Gauss quadrature rules with preassignednodes

In a closed bounded interval [a,b]:1 Gauss-Radau rule:

- One end point of [a,b] is prefixed to be a node; i.e. eitherx1 = a or xN = b; the rest of the nodesx2, x3, . . . , xN ∈ (a,b) .- Degree of polynomial exactness d = 2N − 2.

2 Gauss-Lobatto rule:- Both end points of the interval [a,b] are prefixed to benodes; i.e. x1 = a and xN = b. There restx2, . . . , xN ∈ (a,n).- Degree of polynomial exactness d = 2N − 3.

In general, prefixing nodes reduces the degree ofpolynomial exactness by the number of integration nodesprefixed.

20/31


Recently, Bultheel et. al. [2] give Gauss qudrature ruleswith prefixed nodes and positive weights in any intervalΩ1 ⊂ ℝ bounded or unbounded.

Software:

See Gatushi [1] for Matlab codes.

21/31


1.10 How good are Gauss Quadrature Rules?

Disadvantage:

Different Gauss quadrature rules Q1N1

and Q1N2

have only afew common nodes, for N1 < N2; i.e. the set of nodesX1 = {x1, x2, . . . , xN1

} ⊂ [a,b] andX2 = {z1, z2, . . . , zN2

} ⊂ [a,b].

=⇒ it is difficult to estimate the convergence of the limit:

limN−→∞

∣∣∣I[f ]− Q1

N [f ]∣∣∣ .

Note that: had it been X1 ⊂ X2, we could have obtained∣∣∣I[f ]− Q1

N2[f ]∣∣∣ ≤

∣∣∣I[f ]− Q1

N1[f ]∣∣∣ .

Solution: Either extended Gauss quadrature rules so that thenodes for a lower accuracy can be reused to constructquadrature rules of higher accuracy; or construct embeddedquadrature rules from the start.

22/31


1.11. Embedded quadrature rules - Extensions ofGauss quadrature rules

A) Kronord’s extension [6]: Given Gaussquadrature nodes x1, x2, . . . , xN , between everytwo nodes add one new node:

z1 ∈ (a, x1), z2 ∈ (x1, x2), . . . zN ∈(xN−1, xn), zn+1 ∈ (xN ,b) so that theN + N + 1 = 2N + 1 pointsz1, x1, z2, x2, . . . , xN−1, zN , xN , zN+1 are thenodes of the new quadrature rules; andand the new weights w1,w2, . . . ,w2N+1 are allnonnegative.

=⇒ The degree to exactness of a Gauss-Kronordrule is:

d =

{3N + 1, if N is even3N + 2, if N is odd

Hence, XN ⊂ X2N+1. 23/31


B) Patterson’s extension [2]:To an existing N Gauss quadrature nodesx1, x2, . . . , xN , add m new integration nodesz1, z2, . . . , zm so the new quadrature rule has themaximum degree of accuracy

d = 2(m + N)− N − 1 = N + 2m − 1

and the weights w1,w2, . . . ,wN+m arenonnegative.Hence, XN ⊂ XN+m.

Note:In general, for a given weight function � and integrationdomain [a,b], the construction of convenient embeddedGauss-quadrature rules is not a trivial task.(see, [3, 3, 5])

Software:FORTRAN, C++, Matlab codes in the book ofKythe & Schäferkotter [1].

24/31


Embedded quadrature rules- the Curtis-Clenshawquadrature rule

For integrals on [−1,1] (in general on a closed and boundedinterval [a,b]) the set of nodes

XN =

{

xk ∣ xk = cos(k − 1)�

N − 1, k = 1,2, . . . ,N

}

and the weights are computed from the orthogonal Chebychevpolynomials of first type Tn(x) = cos(n acrcos(x)) usingTheorem 4.Properties:

degree accuracy N − 1;XN ⊂ X2N−1;nodes and weight are very simple to compute; (seeTrefethen [3],Waldvogel[4] for Matlab codes)despite lower polynomial exactness, comparable efficiencywith Gauss quadrature rules ( Trefethen [3]);extensive applications in spectral and pseudo-spectral 25/31


1.12. Quadrature Rules and Probability Integrals

For a well behaved nonnegative function �(x) , if∫ ∞

−∞�(x)dx = 1,

then the expression �(x) = �(x)dx defines a probabilitymeasure. And � is termed a probability density function ofx . In particular, for a random set A

Pr{x ∈ A} = �(x ∈ A) =∫ ∞

−∞1A(x)�(x)dx ,

where

1A(x) ={

1, if x ∈ A0, if x /∈ A.

For function f of the random variable x , its expected value is

E [f ] =∫ ∞

−∞f (x)�(x)dx .

26/31


Standard probability density functions

Given a nonnegative weight function �(x) with the property

�(x) ={

> 0, for x ∈ [a,b]0, if x /∈ [a,b].

such that �0 =∫ b

a �(x)dx set �(x) := �(x)�0

, then the expression�(x) = �(x)dx defines a probability measure.Example:

For �(x) = 1 on [a,b]:

�0 = b−a ⇒ �(x) =1�0

=1

b − a→ the uniform density function.

27/31


For �(x) = e−x2on (−∞,+∞):

�0 =

∫ +∞

−∞e−x2

=√� ⇒ �(x) =

�(x)�0

=e−x2

√�

.

Hence,

F (x) =∫ x

∞

e−z2

√�

dz =

∫ x

∞

e− 12 (√

2z)2

√2√�

√2dz

Setting u =√

2z we have du =√

2dz and

F (x) =∫ x

∞

e−z2

√�

dz =

∫ x

∞

e− 12 u2

√2�

du.

⇒ F defines the standard normal distribution.

28/31


Literature

Davis, P. J.; Rabinowitz, P. Methods of numericalintegration. Dover Publications, 2nd ed., 2007.

Bultheel, A.; Cruz-Barroso, R.; Van Barel, M. OnGauss-type quadrature formulas with prescribed nodesanywhere on the real line. Calcolo, 47(2010) 21–48.

Elhay, S.; Kautsky J. Generalized Kronrod Patterson typeembedded quadratures. Aplikace Matematiky, 37(1992)81–103.

Fernandes, A. D.; Atchley, W. R. Gaussian quadratureformulae for arbitrary positive measure. EvolutionaryBioinformatics, 2(2006) 261 – 269.

Gander, M. J.; Karp, A. H. Stable computation of high orderGauss quadrature rules using discretization for measuresin radiation transfer. J. of Molecular Evolution, 53(4-5):47.

29/31


Gatushi, W. Orthogonal polynomials (in Matlab). J. Comp.Appl. Math. 178(2005) 215–234.

Gatushi, W. Orthogonal polynomials: computation andapproximation. Oxford University Press, New York, 2004.

Gatushi, W. Orthogonal polynomials and quadrature.Electronic Trans. of Num. Math. 9(1999) 65 – 76.

Gatushi, W. Algorithm 726: ORTHOPOL - A package ofroutines for generating orhtogonal polynomials andGauss-type quadrature rules. ACM Trans. on Math.Software, 20(1994) 21 – 62.

Laurie, D. P. Calculation of Gauss-Kronord quadraturerules. Math. Comp. 66(1997) 1133 – 1145.

Kronord, A. S. Nodes and weights of quadrature formulas.Consultants Bureau, New York, 1965.

30/31


Kythe, P. K.; Schäferkotter, M. R. Handbook ofcomputational methods for integration. Chapman &Hall/CRC, 2005.

Patterson, T. N. L. The optimum addition of points toquadrature formulae. Math. Comp. 22(1968) 847 – 856.Errata: Math. Comp. 23(1969) 892.

Trefethen, L. N. Is Gauss Better than Clenshaw-Curtis?SIAM Review 50(2008) 67 – 87.

Waldvogel, J. Fast construction of the Fejer andClenshaw-Curtis quadrature rules. BIT NumericalMathematics 43(2004) 1 – 18.

31/31

orthogonal polynomials, quadratures & sparse-grid methods ... · orthogonal polynomials,...

Documents