theconvergenceofspectralandfinitedifference ... · theconvergenceofspectralandfinitedifference...

THE CONVERGENCE OF SPECTRAL AND FINITE DIFFERENCEMETHODS FOR INITIAL-BOUNDARY VALUE PROBLEMS∗

NATASHA FLYER†‡ AND PAUL N. SWARZTRAUBER†

SIAM J. SCI. COMPUT. c© 2002 Society for Industrial and Applied MathematicsVol. 23, No. 5, pp. 1731–1751

Abstract. The general theory of compatibility conditions for the differentiability of solutions toinitial-boundary value problems is well known. This paper introduces the application of that theoryto numerical solutions of partial differential equations and its ramifications on the performance ofhigh-order methods. Explicit application of boundary conditions (BCs) that are independent ofthe initial condition (IC) results in the compatibility conditions not being satisfied. Since this isthe case in most science and engineering applications, it is shown that not only does the error in aspectral method, as measured in the maximum norm, converge algebraically, but the accuracy of finitedifferences is also reduced. For the heat equation with a parabolic IC and Dirichlet BCs, we provethat the Fourier method converges quadratically in the neighborhood of t = 0 and the boundariesand quartically for large t when the first-order compatibility conditions are violated. For the sameproblem, the Chebyshev method initially yields quartic convergence and exponential convergence fort > 0. In contrast, the wave equation subject to the same conditions results in inferior convergencerates with all spectral methods yielding quadratic convergence for all t. These results naturally directattention to finite difference methods that are also algebraically convergent. In the case of the waveequation, we prove that a second-order finite difference method is reduced to 4/3-order convergenceand numerically show that a fourth-order finite difference scheme is apparently reduced to 3/2-order.Finally, for the wave equation subject to general ICs and zero BCs, we give a conjecture on the errorfor a second-order finite difference scheme, showing that an O(N−2 logN) convergence is possible.

Key words. compatibility conditions, convergence theory, spectral and finite difference methods

AMS subject classifications. 65M06, 65M12, 65M70

PII. S1064827500374169

1. Introduction. The regularity of solutions for initial-boundary value prob-lems (IBVPs) is determined by the compatibility of the initial condition (IC) and theboundary conditions (BCs). IBVPs will have singular solutions at the corners of thespace-time domain if the BCs do not satisfy the partial differential equation (PDE)or any of its higher-order derivatives, even if the IC is C∞. The Cauchy–Kowaleskytheorem [19] provides a solution in the neighborhood of t = 0 to the same problemwithout BCs. If compatibility exists, then the BCs can always be computed by thesolution obtained from the Cauchy–Kowalesky theorem and the IC. In such cases, theIBVP could in fact be posed as an initial-value problem (IVP). The set of compati-bility conditions are derived by equating the time derivatives of the BCs to those ofthe solution to the IVP given by the Cauchy–Kowalesky theorem.

Literature on the theory of compatibility conditions and the regularity of solu-tions for IBVPs is extensive, starting in the 1950s with the work of Ladyzenskaja [10],[11]. In the 1960s, Ladyzenskaja, Solonnikov, and Ural’ceva [12] and Friedman [3]discussed the regularity of the solution for linear and quasi-linear parabolic systems.During this same time, Kreiss [4] and Hersh [7] extended the theory for constant co-efficient hyperbolic systems by providing the necessary and sufficient conditions for

∗Received by the editors June 20, 2000; accepted for publication (in revised form) August 6, 2001;published electronically January 30, 2002.

http://www.siam.org/journals/sisc/23-5/37416.html†National Center for Atmospheric Research, Advanced Study Program, P.O. Box 3000, Boulder,

CO 80307-3000 ([email protected]). NCAR is sponsored by the National Science Foundation.‡Current address: Department of Applied Mathematics, University of Colorado, Boulder, CO

80309 ([email protected]).

1731

1732 NATASHA FLYER AND PAUL N. SWARZTRAUBER

the problem to be well-posed. It was further developed in the 1970s by Rauch andMassey [16] and Sakamoto [17] for hyperbolic systems with time-dependent coeffi-cients. In the 1980s, Temam [20] extended the theory by obtaining the compatibilityconditions for semilinear evolution equations while Smale [18] gave a rigorous proofof the compatibility conditions for the heat and wave equation on a bounded domain.These accomplishments have been confined to the realm of theoretical mathematics.Yet, not satisfying the compatibility conditions of a system can have a significantcomputational impact, particularly for high-order accurate methods, such as spectralmethods, whose performance intrinsically depends on the smoothness of the solution.For nonsmooth initial data, there is a wealth of information on the convergence theoryof numerical schemes in a variety of norms for parabolic and hyperbolic IVPs [13],[5], [21], [22]. These analyses can be applied to the IBVPs if the IBVP can be posedas an IVP with discontinuous ICs through periodic extensions of the ICs to the entirereal axis [6], [9], [14]. However, such periodic extensions may induce singularities thatdo not exist in a smooth nonperiodic solution of an IBVP.

More recently, the impact of noncompatible BCs has been explored by Boyd andFlyer [2] in a computational framework. This paper appeared in a special issue onspectral methods. In the prologue, Karniadakis states “This is the first such effort inbringing the theory of compatibility conditions from the mathematical literature to thenumerical community.” This work illustrated the ubiquity of compatibility conditionsand analyzed the connection between incompatibility and the rate of convergence ofChebyshev spectral series. However, the convergence of the error was not discussed orcompared to alternatives such as the finite difference method. The focus of the paper,as has been in the mathematical literature, is on smoothing the initial condition soas to satisfy the compatibility conditions [2], [5], [18], [15]. The difficulty with thisapproach is that it leaves the fundamental question unanswered: namely, “What isthe rate of convergence to the solution of the original unperturbed problem?” If theIC is smoothed, spectral convergence might be achieved but to a solution that differsalgebraically from the original problem.

The focus of the current work is on the convergence of the approximate solutionto the exact solution. As discussed in the first paragraph, the temporal derivativesof the solution as defined by the IC, differential operator, and Cauchy–Kowaleskytheorem will not equal those determined by the independent BCs. The resulting sin-gularities in the corners of the temporal-spatial domain, which are independent ofthe smoothness of the IC, disrupt the convergence of spectral and finite differencemethods in a manner that differs significantly between parabolic and hyperbolic sys-tems. A detailed analysis of the induced singularities for two model problems andtheir effect on the convergence of both numerical methods is determined. Since anincompatibility exists any time the BC is independent of the IC, algebraic convergencein the maximum norm is the expectation for IBVPs. In most science and engineeringapplications, this is the usual scenario.

In section 2, the infinite set of compatibility conditions for the generalized heatand wave equations with time-dependent BCs is presented. The purpose is to lay thegroundwork for describing the inherent singular nature of IBVPs and how it affects theconvergence rate of the error. In section 3, we examine the prototype parabolic case,namely the heat equation subject to Dirichlet BCs and a smooth parabolic IC. We usethe Fourier method to determine an exact expression for the approximate solution,represented by a truncated Fourier series. Convergence rates are then developed forboth finite difference and Chebyshev spectral methods, the latter of which is normallythe basis set of choice for a problem on a finite domain. For the parabolic example

INITIAL-BOUNDARY VALUE PROBLEMS 1733

in section 3, ut is discontinuous at t = 0. Nevertheless, the Chebyshev method isshown to yield spectral convergence for t > 0, but only quartic convergence in theneighborhood of the discontinuity, which would be the relevant consideration for thecomputation of transient heat flow. Furthermore, the usual Fourier method is onlyquadratically convergent near the discontinuity and quartically convergent elsewhere.Thus, the convergence rate of the error is shown to be nonuniform and algebraic asmeasured in the maximum norm.

The hyperbolic case is discussed in section 4. Discontinuities that are inducedby the incompatibilities now propagate throughout the region. It is shown that allspectral methods will yield algebraic convergence for t > 0. The Fourier methodis used again to derive an exact expression for the error. Similar to the paraboliccase, the results are compared with the centered finite difference and Chebyshevmethods which yield algebraic convergence for smooth ICs. The performance of finitedifference methods is also significantly diminished when the compatibility conditionsare not satisfied. For this reason, the appendix contains the rates of convergence fora second-order finite difference method for a variety of smooth ICs and shows, forexample, that second-order convergence is reduced to 4/3-order.

2. Compatibility conditions for the generalized heat and wave equa-tions. Here compatibility conditions are reviewed to illustrate their fundamental im-pact on the smoothness of the solutions to IBVPs. We simply state the necessaryconditions for the solution to be C∞ on the domain for the heat and wave equationwith time-dependent BCs. The complete proofs of necessity and sufficiency are givenfor parabolic systems in [11], [12], [3], [18] and for hyperbolic systems in [10], [16],[17], [4].

Theorem 2.1. The domain is [0, T ] × Ω, where Ω is a d-dimensional spatialdomain with a boundary ∂Ω which is a C∞ manifold of dimension (d− 1). Let L bean elliptic operator of the form

L =

d∑i=1

d∑j=1

Aij(x)∂

∂xi

∂

∂xj+

d∑j=1

Bj(x)∂

∂xj+

d∑j=1

Cj(x).(2.1)

The generalized linear diffusion problem is

ut = Lu, u(x, 0) = u0(x) ∈ Ω.(2.2)

The generalized wave equation is

utt = Lu, u(x, 0) = u0(x), ut(x, 0) = v0(x) ∈ Ω.(2.3)

Both are subject to the BC

u = f(t) ∈ ∂Ω ∀t.(2.4)

Then, the necessary and sufficient compatibility conditions for u(x,t) to be C2k

for the heat equation are

Lku0 =∂kf

∂tk, k = 0, 1, 2, . . . ∈ [0, 0]× ∂Ω(2.5)

and for the wave equation are

Lku0 =∂2kf

∂t2kand Lkv0 =

∂2k+1f

∂t2k+1, k = 0, 1, 2, . . . ∈ [0, 0]× ∂Ω.(2.6)


In essence (2.5) and (2.6) state that at time t = 0, the temporal derivativesof the solution determined by the application of the differential operator to the ICmust equal the temporal derivatives of the BCs. When these conditions are satisfied,we do not need to impose BCs because they can be derived from the Taylor seriesexpansion of the IC. When (2.5) and (2.6) do not hold, we induce a singularity in the2kth-order derivative of the solution at t = 0 on ∂Ω. Thus, imposing BCs that areindependent of the IC places an infinite set of constraints on the solution which differfrom the constraints enforced by the PDE, resulting in discontinuities in the cornersof the temporal-spatial domain. The manner in which these incompatibilities affectthe accuracy of the numerical method depends on whether the system is parabolic orhyperbolic. In the next section we will discuss the parabolic case, followed in section4 by the hyperbolic case.

3. A parabolic example. Our goal is to study the convergence of the error forapproximate spectral and finite difference solutions to IBVPs when the compatibilityconditions of the system are not satisfied. With this in mind, we first study the effecton parabolic systems, taking as our example the one-dimensional heat equation

ut = uxx(3.1)

subject to BCs

u(0, t) = u(π, t) = 0(3.2)

and IC

u0 = u(x, 0) =π

8x(π − x).(3.3)

The first-order compatibility condition, (k = 1) in (2.5), is not satisfied because

Lu0 =

(∂2u0

∂x2

)= −π

4∀x(3.4)

and

∂u

∂t|x=0,π = 0 ∀t.(3.5)

Thus from (3.1), uxx|x=0,π = ut|x=0,π = 0 yet uxx = −2 in the interior. To seethe impact of the jump discontinuity at the boundary in the second derivative att = 0 on the decay rate of the error we will solve the problem exactly at time t = 0via a truncated Fourier series, then compare this together with finite difference andChebyshev methods against the exact solution for all time.

Time integration is assumed exact throughout, which yields the approximatespectral representation

uN (xi, t) =

N−1∑n=1,odd

cN (n)e−n2t sinnxi(3.6)

of the exact solution, given by

u(xi, t) =

∞∑n=1,odd

e−n2t

n3sinnxi.(3.7)


Gottlieb and Orszag [8] note that for the exact coefficients cN (n), uN (x, t) convergesspectrally to u(x, t). This is also true if u(x, 0) is known for all x because cN (n)can be computed to any accuracy. In this work, we assume, as with most scientificdata, that u(x, 0) is given in tabular form ui = u(xi, t), where xi = iπ/N . In scienceand engineering, we cannot assume we know the function exactly and therefore mustapproach the problem from a numerical standpoint. The coefficients cN (n) are thencomputed in the traditional manner, using trigonometric interpolation, as

cN (n) =2

N

N−1∑i=1

ui sinnxi.(3.8)

With this implementation of the spectral method, it will be shown that the approxi-mate spectral representation (3.6) converges algebraically to the exact solution in themaximum norm, and, furthermore, convergence is nonuniform. This characteristicalgebraic convergence is method-independent due to the inherent singularity inducedby violating the compatibility conditions from explicit application of the BCs. In thenext section, we derive an exact expression for cN (n) which will provide the initialerror in the coefficients for the Fourier method.

3.1. The approximate spectral solution. In what follows, we derive an an-alytical representation of the coefficients cN (n) in (3.6). From (3.7)

u(xi, 0) =

∞∑n=1,odd

1

n3sin

(niπ

N

),(3.9)

u(xi, 0) =

N−1∑n=1,odd

[1

n3sin(n

iπ

N)(3.10)

+

∞∑m=1

1

(2mN + n)3sin

(2mN + n)i

iπ

N

+1

(2mN − n)3 sin(2mN − n)i iπ

N

],

or

u(xi, 0) =

N−1∑n=1,odd

1

n3+

∞∑m=1

1

(2mN + n)3− 1

(2mN − n)3sin

(niπ

N

)(3.11)

and

u(xi, 0) =

N−1∑n=1,odd

1

n3− n

N4Ψ(n/N)

sin

(niπ

N

),(3.12)

where

Ψ(s) = 2∞∑

m=1

3m2 + s2

[m2 − s2]3 .(3.13)

Ψ(s) can be called the alias function since it represents the error in the coefficientsdue to truncating the series after N terms. The initial error in the coefficients is due


to the fact that a sine series cannot represent a nonperiodic polynomial with a finitenumber of terms. For use later, we observe that Ψ(s) is positive and monotonicallyincreasing on the interval [0, 1] with extremes Ψ(0) ≈ 0.4058712 and Ψ(1) = 1.

Equating (3.6) and (3.12), we obtain the desired result

cN (n) =1

n3− nΨ(n)

N4.(3.14)

We study convergence only as it relates to spatial discretization, and therefore tem-poral integrations are represented analytically. The spectral approximation is thengiven by

uN (xi, t) =

N−1∑n=1,odd

e−n2t

[1

n3− nΨ(n/N)

N4

]sinnxi.(3.15)

In the next two sections it will be proved that although the spectral series forboth the exact and approximate solutions decay exponentially, the error decays alge-braically and nonuniformly over the domain. Section 3.2 explores convergence in theneighborhood of the discontinuity induced by the incompatibility of imposing BCs.Section 3.3 looks at convergence in the interior of the domain, bounded away fromthe singularities at (0, 0) and (0, π).

3.2. Quadratic convergence near (0, 0) and (0, π). The discontinuity in utat (0, 0) induces an error in ∂uN/∂t(π/N, 0) that is bounded away from zero for allN. In what follows, we show that this produces an O(N−2) error in the neighborhoodof (0, 0) and likewise near (π, 0). With a derivation similar to that preceding (3.12),the exact solution is given by

u(xi, t) =

N−1∑n=1,odd

e−n2t

n3−

∞∑m=2,even

[e−(mN+n)2t

[mN + n]3− e−(mN−n)2t

[mN − n]3]

sinnxi.(3.16)

Subtracting uN (xi, t), given by (3.15), defines the error

(3.17)

eN (xi, t) =

N−1∑n=1,odd

∞∑m=2,even

[e−(mN+n)2t − e−n2t

[mN + n]3− e−(mN−n)2t − e−n2t

[mN − n]3]

sinnxi.

Then, on the discrete trajectory x1 = π/N and tN = 1/N2

eN

(π

N,1

N2

)=

1

N2IN ,(3.18)

where

(3.19)

IN =1

N

N−1∑n=1,odd

∞∑m=2,even

[e−(m+n/N)2 − e−(n/N)2

[m+ n/N ]3− e−(m−n/N)2 − e−(n/N)2

[m− n/N ]3]

sinnπ

N.

IN defines the midpoint quadrature of the smooth positive function, which is theintegrand in (3.20) below. Therefore,

I = limN→∞

IN =

∫ 1

0

sinπs

∞∑m=2,even

[e−(m+s)2 − e−s2

[m+ s]3− e−(m−s)2 − e−s2

[m− s]3]ds.(3.20)


10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

10−11

10−10

10−9

10−8

10−7

10−6

10−5

10−4

t

|u(x

=π

/N,t

) −

uN

(x=

π/N

,t)|

N=32

N=128

N=64

Fig. 3.1. |eN | for the Fourier method on the interval tε(0, 1] for x = π/N and fixed N .

Numerical evaluation of the midpoint rule (3.20) with N = 512 provides the approx-imate value I ≈ .0876323127. This completes the proof that convergence is at bestquadratic. For each N there exists a point (π/N, 1/N2) at which the error is I/N2.The error as a function of time for x = π/N is plotted in Figure 3.1, where the max-imum occurs at t = 1/N2. Likewise, we can plot the error as a function of space fort = 1/N2 and see that the maximum error occurs at the endpoints as shown in Figure3.2(a). Numerical experiments, in the following sections, will confirm that indeedconvergence is quadratic.

3.3. Quartic convergence for a fixed t > ε > 0. For any fixed t > ε > 0 inthe interior of the domain, bounded away from the singularities at (0, 0) and (0, π),‖eN (xi, t)‖2 converges at best quartically. As was shown in (3.15), the coefficients ofthe approximate trigonometric series have an initial error of O(1/N4). This, in turn,induces quartic convergence for any fixed t.

The difference between (3.7) and (3.15) provides the error in the spectral approx-imation

eN (xi, t) = u(xi, t)− uN (xi, t) = SN (xi, t) + EN (xi, t),(3.21)

where

SN (xi, t) =1

N4

N−1∑n=1,odd

nΨ(n/N)e−n2t sinniπ

N(3.22)

and

EN (xi, t) =

∞∑n=N+1,odd

e−n2t

n3sinn

iπ

N.(3.23)


We will rigorously establish quartic convergence for fixed t > 0 by demonstratingspectral convergence for EN (xi, t) and, at best, quartic convergence for SN (xi, t).Starting with (3.23),

EN (xi, t) =

∞∑n=1,odd

e−(N+n)2t

(N + n)3sin(N + n)xi(3.24)

= e−N2t∞∑

n=1,odd

e−(2N+n2)t

(N + n)3sin(N + n)xi.(3.25)

Therefore,

|EN (xi, t)| < e−N2t∞∑

n=1,odd

1

(N + n)3< e−N2t

∫ ∞

N

ds

s3(3.26)

and finally

|EN (xi, t)| < e−N2t

2N2.(3.27)

Now consider the asymptotic behavior of SN (xi, t). From (3.22),

(3.28)

‖SN (xi, t)‖2 =1

N4

N−1∑n=1,odd

[nΨ(n/N)e−n2t]2

1/2

> Ψ(0)e−t

N4≈ 0.4058712

e−t

N4,

where the last inequality is from the earlier observation that Ψ(s) is monotone in-creasing on the interval [0,1] and consequently Ψ(1/N) > Ψ(0) ≈ 0.4058712.

This demonstrates that convergence in the l2 norm is at best quartic. In contrastto Figure 3.2(a), Figure 3.2(b) shows that for any fixed t, the maximum error, asmeasured by |eN |, occurs in the middle of the domain at x = π/2 as opposed to theendpoint, x = π/N , for small t. The reason the error curves change as t becomes large

is that the exponential term e−n2t dominates, and the error can be approximated bythe lowest wavenumber, n = 1. Thus, for large t, the error is essentially proportionalto sinx which has a maximum at π/2. As will be seen in the next section, numericalexperiments demonstrate that convergence in the l∞ norm is also quartic for t boundedaway from (0, 0) and (0, π).

3.4. Comparison with finite difference methods. Given the nonuniformconvergence of the Fourier method, as illustrated by quadratic convergence for smallt and quartic convergence for large t, it is natural to compare its performance withfourth and second-order finite difference methods that require less computation. Thefinite difference approximations are derived by substituting

ui =

N−1∑n=1,odd

an(t) sinnxi(3.29)

into

uit =ui−1 − 2ui + ui+1

δx2,(3.30)


0 0.5 1 1.5 2 2.5 3 3.510

−12

10−11

10−10

10−9

10−8

10−7

10−6

10−5

10−4

N=32

N=64

N=128

N=256

2

x

) −

uN

(x,t

=1

/N|u

(x,t

=1

/N)|

2

(a)

0 0.5 1 1.5 2 2.5 3 3.510

−11

10−10

10−9

10−8

10−7

10−6

N=32

N=64

N=128

N

x

|u(x

,t=

1)−

u(x

,t=

1)|

(b)

Fig. 3.2. (a) |eN (x,N−2)| on the interval xε(0, π) for t = 1/N2. (b) |eN (x, 1)| on the intervalxε(0, π) for t = 1.

where δx is π/N , yielding

an(t) = cN (n)eλ2nt, where λ2

n =−4N2 sin2( nπ2N )

π2.(3.31)

Likewise, the solution to the fourth-order finite difference approximation

uit =−ui−2 + 16ui−1 − 30ui + 16ui+1 − ui+2

12δx2(3.32)

is given by (3.31), except now

λ2n =

−4N2 sin2( nπ2N )

3π2

[3 + sin2

( nπ2N

)].(3.33)

The values of λn are the only difference between solutions to the finite difference andFourier method for which λn = n.

However, if we compare the performance of the FD4 to the Fourier method forlarge t, it can be seen that there is little difference, as shown in Figure 3.3(a). Forsmall t, a FD2 method performs just as well as the Fourier method and FD4 with onlya slightly larger constant of proportionality, as seen in Figure 3.3(b). This marginalerror reduction may not be sufficient to justify the expense of the Fourier method.However, it is reasonable to speculate that a FD4 scheme on a Chebyshev grid couldgive comparable results to the Chebyshev method that yields an O(N−4) convergenceerror near the endpoints, as will be seen in the next section.

3.5. Comparison with the Chebyshev method. The method of choice forthe interpolation of nonperiodic functions on a finite interval is the Chebyshev method.Unlike the Fourier method, the Chebyshev method does yield spectral convergence


100

101

102

103

10−10

10−9

10−8

10−7

10−6

10−5

10−4

10−3

N

|eN

(π

/2,1

)| 41/N

fd4Fourier

(a)

100

101

102

103

10−6

10−5

10−4

10−3

fd2fd4

FourierN

N

|e(p

i/N

,1/N

2)|

(b)

Fig. 3.3. (a) The error of a fourth-order centered finite difference method versus the error inthe spectral approximation for x = π/2, t = 1. (b) The error of a fourth-order and second-ordercentered finite difference method versus the error in the spectral approximation for x = π/N andt = 1/N2.

for fixed t. The reason is simply that the cosine series representation of a smoothIC does not alias. Nevertheless, convergence in the neighborhood of (0, 0) and (π, 0)is algebraic. To see why this is the case, we implement the Chebyshev method byexpanding the solution in terms of Chebyshev cardinal functions (see [1]).

The error for the Chebyshev method, as a function of space, is qualitatively similarto Figures 3.2(a) and 3.2(b), with the maximum error occurring at the endpoints forsmall t and in the middle of the domain for large t. The error as a function of time issimilar to Figure 3.1 in that it reaches a maximum and then decays, but it differs inthe fact that the decay becomes exponential after the maximum as seen in Figure 3.4.However, there exists a trajectory along which the max error converges algebraically asshown in Figure 3.5(a). The difference is that the max error is converging quarticallyas opposed to quadratically, as seen for the Fourier case. The increase in accuracyoccurs because the Chebyshev grid is quadratically clustered near the endpoints wherethe singularities are occurring, giving an extra factor of O(1/N2). In contrast to theFourier method for large t, the Chebyshev method does indeed give the expectedexponential rate of convergence for the error, as indicated in Figure 3.5(b). This is adirect result of the Chebyshev method’s ability to represent the IC exactly with onlythree polynomials.

Thus unlike the Fourier method, at time t = 0, there is no initial error in thecoefficients due to truncation.

For the parabolic case, we can see that the convergence rate of the error in themaximum norm is nonuniform and algebraic. In general, as long as we are interestedin solutions that are bounded away from singularities induced by violating the com-patibility conditions, the spectral method will yield spectral convergence. However,if we are interested in heat transients such as those that occur on a microprocessor,


10−6

10−5

10−4

10−3

10−2

10−1

100

10−10

10−9

10−8

10−7

10−6

10−5

10−4

10−3

N=32

N=16

N=8

N

t

|e(π

/N,t)|

Fig. 3.4. |eN | for the Chebyshev method on the interval tε(0, 1] for x = π/N and fixed N .

100

101

102

10−8

10−7

10−6

10−5

10−4

10−3

1/N 4

N

||e

||N

∞

CONVERGENCE OF THE MAX ERROR AT THE ENDPOINT − CHEBYSHEV

(a)

4 6 8 10 12 14 16 18 20 2210

−20

10−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

Chebyshev

e−2N

N

|eN|

CONVERGENCE FOR THE MAX ERROR FOR T=0.1

(b)

Fig. 3.5. (a) The max error in the Chebyshev approximation at the endpoint x = π/N as afunction of resolution N . (b) The error in the Chebyshev approximation for x = π/2 and t = 0.1.

a computationally more efficient method with comparable algebraic convergence maybe more attractive.

4. The hyperbolic case. In contrast to the parabolic case, a hyperbolic op-erator will propagate the solution through the time-space domain. As a result, sin-gularities induced by incompatible BCs and the IC are not smoothed as t increases.


To observe the effect on the convergence of the error for spectral methods, we willconsider the one-dimensional wave equation

utt = uxx(4.1)

subject to BCs

u(0, t) = u(π, t) = 0(4.2)

and ICs

u(x, 0) =π

8x(π − x), ut(x, 0) = 0.(4.3)

The first-order compatibility condition, (2.6) for u0 and k = 1, is not satisfied.The difference in the analysis of section 3.1 is that the time dependence of the exactand approximate spectral solution is given by an oscillatory function. Thus,

u(xi, t) =

∞∑n=1,odd

cos(nt)

n3sinnxi(4.4)

with the analogue to (3.15) being

uN (xi, t) =

N−1∑n=1,odd

cos(nt)

[1

n3− n

N4Ψ( nN

)]sinnxi.(4.5)

4.1. Global quadratic convergence. The decay rate of the error for hyper-bolic cases is more dramatic than for parabolic problems. The error induced byviolating the first-order compatibility condition for a hyperbolic problem not onlyproduces an O(N−2) error for the Fourier method in the neighborhood of (0, 0) and(π, 0) but propagates throughout the domain, resulting in global quadratic conver-gence in the maximum norm. The propagation of the error through the x−t plane isalong the characteristic x = t and x = π − t, as shown in Figures 4.1(a) and 4.1(b)for x ∈ [0, π] and t ∈ [0, π].

The proof of quadratic convergence of the error for small t follows that given insection 3.2. For large t, the proof is the same except the error is defined along thediscrete trajectory xN/2 = π/2 and tN = π/2 − 1/N . Numerical experiments, givenin Figure 4.2, show that convergence is quadratic along both of these trajectories,demonstrating the global nature of the error propagation.

4.2. The Chebyshev method. Next, we compare the performance of a Cheby-shev method with the Fourier method that yields quadratic convergence. As noted insection 3.5, we use Chebyshev cardinal functions. Figure 4.3 shows the propagation ofthe error through the x−t plane for the Chebyshev method. As expected, we see thatthe error propagates along the characteristics t = 1±x. The convergence of the erroralong the characteristics for any value of t differs significantly from the parabolic case,where spectral convergence was obtained for t > 0. The convergence of the error, asshown in Figure 4.4, is quadratic, no better than the Fourier series expansion andrequiring O(N2) operations. At first, one might suspect that the convergence ratewould be O(N4), the same as was observed for the heat equation in the neighborhood


00.5

11.5

22.5

33.5

0

0.5

1

1.5

2

2.5

3

3.50

0.5

1

1.5

2

x 10−4

xt

N=

32

(x,t

)||e

(a)

0.5 1 1.5 2 2.5 30

0.5

1

1.5

2

2.5

3

x

t

(b)

Fig. 4.1. (a) Propagation of the error through the x−t plane. (b) Contour plot of (a).

100

101

102

103

10−6

10−5

10−4

10−3

10−2

10−1

N

|e|

N

|eN

(x=π/2,t=π/2−1/N)|

|eN

(x=π/N,t=1/N)|

1/N2

Fig. 4.2. The convergence of the error in the Fourier method on the discrete trajectoriesx = π/2 and t = π/2− 1/N , and x = π/N and t = 1/N .

of the discontinuities (x = 0, t = 0) and (x = π, t = 0). The difference, however, isthat in the heat equation there is only a jump discontinuity in the second derivative ofthe solution at t = 0. For any t > 0, the discontinuity becomes smoothed out due tothe diffusive operator. Thus, in the neighborhood of the singularities for 0 < t < ε, theoriginal jump discontinuities become steep gradients that are analytically continuous.


−0.8 −0.6 −0.4 −0.2 0 0.2 0.4 0.6 0.8

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

x

t

Fig. 4.3. The propagation of the error in the Chebyshev approximation for N = 128.

100

101

102

10−4

10−3

10−2

10−1

N

|eN(x

=t)

|

2

Chebyshev

1/N

Fig. 4.4. The convergence of the error in the Chebyshev approximation along the characteristict = x.

Spectral methods have difficulty resolving these gradients, but the Chebyshev methodgives quartic convergence due to its quadratically clustered grid near the endpoints,as opposed to the uniform grid of the Fourier method. However, in the hyperboliccase, the discontinuities at t = 0 in the second derivative of the analytical solutionare present for all time. Thus, the type of grid used is irrelevant, and any spectralmethod will converge quadratically.


4.3. Comparison with finite difference methods. The quadratic conver-gence of spectral methods naturally refocuses attention on second-order finite differ-ence methods. The second-order centered finite difference method given by (3.30)yields the solution

uN (xi, t) =

N∑n=1,odd

cos(λnt)

1

n3− n

N4Ψ(n/N)

sinnxi,(4.6)

where

λn =2N

πsin

( nπ2N

)(4.7)

is a second-order approximation to the eigenvalues n. The error is then given by

eN (xi, t) = u(xi, t)− uN (xi, t) = TN (xi, t) + SN (xi, t) + EN (xi, t),(4.8)

where

TN (xi, t) =

N∑n=1,odd

cosnt− cosλnt

n3sinnxi,(4.9)

SN (xi, t) =1

N4

N∑n=1,odd

nΨ(n/N) cosλnt sinnxi,(4.10)

EN (xi, t) =

∞∑n=N+1,odd

cosnt

n3sinnxi.(4.11)

Using these formulas, we will analytically prove as well as computationally show thatnot only does a second-order finite difference method not give quadratic convergencebut rather converges only slightly better than linearly. The dominant error term isTN (xi, t) which we demonstrate by first showing that

|SN (xi, t)| < O(N−2) and |EN (xi, t)| < O(N−2).(4.12)

For t = x, we have

|SN (t)| = 1

2N4|

N∑n=1,odd

nΨ(n/N) cosλnt sinnt|(4.13)

≤ 1

2N4

N∑n=1,odd

|nΨ(n/N) cosλnt sinnt|

<1

2N2.

Similarly,

|EN (t)| = 1

2|

∞∑n=N+1,odd

cosnt

n3sin 2nt| ≤ 1

2

∞∑n=N+1,odd

|cosntn3

sin 2nt|(4.14)

<1

2

∫ ∞

N

ds

s3=

1

4N2.


100

101

102

103

10−4

10−3

10−2

10−1

FD2

1/(2N )4/3

N

||e

N(x

,t=

1)|

|∞

(a)

100

101

102

103

10−6

10−5

10−4

10−3

10−2

10−1

N

||u

−u

fd4

||∞

FD4

1/N3/2

(b)

Fig. 4.5. (a) The max error in x for the second-order centered finite difference method at t = 1.(b) The max error in x for the fourth-order centered finite difference method at t = 1.

Surprisingly, Figure 4.5 demonstrates that the error in a second-order finite differ-ence method converges as O(N−4/3). The proof for 4/3 convergence is rather detailedand deferred to Appendix A, where it is first shown that

|TN (t)| ≤ O(N−4/3)(4.15)

along the characteristic t = x for xε(0, π). To derive a lower bound is slightly morecomplicated, but nevertheless for any average value of t along the same characteristic,

1

π

∫ π

0

TN (t)dt >C

N4/3,(4.16)

where C is a constant. These bounds hold for any second-order finite differenceoperator in which there is a discontinuity in the second derivative of the solution. Thefact that the convergence of the second-order finite difference scheme is not quadraticmay not be surprising since the approximation is based on Taylor series expansionsthat assume continuity of the function at least up to the fourth derivative. However,the fact that it is 4/3 is surprising.

For a variety of ICs that would correspond to violating different order compatibil-ity conditions we propose the following conjecture for the upper bounds of the errorterm, TN (xi, t), due to a second-order finite difference approximation.

Let

TN (t) =

N∑n=1,odd

cosλn,N t− cosnt

n3sinnt =

N∑n=1,odd

bn,N (t).(4.17)

When we enforce BCs, we violate some order compatibility condition and the coef-ficients of the error, |bn,N (t)|, will be algebraically decreasing due to the resulting


discontinuity in a derivative of the Taylor expansion of the IC about the boundary.Thus, we can always impose the bound

|bn,N (t)| ≤ c

nβ, β ≥ 1,(4.18)

where c is a constant.From (4.18) and generalizing the analysis in Appendix A we make the following

conjecture whose derivation is relegated to Appendix B.Conjecture. The error term TN (xi, t) is bounded from above by

∑Nn=1 |bn,N (t)| ≤ O(N (1−β)2/3), β < 4,

≤ O(N−2 logN), β = 4,≤ O(N−2), β > 4.

For β < 4, the error from the IC dominates, while for β = 4 both the error fromthe IC and second-order difference approximation contribute equally. In the last casethe error from the finite difference operator dominates.

β > 4 implies that at least the fourth derivative of the IC is continuous. Thus, thebest a second-order method can perform is O(N−2). β < 4 implies that the functionis not C4, and therefore we would expect less than quadratic convergence since theerror in a second-order method depends on the continuity of the fourth derivativeof the function. However, the fact that β = 4 yields a convergence rate that isproportional to log(N) is not only new but quite surprising. The above conjecturehas been supported by the numerical evidence.

For completeness, the convergence rate of the FD4 method that was used insection 3.4 is considered. However, we still see that convergence is less than quadraticand in fact appears to be only O(N−3/2), as demonstrated in Figure 4.5(b). Thereduction in the convergence rates of the finite difference methods demonstrates thesignificant impact of incompatibilities which occur anytime the BCs are independentof the IC. In general, violation of the compatibility conditions will likely lead toalgebraic convergence of spectral methods as well as reduce the convergence rate offinite difference schemes.

5. Conclusions. In this paper, we have applied the theory of compatibilityconditions to the numerical solutions of PDEs and have demonstrated how the subtlesingularities inherent in IBVPs impact the convergence properties of spectral and finitedifference methods. The temporal derivatives of the solution as defined by the initialcondition, differential operator, and Cauchy–Kowalesky theorem will not equal thosedetermined by the independent BCs. The resulting singularities in the corners of thetemporal-spatial domain, which are independent of the smoothness of the IC, disruptthe spectral convergence of the error normally associated with spectral methods, in amanner that differs significantly between parabolic and hyperbolic systems.

For parabolic systems, the Chebyshev spectral method may be considered “self-healing,” leading to a spectral rate of convergence for the error. However, this is nottrue for the Fourier method where aliasing is likely to induce an algebraic error in theinitial Fourier representation. In the neighborhood of the singularities, all methodsyield convergence rates that are algebraic. Thus, as measured in the maximum norm,the convergence of the approximate solution to the exact solution for spectral methodsis both algebraic and nonuniform. Therefore, the usefulness of spectral methods maybe diminished if we are interested in transient solutions for small t.


For hyperbolic systems, the situation is more interesting but worse because thesingularities at t = 0 on the boundary propagate throughout the temporal-spatialdomain along the characteristic lines x± ct. Thus, the algebraic rate of convergencefor the error that existed in the neighborhood of the singularities for the parabolic caseexists for all time in hyperbolic problems. As a result, the spectral method does notyield spectral convergence for hyperbolic IBVPs. This naturally draws attention tofinite difference methods which require only O(N) operations. However, it was shownthat violation of the compatibility conditions can also reduce the order of convergencefor finite difference schemes.

The above study details the impact of the theory of compatibility conditionson numerical calculations for IBVPs. Application is broad because many scientificproblems are calculated on finite domains where independent BCs are imposed. As aresult, singularities in the corners of the temporal-spatial domain are likely to disruptthe accuracy of numerical methods. It is in this regard that the above study hasanalyzed the convergence rate of the error for spectral and finite difference methodsand explored their application to solving IBVPs.

Appendix A: Proof of 4/3 convergence. The term TN (xi, t), which accountsfor the error in the eigenvalue approximation, dominates the error of the second-orderfinite difference scheme. Our goal is therefore to bound the error from above andbelow to prove the 4/3 convergence rate that was computationally observed in section4.3.

We will first prove the upper bound for the error term TN (x, t) along the charac-teristic t = x for xε(0, π)

|TN (t)| ≤ O(N−4/3).(5.1)

Proof.

TN (t) =

N∑n=1,odd

cosλn,N t− cosnt

n3sinnt =

N∑n=1,odd

bn,N (t),(5.2)

where

λn,N = nsin( nπ2N )

nπ2N

.(5.3)

Since the lim supn→N cos(c(n)) sin(d(n)) = 1, we can immediately achieve an upperbound on the coefficients bn,N

|bn,N | ≤ 2

n3.(5.4)

This bound is due to the discontinuity in the second derivative of the IC from violatingthe compatibility conditions. However, because the series is decreasing slowly as 1/n3,this bound is only tight for the tail end of the series, when n is large. To describe thebehavior for the initial terms (i.e., small n), we need to derive a secondary bound.This bound will come from the error in the second-order finite difference operator.Using trigonometric identities, we can rewrite

cosλn,N t− cosnt = 2 sinn+ λn,N

2t sin

n− λn,N2

t.(5.5)


To find an upper bound on the product of the sine functions, we use the known resultderived from the Taylor expansion of sin y/y for yε[0, π] which gives

| sin n− λn,N2

t| < n− λn,N2

t <π2

24

n3

N2t.(5.6)

Therefore,

|bn,N | ≤ |(cosλn,N t− cosnt) sinnt|n3

≤ π2

24

t

N2.(5.7)

The trick now is to divide the summation in (5.2) into two parts,

N∑n=1,odd

bn,N (t) =

Nγ∑n=1,odd

bn,N (t) +

N∑n=Nγ+1,odd

bn,N (t),(5.8)

where 0 < γ < 1. Equation (5.7) gives the tightest bound for the initial terms, while(5.4) gives a tighter bound for the tail end of the series. The bounds are minimized,giving equal contributions, for γ = 2/3. Therefore, we have

|TN (t)| = |N∑

n=1,odd

bn,N (t)| ≤N2/3∑

n=1,odd

|bn,N (t)|+N∑

n=N2/3+1,odd

|bn,N (t)|(5.9)

≤ N2/3π2

24

t

N2+

N∑n=N2/3+1,odd

2

n3(5.10)

≤ π2

24

t

N4/3+

1

N4/3=

(1 +

π2

24t

)N−4/3.(5.11)

For the second part of the proof, we show that TN (x, t) for any average value oft along the characteristic t = x on xε(0, π) is bounded from below by

1

π

∫ π

0

TN (t)dt >C

N4/3.(5.12)

Proof. Beginning with

1

π

∫ π

0

TN (t)dt =1

π

N∑n=1,odd

∫ π

0

cosnt− cosλnt

n3sinnt,(5.13)

then by using trigonometric identities and integrating, we have

1

π

∫ π

0

TN (t)dt =1

2

N∑n=1,odd

f(n(1+λn/n)2 π) + f(n(1−λn/n)

2 π)

n3(5.14)

≥ 1

2

N2/3∑n=1,odd

f(n(1−λn/n)2 π)

n3,(5.15)

where

f(y) =1− cos(y)

y.(5.16)


Next, we will bound the argument of f which is of the form 1− sin yy as shown below.

By inspection, we can show that on yε(0, π), y2/12 < 1 − sin yy < y2/6, which yields

the following bounds:

n(1− λn/n)2

π =π

2n

(1− sin nπ

2Nnπ2N

)(5.17)

<nπ

2

nπ2N

6=π3

48

n3

N2(5.18)

>nπ

2

nπ2N

12=π3

96

n3

N2.(5.19)

In the range 0 < y < 1, f(y) can be bounded by its argument 1/4y < f(y) < 1/2y.Thus, for n < N2/3

1

4

n(1− λn/n)2

π < f

(n(1− λn/n)

2π

)(5.20)

which from (5.19) implies

π3n3

384N2< f

(n(1− λn/n)

2π

).(5.21)

Therefore, from (5.15)

1

π

∫ π

0

TN (t)dt >π3

768N2

N2/3∑n=1,odd

1 >π3

1536N4/3.(5.22)

Appendix B: General upper bounds on the error term TN(t). In general,when we have to enforce BCs we will violate some order compatibility condition,and the coefficients of the error will be algebraically decreasing due to the resultingdiscontinuity in a derivative of the Taylor expansion of the IC about the boundary.Thus, we can always impose the bound

|bn,N (t)| ≤ c

nβ, β ≥ 1,(5.23)

where c is a constant. From the analysis in Appendix A, we can generalize (5.7), theerror bound induced by the second-order difference operator, for any IC as

|bn,N (t)| ≤c2tn(

nN )2

nβ= c2t

n3−β

N2.(5.24)

If we split the summation which would correspond to TN (xi, t) into two parts as wasdone in (5.8), we achieve the following approximation:∑Nγ

n=1 c2tn3−β

N2 ∼ c2t4−βN

−2N (4−β)γ , β < 4,

c2tN−2 logNγ , β = 4,

c3tN−2, β > 4.

Similarly, for the tail end of the series as N becomes large, we have, using (5.23),

∞∑n=Nγ+1,odd

∼ c

β − 1Nγ(1−β).(5.25)

From these approximations, the conjecture in section 4.3 follows for the upper boundon the error term, TN (xi, t), due to the second-order finite difference scheme.


Acknowledgments. The authors gratefully thank Dr. Keith Lindsay at the Na-tional Center for Atmospheric Research for his discussions, help, and contributions,particularly with the convergence proofs in the appendices and the factorizations (5.8)and (5.9).

REFERENCES

[1] J.P. Boyd, Chebyshev and Fourier Spectral Methods, Springer-Verlag, New York, 1989.[2] J.P. Boyd and N. Flyer, Compatibility conditions for time-dependent partial differential

equations and the rate of convergence of Chebyshev and Fourier spectral methods, Comput.Methods Appl. Mech. Engrg., 175 (1999), pp. 281–309.

[3] A. Friedman, Partial Differential Equations of Parabolic Type, Prentice-Hall, EnglewoodCliffs, NJ, 1964.

[4] H.O. Kreiss, Initial boundary value problems for hyperbolic systems, Comm. Pure Appl. Math,23 (1970), pp. 277–298.

[5] H.O. Kreiss, V. Thomee, and O. Widlund, Smoothing of the initial data and rates of conver-gence for parabolic difference equations, Comm. Pure Appl. Math., 23 (1970), pp. 241–259.

[6] H.O. Kreiss and J. Lorenz, Initial-Boundary Value Problems and the Navier-Stokes Equa-tions, Academic Press, Boston, 1989.

[7] R. Hersh, Mixed problems in several variables, J. Math. Mech., 12 (1963), pp. 317–334.[8] D. Gottlieb and S.A. Orszag, Numerical Analysis of Spectral Methods: Theory and Appli-

cations, SIAM, Philadelphia, 1977.[9] B. Gustafsson, H.O. Kriess, and J. Oliger, Time-Dependent Problems and Difference Meth-

ods, Wiley-Interscience, New York, 1995.[10] O. Ladyzenskaja, On the convergence of Fourier series defining a solution of a mixed problem

for hyperbolic equations, Dokl. Akad. Nauk SSSR, 85 (1952), pp. 481–484.[11] O. Ladyzenskaja, On the solvability of the fundamental boundary problems for equations of

parabolic and hyperbolic type, Dokl. Akad. Nauk SSSR, 87 (1954), pp. 395–398.[12] O. Ladyzenskaja, V. Solonnikov, and N. Ural’ceva, Linear and Quasi-Linear Equations

of Parabolic Type, Transl. Math. Monogr. 23, AMS, Providence, RI, 1968.[13] A. Majda, J. McDonough, and S. Osher, The Fourier method for nonsmooth initial data,

Math. Comp., 32 (1978), pp. 1041–1081.[14] A. Majda and S. Osher, Propagation of error into regions of smoothness for accurate dif-

ference approximations to hyperbolic equations, Comm. Pure Appl. Math., 30 (1977), pp.671–705.

[15] S. Osher, Smoothing for spectral methods, in Spectral Methods for Partial Differential Equa-tions, SIAM, Philadelphia, 1984, pp. 209–216.

[16] J.B. Rauch and F.J. Massey, Differentiability of solutions to hyperbolic initial-boundary valueproblems, Trans. Amer. Math. Soc., 189 (1974), pp. 303–318.

[17] R. Sakamoto, Hyperbolic Boundary Value Problems, Cambridge University Press, Cambridge,UK, 1982.

[18] S. Smale, Smooth solutions of the heat and diffusion equations, Comment. Math. Helv., 55(1980), pp. 1–12.

[19] V.I. Smirnov, A Course of Higher Mathematics. Integral Equations and Partial DifferentialEquations, Vol. 4, Pergamon Press, London, 1964.

[20] R. Temam, Behaviour at time t = 0 of the solutions of semi-linear evolution equations, J.Differential Equations, 43 (1982), pp. 73–92.

[21] V. Thomee, Numerical Solution of Partial Differential Equations II, Academic Press, 1970,pp. 585–622.

[22] V. Thomee and L. Wahlbin, Convergence rates of parabolic difference schemes for non-smoothdata, Math. Comp., 28 (1974), pp. 1–13.

theconvergenceofspectralandfinitedifference ... · theconvergenceofspectralandfinitedifference...

Documents