w avelets in scientiﬁc computing - technical … pa˚ dansk w avelets i scientiﬁc computing...

Waveletsin ScientificComputing

Ph.D.Dissertationby

OleMøller [email protected]

http://www.imm.dtu.dk/˜omni

Departmentof MathematicalModelling UNI � CTechnicalUniversityof Denmark TechnicalUniversityof DenmarkDK-2800Lyngby DK-2800LyngbyDenmark Denmark

Preface

This Ph.D.studywascarriedout at theDepartmentof MathematicalModelling,TechnicalUniversity of Denmarkandat UNI � C, the DanishComputingCentrefor Researchand Education. It hasbeenjointly supportedby UNI � C and theDanishNaturalScienceResearchCouncil(SNF)undertheprogramEfficientPar-allel Algorithmsfor OptimizationandSimulation(EPOS).

Theprojectwasmotivatedby adesirein thedepartmentto generateknowledgeaboutwavelet theory, to developandanalyzeparallelalgorithms,andto investi-gatewavelets’applicability to numericalmethodsfor solving partialdifferentialequations.Accordingly, thereportfalls into threeparts:

Part I: Wavelets:BasicTheory and Algorithms.Part II: Fast WaveletTransforms on Supercomputers.Part III: Waveletsand Partial Differ ential Equations.

Waveletanalysisis a youngandrapidly expandingfield in mathematics,andtherearealreadya numberof excellentbookson the subject. Importantexam-ples are [SN96, Dau92, HW96, Mey93, Str94]. However, it would be almostimpossibleto give a comprehensive accountof waveletsin a singlePh.D.study,so we have limited ourselvesto oneparticularwavelet family, namelythe com-pactlysupportedorthogonalwavelets. This family wasfirst describedby IngridDaubechies[Dau88],andit is particularlyattractive becausethereexist fastandaccuratealgorithmsfor the associatedtransforms,themostprominentbeingthepyramidalgorithmwhichwasdevelopedby StephaneMallat [Mal89].

Ourfocusis onalgorithmsandweprovideMatlabprogramswhereapplicable.Thiswill beindicatedby themargin symbolshown here.TheMatlabpackageis ✤availableon theWorld WideWebat

http://www.imm.dtu.dk/˜omni/wapa20.t gz

andits contentsarelistedin AppendixE.We have tried our bestto makethis expositionasself-containedandacces-

sibleaspossible,andit is our sincerehopethat the readerwill find it a help for

iv Preface

understandingtheunderlyingideasandprinciplesof waveletsaswell asa usefulcollectionof recipesfor appliedwaveletanalysis.

I would like to thankthe following peoplefor their involvementandcontri-butionsto thisstudy:My advisorsat theDepartmentof MathematicalModelling,ProfessorVincentA. Barker, ProfessorPerChristianHansen,andProfessorMadsPeterSørensen.In addition,Dr. MarkusHegland,ComputerSciencesLaboratory,RSISE,AustralianNationalUniversity, ProfessorLionel Watkins,DepartmentofPhysics,University of Auckland, Mette Olufsen,Math-Tech, Denmark,IestynPierce,Schoolof ElectronicEngineeringandComputerSystems,University ofWales,andlastbut not leastmy family andfriends.

Lyngby, March1998

OleMøller Nielsen

Abstract

Waveletsin ScientificComputing

Waveletanalysisis a relatively new mathematicaldisciplinewhich hasgeneratedmuchinterestin both theoreticalandappliedmathematicsover the pastdecade.Crucialto waveletsaretheir ability to analyzedifferentpartsof a functionat dif-ferentscalesandthefact thatthey canrepresentpolynomialsup to acertainorderexactly. As a consequence,functionswith fast oscillations,or even discontinu-ities, in localizedregionsmaybe approximatedwell by a linearcombinationofrelatively few wavelets.In comparison,a Fourierexpansionmustusemany basisfunctionsto approximatesucha functionwell. Thesepropertiesof waveletshaveleadto somevery successfulapplicationswithin the field of signalprocessing.Thisdissertationrevolvesaroundtheroleof waveletsin scientificcomputingandit falls into threeparts:

Part I givesan expositionof the theoryof orthogonal,compactlysupportedwaveletsin thecontext of multiresolutionanalysis.Thesewaveletsareparticularlyattractivebecausethey leadto astableandveryefficientalgorithm,namelythefastwavelettransform(FWT).Wegiveestimatesfor theapproximationcharacteristicsof waveletsanddemonstratehow andwhy theFWT canbeusedasafront-endforefficient imagecompressionschemes.

Part II dealswith vector-parallel implementationsof several variantsof theFastWaveletTransform.We developanefficient andscalableparallelalgorithmfor theFWT andderivea modelfor its performance.

Part III is an investigationof thepotentialfor usingthespecialpropertiesofwaveletsfor solvingpartialdifferentialequationsnumerically. Severalapproachesareidentifiedandtwo of themaredescribedin detail. Thealgorithmsdevelopedareappliedto thenonlinearSchrodingerequationandBurgers’equation.Numer-ical resultsrevealthatgoodperformancecanbeachievedprovidedthatproblemsarelarge,solutionsarehighly localized,andnumericalparametersarechosenap-propriately, dependingon theproblemin question.

vi Abstract

Resumepa dansk

Waveletsi ScientificComputing

Waveletteorier en forholdsvisny matematiskdisciplin, somhar vakt stor inter-esseindenforbadeteoretiskog anvendtmatematiki løbetaf detsenestearti. Dealtafgørendeegenskabervedwaveletseratdekananalysereforskelligedeleaf enfunktionpaforskelligeskalatrin,samtatdekanrepræsenterepolynomiernøjagtigtop til en givengrad. Dettefører til, at funktionermedhurtigeoscillationerellersingulariteterindenforlokaliseredeomraderkanapproksimeresgodtmedenlin-earkombinationaf forholdsvisfa wavelets. Til sammenligningskal man med-tagemangeled i en Fourierrækkefor at opna en god tilnærmelsetil denslagsfunktioner. Disseegenskabervedwaveletsharmedheldværetanvendtindenforsignalbehandling.Denneafhandlingomhandlerwaveletsrolle indenforscientificcomputingogdenbestaraf tredele:

Del I giverengennemgangaf teorienfor ortogonale,kompaktstøttedewaveletsmedudgangspunkti multiskalaanalyse.Sadannewaveletser særligtattraktive,fordi de giver anledningtil en stabil og særdeleseffektiv algoritme,kaldetdenhurtigewavelet transformation(FWT). Vi giver estimaterfor approksimations-egenskaberneaf waveletsog demonstrerer, hvordanog hvorfor FWT-algoritmenkanbrugessomførsteled i eneffektiv billedkomprimeringsmetode.

Del II omhandlerforskelligeimplementeringeraf FWT algoritmenpavektor-computereog parallelledatamater. Vi udvikler en effektiv og skalerbarparallelFWT algoritmeogangiverenmodelfor densydeevne.

Del III omfatteret studiumaf mulighedernefor at brugewaveletssærligeegenskabertil at løsepartielledifferentialligningernumerisk. Flere forskelligetilgangeidentificeresog to af dembeskrivesdetaljeret.De udvikledealgoritmeranvendespadenikke-lineæreSchrodingerligning ogBurgersligning. Numeriskeundersøgelserviser, at algoritmernekan væreeffektive under forudsætningafat problemerneer store,at løsningerneer stærktlokaliseredeog at de forskel-lige numeriskemetode-parametrekan vælgespa passendevis afhængigtaf detpagældendeproblem.

viii Resumepadansk

Notation

Symbol Page Description��Genericmatrices �� Genericvectors�� 110 �� ! �"$#&%�'

25"$#(%�' �*) !,+(-/. �10 !,+2�354 � 276 !�8 2:9� 2 16 Filter coefficient for ;< #&%�'

29< #(%�' ��) !,+(-/. � 0 !,+2�354>= 2?6 !@8 2:9< #&AB'

116 Bandwidthof matrixA= 2 16 Filter coefficient for CD 49 VectorcontainingscalingfunctioncoefficientsD5E 163 Vector containingscalingfunction coefficientswith re-

spectto thefunction FG�H(I 2 6 ScalingfunctioncoefficientG 2 3 Coefficient in FourierexpansionJLK23 MON #:PRQS',T 0 !,+4 UUWV K C # V ' UU,X@VJLY62

JLY �1Z\[^]@_�`�a 4 I 0 !,+cb@d C # V ' dJeJ 8 I H � Jgf 8 I H � fhJ 8 I H � fif 8 I H122 Shift-circulantblockmatricesG&G 8 I H � G X 8 I H � X G 8 I H � X&X 8 I H 128 Vectorsrepresentingshift-circulantblockmatricesf16 Waveletgenus- numberof filter coefficientsA*jSk/lm 216 FourierdifferentiationmatrixA jSk/l113 ScalingfunctiondifferentiationmatrixnA jSk/l119 Waveletdifferentiationmatrixo57 Vectorcontainingwaveletcoefficientso E 166 Vectorcontainingwaveletcoefficientswith respectto the

function FX H&I 2 6 Waveletcoefficientp175

p �rq7]5sutOvxw.zy|{67} #&~i' 61 Pointwiseapproximationerror�67} #&~i' 63 Pointwise(periodic)approximationerror� �:��e� F �� 3 Genericfunctions��210 Fouriermatrix��#&%�'25 ContinuousFouriertransform

��#&%�' � T��! � ��#&~i' 6 !@8 9(� X ~ix

x Notation

Symbol Page Description�121 Wavelettransformof a circulantmatrix� 8 I H136 Block matrixof

�� H&I 2 17 Supportof ; H&I 2 and C H(I 2� 4 6 Integerdenotingcoarsestapproximation�6 Integerdenotingfinestapproximation�146 Bandwidth(only in Chapter8)�174 Lengthof period(only in Chapter9)� 8 I H136 Bandwidthof block

� 8 I H(only in Chapter8)�

174 Lineardifferentialoperatory 174 Matrix representationof�� 2 19 � th momentof ; #&~��x'�

5 Lengthof a vector�47 Setof positive integers� 4 27 Setof non-negativeintegers� � #��@'65 Numberof insignificantelementsin waveletexpansion�� #��@'64 Numberof significantelementsin waveletexpansion�174 Matrix representationof nonlinearityP19 Numberof vanishingmoments

P � f N )P85 Numberof processors(only in Chapter6)Pe �¡��15 Orthogonalprojectionof

�onto ¢ HPe£¤¡¥�

15 Orthogonalprojectionof�

onto ¦ HPz§ �¡ �41 Orthogonalprojectionof

�onto

�¢ HP�§£¤¡ �41 Orthogonalprojectionof

�onto

�¦ H¨3 Setof realnumbers© 858

© 8 � � N ) 8 , sizeof a subvectorat depthª© K8 88© K8 � © 8 N P , sizeof a subvectorat depth ª on oneof

Pprocessors«

49 Matrix mappingscalingfunctioncoefficientsto functionvalues: ¬ � « DF } 163 Approximationto thefunction F¢ H 11 th approximationspace,¢ H|® � . # ¨ '�¢ H 7 th periodizedapproximationspace,

�¢ H�® � . #�¯±° � M�² '¦ H 11 th detailspace,¦ H|³ ¢ H and ¦ H�® � . # ¨ '�¦ H 7 th periodizeddetailspace,�¦ H�³ �¢ H and

�¦ H�® � . #�¯±° � M�² '´¶µ58 Wavelettransformmatrix:

o � ´¶µ Dn�60 Wavelettransformof matrix

�:n� � ´·µ�¸h��´·µ�¹n�Bº

69 Wavelettransformedandtruncatedmatrixn�59 Wavelettransformof vector

�:n� � ´ µ �»

11 Setof integers

xi

Symbol Page Description¼ 163 Constantin Helmolzequation½ . � ½x¾ 174 Dispersionconstants¿ kÀ 108 ConnectioncoefficientÁ k109 Vectorof connectioncoefficientsÂ 174 NonlinearityfactorÃ 2 I À 14 Kroneckerdelta�� 171 Thresholdfor vectors�^Ä171 Thresholdfor matrices� 0 187 Specialfixedthresholdfor differentiationmatrixÅ 212 DiagonalmatrixÆ � Æ5Ä � Æ �15 Depthof waveletdecompositionÇ 168 Dif fusionconstant%23 Substitutionfor

~or

25 Variablein FouriertransformÈ 186 Advectionconstant; #&~i' 11,13 Basicscalingfunction; H&I 2 #&~i' 13 ; H(I 2 #(~É' ��) H -/. ; # ) H ~��x'; 2 #(~É' 13 ; 2 � ; 4 I 2 #&~i'�; H&I 2 #&~i' 6, 34 PeriodizedscalingfunctionC #&~i' 13 BasicwaveletC H&I 2 #&~i' 13 C H&I 2 #&~i' �*) H -:. ; # ) H ~>��x'C 2 #(~i' 13 C 2 � C 4 I 2 #(~É'�C H&I 2 #&~i' 6, 34 PeriodizedwaveletÊ � 210 Ê � � 6 8S.&ËÌ- �

xii Notation

Symbolsparametrized by algorithms

Symbol Page DescriptionÍeÎ # � '59 Complexity of algorithm ÏJ Î93 Communicationtime for algorithm ÏÐxÎ # � '76 Executiontime for Algorithm ÏÐ KÎ # � '93 Executiontime for Algorithm Ï on

PprocessorsÐ 4Î # � '

93 Sequentialexecutiontime for Algorithm ÏÑ KÎ # � '94 Efficiency of Algorithm Ï on

Pprocessors© KÎ # � '

97 Speedupof Algorithm Ï onP

processors

Thesymbol Ï is onethefollowing algorithms:

Algorithm ( Ï ) Page DescriptionPWT 53 PartialWaveletTransform;onestepof theFWTFWT 53 FastWaveletTransformFWT2 60 2D FWTMFWT 79 Multiple 1D FWTRFWT 95 ReplicatedFWT algorithmCFWT 97 Communcation-efficientFWT algorithmCIRPWT1 134 CirculantPWT(diagonalblock)CIRPWT2 134 CirculantPWT(off-diagonalblocks)CIRFWT 139 Circulant2D FWTCIRMUL 158 Circulantmatrixmultiplicationin a waveletbasis

MiscellaneousSymbol Page DescriptionÒ F Ò � 63 Infinity normfor functions.

Ò F Ò � �1ZÓ[O] � d F #&~i' dÒ F Ò } I � 165 Pointwiseinfinity normfor functions.Ò F Ò } I � �1ZÓ[O] 2 UU F #&� N ) } ' UUÒ¥ÔÕÒ � 173 Infinity normfor vectors.Ò¥Ô�Ò � �rZÓ[^] 2 d F 2 dÒ F Ò . 11 ) -normfor functions.

Ò F Ò . � #:T d F #(~i' d . X ~i' +&-/.Ò¥ÔÕÒ . ) normfor vectors.Ò¥Ô�Ò . � # � 2 d F 2 d . ' +(-:.Ö ~ ×

36 Thesmallestintegergreaterthan~Ø ~ Ù

39 Thelargestintegersmallerthan~¯±~ ² 205 NearestintegertowardszeroÚ&ÛÝÜ � 205 Modulusoperator(

Ûmod � )¯ � ²ßÞ � ~ Þ 212 The

Ûth elementin vector

�¯ � ²áà I Þ ��â à I Þ 209 The ã � Ûth elementin matrix

�

Contents

I Wavelets:BasicTheory and Algorithms 1

1 Motivation 31.1 Fourierexpansion. . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Waveletexpansion . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Multir esolutionanalysis 112.1 Waveletson thereal line . . . . . . . . . . . . . . . . . . . . . . 112.2 WaveletsandtheFouriertransform. . . . . . . . . . . . . . . . . 252.3 Periodizedwavelets . . . . . . . . . . . . . . . . . . . . . . . . . 33

3 Waveletalgorithms 433.1 Numericalevaluationof ; and C . . . . . . . . . . . . . . . . . . 433.2 Evaluationof scalingfunctionexpansions . . . . . . . . . . . . . 473.3 FastWaveletTransforms . . . . . . . . . . . . . . . . . . . . . . 52

4 Approximation properties 614.1 Accuracy of themultiresolutionspaces. . . . . . . . . . . . . . . 614.2 Waveletcompressionerrors. . . . . . . . . . . . . . . . . . . . . 644.3 Scalingfunctioncoefficientsor functionvalues? . . . . . . . . . 664.4 A compressionexample. . . . . . . . . . . . . . . . . . . . . . . 67

II FastWaveletTransforms on Supercomputers 71

5 Vectorization of the Fast WaveletTransform 735.1 TheFujitsuVPP300. . . . . . . . . . . . . . . . . . . . . . . . . 735.2 1D FWT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745.3 Multiple 1D FWT . . . . . . . . . . . . . . . . . . . . . . . . . . 795.4 2D FWT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 815.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

xiii

xiv

6 Parallelization of the Fast WaveletTransform 856.1 1D FWT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 866.2 Multiple 1D FWT . . . . . . . . . . . . . . . . . . . . . . . . . . 916.3 2D FWT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 956.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

III Waveletsand Partial Differ ential Equations 103

7 Waveletsand differ entiation matrices 1057.1 Previouswaveletapplicationsto PDEs . . . . . . . . . . . . . . . 1057.2 Connectioncoefficients . . . . . . . . . . . . . . . . . . . . . . . 1087.3 Dif ferentiationmatrixwith respectto scalingfunctions . . . . . . 1127.4 Dif ferentiationmatrixwith respectto physicalspace . . . . . . . 1147.5 Dif ferentiationmatrixwith respectto wavelets. . . . . . . . . . . 118

8 2D FastWaveletTransform of a circulant matrix 1218.1 Thewavelettransformrevisited. . . . . . . . . . . . . . . . . . . 1218.2 2D wavelettransformof acirculantmatrix . . . . . . . . . . . . . 1278.3 2D wavelettransformof acirculant,bandedmatrix . . . . . . . . 1468.4 Matrix-vectormultiplicationin awaveletbasis. . . . . . . . . . . 1538.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

9 Examplesof wavelet-basedPDE solvers 1639.1 A periodicboundaryvalueproblem . . . . . . . . . . . . . . . . 1639.2 Theheatequation . . . . . . . . . . . . . . . . . . . . . . . . . . 1689.3 ThenonlinearSchrodingerequation . . . . . . . . . . . . . . . . 1749.4 Burgers’equation . . . . . . . . . . . . . . . . . . . . . . . . . . 1859.5 WaveletOptimizedFiniteDif ferencemethod . . . . . . . . . . . 188

10 Conclusion 199

A Momentsof scalingfunctions 203

B The modulusoperator 205

C Cir culant matricesand the DFT 209

D Fourier differ entiation matrix 215

E List of Matlab programs 219

Bibliography 221

Index 229

Part I

Wavelets:BasicTheory andAlgorithms

Chapter 1

Moti vation

This sectiongivesan introductionto waveletsaccessibleto non-specialistsandserves at the sametime as an introductionto key conceptsand notationusedthroughoutthisstudy.

Thewaveletsconsideredin this introductionarecalledperiodizedDaubechieswaveletsof genusfour andthey constitutea specificbut representative exampleof waveletsin general.For thenotationto beconsistentwith therestof thiswork,wewrite ourwaveletsusingthesymbol

�C H&I À , thetilde signifyingperiodicity.

1.1 Fourier expansion

Many mathematicalfunctionscan be representedby a sum of fundamentalorsimplefunctionsdenotedbasisfunctions.Suchrepresentationsareknown asex-pansionsor series,awell-known examplebeingtheFourierexpansion

��#&~i' � �ä2�3 ! � G 276 8S.&Ë 2/� � ~ ® ¨(1.1)

which is valid for any reasonablywell-behaved function�

with period M . Here,thebasisfunctionsarecomplex exponentials6 8S.&Ë 2/� eachrepresentinga particularfrequency indexedby

�. TheFourierexpansioncanbeinterpretedasfollows: If

�is a periodicsignal,suchasa musicaltone,then(1.1)givesa decompositionof

�asasuperpositionof harmonicmodeswith frequencies

�(measuredby cyclesper

timeunit). This is a goodmodelfor vibrationsof a guitarstringor anair columnin awind instrument,hencetheterm“harmonicmodes”.

Thecoefficients G 2 aregivenby theintegral

G 2 ��å +4 ��#&~i' 6 !@8æ.çË 2:� X ~

4 Motivation

0 1−0.5

0

0.5

x

Figure 1.1: Thefunction è .

−512 −256 0 256 5120

0.1

Figure 1.2: Fouriercoefficientsof è .

Eachcoefficient G 2 canbeconceived astheaverageharmoniccontent(over oneperiod)of

�at frequency

�. Thecoefficient G 4 is theaverageatfrequency

°, which

is just theordinaryaverageof�. In electricalengineeringthis termis known as

the “DC” term. Thecomputationof G 2 is calledthedecompositionof�

andtheserieson theright handsideof (1.1) is calledthereconstructionof

�.

In theory, the reconstructionof�

is exact, but in practicethis is rarely so.Exceptin theoccasionalevent where(1.1) canbe evaluatedanalyticallyit mustbetruncatedin orderto becomputednumerically. Furthermore,oneoftenwantsto save computationalresourcesby discardingmany of thesmallestcoefficientsG 2 . Thesemeasuresnaturallyintroduceanapproximationerror.

To illustrate,considerthesawtoothfunction

�¤#(~É' �êé ~ °ìë�~Óí1°5îðï~�� M °5îWï�ë�~Óí M

1.1Fourierexpansion 5

0 1−0.6

0

0.6

x

Figure 1.3: A function è andatruncatedFourierexpansionwith only 17 terms

which is shown in Figure1.1. TheFouriercoefficients G 2 of thetruncatedexpan-sion

� -/.ä2ñ3 ! � -/./òx+ G 276 8S.&Ë 2/�

areshown in Figure1.2for� � M ° )^ó .

If, for example,we retainonly the M^ô largestcoefficients,we obtainthetrun-catedexpansionshown in Figure1.3. While this approximationreflectssomeofthebehavior of

�, it doesnot do a goodjob for thediscontinuityat

~ � °5îðï. It is

aninterestingandwell-known fact thatsucha discontinuityis perfectlyresolvedby theseriesin (1.1),eventhoughtheindividualtermsthemselvesarecontinuous.However, with only a finite numberof termsthis will not be the case. In addi-tion, andthis is very unfortunate,theapproximationerror is not restrictedto thediscontinuitybut spills into muchof thesurroundingarea.This is known astheGibbsphenomenon.

Theunderlyingreasonfor thepoorapproximationof thediscontinuousfunc-tion liesin thenatureof complex exponentials,asthey all cover theentireintervalanddiffer only with respectto frequency. While suchfunctionsarefine for rep-resentingthebehavior of a guitarstring,they arenot suitablefor a discontinuousfunction. Sinceeachof the Fourier coefficient reflectsthe averagecontentof acertainfrequency, it is impossibleto seewherea singularityis locatedby lookingonly at individual coefficients. The informationaboutpositioncanberecoveredonly by computingall of them.

6 Motivation

1.2 Waveletexpansion

Theproblemmentionedabove is onewayof motivatingtheuseof wavelets.Likethecomplex exponentials,waveletscanbeusedasbasisfunctionsfor theexpan-sion of a function

�. Unlike the complex exponentials,they areableto capture

the positionalinformationabout�

aswell as informationaboutscale. The lat-ter is essentiallyequivalentto frequency information. A waveletexpansionfor aM -periodicfunction

�hastheform

�¤#(~i' � .öõß÷�!,+ä 2�354 G } ÷ I 2 �; } ÷ I 2 #&~i'eø �äH 3 } ÷. ¡ !,+ä 2�3g4 X H&I 2 �C H(I 2 #(~É' � ~ ® ¨

(1.2)

where� 4 is anon-negativeinteger. Thisexpansionis similar to theFourierexpan-

sion (1.1): It is a linearcombinationof a setof basisfunctions,andthewaveletcoefficientsaregivenby

G } ÷ I 2 � å +4 �¤#(~i' �; } ÷ I 2 #&~i' X ~

X H&I 2 � å +4 �¤#(~i' �C H(I 2 #(~É' X ~

Oneimmediatedifferencewith respectto the Fourier expansionis the fact thatnow we have two typesof basisfunctionsandthatbothareindexedby two inte-gers.The

�; } ÷ I 2 arecalledscalingfunctionsandthe�C H(I 2 arecalledwavelets.Both

have compactsupportsuchthat�; H&I 2 #&~i' � �C H&I 2 #&~i' � °for

~úù®êû �) H � �$øýü) HÿþWecall thescaleparameterbecauseit scalesthewidth of thesupport,and

�the

shift parameter becauseit translatesthesupportinterval. Therearegenerallynoexplicit formulasfor

�; H&I 2 and�C H&I 2 but their functionvaluesarecomputableandso

aretheabovecoefficients.Thescalingfunctioncoefficient G } ÷ I 2 canbeinterpretedasa local weightedaverageof

�in the region where

�; } ÷ I 2 is non-zero.On theotherhand,thewaveletcoefficients X H&I 2 representtheoppositeproperty, namelythedetailsof

�thatarelost in theweightedaverage.

In practice,thewaveletexpansion(like theFourierexpansion)mustbe trun-catedatsomefinestscalewhichwedenote

� � M : Thetruncatedwaveletexpansionis . õá÷ !,+ä 2ñ354 G } ÷ I 2 �; } ÷ I 2 #(~i'eø } !,+äH 3 } ÷

. ¡ !,+ä 2�354 X H&I 2 �C H&I 2 #&~i'

1.2Waveletexpansion 7

0 64 128 256 512 1024−6

−4

−2

0

2

4

6

Figure 1.4: Waveletcoefficientsof è .

andthewaveletcoefficientsorderedas�� G } ÷ I 2�� . õá÷ !,+2ñ354 � �� X H(I 2�� . ¡ !,+2�354 � } !,+H 3 } ÷��areshown in Figure1.4. Thewaveletexpansion(1.2) canbeunderstoodasfol-lows: The first sumis a coarserepresentationof

�, where

�hasbeenreplaced

by a linearcombinationof ) } ÷ translationsof thescalingfunction�; } ÷ I 4 . Thiscor-

respondsto a Fourier expansionwhereonly low frequenciesare retained. Theremainingtermsarerefinements.For each a layerrepresentedby ) H translationsof thewavelet

�C H(I 4 is addedto obtainasuccessively moredetailedapproximationof�. It is convenientto definetheapproximationspaces

�¢ H � span� �; H&I 2 � . ¡��2ñ354�¦ H � span� �C H&I 2 � . ¡��2�354

Thesespacesarerelatedsuchthat

�¢ } � �¢ } ÷� �¦ } ÷� �� ¦ } !,+The coarseapproximationof

�belongsto the space

�¢ } ÷ and the successive re-finementsare in the spaces

�¦ H for � � 4 �� 4 ø M � î7îÌî �� M . Together, all ofthesecontributionsconstitutea refinedapproximationof

�. Figure1.5shows the

scalingfunctionsandwaveletscorrespondingto�¢ . � �¦ .

and�¦ ¾

.

8 Motivation

Scalingfunctions in �� : �� "!$#&%'�)(*��+��,

Waveletsin �´ �: �- �� ñ�.��/!0#1%��2(*��+��3,

Waveletsin �´54:- 4 � � �� 6!$#&%'�)(*��727�7"��8

Figure 1.5: Therearefour scalingfunctionsin 9: . andfour waveletsin 9; .but eight

morelocalizedwaveletsin 9; ¾.

1.2Waveletexpansion 9

0 1

V6

W6

W7

W8

W9

V10

~

~

~

~

~

~

Figure 1.6: The top graphis the sumof its projectionsonto a coarsespace 9:2< andasequenceof finerspaces 9;=<

– 9;=>.

Figure1.6showsthewaveletdecompositionof�

organizedaccordingto scale:Eachgraphis a projectionof

�ontooneof theapproximationspacesmentioned

above. Thebottomgraphis thecoarseapproximationof�

in�¢ < . Thoselabeled�¦ <

to�¦ >

aresuccessive refinements.Adding theseprojectionsyields thegraphlabeled

�¢ + 4 .Figure1.4 andFigure1.6 suggestthat many of the wavelet coefficientsare

zero.However, at all scalestherearesomenon-zerocoefficients,andthey revealthepositionwhere

�is discontinuous.If, asin theFouriercase,weretainonly theMOô largestwaveletcoefficients,we obtaintheapproximationshown in Figure1.7.

Becauseof theway waveletswork, theapproximationerroris muchsmallerthanthatof thetruncatedFourierexpansionand,very significantly, is highly localizedat thepointof discontinuity.

10 Motivation

0 1−0.6

0

0.6

x

Figure1.7: A function è anda truncatedwaveletexpansionwith only 17 terms

1.2.1 Summary

Therearethreeimportantfactsto noteaboutthewaveletapproximation:

1. Thegoodresolutionof thediscontinuityisaconsequenceof thelargewaveletcoefficientsappearingat thefinescales.Thelocalhighfrequency contentatthediscontinuityis capturedmuchbetterthanwith theFourierexpansion.

2. The fact that the error is restrictedto a smallneighborhoodof thediscon-tinuity is a resultof the “locality” of wavelets. The behavior of

�at one

locationaffectsonly thecoefficientsof waveletscloseto thatlocation.

3. Mostof thelinearpartof�

is representedexactly. In Figure1.6onecanseethatthelinearpartof

�is approximatedexactlyevenin thecoarsestapprox-

imationspace�¢ < whereonly a few scalingfunctionsareused.Therefore,

nowaveletsareneededto addfurtherdetailsto thesepartsof�.

Theobservationmadeinü

is a manifestationof a propertycalledvanishingmo-mentswhichmeansthatthescalingfunctionscanlocally representlow orderpoly-nomialsexactly. Thispropertyis crucialto thesuccessof waveletapproximationsandit is describedin detailin Sections2.1.5and2.1.6.

TheMatlabfunctionwavecompare conductsthis comparisonexperimentand✤thefunctionbasisdemo generatesthebasisfunctionsshown in Figure1.5.

Chapter 2

Multir esolutionanalysis

2.1 Waveletson the real line

A naturalframeworkfor wavelettheoryismultir esolutionanalysis(MRA) whichis amathematicalconstructionthatcharacterizeswaveletsin ageneralway. MRAyieldsfundamentalinsightsinto wavelettheoryandleadsto importantalgorithmsaswell. Thegoalof MRA is to expressanarbitraryfunction

� ® � . # ¨ 'atvarious

levelsof detail.MRA is characterizedby thefollowing axioms:� ° �@? �� ? ¢ !,+ ? ¢ 4A? ¢ + ? �� ? � . # ¨ ' # � '�BH 3 ! � ¢ H � � . # ¨ ' # = '� ; #&~��x' � 2 ` C is anorthonormalbasisfor ¢ 4 # G '� ® ¢ HED �¤# ) � ' ® ¢ H ò + # X '

(2.1)

This describesa sequenceof nestedapproximationspaces¢ H in� . # ¨ '

suchthattheclosureof theirunionequals

� . # ¨ '. Projectionsof a function

� ® � . # ¨ 'onto ¢ H areapproximationsto

�whichconvergeto

�asGF H . Furthermore,the

space¢ 4 hasanorthonormalbasisconsistingof integral translationsof a certainfunction ; . Finally, the spacesarerelatedby the requirementthat a function

�movesfrom ¢ H to ¢ H òx+ whenrescaledby ) . From(2.1c)wehavethenormalization(in the

� .-norm)

Ò ; Ò .�IEJ å �! � d ; #&~i' d . X ~LK +(-/. � M

12 Multiresolutionanalysis

andit is alsorequiredthat ; hasunit area[JS94, p. 383], [Dau92,p. 175], i.e.å �! � ; #(~É' X ~ � M (2.2)

Remark 2.1 A fifth axiomis oftenaddedto (2.1),namely�MH 3 ! � ¢ H�� ° �However, this is not necessaryas it follows from the other four axiomsin (2.1)[HW96].

Remark 2.2 The nestinggiven in (2.1a) is also usedby [SN96,HW96,JS94,Str94] and manyothers. However, someauthorse.g. [Dau92, Bey93, BK97,Kai94] usethereverseorderingof thesubspaces,making� ° �@? �N�� ? ¢ + ? ¢ 4�? ¢ !,+ ? �� ? � . # ¨ '2.1.1 The detail spacesO5PGiven thenestedsubspacesin (2.1),we define ¦ H to be theorthogonalcomple-mentof ¢ H in ¢ H òx+ , i.e. ¢ H ³ ¦ H and¢ H òx+ � ¢ H ¦ H (2.3)

Considernow two spaces¢ } ÷ and ¢ } , where�RQ � 4 . Applying (2.3) recur-

sively we find that

¢ } � ¢ } ÷� 1S } !,+TH 3 } ÷ ¦ HVU (2.4)

Thusany functionin ¢ } canbeexpressedasa linearcombinationof functionsin ¢ } ÷ and ¦ H � � � 4 �� 4 ø M � î7î7î �� M ; henceit canbeanalyzedseparatelyatdifferentscales.Multiresolutionanalysishasreceivedits namefrom this separa-tion of scales.

Continuingthedecompositionin (2.4) for� 4 F � H and

� F H yields inthelimits �TH 3 ! � ¦ H�� . # ¨ 'It followsthatall ¦ H aremutuallyorthogonal.

2.1Waveletson therealline 13

Remark 2.3 ¦ H canbechosensuch that it is not orthogonalto ¢ H . In that caseMRAwill leadto theso-calledbi-orthogonalwavelets[JS94]. Wewill notaddressthis point further but only mentionthat bi-orthogonalwaveletsare more flexiblethanorthogonalwavelets.We referto [SN96] or [Dau92] for details.

2.1.2 Basicscalingfunction and basicwavelet

Sincethe set� ; #&~ ��x' � 2 ` C is an orthonormalbasisfor ¢ 4 by axiom (2.1c) it

followsby repeatedapplicationof axiom# ) î M X ' that� ; # ) H ~>��x' � 2 ` C (2.5)

is anorthogonalbasisfor ¢ H . Notethat (2.5) is thefunction ; # ) H ~i' translatedby� N ) H , i.e. it becomesnarrowerandtranslationsgetsmalleras grows. Sincethesquarednormof oneof thesebasisfunctionsis

å �! � UU ; # ) H ~��x' UU . X ~ �*) ! H å �! � d ; # V ' d . X@V ��) ! H Ò ; Ò .. ��) ! Hit followsthat � ) H -/. ; # ) H ~�� ' � 2 `�C is anorthonormalbasisfor ¢ HSimilarly, it is shown in [Dau92, p. 135] that thereexists a function C #&~i' suchthat � ) H -:. C # ) H ~��x' � 2 `�C is anorthonormalbasisfor ¦ HWecall ; thebasicscalingfunction and C thebasicwavelet1. It is generallynotpossibleto expresseitherof themexplicitly, but,asweshallsee,thereareefficientandelegantwaysof workingwith them,regardless.It is convenientto introducethenotations ; H(I 2 #(~É' � ) H -/. ; # ) H ~>��x'C H(I 2 #(~É' � ) H -/. C # ) H ~>��x' (2.6)

and ; 2 #&~i' � ; 4 I 2 #&~i'C 2 #&~i' � C 4 I 2 #&~i' (2.7)

1In theliterature W is oftenreferredto asthemother wavelet.


We will usethe long and short forms interchangeablydependingon the givencontext.

SinceC H(I 2 ® ¦ H it followsimmediatelythat C H&I 2 is orthogonalto ; H&I 2 because; H&I 2 ® ¢ H and ¢ H|³ ¦ H . Also, becauseall ¦ H aremutuallyorthogonal,it followsthatthewaveletsareorthogonalacrossscales.Therefore,wehavetheorthogonal-ity relations å �! � ; H(I 2 #&~i' ; H&I À #(~i' X ~ � Ã 2 I À (2.8)

å �! � C 8 I 2 #&~i' C H&I À #(~i' X ~ � Ã 8 I H Ã 2 I À (2.9)

å �! � ; 8 I 2 #&~i' C H&I À #(~i' X ~ � ° � =X�ª (2.10)

whereª � � � �ZY ® »and

Ã 2 I À is theKr oneckerdelta definedasÃ 2 I À � é ° �úù� YM � � Y2.1.3 Expansionsof a function in []\A function

� ® ¢ } canbe expandedin variousways. For example,thereis thepurescalingfunctionexpansion

�¤#(~i' � �äÀ 3 ! � G } I À ; } I À #&~i' � ~ ® ¨(2.11)

where G } I À � å �! � ��#&~i' ; } I À #(~i' X ~ (2.12)

For any� 4 ë �

thereis alsothewaveletexpansion

�¤#(~i' � �äÀ 3 ! � G } ÷ I À ; } ÷ I À #&~i'eø } !,+äH 3 } ÷�äÀ 3 ! � X H(I À C H&I À #&~i' � ~ ® ¨

(2.13)

where G } ÷ I À � å �! � �¤#&~i' ; } ÷ I À #(~i' X ~X H(I À � å �! � �¤#&~i' C H&I À #&~i' X ~ (2.14)


Notethatthechoice� 4 � �

in (2.13)yields(2.11)asa specialcase.We defineÆ � � � � 4 (2.15)

anddenoteÆ

the depth of the wavelet expansion. From the orthonormalityofscalingfunctionsandwaveletswefind that

Ò � Ò .. � �ä2�3 ! � d G } I 2 d . � �ä2�3 ! � d G } ÷ I 2 d . ø } !,+äH 3 } ÷ �ä2ñ3 ! � d X H(I 2 d .which is Parseval’s equationfor wavelets.

Definition 2.1 LetPÉ �¡

andPe£¤¡

denotetheoperatorsthatprojectany� ® � . # ¨ '

orthogonallyonto ¢ H and ¦ H , respectively. Then

#/Pe �¡��Ý'�#(~i' � �äÀ 3 ! � G�H(I À ; H&I À #&~i'#/Pe£¤¡ö�Ý'�#(~i' � �äÀ 3 ! � X H&I À C H&I À #(~É'

where G�H(I À � å �! � �¤#(~É' ; H(I À #(~i' X ~X H(I À � å �! � �¤#(~É' C H&I À #&~i' X ~

and

PÉ õ � � Pe õá÷ �\ø } ! +äH 3 } ÷ PÉ£¤¡¥�

2.1.4 Dilation equationand waveletequation

Since ¢ 4 ? ¢ + , any functionin ¢ 4 canbeexpandedin termsof basisfunctionsof¢ + . In particular, ; #(~i' � ; 4 I 4 #&~i' ® ¢ 4 so

; #&~i' � �ä2�3 ! � � 2 ; + I 2 #(~i' �5^ ) �ä2ñ3 ! � � 2 ; # ) ~��x'


where � 2 ��å �! � ; #&~i' ; + I 2 #&~i' X ~ (2.16)

For compactlysupportedscalingfunctionsonly finitely many � 2 will benonzeroandwe have [Dau92, p. 194]

; #(~i' �5^ ) 0 ! +ä 2ñ354 � 2 ; # ) ~��x'(2.17)

Equation(2.17)is fundamentalfor wavelettheoryandit is known asthedilationequation.

fis anevenpositiveintegercalledthewaveletgenusandthenumbers� 4 � � + � î7î7î � � 0 !,+ arecalledfilter coefficients. The scalingfunction is uniquely

characterized(up to aconstant)by thesecoefficients.In analogyto (2.17)we canwrite a relation for the basicwavelet C . SinceC ® ¦ 4 and ¦ 4 ? ¢ + wecanexpandC as

C #(~i' �_^ ) 0 ! +ä 2ñ354 = 2 ; # ) ~��x'(2.18)

wherethefilter coefficientsare

= 2 ��å �! � C #(~É' ; + I 2 #(~i' X ~ (2.19)

We call (2.18)thewaveletequation.Although the filter coefficients � 2 and = 2 areformally definedby (2.16)and

(2.19),they arenot normallycomputedthatway becausewe do not know ; andC explicitly. However, they canfoundindirectly from propertiesof ; and C , see[SN96, p. 164–173]and[Dau92,p. 195] for details.

TheMatlabfunctiondaubfilt(D) returnsa vectorcontainingthefilter coef-✤ficients � 4 � � + � î7îÌî � � 0 !,+ .It turnsout that = 2 canbeexpressedin termsof � 2 asfollows:

Theorem2.1

= 2 � #�� M ' 2 � 0 !,+/! 2 � � � ° � M � î7îÌî � f � M


Proof: It followsfrom (2.10)thatT �! � ; #&~i' C #&~i' X ~ � °

. Using(2.17)and(2.18)wethenhave

å �! � ; #&~i' C #&~i' X ~ � ) å �! � 0 !,+ä 2ñ354 � 2 ; # ) ~>��x' 0 !,+ä À 354 = À ; # ) ~�� Y ' X ~� 0 !,+ä 2�354 0

!,+ä À 354 � 2 = À å �! � ; # V �� ' ; # V � Y ' X@V` a�b c� Ã 2 I À� 0 !,+ä 2�354 � 2 = 2 � °

Thisrelationis fulfilled if either � 2 � °or = 2 � °

for all�, thetrivial solutions,

or if = 2 � #�� M ' 2 � à ! 2 where ã is anoddintegerprovidedthatwe set � à ! 2 � °for ã ��úù® ¯ ° � f�� M¥² . In thelattercasetheterms�� = � will cancelwith theterms� à ! � = à ! � for � � ° � M � î7î7î � # ã ø M ' N ) � M . An obviouschoiceis ã � f � M . dTheMatlabfunction low2hi computes

� = 2 � 0 !,+2�354 from� � 2 � 0 ! +2ñ354 . ✤

Oneimportantconsequenceof (2.17) and(2.18) is that supp# ; ' � supp

# C ' �¯ ° � f � M�² (seee.g.[Dau92,p. 176] or [SN96, p. 185]). It follows immediatelythat

supp# ; H(I À ' � supp

# C H&I À ' �5e H(I À (2.20)

where e�H(I À � û Y) H � Y øBf � M) H þ (2.21)

Remark 2.4 Theformulationof thedilation equationis not thesamethroughouttheliterature. We haveidentifiedthreeversions:

1. ; #(~É' � � 2 � 2 ; # ) ~�� '2. ; #(~É' � ^ )�� 2 � 2 ; # ) ~>��x'3. ; #(~É' ��) � 2 � 2 ; # ) ~>��x'


Thefirst is usedby e.g. [WA94, Str94], the secondby e.g. [Dau92, Bey93,Kai94], and the third by e.g. [HW96, JS94]. We havechosenthesecondformu-lation, partly becauseit comesdirectly fromtheMRAexpansionof ; in termsof; + I 2 but alsobecauseit leadsto orthonormalityof thewavelettransformmatrices,seeSection3.3.2.

2.1.5 Filter coefficients

In this sectionwe will usepropertiesof ; and C to derive a numberof relationssatisfiedby thefilter coefficients.

Orthonormality property

Using the dilation equation(2.17) we can transformthe orthonormalityof thetranslatesof ; , (2.1c)into a conditionon thefilter coefficients � 2 . From(2.8)wehave theorthonormalityproperty

Ã 4 I Þ � å �! � ; #&~i' ; #&~�� Û ' X ~� å �! � S ^ ) 0 !,+ä 2�3g4 � 2 ; # ) ~>��x' U S ^ ) 0 !,+ä À 3g4 � À ; # ) ~>� ) Û � Y ' U X ~� 0 !,+ä 2�354 0

! +ä À 3g4 � 2 � À å �! � ; # V ' ; # V ø�� ) Û � Y ' X@V � V �*) ~�� 0 !,+ä 2�354 0

! +ä À 3g4 � 2 � À Ã 2 !g. Þ I À� 2gf j Þ lä2�3 2 + j Þ l � 2 � 2 !g. Þ � Û ® »

where� + # Û ' � Z\[^] #(° � ) Û ' and

� . # Û ' � Z=hji #:f � M � f � M ø ) Û ' . Althoughthis holdsfor all

Û ® », it will only yield

f N ) distinctequationscorrespondingtoÛ � ° � M � îÌî7î � f N ) � M becausethesumequalszerotrivially for

Û X f N ) asthereis nooverlapof thenonzero� 2 s. Hencewe have2gf j Þ lä2�3 2 + j Þ l � 2 � 2 !g. Þ � Ã 4 I Þ � Û � ° � M � î7î7î � f N ) � M (2.22)


Similarly, it followsfrom Theorem2.1that2gf j Þ lä2�3,2 + j Þ l = 2 = 2 ! . Þ � Ã 4 I Þ � Û � ° � M � îÌî7î � f N ) � M (2.23)

Conservationof area

RecallthatTL�! � ; #(~i' X ~ � M . Integrationof bothsidesof (2.17)thengives

å �! � ; #&~i' X ~ � ^ ) 0 !,+ä 2�3g4 � 2 å �! � ; # ) ~��x' X ~ � M^ ) 0!,+ä 2�354 � 2 å �! � ; # V ' X@V

or

0 !,+ä 2�3g4 � 2 � ^ ) (2.24)

Thename“conservationof area”is suggestedby Newland[New93, p. 308].

Property of vanishingmoments

Anotherimportantpropertyof thescalingfunctionis its ability to representpoly-nomialsexactlyup to somedegree

P1� M . More precisely, it is requiredthat

~ � � �ä2�3 ! � � �2 ; #&~>��x' � ~ ® ¨ � � � ° � M � îÌî7î � P1� M (2.25)

where � �2 � å �! � ~ � ; #&~��x' X ~ � � ® » � � � ° � M � î7îÌî � Pr� M (2.26)

Wedenote� �2 the � th momentof ; #(~z�Ó�x'

andit canbecomputedby aprocedurewhich is describedin AppendixA.

Equation(2.25) canbe translatedinto a condition involving the wavelet bytakingtheinnerproductwith C #&~i' . Thisyields

å �! � ~ � C #&~i' X ~ � �ä2ñ3 ! � � �2 å �! � ; #&~>��x' C #&~i' X ~ � °


since C and ; are orthonormal. Hence,we have the propertyofP

vanishingmoments: å �! � ~ � C #&~i' X ~ � ° � ~ ® ¨ � � � ° � M � î7î7î � Pr� M (2.27)

The propertyof vanishingmomentscan be expressedin termsof the filtercoefficientsasfollows. Substitutingthewaveletequation(2.18)into (2.27)yields° � å �! � ~ � C #&~i' X ~

� ^ ) 0 !,+ä 2�354 = 2 å �! � ~ � ; # ) ~>��x' X ~� ^ )) � òx+ 0

!,+ä 2ñ354 = 2 å �! � # V ø��x' � ; # V ' X@V � V ��) ~�� ^ )) � òx+ 0

!,+ä 2ñ354 = 2�ä Þ 3g4 J � Û K � Þ å �! � V � ! Þ ; # V ' X@V

� ^ )) � òx+ �ä Þ 354 J �Û K � � ! Þ4 0 !,+ä 2ñ354 = 2 � Þ (2.28)

wherewe haveused(2.26)andthebinomialformula# V øý�x' � � �äÞ 354 J � Û K V � ! Þ � ÞFor � � °

relation(2.28)becomes� 0 !,+2�354u= 2 � °, andusinginductionon � we

obtainP

momentconditionson thefilter coefficients,namely0 !,+ä 2�354 = 2 � � � 0 !,+ä 2�354 #�� M ' 2 � 0 ! +:! 2 � � � ° � � � ° � M � î7î7î � P�� MThisexpressioncanbesimplifiedfurtherby thechangeof variables

Y � f � M �R� .Then ° � 0 !,+ä À 354 #�� M ' 0 !,+/! À � À #:f � M � Y ' �andusingthebinomialformulaagain,we arriveat0 !,+ä À 354 #ñ� M ' À � À Y � � ° � � � ° � M � îÌî7î � P1� M (2.29)


Other properties

Theconditions(2.22),(2.24)and(2.29)compriseasystemoff N ) ø M ø�P

equa-tionsfor the

ffilter coefficients � 2 � � � ° � M � î7î7î � f � M . However, it turnsout

that oneof the conditionsis redundant.For example(2.29)with � � °canbe

obtainedfrom theothers,see[New93, p. 320]. This leavesa total off N ) ø P

equationsfor thef

filter coefficients.Not surprisingly, it canbeshown [Dau92,p. 194] thatthehighestnumberof vanishingmomentsfor this typeof waveletisP � f N )yieldinga totalof

fequationsthatmustbefulfilled.

This systemcanbe usedto determinefilter coefficients for compactlysup-portedwaveletsor usedto validatecoefficientsobtainedotherwise.

The Matlab function filttest checksif a vectorof filter coefficientsfulfils ✤(2.22),(2.24)and(2.29)with

P � f N ) .Finally we notetwo otherpropertiesof thefilter coefficients.

Theorem2.2 0 -/.�!,+ä 2�3g4 � . 2 � 0 -:.�!,+ä 2�354 � . 2 òx+ � M^ )Proof: Adding (2.24)and(2.29)with � � °

yields the first result. Subtractingthemyieldsthesecond. dTheorem2.3 0 -/.�!,+ä À 354 0 !g. À ! .ä Þ 354 � Þ � Þ ò . À òx+ � M)Proof: Using(2.29)twicewe write° � S 0 !,+ä 2�354 #�� M ' 2 � 2 U S 0 !,+ä À 354 #ñ� M ' ! À � À U � 0 !,+ä 2ñ354 0

!,+ä À 354 #�� M ' 2 ! À � 2 � À� 0 !,+ä 2�354 � .2` aNb c3 +

ø 0 !,+ä 2�3542 !,+ä À 354 #�� M ' 2 ! À � 2 � À ø 0 !,+ä 2�354 0 ! +äÀ 3,2 òx+ #�� M ' À ! 2 � 2 � À


Using(2.22)withÛ � °

in thefirst sumandrearrangingtheothersumsyields

° � M ø 0 !,+ä � 3 + 0!,+/! �ä À 354 #ñ� M ' � � À ò �Ì� À ø 0 !,+ä � 3 + 0

!,+/! �ä 2�354 #�� M ' � � 2 � 2 ò �� M ø ) 0 !,+ä � 3 + 0

!,+/! �ä Þ 3g4 #�� M ' � � Þ � Þ ò �All sumswhere � is even vanishby (2.22),so we areleft with the “odd” terms(� ��) Y ø M )

° � M ø ) 0 -/.�!,+ä À 354 #�� M ' . À òx+ 0 !g. À !g.äÞ 354 � Þ � Þ ò,. À òx+� M � ) 0 -/.�!,+ä À 3g4 0 ! . À !g.äÞ 354 � Þ � Þ ò . À òx+

from which theresultfollows. dEquation(2.24)cannow bederiveddirectly from (2.22)and(2.29)andhave

thefollowing Corollaryof Theorem2.2.

Corollary 2.4 0 !,+ä 2ñ354 �5^ )Proof: By amanipulationsimilar to thatof Theorem2.2wewriteS 0 !,+ä 2�354 � 2 U S 0 !,+ä À 354 � À U � 0 !,+ä 2�354 0

!,+ä À 354 � 2 � À� M ø ) 0 -/.�!,+ä À 354 0 !g. À ! .ä Þ 354 � Þ � Þ ò . À òx+� )

whereTheorem2.2wasusedfor thelastequation.Takingthesquarerootoneachsideyieldstheresult. d


2.1.6 Decayof waveletcoefficients

TheP

vanishingmomentshave an importantconsequencefor thewaveletcoef-ficients X H(I 2 (2.14): They decreaserapidly for a smoothfunction. Furthermore,if a functionhasa discontinuityin oneof its derivativesthenthewaveletcoeffi-cientswill decreaseslowly onlycloseto thatdiscontinuityandmaintainfastdecaywherethefunctionis smooth.This propertymakeswaveletsparticularlysuitablefor representingpiecewisesmoothfunctions. Thedecayof waveletcoefficientsisexpressedin thefollowing theorem:

Theorem2.5 LetP � f N ) be thenumberof vanishingmomentsfor a waveletC H(I 2 and let

� ® J K # ¨ '. Thenthewaveletcoefficientsgivenin (2.14)decayas

follows:

d X H&I 2 d ë�J K ) ! H j K ò �f l ZÓ[O]9 `�k ¡ml n UU � j K l #(%�' UUwhere

JLKis a constantindependentof , � , and

�and eñH&I 2 � supp

� C H(I 2 � �¯ � N ) H � #&�$øBf � M ' N ) H ² .Proof: For

~ ®oe H&I 2 we write theTaylorexpansionfor�

around~ � � N ) H .

�¤#(~i' � S K !,+ä � 354 � j � l t � N ) H { #(~��2. ¡ ' �� Q U ø�� j K l #&%�' #&~�� 2. ¡ ' KP$Q (2.30)

where% ® ¯±� N ) H � ~ ² .

Inserting(2.30) into (2.14)andrestrictingthe integral to the supportof C H&I 2yields

X H&I 2 � å k ¡pl n �¤#&~i' C H&I 2 #&~i' X ~� S K !,+ä � 3g4 � j � l t � N ) H { M� Q å k ¡pl n J ~>� �) H K � C H&I 2 #&~i' X ~ U ø

MPRQ å k ¡pl n � j K l #(%�' J ~>� �) H K K C H(I 2 #(~É' X ~Recallthat

%dependson

~, so

� j K l #&%�'is not constantandmustremainunderthe

lastintegral sign.


Considerthe integralswhere � � ° � M � î7î7î � P � M . Using (2.21)andlettingV ��) H ~��weobtain

å j 2 ò 0 !,+ l -/. ¡2 -/. ¡ J ~�� ) H K � ) H -/. C # ) H ~>��x' X ~� ) H -/. å 0 !,+4 q V) Hsr � C # V ' ) ! H X@V� ) ! H j � òx+(-:. l å 0 !,+4 V � C # V ' X@V� ° � � � ° � M � î7î7î � P1� Mbecauseof the

Pvanishingmoments(2.27).Therefore,thewaveletcoefficient is

determinedfrom theremaindertermalone.Hence,

d X H&I 2 d � MPRQ UUUUU å k ¡pl n � j K l #&%�' J ~�� ) H K K ) H -/. C # ) H ~�� ' X ~ UUUUUë MPRQ ZÓ[O]9 `�k ¡pl n UU � j K l #(%�' UU å k ¡pl n UUUUU J ~�� ) H K K ) H -/. C # ) H ~>��x' UUUUU X ~� ) ! H j K òx+(-:. l MPRQ ZÓ[O]9 `�k ¡pl n UU � j K l #&%�' UU å 0 !,+4 UU V K C # V ' UU X@VDefining

J K � MP$Q å 0 !,+4 UU V K C # V ' UU,X@Vweobtainthedesiredinequality. dFromTheorem2.5we seethatif

�behaveslike a polynomialof degreelessthanP

in the interval e�H(I 2 then� j K l I °

and the correspondingwavelet coefficientX H&I 2 is zero. If� j K l

is differentfrom zero,coefficientswill decayexponentiallywith respectto the scaleparameter . If

�hasa discontinuityin a derivative

of order lessthan or equaltoP

, then Theorem2.5 doesnot hold for waveletcoefficients locatedat the discontinuity2. However, coefficientsaway from thediscontinuityarenotaffected.Thecoefficientsin awaveletexpansionthusreflectlocal propertiesof

�andisolateddiscontinuitiesdonotruin theconvergenceaway

fromthediscontinuities.Thismeansthatfunctionsthatarepiecewisesmoothhavemany smallwaveletcoefficientsin their expansionsandmaythusberepresented

2Thereare tvuxw affectedwaveletcoefficientsateachlevel

2.2WaveletsandtheFouriertransform 25

well by relatively few wavelet coefficients. This is the principlebehindwaveletbaseddatacompressionand one of the reasonswhy waveletsare so useful ine.g.signalprocessingapplications.We saw anexampleof this in Chapter1 andSection4.4givesanotherexample.Theconsequencesof Theorem2.5with respectto approximationerrorsof waveletexpansionsaretreatedin Chapter4.

2.2 Waveletsand the Fourier transform

It is oftenusefulto considerthe behavior of theFourier transformof a functionratherthanthefunctionitself. Also in caseit givesriseto someintriguingrelationsandinsightsaboutthebasicscalingfunctionandthebasicwavelet.Wedefinethe(continuous)Fouriertransformas�; #(%�' � å �! � ; #&~i' 6 !@8 9(� X ~ � % ® ¨Therequirementthat ; hasunit area(2.2) immediatelytranslatesinto�; #&°@' �å �! � ; #&~i' X ~ � M (2.31)

Ourpointof departurefor expressing�; atothervaluesof

%is thedilationequation

(2.17).TakingtheFouriertransformonbothsidesyields�; #&%�' � ^ ) 0 ! +ä 2�3g4 � 2 å �! � ; # ) ~��x' 6 !�8 9&� X ~� ^ ) 0 ! +ä 2�3g4 � 2 å �! � ; # V ' 6 !@8 9 j _ ò 2 l -/. X@V N )� M^ ) 0

! +ä 2�3g4 � 276 !@8 2/9 -/. å �! � ; # V ' 6 !@8 j 9 -/. l _ X@V� " J %) K �; J %) K (2.32)

where

"$#&%�' � M^ ) 0 !,+ä 2�354 � 276 !�8 2:9 � % ® ¨(2.33)

"$#(%�'is a )�y -periodic function with someinterestingpropertieswhich can be

deriveddirectly from theconditionson the filter coefficientsestablishedin Sec-tion 2.1.


Lemma 2.6 If C hasP

vanishingmomentsthen"$#(° ' � MX �X % � "R#&%�' d 9/3 Ë � ° � � � ° � M � î7î7î � P1� MProof: Let

% � °in (2.33).Using(2.24)wefind immediately

"$#(° ' � M^ ) 0 ! +ä 2ñ354 � 2 � ^ )^ ) � MNow let

% �zy . Thenby (2.29)

X �X % � "R#&%�' d 9/3 Ë � M^ ) 0!,+ä 2�354 #ñ� ª �x' � � 2�6 !@8 2 Ë

� M^ ) #ñ� ª ' � 0!,+ä 2�354 � � � 2 #�� M ' 2� ° � � � ° � M � î7î7î � P1� M d

Putting� � °in Lemma2.6andusingthe )�y periodicityof

"$#&%�'we obtain

Corollary 2.7

"R# Û y ' �êé M Ûeven° Ûodd

Equation(2.32)canberepeatedfor�; #&% N ) ' yielding�; #&%�' � " J %) K " J %ó K �; J %ó K

After�

suchstepswe have

�; #(%�' � �{H 3 + " J %) H K �; J %) � K


It follows from (2.33) and(2.24) that d "R#&%�' d ë M so the productconvergesfor� F H andwegettheexpression�; #(%�' � �{H 3 + " J %) H K �; #(° 'Using(2.31)wearriveat theproductformula

�; #(%�' � �{H 3 + " J %) H K � % ® ¨(2.34)

Lemma 2.8 �; # )3y Û ' � Ã 4 I Þ � Û ® »

Proof: The caseÛ � °

follows from (2.31). Therefore,letÛ ® »�| � ° � be

expressedin the form }�~��g� where �� and � �� with � odd, i.e.� ��~�� . Thenusing(2.34)weget�� }��~ ��¡ £¢¥¤ ��}� �§¦~ ��¡ ¢ ¤ ��©¨ �ª�� ¦~ ¢ � � � �ª�«� ¢ � � �¬ �ª�«�*®�®N® ¢ � �ª�«�*®�®�®~ ¯since¢ � �ª�«��~°¯ by Corollary2.7. ±A consequenceof Lemma2.8 is thefollowing basicpropertyof

�:

Theorem2.9 �²³ � ¬ � � �´ � �gµ·¶ }��~°� ¬ ��¸ ��¹ º=» ¯ ¹ µ ��¼


Proof: Let ½ � �gµ ��~ �²³ � ¬ � � �g´ � �µ¾¶ }'�We observe that

½ � �µ � is a � -periodic function. Henceit hasthe Fourier seriesexpansion ½ � �gµ ��~ �²¿V� ¬ �§À ¿�Á � �gÂ ¿ÄÃ ¹ µ ��¼ (2.35)

wheretheFouriercoefficients À ¿ aredefinedby

À ¿ ~ Å � ½ � �gµ � Á ¬s� �gÂ ¿�Ã�Æ µ~ Å � �²³ � ¬ � � �g´ � �µ¾¶ }'� Á ¬�� Â ¿ÄÃÇÆ µ~ �²³ � ¬ � Å ³ ¨ ³ � �g´ � �gÈ � Á ¬�� gÂ ¿ÄÉ Á � ��Â ¿ ³Ê Ë�Ì Í�¡ Æ È ¹ È ~ µ¾¶ }~ Å �¬ � � �´ � �gÈ � Á ¬�� gÂ ¿ÄÉ Æ È~ � ��¸ � Å �¬ � �«� � � È � Á ¬�� gÂ ¿ÄÉ Æ È~ � ��¸ � Å �¬ � �«�gÎ � Á ¬�� gÂ ¿ �ÐÏ�ÑÒ � ¬ � Æ Î ¹ Î ~�� È~ � ¬ ��¸ � �� 3��ÓL� ¬ � � ¹ ÓÔ�v�We know from Lemma2.8 that

�� Ó*�Õ~&Ö�� ´ ¿ . Therefore,sinceº×» ¯ by as-

sumption,

À ¿ ~�� ¬ ��¸ � �� Ó£� ¬ � ��~°� ¬ ��¸ � Ö�� ´ ¿sotheFourierseriescollapsesinto oneterm,namelythe“DC” term À � , i.e.½ � �gµ ��~ À �/~°� ¬ ��¸ �from which theresultfollows. ±


Theorem2.9 statesthat if � is a zeroof the function ¢ �Ø � thentheconstantfunctioncanbe representedby a linearcombinationof the translatesof

� �´ � �µ � ,whichagainis equivalentto thezerothvanishingmomentcondition(2.27).If thenumberof vanishingmomentsÙ is greaterthanone,thena similarargumentcanbeusedto show thefollowing moregeneralstatement[SN96, p. 230]

Theorem2.10 If � is a zero of ¢ �gØ � of multiplicity Ù , i.e. ifÆ�ÚÆ Ø Ú ¢ �gØ �6Û Ü � ÂÝ~ ¯ ¹ Þ ~�¯ ¹ � ¹ ®�®N® ¹ ÙzßÝ�then:

1. The integral translatesof�«�µ � can reproducepolynomialsof degree less

than Ù .

2. Thewaveletà �µ � has Ù vanishingmoments.

3.��'á ÚZâ � �3��}'��~°¯ for }ª�ã� , }0ä~Ý¯ and

Þ§å Ù .

4.�à á ÚZâ � ¯æ��~�¯ for

ÞÔå Ù .

2.2.1 The waveletequation

In thebeginningof thissectionweobtaineda relation(2.32)for thescalingfunc-tion in thefrequency domain.Using(2.18)wecanobtainananalogousexpressionfor

�à . �à �Ø ��~ ç �«è ¬ ² ¿Ð� ��é ¿ Å �¬ � �� µ ß0Ó¡� Á ¬s� Ü Ã Æ µ~ �ç � è ¬ ² ¿Ð� � é ¿�Á ¬�� ¿ Ü ¸ � Å �¬ � ��gÈ � Á ¬�� á Ü ¸ � â É Æ È~ ê ¤ Ø� ¦ �� ¤ Ø� ¦where ê �gØ ��~ �ç � è ¬ ² ¿V� � é ¿�Á ¬�� ¿ Ü (2.36)


UsingTheorem2.1we canexpressê �gØ � in termsof ¢ �gØ � .ê �gØ �ë~ �ç � è ¬ ² ¿Ð� � � ß·�� ¿�ì è ¬ ¬ ¿�Á ¬�� ¿ Ü~ �ç � è ¬ ² ¿Ð� � ì è ¬ ¬ ¿�Á ¬�� ¿ á Ü ¨ Â â~ �ç � è ¬ ² À � � ì À Á ¬�� á è ¬ ¬ À â á Ü ¨ Â â~ Á ¬�� á è ¬ â á Ü ¨ Â â �ç � è ¬ ² À � � ì À Á � À á Ü ¨ Â â~ Á ¬�� á è ¬ â á Ü ¨ Â â ¢ �gØ"¶ �«�This leadsto thewaveletequationin thefrequency domain.�à �gØ ��~ Á ¬�� á è ¬ â á Ü ¸ � ¨ Â â ¢ �Øsí � ¶ �«� ��·�Ø�í �æ� (2.37)

An immediateconsequenceof (2.37)is thefollowing lemma:

Lemma 2.11 �à �pî ��}��~�¯ ¹ }ï�o�Proof: Letting

Ø ~ î ��} in (2.37)yields�à �mî ��}'��~ Á ¬�� á è ¬ â á ��Â ³ ¨ Â â ¢ � �3��} ¶ �«� ��·� �3��}'�which is equalto zeroby Lemma2.8for }0ä~Ý¯ . For }�~�¯ we have�à � ¯��~�ß ¢ � �«�but this is alsozeroby Corollary2.7. ±2.2.2 Orthonormality in the fr equencydomain

Theinnerproductsof�

with its integral translatesalsohaveaninterestingformu-lation in thefrequency domain.By Plancherel’s identity [HW96, p. 4] the inner


productin thephysicaldomainequalstheinnerproductin thefrequency domain(exceptfor a factorof �3� ). Henceð ¿ ~ Å �¬ � ��gµ � ��gµ ß0Ó¡� Æ µ~ �� Å �¬ � ��gØ � ��gØ � Á ¬�� Ü ¿ Æ Ø~ �� Å �¬ �_ñññ

��'�Ø � ñññ� Á � Ü ¿òÆ Ø~ �� Å �gÂ� �²³ � ¬ � ñññ��gØ/¶ ��}'� ñññ

� Á � Ü ¿ Æ Ø (2.38)

Define ó �gØ ��~ �²³ � ¬ � ñññ��gØ/¶ ��}'� ñññ

� ¹ Ø ��¼ (2.39)

Thenwe seefrom (2.38)thatð ¿ is the Ó ’ th Fouriercoefficientof

ó �gØ � . Thusó �gØ ��~ �²¿V� ¬ � ð ¿�Á ¬�� ¿ Ü (2.40)

Since�

is orthogonalto its integral translationsweknow from (2.8) thatð ¿ ~õô � Ó=~�¯¯ Óoä~�¯hence(2.40)evaluatesto

ó �gØ ��~¥� ¹ Ø ��¼Thuswehave provedthefollowing lemma.

Lemma 2.12 Thetranslates��gµ ß0Ó¡� , Ó��v� are orthonormalif andonly if

ó �gØ ��ö÷�


We cannow translatetheconditionon

óinto a conditionon ¢ �gØ � . From(2.32)

and(2.39)we haveó � � Ø �ø~ �²³ � ¬ � ñññ

�� Ø/¶ ��}�� ñññ�

~ �²³ � ¬ � ñññ��gØ/¶ ��}�� ñññ

� Û ¢ �Ø�¶ ��}��Û �Splitting thesuminto two sumsaccordingto whether} is evenor oddandusingtheperiodicityof ¢ �gØ � yieldsó � � Ø �ø~ �²³ � ¬ � ñññ

��'�Ø/¶ �3��}'� ñññ� Û ¢ �gØ"¶ �3��}'��Û � ¶�²³ � ¬ � ñññ

��'�Ø/¶ � ¶ �3��}'� ñññ� Û ¢ �Ø�¶ � ¶ ��}'�NÛ �~ Û ¢ �gØ �NÛ � �²³ � ¬ � ñññ

��gØ/¶ ��}'� ñññ� ¶ Û ¢ �gØ"¶ �«��Û � �²³ � ¬ � ñññ

��gØ/¶ � ¶ ��}�� ñññ�~ Û ¢ �gØ �NÛ � ó �Ø � ¶ Û ¢ �Ø�¶ �«��Û � ó �gØ/¶ �«�

If

ó �gØ ��~÷� then Û ¢ �gØ ��Û � ¶ Û ¢ �Ø/¶ �«�NÛ � öù� andtheconverseis alsotrue[SN96,p. 205–206],[JS94,p. 386]. For this reasonwehave

Lemma 2.13ó �gØ ��ö÷� ú Û ¢ �Ø ��Û � ¶ Û ¢ �gØ"¶ �«�NÛ � ö÷�

Finally, Lemma2.12andLemma2.13yieldsthefollowing theorem:

Theorem2.14 Thetranslates�«�µ ßûÓ*� , ÓÔ�v� are orthonormalif andonly ifÛ ¢ �gØ �NÛ � ¶ Û ¢ �gØ�¶ �«��Û � ö÷�

2.2.3 Overview of conditions

Wesummarizenow variousformulationsof theorthonormalityproperty, theprop-ertyof vanishingmomentsandtheconservationof area.

2.3Periodizedwavelets 33

Orthonormality property:Å �¬ � ��gµ � ��gµ ß0Ó¡� Æ µ ~°Ö � ´ ¿ ¹ Ó��o�è ¬ ² ¿V� � ì ¿ ì ¿ ¨ � ³ ~zÖ � ´ ³ ¹ }ï�o�Û ¢ �Ø �NÛ � ¶ Û ¢ �gØ"¶ �«��Û � ö¥� ¹ Ø ��¼Property of vanishingmoments:

ForÞ ~Ý¯ ¹ � ¹ ®N®�® ¹ ÙÝßÝ� and Ù5~5ü í � wehaveÅ �¬ � à �gµ � µ Ú Æ µ ~°¯è ¬ ² ¿Ð� � � ßý�� ¿ ì ¿ Ó Ú ~�¯Æ�ÚÆ Ø Ú ¢ �Ø �AÛ Ü � Â ~°¯

Conservationof area:��gµ ��~ ç � è ¬ ² ¿V� � ì ¿ �� µ ß0Ó¡� ¹ µ �þ¼è ¬ ² ¿Ð� � ì ¿ ~ ç �¢ � ¯æ��~��2.3 Periodizedwavelets

So far our functionshave beendefinedon the entire real line, e.g.ð � ÿ � � ¼�� .

Thereareapplications,suchas the processingof audiosignals,wherethis is areasonablemodelbecauseaudiosignalscanbearbitrarilylongandthetotal lengthmaybeunknown until themomentwhentheaudiosignalstops.However, in mostpracticalapplicationssuchasimageprocessing,datafitting, orproblemsinvolvingdifferentialequations,thespacedomainis a finite interval. Many of thesecasescanbedealtwith by introducingperiodizedscalingfunctionsandwaveletswhichwedefineasfollows:


Definition 2.2 Let� �ûÿ � � ¼�� and à��$ÿ � � ¼Ô� bethebasicscalingfunctionand

the basic waveletfrom a multiresolutionanalysisas definedin (2.1). For anyº�¹ � �v� wedefinethe � -periodicscalingfunction�� g´ À �gµ ��~ �²³ � ¬ � � �g´ À �µ¾¶ }��~Ý� ��¸ � �²³ � ¬ � �«� � � �µ¾¶ }'��ß � � ¹ µ �o¼ (2.41)

andthe � -periodicwavelet�à �g´ À �gµ ��~ �²³ � ¬ � à �g´ À �µ¾¶ }��~�� ¸ � �²³ � ¬ � à � � � �µ¾¶ }'��ß � � ¹ µ ��¼ (2.42)

The � periodicitycanbeverifiedasfollows�� g´ À �µ¾¶ ��~ �²³ � ¬ � � �´ À �gµ¾¶ } ¶ � ��~ �²� � ¬ � � �g´ À �gµ¾¶�� ~ �� ´ À �gµ �andsimilarly

�à �´ À �µ·¶ ��]~ �à �g´ À �gµ � .2.3.1 Someimportant specialcases

1.�� ´ À �µ � is constantfor

º=» ¯ . To seethisnotethat(2.41)yields�� ´ À �gµ ��~ � ��¸ � �²³ � ¬ � �� µ¾¶ }=ß0� ¬ � � �V�~ � ��¸ � �²� � ¬ � �� µ¾¶�� V� ~ �� g´ � �µ � ¹ � �v�where

� ~1}�ß��æ¬ � � is an integer because�æ¬ � � is an integer. Hence,by(2.41)andTheorem2.9wehave

�� g´ � �gµ ��~�� ³ � ¬ � � �g´ � �gµ�¶ }��~°�æ¬ ��¸ � , so�� ´ À �gµ ��~�� ¬ ��¸ � ¹ ºx» ¯ ¹ � �o� ¹ µ �þ¼ (2.43)

2.�à �g´ À �µ �ª~ ¯ for

º » ßý� . To seethis we note first that, by an analysissimilar to theabove,

�à �g´ À �gµ �/~ �à �g´ � �µ � forºª» ¯ and

�à �´ � �µ � hasa Fourierexpansionof the form (2.35). Thesamemanipulationsasin Theorem2.9yield thefollowing Fouriercoefficientsfor

�à �g´ � �gµ � :À ¿ ~Ý� ¬ ��¸ � �à � �3��ÓL� ¬ � � ¹ ÓÔ�v� (2.44)


From Lemma2.11we have that�à �pî ��Ó¡�@~&¯ , ÓÝ� � . Hence À ¿ ~1¯ forº=» ß·� which meansthat�à �´ À �µ ��~z¯ ¹ º=» ß·� ¹ � �v� ¹ µ ��¼ (2.45)

Whenº ~�¯ in (2.44),onefinds from (2.37) that

�à � ��Ó¡� ä~�¯ for Ó odd,so

�à«� ´ ¿ �gµ � is neither ¯ nor anotherconstantfor suchvalueof Ó . This isasexpectedsincetherole of

�à«� ´ ¿ �µ � is to representthedetailsthatarelostwhenprojectinga functionfrom anapproximationat level � to level ¯ .

3.�� ´ À �µ � and

�à �g´ À �µ � areperiodicin the shift parameterÓ with period � � forº� ¯ . We will show this only for�à �g´ À sincetheproof is thesamefor

�� g´ À .Let

º� ¯ , Þ �o� and ¯ » � » � � ßÝ� ; then�à �g´ À ¨ � Ñ Ú �gµ ��~ �²� � ¬ � à �´ À ¨ � Ñ Ú �gµ@¶�� ~ � ��¸ � �²� � ¬ � à � � � �µ¾¶�� ß � ß0� � Þ �~ � ��¸ � �²� � ¬ � à � � � �µ¾¶�� ß Þ ��ß � �~ � ��¸ � �²³ � ¬ � à � � � �µ¾¶ }'��ß � �~ �²³ � ¬ � à �g´ À �gµ¾¶ }'�~ �à �g´ À �gµ � ¹ µ ��¼Hencethereareonly � � distinctperiodizedwavelets:� �à �g´ À� � Ñ ¬ À � � º�� ¯

4. Let � �� ü÷ß�� : Rewriting (2.42)asfollowsyieldsanalternative formula-tion of theperiodizationprocess:�à �´ À �gµ ��~Ý� ��¸ � �²³ � ¬ � à � � � µ·¶ � � }=ß � ��~ �²³ � ¬ � à �g´ À ¬ � Ñ ³ �gµ � (2.46)


Becauseà is compactlysupported,the supportsof the termsin this sumdo not overlapprovidedthat � � is sufficiently large. Let �s� bethesmallestintegersuchthat �� ü÷ßÝ� (2.47)

Thenwe seefrom (2.20) that forº � �� the width of � �g´ À is smallerthan� and(2.46) implies that

�à �g´ À doesnot wrap in sucha way that it overlapsitself. Consequently, theperiodizedscalingfunctionsandwaveletscanbedescribedfor

µ �� ¯ ¹ �� in termsof theirnon-periodiccounterparts:�à �´ À �gµ ��~ ô à �´ À �µ � ¹ µ �� g´ À�� ¯ ¹ ��à �´ À �µ¾¶ �� ¹ µ �� ¯ ¹ �� ¹ µ ä�� g´ ÀTheabove resultsaresummarizedin thefollowing theorem:

Theorem2.15 Let thebasicscalingfunction�

andwaveletà havesupport� ¯ ¹ ü ß×�� , andlet�� g´ À and

�à �g´ À bedefinedasin Definition2.2.Then� º=» ¯ ¹ � �v� ¹ µ ��¼ :�� g´ À �µ ��~ � ¬ ��¸ ��à �g´ À �µ ��~ ¯ ¹ º=» ß·�� º� ¯ ¹ µ �þ¼ : �� ´ À ¨ � Ñ Ú �gµ ��~ �� g´ À �µ ��à �´ À ¨ � Ñ Ú �gµ ��~ �à �g´ À �gµ �� º� �� ! #"%$�& � � ü÷ßÝ��' ¹ µ �� ¯ ¹ �� :�� g´ À �gµ ��~õô � �g´ À �gµ � ¹ µ �� ´ À� �g´ À �gµ@¶ �� ¹ µ ä�� ´ Àand �à �g´ À �gµ ��~õô à �´ À �gµ � ¹ µ �� g´ Àà �´ À �gµ·¶ � � ¹ µ ä�� g´ À


2.3.2 PeriodizedMRA in (*),+.-0/214365�7Many of thepropertiesof the non-periodicscalingfunctionsandwaveletscarryover to theperiodizedversionsrestrictedto theinterval � ¯ ¹ �� . Waveletorthonor-mality, for example,is preservedfor thescales� ¹gº8� ¯ :Å � �à � ´ ¿ �gµ � �à �´ À �gµ � Æ µ ~ Å � �²� � ¬ � à � ´ ¿ �gµ¾¶�� à �´ À �gµ � Æ µ~ �²� � ¬ � Å � ¨ � à � ´ ¿ �gÈ � �à �g´ À �È ß � � Æ È~ �²� � ¬ � Å � ¨ � à � ´ ¿ �gÈ � �à �g´ À �È � Æ È~ Å �¬ � à � ´ ¿ �È � �à �g´ À �È � Æ ÈUsing(2.46) for the secondfunctionandinvoking the orthogonalityrelationfornon-periodicwavelets(2.9)givesÅ � �à � ´ ¿ �gµ � �à �g´ À �µ � Æ µ ~ �²³ � ¬ � Å �¬ � à � ´ ¿ �gµ ��à �´ À ¬ � Ñ ³ �gµ � Æ µ ~zÖ � ´ � �²³ � ¬ � Ö ¿Z´ À ¬ � Ñ ³If ��~ º

then Ö � ´ � ~�� and Ö ¿Z´ À ¬ � Ñ ³ contributesonly when }þ~z¯ and Ó=~ � becauseÓ ¹ � �� ¯ ¹ � � ßÝ�� . Hence,Å � �à � ´ ¿ �gµ � �à �´ À �gµ � Æ µ ~�Ö � ´ � Ö ¿Z´ À (2.48)

asdesired.By asimilaranalysisonecanestablishtherelationsÅ � �� g´ ¿ �gµ � �� g´ À �gµ � Æ µ ~ Ö ¿ ´ À ¹ º � ¯Å � �� ´ ¿ �gµ � �à �g´ À �gµ � Æ µ ~ ¯ ¹ º � � � ¯The periodizedwaveletsandscalingfunctionsrestrictedto � ¯ ¹ �� generatea

multiresolutionanalysisof ÿ � � � ¯ ¹ ��m� analogousto that of ÿ � � ¼Ô� . The relevantsubspacesaregivenby

Definition 2.3 �9 � ~ span� �� g´ À ¹ µ �� ¯ ¹ �� Ñ ¬ À � ��: � ~ span� �à �g´ À ¹ µ �� ¯ ¹ �� mÑ ¬ À � �


It turnsout [Dau92, p. 305] thatthe�9 � arenestedasin thenon-periodicMRA,�9 �<; �9 ; �9 �=;?>@>@>A;_ÿ � � � ¯ ¹ ��m�

andthat the B �� 9 � ~ ÿ � � � ¯ ¹ ��m� . In addition,theorthogonalityrelationsimplythat �9 �DC �: � ~ �9 � ¨ (2.49)

sowehave thedecompositionÿ � � � ¯ ¹ ��p��~ �9 � CFE �G �� : �IH (2.50)

FromTheorem2.15and(2.50)we thenseethatthesystemJ � ¹ ô � �à �g´ ¿ �mÑ ¬ ¿V� �LK �� NM (2.51)

is anorthonormalbasisfor ÿ � � � ¯ ¹ ��p� . Thisbasisis canonicalin thesensethatthespaceÿ � � � ¯ ¹ ��p� is fully decomposedasin (2.50); i.e. theorthogonaldecomposi-tion processcannotbecontinuedfurtherbecause,asstatedin (2.45),

�: � ~PO3¯6Qfor

º_» ßý� . Note that the scalingfunctionsno longerappearexplicitly in theexpansionsincethey have beenreplacedby theconstant� accordingto (2.43).

Sometimesonewantsto usethebasisassociatedwith thedecompositionÿ � � � ¯ ¹ ��p��~ �9 �R� CSE �G�� R� �: � Hfor some �� ¯ . We recall that if �� T"%$�& � � ü ß÷�� then the non-periodicbasisfunctionsdonotoverlap.Thispropertyis exploitedin theparallelalgorithmdescribedin Chapter6.

2.3.3 Expansionsof periodic functions

Letð � �9 � andlet �� satisfy ¯ » �s� » � . Thedecomposition�9 � ~ �9 �R� CSE � ¬ G�� R� �: � H


which is obtainedfrom (2.49), leadsto two expansionsofð, namelythe pure

periodicscalingfunctionexpansionð«�gµ ��~ �IU ¬ ² À � � À � ´ À �� ´ À �µ � ¹ µ �� ¯ ¹ �� (2.52)

andtheperiodicwaveletexpansionð��gµ ��~ � U � ¬ ² À � � À �R� ´ À �� ´ À �gµ � ¶ � ¬ ²�� R� � Ñ ¬ ² À � � Æ �´ À �à �´ À �µ � ¹ µ �V� ¯ ¹ �� (2.53)

If ��"~�¯ then(2.53)becomesð��gµ ��~ À � ´ � ¶ � ¬ ² �� Ñ ¬ ² À � � Æ �g´ À �à �g´ À �µ � (2.54)

correspondingto a truncationof thecanonicalbasis(2.51).Let now

�ðbetheperiodicextensionof

ð, i.e.�ð«�gµ ��~ ð«�gµ ßXW µAY � ¹ µ ��¼ (2.55)

Then�ðis � -periodic�ð«�µ¾¶ � ��~ ð«�µ¾¶ �6ß � W µ ¶ � Y �Ð��~ ð«�µ ßZW µAY ��~ �ð��gµ � ¹ µ ��¼

BecauseW µAY is aninteger, we have��«�µ ß[W µAY �6~ ��«�µ � and

�à �gµ ß\W µAY �6~ �à �gµ �for

µ �þ¼ . Using(2.55)in (2.52)weobtain�ð«�µ ��~ ð��gµ ßXW µAY ��~ �IU ¬ ² À � � À � ´ À �� ´ À �µ ßXW µAY ��~ �]U ¬ ² À � � À � ´ À �� ´ À �gµ � ¹ µ ��¼ (2.56)

andby a similarargument�ð«�µ �Ç~ �IU � ¬ ² À � � À �� ´ À �� R� ´ À �µ � ¶ � ¬ ²�� R� � Ñ ¬ ² À � � Æ �g´ À �à �g´ À �µ � ¹ µ ��¼ (2.57)

Thecoefficientsin (2.52)and(2.53)aredefinedby

À �g´ À ~ Å � ð«�gµ � �� g´ À �µ � Æ µÆ �g´ À ~ Å � ð«�gµ � �à �g´ À �gµ � Æ µ


but it turnsout thatthey are,in fact,thesameasthoseof thenon-periodicexpan-sions.To seethiswe usethefact that

ð��gµ ��~ �ð«�µ � ¹ µ �� ¯ ¹ �� andwriteÆ �g´ À ~ Å � �ð��µ � �à �´ À �gµ � Æ µ~ �²³ � ¬ � Å � �ð��µ ��à �´ À �gµ@¶ }'� Æ µ~ �²³ � ¬ � Å ³ ¨ ³ �ð«�È ß0}'��à �g´ À �gÈ � Æ È~ �²³ � ¬ � Å ³ ¨ ³ �ð«�È ��à �g´ À �gÈ � Æ È~ Å �¬ � �ð��È ��à �g´ À �È � Æ È (2.58)

which is thecoefficient in thenon-periodiccase.Similarly, wefind that

À �g´ À ~ Å � �ð��gµ � �� g´ À �µ � Æ µ ~ Å �¬ � �ð«�µ ��à �g´ À �µ � Æ µHowever, periodicityin

�ðinducesperiodicityin thewaveletcoefficients:Æ �g´ À ¨ � Ñ Ú ~ Å �¬ � �ð��gµ ��à �g´ À ¨ � Ñ Ú �µ � Æ µ~ Å �¬ � �ð��gµ �� ¸ � à � � � �µ ß Þ ��ß � � Æ µ~ Å �¬ � �ð��gÈ ¶ Þ �� ¸ � à � � � È ß � � Æ µ~ Å �¬ � �ð��gÈ ��à �´ À �gÈ � Æ µ~ Æ �g´ À (2.59)

Similarly,

À �g´ À ¨ � Ñ Ú ~ À �´ À (2.60)

For simplicity we will drop thetilde andidentifyð

with its periodicextension�ð

throughoutthisstudy. Finally, wedefine


Definition 2.4 Let Ù=^_ Ñ and Ù^` Ñ denotetheoperatorsthatprojectanyð �vÿ � � � ¯ ¹ ��m�

orthogonallyonto�9 � and

�: � , respectively. Then� Ùa^_ Ñ ð � �µ �ë~ �²À � ¬ � À �´ À �� g´ À �gµ �� Ù^` Ñ ð � �µ �ë~ �²À � ¬ � Æ �g´ À �à �g´ À �µ �where

À �g´ À ~ Å � ð«�gµ � �� g´ À �µ � Æ µÆ �g´ À ~ Å � ð«�gµ � �à �g´ À �gµ � Æ µand Ù=^_ U ð ~RÙa^_ U � ð ¶ � ¬ ²�� Ù8^` Ñ ð

Chapter 3

Waveletalgorithms

3.1 Numerical evaluation of b and cGenerally, thereareno explicit formulasfor thebasicfunctions

�and à . Hence

mostalgorithmsconcerningscalingfunctionsandwaveletsareformulatedin termsof the filter coefficients. A goodexampleis the taskof computingthe functionvaluesof

�and à . Suchanalgorithmis usefulwhenonewantsto makeplotsof

scalingfunctionsandwaveletsor of linearcombinationsof suchfunctions.

3.1.1 Computing d at integers

Thescalingfunction�

hassupporton theinterval � ¯ ¹ üùßÝ�� , with�«� ¯æ�6~¥¯ and�� üoß=� ��~z¯ becauseit is continuousfor ü � î

[Dau92, p. 232].Wewill discard�� ü÷ß_� � in our computations,but, for reasonsexplainedin thenext section,wekeep

�«� ¯æ� .Putting

µ ~÷¯ ¹ � ¹ ®�®�® ¹ üùß � in thedilation equation(2.17)yieldsa homoge-neouslinearsystemof equations,shown herefor üõ~fe .ghhhhi �«� ¯��«� � ��«� ��«��j ��«�mî �

k0llllm ~ ç � ghhhhi ì �ì � ì ì �ì�nøì6o�ì � ì ì �ìqpøì6n�ì�o ì �ì p ì nk0llllmghhhhi �«� ¯æ��«� ��«� �æ��«�rj ��«�pî �

k0llllm ~�s ��t � ¯æ� (3.1)

wherewehave definedthevectorvaluedfunctiont ��µ �Ç~u� �·�µ � ¹ �·�µ ¶ �� ¹ ®�®N® ¹ �·�gµ@¶ ü÷ßû��0vConsiderthentheeigenvalueproblemfor sx� ,sG� t � ¯��~xw t � ¯æ� (3.2)

Equation(3.1)hasasolutionif w=~�� is amongtheeigenvaluesof sG� . Hencethecomputationalproblemamountsto findingtheeigensolutionsof (3.2). It is shown

44 Waveletalgorithms

in [SN96, p. 199] thattheeigenvaluesof sx� includewx~�� ¬ � ¹ � ~�¯ ¹ � ¹ ®�®�® ¹ ü í �·ßÝ�Thepresentcase(3.1) does,indeed,have aneigensolutioncorrespondingto theeigenvalue � , andwe usethe resultfrom Theorem2.9 to fix the inherentmulti-plicativeconstant.Consequently, we choosethesolutionwhereè ¬ ² ¿V� � �«� Ó¡��~¥�3.1.2 Computing d at dyadic rationals.

Given t � ¯�� from(3.1)wecanuse(2.17)againto obtain�

atall midpointsbetweenintegersin the interval, namelythevector t � � í �æ� . Substituting

µ ~ � ¹ o � ¹ p� ¹ ®N®�®into thedilation equationyieldsanothermatrix equationof theform (still shownfor ü ~fe ):t ¤ �� ¦ ~ ghhhhhi �«� � ��«� o � ��«� p� ��«�.y� ��«��z � �

k lllllm ~5ç � ghhhhhi ì ì �ì�o ì � ì ì �ì4pëì�n�ì6o�ì � ì ìqpøì6n�ì�oì4pk lllllmghhhhhi �«� ¯��«� � ��«� ��«��j ��«�mî �

k lllllm ~f{ t � ¯æ�(3.3)

Equation(3.3) is anexplicit formula,hencet � ¯��ë~ { �@t � ¯��t � � í �æ� ~ { t � ¯��Thispatterncontinuesto integersof theform Ó íNî , whereÓ is odd,andwe getthefollowing systemghhhhhhhhhhhhhhhhhi

| �� n �} �� on �| �� pn �} �� yn �| �� zn �} �«� Ä n �| �«� on �} �«� pn �| �«� yn �} �«� zn �

k lllllllllllllllllm~ ç �

ghhhhhhhhhhhhhhhhhiì �ì ì �ì � ì ì �ì6oøì � ì ì �ì6nøì�o�ì � ì ì �ì p ì n ì o ì � ì ì4pøì6n�ì6o�ì �ìqpøì6n�ì�oìqpøì�nì4p

k lllllllllllllllllmghhhhhi �� o � �� p� �� y� ��z � �

k lllllm (3.4)

3.1Numericalevaluationof�

and à 45

Equation(3.4) couldbeusedasit is but we observe that if we split it in twosystems,onewith theequationsmarkedwith | andonewith theequationsmarkedwith } , wecanreusethematrices{Õ� and { . Thispatternrepeatsitselfasfollows:t � n ��~ {Õ� t � � �t � on ��~ { t � � �t � ~ ��~ { �@t � n � t � p~ ��~ { t � n �t � o~ ��~ {Õ� t � on � t � y~ ��~ { t � o n �t � #� �ë~ {Õ� t � ~ � t � z #� �ø~ { t � ~ �t � o #� �ë~ {Õ� t � o~ � t � � #� �ø~ { t � o~ �t � p #� �ë~ {Õ� t � p~ � t � o #� �ø~ { t � p~ �t � y #� �ë~ {Õ� t � y~ � t � p #� �ø~ { t � y~ �This is the reasonwhy we keep

�«� ¯æ� in the initial eigenvalueproblem(3.1): Wecanusethesametwo matricesfor all stepsin thealgorithmandwe cancontinueasfollowsuntil adesiredresolution�� is obtained:

for º ~Ý� ¹ j ¹ ®�®�® ¹I�for Ó=~¥� ¹ j ¹I�2¹ ®�®�® ¹ � � ¬ ßÝ�t ¤ Ó� ��¦ ~ {Õ� t ¤ Ó� � ¬ 3¦t ¤ Ó� � ¶ �� ¦ ~ { t ¤ Ó� � ¬ 3¦

3.1.3 Function valuesof the basicwavelet:

Functionvaluesof à follows immediatelyfrom thecomputedvaluesof�

by thewaveletequation(2.18). However, functionvaluesof

�areonly neededat even

numerators: à �r�þí � � ��~ ç � è ¬ ² ¿Ð� � é ¿ �«� � �§í � � ß0Ó¡�The Matlab functioncascade( ü ,

�) computes

��gµ � and à �µ � atµ ~&Ó í �� , ✤ÓÕ~�¯ ¹ � í �� ¹ ®N®�® ¹ � ü ßÝ� � í �� .

Figure3.1shows�

and à for differentvaluesof ü .


0 1 2 3−0.5

0

0.5

1

1.5φ, D = 4

0 1 2 3−1

−0.5

0

0.5

1

1.5ψ, D = 4

0 1 2 3 4 5−0.5

0

0.5

1

1.5φ, D = 6

0 1 2 3 4 5−1

−0.5

0

0.5

1

1.5ψ, D = 6

0 2 4 6 8 10−0.5

0

0.5

1

1.5φ, D = 10

0 2 4 6 8 10−1

−0.5

0

0.5

1ψ, D = 10

0 5 10 15 20−0.5

0

0.5

1φ, D = 20

0 5 10 15 20

−0.6

−0.3

0

0.3

0.6ψ, D = 20

0 10 20 30 40−0.5

0

0.5

1φ, D = 40

0 10 20 30 40−0.6

−0.3

0

0.3

0.6ψ, D = 40

Figure 3.1: Basicscalingfunctionsandwaveletsplottedfor ��4��6�� .

3.2Evaluationof scalingfunctionexpansions 47

3.2 Evaluation of scalingfunction expansions

3.2.1 Nonperiodic case

Let�

bethebasicscalingfunctionof genusü andassumethat�

is known at thedyadicrationals

�§í �� , � ~�¯ ¹ � ¹ ®�®�® ¹ � ü ß5� �� , for somechosen� �5� . We

wantto computethefunctionð«�gµ ��~ �²À � ¬ � À �´ À � �g´ À �gµ � (3.5)

at thegrid points µ ~ µ ¿ ~°Ó í �� ¹ ÓÔ�v� (3.6)

where � �õ� correspondsto somechosen(dyadic)resolutionof the real line.Using(2.6)wefind that� �g´ À � Ó í �� ë~ � ��¸ � �� Ó í ��ß � �~ � ��¸ � �� ¬ � Ó@ß � �~ � ��¸ � ��Ð� � � ¨,�Ä¬ � ÓGßû� � � � í � � �~ � ��¸ � �� Ó ¹ � � í � � � (3.7)

where �ï� Ó ¹ � ��~�ÓL� � ¨N�Ä¬ � ß � � � (3.8)

Hence,�� Ó ¹ � � servesasanindex into thevectorof pre-computedvaluesof

�. For

this to makesense�� Ó ¹ � � mustbeaninteger, which leadsto therestrictionº ¶ � ß�� ¯ (3.9)

Only üÝßû� termsof (3.5)canbenonzerofor any givenµ ¿ . From(3.7)wesee

thatthesetermsaredeterminedby thecondition¯ å �� Ó ¹ � �� å ü÷ß �Hence,therelevantvaluesof

�are

� ~ � � � Ó*� ¹ � � � Ó¡� ¶ � ¹ ®�®�® ¹ � � � Ó*� ¶ ü�ßû� , where� � � Ó¡��~��pÓL� � ¬ �I� ß ü ¶ � (3.10)

Thesum(3.5),forµ

givenby (3.6),canthereforebewrittenasð ¤ Ó� � ¦ ~Ý� ��¸ � À � á ¿ â ¨ è ¬ �²À � À � á ¿ â À �g´ À � ¤ �ï� Ó ¹ � �� ¦ ¹ ÓÔ�v� (3.11)


3.2.2 Periodic case

Wewantto computethefunctionð«�µ �Ç~ � Ñ ¬ ² À � � À �´ À �� g´ À �µ � ¹ µ �� ¯ ¹ �� (3.12)

forµ ~ µ ¿ ~ÝÓ í � � ¹ Ó ~°¯ ¹ � ¹ ®�®�® ¹ � � ßÝ� where �·�� . Hencewe haveð ¤ Ó� � ¦ ~ �mÑ ¬ ² À � � À �g´ À �� g´ À ¤ Ó� � ¦~ � Ñ ¬ ² À � � À �g´ À ²³�� g´ À ¤ Ó� � ¶ } ¦~ � ��¸ � � Ñ ¬ ² À � � À �´ À ²³�� ¤ �ï� Ó ¹ � � ¶ � � ¨,�Z}� � ¦

with�� Ó ¹ � �@~1ÓL� � ¨,��¬ � ß � �� by the samemanipulationasin (3.7). Now, as-

sumingthatº � �s� where �� is given by (2.47)we have � �� ü&ß � . Using

Lemma3.1(provedbelow), we obtaintheexpressionð ¤ Ó� � ¦ ~z� ��¸ � �mÑ ¬ ² À � � À �g´ À � ¤ � �� Ó ¹ � �� Ñ�� ¦ ¹ Ó=~�¯ ¹ � ¹ ®�®�® ¹ � � ßÝ� (3.13)

Lemma 3.1 Let�

bea scalingfunctionwithsupport � ¯ ¹ üþßÕ�� andlet� ¹ } ¹gº�¹I� �� with

� � ¯ and � � � ü÷ß×� . Then�²³ � ¬ � � ¤ ��¶ � � ¨,�Z}� � ¦ ~ � ¤ � � � � Ñ�� ¦Proof: Since � �*� üõßÝ� , only onetermof thesumcontributesto theresultandthereexistsa unique}þ~z}�� suchthat� ¶ � � ¨,� } �� ¯ ¹ � � �Thenwe know that

�õ¶ � � ¨,�Z}��6�� ¯ ¹ � � ¨N�� but this interval is preciselytherangeof themodulusoperatordefinedin AppendixB, so� ¶ � � ¨,� } � ~ � �õ¶ � � ¨,� }�� Ñ#�� ~ � � � � Ñ�� ¹ }��v�from which theresultfollows. ±


3.2.3 DST and IDST - matrix formulation

Equation(3.13) is a linear mappingfrom � � scalingfunction coefficientsto � �samplesof

ð, soit hasa matrix formulation.Let � � ~u� À �´ � ¹ À �g´ ¹ ®�®�® ¹ À �g´ � Ñ ¬ � v and � ~u� ð�� ¯�� ¹ ð�� í � � � ¹ ®�®�® ¹ ð��Ð� � � ßÝ�� í � � �r� v . Wedenotethemapping � ~f¡ � ´ � � � (3.14)

When � ~ ºthen(3.14)becomes � ~f¡ �´ � � � (3.15)¡ �g´ � is a squarematrix of order ¢�~÷� � . In thecaseof (3.15)we will oftendrop

thesubscriptsandwrite simply ~f¡£� (3.16)

Thishastheform (shown hereforº ~ j , ü ~ î

)¤¥¥¥¥¥¥¥¥¥¥¥¥¥¦§©¨ ��ª§©¨ ~ ª§©¨ �~ ª§©¨ o~ ª§©¨ n~ ª§©¨ p~ ª§©¨ �~ ª§©¨ y~ ª

«I¬¬¬¬¬¬¬¬¬¬¬¬¬ �®�,¯°¤¥¥¥¥¥¥¥¥¥¥¥¥¥¦±©¨ ��ª ±©¨ ��ª ±²¨ ��ª±©¨ ��ª ±²¨ ��ª ±²¨ ��ª±©¨ ��ª ±²¨ ��ª ±©¨ ��ª±²¨ ��ª ±©¨ ��ª ±©¨ ��ª±©¨ ��ª ±©¨ ��ª ±²¨ ��ª±©¨ ��ª ±²¨ ��ª ±©¨ ��ª±²¨ ��ª ±©¨ ��ª ±©¨ ��ª±©¨ ��ª ±©¨ ��ª ±²¨ ��ª

«I¬¬¬¬¬¬¬¬¬¬¬¬¬¤¥¥¥¥¥¥¥¥¥¥¥¥¥¦³ o ´ �³ o ´ ³ o ´ �³ o ´ o³ o ´ n³ o ´ p³ o ´ �³ o ´ y

«I¬¬¬¬¬¬¬¬¬¬¬¬¬Notethatonly valuesof

�at the integersappearin ¡ . Thematrix ´ is non-

singularandwecanwrite �Õ~µ¡ ¬ (3.17)

Wedenote(3.17)thediscretescalingfunction transform (DST)1 and(3.16)theinversediscretescalingfunction transform (IDST) .

TheMatlabfunctiondst(

, ü ) computes(3.17)and idst( � , ü ) computes ✤(3.16).

1DSTshouldnot beconfusedwith thediscretesinetransform.


We now considerthe the computationalproblemof interpolatinga functionð ��¶ � � ¯ ¹ ��m� betweensamplesatresolution� . Thatis,wewantto usethefunctionvalues

ð�� Ó í � � � ¹ Ó ~z¯ ¹ � ¹ ®�®N® ¹ � � ßû� to computeapproximationstoð«� Ó í � �¸· � ¹ Ó ~¯ ¹ � ¹ ®�®�® ¹ � ��· ß5� for some�@¹ � � . Therearetwo steps.Thefirst is to solve the

system ¡ � ´ � � � ~ �for � � . Thesecondis to computethevector

� · definedby � · ~f¡ � · ´ � � � (3.18)

Equation(3.18)is illustratedbelow for thecase�·~ j ¹ � ¹ ~ î.¤¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¦

§©¨ ��ª§©¨ º� ª§©¨ � º� ª§©¨ o º� ª§©¨ n º� ª§©¨ p º� ª§©¨ � º� ª§©¨ y º� ª§©¨ ~ º� ª§©¨ z º� ª§©¨ � º� ª§©¨ Ä º� ª§©¨ � º� ª§©¨ o º� ª§©¨ n º� ª§©¨ p º� ª

«I¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬�� ¯°

¤¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¥¦

±©¨ ��ª ±©¨ ��ª ±²¨ ��ª±©¨ � ª ±©¨ p� ª ±²¨ o � ª±©¨ ��ª ±©¨ ��ª ±²¨ ��ª±©¨ o � ª ±©¨ � ª ±²¨ p� ª±©¨ ��ª ±©¨ ��ª ±©¨ ��ª±©¨ p� ª ±©¨ o � ª ±©¨ � ª±©¨ ��ª ±©¨ ��ª ±©¨ ��ª±©¨ p � ª ±©¨ o � ª ±©¨ � ª±©¨ ��ª ±©¨ ��ª ±©¨ ��ª±©¨ p � ª ±©¨ o � ª ±©¨ � ª±©¨ ��ª ±©¨ ��ª ±©¨ ��ª±©¨ p � ª ±©¨ o � ª ±©¨ � ª±©¨ ��ª ±©¨ ��ª ±©¨ ��ª±©¨ p � ª ±©¨ o � ª ±©¨ � ª±©¨ ��ª ±©¨ ��ª ±²¨ ��ª±©¨ p � ª ±©¨ o � ª ±²¨ � ª

«I¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬¬

¤¥¥¥¥¥¥¥¥¥¥¦³ o ´ �³ o ´ ³ o ´ �³ o ´ o³ o ´ n³ o ´ p³ o ´ �³ o ´ y

«I¬¬¬¬¬¬¬¬¬¬

Figure3.2 show how scalingfunctionscaninterpolatebetweensamplesof asinefunction.

3.2.4 Periodic functions on the interval -¼»½1�¾�5Considertheproblemof expressinga periodicfunction

ðdefinedon theinterval� ì ¹ é � , where

ì ¹ é ��¼ insteadof theunit interval. This canbe accomplishedbymappingtheinterval � ì ¹ é � linearly to theunit interval andthenusethemachineryderivedin Section3.2.3.


0 1 2 3 4 5 6−1

−0.5

0

0.5

1D = 4, Error = 0.14381

0 1 2 3 4 5 6−1

−0.5

0

0.5

1D = 6, Error = 0.04921

0 1 2 3 4 5 6−1

−0.5

0

0.5

1D = 8, Error = 0.012534

Figure 3.2: A sinefunction is sampledin 8 points( ¿<�fÀ ). Scalingfunctionsof genus�f��q��Á arethenusedto interpolatein ��Á points( ¿ ¹ ��Â ).Weimposetheresolution� � on theinterval � ì ¹ é � , i.e.µ ¿ ~�Ó é ß ì� � ¶ ì ¹ Ó=~�¯ ¹ � ¹ ®�®N® ¹ � � ßÝ�

Thelinearmappingof theintervalì » µ » é to theinterval ¯ » È » � is given

by È ~ µ ß ìé ß ì ¹ ì » µ å éhence

È ¿ ~�Ó í � � ¹ Ó ~�¯ ¹ � ¹ ®�®�® ¹ � � ß×� . LetÃ �gÈ ��~ ð��gµ ��~ ð«�V� é ß ì � È ¶ ì � ¹ ¯ » È å �


Thenwehave from (3.13)Ã �È ¿ ��~ Ã � Ó í � � ��~�� ¸ � � Ñ ¬ ² À � � À �g´ À � ¤ � �� Ó ¹ � �Ð� � Ñ#�� ¦and transformingback to the interval � ì ¹ é � yields

ð«�gµ ¿ � ~ Ã �È ¿ � . Thus wehave effectively obtainedanexpansionof

ð �Ä� ì ¹ é � in termsof scalingfunctions“stretched”to fit this interval at its dyadicsubdivisions.

3.3 FastWaveletTransforms

Theorthogonalityof scalingfunctionsandwaveletstogetherwith thedyadiccou-plingbetweenMRA spacesleadto arelationbetweenscalingfunctioncoefficientsandwaveletcoefficientson differentscales.This yieldsa fastandaccuratealgo-rithm dueto Mallat [Mal89] denotedthepyramid algorithm or the fast wavelettransform (FWT). We usethelattername.

Letð �vÿ � � ¼�� andconsidertheprojection� Ù _ Ñ ð � �µ ��~ �²À � ¬ � À �g´ À � �´ À �µ � (3.19)

which is given in termsof scalingfunctionsonly. We know from Definition 2.1that Ù _ Ñ ð ~5Ù _ ÑgÏ�Å ð@¶ Ù ` ÑgÏ�Å ð sotheprojectionalsohasa formulationin termsofscalingfunctionsandwavelets:� Ù _ Ñ ð � �gµ ��~ �²À � ¬ � À � ¬ Ä´ À � � ¬ �´ À �gµ � ¶ �²À � ¬ � Æ � ¬ �´ À à � ¬ �´ À �gµ � (3.20)

Our goalhereis to derive a mappingbetweenthesequenceO À �g´ À Q À �� andthese-quences O À � ¬ �´ À Q À �� ¹ O Æ � ¬ �´ À Q À ��

Thekey to thederivationsis thedilationequation(2.17)andthewaveletequa-tion (2.18).Using(2.17)we derive theidentity� � ¬ Ä´ À �µ ��~ � á � ¬ â ¸ � �� ¬ µ ß � �~ � ��¸ � è ¬ ² ¿Ð� � ì ¿ �«� � � µ ßû� � ßûÓ¡�~ è ¬ ² ¿V� � ì ¿ � �g´ � À ¨ ¿ �µ � (3.21)

3.3FastWaveletTransforms 53

andfrom (2.18)we have à � ¬ �´ À �gµ ��~ è ¬ ² ¿V� � é ¿ � �g´ � À ¨ ¿ �µ � (3.22)

Substitutingthefirst identity into thedefinitionof À �´ À (2.12)weobtain

À � ¬ Ä´ À ~ Å �¬ � ð��gµ � è ¬ ² ¿Ð� � ì ¿ � �´ � À ¨ ¿ �gµ � Æ µ~ è ¬ ² ¿V� � ì ¿ Å �¬ � ð«�µ � � �´ � À ¨ ¿ �gµ � Æ µ~ è ¬ ² ¿V� � ì ¿ À �´ � À ¨ ¿andsimilarly forÆ � ¬ �´ À . Thus,weobtaintherelations

À � ¬ �´ À ~ è ¬ ² ¿Ð� � ì ¿ À �g´ � À ¨ ¿ (3.23)Æ � ¬ �´ À ~ è ¬ ² ¿Ð� � é ¿ À �g´ � À ¨ ¿ (3.24)

which definea linear mappingfrom the coefficientsin (3.19) to the coefficientsin (3.20). We will refer to this asthepartial wavelet transform (PWT) . To de-composethespace

9 � further, oneappliesthemappingto thesequenceO À � ¬ �´ À Q À ��to obtainthenew sequencesO À � ¬ � ´ À Q À �� and O Æ � ¬ � ´ À Q À �� . This patterncanbefur-ther repeatedyielding the full FWT: Applying (3.23)and(3.24) recursively forº ~Æ� ¹ � ß5� ¹ ®�®�® ¹ � � ¶ � , startingwith the initial sequenceO À � ´ À Q À �� , will thenproducethewaveletcoefficientsin theexpansiongivenin (2.13). Notethatoncetheelements

Æ � ¬ �´ À havebeencomputed,they arenotmodifiedin subsequentsteps.ThismeansthattheFWT is veryefficient in termsof computationalwork.

Theinversemappingcanbederivedin asimilar fashion.Equating(3.19)with(3.20)andusing(3.21),(3.22)againwe get�²À � ¬ � À �´ À � �g´ À �gµ ��~ �²³ � ¬ � À � ¬ �´ ³ � � ¬ �´ ³ �gµ � ¶ �²³ � ¬ � Æ � ¬ �´ ³ à � ¬ Ä´ ³ �µ �~ �²³ � ¬ � À � ¬ �´ ³ è ¬

² ¿Ð� � ì ¿ � �´ � ³ ¨ ¿ �gµ � ¶ �²³ � ¬ � Æ � ¬ �´ ³ è ¬ ² ¿Ð� � é ¿ � �´ � ³ ¨ ¿ �gµ �~ è ¬ ² ¿V� � �²³ � ¬ � � À � ¬ �´ ³ ì ¿ ¶ Æ � ¬ �´ ³ é ¿ � � �´ � ³ ¨ ¿ �gµ �


Wenow introducethevariable� ~z��} ¶ Ó in thelastexpression.Since Ó=~ � ßý��}

and ÓÔ�� ¯ ¹ ü÷ßÝ�� , wefind for given�thefollowing boundson } :Ç � ß ü ¶ �� È öR} �� » } » }ò� �� öÊÉ ��ÌË (3.25)

Hence �²À � ¬ � À �g´ À � �´ À �gµ ��~ �²À � ¬ �³ ° á À â²³ � ³ Å á À â � À � ¬ �´ ³ ì À ¬ � ³ ¶ Æ � ¬ �´ ³ é À ¬ � ³ � � �g´ À �gµ �

andequatingcoefficientswe obtainthereconstructionformula

À �´ À ~³ ° á À â²³ � ³ Å á À â À � ¬ �´ ³ ì À ¬ � ³ ¶ Æ � ¬ �´ ³ é À ¬ � ³ (3.26)

We will call this the inversepartial wavelet transform (IPWT). Consequently,the inversefast wavelet transform (IFWT) is obtainedby repeatedapplicationof (3.26)for

º ~�� ¶ � ¹ �s� ¶ � ¹ ®�®N® ¹ � .

3.3.1 Periodic FWT

If the underlyingfunctionð

is periodicwe alsohave periodicity in the scalingfunctionandwaveletcoefficients; À �g´ À ~ À �g´ À ¨ � Ñ Ú by (2.60)and

Æ �´ À ~ Æ �g´ À ¨ � Ñ Ú by(2.59).Hence,it is enoughto consider� � coefficientsof eithertypeat level

º. The

periodicPWTis thus

À � ¬ �´ À ~ è ¬ ² ¿V� � ì ¿ À �g´ÎÍ � À ¨ ¿�Ï ° Ñ (3.27)Æ � ¬ Ä´ À ~ è ¬ ² ¿V� � é ¿ À �g´ÎÍ � À ¨ ¿�Ï ° Ñ (3.28)

where � ~z¯ ¹ � ¹ ®�®N® ¹ � � ¬ ßÝ�HencetheperiodicFWT is obtainedby repeatedapplicationof (3.27)and(3.28)for

º ~�� ¹ �ªß � ¹ ®�®�® ¹ � � ¶ � .If we let � � ~ ¯ then(3.27)and(3.28) canbe applieduntil all coefficients

in the (canonical)expansion(2.54)have beencomputed.Thereis thenonly onescalingfunction coefficient left, namely À � ´ � (the “DC” term), the rest will bewaveletcoefficients.Figure3.3showsanexampleof theperiodicFWT.


À n ´ �À n ´ À n ´ �À n ´ oÀ n ´ nÀ n ´ pÀ n ´ �À n ´ yÀ n ´ ~À n ´ zÀ n ´ �À n ´ � À n ´ �À n ´ oÀ n ´ nÀ n ´ p

Ð À o ´ �À o ´ À o ´ �À o ´ oÀ o ´ nÀ o ´ pÀ o ´ �À o ´ yÆ o ´ �Æ o ´ Æ o ´ �Æ o ´ oÆ o ´ nÆ o ´ pÆ o ´ �Æ o ´ y

Ð À � ´ �À � ´ À � ´ �À � ´ oÆ � ´ �Æ � ´ Æ � ´ �Æ � ´ oÆ o ´ �Æ o ´ Æ o ´ �Æ o ´ oÆ o ´ nÆ o ´ pÆ o ´ �Æ o ´ y

Ð À �´ �À �´ Æ �´ �Æ �´ Æ � ´ �Æ � ´ Æ � ´ �Æ � ´ oÆ o ´ �Æ o ´ Æ o ´ �Æ o ´ oÆ o ´ nÆ o ´ pÆ o ´ �Æ o ´ y

Ð À � ´ �Æ � ´ �Æ Ä´ �Æ Ä´ Æ � ´ �Æ � ´ Æ � ´ �Æ � ´ oÆ o ´ �Æ o ´ Æ o ´ �Æ o ´ oÆ o ´ nÆ o ´ pÆ o ´ �Æ o ´ yFigure 3.3: TheperiodicFWT. TheshadedelementsÑ �g´ À obtainedat eachlevel remainthe sameat subsequentlevels. Here the depthis takento be as large as possible,i.e.Ò �®Ó8�� .


Thereconstructionalgorithm(IPWT) for theperiodicproblemis similarto thenon-periodicversion:

À �´ À ~³ ° á À â²³ � ³ Å á À â À � ¬ �´ÎÍ ³ Ï ° ÑgÏ�Å ì À ¬ � ³ ¶ Æ � ¬ �´ÎÍ ³ Ï ° ÑgÏ�Å é À ¬ � ³ (3.29)

with } �� and } � �� definedasin (3.25)and� ~�¯ ¹ � ¹ ®�®�® ¹ � � ßÝ�Hencethe periodic IFWT is obtainedby applying(3.29) for

º ~Ô� � ¶ � ¹ � � ¶� ¹ ®�®N® ¹ � . We restrictourselvesto algorithmsfor periodicproblemsthroughoutthis report.Hencewewill consideronly theperiodiccasesof FWT, PWT, IFWT,andIPWT.

3.3.2 Matrix representationof the FWT

Let � � ~ � À �g´ � ¹ À �´ ¹ ®�®�® ¹ À �´ � Ñ ¬ � v and similarly, Õ � ~ � Æ �´ � ¹ Æ �´ ¹ ®�®�® ¹ Æ �g´ � Ñ ¬ � v .Since(3.27)and(3.28)are linearmappingsfrom ¼ �mÑ onto ¼ �ÑÏ�Å the PWT canberepresentedas � � ¬ ~ s � � �Õ � ¬ ~ Ö � � �where s � and Ö � are � � ¬ by � � matricescontainingthe filter coefficients. Forü ~×e and

º ~ îwe have

s n ~ØÙÙÙÙÙÙÙÙÙÙÚ

ì � ì ì � ì6o�ì6n�ì4pì � ì ì � ì�o ì�n ì4pì � ì ì � ì�oøì�n ì4pì � ì ì � ì�o ì�n�ìqpì � ì ì � ì6o�ì6n�ì4pì � ì ì � ì�oøì6n�ìqpì6n�ìqp ì � ì ì � ì6oì � ì6oøì�n�ìqp ì � ì Û@ÜÜÜÜÜÜÜÜÜÜÝ

and

Ö n ~ØÙÙÙÙÙÙÙÙÙÙÚ é � é é � é o é n é pé � é é � é o é n é pé � é é � é o é n é pé � é é � é o é n é pé � é é � é o é n é pé � é é � é o é n é pé n é p é � é é � é oé � é o é n é p é � é

Û ÜÜÜÜÜÜÜÜÜÜÝ


Theseareshift-circulantmatrices(seeDefinition8.1). Moreover, therowsareor-thonormalby equations(2.22)and(2.23): � ¿ ì ¿ ì ¿ ¨ � ³ ~RÖ�� ´ ³ and � ¿ é ¿ é ¿ ¨ � ³ ~Ö�� ´ ³ . Ö � is similar, but with é ¿ ’s in placeof

ì ¿ ’s; hencethe rows of Ö � arealsoorthonormal.Moreover, theinnerproductof any row in s � with any row in Ö � iszeroby Theorem2.1,i.e. s � Ö v� ~�¯ .

Wenow combines � and Ö � to obtaina � �<Þ � � matrixß � ~áà s �Ö �£âwhichrepresentsthecombinedmappingà � � ¬ Õ � ¬ â ~ ß � � � (3.30)

Equation(3.30) is the matrix representationof a PWT stepasdefinedby (3.27)and(3.28).Thematrix

ß � is orthogonalsinceß � ß v� ~Êà s �Ö �*â �ãs v� Ö v� ��~Êà s � s v� s � Ö v�Ö � s v� Ö � Ö v� â ~Ôà<ä � åå ä �£âHencethemapping(3.30)is invertedby� � ~ ß v� à � � ¬ Õ � ¬ æâ (3.31)

Equation(3.31)is thematrix representationof anIPWT stepasdefinedin (3.29).The FWT from level � to level ��û~ �0ß?w cannow be expressedas the

matrix-vectorproduct Õï~?çéè�� (3.32)

where�Õ~Ä� � ,Õ�~ ØÙÙÙÙÙÚ � ��Õ �R�Õ �� ¨ ...Õ � ¬

Û ÜÜÜÜÜÝand çÔè ~ �ß �R� �ß �R� ¨ >@>@> �ß � ¬ ß � ®


Thematrix�ß � , of order � � , is givenby

�ß � ~ghhhhhhhhhhhhhhi

ß �ä ¿ á �g´ � â

k llllllllllllllmwhere ä ¿ á �g´ � â denotesthe Ó Þ Ó identity matrix with Ó$~ Ó � º�¹ ��ý~ � � ßÝ� � . Itfollowsthat çéè is orthogonal2 andhencetheIFWT is givenby�=~\êIçÊè@ë v Õ3.3.3 A moregeneralFWT

Supposethat elementsto be transformedmay containotherprime factorsthan� , i.e. ¢ ~ �ª� � , with � ¹ � � � and� �� ~ � . In this case,the FWT can

still bedefinedby mappingsof theform (3.27)and(3.28)with a maximaldepthw=~�� , i.e. � � ~°¯ . It will beconvenientto useaslightly differentnotationfor thealgorithmsin Chapters5, 6, and8. Hence,let

½ � ~�¢ í � � and

À � ¿ ö À � ¬s� ´ ¿Æ � ¿ ö Æ � ¬�� ´ ¿for ��~�¯ ¹ � ¹ ®�®N® ¹ wGßÝ� and Ó=~�¯ ¹ � ¹ ®�®�® ¹ ½ � ßÝ� . We cannow definetheFWT asfollows:

Definition 3.1 Fastwavelettransform(FWT)Let � ¹ � � � with

� ��·~1� and ¢ ~ �ª� � . Let � � ~FO À � ¿ Q�ì ¬ ¿V� � begiven. TheFWTis thencomputedby therecurrenceformulas

À �©¨ ³ ~ è ¬ ² À � � ì À À � Í À ¨ � ³ Ï�í¸îÆ �©¨ ³ ~ è ¬ ² À � � é À À � Í À ¨ � ³ Ï í î(3.33)

where � ~z¯ ¹ � ¹ ®�®N® ¹ w ßÝ� , ½ � ~�¢ í �� , and }§~�¯ ¹ � ¹ ®N®�® ¹ ½ �©¨ ßÝ� .2Theorthonormalityof ïxð is onemotivationfor thechoiceof dilation equationsmentioned

in Remark2.4.


Therelationbetweenthescalingfunctioncoefficientsandtheunderlyingfunc-tion is unimportantfor theFWT algorithmperse: In numericalapplicationsweoftenneedto apply theFWT to vectorsof functionvaluesor evenarbitraryvec-tors.Hencelet ñ ��¼ ì . Thentheexpressionòñã~óçTè.ñ (3.34)

is definedby putting � � ~µñ , performing w stepsof of (3.33),andlettingòñ$~ � O À è¿ Q�ô�õ ¬ ¿Ð� � ¹ O�O Æ � ¿ Q�ô î ¬ ¿Ð� � Q � � è ´ è ¬ �´ÎöÎöÎö ´ If w is omittedin (3.34)it will assumeits maximalvalue � . Notethat ç è ~ äfor w=~°¯ . TheIFWT is writtensimilarly asñ$~ � ç è � v òñ (3.35)

correspondingto therecurrenceformula

À � À ~³ ° á À â²³ � ³ Å á À â À �©¨ Í ³ Ï í¸î ��Å ì À ¬ � ³ ¶ Æ � ¨ Í ³ Ï í�î ��Å é À ¬ � ³

where� ~?w@ßÝ� ¹ wGß0� ¹ ®�®�® ¹ � ¹ ¯ , ½ � ~�¢ í �� , � ~�¯ ¹ � ¹ ®�®N® ¹ ½ � ßÝ� , and } �r� � and} � �� aredefinedasin (3.25).

TheMatlabfunctionfwt ( ñ , ü , w ) computes(3.34)andifwt (òñ , ü , w ) computes ✤

(3.35).

3.3.4 Complexity of the FWT algorithm

Onestepof (3.33),thePWT, involves ü ½ � additionsand ü ½ � multiplications.Thenumberof floatingpointoperationsis therefore

óPWT

� ½ � ��~��ü ½ � (3.36)

Let ¢ be the total numberof elementsto be transformedand assumethat theFWT is carriedout to depth w . TheFWT consistsof w applicationsof thePWTto


successively shortervectorssothetotalwork isó

FWT� ¢ª�ë~ è ¬ ² � � �

óPWT ¤ ¢ � � ¦~ è ¬ ² � � � ��ü ¢ � �~ ��ü�¢ �6ß � õ�6ß �~ î ü�¢ ¤ �6ß �� è ¦ å î ü÷¢ (3.37)ü is normallyconstantthroughoutwaveletanalysis,sothecomplexity is ø � ¢ª� .

TheIFWT hasthesamecomplexity astheFWT. For comparisonwementionthatthecomplexity of thefastFouriertransform(FFT) is ø � ¢ "ù$�& � ¢§� .3.3.5 2D FWT

Usingthedefinitionof ç è from theprevioussection,wedefinethe2D FWT asòú ~?çéè�û ú � çÔè�ü � v (3.38)

whereú ¹ òú ��¼�ý ´ ì . Theparametersw ý and w ì arethetransformdepthsin the

first andseconddimensions,respectively. Equation(3.38)canbeexpressedas þ1D wavelet transformsof therows of

úfollowedby ¢ 1D wavelet transforms

of thecolumnsofú � ç è ü«� v , i.e.òú ~�çÊè�û®êIçÊè�ü ú v ë v

Thereforeit is straightforwardto computethe2D FWT giventhe1D FWT fromDefinition3.1. It followsthattheinverse2D FWT is definedasú ~ ê çéè�û ë v òú çÊè�ü (3.39)

TheMatlabfunctionfwt2 (ú

, ü , w ý , w ì ) computes(3.38)andifwt2 (òú

, ü ,✤ w ý , w ì ) computes(3.39).

Using(3.37)we find thecomputationalwork of the2D FWT asfollows

óFWT2

� þ ¹ ¢§�ø~ þ óFWT

� ¢§� ¶ ¢ óFWT

� þ5�~ þ î ü÷¢ ¤ �/ß �� è�ü ¦ ¶ ¢ î ü÷þ ¤ �6ß �� è�û ¦~ î ü÷þ�¢ ¤ � ß �� è û ß �� è ü ¦ å ÿ ü÷þ�¢ (3.40)

Thecomputationalwork of theinverse2D FWT is thesame.

Chapter 4

Approximation properties

4.1 Accuracy of the multir esolutionspaces

4.1.1 Approximation propertiesof ��We will now discussthepointwiseapproximationerrorintroducedwhen

ðis ap-

proximatedby anexpansionin scalingfunctionsatlevel � . Let �� , �� andassumethat � is � timesdifferentiableeverywhere.

For anarbitrary, but fixed � , we definethepointwiseerroras�� ! #"$�%�& ��(' ��where )�% " �%�& �� is theorthogonalprojectionof � ontotheapproximationspace* � asdefinedin Section2.1.

Recall that �! " � hasexpansionsin termsof scalingfunctionsaswell as intermsof wavelets.Thewaveletexpansionfor �% +"$� is

��! " �,�- .�!�� /013254 /6 �)738 1:9 ��7&8 1 ��%; � 4=<0>?2 ��7 /0132@4 /A > 8 1�B%> 8 1 �� (4.1)

andby letting �C D temporarilywe geta waveletexpansionfor � itself:�E ��F� /01G254 / 6 ��738 1�9 ��738 1 ��%; /0>)2 �)7 /01G254 / A > 8 1�B,> 8 1 �� (4.2)

Then,subtracting(4.2)from (4.1)weobtainanexpressionfor theerror �� in termsof thewaveletsat scalesHJIK� :�� F� /0>?2 � /0132@4 /LA > 8 1�B,> 8 1 �� (4.3)

62 Approximationproperties

Define MFN �PORQ#ST�UWVYX[Z \�]] B �^ > �_�a`5� ]] � ORQ+Sb&Udc e 8 f 4=<hgji B .kl� iHenceORQ#S T�U+VYXmZ \ i B,> 8 1 .�!� i ��^ >)n � MFN andusingTheorem2.5,we find thati A > 8 1�B,> 8 1 .�� ipo MFq ^ 4d>

q ORQ#Sr UWVYX[Z \ ]] �%sq@t �ud� ]] MFN

Recallthat

supp B%> 8 1 �v�xw > 8 1 �zy `^ > ' `{;}|~��^ > �Hence,thereareat most |~�� intervals w > 8 1 containinga givenvalueof � . Thusfor any � only |x�a� termsin theinnersummationin (4.3)arenonzero.Let w > betheunionof all theseintervals,i.e.w > .�!�� )�� T�UWV X[Z �� w > 8 �andlet � q> �� ORQ+Sr U+V X s T t ]] � s

q@t �ud� ]]Thenonefindsa commonboundfor all termsin theinnersum:/01G254 / i A > 8 1�B%> 8 1 �� i�o MFN@MFq ^ 4d> q �|P�}�#� � q> ��Theoutersumcannow beevaluatedusingthefact that� q � .��FI � q �� < ��FI � q �� FI��andwe establishtheboundi �� i�o MFN5MFq �|P��#� � q � �� /0>?2 � ^ 4d>

q� MFN5MFq �|P��#� � q � �� ^ 4 � q��a^ 4 q

Thuswe seethat thatfor anarbitrarybut fixed � theapproximationerrorwill beboundedas i �� .�!� i �K�� ^ 4 � q �This is exponentialdecaywith respectto the resolution � . Furthermore,thegreaterthenumberof vanishingmoments� , thefasterthedecay.

Finally, notethateacherror term A > 8 1�B,> 8 1 .�!� is zerofor ��w > 8 1 . This meansthat �� dependsonly on �E .kl�('&k��}��,'(��;} �|x��+�3�W^ � �� . This is whatwasalsoobservedin Chapter1.

4.1Accuracy of themultiresolutionspaces 63

4.1.2 Approximation propertiesof ��Wenow considertheapproximationerrorin theperiodiccase.Let �� &��'��$�[�andassumethatits periodicextension(2.55)is � timesdifferentiableeverywhere.Furthermore,let �aIK� e where � e is givenasin (2.47)anddefinetheapproxima-tion erroras �� ¡ " �%�& .�!�(' �L�a��'��$�where )�¢ " �%�& �� is theorthogonalprojectionof � ontotheapproximationspace�* � asdefinedin Definition2.4.

Using the periodicwavelet expansion(2.53) andproceedingas in the non-periodiccasewe find that �� F� /0>)2 � � X 4£<0 1G2 e A > 8 1

�B,> 8 1 �� (4.4)

Sincethecoefficients A > 8 1 arethesameasin thenon-periodiccaseby (2.58),Theorem(2.5) appliesand we can repeatthe analysisfrom Section4.1.1 and,indeed,we obtainthat i �� .�� i �K�� ^ 4 � q �(' ��}��'��$�

Wewill now considertheinfinity normof

�� definedby¤ �� ¤ / �¥ORQ#ST�Udc e 8 <hg i �� iA similaranalysisasbeforeyields¤ �� ¤ / o /0>?2 � � X 4=<0 1G2 e i A > 8 1 i ORQ#ST�UWVYXmZ \ ]]]

�B,> 8 1 �� ]]]o MFN-MFq /0>)2 � � X 4=<0 132 e ^ >?n � ^ 4d> sq �v¦§ t ORQ+Sr U+V XmZ \�]] �,s

qlt .u¨� ]]� MFN-MFq ORQ+Sr Udc e 8 <hg ]] �,sq@t .ud� ]] /0 >?2 � ^ 4d>

q� MFN-MFq ORQ+Sr Udc e 8 <hg ]] � s

q@t .ud� ]] ^ 4 � q��a^ 4 qHence ¤ �� ¤ /K�K�� ^ 4 � q �


Finally, considerthe2-normof

�� :¤ �� ¤ �� ª© <e �� .�� A � o © <e ¤ �� ¤ �/ A �� ¤ �� ¤ �/�© <e A �R� ¤ �� ¤ �/hencewe have alsoin thiscase¤ �� ¤ � �K�� ^ 4 � q � (4.5)

4.2 Wavelet compressionerrors

In this Sectionwe will seehow well a function �«� �* � is approximatedby awaveletexpansionwheresmallwaveletcoefficientshavebeendiscarded.Wewillrestrictattentionto thecaseof theinterval ��'��$� .

It wasshown in Section2.1.6thatthewaveletcoefficientscorrespondingto aregionwhere� is smoothdecayrapidlyandarenotaffectedby regionswhere� isnotsosmooth.Wenow investigatetheerrorcommittedwhensmallwaveletcoeffi-cientsarediscarded.Wewill referto thoseastheinsignificantwaveletcoefficientswhile thoseremainingwill be referredto asthe significantwavelet coefficients.Thesedefinitionsarerenderedquantitative in thefollowing.

Let ¬ bea giventhresholdfor separatingtheinsignificantwaveletcoefficientsfrom the significantones. We definean index set identifying the indicesof thesignificantwaveletcoefficientsat level H :�®> �P¯�`R°�� o ` o ^ > ��²± i A > 8 1 ip³ ¬£´Hencetheindicesof theinsignificantcoefficientsaregivenbyµ�®> � e>_¶ �®>With thisnotationwecanwrite an ¬ -truncatedwaveletexpansionfor � asfollows· �¡ #" ��¸ ® .�!�� " 7 4=<0 132 e 6 ��7&8 1

�9 ��738 1 ��%; � 4=<0>)2 �)7 01 U�¹=ºX A > 8 1�B%> 8 1 �� (4.6)

Let »½¼& m¬¾� bethenumberof all significantwaveletcoefficients,i.e.

» ¼ [¬�� 4=<0>?2 �)7�¿ ®> ;a^ ��7where ¿ denotesthecardinalityof

®> . The last termcorrespondsto thescalingfunctioncoefficientsthatmustbepresentin theexpansionbecausethey provide

4.2Waveletcompressionerrors 65

thecoarseapproximationonwhich thewaveletsbuild thefinestructures.If we let»¥�À^ � bethedimensionof thefinestspace

�* � thenwecandefinethesymbol »ÂÁto bethenumberof insignificantcoefficientsin thewaveletexpansion,i.e.

» Á [¬��F�K»Ã��» ¼ m¬��Wethendefinetheerrorintroducedby this truncationas�� ® � ��Ä� �¡ " �J� · �¢ " � ¸ ®� � 4=<0>?2 ��7 01 U�Å ºX A > 8 1$B,> 8 1 ��with thenumberof termsbeingequalto »ÂÁ� [¬�� . The2-normof

�� ® � �� is thenfoundas

¤ �� ® � �� ¤ �� ÆÆÆÆÆÆ� 4£<0>)2 �)7 01 U�Å ºX A > 8 1:B%> 8 1 .�!�WÆÆÆÆÆÆ

�� 4=<0>)2 ��7 01 U�Å ºX i A > 8 1 i �o ¬ � » Á m¬��

sowhenever thethresholdis ¬ then¤ �� ® � �� ¤ � o ¬£Ç » Á [¬�� (4.7)

For the infinity norm the situationis slightly different. For reasonsthat willsoonbecomeclearwenow redefinetheindex setas ®> �P¯p`�°p� o ` o ^ > ��Ä± i A > 8 1 ip³ ¬��+^ >?n � ´and

µ ®> � e>_¶ ®> . Notethatwe now modify thethresholdaccordingto thescaleat which thewavelet coefficient belongs.Applying the infinity norm to

�� ® � with


thenew definitionof > yields¤ �� ® � �� ¤ / � � 4=<0>?2 ��7 01 U�Å ºX ORQ#ST i A > 8 1�B,> 8 1 .�� i �

� MFN � 4=<0>?2 �)7 01 U�Å ºX i A > 8 1 i ^ >)n �o MFN � 4=<0>?2 �)7 01 U�Å@ºX ¬� MFN ¬¾»ÂÁ� m¬¾�

sowhenever thethresholdis ¬ then¤ �� ® � �� ¤ / o MFN ¬�»ÈÁ� m¬�� (4.8)

Thedifferencebetween(4.7) and(4.8) lies first andforemostin thefact that thethresholdis scaledin thelattercase.Thismeansthatthethresholdessentiallyde-creaseswith increasingscalesuchthatmorewaveletcoefficientswill beincludedat thefiner scales.Hence » Á [¬�� will tendto besmallerfor the D -normthanforthe ^ -norm.

Remark 4.1 A heuristicwayof estimating»½¼ is givenasfollows: Let �E �� beasmoothfunctionwith onesingularitylocatedat ��@¼ . ThenA > 8 1 ³ ¬ for thoseHand ` where �@¼��w > 8 1 . Hencewehave |P�� significantcoefficientsat each levelyieldinga total of É- �|��a�+� . Henceif � has Ê singularitieswecanestimate» ¼ toberoughlyproportionalto Ê�É� )|P��#�4.3 Scalingfunction coefficientsor function values?

It wasmentionedin Section3.3.3that the fastwavelet transformcanbeapplieddirectly to sequencesof functionvaluesinsteadof scalingfunctioncoefficients.Sincethis is oftendonein practice(see,for example,[SN96,p. 232]),yet rarelyjustified,we considerit here.

Supposethat ��ª�� R� is approximatelyconstanton theinterval w �38 � givenby (2.21),a conditionthatis satisfiedif � is sufficiently largesincethelengthof

4.4A compressionexample 67w �38 � is )|~��+�3�W^ � . Then �E .�!�FËx�E .Ì?�W^ � � for �Í��w �38 � andwe find that

6 �(8 � � ^ � n � © V " Z � �E .�� 9 .^ � �Î��Ì�� A �Ë ^ � n � �E �Ì��+^ � �=© V " Z � 9 �^ � ��ÏÌ?� A �� ^ 4 � n � �E .Ì?�W^ � � © f 4=<e 9 .kl� A kor

6 �38 � Ë�^ 4 � n � �� Ì��+^ � �In vectornotationthis becomes Ð Ë�^ 4 � n ��Ñandfrom (3.32)follows Ò ËÓ^ 4 � n �+Ô Ñ (4.9)

Hence,theelementsin ÕÑ � Ô Ñ behave approximatelylike thosein

Ò(except

for theconstantfactor)when � is sufficiently large.

4.4 A compressionexample

Oneof themostsuccessfulapplicationsof thewavelet transformis imagecom-pression.Much canbesaidaboutthis importantfield, but anexhaustive accountwould be far beyond the scopeof this thesis.However, with the wavelet theorydevelopedthusfar we cangive a simplebut instructiveexampleof theprinciplesbehindwaveletimagecompression.For details,wereferto [Chu97,SN96, H

�94,

VK95] amongothers.Let Ö bean × Ø�» matrix representationof a digital image.Eachelement

of Ö correspondsto onepixel with its valuecorrespondingto thelight intensity.Figure4.1showsanexampleof a digital imageencodedin Ö .

Applying the2D FWT from (3.38)to Ö yieldsÕÖ � ÔÄÙ$Ú Ö · ÔÄÙ$Û ¸ ¹which is shown in Figure4.2 with | �ÝÜ . The upperleft block is a smoothedapproximationof theoriginal image.Theotherblocksrepresentdetailsatvarious


Figure 4.1: A digital imageÞ . Here ßáàaâ+ã+ä�å3æKàaã�ç$â .

Figure4.2: Thewaveletspectrum èÞ . Here é¨ê�à�é�ë�àaì and íKàaä .

4.4A compressionexample 69

Figure4.3: Reconstructedimage î . Compressionratio ç¢ï�ç$ð+ð .scalesandit canbeseenthatthesignificantelementsarelocatedwheretheorig-inal imagehasedges.The largesmoothareasarewell representedby thecoarseapproximationsono furtherdetailsareneededthere.

Assumingthat the imageis obtainedasfunction valuesof someunderlyingfunctionwhich hassomesmoothregionswe expectmany of the elementsin ÕÖto besmall.Thesimplestcompressionstrategy is to discardsmallelementsin ÕÖ ,andfor this purposewe defineÕÖ ®¡ñ trunc ÕÖK'3¬��xò£� ÕÖa�ôó 8 õ ' ]]] � ÕÖ��öó 8 õ ]]] ³ ¬�÷By inverseFWT oneobtainstheapproximationø � · Ô Ù Ú ¸ ¹ ÕÖ ® Ô Ù Ûwhich is shown in Figure4.3.Thethreshold¬ hasbeenchosensuchthatonly �#ùof thewaveletcoefficientsareretainedin ÕÖ ® . Thisgivestheerrorú

FWT � ¤ ø ��Ö ¤ �¤ Ö ¤ � ��Ü�ûö��üJØ�� 4 �Wepointout thatthereis nonormthataccuratelyreflectsthewaypeopleperceivetheerrorintroducedwhencompressinganimage.Theonly reliablemeasureis thesubjectiveimpressionof theimagequality[Str92, p. 364]. Attemptsat“objective”


Figure 4.4: Reconstructedimage ý ¹ÍþÞ ® ý .

errormeasurementcustomarilymakeuseof thepeaksignal-to-noiseratio(PSNR)which is essentiallybasedon the ^ -norm[Chu97, p. 180], [JS94, p. 404].

In orderto assessthemeritsof theproceduredescribedabove,we now repeatit with thediscreteFouriertransform(DFT): UsingtheFouriermatrix ÿ ë definedin (C.3) in placeof Ô weget �Ö �ªÿÎêÎÖªÿ ¹ëRetaining �#ù of the elementsin

�Ö asbeforeandtransformingbackyields theimageshown in Figure4.4. The discardedelementsin this casecorrespondtohigh frequenciesso thecompressedimagebecomesblurred. Thecorrespondingrelativeerroris now ú

DFT ��jû � � Ø�� 4=<In practicalapplicationsonedoesnotusetheDFT ontheentireimage.Rather,

theDFT is appliedto smallerblockswhicharethencompressedseparately. How-ever, this approachcanleadto artefactsat the block boundarieswhich degradeimagequality. In any case,thisexampleservesto illustratewhy thewavelettrans-form is anattractivealternativeto theFouriertransform;edgesandotherlocalizedsmallscalefeaturesareaddedwhereneeded,andomittedwherethey arenot. Thisis all adirectconsequenceof Theorem2.5.

Finally, we mentionthata practicalimagecompressionalgorithmdoesmuchmorethanmerelythrow away smallcoefficients.Typically, moreelaboratetrun-cation(quantization)schemesarecombinedwith entropycodingto minimizetheamountof dataneededto representthesignal[Chu97, Str92, H

�94].

Part II

FastWaveletTransforms onSupercomputers

Chapter 5

Vectorization of the FastWaveletTransform

Problemsinvolving the FWT aretypically large andwavelet transformscanbetime-consumingeven thoughthe algorithmic complexity is proportionalto theproblemsize.Theuseof high performancecomputersis oneway of speedinguptheFWT.

In this chapterandthenext we will describeour efforts to implementthe1Dandthe 2D FWT on a selectionof high-performancecomputers,especiallytheFujitsu VPP300. For simplicity we let � � � (seeDefinition 3.1) throughoutthesechapters.

TheFujitsuVPP300is a vector-parallelcomputer. Thismeansthatit consistsof a numberof vectorprocessorsconnectedin an efficient network. Goodvec-tor performanceon theindividual processorsis thereforecrucial to goodparallelperformance.In this chapterwe discussthe implementationandperformanceoftheFWT on onenodeof theVPP300.In Chapter6 we discussparallelizationoftheFWT andreportresultsof theparallelimplementationonseveral nodeson theVPP300.Someof thematerialpresentedherehasalsoappearedin [NH95].

5.1 The Fujitsu VPP300

Thevectorunits on the VPP300run with a clock frequency of ��¨^ MHz. Theycanexecute � multiplicationsand � additionsperclock cycle, leadingto a peakperformanceof ^�û ^��+^ Gflop/sper processor. The actualperformance,however,dependsonmany factorssuchas

1. vectorlength

2. memorystride

74 Vectorizationof theFastWaveletTransform

3. arithmeticdensity

4. ratioof arithmeticoperationsto load/storeoperations

5. typeof operations

A vectorprocessoris designedto performarithmeticoperationson vectorsofnumbersthroughhardwarepipelining. Thereis an overheadinvolvedwith eachvectorinstruction,sogoodperformancerequireslongvectorlengths.

As with all moderncomputers,thememoryspeedof theVPPfalls shortof theprocessorspeed.To overcomethis problem,memoryis arrangedin bankswithconsecutive elementsspreadacrossthebanks.Stride-oneaccessthenmeansthatthememorybankshave timeto recoverbetweenconsecutivememoryaccessessothat they arealwaysreadyto deliver a pieceof dataat the rateat which it is re-quested.Furthermore,theVPP300hasaspecialinstructionfor thiswhichis fasterthanany non-uniformmemoryaccess.Finally, on computerarchitecturesusingcache,stride-onemeansthatall elementsin a cacheline will beusedbeforeit isflushed.In eithercase,a stridedifferentfrom onecanleadto poorperformancebecausetheprocessorhasto wait until thedataareready.

Optimal performanceon the VPP300requiresthat � multiplicationsand �additionsoccureveryclockcycle. Therefore,any loopcontainingonly additionormultiplicationcannever run fasterthanhalf thepeakperformance.Also, theuseof loadandstorepipesis crucial.TheVPP300hasoneloadandonestorepipe,soadditionof twovectors,say, canatbestrunat1/4of thepeakperformancebecausetwo loadsareneededat eachiteration. Finally, the typeof arithmeticoperationsis crucial to theperformance.For example,a division takesseven cycleson theVPP. Taking all theseissuesinto account,we seethat goodvectorperformancerequiresoperationsof theform

for � �À��°p»Ã��%� õ ñ�� õ ;��where

�and � arescalarsand � and arevectorsof length » , with » beinglarge.

For detailssee[Uni96, HJ88].

5.2 1D FWT

Thebasicoperationin the1D FWT canbewritten in theform

for � �À��°��%�+^½��J� õ ñ ��J� õ ; � � �� õ��

5.21D FWT 75

» � CPUtime(

�s) Mflop/s��j^�� ¨� Ü��WÜ ��^p�^d�� Ü � �+Üj� ��!� �#"��!"dÜ � ^��WÜd�j� �j�d�� ^$" ��"d^ Üdüdüj^$�j� ��"j� � ÜjÜ��Ü � �� jÜ��¨� � ^dÜj� ��j^� ^��WÜ$� ^jÜd^�� Üj� üjÜdüj� ��Ü��Üjüdü � Ü üj^��^!�d�j� ��d� ü��ü� � ��+^ �� jüdÜ!�d� ��" � �d� ü�� ^jÜd^p��$� ^j�$"��¨��!�� üdüj�üj^��¨^$�$� �£�#"��¨^$"jÜd� Ü!�dÜ��Ü Ü��j�

Table 5.1: Timingsof theFWT. íªàaâ+ð , æ~àaâ � , %Jà�ç$ð¾å$ç+ç+å�&'&�&då$ç�( , and é½à)% .

asdefinedby therecurrenceformulas(3.33).Thearithmeticdensityaswell astheratio of arithmeticoperationsto load/storeoperationsaregood. However, mem-ory is accessedwith stride-twobecauseof theinherentdoubleshift in thewavelettransform,andindicesmustbewrappedbecauseof periodicity. Therefore,opti-malperformanceis notexpectedfor the1D FWT.

Ourimplementationof theFWT ononenodeof theVPP300yieldstheperfor-manceshown in Table5.1. We makethefollowing observationsfrom Table5.1:Firstly, theperformanceis far from optimalevenfor thelargestvalueof » . Sec-ondly, theperformanceimprovesonly slowly as » increases.To understandthelatterpropertywe conducta performanceanalysisof onestep(thePWT) of therecurrenceformulas(3.33). Sincethis is a simpleoperationon onevector, weassumethatthecomputationtime in thevectorprocessorfollowsthemodel �+*G¼,;,*.-#� (5.1)

where � is thenumberof floatingpoint operations,

is thetotal executiontime,*/- is thecomputationtime for onefloatingpoint operationin thepipeline,and *G¼is the startuptime. The performanceexpressedin floating point operationspersecondis then µ � � (5.2)

Letting � go to infinity resultsin thetheoreticallyoptimalperformanceµ /10 243 O576 / � � 243 O576 / �*3¼E;,*/-#� � �*/- (5.3)


Let 8~� ��p'��:� be the fraction ofµ / which is achieved for a given problem

size �:9 . Then �:9 is foundfrom (5.2)withµ �;8 µ / :8*/- � � 9*G¼,;,*/-#�:9

whichhasthesolution

�:9{� 8��)8 *G¼*/-In particular, for 8�� #�+^ wefind

� <.n � � *G¼*/-which is anothercharacteristicperformanceparameterfor thealgorithmin ques-tion. � 9 cannow beexpressedin termsof � <.n � as � 9 �<� <.n � 8�� G�È�)8�� . For ex-ample,to reach80% of themaximumperformance,aproblemsizeof �K�+�� <.n �is required,and � �=" � <.n � is neededto reach90 %. The parameter� <.n � canthereforebeseenasa measureof how quickly theperformanceapproaches

µ / .A largevalueof � <.n � meansthat theproblemmustbe very large in orderto getgoodperformance.Hencewe wish � <�n � to beassmallaspossible.

In orderto estimatethecharacteristicparameters� <.n � andµ / we usemea-

surementsof CPU time in the VPP300. Table5.2 shows the timings of the se-quenceof stepsneededto computethe full FWT with Éª�=>�� #" , » � ^ � ,and | � ^d� (the last result in Table5.1), i.e. the PWT appliedsuccesively tovectorsof length ��K»�':»_�+^p'�û�û�û�'&^ . Usingthesemeasurementswe estimatetheparameters* ¼ and * - (in theleastsquaressense)to be*G¼v�?�+Ü � s and */-¢�Ó��û �j��ü�� s

Consequently µ / � PWT � Ü � � Mflop/s and /� <.n � � PWT � � �d^j^d^ Operations (5.4)

It canbe verified that the valuespredictedby the linear modelcorrespondwellwith thoseobservedin Table5.2.Theexecutiontimeof thePWTthusfollowsthemodel

PWT .�F�F�+*G¼,;,*.-#� PWT /�v�

5.21D FWT 77

� � � s� µ Mflop/s�üj^��^!�$� ^d�!"��¨��üd^j� � ^$" �#� Ü � Ü^jÜd^��$� �� jü��+Üj� ��Üdü � " Ü � �� +^ üj^��¨^$�$�j� �d^$� � Ü �$�Üdüdü � Ü ^jÜd^p��$�¨� �=�#�!� Üj^dÜ� ^��WÜ$� � � ��+^j� ^��¨^ Üp��^��Ü � �� Üdüjü � Üj� �d��^j� ü!�dü��#"j^ � ^ �+Ü$�j� ü$"!� ü�� $"jÜ ��Ü � ��¨� � �!� ��+Ü^d��@� ��"d^d� �#�j� �¨üdü��d^�� !"dÜd� � � � � ��üü��^ ^d�� d� " � ^p�d�^düjÜ ��j^�� jü ��^d��^!� üp��^d� �j� Ü��Ü�� ^jüdÜd� üj� � ^� ^ ��^$�d� � � ��Ü��Ü Ü�� Ü �� ^d� � ^ �� Üd� ^!� ^^ �d� ^!" �Table 5.2: Timingsof thePWT. íªàaâ+ð , æ àaâ � , %Rà�ç�( , and é½à)% .

whichcanbeusedto predicttheexecutiontimeof theFWT to depthÉpë asfollowsFWT �»L� � Ù$Û 4£<0 A 2 e PWT

B » ^ A#C� Ù Û 4£<0 A 2 e *G¼,;,*/-#� PWT

B » ^ A#C� *G¼3Épë ;,*/- Ù Û 4=<0 A 2 e � PWT

B » ^ A Cor, if Épë assumesits maximalvalue

FWT �»L�F�D*G¼ 24E!F � » ;,*/-#� FWT �»Í� (5.5)

Figure5.1showsa plot of (5.5) togetherwith themeasurementsfrom Table5.1.


0 1 2 3 4

x 107

0

2

4

6

x 104

FFWT

TF

WT

Execution time, FWT

Model Measurements

Figure 5.1: The execution time for the FWT plotted as a function of floating pointoperations.Thecirclesarethemeasuredtimeswhile theline is theresultof themodel.

We know from (3.37)that � FWT )»L�HGI��|Î» . Hence 24E!F � »Í�3�!� FWT �»Í��C �for » C D , andweseethattheasymptoticperformanceof theFWT is thesameasthatof thePWT: µ / � FWT �x µ / � PWT � �*/-However, the equationto be solved for .� <.n � � FWT is not linear. Let » <.n � be thevector length correspondingto .� <�n � � FWT. Then .� <.n � � FWT � � FWT �» <.n � �a��|� �» <.n � � �#� by (3.37) and putting

µ � µ / �W^ (correspondingto 8 � ��û ü )in (5.2)weget µ / � FWT^ � /� <.n � � FWT �» <.n � �

FWT �» <�n � ��^�*/- � .� <.n � � FWT*G¼ 2JE$F � » <.n � ;K*/-+ /� <.n � � FWT*/-+ /� <.n � � FWT � *G¼ 2JE$F � » <.n �* - ��|� �» <.n � ��#� � * ¼ 2JE$F � » <.n �For thevalues*G¼ and *.- obtainedfrom timingsof thePWT we find theestimate» <.n � �?�$��"�� whichyields .� <.n � � FWT �� .� <.n � � PWT 24E!F � » <.n � �ÓÜd^ � ��^j� operations

Thisagreeswith theobservedvaluesin Table5.1. It is seen,bothfrom theobser-vationsandthemodel,that thevalueof .� <�n � � FWT is very high comparedto that

5.3Multiple 1D FWT 79

of thePWT(5.4). Theexplanationfor this is revealedby theanalysisjust carriedout: A new vectoroperationis startedup for eachPWT so the startuptime *G¼counts24E!F � » timesregardlessof thevectorlength;nomatterhowlongtheinitialvectorlength » , thePWTwill eventuallybeappliedto shortvectors.

Like the stride-twomemoryaccess,this propertyis inherentin the full 1DFWT andwe concludethat this computationis not particularlysuitablefor vec-torization.However, if thedepth É ë is takento besmallerthan > thentheineffi-ciency dueto shortvectorlengthsbecomeslesssevere.

5.3 Multiple 1D FWT

Considera matrix Ö L�� ê 8 ë . We assumethat Ö is storedby columnsso thatconsecutiveelementsin eachcolumnarelocatedin consecutivepositionsin mem-ory. Applying the1D FWT to every columnof Ö leadsto inefficientdataaccessandlarge � <.n � asdescribedin theprevioussection.By applyingtheFWT to therowsof Ö instead,onecanvectorizeover thecolumnssuchthatall elementswillbe accessedwith stride-onein vectorsof length × . We will refer to this pro-cedureasthemultiple 1D FWT (MFWT). Applying theMFWT to a matrix Öcorrespondsto computingtheexpressionÖK Ô Ù Û � ¹ (5.6)

whereÔ Ù Û is definedasin (3.34).Sincethereare × rows,thenumberof floatingpointoperationsneededare� MFWT �×À':»L� �+��|Î×K» · ��+�+^ Ù Û ¸ (5.7)

Therecurrenceformulasnow taketheform

6 A � <ó 8 õ � f 4£<0 � 2 e � � 6 Aó 8 � � � � õ#�M�/NA A � <ó 8 õ � f 4£<0 � 2 e � � 6 Aó 8 � � � � õ#� � N (5.8)

where O��P��'��d'�û�û�û%':É ë �� , PÄ�P��'��j'�û�û�û%'$× �� , and � � ��'��j'�û�û�û�'�� A � < �ª� .Timingsfor theMFWT with É ë �Q>� �� , »¥��^ � , and | ��^d� , whereonly thevectorizeddimension× is varied,areshown in Table5.3.

Wewill now deriveaperformancemodelfor thiscase.Eachstepof theMFWTappliesa PWTof length � A to the × rowsof Ö . Hence,by (3.36)thenumberofflopsare ^�|R� A × . Vectorizationis achievedby putting P into theinnermostloopsocomputationson eachcolumncanbeassumedto follow the linearmodelfor


× � � s� µ Mflop/s��Ü � � �!"��!�� d��^�� ^d�� ^ ^dÜ��$�!�d� ��!�$�$" ^��Ü�� üd^ � �$�WÜd� �d�j� �$� ��#��^!� ��S�Wüdüj^d� �d��S�#� "p��^^düjÜ ^j�$"düp�� @"��=� ��¨�d^ü��^ �£�#"d�j^d�!�d� ^ � ü$"�� !�$��j^�� d��=��Üd� � � ��d� �#"p�#"^d�� Ü��+Üj�$� � ^d� ��@"$"�� #" �#�Table 5.3: Timings of the MFWT. é�ë¥àT%~àÄç$ð , æ à²â � , í à¥â+ð , and ß àç:ä�åGì+â¾å�&'&�&dåGâ#ð#U!V .vectorization(5.1). Hencetheexecutiontime for onestepof theMFWT (5.8) is� A W*G¼%;a^¾|Î×+*/-d� andtheexecutiontime for theentireMFWT is

MFWT � Ù Û 4=<0 A�2 e » ^ A W* ¼ ; ^¾|Î×+* - �� ^¾» B �¡� �^ Ù Û C W*G¼%;a^¾|Î×+*/-d� (5.9)

Theperformanceis thengivenbyµMFWT � � MFWT

MFWT� ^¾|Î×*3¼%;a^�|�×+*/-

and µ / � MFWT � �#�#*/- asusual.However, we observe thattheperformancemea-sureis independentof thedepth Épë . Theparameter× <.n � is foundby solving�^�*/- � ^¾|Î× <.n �*G¼,;a^�|�× <�n � */-whichhasthesolution × <�n � � *3¼^�|X* -Hence .� <�n � � MFWT � �p|�» B �¡� �^ Ù$Û C × <�n �� ^�» B �� ^ Ù$Û C *G¼3�#*/-

5.42D FWT 81

Using(5.9)andthemeasurementsin Table5.3wegetthenew estimates*G¼v� � ûY� � � s and */-¡��û �d�d��üj� � s

Theseestimatesaredifferentfrom thoseof the1D caseandreflectthefactthattheMFWT algorithmperformsbetteron theVPP300.Consequentlywenow have µ / � MFWT � ^�û ^d^j^ Gflop/s and .� <.n � � MFWT � ��Ü!"dü!"d�d�j� Operations

thelattercorrespondingto a vectorlength × <.n � �K^j�$� . Thesevaluesareclosetobeingoptimalon theVPP300(recall that thepeakperformanceperprocessoris^pû ^��W^ Gflop/s).Finally, since /� <.n � � MFWT growswith » we notethattheMFWTis bestfor matriceswith × I�» .

5.4 2D FWT

Recall from Section6.3 that the 2D wavelet transformis definedby the matrixproduct ÕÖ � Ô Ù Ú Ö� Ô Ù Û � ¹ (5.10)

The expressionÖK Ô Ù:Û � ¹ leadsto vector operationson vectorsof length× andstride-onedataaccessasdescribedin Section5.3. This is not the casefor theexpressionÔ Ù$Ú Ö , becauseit consistsof a collectionof columnwise1Dtransformswhichdonotaccessthememoryefficientlyasdescribedin Section5.2.However(5.10)canberewrittenasÕÖ ¹ � · ÖK Ô Ù Û � ¹ ¸ ¹ Ô Ù Ú � ¹yielding theefficiency of themultiple 1D FWT at thecostof onetransposestep.We call this the split-transposealgorithm . It consistsof the following threestages:

Algorithm 5.1: split-transpose

1. Z«�xÖK ÔÄÙ:Û � ¹2. [Ä�QZ ¹3. ÕÖ ¹ �Q[� Ô Ù Ú � ¹


× � � s� µ Mflop/s��Ü ^dü � �d^�� d��"d� � ^p� �� ^ ü��ü!��¨�d� ��^ �#� � �¨��Ü�� "!��W^d� �� dü �W^dÜ��^!� ^j�$��\" � Üd� ��WÜ$�dÜ �d��d�^düjÜ �£�#��¨�dÜ�� ^��¨��Ü�� ü��ü��^ � � �+Ü � ^j�d� �S�WÜ$�dÜ ��WüdÜ��j^�� Ü��+Üj�$� � ^d� "��+ü�� #�j^dÜTable 5.4: Timings of 2D FWT (FWT2). é ë àT%KàÄç$ð , æ à â � , íáà â+ð , andßáà�ç:ä�åGì+â¾å�&'&'&�å$ç:ð+â#U .

Transpositioncanbeimplementedefficiently on a vectorprocessorby accessingthematrixelementsalongthediagonals[Heg95], sothe2D FWT retainsthegoodvectorperformanceof theMFWT. This is verifiedby thetimingsin Table5.4.

Disregardingthe time for the transpositionstep,a simplemodelfor the 2DFWT executiontime is

FWT2 �×À':»L� � MFWT �×À':»L��; MFWT )»J'$×K� (5.11)

where

MFWT is givenby (5.9).Figure5.2showspredictedexecutiontimesversusmeasuredexecutiontimes,andit is seenthat (5.11)predictstheperformanceofthe2D FWT well.

Theasymptoticperformance µ / � FWT2 cannow bedetermined.Using(3.40),(5.2),and(5.11)we getµ

FWT2 � ��|Î×K» �^_��#�W^ Ù$Ú ��#�W^ Ù$Û �^¾» G�¢��+�+^ Ù Û �( W* ¼ ; ^¾|Î×+* - �%;a^�×� 3�¡��#�W^ Ù Ú �( W* ¼ ; ^¾|Î»]* - �Assumingthat É ê � 24E$F � )×K� and É ë � 24E!F � �»L� (themaximaldepths)andthat× and » arelarge,thisyieldstheestimateµ

FWT2 Ë �ê � ë^ f ê½ë * ¼ ;K* - (5.12)

andletting × C D we obtain µ / � FWT2 �»Í�v� �_�`^ f ë ;K*/- (5.13)

5.42D FWT 83

0 5 10 15

x 107

0

3

6

9

x 104 Execution time, FWT2

FFWT2

TF

WT

2

Model Measurements

Figure 5.2: The measuredtimesfor the2D FWT comparedto theperformancemodel( 5.11).

It is seenthatif » goesto infinity aswell, we obtaintheusualexpression( �#��*/- ),but for thepresentcase,with »²�x��j^�� , wehave from (5.12) µ / � FWT2 ��^pû �� Gflop/s

which is only slightly lessthan µ / � MFWT. Similarly, to find /� <.n � � FWT2 for theproblemathandweuse(5.12)and(5.13)to gettheequation^¾|Î»*G¼,;K�p|�»a*/- � �ê ¦Mb § � ë^ f ê ¦Mb § ë * ¼ ;,* -Fromthiswe find × <.n � � *G¼�p|X*/-�;,*G¼ �$» Ë?"��correspondingto .� <�n � � FWT2 �á�jû ü � ^dÜ�ØÀ��$c . It is seenthat this valueis verycloseto thatof theMFWT despitethefactthatthetranspositionstepwasincludedin themeasuredexecutiontimes.However, thecorrespondingvectorlength × <.n �is muchsmallerascanalsobe seenin Table5.4. The reasonfor this is the factthatwheretheMFWT suffersfrom shortvectorlengthswhen × is small,the2DFWT suffersonly in stage � of the split-transposealgorithm(5.1). Stage

�will

vectorizeover » andyield goodperformancein this case. We seethat the 2DFWT performsvery well indeedon theVPP300,evenfor rectangularmatrices.


5.5 Summary

TheFWT hasbeenimplementedon theVPP300. Theone-dimensionalversionhasa relatively high valueof » <�n � andstride-twomemoryaccessso the perfor-manceis notvery good.The2D FWT canbearrangedsothattheseproblemsareavoided,andaperformanceof morethan �j�¾ù of thetheoreticalpeakperformanceis achievedevenfor relatively smallproblems.This is fortunateasthe2D FWTis computationallymoreintensive thanthe1D FWT andconsequently, it justifiesbettertheuseof asupercomputer.

Chapter 6

Parallelization of the Fast WaveletTransform

With a parallelarchitecture,theaim is to distributethework amongseveralpro-cessorsin orderto computetheresultfasteror to beableto solve largerproblemsthanwhat is possiblewith just oneprocessor. Let

e )»L� be the time it takestocomputetheFWT with a sequentialalgorithmon oneprocessor. Ideally, thetimeneededto computethesametaskon � processors1 is then

e �»L�G�d� . However,thereareanumberof reasonswhy this idealis rarelypossibleto meet:

1. Therewill normallybesomecomputationaloverheadin theform of book-keepinginvolvedin theparallelalgorithm.Thisaddsto theexecutiontime.

2. If d is thefractionof theprogramstatementswhichis parallelizablethentheexecutiontimeon � processorsis boundedfrom below by G�F�edd� e �»L�-;d e )»L�3�j� which is alsolarger thantheideal. This is known asAmdahl’slaw .

3. Theprocessorsmightnotbeassignedthesameamountof work. Thismeansthat someprocessorswill be idle while othersaredoing more than theirfair shareof the work. In that case,the parallel executiontime will bedeterminedby theprocessorwhich is thelastto finish. This is known astheproblemof goodloadbalancing.

4. Processorsmustcommunicateinformationandsynchronizein orderfor thearithmeticto beperformedon thecorrectdataandin thecorrectsequence.This communicationand synchronizationwill delay the computationde-pendingon theamountwhichis communicatedandthefrequency by whichit occurs.

1In this chapterf standsfor the numberof processorsandshouldnot be confusedwith thenumberof vanishingmoments.

86 Parallelizationof theFastWaveletTransform

g �� g ��6 ee 6 e < 6 e � 6 eh 6 e^ 6 e p 6 ei 6 e c 6 ej 6 ek 6 e < e 6 e <)< 6 e < � 6 e < h 6 e < ^ 6 e < pl l6 <e 6 << 6 <� 6 <h 6 <^ 6 <p 6 <i 6 <c A <e A << A <� A <h A <^ A <p A <i A <cl6 �e 6 � < 6 �� 6 �h A �e A � < A �� A �h A <e A << A <� A <h A <^ A <p A <i A <cl6 he 6 h < A he A h < A �e A � < A �� A �h A <e A << A <� A <h A <^ A <p A <i A <c

Table 6.1: Standarddatalayoutresultsin poor loadbalancing.Theshadedsub-vectorsarethosepartswhichdonotrequirefurtherprocessing.Here maàaâ , æ~à�ç$ä , and é�àaì .

In thischapterwewill discussdifferentparallelizationstrategiesfor theFWTswith specialregardto theeffectsof loadbalancing,communication,andsynchro-nization. We will disregardtheinfluenceof thefirst two pointssincewe assumethattheparalleloverheadis smallandthattheFWT persehasno significantun-parallelizablepart. However, in applicationsusing the FWT this problemmaybecomesignificant.Mostof thematerialcoveredin thischapterhasalsoappearedin [NH97].

6.1 1D FWT

We will now addresstheproblemof distributing thework neededto computetheFWT ( �� Ô � ) asdefinedin Definition 3.1 on � processorsdenotedby g ��'��j'�û�û�û%':�� . We assumethat theprocessorsareorganizedin a ring topologysuchthat n g �}�$o q and n g ;��o q arethe left andright neighborsof processorg ,respectively. Assumealso,for simplicity, that » is a multiple of � andthat theinitial vector � is distributedsuchthateachprocessorreceivesthesamenumberof consecutiveelements.Thismeansthatprocessorg holdstheelementsò 6 eõ ÷ õ 'p� � g » � ' g » � ;À�d'�û�û�û%'� g ;À�#� » � ��A questionthat is crucial to the performanceof a parallelFWT is how to chosetheoptimaldistributionof andtheintermediatevectors.

Weconsiderfirst thedatalayoutsuggestedby thesequentialalgorithmin Def-inition 3.1. This is shown in Table6.1. It is seenthatdistributing the resultsofeachtransformstepevenly acrosstheprocessorsresultsin a poor loadbalancing

6.11D FWT 87

g �� g ��6 ee 6 e < 6 e q 6 eh 6 e^ 6 e p 6 ei 6 e c 6 ej 6 ek 6 e < e 6 e <)< 6 e < q 6 e < h 6 e < ^ 6 e < pl l6 <e 6 << 6 <q 6 <h A <e A << A <q A <h 6 <^ 6 <p 6 <i 6 <c A <^ A <p A <i A <cl l

6 qe 6 q < A qe A q < A <e A << A <q A <h 6 qq 6 qh A qq A qh A <^ A <p A <i A <cl l6 he A he A qe A q < A <e A << A <q A <h 6 h < A h < A qq A qh A <^ A <p A <i A <c

Table6.2: Goodloadbalancingis obtainedby usingadifferentdatalayout.Theshadedsub-vectorsarethosepartswhichdonot requirefurtherprocessing.Again,we have maàâ , æPàÓç$ä , and é�àaì .becauseeachstepworkswith thelowerhalf of thepreviousvectoronly. Thepro-cessorscontainingpartsthatarefinishedearlyareidle in thesubsequentsteps.Inaddition,globalcommunicationis requiredin thefirst stepbecauseeveryproces-sor mustknow the valueson every otherprocessorin orderto computeits ownpartof thewavelet transform.In subsequentstepsthis communicationwill takeplaceamongtheactiveprocessorsonly. Thiskind of layoutwasusedin [DMC95]whereit wasobservedthatoptimalloadbalancingcouldnotbeachieved,andalsoin [Lu93] wheretheglobalcommunicationwastreatedby organizingtheproces-sorsof a connectionmachine(CM-2) in apyramidstructure.

However, we canobtainperfectload balancingandavoid global communi-cationby introducinganotherorderingof the intermediateandresultingvectors.This is shown in Table6.2. Processorg will now computeandstoretheelementsò 6 A � <õ ÷ õ and ò A A � <õ ÷ õ where

� � g »�È^ A � < ' g »�Â^ A � < ;��j'�û�û�û%'Hr g ;��s »�Â^ A � < �� (6.1)O²� �p'��d'�û�û�ûE'$É_��


Let now � qA �T� A �d� � »_� r)�Â^ A s . Thenthe recurrenceformulasarealmostthesameas(3.33):

6 A � <õ � f 4=<0 � 2 e � � 6 A � � � q õ�� NA A � <õ � f 4=<0 � 2 e � � 6 A � � � q õ�� N (6.2)

where O��P��'��d'�û�û�û%':ÉJ�� and �� g � qA � < ' g � qA � < ;K�j'�û�û�û%'#r g ;��st� qA � < �� . Thedifferencelies in the periodicwrappingwhich is still global, i.e. elementsfromprocessor� mustbe copiedto processor� � � . However, it turnsout that thisis just a specialcaseof thegeneralcommunicationpatternfor thealgorithms,asdescribedin Section6.1.1.

Notethatthelayoutshown in Table6.2is apermutationof thelayoutshown inTable6.1 becauseeachprocessoressentiallyperformsa local wavelet transformof its data.However, theorderingsuggestedby Table6.1andalsoby Figure3.3is by no meansintrinsic to theFWT sothis permutationis not a disadvantageatall. Rather, onemightargueasfollows:

Local transformsreflectbettertheessenceof thewaveletphi-losophybecauseall scaleinformationconcerninga particularpositionremainson thesameprocessor.

This layout is even likely to increaseperformancefor further processingsteps(suchascompression)becauseit preserveslocality of thedata.

Notealsothatthelocal transformsin thisexamplehave reachedtheirultimateform on eachprocessorafter only threestepsandthat it would not be feasibleto continuethe recursionfurther(i.e. by letting É��u� andsplitting ò 6 he ' 6 h < ÷ intoò 6 ^e ' A ^e ÷ ) becausethen »_� r)�È^ Ù sHGK� , (6.1)no longerholds,andtheresultingdatadistributionwould leadto loadimbalanceaswith thealgorithmmentionedabove.Thus,to maintaingoodloadbalancingwemusthaveanupperboundon É :É o 24E!F q B » � CIn fact, this boundhasto be even more restrictive in order to avoid excessivecommunication.Wewill returnto this in Section6.1.1.

6.11D FWT 89v w7xzy|{~}��v w7x��{~}�� S�M�/� �|� §��/�

v w7x��{ }�� \�

w�}��

w }�� #�

� xzyW�:

�:

Figure6.1: Computationsonprocessor� involve íI�Îâ elementsfrom processor�H�ç .Here íÄà«ä and æ��S�Mm�â AJ� à�V . The lines of width í indicatethe filters asthey areappliedfor differentvaluesof � .

6.1.1 Communication

We will now considertheamountof communicationrequiredfor theparallel1DFWT. Considerthecomputationsdoneby processorg onarow vectorasindicatedin Figure6.1.Thequantitiesin (6.2)canbecomputedwithoutany communicationprovidedthattheindex Ì5;a^!� doesnot referto elementsonotherprocessors,i.e.Ìl; ^$� o r g ;��s »�Â^ A �� o r g ;��s »�Â^ A � < � Ì5;��^ (6.3)

A sufficientcondition(independentof Ì ) for this is

� o r g ;��s »�Â^ A � < � | ^ (6.4)

sinceÌ�La��':|x�a�:� . Weusethiscriterionto separatethelocalcomputationsfromthosethatmayrequirecommunication.

For a fixed � ³ r g ;}��sG»_� r��Â^ A � < s!� |Î�+^ computationsarestill localaslongas(6.3) is fulfilled, i.e.when

Ì o r g ;��s »�Â^ A �a^$�J�� (6.5)

However, when Ì becomeslargerthanthis, theindex Ì�;�^!� will point to elementsresidingon a processorlocatedto the right of processorg . The largestvalueof

90 Parallelizationof theFastWaveletTransformÌ5;a^!� (foundfrom (6.2)and(6.1)) isORQ#S7r.Ì5;a^!��sF�<r g ;��s »�Â^ A ;�|P� �(6.6)

Thelargestvalueof Ì5;a^!� for which communicationis notnecessaryis

r g ;��s »�Â^ A ��Subtractingthis quantityfrom (6.6)we find thatexactly |~�}^ elementsmustbecommunicatedto processorg at eachstepof theFWT asindicatedin Figure6.1.

A tighter bound on ÉIt is a conditionfor goodperformancethat thecommunicationpatterndescribedabove takesplacebetweennearestneighborsonly. Therefore,we want to avoidsituationswhereprocessorg needsdatafrom processorsotherthanits right neigh-bor n g ;��o q soweimposetheadditionalrestrictionORQ+S7r�Ì5; ^$�~s o r g ; ^�s »�Â^ A ��r g ;À��s »�È^ A ;}|P� � o r g ; ^�s »�Â^ A ��|P�a^ o »�Â^ A (6.7)

Sincewe want(6.7) to hold for all O �À��'��j'�û�û�û�':É �� weget|P�a^ o »�Â^ Ù 4=<from whichwe obtainthefinal boundon É :É o 2JE$F q B ^�»r)|~�Ï^�sG� C (6.8)

For »¥��^jüdÜ , | �;� , �K��Ü , for example,we findÉ o üTheboundgivenin (6.8)is notasrestrictiveasit mayseem:Firstly, for theap-

plicationswhereaparallelcodeis calledfor, onenormallyhas »T�áORQ+S7r��F'$|Rs ,secondly, in mostpracticalwavelet applicationsonetakes É to be a fixed smallnumber, say � –ü [Str96], andthirdly, shouldthe needarisefor a large valueofÉ , onecouldusea sequentialcodefor thelaststepsof theFWT asthesewill notinvolve largeamountsof data.


6.2 Multiple 1D FWT

The considerationsfrom the previous sectionarestill valid if we replacesingleelementswith columns. This is a parallel versionof the MFWT describedinSection5.3. Figure6.2shows thedatalayoutof theparallelMFWT algorithm.

� � � � � � � ¡ ¢ £ ¤¥ ¦ § ¨ © £ � � ª © ¢ «

< q heê

ë

Figure 6.2: Multiple FWT. Dataaredistributedcolumnwiseon the processors.TheFWT is organizedrowwisein orderto accessdatawith strideone.

Theamountof necessarycommunicationis now ×<r)| �a^ s elementsinsteadof | �a^ , thecolumnsof Ö aredistributedblockwiseon theprocessors,andthetransformationof therowsof Ö involvestherecursionformulascorrespondingtoÖ Ô ¹ë . Therecursionformulastakethesameform asin Section5.3. Theonlydifferencefrom thesequentialcaseis that � is now givenasin (6.1).

We arenow readyto give the algorithmfor computingonestepof the mul-tiple 1D FWT. The full transformis obtainedby repeatingthis stepfor O��p'��d'�û�û�û%':É�P� . The algorithmfalls naturally into the following threephases:

1. Communication phase: |��Í^ columnsarecopiedfrom theright neighborasthesearesufficient to completeall subsequentcomputationslocally. Wedenotethesecolumnsby theblock ¬6 A � 8 e � f 4 h .

2. Fully local phase:Theinteriorof eachblockis transformed,possiblyover-lappingthecommunicationprocess.

3. Partially remotephase: Whenthecommunicationhascompleted,there-maining elements are computed using ¬6 A � 8 � � q õ 4 ë n s q q N t wheneverÌ5;a^!�I »_�@r.Â^ A s .


Algorithm 6.1: MFWT , level ®°¯ ®�±³²� qA � ëq q Ng � “my processor id” La�� °��:�!———————————–! Communication phase!———————————–send 6 A � 8 e � f 4 h to processor n g ��$o qreceive¬6 A � 8 e � f 4 h fr om processor n g ;À��o q!————————————–! Fully local phase, cf. (6.4)!————————————–for �L��_°S� qA �+^È��|Î�+^6 A � <� 8 õ �?´ f 4£<� 2 e � � 6 A � 8 � � q õ ! O 34µ r.Ì5;a^!��sF��A A � <� 8 õ �;´ f 4=<� 2 e � � 6 A � 8 � � q õ ! ORQ#S7r.Ìl; ^$�~s��Q� qA ��end

!———————————————————————–! Partially remote phase! communication must be finished at this point!———————————————————————–for �L�Q� qA �W^ ��|Î�+^È;�� °S� qA �W^ ��

!—————————! Local part, cf (6.5)!—————————6 A � <� 8 õ �?´Q¶$·N 4 q õ 4£<� 2 e � � 6 A � 8 � � q õ ! O 34µ r.Ì5;a^!��sF�Q� qA ��|�;a^A A � <� 8 õ �;´ ¶$·N 4 q õ 4=<� 2 e � � 6 A � 8 � � q õ ! ORQ#S7r.Ìl; ^$�~s��Q� qA ��!————————————–! Remote part, use ¬6 > � 8 e � f 4 h!————————————–6 A � <� 8 õ � ´ f 4£<� 2 ¶ ·N 4 q õ � � ¬6 A � 8 � � q õ 4 ¶@·N ! O 34µ r.Ì5;a^!��sF�Q� qAA A � <� 8 õ �;´ f 4=<� 2 ¶ ·N 4 q õ � � ¬6 A � 8 � � q õ 4 ¶@·N ! ORQ#S7r.Ìl; ^$�~s��Q� qA ;�|P� �

end


6.2.1 Performancemodel for the multiple 1D FWT

Thepurposeof this sectionis to focuson the impactof theproposedcommuni-cationschemeon performancewith particularregardto speedupandefficiency.Wewill considerthetheoreticallybestachievableperformanceof themultiple1DFWT algorithm.Recallthat(5.6)canbecomputedusing

� MFWT r)»¸sF�+��|Î×K» B �¡� �^ Ù Û C (6.9)

floatingpointoperations.We emphasizethedependency on » becauseit denotesthedimensionoverwhich theproblemis parallelized.

Let *º¹ betheaveragetime it takesto computeonefloatingpointoperationonagivencomputer2. Hence,thetimeneededto compute(5.6)sequentiallyis�e

MFWT r)»¸sF�Q� MFWT r�»»s|* ¹ (6.10)

andthetheoreticalsequentialperformancebecomesµ eMFWT r)»¸sF� � MFWT r�»»s e

MFWT r�»»s (6.11)

In ourproposedalgorithmfor computing(5.6)theamountof doubleprecisionnumbersthatmustbecommunicatedbetweenadjacentneighborsat eachstepofthewavelettransformis ×<r�|��Ï^�s asdescribedin Section6.2.Let * � bethetimeit takesto initiate the communication(latency) and *º¼ the time it takesto sendonedoubleprecisionnumber. Sincethereare É stepsin thewavelet transformasimplemodelfor thetotal communicationtime isM

MFWT �xÉ½r�* � ;�×¾r�|P�a^�s¿*t¼�s (6.12)

Notethat

MMFWT grows linearly with × but that it is independentof thenumber

of processors aswell asthesizeof theseconddimension» !Combiningtheexpressionfor computationtime andcommunicationtime we

obtaina modeldescribingthetotalexecutiontimeon processors( ³ � ) as qMFWT r�»»sv� e

MFWT r�»¸s ; M MFWT (6.13)

2This modelfor sequentialperformanceis simplifiedby disregardingeffectsarisingfrom theuseof cachememory, pipeliningor superscalarprocessors.Adverseeffectsresultingfrom sub-optimaluseof thesefeaturesareassumedto beincludedin À�Á to give anaverageestimateof theactualexecutiontime. Thus,if we estimateÀ�Á from thesequentialmodel(6.10),it will normallybesomewhat larger thanthe nominalvaluespecifiedfor a givencomputer. In caseof the linearmodelfor vectorperformance(5.1)wegetfor exampleÀ ÁHÂ À¿Ã/Ä�ÅRÆÇÀ�È .


andtheperformanceof theparallelalgorithmisµÊÉMFWT r�»»sÌË � MFWT r)»¸s É

MFWT r�»»s (6.14)

Theexpressionsfor performancein (6.11),(6.14),and(6.13)leadto aformulafor thespeedupof theMFWT algorithm:Í É

MFWT r.Î»szË�ÏÑÐMFWT r.Î»sÏ ÉMFWT r.Î»s Ë Ò�Ó �Ô MFWT Õ Ï ÐMFWT r.Î»sTheefficiencyof the parallelimplementationis definedasthespeedupperpro-cessorandwe haveÖ�É

MFWT r.Î»sÌË Í ÉMFWT r/Î¸s Ë ÒÒ�Ó �Ô MFWT Õ Ï ÐMFWT r/Î¸s (6.15)

It canbeseenfrom (6.15)that for constantÎ , theefficiency will decreasewhenthenumberof processors× is increased.

We will now investigatehow the above algorithmscaleswith respectto thenumberof processorswhenthe amountof work per processoris held constant.Thus let ÎÙØ be the constantsizeof a problemon oneprocessor. Thenthe to-tal problemsize becomesÎ Ë ×°ÎÙØ and we find from (6.9) and (6.10) thatÏ ÐMFWT Ú ×�ÎXØ'ÛÜË × Ï ÐMFWT Ú ÎÙØtÛ becausethe computationalwork of the FWT islinearin Î . Thismeansin turnthattheefficiency for thescaledproblemtakestheform Ö É

MFWT Ú ×°Î Ø ÛÝË ÒÒÞÓ É7ßMFWTÉáà�â

MFWT ãåä~æ�ç Ë ÒÒÞÓ Ô MFWT Õ Ï ÐMFWT Ú ÎÙØ'ÛSince

Ö ÉMFWT Ú ×�ÎXØ'Û is independentof × thescaledefficiency is constant.Hence

themultiple1D FWT algorithmis fully scalable.

6.32D FWT 95

6.3 2D FWT

In this sectionwe will considertwo approachesto parallelizethesplit-transposealgorithmfor the2D FWT asdescribedin Section5.4.

Thefirst approachis similar to theway 2D FFTscanbeparallelized[Heg96]in thatit usesthesequentialmultiple1D FWT andaparalleltransposealgorithm;wedenoteit thereplicatedFWT . Thesecondapproachmakesuseof theparallelmultiple1D FWT describedin Section6.2to avoid theparalleltransposition.Wedenotethisapproachthecommunication-efficientFWT .

In bothcaseswe assumethat thetransformdepthis thesamein eachdimen-sion, i.e. è)ËTè�éêËëè ä . Thenwe get from (3.40)and(5.7) that thesequentialexecutiontime for the2D FWT is

Ï ÐFWT2 Ú Î¸ÛzËíì Ï ÐMFWT Ú Î»Û'î (6.16)

6.3.1 ReplicatedFWT

Themoststraightforwardway of dividing the work involvedin the2D FWT al-gorithmamonga numberof processorsis to parallelizealongthefirst dimensionin ï , suchthata sequenceof 1D row transformsareexecutedindependentlyoneachprocessor. This is illustratedin Figure6.3. Sincewe replicateindependentrow transformson the processorswe denotethis approachthe replicatedFWT(RFWT)algorithm. Hereit is assumedthatthematrix ð is distributedsuchthat

ñ ò ó ô õ ö÷ ø ù ú û õ ò ü ý û ô þ ÷ ø ù ú û õ ò ü ý û ô þù õ ÿ þ � � ô � ò ÿ ü ü ò � �ñ ò ó ô õ öÿ ü ü ò � �� æ��

��

�� æ�

Figure 6.3: ReplicatedFWT. Theshadedblock movesfrom processor�

to � .eachprocessorreceivesthesamenumberof consecutive rowsof ð . Thefirst andthelaststagesof Algorithm 5.1arethusdonewithoutany communication.How-ever, theintermediatestage,thetransposition,causesasubstantialcommunicationoverhead.A further disadvantageof this approachis the fact that it reducesthemaximalvectorlengthavailablefor vectorizationfrom to Õ × (andfrom Îto Î Õ × ). This is a problemfor vectorarchitecturessuchastheFujitsuVPP300asdecribedin section5.3.


P1

P2

P3

P4

2 3 4

1 3 4

1

1

2

2

4

3

Figure6.4: Communicationof blocks,first block-diagonalshaded.

A similarapproachwasadoptedin [LS95] wherea2D FWT wasimplementedon theMasPar - a dataparallelcomputerwith 2048processors.It wasnotedthat“the transposeoperationsdominatethe computationtime” anda speedupof nomorethan timesrelative to thebestsequentialprogramwasachieved.

A suitableparalleltransposealgorithmneededfor thereplicatedFWT is onethatmovesdatain wrappedblockdiagonalsasoutlinedin thenext section.

Parallel transposition and data distrib ution

Assumethat the rows of the matrix ï aredistributedover the processors,suchthat eachprocessorgets Õ × consecutive rows, and that the transposeï à

isdistributedsuchthat eachprocessorgets Î Õ × rows. Imaginethat the part ofmatrix ï that resideson eachprocessoris split columnwiseinto × blocks,assuggestedin Figure6.4, thenthe blocksdenotedby � aremoved to processor�during the transpose.In total eachprocessormustsend × � Ò

blocksandeachblockcontains Õ × times Î Õ × elementsof ï . Hence,following thenotationinSection6.2.1,wegetthemodelfor communicationtimeof aparalleltranspositionÔ RFWT � Ú ×�� Ò Û �� Ó QÎ×�� (6.17)

Notethat Ô RFWT grows linearlywith , Î and × (for × large).

Performancemodel for the replicatedFWT

We arenow readyto derive a performancemodelfor the replicatedFWT algo-rithm. Using(6.16)and(6.17)we obtaintheparallelexecutiontimeas

Ï��RFWT Ú Î»Û � Ï ÐFWT2 Ú Î¸Û× Ó Ô RFWT

andthetheoreticalspeedupfor thescaledproblem Î � ×°Î Ø isÍ �RFWT Ú ×�Î Ø Û � ×Ò�Ó Ô RFWT Õ Ï ÐFWT2 Ú ÎXØ'Û (6.18)

Wewill returnto thisexpressionin Section6.3.3.

6.32D FWT 97

6.3.2 Communication-efficientFWT

In thissectionwecombinethemultiple1D FWT describedin Section6.2andthereplicatedFWT ideadescribedin Section6.3.1to geta 2D FWT that combinesthebestof bothworlds. Thefirst stageof Algorithm 5.1 is computedusingtheparallelmultiple 1D FWT asgivenin Algorithm 6.1,so consecutive columnsofð mustbedistributedto theprocessors.However, the last stageusesthe layoutfrom the replicatedFWT, i.e. consecutive rowsaredistributedto theprocessors.This is illustratedin Figure6.5.

� � � � � � ! " � # $ " � ! %÷ ø ù ú û õ ò ü ý û ô þ ÷ ø ù ú û õ ò ü ý û ô þÿ ü ü ò � � ÿ ü ü ò � �ù õ ÿ þ � � ô � òñ ò ó ô õ ö ñ ò ó ô õ ö

& ' (��

��

�� æ�

Figure6.5: Communication-efficientFWT. Datain shadedblockstayonprocessor� .Themainbenefitusingthis approachis that the transposestepis donewith-

out any communicationwhatsoever. Theonly communicationrequiredis thatofthemultiple 1D FWT, namelythe transmissionof Ú*) �+ì Û elementsbetweennearestneighbors,somostof thedatastayon thesameprocessorthroughoutthecomputations.The resultwill thereforebe permutedin the Î -dimensionasde-scribedin Section6.1andorderednormally in theotherdimension.We call thisalgorithmthecommunication-efficientFWT (CFWT) .

Theperformancemodelfor thecommunication-efficientFWT is astraightfor-wardextensionof theMFWT becausethecommunicationpartis thesame,sowegetthetheoreticalspeedupÍ �CFWT Ú ×�ÎXØ'Û � ×Ò�Ó Ô MFWT Õ Ï ÐFWT2 Ú ÎXØ'Û (6.19)

where Ô MFWT and Ï ÐFWT2 Ú Î Ø Û areasgivenin (6.12)and(6.16)respectively.

6.3.3 Comparisonof the 2D FWT algorithms

We cannow comparethe theoreticalperformanceof the RFWT (6.18) and theCFWT(6.19)with regardto their respectivedependencieson × and Î Ø .


0 20 40 60 80 100 120 1400

20

40

60

80

100

120

140Scaled speedup

P

SP

Comm−effic.Replicated Perfect

Figure 6.6: The theoretical scaled speedup of the replicated FWT algo-rithm and the communication-efficient FWT shown together with the line ofperfect speedup. The predicted performancescorrespond to a problem with+ , - �/.1032 , �/.541076 , �/.

. The characteristicparameterswere measuredon an IBM SP2to be 8 � , �:9 .7; s

0 8 � , . �5� ; s0 8=< , >�?

s. The performanceofthecommunication-efficient FWT is muchcloserto the line of perfectspeedupthantheperformanceof thereplicatedFWT, andtheslopeof thecurve remainsconstant.

In caseof the CFWT theratio Ô MFWT Õ Ï ÐFWT2 Ú Î Ø Û is constantwith respectto× whereasthecorrespondingratio for theRFWTin (6.18)goesas @ Ú ×�Û :Ô RFWTÏ ÐFWT2 Ú Î Ø ÛBA Ú ×�� Ò Û �� Ó �DC Ø� � QÎ Ø ��E ) QÎ Ø � @ Ú ×�ÛThis meansthat the efficiency of the RFWT will deteriorateas × grows whileit will stayconstantfor the CFWT. The correspondingspeedupsareshown inFigure6.6.

When × is fixedandtheproblemsize Î Ø grows,then Ô MFWT Õ Ï ÐFWT2 Ú Î Ø Û goesto zero,which meansthat the scaledefficiency of the CFWT will approachtheidealvalue

Ò. For theRFWT thecorrespondingratio approachesa positive con-

stantas Î Ø grows: Ô RFWTÏ ÐFWT2 Ú Î Ø Û F Ú ×G� Ò Û ��E ) × � for Î Ø F H

6.32D FWT 99I 2Mflop/s Efficiency (%) Estim.eff.� �/.J4 �/45K� �/.J4 �/L > MJ> 9 �/4 M L 9 > �. . -J> K � � 4J. 9 .ON 4J. 9 �/.N - �/. > � � 4J. 9 � � 4J. 9 �/.4 � � .ON �J� MJM 41� 9 M � 4J. 9 �/.� > . � NP4 .ON �J� 41� 9 M L 4J. 9 �/.KJ. NQ� MJ> NPL MJ> 41� 9 M � 4J. 9 �/.

Table6.3: Communication-efficientFWT ontheSP2.2 ,RI 2 Ø , 2 Ø , �/.J4 , +S, �T.J4

,6 , � � . IU, � signifiessequentialperformance.Estimatedefficienceis given asV �CFWT W I 2 ØYX[Z I whereV �CFWT W I 2 Ø[X is givenasin (6.19).

This meansthatthescaledefficiency of theRFWT is boundedby a constantlessthanone– no matterhow largetheproblemsize.Theasymptoticscaledefficien-ciesof thetwo algorithmsaresummarizedbelow× F H Î Ø F H

ReplicatedFWT:ÒÒÞÓ @ Ú ×�Û ÒÒ�Ó ã �DC Ø ç]\_^`*a � �

Communication-efficientFWT:

ÒÒ�Ó ßMFWTà â

FWT2 ã ä~æ�ç Ò

6.3.4 Numerical experiments

Wehave implementedthecommunication-efficientFWT ontwo differentMIMDcomputerarchitectures,namelytheIBM SP2andtheFujitsuVPP300.OntheSP2we usedMPI for theparallelismwhereastheproprietaryVPP Fortranwasusedon theVPP300.

TheIBM SP2isaparallelcomputerwhichisdifferentfrom theVPP300.Eachnodeon the SP2is essentiallya workstationwhich doesnot achieve the perfor-manceof a vectorprocessorsuchastheVPP300.High performanceon theSP2must thereforebe achieved througha higher degreeof parallelismthanon theVPP300and scalability to a high numberof processorsis more urgent in thiscase.Themeasuredperformanceson theIBM SP2areshown in Table6.3.


0 5 10 15 20 25 30 350

5

10

15

20

25

30

35Scaled speedup

P

SP

TheoreticalMeasured Ideal

Figure 6.7: Scaledspeedup,communication-efficient FWT (IBM SP2). Thesegraphsdepicthow thetheoreticalperformancemodeldoes,in fact, give a realisticpredictionoftheactualperformance. I 2

Mflop/s Efficiency (%)� - �/. �/K �J�� - �/. �/.JLJ4 M 4 9 K1�. � � .ON . -J- � M 4 9 �/.N . � NP4 - � - 4 M L 9 .JL4 N � MJ> � � �T4 > M L 9 M NTable 6.4: Communication-efficientFWT on theVPP300.

2cb I,+d,R- �/.

,6 , � � .Ie, � signifiessequentialperformance.

It is seenthattheperformancescaleswell with thenumberof processorsand,furthermore,thatit agreeswith thepredictedspeedupasshown in Figure6.7.Theparallelperformanceon theFujitsuVP300is shown in Table6.4.Wehavenotes-timatedthecharacteristicnumbers

��gf��Tf�� < for this machine,but it is neverthelessclearthattheperformancescalesalmostperfectlywith thenumberof processorsalsoin thiscase.

6.4Summary 101

6.4 Summary

Wehave developeda new parallelalgorithmfor computingthe2D wavelettrans-form, thecommunication-efficientFWT.

Our new approachavoids the useof a distributedmatrix transposeandper-forms significantlybetterthanalgorithmsthat requiresucha transpose.This isdueto the fact that the communicationvolumeof a parallel transposeis largerthannecessaryfor computingthe2D FWT.

Thecommunication-efficientFWT is optimalin thesensethatthescaledefficiency is independentof thenumberof proces-sorsandthatit approaches

Òastheproblemsizeis increased.

ImplementationsontheFujitsuVPP300andtheIBM SP2confirmsthescalabilityof theCFWTalgorithm.

Part III

Waveletsand Partial DifferentialEquations

Chapter 7

Waveletsand differentiationmatrices

7.1 Previouswaveletapplications to PDEs

Even thoughthe field of wavelet theoryhashada greatimpacton otherfields,suchassignalprocessing,it is not yet clearwhetherit will have a similar impactonnumericalmethodsfor solvingpartialdifferentialequations(PDEs).

In the early ninetiespeoplewerevery optimistic becauseit seemedthat thenice propertiesof waveletswould automaticallyleadto efficient solutionmeth-ods for PDEs. The reasonfor this optimismwas the fact that many nonlinearPDEshave solutionscontaininglocal phenomena(e.g.formationof shocks)andinteractionsbetweenseveral scales(e.g. turbulence). Suchsolutionscan oftenbe well representedin wavelet bases,as explainedin the previous chapters. Itwasthereforebelievedthatefficientwavelet-basednumericalschemesfor solvingthesePDEswould follow from waveletcompressionproperties[BMP90,LT90b,LPT92,HPW94, GL94,CP96,PW96, Fh 96, DKU96].

However, this earlyoptimismremainsto behonored.Waveletshave not hadthe expectedimpacton differentialequations,partly becausethe computationalwork is not necessarilyreducedby applyingwaveletcompression- even thoughthesolutionis sparselyrepresentedin a waveletbasis.In thefollowing chapterswe will discusssomeof the mostpromisingapproachesandshedsomelight ontheobstaclesthatmustbe overcomein orderto obtainsuccessfulwavelet-basedPDEsolvers.

Schematically, wavelet-basedmethodsfor PDEscanbeseparatedinto thefol-lowing classes:

106 Waveletsanddifferentiationmatrices

Class1: Methodsbasedon scalingfunction expansions

The unknown solution is expandedin scalingfunctionsat somechosenlevel iandis solved usinga Galerkinapproach.Becauseof their compactsupport,thescalingfunctionscanberegardedasalternativesto splinesor thepiecewisepoly-nomialsusedin finite elementschemes.While this approachis importantin itsown right, it cannotexploit waveletcompression.Hencemethodsin thiscategoryarenot adaptive [LT90a,QW93,WA94, LR94,Jam93,RE97]. However, this ap-proachhasmany pointsof interest.LelandJameson[Jam93]hasshown thatoneobtainsamethodwhichexhibitssuperconvergenceat thegrid points,theorderofapproximationbeingtwice aslargeasthatof the projectionof thesolutionontothe spacespannedby scalingfunctions. JohanWalden [Wal96] hasshown thatthesizeof thedifferentiationfilters grows fasterthantheoptimal centeredfinitedifferencemethodof thesameorder. In the limit, astheordergoesto infinity, itis shown that jlknm Ú=o �Rp ÛYk Ú=o ÛQq o F Ú � Ò ÛYr Õ p for ) F H . Finally, we mentionthatTeresaReginskaandothers[RE97,EBR97] usescalingfunctionsto regular-ize thesolutionof thesidewaysheatequation.This is anill-posedproblemin thesensethatthesolutiondoesnot dependcontinuouslyon theinitial condition. Byexpandingthe solutionin scalingfunctions,high frequency componentscanbefilteredaway andcontinuousdependenceof theinitial conditionis restored.

Class2: Methodsbasedon waveletexpansions

The PDE is solved usinga Galerkinapproachasin the first class. In this case,however, theunknownsolutionisexpressedin termsof waveletsinsteadof scalingfunctionssowaveletcompressioncanbeapplied;eitherto thesolution[LT90b],thedifferentialoperator[BCR91,Bey92, EOZ94, XS92], or both [CP96, BK97,PW96]. Severaldifferentapproacheshavebeenconsideredfor exploiting thespar-sity of a wavelet representation.Oneis to performall operationsin thewaveletdomain[BN96, BMP90,Wal96]. Theoperatorsaresparsebut thenumberof sig-nificantcoefficientsin the solutions ÎBs Úut Û hasto be very small comparedto thedimensionof theproblemÎ for themethodsto beefficient. Thisis especiallytruefor non-linearoperations.However, recentwork by Beylkin andKeiser[BK97]suggeststhatsomenonlinearitiesmaybeefficiently treatedin thewaveletdomain.

In anotherapproachlinearoperationssuchasdifferentiationaredonein thewaveletdomainandnonlinearoperationssuchassquaringin thephysicaldomain.This is by far themostcommonapproachandit is usedby [Kei95, CP96, FS97,PW96, EOZ94,VP96] andtheauthor. This approachinvolvesa numberof trans-formationsbetweenthe physicaldomainand the wavelet domainin eachtimestep,andthis canintroduceconsiderableoverhead.Hence,thewaveletcompres-sionpotentialof thesolutionmustbevery largefor this approachto befeasible.

7.1Previouswaveletapplicationsto PDEs 107

An interestingaspectof thewaveletapproachis thatcertainoperatorsrepre-sentedwith respectto awaveletbasisbecomesparserwhenraisedto higherpow-ers[EOZ94]. Fromthispropertyonecanobtainanefficienttime-steppingschemefor certainevolutionequations.Thismethodhasbeenemployedin [CP96,Dor95]to solve theheatequation.

Finally, we mention that several papershave exploited the so-calledNon-standard or BCRform of thedifferentialoperator[BCR91, Bey92, Dor95,BK97].This form is sparserthanthestandardform, but to applythenon-standardform ofthedifferentialoperatorto a functiononeneedsto know, at all scales,thescalingfunctioncoefficientsof thefunctionaswell asits waveletcoefficients.Thisrepre-sentationis redundantand,eventhoughthefunctionmaybesparsein its waveletrepresentation,thescalingfunctionrepresentationmaynotbesparse.

Wewill referto methodsfrom Classv and ì asprojection methods.

Class3: Waveletsand finite differ ences

In thethird approachwaveletsareusedto derive adaptive finite differencemeth-ods.Insteadof expandingthesolutionin termsof wavelets,thewavelettransformis usedto determinewherethefinite differencegrid mustberefinedor coarsenedto optimally representthesolution. Thecomputationaloverheadis low becauseoneworkswith pointvaluesin thephysicalrepresentation.Oneapproachwasde-velopedby LelandJameson,[Jam94,Jam96] underthenameWaveletOptimizedFinite Dif ferenceMethod (WOFD). Walden describesa filter bank methodin[Wal96], MattsHolmstromhasintroducedtheSparsePointRepresentation(SPR)in [Hol97], andalso[PK91] have usedwaveletsto localizewhereto apply gridrefinement.

Class4: Other methods

Thereareafew approachesthatusewaveletsin waysthatdonotfit into any of thepreviousclasses.Examplesareoperatorwavelets[JS93,EL], anti-derivativesofwavelets[XS92], themethodof travelling wavelets[PB91, WZ94], andwavelet-preconditioning[Bey94].

Operatorwaveletsarewaveletsthatare(bi-)orthogonalwith respectto anin-nerproductdesignedfor the particulardifferentialoperatorin question.This isnot a generalmethodsinceit worksonly for certainoperators.Theuseof anti-derivativesof waveletsis similar to thatof operatorwavelets.

In the methodof travelling wavelets,an initial condition is expandedusingonly a few waveletswhich arethenpropagatedin time. A disadvantageof this


methodis that thesefew waveletsmaybe unableto expressthe solutionafter ithasbeenpropagatedfor a time.

Finally, in [Bey94] it is shown that any finite differencematrix representa-tion of periodizeddifferentialoperatorscanbepreconditionedso that thecondi-tion numberis @ Ú v�Û . Furthermore,if the differencematrix is representedin awavelet basis(the standardform) then the preconditioneris a diagonalmatrix.Thus,waveletsplay anauxiliary role in that they provide a meansto reducetheconditionnumberof theoperator.

In this chapterandChapter8 we describethenecessarybackgroundfor methodsbelongingto Class v and ì . In Chapter9 we give someapplicationsof thesemethodsto PDEs,andwealsoillustratetheWOFDmethodfrom Classw . Wewillnotdiscussfurthermethodsbelongingto class4.

7.2 Connectioncoefficients

A naturalstartingpoint for theprojectionmethodsis thetopic of two-termcon-nectioncoefficients. Thedescriptionadoptedhereis similar to theonedescribedin [LRT91] and [Bar96]. An alternative methodis describedin [Kun94]. Wedefinetheconnectioncoefficientsasx � æ[y � �z y � y { ��|~}C } k ã

� æ�çz y � Ú=o Û[k ã � � çz y { Ú�o Û:q o f��1f��*f��where qSØ and q � areordersof differentiation.We will assumefor now thatthesederivativesarewell-defined.

Usingthechangeof variableo�� Ú ì z o � � Û oneobtainsx � æ[y � �z y � y { � ì z � |~}C } k ã� æ�ç Ú�o ÛYk ã � � ç Ú�o � � Ó � ÛQq o � ì z � x � æ[y � �� y � y { C �

where q � q Ø Ó q � . Repeatedintegrationby partsyields the identityx � æ[y � �� y � y � �Ú ��v�Û � æ x � y �� y � y � becausethescalingfunctionshavecompactsupport.Hencex � æ*y � �z y � y { � Ú ��v�Û � æ ì z � x � y �� y � y { C �

Thereforeit is sufficient to consideronly oneorderof differentiationÚ qSÛ andoneshift parameterÚ � � � Û andwe definex �� |~}C } k Ú=o Û[k ã

� ç� Ú=o ÛQq o f ��(7.1)

7.2Connectioncoefficients 109

Consequently x � æ[y � �z y � y { � Ú ��v�Û � æ ì z � x �{ C � (7.2)

andwenotethepropertyx �� Ú ��v�Û � x � C � f �� ì�� ) f ) �)ìJ� (7.3)

which is obtainedusingthe changeof variable o�� o � � in (7.1) followedbyrepeatedintegrationby parts.

In the following we will restrictour attentionto the problemof computing(7.1).Thesupportsof k and k ã � ç� overlaponly for � Ú[) �Üì Û�� ) �Üì sothereare ì ) �Rw nonzeroconnectioncoefficientsto bedetermined.Let� � �� x ��T� a C �� C aandassumethat k � � � Ú�¡ Û . Thentaking the identity (3.21) with

� � v anddifferentiatingit q timesleadstok ã � ç� Ú=o Û � a C Ø¢ r � �¤£ r k ã

� çØ y � � h¥r Ú�o Û � ì �T¦ ì a C Ø¢ r � �¤£ r k ã� ç� � h¥r Ú ì o Û (7.4)

Substitutingthedilationequation(2.17)and(7.4) into (7.1)yieldsx �� |~}C }§ ¦ ì a C Ø¢ ¨ � � £ ¨ k ¨ Ú ì o Ûª© § ì �O¦ ì a C Ø¢ s � � £ s[k ã � ç� � h s Ú ì o Ûu©«q o� ì � h Ø a C Ø¢ ¨ � � a C Ø¢ s � � £ ¨ £ s |�}C } k

¨ Ú ì o Û[k ã � ç� � h s Ú ì o Û:q o f o¬� ì o� ì � a C Ø¢ ¨ � � a C Ø¢ s � � £ ¨ £ s | }C } k¨ Ú=o ÛYk ã � ç� � h s Ú�o ÛQq o f o¬�o �R®� ì � a C Ø¢ ¨ � � a C Ø¢ s � � £ ¨ £ s | }C } k Ú=o Û[k ã

� ç� � h s C ¨ Ú�o ÛQq oor a C Ø¢ ¨ � � a C Ø¢ s � � £ ¨ £ s x �� h s C ¨ � vì � x �� f �¯�� ì�� ) f ) �,ì5� (7.5)


Let

� � ì � Ó«° �±® . We know thatx �� is nonzeroonly for

�� ì²� ) f ) �,ì5� andthat

° � ® Ó � �)ì � aswell as ® mustberestrictedto

��³Qf ) ��v/� . This is fulfilledfor ´¶µJ· Ú ³¸f ì � � � Û3��®��´º¹_» Ú*) �Üì f ) �Üì Ó ì � � � Û . Let ¼ � ì � � � anddefine£¾½ �

¨ � ã ½ ç¢¨ � ¨ æ/ã ½ ç £¨ £ ¨ C ½

where ®�Ø Ú ¼ Û � ´¶µ5· Ú ³¸f ¼7Û and ® � Ú ¼7Û � ´º¹¿» Ú[) �Àv f ) ��v Ó ¼ Û . Hence(7.5)becomes a C �¢� � � C a £ �

� C � x �� vì � x �� f ��R� ì�� ) f ) �)ìJ�Thematrix-vectorform of this relationisÁ�Â �)ì C �/ÃDÄ � � �ÆÅ (7.6)

whereÂ

is a Ú ì ) �Rw�Û7Ç Ú ì ) �Rw�Û matrixwith theelements� Â � � y � � £ � � C � f �*f�� ì�� ) f ) �)ì5�Wenotethefollowing propertiesof £ :È Becauseof theorthogonalityproperty(2.22)we have£¾½ �ÊÉ v for ¼ � ³³

for ¼ ��Ë ì f Ë�Ì f Ë �î�î#îÈ Also, £5½ � £ C ½ soweneedonly compute£¾½ for ¼¬Í ³ .È Finally, aconsequenceof Theorem2.3 is that¢½ odd£ ½ � v

Henceall columnsaddto one,which meansthatÂ

hastheleft eigenvector� v f v f î�î#î f vT� correspondingto theeigenvalue v , q � ³ in (7.6).

Consequently,Â

hasthestructureshown herefor ) � :ÎÏÏÏÏÏÏÏÏÏÏÏÏÐ³ £1Ñ³ £¾Ò ³ £¸Ñv £ Ø ³ £:Ò ³ £QÑ³ £ Ø v £ Ø ³ £1Ò ³ £1Ñ³ £¾Ò ³ £ Ø v £ Ø ³ £¾Ò ³£1Ñ ³ £:Ò ³ £ Ø v £ Ø ³£¸Ñ ³ £1Ò ³ £ Ø v£QÑ ³ £¾Ò ³£1Ñ ³

ÓÕÔÔÔÔÔÔÔÔÔÔÔÔÖ

7.2Connectioncoefficients 111

Equation(7.6) hasa non-trivial solution if ì C � is an eigenvalueofÂ

. Nu-mericalcalculationsfor ) �×Ì f f î#î#î f w ³ indicatethat ì C � is an eigenvalueforq � ³Qf v f î�î#î f ) ��v andthatthedimensionof eachcorrespondingeigenspaceisv . Henceoneadditionalequationis neededto normalizethesolution.

To this endwe usethepropertyof vanishingmoments.Recallthat × � )ÙØ ì .Assumingthat qÛÚ × wehave from (2.25)thato � � }¢�� C }

�� k Ú�o � � ÛDifferentiatingbothsidesof this relation q timesyieldsq¥Ü � }¢�� C }

�� k ã � ç Ú=o � � ÛMultiplying by k Ú=o Û andintegratingwe thenobtainq¥Ü |~}C } k Ú�o Û:q o � }¢�Ý� C }

�� |~}C } k Ú=o ÛYk ã� ç Ú�o � � ÛQq o� a C �¢�Ý� � C a

�� | }C } k Ú�o ÛYk ã� ç Ú�o � � ÛQq o

Hencewe get a C �¢�� C a �� x �� q¥Ü (7.7)

which closesthe thesystem(7.6). Thecomputationof themomentsneededforthisequationis describedin Appendix(A).

� �is thenfoundasfollows: Let Þ � be

aneigenvectorcorrespondingto theeigenvalue ì C � in (7.6). Then� � � p¥Þ � for

someconstantp , which is fixedaccordingto (7.7).

TheMatlabfunctionconn( q , ) ) computes� �

for a givenwaveletgenus) . ✤

Remark 7.1 There is oneexceptionto thestatementthat ì C � is an eigenvalueofÂfor q � ³Qf v f î#î#î f ) ��v : Let ) ��Ì . Thentheeigenvaluesof

ÂarevE f vÌ Ó Sî Ì¸ß PàáÇâv ³ C1ã � f vÌ �R�î ÌQß ¾àºÇâv ³ C1ã � f vì f v

ConsequentlyØä is notaneigenvalueofÂ

andconnectioncoefficientsfor thecom-bination ) ��Ì f q � ì are notwell defined.


7.2.1 Differ entiability

The questionof differentiability of k (andhenceå ) is nontrivial and not fullyunderstood[SN96, p. 199–200],seee.g.[Dau92,p. 215–249]and[SN96, p. 242–245] for adiscussion.However, somebasicresultsaregivenin [Eir92] andstatedin Table7.1. Thespace

�Bæ Ú=¡ Û denotesthespaceof functionshaving continuous

) ì Ì E v ³ v#ì v Ì vç v E ì ³è � ³ v v v v ì ì ì ìé ³ ³ v v ì ì ì ì w wTable 7.1: Regularity of scalingfunctionsand wavelets,where ê 0�ëíì�î æ Wgï X andê 0�ëðìÙñÛò Wgï X .derivativesof order � è . Thespaceó ò Ú�¡ Û is a Sobolev spacedefinedasó ò Ú�¡ Û ��Pô ��õ � Ú�¡ Û3ö ô ã � ç �âõ � Ú�¡ Û f ÷ q ÷ � é �see,e.g.[Fol95, p. 190]. This latterconceptis ageneralizationof ordinarydiffer-entiability, henceè � é .

As we shallseenumericalexperimentsrevealthattheconnectioncoefficientswork for higherordersof differentiationthanthosespecifiedin Table7.1. SeeSection7.4.

7.3 Differ entiation matrix with respectto scalingfunc-tions

Let ô be a function in øúù�û � � Ú�¡ Û , i �Êü � . The connectioncoefficientsde-scribedin theprevioussectioncanbeusedto evaluatethe q th orderderivativeofô in termsof its scalingfunctioncoefficients.Dif ferentiatingbothsidesof (2.11)q timeswe obtain ô ã � ç Ú=o Û � }¢�Ý� C }þý ù y

� k ã � çù y � Ú=o Û f o � ¡ (7.8)ô ã � ç will in generalnotbelongto ø¥ù sowe project ô ã � ç backonto ø¥ùÚ ×�ÿ�� ô ã � ç Û Ú=o Û � }¢r � C } ý ã

� çù y r k ù y r Ú=o Û f o � ¡ (7.9)

7.3Dif ferentiationmatrixwith respectto scalingfunctions 113

where,accordingto (2.12),

ý ã� çù y r ��|R}C } ô ã

� ç Ú�o ÛYkDù y r Ú�o Û:q o (7.10)

Substituting(7.8) into (7.10)we find

ý ã� çù y r � }¢�� C } ý ù y

� | }C } k ù y r Ú=o Û[k ã� çù y � Ú=o ÛQq o� }¢�� C } ý ù y

� x � y �ù y r y �� }¢�� C } ý ù y� ì ù � x �� C r� }¢� � C } ý ù y � h¥r ì ù

� x �� f � H Ú�p�Ú HWe used(7.2) for the secondlast equality. Since

x �� is only nonzerofor

� �� ì�� ) f ) �)ìJ� we find that

ý ã� çù y r � a C �¢� � � C a ý ù y � húr ì ù

� x �� f i f p �� (7.11)

Recallfrom (2.60)thatif ô is v -periodicthen

ý ù y� � ý ù y

� h ½ � � f �[f ¼ ��and

ý ã� çù y r � ý ã

� çù y r�h ½ � � f p f ¼ ��Hence,it is sufficientto considerì ù coefficientsof eithertypeand(7.11)becomes

ý ã� çù y r � a C �¢� � � C a ý ù y � � h¥r � � � ì ù

� x �� f p � ³Qf v f î#î#î f ì ù � v (7.12)

Thissystemof equationscanberepresentedin matrix-vectorform� ã � ç �� ã � ç � (7.13)


where � � ã � ç � r y � � húr � � � � ì ù � x �� f p � ³¸f v f î#î#î f ì ù ��v f� � ì�� ) f w � ) f î#î#î f ) �Kìand � ã � ç � �

ý ã� çù y � f ý ã

� çù y Ø f î�î#î f ý ã� çù y � � C Ø � .We will refer to thematrix � ã � ç asthediffer entiation matrix of order q . It

canbeseenfrom (7.3) that � ã � ç is symmetricfor q evenandskew-symmetricforq odd. Furthermore,it follows that � ã � ç is circulantasdefinedin Definition C.1andthat it hasbandwidthì ) ��w . The differentiationmatrix hasthe followingstructure(shown for ) �~Ì and i � w ):� ã � ç � ì Ò

ÎÏÏÏÏÏÏÏÏÏÏÐ� ^â � ^ æ � ^� � � � ã C Ø ç ^ � ^� ã C Ø ç ^ � ^ æã C Ø ç ^ � ^ æ � ^â � ^ æ � ^� � � � ã C Ø ç ^ � ^�ã C Ø ç ^ � ^� ã C Ø ç ^ � ^ æ � ^â � ^ æ � ^� � � �� ã C Ø ç ^ � ^� ã C Ø ç ^ � ^ æ � ^â � ^ æ � ^� � �� ã C Ø ç ^ � ^� ã C Ø ç ^ � ^ æ � ^â � ^ æ � ^� �� ã C Ø ç ^ � ^� ã C Ø ç ^ � ^ æ � ^â � ^ æ � ^�� ^� � � � ã C Ø ç ^ � ^� ã C Ø ç ^ � ^ æ � ^â � ^ æ� ^ æ � ^� � � � ã C Ø ç ^ � ^� ã C Ø ç ^ � ^ æ � ^â

Ó ÔÔÔÔÔÔÔÔÔÔÖAn importantspecialcaseis q � v , andwedefine� �� ã Ø ç (7.14)

7.4 Differ entiation matrix with respectto physicalspace

Wewill restrictourattentionto theperiodiccase.Recallfrom Section3.2.3that� � � f � �� C Ø �where

�arethegrid valuesof a function ô ��øúù definedon theunit interval. � is

thevectorof scalingfunctioncoefficientscorrespondingto ô , and is thematrixdefinedin (3.16)which mapsfrom thescalingfunctioncoefficientsontothegridvalues.Similarly, theprojectionof ô ã � ç onto

�øúù satisfies� ã � ç � � ã � ç

7.4Dif ferentiationmatrixwith respectto physicalspace 115

Thenit followsfrom (7.13)that� ã � ç �� ã � ç C Ø �Wecall � ã � ç thedifferentiationmatrixwith respectto thecoefficientspace,by

virtueof (7.13),and �� ã � ç C Ø thedifferentiationmatrixwith respectto physicalspace. The matrices and � ã � ç areboth circulantwith the samedimensionsÚ ì ) �Gw�Û�Ç Ú ì ) �~w�Û , so they arediagonalizedby thesamematrix, namelytheFourier matrix � � a C Ò definedin (C.3). Therefore,they commute,accordingtoTheoremC.5,andwefind that�� ã � ç C Ø �� ã � ç � C Ø �� ã � çand � ã � ç �� ã � ç � (7.15)

Hence� ã � ç is thedifferentiationmatrixwith respectto bothcoefficientspaceandphysicalspace.

If ô is an arbitrary1-periodicfunction then� ã � ç will generallynot be exact

dueto approximationerrorsandwe define� ã � ç Ú ô f iÝÛ � ´¶µ5·r � � y Ø y��gy � � C Ø �� ã� ç�� r � ô ã � ç � pì ù � ��

Jameson[Jam93] establishesthefollowing convergenceresultfor thedifferentia-tion matrix � ã Ø ç : ô �� a Ú=¡ Û � � ã Ø ç Ú ô f iÝÛ¤� � ì C ù a (7.16)

where

�is a constant.Assumethatsimilar resultshold for higherordersof dif-

ferentiation,i.e. � ã � ç Ú ô f iÝÛ � � ì C ù�� f � � ¡ f q��v (7.17)

where

�possiblydependson q and ) . Taking the logarithmon both sidesof

(7.17)yields �! #" � Ú � ã � ç Ú ô f iÝÛtÛ � �$ %" � Ú � ì C ù�� Û � �$ #" � Ú � Û � i �andwecanfind theconvergencerate

�from thedifferences�$ #" � Ú � ã � ç Ú ô f iÝÛtÛ � �$ %" � Ú � ã � ç Ú ô f i Ó v�ÛºÛ � �


)�& q v ì w Ì à ß E ' v ³ vPv v�ìÌ Ì � ì � � � � � � � � � Ì Ì ì ì � � � � � � �E E Ì Ì ì ì � � � � �v ³ v ³ E E Ì Ì ì ì � � �v#ì v#ì v ³ v ³ E E Ì Ì ì ì �Table 7.2: Convergencerates ( (rounded)for the differentiationmatrix shown fordifferent valuesof

6and ordersof differentiation ) . Here, * W,+ X ,.-0/21 W N%3 + X , and4�, K:0[NQ0 9/9�9 0�4 . Note that the case

6 , NQ0 ) , .is not valid as mentionedin Re-

mark7.1.

Numericalresultsaresummarizedin Table7.2andit is verifiedthat(7.16)applies.For higherordersof differentiation,thereis still convergence,but with lowercon-vergencerates.We observe that

�appearsto begivenby� � ) �)ì65uq Ø ì#7 f q � ³Qf v f î�î#î f ) ��v

Moreover, we observe thatnumericaldifferentiationis successfullycarriedout tomuchhigherordersthanthosesuggestedin Table7.1.

Theconvergencerate

� � ) for q � v canbeachievedalsofor higherordersby redefiningthedifferentiationprocessfor qºÍ�v . Let� ã � ç �� i.e. the q th derivative of ô is approximatedby repeatedapplicationof the firstorderdifferentiationmatrix. Define� ã � ç Ú ô f iÝÛ � ´¶µJ·r � � y Ø y��gy � � C Ø �� ã

� ç � r � ô ã � ç � pì ù � ��Then � ã � ç Ú ô f iÝÛ¤� � ì C ù � f � � ¡with

� � ) . This is confirmedin Table7.3.Let 8 Ú � � Û denotethebandwidthof � � . We know from (7.13)that 8 Ú � Û �ì ) �Rw , andthegeneralformulaturnsout to be:8 Ú � � Û � ´º¹¿» Ú q98 Ú � Û �Rq Ó v f ì ù Û

7.4Dif ferentiationmatrixwith respectto physicalspace 117

)�& q v ì w Ì à ß E ' v ³ v¾v v#ìÌ Ì Ì Ì Ì Ì Ì Ì Ì Ì Ì Ì Ì E E E E E E E E E E E E Ev ³ v ³ v ³ v ³ v ³ v ³ v ³ v ³ v ³ v ³ v ³ v ³ v ³v�ì v�ì v�ì v�ì v#ì v#ì v#ì v�ì v�ì v�ì v�ì v#ì v#ìTable 7.3: Convergencerates(rounded)for : � shown for differentvaluesof

6and

ordersof differentiation) . Again, * W�+ X ,;-0/21 W N%3 + X , and4º, K10*NQ0 9T9�9 0�4 .

)�& q v ì w Ì à ß E ' v ³ v¾v v#ìÌ à ' vOw v ß ì¸v ìPà ì ' wPw w ß Ì v Ì à Ì ' ' v ß ì¾à w¾w Ì v Ì ' à ß Pà ß w E v E#' ' ßE vçw ì¾à w ß Ì ' ¸v ß w E à ' ß v ³ ' v�ì¸v vOwPw v Ì àv ³ v ß w¾w Ì ' ¾à E v ' ß v¾vçw v�ì ' v Ì à vç¸v v ßPß v ' wv�ì ìQv Ì v ¸v E v v ³ v v#ìQv v Ì v vç¸v v E v ì ³ v ì$ìQv ì Ì vTable7.4: Bandwidthsof : � shown for thevaluesof ) and

6in Figure7.3.

Hencetheimprovedconvergencefor q¬Ícv comesat thecostof increasedband-width. Table 7.4 gives the bandwidthscorrespondingto the convergenceratesgivenin Table7.3assumingthat i is sufficiently large.

The Matlab function diftest( ) ) computesthe convergenceratesfor � ã � ç ✤and � � .7.4.1 Differ entiation matrix for functions with period <If thefunctionto bedifferentiatedis periodicwith period

õ,ô Ú�o Û ��ô Ú�o Ó õ Û f o �=�

thenwe canmaponeperiodto theunit interval andapply thevarioustransformmatricesthereasdescribedin Section3.2.4.Thus,let > � o Ø õ anddefine? Ú >áÛA@ ô Ú õ > Û ��ô Ú=o Û


Then ? is v -periodicandwe definethevectorB � � ? � f ? Ø f î#î�î f ? � � C Ø � by? r � Ú ×DCÿ�� ? Û Ú p Ø ì ù Û f p � ³¸f v f î#î�î f ì ù ��vLet

� � B . Then�

approximatesô Ú�o Û at o � p õ Ø ì ù . From(7.15)we haveB ã � ç �� ã � ç BHence,by thechainrule ô ã � ç Ú=o Û � võ � ? ã � ç Ú >áÛwehave � ã � ç � võ � B ã � ç � võ � � ã � ç B � võ � � ã � ç � (7.18)

The Matlab function difmatrix( q , E ,

õ, ) ) computesthe differentiation✤

matrix � ã � ç Ø õ � for waveletgenus) andsize E andperiod

õ.

7.5 Differ entiation matrix with respectto wavelets

We will restrict our attentionto the periodiccase. Let ô be a function in

�øúù .Proceedingasin theprevioussection,wedifferentiatebothsidesof (2.53) q timesandobtain ô ã � ç Ú=o Û � � � â C Ø¢ �� ý ù â y

��k ã � çù â y � Ú�o Û Ó ù C Ø¢z � ù â �GF C Ø¢ �� q z y �H�å ã � çz y � Ú�o Û (7.19)

Projectingô ã � ç onto

�øúù yields

Ú ×�ÿ�� ô ã � ç Û Ú�o Û � � � â C Ø¢ �Ý� � ý ã� çù â y � �kDù â y � Ú=o Û Ó ù C Ø¢z � ù â �IF C Ø¢ �Ý� � q ã � çz y � �å z y � Ú=o Û

where

ý ã� çù â y � � |~}C } ô ã

� ç Ú=o Û �k ù â y � Ú=o ÛQq o f � � ³¸f v f î#î�î f ì ù â ��vq ã � çz y � � |~}C } ô ã� ç Ú=o Û �å z y � Ú=o ÛQq o f � � i � f i � Ó v f î#î#î f i¶��v� � ³¸f v f î#î#î f ì z ��v (7.20)

7.5Dif ferentiationmatrixwith respectto wavelets 119

Recallthatgiven thescalingfunctioncoefficientsof ô ã � ç on the finestlevel, theFWT (3.33)canbeusedto obtainthewaveletcoefficientsabove. HenceJ ã � ç �LK � ã � çwhere� ã � ç is definedasin Section7.3and

J ã � ç containsthecoefficientsin (7.20).Using(7.13)and(3.35)wethenobtainJ ã � ç �LKM� ã � ç � �LKN� ã � ç K àOJor J ã � ç � P� ã � ç J (7.21)

wherewehave defined P� ã � ç �LKQ� ã � ç K à(7.22)P� ã � ç is the differentiationmatrix with respectto the wavelet coefficients. We

observe that P� ã � ç is obtainedasa 2D FWT of thedifferentiationmatrix � ã � ç .In contrastto � ã � ç , thematrix P� ã � ç is not circulant. Instead,it hasa charac-

teristicfingerbandpatternasillustratedin Figure(7.1). As will bedescribedin

Chapter8, onecantakeadvantageof thisstructureto compute P� ã � ç efficiently.


8 16 32 64

8

16

32

64

nz = 1360

V3

~W

3

~W

4

~W

5

~

V3

~

W3

~

W4

~

W5

~

Figure 7.1: Sparsitypatternof R: ã � ç when6 , N

, S , �,4¬,�>

, T , K , ) , � . Thepatternis identicalfor )�U � .

Chapter 8

2D FastWaveletTransform of acirculant matrix

In this chapterwe will describean algorithmfor computingthe2D fastwavelettransform(2D FWT) of a circulant E ÇVE matrix

Â. The2D FWT is definedin

(3.38)andcirculantmatricesarediscussedin AppendixC.Recallthatthe2D FWT is amapping

Â F W givenas

W �XK Â KQYwhereK is definedasin (3.34).Wewill show thatthiscanbedonein @[Z0E]\ stepsandthat W canberepresentedusing @^Z_E`\ elementsin a suitabledatastructure.Usingthis structurewe describeanefficientalgorithmfor computingthematrix-vectorproduct

WLawhere a is an arbitraryvectorof length E . This algorithmalso hascomplex-ity @[Z0E]\ andwill be usedin Chapter9 in a wavelet methodfor solving PDEsdevelopedin 9.3.1.

This chapteris basedon [Nie97] andtheapproachfollows ideasproposedbyPhilippeCharton[Cha96]. However, many of the details,thedatastructureandthecomplexity analysisare,to ourknowledge,presentedherefor thefirst time.

8.1 The wavelet transform revisited

Let E and b beintegersof theform givenin Definition 3.1 andlet �Hc bea givenvectorwith elementsd ý ce�f ý c ghfjijikilf ý cmonpgrq andlet

J c bedefinedsimilarly.

122 2D FastWaveletTransformof a circulantmatrix

Recallthatthe1D FWT is defined(andcomputed)by therecurrenceformulasfrom Definition3.1: s c2t gu v w npgx y{z e}| y s c ~ y tp� u�� c2t gu v w npgx y{z e}� y s c ~ y tp� uA�,� �for � vQ� fj�#fjikiji�f b]� � , � vM� fk�#fjijikilf�� c2t g � � , and � c v�� c . Figure8.1illustrateshow a vectoris transformed.

� e� g� g

� g� ��

� g� ��

� � �

Figure 8.1: Thestepsof a1D wavelettransformfor � ,�� and � ,�� .The2D FWT definedin (3.38)hasalsoarecursiveformulation.The2D recur-

renceformulasarestraightforwardgeneralizationsof the1D formulas,andtheydecomposea matrix analogouslyto theway the1D recurrenceformulasdecom-poseavector.

The recursionis initialized by assigninga matrix (assumedto be squareforsimplicity) � ¡ m£¢ m to the initial block1 which we denote¤¥¤ e¦¢ e . This blockis thensuccessively split into smallerblocksdenoted¤¥¤ c ¢ § , ¨]¤ c ¢ § , ¤O¨ c ¢ § , and¨©¨ c ¢ § for � frª v �#f � fjijiki�f b . Theblock dimensionsaredeterminedby thesuper-scripts� frª : A blockwith indices� frª has� c v«�� c rowsand � § v«�� § columns.

1Wepointout that,throughoutthischapter, weusetwo-charactersymbolsfor certainmatricesandblock-matrices.

8.1Thewavelettransformrevisited 123

Thestepsof the2D wavelettransformfor b v�¬ areshown in Figure8.2with beingthe aggregationof all blocksafter the final stepshown in the upperrightcornerin thefigure. (Comparisonwith the 1D wavelet transformshown in Fig-ure8.1canbehelpful for understandingthis scheme).Notethateachstepof thetransformproducesblocksthat have attainedtheir final values,namelythoseofthe type ¨©¨ c ¢ § , andsubsequentstepswork on blocksof the type ¤¥¤ §G¢ § , ¤O¨ c ¢ § ,and ¨®¤ c ¢ § . Theformulasfor thesethreetypesof blocksaregivenbelow.

¤¥¤°¯,¯

±

�

²¤¥¤´³,³ ¤O¨µ³,³¨]¤]³,³ ö¨�³,³ ¨©¨�³,³

¨©¨�³,³

ö¨;¶,¶¤¥¤[¶,¶¨]¤^¶,¶

¤O¨�¶,¶ ¤O¨=¶ ³¨©¨·¶ ³

ö¨µ³ ¶¨]¤`³ ¶

ö¨;¶,¶ ¨©¨·¶ ³ö¨µ³ ¶

¸�¸º¹$¹»�» ¹$¹¸¼»�¹$¹ »0¸ ¹!¹¸¼»�½$¹¸¼»p¾ ¹

¸�¸º½$¹¸�¸¿¾ ¹

¸�¸º¹$½ ¸À¸O¹ ¾»0¸ ¹!½ »_¸ ¹ ¾�

Figure 8.2: The stepsof a 2D wavelet transformfor � ,L�and � ,X�

. Theresultingmatrix Á hasthecharacteristic“arrow-shaped”blockstructure.


¤¥¤ c ¢ c � ¤¥¤ cÂtg0¢ c2t g

¨®¤ c2t g0¢ c2t g¤O¨ c2t g0¢ c2t g¨©¨ c2t g_¢ cÂt g

Figure 8.3: The transformof a squareblock on the diagonalyields four new squareblocks.Thisoperationcorrespondsto thedecompositionsgivenin ( 8.1) to ( 8.4).

Blockson the diagonal

Considerthe squareblocks ¤¥¤ c ¢ c in Figure 8.2. Eachstepof the 2D wavelettransformsplitssuchablock into four new blocksdenoted¤¥¤ c2t g0¢ c2t g , ¤O¨ c2t g0¢ c2t g ,¨®¤ c2t g0¢ c2t g , and ¨o¨ cÂt g0¢ c2t g . Figure8.3 illustratesthedecompositionof this type.Let ÃlÃ c ¢ §u ¢ Ä v dÂ¤¥¤ c ¢ § q u ¢ Ä andsimilarly for Ã9Å c2t gu ¢ Ä , ÅÆÃ c2t gu ¢ Ä , and ÅOÅ c2t gu ¢ Ä . Therecur-renceformulasfor this decompositionarethengivenasfollows:

ÃlÃ c2t g0¢ c2t gu ¢ Ä v w npgx yÇz e w npgx ÈÉz e | y | È ÃlÃ c ¢ c~ y tp� u£�,� � ¢ ~ È tp� Ä �� (8.1)

Ã9Å c2t g0¢ c2t gu ¢ Ä v w npgx yÇz e w npgx ÈÉz eÊ| y � È ÃlÃ c ¢ c~ y tp� u£� � � ¢ ~ È tp� Ä � � � (8.2)

ÅÆÃ c2t g0¢ c2t gu ¢ Ä v w npgx yÇz e w npgx ÈÉz e � y | È ÃlÃ c ¢ c~ y tp� u£� � � ¢ ~ È tp� Ä � � � (8.3)

Å�Å c2t g0¢ c2t gu ¢ Ä v w npgx yÇz e w npgx ÈÉz e � y � È ÃlÃ c ¢ c~ y tp� u£� � � ¢ ~ È tp� Ä � � � (8.4)

for � v�� fj�#fkijijilf bË� � and � fhÌ v�� fj�#fjikijilf�� c2t g � � .

8.1Thewavelettransformrevisited 125

¨]¤ c ¢ § � ¨]¤ c ¢ § t g ¨©¨ c ¢ § t g

Figure8.4: Transformof ablockbelow thediagonalyieldstwo new rectangularblocks.Thisoperationcorrespondsto therow transformsgivenin ( 8.5)and( 8.6).

Blocksbelowthe diagonal

Blocksof the type ¨]¤ c ¢ § (ª�Í � ) aresplit in onedirectiononly asindicatedinFigure8.4. Therecurrenceformulasarethe1D formulasappliedto eachrow oftheblock ¨]¤ c ¢ § :

ÅÊÃ c ¢ § t gu ¢ Ä v w npgx ÈÉz eÊ| È ÅÆÃ c ¢ §u ¢ ~ È tÎ� Ä ��ÐÏ (8.5)

ÅOÅ c ¢ § t gu ¢ Ä v w npgx ÈÉz e � È ÅÆÃ c ¢ §u ¢ ~ È tÎ� Ä ��ÐÏ (8.6)

for ª v � f �ÒÑ �#fjijikilf b�� , � v�� fk�#fjijikilf�� c � � , and Ì v� fj�%fjijiji¿f�� § t g � � .


¤O¨ c ¢ § � ¤O¨ c2t g_¢ §¨©¨ cÂt g0¢ §

Figure8.5: Transformof ablockabove thediagonalyieldstwo new rectangularblocks.Thisoperationcorrespondsto thecolumntransformsgivenin ( 8.7)and( 8.8).

Blocksabovethe diagonal

For blocks ¤O¨ c ¢ § with � ÍÓª we have a splitting as shown in Figure8.5. Therecurrenceformulasarethe1D formulasappliedto eachcolumnof ¤O¨ c ¢ § :

Ã�Å cÂt g0¢ §u ¢ Ä v w npgx y{z eA| y Ã�Å c ¢ §~ y tp� u�� ¢ Ä (8.7)

Å�Å cÂt g0¢ §u ¢ Ä v w npgx y{z e � y Ã�Å c ¢ §~ y tp� u�� ¢ Ä (8.8)

for � v ªHfGª Ñ �%fjijijilf b�� , � v�� fj�%fjijijilfÀ� c2t g � � , and Ì v � fj�#fkijiji¿f�� § � � .

8.22D wavelettransformof acirculantmatrix 127

8.2 2D wavelet transform of a circulant matrix

The2D FWT of circulantmatricesgiveriseto structuredmatrixblockswhichwewill call shift-cir culant matrices. We begin by giving thedefinition.

Let Ô be an Õ Ö Õ circulant matrix as definedin Definition C.1 and let× | u©Ø�u z eh¢�g_¢�Ù�Ù�Ù,¢ Ú�npg bethefirst columnof Ô . ThendÂ� q u ¢ Ä v | ~ u nHÄ �$Û f � fhÌ v� fj�%fjijijilf Õ � �A shift-circulantmatrix is a generalizationof a circulantmatrix which we defineasfollows

Definition 8.1(Shift-cir culant matrix)

1. Let � be an Õ Ö � matrix where Õ Í � with Õ divisible by � , andlet

× | u©Ø�u z eh¢�g0¢�Ù�Ù�Ù,¢ Ú�npg be the first columnof � . Then � is column-shift-circulant ifdÂ� q u ¢ Ä v | ~ u nÝÜ�Ä �!Û f � v�� fj�#fkijijilf Õ � �#fAÌ v�� fj�%fjijiji¿f � � �where Þ v Õ �À� .

2. Let � bean Õ Ö � matrix where � Í Õ with � divisibleby Õ , andlet× | Ä Ø Ä z eh¢�g0¢�Ù�Ù�Ù,¢ m©npg bethefirst row of � . Then � is row-shift-circulant ifdÂ� q u ¢ Ä v | ~ ÄßnHÜ u��$à f � v� fj�%fjijijilf Õ � �#f�Ì v� fj�%fjijijilf � � �where Þ v«�� Õ .

ThenumberÞ is a positiveinteger that denotestheamountby which columnsorrowsare shifted.

A column-shift-circulantá[Ö � matrix ( Þ v�� ) hastheformâããä | e | �| g | �| � | e| � | gåjææç

A row-shift-circulant� Ö´á matrix ( Þ v� ) hastheformè | e | g | � | �| � | � | e | gêéNotethata circulantmatrix is bothcolumn-shift-circulantandrow-shift-circulantwith Þ v � .


Let× | Ä Ø Ä z eh¢�g0¢�Ù�Ù�Ù$¢ monpg be the first columnof a circulantmatrix � . Using this

columnvectorasa point of departurewe will now show how to computea rep-resentationof using only one vectorper block. Note that accordingto therecurrenceequations(8.1) to (8.8), the operationscanbe divided into 2D trans-formsof blocksonthediagonaland1D row or columntransformsof off-diagonalblocks.Wewill treatthesecasesseparately.

8.2.1 Blockson the diagonal

Lemma 8.1 Let ¤¥¤ c ¢ c bea � c Ö � c circulantmatrix. Then ¤¥¤ c2t g0¢ c2t g , ¤O¨ c2t g0¢ c2t g ,¨®¤ c2t g0¢ c2t g , and ¨o¨ c2t g0¢ § t g definedby (8.1) to (8.4), respectively, are circulantmatrices.

Proof: We will prove the lemmafor ¤O¨ c2t g_¢ cÂt g only sincethe other casesarecompletelyanalogous.

By assumption¤¥¤ c ¢ c is circulant,i.e.ÃlÃ c ¢ cu ¢ Ä v sGs c ¢ c~ u nHÄ � � �where�À�Hc ¢ c is thefirst columnof ¤¥¤ c ¢ c . Equation(8.2) thenbecomes

Ã�Å c2t g0¢ c2t gu ¢ Ä v w npgx y{z e w npgx ÈÉz eë| y � È srs c ¢ c~$~ y tp� u�� n ~ È tp� Ä �� (8.9)

Consideringnow the index of the typical term of (8.9) and using LemmaB.1andB.2 derivedin AppendixB wefind thatìÇìrí Ñ � �]î�ï � � ìGð Ñ � Ì î0ï � î�ï � v ì{ì ��ñ �Ó� Ìlò î0ï � Ñ í � ð î�ï �v ì � ì �Ó� Ì î ï �!ó � Ñ í � ð î�ï �v ì � ì �Ó� Ì î ï �õô ³ Ñ í � ð î ï �Thisexpressiondoesnot dependon theindividualvaluesof � and Ì but only ontheirdifference. Therefore,Ã�Å c2t g0¢ c2t gu ¢ Ä dependsonly on

ì �ö� Ì î�ï �õô ³ whichprovesthat ¤O¨ c2t g0¢ c2t g is circulant.HenceÃ�Å c2t g0¢ c2t gu ¢ Ä v s � c2t g_¢ cÂt g~ u nHÄ �� Éô ³ for somecolumnvec-

tor � � cÂt g0¢ c2t g . ÷Having establishedthat ¤O¨ c2t g0¢ c2t g is circulant,wecannow givea formulafor

the vector � � c2t g0¢ c2t g . Putting Ì vø� in equation(8.9) andusingLemmaB.1 we


obtainthefirst columnof ¤O¨ c2t g0¢ c2t g :s � c2t g_¢ cÂt gu v Ã9Å c2t g0¢ c2t gu ¢ e v w npgx yÇz e w npgx ÈÉz eÊ| y � È sGs c ¢ c~!~ � u t y � � � n ~ È � � �$� � �v w npgx yÇz e w npgx ÈÉz eÊ| y � È sGs c ¢ c~ � u t y n È � � � (8.10)

where � v�� fj�#fkijijilf�� c2t g � � . A computationallymoreefficient expressionfors � cÂt g0¢ c2t gu canbe derived by rearrangingthe termsin (8.10)so that we sumoverthedifferencesin the indices.More precisely, we split thedoublesuminto termswhere

í � ð Í � andtermswhereí � ð£ù � andrearrangetheseseparately. Thus

let ú v í � ð . Thetermsin (8.10)withí � ð Í � are

w npgx y{z eyx ÈÉz e | y � È srs c ¢ c~ � u t y n È ��hû � v w npgx yÇz e

yx ÈÉz e | È týü � È srs c ¢ c~ � u týü �,�0� (8.11)

Thefollowing tableshows ú asa functionofí

andð(for Å vþ ):í¼ÿ%ð � � � ¬ á �� ¬ ¬ � � �á á ¬ � � �� á ¬ � � �

Summingover thediagonalsyieldsanalternativeexpressionfor (8.11):

w npgx y{z eyx Èõz e¥| È týü � È srs c ¢ c~ � u týü �� v�w npgx ü z e � w nÎg_n üx ÈÉz e | È tHü � È�� srs c ¢ c~ � u týü �� (8.12)

Similarly, wetakethetermsfrom (8.10)withí � ð¿ù � andset ú v ð � í�� :

w nÎgx yÇz e w npgxÈõzpy t g�|y � y týü srs c ¢ c~ � u n ü � � � v w npgx ü z g � w nÎg_n üx yÇz eö| y � y tHü � srs c ¢ c~ � u n ü � � � (8.13)

Now let � ü�� v w npg0n üx y{z e | y � y týü f ú v�� fj�#fjikijilf Å � �


Combining(8.12)and(8.13)wecanrewrite (8.10)ass � c2t g0¢ c2t gu v� e�� srs c ¢ c� u Ñ w npgx ü z g�� ü�� srs c ¢ c~ � u n ü �,�0� Ñ � ü�� srs c ¢ c~ � u týü �� (8.14)

for � v�� fk�#fjijikilf�� c2t g � � .Thevectors��HcÂt g0¢ c2t g f � � c2t g0¢ c2t g , and

�h� c2t g0¢ c2t g arecomputedsimilarly but withthefilters

� ü�� , � ü�� , � ü�� , respectively. TheformulasaresGs c2t g_¢ cÂt gu v� e�� sGs c ¢ c� u Ñ w npgx ü z g � ü�� sGs c ¢ c~ � u n ü � � � Ñ � ü�� sGs c ¢ c~ � u týü � � � (8.15)� s c2t g_¢ cÂt gu v� e�� srs c ¢ c� u Ñ w nÎgx ü z g�� ü�� srs c ¢ c~ � u n ü � � � Ñ � ü�� srs c ¢ c~ � u tHü � � � (8.16)�G� c2t g_¢ cÂt gu v� e�� srs c ¢ c� u Ñ w npgx ü z g � ü�� sGs c ¢ c~ � u n ü �� Ñ � ü�� srs c ¢ c~ � u týü �� (8.17)

It followsfrom (2.22)that� e��¥v � e�� v �� ü�� v � ü�� v � f ú even� ü�� v � ü�� v � f ú even f ú � �

sothecomputationalwork in computing(8.14)–(8.17)is reducedaccordingly.

8.2.2 Blocksbelow the diagonal

Wenow turn to thetaskof computingthe lower off-diagonalblocks ¨]¤ c ¢ § t g and¨©¨ c ¢ § t g , ª[Í � definedby (8.5)and(8.6),respectively. Theoperationsdiffer fromthoseof the diagonalcasesby beingappliedin onedimensiononly. Thereforeblockstendto becomemorerectangularwhichmeansthatthey arenotnecessarilycirculantin theordinarysense.However, asweshallsee,thetypical block is stillrepresentedby asinglevector, onewhichisshiftedtomatchtherectangularshape.Theblock is thenacolumn-shift-circulantmatrix in thesenseof Definition8.1.


Lemma 8.2 Let ¨®¤ c ¢ § , ª]Í � , bea � c Ö � § column-shift-circulantmatrix. Then¨]¤ c ¢ § t g and ¨©¨ c ¢ § t g definedby (8.5) and (8.6), respectively, are column-shift-circulantmatrices.

Proof: We will give theproof for thecaseof ¨®¤ c ¢ § only. By assumption]¤ c ¢ §is column-shift-circulant,i.e. ÅÆÃ c ¢ §u ¢ Ä v � s c ¢ §~ u nHÜ�Ä � � �where

� � c ¢ § is thefirst columnof ¨]¤ c ¢ § and Þ v � c � � § v � §Çn c . Equation(8.5)thenbecomes ÅÆÃ c ¢ § t gu ¢ Ä v w npgx Èõz e | È ÅÆÃ c ¢ §u ¢ ~ È tp� Ä �,� Ï

v w npgx Èõz eë| È � s c ¢ §~ u nHÜ ~ È tp� Ä � �ÐÏ � � � (8.18)

We considertheindex of thetypical termof (8.18)anduseLemmasB.2 andB.1in AppendixB to obtainthefollowing:ì �ö��Þ ìrð Ñ � Ì î0ï Ï î0ï � v ì � � ì Þ ñ ð Ñ � Ìlò î Ü ï Ï î0ï �v ì � � ì Þ ð Ñ � Þ Ì î�ï � î0ï �v ì � ��Þ ð � � Þ Ì î�ï �Therefore ÅÆÃ c ¢ § t gu ¢ Ä v w nÎgx Èõz e | È � s c ¢ §~ u nHÜ È n � Ü�Ä � � � (8.19)

Equation(8.19)establishestheexistenceof avector,� � c ¢ § t g say, suchthat ¨]¤ c ¢ § t g

hasthedesiredcolumn-shift-circulantformÅÆÃ c ¢ § t gu ¢ Ä v � s c ¢ § t g~ u n � ÜÀÄ � � �for � v�� fk�#fjijiki�f�� c � � and Ì v � fj�#fkijiji¿f�� § t g � � . ÷

Wecannow look for anexplicit formulafor thevector� � c ¢ § t g . Takingthefirst

column( Ì v�� ) of ÅÆÃ c ¢ § t gu ¢ Ä in (8.19)givestheresult:� s c ¢ § t gu v ÅÆÃ c ¢ § t gu ¢ e v w npgx Èõz eÊ| È � s c ¢ §~ u nHÜ È � � � f � v�� fj�%fjijijilfÀ� c � � (8.20)


An analysissimilar to the above establishesthat ¨o¨ c ¢ § t g is column-shift-circulantwith respectto thevector

�h� c ¢ § t g givenby�G� c ¢ § t gu v ÅOÅ c ¢ § t gu ¢ e v w nÎgx ÈÉz e � È � s c ¢ §~ u nHÜ È � � � f � v�� fj�#fkijiji¿f�� c � � (8.21)

Sincethe initial block ¨]¤ c ¢ c is circulantaccordingto Lemma8.1, it is alsocolumn-shift-circulantwith Þ v � c � � c v � andwe canusethe columnvector� � c ¢ c ascomputedfrom (8.16)directly in (8.20)and(8.21)for thecase� v ª .8.2.3 Blocksabove the diagonal

Theupperoff-diagonalblocks ¤O¨ c2t g_¢ § and ¨©¨ c2t g0¢ § , � Í�ª , arecomputedaccord-ing to (8.7)and(8.8). Thesituationis completelyanalogousto (8.20)and(8.21)but the blocksarenow row-shift-circulant matricesrepresentedby row vectors� � c2t g0¢ § and

�h� c2t g0¢ § , respectively, asstatedby Lemma8.3.

Lemma 8.3 Let ¤O¨ c ¢ § , � ÍQª , be a � c Ö � § row-shift-circulant matrix. Then¤O¨ c2t g0¢ § and ¨o¨ cÂt g0¢ § definedby(8.7)and(8.8),respectively, arerow-shift-circulantmatrices.

Proof: Wewill givetheproof for thecaseof ¤O¨ c ¢ § only. Theproof is completelysimilar to thatof Lemma8.2,but theassumptionis now that ¤O¨ c ¢ § is row-shift-circulant,i.e. Ã9Å c ¢ §u ¢ Ä v s � c ¢ §~ Ä�nHÜ u}� �IÏwhere � � c ¢ § is thefirst row of ¤O¨ c ¢ § and Þ v � § � � c v � c n#§ . Equation(8.7) thenbecomes Ã9Å cÂt g0¢ §u ¢ Ä v w npgx y{z eA| y Ã9Å c ¢ §~ y tÎ� u}�� ¢ Ä

v w npgx y{z e | y s � c ¢ §~ Ä�nHÜ ~ y tp� uA�,� � �,� Ïv w npgx y{z eA| y s � c ¢ §~ Ä�nHÜ y n � Ü uo� �IÏ

Therefore¤O¨ c2t g_¢ § hasthedesiredrow-shift-circulantformÃ�Å c2t g0¢ §u ¢ Ä v s � c2t g0¢ §~ Ä�n � Ü u}� �IÏfor � v�� fk�#fjijikilf�� c2t g � � and Ì v�� fj�#fkijijilf�� § � � . ÷


Theformulasfor therow vectors� � c2t g0¢ § and�h� cÂt g0¢ § follow by takingthefirst

rows( � v�� ) of ¤O¨ c2t g0¢ §u ¢ Ä and ¨©¨ c2t g_¢ §u ¢ Ä , respectively:

s � c2t g0¢ §Ä v ÃlÃ c2t g_¢ §eh¢ Ä v w npgx y{z e}| y s � c ¢ §~ ÄßnÝÜ y � �IÏ f£Ì v�� fj�#fkijijilf�� § � � (8.22)

�r� c2t g0¢ §Ä v Å�Å c2t g_¢ §eh¢ Ä v w npgx y{z e � y s � c ¢ §~ ÄßnÝÜ y �,�IÏ Ì v�� fj�#fjikijilf�� § � � (8.23)

However, oneminor issueremainsto bedealtwith beforea viablealgorithmcanbeestablished:While the initial blocks ¤O¨ c ¢ c definedaccordingto (8.2) arecirculantandthereforealsorow-shift-circulant(with Þ v � ), they arerepresentedby columnvectorswhencomputedaccordingto equation(8.14).However, (8.22)and(8.23)work with a row vector � � c ¢ § . Thereforewe mustmodify each� � c ¢ c sothatit representsthefirst rowof ¤O¨ c ¢ c insteadof thefirst column.FromDefinitionC.1of a circulantmatrixwe have thatÃ�Å c ¢ cu ¢ Ä v s � c ¢ c~ u nHÄ � � � fVÌ v�� fk�#fjijikilf�� c � �where� � c ¢ c is thefirst columnof ¤O¨ c ¢ c . Putting � v�� thenyieldsthefirst row:Ã�Å c ¢ ceh¢ Ä v s � c ¢ c~ nHÄ � � � fVÌ v � fj�#fjikiji�fÀ� c � �To obtaina row representationfor ¤O¨ c ¢ c we can thereforetakethe result fromequation(8.14)andconvert it asfollowss � c ¢ cÄ�� s � c ¢ c~ nHÄ � � �Alternatively, we canmodify equation(8.14)to producetherow vectordirectly:s � c2t g0¢ c2t gÄ v� e�� srs c ¢ c~ n � Ä � � � Ñ w npgx ü z g�

� ü�� sGs c ¢ c~ n � Äßn ü � � � Ñ � ü�� srs c ¢ c~ n � Ä týü � � � (8.24)

for Ì v�� fj�#fjikijilf�� c2t g � � .Theequationsfor blocksabove thediagonal((8.22)and(8.23))andequations

for blocksbelow thediagonal((8.20)and(8.21))cannow becomputedwith thesamealgorithm.


8.2.4 Algorithm

Wewill now stateanalgorithmfor the2D wavelettransformof acirculantmatrix� . Let CIRPWT1beafunctionthatimplements2D decompositionsof blocksonthediagonalaccordingto (8.15),(8.16),(8.17),and(8.24):d �� c2t g0¢ c2t g fÇ� � c2t g0¢ c2t g f � � c2t g0¢ c2t g f �h� cÂt g0¢ c2t g q v CIRPWT1ñ �� c ¢ c òMoreover, let CIRPWT2beafunctionthatimplements1D decompositionsof theform describedin equations(8.20)and(8.21).This functioncanalsobeusedforcomputationsof (8.22)and(8.23)asmentionedabove.

d � � c ¢ § t g f �h� c ¢ § t g q v CIRPWT2ñ � � c ¢ § òhfOª[Í �d � � c2t g_¢ § f �h� c2t g0¢ § q v CIRPWT2ñ � � c ¢ § òhf � Í;ªWith thesefunctionstheexampleshownin Figure8.2canbecomputedasfollows:Let �À� eh¢ e be the column vector representingthe initial circulant matrix ¤¥¤ eh¢ e .Then d �À� g0¢�g f{� � g0¢�g f � � g0¢�g f �h� g0¢�g q v CIRPWT1ñ �� eh¢ e ò

d �À� � ¢ � f{� � � ¢ � f � � � ¢ � f �h� � ¢ � q v CIRPWT1ñ �� g0¢�g òd � � g0¢ � f �h� g0¢ � q v CIRPWT2ñ � � g0¢�g òd � � � ¢�g f �h� � ¢�g q v CIRPWT2ñ � � g0¢�g òd �À� � ¢ � f{� � � ¢ � f � � � ¢ � f �h� � ¢ � q v CIRPWT1ñ �� ¢ � òd � � � ¢ � f �h� � ¢ � q v CIRPWT2ñ � � � ¢ � òd � � � ¢ � f �h� � ¢ � q v CIRPWT2ñ � � � ¢ � òd � � g0¢ � f �h� g0¢ � q v CIRPWT2ñ � � g0¢ � òd � � � ¢�g f �h� � ¢�g q v CIRPWT2ñ � � � ¢�g ò


In general,thealgorithmis

For � v�� fk�#fjijiki�f bË� �d �À� c2t g0¢ c2t g f{� � c2t g0¢ c2t g f � � c2t g_¢ cÂt g f �h� c2t g0¢ c2t g q v CIRPWT1 ñ �À� c ¢ c òFor ª v � f �� %fjijijilfj�d � � §G¢ c2t g f �h� §r¢ c2t g q v CIRPWT2 ñ � � §G¢ c òd � � cÂt g0¢ § f �h� c2t g0¢ § q v CIRPWT2 ñ � � c ¢ § òend

end

This algorithmdescribestheprocessof computingthe2D wavelet transformasdescribedin theprevioussection.However, it doesnotdistinguishamongvec-torsthatshouldbekeptin thefinal resultandvectorsthataremerelyintermediatestagesof the transform.Thevectors

� � §G¢ c and � � c ¢ § , for example,arepartof thefinal resultfor � v b only, soall othervectorscanbediscardedat somepoint.

In practicewe preferan algorithmthat makesexplicit useof a fixedstorageareaandthatdoesnotstoreunnecessaryinformation.Therefore,wewill introducea modifiednotationthatis suitablefor suchanalgorithmandalsoconvenientforthematrix-vectormultiplicationwhich is describedin Section8.4.


8.2.5 A data structur e for the 2D wavelet transform

As describedin Section8.1 the result of a 2D wavelet transformis a blockmatrixwith a characteristicblock structure(seeFigure8.2). We now introduceanew notationfor theseblocksasfollows

c ¢ § v �� ¤¥¤�� ¢ � for � frª v��¤O¨�� ¢ � n#§ t g for � v�� fk��;ª�� b¨]¤ � n c2t g0¢ � for �� b frª v�¨©¨ � n c2t g0¢ � n#§ t g for �� frª�� b (8.25)

This is illustratedin Figure8.6for thecaseb v�¬ . e0e ehg e � e �

g0g g � g � �_� � �

�0�

gÐe � e

� e

� g

� g � �

Figure 8.6: The new notationfor the block structureof Á (for � , �) asdefinedby

(8.25).Comparewith Á in Figure8.2.

We can now use indices � fGª v � fj�%fjijiji¿f b to label the blocks of in astraightforwardmanner. It follows from this definition that eachblock c ¢ § isan � c by � § matrixwith� y v � �� v � � for

í v�� n y t g v � � n y t g for �� í � b (8.26)


Note that � e v � g . Becauseall the blocks c ¢ § areeithercirculantor shift-circulant,wecanrepresentthemby vectors! c ¢ § with" c ¢ §u v �$# c ¢ §u ¢ e for � Í�ª# c ¢ §eh¢ u for � ù ª (8.27)

where � vX� fj�#fkijiji¿f�%�&(' ñ_� c0f � § ò � � . It followsfrom (8.26)thatthe lengthof! c ¢ § is � � c for � Í�ª� § for � ù ª (8.28)

andtheshift parameterÞ is now givenaccordingto Definition8.1asÞ v � � c �À� § for � Í;ª� § �� c for � ù ªEquation(8.27)suggestsadatastructureconsistingof thevectorvariables! c ¢ §

which refer to theactualarraysrepresentingthefinal stageof thewavelet trans-form (e.g.

�h� � n c2t g_¢ � n#§ t g ). Thisstructurecanalsobeusedto storetheintermediatevectorsfrom therecursionif we allow thevariables! c ¢ § to assumedifferentval-ues(anddifferentlengths)duringthecomputation– for exampleby usingpointervariables.This is demonstratedin Figure8.7.


± ²!*)�)z �h� ³,³

!�¶,¶z �h� ¶,¶ !�¶ )z �h� ¶ ³!+) ¶z �h� ³ ¶

! ³,³,�h� )�)! ¯,¯,�� )�)! ³2¯,� � )�) ! ¯r³,� � )�)! ¶ ¯,� � ¶ )! ),¯,� � ³-)! ¶ ³,�h� ¶ )! )r³,�h� ³-)! ³ ¶, �h� ) ¶ ! ³-), �h� )r³!�¯ ¶, � � ) ¶ !¿¯�), � � )r³

!*)�)z �h� ³,³! ¶,¶, �h� ¶,¶ ! ¶ ), �h� ¶ ³!.) ¶, �h� ³ ¶

! ³,³,�À� ¶,¶!�¶ ³,� � ¶,¶! )r³,� � ³ ¶! ³ ¶, � � ¶,¶ !A³-), � � ¶ ³

!.)�), �h� ³,³! ¶,¶, �À� ³,³ ! ¶ ), � � ³,³!.) ¶, � � ³,³

�

!*)�), �À� ¯,¯

Ô #

Figure8.7: Theuseof (pointer)variables/ c ¢ § to implementthe2D wavelettransformofa circulantmatrix. Intermediateresultsarereferencedby variables/ c ¢ § beforethey attaintheir final values.For example,considertheshadedparts: / � ¢ � storesfirst 021 g0¢�g which isthensplit furtherinto 031 � ¢�g and 11 � ¢�g . Then 1�1 � ¢�g is storedin / � ¢ � overwriting 031 g0¢�g .


FromFigure8.7wearriveatthefinal formulationof thealgorithmin Section8.2.4.

Algorithm 8.1: Cir culant 2D wavelet transform (CIRFWT)! � ¢ � � �À� eh¢ eFor ª v b f bË� �#fjijikilfj�d4! §Çnpg0¢ §{nÎg f ! §Çnpg0¢ § f ! §r¢ §Çnpg f ! §G¢ § q � CIRPWT1 ñ ! §G¢ § ò

For � v ª Ñ �#frª Ñ � fjijiji bd4! c ¢ §Çnpg f ! c ¢ § q � CIRPWT2 ñ ! c ¢ § òd4! §Çnpg0¢ c f ! §G¢ c q � CIRPWT2 ñ ! §r¢ c òend

end

whereCIRPWT1is derivedfrom (8.15),(8.16),(8.17),and(8.24):" §Çnpg0¢ §Çnpgu � � e�� " §r¢ §� u Ñ w npgx ü z g � ü�� " §r¢ §~ � u n ü � à Ï Ñ � ü�� " §G¢ §~ � u týü � à Ï (8.29)" §Çnpg0¢ §u � � e�� " §r¢ §~ n � uA� à Ï Ñ w npgx ü z g � ü�� " §r¢ §~ n � u n ü � à Ï Ñ � ü�� " §r¢ §~ n � u týü � à Ï (8.30)" §r¢ §Çnpgu � � e�� " §r¢ §� u Ñ w npgx ü z g

� ü�� " §r¢ §~ � u n ü � à Ï Ñ � ü�� " §r¢ §~ � u tHü � à Ï (8.31)" §r¢ §u � � e�� " §r¢ §� u Ñ w nÎgx ü z g � ü�� " §r¢ §~ � u n ü � à Ï Ñ � ü�� " §r¢ §~ � u týü � à Ï (8.32)

where � vL� fj�#fkijiji¿f � §Çnpg � � . For � Í ª CIRPWT2is derivedfrom (8.20)and(8.21): " c ¢ §{nÎgu � w nÎgx ÈÉz eÆ| È " c ¢ §~ u nHÜ È � à � (8.33)" c ¢ §u � w nÎgx ÈÉz e � È " c ¢ §~ u nHÜ È � à � (8.34)

where� v�� fj�#fkijijilf � §{npg � � . For � ù ª CIRPWT2is" c npg0¢ §u � w npgx Èõz eë| È " c ¢ §~ u nHÜ È � à Ï (8.35)" c ¢ §u � w npgx Èõz e � È " c ¢ §~ u nHÜ È � à Ï (8.36)


where � v � fk�#fjijiki�f � c npg � � . Note that we useexactly the samecodeforCIRPWT2by exchangingtheindices� andª in Algorithm 8.1.Ourexamplenowtakestheform d4! � ¢ � f ! � ¢ � f ! � ¢ � f ! � ¢ � q � CIRPWT1ñ ! � ¢ � ò

d4! g0¢�g f ! g0¢ � f ! � ¢�g f ! � ¢ � q � CIRPWT1ñ ! � ¢ � òd4! � ¢�g f ! � ¢ � q � CIRPWT2ñ ! � ¢ � òd4! g0¢ � f ! � ¢ � q � CIRPWT2ñ ! � ¢ � òd4! eh¢ e f ! eh¢�g f ! g0¢ e f ! g0¢�g q � CIRPWT1ñ ! g0¢�g òd4! � ¢ e f ! � ¢�g q � CIRPWT2ñ ! � ¢�g òd4! eh¢ � f ! g0¢ � q � CIRPWT2ñ ! g0¢ � òd4! � ¢ e f ! � ¢�g q � CIRPWT2ñ ! � ¢�g òd4! eh¢ � f ! g0¢ � q � CIRPWT2ñ ! g0¢ � ò

8.2.6 Computational work

We will now derive anestimateof thecomplexity of Algorithm 8.1,but first weneedto establishthefollowing lemma:

Lemma 8.4 � npgx y{z e �y v � �

Proof: Using(8.26),wewrite� nÎgx yÇz e �y v �� Ñ � npgx y{z g �� n y t gv �65 �� Ñ �x y{z � �� y87v � è �� Ñ � � g��9 ô ³� � g� � �� év � è �� Ñ � � �� év � � ÷


Equations(8.29)–(8.32)eachrequire ñ á ñ Å�� ò Ñ � ò � §Çnpg flopssothecomplexityof CIRPWT1is :

CIRPWT1ñ_� ò vXñ � þ9ñ Å�� ò Ñ�; ò �with � v � §{npg .Equations(8.33)–(8.36)eachrequire� Å � flopssothecomplexity of CIRPWT2is :

CIRPWT2ñ_� ò v á�Å �with � v � §{npg or � v�� c npg .Thetotal complexity of Algorithm 8.1 is thus:

CIRFWT ñ_� ò v �x § z g 5 : CIRPWT1ñ0� §Çnpg ò Ñ �xc z § t g : CIRPWT2ñ0� §Çnpg ò Ñ : CIRPWT2ñ_� c npg ò 7Hence:

CIRFWT ñ0� ò v �x § z g 5 ñ � þ9ñ Å � ��ò Ñ<; ò � §Çnpg Ñ �xc z § t g á9Å � §Çnpg Ñ;á�Å � c npg 7ù á9Å 5 �x § z g á � §Çnpg Ñ �x c z � � §{npg Ñ � c npg 7ù á9Å 5 ñ á6Ñ b�� ò �x § z g � §{nÎg Ñ �x c z g � c npg 7v á9Å è ñ áêÑµb�� ò � � Ñ � � éwherewe have usedLemma8.4 twice. Consequently, we obtain the followingboundfor thecomplexity of Algorithm 8.1::

CIRFWT ñ_� ò ù � Å �®ñ á�Ñ b ò (8.37)


8.2.7 Storage

In this sectionwe will investigatethestoragerequirementfor asa functionofthetransformdepth b whenthedatastructureproposedin Section8.2.5is used.

Lemma 8.5 Thenumberof elementsneededto representtheresultof b stepsofthewavelettransformof a circulant � Ö � matrixascomputedbyAlgorithm8.1is � m ñ b ò v«�=5 � Ñ �x y{z g í� � n y>7 f b v�� fj�#fjikijilf b u.�3? (8.38)

Proof: We will prove(8.38)by induction:Induction start: For b vö� thereis only oneblock, namely eh¢ e

which is rep-resentedby thevector ! eh¢ e of length � . Thus, � m ñG� ò v � and(8.38)thereforeholdsfor b v� .Induction step: Theinductionhypothesisis� m ñ b�� ßò vX�65 � Ñ � npgx yÇz g í� � npg0n y 7 (8.39)

! e0e ! e¦g ! e �! gÐe! � e

Depth � Depth ¬! e0e ! ehg ! e � ! e �! gÐe ! g0g ! g � ! g �! � e! � e! � g! � g

Figure 8.8: Thedifferencesbetweena wavelettransformof depth ��@BA and � (shownfor � ,�� ).Thestoragedifferencesbetweena wavelet transformof depth b^� � and b lie intheuppermostandleftmostblocksof thestructureasindicatedin Figure8.8.Therestarethesame.


Depth � Depth ¬Vector Length Vector Length! e0e �� á ! e_e �� ;! ehg f ! gIe f ! g0g �� ;! ehg f ! gÐe �� á ! e � f ! g � f ! � e f ! � g �� á! e � f ! � e �� ! e � f ! g � f ! � e f ! � g ��

Table8.1: Thenumberof elementsin thevectorsshown in Figure 8.8( � ,�� ).Recallthateachblock is representedby avector. Wewill subtractthenumber

of elementsin the affectedvectorsat depth b´� � from (8.39)andthenaddtheelementsfrom vectorsresultingfrom a transformto depth b . Thevectorlengthsfor theexample b v�¬ areshown in Table8.1Accordingto (8.26)– (8.28)thevector ! e_e atdepth b�� hasthelength� e v �� npg (8.40)

This is replacedby á new vectors( ! e_e f ! ehg f ! gIe f ! g0g ) at depth b eachhaving thelength �� . We write this numberasá � �� v � è �� npg Ñ �� nÎg é (8.41)

The blocksabove the diagonalthat arereplacedarethosein the upperrow.They arerepresentedby a vectorof length �� , a vectorof length �� á down toa vectorof length �� npg . Thenumberof elementsin vectorscorrespondingtocolumnblocksis thesame.Hencethenumberof elementswemustsubtractfrom(8.39)is � � npgx y{z g �� y v � � nÎgx yÇz g �� n y v � nÎgx yÇz g �� npg0n y (8.42)

Thesplittingof theseoff-diagonalblockswill createtwice asmany blockssothenumberof elementsintroducedwill be� � npgx y{z g �� npg0n y v�� x y{z � �� n y (8.43)

Now we subtractfrom (8.39)thetotal numberof elementsin thevectorsthatarereplaced((8.40)and(8.42))andaddthenumberof elementsintroduced((8.41)and(8.43)),i.e.


� m ñ b ò v � m ñ b°� ��ò � �� npg � � npgx y{z g �� npg0n y Ñ � è �� nÎg Ñ �� npg é Ñ � �x yÇz � �� n yv � 5 � Ñ � npgx yÇz g í� � nÎg_n y � �� npg � � npgx y{z g �� npg0n y Ñ �� npg Ñ �� npg Ñ � �x y{z � �� n yC7v �=5 � Ñ � npgx yÇz g í � �� nÎg_n y Ñ �� npg Ñ � �x y{z � �� n y>7v �=5 � Ñ �x yÇz � í � �� n y Ñ �� npg Ñ � �x y{z � �� n y87v � 5 � Ñ �x yÇz � í� � n y Ñ �� npg 7v �=5 � Ñ �x yÇz g í� � n yD7which is thedesiredformula. ÷Weconsidernow thesumin (8.38)anddefineE ñ b ò v �x yÇz g í� � n y f b v�� fj�%f � fkijijiWith a changeof variableswe canrewrite

EasE ñ b ò v � npgx yÇz e b�� í� y

Thisis anarithmetic-geometricserieswhichhastheclosedform [Spi93, p. 107]E ñ b ò v b ñ � � ñ � �� ò � ò� � � �� µb ñ � �� ò � nÎg Ñ ñ b°� �ßò ñ � �� ò ��9ñ � � � �� ò �v � b ñ � � ñ � �� ò � ò � ��ñ � � b ñ � �� ò � npg Ñ ñ bË� �ßò ñ � �� ò � òv � b°� � �;á�b ñ � �� ò � Ñ � b ñ � �� ò � npg Ñ �9ñ � �� ò �v � b°� � Ñ �9ñ � �� ò �


0 2 4 6 8−2

0

4

8

12

16

λ

f

f(λ)2λ−2

Figure 8.9: Thefunction FHGI�>I behavesasymptoticallylike J#�K@LJ .We observe that

E ñ b ò behavesasymptoticallyas � b^� � . Figure8.9shows a plotofE ñ b ò togetherwith its asymptote.Thestoragerequirementcannow beexpressedas� m ñ b ò v �Vñ � Ñ E ñ b òÇòv �VñG� b°� � Ñ �9ñ � �� ò � òhf b v � fj�#fkijiji¿f b u.�3?

andwehave thebound � m ñ b òM� � b � f b ÍX� (8.44)


8.3 2Dwavelettransform of acirculant, bandedma-trix

An importantspecialcaseof a circulantmatrix is when � is a bandedcirculantmatrix suchas the differentiationmatrix ¨ given in (7.14). In this caseeachcolumnof � consistsof a piecewhich is zeroanda piecewhich is regardedasnon-zero.In certaincolumnsthenon-zeropartis wrappedaround.Consequently,it is sufficient to storeonly the non-zeropart of the first columnalongwith anindex N determininghow it mustbealignedin thefirst columnrelative to thefull-lengthvector. Thelengthof thenon-zeropartis thebandwidth of � , andwewilldenoteit by O in this chapter.

It turnsout thateachblockof the2D wavelettransformretainsabandedstruc-ture,sothevectorrepresentingit needonly includethenon-zeropart. Thereforethestoragerequirementscanbe considerablylessthanthat givenby (8.38). Anexampleof this structureis givenin Figure8.10.

8 16 32 64

8

16

32

64

nz = 1360

Figure8.10: Thestructureof awavelettransformof a P �KQ P � circulantbandedmatrix.Here R , �

, S , � and � , �. This is thesameblock structureasin Figure8.6 andin

Figure7.1.

8.32D wavelettransformof acirculant,bandedmatrix 147

For eachblock,weknow thenon-zerovaluesof thefirst column(or row in thecaseof blocksabovethediagonal)representedby thevectorTVU ¢ § vXW�Y U ¢ §Z\[ Y U ¢ §]^[`_`_`_H[ Y U ¢ §a8b ]dcfe ,theamountby which it is shifted( Þ ) andhow it is alignedwith respectto thefirstelementof theblock ( N ). Theblock � ¢ ]

, say, hasthegeneralstructure

� ¢ ] v

ghhhhhhhhhhhhhhhhhhhhhhhhhhhhi

Y � ¢ ]�Y � ¢ ]�Y � ¢ ]j Y � ¢ ]ZY � ¢ ]]Y � ¢ ]�Y � ¢ ]�Y � ¢ ]j Y � ¢ ]ZY � ¢ ]]Y � ¢ ]�Y � ¢ ]�Y � ¢ ]j Y � ¢ ]ZY � ¢ ]]Y � ¢ ]�Y � ¢ ]�Y � ¢ ]Z Y � ¢ ]jY � ¢ ]]

k-llllllllllllllllllllllllllllm(8.45)

In this examplewe have N v ¬ , � � v � þ , � ] v á , Þ v á (from Definition 8.1),bandwidthO � ¢ ] v � , and"p� ¢ ]u v � Y � ¢ ]~ u tp� � ³-n for

ì ��Ñ � î ]po Wõ� [ á c� otherwise

with # � ¢ ]u ¢ Ä v " � ¢ ]~ u b j Ä � ³-nIf thematrix isnotbandedwehavethespecialcaseT U ¢ § v ! U ¢ § and N v � so(8.38)appliesexactly.

8.3.1 Calculation of bandwidths

GiventhebandwidthO of theoriginalmatrix � it is possibleto derivea formulafor thebandwidthsof eachblockof thewavelettransform.

Let OqU ¢ § be the bandwidthof block U ¢ § shown in Figure8.6. We canthenusethe recurrenceformulasfor the 2D wavelet transformto obtain the desiredformulas.


Blockson the diagonal

We start with the blocks on the diagonalgiven by equation(8.1) to (8.4) andconsideragainonly the � � blockasthetypicalcase.Let usrecall(8.29):" § b ] ¢ § b ]u � � Z�� " §r¢ §� u Ñ w b ]x ü z ]

� ü�� " §r¢ §~ � u b ü � à Ï Ñ � ü�� " §r¢ §~ � u týü � à Ï for � v � [ � [`_r_`_H[ � § b ] � � . Assumethat ! §r¢ § is banded,i.e. zero outsideabandof length O §G¢ § as shown in Figure 8.11. Then we canusethe recurrenceformula to computethe bandwidthof ! § b ] ¢ § b ] . Without loss of generalitywemayassumethat thenonzeropart is wholly containedin thevector; i.e. thereisnowrap-around.

sKtvu tw tvu t xzy

Figure8.11: Thebandof / §G¢ § haslength R §r¢ § andstartsat index { ] .Since ! §G¢ § is zerooutsidethe interval startingat � ] of length O §r¢ § , we see

that" § b ] ¢ § b ]u will be zeroonly if � � �;ú Í � ] Ñ|O §r¢ § or � � Ñ;ú ù � ] for allú WÉ� [ Å � � c . This leadsimmediatelyto theinequalities� ��Ñ ñ Å � ��ò ù � ]� � � ñ Å � ��òNÍ � ] Ñ}O §G¢ §

or � Í � ] Ñ~O §r¢ § ÑµÅ � �� or� ù � ] �µÅLÑ ��Thelengthof this interval is thenthebandwidthof ! § b ] ¢ § b ] :O § b ] ¢ § b ] v � ] Ñ~O §G¢ § ÑµÅ � �� ] �µÅ Ñ ��v O §r¢ §� ÑµÅ � � (8.46)


However, sincethebandwidthis anintegerthefractionmustberoundedeitherupor down if O §r¢ § is odd.Which of theseoperationsto choosedependson � ] asillustratedin Figure8.12.

0

4

12

3

0

5

12

34

5

�� ^��A:

B:

w tvu t

Figure 8.12: The computationin equation(8.14)canbe viewed asa sliding filter oflength J�S�@�A appliedto the bandof / §r¢ § . The numbersindicateoffset with respecttoJ�{ , G�{ ] @LS��AvI3��J . Theresultingbandwidthdependson how theinitial bandwidthis alignedwith this slidingfilter. In this example R §G¢ § ,~- , S ,;� so R § b ] ¢ § b ] is either

-(case� ) or P (case� ) dependingon how theconvolutionshappento align with thenon-zeroblock. Thuscase� correspondsto roundingupandcase� correspondsto roundingdown in ( 8.46).

Wehavechosenalwaysto roundupwardsbecauseit yieldsanupperboundforthebandwidth.ThustheformulabecomesO § b ] ¢ § b ] v6� O §G¢ §�z� Ñ Å � � (8.47)

Weobserve thatequation(8.47)hastwo fixedpoints,namelyO § b ] ¢ § b ] v O §G¢ § v � � Å � �� Å � �


and it turnsout that thereis convergenceto one of thesevaluesdependingonwhetherthe initial O §r¢ § is smallerthan � Åö� � or larger than � Åö� � . However,theimportantfact is that thesefixedpointsaremore relatedto thewaveletgenusÅ thanto theoriginalbandwidthO .Blocksbelowand abovethe diagonal

Thebandwidthsof theblocksbelow thediagonalarefoundfrom recurrencefor-mulasof theform " U ¢ § b ]u v w b ]x ÈÉz Z | È " U ¢ §~ u b Ü È � à �Again, we disregardwrappingand,proceedingasin thecaseof ! § b ] ¢ § b ] above,we find that

" U ¢ § b ]u is zeroonly if � � Þ ð Í � ] Ñ�OqU ¢ § or � � Þ ðëù � ] for allð Wõ� [ Å � � c . This leadsto theinequalities� ù � ]��Þ ñ ÅL� ��ò Í � ] Ñ~O U ¢ §Consequently, theinterval lengthfor which

" U ¢ § b ]u �v�� isO U ¢ § b ] v � ] Ñ}O U ¢ § Ñ�Þ ñ ÅL� �ßò �;� ]v O U ¢ § Ñ�Þ ñ Å � �ßòPerformingsimilar computationsfor blocksabove thediagonalyieldsthere-

sult O U ¢ § b ] v O § b ] ¢ U v O U ¢ § Ñ�Þ ñ ÅL� �ßò (8.48)

SinceÞ is theratiobetween� U and � § , equation(8.48)showsthatthebandwidthgrows exponentiallyas the differencebetween� and ª increases.Figures8.13and8.14show examplesof thebandwidthsfor differenttransformlevels.


O v�¬

±

�

²� ��

�

þþþ

þ ;;;;

þ ;;

þþþ þ�� á�� á

� � á� � á

Figure 8.13: Thebandwidthsfor transformdepths� ,��C� A � J �{� . Theinitial bandwidthis�

and S ,;� . Thesebandwidthsareupperboundssinceactualbandwidthsmaybeonelessthanthepredictedvaluesascanbeseenin Figure8.10.


þ � � áþ ;

�

�� á ;

þþþþ �� #þ

�� %þ

� � � �%þ� � � �%þ

Figure 8.14: Thebandwidthsfor transformdepth � ,;� .

8.4Matrix-vectormultiplicationin awaveletbasis 153

8.4 Matrix-v ector multiplication in a waveletbasis

We now turn to the problemof computingthe matrix-vectorproduct � v ��where � [ �� µ¡�� and is givenasin (8.25). This systemhastheform shownin Figure8.15.

# Z2Z # Z�] # Z � # Z �# ]3] # ] � # ] �# �_� # � �

# �_�

# ]�Z# � Z

# � Z# � ]

# � ] # � �

#

� �� ]� Z�

v� �� ]� Z�

Figure 8.15: Thestructureof � , Á�� for � ,�� .


Thevector � maybecomputedblock-wiseasfollows� U v �x § z Z U ¢ § � § [ � v�� [ � [r_`_`_q[¡ (8.49)

where is the depthof the wavelet transform. The symbols � § , � U , and U ¢ §denotethe differentblocks of the � , � , and as indicatedin Figure8.6. Thecomputationin (8.49)is thusbrokendown into the tasksof computingtheprod-uctswhichwewill denoteby� U ¢ § v U ¢ § � § [ � [ ª v�� [ � [`_r_`_H[¡ � � (8.50)

In thefollowing we distinguishbetweenblockson or below thediagonal( � Í ª )andblocksabove thediagonal( � ù ª ).8.4.1 Blockson or below the diagonal

Let TVU ¢ § , � Í ª , be thevectorof length OqU ¢ § representingthenonzeropartof thefirst columnof U ¢ § , i.e." U ¢ §u v£¢ Y U ¢ §~ u t¥¤ b ] � à � for

ì ��Ñ�NÆ� � î � � WÉ� [ OqU ¢ § � � c� otherwise

and # U ¢ §u ¢ Ä v " U ¢ §~ u b ÜÀÄ � à �where� v�� [ � [r_`_`_H[ � U , Þ v«� U �À� § , and N is theoffsetrelative to theupperleftelement(see(8.45)).Fromequation(8.50)we seethatthetypical elementof � U ¢ §canbecomputedcolumnwiseas� U ¢ §u v � Ï b ]x Ä z Z # U ¢ §u ¢ Ä � § Äv � Ï b ]x Ä z Z " U ¢ §~ u b Ü�Ä � à � � § Äv � Ï b ]x Ä z Z Y U ¢ §~ u b ÜÀÄ t¥¤ b ] � à � � § Ä (8.51)

For each Ì this computationis only valid for those � WÉ� [ � U¥� � c whereY U ¢ §~ u b ÜÀÄ t¥¤ b ] � à � is defined,namelywhere� � ì �ö��Þ Ì Ñ�Në� � î � � � O U ¢ § � � (8.52)


Letí

andðbedefinedsuchthat

í � � � ð whenever (8.52)is satisfied.Thenwecanfind

ífrom therequirementìGí ��Þ Ì Ñ�ND� � î � � v ��¦í v ì Þ Ì ��NDÑ � î � �

andthelastrow asð v ìrí Ñ}O*U ¢ § � � î � � . Letting� U ¢ §y¨§ È vXW � U ¢ §y [ � U ¢ §y t ] [r_`_`_H[ � U ¢ §È c

thenwecanwrite thecomputation(8.51)compactlyas� U ¢ §y¨§ È v � U ¢ §y¨§ È Ñ � § Ä Y U ¢ §Z § a �ª© Ï b ] [ Ì v�� [ � [`_r_`_H[ � § � � (8.53)

Whení��ð

thebandis wrappedand(8.53)mustbemodifiedaccordingly.If thevector � is a waveletspectrumthenmany of its elementsarenormally

closeto zeroasdescribedin Chapter4. Thereforewewill designthealgorithmtodisregardcomputationsinvolving elementsin � § where«« � § Ä «« ù~¬Thealgorithmis givenbelow

Algorithm 8.2: �® u t�¯±° ® u t³² t , ´�µ·¶For Ì v�� to � § � �

if ¸ � § Ä ¸ �~¬ thení v ì Þ Ì ��N¥Ñ � î � �ð v ìrí Ñ}OqU ¢ § � � î � �ifí´ù ð

then� U ¢ §y�§ È v � U ¢ §y¨§ È Ñ � § Ä Y U ¢ §else (wrap)� U ¢ §Z § È v � U ¢ §Z § È Ñ � § Ä Y U ¢ §a �4© Ï b Èª§ a8b ]� U ¢ §y�§ � � b ] v � U ¢ §y�§ � � b ] Ñ � Ä Y U ¢ §Z § a �ª© Ï b È b ]end

endend

8.4.2 Blocksabove the diagonal

Let YCU ¢ § , � ù ª , be the vectorof length O U ¢ § representingthe nonzeropartof thefirst row of U ¢ § , i.e." U ¢ §Ä v£¢ Y U ¢ §~ Ä t¥¤ b ] � à Ï for

ì Ì Ñ�Në� � î � Ï WÉ� [ O U ¢ § � � c� otherwise(8.54)


where # U ¢ §u ¢ Ä v " U ¢ §~ Ä b Ü uA� à Ï (8.55)

with Þ vX� § �À� U . An example(shown for ] ¢ � ) is¹ºººººººººº»M¼ ¶ ¼ ) ¼¾½3¼¾¿d¼ n ¼¾À ¼¾Á3¼¾Â¾¼ ³2¯ Z ¼ ¯ ¼ ³¼ ¯ ¼ ³ ¼ ¶ ¼ ) ¼¾½¾¼¾¿d¼ n ¼¾À ¼dÁ¾¼¾Â¾¼ ³2¯ Z¼ ¯ ¼ ³ ¼ ¶ ¼ ) ¼¾½¾¼¾¿3¼ n ¼¾À=¼¾Á¾¼¾Â3¼ ³2¯ Z¼ ¯ ¼ ³ ¼ ¶ ¼ ) ¼¾½3¼¾¿d¼ n ¼¾ÀÃ¼¾Á3¼¾Âd¼ ³2¯ Z¼ ¯ ¼ ³ ¼ ¶ ¼ ) ¼¾½d¼¾¿¾¼ n ¼¾À=¼¾Ád¼¾Â3¼ ³2¯ Z¼ ¯ ¼ ³ ¼ ¶ ¼ ) ¼¾½¾¼¾¿3¼ n ¼dÀ=¼¾Á3¼¾Â¾¼ ³2¯ Z¼ ³2¯ Z ¼ ¯ ¼ ³ ¼ ¶ ¼ ) ¼¾½3¼¾¿¾¼ n ¼dÀÃ¼¾Á¾¼dÂ¼ n ¼¾À=¼¾Á3¼¾Â¾¼ ³2¯ Z ¼ ¯ ¼ ³ ¼ ¶ ¼ ) ¼¾½¾¼d¿

Ä ÅÅÅÅÅÅÅÅÅÅÆHere Þ v á [ N v�¬ , � ] v ; , � � v ¬%� , O ] ¢ � v � � (paddedwith onezero). The

superscripts� [ ª hasbeendroppedin this example.In orderto beableto disregardsmallelementsin � asin theprevioussection,

we would like to convert therow representationto a columnorientedformat. Asindicatedin the exampleabove, the block U ¢ § canbecharacterizedcompletelyby Þ columnvectorseachwith length OqU ¢ § � Þ togetherwith somebook-keepinginformation.Wewill assumethat OqU ¢ § is amultipleof Þ , possiblyobtainedthroughpaddingwith zerosassuggestedin theexampleabove.

Now we choosethesevectorsfrom thecolumnsO U ¢ § ��N [ O U ¢ § ��NÆ� � [`_`_r_H[ O U ¢ § ��Në��Þ Ñ �whicharethe Þ lastcolumnswherethetoprow is non-zero:theshadedareain theexampleabove. Let thesecolumnvectorsbedenotedÇ U ¢ §§ ¢ È for

� v� [ � [`_`_`_H[ Þ � � .From(8.54)and(8.55)we getÇ U ¢ §u ¢ È v # U ¢ §u ¢ a �ª© Ï b ¤ b Èv " U ¢ §~ a �ª© Ï b ¤ b È b Ü u}� à Ïv Y U ¢ §~ a �4© Ï b ¤ b È b Ü u tÉ¤ b ] � à Ïv Y U ¢ §~ a �4© Ï b Ü u b È b ] � à Ï (8.56)

for � vL� [ � [`_`_r_*[ OqU ¢ § � Þ]� � and� v � [ � [`_`_`_H[ Þ � � . In theexampleabove we

have Ç U ¢ §§ ¢ Z v âä �YËÊY � åç [ Ç U ¢ §§ ¢ ] v âä Y ]pZY oY � åç [ Ç U ¢ §§ ¢ � v âä Y�ÌY³ÍY ] åç [ Ç U ¢ §§ ¢ � v âä Y(ÎY jY Z åç


Equation(8.50)canthenbecomputedcolumnwiseusing Ç U ¢ §§ ¢ È insteadof U ¢ § . Thetypicalelementof � U ¢ § is � U ¢ §u v � Ï b ]x Ä z Z # U ¢ §u ¢ Ä � § Äv � Ï b ]x Ä z Z Ç U ¢ §Ï ¢ È¡� § ÄLet

íand

ðbedefinedsuchthat

í � � � ð whenever � � Ð�� OqU ¢ § � ÞV� � andlet Ì be fixed. Our taskis now to determine

�, togetherwith

íand

ð, suchthatÇ U ¢ §Ï ¢ È v # U ¢ §u ¢ Ä . Thereforewe put Ð v�� andlook for

í,�

suchthatÇ U ¢ §Z ¢ È v # U ¢ §y ¢ Ä (8.57)

In otherwords: For Ì given,we want to know which vectorto useandat whichrow it mustbealigned.

Next we insert the definitions(8.55) and (8.56) in (8.57) anduse(8.54) toobtaintheequationY U ¢ §~ a �4© Ï b È b ] � à Ï v " U ¢ §~ Ä b Ü y � à Ï v�Y U ¢ §~ Ä b Ü y tÉ¤ b ] � à Ïwhich is fulfilled wheneverì O U ¢ § � � � � î � Ï v ì Ì ��Þ í Ñ<NÆ� � î � Ï ¦ì O U ¢ § � � � Ì Ñ�Þ í ��N#î � Ï v �Let O v OqU ¢ § � Ì ��N . Thenwe canwrite therequirementasì OV� � Ñ�Þ í î � Ï v�� (8.58)

Sinceweneedí

in theinterval Wõ� [ � U¼� � c werewrite (8.58)usingLemmaB.2:� v ì O·� � Ñ�Þ í î � Ïv ì ñ O®� � Ñ�Þ í ò � Þlî � Ï ó Üv ìrí Ñ ñ O·� � ò � Þ}î � �from whichwe get í v ì ñ � ��O ò � Þ£î � �For thisto bewell-definedÞ mustbeadivisorin ñ � �BO ò , i.e.

ì � �~OAî Ü v� sowemustchoose � v ì OAî Ü


Expandingtheexpressionfor O we obtainthedesiredexpressionsÑ Ò ì O UfÓ ÔÖÕ Ì Õ Nýî Ü (8.59)× Ò Ø�ÙdÑ Õ O UfÓ Ô+Ú�Û�Ú NCÜ�Ý�ÞHß �áà (8.60)

Finally, â is obtainedas â ÒãØ¾× Ú~ä UfÓ Ô Ý�Þ Õ�å ß �áà (8.61)

Let Ç UfÓ ÔÏ Ó æ bedefinedby (8.56). Using (8.59),(8.60)and(8.61)we canformulatethealgorithmas

Algorithm 8.3: ç�èpé ê�ë±ì èpé ê�í ê , îðïòñFor Û Ò�ó to ô Ô Õ�å

if õ öÉ÷øõúù~û thenÑüÒýØ äqþ Ó Ô Õ Û Õ�ÿ ß��×�Ò�ØÙ¾Ñ Õ äqþ Ó Ô Ú�Û�Ú ÿ Ü�Ý Þ*ß�� àâ ÒýØ¾× Ú~äqþ Ó Ô Ý�Þ Õ�å ß�� àif×�� â then� þ Ó Ô�� Ò � þ Ó Ô�� Ú ö8Ô÷ � þ Ó Ô� Ó æ

else (wrap)� þ Ó Ô� � Ò � þ Ó Ô� � Ú ö Ô ÷ � þ Ó Ô à�� à�� Ó æ� þ Ó Ô�� à �� Ò � þ Ó Ô�� à �� Ú ö8Ô÷ � þ Ó Ô� � à�� Ó æend

endend

8.4.3 Algorithm

Thefull algorithmfor computing(8.49)is

Algorithm 8.4: Matrix-v ector multiplication (CIRMUL)� Ò�óFor � Ò ó to �

For � Ò�ó to �If �! "� then� þ Ò � þ Ú � þ Ó Ô computed with Algorithm 8.2else � þ Ò � þ Ú � þ Ó Ô computed with Algorithm 8.3


8.4.4 Computational work

We are now readyto look at the computationalwork requiredfor the matrix-vectormultiplication �$#&%(' . We take(8.49)asthepointof departureandstartby consideringthetypicalblock % þ*) + ' +Thelengthof ' + is ô + so for blockson andbelow thediagonalof % ( �, &� ) inAlgorithm 8.2thereare ä þ*) + ô + multiplicationsandthesamenumberof additions.Hencethework is - ä þ*) + ô +floatingpointoperations.For blocksabove thediagonal(��ù&� ) in Algorithm 8.3thereare ô + ä þ*) + Ý�Þ # ä þ*) + ô þ multiplicationsandadditions,sothework hereis- ä þ*) + ô þThetotalwork canthereforebewrittenasfollows./ +10 � - ä +�) + ô + Ú ./ þ20 � þ ��/ +30 � - ä þ*) + ô + Ú ./ +30 � + ��/ þ20 � - ä þ*) + ô þwherethefirst sumcorrespondsto blockson thediagonal,thesecondto blocksbelow thediagonal,andthethird to blocksabove thediagonal.Since ä þ4) + # ä +�) þwecanswaptheindicesof thelastdoublesumto find theidentity./ +30 � + ��/ þ20 � - ä þ*) + ô þ # ./ þ�0 � þ ��/ +30 � - ä þ4) + ô +Hencewe may write the total work of the matrix-vectormultiplication with thewavelettransformof acirculantmatrixas5

CIRMUL# - ./ +10 � ä +�) + ô + Ú76 ./ þ20 � þ ��/ +30 � ä þ*) + ô + (8.62)

Werecallthat ô + #98 ôðÝ - . for � # óôðÝ - . � +3: � for å<; � ; �


andthat ä þ*) + is givenby therecurrenceformulasä + �� ) + �� # = ä +�) + Ý -?> ÚA@9B åä þ*) + �� # ä þ*) + Ú - þ � + Ù @9B å Üä . : � ) . : � # ä (thebandwidthof theoriginalmatrix C )

Tables8.2,8.3,and8.4show5

CIRMUL evaluatedfor variousvaluesof ä , @ , � , andô . � #&DFE ä #&Gô @ # - @ # 6 @ #IH @ #&JD - K J 6 å 6 K�- å J ó J - ó H 6H 6 å G?H?J D ó K�- 6 G K H G KML?-å - J D å D?H H å 6?6 L?- J ó å -?- J?J- GNH H -OK�- å -N- J?J å JNG?H ó - 6 G K HG å - å - G 6?6 - 6 G K H D K å - ó 6 L å G -å ó - 6 - G ó J?J 6 L å G - K 6 - 6 ó L J?D ó 6- ó 6 J G ó å K H L JND ó 6 å 6 J 6 J ó å L HNH ó JTable 8.2: The numberof floating point operationsP CIRMUL as a function of Q fordifferentvaluesof R .

ô # - G?HSE @ # 6� ä #TD ä # 6 ä #&G ä #&Hó å G?DNH - ó 6 J - G?H ó D ó KM-å G å - ó G å - ó H å 6N6 H å 6N6- J 6?6 J J 6?6 J LN- å H LN- å HD å³å G - ó å³å G - ó å -?- JNJ å -?- J?J6 å 6 G LN- å 6 G LN- å G?D?H ó å G?DNH óG å K H?H 6 å K H?H 6 å J 6 D - å J 6 D -Table 8.3: Thenumberof floatingpointoperationsP CIRMUL shown for differentvaluesof U and V . Notethat VXWAY�Z and VXWAY�Z,[]\ ( Z_^a` ) yield thesamevaluesfor Ucb7d .This is adirectconsequenceof theroundingdonein (8.62).

8.5Summary 161ô # - G?HSE ä #IG� @ # - @ # 6 @ #&H @ #&Jó - G?H ó - G?H ó - G?H ó - G?H óå 6 ó L H H å 6N6 J å LN- å ó - 6 ó- G å - ó L?- å H å D?D å - å K 6 ó JD H -OK�- å -N- J?J å J?G?H ó - 6 G K H6 K D?H ó å GND?H ó - D?H?J ó D å J ó JG J 6 å H å J 6 D - - J K DNH D?JND?DNHTable 8.4: Thenumberof floatingpoint operationsP CIRMUL shown for differentvaluesof U and R .

Table8.2 shows that5

CIRMUL dependslinearly on ô . Moreover, Tables8.3and8.4show thatthecomputationalworkgrowswith thebandwidthä , thewaveletgenus@ , andthe transformdepth � . Supposethat ä is given. Thenwe seethatthematrix-vectormultiplicationis mostefficientif wetakenostepsof thewavelettransform( � # ó ). Consequently, any work reductionmustbesoughtin trunca-tion of thevector ' . This canbe justifiedbecause' will oftenbea 1D wavelettransformof thesamedepth( � ) asthematrix % . Therefore,dependingon � andtheunderlyingapplication,we expectmany elementsin ' to becloseto zerosowe maybeableto discardthemin orderto reducethecomputationalwork. Con-siderTable8.3. If we take ô # - G?H , ä #9D , � # 6 asanexample,thequestionboilsdown to whethersucha truncationof ' canreduce14592operationsto lessthanthe 1536operationsrequiredfor � # ó (no transform).Assumingthat theworkdependslinearlyonthenumberof non-zeroelementsin ' , thismeansthat 'mustbereducedby a factorof å ó (at least)beforeany work reductionis obtained.

8.5 Summary

Wehavederivedanalgorithmfor computingthe2D wavelettransformof acircu-lant ôfe ô matrix g in h Ù ô\Ü steps.More specificallywederivedthebound5

CIRFWTÙ ô\Ü � - @ ô Ù 6�Ú � Ü

Thealgorithmworkswith a datastructurethat requiresno morethan- �øô ele-

mentsinsteadof ôji . If C is alsobandedthe storagerequirementsarereducedfurther. Thenwe have shown how a bandedmatrix-vectormultiplication usingtheproposedstorageschemecanbe implementedandthecomplexity of this al-


gorithmwasanalyzed.It wasfoundthatthework neededto computethematrix-vectormultiplicationis linearin ô andgrowswith thebandwidth( ä ), thewaveletgenus( @ ), andthedepthof thewavelettransform� . On theotherhand,theworkis reducedwhensmallelementsin ' arediscarded.

Chapter 9

Examplesof wavelet-basedPDEsolvers

We considerherethe taskof solving partial differentialequationsin onespacedimensionusingwavelets.We assumethat theboundaryconditionsareperiodicandmakeuseof theperiodizedwaveletsdescribedin Section2.3andthematerialdevelopedin Chapters7 and8. Sections9.1 and9.2 treat linear problemsandserveto setthestagefor Sections9.3and9.4whichdealwith nonlinearproblems.

Webegin by consideringaperiodic boundary valueproblembecauseit is asimpleandillustrativeexample.

9.1 A periodic boundary valueproblem

Considerthe1D HelmholtzequationBlk�m m³Ú]nok # p Ù öVÜk Ù öVÜ # k Ù ö Ú å Ürq örsut (9.1)

wheren sut and p Ù öVÜ #&p Ù ö Ú å Ü .We look for å -periodicsolutions,soit sufficesto considerk Ù öVÜ on theintervaló ; ö � å .

9.1.1 Representationwith respectto scalingfunctions

Webegin thediscretizationof (9.1)by replacingk Ù öVÜ with theapproximation

kwv Ù öVÜ # i�x ��/ � 0 � Ù�y�z Ü v ) �|{} v ) � Ù öVÜ E ~ su� � (9.2)

164 Examplesof wavelet-basedPDEsolvers

Following theapproachthatleadto (7.9)we find thatk m mv Ù öVÜ # i�x ��/ � 0 � Ù�yM� i1�z Ü v ) �|{} v ) � Ù öVÜ (9.3)

whereÙ�y � i1�z Ü v ) � is givenasin (7.12),i.e.Ù�y � i��z Ü v ) � #�� i1�*� z�� # � � i/÷ 0 i � � Ù�y z Ü v )�� ÷ : �� x - v æ�� æ÷ E × # ó E å E��oE - v B å

with � æ÷ definedin (7.1).We canusetheGalerkinmethodto determinethecoefficients

Ù�y�z Ü v ) � . Multi-plying (9.1)by {} v ) Ù öVÜ andintegratingover theunit interval yieldstherelationB�� k m mv Ù öVÜ {} v ) Ù öVÜF�Cö Ú$n_� �� k�v Ù öVÜ {} v ) Ù ö ÜO�Cö # � �� p Ù öVÜ {} v ) Ù ö ÜO�CöUsing (9.2), (9.3), and the orthonormalityof the periodizedscaling functions(2.48)weget B Ù�y � i��z Ü v ) Ú]n Ù�y�z Ü v ) # Ù�y�� Ü v ) E â # ó E å E��|E - v B åwhere Ù�y�� Ü v ) # � �� p Ù öVÜ {} v ) Ù öVÜF�Cö (9.4)

In vectornotationthisbecomesB � � i��z Ú]n � z # � � (9.5)

andusing(7.13)we arriveat thelinearsystemof equationsg � z # � � (9.6)

where g # B � � i1� Ú]no� (9.7)

Alternatively, we canreplace� � i1� by � i , where� is givenin (7.14),andobtaing � z # � � (9.8)

where g # B � i Ú$n�� (9.9)

Equations(9.6)and(9.8)representthescalingfunctiondiscretizationsof (9.1).Hencethis methodbelongsto Class å describedin Section7.1. We observe that(9.6)hasa uniquesolutionif n doesnot belongto thesetof eigenvaluesof � � i1� .Similarly (9.8)hasauniquesolutionif n doesnotbelongto thesetof eigenvaluesof � i .

9.1A periodicboundaryvalueproblem 165

B��*�N� i Ù� kcB]k v v ) ¡ Üg # B � � i�� Ú]no� g # B � i Ú$n��~ @ #&H @ #&J @ # 6 @ #&H @ #&J- å � LNL DS� 6 ó å ��J ó DF� ó G 6 � å JD 6 ��J å JS� å G GS� K å JF��H ó åËå � 6 -6 JS��G K å DF��J K L ��HNG å 6 � 6 H å L � - åG å - �¢G å å L ��J å å DS��HND - ó � 6 D -OK � å HH å HS�¢G ó - GF� K�L å K ��HND - HF� 6 - D?GF� å GK - ó �¢G ó D å � K�L - å ��HND D - � 6 - 6 - � 6 JJ - 6 �¢G ó 6 ó � ó D - GS��HND D?JF��H å 6 - � ó åL - JS�¢G ó D?GF� 6 J -NL ��HND D L � -£K 6 ó ��J?GTable 9.1: Theerror ¤¦¥�§�¨ i�©1ª¬« ¤ « v ª v ) ¡® shown for differentvaluesof ¯ , R andthechoiceof differentiationmatrix. Convergenceratesareseenfrom thedifferencesbetweensuccessive measurements.The value R°W²± is omittedin the caseof (9.7) becausethecombinationR³W"±F´¬µ,W]Y is invalid asdescribedin Remark7.1.

Accuracy

Let p Ù öVÜ # Ù 6£¶ i Ú]n ÜF·1¸*¹ Ù - ¶ öVÜ . Thesolutionof (9.1) is thenk Ù öVÜ # ·�¸*¹ Ù - ¶ öVÜDefine k ¡ # ºj»�¼��½£¾�¿ � õ k Ù öVÜ`õ (9.10) k v ) ¡ # ºj»�¼� 0 � ) � )�À�À�ÀÁ) i x ��ÃÂÂ k Ù¾× Ý - v Ü ÂÂ (9.11)

Table9.1shows how theerror kcB"k�v v ) ¡ dependson ~ , @ , andthechoice

of � � i1� or � i . Until theonsetof roundingerrors,theconvergenceratesobtainedin Chapter7 arerecovered.More precisely, k_B]k�v v ) ¡ # h�Ä - � v � � � i1��Å in thecaseof (9.7) k_B]k�v v ) ¡ # h Ä - � v � Å in thecaseof (9.9)

Table9.2showshow theerror kcB]kwv ¡ dependson ~ , @ , andthechoiceof� � i1� or � i . In this case,we find k_B7kwv ¡ # h�Ä - � v � � i�Å in thecaseof (9.7) k_B7kwv ¡ # h�Ä - � v � � i�Å in thecaseof (9.9)


BX�*�?� i�Æ k¦B]k v ¡ÈÇg # B � � i1�ÊÉ n�� g # B � iËÉ no�~ @ #ÌH @ #ÌJ @ # 6 @ #IH @ #ÌJ- å � å G - � å J ó � L?L å ��HNJ - � å óD DS�¢D?H GF��J 6 - � 6 å DF� K J GF��H L6 HS� 6 G L ��H å 6 � å H HF��H å L ��G 6G L �¢G?D å DF��G å HS� ó J L ��GNJ å DF� 6 LH å - �¢G?H å K � 6 L JS� ó H å - �¢G K å K � 6 JK å GS�¢G?H - å � 6 J å ó � ó H å GS�¢G K - å � 6 JJ å JS�¢G K - GF� 6 J å - � ó H å JS�¢G K - GF� 6 JL - å �¢G K -?L � 6 G å 6 � ó H - å �¢G K -?L � 6 JTable 9.2: The error ¤¦¥�§�¨ i ©1ª�« ¤ « v ª ¡ shown for differentvaluesof ¯ , R andthedifferentiationmatrix. Convergenceratesareseenfrom thedifferencesbetweensucces-sivemeasurements.Thevalue R³W"± is omittedin caseof (9.7)becausethecombinationRÍW"±F´¬µÎW]Y is invalid asdescribedin Remark7.1.

whichagreeswith theapproximationpropertiesdescribedin (4.5).Thehigh convergenceobserved in Table9.1 meansthat the solution k Æ ö Ç is

approximatedto @ th orderin thenorm �ÏÐ v ) ¡ even thoughthe the subspace{Ñ v

canonly representexactly polynomialsup to degree @cÒ - B å , asexplainedinSection4.1. Thisphenomenon,known assuperconvergence, is alsoencounteredin thefinite elementmethod,[WM85, p. 106], [AB84, p. 231].

9.1.2 Representationwith respectto wavelets

Taking(9.6)aspointof departureandusingtherelationsÓÔz # Õ � zÓ|� # Õ � �yields g Õ×Ö ÓÔz #(ÕØÖ Ó|�Let Ùg #(Õ g Õ Ö #ÚÕ Û B � � i1� É n��ÝÜ Õ Ö # B Ù� � i1� É no�

9.1A periodicboundaryvalueproblem 167

where

Ù� � i1� is definedaccordingto (7.22).ThenÙg ÓÝz # Óo� (9.12)

which is thewaveletdiscretizationof (9.1). This methodbelongsto Class-

de-scribedin Section7.1. Hence,thereis a potentialfor usingwaveletcompressionto reducethecomputationalcomplexity of solving(9.12).

9.1.3 Representationwith respectto physical space

Multiplying (9.5)by Þ andusing(3.17)yieldsBàß � i1� É noß #Íáwhere á]# Þ � � #(â Ä�ãÎäå x p Å Æ ö Ç�æ i x �� 0 � (9.13)

Fromtherelation ß � i1� #&� � i1� ß (7.15)we obtaintheequationg ß #&á (9.14)

where g is givenby (9.7).An importantvariantof (9.14)is obtainedby redefiningá asthevectorá]#³çNp Æ ö Ç¬è i x �� 0 �

Thisproducesacollocationschemefor thesolutionof (9.1).Becausethismethodis essentiallybasedonscalingfunctionsanddoesnotuse

waveletcompressionit belongsto Classå describedin Section7.1.

9.1.4 Hybrid representation

Wementionasecondpossibilityfor discretizing(9.1)usingwavelets.It is simpleandseveral authorsfollow this approach,e.g. [EOZ94, CP96, PW96]. This isessentiallyacombinationof theapproachesthatleadto (9.14)and(9.12),andweproceedasfollows: Multiplying (9.14)from theleft by Õ andusingtheidentityÕ Ö Õ # � oneobtains Õ g Õ Ö Õ ß #(Õéá


DefiningthewavelettransformedvectorsasÙá # Õéá andÙß # Õ ß

thenyields Ùg Ùß # Ùá (9.15)

where

Ùg is thesameasin (9.12).Thisapproachbypassesthescalingcoefficientrepresentationandreliesonthe

fact that theFWT canbeapplieddirectly to functionvaluesof p . Indeed,usingthis approach,the differentiationmatrix g might aswell be derived from finitedifferences,finite elementsor spectralmethods.

FromSection4.3weknow thattheelementsin

Ùß will behave similarly asthetruewaveletcoefficients

ÓÝz. Therefore,waveletcompressionis aslikely in this

caseaswith thepurewaveletrepresentation(9.12).HencethismethodbelongstoClass

-describedin Section7.1.

9.2 The heatequation

Weconsidernow theperiodicinitial-valueproblemfor theheatequationkÊê # ë k ¾�¾ É p Æ ö Ç Erì ù ók Æ ö E ó Ç # í Æ ö Çk Æ ö E¬ì Ç # k Æ ö É�å E¬ì Ç E�ì ó î�ïð örsut (9.16)

where ë is a positive constant,p Æ ö Ç #ñp Æ ö ÉXå Ç and í Æ ö Ç #òí Æ ö É å Ç . Thediscretizationstrategiesaretime-dependentanaloguesof thosein Section9.1.

9.2.1 Representationwith respectto scalingfunctions

We considerfirst theGalerkinmethodfor (9.16)andproceedasin Section9.1.1.Thetime-dependentversionof (9.2) iskwv Æ ö E¬ì Ç # i�x ��/ � 0 � Æ yz Ç v ) � Æ ì Ç {} v ) � Æ ö Ç (9.17)

After an analysissimilar to that which lead to (9.5), we arrive at the Galerkindiscretizationof (9.16):�� ì � z Æ ì Ç # ë�� i1� � z Æ ì Ç É"� � E ì ù ó� z Æ ó Ç # ��ó (9.18)

9.2Theheatequation 169

where� � is givenby (9.4)and

Æ y ó Ç v ) � # � �� í Æ ö Ç {} v ) � Æ ö Ç �Cö E ôj# ó E å E��|E - v B å (9.19)

9.2.2 Representationwith respectto wavelets

Multiplying (9.18) from the left by Õ and insertingthe identity Õ Ö Õ # �yields �� ì Õ � z Æ ì Ç #&ëÝÕf� � i1� ÕØÖoÕ � z Æ ì Ç É Õ � � E ì ù óFromtheidentities Ó|z Æ ì Ç # Õ � z Æ ì ÇÓo� Æ ì Ç # Õ � � Æ ì ÇÙ� � i1� # Õf� � i1� Õ Öwethenobtain �� ì ÓÝz Æ ì Ç #&ë

Ù� � i1� Ó|z Æ ì Ç É Ó|� E ì ù ó (9.20)

with theinitial condition Ó|z Æ ó Ç #�Õ �Fó9.2.3 Representationwith respectto physical space

Proceedingasabove but this time multiplying (9.18)from theleft by Þ yields�� ì Þ � z Æ ì Ç #&ë Þ � � i�� Þ �� Þ � z Æ ì Ç É Þ � � E ì ù óUsingtherelations ß Æ ì Ç # Þ � z Æ ì Çá # Þ � � # â Ä¬ãõäå x p Å Æ ö Ç�æ i x �� 0 �� i1� # Þ � � i1� Þ ��wefind �� ì ß Æ ì Ç #Ìë�� i1� ß Æ ì Ç É áöE ì ù ó (9.21)


with theinitial conditionß Æ ó Ç #Í÷]# â Ä ãõäå x í Å Æ ö Ç æ i x �� 0 � (9.22)

Also in thiscasewe canproducea collocationschemeby redefiningá # çMp Æ ö Ç¬è i x �� 0 �÷ # çMí Æ ö Ç�è i x �� 0 �9.2.4 Hybrid representation

Multiplying (9.21)from theleft with Õ andproceedingasin Section9.1.4yields�� ì Ùß Æ ì Ç #&ëÙ� � i�� Ùß Æ ì Ç É

ÙáøE ì ù ó (9.23)

with theinitial condition Ùß Æ ó Ç #ÚÕù÷9.2.5 Time steppingin the waveletdomain

A numberof time-steppingschemesareavailablefor thesystem(9.20).We con-siderherethebackwardEulermethoddefinedbyÆ Ó z Ç ÷ : � B Æ Ó z Ç ÷ú ì #&ë Ù� � i1� Æ Ó|z Ç ÷ : � É Ó|� (9.24)

where Æ ÓÝz Ç ÷ # ÓÝz Æ�û ú ì Ç . This leadsto therecursion

Æ Ó|z Ç ÷ : � # Ùg �� Æ�Æ Ó|z Ç ÷ É ú ì Óü� Ç E û # ó E å E��|E û � B åÆ ÓÝz Ç � # ÓÝz Æ ó Ç (9.25)

whereû � su� and Ùg # �rB ë ú ì Ù� � i1�ThebackwardEulertime-steppingschemesfor (9.18),(9.21),and(9.23)arecom-pletelyanalogous.

Thematrix

Ùg hasthecharacteristic“finger-band”patternshown in Figure7.1,

andit turnsout that so does

Ùg �� for smallú ì whensmall entriesareremoved

[CP96]. Figure9.1showsanexampleof

Ùg �� whereelementssmallerthan å ó ��have beenremoved.


0 512 1024

0

512

1024

nz = 125056

Figure9.1: ýþ �� whereelementssmallerthan \�d ��1� havebeenremoved. RÍW]ÿ , U W �,¯ WÌ\�d , ��|W]Y ��1� , and �®W]d��d�� .

Moreover, we canexpectmany elementsinÓÝz

andÓo�

to be small by virtueof Theorem2.5. Consequently, thereis a potentialfor compressionof both thecoefficient matrix aswell asthesolutionvector. This is true for both(9.20)and(9.23)but wegivethealgorithmfor (9.20)only sincethecase(9.23)is completelyanalogous.

Let truncÆ Ó E û Ç be a function that returnsonly the significantelementsin agivenvector

Ó, i.e. let

truncÆ Ó E û Ç #Íç � +�) � E õ � +�) � õ8ù}û èSimilarly, let

truncÆ g E û Ç #³ç�� g � � ) ÷ E õ � g �� ) ÷ õ>ù�û èLet û å be the compressionthresholdfor the vectorand û� be the compressionthresholdfor thematrixanddefine

Ó�� # truncÆ Ó E û å Ç and

Ùg �� # truncÆÙg E û� Ç .

Following ideasgivenin [CP96] we cannow give a computationalprocedurefor computing(9.25)usingwaveletcompression.

Algorithm 9.1å�� ÆÙg �� Ç � � �

trunc Æ Õ g �� Õ Ö E û� Ç- � Ó �� trunc Æ Õ � � E û å ÇD � Æ ÓÝz Ç �� trunc Æ Õ ��ó E û å Ç


for û # ó E å E��oE û � B å6 � Æ Ó|z Ç ÷ : � � ÆÙg �� Ç � � Ä Æ ÓÝz Ç ��÷ É ú ì Ó �� ÅG � Æ Ó z Ç ��÷ : � �

trunc Æ¬Æ Ó z Ç ÷ : � E û å ÇendH � Æ � z Ç ÷� � ÕØÖ Æ ÓÝz Ç ��÷�K � Æ ß Ç �� ) � �÷� � Þ Æ � z Ç ÷�

A few commentsareappropriate:� Step � : We haveÙg �� # Ä Õ g Õ Ö Å �� # Ä Õ Ö Å �� g �� Õ �� #ÚÕ g �� Õ ÖHence,step å canbedoneusingthe2D FWT (3.38). However, since g iscirculantthen g �� is alsocirculantby TheoremC.6andit canbecomputed

efficiently usingtheFFT asexplainedin AppendixC. Consequently,

Ùg ��canbecomputedandstoredefficiently usingAlgorithm 8.1.� Steps� �! : ��ó , � � arebothcomputedaccordingto (3.17),i.e.by thequadra-tureformulas � � # Þ �� á� z # Þ �� ÷where á # çNp Æ ö � Ç¬è i x �� 0 �÷ # çNí Æ ö � Ç�è i�x �� 0 �� Step " : It is essentialfor thesuccessof this algorithmthatthecomputationof the matrix-vectorproduct fully exploits the compressedform of bothmatrix andvectors. This canbe done,for example,usingAlgorithm 8.4.In [CP96,p. 7] fastmultiplicationis basedon a generalsparseformat forbothmatrixandvector.� Steps #$�&% : Finally, the computedvectorof wavelet coefficients is trans-formedbackinto the physicaldomain. We denotethe computedsolutionß � � ) ��÷� , becauseit dependson boththresholds.


To testAlgorithm 9.1wehave appliedit to theproblemk ê # ë k ¾�¾ É ·�¸4¹ Æ - ¶ ö Ç E ì ù ók Æ ö E ó Ç # ó q örsut (9.26)

Thenumericalparametersare ~u# å ó , ú ì!# å Ò - � � , û � # - � � (makingó ; ì ;�å ),ë�# ó � ó å Ò�¶ , � #ÚD , and @ #ÚJ . Thevector ß �� ) � �÷'� is theresultof Algorithm 9.1

giventhethresholdsû� and û å . Hence,we definetherelative compressionerroras ( � � ) �� # )) ß �� ) � �÷'� B7ß � ) �÷� )) ¡)) ß � ) �÷� )) ¡where ß ¡ # ºr»�¼� 0 � ) � )�À�À�À*) i x �� õ k õ

Table9.3showsthepercentageof significantelementsin ÆÙg �� Ç � � and Æ ÓÝz Ç ��÷�

for variousvaluesof û å and û� . Moreover, therelative errorintroducedby com-pression

( �� ) � � is givenfor eachcase.It is seenthatsignificantcompressioncanbeachievedin boththematrixandthesolutionvector.

û å # ó % elem û'� # ó % elemû � ÆÙg �� Ç �� ( �� ) � � û å Æ Ó z Ç � �÷'� ( �� ) � �* ó �� *�L � ó H * ó ��1� * ó �� L GF� ó - * ó ��1�* ó �� * HF��GNH * ó �� i * ó �� 6 L � *�- * ó �� * ó �,+ * 6 � ó J * ó �� À - * ó �,+ D K � KML * ó ��+* ó �,. *&* ��G L * ó ��+ * ó �,. -?- � *�K * ó ��.* ó �0/ L � - J * ó �1/ À - * ó �0/ *&* ��J - * ó �1/* ó �,2 K � - ó * ó ��2 * ó �,2 GF� L H * ó ��2* ó � - GF� * J * ó ��3 À - * ó � - 6 � - ó * ó � -

Table 9.3: Percentageof elementsretainedin © ýþ �� and © 4 z ��÷'� togetherwith theresultingcompressionerror.


9.3 The nonlinear Schrodinger equation

Weconsidernow theperiodicinitial-valueproblemfor thenonlinear Schrodingerequation:

k�ê # 5 k É �76�õ k õ i k E³ì ù ók Æ ö E ó Ç # í Æ ö Çk Æ ö E¬ì Ç # k Æ ö É98 E�ì Ç E ì ó î ïð örsut (9.27)

whereí Æ ö Ç #Ìí Æ ö É:8 Ç and 5I# B �-<; i = i= ö i B n -Thisequationdescribesthepropagationof a pulseenvelopein anopticalfiber. ; iis adispersionparameter, n is a measureof intensityloss,and 6 is themagnitudeof thenonlineartermwhichcounteractsdispersionfor certainwaveforms[Agr89,FJAF78, LBA81, PAP86]. Sometimesthe term �2 ;?>A@CB@ ¾ B , which representsthirdorderdispersion,is addedto 5 .

Becauseof periodicitywecanrestrictthespatialdomainof (9.27)to theinter-val � B 8 Ò - E 8 Ò - � . Let ô # - v anddefinea grid consistingof thepointsö #EDGFô B *-<H 8 E F # ó E * E��oE ô B *Definethevector ß Æ ì Ç suchthatk Æ ì Ç # k v Æ ö E¬ì Ç E F # ó E * E��|E ô B *where k�v Æ ö E�ì Ç is an approximatesolutionof (9.27) of the form (9.17). Definesimilarly thevectorsß � i1� Æ ì Ç and á Æ ì Ç ask � i�� Æ ì Ç # k m mv Æ ö E¬ì Ç E F # ó E * E��|E ô B *p Æ ì Ç # p Æ ö E¬ì Ç E F # ó E * E��|E ô B *Hencetheelementsin ß approximatefunctionvaluesof k in theinterval � B 8 Ò - E 8 Ò - � .With regardto our usingtheinterval � 8 Ò - E 8 Ò - � insteadof � ó E * � we mentionthatthe mappingsÞ � and Õ � areunchangedasdescribedin Section3.2.4but thedifferentiationmatrixmustbescaledaccordingto (7.18).Hence,we arriveat thefollowing initial problemfor ß formulatedwith respectto physicalspace:�� ì ß Æ ì Ç # I ß Æ ì Ç ÉKJ Æ ß Æ ì Ç¬Ç ß Æ ì Ç E ì óß Æ ó Ç # ÷ML9��í Æ ö � Ç E�í Æ ö � Ç E��|E�í Æ ö � �� Ç � Ö (9.28)

9.3ThenonlinearSchrodingerequation 175

where I # B �- ; i � � i1�8 i B n - �J Æ ß Æ ì Ç�Ç # �N6 diagÆ õ k Æ ì Ç õ i E F # ó E * E��oE ô B * Ç9.3.1 Waveletsplit-stepmethod

A well establishedtime-steppingschemefor solving(9.27)in opticsapplicationsis thesplit-step method which is alsoknown asthe beampropagationmethod.Traditionallyit isappliedin conjunctionwith aspatialdiscretizationschemebasedon a Fourier spectralmethodin which caseit is known asthe Fourier split-stepmethod,or split-stepFFT, (See,for example,[Agr89, p. 44–48]or [New92, p.413–423]). However, we will usesplit-steppingwith the spatialdiscretizationdescribedabove andcall it the wavelet split-step method. A similar approachhasbeeninvestigatedin [GL94, PW96].

Thesplit-stepmethodderivesfrom thefact thatthesolutionof problem(9.28)satisfiestheidentityß Æ ì É ú ì Ç #PO�¼1QRD ú ìCI É � ê :TS êê J Æ ß Æ�U Ç�Ç � U H ß Æ ì ÇWenow introducetheapproximations� ê :TS êê J Æ ß Æ�U Ç�Ç � UWV ú ì- � J Æ ß Æ ì Ç¬Ç ÉMJ Æ ß Æ ì É ú ì Ç�Ç �and O�¼1QXD ú ìCI É ú ì- � J Æ ß Æ ì Ç¬Ç ÉKJ Æ ß Æ ì É ú ì Ç�Ç � H VO�¼1QXD ú ì- I H O�¼1QXD ú ì- � J Æ ß Æ ì Ç�Ç ÉMJ Æ ß Æ ì É ú ì Ç�Ç � H O�¼1QXD ú ì- I HUsingtheseweobtainthetime-steppingprocedureß ÷ : � # YPO�¼1QXD ú ì- � J Æ ß ÷ Ç ÉMJ Æ ß ÷ : � Ç � H Y ß ÷ E û # ó E * E��|E û � B *ß � # ÷

(9.29)

whereß ÷ # ß Æ�û ú ì Ç , ÷ is thevectordefinedin (9.28),andwhereY #:O�¼ZQXD ú ì- I H


Thevector ß ÷ : � appearsalsoon the right-handsideof (9.29)so eachtime steprequiresaniterativeprocedure.Oursis thefollowing: For eachû letß � � �÷ : � # ß ÷anditerateuntil convergence,ß �\[ : � �÷ : � #]Y^O�¼ZQ D ú ì- � J Æ ß ÷ Ç ÉKJ Æ ß �_[ �÷ : � Ç � H Y ß ÷ E `È# ó E * E��Definingthefunction

linstepÆ�a Ç #]Y awehave thefollowing algorithmfor (9.29):

Algorithm 9.2: Split-stepmethod*: ß � � ÷"#ÚçNí Æ ö � Ç�è i x �� 0 �-: Y � O�¼ZQ Ä S êi I ÅD : J � � J Æ ß � Ç

for û # ó E * E��|E û � B *6 : a ÷ � linstepÆ ß ÷ ÇG : J � � J �Iterate until convergenceH : ß ÷ : � � linstep Ä O�¼ZQ Ä S êi � J � ÉKJ � � Å a ÷ ÅK

: J � � J Æ ß ÷ : � Ç9.3.2 Matrix exponentialof a circulant matrixJ Æ ß ÷ Ç is adiagonalmatrixfor everyvalueof û , sotheexponentiationrequiredinstep H of Algorithm 9.2canbedonecheaply. Thematrix I , on theotherhand,isnotdiagonalbut it is circulant.Hence,with regardto step

-, we needa procedure

for computingthematrixexponentialY #cb g (9.30)

where g is thecirculantmatrix g # ú ì- I


Standardmethodsfor computing(9.30)aremostly basedon matrix multiplica-tions [GL89] which have complexity h Æ ô > Ç . Hencethe computationof (9.30)canbeveryexpensive.

However, by using the fact that g is circulant one can compute(9.30) inh Æ ô �4�?� i ô Ç floating point operations.From TheoremC.4 we have the factor-ization g #ed ��Mfhg d �where d � is theFouriermatrix definedin (C.3), fig # diagÆ�jk Ç , jkT#ld � k , andk is thefirst columnin g . By a standardmatrix identityY #9O�¼ZQ Æ g Ç #ed �� O�¼ZQ Æ d � g d �� Ç d � #ed �� O�¼ZQ Æ fmg Ç d �Thematrix O�¼ZQ Æ f g Ç is diagonalsoweknow fromTheoremC.4that Y is circulant.Hencewe canwrite Y #ed �� fin d �wherefin # diagÆ jo Ç , jo�#ed � o , and o is thefirst columnin Y . Equatingfin andO�¼1Q Æ fig Ç yields

joj#9�pb�qg�r�Esb�qg � E��|Esb,qgut<v � � ÖHencewe have thefollowing fastalgorithmfor computing(9.30):

Algorithm 9.3: Exponential of circulant matrix wk � � g � � ) � ! Extract first column of gjk � d � k ! FFTjo � çO�¼1Q Æ jx � Ç¬è � �� 0 � ! Pointwise exponentiationo � d �� jo ! IFFT�yY � � ) ÷ � b � � �C÷ � t E{zuE û # ó E * E��oE ô B *! Assemble Y

Thematricesg and Y neednot bestoredexplicitly. Hencethefirst andthelaststepsherecanbeomittedin practice. In step

-of Algorithm 9.2,wherewe

useAlgorithm 9.3, only o is computed.The computationalcomplexity followsfrom thatof theFFT andits inverse.

In theproblemathand g # S êi I , where I is circulantandbanded.It followsfrom thedefinitionof thematrixexponentialY #PO�¼1Q Æ g Ç # ¡/ � 0 � g � Ò ô�| (9.31)


that Y is dense.However, it turnsout thatmany entriesin Y arevery smallandthat Y assumescirculantbandform if truncatedwith asmallthreshold.In [GL94,PW96] Y is approximatedby truncatingthe series(9.31) to a few terms. Thisalsoyieldsacirculantbandform,but theprocessis lessexactandnotsignificantlyfasterthantheonepresentedhere.

9.3.3 Compression

We canexploit the compressionpotentialof waveletsby an appropriateimple-mentationof linstep.Let ÙY #(Õ}YuÕ Ö (9.32)

and ÆÙY Ç � � # truncÆ

ÙYuE û� ÇThentheproduct Y a thatis computedby linstepcanbeformulatedasY a #ÚÕ Ö ÙYuÕ a #ÚÕ Ö ÙY Ùa~V Õ Ö Æ

ÙY Ç � � Ùa � �where

Ùa � � # truncÆ Ùa E û å Ç . Hencewe obtain the following versionof linstep

whichcanbeusedin step6 and H of Algorithm 9.2.

Algorithm 9.4: �� linstep�C��Ùa � Õ aÆ Ùa Ç ��

truncÆ Ùa E û å ÇÙß � ÆÙY Ç � � Æ Ùa Ç ��ß � Õ Ö Ùß

Since Y is circulant,

ÙY can be computedefficiently using Algorithm 8.1.Since Y is dense

ÙY is densetoo, but under truncation,even with very smallthresholds,it assumesthe finger-bandpatternassociatedwith differentialoper-atorsin wavelet bases.Figure9.2 shows

ÙY truncatedto machineprecisionfor� # ó E * E - E�DSE 6 E�G , thecase� # ó correspondingto Y .


0 256

0

256

nz = 5888

λ=0

0 128 256

0

128

256

nz = 9728

λ=1

0 64 128 256

0

64

128

256

nz = 12928

λ=2

0 32 64 128 256

0

32

64

128

256

nz = 16320

λ=3

0 16 32 64 128 256

01632

64

128

256

nz = 19648

λ=4

0 16 32 64 128 256

01632

64

128

256

nz = 22800

λ=5

Figure9.2: ý� �� for U W]dO´�\�´¬Y£´ � ´�±S´�� truncatedto theprecision��(W]Y�� Y�Y�d�±��à\�d ��72 .Theremainingparametersare R³W]ÿ , ¯ W]ÿ , ��|WÌ\��Y � � , V�WM��d , � i WÌ¤ Y , and � W]d .


% elem.retained� ÙY �� Û ÙYR� Ü ��ó JS� L J *�- � *&** * 6 �¢J 6 *�K � LOK- *�L � K D -N- � 6 HD - 6 � L ó -£K �¢D 66 -?L � L J D - � 6 -G D 6 � K�L D K � * DTable9.4: Thepercentageof elementsretainedin ý� �� and © ý� � � � for ��(W]Y�� Y�Y�d�±��\�d ��72 , RfW ÿ , ¯&W°ÿ , �� WØ\��Y � � , V³W��d , � i WØ¤ Y , �&W d . It is seenthat thecompressionpotentialof ý� � is slightly lowerthanthatof ý� .

Recallfrom Chapter7 thatthedifferentiationprocesshashigherconvergenceif � � i1� is replacedby � i , but that this comesat thecostof a largerbandwidth.Wewantto comparethesetwo choiceswith respectto compressionpotential.LetI � # B �- ; i � i8 i B n - �Y � # O�¼1QXD ú ì- I � HÙYR�ù# Õ�Y � ÕfÖThe percentagesof elementsretainedin

ÙY and

ÙYR� with truncationto machineprecisionareshown in Table9.4for � # ó E * E - E�DSE 6 E�G .

Figure9.3 shows

ÙY �� for differentchoicesof û� andTable9.5 shows thecorrespondingpercentagesof elementsretainedin

ÙY and

ÙYR� . It is observedthatthecompressionpotentialof Y � is slightly lower thanthatof Y , sowe will useonly Y in thefollowing.


0 32 64 128 256

0

32

64

128

256

nz = 4992

εM

=0.0001

0 32 64 128 256

0

32

64

128

256

nz = 11008

εM

=1e−07

0 32 64 128 256

0

32

64

128

256

nz = 13504

εM

=1e−10

0 32 64 128 256

0

32

64

128

256

nz = 14976

εM

=1e−13

Figure 9.3: ý� � � for ��fW \�d ��3 ´�\�d �1/ ´�\�d �� ´�\�d �� > . The remainingparametersareR&W]ÿ , ¯�W]ÿ , U W �, ��|WÌ\��Y � � , VrWG��d , � i WI¤ Y , and � W]d .

% elem.retainedû� ÙY �� Û ÙYR� Ü ��* ó ��3 K �¢H - HF� - G* ó �1/ * HS�¢J ó *�K ��DNJ* ó �� - ó �¢H * -1* ��D L* ó �� > -N- �¢J?G - 6 ��H *Table 9.5: The percentageof elementsretainedin ý� � � and © ý� � � � for �� W\�d ��3 ´�\�d �1/ ´�\�d �� ´�\�d �� > . The remainingparametersare R W ÿ , ¯ W ÿ , U�W �

,��àW°\s�Y � � , V&Wl��d , � i W°¤ Y , and �ÌW d . Hence,the first columncorrespondstothematricesshown in Figure9.3.


Thresholds % elem.retainedû� û å ( � � ) �� ÙY �� Ùß ��÷'� CPUtime(s)ó ó - � - D e * ó ��2 * óËó * óËó L H?D* ó �� - ó - � - D e * ó ��2 DNJ * óËó G 6 ** ó �� * ó �� - - � - D e * ó ��2 6 * óËó DND?D* ó �� * ó �� i - � - D e * ó ��2 6 J L - G K* ó �� * ó ��+ - � - 6 e * ó ��2 6 D ó *�K D* ó �� * ó �1/ HS��H - e * ó ��2 6 * H * 6 ** ó �� * ó ��2 HS��JNJ e * ó � - 6 * ó *�- HTable 9.6: Error, compressionandexecutiontime datafor a problemwith known solu-tion (compressionof ý� ÷� is givenfor �w� W]Y�d�d�d ).

Thewaveletsplit-stepmethodasdefinedby Algorithm 9.2andtheimplemen-tationof linstepgiven in Algorithm 9.4 hasbeenimplementedin Fortran90 ona CRAY C92A. Algorithm 9.3 was usedfor computingthe matrix exponentialY #lO�¼ZQ Ä S êi I Å , Algorithm 8.1 wasusedfor (9.32)usingthedatastructurede-scribedin Section8.2.5,andAlgorithm 8.4 wasusedfor computingtheproduct

ÆÙY Ç �� Æ Ùa Ç � �

Finally, thetransformsÕ a and Õ Ö Ùa wereimplementedusingtheFWT andtheIFWT definedin Section3.3. Hence,the algorithm hasthe complexity h Æ ô Çregardingbothstorageandfloatingpointoperations.

To test the performanceof this algorithm, we have appliedit to a caseofproblem(9.27)with a known solution. The dataare ; i # B - , n # ó , 6 # -and í Æ ö Ç #lb þ ¾ sechÆ ö Ç , which yield thesolution k Æ ö E�ì Ç #lb þ ¾ sechÆ ö B - ì Ç . Thenumericaldataare 8 #(H ó , ~"# *�*

(making ô # - ó 6 J ), � #�D , û � # - óËó³ó, andú ì # ó � óËó * . Thewaveletsusedwerethoseof genus@ #ÚJ . Table9.6shows the

relative error, percentageof compressionachieved in

ÙY and

Ùß ÷'� , andmeasuredCPU time of û � time stepsasfunctionsof the thresholdparametersû � and û å .Therelativeerroris definedhereas(�� ) � � # ºr»�¼ � 0 � ) � )�À�À�À*) i x �� õ � ß ÷'� � � B]k Æ ö � E * Ç õºj»�¼ � 0 � ) � )�À�À�ÀÁ) i x �� õ � ß ÷� � � õIt is seenthatwaveletcompressionin thiscasecanyield aspeedupof aboutsevenwith arelativeerrorof about

K e * ó � - .


% elem.retained� ÙY �� Ùß ��÷'� CPUtime(s)ó DS� ó J 6N6 � L£K 6 L* DS� * D - GS� - ó K H- DS� 6 K * JS� ó - * ó KD 6 � ó - * GS��JND * 6 *6 6 �¢H ó * 6 ��DNH * J óG GS� - 6 * DS� 6 D -?- JH GS� L ó * DS� * J - J KK K � -?- * DS� KNK D 6 HTable 9.7: Compressionratiosof ý� and ý� ÷'� ( � � W&Y�d�d�d ) andexecutiontime in (CPUseconds)of thewaveletsplit-stepalgorithmfor differentvaluesof U . Here �� W³\�d �� -and � å WÌ\�d �1/ .

Table9.7 shows how performancedependson the transformdepth � . It isseenthat even thoughthe compressionpotentialof

Ùß ÷� increaseswith � , it isnot enoughto balancethe increasedwork inherentin the matrix vector multi-plications. This shouldbe comparedto Table8.3 which indicatesthat the workgrowswith � unless

Ùß ÷� is compressedexcessively. Thegainfrom compressionis outweighedby thecostof thewavelet transformin Algorithm 9.4. andthatofcomputingAlgorithm 8.4.

To illustratearealisticapplication,weshow in Figure9.4acomputed“breathersolution” obtainedfrom the data ; i # B - , n # ó , 6 # -

, í Æ ö Ç # -sechÆ ö Ç ,8 # H ó , ô # - ó 6 J , � # D , ú ì®# ó � óËó * and @ # J . Thefiguregivesonly the

middlethird of the ö -interval.


0

1

2

−100

10

0

5

10

15

x

t

|u|2

Figure 9.4: A “breathersolution” of problem(9.27).

9.4Burgers’equation 185

9.3.4 FFT split-stepmethod

To assesstheperformanceof thewavelet split-stepmethodwe compareit to animplementationof the FFT split-stepmethodwhich is the de factostandardforsolvingthenonlinearSchrodingerequation[New92].

The FFT split-stepmethodexploits the fact that the Fourier differentiationmatrix � �_� �� , givenin AppendixD, is diagonal.Hence,I � # B��- ; i � � i��8 i B n - �is diagonaland

jY #PO�¼1Q Æ ú ì- I � Ç (9.33)

is alsodiagonal.TheFFTsplit-stepalgorithmis thereforegivenby Algorithm 9.2but with step

-replacedby (9.33)andlinstepÆ�a Ç (usedin steps6 and H ) redefined

as

Algorithm 9.5: �� linstep�C��ja � dm� ajß � jY jaß � dW�T�� jßThe product jY ja is computedin operationsso the complexity for large

is dominatedby thatof theFFT andhenceis ¡£¢� ]¤ ¥�¦1§� ©¨ . This shouldbecom-paredto thewaveletsplit-stepmethodwheretheFWT aswell asthematrix-vectorproducthaslinearcomplexity, thelatter, however, involving a largeconstant.

ReplacingAlgorithm 9.4 by Algorithm 9.5 rendersAlgorithm 9.2 the FFTsplit-stepmethod.Theperformanceof theFFT split-stepmethodwith thesameparametersasin Table9.6 yields anexecutiontime of

*�ªs with a relative error

ofªZ«¬ª&¯®]*�° � 2 . The FFT routine usedfor this purposewas a library routine

optimizedfor vectorizationon theCRAY C92A. In contrast,the implementationof Algorithm 8.4did notvectorizewell dueto shortvectorlengthsandinefficientmemoryaccess.Table9.8 shows how the executiontime grows with for theFFT split-stepmethodandthewaveletsplit-stepmethod.In thewaveletcasetheparametersare ±l²]³ , ´W²

, µ'¶l²¸· ° �T�� , and µ�¹ asgivenin thetable.

9.4 Burgers’ equation

We considernow Burgers’ equationwhich is frequentlyusedto studyadaptivemethodsbecauseit modelsformationof shocks.Theperiodicinitial-valueprob-


CPUtime(s) FFT µ ¹ ²¸· ° ��º µ ¹ ²c· ° �1»· ª ³ · &¼ &ªª&¼�½ ª ¾Z¿ &À¼ · ª ¾ ½'¾ ¼&¼· °&ª¾ ½ · ° · ³ ª�°'¾ ³ ·&· · ¿ · ¾ ·¾,°&À�½ ª · ª�À&¼ ª&�½³1· À�ª ¾�ª ¼�'¾ ¾��½· ½& ³ ¾ ³ ¼ · °'¾� ³ ½�¼Table9.8: Executiontime(CPUseconds)for theFFTsplit-stepmethodandthewaveletsplit-stepmethodboth appliedto (9.27). The latter methodis shown for two differentcompressionthresholds.

lemfor aparticularform of Burgers’equationisÁTÂ ² Ã ÁTÄCÄ�Å ¢ ÁiÆ¯Ç ¨ ÁTÄ�ÈeÉËÊ °Á ¢�Ì È ° ¨}² ÍÎ¢�Ì�¨Á ¢�Ì ÈÏÉ ¨Ð² Á ¢�Ì Æ · ÈuÉ ¨ È ÉËÑ °eÒÔÓÕ Ì×Ö~Ø (9.34)

where Ã is a positive constant,Ç Ö]Ø and Í$¢�Ì�¨�²ÙÍÎ¢�Ì Æ ·'¨ . Burgers’equationwith Ç ² °

describesthe evolution of Á undernonlinearadvection and lineardissipation.A nonzerovalueof Ç addslinearadvectionto thesystem.

This problemis discretizedin the samemanneras(9.42),andwe obtainthesystem ÚÚ É�Û ¢ É ¨}² Ü Û ¢ É ¨ ÆKÝ ¢ Û ¢ É ¨Ï¨ Û ¢ É ¨ È ÉÞÑ °Û ¢ ° ¨�² ßMàâáãÍ$¢�Ìåä�¨ È Í$¢�Ì � ¨ È «�«�« È Í$¢�Ì � �T� ¨�æ�çwhere Ü ² ÃTè:é §�ê Å¯Ç1ëÝ ¢ Û ¢ É ¨u¨}² Å diag¢�è Û ¢ É ¨u¨with è ²]è é � ê . Thishasthesameform as(9.28)andanappropriatemodificationof Algorithm 9.2canbeusedto solve it.

The matrix-vectorproduct Û é � ê ²�è Û canbe computedanalogouslyto thecomputationof the matrix-vectormultiplicationsin Algorithm 9.4. Hencewehave

9.4Burgers’equation 187

Algorithm 9.6: ì£íNîuï�ð ñ�ìòÛ ó ô{Û¢ òÛ ¨�õ�ö ó trunc¢ òÛ È µ ¹ ¨òÛ é � ê ó ¢ òè:¨ õ�÷ ¢ òÛ ¨ õ öÛ é � ê ó ô ç òÛ é � êwhere

òè ² ô è ô ç and ¢ òèø¨ õ ÷ ² trunc¢ òè È µùú¨Unlike û , whichis dense,thematrix è is abandedmatrix(seeSection7.3)so

òèwill havethe“finger band”structurewithoutapplyingany compression.However,thewavelet transformintroducesroundingerrorswhich we remove by choosingµù to bea fixedsmallprecision,namelyµùM²¸· ° �T�7ü , asthis is sufficient to retainthesparsityof

òè . HenceweuseAlgorithms8.1and8.4for computingòè andthe

product ¢ òè:¨ õ ÷ ¢ òÛ ¨ õ�ö , respectively.To testthis procedureon (9.34)we have chosenthevaluesÃ©² °Z«¬°&°&¼

, Ç ² °,

and ÍÎ¢�Ì�¨W²}ý�þ ÿ$¢ ª�� Ì�¨ . The numericaldataare� ²^· ° , ´c²

, µ¶ ²^· ° �T�� ,µ�¹G² · ° �T� ä , � É ² °Z«¬°&°&¼, � � ² · °&° , and ± ²â³ . The implementationwasdone

in Fortran90on a200MHz PCusingtheLinux operatingsystem.Figure9.5 shows the evolution of Á ¢�Ì ÈÏÉ ¨ for

°�� É�� °1«p¼. It is seenthat a

largegradientformsat Ì¯² °1«p¼whereasÁ is very smoothelsewhere.Hencewe

expecta largewaveletcompressionpotential.Table9.9 shows the influenceon performanceof the parameterµ�¹ with µ¶

(and µù ) fixed. It is seenthat theexecutiontime dropsasfewer elementsarere-tainedin

òÛ�� . However, weobserve thatthesplit-stepmethodceasesto convergewhenthecompressionerrorbecomestoo large.

Table9.10shows the influenceon performanceof the transformdepth ´ forfixedvaluesof µ�¹ and µ'¶ . It is seenthat theexecutiontime is reducedby com-pressionas ´ grows from

°to¾. For ´ Ê ¾

theexecutiontime increasesbecausethecompressionisnotenoughto balancetheincreasedwork inherentin thematrixvectormultiplications.

However, astheproblemsizegrows, sodoesthecompressionpotential. Ta-ble 9.11 shows the bestexecutiontimes and compressionrationsobtainedfordifferentvaluesof . It is seenthat by adjustingthe parameters , µ ¹ and µ ¶onecanobtainvery largecompressionratiosanda CPUtime which grows evenslower than . Consequently, we seethat for this particularproblem,waveletcompressionleadsto a feasiblesolutionmethodprovidedthat theparameters ,± , ´ , µ�¹ , and µ¶ arechosenappropriately.


0

0.25

0.5

00.5

1

−1

0

1

t

x

u

Figure 9.5: Solutionof (9.34)shown for �� . � �!��" , #$�%� ,&('*) ��+(��,.-0/ ' "�1 ) + .9.5 WaveletOptimized Finite Differ encemethod

In this sectionwe will outlinea completelydifferentapproachto solvingpartialdifferentialequationswith the aid of wavelets. This methodis due to LelandJameson[Jam94,Jam96]andis calledthe Wavelet OptimizedFinite Dif ferenceMethod(WOFD). It worksby usingwaveletsto generateanirregulargrid whichis thenexploited for thefinite differencemethod.Hencethis methodbelongstoClass

asdescribedin Section7.1.

9.5WaveletOptimizedFiniteDif ferencemethod 189

% elem.retained CPUtime(s)µ�¹ Rel erròû õ32 òè õù òÛ õ�ö�� Total WT Mult· ° �T�7ü 1« ³ ® · ° �1» · ¼1«p½& ªZ«¬½&° À�ÀZ«¬ª&ª ·&· ¼ ·�³ À ·· ° �T� § ¾ « ³ ® · ° �1» · ¼1«p½& ªZ«¬½&° À�ªZ«¬½ ³ · ° · ·�³ ¿�¿· ° �T�� ª1« ¿ ® · ° �1» · ¼1«p½& ªZ«¬½&° ¿ Z«¬½& ³ À ·�³ ½�½· ° �T� ä ¾ «pªW® · ° ��4 · ¼1«p½& ªZ«¬½&° ¿ · « ³ ³ ¼ ·�³ ¼�¿· ° �,º ¾ «p½W® · ° �65 · ¼1«p½& ªZ«¬½&° ½�¼Z« ³ ª ¿ · ·�³ ¼�°· ° �87 No convergence

Table 9.9: Effect of the parameter9¹ on performance.The remainingparametersare: �<; , 9 ¶ � �� T� § , = �<"��?>A@ , B �C@ , DE F�C�� , and G � � �� . Secondcolumnshows therelative errorat GH�%G � in the infinity normcomparedto a referencesolutioncomputedusingthe FFT split-stepmethod.Columns ; –� show the percentageof elementsretainedin IJ õK2 , IL õ�÷ , and IM õ�ö�� , respectively. ColumnsN –@ show the totalexecutiontime, thetime spentin thewavelettransforms(FWT andIFWT), andthe timespentin computingtheproductsusingAlgorithm 8.4.

9.5.1 Finite differ enceson an irr egular grid

We begin by defininga finite differencemethodfor anirregulargrid. Let Á beatwicedifferentiable· -periodicfunctionandlet therebeasetof grid points° ² Ì ä � Ì � � Ì § ��O?O?OP� Ì � � � � ·whicharenotnecessarilyequidistant.A wayto approximatederivativesof Á is toconstructaLagrangianinterpolatingpolynomialthroughQ pointsanddifferentiateit. We consideronly odd Q Ñ

becauseit makesthe algorithmsimpler. LetR ²c¢SQ Å ·'¨�T ª anddefine

ÁPU ¢�Ì�¨�² V0W6XYZ�[\V � X Á ¢�Ì Z ¨^] X`_ VS_ Z ¢�Ì�¨] XP_ VS_ Z ¢�Ì Z ¨ (9.35)

where,

] XP_ VS_ Z ¢�Ì�¨�² V0W\Xabc[\V � Xbed[fZ ¢�Ì Å Ì b ¨ (9.36)

Becauseof periodicity, all indicesare assumedto be computedmodulo . Itfollows that ÁPU ¢�Ì V ¨Þ² Á ¢�Ì V ¨ for gú² ° È · È «�«�« È Å · ; i.e. ÁPU interpolatesÁ at thegrid points.


% elem.retained CPUtime(s)´ Rel erròû õ32 òè õ�÷ òÛ õ�ö�� Total WT Mult° · « ³ ® · ° �1» · ÀZ« · À °1«p½& À�ÀZ«¬À&¼ ³ À ° ³ · · « ³ ® · ° �65 · ¼Z«¬¼& · «pª0¿ À¾T« ³ ¿ ½'¾ · ° ¾ ³ª · «¬ª ® · ° �65 · ¿�«¬ ³ · «pÀ& ¿ · «¬À&ª ¿�¾ · ½ ¼& ¾T«¬½ ® · ° �65 · ¼Z«¬½& ª1«p½&° ½�¼Z« ³ ª ¿�¾ ·�³ ¼&°¾ ¼Z«¬½ ® · ° �65 · ¼Z«¬ª& 1«pª ³ ½�ªZ«¬°&½ ¿¼ ·�³ ¼ ·¼ ¼Z«¬½ ® · ° �65 · ¼Z«¬¼'¾ 1«pÀ&½ ½�°Z«y¾�° ³Z· · À ¼&¼½ ¿�«y¾×® · ° �65 · ½Z«¬°&À ¾ «p½'¾ ¼ ³ « ³ ¾ ³ ¼ · À ¼&À¿ Z«¬½ ® · ° �65 · ½Z«\¿ · ¼1«p&ª ¼�Z«\¿½ ³&³ · À ½&ª³ ªZ«¬ª ® · ° ��h · ¿�«¬0¿ ½1«p°&° ¼ ³ « ³ À ³&³ · À ¿¼

Table 9.10: 9 ¹i�j�� º , 9�¶k�l�m� �T� § , =C�n"��?>A@ , Bo�p@ , DE q�r�� , G � �j�� .Columns" –@ have thesamemeaningsasthosein Table9.9.

Differentiationof (9.35)

Útimesyields

Á é0s êU ¢�Ì�¨�² V0W\XYZ�[\V � X Á ¢�Ì Z ¨ ] é0s êXP_ VS_ Z ¢�Ì�¨] XP_ VS_ Z ¢�Ì Z ¨ (9.37)

ReplacingÌ by Ì V in (9.37)yieldsa Q -point differenceapproximationfor Á éts ê ¢�Ì�¨centeredat Ì V . Let Û ²lá Á ¢�Ì ä ¨ ÈCÁ ¢�Ì � ¨ È «�«�« ÈCÁ ¢�Ì � �T� ¨�æ . Thederivatives Á é0s ê ¢�Ì ¨ canthenbeapproximatedatall of thegrid pointsby

Û é0s ê ²]è:éts êu Û (9.38)

wherethedifferentiationmatrix è:é0s êu is definedby

áãè é0s êu æ Ve_ Z ² ] éts êXP_ VS_ Z ¢�Ì V ¨] X`_ Ve_ Z ¢�Ì Z ¨ È R ²¸¢eQ Å ·'¨�T ª (9.39)

Regardingthefirst andsecondorderderivativeswe find that

] é � êXP_ VS_ Z ¢�Ì�¨ ²ÚÚ Ìv] XP_ VS_ Z ¢�Ì�¨ ² V0W\XYb�[\V � Xbed[fZ

VtW\Xaw [\V � Xw d[xZy_ b ¢�Ì Å Ì w ¨ (9.40)


% elem.retained CPUtime(s) µ�¹ µ¶ ´ Relerròû õ32 òè õ�÷ òÛ õ�ö�� FWT FFT· °&ª'¾ · ° �T� ä · ° �T�� ¼ ® · ° �65 · ½ ¼1«pª&° ³ ª �½ 0¿ª�°'¾ ³ · ° ��º · ° �T�� ¼ ® · ° �65 · ª1«p½&° ½�¼ ½�¼ ¿ ³¾,°&À&½ · ° ��7 · ° �T� ä ¼ ® · ° �65 À · «p&° ¼¾ · ª�¿ · ¿½³1· À&ª · ° ��h · ° ��7 ¼ ¾×® · ° ��4 ¼ °1«pÀ&À ¾ · ¿ ¾,°'¾· ½� ³ ¾ · ° ��h · ° �65 ½ ³ ® · ° �65 °1«p¼ ³ ª &�ª ³ À&°

Table9.11: Bestexecutiontimesfor thewaveletsplit-stepmethodfor differentproblemsizes.The fixedparametersare Bz�n@ , DE ^�r�� , and G � �{�� . Column � showsthe problemsize. Columns " –> show the parametersthat werechosento obtaingoodperformancefor each = . Columns � –@ have the samemeaningsas the correspondingcolumnsin Table9.9. Column |�� shows theexecutiontime usingthewaveletsplit-stepmethodwith therespectiveparametersandcolumn �� showstheexecutiontimeusingtheFFTsplit-stepmethod.

and

] é §�êXP_ VS_ Z ¢�Ì�¨ ²Ú §Ú Ì § ] XP_ VS_ Z ¢�Ì�¨ ² VtW\XYb�[\V � Xbed[fZ VtW\XYw [\V � Xw d[xZy_ b

V0W6Xa [6V � X d[fZy_ b�_ w ¢�Ì Å Ì ¨ (9.41)

It will berecalledthatfor equidistantgridswith steplength Í theerroris describedby }}} Á é0s êU ¢�Ì V ¨ ÅMÁ éts ê ¢�Ì V ¨ }}} ²]¡W¢�Í u � � ¨ È Ú ²¸· È ªprovidedthat Á is sufficiently smooth.

9.5.2 The nonlinear Schrodinger equationrevisited

Consideragainthe periodic initial-valueproblemfor the nonlinearSchrodingerequation Á Â ² ~ Á Æ g�� Á � § Á$ÈÏÉúÊ °Á ¢�Ì È ° ¨}² Í$¢�Ì ¨Á ¢�Ì ÈÏÉ ¨ ² Á ¢�Ì Æi��ÈÏÉ ¨ È×ÉÞÑ ° ÒÔÓÕ ÌRÖ~Ø (9.42)

where ~P² Å gª`� §�� §� Ì § Ån� ª


Proceedingasin Section9.3, but now usingthe differentiationmatricesde-finedin (9.39)weobtainÚÚ É Û ¢ É ¨}² Ü Û ¢ É ¨ ÆKÝ ¢ Û ¢ É ¨Ï¨ Û ¢ É ¨ È ÉÞÑ °Û ¢ ° ¨�² ßMàâáãÍ$¢�Ì ä ¨ È Í$¢�Ì � ¨ È «�«�« È Í$¢�Ì � �T� ¨�æ�ç (9.43)

where

Ü ² Å gª � § è é §�êu� § Å � ª ëÝ ¢ Û ¢ É ¨Ï¨}² g�� diag¢�� Á b ¢ É ¨�� § ÈP� ² ° È · È «�«�« È Å ·¨The split-stepmethodcould now be usedfor time stepping. However, this

involvestruncationof matricesandsincewe no longerrepresentthe solutioninthewaveletdomainwedonotseekadaptivity throughthesemeans.Insteadweusea standardfinite-differenceapproachmethodsuchastheCrank-Nicolsonmethod[Smi85, p. 19]. Then(9.43)is approximatedbyÛ ¢ É Æ � É ¨ Å Û ¢ É ¨� É ² Ü Û ¢ É$Æ � É ¨ Æ Û ¢ É ¨ª ÆÝ � Û ¢ É Æ � É ¨ Æ Û ¢ É ¨ª � Û ¢ É Æ � É ¨ Æ Û ¢ É ¨ªandwe obtainthetime-steppingprocedure� Û� W � ² � Û� Æ (9.44)� ÉuÝ �úÛ� W � Æ Û�ª � Û� W � Æ Û�ª È �©² ° È · È «�«�« È � � Å ·Û ä ² ßwhere

� ² ëWÅn� Â§ Ü , �}² ë Æn� Â§ Ü , and Û� ² Û ¢K�� É ¨ .As with thesplit-stepmethod,aniterativeprocedureis required.Thecompu-

tationof Û� W � canbehandledby thestepsÛ é ä ê W � ² Û anditerateuntil convergence� Û é0� W � ê W � ² � Û� Æ (9.45)� É Ý�� Û ét� ê W � Æ Û�ª � Û é0� ê W � Æ Û�ª È � ² ° È · È «�«�«We have solvedthesystem(9.45)in Matlabby LU-factorizationof

�andsubse-

quentforwardandbackwardsubstitutionsin eachiterationstep.


9.5.3 Grid generationusingwavelets

Theelementsof Û ¢ É ¨ approximatethefunctionvaluesÁ ¢�Ì Z ÈÏÉ ¨ È�� ² ° È · È «�«�« È Å· . Thesuccessof anadaptivemethodreliesonaprocedurefor determiningagridwhich is densewhereÁ is erraticandsparsewhereÁ is smooth.

Recallfrom Theorem2.5thatthewaveletcoefficientsof Á satisfytheinequal-ity � Ú�� _ Z � � ·��q� � ª � � é � W �� êf�F�� U¢¡¤£ ¥ }} Á é � ê ¢K¦&¨ }}where ] ² ± T ª . Notethatwe have takentheinterval length � into account.

Theerrorat � T ª � dependson thesizeof theneighboringintervals,anda largevalueof � Ú�� _ Z � is an indicationthat thegrid spacing·�T ª � is too coarseto resolveÁ properlyin the interval § � _ Z . Hencewhena largevalueof � Ú�� _ Z � arises,we addpointswith spacing ·�T ª � W � aboutposition � T ª � to reducethe error locally. In[Jam94] it is suggestedthatonly afew pointsbeaddedat location � T ª � . However,in thelight of Theorem2.5wehave foundit reasonableto distributepointsevenlyover the entireinterval § � _ Z becausethe large gradientcanbe locatedanywherewithin thesupportof thecorrespondingwavelet.

This canbe realizedin practiceby introducingan equidistantgrid at somefinest level

�. If the solution vector Û is definedfor a coarsergrid, it is then

interpolatedto valuesonthefinegrid. Thenthevectorof waveletcoefficients ¨ iscomputedas ¨ ² ôC©mÛTheWOFD grid is generatedby choosingthosegrid pointswhich correspondtolargewaveletcoefficientsasdescribedabove. After thegrid isgenerated,thefinitedifferenceequationsare constructedaccordingto (9.44) anda numberof timestepsaretaken.Hence,we proposethefollowing algorithmfor grid generation:

Algorithm 9.7: Wavelet-basedgrid generation

Interpolate Û from current coarse grid to fine equidistant grid at scale�

.

Construct new coarse grid from this expanded Û :Initialize new grid (delete old points)Compute wavelet coefficients

Ú�� _ Z of Á ( Ûiª<«�ª ¨ ).Insert grid points where � Ú�� _ Z � Ê µ .Construct matrices

�and � and factorize

�.

Perform several time steps on new grid.


How many time stepsto takebetweengrid evaluationsdependson theprob-lem at hand: A rapidly changingsolutionwill requirefrequentgrid evaluationswhereasa slowly changingsolutioncanusethe samegrid for many time steps.We have had good resultswith the following heuristic: If the numberof gridpointsincreasesmorethana given thresholdfrom onegrid to thenext, thenthepreviousseriesof timestepsis repeatedon thenew grid.

Theinitializationof Algorithm 9.7 isÁ b ²]Í$¢�Ì b ¨ È � ² ° È · È «�«�« È ª�¬ Å ·9.5.4 Results

Figure9.6 shows the computedsolutionof (9.42) at É ² °1«p¼togetherwith the

grid generatedby theWOFDmethod.Thesolutioncorrespondsto theparameters�( ² Å ª , � ² ª, � ² °

, � ² ½¾, and Í¯®�Ì(°Þ² ª

sech®�Ì(° . Thenumericaldataare� ²¸· ° (making ±�²¸· °�ª'¾ ), ´R² � , µi²¸· °�² h , QW² ¼, � É ²¸·�T ª�³ ä , � ³ ² ¼ · ª and

thewaveletsusedarethoseof genus±l²]³ . It is seenthattheWOFDmethodhas

−40 −30 −20 −10 0 10 20 30 40−2

0

2

4

6

8

10

12NLS WOFD (t = 0.5)

x

|u|2

Solution WOFD grid

Figure9.6: A WOFDsolutionof ( 9.42).

generateda grid which capturesthebehavior of thepulse,thegrid densitybeinglow wherethesolutionis smoothandhighwhereit is not.


Mflops CPU(s) ´ õ ® Û ° pointsWOFD®Nµi²¸· °�² h ° · °&ª À�¿ µ · °�² h µ · &°WOFD®Nµi² ° ° ¿ °&° · ¾Z¿ ° · °�ª'¾FFTsplit-step

½ · À�½ Å · °�ª'¾Table 9.12: Theperformanceof theWOFD methodcomparedto thatof theFFT split-step. The test problemis (9.42) and the numericaldataare B �<@ , : �C¶r� �� ,�E·H ¸¹�� , º��»� , DE ¼�½��¾�" ³ ä and 9q�½�� ² h .Let Û õ betheresultof Algorithm 9.7using µ asthresholdanddefinethecompres-sionerror ´�õ�²l¿¿ Û ä Å Û õA¿¿�Àwherethevector Û ä ( µi² °

) correspondsto thesolutionobtainedusingfinite dif-ferenceson thefinestgrid. For comparison,we have alsoimplementedtheFFT 1

split-stepmethoddefinedin Section9.3.4in Matlab. Table9.12shows thenum-berof Mflops (measuredin Matlab)neededfor eachmethod,thecorrespondingexecutiontime,andthecompressionerror ´ õ whereapplicable.It is seenthattheflop countis reducedsignificantlyby theWOFDmethodbut theCPUtimeis lessso. The latter behavior is probablydueto inefficient memoryreferencesin theupdatingof thefinite differencematrices.However, thisexperimentsuggeststhattheWOFDmethodis potentiallyveryefficient for thisproblem.

TheMatlab functionsnlswofd andnlsfft demonstratetheWOFD method ✤andtheFFTsplit-stepmethod,respectively.

9.5.5 Burgers’ equationrevisited

We now illustrate the adaptivity of WOFD by consideringa problemwhereagradientcangrow andmove with time. TheproblemconcernsBurgers’equationasgivenin (9.34).

To testAlgorithm 9.7on (9.34)we have chosenthevaluesÃ×² °Z«¬°&°&ª, Ç ² °

,Í¼®�Ì(°m²Ðý�þ ÿ�® ª�� Ì(° . The numericaldataare� ²�·�· , ´P² �

, µ¯²�· ° ² 4 , Qø² ,� É ² ·�T ª º , � ³ ² ª&¼�½

and ±{² ½. Figure9.7 show theevolution of Á together

1The FFT in Matlab doesnot precomputethe exponentials.Henceit is not asefficient asalibrary FFT.


0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1Grid modifications

Pts added

Pts removed

0 0.2 0.4 0.6 0.8 1−1

−0.5

0

0.5

1Burgers’ equation (t = 0.0625)

x

u

Solution

WOFD grid

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1Grid modifications

Pts added

Pts removed

0 0.2 0.4 0.6 0.8 1−1

−0.5

0

0.5


x

u

Solution

WOFD grid

0 0.2 0.4 0.6 0.8 10

0.2

0.4

0.6

0.8

1Grid modifications

Pts added

Pts removed

0 0.2 0.4 0.6 0.8 1−1

−0.5

0

0.5


x

u

Solution

WOFD grid

Figure 9.7: Solutionof (9.34) with �»�Á��" and #��Á� shown togetherwith theWOFD grid at times ��%��N�"��Â"��.��Â>�;�Ã�� . Thenumericaldataare ¶¹�Ä�� , : �!¶ ,9$�{�� ² 4 , ºF�Ä; , DE ^�{��¾�" º , and Bl�ÄN . The lower graphsindicatepointsthat havebeenadded(+) andpointsthathave beenremoved( Å ) at time .with the WOFD grid for differentvaluesof t. It is seenthat pointsconcentratearoundthelargegradientasit formsandthatthegrid becomesverysparsewherethe solution is smooth. Table 9.13 shows the numberof Mflops (measuredinMatlab)usedfor µi²�Æ and µi²c·?Æ ² 4 .

Finally, Figure 9.9 shows the solution to (9.34) at É ² Æ\Ç ¼ with the sameparametersasbeforeexcept that we have now introducedlinear advectioncor-respondingto the parameterÇ ²ÈÆ\ÇÉÆ�Ê . It is seenthat the WOFD methodhascapturedtheshockalsoin thiscase.

Mflops CPU(s) ´ õ ® Û ° pointsWOFD®Nµ�²¸·?Æ ² 4 ° ·?Ê ·AÇ ¿ µ ·?Æ ² h µ Ê�ÆAÆWOFD®Nµ�²�Æ�° ¾�½ Ê6Ç À Æ Ê�Æ ¾ ³

Table 9.13: The performanceof the WOFD method. The testproblemis (9.34)with�Ë�n��c��" and #F�p� andthe numericaldataare Bz�rN , : �p¶»�{�� , �F·Ì �¸�� ,º��Í; , DE ¼�½��¾�" º , and 9v�Î�� ² 4 .


0 0.2 0.4 0.6 0.8 1−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8Burgers equation WOFD (t = 0.5)

x

u

Solution WOFD grid

Figure 9.8: Solutionof (9.34) with ��Á��" and #Ï�Á� shown togetherwith theWOFD grid at Ð�j�� . The numericaldataare ¶½�Ñ�� , : �j¶ , 9Ò�k�m� ² 4 , º��l; ,DE ��½�m¾�" º , and B%�ÍN .TheMatlabprogramburgerwofd demonstratestheWOFDmethodfor burgers ✤equation.

Thewaveletoptimizedfinite differencemethod(WOFD) is a promisingwayof solving partial differentialequationswith the aid of wavelets. However, themethodinvolvesmany heuristicsandmuchremainsto beinvestigated:Ó Wehavefoundthatthemethodis sensitiveto thethechoiceof pointsplaced

at location � TÔÊ � . Theoptimalstrategy still remainsto befound.Ó The reconstructionon the finest grid is presentlydonewith cubic splineinterpolations.This is not necessarilythebestapproach,andotherinterpo-lationschemesshouldbeconsidered.Alternatively, if thewavelettransformcouldbecomputeddirectly from functionvalueson the irregulargrid, the


0 0.2 0.4 0.6 0.8 1−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8Burgers equation WOFD (t = 0.5)

x

u

Solution WOFD grid

Figure 9.9: Solutionof (9.34)with �H�{��" and #Ë�{��" shown togetherwith theWOFD grid at ��l�� . The numericaldataare ¶½�Á�� , : �j¶ , 9Ò�Á�� ² 4 , ºÕ�o; ,DÖ ×�!�m¾�" º , and B{�½N . Thegrid pointsclusteraroundgradientwhich hasmovedfrom) �»�� to )�Ø ��N .

reconstructioncouldbeavoidedaltogether. Researchtowardsthis endis inprogress[Swe96].Ó As mentionedearlier, the numberof time stepsbetweenconsecutive gridevaluationsshouldbedeterminedadaptively basedon thespeedwith whichthesolutionevolves.Also thispointneedsfurtherinvestigation.

Chapter 10

Conclusion

In part · weexposedthetheoryfor compactlysupportedorthogonalwaveletsandtheir periodizedformsandgave estimatesof their approximationcharacteristics.Furthermore,we demonstratedthatthefastwavelettransformis a viablealterna-tive to the fastFourier transformwhenever oneencounterstransientphenomenain functionsor, asGilbert Strang[SN96] putsit, waveletsare bestfor piecewisesmoothfunctions.

In part2 we showedhow onecanimplementthe fastwavelet transformeffi-cientlyon a vectorcomputeraswell ason parallelarchitectures.Thereasonwhythewavelet transformlendsitself well to parallelizationis a consequenceof thelocality of wavelets.Hence,little communicationis required.We observedverygoodactualperformanceontheFujitsuVPP300aswell astheIBM PS2andit wasshown thatourparallelalgorithmis optimalin thesensethatthescaledefficiencyis independentof thenumberof processorsandit approachesoneastheproblemsizeis increased.

In part 3 we developedseveral wavelet-basedalgorithmsfor solving partialdifferentialequations.We derived wavelet differentiationmatricesandshowedhow onecancomputethe wavelet transformof a circulant ± Ù»± matrix

�inÚ ®Û±Ò° stepsusingstorageboundedby Ê0´P± elements.Further, it wasshown that

if�

is alsobanded,which is thecasefor a differentiationmatrix, thentheworkneededfor computinga matrix-vectorproductin the wavelet domainis

Ú ®Û±Ë° .Moreover, it wasshown that the complexity grows with the transformdepth ´ .However, thewaveletcompressionpotentialgrowswith ´ too. Consequently, thefeasibilityof performingsucha multiplicationdependson whetherthegainfromwaveletcompressionoutweighstheincreasedcostof doingthemultiplicationinthewaveletdomain.This is thecentralquestionfor wavelet-basedsolvers.

200 Conclusion

Examplesof thenumericalsolutionof partialdifferentialequationsweregiven.In particular, we developeda waveletsplit-stepmethodfor the nonlinearSchro-dingerequationaswell asBurgers’equation.

It wasobservedthattheproblemsizeaswell asthecompressionpotentialmustbevery largefor thismethodto becompetitivewith traditionalmethods.Whereasit is straightforward to obtain thewaveletexpansionof a knownfunction,it canbeexceedinglydifficult to efficientlyobtain thewaveletcoefficientsof a solutionto a partial differentialequation.Similarconclusionswerereachedby [Wal96,p.III-29, V-1], [FS97, Jam94,VP96, CP96,PW96].

Finally, we outlineda differentapproachcalledthe wavelet optimizedfinitedifferencemethod.This methodusesthewavelet transformasa meansfor gen-eratinganadaptive grid for a traditionalfinite differencemethod,andthe resultsfor thenonlinearSchrodingerequationandBurgers’equationarepromising.Thepertinentquestionis not whetheranadaptive grid is betterthana fixedgrid, butwhetherit canbe generatedbetterandmorecheaplythroughothermeansthanwavelets. However, this exampleindicatesthat waveletscanplay the role asapowerful tool for dynamicanalysisof thesolutionasit evolves.

In bothcaseswe observed that theproblemsizehasto bevery largeandthesolutionmustbeextremelysparsein a waveletbasisbeforewavelet-basedmeth-odsfor solvingpartialdifferentialequationshaveachanceto outperformclassicalmethods– andeven then,the advantagerelieson appropriatechoicesof severalcritical parameters.

We concludeby pointingout whatwe seeasthemainobstaclesfor obtainingtruly feasibleandcompetitive wavelet-basedsolversfor partialdifferentialequa-tions. Part of the problemis the fact that thereis alwaysa finest level or gridinherentall waveletapproaches.Futureresearchcouldbedirectedatmethodsforavoiding the computationsin thespaceof scalingfunctionor on thefinestgrid.This would for examplerequirea wavelet transformof datathatarenot equallyspaced.

Anotherintrinsicproblemis thatcompressionerrorstendto accumulatewhenappliedto aniterativeprocess.Thisputsa severerestrictionon theamountof ad-missiblecompression.An errorwhich is acceptablefor a compressedsignalmaybedetrimentalfor a solutionto a partialdifferentialequationwhenaccumulated.Also, whenerrorsareintroduced,thesolutionmaybecomelesssmoothwhich inturnmeansthatthesubsequentwaveletcompressionpotentialdrops.

Finally, many algebraicoperationsin thewaveletdomaintendto requirecom-plicateddatastructures[BK97, Hol97]. Suchdatastructuresmayinvolvepointersandother typesof indirect addressingaswell asshortvectors. For this reasonimplementationsareunlikely to performwell on computerarchitecturessuchasvectorprocessors.

201

Much researchin this field is on its way andit is too early to saywhethertheproblemsmentionedabovewill besolved.However, thereisnodoubtthatwaveletanalysishasearnedits placeasanimportantalternativeto Fourieranalysis— onlythescopeof applicabilityremainsto besettled.

Appendix A

Momentsof scalingfunctions

Considertheproblemof computingthemomentsasgivenin (2.26):Ü ub ²!Ý À² À Ì uyÞ ®�Ì Å»� ° Ú Ì È ��È QWÖÕß (A.1)

By thenormalization(2.2)we notefirst thatÜ äb ²c· È � ÖÕß (A.2)

Let � ²�Æ . Thedilationequation(2.17)thenyieldsÜ uä ² Ý À² À Ì u Þ ®�Ìà° Ú Ì² á Ê ù ²x³Y Z�[ ä�â Z Ý À² À Ì u Þ ®3Ê&Ì Å»� ° Ú Ì² á ÊÊ u W ³ ù ²f³Y Z�[ ä â Z Ý À² À¹ã u Þ ® ã Å»� ° Ú ã È ã ²�Ê�Ìor Ü uä ² á ÊÊ u W ³ ù ²f³Y Z�[ ä×â Z Ü uZ (A.3)

To reducethe numberof unknowns in (A.3) we will eliminateÜ uZ for �jä²åÆ .

Using the variabletransformationã ² Ì Ån� in (A.1) and following a similarapproachasin thederivationonpage20yieldsÜ ub ² Ý À² À ® ã Ææ� ° u�Þ ® ã ° Ú ã² uY [ ä � Q � � � u ² Ý À² À¹ã Þ ® ã ° Ú ã

204 Momentsof scalingfunctions

or Ü ub ² uY [ ä � Q � � � u ² Ü ä (A.4)

Substituting(A.4) into (A.3) we obtainÜ uä ² á ÊÊ u W ³ ù ²x³Y Z�[ ä â Z uY [ ä � Q � � � u ² Ü ä² á ÊÊ u W ³ u ²f³Y [ ä � Q� � Ü ä ù ²f³Y Z�[ ä â Z � u ² Æ á ÊÊ u W ³ Ü uä ù ²f³Y Z�[ ä â Zç è?é êë Solvingfor

Ü �ä yieldsÜ uä ² á ÊÊ\®3Ê u Å ·�° u ²f³Y [ ä � Q � � Ü ä ù ²f³Y Z�[ ä â Z � u ² (A.5)

Equation(A.5) cannow beusedto determinethe Q th momentofÞ ®�Ì(° , Ü uä for

any Q Ê Æ . For Q¯²ìÆ use(A.2). ThetranslatedmomentsÜ ub arethenobtained

from (A.4).

Appendix B

The modulusoperator

Let �~ÖÕß then �©²»Q ��Ææí (B.1)

whereQ Èî�ZÈ�í ÖÕß . Wedenote� thequotientof � dividedby Q and í theremainderof thatdivision. The � and í arenot uniquelydeterminedfrom Q and � but givenQ È � wespeakof theuniqueequivalenceclassconsistingof all valuesof í fulfillingfulfilling (B.1) with � ÖÕß .

However, one representative of this equivalencestandsout. It is called theprincipal remainderandit is definedasí ²�� ï6ð Q×²ñ� Å QHò �Q�ówhere á O æ denotesthenearestintegertowardszero.

This is the way modulusis implementedin many programminglanguagessuchasMatlab. While mathematicallycorrect,it hasthe inherentinconveniencethat a negative í is chosenfor � � Æ . In many applicationssuchas periodicconvolution we think of í asbeingthe index of anarrayor a vector. Therefore,wewish to choosearepresentativewhereí ÖMá�Æ È Q Å ·�æ for all �~ÖÕß . Thiscanbeaccomplishedby defining í ²nô3�¯õ u ²�� Å QHö �Q�÷ (B.2)

where ø¤�¼T�Qxù denotesthenearestintegerbelow �¯TîQ . We have introducedtheno-tation ô3�¯õ u in orderto avoid confusion,andwe notethatô3�¯õ u ú � ��ï6ð Q for �Hû½Æ\ü¼QFû!ý

206 Themodulusoperator

For practicalpurposesô3þ¯õ�ÿ shouldnot be implementedasin (B.2); rather, itshouldbewrittenusingthebuilt-in modulusfunctionmodifying theresultwhen-everneeded.

TheprogramminglanguageFortran90includesbothdefinitions,soMOD(n,p)is the ordinarymodulusoperator, whereMODULO(n,p) implementsôKþ¼õ�ÿ (see[MR96, p 177]).

Definition B.1 Let þ begivenasin (B.1). If � ú�� , wesaythat � is a divisor in þandwrite ��þIt followsthatfor all ��ü��\üîþ�ü�� wehave�� Kþ�� and �� 3þ��Lemma B.1 Let þ��îü�þ��?ü�� . Then� þ�� þ!�mõ#" ú � þ��$� � þ��mõ%"mõ#"� þ��þ!�mõ#" ú � þ�� þ!�mõ#"mõ%"Proof: We write þ�� asin (B.1) andfind� þ��&�æþ��mõ%" ú � þ��$�� '�(��*)��+�,��õ#"ú � þ��$�-�+�yõ%"ú � þ��$� � þ��mõ%"yõ%"and � þ��þ��mõ%" ú � þ��. /�(��0)1�+�,��õ%"ú � þ � � � �*)»þ � � � õ "ú � þ��2�3�mõ#"ú � þ�� þ��?õ#"�õ#" 4

207

Lemma B.2 Let 5(ü�þ ü�� . Then5 � þ¯õ#" ú � 5fþ�õ#62"Proof: Wewrite þ asin (B.1). Then�7�8 Kþ��for �6ü�þ�ü��9�� . When � is adivisor in þ�� thenalso5(�7�:5� Kþ��from whichwe get 5(� ú � 5fþ�õ#62" 4

208 Themodulusoperator

Appendix C

Cir culant matricesand the DFT

We statesomepropertiesof circulantmatricesanddescribetheir relationto con-volutionandthediscreteFouriertransform(DFT).

Definition C.1 (Cir culant matrix) Let ; be an < =>< matrix and let ? ú@BADC ü A ��ü+E+E+E�ü AGFIH �KJ/L bethefirst columnof ; . Then ; is circulant if@ ;7J/MON P ú A(Q M H PSR/T ü�UHü�þ úV� ü?ý�ü+E3E+E�ü.<W�½ýA circulant X�=�X matrixhastheformYZZ[ A C AG\]A � A �A � A C AG\]A �A � A � AGC]A \A \ A � A � AGC

^+__`Sinceall columnsareshiftedversionsof ? , it sufficesto storethe < elementsof? insteadof the < � elementsof ; . Also, computationalwork canbesavedusingthecirculantstructure.

Matrix vectormultiplicationwith a circulantmatrix is closelyrelatedto dis-crete(cyclic) convolutionwhich is definedasfollows

Definition C.2 (Discreteconvolution) Let acb @Bd C3e d � e E3E+E e d FIH � J L and definef and g similarly. Then ghb�aji fis the(cyclic)convolutiondefinedbyk Mlb FIH �m P3n C d P:o Q M H PSR T e Upb � e+q:e E+E+E e <W� q

(C.1)

210 CirculantmatricesandtheDFT

ThediscreteFouriertransform(DFT) is closelyrelatedto convolutionandcircu-lantmatrices.Wedefineit asfollows:

Definition C.3 (DiscreteFourier transform) Let r dtsvu FwH �s n C be a sequenceof <complex numbers.Thesequencer$xd 6 u FIH �6yn C definedbyxd 60b FIH �m s n C dts{z H 6 sF e 5�b � e3q�e E+E3E e <W� qwhere

z$F b}|.~ �K�+� F , is thediscreteFourier transformof r dts�u FIH �s n C .

Therearea numberof variantsof theDFT in theliterature,andno singledefini-tion hasaclearadvantageover theothers.A particularvariantshouldthereforebechosensuchthatit suitsthecontext best.Thechoicewehavemadehereis mainlymotivatedby thefactthatit yieldsasimpleform of theconvolutiontheorem(The-oremC.1).

Let a�b @�d(C e d � e E+E3E e d(FIH �KJ/L and xa�b @ xd(C e xd � e E+E+E e xd�FwH �KJ/L . ThentheDFT canbewritten in matrix-vectorform as xa�b}� F a (C.2)

where @ � F J 6�N s b z H 6 sF b}| H ~ �%�36 s � F (C.3)

is the <�=�< Fouriermatrix.Theinverseof � F satisfiestherelation� H �F b q< � F (C.4)

This is seenfrom thematrixproduct@ � F J MON 6 @ � F J 6�N P b FwH �m 6yn C z 6��P H M��F b�� < U�b�þ� U��b�þConsequently, a�b}� H �F xa andwe have

Definition C.4 (InversediscreteFourier transform) Let r d�s'u FIH �s n C and r$xd 6 u FIH �6yn Cbegivenasin DefinitionC.3.ThentheinversediscreteFourier transform(IDFT)is definedas d�s b q< FIH �m 6yn C xd 6 z 6 sF e � b � e+q�e E3E+E e <W� q

(C.5)

211

BoththeDFT andtheIDFT canbecomputedin �� #<��/�� <h� stepsusingthefastFouriertransformalgorithm(FFT).

Thelink betweenDFT andconvolution is embodiedin theconvolution theo-rem, whichwe stateasfollows:

TheoremC.1 (Convolution theorem)ghb�aji f � xgjb diag xa�� xfProof: Webegin by writing thecomponentsof a and f in termsof thoseof xa andxf , i.e. d P b q< FIH �m 6yn C xd 6 z 6�PFo Q M H P3R T b q< FIH �m 6yn C xo 6 z 6 Q M H PSR TF b q< FwH �m 6yn C �xo 6 z H 6#PF � z 6�MFsince�+�� K�K :¡ � þ¯õ F0¢ <h�wbl�+�8�� £�K �¡¼þ ¢ <¤� . Consequently,k M b FIH �mP3n C�¥ q< FIH �m 6yn C xd 6 z 6�PF ) q< FwH �m s n C xo s¦z H8s PF z s MF¨§

b q< � FIH �m 62n C FIH �m s n C xd 6 xo s z s MF FIH �m P+n C z PS�©6 HDs �Fb q< FIH �m 62n C xd 6 xo 6 z 6#MF

Hence xk 60bªxd 6txo�6 for 5«b � e+q:e E+E+E e <W� qby thedefinitionof theIDFT (C.5).

4Corollary C.2 Theconvolutionoperator is commutativea�i f b f i*aProof: UsingTheoremC.1we geta�i f b>� H �F diag xaI� xf b}� H �F diag xf � xa�b f i�a 4


Matrix multiplicationwith acirculantmatrix is equivalentto a convolution:

Lemma C.3 Let ; and ? bedefinedasin DefinitionC.1and al��¬ F then;a�b®?hi*aProof: @ ;a&J'M b FIH �m P+n C @ ;7J/MON P d P b FwH �m P3n C A(Q M H P3R T d Pb @ ahi*?$J M b @ ?¯i�a&J M e U°b � e3q�e E+E3E e <W� q 4We now usetheconvolution theoremto obtainanalternative andusefulcharac-terizationof circulantmatrices.

TheoremC.4; is circulant � ;°b�� H �F�±² � F³e ±´² b diag x?µ�Proof:¶ : Let ; bean <·=�< circulantmatrixandlet a and f bearbitraryvectorsof

length < . Then,by LemmaC.3andTheoremC.1f b ;�ab ?¯i*ab � H �F diag x?&� xab � H �F diag x?&�y� F athefactorizationfollows.¸ : Let ;·b¹� H �F ± ² � F andlet º(P7b¼»�½¾N P bethe þ ’ th unit vectorof length < .Theelement

@ ;�J'M¿N P canthenbeextractedasfollows@ ;´J/MON P b @ � H �F�± ² � F J'M¿N Pb q< º LM � F L ± ² � F º�Pb q< FwH �m 6yn C z 6�MF xA 6 z H 6�PFb q< FwH �m 6yn C xA 6 z 6��{M H P3�Fb A�Q M H PÀR T

Hence; is circulantaccordingto DefinitionC.1. 4

213

TheoremC.5 Circulantmatriceswith thesamedimensionscommute.

Proof: Let ; and Á be circulant <]=�< matricesaccordingto Definition C.1.Thenwe have from TheoremC.4;ÁÂb>� H �F�± ² � F � H �F1±Ã � F b}� H �F�± ² ±Ã � FSince

± ²and

±�Ãarediagonal,they commuteandwefind;«ÁÂb}� H �F�±Ã#± ² � F b>� H �F-±Ã � F � H �F�± ² � F b�Á�; 4

TheoremC.6 ; is circulant � ; H � is circulant

with ; H � b}� H �F ± H �² � F e ± ² b diag x?µ�Proof: Usingthefactorizationgivenin TheoremC.4wefind; H � b �� H �F ± ² � F � H � b � H �F ± H �² � F 4Since

± ²is diagonal,

± H �² is computedin < operations.Hencethe computa-tional complexity of computing ; H � is dominatedby that of the FFT which is� �<V�/�� <h� .

Appendix D

Fourier differentiation matrix

Let Ä be aq

periodic function which is Å timesdifferentiableandconsideritsFourierexpansion Ä$ d �¿b Æm6yn H Æ�Ç 6+| ~ �K�+6#È e d ��¬ (D.1)

whereÇ 60b}É �C Ä d �y| H ~ �K�36�È Å d .WeapproximateÄ by thetruncatedexpansionÄ F d �¿b F �� H �m6yn H�F �#� Ç 6+| ~ �%�36�È e d �Ê¬

DifferentiatingÅ timesthenyieldsÄ �©Ë#�F d �&b F �#� H �m62n HGF �� Ç �©Ë#�6 | ~ �K�36�È e d ��¬where

Ç �ÌË#�6 b} £�K �¡�5�� Ë Ç 6 e 5�b¼� < e � < ) q:e E+E+E e < � qHenceÄ �©Ë#�F d � approximatesÄ �©Ë#� d � .

Let <�b¹ �Í for Î-��Ï anddts b � ¢ < ,

� bÐ�Ñ< ¢ e �9< ¢ ) q�e E+E+E e < ¢ � q.

DefinethevectorsÒ and Ò �©Ë#� asÄ s b�Ä F d�s �&b F �#� H �m6yn HGF �#� Ç 6Ó| ~ �K�+6#È.Ô bF �� H �m62n HGF �#� Ç 6 z 6

sF(D.2)

Ä �ÌË#�s b�Ä �ÌË��F dts �¿b F �#� H �m62n HGF �� Ç �ÌË��6 | ~ �K�36�È Ô b F �� H �m62n HGF �#� Ç �ÌË#�6 z 6sF

(D.3)

216 Fourierdifferentiationmatrix

for � b>� < e � < ) q:e E+E+E e < � qThen

Ç 60bq< F �#� H �ms n HGF �#� Ä s�z H 6 sF e 5�b¼� < e � < ) q:e E+E+E e < � q

We will now derive a matrix expressionfor computingÒ �©Ë#� usingtheFouriermatrix � F describedin (C.3). Theproblemis that theFouriermatrix is definedfor vectorsindex by

� b � e+q:e E+E+E e <Õ� qwhereasour vectorshereareshiftedby< ¢ . Applying � FÖ¢ < directly to thevector Ò yieldsq< FIH �m s n C Ä s�HGF �#� z H 6 sF b q< F �#� H �msB× n HGF �#� Ä s ×{z H 6 s ×F z H 6 F �#�FØ Ù+Ú Ûn�� H �K�/Ü bÝ y� q � 6 Ç 6 e 5�b � e3q�e E+E3E e <W� q

Thecoefficientsare < -periodic, Ç 6 b Ç 62Þ F , sowe canextendthedefinitionof Ç �©Ë#�6 to theinterval

@ � e <W� q J asfollows:

Ç �ÌË��6 b � �¡�5 Ç 6 for 5�b � e+q:e E+E+E e < ¢ 9� q �¡& £5ß�l<¤� Ç 6 for 5�b}< ¢ e < ¢ Ñ) q�e E3E+E e <W� q (D.4)

Let à b r Ç 6 uFIH �6yn Cà �ÌË�� b á Ç �©Ë#�6�â FIH �6yn C

anddefinetheFourier differ entiation matrix ã �©Ë#�ä asthediagonalmatrixå ã �©Ë#�ä�æ 6�N 6 b � :¡�5 for 57b � e+q�e E+E+E e < ¢ ç� q :¡& K5´��<¤� for 57b}< ¢ e < ¢ Ñ) q:e E+E+E e <W� q (D.5)

then(D.4) hasthevectorformulationà �©Ë#� b�ã �ÌË��ä à

217

Applying <h� H �F to

à �©Ë#� thenyieldsFIH �m 6yn C 2� q � 6 Ç �©Ë#�6 z 6sF b FIH �m 62n C Ç �ÌË#�6 z 6

sF z H 6 F �#�Fb FIH �m 62n C Ç �ÌË#�6 z 6��

s�HGF �#�#�Fb Ä �ÌË��s�HGF �#� e � b � e+q�e E+E+E e <W� qHence< and

q ¢ < canceloutandwe arriveatÒ �ÌË#� b}� H �F ã �©Ë#�ä � F ÒTheshift in (D.5) canfor examplebe donein MatlabusingFFTSHIFTor a

similar function.If theperiodis differentfromq, say è , then ã �ÌË#�ä mustbescaled

asdescribedin Section7.4.1.

218 Fourierdifferentiationmatrix

Appendix E

List of Matlab programs

ThefollowingMatlabprogramsdemonstrateselectedalgorithmsandareavailableon theWorld WideWebat ✤

http://www.imm.dtu.dk/˜omni/wapa20.t gz

Moredocumentationis foundin theenclosedREADME file.

Function Page Descriptionwavecompare 10 Comparesa wavelet approximation to a

Fourierapproximationbasisdemo 10 Generatesanddisplaysperiodicwaveletsdaubfilt 16 Returnsa vectorcontainingthe filter coeffi-

cientsA C3e A � e E+E3E e ADé H �

low2hi 17 Computesr:ê.6 u é H �6yn C from r A 6 u é H �6yn Cfilttest 21 Checksvectorof filter coefficientscascade 45 Computesfunctionvaluesfor ë d � and ì0 d �dst, idst 49 ComputestheDSTandits inversefwt, ifwt 59 ComputestheFWT andits inversefwt2, ifwt2 60 Computesthe2D FWT andits inverseconn 111 Computesí Ë for a givenwaveletgenusîdiftest 117 Testsconvergenceratesfor ã �ÌË#�

and ã Ëdifmatrix 118 Computesthedifferentiationmatrix ã �ÌË#� ¢ è Ënlsfft 195 Demonstratesthe FFT split-stepmethodfor

theNonlinearSchrodingerequationnlswofd 195 Demonstratesthe WOFD method for the

nonlinearSchrodingerequationburgerwofd 197 Demonstratesthe WOFD methodfor Burg-

ers’equation

Bibliography

[AB84] O. AxelssonandV. A. Barker. Finite ElementSolutionof BoundaryValue Problems. ComputerScienceandApplied Mathematics.Aca-demicPress,1984.

[Agr89] G. P. Agrawal. NonlinearFiber Optics. AcademicPress,1989.

[Bar96] V. A. Barker. Computingconnectioncoefficients. TechnicalReportIMM-REP-1996-5,TechnicalUniversityof Denmark,1996.

[BCR91] G. Beylkin, R. Coifman,andV. Rohklin. Fastwavelet transformsandnumericalalgorithms.Comm.Pure Appl.Math., 44:141–183,1991.

[Bey92] G. Beylkin. On the representationof operatorsin basesof compactlysupportedwavelets.SIAMJ. Num.Anal., 29(6):1716–1740,1992.

[Bey93] G. Beylkin. Waveletsandfastnumericalalgorithms.In DifferentPer-spectivesonWavelets, pages89–117.AMS, 1993.

[Bey94] G.Beylkin. Onwavelet-basedalgorithmsfor solvingdifferentialequa-tions”,. In J.J.BenedettoandM. W. Frazier, editors,Wavelets- Mathe-maticsandApplications, chapter12,pages449–466.CRCPress,1994.

[BK97] G.Beylkin andJ.M. Keiser. Ontheadaptivenumericalsolutionof non-linearpartial differentialequationsin waveletbases.J. Comp.Phys.,132:233–259,1997.

[BMP90] E. Bacry, S.Mallat, andG. Papanicolaou.A waveletbasedspace-timeadaptivenumericalmethodfor partialdifferentialequations.Technicalreport,CourantInstituteof MathematicalSciences,New York, 1990.

[BN96] S.BertoluzzaandG. Naldi. A waveletcollocationmethodfor thenu-merical solutionof partial differentialequations. Appl. Comp.Har-monicAnal., 3:1–9,1996.

221

222 Bibliography

[Cha96] P. Charton. Produitsde matricesrapidesen basesd’ondelettes:ap-plicationa la resolutionnumeriqued’equationsauxderivespartielles.PhDthesis,UniversitedeParisNord,1996.

[Chu97] C.K. Chui.Wavelets:AMathematicalTool for SignalAnalysis. SIAM,1997.

[CP96] P. Chartonand V. Perrier. A pseudo-wavelet schemefor the two-dimensionalNavier-Stokesequations. Matematica Aplicadae Com-putational, 15(2):139–160,1996.

[Dau88] I. Daubechies.Orthonormalbasesof compactlysupportedwavelets.Comm.Pure Appl.Math., XLI(7):909–996,1988.

[Dau92] I. Daubechies.TenLecturesonWavelets. SIAM, 1992.

[DKU96] W. Dahmen,A. Kunoth,andK. Urban.A waveletGalerkinmethodfortheStokesequations.Computing, 56:259–301,1996.

[DMC95] B. K. Das,R. N. Mahapatra,andB. N. Chatterli.Modellingof wavelettransformon multistageinterconnectionnetwork. In ProceedingsoftheAustralasianConferenceonParallel andReal-TimeSystems, pages134–142,Freemantle,WesternAustralia,28-29September1995.

[Dor95] M. Dorobantu.Wavelet-basedAlgorithmsfor FastPDE Solvers. PhDthesis,RoyalInstitueof Technology, Stockholm,1995.

[EBR97] L. Elden, F. Berntsson,andT. Reginska. Wavelet andFouriermeth-odsfor solving the sidewaysheatequation. Preprint.DepartmentofMathematics,LinkobingUniversity, 1997.

[Eir92] T. Eirola. Sobolev characterizationof solutionsof dilation equations.SIAMJ. Math.Anal., 23(4):1015–1030,1992.

[EL] F. EkstedtandM. Lindberg. Diagonalizationof homogeneouslinearoperatorsin biorthogonalwaveletbases.ftp.math.chalmers.se/pub/users/mlind/diag.a4.ps.gz.

[EOZ94] B. Engquist,S.Osher, andS.Zhong.Fastwaveletbasedalgorithmsforlinearevolutionequations.SIAMJ. Sci.Comp., 15(4):755–775,1994.

[F Þ 96] M. Fargeet al. Waveletsandturbulence.Proc. IEEE, 84(4):639–669,1996.

223

[FJAF78] M. D. FeitandJrJ.A. Fleck.Light propagationin graded-index opticalfibers.Appl.Optics, 17(24):3990–3998,1978.

[Fol95] G.B. Folland.Introductionto Partial DifferentialEquations. PrincetonUniversityPress,1995.

[FS97] J.Frolich andK. Schneider. An adaptive wavelet-vaguelettealgorithmfor thesolutionof pdes.J. Comp.Phys., 130(2):174–190,1997.

[GL89] G. H. GolubandC. Van Loan. Matrix Computations. JohnHopkinsUniversityPress,2 edition,1989.

[GL94] L. GagnonandJ. M. Lina. Symmetricdaubechies’waveletsandnu-mericalsolutionof nlsequations.J. Phys.A: MathGen, 27:8207–8230,1994.

[H Þ 94] C. Hsiaoet al. Nearoptimalcompressionof orthonormalwaveletex-pansions. In J. J. Benedettoand M. W. Frazier, editors,Wavelets-MathematicsandApplications, chapter11,pages425–446.CRCPress,1994.

[Heg95] M. Hegland. An implementationof multiple and multi-variatefastFourier transformson vector processors. SIAM J. Sci. Comp.,16(2):271–288,1995.

[Heg96] M. Hegland. Real and complex fast Fourier transformson the Fu-jitsu VPP500.Parallel Comp., 22:539–553,1996.

[HJ88] R. W. Hockney andC. R. Jesshope.Parallel Computers2. IOP Pub-lishingLtd, 1988.

[Hol97] M. Holmstrom. WaveletBasedMethodsfor Time DependentPDEs.PhDthesis,Departmentof ScientificComputing,UppsalaUniversity,1997.

[HPW94] F. Heurtaux,F. Planchon,andMladenVictor Wickerhauser. Scalede-compositionin burgers’equation.In J.J.BenedettoandM. W. Frazier,editors,Wavelets- Mathematicsand Applications, chapter14, pages505–523.CRCPress,1994.

[HW96] E. HernandezandG. Weiss. A First Courseon Wavelets. CRCPress,1996.

[Jam93] L. Jameson.On theDaubechies-basedwaveletdifferentiationmatrix.J. Sci.Comp., 8(3):267–305,1993.

224 Bibliography

[Jam94] L. Jameson.Onthewavelet-optimizedfinite differencemethod.Tech-nicalReportNASA CR-191601,ICASEReportNo. 94-9,1994.

[Jam96] L. Jameson.Thewavelet-optimizedfinite differencemethod.In G. Er-lebacherandM. Y. HussainianL. Jameson,editors,Wavelets.TheoryandApplications, chapter1.5, pages16–22.Oxford UniversityPress,1996.

[JS93] B. JawerthandW. Sweldens.Wavelet resolutionanalysisadaptedforthefastsolutionof boundaryvalueordinarydifferentialequations.InProceedingsof thesixthCopperMountainMultigrid Conference, 1993.

[JS94] B. JawerthandW. Sweldens.An overview of waveletbasedmultires-olutionanalysis.SIAMReview, 36(3):377–412,1994.

[Kai94] G. Kaiser. A Friendly Guideto Wavelets. Birkhauser, 94.

[Kei95] J.M. Keiser. WaveletBasedApproach toNumericalSolutionof Nonlin-earPartial DifferentialEquations. PhDthesis,Universityof Colorado,1995.

[Kun94] A. Kunoth.Computingintegralsof refinablefunctions- documentationof theprogram.Technicalreport,SINTEF, 1994.

[LBA81] M. Lax, J.H.Batteh,andG.P. Agrawal. Channelingof intenseelectro-magneticbeams.J. Appl.Phys., 52(1):109–125,1981.

[LPT92] J. Liandrat,V. Perrier, andP. H. Tchamitchan.Numericalresolutionof nonlinearpartial differentialequationsusingthewaveletapproach.In Mary Beth Ruskaiet al., editors,Waveletsand their applications,pages227–238.JonesandBartlett,1992.

[LR94] G. Leaf andJ.M. Restrepo.Wavelet-Galerkindiscretizationof hyper-bolic equations.Technicalreport,ArgonneNationalLaboratory, USA,1994.

[LRT91] A. Latto,H. L. Resnikoff, andE. Tenenbaum.Theevaluationof con-nectioncoefficients of compactlysupportedwavelets. Technicalre-port,AWARE,Inc,OneMemorialDrive,Cambridge,MA 02142-1301,USA, 1991.

[LS95] R. LangandA. Spray. The2D wavelettransformonamassively paral-lel machine.In Proceedingsof theAustralasianConferenceon Paral-lel andReal-TimeSystems, pages325–332,Freemantle,WesternAus-tralia,28-29September19951995.

225

[LT90a] A. Latto andE. Tenenbaum.Compactlysupportedwaveletsand thenumericalsolutionof burgers’equation.C.R.Acad.Sci.Paris, t. 311,Serie I, pages903–909,1990.

[LT90b] J.LiandratandP. Tchamitchian.Resolutionof the1D regularizedburg-ers equationusing spatialwavelet approximation. TechnicalReport90-83,ICASE,1990.

[Lu93] J. Lu. Parallelizingmallatalgorithmfor 2-D wavelet transforms.Inf.Proc.Letters, 45:255–259,1993.

[Mal89] S.G. Mallat. A theoryfor multiresolutionsignaldecomposition:Thewaveletrepresentation.IEEE Trans.PAMI, 11(7):674–693,1989.

[Mey93] Y. Meyer. Wavelets:AlgorithmsandApplications. SIAM, 1993.

[MR96] M. Metcalf and J. Reid. Fortran 90/95 explained. Oxford SciencePublications,1996.

[New92] A. C. Newell. NonlinearOptics. Addison-Wesley, 1992.

[New93] D. E. Newland. RandomVibrations, Spectral & Wavelet Analysis.LongmanScientific& Technical,1993.

[NH95] O. M. NielsenandM. Hegland. Two-dimensionalfastwavelet trans-form on theVP2200.Semi-annualreporton theFujitsu-ANUparallelmathematicalsubroutinelibrary project,AustralianNationalUniver-sity, 1995.

[NH97] O. M. NielsenandM. Hegland. A scalableparallel2D wavelet trans-form algorithm. ANU ComputerScienceTechnicalReportsTR-CS-97-21, AustralianNational University, 1997. Submittedto ParallelComputing.

[Nie97] O. M. Nielsen.Fast2D wavelettransformof circulantmatrices.Tech-nical Report IMM-REP-1997-8, TechnicalUniversity of Denmark,1997.

[PAP86] M. J.Potasek,G.P. Agrawal,andS.C.Pinault.Analyticandnumericalstudyof pulsebroadeningin nonlineardispersive optical fivbers. J.Opt.Soc.Am.B, 3(2):205–211,1986.

[PB91] V. PerrierandC. Basdevant. Travelling waveletsmethod. In Proc. oftheUS-French ’WaveletsandTurbulence’Workshop. Princeton,1991.

226 Bibliography

[PK91] D. PathriaandG. Karniadakis. Wavelet-aidedspectralelementflowsimulations.PreprintPrinceton,1991.

[PW96] I. PierceandL. Watkins. Modelling opticalpulsepropagationin non-linear mediausingwavelets. Technicalreport,Schoolof ElectronicEngineeringandComputerSystems,Universityof Wales,Bangor, UK,1996.

[QW93] S.QuianandJ. Weiss. Waveletsandthenumericalsolutionof partialdifferentialequations.J. comp.phys., 106:155–175,1993.

[RE97] T. ReginskaandL. Elden. Solving the sidewaysheatequationby awavelet-Galerkinmethod.Inv. Prob., 13:1093–1106,1997.

[Smi85] G.D. Smith. NumericalSolutionof Partial Differential Equations:Fi-niteDifferenceMethods. OxfordUniversityPress,1985.

[SN96] G. Strangand T. Nguyen. Waveletsand Filter Banks. Wellesley-CambridgePress,1996.

[Spi93] M. R. Spiegel. MathematicalHandbookof Formulas and Tables.Schaum’sOutlineSeries,1993.

[Str92] G. Strang. Wavelet transformsversusFouriertransforms. Technicalreport,MIT, Departmentof Mathematics,MIT, CambridgeMA 02139,1992.

[Str94] R. S. Stricharz. Constructionof orthonormalwavelets. In JohnJ.Benedettoand Michael W. Frazier, editors,Wavelets- MathematicsandApplications, chapter1, pages23–51.CRCPress,1994.

[Str96] G.Strang.Creatingandcomparingwavelets.Availablefrom theauthor([email protected] ), 1996.

[Swe] W. Sweldens.Waveletresources.Availableathttp://www.mathsoft.com/wavelets .html .

[Swe96] W. Sweldens. Wavelets: What next ? Proc. IEEE, 84(4):680–685,1996.

[Uni96] AustralianNationalUniversity. ANU VPP300.Availableathttp://anusf.anu.edu.au/VPP/ , 1996.

[VK95] M. Vetterli andJ.Kovacevic. WaveletsandSubbandCoding. PrenticeHall, 1995.

227

[VP96] O. V. Vasilyev and S. Paolucci. A dynamicallyadaptive multilevelwaveletcollocationmethodfor solvingpartialdifferentialequationinafinite domain.J. comp.phys., 125:498–512,1996.

[WA94] J. R. Williams and K. Amaratunga. Wavelet-Galerkinsolutionsforone-dimensionalpartialdifferentialequations.Int J. Num.Meth.Eng.,37:2703–2716,1994.

[Wal96] J.Walden. WaveletSolversfor HyperbolicPDEs. PhDthesis,Depart-mentof ScientificComputing,UppsalaUniversity, 1996.

[WM85] R. Wait andA. R. Mitchell. Finite ElementAnalysisandApplications.JohnWiley & SonsLtd., 1985.

[WZ94] L. WatkinsandY. R. Zhou. Modelling propagationin optical fibersusingwavelets.J. LightwaveTech., 12(9):1536–1542,1994.

[XS92] J. C. Xu andW. C. Shann. Galerkin-wavelet methodsfor two pointboundaryvalueproblems.In Num.Math. [Swe], pages123–144.

Index

2D FWT, 60of circulantmatrix,121,127of circulant,bandedmatrix,146parallelizationof, 95vectorperformance,81

2D circulantwavelettransformalgorithm,139complexity, 140storage,142storagerequirement,145

accuracyof themultiresolutionspaces,61of waveletdifferentiationmatrix,

115of wavelet-Galerkinmethod,165

Amdahl’s law, 85approximatingscalingfunctioncoef-

ficientsbyscaledfunctionval-ues,66

approximationproperties,61–70of ïð Í , 63ofð Í , 61

approximationspaces,11–13,38arithmetic-geometricseries,144

bandedcirculantmatrix,146bandwidth,114,116,146basicscalingfunction,13basicwavelet,13Burgers’equation,185,195

calculationof bandwidths,147cascadealgorithm,43

CFWT, 97circulant2D wavelettransform

algorithm,139complexity of, 140storage,142storagerequirement,145

circulantmatrix,209andconvolution,212andDFT, 212commutation,213inverse,213wavelettransformof, 121,127

CIRFWT, 139CIRMUL, 158column-shift-circulantmatrix,127communication,89communication-efficientFWT, 95,97

performancemodelfor, 97complexity

of 2Dcirculantwavelettransform,140

of 2D FWT, 60of FWT, 59of matrix-vectormultiplicationin

thewaveletdomain,159of waveletsplit-stepmethod,182

compression,64,67–70error, 69,173of ; H � , 171of matrix,171of solutionto Burgers’equation,

189

229

230 Index

of solutionto thenonlinearSchrodingerequation,178

of vector, 171connectioncoefficients,108conservationof area,33convolution,209convolutiontheorem,211

datastructurefor 2D wavelet trans-form, 136

depth,15DFT, 210

andcirculantmatrices,212differentialequations

examples,163–198differentiationmatrix,114

convergencerate,115,165Fourier, 215with respectto physicalspace,115with respectto scalingfunctions,

114with respectto wavelets,119WOFD,190

dilationequation,15,16,33in frequency domain,25

discreteFouriertransform,210DST, 49

evaluationof scalingfunctionexpan-sions,47

expansionFourier, 3periodicscalingfunctions,39periodicwavelets,39scalingfunctions,14,47wavelets,6, 14

fastwavelettransform,52–60complexity of, 59of circulantmatrix,127of circulant,bandedmatrix,146

FFTsplit-stepmethod,185

filter coefficients,16finite differencesonanirregulargrid,

189Fourierdifferentiationmatrix,215,216Fourierexpansion,3Fouriertransform

continuous,25discrete,210inversediscrete,210

FujitsuVPP300,73,99FWT, 52,59,121

complexity of, 59Definitionof, 58Matrix formulation,56parallelizationof, 86periodic,54vectorperformance,74

genus,16grid generationusingwavelets,193

heatequation,168hybridrepresentation,170with respecttophysicalspace,169with respectto scalingfunctions,

168with respectto wavelets,169

Helmholzequation,163hybridrepresentation,167with respecttophysicalspace,167with respectto scalingfunctions,

163with respectto wavelets,166

IBM SP2,99IDFT, 210IDST, 49IFWT, 54,59

periodic,56ñyò N s , 17inversediscreteFouriertransform,210inversefastwavelettransform,54

231

inversepartialwavelettransform,54IPWT, 54

periodic,56

Kroneckerdelta,14ó, 15

Mallat’s pyramidalgorithm,seefastwavelettransform

Matlabprograms,219matrix

banded,146circulant,209column-shift-circulant,127row-shift-circulant,127

matrixexponentialof acirculantma-trix, 176

matrix-vectormultiplicationin awaveletbasis,153,158

complexity, 159MFWT, 79

parallelalgorithmfor, 91parallelizationof, 91performancemodelfor, 93vectorizationof, 79

modulusoperator, 205moments

of scalingfunctions,203vanishing,19

motivation,3multiple1D FWT, 79multiresolutionanalysis,11

nonlinearSchrodingerequation,174,191

numericalevaluationof ë and ì , 43

orthogonality, 14,33in frequency domain,30

parallelperformanceof the2D FWT, 96–100

of theMFWT, 93paralleltranspositionanddatadistri-

bution,96parallelization,85

of the2D FWT, 95of theFWT, 86of theMFWT, 91

Parseval’sequationfor wavelets,15partialdifferentialequations,seedif-

ferentialequationspartialwavelettransform,53PDEs,seedifferentialequationsperformancemodel

for thecommunication-efficientFWT,97

for theMFWT, 93for thereplicatedFWT, 96

periodicboundaryvalueproblem,163periodicfunctions

on theinterval@�A e ê�J , 50

periodicFWT, 54periodicIFWT, 56periodicinitial-valueproblem,168,174,

185periodicIPWT, 56periodicPWT, 54periodicscalingfunction

expansion,39periodicwavelet

expansion,39periodizedwavelets,33ë , 13Projection

on ïð ò and ïô ò , 41on

ð òand

ô ò, 15

projectionmethods,107propertyof vanishingmoments,20ì , 13� th momentof ë$ d ��5�� , 19PWT, 53

periodic,54

232 Index

pyramidalgorithm, seefast wavelettransform

replicatedFWT, 95performancemodelfor, 96

RFWT, 95row-shift-circulantmatrix,127

scaleparameter, 6scalingfunction,13

expansion,14,47evaluationof, 47

in frequency domain,25momentsof, 203periodized,33supportof, 17

shift parameter, 6shift-circulantmatrices,127SP2,99split-stepmethod,175

FFT, 185wavelet,175

split-transposealgorithm,81storageof 2D circulantwavelettrans-

form, 142storagerequirement,145support,17survey of waveletapplicationstoPDEs,

105

timesteppingin thewaveletdomain,170

twotermconnectioncoefficients,108ð Íapproximationpropertiesof, 61definitionof, 11ïð Íapproximationpropertiesof, 63definitionof, 38

vanishingmoments,19,20,33vectorperformance

of the2D FWT, 81of theFWT, 74

vectorizationof the2D FWT, 81of theFWT, 73of theMFWT, 79

VPP300,99ô Ídefinitionof, 12ïô Ídefinitionof, 38

wavelet,13algorithms,43–60basic,13differentiationmatrices,105–119expansion,6, 14genus,16in frequency domain,25periodized,33supportof, 17

waveletcompression,seecompressionwaveletdifferentiationmatrix,119waveletequation,15,16

in frequency domain,29waveletoptimizedfinitedifferencemethod,

188waveletsplit-stepmethod,175

complexity of, 182wavelettransform,seefastwavelettrans-

formwavelets

on therealline, 11on thetheunit interval, 33

WOFD, seewaveletoptimizedfinitedifferencemethod

w avelets in scientiﬁc computing - technical … pa˚ dansk w avelets i scientiﬁc computing...

Documents