![Page 1: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/1.jpg)
28 Sep 2015
Big O Theory Club , Georgia Tech
FAST MULTIPOLE
METHODS (FMM)
SKANDA PRASAD
![Page 2: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/2.jpg)
In Gravitation or Electrostatics
Φ 𝒚𝑗 = 𝑘
𝑖=1
𝑁𝑞𝑖
𝒙𝑖 − 𝒚𝑗
OR
𝜱 𝒚 = 𝑘. 𝑅𝒒(Matrix-Vector product form)
Similar problems in acoustics, scattering, magnetostatics, illumination, thermodynamics etc.
Essentially, problems of the form
𝑢 𝒚 = 𝐾 𝒙, 𝒚 𝑤 𝒙 𝑑𝒙
Or
𝑢 𝒚 = 𝑖=1𝑁 𝑤𝑖𝐾 𝒚, 𝒙𝑖
Can approximate integrals as sums.
THE CLASSIC N-BODY PROBLEM
![Page 3: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/3.jpg)
Highly Complex
Matrix-Vector Product
O(MN) for dense matrices
O(N2) for N-body problem
Complex in time AND memory!
Fast Summation Methods
Achieve in O(Nα), with 1 ≤ α < 2
Eg: FFT
Sequential Algorithms
○ Appel (1985), Barnes and Hut (1986)
THE PROBLEM??
![Page 4: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/4.jpg)
Wickedly Fast ( O(N log N) )…
Exact
… BUT:
Restrictive Problem Structures.
Only uniform sample grids.
Very high sample rates required
Not great for sparse or uneven
structures.
Not robust
Can’t work for 2D manifolds in
3D space.
WHAT ABOUT THE FFT?
![Page 5: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/5.jpg)
FMMS TO THE RESCUE!
• Greengard and Rokhlin, 1985.
• Original Aim: Calculate N-body / MoM problems
faster.
• One of the Top 10 Algorithms of the 20th Century
(SIAM)
• Monte Carlo Method
• Simplex Method for LP
• Krylov Subspace iteration
• Matrix Decompositions
• Integer relation detection algo.
• Fortran
• QR Algorithm
• Quicksort
• FFT
• The FMM
![Page 6: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/6.jpg)
Reduced computational cost From O(N2) to O(N log N) – or even O(N).
Reduced Memory requirements O(N2) to ~ O(N log N).
Just-in-time storage possible
Numerically exact solution. Can achieve machine precision.
Pre-computation to achieve speedup.
FMMS TO THE RESCUE!
![Page 7: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/7.jpg)
FMM - CHARACTERISTICS
• Common uses:
• Method of Moments (MoM) and N-Body problems.
• Computational EMag, Large scale scattering problems.
• Tree Code:
• Specified accuracy, є.
• Hierarchical subdivision of space.
• Far field expansion of ‘kernel’ K(x, y), separating source and
target influences.
• (Optional) Converting Far field expansions into local
expansions.
![Page 8: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/8.jpg)
FMM – THE BASIC INGREDIENTS
• SUGAR: Finite Rank / Degenerate
Kernels
• SPICE: Error Bounding through
truncated series expansion.
• EVERYTHING NICE: Hierarchical
Partitioning / Recursive computation
• For MLFMA
• … And CHEMICAL X:
Translation + Precomputation Takes the FMM from O(N logN) to O(N)
![Page 9: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/9.jpg)
Finite Rank Approximation.
𝐾 𝒙, 𝒚 = 𝑘=0𝑃−1𝜙𝑘 𝒙 𝜓𝑘 𝒚
• Decouples source and target influences
• Is a low rank approximation of the field.
Example:
1
𝑦− 𝑥𝑖=1
𝑦− 𝑥∗. 𝑚=0∞ 𝑥𝑖− 𝑥∗
𝑦− 𝑥∗
𝑚
“Multipole Expansion”
Regular (R) & Singular (S) Expansions
SUGAR – DEGENERATE KERNELS
![Page 10: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/10.jpg)
Multipole Expansions: Guaranteed far field convergence Far Field: Target box is ‘well separated’ from Source box.
How far is ‘well separated’? Ask MAC – Multipole Acceptance Criterion
Rigorous Error bounds Error bounding is NOT a matter of trial and error - Can be ANALYTICALLY
bounded
Can a priori choose truncation number.
SPICE – ERROR BOUNDING
![Page 11: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/11.jpg)
Single Evaluation
Centre.
THE MIDDLEMAN APPROACH
![Page 12: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/12.jpg)
MULTIPLE EVALUATION CENTERS
“Approximation” can be as exact as we’d like
![Page 13: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/13.jpg)
Given a tolerance ‘ε’, choose
expansion rank, P.
SINGLE LAYER FMM / BARNES-HUT
![Page 14: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/14.jpg)
Given a tolerance ‘ε’, choose expansion rank, P.
Partition the source space into cells
Each cell has O(1) no. of sources.
IMPORTANT: Assumed uniform distribution
SINGLE LAYER FMM / BARNES-HUT
![Page 15: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/15.jpg)
Given a tolerance ‘ε’, choose expansion rank, P.
Partition the source space into cells
Each cell has O(1) no. of sources.
IMPORTANT: Assumed uniform distribution
Form multipole expansions about each cell centre.
SINGLE LAYER FMM / BARNES-HUT
![Page 16: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/16.jpg)
Given a tolerance ‘ε’, choose expansion rank, P.
Partition the source space into cells
Each cell has O(1) no. of sources.
IMPORTANT: Assumed uniform distribution
Form multipole expansions about each cell centre.
For each target, do:
Aggregate contributions from each far-field cell
SINGLE LAYER FMM / BARNES-HUT
![Page 17: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/17.jpg)
Given a tolerance ‘ε’, choose expansion rank, P.
Partition the source space into cells
Each cell has O(1) no. of sources.
IMPORTANT: Assumed uniform distribution
Form multipole expansions about each cell centre.
For each target, do:
Aggregate contributions from each far-field cell
Individually add the contributions from each near field source point.
SINGLE LAYER FMM / BARNES-HUT
![Page 18: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/18.jpg)
SINGLE LAYER FMM
![Page 19: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/19.jpg)
EVERYTHING NICE – HIERARCHICAL PARTITIONING
![Page 20: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/20.jpg)
![Page 21: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/21.jpg)
EVERYTHING NICE – HIERARCHICAL PARTITIONING
L = logb (N) levels.
Near Field
Interaction List
Far Field
![Page 22: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/22.jpg)
AND NOW…
… FOR
CHEMICAL X
![Page 23: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/23.jpg)
Multipole to Multipole Translation (M2M): Or “Outgoing-to-Outgoing”
Transforms the multipole expansion of a node’s children to its own multipole expansion.
Multipole to Local Translation (M2L): Transforms the multipole expansion of a node into
the local expansion of a node in its interaction list
Local to Local Translation (L2L): Or “Incoming / local expansion”
Transforms the local expansion of a node’s parent to its own local expansion
TRANSLATION OPERATORS
![Page 24: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/24.jpg)
1. Aggregation (Multipole
Expansion)
2. Translation
1. Outgoing-to-outgoing
2. Outgoing-to-local/incoming
3. Disaggregation
TRANSLATION OPERATORS
12 3
![Page 25: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/25.jpg)
PREPARATION:
1. Choose 𝑝 ≈ 𝑙𝑜𝑔𝑏 𝜖𝒅 1
𝑜𝑟 𝑝 ≈ log2 1𝜖
‘b’ varies with problem.
‘d’ – leaf cell size
2. Choose number of levels, 𝐿 ≈ log2𝑁 .
3. Partition the space so that each leaf cell has O(1) sources.
PRECOMPUTATION:
1. Form multipole expansions (moments) at the tree’s finest scale.
2. UPWARD PASS: Translate expansions from lower levels to next coarser level.
Multipole expansions of each leaf cell translated to parent cell.
THE MULTI-LAYER FMM ALGORITHM (MLFMA)
![Page 26: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/26.jpg)
ALGORITHM:
3. DOWNWARD PASS: For each (target) region, Transform multipole expansions of all far-
field cells in the interaction list (at current level) into a local expansions about current
cell centre.
Algorithm bottleneck, but still O(N).
3. DOWNWARD PASS: For each region, translate the local expansion down the tree to the
centre of each of its 2d children.
4. Repeat 3 and 4 till finest level;
5. Evaluate series expansions at each target location and aggregate near neighbour
interactions directly.
THE MULTI-LAYER FMM ALGORITHM (MLFMA)
![Page 27: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/27.jpg)
Asymptotic Cost Getting to O(N)
Stage 1: Compute all outgoing expansions O ( N log N ) Easy
Stage 2: Compute Far Field Potentials O ( N log N ) ??
Stage 3: Compute Near Field Potentials O ( N ) No need to fix
COMPUTATIONAL COMPLEXITY
… BECAUSE IT WON’T BE BIG-O WITHOUT IT
Single Layer FMM
Need to compute all interactions
Too much “communication” between boxes
Multi-level FMM
O(N log N) - Each particle (N) communicates with all levels above it.
Tradeoff: decimate for smaller cells vs increasing # of levels
![Page 28: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/28.jpg)
LevelP = expansion size,
L ~ log N, d = dimensions# (Opers)
L (leaf) “outgoing from sources” N.P
L - 1 “outgoing from outgoing” Nboxes.P2
L – 2 - do -(Nboxes/4).P2
L – 3 - do - (Nboxes/16).P2
… … …
COMPUTATIONAL COMPLEXITY
Single Layer FMM
Need to compute all interactions
Too much “communication”
between boxes
Multi-level FMM
O(N log N) - Each particle (N)
communicates with all levels
above it.
Tradeoff: decimate for smaller
cells vs increasing # of levels Cost of computing all outgoing expansions = O(N)
![Page 29: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/29.jpg)
COMPUTATIONAL COMPLEXITY
![Page 30: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/30.jpg)
PUTTING IT ALL TOGETHER …
![Page 31: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/31.jpg)
Standard M-V
Product
Middle approach
Middleman with Target
Clusters
Middleman with Source
Clusters
Single Layer FMM
PUTTING IT ALL TOGETHER
![Page 32: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/32.jpg)
ADVANTAGES OF FMM
• O( N log N ) complexity, Going down to O(N)
• Non-uniform distributions supported, (slightly ↑sed complexity).
• (Rigourous) Error Bounding to arbitrary precision.
• Massive Memory saving.
• FMM parallelizes very well.
• Compare with FFT
• FFT – Exact, but Brittle.
• FMM – Approximate but robust. Rigorous a priori error bound possible.
![Page 33: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/33.jpg)
Adaptive FMMs – 2 extra steps1. Partition carefully
2. Some exceptional boxes.
Fast Elliptic Solvers for PDEs PDE Integral Equation Sum Approx. FMM
SOME INTERESTING EXTENSIONS
![Page 34: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/34.jpg)
Different Kernels Cauchy Kernel (Electrostatics)
Laplace’s Equation and Green’s Function
Helmholtz Equation
3D Laplace’s Equation
Solving certain PDEs Recast as Integral Equations
Stealth / low-RCS testing
CFD and Thermodynamics?
…What’s next?
EXPANDING HORIZONS
![Page 35: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/35.jpg)
… Want to learn someone who isn’t a dummy?
Beatson & Greengard, A short course on FMMs
Raykar, A Short Primer on the FMM
LeXing, A pedestrian introduction to FMMs
Darve, The FMM: Numerical Implementation
… Want some CS / implementation specific stuff?
Chandramowlishwaran, The FMM at Exascale. (PhD Dissertation, Georgia Tech, 2013)
… feeling especially mathochistic?
Greengard & Rokhlin, A new version of the FMM for the 3D Laplace Equation.
Cheng, Greengard & Rokhlin, A fast adaptive multipole algorithm in 3 dimensions.
… not feeling punished enough?
FINE! Greengard’s original thesis it is then.
IF I HAVEN’T BORED YOU YET…
![Page 36: FAST MULTIPOLE METHODS (FMM)€¦ · 28.11.2018 · • Specified accuracy, є. • Hierarchical subdivision of space. • Far field expansion of ‘kernel’ K(x, y), separating](https://reader036.vdocuments.net/reader036/viewer/2022090609/605f2b18b2dbc44111469215/html5/thumbnails/36.jpg)
36