particle advection defined & usage
DESCRIPTION
Particle Advection Defined & Usage. Particle advection is a foundational visualization algorithm. Advecting particles creates integral curves. Particle advection is a foundational visualization algorithm. Courtesy C. Garth, Kaiserslautern. - PowerPoint PPT PresentationTRANSCRIPT
Particle Advection Defined & Usage
Particle advection is a foundational visualization algorithm
• Advecting particles creates integral curves
Particle advection is a foundational visualization algorithm
Courtesy C. Garth, Kaiserslautern
Particle advection is a foundational visualization algorithm
Streamlines in the “fish tank”
Image credit:A. Sanderson, Univ. Utah
Particle advection is a foundational visualization algorithmPoincaré analysis
of a tokamak
Particle advection is a foundational visualization algorithm
Stream surface
around a tokamak
Lagrangian Methods
Visualize manifolds of maximal stretching in a flow, as indicated by dense particles
Finite-Time Lyapunov Exponent (FTLE)
Courtesy Garth
www.vacet.org
Lagrangian Methods Visualize manifolds of maximal
stretching in a flow, as indicated by dense particles
Forward in time: FTLE+ indicates divergence
Backward in time: FTLE- indicates convergence
Courtesy Garth
Particle Advection In A Parallel Setting
Parallel Particle Advection: How To Load Balance?
N particles (P1, P2, … Pn), M Processing Elements/PEs (PE1, …, PEm) (Processing Element = instance of your program)
Each particle takes a variable number of steps, S1, S2, … Sn
Total number of steps is ΣSi We cannot do less work than this (ΣSi)
Goal: Distribute the ΣSi steps over M PEs such that problem finishes in minimal time
Goal: Distribute the ΣSi steps over M PEs such that problem finishes in minimal time
Sounds sort of like a bin-packing problem, but… particles can move from PE to PE (i.e. work
can move from bin to bin) path of particle is data dependent and
unknown a priori (i.e. we don’t know Si beforehand)
big data significantly complicates this picture…. … data may not be readily available, introducing
starvation
Parallel Particle Advection: How To Load Balance?
Advecting particles
Decomposition of large data set into blocks on filesystem
?
What is the right strategy for getting particle and data together?
Strategy: load blocks necessary for advection
Decomposition of large data set into blocks on filesystem
Go to filesystem and read block
Decomposition of large data set into blocks on filesystem
Strategy: load blocks necessary for advection
“Parallelize over Particles” Basic idea: particles are partitioned over
PEs, blocks of data are loaded as needed. Positives:
Indifferent to data size: a serial program can process data of any size
Trivial parallelization (partition particles over processors)
Negative: Redundant I/O (both across PEs and within a
PE) is a significant problem.
“Parallelize over data” strategy:parallelize over blocks and pass particles
PE1 PE2
PE4PE3
“Parallelize over Data” Basic idea: data is partitioned over PEs,
particles are communicated as needed. Positives:
Only load data one time Great for in situ processing
Negative: Starvation!
Contracts are needed to enable these different processing techniques
Parallelize-over-seeds One execution per block Only limited data available at one time
Parallelize-over-data One execution total Entire data set available at one time
Both parallelization schemes have serious flaws.
Two approaches:
Parallelizing Over I/O EfficiencyData Good BadParticles Bad Good
Parallelizeover particles
Parallelizeover dataHybrid algorithms
The master-slave algorithm is an example of a hybrid technique.
Uses “master-slave” model of communication One process has unidirectional control over
other processes Algorithm adapts during runtime to avoid
pitfalls of parallelize-over-data and parallelize-over-particles. Nice property for production visualization tools. (Implemented in VisIt)
D. Pugmire, H. Childs, C. Garth, S. Ahern, G. Weber, “Scalable Computation of
Streamlines on Very Large Datasets.” SC09, Portland, OR, November, 2009
Master-Slave Hybrid Algorithm• Divide processors into groups of N
• Uniformly distribute seed points to each group
Master:- Monitor workload- Make decisions to optimize resource
utilization
Slaves:- Respond to commands from
Master- Report status when work
complete
SlaveSlaveSlave
Master
SlaveSlaveSlave
Master
SlaveSlaveSlave
Master
SlaveSlaveSlave
MasterP0P1P2P3
P4P5P6P7
P8P9P10P11
P12P13P14P15
Master Process Pseudocode
Master(){ while ( ! done ) { if ( NewStatusFromAnySlave() ) { commands = DetermineMostEfficientCommand()
for cmd in commands SendCommandToSlaves( cmd ) } }}
What are the possible commands?
Commands that can be issued by master
Master Slave
Slave is given a particle that is contained in a block that is already loaded
1. Assign / Loaded Block2. Assign / Unloaded Block3. Handle OOB / Load4. Handle OOB / Send
OOB = out of bounds
Master Slave
Slave is given a particle and loads the block
Commands that can be issued by master
1. Assign / Loaded Block2. Assign / Unloaded Block3. Handle OOB / Load4. Handle OOB / Send
OOB = out of bounds
Master Slave
Load
Slave is instructed to load a block. The particle advection in that block can then proceed.
Commands that can be issued by master
1. Assign / Loaded Block2. Assign / Unloaded Block3. Handle OOB / Load4. Handle OOB / Send
OOB = out of bounds
Master Slave
Send to J
Slave J
Slave is instructed to send the particle to another slave that has loaded the block
Commands that can be issued by master
1. Assign / Loaded Block2. Assign / Unloaded Block3. Handle OOB / Load4. Handle OOB / Send
OOB = out of bounds
Master Process Pseudocode
Master(){ while ( ! done ) { if ( NewStatusFromAnySlave() ) { commands = DetermineMostEfficientCommand()
for cmd in commands SendCommandToSlaves( cmd ) } }}
Driving factors… You don’t want PEs to sit idle
(The problem with parallelize-over-data) You don’t want PEs to spend all their
time doing I/O (The problem with parallelize-over-seeds)
you want to do “least amount” of I/O that will prevent “excessive” idleness.
The heuristics behind the heuristics Rules:
If no PE has the block, then load it! If a PE does have the block…
If it is not busy, then give it the particle If it is busy, then wait “a while”. If you’ve waited a “long time”, then load it
yourself.
Master-slave in action
S0
S0
S1
S1S2
S3
S4
Iteration
Action
0 S0 reads B0,S3 reads B1
1 S1 passes points to S0,S4 passes points to S3,S2 reads B0
0: Read
0: Read1: Pass
1: Pass1: Read
Algorithm Test Cases
- Core collapse supernova simulation- Magnetic confinement fusion simulation- Hydraulic flow simulation
Workload distribution in parallelize-over-data
Starvation
Workload distribution in parallelize-over-particles
Too much I/O
Workload distribution in master-slave algorithm
Just right
ParticlesData Hybrid
Workload distribution in supernova simulation
Parallelization by:
Colored by PE doing integration
Astrophysics Test Case: Total time to compute 20,000 Streamlines
Sec
onds
Sec
onds
Number of procs Number of procs
Uniform Seeding
Non-uniform Seeding
Data HybridParticles
Astrophysics Test Case: Number of blocks loaded
Blo
cks
load
ed
Blo
cks
load
ed
Number of procs Number of procs
Data Hybrid
Uniform Seeding
Non-uniform Seeding
Particles
Summary: Master-Slave Algorithm
First ever attempt at a hybrid algorithm for particle advection
Algorithm adapts during runtime to avoid pitfalls of parallelize-over-data and parallelize-over-particles. Nice property for production visualization tools.
Implemented inside VisIt visualization and analysis package.
Benefits of an Extended Memory Hierarchy
Emerging I/O Configuration
Emerging I/O ConfigurationCommon I/O Configuration
Simulation Codes
SimulationCode
GlobalFile System
Node
Checkpoint/Restart
SimulationCode
Node
Checkpoint/Restart
SSD
Parallelize-over-Seeds Algorithm
IntegrationI/O
Memory Cache
GlobalFile System
Streamlines
Node
Parallelize-over-Seeds Algorithm with the Extended Memory Hierarchy
IntegrationI/O
Memory Cache
GlobalFile System
Streamlines
Local File SystemNode
I/O Configurations
A Real World Particle Advection System
Goal Efficient code for a variety of particle
advection based techniques Cognizant of use cases with >>10K
particles. Need handling of every particle, every
evaluation to be efficient. Want to support diverse flow techniques:
flexibility/extensibility is key. Fit within data flow network design (i.e. a
filter)
Motivating examples of system FTLE Stream surfaces Streamline Dynamical Systems (e.g. Poincaré Maps) Statistics based analysis + more
Design PICS filter: parallel integral curve system Execution:
Instantiate particles at seed locations Step particles to form integral curves
Analysis performed at each step Termination criteria evaluated for each step
When all integral curves have completed, create final output
Design Five major types of extensibility:
How to parallelize? How do you evaluate velocity field? How do you advect particles? Initial particle locations? How do you analyze the particle paths?
Inheritance hierarchy
avtPICSFilter
Streamline Filter
Your derived type of PICS
filter
avtIntegralCurve
avtStreamlineIC
Your derived type of integral
curve
We disliked the “matching inheritance” scheme, but this achieved all of our design goals cleanly.
#1: How to parallelize?
avtICAlgorithm
avtParDomIC-Algorithm
(parallel over data)
avtSerialIC-Algorithm
(parallel over seeds)
avtMasterSlave-
ICAlgorithm
#2: Evaluating velocity field
avtIVPField
avtIVPVTKField
avtIVPVTK- TimeVarying-
FieldavtIVPM3DC1
Field
avtIVP-<YOUR>HigherOrder-Field
IVP = initial value problem
#3: How do you advect particles?
avtIVPSolver
avtIVPDopri5 avtIVPEuler avtIVPLeapfrog
avtIVP-M3DC1Integrato
r
IVP = initial value problem
avtIVPAdams-Bashforth
#4: Initial particle locations avtPICSFilter::GetInitialLocations() = 0;
#5: How do you analyze particle path? avtIntegralCurve::AnalyzeStep() = 0;
All AnalyzeStep will evaluate termination criteria avtPICSFilter::CreateIntegralCurveOutput(
std::vector<avtIntegralCurve*> &) = 0;
Examples: Streamline: store location and scalars for current
step in data members Poincare: store location for current step in data
members FTLE: only store location of final step, no-op for
preceding steps NOTE: these derived types create very
different types of outputs.
Putting it all togetherPICS Filter
avtICAlgorithmavtIVPSolv
er
avtIVPFieldVector<
avtIntegral-Curve>
Integral curves sent to other processors with some derived types of avtICAlgorithm.
::CreateInitialLocations() = 0;
::AnalyzeStep() = 0;