a.c. bauer, m.s. shephard, e. seol and j. wan, ([email protected]) scientific computation...
TRANSCRIPT
A.C. Bauer, M.S. Shephard, E. Seol and J. Wan, ([email protected])
Scientific Computation Research Center Rensselaer Polytechnic Institute, Troy, NY 12180
Outline: TSTT simulation infrastructure development Structures to support the construction of adaptive loops Mesh modification tools for adaptive mesh control Adaptive loop for DOE accelerator modeling code Adaptive loop for evolving geometry problem solved using commercial
analysis engine
The TSTT Interface for Mesh-Based Simulations
Terascale Simulation Tools & Technologies (TSTT) Center
Goal: To enable high-fidelity calculations based on multiple coupled physical processes and multiple physical scales Adaptive methods Composite or hybrid solution strategies High-order discretization strategies
Barrier: The lack of easy-to-use interoperable meshing, discretization and adaptive tools requires too much software expertise by application scientists
Organizations: DOE Labs: Argonne, Brookhaven, Lawrence Livermore, Oak Ridge, Pacific
Northwest, Sandia Universities: Rensselaer Polytechnic Institute, SUNY Stony Brook, University of
British Columbia Project Managers:
David Brown: [email protected] Lori Freitag Diachin: [email protected] Jim Glimm: [email protected]
More information: www.tstt-scidac.org
TSTT Interoperability Goal
Accomplishing this goal requires: An abstract data model that encompasses a
broad spectrum of mesh types and usage scenarios
A set of common interfaces● Implementation and data structure
neutral● Small enough to encourage adoption● Flexible enough to support a broad
range of functionality
Accomplishing this goal requires: An abstract data model that encompasses a
broad spectrum of mesh types and usage scenarios
A set of common interfaces● Implementation and data structure
neutral● Small enough to encourage adoption● Flexible enough to support a broad
range of functionality
GeometryInformation(Level A)
Full GeometryMeshes(Level B)
MeshComponents(Level C)
To provide interchangeable and interoperable access to different meshmanagement and discretization strategies Ease experimentation with different
technologies Combine technologies together for hybrid
solution techniques
A different way of thinking about simulation code development
TSTT Simulation Infrastructures
7 Interacting structures Mathematical problem definition
1) Space/time domain ***2) Physical and mathematical model attributes
First level of discretization3) Mesh ***
Second level of discretization4) Discretization operators acting on mathematical equations and degrees-of-
freedom (dof) Application of #4 over mesh entities
5) Algebraic system contributors Solving for dof values
6) Algebraic system Description of the physical parameters
7) Fields ***
*** Main TSTT focus to date
Space/Time Domain
Overall domain definition - geometric model and timedomain Want a “true definition” of the domain - not a specific
approximation Define other structures in relation to the domain definition
Problem attributes defined over portions of the domain Mesh represents discretization of domain Fields are discretizations of physical tensors defined over the
domain
At SCOREC, direct geometric model access via topology Shape independent abstraction Need general combination of solids, surfaces and curves Non-manifold topology representations known Effectively supported by commercial solid modelers
Topology built from topology in modeling source Geometric queries passed through modeler API - supports
auto-meshing
Attributes
Problem definition attributes Governing equations, material properties, loads, b.c.’s
Solution strategy attributes Meshing parameters, PDE discretization, basis functions, etc.
Attributes applied to geometric model
Problem definition attributes: Are tensors Have distributions Have relationships
Classification used to retrieve attributes
TSTT Mesh Data Model
Data model consists of entities and entity sets Entity definition
Unique topology Canonical ordering defines adjacency
relationships Entity set definition
Arbitrary grouping of TSTT mesh entities There is a single “Root Set” Relationship among entity sets
Contained-in Hierarchical
Mesh Definition Simple meshes are a set of connected,
non-overlapping entities related through adjacency relationships
Meshes can contain collections of simple meshes
TSTT Mesh Interface Functionality
Core functionality The smallest set of functions required to be considered TSTT compliant –
extensions for implementation of advanced functionality Provides basic access to vertex coordinates and adjacency information
Mesh loading and saving Accessing global information such as the root set, geometric dimension, number of
entities of a given type or topology Primitive array access for entity sets Global entity arrays for entity sets
Traversal functionality Advanced traversal functions provide additional flexibility and performance
Single Entity Traversal Iterator functionality (init, next, reset, end) Query (topology, type, adjacency)
Workset Traversal Block iterators allow access of entities in a user-defined array size Array based query functions for topology, type, adjacency Increases performance by reducing the number of calls through the interface Interface is not as intuitive
TSTT Mesh Modification Operators
Simple mesh modification operations are also supported Modifications
Changing vertex location Add and delete entities
Caveats No validity checks Requires simple classification mechanisms
against the geometric model Intended to support higher-level
functionality Mesh quality improvement Adaptive schemes with validity checking Front tracking procedures Basic mesh generation algorithms
Domain Discretization and Mesh Representation
Geometric model discretized into a mesh Discretized geometric domain defined as “union” of the
mesh entities Maintain link to shape information Support the field’s distributions used in equation
discretization Maintain the dof multiplying the distributions and support
assembly of global system
Mesh topology provides flexible support for these functions Mesh entities, Md, of dimension d Mesh entities are constructed from lower dimensional
mesh entities e.g. A face is constructed from edges which are constructed
from vertices Entities are adjacent if one is used to construct the other Typical complete mesh topological structures:
Mesh/Model Relationship
Relationship is termed “classification” Mesh classification: unique
association of a mesh entity, Midi,
to a geometric model entity, Gjdj,
where di≤dj is denoted by
Midi Gj
dj indicates the left-hand entity
(or set) represents a portion of the right-hand entity in the discretization
Multiple Midi classified on a Gj
dj
Boundary mesh entities are identifiedin terms of their classifications
Classification critical to supporting adaptive simulations and high level problem definitions
Reverse classification is the set of equal order mesh entities classified on a specific model entity
Simulation Fields
The discretization of the tensors used in the PDE in terms of distribution functions times degrees of freedom written over mesh entities
Field definition Tensor qualification - order, symmetries,
spatial dimension, coordinate system, constraints
Distribution functions - equations, mesh entities they act over, dof used
dof - mesh entities they are on, values after evaluation
ljki
Num
iiljkljk DOFtxNFT
N
...,1
...... ),(
* Typical but not unique form
Fields Relationship With Other Simulation Structures
Relationship with the space/time domain Tensor qualifiers specified over the space/time domain
Specifies the spatial domain dimension for the tensor Constraints (e.g. boundary conditions, initial conditions, divergence free/solenoidal field,
etc.) can be specified with attributes
Relationship with the mesh Information used to discretize the tensor specified over the mesh Distributions -- defined over mesh entities that are the same dimension as the
tensor dimension DOFS -- associated with mesh entities that are the dimension of the distribution
mesh entities and lower
DOF locations on associated entities
5 point FD stencil
FV with 1 distribution per element
C0 FEM quadratic distributions
Functionality of Field Library
Load, save, create and delete functions Point-wise interrogation functions using
combinations of operators Tensor operators
Addition Multiplication Transpose
Differential operators Identity Derivatives (x,y,z,t) Gradient
Integral functions Domain defined through TSTT entity sets
and/or model entities Uses point-wise interrogation operators for
integration of general quantities Mapping fields between meshes
Requires mesh information and above operations
Curl Divergence Laplacian
Locally adapted mesh
Meshes sharing a boundary
TSTT Interface Implementations
Implementations are software libraries that can be used through the TSTT interface Implementations are required to contain a core functionality Includes extensions allowing certain software requirements to be met Multiple implementations can be used simultaneously Tests to verify each implementations compliance with TSTT interface
standard
Implementations of the TSTT Mesh Interface currently available at https://svn.scorec.rpi.edu/wsvn/TSTT FMDB - http://www.scorec.rpi.edu/AOMD FronTier - http://frontier.ams.sunysb.edu GRUMMP - http://tetra.mech.ubc.ca/GRUMMP MOAB - http://cubit.sandia.gov/MOAB NWGrid - http://www.emsl.pnl.gov/nwgrid/index_nwgrid.html
SCOREC Involvement With TSTT
Development of interface standards TSTT Mesh Interface TSTT Geometry Interface TSTT Fields Interface
Software development FMDB (Flexible Mesh DataBase) library which is compliant with the TSTT
Mesh Interface FANS (Field Approximation for Numerical Simulations) library which is
being developed in conjunction with development of the TSTT Fields Interface
Model library which is being developed in conjunction with development of the TSTT Geometry Interface
Notable uses of software Development of adaptive loops for mesh-based simulations Multiphysics simulations
Constructing Adaptive Solution Procedures
Components of an an adaptive mesh control loop High level problem definition (domain and attributes) Domain discretization (mesh generation) Equation discretization Solvers to deal with the large systems of equations Error estimators/indicators Correction indicators Spatial and/or equation discretization enrichment methods
Implementation Tightly coupled using a single set of structures
Advantage: Computationally efficient if done well Disadvantage: Complex algorithm and code development
Loosely coupled building on existing components Advantage: Ability to use existing analysis codes and adaptive tools and to
mix and match tools Disadvantage: Overhead of multiple structures and data conversion
Information needed by an adaptive loop for proper creation of input for a code Overall Domain Definition - Geometric Model
Want a “true definition” of the domain - not a specific approximation Physical Attributes
Sets of tensors needed to qualify the loads, material properties, boundary conditions and initial conditions
Domain Discretizations (meshes) First piece of the two level discretization that is input to analysis codes
Simulation Fields Discretizations The discretized versions of tensor variables defined in terms of distributions
and degrees of freedom over the meshes as dictated by the PDE discretization process performed within the analysis code
The goal of the TSTT development is to provide all of the above information in a uniform manner
Structures/Services for Adaptive Loops
Adaptive Loop Driver Code Input and Output
Input Domain definition
Mathematical problem attributes Adaptive loop attributes/a priori
mesh controls Adaptive goal Geometry approximation
controls for mesh Mesh size limits (memory,
computation constraints) a posteriori mesh controls
Field information Error estimates Convergence rates Adaptive strategies
Model modifications through mesh motion Output
Simulation code input Adaptive loop convergence information Mesh
Simulation Code Input and Output
Input in form manageable by simulation code Sufficient mesh representation
Constructed from complete mesh representation from interface
Usually only nodes and elements Boundary and possibly initial condiitons
Element “sides” and nodal boundary conditions from classification/reverse classification information
Tensor values from field and attributes Other problem attributes
From complete problem definition Ideally code input files do not change
Output Computed field information
May include error estimates/indicators Field information from visualization output may be
sufficient Ideally code output files do not change
Adaptive Mesh Control
Need adaptive mesh control when Discretization is inadequate - mesh refinement Discretization computationally inefficient - mesh coarsening Element shape becomes unacceptable – mesh improvement
Two approaches for h-type mesh adaptation Re-meshing
Define new mesh size field Provide mesh generator with domain definition and mesh size field Map any history dependent solution fields from the old to the new mesh
Local mesh modifications Determine local operations needed to perform desired modifications Execute the local modifications Apply incremental history dependent solution field updates
Mesh Modification Operators
Single step operators Face swap Edge swap Edge collapse Region collapse Edge split Vertex motion Curving interior mesh entity
Compound operators modify two or more entities to allow the modification of the problem entity
Examples of two step compound operators Swap entity A first, and then swap entity B Collapse entity A first, and then collapse entity B Swap entity A first, and then collapse entity B Swap entity A first, and then curve new entity B Swap entity A first, and then reposition vertex B Etc.
Moving refinement vertices to boundary required during mesh modification (see IJNME paper, vol58 pp247-276, 2003 )
Coarse initial mesh and the mesh after multiple refinement/coarsening
Operations to move refinement vertices
Accounting for Curved Domains During Refinement
xy z
Iterations ofadaptations
# ofvertices tobe snapped
# of verticessnapped by a
reposition
# of verticessnapped by local
modifications
# of vertices snappedrequiring local re-
triangulations1 342 204 136 22 485 369 110 63 340 286 52 24 74 34 40 -5 26 3 23 -
Field Transfer on Adapted Meshes
Two approaches Global
Computationally expensive Procedures tend to diffuse information - accuracy loss
Local Performed with each local mesh modification Limited number of elements involved - efficient No accuracy loss with some operations, others easier to control
due to local nature
Edge marked for collapse
Zone updated by the operation
Mesh before collapse Mesh after collapse
Adaptive Loop for Accelerator Design
Collaboration with SLAC (I. Malik, K. Ko and Z. Li) Complex CAD geometry Physics modeling by the Omega3P code from SLAC
Solves Maxwell’s equations in frequency domain Determine power loss for specific modes
High level modeling accuracy needed E.g., 0.1% error in frequency predictions
Adaptive mesh control needed to provide desired accuracy Adaptive loop constructed using unmodified Omega3P
Providing the Adaptive Loop Components
Components added to construct adaptive loop High level problem definition Automatic mesh generation directly from CAD a-posteriori error indication and convergence rate to define mesh size field
a-posteriori error indicator constructed from knowledge of the tensor and its discretized field
General mesh modification to construct requested mesh size field accounting for the CAD geometry
Approach to the coupling of the components Interoperability procedures to provide information needed by components
Geometry - integration to multiple CAD systems Mesh - integration to various mesh generation and modification procedures Field - integration with different analysis procedures
Interoperability procedures used in this example from TSTT SCOREC Simmetrix
Integration of Components
Using geometry operatorsmeans alternate solid modelers can be inserted
Using geometry operatorsmeans alternate solid modelers can be inserted
Using TSTT mesh operatorsmeans alternate mesh generatorsand mesh adaptation procedurescan be inserted
Using TSTT mesh operatorsmeans alternate mesh generatorsand mesh adaptation procedurescan be inserted
Using TSTT field operators allows easy construction of alternative error estimators
Operator Interfaces
Projection-based error estimatorused to construct new mesh sizefield given to mesh modification
Mesh adaptation based on local modification linkeddirectly to CAD
Mesh adaptation based on local modification linkeddirectly to CAD
UnalteredSLAC code
UnalteredSLAC code
Error estimatorsfrom RPI and SLAC
Error estimatorsfrom RPI and SLAC
Adaptive Results for Trispal Model
1.0560
1.0580
1.0600
1.0620
1.0640
1.0660
1.0680
1.0700
1.0720
0 1E+05 2E+05 3E+05 4E+05 5E+05Number of DOF
F (
GH
z)
12000.0
12500.0
13000.0
13500.0
14000.0
0 1E+05 2E+05 3E+05 4E+05 5E+05
Number of DOF
Q
Frequency Convergence Q Convergence
Distribution of Wall-loss
Level 0 Level 2Level 1
Metal Forming Simulation
Large plastic deformation Meshes often become invalid Evolving geometry Evolving contact
Components of automated simulation Commercial analysis engine (DEFORMTM) Monitoring of mesh discretization errors and element shapes Mesh model topology update Construct mesh size field based on discretization errors and
geometry approximation General mesh modification to obtain the desired mesh size field Adjust mesh size and shape to control geometric approximations Local field transfer
Model Topology Update
Geometric components in forming simulations Workpiece Dies Die motions
Model topology needs to be updated Contact conditions change as simulation proceeds Mesh updates require complete model topology Simulation engine tracts only nodal contact
Must update model topology based on this information Model update procedure
Maintain non-manifold model representation Simulation contact information and “mesh geometry” Uses initial classification to build topology and then corrects ambiguities Mesh classified against updated model topology
Mesh modifications controlled Attributes properly associated to mesh
Steering Link Problem
Adapted based on the error indicators on the effective strain
Step 120
Step 160
Step 180
Step 200
Step 220
Step 240
Step 260 Step 279 (Completion)
Summary of TSTT Work
TSTT Mesh 0.6 Interface complete Working document describing the mesh and geometry data models as
well as the overall vision Implementations of TSTT Mesh 0.6 Interface available
https://svn.scorec.rpi.edu/wsvn/TSTT FMDB - http://www.scorec.rpi.edu/AOMD FronTier - http://frontier.ams.sunysb.edu GRUMMP - http://tetra.mech.ubc.ca/GRUMMP MOAB - http://cubit.sandia.gov/MOAB NWGrid - http://www.emsl.pnl.gov/nwgrid/index_nwgrid.html
Preliminary interfaces for geometry, field data, and mesh/geometry data model manager
C, C++, and Fortran language interoperability through SIDL/Babel (CCA) Analyzing performance ramifications of SIDL/Babel language
interoperability tools (joint with the CCA) TSTT Information
http://www.tstt-scidac.org
Adaptive loops built from available components Software for multiple sources combined to create functional adaptive
loops that provide superior results Initial SLAC adaptive loop constructed in less than 3 months
By people with little background with components Included development of an error estimation and correction
indication routines
Approach taken focused on optimizing time to getting adaptive results (not the computing resources needed for the simulation) Did not alter Omega3P or DEFORM codes
Used meshing components and operators to create input files Used field operators to extract results from output file
Closing Remarks on Adaptive Loops