polynomial chaos based uncertainty...
TRANSCRIPT
Overview UQ Big Picture PCES References
Polynomial Chaos BasedUncertainty Propagation
Lecture 1: Uncertainty and Spectral Expansions
Bert Debusschere†
†[email protected] National Laboratories,
Livermore, CA
Aug 11, 2014 – UQ Summer School, USC
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Acknowledgement
Cosmin Safta Sandia National LaboratoriesKhachik Sargsyan Livermore, CAKenny ChowdharyHabib NajmOmar Knio Duke University, Raleigh, NCRoger Ghanem University of Southern California
Los Angeles, CAOlivier Le Maître LIMSI-CNRS, Orsay, Franceand many others ...
This material is based upon work supported by the U.S. Department of Energy, Officeof Science, Office of Advanced Scientific Computing Research (ASCR), ScientificDiscovery through Advanced Computing (SciDAC) and Applied Mathematics Research(AMR) programs.
Sandia National Laboratories is a multi-program laboratory managed and operated bySandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for theU.S. Department of Energy’s National Nuclear Security Administration under contractDE-AC04-94AL85000.
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Goals of this tutorial
• An overview of the key aspects of uncertaintyquantification
• 2 Lectures• Lecture 1: Context and Fundamentals
• Introduction to uncertainty• The various aspects of UQ• Spectral representation of random variables
• Lecture 2: Forward Propagation and InverseProblems
• Please do not hesitate to ask questions!!
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Outline
1 Tutorial Overview
2 The Many Aspects of Uncertainty Quantification
3 Spectral Representation of Random Variables
4 References
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Aspects of Uncertainty Quantification
• Enabling predictive simulation• Types of uncertainty• UQ methodologies
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Predictive simulations enable science-baseddesign
• Empirical design can be inefficient andcostly• Trial and error does not work well for complex
systems• Experiments not always feasible or
permissible• Reliability of nuclear weapons• Climate change mitigation approaches
• Predictive simulations provide insight intothe physics that drive complex systems• Identification of key mechanisms• Allows for rigorous optimization strategies
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Model validation requires targeted experimentsModel validation requires targeted experiments
DLR-A Flame: Red = 15,200!Fuel: 22.1% CH4, 33.2% H2, 44.7% N2!
Coflow: 99.2% Air, 0.8% H2O!Detailed Chemistry and Transport: 12-Step
Mechanism (J.-Y. Chen, UC Berkeley)!
Experiment!
LES!
x/d = 10!
MEAN!RMS!
Temperature!
Mixture Fraction!
Comparisons with 1D Raman/Rayleigh/CO-LIF
line images (Barlow et al. )!
H2O Mass Fraction!
CO Mass Fraction!
Large Eddy Simulation (LES) validation on turbulent flame (courtesy J. Oefelein, SNL)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Predictive simulation requires careful assessment ofall sources of error and uncertainty
• Numerical errors• Grid resolution• Time step• Time integration order• Spatial derivative order• Tolerance on iterative solvers• Condition number of matrices
• Uncertainties• Initial and boundary conditions• Model parameters• Model equations• Coupling between models• Sampling noise in particle models• Stochastic forcing terms
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Governing equations for LES simulation of multiphasereacting flow
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Constitutive models provide closure for governingequations
Smagorinsky sub-grid scale model
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Dynamic modeling and reacting flows involveadditional complexity
Does additional complexity provide more accuracy and less uncertainty?Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Experimental data is used to calibrate models
Predictive Simulation
Theory
Experiments
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
UQ methods extract information from all sources toenable predictive simulation
Predictive Simulation
Theory
Experiments
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)p(λ, I)
P(D)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)p(λ, I)
P(D)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)p(λ, I)
P(D)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)p(λ, I)
P(D)
Debusschere – SNL UQ
Inference
Forward Propagation
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ• UQ not just about propagating uncertainties• The term UQ covers a wide range of methods
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
UQ assesses confidence in model predictions andallows resource allocation for fidelity improvements
• Parameter inference• Determine parameters from data• Characterize uncertainties in inferred parameters
• Propagate input uncertainties through computationalmodels• Account for uncertainty from all sources• Resolve coupling between sources
• Analysis• Sensitivity analysis• Attributions
• Model calibration, validation, selection, averaging
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Types of uncertainty
• Epistemic uncertainty• Variable has one particular value, but it is not known• Reducible: by taking more measurements, we can
get to know the value of the variable better• Examples
• The mass of the planet Neptune• Ocean temperature at a particular point and time
• Aleatory uncertainty• Intrinsic or inherent uncertainty: variable is random;
different value each time it is observed• Irreducible: taking more measurements will not
reduce uncertainty in the value of the variable• Examples:
• Collisions interactions in molecular systems• Sampling noise
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Point of view matters
• Epistemic versus aleatory sometimes depends onscale level• In fully resolved turbulent simulations, velocity field
near a wall is deterministic• In mesoscale channel flow, near wall velocity field is
turbulent forcing term (aleatory)• In macroscale channel flow, near wall velocity field
provides friction term (epistemic)
• These lectures follow the Bayesian view: probabilityrepresents the degree of belief in the value of avariable
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Many UQ approaches are available, filling specificneeds
Estimation of model/parametric uncertainty• Expert opinion, data collection• Regression analysis, fitting, parameter estimation• Bayesian inference of uncertain models/parameters
Forward propagation of uncertainty in models• Local sensitivity analysis (SA) and error propagation• Fuzzy logic; Evidence theory — interval math• Probabilistic framework — Global SA / stochastic UQ
• Random sampling, statistical methods (e.g.Monte-Carlo)
• Spectral Polynomial Chaos (PC) Galerkin methods– Intrusive or non-intrusive
• Collocation, interpolants, regression, ... PC/otherDebusschere – SNL UQ
Overview UQ Big Picture PCES References
Outline
1 Tutorial Overview
2 The Many Aspects of Uncertainty Quantification
3 Spectral Representation of Random Variables
4 References
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
This section focuses on representation of randomvariables with Polynomial Chaos Expansions (PCEs)
Predictive Simulation
Theory
Experiments
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)p(λ, I)
P(D)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)p(λ, I)
P(D)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)p(λ, I)
P(D)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)p(λ, I)
P(D)
Debusschere – SNL UQ
Inference
Forward Propagation
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ
Introduction Introduction UQ Software PCES Summary References
Formulation
dudt
= f (u;λ)
P(λ|D, I) =P(D|λ, I)P(λ, I)
P(D)
λ u
P(u) P(λ)
Debusschere – SNL UQ
• PCEs are a way to compactly represent randomvariables
• Often used as a way to characterize parametricuncertainties
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Polynomial Chaos Expansions (PCEs)
• Representation of random variables with PCEs• Projection of random variables onto PCEs• PC basis types• Convergence of PCEs• Analysis of PCEs
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Polynomial Chaos Expansions Represent RandomVariables
u =P∑
k=0
ukψk (ξ)
• u: Random Variable (RV)represented with 1D PCE
• uk : PC coefficients(deterministic)
• ψk : 1D Hermite polynomialof order k
• ξ: Gaussian RV0.0 0.5 1.0 1.5 2.0
u0
1
2
3
4
5
Prob. Dens. [-]
u = 0.5 + 0.2ψ1(ξ) + 0.1ψ2(ξ)Expansion in terms of functions of random variablesmultiplied with deterministic coefficients• Set of deterministic PC coefficients fully describes RV• Separates randomness from deterministic dimensions
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PCEs are a functional map from standard RVs to therepresented RV
4 2 0 2 40.5
0
0.5
1
1.5
4 2 0 2 40
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.5
00.5
11.5
0
0.51
1.52
2.5
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
u = 0.5 + 0.2ψ1(ξ) + 0.01ψ2(ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
One-Dimensional Hermite Polynomials
ψ0(ξ) = 1
ψk (ξ) = (−1)keξ2/2 dk
dξk e−ξ2/2, k = 1,2, . . .
ψ1(ξ) = x , ψ2(ξ) = ξ2 − 1, ψ3(ξ) = ξ3 − 3ξ, . . .
The Hermite polynomials form an orthogonal basis over
[−∞,∞] with respect to the inner product
〈ψiψj〉 ≡1√2π
∫ ∞−∞
ψi(ξ)ψj(ξ)w(ξ)dξ = δij⟨ψ2
i
⟩where w(ξ) = e−ξ2/2 is the weight function.Note that e−ξ
2/2√
2πis the density of a standard normal
random variableDebusschere – SNL UQ
Overview UQ Big Picture PCES References
Multidimensional Hermite Polynomials
The multidimensional Hermite polynomial Ψi(ξ1, . . . , ξn) isa tensor product of the 1D Hermite polynomials, with asuitable multi-index αi = (αi
1, αi2, . . . , α
in),
Ψi(ξ1, . . . , ξn) =n∏
k=1
ψαik(ξk )
For example, 2D Hermite polynomials:i p Ψi αi
0 0 1 (0,0)1 1 ξ1 (1,0)2 1 ξ2 (0,1)3 2 ξ2
1 − 1 (2,0)4 2 ξ1ξ2 (1,1)5 2 ξ2
2 − 1 (0,2)... ... ... ...
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Multidimensional Polynomial Chaos Expansion
u =P∑
k=0
uk Ψk (ξ1, . . . , ξn)
• u: Random Variable (RV) represented with multi-D PCE• uk : PC coefficients (deterministic)• Ψk : Multi-D Hermite polynomials up to order p• ξi : Gaussian RV• n: Dimensionality of stochastic space• P + 1: Number of PC terms: P + 1 = (n+p)!
n!p!
The number of dimensions represents the number ofindependent inputs, degrees of freedom for u• E.g. one stochastic dimension per uncertain model
parameter• Contributions from each uncertain input can be identified
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Polynomial Chaos Expansions (PCEs)
• Representation of random variables with PCEs• Projection of random variables onto PCEs• PC basis types• Convergence of PCEs• Analysis of PCEs
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Obtaining PC coefficients for arbitrary randomvariables
• This is a hard problem in general• Random Variable can be specified in a variety of
ways, but often incomplete• Probability Density Function (PDF)• Samples• Expert opinion (e.g. “somewhere between 2 and 4”)
• Particular case of a random variable specified by aPDF is generally tractable
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
The PC basis functions are orthogonal with respect tothe probability measure of the associated RVs.
〈ΨiΨj〉 ≡∫. . .
∫Ψi(ξ)Ψj(ξ)g(ξ1)g(ξ2) · · · g(ξn)dξ1dξ2 · · · dξn
=n∏
k=1
⟨ψαi
k(ξk )ψ
αjk(ξk )
⟩= δij
⟨Ψ2
i
⟩where,
g(ξ) =e−ξ2/2√
2π
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Orthogonality enables a Galerkin projection todetermine the PC coefficients.
u1
E
u
u
uo
ψ1
ψo
1
u1
E
u
u
uo
ψ1
ψo
1
u1
E
u
u
uo
ψ1
ψo
1
u1
E
u
u
uo
ψ1
ψo
1
u1
E
u
u
uo
ψ1
ψo
1
u1
E
u
u
uo
ψ1
ψo
1
u1
E
u
u
uo
ψ1
ψo
�
1
u ≈P∑
k=0
uk Ψk ⇒ 〈Ψiu〉 =P∑
k=0
uk 〈ΨiΨk〉 = ui⟨
Ψ2i
⟩⇒ ui =
〈uΨi〉⟨Ψ2
i
⟩Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Galerkin projection requires functional relationshipbetween random variable and germ of PCE.
0 2 4 6 80
1
2
3
4
5
ξ
p(ξ)
u(ξ)
p(u)
η
1
ξ
p(ξ)
u(ξ)
p(u)
η
u
1
u =P∑
k=0
uk Ψk (ξ) ⇒ ui =〈u Ψi(ξ)〉⟨
Ψ2i
⟩Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Cumulative Distribution Function (CDF) maps arbitraryrandom variable to a uniform random variable
• Consider u with PDF p(u)
• CDF of u is given by
F (u) =
∫ u
−∞p(s)ds
• F (u) maps u to η, uniformon [0,1]
0 2 4 6 80
0.2
0.4
0.6
0.8
1
0 2 4 6 80
1
2
3
4
5
ξ
p(ξ)
u(ξ)
p(u)
η
1
ξ
p(ξ)
u(ξ)
p(u)
η
u
1
ξ
p(ξ)
u(ξ)
p(u)
η
u
1
ξ
p(ξ)
u(ξ)
p(u)
η
1
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Inverse CDF mapping enables Galerkin Projection
• η = F (u)
• η = Φ(ξ)maps uniformη to normalRV ξ
• u =F−1(Φ(ξ))
4 2 0 2 40
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
4 2 0 2 40
0.2
0.4
0.6
0.8
1
0 2 4 6 80
0.2
0.4
0.6
0.8
1
0 2 4 6 80
1
2
3
4
5
ξ
p(ξ)
u(ξ)
p(u)
η
1
ξ
p(ξ)
u(ξ)
p(u)
η
1
ξ
p(ξ)
u(ξ)
p(u)
η
1
ξ
p(ξ)
u(ξ)
p(u)
η
1
ξ
p(ξ)
u(ξ)
p(u)
η
u
1
ξ
p(ξ)
u(ξ)
p(u)
η
u
1
ξ
p(ξ)
u(ξ)
p(u)
η
1
ξ
p(ξ)
u(ξ)
p(u)
η
1
〈u Ψi(ξ)〉 =
∫F−1(Φ(ξ))︸ ︷︷ ︸
u
Ψi(ξ)w(ξ)dξ
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: PC Expansion for a Normal RV
• Wiener-HermitePCE constructedfor a Normal RV
• PCE-sampledPDF superposedon true PDF
• Order = 1 0
0.1
0.2
0.3
0.4
0.5
5 6 7 8 9 10 11 12 13 14 15
Normal; WH PC order = 1
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ
+ u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: PC Expansion for a Normal RV
• Wiener-HermitePCE constructedfor a Normal RV
• PCE-sampledPDF superposedon true PDF
• Order = 2 0
0.1
0.2
0.3
0.4
0.5
5 6 7 8 9 10 11 12 13 14 15
Normal; WH PC order = 2
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ + u2(ξ2 − 1)
+ u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: PC Expansion for a Normal RV
• Wiener-HermitePCE constructedfor a Normal RV
• PCE-sampledPDF superposedon true PDF
• Order = 3 0
0.1
0.2
0.3
0.4
0.5
5 6 7 8 9 10 11 12 13 14 15
Normal; WH PC order = 3
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ)
+ u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: PC Expansion for a Normal RV
• Wiener-HermitePCE constructedfor a Normal RV
• PCE-sampledPDF superposedon true PDF
• Order = 4 0
0.1
0.2
0.3
0.4
0.5
5 6 7 8 9 10 11 12 13 14 15
Normal; WH PC order = 4
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: PC Expansion for a Normal RV
• Wiener-HermitePCE constructedfor a Normal RV
• PCE-sampledPDF superposedon true PDF
• Order = 5 0
0.1
0.2
0.3
0.4
0.5
5 6 7 8 9 10 11 12 13 14 15
Normal; WH PC order = 5
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: WH PCE for a Normal RV
0
2
4
6
8
10
0 1 2 3 4 5
Normal RV
WH PC order54321
PC mode amplitudesu0–u5
6
7
8
9
10
11
12
13
14
-4 -3 -2 -1 0 1 2 3 4
Normal RV
WH PC order12345
PC function u(ξ)Order 1–5
• First order Wiener-Hermite PCE exact for a normalRV
• Linear function of ξ• Higher order terms are negligible
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: WH PCE for a Lognormal RV
• Wiener-HermitePCE constructedfor a LognormalRV
• PCE-sampledPDF superposedon true PDF
• Order = 1 0
0.2
0.4
0.6
0.8
1
1.2
1.4
0 1 2 3 4 5 6
Lognormal; WH PC order = 1
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ
+ u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: WH PCE for a Lognormal RV
• Wiener-HermitePCE constructedfor a LognormalRV
• PCE-sampledPDF superposedon true PDF
• Order = 2 0
0.2
0.4
0.6
0.8
1
1.2
1.4
0 1 2 3 4 5 6
Lognormal; WH PC order = 2
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ + u2(ξ2 − 1)
+ u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: WH PCE for a Lognormal RV
• Wiener-HermitePCE constructedfor a LognormalRV
• PCE-sampledPDF superposedon true PDF
• Order = 3 0
0.2
0.4
0.6
0.8
1
1.2
1.4
0 1 2 3 4 5 6
Lognormal; WH PC order = 3
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ)
+ u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: WH PCE for a Lognormal RV
• Wiener-HermitePCE constructedfor a LognormalRV
• PCE-sampledPDF superposedon true PDF
• Order = 4 0
0.2
0.4
0.6
0.8
1
1.2
1.4
0 1 2 3 4 5 6
Lognormal; WH PC order = 4
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: WH PCE for a Lognormal RV
• Wiener-HermitePCE constructedfor a LognormalRV
• PCE-sampledPDF superposedon true PDF
• Order = 5 0
0.2
0.4
0.6
0.8
1
1.2
1.4
0 1 2 3 4 5 6
Lognormal; WH PC order = 5
u =P∑
k=0
uk Ψk (ξ)
= u0 + u1ξ + u2(ξ2 − 1) + u3(ξ3 − 3ξ) + u4(ξ4 − 6ξ2 + 3)
+ u5(ξ5 − 10ξ3 + 15ξ)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PC Illustration: WH PCE for a Lognormal RV
0
0.2
0.4
0.6
0.8
1
1.2
1.4
0 1 2 3 4 5
Lognormal RV
WH PC order54321
PC mode amplitudesu0–u5
-4
-2
0
2
4
6
8
10
12
14
16
-4 -3 -2 -1 0 1 2 3 4
Lognormal RV
WH PC order12345
PC function u(ξ)Order 1–5
• Fifth-order Wiener-Hermite PCE represents the givenLognormal well
• Higher order terms are negligible
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Polynomial Chaos Expansions (PCEs)
• Representation of random variables with PCEs• Projection of random variables onto PCEs• PC basis types• Convergence of PCEs• Analysis of PCEs
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Generalized Polynomial Chaos
PC Type Domain Density w(ξ) Polynomial Free parameters
Gauss-Hermite (−∞,+∞) 1√2π
e−ξ22 Hermite none
Legendre-Uniform [−1, 1] 12 Legendre none
Gamma-Laguerre [0,+∞) xαe−ξ
Γ(α+1)Laguerre α > −1
Beta-Jacobi [−1, 1] (1+ξ)α(1−ξ)β
2α+β+1B(α+1,β+1)Jacobi α > −1, β > −1
Inner product: 〈ψiψj〉 ≡∫ b
a ψi(ξ)ψj(ξ)w(ξ)dξ = δij⟨ψ2
i
⟩• Wiener-Askey scheme provides a hierarchy of possible
continuous PC bases [Xiu and Karniadakis, 2002]• Legendre-Uniform is special case of Beta-Jacobi
• Input parameter domain often dictates the mostconvenient choice of PC
• Polynomials can also be tailored to be orthogonal w.r.t.chosen, arbitrary density
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Normal Distribution
4 3 2 1 0 1 2 3 4x
0.0
0.1
0.2
0.3
0.4
0.5
PD
F
• Most commonly used density in PCEs• Support on (−∞,+∞)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Uniform Distribution
1.5 1.0 0.5 0.0 0.5 1.0 1.5x
0.0
0.2
0.4
0.6
0.8
1.0
PD
F
• Appropriate for variables with sharp bounds on theirdistribution
• Support on [−1,1]
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Gamma Distribution
0 2 4 6 8 10 12 14 16x
0.0
0.2
0.4
0.6
0.8
1.0
PD
F
α=0
α=1
α=2
α=3
α=4
• Useful to represent quantities that are strictly positive• Support on [0,+∞)
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Beta Distribution
1.0 0.5 0.0 0.5 1.0x
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
PD
F
α=β=−0.5
α=β=0.0
α=β=1.0
α=4,β=0
α=0,β=2
α=1,β=4
• Good for quantities that vary between set boundaries• Can be tailored to preferentially weight some areas• Support on [−1,1]
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Polynomial Chaos Expansions (PCEs)
• Representation of random variables with PCEs• Projection of random variables onto PCEs• PC basis types• Convergence of PCEs• Analysis of PCEs
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
More Formally: Probability Spaces and RandomVariables
• Let (Θ,S,P) be a probability space• Θ is an event space• S is a σ-algebra on Θ• P is a probability measure on (Θ,S)
• Random variables are functions Ξ : Θ→ R with ameasure corresponding to their image:• if Ξ−1(A) ∈ S, then define µ(A) = P(Ξ−1(A))• p(ξ) = dµ/dξ: the density of the random variable Ξ
(with respect to Lebesgue measure on R)• Expectation: 〈f 〉 =
∫f dµ =
∫f p(ξ) dξ
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Convergence of PC expansions
• General convergence theorems are subject ofongoing research
• Depends on the underlying random variables ξ• Wiener-Hermite Chaos has been well-studied• Generalized PC less so
• Ernst et al. 2011:• Let ξ : Θ→ RN such that for i = 1, . . . ,N eachξi : Θ→ R, be a set of random variables
• S(ξ): σ-algebra generated by the set ξ• L2(Θ,S(ξ),P): Hilbert space of real random variables
defined on (Θ,S(ξ),P) with finite second moments• Any random variable in this σ-algebra can be
represented with a Polynomial Chaos expansion withgerm ξ
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Convergence of PC expansions
• Proof does not state that any RV with finite variancecan be represented with a PCE to arbitrary precision• In practice, one rarely knows how many degrees of
freedom a RV has• Also, RV specification is rarely complete• In an engineering sense, the choice of the germ and
the PCE order is seen as a model choice to representwhat is known about a RV
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
How do I know my PCE is converged?
• Approximation error in PCE is topic of a lot ofresearch
• Rules of thumb:• Higher order PC coefficients should decay• Increase order until results no longer change• Not always fail-proof ...
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Polynomial Chaos Expansions (PCEs)
• Representation of random variables with PCEs• Projection of random variables onto PCEs• PC basis types• Convergence of PCEs• Analysis of PCEs
• Moments• Probability Density Functions
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Moments of RVs described with PCEs
u =P∑
k=0
uk Ψk (ξ)
• Expectation: 〈u〉 = u0
• Variance σ2
σ2 =⟨
(u − 〈u〉)2⟩ =
⟨(
P∑k=1
uk Ψk (ξ))2
⟩
=
⟨P∑
k=1
P∑j=1
ujuk Ψj(ξ)Ψk (ξ)
⟩
=P∑
k=1
P∑j=1
ujuk 〈Ψj(ξ)Ψk (ξ)〉 =P∑
k=1
u2k
⟨Ψk (ξ)2⟩
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Kernel Density Estimation to Get Probability DensityFunction of Random Variable Corresponding to PCEs
• PCE u =∑P
k=0 uk Ψk (ξ) is cheap to sample• Brute-force sampling and bin samples into histogram• Use Kernel Density Estimation (KDE) to get smoother
PDF with fewer samples ui
PDF(u) =1
Nsh
Ns∑i=1
K(
u − ui
h
)K is the kernel, h is the bandwidth.
• Gaussian KDE is commonly used with
K (x) = 1√2π
e−x22 , leading to
PDF(u) =1√
2πNsh
Ns∑i=1
exp(−(u − ui)2
2h2
)Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Comparison of histograms and KDE
Source Wikipedia, Drleft; licensed under the Creative Commons Attribution-ShareAlike
3.0 License
• Bandwidth h needs to be chosen carefully to avoidoversmoothing
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
KDE requires judicious choice of bandwidth h
−5 −4 −3 −2 −1 0 1 2 3 4
Random variable
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
PD
F
True PDFw=wopt=0.16
w=2wopt
w=wopt/2
w=wopt/4
binning
• Scott’s rule-of-thumb optimal for Gaussian RV: h = N− 1d+4
• Smaller h makes KDE more accurate, but more noisy
• Binning is similar to smaller h case, for proper choice of Nbins
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
KDE requires judicious choice of bandwidth h
−6 −4 −2 0 2 4 6
Random variable
0.00
0.05
0.10
0.15
0.20
0.25
0.30
PD
F
True PDFw=wopt=0.16
w=2wopt
w=wopt/2
w=wopt/4
binning
• Scott’s rule-of-thumb optimal for Gaussian RV: h = N− 1d+4
• Smaller h makes KDE more accurate, but more noisy
• Binning is similar to smaller h case, for proper choice of Nbins
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
KDE requires judicious choice of bandwidth h
−10 −5 0 5 10
Random variable
0.00
0.05
0.10
0.15
0.20
0.25
0.30
PD
F
True PDFw=wopt=0.16
w=2wopt
w=wopt/2
w=wopt/4
binning
• Scott’s rule-of-thumb optimal for Gaussian RV: h = N− 1d+4
• Smaller h makes KDE more accurate, but more noisy
• Binning is similar to smaller h case, for proper choice of Nbins
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Extra Material
• Analytical approach to obtain PDF of random variable
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Probability Density Function of Random VariableCorresponding to PCEs – 1
• Assume one-dimensional PCE
u =P∑
k=0
uk Ψk (ξ) = f (ξ)
• To evaluate PDF of u: pu(·)
pu(u)du =N∑i
pξ(ξ i)dξ
where ξ1, . . . , ξN are the N roots of f (ξ)− u = 0Many possible ξ may give you this particular u and allof them contribute to the probability density at u.
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Probability Density Function of Random VariableCorresponding to PCEs – 2
• More compactly
pu(u) =∑ξ∈Ru
pξ(ξ)
|Df (ξ)|
where Ru = ξ : f (ξ)− u = 0 and |Df (ξ)| = |df/dξ|evaluated at ξ
• Hard to generalize to multi-D PC expansions
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PCE to PDF for monotonic mapping between ξ and u
pu(u) =∑ξ∈Ru
pξ(ξ)
|Df (ξ)|
Ru = ξ : f (ξ)− u = 0|Df (ξ)| = |df/dξ|
4 2 0 2 40.5
0
0.5
1
1.5
4 2 0 2 40
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
0.5
00.5
11.5
0
0.51
1.52
2.5
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
PCE to PDF for non-monotonic mapping between ξand u
pu(u) =∑ξ∈Ru
pξ(ξ)
|Df (ξ)|
Ru = ξ : f (ξ)− u = 0|Df (ξ)| = |df/dξ| 4 2 0 2 4
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
02
46
0
0.51
1.52
2.5 4 2 0 2 4
0
2
4
6
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
ξ
p(ξ)
u(ξ)
p(u)
u
1
Debusschere – SNL UQ
Overview UQ Big Picture PCES References
Further Reading
• N. Wiener, “Homogeneous Chaos", American Journal of Mathematics, 60:4, pp. 897-936, 1938.
• M. Rosenblatt, “Remarks on a Multivariate Transformation", Ann. Math. Statist., 23:3, pp. 470-472, 1952.
• R. Ghanem and P. Spanos, “Stochastic Finite Elements: a Spectral Approach", Springer, 1991.
• O. Le Maître and O. Knio, “Spectral Methods for Uncertainty Quantification with Applications toComputational Fluid Dynamics", Springer, 2010.
• D. Xiu, “Numerical Methods for Stochastic Computations: A Spectral Method Approach", Princeton U.Press, 2010.
• O.G. Ernst, A. Mugler, H.-J. Starkloff, and E. Ullmann, “On the convergence of generalized polynomialchaos expansions,” ESAIM: M2AN, 46:2, pp. 317-339, 2011.
• D. Xiu and G.E. Karniadakis, “The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations",SIAM J. Sci. Comput., 24:2, 2002.
• Le Maître, Ghanem, Knio, and Najm, J. Comp. Phys., 197:28-57 (2004)
• Le Maître, Najm, Ghanem, and Knio, J. Comp. Phys., 197:502-531 (2004)
• B. Debusschere, H. Najm, P. Pébay, O. Knio, R. Ghanem and O. Le Maître, “Numerical Challenges in theUse of Polynomial Chaos Representations for Stochastic Processes", SIAM J. Sci. Comp., 26:2, 2004.
• S. Ji, Y. Xue and L. Carin, “Bayesian Compressive Sensing", IEEE Trans. Signal Proc., 56:6, 2008.
• A. Saltelli and S. Tarantola and F. Campolongo and M. Ratto, “Sensitivity Analysis in Practice: A Guide toAssessing Scientific Models”. John Wiley & Sons, 2004.
• K. Sargsyan, B. Debusschere, H. Najm and O. Le Maître, “Spectral representation and reduced ordermodeling of the dynamics of stochastic reaction networks via adaptive data partitioning". SIAM J. Sci.Comp., 31:6, 2010.
• K. Sargsyan, B. Debusschere, H. Najm and Y. Marzouk, “Bayesian inference of spectral expansions forpredictability assessment in stochastic reaction networks," J. Comp. Theor. Nanosc., 6:10, 2009.
• D.S. Sivia, “Data Analysis: A Bayesian Tutorial,” Oxford Science, 1996.
Debusschere – SNL UQ