uncertainty quantification framework for modeling...
TRANSCRIPT
![Page 1: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/1.jpg)
Uncertainty Quantification Framework for Modeling Prediction
Michael Frenklach
Columbia UniversityOctober 10, 2014
Supported by: NSF, AFOSR, DOE‐NNSA (PSAAP II)
Collaborators:
‒ Andy Packard‒ W. A. Lester, Jr. (DFT)‒ P. Westmoreland (ALS)‒ N. Slavinskaya (SynGas)‒ R. Feely, P. Seiler, T. Russi, D. Yeates, X. You, F. Lei,
D. Edwards, D. Zubarev, W. Speight
![Page 2: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/2.jpg)
• Introduction: UQ-predictive modeling
• Bound-To-Bound Data Collaboration
• Introductory case: Energetics of water clusters
• Full-blown case: Combustion of natural gas
![Page 3: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/3.jpg)
![Page 4: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/4.jpg)
“Model predicts data” ?
“Model predicts reasonably well the experimental behavior”
“…excellent agreement between model and data.”
“Model falls short in predicting experimental data”
“The model predictions match reasonably well the experimental data”
“The model well predicts the data”
“Model matches the experimental data”
“The prediction matches very well with experimental data”
“Simulation agrees well with the data”
“Good agreement was found between the model and the data”
![Page 5: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/5.jpg)
![Page 6: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/6.jpg)
WWW 2012 Lyon, France
We develop a temporal modeling framework adapted from physics and signal processing …
The results … indicate that by using our framework …. we can achieve significant improvements in prediction compare to baseline models …
![Page 7: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/7.jpg)
•Predictive UQ‐Predictive
•Physics‐based models with the focus on data
•Validation is part of the process
•Dimensionality reduction is part of the process
•Practicality use of surrogate models (Emulators)
•Data/Models• Access, sharing, documentation, …
• Reproducibility
![Page 8: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/8.jpg)
model(dif eq, nonlinear)
![Page 9: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/9.jpg)
model(dif eq, nonlinear)
![Page 10: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/10.jpg)
![Page 11: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/11.jpg)
![Page 12: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/12.jpg)
![Page 13: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/13.jpg)
x2
x3
x1,min x1 x1,max
prior knowledge on parametersexperimental uncertainties
B2B‐DC
y2
y3
L1 y1 U1
HD
─ an optimization‐based framework for combining models and data to ascertain the collective information content
![Page 14: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/14.jpg)
Phys Rev Lett 112: 253003 (2014)
![Page 15: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/15.jpg)
•empirical: force‐field – guessed potential, empirically fitted; …
•semi‐empirical HF – quantum “core” with some terms replaced by parameters fitted to data (AM1, RM1, PM3, PM6, ZINDO, …)
•DFT with fitted parameters: meta‐GGA (Truhlar, M05,M06,M11,…), double‐hybrid DFT (Grimme), ...
•“static” outcome: the optimized model needs (constant) retuning
•the optimum is not unique!•partial loss of information (two‐step process)
DFT HF DFT MP2XC X X C CX C CX1 1E E E E E
trainingdata
parameters
blindprediction
model‐based UQ
![Page 16: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/16.jpg)
GGA HF GGA MP2XC X X C C 1 1E E E E E Model :
use E intervals computed for
dimer, trimer, tetramer, and
pentamer
to predict E interval of hexamer
Data Solve for:
![Page 17: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/17.jpg)
GGA HF GGA MP2XC X X C C1 1E E E E E
VIP
high‐level theory result (or experiment)
Combined Feasible SetHexamer prediction (kcal/mol):Source Min Max Range
Over Feasible Set 267.7 269.7 2.0
Segarra‐Marti et al. 265.9 270.0 4.1
![Page 18: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/18.jpg)
•mixture of mostly methane with other light gases
• lowest emissions among fossil fuels; no soot; smallest carbon footprint
•various, expanding sources (biofuels, artificial synthesis,…)
•plenty and cheap; booming US (and world) economy
• technology issues/needs• varying compositions – hard to categorize empirically
• prediction needs: emissions, combustion efficiency, …
![Page 19: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/19.jpg)
Methane Combustion: CH4 + 2 O2 CO2 + 2 H2O
300+ reactions, 50+ species
Foundation
•A physically‐based model
• The network is complex, but the governing equations (rate laws) are known
•Uncertainty exists, but much is known where the uncertainty lies (rate parameters)
•Numerical simulations with parameters fixed to certain values may be performed “reliably”
• There is an accumulating experimental portfolio on the system
• The model is reduced in size for applications
theoryexperiments
numerical simulations
![Page 20: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/20.jpg)
flow‐reactor measurements
theoreticalrate constants
laboratory flame measurements
PREDICTION:ignition delay in HCCI engine
![Page 21: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/21.jpg)
Dataset unitU = ( U, L, M )
Dataset unitU2 = (U2, L2, M2)
Dataset unitU3 = (U3, L3, M3)
Dataset unitU4 = (U4, L4, M4)
Dataset unitU5 = (U5, L5, M5)
Dataset unitUe = ( Ue, Le, Me )
y1
y2
y3
x1
x2
x3Dataset{Ue}
Model parameters
Feasible set of x, FIf empty, inconsistent, otherwise, consistent
Feasible set of x, FIf empty, inconsistent, otherwise, consistent
Experiment, EL y U
Model, M(x)
Dataset imposes constraints
,e e eL M U e x
xe forms n‐dimensional hypercube
,min ,max:ni i ix x x x
HD
![Page 22: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/22.jpg)
yupper
ylower
yexpt
M(x1,x2)
H
prior knowledge
bounds on x1feasible set
F
![Page 23: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/23.jpg)
a set of individual uncertainties does not represent the true compound uncertainty
a realistic feasible set:
![Page 24: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/24.jpg)
• build surrogate models for individual responses y(rather than for overall objective)
• construct global objective from individual responses (higher fidelity)
modelresponse
ODEModel
x1x2
T P C
y
parameters
2 22 1 20 11 1 2 1,2 1,1 2, 22x x xy x xa a a a a ax x surrogatemodel
2mincomputed observed xall responses
yyw
![Page 25: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/25.jpg)
• build surrogate models for individual responses(rather than for overall objective)
• construct global objective from individual responses (higher fidelity)
modelresponse
ODEModel
x1x2
T P C
y
parameters
2 22 1 20 11 1 2 1,2 1,1 2, 22x x xy x xa a a a a ax x surrogatemodel
2mincomputed observed xall responses
yyw
0
0.2
0.4
0.6
0.8
1
45 2 17 11 3 9 58 1 29 33 47 4 73 82 5 6 98 …
|sensitivity| (×uncertainty)
active variables
dimensionality reduction
![Page 26: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/26.jpg)
• build surrogate models for individual responses(rather than for overall objective)
• construct global objective from individual responses (higher fidelity)
modelresponse
ODEModel
x1x2
T P C
y
parameters
2mincomputed observed xall responses
yyw
dimensionality reduction
flamespeed P2 x1,x2,x7,x23,…•••
ignitiondelay P2 x1,x4,x5,x17,…•••
speciesconc P2 x3,x4,x7,x12,…•••
dimensionality of individual response
dimensionality of optimization
![Page 27: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/27.jpg)
Uncertainty is constrained by:• prior knowledge of parameters, xH, the “H cube” • observed data/models, M(x)D, the “D cube”
Prediction model: f(x)‒establish possible range of f(x), constrained by
: maxx
UL
M x
r f rr x
: minx
UL
M x
p f pp x
min maxx x
M x M x
f x f x
Hard‐to‐solve optimization
Computable bounds, easily verified as valid
Hard‐to‐solve optimization
Computable bounds, easily verified as valid
inner innerouter outer
If f and M are quadratic, then the min and max problems SDPand p’s and r’s bounds are
• computable • easily verified as valid• same for their global sensitivities
![Page 28: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/28.jpg)
all parameters are within prior bounds,
A dataset is consistent if the Feasible Set is nonempty;i.e., there exists a parameter vector that satisfies:
1,min 1 1,max
2,min 2 2,max
x x xx x x
e e eL M U x
all model predictions are within experimental boundsx1
x2
x3
y1
y2
y3
H
D
numerical measure of consistency
1 1 ,
max
e e e
DH
L M U e
C
x
x
![Page 29: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/29.jpg)
0Consistent
Feasible Set
0Inconsistent
Feasible Set
J. Phys. Chem. A 108:9573 (2004)
![Page 30: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/30.jpg)
Wiesner et al. 199627 active variables
Lemon et al. 200334 active variables
0
inconsistent consistent
cons
iste
ncy
mea
sure
+0.24
0.03
J. Phys. Chem. A 110:6803 (2006)
![Page 31: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/31.jpg)
prediction
para
model
moeter
delm
prediction
parameterinterval
uncertainty
ex
p
p
re
er
dict
imeninterval
uncert
io
aintyt
n
![Page 32: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/32.jpg)
1:2experiment
uncertainty
predictioninterva M M M M
L U Ll
U
max min max min
1:2parameter
uncertaint
predictioninterv M M M M
xa
xy
x xl
![Page 33: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/33.jpg)
ofto
i
jYYpredictiuncertainty in
uncertainty in ob servingngSensitivity
Yi Yj
![Page 34: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/34.jpg)
-0.1 0.0 0.1 0.2 0.3
H + O2 → O + OH
OH + CO → H + CO2
CH3 + CH3 → H + C2H5
H + C2H4 + (M) → C2H5 + (M)
H + CH3 + (M) → CH4 + (M)
OH + CH3 → CH2* + H2O
C2H5 + O2 → HO2 + C2H4
H + O2 + H2O → HO2 + H2O
HCO + H2O → H + CO + H2O
H + HCO → H2 + CO
HO2 + CH3 → OH + CH3O
O + H2 → H + OH
H + OH + M → H2O + M
H + HO2 → OH + OH
sensitivity“w.r.t. value” vs “w.r.t. uncertainty”
laminar flame speed in a stoichiometric atmospheric C2H6‐air mixture
J. Phys. Chem. A 112:2579 (2008)
Lower00.20.40.6
1 11 21 31 41 51 61 71
lower0
0.001
0.002
1 11 21 31 41 51 61 71 81 91101
sensitivity of methanedataset consistency
to uncertainty in experimental observations
to uncertainty in model parameters
upper boundlower bound
upper boundlower bound
![Page 35: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/35.jpg)
Initial prediction from prior info
Final prediction
Mp(x1,x2)
![Page 36: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/36.jpg)
p pMM x x
p
p
p
pmax
m
:
n
,
i
e e e e e
M
L
M M
d U
M
M e
x
x
x x
xx
x x
subject tofeasible set
is the range of values Mp takes over the set of feasible values of parameters
prediction interval
![Page 37: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/37.jpg)
35th Combust. Symp. 2014
PREDICTION FEATURE PREDICTION INTERVAL
O Peak Value [2.7, 4.3] × 10‐2
O Peak Location [1.9, 2.2] cmOH Peak Value [3.0, 3.6] × 10‐2
OH Peak Location [1.60, 1.67] cmC2H3 Peak Value [0.09, 1.15] × 10‐4
C2H3 Peak Location [0.6, 3.9 ] × 10‐2 cm
![Page 38: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/38.jpg)
Including Experimental ObservationsOnly prior knowledge
prior knowledge
feasible set
1I Posterior Range
Prior Range
Information Gain:
Int. J. Chem. Kinet. 36:57 (2004)
0
0.2
0.4
0.6
0.8
1
ig.1
a
ig.2
ig.6
b
ig.t2
ig.s
t1b
ig.s
t3b
ig.s
t4b
ch3.
c1b
ch3.
t1b
ch3.
t2
ch3.
t3
ch3.
t4
ch3.
stc7
oh.1
b
oh.3
a
co.t1
a
co.c
1b
oh.3
c
co.t1
c
co.c
1d
oh.s
t8
co.s
c8
bco.
t2
bco.
t4
bco.
t6
bch2
o.t1
bch2
o.t3 f1 f3 f6
stf8
sch.
c11
sch.
c13
nfr1
nfr3 nf7
nf12 nfr5
![Page 39: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/39.jpg)
arg min curexperim
curparameen t
pts
ters
o
:
0
TCC
parametersexpr
c
emen
ur
tssubject to
Given a budget T, determine the best strategy for reducing the uncertainty in model prediction
1a
C
currentcost
uncertainty,
16
18
20
22
05
1015
20
050
100150
200
50
100
200
0
total cost
range predicted for C2H6 flame speed (cm/s)
J. Phys. Chem. A 112:2579 (2008)
![Page 40: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/40.jpg)
Slavinskaya, et al., 2013, 2014
flame speeds ignition delays
![Page 41: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/41.jpg)
Factor
Compute gradients of f(x) at points of
Perform SVD of F; this gives S
Sample r‐subspace ofH to build surrogate design
Tf g Sx x
C2H2 flat flame (ALS)
12 responses, 51 active variables
Yeates et al. 7th US Combust. 2011
While a M(x) formally depends on all n active variables, in reality it mostly vary in r « nlinear combinations of the variables.
For creating a surrogate of M(x) we would like to do the design in the r‐dimensional space.
![Page 42: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/42.jpg)
• Even in case of rigorous Bayesian, use of a prescribed prior (e.g., Gaussian) underestimates the uncertainty in prediction (Phillip Stark, “Constraints versus Priors”, 2012)AND we unlikely to have Gaussian priors!
• Approximations, even seemingly “harmless”, may lead to substantial differences in prediction of uncertainty (Russi et al, Chem. Phys. Lett. 2010)
• Optimization‐based methods, transferring uncertainty in two‐steps — from data to parameters and then from parameters to prediction — necessarily overestimate the predicted uncertainty
Baulch et al. 2005
![Page 43: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/43.jpg)
An ongoing collaborative study withJerome Sacks, National Institute of Statistical SciencesRui Paulo, ISEG Technical University of LisbonGonzalo Garcia‐Donato, Universidad de Castilla‐La Mancha, Spain
B2B‐DC prediction for this blind target is [ 1.89 2.12]
BayesianExample: GRI‐Mech 3.0• 102 active variables• 76 experimental targets• predicting one “blind”
Example: H2/O2• 21 active variables• 12 experimental targets• predicting one “blind”
Bayesian
B2Bsampling F
![Page 44: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/44.jpg)
•current inability of truly predictive modeling– conflicting data in/among sources– poor documentation of data/models– no uncertainty reporting or analysis– not much focus on integration of data
• resistance to data sharing– no personal incentives– no easy‐to‐use technology
•no recognition of the problem
![Page 45: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/45.jpg)
• is mathematically rigorous, numerically efficient, and UQ‐richapproach to analysis of practical systems
• is data‐centric, handles heterogeneous data, and is easily scalable to a large number of data sets
• is scalable to a large number of parameters through Solution Mapping features, combined with the Active Space Discovery
•establishes a clear measure of consistency among data and models, and identifies the cause of inconsistency if detected
•“measures” information content of an experiment‒ assess an impact of a given or planned experiment (“what if”)‒ design new experiments/theory that impact the most
•reduces uncertainties of known and predicts correctly uncertainty of unknown
![Page 46: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/46.jpg)
![Page 47: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/47.jpg)
![Page 48: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/48.jpg)
•What causes/skews model predictiveness?
•Are there new experiments to be performed, old repeated, theoretical studies to be carried out?
•What impact could a planned experiment have?
•What is the information content of the data?
•What would it take to bring a given model to a desired level of accuracy?
![Page 49: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/49.jpg)
from algorithm-centric view
to data-centric view
outputinput codedata data
![Page 50: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/50.jpg)
2
iidata model
k2 k1
devi
atio
ns
![Page 51: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/51.jpg)
(top) Cool et al., Rev. Sci. Instrum. 76, 094102 (2005)(bottom) Courtesy of Sandia CRF ‐ http://www.sandia.gov/ERN/images/CRF‐Science.jpg
Integrated signal
![Page 52: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/52.jpg)
Concentration profile
Data Analysisy = f(x,c)
Calibration, c
Model
Assumptions
Integrated signal
![Page 53: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/53.jpg)
Concentration profile
Data Analysisy = f(x,c)
Calibration, c
Model
Assumptions
Integrated signal
![Page 54: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/54.jpg)
Data Analysisy = f(x,c)
Calibration, c
Model
Integrated signal
Instrumental Model
![Page 55: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/55.jpg)
O, OH, C2H3
– Peak Value– Peak Location
PREDICTION FEATURE PREDICTION INTERVALO Peak Value [2.73, 4.29] × 10‐2
O Peak Location [1.87, 2.20] cmOH Peak Value [2.97, 3.59] × 10‐2
OH Peak Location [1.60, 1.67] cmC2H3 Peak Value [0.09, 1.15] × 10‐4
C2H3 Peak Location [0.60, 3.90] × 10‐2 cm
![Page 56: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/56.jpg)
hypothesis collectionof data model
predictionhypothesis collection
of data model-experiments-theory
![Page 57: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/57.jpg)
hypothesis collectionof data model
predictionhypothesis collection
of data model
SENSITIVITY ANALYSIS:What conditions will maximize sens to k?
![Page 58: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/58.jpg)
hypothesis collectionof data model
predictionhypothesis collection
of data model
UNCERTAINTY QUANTIFICATION:What experiment will be most informative?
![Page 59: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/59.jpg)
?
flame codeinput
conditions
reaction model
validation datathermo and transport data
prediction
![Page 60: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/60.jpg)
Volume of sphereVolume of cube
dimension
![Page 61: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/61.jpg)
pM x pM x
pM x
![Page 62: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/62.jpg)
Data analysis performed
in isolationleads to loss of information
![Page 63: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/63.jpg)
hypothesis|data data|hypothesis hypothesis
![Page 64: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/64.jpg)
priorlikelihoodposterior
hypothesis|data data|hypothesis hypothesis
model /analysis
![Page 65: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/65.jpg)
priorlikelihoodposterior
hypothesis|data data|hypothesis hypothesis
model /analysis
![Page 66: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/66.jpg)
• Quadratic surrogates enable mathematically rigorous, numerically efficient, and UQ‐rich approach of B2B‐DC to practical systems
• Quadratics work in practice because model parameters are limited‒ by physical constraints; e.g., 0 < k < collision limit‒ by reaction theory / chemical analogy‒ by prior experimental / theoretical studies‒ and can be linearized; e.g., by the log transformation
• And if they do not work, then‒ rational quadratics (“native” with B2B framework)‒ a two‐level surrogate approach
first, use machine learning to build “high‐order surrogates”,e.g., Gaussian Process, Kriging, ‐SVM, Polynomial Chaos
then, build/use on‐demand piece‐wise quadratics from the high‐order surrogates
J. Phys. Chem. A 110:6803 (2006)
![Page 67: Uncertainty Quantification Framework for Modeling Predictionccmsc.utah.edu/images/publications/presentation... · •Uncertainty exists, but much is known where the uncertainty lies](https://reader034.vdocuments.net/reader034/viewer/2022043020/5f3c840ee15b077eb946d526/html5/thumbnails/67.jpg)
L y U
y2
y3
Dy1
L ‒ M(x) U +
y = M(x) +
L′ M(x) U′
surrogate model
fitting error
D