sensitivity indices based on a generalized functional anova · sensitivity indices based on a...

FrameworkHierarchically Orthogonal Decomposition

Generalized Sensitivity IndicesEstimation procedureNumerical examples

Sensitivity indices based ona generalized functional ANOVA

Clémentine PRIEURUniversity of Grenoble, France

joint work withGaëlle CHASTAING (Grenoble) and Fabrice GAMBOA (Toulouse)

Raleigh, April 2, 2012

Clémentine PRIEUR University of Grenoble, France Sensitivity indices based on a generalized functional ANOVA

Introduction

Goal : providing tools for performing a sensitivity analysis when thereexist dependences among inputs.

Several approaches :

a variance-based approach (three first talks),a distribution-based approach (last talk).

Just a few words on the second approach as E. Borgonovo’s talk iscancelled.

−→ a distribution-based approach

Introduction

Importance measures relying on the entire output probability distributionfunction : Y = f (X1, . . . ,Xd)

[1] Borgonovo, E. (2007). A new uncertainty importance measure. ReliabilityEngineering and System Safety, 92, 771-784.

δi =12

EXi [Si (Xi )] =12

∫ [∫|fY (y)− fY |Xi=xi (y)|dy

]fXi (xi )dxi

Figure: Graphical interpretation of the separation measure building δi .

Introduction

Importance measures relying on the entire output probability distributionfunction : Y = f (X1, . . . ,Xd)

[1] Borgonovo, E. (2007). A new uncertainty importance measure. ReliabilityEngineering and System Safety, 92, 771-784.

δi =12

EXi [Si (Xi )] =12

∫ [∫|fY (y)− fY |Xi=xi (y)|dy

]fXi (xi )dxi

Figure: Graphical interpretation of the separation measure building δi .

Introduction

Several approaches :a variance-based approach (three first talks),a distribution-based approach (last talk).

Let’s focus on the first approach.

Main idea : decomposition of the function of interest.

Rd → Rx = (x1, . . . , xd) 7→ f (x) = f (x1, . . . , xd)

f (x) = f0 +∑d

i=1 fi (xi ) +∑

1≤i<j≤d fi,j(xi , xj) + · · ·+∑1≤i1<...<il≤d fi1,i2,...,il (xi1 , xi2 , . . . , xil ) + . . .+

f1,2,...,d(x1, x2, . . . , xd) .

Such a decomposition is not unique !

Introduction

Several approaches :a variance-based approach (three first talks),a distribution-based approach (last talk).

Let’s focus on the first approach.

Main idea : decomposition of the function of interest.

Rd → Rx = (x1, . . . , xd) 7→ f (x) = f (x1, . . . , xd)

f (x) = f0 +∑d

i=1 fi (xi ) +∑

1≤i<j≤d fi,j(xi , xj) + · · ·+∑1≤i1<...<il≤d fi1,i2,...,il (xi1 , xi2 , . . . , xil ) + . . .+

f1,2,...,d(x1, x2, . . . , xd) .

Such a decomposition is not unique !

Reminders

Theorem (Hoeffding’s decomposition)

f : [0, 1]d → R,∫[0,1]d

f 2(x)dx <∞

f admits a unique decomposition

f0 +∑d

i=1 fi (xi ) +∑

1≤i<j≤d fi,j(xi , xj) + · · ·+ f1,...,d(x1, . . . , xd)

under the constraintsf0 constant,∀ 1 ≤ s ≤ d, ∀ 1 ≤ i1 < . . . < is ≤ d, ∀ 1 ≤ p ≤ s∫ 1

0fi1,...,is (xi1 , . . . , xis )dxip = 0

Reminders

Let Y = f (X1, . . . ,Xd) with X1, . . . ,Xd i.i.d. r.v. ∼ U ([0, 1]).

From Hoeffding’s decomposition one deduces the followingdecomposition of V (Y ) :

d∑i=1

V (fi (Xi )) +∑

1≤i<j≤n

V (fi,j(Xi ,Xj)) + · · ·+ V (f1,...,d(X1, . . . ,Xd)) .

One has moreover :f0 = E (Y ),∀ i = 1, . . . , d , fi (Xi ) = E (Y |Xi )− f0,∀ 1 ≤ i < j ≤ d , fi,j(Xi ,Xj) = E (Y |Xi ,Xj)− (f0 + fi (Xi ) + fj(Xj)),. . .,f1,...,d(X1, . . . ,Xd) = E (Y |X1, . . . ,Xd)−(f0 +

∑di=1 fi (Xi ) + · · ·+

∑1≤i1<···<id−1≤d fi1,...,id−1(Xi1 , . . . ,Xid−1)

Reminders

d∑i=1

V (fi (Xi )) +∑

1≤i<j≤n

V (fi,j(Xi ,Xj)) + · · ·+ V (f1,...,d(X1, . . . ,Xd)) .

∑di=1 fi (Xi ) + · · ·+

∑1≤i1<···<id−1≤d fi1,...,id−1(Xi1 , . . . ,Xid−1)

Reminders

d∑i=1

V (fi (Xi )) +∑

1≤i<j≤n

V (fi,j(Xi ,Xj)) + · · ·+ V (f1,...,d(X1, . . . ,Xd)) .

∑di=1 fi (Xi ) + · · ·+

∑1≤i1<···<id−1≤d fi1,...,id−1(Xi1 , . . . ,Xid−1)

Reminders

d∑i=1

V (fi (Xi )) +∑

1≤i<j≤n

V (fi,j(Xi ,Xj)) + · · ·+ V (f1,...,d(X1, . . . ,Xd)) .

∑di=1 fi (Xi ) + · · ·+

∑1≤i1<···<id−1≤d fi1,...,id−1(Xi1 , . . . ,Xid−1)

Reminders

d∑i=1

V (fi (Xi )) +∑

1≤i<j≤n

V (fi,j(Xi ,Xj)) + · · ·+ V (f1,...,d(X1, . . . ,Xd)) .

∑di=1 fi (Xi ) + · · ·+

∑1≤i1<···<id−1≤d fi1,...,id−1(Xi1 , . . . ,Xid−1)

Remark

The assumption on the inputs was X1, . . . ,Xd i.i.d. ∼ U ([0, 1]).

i.i.d. independent (i.) identically distributed (i.d.)

Remark : one can easily do without the i.d. assumption. The purposehere is to handle the non-independent case, which is much more difficult.

One can still decompose Y as f0 +∑d

i=1 fi (Xi ) + · · ·+ f1,...,d(X1, . . . ,Xd)with

f0 = E (Y ),∀ i = 1, . . . , d , fi (Xi ) = E (Y |Xi )− f0,∀ 1 ≤ i < j ≤ d , fi,j(Xi ,Xj) = E (Y |Xi ,Xj)− (f0 + fi (Xi ) + fj(Xj)),. . .,f1,...,d(X1, . . . ,Xd) = E (Y |X1, . . . ,Xd)−(f0 +

∑di=1 fi (Xi ) + · · ·+

∑1≤i1<···<id−1≤d fi1,...,id−1(Xi1 , . . . ,Xid−1)

But V (Y )6=∑d

i=1 V (fi (Xi )) + · · ·+ V (f1,...,d(X1, . . . ,Xd)).

Remark

i=1 fi (Xi ) + · · ·+ f1,...,d(X1, . . . ,Xd)with

∑di=1 fi (Xi ) + · · ·+

∑1≤i1<···<id−1≤d fi1,...,id−1(Xi1 , . . . ,Xid−1)

But V (Y )6=∑d

i=1 V (fi (Xi )) + · · ·+ V (f1,...,d(X1, . . . ,Xd)).

Remark

i=1 fi (Xi ) + · · ·+ f1,...,d(X1, . . . ,Xd)with

∑di=1 fi (Xi ) + · · ·+

∑1≤i1<···<id−1≤d fi1,...,id−1(Xi1 , . . . ,Xid−1)

But V (Y )6=∑d

i=1 V (fi (Xi )) + · · ·+ V (f1,...,d(X1, . . . ,Xd)).

Remark

i=1 fi (Xi ) + · · ·+ f1,...,d(X1, . . . ,Xd)with

∑di=1 fi (Xi ) + · · ·+

∑1≤i1<···<id−1≤d fi1,...,id−1(Xi1 , . . . ,Xid−1)

But V (Y )6=∑d

i=1 V (fi (Xi )) + · · ·+ V (f1,...,d(X1, . . . ,Xd)).

What can we do?

Several alternatives were proposed in the literature of the last decade.

These alternatives are reviewed e.g., in the introduction of the followingpaper :

[2] Mara, T. A. and Tarantola, S. (2011). Variance-based sensitivity indices formodels with dependent inputs. Reliability Engineering and System Safety. InPress.http://www.sciencedirect.com/science/journal/aip/09518320

An important finding is (see Li et al.) :"Adopting the same definition of sensitivity indices given by thevariance-based methods for a given subset of inputs can lead tocontributions from other correlated inputs contaminating theresult."

[3] Li, G. et al. (2010). Global Sensitivity Analysis for Systems withIndependent and/or Correlated Inputs. J. Phys. Chem. A, 114, 6022-6032.

a covariance-based approach

Our approach follows some common lines with the one of Li et al., thatare :

the use of an adequate decomposition f =∑C⊆1,...,d fC ,

the variance decomposition of Y makes appear covariance terms ofthe form

Cov (fA(XA), fB(XB))

where A, B ⊆ 1, . . . , d,we introduce new indices reflecting the total, structural andcorrelative contributions for XC , C ⊆ 1, . . . , d.

The main question is how choosing the decomposition. There existseveral choices : High Dimensional Model Representation HDMR (see Liet al., Polynomial Chaos Expansion (see Sudret et al.), HierarchicallyOrthogonal Decomposition (our approach), . . .

The new indices are consistent with the Sobol’s ones if the inputs areindependent.

Contents

1 Framework

2 Hierarchically Orthogonal Decomposition

3 Generalized Sensitivity Indices

4 Estimation procedure

5 Numerical examples

Contents

1 Framework

Framework

Let ν be any σ-finite product measure on Rd ,ν(dx) = ν1(dx1)⊗ · · · ⊗ νd(dxd).

One assumes Y = f (X ) = f (X1, . . . ,Xd) with (X1, . . . ,Xd) ∼ PX andf : Rd → R satisfying

∫Rd f 2(x)PX (dx) < +∞.

Assumption (Dependence structure)

PX << ν , pX (x) =dPX

dν(x)

∃ 0 < M , ∀ C ⊆ 1, . . . , d , pX ≥ M pXC pXC ν a.s. ,

where pC (resp. pC) denotes the marginal density of XC = (Xi , i ∈ C)(resp. XC = (Xi , i /∈ C)).

Framework

Let ν be any σ-finite product measure on Rd ,ν(dx) = ν1(dx1)⊗ · · · ⊗ νd(dxd).

One assumes Y = f (X ) = f (X1, . . . ,Xd) with (X1, . . . ,Xd) ∼ PX andf : Rd → R satisfying

∫Rd f 2(x)PX (dx) < +∞.

Assumption (Dependence structure)

PX << ν , pX (x) =dPX

dν(x)

∃ 0 < M , ∀ C ⊆ 1, . . . , d , pX ≥ M pXC pXC ν a.s. ,

where pC (resp. pC) denotes the marginal density of XC = (Xi , i ∈ C)(resp. XC = (Xi , i /∈ C)).

Framework

A first example in Rd :

• Let ν be a multivariate normal distribution Nd(m,Σ) with Σ diagonal.Let 0 < α < 1 and let PX be the following gaussian mixtureαNd(m,Σ) + (1− α)Nd(µ,Ω), with

(Ω−1 − Σ−1

)positive definite.

Framework

Let now d = 2. Let ν be Lebesgue measure on R2. Let C denote thecopula associated to PX (unique thanks to Sklar’s theorem). The copuladensity of PX is denoted by c .

FX (x1, x2) = C (FX1(x1),FX2(x2))

pX (x1, x2) = c (FX1(x1),FX2(x2)) pX1(x1)pX2(x2)

Proposition (necessary and sufficient conditions for d = 2)

Assumptions (i), (ii) and (iii) below are equivalent.(i) ∃M > 0 such that pX ≥ MpX1pX2 ν a.s.(ii) ∃M > 0 such that ∀ (u, v) ∈ [0, 1]2, c(u, v) ≥ M

(iii) there exist a copula C and 0 < M ≤ 1 such thatC (u, v) = Muv + (1−M)C (u, v)

Framework

Let now d = 2. Let ν be Lebesgue measure on R2. Let C denote thecopula associated to PX (unique thanks to Sklar’s theorem). The copuladensity of PX is denoted by c .

FX (x1, x2) = C (FX1(x1),FX2(x2))

pX (x1, x2) = c (FX1(x1),FX2(x2)) pX1(x1)pX2(x2)

Proposition (necessary and sufficient conditions for d = 2)

Assumptions (i), (ii) and (iii) below are equivalent.(i) ∃M > 0 such that pX ≥ MpX1pX2 ν a.s.(ii) ∃M > 0 such that ∀ (u, v) ∈ [0, 1]2, c(u, v) ≥ M

(iii) there exist a copula C and 0 < M ≤ 1 such thatC (u, v) = Muv + (1−M)C (u, v)

Framework

Remark : the assumption does not depend on the marginal distributions.

Copula-based examples :

? Morgenstern copula Cθ(u, v) = uv (1 + θ(1− u)(1− v)) satisfies (iii)as soon as −1 < θ < 1,

? Archimedian copulas C (u, v) = ϕ−1 (ϕ(u) + ϕ(v)) , u, v ∈ [0, 1] withϕ (the generator) non negative, two times differentiable on [0, 1],ϕ(1) = 0 , ϕ′(u) < 0 and ϕ”(u) > 0 ∀ u ∈ [0, 1].A sufficient condition for (ii) to hold is

∃M1,M2 > 0 such that

−ϕ′(u) ≥ M1 ∀ u ∈ [0, 1]

1ϕ′(u)2

)≥ M2 ∀ u ∈ [0, 1]

Framework

Remark : the assumption does not depend on the marginal distributions.

Copula-based examples :

? Morgenstern copula Cθ(u, v) = uv (1 + θ(1− u)(1− v)) satisfies (iii)as soon as −1 < θ < 1,

? Archimedian copulas C (u, v) = ϕ−1 (ϕ(u) + ϕ(v)) , u, v ∈ [0, 1] withϕ (the generator) non negative, two times differentiable on [0, 1],ϕ(1) = 0 , ϕ′(u) < 0 and ϕ”(u) > 0 ∀ u ∈ [0, 1].A sufficient condition for (ii) to hold is

∃M1,M2 > 0 such that

−ϕ′(u) ≥ M1 ∀ u ∈ [0, 1]

1ϕ′(u)2

)≥ M2 ∀ u ∈ [0, 1]

Framework

0.00.20.40.60.81.0

copula[, 2]

Figure: Independent copula.

0.00.20.40.60.81.0

copula[, 2]

Figure: Morgenstern copula withparameter 0.99.

Framework

0.00.20.40.60.81.0

copula[, 2]

Figure: Frank copula withparameter 30.

0.00.20.40.60.81.0

copula[, 2]

Figure: Gaussian copula.

Contents

1 Framework

Hierarchically Orthogonal Decomposition HOD

Theorem (Generalization of Stone, Hooker)

Under our dependence assumption, there exists a unique decompositionof the form

f (X ) = f0 +∑

i fi (Xi ) +∑

i<j fij(Xi ,Xj) + · · ·+ f1,...,d(X1, . . . ,Xd)

=∑C⊆1,...,d fC(XC)

under the following constraints : ∀ B ⊂ A

E(fA(XA)fB(XB)) =

∫fA(xA)fB(xB)pX (x)ν(dx) = 0 .

[4] Stone, C. J. (1994). The use of Polynomial Splines and their tensor productsin multivariate function estimation. The Annals of Statistics, 22, 1, 118-171.

[5] Hooker, G. (2007). Generalized functional ANOVA diagnostics forhighdimensional functions of dependent variables. Journal of Computationaland Graphical Statistics, 16, 3, 709-732.

Contents

1 Framework

Generalized Sensitivity Indices

Hence we derive

V (Y ) =∑C

[V (fC) +∑

A∩C6=A,C

Cov(fA, fC)]

The generalized sensitivity indice associated with C ⊆ 1, . . . , d is thendefined by

SC =V (fC(XC)) +

∑A∩C6=A,C Cov(fA(XA), fC(XC))

V (Y )

One has moreover ∑C⊆1,...,d

SC = 1 .

For independent inputs one recover Sobol indices.Clémentine PRIEUR University of Grenoble, France Sensitivity indices based on a generalized functional ANOVA

Hence we derive

V (Y ) =∑C

[V (fC) +∑

A∩C6=A,C

Cov(fA, fC)]

SC =V (fC(XC)) +

V (Y )

SC = 1 .

Hence we derive

V (Y ) =∑C

[V (fC) +∑

A∩C6=A,C

Cov(fA, fC)]

SC =V (fC(XC)) +

V (Y )

SC = 1 .

Contents

1 Framework

Estimation procedure

• Case d = 2 :

Define for i = 1, 2H0

i =hi (Xi ) , Eh2

i (Xi ) <∞ , Ehi (Xi ) = 0⊂ L2

(R2,B(R2),PX

Let PH0ibe the orthogonal projection operator onto H0

One can prove that f1, f2 in the HOD of f (X1, . . . ,Xd) is the uniquesolution of the following system :(

Id PH01

)(f1(X1)f2(X2)

(E (Y |X1)− E(Y )E (Y |X2)− E(Y )

)One way to estimate f1 and f2 (and then to deduce f12) is to solve theabove system (e.g., with a Gauss-Seidel algorithm). Conditionalexpectations are estimated by local polynomial regression. Empiricalvariance and covariance estimation are then used to estimate the newsensitivity indices.

• Case d = 2 :

i =hi (Xi ) , Eh2

i (Xi ) <∞ , Ehi (Xi ) = 0⊂ L2

(R2,B(R2),PX

Id PH01

)(f1(X1)f2(X2)

(E (Y |X1)− E(Y )E (Y |X2)− E(Y )

• Case d = 2 :

i =hi (Xi ) , Eh2

i (Xi ) <∞ , Ehi (Xi ) = 0⊂ L2

(R2,B(R2),PX

Id PH01

)(f1(X1)f2(X2)

(E (Y |X1)− E(Y )E (Y |X2)− E(Y )

• Independent Pairs of Correlated Inputs (IPCI) case :

We assume X =(X (1), . . . ,X (p)

)with X (i) = (X2i−1,X2i ), 2p = d . We

assume moreover that the X (i)s are independent random variables.

We first apply Hoeffding’s decomposition for independent r.v. :

f (X ) = f0 +

p∑i=1

fi (X (i)) + · · ·+ f1,...,p(X (1), . . . ,X (p)) .

[6] Jacques, J., Lavergne, C. and Devictor, N. (2006). Sensitivity analysis inpresence of model uncertainty and correlated inputs. Reliability Engineeringand System Safety, 91, 1126-1134.

Then, we apply our HOD to each fi :

fi (X (i)) = ϕ(i)0 + ϕ

(i)1 (X2i−1) + ϕ

(i)2 (X2i ) + ϕ

(i)12(X2i−1,X2i ) .

We define for all i = 1, . . . , p

Si,1 =V (ϕ

(i)1 ) + Cov(ϕ

(i)1 , ϕ

(i)2 )

V (Y )

Si,2 =V (ϕ

(i)2 ) + Cov(ϕ

(i)1 , ϕ

(i)2 )

V (Y )

Si,12 =V (ϕ

(i)12)

V (Y )

Contents

1 Framework

Numerical examples

Y = X1 + X2 + X1X2

αN2(m,Σ) + (1− α)N2(µ,Ω)

with ν ∼ N (m,Σ) and

m = µ = 02, α = 0.2, Σ = I2, Ω =

(0.5 0.40.4 0.5

S1 S2 S12P

new indices 0.42± 0.041 0.41± 0.043 0.17± 0.026 1± 9.10−16

Sobol indices 0.64± 0.045 0.65± 0.044 0.41± 0.038 1.7± 0.09

analytical val-ues

0.39 0.39 0.22 1

Numerical examples

Y = 5X1 + 4X2 + 3X3 + 2X4

X (1) = (X1,X3), X (2) = (X2,X4)

X (i) ∼ 0.2N2(0, I2) + 0.8N2(0,

(v (i)1 ρ(i)

ρ(i) v (i)2

)) with

v (1)1 = v (1)

2 = 0.5, v (2)1 = 0.7, v (2)

2 = 0.7, ρ(1) = 0.4 and ρ(2) = 0.37.

S1 S2 S3 S4 S13 S24

Sensitivity indices

Analytical indexDVP index

Conclusion, perspectives

• Conclusiona hierarchically orthogonal decomposition,new indices, including a correlative term and an uncorrelated term,implementation on IPCI models.

• Perspectivesestimation in a more general case,see e.g., [7] Li, G. and Rabitz, H. (2011). General formulation of HDMRcomponent functions with independent and correlated variables. J. Math.Chem., 50, 1, 99-130.

a complete study including the estimation of the dependencestructure of the inputs.

Thank for listening . . .

Joint MASCOT-SAMO Conferenceorganized by University of Nice, Electricity of France and JRC Ispra

in Nice (FRANCE), July 1st, 2nd, 3rd and 4th 2013

http://www.gdr-mascotnum.fr/rencontres.html

sensitivity indices based on a generalized functional anova · sensitivity indices based on a...

Documents