1

Use of sensitivity analysis in hydrological modeling

Thorsten Wagener & Francesca Pianosi

thorsten.wagener@bristol.ac.uk

NE/J017450/1

Bristol, UK

2

We have a growing number of Engineering & Geography Water Staff at Bristol

3

I broke my talk into three parts

1.  Sensitivity Analysis For Everybody (SAFE)

2.  Long term recharge sensitivity

3.  Short term sensitivity to data and parameters

4

SENSITIVITY ANALYSIS FOR EVERYBODY (SAFE)

Pianosi, F., Sarrazin, F. and Wagener, T. (2015). A Matlab toolbox for Global Sensitivity Analysis. Environmental Modelling & Software, 70. http://dx.doi.org/10.1016/j.envsoft.2015.04.009

5

Sensitivity Analysis (SA) is a set of mathematical techniques to investigate how the variation in the output of a numerical model can be attributed to variations of its inputs

6

boundary conditions

parameters input

forcing

model

output

Response (output) Factor

(input)

GSA provides a formal, structured approach to: > support model calibration and verification > investigate propagation of uncertainty through the model > identify dominant controls of the model (system)

7

The question in SA is not “how well does the model predict?” but rather “why does the model predict so?”

X-Ray Vision: Fish Inside out: http://www.mnh.si.edu/exhibits/x-ray-vision/

We can classify SA methods by computational demand and purpose

8

Pianosi et al. (In Review) Env. Mod. & Software

We can classify Sensitivity Analysis methods by computational demand and purpose

9

Pianosi et al. (In Review) Env. Mod. & Software

The Matlab SAFE Toolbox

10

http://bristol.ac.uk/cabot/resources/safe-toolbox/

•  Developed at University of Bristol within the NERC-funded CREDIBLE Project on Uncertainty and Risk in Natural Hazard assessment [NE/J017450/1] credible.bris.ac.uk/about-us/

•  Freely available for academic, non-commercial purpose since December, 2014 •  Works under Matlab, Octave and R on Windows, Linux and Mac OS X

•  Currently implemented methods: –  - EET (Morris method)

- Variance-Based (Sobol’ method) - FAST - Regional Sensitivity Analysis - PAWN - DYNIA

11

modular structure à facilitates

multi-method approach

minimum dependency on Matlab version, etc. à reduce obsolescence

TS DDF CFR CWH BETA LP FC PERC K0 K1 K2 UZL MB0

0.2

0.4

0.6

0.8

1

Sensitivity

many visualization functions

more comments than commands

tutorial scripts (workflows) to get started

à learn by doing

functions to assess

robustness and convergence

http://bristol.ac.uk/cabot/resources/safe-toolbox/

Features

GSA steps folders in SAFE Toolbox

SAMPLING INPUT SPACE

GSA steps X!

12…

N1 2 … M

sampling"input samples functions for generic sampling strategies

(e.g. Latin Hypercube) and ad hoc sampling (e.g. One-At-the-Time)

folders in SAFE Toolbox

MODEL EVALUATION

SAMPLING INPUT SPACE

GSA steps X!

12…

N1 2 … M

sampling"

Y!12…

N1 … P

input samples

output samples

functions for generic sampling strategies (e.g. Latin Hypercube) and ad hoc sampling (e.g. One-At-the-Time)

folders in SAFE Toolbox

(*)

15

POST PROCESSING

MODEL EVALUATION

SAMPLING INPUT SPACE

Elementary Effects Test

Regional Sensitivity Analysis

Variance-Based Sensitivity Analysis

…

GSA steps

methods

X!12…

N1 2 … M

S!1…P1 2 … M

sampling"

EET"

RSA"

VBSA"

visualization"

Y!12…

N1 … P

input samples

output samples

sensitivity indices

and plots

util"

example"

functions for generic sampling strategies (e.g. Latin Hypercube) and ad hoc sampling (e.g. One-At-the-Time)

folders in SAFE Toolbox

functions to compute and plot indices and analyze their convergence within a specific GSA method, e.g. EET_indices.m EET_convergence.m EET_plot.m

generic plotting functions that can be used on their own or within different GSA methods

shared utility functions

functions implementing numerical models used in the workflow examples

other methods to be plugged in …

(*)

x3"

y "

0.2 0.4 0.6 0.8−0.5

0

0.5

1

Example application to flood inundation modelling

16

We can formally include both discretely (e.g. resolution) and continuously (e.g. parameters) varying inputs in our sensitivity analysis

Savage et al. (In Review) WRR

SAFE is freely available for non-commercial use

•  We have almost 200 users by now

•  We have a wide range of case studies and are looking for other application opportunities

•  We run an annual summer school where we teach sensitivity analysis and other modeling techniques

17

http://bristol.ac.uk/cabot/resources/safe-toolbox/

LONG-TERM RECHARGE SENSITIVITY

Hartmann, A., Gleeson, T., Rosolem, R., Pianosi, F., Wada, Y. and Wagener, T. 2015. A large-scale simulation model to assess karstic groundwater recharge over Europe and the Mediterranean. Geoscientific Model Devel., 8. DOI:10.5194/gmd-8-1729-2015

18

www.hydro.uni-freiburg.de/mitarbeiter/hartmann

Global distribution of major outcrops of carbonate rocks

19 https://en.wikipedia.org/wiki/Karst#/media/File:Carbonate-outcrops_world.jpg

Karst regions cover about 10% of the Earth's continental area, and partially supply almost a quarter of the world's population with freshwater

Hartmann et al., 2014, Reviews in Geophysics

Global hydrological models do not represent this subsurface heterogeneity

20

e.g.  PCR-­‐GLOBWB          

Hartmann et al., 2015, Geoscientific Model Dev.

e.g.  VarKarst-­‐R  

This can lead to unrealistic recharge estimates in karstic regions

21 Hartmann et al., 2015, Geoscientific Model Dev.

VarKarst is closer to other estimates of recharge amounts in karst regions than ‘homogeneous’ models

We developed a model that considers this heterogeneity and applied it to Europe

22

But how can we estimate model parameters at this scale (without calibration to runoff)?

23

We use a Winter-type hydrologic landscape unit apporach based largely on climate and topography for classification

Hartmann et al., 2015, Geoscientific Model Dev.

We find that simple and weak constraints strongly reduce the feasible parameter space

24

AE flux bias of less than 75%, positive correlation with soil moisture and AE, and prior constraints on parameters

Hartmann et al., 2015, Geoscientific Model Dev.

We see significant difference in recharge estimates between the models

25

Typically we estimate higher recharge using the heterogeneous subsurface representation

Mean future change of input variables

•  Comparison  of  :me  periods  1991-­‐2010  to  2080-­‐2099  

•  5  climate  models    (ISI-­‐MIP)  

•  RCP8.5  (worst  case)    

P: precipitation EPT: potential evaporation HINT: higher intensity events

Some conclusions of this work so far

•  Weak constraints on the model dynamics are very effective in reducing parameter uncertainty

•  Constraining the parameter space this way is likely more realistic than traditional calibration using some statistical performance metric

•  Subsurface heterogeneity has a significant impact on recharge estimates

•  Recharge is sensitive to precipitation type, not just total amounts

27

SHORT TERM SENSITIVITY TO DATA AND PARAMETERS

Pianosi, F. and Wagener, T. (In Review) Understanding the time-varying importance of different uncertainty sources in hydrological modeling using sensitivity analysis. Hydrological Processes.

28

A range of uncertainties will impact hydrologic model simulations

29 [Courtesy of Keith Beven]

For example, missing rainfall events can e.g. strongly influence calibration results

30

WASMOD application to Pasa La Ceiba, Honduras (from Ida Westerberg, Uppsala)

We want to understand the relative importance of uncertainty in data and parameters in time

We test our approach on several US basins. Uncertainty characterization is as follows: -  Wide parameter ranges for all catchments -  Precipitation: storm-dependent multipliers drawn from

a uniform distribution over the interval [0.6,1.4] -  PET: multiplier drawn from a uniform distribution over

[0.8,1.2] -  Flow: Lag-one autocorrelation and a Gaussian error

31

We test the idea on a version of the (lumped) HBV model

32 32

Temperature

Precipitation

Separation (Ts)

Snowpack (CFMAX,

CFR, CWH)

Soil Moisture Accounting

(FC, LP, BETA)

Upper zone

(K1, K0, UZL, PERC)

Lower zone (K2)

rainfall snowfall

evapotranspiration

Transfer function

(MAXBAS)

Potential evapotranspiration

flow

Flow

RMSE

We apply a sensitivity analysis approach to a running mean of model performance (RMSE)

•  The approach (called PAWN) uses the full output distribution

•  Results are indices ranging between 0 and 1

•  A higher index means more sensitivity

•  Applied as 30 day moving average

33 [Pianosi and Wagener, 2015, EM&S]

O N D J F M M J J A S O N D J F M M J J A S O N D J F M M J J A S O N D J

snow

soil

route

0

0.2

0.4

0.6

0.8

1English River at Kalona in Iowa (USGS 05455500) (incl. snow)

O N D J F M M J J A S O N D J F M M J J A S O N D J F M M J J A S O N D J

snow

soil

route

0

0.2

0.4

0.6

0.8

1English River at Kalona in Iowa (USGS 05455500) (incl. snow)

O N D J F M M J J A S O N D J F M M J J A S O N D J F M M J J A S O N D J

rain

evap

snow

soil

route

flow

0

0.2

0.4

0.6

0.8

1

We can zoom in on some of the events

36

French Broad River at Ashville, NC (USGS 03451500) (no snow)

O N D J F M M J J A S O N D J F M M J J A S O N D J F M M J J A S O N D J

rain

evap

soil

route

flow

0

0.2

0.4

0.6

0.8

1O N D J F M M J J A S O N D J F M M J J A S O N D J F M M J J A S O N D J

soil

route

0

0.2

0.4

0.6

0.8

1

O N D J F M M J J A S O N D J F M M J J A S O N D J F M M J J A S O N D J

rain

evap

soil

route

flow

0

0.2

0.4

0.6

0.8

1

Guadalupe Rv near Spring Branch, TX (USGS 08167500) (no snow)

O N D J F M M J J A S O N D J F M M J J A S O N D J F M M J J A S O N D J

soil

route

0

0.2

0.4

0.6

0.8

1

Preliminary conclusions of this work so far

•  Relative importance of different sources of uncertainty changes in time, but also across catchment with different characteristics (snow affected or not; high or low variability of flow,...)

•  Future research: link the range of allowed variability of different sources of errors to the estimated sensitivity so to find at what level of error the uncertainty source becomes influential

•  The method provides information that has to be carefully interpreted

•  We ultimately strive for a multi-method approach: combining data- and model-based analyses

39

In summary, sensitivity analysis has to be an inherent step in any model application (transparency!). We have to understand how our models reproduce the system under study – especially for problems of environmental change.

40 thorsten.wagener@bristol.ac.uk

Sensitivity Analysis •  Pianosi, F., Sarrazin, F. and Wagener,

T. (2015). A Matlab toolbox for Global Sensitivity Analysis. Environmental Modelling & Software, 70. http://dx.doi.org/10.1016/j.envsoft.2015.04.009

•  Pianosi, F. and Wagener, T. 2015. A simple and efficient method for global sensitivity analysis based on cumulative distribution functions. Environmental Modelling & Software, 67, 1-11. doi:10.1016/j.envsoft.2015.01.004

•  Singh, R., T. Wagener, R. Crane, M.E. Mann, and L. Ning 2014. A vulnerability driven approach to identify adverse climate and land use change combinations for critical hydrologic indicator thresholds. Water Resour. Res., 50, 3409–3427, doi:10.1002/2013WR014988.

• 

Karst •  Hartmann, A., Gleeson, T., Rosolem,

R., Pianosi, F., Wada, Y. and Wagener, T. 2015. A large-scale simulation model to assess karstic groundwater recharge over Europe and the Mediterranean. Geoscientific Model Devel., 8. DOI:10.5194/gmd-8-1729-2015

•  Hartmann, A., Goldscheider, N., Wagener, T., Lange, J. and Weiler, M. 2014. Karst water resources in a changing world: Review of hydrological modeling approaches. Reviews of Geophysics, DOI:10.1002/2013RG000443.

41