guidelines for rainfall-runoff modelling - · pdf fileguidelines for rainfall-runoff modelling...

Towards best practice model application

Vaze, J., Jordan, P., Beecham, R., Frost, A., Summerell, G.

Guidelines for rainfall-runoff modelling


2

Copyright Notice

© 2012 eWater Ltd

Legal Information

This work is copyright. You are permitted to copy and reproduce the information, in an unaltered form, for

non-commercial use, provided you acknowledge the source as per the citation guide below. You must not

use the information for any other purpose or in any other manner unless you have obtained the prior written

consent of eWater Ltd.

While every precaution has been taken in the preparation of this document, the publisher and the authors

assume no responsibility for errors or omissions, or for damages resulting from the use of information

contained in this document. In no event shall the publisher and the author be liable for any loss of profit or

any other commercial damage caused or alleged to have been caused directly or indirectly by this

document.

Citing this document

Vaze, J., Jordan, P., Beecham, R., Frost, A., Summerell, G. (eWater Cooperative Research Centre 2011)

Guidelines for rainfall-runoff modelling: Towards best practice model application.

Publication date: March 2012 (Version 1.0)

ISBN 978-1-921543-51-7

Acknowledgments

eWater CRC acknowledges and thanks all partners to the CRC and individuals who have contributed to the

research and development of this publication.

We acknowledge the inputs from the hydrology group in DERM, Queensland, and Mark Alcorn from SA

Department for Water. We thank Matthew Bethune, Peter Wallbrink, Dugald Black, Jin Teng, Jean-Michel

Perraud, Melanie Ryan, Bill Wang, David Waters, Richard Silberstein, Geoff Podger, David Post, Cuan

Petheram, Francis Chiew and Andrew Davidson for useful discussions.

eWater CRC gratefully acknowledges the Australian Government’s financial contribution to this project

through its agencies, the Department of Innovation, Industry, Science and Research, the Department of

Sustainability, Environment, Water, Population and Communities and the National Water Commission

For more information:

Innovation Centre, Building 22

University Drive South

Bruce, ACT, 2617, Australia

T: +61 2 6201 5834 (outside Australia)

Support: 1300 5 WATER (1300 592 937)

E: [email protected]

www.ewater.com.au


3

Contents

1 Introduction ................................................................. 5

1.1 Background ........................................................................................................................... 5

1.2 Definition of Best Practice ..................................................................................................... 5

1.3 Scope .................................................................................................................................... 6

2 Overview of procedure for rainfall-runoff modelling .... 8

2.1 Problem definition ................................................................................................................. 8

Problem statement and setting objectives ............................................................................ 8

Understanding the problem domain ...................................................................................... 8

Metrics and criteria and decision variables ........................................................................... 9

Performance across multiple catchments and subcatchments ............................................. 9

2.2 Option modelling ................................................................................................................... 9

Methodology development .................................................................................................... 9

Collate and review data ...................................................................................................... 10

Setting up and building a model ......................................................................................... 10

Calibration and Validation ................................................................................................... 10

Sensitivity/uncertainty analysis ........................................................................................... 12

Documentation and Provenance ........................................................................................ 12

Model acceptance and accreditation .................................................................................. 13

Use of accepted/accredited model...................................................................................... 13

3 Model choice ............................................................. 14

3.1 Model selection ................................................................................................................... 14

3.2 Available models ................................................................................................................. 15

Empirical methods .............................................................................................................. 15

Large scale energy-water balance equations ..................................................................... 16

Conceptual Rainfall-Runoff Models .................................................................................... 16

Landscape daily hydrological models ................................................................................. 17

Fully distributed physically based hydrological models which explicitly model hillslope and

catchment processes .......................................................................................................... 17

4 Collate and Review Data........................................... 20

4.1 Catchment details ............................................................................................................... 21

Location of gauges (streamflow, rainfall and evaporation) ................................................. 21

Topography and Catchment Areas ..................................................................................... 21

Soil types ............................................................................................................................ 21

Vegetation ........................................................................................................................... 21


4

Water Management Infrastructure ...................................................................................... 22

4.2 Flow data ............................................................................................................................ 22

4.3 Rainfall ................................................................................................................................ 23

4.4 Evapotranspiration .............................................................................................................. 24

5 Statistical Metrics for Testing Performance .............. 25

6 Calibration and validation .......................................... 27

6.1 Calibration ........................................................................................................................... 27

6.2 Validation ............................................................................................................................ 27

6.3 Calibration and Validation of Models to Single Gauge Sites, Multiple Gauge Sites and

Regionalisation of Model Parameter Sets ......................................................................... 29

6.4 Automated, Manual and Hybrid Calibration Strategies ....................................................... 30

Manual Calibration .............................................................................................................. 30

Automated Calibration ........................................................................................................ 31

Hybrid Calibration Strategies .............................................................................................. 32

Selection of Objective Functions for Automated and Hybrid Calibration ............................ 33

6.5 Calibration of Regression Models ....................................................................................... 37

7 Uncertainty and Sensitivity Analysis ......................... 38

7.1 Sensitivity Analysis ............................................................................................................. 39

7.2 Application of Multiple Parameter Sets ............................................................................... 39

7.3 More Advanced Quantitative Uncertainty Analysis ............................................................. 40

7.4 Consideration of Uncertainty in Practical Applications of Rainfall Runoff Models .............. 40

8 Concluding remarks .................................................. 42

9 References ................................................................ 43


5

1 Introduction

1.1 Background

Reliable estimates of stream flow generated from catchments are required as part of the

information sets that help policy makers make informed decisions on water planning and

management. The characteristics of the streamflow time series that influence water resources

system modelling and planning can include the sequencing of flows on daily and longer time

steps, spatial and temporal variability of flows, seasonal distribution and characteristics of high

and low flows.

The best available estimate of streamflow would be expected to come from water level

observations made at a gauging station, converted to flow estimates using a well defined and

stable rating curve. However, these observations are only available for limited number of

gauging locations and for limited time span. Estimates for ungauged locations and for a much

longer time period are needed for contemporary water management, and ways to make

estimates for future possible conditions are also required.

A range of methods are available to estimate streamflow from catchments, using observed

data wherever possible, or estimating by empirical and statistical techniques, and more

commonly using rainfall-runoff models. The modelling approach used to estimate streamflow

by different water agencies and consultants varies across Australia and depends on the

purpose of the modelling, time and money available, and the tools and skills available within

the organisation. With increasing levels of inter-agency collaboration in water planning and

management, development of a best practice approach in rainfall runoff modelling is desirable

to provide a consistent process, and improve interpretation and acceptability of the modelling

results.

The purpose of this document is to provide guidance on the best practice for implementing fit

for purpose rainfall-runoff models, covering topics such as setting modelling objectives,

identifying data sources, quality assuring data and understanding its limitations, model

selection, calibration approaches, and performance criteria for assessing fitness for purpose

1.2 Definition of Best Practice

Best Practice Modelling can be defined as a series of quality assurance principles and actions

to ensure that model development, implementation and application are the best achievable,

commensurate with the intended purpose (Black et al., 2011).

What is in practice “best achievable, commensurate with the intended purpose” may be

subject to data availability, time, budget and other resourcing constraints. Hence, what is

meant by the term “Best Practice Modelling” can vary. Not only does it depend on the

circumstances of the project, particularly providing results that are fit for the intended purpose,

but it also depends to a great degree on interpretation in peer review. This, in turn, will be

influenced by the general state of knowledge and technology in the modelling field, which can


6

be expected to progressively develop over time (such as new remote sensing data sources

coming on line, and new computing hardware and software), as well as data, time, budget and

resourcing constraints. Best Practice Modelling provides for a strategic approach to

modelling which enables the trade-offs that may be imposed by these constraints to be better

managed, and assists in identifying priorities for addressing model and data limitations.

1.3 Scope

The eWater CRC has prepared generic Best Practice Modelling guidelines (Black et al., 2011).

They aim to provide for an integrated approach that enables interactions and feedbacks

between all domains relevant to water management (e.g. hydrological, ecological,

engineering, social, economic and environmental) to be considered.

The procedure in that guidance is intended to be flexible enough to accommodate variations in

the meaning of the term “Best Practice Modelling” and also allow for continuous improvement

as the state of knowledge and technology in the modelling field develops and improves.

The eWater CRC will also provide guidelines to support the BPM guidelines in specific areas of

hydrological modelling that relate to the products that they are developing. This guideline is

intended to address rainfall-runoff model application with the objectives being to provide water

managers, consultants and research scientists with information on rainfall-runoff models and

how to choose one that is fit for purpose, the data available to develop them, and the

calibration and validation methodologies.

There are a number of different purposes that a rainfall runoff model may be applied within an

overall water resources or catchment modelling framework, such as eWater Source. Most of

these purposes relate to providing information to support decision making for some water

management policy. In particular, this can involve:

• Understanding the catchment yield, and how this varies in time and space, particularly

in response to climate variability: seasonally, inter-annually, and inter-decadally.

• Estimating the relative contributions of individual catchments to water availability over a

much larger region, e.g. valley or basin scale.

• Estimating how this catchment yield and water availability might change over time in

response to changes in the catchment, such as increasing development of farm dams,

or changes in land-use and land management.

In some instances with a high quality network of long term stream gauges, most of this type of

information can be estimated from the observations. However, the more common case is

where there is some combination of short term stations, variable quality data, and gaps in

spatial coverage. In these cases, spatial and temporal gaps in the information can be

estimated using rainfall runoff models to:

• Infill gaps caused by missing or poor quality data in an observed data series for a

gauged catchment.

• Estimate flows for a gauged catchment for periods before the observed flow record

started or after when the observed flow record ends.

• Estimate flows for an ungauged catchment.

• Estimating flows from ungauged subcatchments within an overall gauged catchment.


7

• Forecast flows for some immediate future period (typically for a period of between a few

days and a few months), conditioned on current (or recent) observations of the

catchment state.

• Assess the impacts of human influences within a gauged catchment (for example

landuse or vegetation cover change) and simulating the flows that would have occurred

for the historical climate sequence with modified catchment conditions. This may

include assessment of catchment conditions that may be non-stationary in either the

observed record or for the simulation.

• Assess the potential impact of climate variability and/or climate change on flows from a

gauged catchment.

In some cases, several of the above purposes may be satisfied by rainfall runoff modelling for

the same catchment. There are similarities in the approach that is taken to rainfall runoff

modelling, even though the purpose may differ.


8

2 Overview of procedure for

rainfall-runoff modelling The generic guidelines (Black et al., 2011) outline a procedure for applying a hydrological

model. This can be summarised as occurring in 4 phases:

1 Project management,

2 Problem definition,

3 Option modelling,

4 Compare Options and select the best.

This guidance deals only with problem definition and option modelling because the first and

last phases are discussed sufficiently for the purpose of rainfall-runoff modelling in the generic

guidelines. A further reason is that rainfall-runoff modelling is usually only a part of a larger

hydrological modelling project and these phases would be most appropriately considered in

the context of that larger project. Specific aspects of project management and option

comparison that are directly applicable to the development of a rainfall-runoff model, such as

accreditation, are dealt with at appropriate points in this guidance.

2.1 Problem definition

Problem statement and setting objectives

The problem to be addressed must be clearly articulated to minimise the risk that the wrong

tool will be used for the job. The problem statement will give direction on what objectives will be

considered in developing the rainfall-runoff model. As many water management decisions will

often have more than one goal it will be important to ensure these are all identified.

Sometimes it can be useful to express objectives in a hierarchy that shows primary objectives,

secondary objectives and so on. In this regard, consideration should also be given to

possible additional future objectives and goals that could be met based on the current project

or on future projects that build upon the model established in the current project. The decision

on which option offers the best solution should be based upon whether, or how well, each

option meets the agreed objectives (also see section 2.2.1 and 2.2.2 in the generic

guidelines).

Understanding the problem domain

The choice of the rainfall-runoff model will vary based on the purpose the modelling is being

done for, e.g., to understand seasonal low flow characteristics for an in-stream environmental

need; or to assess over-bank flow frequency; or to estimate overall catchment yield on an

average annual basis. The model selected, data required, and calibration approach adopted


9

should reflect this requirement. Where the same model may be used for two or more different

purposes, there may also be a need to focus the calibration on a number of different flow

regimes. If ‘rough’ flow estimates are required over large areas and the runoff generation

methodology should be consistent then the data and modelling process will differ again.

Metrics and criteria and decision variables

Model calibration is a process of systematically adjusting model parameter values to get a set

of parameters which provides the best estimate of the observed streamflow (in the case of

rainfall-runoff models). The process of determining which particular set of parameter values

are best for the intended purpose depends on what comparison metrics are used. Metrics

should be used to quantify the acceptability of the developed model. In all cases graphical

assessment and statistical results of some sort will be assessed to identify the ability of the

calibrated model to reproduce the flows calibrated against.

Different metrics will be more effective in determining model appropriateness to meet different

objectives. What these are should be considered when the problem is being defined.

Understanding appropriate metrics allows model acceptance criteria to be identified.

Performance across multiple catchments and subcatchments

In some situations, the purpose of rainfall runoff modelling is to produce an estimate of the

runoff at a single location where there is a streamflow gauge. If this is the case, the calibration

and validation process may be performed for the single gauged catchment. This approach is

justifiable in situations where gauged data is available for most of the period that flow results

are required for and the purpose of the rainfall runoff model is to infill missing data during the

period of record. It may also be justifiable where there is a requirement to extend the period of

record at the single gauge.

A much more common situation is that flow time series estimates are required at several

locations and that gauged streamflow data is also available at several locations. The locations

where flow estimates are required may or may not overlap with the locations where the flow

data is also available. An objective of any project that involves the application of rainfall runoff

models to multiple catchments or subcatchments should be to demonstrate consistency in the

rainfall runoff model response between those catchments and to explain systematic

differences in the hydrological response between catchments and subcatchments in an

appropriate and logical manner.

2.2 Option modelling

This section describes the process of developing a rainfall-runoff model, further details on key

components are provided in later sections.

Methodology development

The models and methodology employed should be appropriate for the purpose that the model

will be used for. The choices made will be directed by the problem definition developed and

any other information at hand to the modeller. Detail on the models available and guidance on

selecting models and methodology that is fit for purpose is provided in Section 3.


10

Collate and review data

The amount and quality of data available to develop a model should be determined at the

outset of the project. This can influence the selection of models, the performance criteria, and

the approach to calibrate models. A bare minimum data set sufficient to make an approximate

estimate of mean annual catchment yield would include catchment area along with spatial and

temporal characteristics of rainfall and potential evapotranspiration (PET). A comprehensive

data set would include long-term streamflow measurements and rainfall and PET data

collected at one or more locations within the catchment along with land use coverage,

vegetation cover and impervious area information, including changes over time.

The quality of the data should be reviewed prior to using to detect errors, non-stationarity if

any, and understand uncertainties that may influence estimates. Some methods are

discussed in section 4.

Setting up and building a model

The catchment characteristics are considered along with the knowledge on data available and

any other information available to the modeller. The rainfall-runoff model chosen is

conceptualised and an initial parameter set is identified.

When the model is first set up consideration should be given to all constraints which are

limiting and their effects on the modelling. Section 5 provides more details associated with this

step.

Calibration and Validation

Model calibration is a process of systematically adjusting model parameter values to get a set

of parameters which provides the best estimate of the observed streamflow (in the case of

rainfall-runoff models).

The term “validation”, as applied to models, typically means confirmation to some degree that

the calibration of the model is acceptable for the intended purpose (Refsgaard and Henriksen,

2004). In the context of rainfall runoff modelling, validation is a process of using the calibrated

model parameters to simulate runoff over an independent period outside the calibration period

(if enough data is available) to determine the suitability of the calibrated model for predicting

runoff over any period outside the calibration period. If there is not enough data available, the

validation may be performed by testing shorter periods within the full record.

It is normal in research studies to split the observed data sets into calibration and validation

period prior to the study, to demonstrate the performance of the model under both sets of

conditions. Use of this approach can cause problems in practical applications if a model

demonstrates acceptable performance for the calibration data set but produces unsatisfactory

results for the validation data set. An alternative approach in this situation is to calibrate the

rainfall runoff model to all available data but to demonstrate that the performance of the model

is satisfactory over different sub-sets of the period that observed data is available.

Further discussion of model calibration and validation is provided in Section 6.

It is a very common situation in a project that involves rainfall runoff modelling for flow time

series to be required for several catchments or subcatchments within the model domain and

for data to be available from two or more stream flow gauges to facilitate calibration and


11

validation. At locations where gauged flows are available and flow estimates are required, two

options are available to the modeller:

• The rainfall runoff models may be calibrated independently for each gauged catchment.

In this case, independent parameter sets will be derived for the rainfall runoff models of

each catchment; or

• A joint calibration may be performed, with rainfall runoff models calibrated with

consistent parameters to fit to the gauge records from two or more gauges. In this case,

a single set of rainfall runoff model parameters will be produced for all of the catchments

that represent a compromise to fit the flows at all of the gauges within that group.

Consideration should be given at the outset of modelling to the approach that will be used for

dealing with flows from multiple catchments and subcatchments and from multiple gauges.

The strategy for dealing with this issue should be documented at this point and revised, if

necessary, during the process of calibrating and validating the models.

Calibration of a rainfall runoff model normally involves running the model may times, trialling

different values of parameters, with the aim of improving the fit of the model to the calibration

data. Calibration can be facilitated:

• Manually, with the modeller setting the parameter values, running the model to inspect

the results and then repeating this process many times;

• Using automated optimisation, with an optimiser algorithm running the model hundreds

or thousands of times with different parameter values; or

• Using a hybrid approach of automated optimisation phases, interspersed with manually

implemented trials of parameter sets.

Defining the calibration and validation approach before commencing a modelling project can

maximise the efficiency of the calibration process, whilst avoiding the temptation to “overfit”

the model to noise in the observational data. A calibration strategy should therefore outline

the:

• gauge locations where model calibration and validation will be performed;

• viable or allowable ranges for each model parameter value;

• known constraints, dependencies or relationships between parameter values (for

example, the total of the three partial area parameters in AWBM, A1, A2 and A3 must

sum to 1);

• period for calibration at each gauge location;

• period for validation at each gauge location;

• expected level of uncertainty in observations introduced by measurement uncertainty;

• metrics to be used to test calibration and validation performance;

• whether manual or automated calibration strategy will be adopted, or how a hybrid

strategy of progressive manual and automated calibration will be implemented.

• If an automated or hybrid optimisation strategy is to be used, further details should be

defined at the outset of the calibration process on:

• algorithms to be used for optimisation of parameter values;

• objective function(s) that will be used to test the calibration performance;


12

• weightings that may be applied in computation of objective functions, to encourage

fitting to different parts of the flow regime (typically the relative weightings to high,

medium and low flows); and

• the set of model parameters that will be optimised during calibration and constraints on

the allowable range of values for each parameter.

Ideally, calibration strategy should be documented prior to the commencement of the

calibration process. It may be appropriate for the calibration strategy to be reviewed during the

calibration.

Sensitivity/uncertainty analysis

Relevant sources of uncertainty in typical order of importance include:

1 Model input data including parameters, constants and driving data sets,

2 Model assumptions and simplifications of what the model is representing,

3 The science underlying the model,

4 Stochastic uncertainty (this is addressed under “variability” below),

5 Code uncertainty such as numerical approximations and undetected software bugs.

The potential impacts of the above sources of uncertainty on the decisions that will be made

using the model should be considered early in the modelling process and then re-examined

once the model has been calibrated, validated and applied for scenario runs. Uncertainty

becomes more important for estimation of events in the tails of the probability distribution,

floods and droughts, than it is for consideration of events that are closer to the centre of the

probability distribution (such as estimation of the mean annual runoff from a catchment).

Documentation and Provenance

Documentation is an important requirement for model acceptance. Its role is:

1 To keep a record of what was done so that it can be reviewed and reproduced,

2 To provide source or background material for further work and research,

3 To effectively communicate the results from models, and

4 To effectively communicate the assumptions made during the modelling process and

the decisions made by the modeller during implementation of the model.

Good documentation supports a study and it will also enable someone coming along later to

see what decisions were made, what was done to underpin the decisions and why, particularly

if aspects of the project are revisited at some later time.

Provenance, as it might relate to hydrological modelling studies simply means the ability to

trace the source/lineage of the data, model and modelling results. Reasons for providing

provenance in rainfall runoff modelling include:

1 Accountability and a full audit trail for all modelled results.

2 Repeatability; ability to re-create a results data set using current data or better

understanding.


13

3 Reproducibility; ability to re-create a results data set exactly using all original data,

workflow ordering, assumptions and parameters.

4 Versioning of both entire workflow and systems implementation. Versioning of the

subcomponents and data sets will be the responsibility of those who govern them but

must be captured by the system.

The degree of provenance required depends on the application and/or how the modelling

system is intended to be used by the individual or organisation in future. Current best practise

provenance is to save all input data and model/parameters version and workflow history such

that the outputs can be reproduced in future if required. In the future the ability to register and

resolve the type and identity of objects within the modelling process should reduce the

requirement to capture and archive these objects, especially as modellers take greater

advantage of services based point of truth data streams and modelling systems, and rely less

on ad hoc locally managed resources.

Model acceptance and accreditation

The aim of model acceptance is to gain agreement that the model is fit for purpose.

Information available from the model accreditation process (Reporting, QA documentation,

Peer review) provides model development details and review results which will enhance

model acceptance.

Peer review plays an important part, especially with stakeholders that are external to the

organisation undertaking the model development. It is important for establishing the

credibility, reliability and robustness of results and the methodology used to obtain the results.

It is undertaken by people with specialist understanding in fields relevant to the project.

Use of accepted/accredited model

Once a calibrated model is evaluated against good quality data and has undergone thorough

review process (model acceptance and accreditation), it can be used for modelling to support

water management planning and policy decisions (provided that the model was accredited for

similar purpose).


14

3 Model choice

3.1 Model selection

Model selection is made based on an understanding of the objectives and the system being

modelled (http://www.toolkit.net.au/Tools/Category-Model_development; CRCCH 2005a, b).

The WMO (2008, 2009) report include the following factors and criteria as being relevant when

selecting a model:

1 The general modelling objective; e.g. hydrological forecasting, assessing human

influences on the natural hydrological regime or climate change impact assessment.

2 The type of system to be modelled; e.g. small catchment, river reach, reservoir or large

river basin.

3 The hydrological element(s) to be modelled; e.g. floods, daily average discharges,

monthly average discharges, water quality, amongst others.

4 The climatic and physiographic characteristics of the system to be modelled.

5 Data availability with regard to type, length and quality of data versus data requirements

for model calibration and operation.

6 Model simplicity, as far as hydrological complexity and ease of application are

concerned.

7 The possible transposition of model parameter values from smaller sub catchments of

the overall catchment or from neighbouring catchments.

8 The ability of the model to be updated conveniently on the basis of current

hydrometeorological conditions.

Other things that should be considered are:

1 The level of modelling expertise available.

2 Whether the model is going to be used on its own, or if it is going to be used in

conjunction with other models.

3 Freedom of choice may be limited by a desire to minimise problems of different models

for much the same purpose in the same project area, or to avoid problems of different

models in adjoining project areas, particularly where the models are linked in some way

in the future or results compared in some way.

4 Whether uncertainty will be explicitly modelled. If uncertainty is to be explicitly included,

what types of uncertainty are to be modelled (e.g. climatic uncertainty, uncertainty in

climate change projections, uncertainty in rainfall runoff model parameter values); what

approaches will be used to generate the replicates to represent uncertainty and how

many replicates will be required to adequately quantify uncertainty.

5 Whether simulation or optimisation, or a combination of both, is adopted.

http://www.toolkit.net.au/Tools/Category-Model_development


15

6 Whether the model is to be used for hindcasting or forecasting when being applied in

predictive mode.

In essence the governing principle in choosing a model should be that it should not have more

parameters requiring calibration or a greater level of detail than the available data can support,

to minimise problems of spurious results and false calibrations.

3.2 Available models

Rainfall runoff models can be represented by a range of approaches, in order of increasing

complexity as:

• simple empirical methods (e.g., curve number and regression equations);

• large scale energy-water balance equations (e.g., Budyko curve);

• conceptual rainfall-runoff models (e.g. SIMHYD, Sacramento, AWBM)

• landscape daily hydrological models (e.g., VIC, WaterDyn);

• fully distributed physically based hydrological models which explicitly model hillslope

and catchment processes (e.g., SHE, TOPOG).

These categories have been used for ease of description, and there is overlap between these

model types. Although these approaches vary in terms of the complexity with which they

represent the rainfall-runoff transformation processes, all of them conceptualise the real

processes using some sets of mathematical equations (and hence are all conceptual models

of the physical environment). Similarly, conceptual rainfall-runoff models run in distributed

mode can be classed as being landscape daily hydrological models. This section provides a

discussion of the characteristics of each of these model types, along with a broad assessment

of the strengths and weaknesses of each approach for rainfall runoff modelling (Table 3-1).

Empirical methods

Empirical methods to rainfall runoff modelling typically involve the fitting and application of

simple equation(s) that relate drivers of runoff response to flow at the catchment outlet.

Empirical equations are most often derived using regression relationships.

Common predictor variables may include rainfall for the catchment, flow observed at another

gauge in the vicinity, evapotranspiration, groundwater levels, vegetation cover and the

impervious area within the catchment. Where rainfall is used as a predictor variable,

regression relationships derived almost always include a non-linear relationship between

rainfall and runoff.

All catchments incorporate storage elements, including interception by vegetation, storage

within the soil column, groundwater storage and storage within stream channels. Catchment

storage typically results in runoff from the catchment being an integrated function of the

climatic conditions for the catchment over some period prior to the period for which runoff is to

be calculated by the model. Therefore, empirical models that produce acceptably accurate

simulations of runoff are either applied at sufficiently long time steps that changes in internal

water storage within the catchment can be ignored (e.g. annual time step) or applied to

represent an integration of the climatic conditions that occurred over some time period prior.

As a practical example, for most catchments a regression model that only includes daily

rainfall on the current day is likely to produce a very poor estimation of daily runoff but a model


16

for predicting daily runoff that used individual values of daily rainfall for several days prior may

produce acceptable runoff estimates.

Empirical regression relationships are often developed using spreadsheets. They can also be

fitted using more sophisticated statistical analysis packages, which may more easily facilitate

the investigation of predictor variables. For general information on the development of

regression relationships, the modeller is referred to NIST/SEMATECH e-Handbook of

Statistical Methods (NIST and SEMATECH, 2010) or to a University Level statistics text book.

Empirical regression equations are best suited to situations where there are two flow gauges

on the same stream with partially overlapping periods of record, which are therefore subject to

similar climatic drivers, and the regression equation is used to extend the simulated flow to the

combined period of record from both sites. They can also produce adequate simulations for

neighbouring gauged catchments with overlapping periods of record in situations where the

two catchments are subject to similar rainfall timeseries and are relatively similar

hydrologically.

Large scale energy-water balance equations

The large scale energy-water balance methods are based on the hypothesis of available

energy and water governing large scale water balance (precipitation, evaporation and runoff).

These are usually developed using large scale observed data sets, eg. the Budyko curve

(Budyko, 1958) was developed using mainly European data, and numerous other forms have

been proposed to improve estimates in local regions and to account for different land cover

types (Arora, 2002). One of the more popular forms of the Budyko method is the Fu (1981)

rational function equation (Zhang et al., 2004) where a single parameter, α, in the equation can

be calibrated against local data to tune the method for the local conditions. The inputs to these

equations are rainfall and potential evapotranspiration (PET) and the output is runoff at mean

annual time step.

Conceptual Rainfall-Runoff Models

Conceptual rainfall runoff models represent the conversion of rainfall to runoff,

evapotranspiration, movement of water to and from groundwater systems and change in the

volume of water within the catchment using a series of mathematical relationships. Conceptual

rainfall runoff models almost always represent storage of water within the catchment using

several conceptual stores (or “buckets”) that can notionally represent water held within the soil

moisture, vegetation, groundwater or within stream channels within the catchment. Fluxes of

water between these stores and in and out of the model are controlled by mathematical

equations.

Most applications of conceptual rainfall runoff models treat the model in a spatially lumped

manner, assuming that the time series of climatic conditions (notably rainfall and potential

evapotranspiration) and the model parameter values are consistent across the catchment.

There have been implementations in more recent times of conceptual rainfall runoff models in

semi-spatially distributed and spatially distributed frameworks. In distributed application, the

catchment is defined by grid cells or subcatchments within the catchment that are assigned

the same rainfall runoff parameter values but different time series of climatic inputs so that

different grid cells or subcatchments within the catchment produce different contributions to

the overall runoff. This is effectively a series of lumped rainfall runoff models, with lumped sets

of model parameters that are applied with spatially distributed rainfall.


17

Conceptual rainfall-runoff models have been widely used in Australia for water resources

planning and operational management because they are relatively easily calibrated and they

provide good estimates of flows in gauged and ungauged catchments, provided good climate

data is available.

In Australia there are six widely used conceptual rainfall-runoff models; AWBM (Boughton

2004), IHACRES (Croke et al. 2006), Sacramento (Burnash et al. 1973), SIMHYD (Chiew et

al. 2002), SMARG (Vaze et al., 2004) and GR4J (Perrin et al. 2003). The input data into the

models are daily rainfall and PET, and the models simulate daily runoff. The models are typical

of lumped conceptual rainfall-runoff models, with interconnected storages and algorithms that

mimic the hydrological processes used to describe movement of water into and out of

storages. They vary in terms of the complexity of the catchment processes that they try to

simulate and in terms of the number of calibration parameters which vary from four to

eighteen.

Landscape daily hydrological models

These models are based on the concept of landscape processes and they model the typical

landscape processes using simplified physical equations (VIC model, Liang et al., 1994;

2CSALT, Stenson et al., 2011; AWRA-L, Van Dijk, 2010). A catchment is usually

conceptualised as a combination of landscapes which are delineated using some combination

of outputs from digital elevation model analysis, underlying geology, soil types and land use.

Often these models have been designed to reproduce other variables in addition to streamflow

(e.g. distributed evapotranspiration, soil moisture, recharge, salinity), and as a result have a

greater complexity to methods that target streamflow alone.

Fully distributed physically based hydrological models which

explicitly model hillslope and catchment processes

The physically based models are based on our understanding of the physics of the

hydrological processes which control the catchment response and use physically based

equations to describe these processes. A discretisation of spatial and temporal coordinates is

made at a very fine scale for the entire catchment and the physical equations are solved for

each discretised grid to obtain a solution.


18

Table 3–1 Assessment of Strengths and Weaknesses of Different Rainfall Runoff Model Structures

Criteria Model Type

Empirical Large Scale

Energy-Water

Balance

Conceptual Landscape

Daily

Fully

Distributed

Physically

Based

Typical Run Time Step

Can be daily if daily flow from another gauge is used as a predictor variable. Otherwise typically only applied at annual (or longer) time scale

Typically only applied for mean annual runoff, although pattern of flows from a nearby gauge may be used to disaggregate annual totals to monthly or daily time steps

Daily, although shorter run time steps are possible if sufficient climatic data is available at this shorter time step

Daily, although shorter run time steps are possible if sufficient climatic data is available at this shorter time step

Minutes to hours to maintain numerical stability, although often forced with daily data and assumed patterns used to disaggregate to shorter time steps

Typical Number of Parameters

1 to 5 2 to 4 4 to 20 10 to 100 10 to 1000's

Risk of over-fitting or over-parameterising the model.

Low Very Low Moderate High Very High

Need for high resolution spatial data layers

None to Moderate

Low to Moderate

Low High Very High

Strength of Apparent Connection between Model Parameters and Measurable Physical Catchment Characteristics

None None Weak for most parameters (although impervious area or interception may be exceptions)

Moderately weak

Claimed to be strong by proponents but can be difficult to validate this claim

Run time on typically available computer platforms for 100 years of daily data

<1 second <1 second <1 to 60 seconds

10 seconds to 100 minutes

1 minute to several hours


19

Criteria Model Type

Ability to implement multiple runs for automated calibration

Not typically required - optimum parameters can be obtained by least squares fitting that does not require multiple runs

Not typically required

Very Good. Run times are typically sufficiently low to facilitate this and tools are available (Rainfall Runoff Library and Source) to facilitate this

Good. Run times likely to be sufficiently low to facilitate this in most circumstances, however tools for calibrating such models using automated routines are not as widely available

Poor. Run times are generally too long to consider automated calibration

Typical Performance in Regionalisation

Moderate at annual time steps. Usually very poor at shorter time steps (e.g. Daily)

Good at annual time steps. Usually very poor at shorter time steps (e.g. Daily)

Moderate at daily time steps

Proponents claim to be superior for regionalisation to conceptual rainfall runoff models

Proponents claim this to be a strength of distributed models but in reality the large number of parameters required may compromise the application of distributed models to regionalisation

Representation of non-stationarity in catchment conditions

Not possible

Often applied to explicitly represent non-stationarity in vegetation cover for mean annual runoff signal

Usually difficult, due to lack of physical meaning for many model parameters

Possible Possible

Typical performance of model when applying to a very different climatic period to that used for calibration

Poor Moderate when used to estimate impact on mean annual flow but flows disaggregated to shorter time steps are likely to be poorly estimated

Variable - can be good in some catchments but poor in others



Typical level of expertise with this type of model within Australian water industry

Strong Moderate Strong Weak Very weak

Likelihood that previously calibrated models are available for catchment to be modelled.

Moderate to Low

Moderate Very High Low Very low


20

4 Collate and Review Data Climatic data is the most important driver of any rainfall runoff modelling process. The

calibration and validation of models also involves comparison to observed streamflow data.

Major causes of difficulty in calibrating rainfall-runoff models are errors and uncertainties in the

input data (see Kavetski et al, 2003). A discussion of these problems can be found in the

collection of papers in Duan et al 2003. Checks should therefore be performed on the input

data and the comparison data set for calibration and validation to be used in rainfall runoff

modelling before any attempt is made to apply or calibrate the models. The intent here is to

investigate the integrity of the data, whether observations are in the first instance plausible

(e.g., is the volume in a hydrograph less than the product of the rainfall and catchment area).

Investigations into data to be used for rainfall runoff modelling should include checks of:

• Stationarity of the data time series , i.e. has there been any systematic or step change in

the statistical properties over the time of data collection, and if so why;

• Spatial coherence of data, i.e., is the data consistent with regional spatial and temporal

patterns and trends;

• Accuracy of the spatial location of the gauging site;

• Consistency in the approach used to date and time stamp the data, particularly for data

provided by different agencies;

• Procedures use for spatially interpolation of point observations to gridded data

estimates or estimated series across catchment areas

e.g., time series plots at different levels of temporal aggregation, ranked plots, residual mass

curves, double mass curves, etc. This will pick up patterns as well as identify anomalies which

may be potential data QA issues.

Other checks and analysis, including regional consistency of runoff depths, rain-runoff ratios,

rating confidence limits, period of record, whether rainfall and PET is observed or interpolated,

base-flow characteristics, checks for stationarity and variability over time, etc would also be

useful. It is important that prior knowledge is considered.

One major factor which will apply across all types of time series data used is that the time base

must be kept consistent so that the data applies to the same time period. An example is

where flow data time steps should be matched to the rainfall data time step. In Australia, daily

rainfall data is commonly recorded as the depth of rainfall that occurred in the 24 hours

preceding 9 am on the date of the recorded data. In contrast, daily streamflow totals are often

quoted for the 24 hour period commencing on the nominated date, resulting in the recorded

flow data being offset by 1 day forward of the rainfall data. Where possible the flow data should

be extracted at a time step to match the rainfall dataset. HYDSTRA databases allow this where

the records are at short time intervals. In other cases shifting the recorded time series by one

day for either the rainfall or flow time series may be required to produce consistent time series

for modelling.

The remainder of Section 4 outlines the data types, sources, availability, accuracy,

manipulations (such as gap filling) and any other issues.


21

4.1 Catchment details

Location of gauges (streamflow, rainfall and evaporation)

The streamflow recorded at the catchment outlet is a combined response to the spatial

distribution of rainfall and evaporation across the catchment. There are uncertainties

associated with the streamflow measurements due to rating curve errors as well as due to

extrapolation outside the limits of the rating curve. There is spatial variability in rainfall (and to

smaller extent evaporation) across a catchment which is not captured when undertaking

lumped catchment modelling using a single rain gauge. There might be problems with the

location of the rain gauge in terms of capturing a representative rainfall for all the rainfall

events especially for catchments with high rainfall gradients.

Topography and Catchment Areas

The catchment area for a catchment represents the contributing area to the catchment outlet

where the streamflow is measured. The catchment boundaries (and the corresponding

catchment area) can either be derived from topographic map layers or using the catchment

digital elevation model (DEM) and a standard package such as ARCGIS. It is usually easier to

determine catchment area for the catchments located in steeper terrain compared to those

located in very flat areas (especially when using DEM).

Slope and dominant aspect may provide useful explanatory variables for estimating routing

parameters or for regionalisation of rainfall runoff parameters between catchments.

Soil types

A catchments rainfall-runoff response is related to the soil types in the catchment. The surface

soil characteristics determine the infiltration rates and so the contributions from different flow

components (surface runoff, throughflow and base flow). Soils information can be obtained

from any soils field work that has been undertaken in the catchment or from large scale soil

properties maps (e.g. Australian Soils Atlas, Northcote et al., 1960). In most practical

applications of conceptual rainfall-runoff models in Australia, soil information is seldom directly

used as input in the calibration process because the inherent spatial variability in soil

properties within a catchment is typically sufficiently large that it has been difficult to

demonstrate statistically robust relationships between conceptual model parameters and soil

properties.

Vegetation

Land cover/vegetation cover in a catchment can often be correlated with the amount of

interception storage/loss and actual evapotranspiration in a catchment. The land cover across

the catchment can be derived from large scale vegetation mapping based on satellite imagery

or remotely sensed data. Vegetation cover data has not typically been used explicitly in

directly determining rainfall runoff model parameters, although there have been some recent

studies which have demonstrated the importance of catchment land cover in rainfall-runoff

model calibration and for predictions in ungauged basins (Zhang and Chiew, 2008; Vaze et al.,

2011c).


22

Water Management Infrastructure

Water management infrastructure within a catchment can allow humans to make very

substantial modifications to flows within a catchment. Water management infrastructure may

include large dams, farm dams and off stream storages, extractions, man-made canals or

diversion pipelines and discharges from sewage treatment plants. Locations of these

infrastructures should be identified where they exist within the catchment so that their potential

impact on streamflows may be assessed. Recorded flows at the catchment outlet may require

adjustment to allow for the influence of water management infrastructure located upstream of

each of the flow gauging locations.

4.2 Flow data

Reliable measurements of streamflow data are critical for successfully calibrating a

rainfall-runoff model to a catchment. The streamflow data for all the gauged locations can be

obtained from the respective state government water management agencies or from the

Bureau of Meteorology (in Australia). Considerations in checking streamflow data include:

• the agency collecting the data and the quality assurance procedures (if any)

implemented by that organisation during data collection;

• reliability of the rating of levels to flows for the gauge;

• the accuracy, extent and currency of cross sections surveyed at the gauge site.

(Surveyed cross sections may only extent to the top of the stream bank and gauging for

flows extending onto the floodplain may use a cross section that is inaccurate);

• vegetation and substrate material for the channel bed, channel banks and floodplain

and the influence of assumptions made about these on gauged flows;

• the ratio of the highest flow outputs to the highest flow that the gauge has been rated for;

• how hydraulically stable (variable over time) the rating site is;

• examination of potential backwater effects for the site from influences that are

downstream of the site, such as stream confluences, bridge crossings, culverts, dams

or weirs;

• hysteresis effects leading to different flow rates for the same recorded level on rising

and falling limbs of hydrographs;

• how well maintained the gauging site and instrumentation used for measuring water

levels has been;

• any changes to the gauging instrumentation over time;

• the length of time since the last rating at high flows;

• length of record at the site;

• availability of quality codes with the flow data record;

• proportion of missing data;

• trends in when data is missing from the record (i.e. Is there any bias toward high or low

flow periods, particular seasons, or are the gaps just random?) and how this might

influence any infilling procedures; and


23

• if there are a number of gauges closely located that basically represent the same

catchment the data sets may be able to be combined to give a longer record for the site.

Assessment of the above factors will inform whether the data is useful in calibration of the

model, independent validation of the model or whether the data should be ignored.

4.3 Rainfall

Rainfall is the main driver of runoff and so reliable measurements of rainfall are critical for

successfully calibrating a rainfall-runoff model to a catchment. There are several sources for

obtaining climatic data for a particular catchment:

• Site observations from Bureau of Meteorology climate database.

• Site observations taken from monitoring sites collected by other organisations that may

exist outside of the Bureau of Meteorology database. Many jurisdictional databases

contain rainfall records.

• Gridded data products, such as the Bureau of Meteorology’s Australian Water

Availability Project (AWAP) or Queensland Centre for Climate Applications’ SILO data

set.

It is important to be aware of how this data has been collected and what data quality control

methods have already been applied to the data prior provision of the data set as this may

influence the modelling results. This is particularly relevant to gridded products, such as SILO

and AWAP (SILO, Jeffrey et al., 2001; AWAP, Jones et al., 2009), as these data sources

generally use different algorithms to convert time-series observations at data points to gridded

data products.

In a small catchment, considerably better results may be obtained from using rainfall station

data from the BOM (http://www.bom.gov.au/climate/) or locally collected data than a gridded

data set that smoothes observations from a smaller number of more sparsely located sites. In

some cases it may be appropriate to adjust the station data, normally by a percentage, if the

mean catchment rainfall can be defined using other sources e.g. isohyetal detail.

In large catchments there is spatial variability in rainfall across a catchment which is not

captured when undertaking rainfall-runoff modelling using rainfall time series from the rain

gauges. If using a single rain gauge, there might be problems with the location of the rain

gauge in terms of capturing a representative rainfall for all the rainfall events. If using a

spatial rainfall product (SILO or AWAP in Australia), there can be uncertainties introduced

because of the method used for interpolating rainfall between rain gauges and changes in the

rain gauge network over time. Interpolation methods currently used are more suited to areas

where rainfall varies less over space and in time. They do not account well for orographic

effects, and rainfall networks in Australia historically have not captured the spatial and

temporal variations in tropical and monsoonal areas well.

Vaze et al., 2010b discusses testing carried out considering the effects of using different

rainfall data sets on the calibration and simulation of conceptual rainfall-runoff models. They

conclude that considerable improvements can be made in the modelled daily runoff and mean

annual runoff with better spatial representation of rainfall. Where a single lumped

catchment-average daily rainfall series is used, care should be taken to obtain a rainfall series

that best represents the spatial rainfall distribution across the catchment. However where only

estimates of runoff at the catchment outlet are required, there is little advantage in driving a


24

rainfall-runoff model with different rainfall inputs from different parts of the catchment

compared to using a single lumped rainfall series for the catchment.

4.4 Evapotranspiration

The measured pan evaporation data can be obtained for all the locations with the evaporation

gauges installed (in Australia from the Bureau of Meteorology (BoM) basic records). In

Australia there are also some spatial climate products which use point evaporation

measurements recorded by the BoM and use an interpolation schemes to produce spatial

evaporation surfaces (SILO, Jeffrey et al., 2001; AWAP, Jones et al., 2009).

The network of pan evaporation recording stations in Australia is sparse in comparison to

stream flow and rainfall networks, although there is some compensation in that typically

potential evapotranspiration exhibits substantially higher spatial correlation than rainfall or

stream flow. This limits the ability to accurately represent the true spatial and temporal

variability in evaporation in models however the spatial variability in evaporation is much

smaller compared to the variability in rainfall.

The BoM network records pan evaporation. Modelling requires potential evapotranspiration

(PET). There are a number of methods to convert pan evaporation to PET including Penman

Monteith, Morton’s and accepted pan factors. These use climatic variables in the conversion

calculation including solar radiation, temperature, vapour pressure, and wind speed which are

recorded at some pan recording stations but not all. This further limits the network available to

draw data from.

When all the required data is available the conversion calculations will use the records but

often some variable is missing and estimates of that variable are made and used. Where

there is no data for the climatic variables, calculated pan to PET conversion factors from a

nearby station can be used to derive PET from pan evaporation.

Commonly the spatial products have interpolated layers for a range of climatic factors and the

spatial PET layer is calculated from data in these layers rather than interpolating PET

calculated at recording stations.


25

5 Statistical Metrics for Testing

Performance There are many performance measures used to consider the acceptability of a rainfall-runoff

model. In all cases visual assessment and statistical results of some sort will be assessed to

identify the ability of the model to reproduce the flows it is calibrated or validated against. All

may contribute to best practice and which measures are more appropriate will be directed by

the modelling objective. A number of commonly used visual assessment techniques are

outlined in Table 5–1. Statistical performance measures and their relevance in various study

types are listed below in Table 5–2.

Table 5–1 Plots for assessing model performance

Plot Assessment and Purpose

Daily and monthly plots (linear and log)

Used to check the general size, shape and timing of hydrographs. Linear plots will better show medium and high flows and log plots low flows. Baseflow and recession characteristics can be reviewed. If recessions are frequently too flat then this can indicate that the interflow and baseflow are not represented correctly.

Scatter Plot

Scatter plots show the ability of the model to match flows on actual time steps. They show the flow ranges where the model is more accurate. Linear and log plots will show the variability across the various flow ranges. Often a line of best fit is shown to indicate the bias of the model in estimating flows.

Ranked Plots Commonly referred to as frequency of excedence or flow duration graphs, they show the percentage of time a flow is exceeded over the modelled period. They show whether the modelled output can replicate the observed flow regime. Flow duration curves are effective diagnostics to ensure that both the variability and the seasonal pattern are captured.

Cumulative mass or cumulative residual mass curves

Scatter plots and flow duration curves do not examine the time sequence of events. A model could appear to be replicating the flow regime however the replication of regimes during wet and dry periods may not be adequate. A cumulative residual mass curve is a cumulative plot of residuals (flow value - mean of all values). A residual, and therefore slope of the curve, will be positive during wet periods as flows are higher than average and during dry periods the slope will be negative. If the curves diverge there may be a data issue. If they diverge consistently in all wet or all dry periods it is likely that model parameterisation for wet periods or dry periods may not be appropriate.

Plotting average daily or monthly flows (average of all Days, average of all Januaries)

A simple diagnostic to ensuring that the model can replicate seasonality characteristics.


26

Table 5–2 Statistical performance measures (metrics) and their relevance in various study types (Y – Yes, N – No)

Metric Purpose

Runoff Yield

Climate change

Landuse change

Low flow

Water quality

Peak flow / floods

Difference in total runoff Y Y Y N N Y

Difference in total runoff over different seasons of the year*

Y Y Y Y Y Y

Difference in total runoff contained within high, medium and low parts of the flow duration curve

Y Y Y Y Y Y (high flows)

Difference in proportion of time that cease to flow occurs

N Y Y Y Y N

Difference in the slope of logarithm of flow versus time for baseflow recession periods

N N Y Y Y N

Mean square error between observed and modelled runoff

Y Y Y N N Y

Coefficient of determination (often referred to as r²)

Y Y Y N N Y

Nash Sutcliffe coefficient of efficiency on daily flows

Y Y Y N N Y

Nash Sutcliffe coefficient of efficiency on monthly accumulated flows

Y Y Y N N N

Nash Sutcliffe coefficient of efficiency calculated using logarithm transformed flows

N Y Y Y Y N

* Definition of seasons to be used will vary depending upon the climatic zone that the catchment is in. For

tropical areas, two seasons (a “wet” season from December-April and “dry” season from May-November) may

be appropriate. In Southern Australia, it may be appropriate to consider the four conventional calendar

seasons (Dec-Feb, Mar-May, Jun-Aug and Sep-Nov).

** Definitions of high, medium and low flow ranges will depend upon the purpose of the study and the

catchment. Typical ranges might be High flows: days in observed data in the 0 to 20% probability of

exceedance range; Medium flows: days in observed data in the 20 to 80% probability of exceedance range;

Low flows: days in observed data with greater than 80% probability of exceedance and above the cease to

flow level at the gauge. Adjustment of the low and medium flow ranges may be required particularly in

response to the probability of cease to flow conditions at the gauge.


27

6 Calibration and validation

6.1 Calibration

Model calibration is a process of optimising or systematically adjusting model parameter

values to get a set of parameters which provides the best estimate of the observed streamflow.

Virtually all rainfall runoff models must be calibrated to produce reliable estimates of

streamflow because there has been little evidence identified of strong links between physical

characteristics of catchments and the parameters of rainfall runoff models (Beven, 1989).

Models should always be calibrated to observed data to demonstrate that the model can

produce observed flow time series with an acceptable level of accuracy. The acceptable level

of accuracy will depend upon the statistics of the flow data to be reproduced, which is

determined by the purpose that the model will be applied for.

A model may be available that has been previously calibrated for a catchment as part of

another study. In this situation, the calibration performance of the model should be re-tested

before it is applied because the purpose for developing the model may be different between

the earlier and later applications, which may influence the calibration objectives.

When calibrating a model it should always be kept in mind that there are always going to be

tradeoffs, for example between getting wet, dry, and average conditions correct, and those

tradeoffs will be driven by the purposes the model will be used for.

6.2 Validation

Model validation is a process of using the calibrated model parameters to simulate runoff over

an independent period outside the calibration period (if enough data is available) to determine

the suitability of the calibrated model for predicting runoff over any period outside the

calibration period. If there is not enough data available, the validation may be performed by

testing shorter periods within the full record.

Model validation is one of the most important steps in rainfall-runoff modelling as the

performance of the calibrated model in the validation period provides us confidence in the

modelling results when the calibrated model is used for simulating streamflow outside the

measured streamflow period or when the model is used for predicting streamflow under future

climate change scenarios.

Validation has often been achieved using a “split sample” process, whereby a period of

observed data (say the first two-thirds of the available record) are used for calibration and the

remaining one-third are used for validation. The model that was calibrated using the calibration

data set is run for the validation period without changing the model parameters and the

goodness of fit statistics are computed for the validation period. The split sample approach

assumes that both the catchment and the climatic conditions that it is subject to are stationary

in nature across the entire period that recorded data is available for. Evidence of stationarity

(or non-stationarity) in catchment conditions that would affect the hydrological response during

the period of recorded data should be checked using independent data sources (such as aerial

photography, satellite imagery, landuse, topographic or other spatial information).


28

A more sophisticated calibration approach can involve multiple calibration and validation

periods. As in the simple split sample approach, the model is calibrated to a calibration period

and then performance is tested over the validation period without changing the model

parameter values. This approach is then repeated multiple times, with each replicate using

different start and end dates for the respective calibration and validation periods. This allows a

range of model performance statistics for calibration and validation periods to be reported.

There will be some instances with this calibration and validation approach where the calibrated

parameters perform well against the calibration data set, but performs poorly against the

validation data set. In research type investigations, where the modeller may be comparing

different rainfall-runoff models, calibration methods, or objective functions, the validation

results can be used directly to help decide the best model or method or objective function.

However, in practical applications, a modeller may have to decide either not to change the

calibrated parameters and report the poor results, or to recalibrate the model because the

performance is unacceptable.

The modeller may choose the latter option, and may then recalibrate and compare against the

‘validation data set’ several times until the calibrated parameters perform acceptably against

both data sets. However, as the validation data set has now been used to change the

calibrated parameters, it is no longer an independent data set and has in effect indirectly

become part of the calibration data set.

This risk of having much poorer performance in validation than calibration may be mitigated by

ensuring as far as possible both data sets have similar flow distributions, An arbitrary

approach to splitting the data, e.g., at the midpoint, may result in half of the data being in a

much wetter period. A model calibrated to these conditions would not be expected to perform

well under the drier conditions in the validation data set. More alternate approaches should be

considered on how to split the data set, perhaps into non-contiguous periods, to ensure overall

flow distributions are similar in each period.

Data is a valuable resource, and should be used to greatest effect. In most Australian

conditions, long data sets are needed to adequately represent climatic variability. An

alternative approach to having split samples is to use the complete data set to calibrate the

model, then to report its performance for different sub-periods, e.g., first half and last half, or

decadally, or driest X year period and wettest X year period. The objective would be to have a

comparatively persistent performance across all these periods. This does not necessarily give

you an independent assessment of performance, but does report on performance under

different conditions.

Transposition of model parameter values from gauged to ungauged catchments may be

tested using a spatial variant on split sample validation. Under this approach, component

models from a gauged catchment with the calibrated parameter values for that catchment can

be applied to another gauged catchment to test the uncertainty and bias introduced from

transposition. Uncertainty ranges can be established by testing contributions flow series

produced by model outputs with parameter sets adopted from several different gauged

catchments. Examples of the performance of these transposition approaches are discussed in

Viney et al. (2009) and Chiew (2010).

Generally the same metrics used to assess the performance of the model during calibration

are also used to assess model performance during validation. The model performance during

validation is almost always poorer than during calibration because model parameters are

deliberately not specifically fitted to the data for the validation period.


29

6.3 Calibration and Validation of Models to Single Gauge Sites,

Multiple Gauge Sites and Regionalisation of Model

Parameter Sets

It is a very common situation in a project that involves rainfall runoff modelling for flow time

series to be required for several catchments or subcatchments within the model domain and

for data to be available from two or more stream flow gauges to facilitate calibration and

validation. At locations where gauged flows are available and flow estimates are required, two

options are available to the modeller:

• The rainfall runoff models may be calibrated independently for each gauged catchment.

In this case, independent parameter sets will be derived for the rainfall runoff models of

each catchment; or

• A joint calibration may be performed, with rainfall runoff models calibrated with

consistent parameters to fit to the gauge records from two or more gauges. In this case,

a single set of rainfall runoff model parameters will be produced for all of the catchments

that represents a compromise to fit the flows at all of the gauges within that group.

The advantage of the joint calibration approach is that, assuming some degree of

homogeneity in the rainfall runoff response of the selected gauged catchments, the parameter

sets produced should be more robust when applied to other catchments with similar response

that were not used for the calibration.

If an automated calibration process is used for joint calibration of multiple catchments, the

objective function used for automated calibration to the gauged catchments will be a weighted

average of the objective function values produced at the individual gauges. Options for

selecting the weighting values are:

• All gauged catchments contribute equally to the overall objective function;

• Weights are assigned according to the length of available record (e.g. number of days

with data) at each site;

• Weights are assigned according to the inverse of the correlation coefficient in gauged

flows between one gauge and one or more of the other gauges in the set (i.e. gauges

with strongly correlated recorded flows are assigned lower weighting factors than

gauges that have weaker correlations with other gauges);

• Some combination of the above factors.

There are three main methods of developing flow data sets in residual ungauged catchments

between upstream and downstream gauges:

1 Calibrate a model to the difference in flow between the gauged upstream flows routed to

downstream (adjusted for known transmission losses) and downstream gauges.

2 Adjust a flow data set from a nearby catchment using either recorded or generated data,

3 Apply parameter values from other calibrated models and use catchment appropriate

climate data.

There are two main methods of developing flow data sets in ungauged catchments:

1 Develop a regression equation between flows for the ungauged catchment and gauged

catchments and apply this equation to transpose the flow, or


30

2 Apply parameter values from other calibrated models and use catchment appropriate

climate data.

Generally in the second case parameters for a neighbouring or nearby catchment are used but

climate data and catchment characterises of the catchment of interest are applied in the

model. Many studies have shown that selecting a donor catchment based on spatial proximity

gives similar or better results than selecting a donor catchment based on catchment attributes

(Merz and Bloschl 2004, Oudin et al 2008; Parajka et al. 2005; Zhang and Chiew 2009).

6.4 Automated, Manual and Hybrid Calibration Strategies

Calibration of hydrological models can be conducted using manual or automated methods, or

a combination of the two approaches (see Boyle et al, 2000 and Bárdossy, 2007 for

frameworks for combining manual and automated methods of model calibration). Calibration

involves the adjustment of model parameter values to improve the fit of model output data to

observations to a level that is acceptable.

In case of manual calibration, definition of “goodness of fit” is usually produced as a

combination of statistical indices and visual inspection of the observed and simulated

hydrographs. Whereas in case of automated calibration, definition of “goodness of fit” is

usually produced using an objective function. The objective function translates the observed

and modelled outputs into a single number, so that the results of successive calibration

iterations can be compared. Automated calibration routines use a defined algorithm that runs

the model multiple times, adjusting model parameter values according to a strategy that is

intended to improve the value of the objective function.

The sections that follow give information on the calibration methods available and their

relevance in various study types (shown in Table 3-1) which dealt with model choice

appropriate for intended purposes.

Manual Calibration

Manual calibration involves the modeller selecting a set of parameters for their model, running

the model once and then examining the output statistics from the model (from the list

discussed in Section 5). The modeller would then revise the values of one or more parameters

and repeat the above process. This may continue many times until the model achieves the

desired level of performance.

The match between simulated and observed streamflow can be visually assessed as either a

time series, or as flow duration curves or residual mass curves. The visual assessment can

identify general deficiencies in the matching of the hydrologic regime, e.g., high flow events

under or over estimated, baseflows under or over-estimated or the seasonal response of the

model not captured appropriately. Software that stores the results of conceptual storages and

fluxes for graphing, and interpretation of these results in the context of model structure is also

useful to identify which parameter values need adjusting and in which direction in order to

improve results.

Guidelines are available from the developers of the Sacramento model that describe how to

estimate key parameter values directly from analysis of recorded hydrographs (Burnash,

1995). A range of realistic parameter values has also been recommended to guide initial

estimates.


31

Strengths:

1 Encourages a deeper understanding of model structure and its applicability to

catchment hydrology, rather than treating as a black box.

2 Allows for hydrologist to consider performance against a broad range of performance

metrics, and make appropriate adjustments.

3 Takes into account understanding of the data and the catchment.

4 Allows a logical checking at each change.

5 Produces a greater appreciation of strengths and limitations of calibrated result.

Weaknesses:

1 Repeatability is limited. Different people may get different parameters and output flow

time series.

2 More effort and time required to complete a calibration.

3 Difficult to manually calibrate models with more than about 10 parameters.

4 Only amenable to calibration for one gauge at a time and difficult to simultaneously

calibrate flows from multiple gauged catchments that may have similar hydrological

response. As a result, in catchments with multiple gauges very different parameter sets

for independently calibrated catchments may be produced that don’t represent the

underlying similarity in hydrological response that would be expected from those

catchments.

Automated Calibration

In computer optimisation routines, an objective function, which itself is selected by the

modeller is used to formulate the calibration problem and the computer undertakes the

calculations.

Strengths:

1 Can operate with one simple objective function or use of multiple objective functions.

2 Can undertake calibrations for multiple catchments in a short period of time.

3 Relatively easy.

4 Repeatable. Different people will get same parameter values.

Weaknesses:

1 Is confined by the optimisation routine chosen and how the objectives are set.

2 Is dependent on the computer routine being set up accurately to reflect the choices in 1.

3 Lack ability to check the relationships between the calibrated parameter values

produced as the calibration proceeds, which may cause investigation of sets of

parameters that are infeasible (unless appropriate checks are build within the

calibration algorithms).

4 Software is required to automate the optimisation process.


32

5 Parameter values commonly become “trapped” against the minimum and maximum

constraints of the allowable parameter ranges set by the user. If the user does not check

for this, the parameter set chosen may be sub-optimal as the best parameter set may

have parameter values that lie outside the constraints set by the user at the time the

optimisation is initiated.

There are two global optimisation methods included in Source: Shuffled Complex Evolution

(SCE) and Uniform random sampling. The analysis undertaken as part of the testing with data

from 200+ catchments in southeast Australia showed that there is an advantage in following a

global optimiser with a local optimiser to fine tune the calibrated parameter values. The

Rosenbrock method is included in the framework as a local optimiser. The testing results

suggest that the combination of SCE followed by the Rosenbrock should be used (Vaze et al.,

2011a,d).

Hybrid Calibration Strategies

Best practice for model calibration would normally involve a hybrid of manual and automated

calibration approaches. A typical hybrid strategy would involve:

Implementation of an automated calibration routine using a global optimiser (such as SCE),

with most (if not all) of the possible parameters allowed to vary within the widest ranges that

are physically plausible for that model and catchment.

Visual inspection of the time series and other statistical measures (see Section 5) from the run

with the parameter set that produced the optimum value of the objective function. This

inspection should aim to identify aspects of the model that are performing well and those that

are performing poorly, to provide insight on the particular parameters that are influencing the

(poor or good) quality of the fit.

Implementation of one or many trials of manual calibration, adjusting the value of one or two

parameters at a time that are likely to be particularly influential on aspects of the modelled

results that the modeller observes are undesirable. The outcome of these manual calibration

runs would provide insight about fixed values for particular parameters, or tighter ranges for

constraining the value of particular parameters in subsequent automated trials. An example

might be that the global optimiser implemented to optimise the NSE finds the baseflow

recession parameter (KBase) for an AWBM model as 0.95/day, but examination of the flow

duration curve and time series reveals that adjusting the parameter to 0.99/day produces a

much better fit to baseflow recessions without compromising the fit to high flows. The modeller

could then choose to either fix KBase at 0.99/day for subsequent trials or to constrain the

KBase parameter within a tighter range (say 0.985 to 1.0/day) for subsequent trials.

Implementation of further trials of automated calibration using a global optimiser but this time

with tighter constraints on most or all of the parameters. Appropriate ranges for parameters

could be assessed either from the previous (manual) calibration step, or from the ranges of

each parameter from the 20, 50 or 100 runs producing the best value of the objective function

from the first automated calibration.

Visual inspection of the time series and statistical measures produced by runs that produced

the optimum value of the objective function and also from runs that produced near to the

optimum value.

Steps 2 through 5 may be repeated several times by the modeller, allowing different sub-sets

of parameters to be fixed and allowing for different allowable ranges for each parameter.


33

Implementation of a local automated optimiser, such as a Rosenbrook search, starting from

the parameter set that is most desirable to the modeller identified in Step 5.

Over the last decade or so, there has been a debate in some parts of the hydrological

modelling community with calibration (particularly automated calibration) to a single objective

function value. This has led to the development of calibration approaches that consider

multiple objectives (Gupta et al., 1998; Madsen et al., 2000). Viney et al. (2009) have

investigated the use of hybrid objective functions (Nash Sutcliffe Coefficient of Efficiency with

various bias constraints) in optimisation of rainfall runoff models for the purposes of estimation

of catchment runoff. Kuczera et al, (2006) and Kavetski et al. (2006a, 2006b) have suggested

an alternative Bayesian approach to model calibration that attempts to deal with

multi-objective optimisation by considering possible calibration solutions that overcome many

of the limitations of single objective optimisation. While these approaches have been

demonstrated in the research domain, practical tools for use by general practitioners for

multi-objective calibration are generally not easily accessible.

Selection of Objective Functions for Automated and Hybrid

Calibration

Automated calibration requires the use of an objective function to direct the optimisation

process. The Source calibration tool implements single objective function optimisation. The

calibration tool reduces the comparison between the observed and modelled data during the

calibration period to a single number to be optimised. Other tools (such as Insight) perform

multiple objective optimisation, which searches for many options of possible solutions that are

optimum for at least one of the multiple objectives that are chosen. Multiple objective

optimisation has not been implemented within the Source calibration tool, although it may be

introduced in later versions. Multiple objective optimisation has the advantage that it explicitly

presents the modeller with many possible parameter sets and it then allows the modeller to

make the choice after the optimisation has been run between parameter sets.

Objective functions are normally chosen as one of the statistics that are discussed in Section

5, or a combination of two statistics from this table. The objective functions that are currently

implemented within the Source calibration tool are:

1 Match to Nash Sutcliffe Coefficient of Efficiency of Daily Flows

2 Minimise Bias between Observed and Modelled

3 Match to Nash Sutcliffe Coefficient of Efficiency of Daily Flows but Penalise Biased

Solutions (NSE-Bias)

4 Match to Nash Sutcliffe Coefficient of Efficiency of Monthly Flows

5 Match to Nash Sutcliffe Coefficient of Efficiency of Monthly Flows but Penalise Biased

Solutions

6 Combined Match to Nash Sutcliffe Coefficient of Efficiency and Match to Flow Duration

Curve (Daily) (NSE-FDC)

7 Combined Match to Nash Sutcliffe Coefficient of Efficiency and Match to Logarithm of

Flow Duration Curve (Daily) (NSE-logFDC)

The choice of any particular objective function will depend on the intended application. Each of

the pre-defined objective functions are formulated to put emphasis (reproduce as closely as

possible) on different flow characteristics.


34

The Nash-Sutcliffe Efficiency (NSE) is commonly used in hydrological modelling to describe

the agreement between the modelled streamflow and observed streamflow time series. A NSE

value of 1.0 indicates that all the modelled daily streamflows are the same as the observed

daily streamflows, and an NSE value of less than zero indicates that that the model simulations

are poorer than simply using the mean observed daily streamflow as the streamflow estimate

for every single day. The daily NSE tends to produce solutions that match high and moderate

flows very well but often will produce poor fits to low flows. NSE will also tend to favour

solutions that provide a good match to the timing and shape of runoff events.

The minimise bias (number 2 in the list) objective function will produce a match on the overall

volume of flow generated but often will produce a poor fit to the timing of flows. It should

normally only be used to refine calibrations determined from other methods with the local

(Rosenbrook) optimiser.

The NSE-Bias objective function (number 3 in the list) formulation makes sure that the models

are calibrated predominantly to optimise NSE while ensuring a low bias in the total streamflow.

It avoids solutions that produce biased estimates of overall runoff, which can produce marginal

improvements in low flow performance over the NSE objective function. However, NSE-Bias

will still be strongly influenced by moderate and high flows and by the timing of runoff events,

which can still often result in poor fits to low flows.

The NSE of monthly flows and NSE-Bias of monthly flows (numbers 4 and 5 on the list above)

can be useful for initial calibration because they tend to find solutions that will match the overall

movement of water through the conceptual stores in the rainfall runoff model, without being

influenced by the timing of individual runoff events.

The NSE-FDC and NSE-logFDC (numbers 6 and 7 on the list above) are hybrid objective

functions that compromise between the fit to the timing of high and moderate flows from the

NSE component and the fit to the shape of the whole flow duration curve (FDC). The

NSE-logFDC (number 7) will produce a closer fit to low flows than the NSE-FDC (number 6).

They require the modeller to select a weighting before the implementation of the optimisation

to the NSE and FDC / logFDC components. Higher weightings (between 0.5 and 1) shift the

balance toward the NSE component (and consequently toward matching timing and high

flows), whilst lower weightings (between 0 and 0.5) shift the balance toward the FDC or

logFDC component (and consequently toward ignoring timing but matching the prevalence of

baseflows). The modeller should therefore consider the influence that the weighting of the two

components in the combined objective function has on the outcomes of their automated

calibration. The weighting can be adjusted during successive runs of automated calibration:

e.g. commencing with a value of 0.5, then modifying to a value of say 0.2 if there is a desire for

more emphasis to be placed on matching the flow duration curve, particularly for low flows.

Automated calibration using four of the objective functions were tested with a global optimiser

(SCE) followed by a local optimiser (Rosenbrook) for a large number of catchments in Eastern

Australia. Testing of the results of the calibration with each of the four objective functions

indicate that the NSE-Bias objective function provides the best estimates of daily flows, timing

and volume ratios in majority of the catchments when compared to the other three objective

functions. If daily sequencing is not important for the intended application, any of the other

three objective functions can be used. If the main interest is to reproduce the low flows whilst

compromising the fit to moderate and high flows, the NSE- logFDC objective function should

be used (see Figure 1 on the next page).

The above discussion underlines the importance at the end, and often during, the calibration

process of testing the performance of parameter values produced by automated calibration

against multiple metrics that are appropriate for the problem that the calibrated rainfall runoff


35

model is being applied to. Following rainfall runoff calibration, the model should perform

adequately against a number of different metrics that represent the characteristics of interest

from Table 5–1 and Table 5–2. Automated calibration presents a risk of selecting model

parameters that reproduce aspects of the flow regime that are well detected by the selected

objective function used in the optimisation at the cost of ignoring other important aspects of the

flow regime.

Further Guidance on Calibration and Validation of Conceptual Rainfall Runoff Models

This section contains a number of miscellaneous guidance about calibration and validation of

conceptual rainfall runoff models.

There are a number of potential problems that can occur during calibration, particularly during

implementation of automated calibration. The following lists a number of potential problems

and issues that should be considered:

Figure 1: Calibration results for different objective functions

• Values of one or more parameters can become “trapped” near the minimum or

maximum values of the range that the modeller has allowed. The modeller should

always check the value of each parameter against the range of values that the particular

parameter was allowed to vary within. A parameter value that is close to one of the limits

(say within 1% of the minimum or maximum value) may indicate that the allowable

0.01

0.1

1

10

100

1 10 100 1000 10000

Days

Daily f

low

du

rati

on

cu

rve (

mm

) Observed

NSE low flow fdc

NSE

NSE fdc

NSE Bias

0

20

40

60

80

1 10 100 1000 10000

Days

Daily f

low

du

rati

on

cu

rve (

mm

)


36

range for the parameter provided as an input to the automated calibration process was

too narrowly defined and that widening the allowable range for the parameter may allow

the automated optimisation to find a significantly better overall result. Of course, there

may be a physical constraint that means that the parameter range and optimum value is

appropriate, e.g. the depth of soil moisture stores cannot be negative and the optimum

value for a particular parameter may then fall close to 0 mm.

• The model may produce a good fit to the objective function but poor performance on

other statistics. As discussed in Section 5, the performance of the model should always

be checked against a number of different statistical measures.

• The model may produce a good fit to calibration data but poor fit to validation data. This

may indicate some inherent non-stationarity in the hydrological response of the

catchment.

• The model may produce an inappropriate split between surface and baseflow

contributions. In this case, objective functions that are weighted toward matching the

baseflow contribution or constraining the ranges of parameters that particularly control

the baseflow response should be considered.

• If an automated calibration routine is implemented, the model may fail to converge to an

acceptable solution. In this case, the input data (particularly rainfall and streamflow)

should be checked. If this appears reasonable, the calibration should be re-run for a

much larger number of iterations and possibly with wider ranges on the parameters to

let it find an optimal solution.

• Local optimisers (Rosenbrock) may find only a local optimum solution and miss a

solution that would otherwise produce a much better fit. This is why global optimisers

should always be implemented as the first step of any automated calibration routine and

the local optimiser only used as the final step in the optimisation process to “sharpen”

the fit.

• The structure of most rainfall runoff models is such that there is a strong degree of

interaction between model parameters. In other words, changing the value of one

parameter of the model can result in changing the value of one or more of the other

parameters of the model to produce a fit to the observed data that is equivalent or

almost equivalent.

• Performance should be tested against a range of metrics for different flow regimes

(high, medium, low), as well as for wet and dry periods. A good all-round performance

indicates that the calibration is representing the catchment under most conditions.

• If the purpose of the modelling is to derive time series of constituent loads for water

quality modelling, it is usually the case that runoff and constituent loads are dominated

by episodic events. Therefore, in looking at the performance of the model, higher

weighting should be placed on statistics that emphasise runoff events than on low flows.

The baseflow index (the ratio of mean annual baseflow to mean annual flow) produced

by the modelled outputs should be similar to the baseflow index from gauged

streamflow data. As a guide, the baseflow index should be within 10% in the modelled

and observed data. Significant differences in baseflow index may result in inaccurate

predictions of loads, particularly when an Event Mean Concentration (EMC)/Dry

Weather Concentration (DWC) generation model is adopted. The baseflow index

produced by the model can normally be corrected relatively easily by manually

calibrating parameters in the model that control recharge to and leakage from baseflow


37

stores, or constraining the range of allowable values of these parameters in an

automated calibration approach.

• In case of nested catchments, some gauges in the catchment will be better rated and

provide more accurate flow measurements than others, particularly in larger flow

events. In deriving a combined objective function to fit parameters across the whole

catchment, weightings should be adjusted in the combined objective function toward

gauges with more reliable data in preference to gauges with less reliable data.

• A check should be performed that the mean annual runoff from the model at gauging

sites is close to the observed mean annual runoff. As a guide, the mean annual runoff

should match to within 5%.

• Gauges should be identified that appear to be outliers, where a good calibration

performance is difficult to obtain. This may identify problems that are not immediately

evident in the data, such as additional inflow or extractions not included in the model,

losses/gains to/from groundwater or rating errors at high flows.

• Comparison of the data over the model calibration and validation periods should be

used to identify other issues such as potential changes in instrumentation, rating or

instrument malfunction that may not have been detected during earlier quality

assurance checks on the recorded data.

• Land use change or events such as bushfire can be evident in the observed data but

may not have been included in the model, which will impact on the performance of the

calibration. Measures to address this should be considered, with options of either

adjustment to the structure of the model to explicitly represent these effects, or

calibrating the model on a different period of data.

• It is generally more difficult for a model calibrated over a wet period to predict runoff over

a dry period compared to a model calibrated over a dry period to predict runoff over a

wet period (Vaze et al., 2010). If possible it is suggested that the calibration flow data

should include both wet and dry periods (rainfall periods about 20% higher and lower

than the long term mean annual rainfall for the catchment).

6.5 Calibration of Regression Models

Empirical regression models are most commonly applied to provide an estimate of flow at one

gauge location, based upon recorded flows at another nearby flow gauge. They may also

incorporate other drivers of runoff response, such as rainfall on the previous day and possibly

total rainfall for a defined number of preceding days.

Regression models are commonly fitted by minimising the least square of the residuals

between flows from the recorded and modelled flow time series. Least squares fitting can be

easily conducted within a spreadsheet, using the common tools provided to undertake those

analyses.

Regression models almost always assume that the catchment is stationary in terms of its

runoff generation characteristics. Care should also be taken to confirm that the period used to

fit the model is climatically representative of the overall simulation period. If the simulation

period is substantially wetter or drier than the model calibration period it is very likely that a

regression model will produce biased results, due to the inherently non-linear rainfall to runoff

conversion relationship that is observed in most catchments.


38

7 Uncertainty and Sensitivity Analysis Relevant sources of uncertainty in typical order of importance include:

1 Model input data including parameters, constants and driving data sets,

2 Model assumptions and simplifications of what the model is representing,

3 The science underlying the model,

4 Stochastic uncertainty (this is addressed under “variability” below),

5 Code uncertainty such as numerical approximations and undetected software bugs.

The potential impacts of the above sources of uncertainty on the decisions that will be made

using the model should be considered early in the modelling process and then re-examined

once the model has been calibrated, validated and applied for scenario runs. Uncertainty

becomes more important for estimation of events in the tails of the probability distribution,

floods and droughts, than it is for consideration of events that are closer to the centre of the

probability distribution (such as estimation of the mean annual runoff from a catchment).

The major uncertainty is most likely associated with the input data sets. There are

uncertainties associated with the measured rainfall, PET and streamflow (associated with the

rating curve). There is spatial variability in rainfall across a catchment which is not captured

when undertaking lumped catchment modelling. If using a single rain gauge, there might be

problems with the location of the rain gauge in terms of capturing a representative rainfall for

all the rainfall events. If using a spatial rainfall product (SILO, Jeffrey et al., 2001 or BAWAP,

Jones et al., 2009), there can be uncertainties introduced because of the method used for

interpolating rainfall between rain gauges and changes in the rain gauge network over time. It

is up to the modeller to check the integrity of the input data and be aware of its limitations. If

data problems cannot be fixed the implications for model calibration (particularly where

optimisation is involved) and interpreting model results should be recognised.

Rainfall runoff models are inherently a simplification of the actual hydrological processes that

are occurring within a catchment. The assumptions and simplification reflected in the structure

of the hydrological model introduce uncertainty into the predictions produced by the model,

which is sometimes referred to as structural uncertainty. Even if the real catchment did behave

in a manner that was identical to the structure represented by the rainfall runoff model, there

would be uncertainty introduced by the unknown values of the parameters of the rainfall runoff

model.

The risks presented by uncertainty introduced by numerical approximations and undetected

software bugs can be mitigated by rigorous testing of the computer software. There is an entire

discipline of the information technology industry devoted to development and application of

tests for software programs. The testing regime for a rainfall runoff model should include:

• Numerical testing of the results of the software code against an independently “coded”

solution that has been constructed in a spreadsheet or in a previous implementation

(e.g. in FORTRAN);

• Testing of the user interface;


39

• Testing the results that are produced by the model for extreme values of each of the

model parameters, i.e. parameter values at the minimum and maximum values of their

feasible limits;

• Testing that the model internally preserves a mass balance for each time step, i.e. the

change in water stored within the conceptual stores in the model is equal to rainfall less

evaporation less seepage losses less total runoff;

• Testing that one version of the model replicates the results of a previous version,

particularly where there is ongoing development of the model and/or the framework that

the model is implemented within.

7.1 Sensitivity Analysis

A relatively simple means of producing a quantitative estimate of uncertainty from a rainfall

runoff model is sensitivity analysis. In sensitivity analysis, one parameter of the model is

typically varied at a time and the model re-run to test the change in the output produced by the

change in the single input parameter. A typical approach might be to modify the value of each

of the model parameters by +10% to check the percentage change in the mean annual runoff,

followed by modifying each of the model parameters by -10% to check the mean annual runoff.

This analysis would provide the modeller with an appreciation of how sensitive the mean

annual runoff generated for the catchment is to a simple change in each of the input parameter

values. Consistent with the discussion in Section 5, several metrics that reflect the purpose of

the model should be checked during the sensitivity analysis.

Some deficiencies of sensitivity analysis are that:

• It only tests for the influence of parameter uncertainty and ignores uncertainty

introduced by the model structure;

• The common method of testing for the same proportional change in each parameter

(e.g. the +/- 10% change in each parameter value in the example above) does not

usually reflect the fact that some parameters are inherently more difficult to estimate via

model calibration and therefore more uncertain than other parameters. It can therefore

produce misleading inferences because small changes in the value of one parameter

may produce large changes in the output but because of this sensitivity the value of this

parameter may be defined to within a relatively narrow range;

• There is typically strong interaction between parameters of rainfall runoff models, so

that the change in the value of one parameter would be compensated for by the change

in the value of one or more of the other parameters of the model. Sensitivity analysis on

single parameters ignores parameter interaction.

7.2 Application of Multiple Parameter Sets

The process of calibrating rainfall runoff models using an automated optimiser normally

involves a search of many thousands of possible parameter sets. While the focus of

optimisation is often on the single parameter set that best calibrates the model, the

optimisation process will normally return a large number (typically hundreds) of parameter sets

that produce a performance from the model that is almost as good as the optimum parameter

set. The results of the rainfall runoff model produced by these “almost as good” parameter sets

can be used to quantify the uncertainty in the model predictions produced by parameter


40

uncertainty. A typical approach, if 2000 model runs had been completed during the automated

calibration process, would be to select the 100 model runs that produced the best value of the

selected objective function to characterise the uncertainty in the time series produced by the

rainfall runoff model.

This approach avoids the second and third deficiencies of sensitivity analysis because:

• Parameters that are well defined via the calibration process will be represented by a

relatively tight distribution of values amongst the parameter sets that almost produce

the optimum calibration;

• Interaction between parameter values is retained by selecting each parameter set.

However, a limitation of this method is that uncertainty introduced by model structure is

ignored, which was also the case for sensitivity analysis.

7.3 More Advanced Quantitative Uncertainty Analysis

A further advancement on the approach of testing many possible parameter sets that almost

calibrate the model would be to calibrate several different rainfall runoff model structures to the

same gauged data time series, which would return multiple possible parameter sets for each

of the model structures, and then to choose combinations of model structure and parameter

set that almost produce the optimum value of the objective function. An example might involve

automated calibration of AWBM, SMARG, SimHyd, Sacramento, GR4J and IHACRES models

to a particular catchment and then selecting the top 300 sets of parameters (along with the

model structure) that produced the optimum value of the objective function from amongst all

the model runs with all six selected model structures.

The Bayesian Total Error Analysis methodology (BATEA) provides the opportunity to directly

address all sources of uncertainty (input, model and response error) in the calibration of

conceptual rainfall-runoff (CRR) models. BATEA has demonstrated the potential to overcome

the parameter biases introduced by poor conceptualisations of these sources of errors and

enhance regionalisation capabilities of hydrological models (Thyer et al., 2007). Application of

BATEA is currently within the domain of applied research.

7.4 Consideration of Uncertainty in Practical Applications of

Rainfall Runoff Models

A hierarchy of techniques exist, as described above, for quantitative estimation of uncertainty

for practical applications of rainfall runoff models.

The potential impact of uncertainty in rainfall runoff modelling on the decisions that are to be

made using that model should be assessed before quantitative estimation of uncertainty is

undertaken. In many situations, decisions will be made informed from the “best estimate”

rainfall runoff model results and will implicitly ignore uncertainty.

Sensitivity analysis, despite its limitations, at least can be informative about the possible range

of solutions that might be produced due to uncertainty in the parameter values. Consideration

of uncertainty in this reasonably simplistic manner can often provide an indicative range of the

possible range of solutions that may be produced.

Application of multiple parameter sets is facilitated using existing tools that are available to

hydrological modellers (including PEST and the Source calibration tool) and it overcomes two


41

of the major limitations of simple sensitivity analysis. In situations where quantifying

uncertainty, at least the uncertainty introduced by model parameter uncertainty, is important

then this is a viable approach that should be applied by the modeller.

More advanced uncertainty analysis techniques, including BATEA, are likely to advance as

further research is conducted but the tools to implement these techniques are not currently

available to most practitioners. If uncertainty in the model results is a primary driver of the

decisions to be made on the basis of the rainfall runoff model results then these more

advanced techniques of uncertainty quantification should be adopted.


42

8 Concluding remarks There are a number of methods of undertaking rainfall-runoff modelling and all have strengths

and weaknesses given the data available and the purpose that the model output will be used

for. Simple mean annual flow assessments may utilise simple equations based on the size of

the project and outcomes required.

Generally where the model results will be used for planning processes on larger scale

catchments more detailed models are appropriate. As indicated earlier, conceptual lumped

parameter models are generally the models of choice in Australia.

More complex models than these tend to be data intensive but should be considered when the

results (generated flows or some other parameter such a water quality constituent) may be

heavily biased if a simpler method is used to model rainfall-runoff.

The conceptual rainfall-runoff models can generally be calibrated to reproduce the daily

observed streamflow well and the transfer of parameter values from a gauged catchment

nearby can model runoff reasonably well in ungauged areas.

Current areas of research that are likely to further develop best practice in rainfall runoff

modelling are:

• More advanced methods for regionalisation of rainfall runoff model parameters from

gauged to ungauged catchments;

• Development of more sophisticated approaches for use of ensembles of results from

several different rainfall runoff models to provide an optimum estimate of the flow time

series;

• Development of more sophisticated and robust approaches for using parameter sets

derived from multiple gauged (donor) catchments in ungauged catchments;

• Use of remotely sensed data sets, such as actual evapotranspiration, soil moisture and

vegetation cover to inform the structure, parameter values and calibration of rainfall

runoff models.


43

9 References Arora, V. K., 2002. The use of the aridity index to assess climate change effect on annual

runoff. Journal of Hydrology, 265, 164– 177.

Bárdossy, A. (2007) Calibration of hydrological model parameters for ungauged catchments,

Hydrology and Earth System Sciences, 11 (2): 703-710

Black, D., Wallbrink, P., Jordan, P., Waters, D., Carroll, C., Blackmore, J. (2011). Guidelines

for water management modelling: Towards best-practice model application.

eWater Cooperative Research Centre. ISBN 978-1-921543-46-3.

Beven, K. (1989). Changing ideas in hydrology – the case of physically-based models. Journal

of Hydrology, 105, pp 157-172.

Boughton, W. C. (2004). The Australian water balance model. Environmental Modelling and

Software, 19, 943–956.

Boyle, D.P., Gupta, H.V. and Sorooshian, S. (2000) Toward improved calibration of hydrologic

models: Combining the strengths of manual and automatic methods, Water

Resources Research, 36 (12): 3663-3674.

Budyko, M. I. (1958). The Heat Balance of the Earth's Surface, translated from Russian by N.

A. Stepanova, 259 pp. U.S. Dep. of Commerce., Washington.

Burnash, R. J. C., Ferral, R. L. and McGuire, R. A. (1973). A Generalised Streamflow

Simulation System – Conceptual Modelling for Digital Computers. Joint Federal

and State River Forecast Center, Sacramento, Technical Report, 204 pp.

Burnash, R.J.C. (1995). The NWS River Forecast System - Catchment Modeling, pp311-366

in V.P.Singh (ed) Computer Models of Watershed Hydrology, Water Resources

Publications (Colarado, US).

Hydrology, 311-366.Chiew, F. H. S., Peel, M. C. and Western, A. W. (2002). Application and

testing of the simple rainfall-runoff model SIMHYD. In: Singh, V. P. and Frevert,

D. K., (eds.), Mathematical Models of Small Watershed Hydrology and

Applications. Water Resources Publications, Littleton, USA, pp. 335–367.

Chiew, F.H.S. (2010). Lumped conceptual rainfall-runoff models and simple water balance

methods: overview and applications in ungauged and data limited regions.

Geography Compass, In Press.

CRCCH (2005a), General approaches to modelling and practical issues of model choice.

Cooperative Research Centre on Catchment Hydrology, (Canberra).

http://www.toolkit.net.au/tools/ModelChoice/MC-2.pdf

CRCCH (2005b), Water Quality Models: Sediments and Nutrients, Cooperative Research

Centre on Catchment Hydrology, (Canberra).

http://www.toolkit.net.au/tools/ModelChoice/MC-1.pdf

Croke, B. F. W., Andrews, F., Jakeman, A. J., Cuddy, S. M. and Luddy, A. (2006). IHACRES

Classic Plus: A redesign of the IHACRES rainfall-runoff model. Environmental

Modelling and Software, 21, 426–427.


44

Duan, Q. Y., Gupta, V. K. and Sorooshian, S. (1993). Shuffled complex evolution approach for

effective and efficient global minimization. J. Optimiz. Theory Appl. 76(3),

501–521.

Duan, Q, H.V. Gupta, S. Sorooshian, A.N. Rousseau and R Turcotte (ed.) (2003). Calibration

of Watershed Models., Water Science and Application No 6, American

Geophysical Union, Washington D.C , USA.

Fu, B.P., 1981. On the calculation of the evaporation from land surface (in Chinese). Scientia

Atmospherica Sinica, 5, 23– 31.

Gupta, H. V., S. Sorooshian, and P. O. Yapo, (1998) Toward improved calibration of

hydrologic models: multiple and noncommensurable measures of information,

Water. Resourc.Res., 34:751-763.

Jeffrey, S. J., Carter, J. O., Moodie, K. B. and Beswick, A. R. (2001). Using spatial interpolation

to construct a comprehensive archive of Australian climate. Environmental

Modelling and Software, 16, 309–330.

Jones, D. A., Wang, W. and Fawcett, R. (2009). High-quality spatial climate data-sets for

Australia, Australian Meteorological and Oceanographic Journal 58:4, pp.

233-248.

Kavetski, D and Kuczera, G. and Franks, S.W. (2006a) Bayesian analysis of input uncertainty

in hydrological modeling: 1. Theory, Water Resources Research, 42 (3):

W03407.

Kavetski, D and Kuczera, G. and Franks, S.W. (2006b) Bayesian analysis of input uncertainty

in hydrological modeling: 2. Application, Water Resources Research, 42 (3):

W03408.

Kuczera, G, Kavetski D, Franks S and Thyer M (2006) Towards a Bayesian total error analysis

of conceptual rainfall-runoff models: Characterising model error using

storm-dependent parameters, J. Hydrology 331(1-2): 161-177.

Kavetski,D., S.W. Franks and G. Kuzera (2003) Confronting Input Uncertainty in

Environmental Modelling. In ‘Calibration of Watershed Models’, Water Science

and Application No 6, American Geophysical Union, Washington D.C , USA.

pp49-68

Liang, X., Lettenmaier, D.P., Wood, E.F., and Burges, S.J., 1994. A Simple hydrologically

Based Model of Land Surface Water and Energy Fluxes for GSMs, J. Geophys.

Res., 99(D7), 14,415-14,428.

Madsen, H. (2000) Automatic calibration of a conceptual rainfall-runoff model using multiple

objectives, J. Hydrology, 235 (3-4): 276-288.

Merz, R., and Bloschl, G. (2004). Regionalisation of catchment model parameters, Journal of

Hydrology, 287, 95-123.

Nash, J.E. and Sutcliffe, J. V. (1970). River flow forecasting through conceptual models, I, A

discussion of principles. J. Hydrol, 10, 282–290.

NIST and SEMATECH (2010). NIST/SEMATECH e-Handbook of Statistical Methods,

http://www.itl.nist.gov/div898/handbook/, United States Department of

Commerce, Last updated 23 June 2010, Accessed 7 July 2011.

http://www.itl.nist.gov/div898/handbook/


45

Northcote, K. H. with Beckmann, G. G., Bettenay, E., Churchward, H. M., Van Dijk, D. C.,

Dimmock, G. M., Hubble, G. D., Isbell, R. F., McArthur, W. M., Murtha, G. G.,

Nicolls, K. D., Paton, T. R., Thompson, C. H., Webb, A. A. and Wright, M. J.

(1960-1968). Atlas of Australian Soils, Sheets 1 to 10. With explanatory data

(CSIRO Aust. and Melbourne University Press: Melbourne).

Oudin, L., Andreassian, V., Perrin, C., Michel, C. and Le Moine, N. (2008). Spatial proximity,

physical similarity, regression and ungauged catchments: A comparison of

regionalisation approaches based on 913 French catchments. Water Resources

Research, 44, W03413, doi:10.1029/2007WR006240.

Parajka, J., Merz, R. and Bloschl, G. (2005). A comparison of regionalisation methods for

catchment model parameters, Hydrology and Earth System Sciences, 9,

157-171.

Perrin, C., Michel, C. and Andreassian, V. (2003). Improvement of a parsimonious model for

streamflow simulations. Journal of Hydrology, 279, 275–289.

Refsgaard and Henriksen (2004) Modelling guidelines--terminology and guiding principles,

Advances in Water Resources 27, 71-82.

Rosenbrock, H. H. (1960). An automatic method for finding the greatest or least value of a

function, Computer Journal, 3, 175–184.

Stenson, M. P, Littleboy, M. and Gilfedder, M. (2011). Estimation of water and salt generation

from unregulated upland catchments. Environmental Modelling & Software, 26,

1268-1278.

Thyer, M.A., Renard, B., Kavetski, D., Kuczera, G. and Srikanthan, S. (2007) Bayesian Total

Error Analysis For Hydrological Models: Preliminary Evaluation Using Multi-Site

Catchment Rainfall Data, In Oxley, L. and Kulasiri, D. (eds) Proc. MODSIM 2007

International Congress on Modelling and Simulation. Modelling and Simulation

Society of Australia and New Zealand, December 2007, pp. 74-80. ISBN :

978-0-9758400-4-7.

Van Dijk, A. I. J. M. (2010), AWRA Technical Report 3. Landscape Model (version 0.5)

Technical Description, WIRADA / CSIRO Water for a Healthy Country Flagship,

Canberra

(http://www.clw.csiro.au/publications/waterforahealthycountry/2010/wfhc-aus-wa

ter-resources-assessment-system.pdf).

Vaze, J., Barnett, P., Beale, G.T.H., Dawes, W., Evans, R., Tuteja, N.K., Murphy, B., Geeves,

G. and Miller, M. (2004). Modelling the effects of landuse change on water and

salt delivery from a catchment affected by dryland salinity in south-east Australia,

Hydrological Processes, Vol. 18, pp. 1613 - 1637.

Vaze, J., Post, D. A., Chiew, F. H. S., Perraud, J.-M., Viney, N., Teng, J., (2010) Climate

nonstationarity - Validity of calibrated rainfall-runoff models for use in climate

change studies, J. Hydrol. (2010), doi:10.1016/j.jhydrol.2010.09.018.

Vaze J, Chiew FHS, Perraud JM, Viney NR, Post DA, Teng J, Wang B, Lerat J, Goswami M.

(2011a). Rainfall-runoff modelling across southeast Australia: datasets, models

and results. Australian Journal of Water Resources, Vol 14, No 2, pp. 101-116.


46

Vaze, J., Post, D. A., Chiew, F. H. S., Perraud, J.-M., Teng, J., Viney, N. (2011b). Conceptual

Rainfall–Runoff Model Performance with Different Spatial Rainfall Inputs, Journal

of Hydrometeorology, Vol. 12, No. 5. pp 1100-1112, doi:

10.1175/2011JHM1340.1.

Vaze, J., Perraud, J-M., Teng, J., Chiew, F.H.S., Wang, B. (2011c). Estimating regional model

parameters using spatial land cover information – implications for predictions in

ungauged basins. 19th MODSIM Congress, Perth, Australia, 12-16 December

2011. http://mssanz.org.au/modsim11.

Vaze, J., Perraud, JM., Teng, J., Chiew, F. H. S., Wang, B., Yang, Z. (2011d). Catchment

Water Yield Estimation Tools (CWYET). 34th IAHR World Congress, 27th June to

1st July, Brisbane, Australia.

Viney N, Vaze J, Chiew F, Perraud, J, Post D and Teng J. (2009). Comparison of multi-model

and multi-donor ensembles for regionalisation of runoff generation using five

lumped rainfall-runoff models in: 18th World IMACS Congress and MODSIM09

International Congress on Modelling and Simulation, July 2009, Cairns:

Modelling and Simulation Society of Australian and New Zealand and

International Association for Mathematics and Computers in Simulation:

3428-3434.

WMO (2008) Guide to Hydrological Practices. Volume I: Hydrology – From Measurement to

Hydrological Information. WMO No. 168. 6th ed. World Meteorological

Organisation, Geneva, Switzerland. ISBN 978-92-63-10168-6.

WMO (2009) Guide to Hydrological Practices. Volume II: Management of Water Resources

and Application of Hydrological Practices. WMO No. 168. 6th ed. World

Meteorological Organisation, Geneva, Switzerland. ISBN 978-92-63-10168-6.

Zhang, L., Hickel, K., Dawes, W.R., Chiew, F.H.S., Western, A.W., Briggs, P.R., 2004. A

rational function approach for estimating mean annual evapotranspiration. Water

Resources Research, 40(2), W02502.

Zhang, Y. Q. and Chiew, F. H. S. (2008). Can remote sensing data improve short-term

rainfall-runoff simulation?, paper presented at Water Down Under 2008, April

2008, Adelaide: Engineers Australia.

Zhang, Y. and Chiew, F. H. S. (2009). Relative merits of different methods for runoff

predictions in ungauged catchments. Water Resources Research, 45, W07412,

doi:10.1029/2008WR007504.

http://mssanz.org.au/modsim11

eWater Cooperative Research Centre eWater Limited ABN 47 115 422 903

Innovation Centre, Building 22

University Drive South Bruce, ACT, 2617, Australia

T: +61 2 6201 5168 [email protected]

www.ewater.com.au

© 2012 eWater Ltd

guidelines for rainfall-runoff modelling - · pdf fileguidelines for rainfall-runoff modelling...

Documents