ewater modelling guidelines rrm (v1 mar 2012)
DESCRIPTION
Water-Modelling-GuidelinesTRANSCRIPT
-
Towards best practice model application
Vaze, J., Jordan, P., Beecham, R., Frost, A., Summerell, G.
Guidelines for rainfall-runoff modelling
-
Guidelines for rainfall-runoff modelling
2
Copyright Notice
2012 eWater Ltd
Legal Information
This work is copyright. You are permitted to copy and reproduce the information, in an unaltered form, for
non-commercial use, provided you acknowledge the source as per the citation guide below. You must not
use the information for any other purpose or in any other manner unless you have obtained the prior written
consent of eWater Ltd.
While every precaution has been taken in the preparation of this document, the publisher and the authors
assume no responsibility for errors or omissions, or for damages resulting from the use of information
contained in this document. In no event shall the publisher and the author be liable for any loss of profit or
any other commercial damage caused or alleged to have been caused directly or indirectly by this
document.
Citing this document
Vaze, J., Jordan, P., Beecham, R., Frost, A., Summerell, G. (eWater Cooperative Research Centre 2011)
Guidelines for rainfall-runoff modelling: Towards best practice model application.
Publication date: March 2012 (Version 1.0)
ISBN 978-1-921543-51-7
Acknowledgments
eWater CRC acknowledges and thanks all partners to the CRC and individuals who have contributed to the
research and development of this publication.
We acknowledge the inputs from the hydrology group in DERM, Queensland, and Mark Alcorn from SA
Department for Water. We thank Matthew Bethune, Peter Wallbrink, Dugald Black, Jin Teng, Jean-Michel
Perraud, Melanie Ryan, Bill Wang, David Waters, Richard Silberstein, Geoff Podger, David Post, Cuan
Petheram, Francis Chiew and Andrew Davidson for useful discussions.
eWater CRC gratefully acknowledges the Australian Governments financial contribution to this project
through its agencies, the Department of Innovation, Industry, Science and Research, the Department of
Sustainability, Environment, Water, Population and Communities and the National Water Commission
For more information:
Innovation Centre, Building 22
University Drive South
Bruce, ACT, 2617, Australia
T: +61 2 6201 5834 (outside Australia)
Support: 1300 5 WATER (1300 592 937)
www.ewater.com.au
-
Guidelines for rainfall-runoff modelling
3
Contents
1 Introduction ................................................................. 5
1.1 Background ........................................................................................................................... 5
1.2 Definition of Best Practice ..................................................................................................... 5
1.3 Scope .................................................................................................................................... 6
2 Overview of procedure for rainfall-runoff modelling .... 8
2.1 Problem definition ................................................................................................................. 8
Problem statement and setting objectives ............................................................................ 8
Understanding the problem domain ...................................................................................... 8
Metrics and criteria and decision variables ........................................................................... 9
Performance across multiple catchments and subcatchments ............................................. 9
2.2 Option modelling ................................................................................................................... 9
Methodology development .................................................................................................... 9
Collate and review data ...................................................................................................... 10
Setting up and building a model ......................................................................................... 10
Calibration and Validation ................................................................................................... 10
Sensitivity/uncertainty analysis ........................................................................................... 12
Documentation and Provenance ........................................................................................ 12
Model acceptance and accreditation .................................................................................. 13
Use of accepted/accredited model...................................................................................... 13
3 Model choice ............................................................. 14
3.1 Model selection ................................................................................................................... 14
3.2 Available models ................................................................................................................. 15
Empirical methods .............................................................................................................. 15
Large scale energy-water balance equations ..................................................................... 16
Conceptual Rainfall-Runoff Models .................................................................................... 16
Landscape daily hydrological models ................................................................................. 17
Fully distributed physically based hydrological models which explicitly model hillslope and
catchment processes .......................................................................................................... 17
4 Collate and Review Data........................................... 20
4.1 Catchment details ............................................................................................................... 21
Location of gauges (streamflow, rainfall and evaporation) ................................................. 21
Topography and Catchment Areas ..................................................................................... 21
Soil types ............................................................................................................................ 21
Vegetation ........................................................................................................................... 21
-
Guidelines for rainfall-runoff modelling
4
Water Management Infrastructure ...................................................................................... 22
4.2 Flow data ............................................................................................................................ 22
4.3 Rainfall ................................................................................................................................ 23
4.4 Evapotranspiration .............................................................................................................. 24
5 Statistical Metrics for Testing Performance .............. 25
6 Calibration and validation .......................................... 27
6.1 Calibration ........................................................................................................................... 27
6.2 Validation ............................................................................................................................ 27
6.3 Calibration and Validation of Models to Single Gauge Sites, Multiple Gauge Sites and Regionalisation of Model Parameter Sets ......................................................................... 29
6.4 Automated, Manual and Hybrid Calibration Strategies ....................................................... 30
Manual Calibration .............................................................................................................. 30
Automated Calibration ........................................................................................................ 31
Hybrid Calibration Strategies .............................................................................................. 32
Selection of Objective Functions for Automated and Hybrid Calibration ............................ 33
6.5 Calibration of Regression Models ....................................................................................... 37
7 Uncertainty and Sensitivity Analysis ......................... 38
7.1 Sensitivity Analysis ............................................................................................................. 39
7.2 Application of Multiple Parameter Sets ............................................................................... 39
7.3 More Advanced Quantitative Uncertainty Analysis ............................................................. 40
7.4 Consideration of Uncertainty in Practical Applications of Rainfall Runoff Models .............. 40
8 Concluding remarks .................................................. 42
9 References ................................................................ 43
-
Guidelines for rainfall-runoff modelling
5
1 Introduction
1.1 Background
Reliable estimates of stream flow generated from catchments are required as part of the
information sets that help policy makers make informed decisions on water planning and
management. The characteristics of the streamflow time series that influence water resources
system modelling and planning can include the sequencing of flows on daily and longer time
steps, spatial and temporal variability of flows, seasonal distribution and characteristics of high
and low flows.
The best available estimate of streamflow would be expected to come from water level
observations made at a gauging station, converted to flow estimates using a well defined and
stable rating curve. However, these observations are only available for limited number of
gauging locations and for limited time span. Estimates for ungauged locations and for a much
longer time period are needed for contemporary water management, and ways to make
estimates for future possible conditions are also required.
A range of methods are available to estimate streamflow from catchments, using observed
data wherever possible, or estimating by empirical and statistical techniques, and more
commonly using rainfall-runoff models. The modelling approach used to estimate streamflow
by different water agencies and consultants varies across Australia and depends on the
purpose of the modelling, time and money available, and the tools and skills available within
the organisation. With increasing levels of inter-agency collaboration in water planning and
management, development of a best practice approach in rainfall runoff modelling is desirable
to provide a consistent process, and improve interpretation and acceptability of the modelling
results.
The purpose of this document is to provide guidance on the best practice for implementing fit
for purpose rainfall-runoff models, covering topics such as setting modelling objectives,
identifying data sources, quality assuring data and understanding its limitations, model
selection, calibration approaches, and performance criteria for assessing fitness for purpose
1.2 Definition of Best Practice
Best Practice Modelling can be defined as a series of quality assurance principles and actions
to ensure that model development, implementation and application are the best achievable,
commensurate with the intended purpose (Black et al., 2011).
What is in practice best achievable, commensurate with the intended purpose may be
subject to data availability, time, budget and other resourcing constraints. Hence, what is
meant by the term Best Practice Modelling can vary. Not only does it depend on the
circumstances of the project, particularly providing results that are fit for the intended purpose,
but it also depends to a great degree on interpretation in peer review. This, in turn, will be
influenced by the general state of knowledge and technology in the modelling field, which can
-
Guidelines for rainfall-runoff modelling
6
be expected to progressively develop over time (such as new remote sensing data sources
coming on line, and new computing hardware and software), as well as data, time, budget and
resourcing constraints. Best Practice Modelling provides for a strategic approach to
modelling which enables the trade-offs that may be imposed by these constraints to be better
managed, and assists in identifying priorities for addressing model and data limitations.
1.3 Scope
The eWater CRC has prepared generic Best Practice Modelling guidelines (Black et al., 2011).
They aim to provide for an integrated approach that enables interactions and feedbacks
between all domains relevant to water management (e.g. hydrological, ecological,
engineering, social, economic and environmental) to be considered.
The procedure in that guidance is intended to be flexible enough to accommodate variations in
the meaning of the term Best Practice Modelling and also allow for continuous improvement
as the state of knowledge and technology in the modelling field develops and improves.
The eWater CRC will also provide guidelines to support the BPM guidelines in specific areas of
hydrological modelling that relate to the products that they are developing. This guideline is
intended to address rainfall-runoff model application with the objectives being to provide water
managers, consultants and research scientists with information on rainfall-runoff models and
how to choose one that is fit for purpose, the data available to develop them, and the
calibration and validation methodologies.
There are a number of different purposes that a rainfall runoff model may be applied within an
overall water resources or catchment modelling framework, such as eWater Source. Most of
these purposes relate to providing information to support decision making for some water
management policy. In particular, this can involve:
Understanding the catchment yield, and how this varies in time and space, particularly
in response to climate variability: seasonally, inter-annually, and inter-decadally.
Estimating the relative contributions of individual catchments to water availability over a
much larger region, e.g. valley or basin scale.
Estimating how this catchment yield and water availability might change over time in
response to changes in the catchment, such as increasing development of farm dams,
or changes in land-use and land management.
In some instances with a high quality network of long term stream gauges, most of this type of
information can be estimated from the observations. However, the more common case is
where there is some combination of short term stations, variable quality data, and gaps in
spatial coverage. In these cases, spatial and temporal gaps in the information can be
estimated using rainfall runoff models to:
Infill gaps caused by missing or poor quality data in an observed data series for a
gauged catchment.
Estimate flows for a gauged catchment for periods before the observed flow record
started or after when the observed flow record ends.
Estimate flows for an ungauged catchment.
Estimating flows from ungauged subcatchments within an overall gauged catchment.
-
Guidelines for rainfall-runoff modelling
7
Forecast flows for some immediate future period (typically for a period of between a few
days and a few months), conditioned on current (or recent) observations of the
catchment state.
Assess the impacts of human influences within a gauged catchment (for example
landuse or vegetation cover change) and simulating the flows that would have occurred
for the historical climate sequence with modified catchment conditions. This may
include assessment of catchment conditions that may be non-stationary in either the
observed record or for the simulation.
Assess the potential impact of climate variability and/or climate change on flows from a
gauged catchment.
In some cases, several of the above purposes may be satisfied by rainfall runoff modelling for
the same catchment. There are similarities in the approach that is taken to rainfall runoff
modelling, even though the purpose may differ.
-
Guidelines for rainfall-runoff modelling
8
2 Overview of procedure for
rainfall-runoff modelling The generic guidelines (Black et al., 2011) outline a procedure for applying a hydrological
model. This can be summarised as occurring in 4 phases:
1 Project management,
2 Problem definition,
3 Option modelling,
4 Compare Options and select the best.
This guidance deals only with problem definition and option modelling because the first and
last phases are discussed sufficiently for the purpose of rainfall-runoff modelling in the generic
guidelines. A further reason is that rainfall-runoff modelling is usually only a part of a larger
hydrological modelling project and these phases would be most appropriately considered in
the context of that larger project. Specific aspects of project management and option
comparison that are directly applicable to the development of a rainfall-runoff model, such as
accreditation, are dealt with at appropriate points in this guidance.
2.1 Problem definition
Problem statement and setting objectives
The problem to be addressed must be clearly articulated to minimise the risk that the wrong
tool will be used for the job. The problem statement will give direction on what objectives will be
considered in developing the rainfall-runoff model. As many water management decisions will
often have more than one goal it will be important to ensure these are all identified.
Sometimes it can be useful to express objectives in a hierarchy that shows primary objectives,
secondary objectives and so on. In this regard, consideration should also be given to
possible additional future objectives and goals that could be met based on the current project
or on future projects that build upon the model established in the current project. The decision
on which option offers the best solution should be based upon whether, or how well, each
option meets the agreed objectives (also see section 2.2.1 and 2.2.2 in the generic
guidelines).
Understanding the problem domain
The choice of the rainfall-runoff model will vary based on the purpose the modelling is being
done for, e.g., to understand seasonal low flow characteristics for an in-stream environmental
need; or to assess over-bank flow frequency; or to estimate overall catchment yield on an
average annual basis. The model selected, data required, and calibration approach adopted
-
Guidelines for rainfall-runoff modelling
9
should reflect this requirement. Where the same model may be used for two or more different
purposes, there may also be a need to focus the calibration on a number of different flow
regimes. If rough flow estimates are required over large areas and the runoff generation
methodology should be consistent then the data and modelling process will differ again.
Metrics and criteria and decision variables
Model calibration is a process of systematically adjusting model parameter values to get a set
of parameters which provides the best estimate of the observed streamflow (in the case of
rainfall-runoff models). The process of determining which particular set of parameter values
are best for the intended purpose depends on what comparison metrics are used. Metrics
should be used to quantify the acceptability of the developed model. In all cases graphical
assessment and statistical results of some sort will be assessed to identify the ability of the
calibrated model to reproduce the flows calibrated against.
Different metrics will be more effective in determining model appropriateness to meet different
objectives. What these are should be considered when the problem is being defined.
Understanding appropriate metrics allows model acceptance criteria to be identified.
Performance across multiple catchments and subcatchments
In some situations, the purpose of rainfall runoff modelling is to produce an estimate of the
runoff at a single location where there is a streamflow gauge. If this is the case, the calibration
and validation process may be performed for the single gauged catchment. This approach is
justifiable in situations where gauged data is available for most of the period that flow results
are required for and the purpose of the rainfall runoff model is to infill missing data during the
period of record. It may also be justifiable where there is a requirement to extend the period of
record at the single gauge.
A much more common situation is that flow time series estimates are required at several
locations and that gauged streamflow data is also available at several locations. The locations
where flow estimates are required may or may not overlap with the locations where the flow
data is also available. An objective of any project that involves the application of rainfall runoff
models to multiple catchments or subcatchments should be to demonstrate consistency in the
rainfall runoff model response between those catchments and to explain systematic
differences in the hydrological response between catchments and subcatchments in an
appropriate and logical manner.
2.2 Option modelling
This section describes the process of developing a rainfall-runoff model, further details on key
components are provided in later sections.
Methodology development
The models and methodology employed should be appropriate for the purpose that the model
will be used for. The choices made will be directed by the problem definition developed and
any other information at hand to the modeller. Detail on the models available and guidance on
selecting models and methodology that is fit for purpose is provided in Section 3.
-
Guidelines for rainfall-runoff modelling
10
Collate and review data
The amount and quality of data available to develop a model should be determined at the
outset of the project. This can influence the selection of models, the performance criteria, and
the approach to calibrate models. A bare minimum data set sufficient to make an approximate
estimate of mean annual catchment yield would include catchment area along with spatial and
temporal characteristics of rainfall and potential evapotranspiration (PET). A comprehensive
data set would include long-term streamflow measurements and rainfall and PET data
collected at one or more locations within the catchment along with land use coverage,
vegetation cover and impervious area information, including changes over time.
The quality of the data should be reviewed prior to using to detect errors, non-stationarity if
any, and understand uncertainties that may influence estimates. Some methods are
discussed in section 4.
Setting up and building a model
The catchment characteristics are considered along with the knowledge on data available and
any other information available to the modeller. The rainfall-runoff model chosen is
conceptualised and an initial parameter set is identified.
When the model is first set up consideration should be given to all constraints which are
limiting and their effects on the modelling. Section 5 provides more details associated with this
step.
Calibration and Validation
Model calibration is a process of systematically adjusting model parameter values to get a set
of parameters which provides the best estimate of the observed streamflow (in the case of
rainfall-runoff models).
The term validation, as applied to models, typically means confirmation to some degree that
the calibration of the model is acceptable for the intended purpose (Refsgaard and Henriksen,
2004). In the context of rainfall runoff modelling, validation is a process of using the calibrated
model parameters to simulate runoff over an independent period outside the calibration period
(if enough data is available) to determine the suitability of the calibrated model for predicting
runoff over any period outside the calibration period. If there is not enough data available, the
validation may be performed by testing shorter periods within the full record.
It is normal in research studies to split the observed data sets into calibration and validation
period prior to the study, to demonstrate the performance of the model under both sets of
conditions. Use of this approach can cause problems in practical applications if a model
demonstrates acceptable performance for the calibration data set but produces unsatisfactory
results for the validation data set. An alternative approach in this situation is to calibrate the
rainfall runoff model to all available data but to demonstrate that the performance of the model
is satisfactory over different sub-sets of the period that observed data is available.
Further discussion of model calibration and validation is provided in Section 6.
It is a very common situation in a project that involves rainfall runoff modelling for flow time
series to be required for several catchments or subcatchments within the model domain and
for data to be available from two or more stream flow gauges to facilitate calibration and
-
Guidelines for rainfall-runoff modelling
11
validation. At locations where gauged flows are available and flow estimates are required, two
options are available to the modeller:
The rainfall runoff models may be calibrated independently for each gauged catchment.
In this case, independent parameter sets will be derived for the rainfall runoff models of
each catchment; or
A joint calibration may be performed, with rainfall runoff models calibrated with
consistent parameters to fit to the gauge records from two or more gauges. In this case,
a single set of rainfall runoff model parameters will be produced for all of the catchments
that represent a compromise to fit the flows at all of the gauges within that group.
Consideration should be given at the outset of modelling to the approach that will be used for
dealing with flows from multiple catchments and subcatchments and from multiple gauges.
The strategy for dealing with this issue should be documented at this point and revised, if
necessary, during the process of calibrating and validating the models.
Calibration of a rainfall runoff model normally involves running the model may times, trialling
different values of parameters, with the aim of improving the fit of the model to the calibration
data. Calibration can be facilitated:
Manually, with the modeller setting the parameter values, running the model to inspect
the results and then repeating this process many times;
Using automated optimisation, with an optimiser algorithm running the model hundreds
or thousands of times with different parameter values; or
Using a hybrid approach of automated optimisation phases, interspersed with manually
implemented trials of parameter sets.
Defining the calibration and validation approach before commencing a modelling project can
maximise the efficiency of the calibration process, whilst avoiding the temptation to overfit
the model to noise in the observational data. A calibration strategy should therefore outline
the:
gauge locations where model calibration and validation will be performed;
viable or allowable ranges for each model parameter value;
known constraints, dependencies or relationships between parameter values (for
example, the total of the three partial area parameters in AWBM, A1, A2 and A3 must
sum to 1);
period for calibration at each gauge location;
period for validation at each gauge location;
expected level of uncertainty in observations introduced by measurement uncertainty;
metrics to be used to test calibration and validation performance;
whether manual or automated calibration strategy will be adopted, or how a hybrid
strategy of progressive manual and automated calibration will be implemented.
If an automated or hybrid optimisation strategy is to be used, further details should be
defined at the outset of the calibration process on:
algorithms to be used for optimisation of parameter values;
objective function(s) that will be used to test the calibration performance;
-
Guidelines for rainfall-runoff modelling
12
weightings that may be applied in computation of objective functions, to encourage
fitting to different parts of the flow regime (typically the relative weightings to high,
medium and low flows); and
the set of model parameters that will be optimised during calibration and constraints on
the allowable range of values for each parameter.
Ideally, calibration strategy should be documented prior to the commencement of the
calibration process. It may be appropriate for the calibration strategy to be reviewed during the
calibration.
Sensitivity/uncertainty analysis
Relevant sources of uncertainty in typical order of importance include:
1 Model input data including parameters, constants and driving data sets,
2 Model assumptions and simplifications of what the model is representing,
3 The science underlying the model,
4 Stochastic uncertainty (this is addressed under variability below),
5 Code uncertainty such as numerical approximations and undetected software bugs.
The potential impacts of the above sources of uncertainty on the decisions that will be made
using the model should be considered early in the modelling process and then re-examined
once the model has been calibrated, validated and applied for scenario runs. Uncertainty
becomes more important for estimation of events in the tails of the probability distribution,
floods and droughts, than it is for consideration of events that are closer to the centre of the
probability distribution (such as estimation of the mean annual runoff from a catchment).
Documentation and Provenance
Documentation is an important requirement for model acceptance. Its role is:
1 To keep a record of what was done so that it can be reviewed and reproduced,
2 To provide source or background material for further work and research,
3 To effectively communicate the results from models, and
4 To effectively communicate the assumptions made during the modelling process and
the decisions made by the modeller during implementation of the model.
Good documentation supports a study and it will also enable someone coming along later to
see what decisions were made, what was done to underpin the decisions and why, particularly
if aspects of the project are revisited at some later time.
Provenance, as it might relate to hydrological modelling studies simply means the ability to
trace the source/lineage of the data, model and modelling results. Reasons for providing
provenance in rainfall runoff modelling include:
1 Accountability and a full audit trail for all modelled results.
2 Repeatability; ability to re-create a results data set using current data or better
understanding.
-
Guidelines for rainfall-runoff modelling
13
3 Reproducibility; ability to re-create a results data set exactly using all original data,
workflow ordering, assumptions and parameters.
4 Versioning of both entire workflow and systems implementation. Versioning of the
subcomponents and data sets will be the responsibility of those who govern them but
must be captured by the system.
The degree of provenance required depends on the application and/or how the modelling
system is intended to be used by the individual or organisation in future. Current best practise
provenance is to save all input data and model/parameters version and workflow history such
that the outputs can be reproduced in future if required. In the future the ability to register and
resolve the type and identity of objects within the modelling process should reduce the
requirement to capture and archive these objects, especially as modellers take greater
advantage of services based point of truth data streams and modelling systems, and rely less
on ad hoc locally managed resources.
Model acceptance and accreditation
The aim of model acceptance is to gain agreement that the model is fit for purpose.
Information available from the model accreditation process (Reporting, QA documentation,
Peer review) provides model development details and review results which will enhance
model acceptance.
Peer review plays an important part, especially with stakeholders that are external to the
organisation undertaking the model development. It is important for establishing the
credibility, reliability and robustness of results and the methodology used to obtain the results.
It is undertaken by people with specialist understanding in fields relevant to the project.
Use of accepted/accredited model
Once a calibrated model is evaluated against good quality data and has undergone thorough
review process (model acceptance and accreditation), it can be used for modelling to support
water management planning and policy decisions (provided that the model was accredited for
similar purpose).
-
Guidelines for rainfall-runoff modelling
14
3 Model choice
3.1 Model selection
Model selection is made based on an understanding of the objectives and the system being
modelled (http://www.toolkit.net.au/Tools/Category-Model_development; CRCCH 2005a, b).
The WMO (2008, 2009) report include the following factors and criteria as being relevant when
selecting a model:
1 The general modelling objective; e.g. hydrological forecasting, assessing human
influences on the natural hydrological regime or climate change impact assessment.
2 The type of system to be modelled; e.g. small catchment, river reach, reservoir or large
river basin.
3 The hydrological element(s) to be modelled; e.g. floods, daily average discharges,
monthly average discharges, water quality, amongst others.
4 The climatic and physiographic characteristics of the system to be modelled.
5 Data availability with regard to type, length and quality of data versus data requirements
for model calibration and operation.
6 Model simplicity, as far as hydrological complexity and ease of application are
concerned.
7 The possible transposition of model parameter values from smaller sub catchments of
the overall catchment or from neighbouring catchments.
8 The ability of the model to be updated conveniently on the basis of current
hydrometeorological conditions.
Other things that should be considered are:
1 The level of modelling expertise available.
2 Whether the model is going to be used on its own, or if it is going to be used in
conjunction with other models.
3 Freedom of choice may be limited by a desire to minimise problems of different models
for much the same purpose in the same project area, or to avoid problems of different
models in adjoining project areas, particularly where the models are linked in some way
in the future or results compared in some way.
4 Whether uncertainty will be explicitly modelled. If uncertainty is to be explicitly included,
what types of uncertainty are to be modelled (e.g. climatic uncertainty, uncertainty in
climate change projections, uncertainty in rainfall runoff model parameter values); what
approaches will be used to generate the replicates to represent uncertainty and how
many replicates will be required to adequately quantify uncertainty.
5 Whether simulation or optimisation, or a combination of both, is adopted.
-
Guidelines for rainfall-runoff modelling
15
6 Whether the model is to be used for hindcasting or forecasting when being applied in
predictive mode.
In essence the governing principle in choosing a model should be that it should not have more
parameters requiring calibration or a greater level of detail than the available data can support,
to minimise problems of spurious results and false calibrations.
3.2 Available models
Rainfall runoff models can be represented by a range of approaches, in order of increasing
complexity as:
simple empirical methods (e.g., curve number and regression equations);
large scale energy-water balance equations (e.g., Budyko curve);
conceptual rainfall-runoff models (e.g. SIMHYD, Sacramento, AWBM)
landscape daily hydrological models (e.g., VIC, WaterDyn);
fully distributed physically based hydrological models which explicitly model hillslope
and catchment processes (e.g., SHE, TOPOG).
These categories have been used for ease of description, and there is overlap between these
model types. Although these approaches vary in terms of the complexity with which they
represent the rainfall-runoff transformation processes, all of them conceptualise the real
processes using some sets of mathematical equations (and hence are all conceptual models
of the physical environment). Similarly, conceptual rainfall-runoff models run in distributed
mode can be classed as being landscape daily hydrological models. This section provides a
discussion of the characteristics of each of these model types, along with a broad assessment
of the strengths and weaknesses of each approach for rainfall runoff modelling (Table 3-1).
Empirical methods
Empirical methods to rainfall runoff modelling typically involve the fitting and application of
simple equation(s) that relate drivers of runoff response to flow at the catchment outlet.
Empirical equations are most often derived using regression relationships.
Common predictor variables may include rainfall for the catchment, flow observed at another
gauge in the vicinity, evapotranspiration, groundwater levels, vegetation cover and the
impervious area within the catchment. Where rainfall is used as a predictor variable,
regression relationships derived almost always include a non-linear relationship between
rainfall and runoff.
All catchments incorporate storage elements, including interception by vegetation, storage
within the soil column, groundwater storage and storage within stream channels. Catchment
storage typically results in runoff from the catchment being an integrated function of the
climatic conditions for the catchment over some period prior to the period for which runoff is to
be calculated by the model. Therefore, empirical models that produce acceptably accurate
simulations of runoff are either applied at sufficiently long time steps that changes in internal
water storage within the catchment can be ignored (e.g. annual time step) or applied to
represent an integration of the climatic conditions that occurred over some time period prior.
As a practical example, for most catchments a regression model that only includes daily
rainfall on the current day is likely to produce a very poor estimation of daily runoff but a model
-
Guidelines for rainfall-runoff modelling
16
for predicting daily runoff that used individual values of daily rainfall for several days prior may
produce acceptable runoff estimates.
Empirical regression relationships are often developed using spreadsheets. They can also be
fitted using more sophisticated statistical analysis packages, which may more easily facilitate
the investigation of predictor variables. For general information on the development of
regression relationships, the modeller is referred to NIST/SEMATECH e-Handbook of
Statistical Methods (NIST and SEMATECH, 2010) or to a University Level statistics text book.
Empirical regression equations are best suited to situations where there are two flow gauges
on the same stream with partially overlapping periods of record, which are therefore subject to
similar climatic drivers, and the regression equation is used to extend the simulated flow to the
combined period of record from both sites. They can also produce adequate simulations for
neighbouring gauged catchments with overlapping periods of record in situations where the
two catchments are subject to similar rainfall timeseries and are relatively similar
hydrologically.
Large scale energy-water balance equations
The large scale energy-water balance methods are based on the hypothesis of available
energy and water governing large scale water balance (precipitation, evaporation and runoff).
These are usually developed using large scale observed data sets, eg. the Budyko curve
(Budyko, 1958) was developed using mainly European data, and numerous other forms have
been proposed to improve estimates in local regions and to account for different land cover
types (Arora, 2002). One of the more popular forms of the Budyko method is the Fu (1981)
rational function equation (Zhang et al., 2004) where a single parameter, , in the equation can
be calibrated against local data to tune the method for the local conditions. The inputs to these
equations are rainfall and potential evapotranspiration (PET) and the output is runoff at mean
annual time step.
Conceptual Rainfall-Runoff Models
Conceptual rainfall runoff models represent the conversion of rainfall to runoff,
evapotranspiration, movement of water to and from groundwater systems and change in the
volume of water within the catchment using a series of mathematical relationships. Conceptual
rainfall runoff models almost always represent storage of water within the catchment using
several conceptual stores (or buckets) that can notionally represent water held within the soil
moisture, vegetation, groundwater or within stream channels within the catchment. Fluxes of
water between these stores and in and out of the model are controlled by mathematical
equations.
Most applications of conceptual rainfall runoff models treat the model in a spatially lumped
manner, assuming that the time series of climatic conditions (notably rainfall and potential
evapotranspiration) and the model parameter values are consistent across the catchment.
There have been implementations in more recent times of conceptual rainfall runoff models in
semi-spatially distributed and spatially distributed frameworks. In distributed application, the
catchment is defined by grid cells or subcatchments within the catchment that are assigned
the same rainfall runoff parameter values but different time series of climatic inputs so that
different grid cells or subcatchments within the catchment produce different contributions to
the overall runoff. This is effectively a series of lumped rainfall runoff models, with lumped sets
of model parameters that are applied with spatially distributed rainfall.
-
Guidelines for rainfall-runoff modelling
17
Conceptual rainfall-runoff models have been widely used in Australia for water resources
planning and operational management because they are relatively easily calibrated and they
provide good estimates of flows in gauged and ungauged catchments, provided good climate
data is available.
In Australia there are six widely used conceptual rainfall-runoff models; AWBM (Boughton
2004), IHACRES (Croke et al. 2006), Sacramento (Burnash et al. 1973), SIMHYD (Chiew et
al. 2002), SMARG (Vaze et al., 2004) and GR4J (Perrin et al. 2003). The input data into the
models are daily rainfall and PET, and the models simulate daily runoff. The models are typical
of lumped conceptual rainfall-runoff models, with interconnected storages and algorithms that
mimic the hydrological processes used to describe movement of water into and out of
storages. They vary in terms of the complexity of the catchment processes that they try to
simulate and in terms of the number of calibration parameters which vary from four to
eighteen.
Landscape daily hydrological models
These models are based on the concept of landscape processes and they model the typical
landscape processes using simplified physical equations (VIC model, Liang et al., 1994;
2CSALT, Stenson et al., 2011; AWRA-L, Van Dijk, 2010). A catchment is usually
conceptualised as a combination of landscapes which are delineated using some combination
of outputs from digital elevation model analysis, underlying geology, soil types and land use.
Often these models have been designed to reproduce other variables in addition to streamflow
(e.g. distributed evapotranspiration, soil moisture, recharge, salinity), and as a result have a
greater complexity to methods that target streamflow alone.
Fully distributed physically based hydrological models which
explicitly model hillslope and catchment processes
The physically based models are based on our understanding of the physics of the
hydrological processes which control the catchment response and use physically based
equations to describe these processes. A discretisation of spatial and temporal coordinates is
made at a very fine scale for the entire catchment and the physical equations are solved for
each discretised grid to obtain a solution.
-
Guidelines for rainfall-runoff modelling
18
Table 31 Assessment of Strengths and Weaknesses of Different Rainfall Runoff Model Structures
Criteria Model Type
Empirical Large Scale
Energy-Water
Balance
Conceptual Landscape
Daily
Fully
Distributed
Physically
Based
Typical Run Time Step
Can be daily if daily flow from another gauge is used as a predictor variable. Otherwise typically only applied at annual (or longer) time scale
Typically only applied for mean annual runoff, although pattern of flows from a nearby gauge may be used to disaggregate annual totals to monthly or daily time steps
Daily, although shorter run time steps are possible if sufficient climatic data is available at this shorter time step
Daily, although shorter run time steps are possible if sufficient climatic data is available at this shorter time step
Minutes to hours to maintain numerical stability, although often forced with daily data and assumed patterns used to disaggregate to shorter time steps
Typical Number of Parameters
1 to 5 2 to 4 4 to 20 10 to 100 10 to 1000's
Risk of over-fitting or over-parameterising the model.
Low Very Low Moderate High Very High
Need for high resolution spatial data layers
None to Moderate
Low to Moderate
Low High Very High
Strength of Apparent Connection between Model Parameters and Measurable Physical Catchment Characteristics
None None Weak for most parameters (although impervious area or interception may be exceptions)
Moderately weak
Claimed to be strong by proponents but can be difficult to validate this claim
Run time on typically available computer platforms for 100 years of daily data
-
Guidelines for rainfall-runoff modelling
19
Criteria Model Type
Ability to implement multiple runs for automated calibration
Not typically required - optimum parameters can be obtained by least squares fitting that does not require multiple runs
Not typically required
Very Good. Run times are typically sufficiently low to facilitate this and tools are available (Rainfall Runoff Library and Source) to facilitate this
Good. Run times likely to be sufficiently low to facilitate this in most circumstances, however tools for calibrating such models using automated routines are not as widely available
Poor. Run times are generally too long to consider automated calibration
Typical Performance in Regionalisation
Moderate at annual time steps. Usually very poor at shorter time steps (e.g. Daily)
Good at annual time steps. Usually very poor at shorter time steps (e.g. Daily)
Moderate at daily time steps
Proponents claim to be superior for regionalisation to conceptual rainfall runoff models
Proponents claim this to be a strength of distributed models but in reality the large number of parameters required may compromise the application of distributed models to regionalisation
Representation of non-stationarity in catchment conditions
Not possible
Often applied to explicitly represent non-stationarity in vegetation cover for mean annual runoff signal
Usually difficult, due to lack of physical meaning for many model parameters
Possible Possible
Typical performance of model when applying to a very different climatic period to that used for calibration
Poor Moderate when used to estimate impact on mean annual flow but flows disaggregated to shorter time steps are likely to be poorly estimated
Variable - can be good in some catchments but poor in others
Variable - can be good in some catchments but poor in others
Variable - can be good in some catchments but poor in others
Typical level of expertise with this type of model within Australian water industry
Strong Moderate Strong Weak Very weak
Likelihood that previously calibrated models are available for catchment to be modelled.
Moderate to Low
Moderate Very High Low Very low
-
Guidelines for rainfall-runoff modelling
20
4 Collate and Review Data Climatic data is the most important driver of any rainfall runoff modelling process. The
calibration and validation of models also involves comparison to observed streamflow data.
Major causes of difficulty in calibrating rainfall-runoff models are errors and uncertainties in the
input data (see Kavetski et al, 2003). A discussion of these problems can be found in the
collection of papers in Duan et al 2003. Checks should therefore be performed on the input
data and the comparison data set for calibration and validation to be used in rainfall runoff
modelling before any attempt is made to apply or calibrate the models. The intent here is to
investigate the integrity of the data, whether observations are in the first instance plausible
(e.g., is the volume in a hydrograph less than the product of the rainfall and catchment area).
Investigations into data to be used for rainfall runoff modelling should include checks of:
Stationarity of the data time series , i.e. has there been any systematic or step change in
the statistical properties over the time of data collection, and if so why;
Spatial coherence of data, i.e., is the data consistent with regional spatial and temporal
patterns and trends;
Accuracy of the spatial location of the gauging site;
Consistency in the approach used to date and time stamp the data, particularly for data
provided by different agencies;
Procedures use for spatially interpolation of point observations to gridded data
estimates or estimated series across catchment areas
e.g., time series plots at different levels of temporal aggregation, ranked plots, residual mass
curves, double mass curves, etc. This will pick up patterns as well as identify anomalies which
may be potential data QA issues.
Other checks and analysis, including regional consistency of runoff depths, rain-runoff ratios,
rating confidence limits, period of record, whether rainfall and PET is observed or interpolated,
base-flow characteristics, checks for stationarity and variability over time, etc would also be
useful. It is important that prior knowledge is considered.
One major factor which will apply across all types of time series data used is that the time base
must be kept consistent so that the data applies to the same time period. An example is
where flow data time steps should be matched to the rainfall data time step. In Australia, daily
rainfall data is commonly recorded as the depth of rainfall that occurred in the 24 hours
preceding 9 am on the date of the recorded data. In contrast, daily streamflow totals are often
quoted for the 24 hour period commencing on the nominated date, resulting in the recorded
flow data being offset by 1 day forward of the rainfall data. Where possible the flow data should
be extracted at a time step to match the rainfall dataset. HYDSTRA databases allow this where
the records are at short time intervals. In other cases shifting the recorded time series by one
day for either the rainfall or flow time series may be required to produce consistent time series
for modelling.
The remainder of Section 4 outlines the data types, sources, availability, accuracy,
manipulations (such as gap filling) and any other issues.
-
Guidelines for rainfall-runoff modelling
21
4.1 Catchment details
Location of gauges (streamflow, rainfall and evaporation)
The streamflow recorded at the catchment outlet is a combined response to the spatial
distribution of rainfall and evaporation across the catchment. There are uncertainties
associated with the streamflow measurements due to rating curve errors as well as due to
extrapolation outside the limits of the rating curve. There is spatial variability in rainfall (and to
smaller extent evaporation) across a catchment which is not captured when undertaking
lumped catchment modelling using a single rain gauge. There might be problems with the
location of the rain gauge in terms of capturing a representative rainfall for all the rainfall
events especially for catchments with high rainfall gradients.
Topography and Catchment Areas
The catchment area for a catchment represents the contributing area to the catchment outlet
where the streamflow is measured. The catchment boundaries (and the corresponding
catchment area) can either be derived from topographic map layers or using the catchment
digital elevation model (DEM) and a standard package such as ARCGIS. It is usually easier to
determine catchment area for the catchments located in steeper terrain compared to those
located in very flat areas (especially when using DEM).
Slope and dominant aspect may provide useful explanatory variables for estimating routing
parameters or for regionalisation of rainfall runoff parameters between catchments.
Soil types
A catchments rainfall-runoff response is related to the soil types in the catchment. The surface
soil characteristics determine the infiltration rates and so the contributions from different flow
components (surface runoff, throughflow and base flow). Soils information can be obtained
from any soils field work that has been undertaken in the catchment or from large scale soil
properties maps (e.g. Australian Soils Atlas, Northcote et al., 1960). In most practical
applications of conceptual rainfall-runoff models in Australia, soil information is seldom directly
used as input in the calibration process because the inherent spatial variability in soil
properties within a catchment is typically sufficiently large that it has been difficult to
demonstrate statistically robust relationships between conceptual model parameters and soil
properties.
Vegetation
Land cover/vegetation cover in a catchment can often be correlated with the amount of
interception storage/loss and actual evapotranspiration in a catchment. The land cover across
the catchment can be derived from large scale vegetation mapping based on satellite imagery
or remotely sensed data. Vegetation cover data has not typically been used explicitly in
directly determining rainfall runoff model parameters, although there have been some recent
studies which have demonstrated the importance of catchment land cover in rainfall-runoff
model calibration and for predictions in ungauged basins (Zhang and Chiew, 2008; Vaze et al.,
2011c).
-
Guidelines for rainfall-runoff modelling
22
Water Management Infrastructure
Water management infrastructure within a catchment can allow humans to make very
substantial modifications to flows within a catchment. Water management infrastructure may
include large dams, farm dams and off stream storages, extractions, man-made canals or
diversion pipelines and discharges from sewage treatment plants. Locations of these
infrastructures should be identified where they exist within the catchment so that their potential
impact on streamflows may be assessed. Recorded flows at the catchment outlet may require
adjustment to allow for the influence of water management infrastructure located upstream of
each of the flow gauging locations.
4.2 Flow data
Reliable measurements of streamflow data are critical for successfully calibrating a
rainfall-runoff model to a catchment. The streamflow data for all the gauged locations can be
obtained from the respective state government water management agencies or from the
Bureau of Meteorology (in Australia). Considerations in checking streamflow data include:
the agency collecting the data and the quality assurance procedures (if any)
implemented by that organisation during data collection;
reliability of the rating of levels to flows for the gauge;
the accuracy, extent and currency of cross sections surveyed at the gauge site.
(Surveyed cross sections may only extent to the top of the stream bank and gauging for
flows extending onto the floodplain may use a cross section that is inaccurate);
vegetation and substrate material for the channel bed, channel banks and floodplain
and the influence of assumptions made about these on gauged flows;
the ratio of the highest flow outputs to the highest flow that the gauge has been rated for;
how hydraulically stable (variable over time) the rating site is;
examination of potential backwater effects for the site from influences that are
downstream of the site, such as stream confluences, bridge crossings, culverts, dams
or weirs;
hysteresis effects leading to different flow rates for the same recorded level on rising
and falling limbs of hydrographs;
how well maintained the gauging site and instrumentation used for measuring water
levels has been;
any changes to the gauging instrumentation over time;
the length of time since the last rating at high flows;
length of record at the site;
availability of quality codes with the flow data record;
proportion of missing data;
trends in when data is missing from the record (i.e. Is there any bias toward high or low
flow periods, particular seasons, or are the gaps just random?) and how this might
influence any infilling procedures; and
-
Guidelines for rainfall-runoff modelling
23
if there are a number of gauges closely located that basically represent the same
catchment the data sets may be able to be combined to give a longer record for the site.
Assessment of the above factors will inform whether the data is useful in calibration of the
model, independent validation of the model or whether the data should be ignored.
4.3 Rainfall
Rainfall is the main driver of runoff and so reliable measurements of rainfall are critical for
successfully calibrating a rainfall-runoff model to a catchment. There are several sources for
obtaining climatic data for a particular catchment:
Site observations from Bureau of Meteorology climate database.
Site observations taken from monitoring sites collected by other organisations that may
exist outside of the Bureau of Meteorology database. Many jurisdictional databases
contain rainfall records.
Gridded data products, such as the Bureau of Meteorologys Australian Water
Availability Project (AWAP) or Queensland Centre for Climate Applications SILO data
set.
It is important to be aware of how this data has been collected and what data quality control
methods have already been applied to the data prior provision of the data set as this may
influence the modelling results. This is particularly relevant to gridded products, such as SILO
and AWAP (SILO, Jeffrey et al., 2001; AWAP, Jones et al., 2009), as these data sources
generally use different algorithms to convert time-series observations at data points to gridded
data products.
In a small catchment, considerably better results may be obtained from using rainfall station
data from the BOM (http://www.bom.gov.au/climate/) or locally collected data than a gridded
data set that smoothes observations from a smaller number of more sparsely located sites. In
some cases it may be appropriate to adjust the station data, normally by a percentage, if the
mean catchment rainfall can be defined using other sources e.g. isohyetal detail.
In large catchments there is spatial variability in rainfall across a catchment which is not
captured when undertaking rainfall-runoff modelling using rainfall time series from the rain
gauges. If using a single rain gauge, there might be problems with the location of the rain
gauge in terms of capturing a representative rainfall for all the rainfall events. If using a
spatial rainfall product (SILO or AWAP in Australia), there can be uncertainties introduced
because of the method used for interpolating rainfall between rain gauges and changes in the
rain gauge network over time. Interpolation methods currently used are more suited to areas
where rainfall varies less over space and in time. They do not account well for orographic
effects, and rainfall networks in Australia historically have not captured the spatial and
temporal variations in tropical and monsoonal areas well.
Vaze et al., 2010b discusses testing carried out considering the effects of using different
rainfall data sets on the calibration and simulation of conceptual rainfall-runoff models. They
conclude that considerable improvements can be made in the modelled daily runoff and mean
annual runoff with better spatial representation of rainfall. Where a single lumped
catchment-average daily rainfall series is used, care should be taken to obtain a rainfall series
that best represents the spatial rainfall distribution across the catchment. However where only
estimates of runoff at the catchment outlet are required, there is little advantage in driving a
-
Guidelines for rainfall-runoff modelling
24
rainfall-runoff model with different rainfall inputs from different parts of the catchment
compared to using a single lumped rainfall series for the catchment.
4.4 Evapotranspiration
The measured pan evaporation data can be obtained for all the locations with the evaporation
gauges installed (in Australia from the Bureau of Meteorology (BoM) basic records). In
Australia there are also some spatial climate products which use point evaporation
measurements recorded by the BoM and use an interpolation schemes to produce spatial
evaporation surfaces (SILO, Jeffrey et al., 2001; AWAP, Jones et al., 2009).
The network of pan evaporation recording stations in Australia is sparse in comparison to
stream flow and rainfall networks, although there is some compensation in that typically
potential evapotranspiration exhibits substantially higher spatial correlation than rainfall or
stream flow. This limits the ability to accurately represent the true spatial and temporal
variability in evaporation in models however the spatial variability in evaporation is much
smaller compared to the variability in rainfall.
The BoM network records pan evaporation. Modelling requires potential evapotranspiration
(PET). There are a number of methods to convert pan evaporation to PET including Penman
Monteith, Mortons and accepted pan factors. These use climatic variables in the conversion
calculation including solar radiation, temperature, vapour pressure, and wind speed which are
recorded at some pan recording stations but not all. This further limits the network available to
draw data from.
When all the required data is available the conversion calculations will use the records but
often some variable is missing and estimates of that variable are made and used. Where
there is no data for the climatic variables, calculated pan to PET conversion factors from a
nearby station can be used to derive PET from pan evaporation.
Commonly the spatial products have interpolated layers for a range of climatic factors and the
spatial PET layer is calculated from data in these layers rather than interpolating PET
calculated at recording stations.
-
Guidelines for rainfall-runoff modelling
25
5 Statistical Metrics for Testing
Performance There are many performance measures used to consider the acceptability of a rainfall-runoff
model. In all cases visual assessment and statistical results of some sort will be assessed to
identify the ability of the model to reproduce the flows it is calibrated or validated against. All
may contribute to best practice and which measures are more appropriate will be directed by
the modelling objective. A number of commonly used visual assessment techniques are
outlined in Table 51. Statistical performance measures and their relevance in various study
types are listed below in Table 52.
Table 51 Plots for assessing model performance
Plot Assessment and Purpose
Daily and monthly plots (linear and log)
Used to check the general size, shape and timing of hydrographs. Linear plots will better show medium and high flows and log plots low flows. Baseflow and recession characteristics can be reviewed. If recessions are frequently too flat then this can indicate that the interflow and baseflow are not represented correctly.
Scatter Plot
Scatter plots show the ability of the model to match flows on actual time steps. They show the flow ranges where the model is more accurate. Linear and log plots will show the variability across the various flow ranges. Often a line of best fit is shown to indicate the bias of the model in estimating flows.
Ranked Plots Commonly referred to as frequency of excedence or flow duration graphs, they show the percentage of time a flow is exceeded over the modelled period. They show whether the modelled output can replicate the observed flow regime. Flow duration curves are effective diagnostics to ensure that both the variability and the seasonal pattern are captured.
Cumulative mass or cumulative residual mass curves
Scatter plots and flow duration curves do not examine the time sequence of events. A model could appear to be replicating the flow regime however the replication of regimes during wet and dry periods may not be adequate. A cumulative residual mass curve is a cumulative plot of residuals (flow value - mean of all values). A residual, and therefore slope of the curve, will be positive during wet periods as flows are higher than average and during dry periods the slope will be negative. If the curves diverge there may be a data issue. If they diverge consistently in all wet or all dry periods it is likely that model parameterisation for wet periods or dry periods may not be appropriate.
Plotting average daily or monthly flows (average of all Days, average of all Januaries)
A simple diagnostic to ensuring that the model can replicate seasonality characteristics.
-
Guidelines for rainfall-runoff modelling
26
Table 52 Statistical performance measures (metrics) and their relevance in various study types (Y Yes, N No)
Metric Purpose
Runoff Yield
Climate change
Landuse change
Low flow
Water quality
Peak flow / floods
Difference in total runoff Y Y Y N N Y
Difference in total runoff over different seasons of the year*
Y Y Y Y Y Y
Difference in total runoff contained within high, medium and low parts of the flow duration curve
Y Y Y Y Y Y (high flows)
Difference in proportion of time that cease to flow occurs
N Y Y Y Y N
Difference in the slope of logarithm of flow versus time for baseflow recession periods
N N Y Y Y N
Mean square error between observed and modelled runoff
Y Y Y N N Y
Coefficient of determination (often referred to as r)
Y Y Y N N Y
Nash Sutcliffe coefficient of efficiency on daily flows
Y Y Y N N Y
Nash Sutcliffe coefficient of efficiency on monthly accumulated flows
Y Y Y N N N
Nash Sutcliffe coefficient of efficiency calculated using logarithm transformed flows
N Y Y Y Y N
* Definition of seasons to be used will vary depending upon the climatic zone that the catchment is in. For
tropical areas, two seasons (a wet season from December-April and dry season from May-November) may
be appropriate. In Southern Australia, it may be appropriate to consider the four conventional calendar
seasons (Dec-Feb, Mar-May, Jun-Aug and Sep-Nov).
** Definitions of high, medium and low flow ranges will depend upon the purpose of the study and the
catchment. Typical ranges might be High flows: days in observed data in the 0 to 20% probability of
exceedance range; Medium flows: days in observed data in the 20 to 80% probability of exceedance range;
Low flows: days in observed data with greater than 80% probability of exceedance and above the cease to
flow level at the gauge. Adjustment of the low and medium flow ranges may be required particularly in
response to the probability of cease to flow conditions at the gauge.
-
Guidelines for rainfall-runoff modelling
27
6 Calibration and validation
6.1 Calibration
Model calibration is a process of optimising or systematically adjusting model parameter
values to get a set of parameters which provides the best estimate of the observed streamflow.
Virtually all rainfall runoff models must be calibrated to produce reliable estimates of
streamflow because there has been little evidence identified of strong links between physical
characteristics of catchments and the parameters of rainfall runoff models (Beven, 1989).
Models should always be calibrated to observed data to demonstrate that the model can
produce observed flow time series with an acceptable level of accuracy. The acceptable level
of accuracy will depend upon the statistics of the flow data to be reproduced, which is
determined by the purpose that the model will be applied for.
A model may be available that has been previously calibrated for a catchment as part of
another study. In this situation, the calibration performance of the model should be re-tested
before it is applied because the purpose for developing the model may be different between
the earlier and later applications, which may influence the calibration objectives.
When calibrating a model it should always be kept in mind that there are always going to be
tradeoffs, for example between getting wet, dry, and average conditions correct, and those
tradeoffs will be driven by the purposes the model will be used for.
6.2 Validation
Model validation is a process of using the calibrated model parameters to simulate runoff over
an independent period outside the calibration period (if enough data is available) to determine
the suitability of the calibrated model for predicting runoff over any period outside the
calibration period. If there is not enough data available, the validation may be performed by
testing shorter periods within the full record.
Model validation is one of the most important steps in rainfall-runoff modelling as the
performance of the calibrated model in the validation period provides us confidence in the
modelling results when the calibrated model is used for simulating streamflow outside the
measured streamflow period or when the model is used for predicting streamflow under future
climate change scenarios.
Validation has often been achieved using a split sample process, whereby a period of
observed data (say the first two-thirds of the available record) are used for calibration and the
remaining one-third are used for validation. The model that was calibrated using the calibration
data set is run for the validation period without changing the model parameters and the
goodness of fit statistics are computed for the validation period. The split sample approach
assumes that both the catchment and the climatic conditions that it is subject to are stationary
in nature across the entire period that recorded data is available for. Evidence of stationarity
(or non-stationarity) in catchment conditions that would affect the hydrological response during
the period of recorded data should be checked using independent data sources (such as aerial
photography, satellite imagery, landuse, topographic or other spatial information).
-
Guidelines for rainfall-runoff modelling
28
A more sophisticated calibration approach can involve multiple calibration and validation
periods. As in the simple split sample approach, the model is calibrated to a calibration period
and then performance is tested over the validation period without changing the model
parameter values. This approach is then repeated multiple times, with each replicate using
different start and end dates for the respective calibration and validation periods. This allows a
range of model performance statistics for calibration and validation periods to be reported.
There will be some instances with this calibration and validation approach where the calibrated
parameters perform well against the calibration data set, but performs poorly against the
validation data set. In research type investigations, where the modeller may be comparing
different rainfall-runoff models, calibration methods, or objective functions, the validation
results can be used directly to help decide the best model or method or objective function.
However, in practical applications, a modeller may have to decide either not to change the
calibrated parameters and report the poor results, or to recalibrate the model because the
performance is unacceptable.
The modeller may choose the latter option, and may then recalibrate and compare against the
validation data set several times until the calibrated parameters perform acceptably against
both data sets. However, as the validation data set has now been used to change the
calibrated parameters, it is no longer an independent data set and has in effect indirectly
become part of the calibration data set.
This risk of having much poorer performance in validation than calibration may be mitigated by
ensuring as far as possible both data sets have similar flow distributions, An arbitrary
approach to splitting the data, e.g., at the midpoint, may result in half of the data being in a
much wetter period. A model calibrated to these conditions would not be expected to perform
well under the drier conditions in the validation data set. More alternate approaches should be
considered on how to split the data set, perhaps into non-contiguous periods, to ensure overall
flow distributions are similar in each period.
Data is a valuable resource, and should be used to greatest effect. In most Australian
conditions, long data sets are needed to adequately represent climatic variability. An
alternative approach to having split samples is to use the complete data set to calibrate the
model, then to report its performance for different sub-periods, e.g., first half and last half, or
decadally, or driest X year period and wettest X year period. The objective would be to have a
comparatively persistent performance across all these periods. This does not necessarily give
you an independent assessment of performance, but does report on performance under
different conditions.
Transposition of model parameter values from gauged to ungauged catchments may be
tested using a spatial variant on split sample validation. Under this approach, component
models from a gauged catchment with the calibrated parameter values for that catchment can
be applied to another gauged catchment to test the uncertainty and bias introduced from
transposition. Uncertainty ranges can be established by testing contributions flow series
produced by model outputs with parameter sets adopted from several different gauged
catchments. Examples of the performance of these transposition approaches are discussed in
Viney et al. (2009) and Chiew (2010).
Generally the same metrics used to assess the performance of the model during calibration
are also used to assess model performance during validation. The model performance during
validation is almost always poorer than during calibration because model parameters are
deliberately not specifically fitted to the data for the validation period.
-
Guidelines for rainfall-runoff modelling
29
6.3 Calibration and Validation of Models to Single Gauge Sites,
Multiple Gauge Sites and Regionalisation of Model
Parameter Sets
It is a very common situation in a project that involves rainfall runoff modelling for flow time
series to be required for several catchments or subcatchments within the model domain and
for data to be available from two or more stream flow gauges to facilitate calibration and
validation. At locations where gauged flows are available and flow estimates are required, two
options are available to the modeller:
The rainfall runoff models may be calibrated independently for each gauged catchment.
In this case, independent parameter sets will be derived for the rainfall runoff models of
each catchment; or
A joint calibration may be performed, with rainfall runoff models calibrated with
consistent parameters to fit to the gauge records from two or more gauges. In this case,
a single set of rainfall runoff model parameters will be produced for all of the catchments
that represents a compromise to fit the flows at all of the gauges within that group.
The advantage of the joint calibration approach is that, assuming some degree of
homogeneity in the rainfall runoff response of the selected gauged catchments, the parameter
sets produced should be more robust when applied to other catchments with similar response
that were not used for the calibration.
If an automated calibration process is used for joint calibration of multiple catchments, the
objective function used for automated calibration to the gauged catchments will be a weighted
average of the objective function values produced at the individual gauges. Options for
selecting the weighting values are:
All gauged catchments contribute equally to the overall objective function;
Weights are assigned according to the length of available record (e.g. number of days
with data) at each site;
Weights are assigned according to the inverse of the correlation coefficient in gauged
flows between one gauge and one or more of the other gauges in the set (i.e. gauges
with strongly correlated recorded flows are assigned lower weighting factors than
gauges that have weaker correlations with other gauges);
Some combination of the above factors.
There are three main methods of developing flow data sets in residual ungauged catchments
between upstream and downstream gauges:
1 Calibrate a model to the difference in flow between the gauged upstream flows routed to
downstream (adjusted for known transmission losses) and downstream gauges.
2 Adjust a flow data set from a nearby catchment using either recorded or generated data,
3 Apply parameter values from other calibrated models and use catchment appropriate
climate data.
There are two main methods of developing flow data sets in ungauged catchments:
1 Develop a regression equation between flows for the ungauged catchment and gauged
catchments and apply this equation to transpose the flow, or
-
Guidelines for rainfall-runoff modelling
30
2 Apply parameter values from other calibrated models and use catchment appropriate
climate data.
Generally in the second case parameters for a neighbouring or nearby catchment are used but
climate data and catchment characterises of the catchment of interest are applied in the
model. Many studies have shown that selecting a donor catchment based on spatial proximity
gives similar or better results than selecting a donor catchment based on catchment attributes
(Merz and Bloschl 2004, Oudin et al 2008; Parajka et al. 2005; Zhang and Chiew 2009).
6.4 Automated, Manual and Hybrid Calibration Strategies
Calibration of hydrological models can be conducted using manual or automated methods, or
a combination of the two approaches (see Boyle et al, 2000 and Brdossy, 2007 for
frameworks for combining manual and automated methods of model calibration). Calibration
involves the adjustment of model parameter values to improve the fit of model output data to
observations to a level that is acceptable.
In case of manual calibration, definition of goodness of fit is usually produced as a
combination of statistical indices and visual inspection of the observed and simulated
hydrographs. Whereas in case of automated calibration, definition of goodness of fit is
usually produced using an objective function. The objective function translates the observed
and modelled outputs into a single number, so that the results of successive calibration
iterations can be compared. Automated calibration routines use a defined algorithm that runs
the model multiple times, adjusting model parameter values according to a strategy that is
intended to improve the value of the objective function.
The sections that follow give information on the calibration methods available and their
relevance in various study types (shown in Table 3-1) which dealt with model choice
appropriate for intended purposes.
Manual Calibration
Manual calibration involves the modeller selecting a set of parameters for their model, running
the model once and then examining the output statistics from the model (from the list
discussed in Section 5). The modeller would then revise the values of one or more parameters
and repeat the above process. This may continue many times until the model achieves the
desired level of performance.
The match between simulated and observed streamflow can be visually assessed as either a
time series, or as flow duration curves or residual mass curves. The visual assessment can
identify general deficiencies in the matching of the hydrologic regime, e.g., high flow events
under or over estimated, baseflows under or over-estimated or the seasonal response of the
model not captured appropriately. Software that stores the results of conceptual storages and
fluxes for graphing, and interpretation of these results in the context of model structure is also
useful to identify which parameter values need adjusting and in which direction in order to
improve results.
Guidelines are available from the developers of the Sacramento model that describe how to
estimate key parameter values directly from analysis of recorded hydrographs (Burnash,
1995). A range of realistic parameter values has also been recommended to guide initial
estimates.
-
Guidelines for rainfall-runoff modelling
31
Strengths:
1 Encourages a deeper understanding of model structure and its applicability to
catchment hydrology, rather than treating as a black box.
2 Allows for hydrologist to consider performance against a broad range of performance
metrics, and make appropriate adjustments.
3 Takes into account understanding of the data and the catchment.
4 Allows a logical checking at each change.
5 Produces a greater appreciation of strengths and limitations of calibrated result.
Weaknesses:
1 Repeatability is limited. Different people may get different parameters and output flow
time series.
2 More effort and time required to complete a calibration.
3 Difficult to m