guidelines for rainfall-runoff modelling: towards best ... · information on rainfall-runoff models...

eWater Cooperative Research Centre eWater Limited ABN 47 115 422 903 Innovation Centre, Building 22 University Drive South Bruce ACT 2617 Phone: +61 2 6201 5168 [email protected] www.ewater.com.au

Guidelines for Rainfall-Runoff Modelling: Towards Best Practice Model Application

December 2011

Jai Vaze, Phillip Jordan, Richard Beecham, Andrew Frost, Gregory Summerell

Interim Version 1.0

eWater CRC

Copyright Notice

© eWater Ltd 2011

Legal Information

This work is copyright. You are permitted to copy and reproduce the information, in an unaltered form, for non-commercial use, provided you acknowledge the source as per the citation guide below. You must not use the information for any other purpose or in any other manner unless you have obtained the prior written consent of eWater Ltd.

While every precaution has been taken in the preparation of this document, the publisher and the authors assume no responsibility for errors or omissions, or for damages resulting from the use of information contained in this document. In no event shall the publisher and the author be liable for any loss of profit or any other commercial damage caused or alleged to have been caused directly or indirectly by this document.

Citing this document

Vaze, J., Jordan, P., Beecham, R., Frost, A., Summerell, G. (eWater Cooperative Research Centre 2011) Guidelines for Rainfall-Runoff Modelling: Towards Best Practice Model Application.

Publication date: December 2011 (Interim Version 1.0)

ISBN 978-1-921543-51-7

Acknowledgments

eWater CRC acknowledges and thanks all partners to the CRC and individuals who have contributed to the research and development of this publication.

We acknowledge the inputs from the hydrology group in DERM, Queensland, and Mark Alcorn from SA Department for Water. We thank Matthew Bethune, Peter Wallbrink, Dugald Black, Jin Teng, Jean-Michel Perraud, Melanie Ryan, Bill Wang, David Waters, Richard Silberstein, Geoff Podger, David Post, Cuan Petheram, Francis Chiew and Andrew Davidson for useful discussions.

eWater CRC gratefully acknowledges the Australian Government’s financial contribution to this project through its agencies, the Department of Innovation, Industry, Science and Research, the Department of Sustainability, Environment, Water, Population and Communities and the National Water Commission.

For more information:

Innovation Centre, Building 22 University Drive South Bruce, ACT, 2617, Australia T: 1300 5 WATER (1300 592 937) T: +61 2 6201 5834 (outside Australia) E: [email protected] www.ewater.com.au

Best Practice Guidelines for Rainfall-Runoff Modelling

Table of Contents

Table of Contents .......................................................................................................................... 2

1 Introduction ............................................................................................................................ 5

1.1 Background ................................................................................................................... 5

1.2 Definition of Best Practice ............................................................................................. 5

1.3 Scope ............................................................................................................................ 6

2 Overview of procedure for rainfall-runoff modelling .............................................................. 8

2.1 Problem definition .......................................................................................................... 8

2.1.1 Problem statement and setting objectives ............................................................ 8 2.1.2 Understanding the problem domain ...................................................................... 8 2.1.3 Metrics and criteria and decision variables ........................................................... 9 2.1.4 Performance across multiple catchments and subcatchments ............................. 9

2.2 Option modelling ........................................................................................................... 9

2.2.1 Methodology development .................................................................................... 9 2.2.2 Collate and review data ....................................................................................... 10 2.2.3 Setting up and building a model .......................................................................... 10 2.2.4 Calibration and Validation ................................................................................... 10 2.2.5 Sensitivity/uncertainty analysis............................................................................ 12 2.2.6 Documentation and Provenance ......................................................................... 12 2.2.7 Model acceptance and accreditation ................................................................... 13 2.2.8 Use of accepted/accredited model ...................................................................... 13

3 Model choice ....................................................................................................................... 14

3.1 Model selection ........................................................................................................... 14

3.2 Available models ......................................................................................................... 15

3.2.1 Empirical methods ............................................................................................... 15 3.2.2 Large scale energy-water balance equations ..................................................... 16 3.2.3 Conceptual Rainfall-Runoff Models ..................................................................... 16 3.2.4 Landscape daily hydrological models ................................................................. 17 3.2.5 Fully distributed physically based hydrological models which explicitly model hillslope and catchment processes ..................................................................................... 17

4 Collate and Review Data ..................................................................................................... 21

4.1 Catchment details ........................................................................................................ 22

4.1.1 Location of gauges (streamflow, rainfall and evaporation) ................................. 22 4.1.2 Topography and Catchment Areas ..................................................................... 22 4.1.3 Soil types ............................................................................................................. 22 4.1.4 Vegetation ........................................................................................................... 22 4.1.5 Water Management Infrastructure ...................................................................... 23

4.2 Flow data ..................................................................................................................... 23

4.3 Rainfall ......................................................................................................................... 24

4.4 Evapotranspiration ...................................................................................................... 25

5 Statistical Metrics for Testing Performance......................................................................... 26

Page 2

eWater CRC

6 Calibration and validation .................................................................................................... 28

6.1 Calibration ................................................................................................................... 28

6.2 Validation ..................................................................................................................... 28

6.3 Calibration and Validation of Models to Single Gauge Sites, Multiple Gauge Sites and

Regionalisation of Model Parameter Sets ............................................................................... 30

6.4 Automated, Manual and Hybrid Calibration Strategies ............................................... 31

6.4.1 Manual Calibration .............................................................................................. 31 6.4.2 Automated Calibration ......................................................................................... 32 6.4.3 Hybrid Calibration Strategies ............................................................................... 33 6.4.4 Selection of Objective Functions for Automated and Hybrid Calibration ............ 34 6.4.5 Further Guidance on Calibration and Validation of Conceptual Rainfall Runoff Models 36

6.5 Calibration of Regression Models ............................................................................... 39

7 Uncertainty and Sensitivity Analysis ................................................................................... 40

7.1 Sensitivity Analysis ...................................................................................................... 41

7.2 Application of Multiple Parameter Sets ....................................................................... 42

7.3 More Advanced Quantitative Uncertainty Analysis ..................................................... 42

7.4 Consideration of Uncertainty in Practical Applications of Rainfall Runoff Models ...... 43

8 Concluding remarks ............................................................................................................ 44

9 References .......................................................................................................................... 45


Page 4

eWater CRC

1 Introduction

1.1 Background Reliable estimates of stream flow generated from catchments are required as part of the information sets that help policy makers make informed decisions on water planning and management. The characteristics of the streamflow time series that influence water resources system modelling and planning can include the sequencing of flows on daily and longer time steps, spatial and temporal variability of flows, seasonal distribution and characteristics of high and low flows.

The best available estimate of streamflow would be expected to come from water level observations made at a gauging station, converted to flow estimates using a well defined and stable rating curve. However, these observations are only available for limited number of gauging locations and for limited time span. Estimates for ungauged locations and for a much longer time period are needed for contemporary water management, and ways to make estimates for future possible conditions are also required.

A range of methods are available to estimate streamflow from catchments, using observed data wherever possible, or estimating by empirical and statistical techniques, and more commonly using rainfall-runoff models. The modelling approach used to estimate streamflow by different water agencies and consultants varies across Australia and depends on the purpose of the modelling, time and money available, and the tools and skills available within the organisation. With increasing levels of inter-agency collaboration in water planning and management, development of a best practice approach in rainfall runoff modelling is desirable to provide a consistent process, and improve interpretation and acceptability of the modelling results.

The purpose of this document is to provide guidance on the best practice for implementing fit for purpose rainfall-runoff models, covering topics such as setting modelling objectives, identifying data sources, quality assuring data and understanding its limitations, model selection, calibration approaches, and performance criteria for assessing fitness for purpose

1.2 Definition of Best Practice Best Practice Modelling can be defined as a series of quality assurance principles and actions to ensure that model development, implementation and application are the best achievable, commensurate with the intended purpose (Black et al., 2011).

What is in practice “best achievable, commensurate with the intended purpose” may be subject to data availability, time, budget and other resourcing constraints. Hence, what is meant by the term “Best Practice Modelling” can vary. Not only does it depend on the circumstances of the project, particularly providing results that are fit for the intended purpose, but it also depends to a great degree on interpretation in peer review. This, in turn, will be influenced by the general state of knowledge and technology in the modelling field, which can be expected to progressively develop over time (such as new remote sensing data sources coming on line, and new computing


hardware and software), as well as data, time, budget and resourcing constraints. Best Practice Modelling provides for a strategic approach to modelling which enables the trade-offs that may be imposed by these constraints to be better managed, and assists in identifying priorities for addressing model and data limitations.

1.3 Scope The eWater CRC has prepared generic Best Practice Modelling guidelines (Black et al., 2011). They aim to provide for an integrated approach that enables interactions and feedbacks between all domains relevant to water management (e.g. hydrological, ecological, engineering, social, economic and environmental) to be considered.

The procedure in that guidance is intended to be flexible enough to accommodate variations in the meaning of the term “Best Practice Modelling” and also allow for continuous improvement as the state of knowledge and technology in the modelling field develops and improves.

The eWater CRC will also provide guidelines to support the BPM guidelines in specific areas of hydrological modelling that relate to the products that they are developing. This guideline is intended to address rainfall-runoff model application with the objectives being to provide water managers, consultants and research scientists with information on rainfall-runoff models and how to choose one that is fit for purpose, the data available to develop them, and the calibration and validation methodologies.

There are a number of different purposes that a rainfall runoff model may be applied within an overall water resources or catchment modelling framework, such as eWater Source IMS. Most of these purposes relate to providing information to support decision making for some water management policy. In particular, this can involve:

• Understanding the catchment yield, and how this varies in time and space, particularly in response to climate variability: seasonally, inter-annually, and inter-decadally.

• Estimating the relative contributions of individual catchments to water availability over a much larger region, e.g. valley or basin scale.

• Estimating how this catchment yield and water availability might change over time in response to changes in the catchment, such as increasing development of farm dams, or changes in land-use and land management.

In some instances with a high quality network of long term stream gauges, most of this type of information can be estimated from the observations. However, the more common case is where there is some combination of short term stations, variable quality data, and gaps in spatial coverage. In these cases, spatial and temporal gaps in the information can be estimated using rainfall runoff models to:

Infill gaps caused by missing or poor quality data in an observed data series for a gauged catchment.

Estimate flows for a gauged catchment for periods before the observed flow record started or after when the observed flow record ends.

Estimate flows for an ungauged catchment.

Page 6

eWater CRC

Estimating flows from ungauged subcatchments within an overall gauged catchment.

Forecast flows for some immediate future period (typically for a period of between a few days and a few months), conditioned on current (or recent) observations of the catchment state.

Assess the impacts of human influences within a gauged catchment (for example landuse or vegetation cover change) and simulating the flows that would have occurred for the historical climate sequence with modified catchment conditions. This may include assessment of catchment conditions that may be non-stationary in either the observed record or for the simulation.

Assess the potential impact of climate variability and/or climate change on flows from a gauged catchment.

In some cases, several of the above purposes may be satisfied by rainfall runoff modelling for the same catchment. There are similarities in the approach that is taken to rainfall runoff modelling, even though the purpose may differ.


2 Overview of procedure for rainfall-runoff modelling

The generic guidelines (Black et al., 2011) outline a procedure for applying a hydrological model. This can be summarised as occurring in 4 phases:

1. Project management, 2. Problem definition, 3. Option modelling, 4. Compare Options and select the best.

This guidance deals only with problem definition and option modelling because the first and last phases are discussed sufficiently for the purpose of rainfall-runoff modelling in the generic guidelines. A further reason is that rainfall-runoff modelling is usually only a part of a larger hydrological modelling project and these phases would be most appropriately considered in the context of that larger project. Specific aspects of project management and option comparison that are directly applicable to the development of a rainfall-runoff model, such as accreditation, are dealt with at appropriate points in this guidance.

2.1 Problem definition

2.1.1 Problem statement and setting objectives

The problem to be addressed must be clearly articulated to minimise the risk that the wrong tool will be used for the job. The problem statement will give direction on what objectives will be considered in developing the rainfall-runoff model. As many water management decisions will often have more than one goal it will be important to ensure these are all identified.

Sometimes it can be useful to express objectives in a hierarchy that shows primary objectives, secondary objectives and so on. In this regard, consideration should also be given to possible additional future objectives and goals that could be met based on the current project or on future projects that build upon the model established in the current project. The decision on which option offers the best solution should be based upon whether, or how well, each option meets the agreed objectives (also see section 2.2.1 and 2.2.2 in the generic guidelines).

2.1.2 Understanding the problem domain

The choice of the rainfall-runoff model will vary based on the purpose the modelling is being done for, e.g., to understand seasonal low flow characteristics for an in-stream environmental need; or to assess over-bank flow frequency; or to estimate overall catchment yield on an average annual basis. The model selected, data required, and calibration approach adopted should reflect this requirement. Where the same model may be used for two or more different purposes, there may also be a need to focus the calibration on a number of different flow regimes. If ‘rough’ flow estimates are required

Page 8

eWater CRC

over large areas and the runoff generation methodology should be consistent then the data and modelling process will differ again.

2.1.3 Metrics and criteria and decision variables

Model calibration is a process of systematically adjusting model parameter values to get a set of parameters which provides the best estimate of the observed streamflow (in the case of rainfall-runoff models). The process of determining which particular set of parameter values are best for the intended purpose depends on what comparison metrics are used. Metrics should be used to quantify the acceptability of the developed model. In all cases graphical assessment and statistical results of some sort will be assessed to identify the ability of the calibrated model to reproduce the flows calibrated against.

Different metrics will be more effective in determining model appropriateness to meet different objectives. What these are should be considered when the problem is being defined. Understanding appropriate metrics allows model acceptance criteria to be identified.

2.1.4 Performance across multiple catchments and subcatchments

In some situations, the purpose of rainfall runoff modelling is to produce an estimate of the runoff at a single location where there is a streamflow gauge. If this is the case, the calibration and validation process may be performed for the single gauged catchment. This approach is justifiable in situations where gauged data is available for most of the period that flow results are required for and the purpose of the rainfall runoff model is to infill missing data during the period of record. It may also be justifiable where there is a requirement to extend the period of record at the single gauge.

A much more common situation is that flow time series estimates are required at several locations and that gauged streamflow data is also available at several locations. The locations where flow estimates are required may or may not overlap with the locations where the flow data is also available. An objective of any project that involves the application of rainfall runoff models to multiple catchments or subcatchments should be to demonstrate consistency in the rainfall runoff model response between those catchments and to explain systematic differences in the hydrological response between catchments and subcatchments in an appropriate and logical manner.

2.2 Option modelling This section describes the process of developing a rainfall-runoff model, further details on key components are provided in later sections.

2.2.1 Methodology development

The models and methodology employed should be appropriate for the purpose that the model will be used for. The choices made will be directed by the problem definition developed and any other information at hand to the modeller. Detail on the models


available and guidance on selecting models and methodology that is fit for purpose is provided in Section 3.

2.2.2 Collate and review data

The amount and quality of data available to develop a model should be determined at the outset of the project. This can influence the selection of models, the performance criteria, and the approach to calibrate models. A bare minimum data set sufficient to make an approximate estimate of mean annual catchment yield would include catchment area along with spatial and temporal characteristics of rainfall and potential evapotranspiration (PET). A comprehensive data set would include long-term streamflow measurements and rainfall and PET data collected at one or more locations within the catchment along with land use coverage, vegetation cover and impervious area information, including changes over time.

The quality of the data should be reviewed prior to using to detect errors, non-stationarity if any, and understand uncertainties that may influence estimates. Some methods are discussed in section 4.

2.2.3 Setting up and building a model

The catchment characteristics are considered along with the knowledge on data available and any other information available to the modeller. The rainfall-runoff model chosen is conceptualised and an initial parameter set is identified.

When the model is first set up consideration should be given to all constraints which are limiting and their effects on the modelling. Section 5 provides more details associated with this step.

2.2.4 Calibration and Validation

Model calibration is a process of systematically adjusting model parameter values to get a set of parameters which provides the best estimate of the observed streamflow (in the case of rainfall-runoff models).

The term “validation”, as applied to models, typically means confirmation to some degree that the calibration of the model is acceptable for the intended purpose (Refsgaard and Henriksen, 2004). In the context of rainfall runoff modelling, validation is a process of using the calibrated model parameters to simulate runoff over an independent period outside the calibration period (if enough data is available) to determine the suitability of the calibrated model for predicting runoff over any period outside the calibration period. If there is not enough data available, the validation may be performed by testing shorter periods within the full record.

It is normal in research studies to split the observed data sets into calibration and validation period prior to the study, to demonstrate the performance of the model under both sets of conditions. Use of this approach can cause problems in practical applications if a model demonstrates acceptable performance for the calibration data set but produces unsatisfactory results for the validation data set. An alternative approach in this situation is to calibrate the rainfall runoff model to all available data but

Page 10

eWater CRC

to demonstrate that the performance of the model is satisfactory over different sub-sets of the period that observed data is available.

Further discussion of model calibration and validation is provided in Section 6.

It is a very common situation in a project that involves rainfall runoff modelling for flow time series to be required for several catchments or subcatchments within the model domain and for data to be available from two or more stream flow gauges to facilitate calibration and validation. At locations where gauged flows are available and flow estimates are required, two options are available to the modeller:

• The rainfall runoff models may be calibrated independently for each gauged catchment. In this case, independent parameter sets will be derived for the rainfall runoff models of each catchment; or

• A joint calibration may be performed, with rainfall runoff models calibrated with consistent parameters to fit to the gauge records from two or more gauges. In this case, a single set of rainfall runoff model parameters will be produced for all of the catchments that represent a compromise to fit the flows at all of the gauges within that group.

Consideration should be given at the outset of modelling to the approach that will be used for dealing with flows from multiple catchments and subcatchments and from multiple gauges. The strategy for dealing with this issue should be documented at this point and revised, if necessary, during the process of calibrating and validating the models.

Calibration of a rainfall runoff model normally involves running the model may times, trialling different values of parameters, with the aim of improving the fit of the model to the calibration data. Calibration can be facilitated:

• Manually, with the modeller setting the parameter values, running the model to inspect the results and then repeating this process many times;

• Using automated optimisation, with an optimiser algorithm running the model hundreds or thousands of times with different parameter values; or

• Using a hybrid approach of automated optimisation phases, interspersed with manually implemented trials of parameter sets.

Defining the calibration and validation approach before commencing a modelling project can maximise the efficiency of the calibration process, whilst avoiding the temptation to “overfit” the model to noise in the observational data. A calibration strategy should therefore outline the:

• gauge locations where model calibration and validation will be performed;

• viable or allowable ranges for each model parameter value; • known constraints, dependencies or relationships between parameter values

(for example, the total of the three partial area parameters in AWBM, A1, A2 and A3 must sum to 1);

• period for calibration at each gauge location;

• period for validation at each gauge location;


• expected level of uncertainty in observations introduced by measurement uncertainty;

• metrics to be used to test calibration and validation performance;

• whether manual or automated calibration strategy will be adopted, or how a hybrid strategy of progressive manual and automated calibration will be implemented.

• If an automated or hybrid optimisation strategy is to be used, further details should be defined at the outset of the calibration process on:

• algorithms to be used for optimisation of parameter values;

• objective function(s) that will be used to test the calibration performance;

• weightings that may be applied in computation of objective functions, to encourage fitting to different parts of the flow regime (typically the relative weightings to high, medium and low flows); and

• the set of model parameters that will be optimised during calibration and constraints on the allowable range of values for each parameter.

Ideally, calibration strategy should be documented prior to the commencement of the calibration process. It may be appropriate for the calibration strategy to be reviewed during the calibration.

2.2.5 Sensitivity/uncertainty analysis

Relevant sources of uncertainty in typical order of importance include:

5. Model input data including parameters, constants and driving data sets, 6. Model assumptions and simplifications of what the model is representing, 7. The science underlying the model, 8. Stochastic uncertainty (this is addressed under “variability” below), 9. Code uncertainty such as numerical approximations and undetected software

bugs. The potential impacts of the above sources of uncertainty on the decisions that will be made using the model should be considered early in the modelling process and then re-examined once the model has been calibrated, validated and applied for scenario runs. Uncertainty becomes more important for estimation of events in the tails of the probability distribution, floods and droughts, than it is for consideration of events that are closer to the centre of the probability distribution (such as estimation of the mean annual runoff from a catchment).

2.2.6 Documentation and Provenance

Documentation is an important requirement for model acceptance. Its role is:

1. To keep a record of what was done so that it can be reviewed and reproduced, 2. To provide source or background material for further work and research, 3. To effectively communicate the results from models, and

Page 12

eWater CRC

4. To effectively communicate the assumptions made during the modelling process and the decisions made by the modeller during implementation of the model.

Good documentation supports a study and it will also enable someone coming along later to see what decisions were made, what was done to underpin the decisions and why, particularly if aspects of the project are revisited at some later time.

Provenance, as it might relate to hydrological modelling studies simply means the ability to trace the source/lineage of the data, model and modelling results. Reasons for providing provenance in rainfall runoff modelling include:

1. Accountability and a full audit trail for all modelled results. 2. Repeatability; ability to re-create a results data set using current data or better

understanding. 3. Reproducibility; ability to re-create a results data set exactly using all original

data, workflow ordering, assumptions and parameters. 4. Versioning of both entire workflow and systems implementation. Versioning of

the subcomponents and data sets will be the responsibility of those who govern them but must be captured by the system.

The degree of provenance required depends on the application and/or how the modelling system is intended to be used by the individual or organisation in future. Current best practise provenance is to save all input data and model/parameters version and workflow history such that the outputs can be reproduced in future if required. In the future the ability to register and resolve the type and identity of objects within the modelling process should reduce the requirement to capture and archive these objects, especially as modellers take greater advantage of services based point of truth data streams and modelling systems, and rely less on ad hoc locally managed resources.

2.2.7 Model acceptance and accreditation

The aim of model acceptance is to gain agreement that the model is fit for purpose. Information available from the model accreditation process (Reporting, QA documentation, Peer review) provides model development details and review results which will enhance model acceptance.

Peer review plays an important part, especially with stakeholders that are external to the organisation undertaking the model development. It is important for establishing the credibility, reliability and robustness of results and the methodology used to obtain the results. It is undertaken by people with specialist understanding in fields relevant to the project.

2.2.8 Use of accepted/accredited model

Once a calibrated model is evaluated against good quality data and has undergone thorough review process (model acceptance and accreditation), it can be used for modelling to support water management planning and policy decisions (provided that the model was accredited for similar purpose).


3 Model choice

3.1 Model selection Model selection is made based on an understanding of the objectives and the system being modelled (http://www.toolkit.net.au/Tools/Category-Model_development; CRCCH 2005a, b). The WMO (2008, 2009) report include the following factors and criteria as being relevant when selecting a model:

1. The general modelling objective; e.g. hydrological forecasting, assessing human influences on the natural hydrological regime or climate change impact assessment.

2. The type of system to be modelled; e.g. small catchment, river reach, reservoir or large river basin.

3. The hydrological element(s) to be modelled; e.g. floods, daily average discharges, monthly average discharges, water quality, amongst others.

4. The climatic and physiographic characteristics of the system to be modelled. 5. Data availability with regard to type, length and quality of data versus data

requirements for model calibration and operation. 6. Model simplicity, as far as hydrological complexity and ease of application are

concerned. 7. The possible transposition of model parameter values from smaller sub

catchments of the overall catchment or from neighbouring catchments. 8. The ability of the model to be updated conveniently on the basis of current

hydrometeorological conditions. Other things that should be considered are:

1. The level of modelling expertise available. 2. Whether the model is going to be used on its own, or if it is going to be used in

conjunction with other models. 3. Freedom of choice may be limited by a desire to minimise problems of different

models for much the same purpose in the same project area, or to avoid problems of different models in adjoining project areas, particularly where the models are linked in some way in the future or results compared in some way.

4. Whether uncertainty will be explicitly modelled. If uncertainty is to be explicitly included, what types of uncertainty are to be modelled (e.g. climatic uncertainty, uncertainty in climate change projections, uncertainty in rainfall runoff model parameter values); what approaches will be used to generate the replicates to represent uncertainty and how many replicates will be required to adequately quantify uncertainty.

5. Whether simulation or optimisation, or a combination of both, is adopted. 6. Whether the model is to be used for hindcasting or forecasting when being

applied in predictive mode.

Page 14

http://www.toolkit.net.au/Tools/Category-Model_development

eWater CRC

In essence the governing principle in choosing a model should be that it should not have more parameters requiring calibration or a greater level of detail than the available data can support, to minimise problems of spurious results and false calibrations.

3.2 Available models Rainfall runoff models can be represented by a range of approaches, in order of increasing complexity as:

• simple empirical methods (e.g., curve number and regression equations); • large scale energy-water balance equations (e.g., Budyko curve);

• conceptual rainfall-runoff models (e.g. SIMHYD, Sacramento, AWBM)

• landscape daily hydrological models (e.g., VIC, WaterDyn);

• fully distributed physically based hydrological models which explicitly model hillslope and catchment processes (e.g., SHE, TOPOG).

These categories have been used for ease of description, and there is overlap between these model types. Although these approaches vary in terms of the complexity with which they represent the rainfall-runoff transformation processes, all of them conceptualise the real processes using some sets of mathematical equations (and hence are all conceptual models of the physical environment). Similarly, conceptual rainfall-runoff models run in distributed mode can be classed as being landscape daily hydrological models. This section provides a discussion of the characteristics of each of these model types, along with a broad assessment of the strengths and weaknesses of each approach for rainfall runoff modelling (Table 3-1).

3.2.1 Empirical methods

Empirical methods to rainfall runoff modelling typically involve the fitting and application of simple equation(s) that relate drivers of runoff response to flow at the catchment outlet. Empirical equations are most often derived using regression relationships.

Common predictor variables may include rainfall for the catchment, flow observed at another gauge in the vicinity, evapotranspiration, groundwater levels, vegetation cover and the impervious area within the catchment. Where rainfall is used as a predictor variable, regression relationships derived almost always include a non-linear relationship between rainfall and runoff.

All catchments incorporate storage elements, including interception by vegetation, storage within the soil column, groundwater storage and storage within stream channels. Catchment storage typically results in runoff from the catchment being an integrated function of the climatic conditions for the catchment over some period prior to the period for which runoff is to be calculated by the model. Therefore, empirical models that produce acceptably accurate simulations of runoff are either applied at sufficiently long time steps that changes in internal water storage within the catchment can be ignored (e.g. annual time step) or applied to represent an integration of the climatic conditions that occurred over some time period prior. As a practical example, for most catchments a regression model that only includes daily rainfall on the current day is likely to produce a very poor estimation of daily runoff but a model for predicting


daily runoff that used individual values of daily rainfall for several days prior may produce acceptable runoff estimates.

Empirical regression relationships are often developed using spreadsheets. They can also be fitted using more sophisticated statistical analysis packages, which may more easily facilitate the investigation of predictor variables. For general information on the development of regression relationships, the modeller is referred to NIST/SEMATECH e-Handbook of Statistical Methods (NIST and SEMATECH, 2010) or to a University Level statistics text book.

Empirical regression equations are best suited to situations where there are two flow gauges on the same stream with partially overlapping periods of record, which are therefore subject to similar climatic drivers, and the regression equation is used to extend the simulated flow to the combined period of record from both sites. They can also produce adequate simulations for neighbouring gauged catchments with overlapping periods of record in situations where the two catchments are subject to similar rainfall timeseries and are relatively similar hydrologically.

3.2.2 Large scale energy-water balance equations

The large scale energy-water balance methods are based on the hypothesis of available energy and water governing large scale water balance (precipitation, evaporation and runoff). These are usually developed using large scale observed data sets, eg. the Budyko curve (Budyko, 1958) was developed using mainly European data, and numerous other forms have been proposed to improve estimates in local regions and to account for different land cover types (Arora, 2002). One of the more popular forms of the Budyko method is the Fu (1981) rational function equation (Zhang et al., 2004) where a single parameter, α, in the equation can be calibrated against local data to tune the method for the local conditions. The inputs to these equations are rainfall and potential evapotranspiration (PET) and the output is runoff at mean annual time step.

3.2.3 Conceptual Rainfall-Runoff Models

Conceptual rainfall runoff models represent the conversion of rainfall to runoff, evapotranspiration, movement of water to and from groundwater systems and change in the volume of water within the catchment using a series of mathematical relationships. Conceptual rainfall runoff models almost always represent storage of water within the catchment using several conceptual stores (or “buckets”) that can notionally represent water held within the soil moisture, vegetation, groundwater or within stream channels within the catchment. Fluxes of water between these stores and in and out of the model are controlled by mathematical equations.

Most applications of conceptual rainfall runoff models treat the model in a spatially lumped manner, assuming that the time series of climatic conditions (notably rainfall and potential evapotranspiration) and the model parameter values are consistent across the catchment. There have been implementations in more recent times of conceptual rainfall runoff models in semi-spatially distributed and spatially distributed frameworks. In distributed application, the catchment is defined by grid cells or subcatchments within the catchment that are assigned the same rainfall runoff

Page 16

eWater CRC

parameter values but different time series of climatic inputs so that different grid cells or subcatchments within the catchment produce different contributions to the overall runoff. This is effectively a series of lumped rainfall runoff models, with lumped sets of model parameters that are applied with spatially distributed rainfall.

Conceptual rainfall-runoff models have been widely used in Australia for water resources planning and operational management because they are relatively easily calibrated and they provide good estimates of flows in gauged and ungauged catchments, provided good climate data is available.

In Australia there are six widely used conceptual rainfall-runoff models; AWBM (Boughton 2004), IHACRES (Croke et al. 2006), Sacramento (Burnash et al. 1973), SIMHYD (Chiew et al. 2002), SMARG (Vaze et al., 2004) and GR4J (Perrin et al. 2003). The input data into the models are daily rainfall and PET, and the models simulate daily runoff. The models are typical of lumped conceptual rainfall-runoff models, with interconnected storages and algorithms that mimic the hydrological processes used to describe movement of water into and out of storages. They vary in terms of the complexity of the catchment processes that they try to simulate and in terms of the number of calibration parameters which vary from four to eighteen.

3.2.4 Landscape daily hydrological models

These models are based on the concept of landscape processes and they model the typical landscape processes using simplified physical equations (VIC model, Liang et al., 1994; 2CSALT, Stenson et al., 2011; AWRA-L, Van Dijk, 2010). A catchment is usually conceptualised as a combination of landscapes which are delineated using some combination of outputs from digital elevation model analysis, underlying geology, soil types and land use. Often these models have been designed to reproduce other variables in addition to streamflow (e.g. distributed evapotranspiration, soil moisture, recharge, salinity), and as a result have a greater complexity to methods that target streamflow alone.

3.2.5 Fully distributed physically based hydrological models which explicitly model hillslope and catchment processes

The physically based models are based on our understanding of the physics of the hydrological processes which control the catchment response and use physically based equations to describe these processes. A discretisation of spatial and temporal coordinates is made at a very fine scale for the entire catchment and the physical equations are solved for each discretised grid to obtain a solution.


Table 3–1 Assessment of Strengths and Weaknesses of Different Rainfall Runoff Model Structures

Criteria Model Type

Empirical Large Scale Energy-Water Balance

Conceptual Landscape Daily

Fully Distributed Physically Based

Typical Run Time Step

Can be daily if daily flow from another gauge is used as a predictor variable. Otherwise typically only applied at annual (or longer) time scale

Typically only applied for mean annual runoff, although pattern of flows from a nearby gauge may be used to disaggregate annual totals to monthly or daily time steps

Daily, although shorter run time steps are possible if sufficient climatic data is available at this shorter time step

Daily, although shorter run time steps are possible if sufficient climatic data is available at this shorter time step

Minutes to hours to maintain numerical stability, although often forced with daily data and assumed patterns used to disaggregate to shorter time steps

Typical Number of Parameters

1 to 5 2 to 4 4 to 20 10 to 100 10 to 1000's

Risk of over-fitting or over-parameterising the model.

Low Very Low Moderate High Very High

Need for high resolution spatial data layers

None to Moderate

Low to Moderate

Low High Very High

Strength of Apparent Connection between Model Parameters and Measurable Physical Catchment Characteristics

None None Weak for most parameters (although impervious area or interception may be exceptions)

Moderately weak

Claimed to be strong by proponents but can be difficult to validate this claim

Run time on typically available computer platforms for 100 years of daily data

<1 second <1 second <1 to 60 seconds

10 seconds to 100 minutes

1 minute to several hours

Page 18

eWater CRC

Criteria Model Type




Ability to implement multiple runs for automated calibration

Not typically required - optimum parameters can be obtained by least squares fitting that does not require multiple runs

Not typically required

Very Good. Run times are typically sufficiently low to facilitate this and tools are available (Rainfall Runoff Library and Source IMS) to facilitate this

Good. Run times likely to be sufficiently low to facilitate this in most circumstances, however tools for calibrating such models using automated routines are not as widely available

Poor. Run times are generally too long to consider automated calibration

Typical Performance in Regionalisation

Moderate at annual time steps. Usually very poor at shorter time steps (e.g. Daily)

Good at annual time steps. Usually very poor at shorter time steps (e.g. Daily)

Moderate at daily time steps

Proponents claim to be superior for regionalisation to conceptual rainfall runoff models

Proponents claim this to be a strength of distributed models but in reality the large number of parameters required may compromise the application of distributed models to regionalisation

Representation of non-stationarity in catchment conditions

Not possible

Often applied to explicitly represent non-stationarity in vegetation cover for mean annual runoff signal

Usually difficult, due to lack of physical meaning for many model parameters

Possible Possible


Criteria Model Type




Typical performance of model when applying to a very different climatic period to that used for calibration

Poor Moderate when used to estimate impact on mean annual flow but flows disaggregated to shorter time steps are likely to be poorly estimated

Variable - can be good in some catchments but poor in others



Typical level of expertise with this type of model within Australian water industry

Strong Moderate Strong Weak Very weak

Likelihood that previously calibrated models are available for catchment to be modelled.

Moderate to Low

Moderate Very High Low Very low

Page 20

eWater CRC

4 Collate and Review Data

Climatic data is the most important driver of any rainfall runoff modelling process. The calibration and validation of models also involves comparison to observed streamflow data. Major causes of difficulty in calibrating rainfall-runoff models are errors and uncertainties in the input data (see Kavetski et al, 2003). A discussion of these problems can be found in the collection of papers in Duan et al 2003. Checks should therefore be performed on the input data and the comparison data set for calibration and validation to be used in rainfall runoff modelling before any attempt is made to apply or calibrate the models. The intent here is to investigate the integrity of the data, whether observations are in the first instance plausible (e.g., is the volume in a hydrograph less than the product of the rainfall and catchment area). Investigations into data to be used for rainfall runoff modelling should include checks of:

• Stationarity of the data time series , i.e. has there been any systematic or step change in the statistical properties over the time of data collection, and if so why;

• Spatial coherence of data, i.e., is the data consistent with regional spatial and temporal patterns and trends;

• Accuracy of the spatial location of the gauging site;

• Consistency in the approach used to date and time stamp the data, particularly for data provided by different agencies;

• Procedures use for spatially interpolation of point observations to gridded data estimates or estimated series across catchment areas

e.g., time series plots at different levels of temporal aggregation, ranked plots, residual mass curves, double mass curves, etc. This will pick up patterns as well as identify anomalies which may be potential data QA issues.

Other checks and analysis, including regional consistency of runoff depths, rain-runoff ratios, rating confidence limits, period of record, whether rainfall and PET is observed or interpolated, base-flow characteristics, checks for stationarity and variability over time, etc would also be useful. It is important that prior knowledge is considered.

One major factor which will apply across all types of time series data used is that the time base must be kept consistent so that the data applies to the same time period. An example is where flow data time steps should be matched to the rainfall data time step. In Australia, daily rainfall data is commonly recorded as the depth of rainfall that occurred in the 24 hours preceding 9 am on the date of the recorded data. In contrast, daily streamflow totals are often quoted for the 24 hour period commencing on the nominated date, resulting in the recorded flow data being offset by 1 day forward of the rainfall data. Where possible the flow data should be extracted at a time step to match the rainfall dataset. HYDSTRA databases allow this where the records are at short time intervals. In other cases shifting the recorded time series by one day for either the rainfall or flow time series may be required to produce consistent time series for modelling.


The remainder of Section 4 outlines the data types, sources, availability, accuracy, manipulations (such as gap filling) and any other issues.

4.1 Catchment details

4.1.1 Location of gauges (streamflow, rainfall and evaporation)

The streamflow recorded at the catchment outlet is a combined response to the spatial distribution of rainfall and evaporation across the catchment. There are uncertainties associated with the streamflow measurements due to rating curve errors as well as due to extrapolation outside the limits of the rating curve. There is spatial variability in rainfall (and to smaller extent evaporation) across a catchment which is not captured when undertaking lumped catchment modelling using a single rain gauge. There might be problems with the location of the rain gauge in terms of capturing a representative rainfall for all the rainfall events especially for catchments with high rainfall gradients.

4.1.2 Topography and Catchment Areas

The catchment area for a catchment represents the contributing area to the catchment outlet where the streamflow is measured. The catchment boundaries (and the corresponding catchment area) can either be derived from topographic map layers or using the catchment digital elevation model (DEM) and a standard package such as ARCGIS. It is usually easier to determine catchment area for the catchments located in steeper terrain compared to those located in very flat areas (especially when using DEM).

Slope and dominant aspect may provide useful explanatory variables for estimating routing parameters or for regionalisation of rainfall runoff parameters between catchments.

4.1.3 Soil types

A catchments rainfall-runoff response is related to the soil types in the catchment. The surface soil characteristics determine the infiltration rates and so the contributions from different flow components (surface runoff, throughflow and base flow). Soils information can be obtained from any soils field work that has been undertaken in the catchment or from large scale soil properties maps (e.g. Australian Soils Atlas, Northcote et al., 1960). In most practical applications of conceptual rainfall-runoff models in Australia, soil information is seldom directly used as input in the calibration process because the inherent spatial variability in soil properties within a catchment is typically sufficiently large that it has been difficult to demonstrate statistically robust relationships between conceptual model parameters and soil properties.

4.1.4 Vegetation

Land cover/vegetation cover in a catchment can often be correlated with the amount of interception storage/loss and actual evapotranspiration in a catchment. The land cover across the catchment can be derived from large scale vegetation mapping based on satellite imagery or remotely sensed data. Vegetation cover data has not typically been used explicitly in directly determining rainfall runoff model parameters, although there

Page 22

eWater CRC

have been some recent studies which have demonstrated the importance of catchment land cover in rainfall-runoff model calibration and for predictions in ungauged basins (Zhang and Chiew, 2008; Vaze et al., 2011c).

4.1.5 Water Management Infrastructure

Water management infrastructure within a catchment can allow humans to make very substantial modifications to flows within a catchment. Water management infrastructure may include large dams, farm dams and off stream storages, extractions, man-made canals or diversion pipelines and discharges from sewage treatment plants. Locations of these infrastructures should be identified where they exist within the catchment so that their potential impact on streamflows may be assessed. Recorded flows at the catchment outlet may require adjustment to allow for the influence of water management infrastructure located upstream of each of the flow gauging locations.

4.2 Flow data Reliable measurements of streamflow data are critical for successfully calibrating a rainfall-runoff model to a catchment. The streamflow data for all the gauged locations can be obtained from the respective state government water management agencies or from the Bureau of Meteorology (in Australia). Considerations in checking streamflow data include:

• the agency collecting the data and the quality assurance procedures (if any) implemented by that organisation during data collection;

• reliability of the rating of levels to flows for the gauge;

• the accuracy, extent and currency of cross sections surveyed at the gauge site. (Surveyed cross sections may only extent to the top of the stream bank and gauging for flows extending onto the floodplain may use a cross section that is inaccurate);

• vegetation and substrate material for the channel bed, channel banks and floodplain and the influence of assumptions made about these on gauged flows;

• the ratio of the highest flow outputs to the highest flow that the gauge has been rated for;

• how hydraulically stable (variable over time) the rating site is; • examination of potential backwater effects for the site from influences that are

downstream of the site, such as stream confluences, bridge crossings, culverts, dams or weirs;

• hysteresis effects leading to different flow rates for the same recorded level on rising and falling limbs of hydrographs;

• how well maintained the gauging site and instrumentation used for measuring water levels has been;

• any changes to the gauging instrumentation over time; • the length of time since the last rating at high flows;

• length of record at the site;

• availability of quality codes with the flow data record;


• proportion of missing data;

• trends in when data is missing from the record (i.e. Is there any bias toward high or low flow periods, particular seasons, or are the gaps just random?) and how this might influence any infilling procedures; and

• if there are a number of gauges closely located that basically represent the same catchment the data sets may be able to be combined to give a longer record for the site.

Assessment of the above factors will inform whether the data is useful in calibration of the model, independent validation of the model or whether the data should be ignored.

4.3 Rainfall Rainfall is the main driver of runoff and so reliable measurements of rainfall are critical for successfully calibrating a rainfall-runoff model to a catchment. There are several sources for obtaining climatic data for a particular catchment:

• Site observations from Bureau of Meteorology climate database. • Site observations taken from monitoring sites collected by other organisations

that may exist outside of the Bureau of Meteorology database. Many jurisdictional databases contain rainfall records.

• Gridded data products, such as the Bureau of Meteorology’s Australian Water Availability Project (AWAP) or Queensland Centre for Climate Applications’ SILO data set.

It is important to be aware of how this data has been collected and what data quality control methods have already been applied to the data prior provision of the data set as this may influence the modelling results. This is particularly relevant to gridded products, such as SILO and AWAP (SILO, Jeffrey et al., 2001; AWAP, Jones et al., 2009), as these data sources generally use different algorithms to convert time-series observations at data points to gridded data products.

In a small catchment, considerably better results may be obtained from using rainfall station data from the BOM (http://www.bom.gov.au/climate/) or locally collected data than a gridded data set that smoothes observations from a smaller number of more sparsely located sites. In some cases it may be appropriate to adjust the station data, normally by a percentage, if the mean catchment rainfall can be defined using other sources e.g. isohyetal detail.

In large catchments there is spatial variability in rainfall across a catchment which is not captured when undertaking rainfall-runoff modelling using rainfall time series from the rain gauges. If using a single rain gauge, there might be problems with the location of the rain gauge in terms of capturing a representative rainfall for all the rainfall events. If using a spatial rainfall product (SILO or AWAP in Australia), there can be uncertainties introduced because of the method used for interpolating rainfall between rain gauges and changes in the rain gauge network over time. Interpolation methods currently used are more suited to areas where rainfall varies less over space and in time. They do not account well for orographic effects, and rainfall networks in Australia historically have not captured the spatial and temporal variations in tropical and monsoonal areas well.

Page 24

eWater CRC

Vaze et al., 2010b discusses testing carried out considering the effects of using different rainfall data sets on the calibration and simulation of conceptual rainfall-runoff models. They conclude that considerable improvements can be made in the modelled daily runoff and mean annual runoff with better spatial representation of rainfall. Where a single lumped catchment-average daily rainfall series is used, care should be taken to obtain a rainfall series that best represents the spatial rainfall distribution across the catchment. However where only estimates of runoff at the catchment outlet are required, there is little advantage in driving a rainfall-runoff model with different rainfall inputs from different parts of the catchment compared to using a single lumped rainfall series for the catchment.

4.4 Evapotranspiration The measured pan evaporation data can be obtained for all the locations with the evaporation gauges installed (in Australia from the Bureau of Meteorology (BoM) basic records). In Australia there are also some spatial climate products which use point evaporation measurements recorded by the BoM and use an interpolation schemes to produce spatial evaporation surfaces (SILO, Jeffrey et al., 2001; AWAP, Jones et al., 2009).

The network of pan evaporation recording stations in Australia is sparse in comparison to stream flow and rainfall networks, although there is some compensation in that typically potential evapotranspiration exhibits substantially higher spatial correlation than rainfall or stream flow. This limits the ability to accurately represent the true spatial and temporal variability in evaporation in models however the spatial variability in evaporation is much smaller compared to the variability in rainfall.

The BoM network records pan evaporation. Modelling requires potential evapotranspiration (PET). There are a number of methods to convert pan evaporation to PET including Penman Monteith, Morton’s and accepted pan factors. These use climatic variables in the conversion calculation including solar radiation, temperature, vapour pressure, and wind speed which are recorded at some pan recording stations but not all. This further limits the network available to draw data from.

When all the required data is available the conversion calculations will use the records but often some variable is missing and estimates of that variable are made and used. Where there is no data for the climatic variables, calculated pan to PET conversion factors from a nearby station can be used to derive PET from pan evaporation.

Commonly the spatial products have interpolated layers for a range of climatic factors and the spatial PET layer is calculated from data in these layers rather than interpolating PET calculated at recording stations.


5 Statistical Metrics for Testing Performance

There are many performance measures used to consider the acceptability of a rainfall-runoff model. In all cases visual assessment and statistical results of some sort will be assessed to identify the ability of the model to reproduce the flows it is calibrated or validated against. All may contribute to best practice and which measures are more appropriate will be directed by the modelling objective. A number of commonly used visual assessment techniques are outlined in Table 5–1. Statistical performance measures and their relevance in various study types are listed below in Table 5–2.

Table 5–1 Plots for assessing model performance

Plot Assessment and Purpose

Daily and monthly plots (linear and log)

Used to check the general size, shape and timing of hydrographs. Linear plots will better show medium and high flows and log plots low flows. Baseflow and recession characteristics can be reviewed. If recessions are frequently too flat then this can indicate that the interflow and baseflow are not represented correctly.

Scatter Plot

Scatter plots show the ability of the model to match flows on actual time steps. They show the flow ranges where the model is more accurate. Linear and log plots will show the variability across the various flow ranges. Often a line of best fit is shown to indicate the bias of the model in estimating flows.

Ranked Plots Commonly referred to as frequency of excedence or flow duration graphs, they show the percentage of time a flow is exceeded over the modelled period. They show whether the modelled output can replicate the observed flow regime. Flow duration curves are effective diagnostics to ensure that both the variability and the seasonal pattern are captured.

Cumulative mass or cumulative residual mass curves

Scatter plots and flow duration curves do not examine the time sequence of events. A model could appear to be replicating the flow regime however the replication of regimes during wet and dry periods may not be adequate. A cumulative residual mass curve is a cumulative plot of residuals (flow value - mean of all values). A residual, and therefore slope of the curve, will be positive during wet periods as flows are higher than average and during dry periods the slope will be negative. If the curves diverge there may be a data issue. If they diverge consistently in all wet or all dry periods it is likely that model parameterisation for wet periods or dry periods may not be appropriate.

Plotting average daily or monthly flows (average of all Days, average of all Januaries)

A simple diagnostic to ensuring that the model can replicate seasonality characteristics.

Page 26

eWater CRC

Table 5–2 Statistical performance measures (metrics) and their relevance in various study types (Y – Yes, N – No)

Metric Purpose

Runoff Yield

Climate change

Landuse change

Low flow

Water quality

Peak flow / floods

Difference in total runoff Y Y Y N N Y

Difference in total runoff over different seasons of the year* Y Y Y Y Y Y

Difference in total runoff contained within high, medium and low parts of the flow duration curve

Y Y Y Y Y Y (high

flows)

Difference in proportion of time that cease to flow occurs N Y Y Y Y N

Difference in the slope of logarithm of flow versus time for baseflow recession periods

N N Y Y Y

N

Mean square error between observed and modelled runoff Y Y Y N N Y

Coefficient of determination (often referred to as r²)

Y Y Y N N Y

Nash Sutcliffe coefficient of efficiency on daily flows Y Y Y N N Y

Nash Sutcliffe coefficient of efficiency on monthly accumulated flows Y Y Y N N N

Nash Sutcliffe coefficient of efficiency calculated using logarithm transformed flows

N Y Y Y Y

N

* Definition of seasons to be used will vary depending upon the climatic zone that the catchment is in. For tropical areas, two seasons (a “wet” season from December-April and “dry” season from May-November) may be appropriate. In Southern Australia, it may be appropriate to consider the four conventional calendar seasons (Dec-Feb, Mar-May, Jun-Aug and Sep-Nov).

** Definitions of high, medium and low flow ranges will depend upon the purpose of the study and the catchment. Typical ranges might be High flows: days in observed data in the 0 to 20% probability of exceedance range; Medium flows: days in observed data in the 20 to 80% probability of exceedance range; Low flows: days in observed data with greater than 80% probability of exceedance and above the cease to flow level at the gauge. Adjustment of the low and medium flow ranges may be required particularly in response to the probability of cease to flow conditions at the gauge.


6 Calibration and validation

6.1 Calibration Model calibration is a process of optimising or systematically adjusting model parameter values to get a set of parameters which provides the best estimate of the observed streamflow. Virtually all rainfall runoff models must be calibrated to produce reliable estimates of streamflow because there has been little evidence identified of strong links between physical characteristics of catchments and the parameters of rainfall runoff models (Beven, 1989). Models should always be calibrated to observed data to demonstrate that the model can produce observed flow time series with an acceptable level of accuracy. The acceptable level of accuracy will depend upon the statistics of the flow data to be reproduced, which is determined by the purpose that the model will be applied for.

A model may be available that has been previously calibrated for a catchment as part of another study. In this situation, the calibration performance of the model should be re-tested before it is applied because the purpose for developing the model may be different between the earlier and later applications, which may influence the calibration objectives.

When calibrating a model it should always be kept in mind that there are always going to be tradeoffs, for example between getting wet, dry, and average conditions correct, and those tradeoffs will be driven by the purposes the model will be used for.

6.2 Validation Model validation is a process of using the calibrated model parameters to simulate runoff over an independent period outside the calibration period (if enough data is available) to determine the suitability of the calibrated model for predicting runoff over any period outside the calibration period. If there is not enough data available, the validation may be performed by testing shorter periods within the full record.

Model validation is one of the most important steps in rainfall-runoff modelling as the performance of the calibrated model in the validation period provides us confidence in the modelling results when the calibrated model is used for simulating streamflow outside the measured streamflow period or when the model is used for predicting streamflow under future climate change scenarios.

Validation has often been achieved using a “split sample” process, whereby a period of observed data (say the first two-thirds of the available record) are used for calibration and the remaining one-third are used for validation. The model that was calibrated using the calibration data set is run for the validation period without changing the model parameters and the goodness of fit statistics are computed for the validation period. The split sample approach assumes that both the catchment and the climatic conditions that it is subject to are stationary in nature across the entire period that recorded data is available for. Evidence of stationarity (or non-stationarity) in catchment conditions that would affect the hydrological response during the period of

Page 28

eWater CRC

recorded data should be checked using independent data sources (such as aerial photography, satellite imagery, landuse, topographic or other spatial information).

A more sophisticated calibration approach can involve multiple calibration and validation periods. As in the simple split sample approach, the model is calibrated to a calibration period and then performance is tested over the validation period without changing the model parameter values. This approach is then repeated multiple times, with each replicate using different start and end dates for the respective calibration and validation periods. This allows a range of model performance statistics for calibration and validation periods to be reported.

There will be some instances with this calibration and validation approach where the calibrated parameters perform well against the calibration data set, but performs poorly against the validation data set. In research type investigations, where the modeller may be comparing different rainfall-runoff models, calibration methods, or objective functions, the validation results can be used directly to help decide the best model or method or objective function. However, in practical applications, a modeller may have to decide either not to change the calibrated parameters and report the poor results, or to recalibrate the model because the performance is unacceptable.

The modeller may choose the latter option, and may then recalibrate and compare against the ‘validation data set’ several times until the calibrated parameters perform acceptably against both data sets. However, as the validation data set has now been used to change the calibrated parameters, it is no longer an independent data set and has in effect indirectly become part of the calibration data set.

This risk of having much poorer performance in validation than calibration may be mitigated by ensuring as far as possible both data sets have similar flow distributions, An arbitrary approach to splitting the data, e.g., at the midpoint, may result in half of the data being in a much wetter period. A model calibrated to these conditions would not be expected to perform well under the drier conditions in the validation data set. More alternate approaches should be considered on how to split the data set, perhaps into non-contiguous periods, to ensure overall flow distributions are similar in each period.

Data is a valuable resource, and should be used to greatest effect. In most Australian conditions, long data sets are needed to adequately represent climatic variability. An alternative approach to having split samples is to use the complete data set to calibrate the model, then to report its performance for different sub-periods, e.g., first half and last half, or decadally, or driest X year period and wettest X year period. The objective would be to have a comparatively persistent performance across all these periods. This does not necessarily give you an independent assessment of performance, but does report on performance under different conditions.

Transposition of model parameter values from gauged to ungauged catchments may be tested using a spatial variant on split sample validation. Under this approach, component models from a gauged catchment with the calibrated parameter values for that catchment can be applied to another gauged catchment to test the uncertainty and bias introduced from transposition. Uncertainty ranges can be established by testing contributions flow series produced by model outputs with parameter sets adopted from


several different gauged catchments. Examples of the performance of these transposition approaches are discussed in Viney et al. (2009) and Chiew (2010).

Generally the same metrics used to assess the performance of the model during calibration are also used to assess model performance during validation. The model performance during validation is almost always poorer than during calibration because model parameters are deliberately not specifically fitted to the data for the validation period.

6.3 Calibration and Validation of Models to Single Gauge Sites, Multiple Gauge Sites and Regionalisation of Model Parameter Sets

It is a very common situation in a project that involves rainfall runoff modelling for flow time series to be required for several catchments or subcatchments within the model domain and for data to be available from two or more stream flow gauges to facilitate calibration and validation. At locations where gauged flows are available and flow estimates are required, two options are available to the modeller:

• The rainfall runoff models may be calibrated independently for each gauged catchment. In this case, independent parameter sets will be derived for the rainfall runoff models of each catchment; or

• A joint calibration may be performed, with rainfall runoff models calibrated with consistent parameters to fit to the gauge records from two or more gauges. In this case, a single set of rainfall runoff model parameters will be produced for all of the catchments that represents a compromise to fit the flows at all of the gauges within that group.

The advantage of the joint calibration approach is that, assuming some degree of homogeneity in the rainfall runoff response of the selected gauged catchments, the parameter sets produced should be more robust when applied to other catchments with similar response that were not used for the calibration.

If an automated calibration process is used for joint calibration of multiple catchments, the objective function used for automated calibration to the gauged catchments will be a weighted average of the objective function values produced at the individual gauges. Options for selecting the weighting values are:

• All gauged catchments contribute equally to the overall objective function;

• Weights are assigned according to the length of available record (e.g. number of days with data) at each site;

• Weights are assigned according to the inverse of the correlation coefficient in gauged flows between one gauge and one or more of the other gauges in the set (i.e. gauges with strongly correlated recorded flows are assigned lower weighting factors than gauges that have weaker correlations with other gauges);

• Some combination of the above factors. There are three main methods of developing flow data sets in residual ungauged catchments between upstream and downstream gauges:

Page 30

eWater CRC

1. Calibrate a model to the difference in flow between the gauged upstream flows routed to downstream (adjusted for known transmission losses) and downstream gauges.

2. Adjust a flow data set from a nearby catchment using either recorded or generated data,

3. Apply parameter values from other calibrated models and use catchment appropriate climate data.

There are two main methods of developing flow data sets in ungauged catchments:

1. Develop a regression equation between flows for the ungauged catchment and gauged catchments and apply this equation to transpose the flow, or

2. Apply parameter values from other calibrated models and use catchment appropriate climate data.

Generally in the second case parameters for a neighbouring or nearby catchment are used but climate data and catchment characterises of the catchment of interest are applied in the model. Many studies have shown that selecting a donor catchment based on spatial proximity gives similar or better results than selecting a donor catchment based on catchment attributes (Merz and Bloschl 2004, Oudin et al 2008; Parajka et al. 2005; Zhang and Chiew 2009).

6.4 Automated, Manual and Hybrid Calibration Strategies Calibration of hydrological models can be conducted using manual or automated methods, or a combination of the two approaches (see Boyle et al, 2000 and Bárdossy, 2007 for frameworks for combining manual and automated methods of model calibration). Calibration involves the adjustment of model parameter values to improve the fit of model output data to observations to a level that is acceptable.

In case of manual calibration, definition of “goodness of fit” is usually produced as a combination of statistical indices and visual inspection of the observed and simulated hydrographs. Whereas in case of automated calibration, definition of “goodness of fit” is usually produced using an objective function. The objective function translates the observed and modelled outputs into a single number, so that the results of successive calibration iterations can be compared. Automated calibration routines use a defined algorithm that runs the model multiple times, adjusting model parameter values according to a strategy that is intended to improve the value of the objective function.

The sections that follow give information on the calibration methods available and their relevance in various study types (shown in Table 3-1) which dealt with model choice appropriate for intended purposes.

6.4.1 Manual Calibration

Manual calibration involves the modeller selecting a set of parameters for their model, running the model once and then examining the output statistics from the model (from the list discussed in Section 5). The modeller would then revise the values of one or more parameters and repeat the above process. This may continue many times until the model achieves the desired level of performance.


The match between simulated and observed streamflow can be visually assessed as either a time series, or as flow duration curves or residual mass curves. The visual assessment can identify general deficiencies in the matching of the hydrologic regime, e.g., high flow events under or over estimated, baseflows under or over-estimated or the seasonal response of the model not captured appropriately. Software that stores the results of conceptual storages and fluxes for graphing, and interpretation of these results in the context of model structure is also useful to identify which parameter values need adjusting and in which direction in order to improve results.

Guidelines are available from the developers of the Sacramento model that describe how to estimate key parameter values directly from analysis of recorded hydrographs (Burnash, 1995). A range of realistic parameter values has also been recommended to guide initial estimates.

Strengths:

1. Encourages a deeper understanding of model structure and its applicability to catchment hydrology, rather than treating as a black box.

2. Allows for hydrologist to consider performance against a broad range of performance metrics, and make appropriate adjustments.

3. Takes into account understanding of the data and the catchment. 4. Allows a logical checking at each change. 5. Produces a greater appreciation of strengths and limitations of calibrated result.

Weaknesses:

1. Repeatability is limited. Different people may get different parameters and output flow time series.

2. More effort and time required to complete a calibration. 3. Difficult to manually calibrate models with more than about 10 parameters. 4. Only amenable to calibration for one gauge at a time and difficult to

simultaneously calibrate flows from multiple gauged catchments that may have similar hydrological response. As a result, in catchments with multiple gauges very different parameter sets for independently calibrated catchments may be produced that don’t represent the underlying similarity in hydrological response that would be expected from those catchments.

6.4.2 Automated Calibration

In computer optimisation routines, an objective function, which itself is selected by the modeller is used to formulate the calibration problem and the computer undertakes the calculations.

Strengths:

1. Can operate with one simple objective function or use of multiple objective functions.

2. Can undertake calibrations for multiple catchments in a short period of time. 3. Relatively easy.

Page 32

eWater CRC

4. Repeatable. Different people will get same parameter values.

Weaknesses:

1. Is confined by the optimisation routine chosen and how the objectives are set. 2. Is dependent on the computer routine being set up accurately to reflect the

choices in 1. 3. Lack ability to check the relationships between the calibrated parameter values

produced as the calibration proceeds, which may cause investigation of sets of parameters that are infeasible (unless appropriate checks are build within the calibration algorithms).

4. Software is required to automate the optimisation process. 5. Parameter values commonly become “trapped” against the minimum and

maximum constraints of the allowable parameter ranges set by the user. If the user does not check for this, the parameter set chosen may be sub-optimal as the best parameter set may have parameter values that lie outside the constraints set by the user at the time the optimisation is initiated.

There are two global optimisation methods included in Source: Shuffled Complex Evolution (SCE) and Uniform random sampling. The analysis undertaken as part of the testing with data from 200+ catchments in southeast Australia showed that there is an advantage in following a global optimiser with a local optimiser to fine tune the calibrated parameter values. The Rosenbrock method is included in the framework as a local optimiser. The testing results suggest that the combination of SCE followed by the Rosenbrock should be used (Vaze et al., 2011a,d).

6.4.3 Hybrid Calibration Strategies

Best practice for model calibration would normally involve a hybrid of manual and automated calibration approaches. A typical hybrid strategy would involve:

Implementation of an automated calibration routine using a global optimiser (such as SCE), with most (if not all) of the possible parameters allowed to vary within the widest ranges that are physically plausible for that model and catchment.

Visual inspection of the time series and other statistical measures (see Section 5) from the run with the parameter set that produced the optimum value of the objective function. This inspection should aim to identify aspects of the model that are performing well and those that are performing poorly, to provide insight on the particular parameters that are influencing the (poor or good) quality of the fit.

Implementation of one or many trials of manual calibration, adjusting the value of one or two parameters at a time that are likely to be particularly influential on aspects of the modelled results that the modeller observes are undesirable. The outcome of these manual calibration runs would provide insight about fixed values for particular parameters, or tighter ranges for constraining the value of particular parameters in subsequent automated trials. An example might be that the global optimiser implemented to optimise the NSE finds the baseflow recession parameter (KBase) for an AWBM model as 0.95/day, but examination of the flow duration curve and time


series reveals that adjusting the parameter to 0.99/day produces a much better fit to baseflow recessions without compromising the fit to high flows. The modeller could then choose to either fix KBase at 0.99/day for subsequent trials or to constrain the KBase parameter within a tighter range (say 0.985 to 1.0/day) for subsequent trials.

Implementation of further trials of automated calibration using a global optimiser but this time with tighter constraints on most or all of the parameters. Appropriate ranges for parameters could be assessed either from the previous (manual) calibration step, or from the ranges of each parameter from the 20, 50 or 100 runs producing the best value of the objective function from the first automated calibration.

Visual inspection of the time series and statistical measures produced by runs that produced the optimum value of the objective function and also from runs that produced near to the optimum value.

Steps 2 through 5 may be repeated several times by the modeller, allowing different sub-sets of parameters to be fixed and allowing for different allowable ranges for each parameter.

Implementation of a local automated optimiser, such as a Rosenbrook search, starting from the parameter set that is most desirable to the modeller identified in Step 5.

Over the last decade or so, there has been a debate in some parts of the hydrological modelling community with calibration (particularly automated calibration) to a single objective function value. This has led to the development of calibration approaches that consider multiple objectives (Gupta et al., 1998; Madsen et al., 2000). Viney et al. (2009) have investigated the use of hybrid objective functions (Nash Sutcliffe Coefficient of Efficiency with various bias constraints) in optimisation of rainfall runoff models for the purposes of estimation of catchment runoff. Kuczera et al, (2006) and Kavetski et al. (2006a, 2006b) have suggested an alternative Bayesian approach to model calibration that attempts to deal with multi-objective optimisation by considering possible calibration solutions that overcome many of the limitations of single objective optimisation. While these approaches have been demonstrated in the research domain, practical tools for use by general practitioners for multi-objective calibration are generally not easily accessible.

6.4.4 Selection of Objective Functions for Automated and Hybrid Calibration

Automated calibration requires the use of an objective function to direct the optimisation process. The Source calibration tool implements single objective function optimisation. The calibration tool reduces the comparison between the observed and modelled data during the calibration period to a single number to be optimised. Other tools (such as Insight) perform multiple objective optimisation, which searches for many options of possible solutions that are optimum for at least one of the multiple objectives that are chosen. Multiple objective optimisation has not been implemented within the Source calibration tool, although it may be introduced in later versions. Multiple objective optimisation has the advantage that it explicitly presents the modeller with many possible parameter sets and it then allows the modeller to make the choice after the optimisation has been run between parameter sets.

Page 34

eWater CRC

Objective functions are normally chosen as one of the statistics that are discussed in Section 5, or a combination of two statistics from this table. The objective functions that are currently implemented within the Source calibration tool are:

1. Match to Nash Sutcliffe Coefficient of Efficiency of Daily Flows 2. Minimise Bias between Observed and Modelled 3. Match to Nash Sutcliffe Coefficient of Efficiency of Daily Flows but Penalise

Biased Solutions (NSE-Bias) 4. Match to Nash Sutcliffe Coefficient of Efficiency of Monthly Flows 5. Match to Nash Sutcliffe Coefficient of Efficiency of Monthly Flows but Penalise

Biased Solutions 6. Combined Match to Nash Sutcliffe Coefficient of Efficiency and Match to Flow

Duration Curve (Daily) (NSE-FDC) 7. Combined Match to Nash Sutcliffe Coefficient of Efficiency and Match to

Logarithm of Flow Duration Curve (Daily) (NSE-logFDC) The choice of any particular objective function will depend on the intended application. Each of the pre-defined objective functions are formulated to put emphasis (reproduce as closely as possible) on different flow characteristics.

The Nash-Sutcliffe Efficiency (NSE) is commonly used in hydrological modelling to describe the agreement between the modelled streamflow and observed streamflow time series. A NSE value of 1.0 indicates that all the modelled daily streamflows are the same as the observed daily streamflows, and an NSE value of less than zero indicates that that the model simulations are poorer than simply using the mean observed daily streamflow as the streamflow estimate for every single day. The daily NSE tends to produce solutions that match high and moderate flows very well but often will produce poor fits to low flows. NSE will also tend to favour solutions that provide a good match to the timing and shape of runoff events.

The minimise bias (number 2 in the list) objective function will produce a match on the overall volume of flow generated but often will produce a poor fit to the timing of flows. It should normally only be used to refine calibrations determined from other methods with the local (Rosenbrook) optimiser.

The NSE-Bias objective function (number 3 in the list) formulation makes sure that the models are calibrated predominantly to optimise NSE while ensuring a low bias in the total streamflow. It avoids solutions that produce biased estimates of overall runoff, which can produce marginal improvements in low flow performance over the NSE objective function. However, NSE-Bias will still be strongly influenced by moderate and high flows and by the timing of runoff events, which can still often result in poor fits to low flows.

The NSE of monthly flows and NSE-Bias of monthly flows (numbers 4 and 5 on the list above) can be useful for initial calibration because they tend to find solutions that will match the overall movement of water through the conceptual stores in the rainfall runoff model, without being influenced by the timing of individual runoff events.

The NSE-FDC and NSE-logFDC (numbers 6 and 7 on the list above) are hybrid objective functions that compromise between the fit to the timing of high and moderate


flows from the NSE component and the fit to the shape of the whole flow duration curve (FDC). The NSE-logFDC (number 7) will produce a closer fit to low flows than the NSE-FDC (number 6). They require the modeller to select a weighting before the implementation of the optimisation to the NSE and FDC / logFDC components. Higher weightings (between 0.5 and 1) shift the balance toward the NSE component (and consequently toward matching timing and high flows), whilst lower weightings (between 0 and 0.5) shift the balance toward the FDC or logFDC component (and consequently toward ignoring timing but matching the prevalence of baseflows). The modeller should therefore consider the influence that the weighting of the two components in the combined objective function has on the outcomes of their automated calibration. The weighting can be adjusted during successive runs of automated calibration: e.g. commencing with a value of 0.5, then modifying to a value of say 0.2 if there is a desire for more emphasis to be placed on matching the flow duration curve, particularly for low flows.

Automated calibration using four of the objective functions were tested with a global optimiser (SCE) followed by a local optimiser (Rosenbrook) for a large number of catchments in Eastern Australia. Testing of the results of the calibration with each of the four objective functions indicate that the NSE-Bias objective function provides the best estimates of daily flows, timing and volume ratios in majority of the catchments when compared to the other three objective functions. If daily sequencing is not important for the intended application, any of the other three objective functions can be used. If the main interest is to reproduce the low flows whilst compromising the fit to moderate and high flows, the NSE- logFDC objective function should be used (see Figure 1 on the next page).

The above discussion underlines the importance at the end, and often during, the calibration process of testing the performance of parameter values produced by automated calibration against multiple metrics that are appropriate for the problem that the calibrated rainfall runoff model is being applied to. Following rainfall runoff calibration, the model should perform adequately against a number of different metrics that represent the characteristics of interest from Table 5–1 and Table 5–2. Automated calibration presents a risk of selecting model parameters that reproduce aspects of the flow regime that are well detected by the selected objective function used in the optimisation at the cost of ignoring other important aspects of the flow regime.

6.4.5 Further Guidance on Calibration and Validation of Conceptual Rainfall Runoff Models

This section contains a number of miscellaneous guidance about calibration and validation of conceptual rainfall runoff models.

There are a number of potential problems that can occur during calibration, particularly during implementation of automated calibration. The following lists a number of potential problems and issues that should be considered:

Page 36

eWater CRC

0.01

0.1

1

10

100

1 10 100 1000 10000Days

Dai

ly fl

ow d

urat

ion

curv

e (m

m) Observed

NSE low flow fdcNSE NSE fdcNSE Bias

0

20

40

60

80

1 10 100 1000 10000Days

Dai

ly fl

ow d

urat

ion

curv

e (m

m)

Figure 1: Calibration results for different objective functions

• Values of one or more parameters can become “trapped” near the minimum or maximum values of the range that the modeller has allowed. The modeller should always check the value of each parameter against the range of values that the particular parameter was allowed to vary within. A parameter value that is close to one of the limits (say within 1% of the minimum or maximum value) may indicate that the allowable range for the parameter provided as an input to the automated calibration process was too narrowly defined and that widening the allowable range for the parameter may allow the automated optimisation to find a significantly better overall result. Of course, there may be a physical constraint that means that the parameter range and optimum value is appropriate, e.g. the depth of soil moisture stores cannot be negative and the optimum value for a particular parameter may then fall close to 0 mm.

• The model may produce a good fit to the objective function but poor performance on other statistics. As discussed in Section 5, the performance of the model should always be checked against a number of different statistical measures.

• The model may produce a good fit to calibration data but poor fit to validation data. This may indicate some inherent non-stationarity in the hydrological response of the catchment.


• The model may produce an inappropriate split between surface and baseflow contributions. In this case, objective functions that are weighted toward matching the baseflow contribution or constraining the ranges of parameters that particularly control the baseflow response should be considered.

• If an automated calibration routine is implemented, the model may fail to converge to an acceptable solution. In this case, the input data (particularly rainfall and streamflow) should be checked. If this appears reasonable, the calibration should be re-run for a much larger number of iterations and possibly with wider ranges on the parameters to let it find an optimal solution.

• Local optimisers (Rosenbrock) may find only a local optimum solution and miss a solution that would otherwise produce a much better fit. This is why global optimisers should always be implemented as the first step of any automated calibration routine and the local optimiser only used as the final step in the optimisation process to “sharpen” the fit.

• The structure of most rainfall runoff models is such that there is a strong degree of interaction between model parameters. In other words, changing the value of one parameter of the model can result in changing the value of one or more of the other parameters of the model to produce a fit to the observed data that is equivalent or almost equivalent.

• Performance should be tested against a range of metrics for different flow regimes (high, medium, low), as well as for wet and dry periods. A good all-round performance indicates that the calibration is representing the catchment under most conditions.

• If the purpose of the modelling is to derive time series of constituent loads for water quality modelling, it is usually the case that runoff and constituent loads are dominated by episodic events. Therefore, in looking at the performance of the model, higher weighting should be placed on statistics that emphasise runoff events than on low flows. The baseflow index (the ratio of mean annual baseflow to mean annual flow) produced by the modelled outputs should be similar to the baseflow index from gauged streamflow data. As a guide, the baseflow index should be within 10% in the modelled and observed data. Significant differences in baseflow index may result in inaccurate predictions of loads, particularly when an Event Mean Concentration (EMC)/Dry Weather Concentration (DWC) generation model is adopted. The baseflow index produced by the model can normally be corrected relatively easily by manually calibrating parameters in the model that control recharge to and leakage from baseflow stores, or constraining the range of allowable values of these parameters in an automated calibration approach.

• In case of nested catchments, some gauges in the catchment will be better rated and provide more accurate flow measurements than others, particularly in larger flow events. In deriving a combined objective function to fit parameters across the whole catchment, weightings should be adjusted in the combined objective function toward gauges with more reliable data in preference to gauges with less reliable data.

• A check should be performed that the mean annual runoff from the model at gauging sites is close to the observed mean annual runoff. As a guide, the mean annual runoff should match to within 5%.

Page 38

eWater CRC

• Gauges should be identified that appear to be outliers, where a good calibration performance is difficult to obtain. This may identify problems that are not immediately evident in the data, such as additional inflow or extractions not included in the model, losses/gains to/from groundwater or rating errors at high flows.

• Comparison of the data over the model calibration and validation periods should be used to identify other issues such as potential changes in instrumentation, rating or instrument malfunction that may not have been detected during earlier quality assurance checks on the recorded data.

• Land use change or events such as bushfire can be evident in the observed data but may not have been included in the model, which will impact on the performance of the calibration. Measures to address this should be considered, with options of either adjustment to the structure of the model to explicitly represent these effects, or calibrating the model on a different period of data.

• It is generally more difficult for a model calibrated over a wet period to predict runoff over a dry period compared to a model calibrated over a dry period to predict runoff over a wet period (Vaze et al., 2010). If possible it is suggested that the calibration flow data should include both wet and dry periods (rainfall periods about 20% higher and lower than the long term mean annual rainfall for the catchment).

6.5 Calibration of Regression Models Empirical regression models are most commonly applied to provide an estimate of flow at one gauge location, based upon recorded flows at another nearby flow gauge. They may also incorporate other drivers of runoff response, such as rainfall on the previous day and possibly total rainfall for a defined number of preceding days.

Regression models are commonly fitted by minimising the least square of the residuals between flows from the recorded and modelled flow time series. Least squares fitting can be easily conducted within a spreadsheet, using the common tools provided to undertake those analyses.

Regression models almost always assume that the catchment is stationary in terms of its runoff generation characteristics. Care should also be taken to confirm that the period used to fit the model is climatically representative of the overall simulation period. If the simulation period is substantially wetter or drier than the model calibration period it is very likely that a regression model will produce biased results, due to the inherently non-linear rainfall to runoff conversion relationship that is observed in most catchments.


7 Uncertainty and Sensitivity Analysis

Relevant sources of uncertainty in typical order of importance include:

1. Model input data including parameters, constants and driving data sets, 2. Model assumptions and simplifications of what the model is representing, 3. The science underlying the model, 4. Stochastic uncertainty (this is addressed under “variability” below), 5. Code uncertainty such as numerical approximations and undetected software

bugs. The potential impacts of the above sources of uncertainty on the decisions that will be made using the model should be considered early in the modelling process and then re-examined once the model has been calibrated, validated and applied for scenario runs. Uncertainty becomes more important for estimation of events in the tails of the probability distribution, floods and droughts, than it is for consideration of events that are closer to the centre of the probability distribution (such as estimation of the mean annual runoff from a catchment).

The major uncertainty is most likely associated with the input data sets. There are uncertainties associated with the measured rainfall, PET and streamflow (associated with the rating curve). There is spatial variability in rainfall across a catchment which is not captured when undertaking lumped catchment modelling. If using a single rain gauge, there might be problems with the location of the rain gauge in terms of capturing a representative rainfall for all the rainfall events. If using a spatial rainfall product (SILO, Jeffrey et al., 2001 or BAWAP, Jones et al., 2009), there can be uncertainties introduced because of the method used for interpolating rainfall between rain gauges and changes in the rain gauge network over time. It is up to the modeller to check the integrity of the input data and be aware of its limitations. If data problems cannot be fixed the implications for model calibration (particularly where optimisation is involved) and interpreting model results should be recognised.

Rainfall runoff models are inherently a simplification of the actual hydrological processes that are occurring within a catchment. The assumptions and simplification reflected in the structure of the hydrological model introduce uncertainty into the predictions produced by the model, which is sometimes referred to as structural uncertainty. Even if the real catchment did behave in a manner that was identical to the structure represented by the rainfall runoff model, there would be uncertainty introduced by the unknown values of the parameters of the rainfall runoff model.

The risks presented by uncertainty introduced by numerical approximations and undetected software bugs can be mitigated by rigorous testing of the computer software. There is an entire discipline of the information technology industry devoted to development and application of tests for software programs. The testing regime for a rainfall runoff model should include:

Page 40

eWater CRC

• Numerical testing of the results of the software code against an independently “coded” solution that has been constructed in a spreadsheet or in a previous implementation (e.g. in FORTRAN);

• Testing of the user interface; • Testing the results that are produced by the model for extreme values of each

of the model parameters, i.e. parameter values at the minimum and maximum values of their feasible limits;

• Testing that the model internally preserves a mass balance for each time step, i.e. the change in water stored within the conceptual stores in the model is equal to rainfall less evaporation less seepage losses less total runoff;

• Testing that one version of the model replicates the results of a previous version, particularly where there is ongoing development of the model and/or the framework that the model is implemented within.

7.1 Sensitivity Analysis A relatively simple means of producing a quantitative estimate of uncertainty from a rainfall runoff model is sensitivity analysis. In sensitivity analysis, one parameter of the model is typically varied at a time and the model re-run to test the change in the output produced by the change in the single input parameter. A typical approach might be to modify the value of each of the model parameters by +10% to check the percentage change in the mean annual runoff, followed by modifying each of the model parameters by -10% to check the mean annual runoff. This analysis would provide the modeller with an appreciation of how sensitive the mean annual runoff generated for the catchment is to a simple change in each of the input parameter values. Consistent with the discussion in Section 5, several metrics that reflect the purpose of the model should be checked during the sensitivity analysis.

Some deficiencies of sensitivity analysis are that:

• It only tests for the influence of parameter uncertainty and ignores uncertainty introduced by the model structure;

• The common method of testing for the same proportional change in each parameter (e.g. the +/- 10% change in each parameter value in the example above) does not usually reflect the fact that some parameters are inherently more difficult to estimate via model calibration and therefore more uncertain than other parameters. It can therefore produce misleading inferences because small changes in the value of one parameter may produce large changes in the output but because of this sensitivity the value of this parameter may be defined to within a relatively narrow range;

• There is typically strong interaction between parameters of rainfall runoff models, so that the change in the value of one parameter would be compensated for by the change in the value of one or more of the other parameters of the model. Sensitivity analysis on single parameters ignores parameter interaction.


7.2 Application of Multiple Parameter Sets

The process of calibrating rainfall runoff models using an automated optimiser normally involves a search of many thousands of possible parameter sets. While the focus of optimisation is often on the single parameter set that best calibrates the model, the optimisation process will normally return a large number (typically hundreds) of parameter sets that produce a performance from the model that is almost as good as the optimum parameter set. The results of the rainfall runoff model produced by these “almost as good” parameter sets can be used to quantify the uncertainty in the model predictions produced by parameter uncertainty. A typical approach, if 2000 model runs had been completed during the automated calibration process, would be to select the 100 model runs that produced the best value of the selected objective function to characterise the uncertainty in the time series produced by the rainfall runoff model.

This approach avoids the second and third deficiencies of sensitivity analysis because:

• Parameters that are well defined via the calibration process will be represented by a relatively tight distribution of values amongst the parameter sets that almost produce the optimum calibration;

• Interaction between parameter values is retained by selecting each parameter set.

However, a limitation of this method is that uncertainty introduced by model structure is ignored, which was also the case for sensitivity analysis.

7.3 More Advanced Quantitative Uncertainty Analysis A further advancement on the approach of testing many possible parameter sets that almost calibrate the model would be to calibrate several different rainfall runoff model structures to the same gauged data time series, which would return multiple possible parameter sets for each of the model structures, and then to choose combinations of model structure and parameter set that almost produce the optimum value of the objective function. An example might involve automated calibration of AWBM, SMARG, SimHyd, Sacramento, GR4J and IHACRES models to a particular catchment and then selecting the top 300 sets of parameters (along with the model structure) that produced the optimum value of the objective function from amongst all the model runs with all six selected model structures.

The Bayesian Total Error Analysis methodology (BATEA) provides the opportunity to directly address all sources of uncertainty (input, model and response error) in the calibration of conceptual rainfall-runoff (CRR) models. BATEA has demonstrated the potential to overcome the parameter biases introduced by poor conceptualisations of these sources of errors and enhance regionalisation capabilities of hydrological models (Thyer et al., 2007). Application of BATEA is currently within the domain of applied research.

Page 42

eWater CRC

7.4 Consideration of Uncertainty in Practical Applications of Rainfall Runoff Models

A hierarchy of techniques exist, as described above, for quantitative estimation of uncertainty for practical applications of rainfall runoff models.

The potential impact of uncertainty in rainfall runoff modelling on the decisions that are to be made using that model should be assessed before quantitative estimation of uncertainty is undertaken. In many situations, decisions will be made informed from the “best estimate” rainfall runoff model results and will implicitly ignore uncertainty.

Sensitivity analysis, despite its limitations, at least can be informative about the possible range of solutions that might be produced due to uncertainty in the parameter values. Consideration of uncertainty in this reasonably simplistic manner can often provide an indicative range of the possible range of solutions that may be produced.

Application of multiple parameter sets is facilitated using existing tools that are available to hydrological modellers (including PEST and the Source calibration tool) and it overcomes two of the major limitations of simple sensitivity analysis. In situations where quantifying uncertainty, at least the uncertainty introduced by model parameter uncertainty, is important then this is a viable approach that should be applied by the modeller.

More advanced uncertainty analysis techniques, including BATEA, are likely to advance as further research is conducted but the tools to implement these techniques are not currently available to most practitioners. If uncertainty in the model results is a primary driver of the decisions to be made on the basis of the rainfall runoff model results then these more advanced techniques of uncertainty quantification should be adopted.


8 Concluding remarks

There are a number of methods of undertaking rainfall-runoff modelling and all have strengths and weaknesses given the data available and the purpose that the model output will be used for. Simple mean annual flow assessments may utilise simple equations based on the size of the project and outcomes required.

Generally where the model results will be used for planning processes on larger scale catchments more detailed models are appropriate. As indicated earlier, conceptual lumped parameter models are generally the models of choice in Australia.

More complex models than these tend to be data intensive but should be considered when the results (generated flows or some other parameter such a water quality constituent) may be heavily biased if a simpler method is used to model rainfall-runoff.

The conceptual rainfall-runoff models can generally be calibrated to reproduce the daily observed streamflow well and the transfer of parameter values from a gauged catchment nearby can model runoff reasonably well in ungauged areas.

Current areas of research that are likely to further develop best practice in rainfall runoff modelling are:

• More advanced methods for regionalisation of rainfall runoff model parameters from gauged to ungauged catchments;

• Development of more sophisticated approaches for use of ensembles of results from several different rainfall runoff models to provide an optimum estimate of the flow time series;

• Development of more sophisticated and robust approaches for using parameter sets derived from multiple gauged (donor) catchments in ungauged catchments;

• Use of remotely sensed data sets, such as actual evapotranspiration, soil moisture and vegetation cover to inform the structure, parameter values and calibration of rainfall runoff models.

Page 44

eWater CRC

9 References

Arora, V. K., 2002. The use of the aridity index to assess climate change effect on annual runoff. Journal of Hydrology, 265, 164– 177.

Bárdossy, A. (2007) Calibration of hydrological model parameters for ungauged catchments, Hydrology and Earth System Sciences, 11 (2): 703-710

Black, D., Wallbrink, P., Jordan, P., Waters, D., Carroll, C., Blackmore, J. (2011). Guidelines for water management modelling: Towards best-practice model application. eWater Cooperative Research Centre. ISBN 978-1-921543-46-3.

Beven, K. (1989). Changing ideas in hydrology – the case of physically-based models. Journal of Hydrology, 105, pp 157-172.

Boughton, W. C. (2004). The Australian water balance model. Environmental Modelling and Software, 19, 943–956.

Boyle, D.P., Gupta, H.V. and Sorooshian, S. (2000) Toward improved calibration of hydrologic models: Combining the strengths of manual and automatic methods, Water Resources Research, 36 (12): 3663-3674.

Budyko, M. I. (1958). The Heat Balance of the Earth's Surface, translated from Russian by N. A. Stepanova, 259 pp. U.S. Dep. of Commerce., Washington.

Burnash, R. J. C., Ferral, R. L. and McGuire, R. A. (1973). A Generalised Streamflow Simulation System – Conceptual Modelling for Digital Computers. Joint Federal and State River Forecast Center, Sacramento, Technical Report, 204 pp.

Burnash, R.J.C. (1995). The NWS River Forecast System - Catchment Modeling, pp311-366 in V.P.Singh (ed) Computer Models of Watershed Hydrology, Water Resources Publications (Colarado, US).

Hydrology, 311-366.Chiew, F. H. S., Peel, M. C. and Western, A. W. (2002). Application and testing of the simple rainfall-runoff model SIMHYD. In: Singh, V. P. and Frevert, D. K., (eds.), Mathematical Models of Small Watershed Hydrology and Applications. Water Resources Publications, Littleton, USA, pp. 335–367.

Chiew, F.H.S. (2010). Lumped conceptual rainfall-runoff models and simple water balance methods: overview and applications in ungauged and data limited regions. Geography Compass, In Press.

CRCCH (2005a), General approaches to modelling and practical issues of model choice. Cooperative Research Centre on Catchment Hydrology, (Canberra). http://www.toolkit.net.au/tools/ModelChoice/MC-2.pdf

CRCCH (2005b), Water Quality Models: Sediments and Nutrients, Cooperative Research Centre on Catchment Hydrology, (Canberra). http://www.toolkit.net.au/tools/ModelChoice/MC-1.pdf


Croke, B. F. W., Andrews, F., Jakeman, A. J., Cuddy, S. M. and Luddy, A. (2006). IHACRES Classic Plus: A redesign of the IHACRES rainfall-runoff model. Environmental Modelling and Software, 21, 426–427.

Duan, Q. Y., Gupta, V. K. and Sorooshian, S. (1993). Shuffled complex evolution approach for effective and efficient global minimization. J. Optimiz. Theory Appl. 76(3), 501–521.

Duan, Q, H.V. Gupta, S. Sorooshian, A.N. Rousseau and R Turcotte (ed.) (2003). Calibration of Watershed Models., Water Science and Application No 6, American Geophysical Union, Washington D.C , USA.

Fu, B.P., 1981. On the calculation of the evaporation from land surface (in Chinese). Scientia Atmospherica Sinica, 5, 23– 31.

Gupta, H. V., S. Sorooshian, and P. O. Yapo, (1998) Toward improved calibration of hydrologic models: multiple and noncommensurable measures of information, Water. Resourc.Res., 34:751-763.

Jeffrey, S. J., Carter, J. O., Moodie, K. B. and Beswick, A. R. (2001). Using spatial interpolation to construct a comprehensive archive of Australian climate. Environmental Modelling and Software, 16, 309–330.

Jones, D. A., Wang, W. and Fawcett, R. (2009). High-quality spatial climate data-sets for Australia, Australian Meteorological and Oceanographic Journal 58:4, pp. 233-248.

Kavetski, D and Kuczera, G. and Franks, S.W. (2006a) Bayesian analysis of input uncertainty in hydrological modeling: 1. Theory, Water Resources Research, 42 (3): W03407.

Kavetski, D and Kuczera, G. and Franks, S.W. (2006b) Bayesian analysis of input uncertainty in hydrological modeling: 2. Application, Water Resources Research, 42 (3): W03408.

Kuczera, G, Kavetski D, Franks S and Thyer M (2006) Towards a Bayesian total error analysis of conceptual rainfall-runoff models: Characterising model error using storm-dependent parameters, J. Hydrology 331(1-2): 161-177.

Kavetski,D., S.W. Franks and G. Kuzera (2003) Confronting Input Uncertainty in Environmental Modelling. In ‘Calibration of Watershed Models’, Water Science and Application No 6, American Geophysical Union, Washington D.C , USA. pp49-68

Liang, X., Lettenmaier, D.P., Wood, E.F., and Burges, S.J., 1994. A Simple hydrologically Based Model of Land Surface Water and Energy Fluxes for GSMs, J. Geophys. Res., 99(D7), 14,415-14,428.

Madsen, H. (2000) Automatic calibration of a conceptual rainfall-runoff model using multiple objectives, J. Hydrology, 235 (3-4): 276-288.

Merz, R., and Bloschl, G. (2004). Regionalisation of catchment model parameters, Journal of Hydrology, 287, 95-123.

Page 46

eWater CRC

Nash, J.E. and Sutcliffe, J. V. (1970). River flow forecasting through conceptual models, I, A discussion of principles. J. Hydrol, 10, 282–290.

NIST and SEMATECH (2010). NIST/SEMATECH e-Handbook of Statistical Methods, http://www.itl.nist.gov/div898/handbook/, United States Department of Commerce, Last updated 23 June 2010, Accessed 7 July 2011.

Northcote, K. H. with Beckmann, G. G., Bettenay, E., Churchward, H. M., Van Dijk, D. C., Dimmock, G. M., Hubble, G. D., Isbell, R. F., McArthur, W. M., Murtha, G. G., Nicolls, K. D., Paton, T. R., Thompson, C. H., Webb, A. A. and Wright, M. J. (1960-1968). Atlas of Australian Soils, Sheets 1 to 10. With explanatory data (CSIRO Aust. and Melbourne University Press: Melbourne).

Oudin, L., Andreassian, V., Perrin, C., Michel, C. and Le Moine, N. (2008). Spatial proximity, physical similarity, regression and ungauged catchments: A comparison of regionalisation approaches based on 913 French catchments. Water Resources Research, 44, W03413, doi:10.1029/2007WR006240.

Parajka, J., Merz, R. and Bloschl, G. (2005). A comparison of regionalisation methods for catchment model parameters, Hydrology and Earth System Sciences, 9, 157-171.

Perrin, C., Michel, C. and Andreassian, V. (2003). Improvement of a parsimonious model for streamflow simulations. Journal of Hydrology, 279, 275–289.

Refsgaard and Henriksen (2004) Modelling guidelines--terminology and guiding principles, Advances in Water Resources 27, 71-82.

Rosenbrock, H. H. (1960). An automatic method for finding the greatest or least value of a function, Computer Journal, 3, 175–184.

Stenson, M. P, Littleboy, M. and Gilfedder, M. (2011). Estimation of water and salt generation from unregulated upland catchments. Environmental Modelling & Software, 26, 1268-1278.

Thyer, M.A., Renard, B., Kavetski, D., Kuczera, G. and Srikanthan, S. (2007) Bayesian Total Error Analysis For Hydrological Models: Preliminary Evaluation Using Multi-Site Catchment Rainfall Data, In Oxley, L. and Kulasiri, D. (eds) Proc. MODSIM 2007 International Congress on Modelling and Simulation. Modelling and Simulation Society of Australia and New Zealand, December 2007, pp. 74-80. ISBN : 978-0-9758400-4-7.

Van Dijk, A. I. J. M. (2010), AWRA Technical Report 3. Landscape Model (version 0.5) Technical Description, WIRADA / CSIRO Water for a Healthy Country Flagship, Canberra (http://www.clw.csiro.au/publications/waterforahealthycountry/2010/wfhc-aus-water-resources-assessment-system.pdf).

Vaze, J., Barnett, P., Beale, G.T.H., Dawes, W., Evans, R., Tuteja, N.K., Murphy, B., Geeves, G. and Miller, M. (2004). Modelling the effects of landuse change on water and salt delivery from a catchment affected by dryland salinity in south-east Australia, Hydrological Processes, Vol. 18, pp. 1613 - 1637.

http://www.itl.nist.gov/div898/handbook/


Page 48

Vaze, J., Post, D. A., Chiew, F. H. S., Perraud, J.-M., Viney, N., Teng, J., (2010) Climate nonstationarity - Validity of calibrated rainfall-runoff models for use in climate change studies, J. Hydrol. (2010), doi:10.1016/j.jhydrol.2010.09.018.

Vaze J, Chiew FHS, Perraud JM, Viney NR, Post DA, Teng J, Wang B, Lerat J, Goswami M. (2011a). Rainfall-runoff modelling across southeast Australia: datasets, models and results. Australian Journal of Water Resources, Vol 14, No 2, pp. 101-116.

Vaze, J., Post, D. A., Chiew, F. H. S., Perraud, J.-M., Teng, J., Viney, N. (2011b). Conceptual Rainfall–Runoff Model Performance with Different Spatial Rainfall Inputs, Journal of Hydrometeorology, Vol. 12, No. 5. pp 1100-1112, doi: 10.1175/2011JHM1340.1.

Vaze, J., Perraud, J-M., Teng, J., Chiew, F.H.S., Wang, B. (2011c). Estimating regional model parameters using spatial land cover information – implications for predictions in ungauged basins. 19th MODSIM Congress, Perth, Australia, 12-16 December 2011. http://mssanz.org.au/modsim11.

Vaze, J., Perraud, JM., Teng, J., Chiew, F. H. S., Wang, B., Yang, Z. (2011d). Catchment Water Yield Estimation Tools (CWYET). 34th IAHR World Congress, 27th June to 1st July, Brisbane, Australia.

Viney N, Vaze J, Chiew F, Perraud, J, Post D and Teng J. (2009). Comparison of multi-model and multi-donor ensembles for regionalisation of runoff generation using five lumped rainfall-runoff models in: 18th World IMACS Congress and MODSIM09 International Congress on Modelling and Simulation, July 2009, Cairns: Modelling and Simulation Society of Australian and New Zealand and International Association for Mathematics and Computers in Simulation: 3428-3434.

WMO (2008) Guide to Hydrological Practices. Volume I: Hydrology – From Measurement to Hydrological Information. WMO No. 168. 6th ed. World Meteorological Organisation, Geneva, Switzerland. ISBN 978-92-63-10168-6.

WMO (2009) Guide to Hydrological Practices. Volume II: Management of Water Resources and Application of Hydrological Practices. WMO No. 168. 6th ed. World Meteorological Organisation, Geneva, Switzerland. ISBN 978-92-63-10168-6.

Zhang, L., Hickel, K., Dawes, W.R., Chiew, F.H.S., Western, A.W., Briggs, P.R., 2004. A rational function approach for estimating mean annual evapotranspiration. Water Resources Research, 40(2), W02502.

Zhang, Y. Q. and Chiew, F. H. S. (2008). Can remote sensing data improve short-term rainfall-runoff simulation?, paper presented at Water Down Under 2008, April 2008, Adelaide: Engineers Australia.

Zhang, Y. and Chiew, F. H. S. (2009). Relative merits of different methods for runoff predictions in ungauged catchments. Water Resources Research, 45, W07412, doi:10.1029/2008WR007504.

http://mssanz.org.au/modsim11

guidelines for rainfall-runoff modelling: towards best ... · information on rainfall-runoff models...

Documents