guidelines for modelling water sharing rules in ewater source

Towards best practice model application

Black, D.C. and Podger, G.M.

Guidelines for modelling water sharing rules in eWater Source


ii

Copyright Notice

© 2012 eWater Ltd

Legal Information

This work is copyright. You are permitted to copy and reproduce the information, in an unaltered form, for

non-commercial use, provided you acknowledge the source as per the citation guide below. You must not

use the information for any other purpose or in any other manner unless you have obtained the prior

written consent of eWater Ltd.

While every precaution has been taken in the preparation of this document, the publisher and the authors

assume no responsibility for errors or omissions, or for damages resulting from the use of information

contained in this document. In no event shall the publisher and the author be liable for any loss of profit or

any other commercial damage caused or alleged to have been caused directly or indirectly by this

document.

Citing this document

Black, D.C. and Podger, G.M. (2012) Guidelines for modelling water sharing rules in eWater Source:

Towards best practice model application. eWater Cooperative Research Centre.

Publication date: July 2012

ISBN 978-1-921543-74-6

Acknowledgments

eWater CRC acknowledges and thanks all partners to the CRC and individuals who have contributed to

the research and development of this publication. In particular, the contributions of the following individuals

to the drafting of this document have been invaluable and are gratefully acknowledged: Andrew Close,

Paul Harding, Barry James and Chris Ribbons.

eWater CRC gratefully acknowledges the Australian Government’s financial contribution to this project

through its agencies, the Department of Industry, Innovation, Science, Research and Tertiary Education,

the Department of Sustainability, Environment, Water, Population and Communities, and the National

Water Commission

For more information:

UC Innovation Centre, Building 22

University Drive South

Bruce, ACT, 2617, Australia

T: 1300 5 WATER (1300 592 937)

T: +61 2 6201 5834 (outside Australia)

E: [email protected]

www.ewater.com.au


iii

Contents

1 Introduction ................................................................. 1

1.1 Background ........................................................................................................................... 1

1.2 Scope .................................................................................................................................... 2

2 Procedure for quality assured modelling of water sharing

rules ............................................................................ 3

2.1 Project administration ........................................................................................................... 3

Peer review ........................................................................................................................... 3

Stakeholder consultation....................................................................................................... 4

2.2 Problem definition ................................................................................................................. 5

Problem statement ................................................................................................................ 5

Objectives ............................................................................................................................. 6

Understanding problem domain ............................................................................................ 6

System definition................................................................................................................... 7

Conceptual models ............................................................................................................... 7

Metrics and criteria................................................................................................................ 8

Decision variables ................................................................................................................. 8

Uncertainty and risk .............................................................................................................. 9

Preliminary assessment ...................................................................................................... 10

2.3 Option modelling ................................................................................................................. 10

Methodology development .................................................................................................. 10

Gather and clean up data ................................................................................................... 10

Setting up and building a model ......................................................................................... 14

Calibrate model ................................................................................................................... 15

Validate model .................................................................................................................... 17

Sensitivity/uncertainty analysis ........................................................................................... 17

Develop, test and explore options....................................................................................... 19

Reporting and communication of scenario results .............................................................. 19

Model acceptance ............................................................................................................... 21

2.4 Compare options and select the “best” ............................................................................... 23

Option selection criteria ...................................................................................................... 23

3 References ................................................................ 24


1

1 Introduction

1.1 Background

Water sharing rules are frequently encapsulated in some form of water management plan, or

policy, which often has statutory status. Water sharing rules are developed for individual river

systems and this can occur for a variety of reasons. For example, the aims could be to

maintain or improve ecological functions, sustain the regional economy and protect the

social values and benefits of the river system. Fundamental to developing water sharing

rules in this situation is an understanding of environmental water needs, water entitlements

including their priority of access, basic landholder rights, allocation of water and operation of

water accounts. In regulated systems these rules are implemented in practice as operating

rules for dams, rules for water allocation, rules governing access to water and water

accounts, while in unregulated systems, implementation is via rules governing access to

water and water accounts. Rules governing access to water may be attached to licences.

Modelling of water sharing rules entails representing the water resource system, its water

users, infrastructure details, environmental assets and processes for implementation of

these rules.

Water sharing rules need to be developed considering the natural variability of climate and

stream flow and how these might constrain water availability. Access by water users is also

governed by water use accounting procedures that might limit annual use or the volumes of

water held in water accounts or carried over from year to year. Therefore, modelling the

assessment of water availability (ie resource assessment), trigger thresholds for restrictions,

and water use accounting procedures is integral to the modelling of water sharing rules.

At the same time, water use patterns may be influenced by expectations of future water

availability. For example in low water availability years water users might expect water

availability to improve in the near term (eg irrigators planting more irrigated crop than current

water availability can support on the expectation that availability will improve sufficiently

during the irrigation season to support the full crop). Also during high water availability years

water users might expect to get access to large volumes of off allocation flow, be able to

store that water and plant greater areas. This risk taking behaviour may be affected by

changes in, or introduction of, water sharing rules.

Modelling water sharing rules is often the main focus of water management modelling

studies, and the process of arriving at an agreed set of rules that can impact water users,

including the environment, is usually socially and politically sensitive. This sensitivity is a

major factor underlying Australian water management agencies’ needs for best practice

modelling guidelines.

To meet its responsibilities to its partners and to meet this need for guidelines, eWater is

developing a family of guidelines, comprising Guidelines for water management modelling

(Black et al, 2011) and a series of guidelines covering application of various aspects of

eWater Source. These guidelines are one member of this series. They provide guidance


2

directly relevant to modelling water sharing rules and support best practice application of

eWater Source for this purpose.

These guidelines complement eWater’s generic guidelines and should be used in

conjunction with them, as they are not intended to be a stand-alone document. They also

need to be used together with other guidelines in the series as needed. In common with the

other guidelines in this series, and the generic guidelines, these guidelines are mainly

intended for use by practising modellers with appropriate background; ie they are not a text

book.

It is emphasised that the procedure in the generic guidelines is intended to allow flexibility in

the way it is used, and this flexibility also extends to these guidelines.

1.2 Scope

These guidelines provide guidance relevant to modelling the practical implementation of

water sharing rules via river and dam operating rules and rules governing access to water.

They also provide guidance relevant to modelling the assessment of water availability (ie

resource assessment) and water use accounting procedures. However, the closely related

topic of modelling restriction trigger thresholds in storages (eg target curves) is covered in

the companion guidelines on storage modelling, and is outside the scope of these

guidelines.

eWater’s Guidelines for water management modelling (Black et al, 2011) describe a

procedure for quality assured model application which can be summarised as comprising

four phases:

1 Project management,

2 Problem definition,

3 Option modelling,

4 Compare options and select the most appropriate.

These guidelines deal mainly with Phases 2 and 3. From the point of view of modelling water

sharing rules, the generic guidelines provide sufficient information relevant to Phases 1 and

4, particularly as this activity is usually just one of several comprising a project as a whole,

and Phases 1 and 4 would be considered in more detail in the planning and undertaking of

the overall project. However, guidance on aspects of Phases 1 and 4 specifically relating to

modelling water sharing rules is provided, where the need has been identified, at appropriate

points in these guidelines.


3

2 Procedure for quality assured

modelling of water sharing rules

2.1 Project administration

The guidance on steps in the Project Administration phase given in eWater’s Guidelines for

water management modelling (Black et al, 2011) adequately covers the requirements in

relation to modelling water sharing rules. Where this modelling is seen to be sensitive, the

guidance provided should be read from this perspective. This applies particularly to peer

review and stakeholder consultation, where needs in relation to modelling water sharing

rules could determine the needs for the project as a whole. These two aspects are discussed

further below.

Peer review

From the point of view of modelling water sharing rules, peer review should occur at a

number of steps, including (but not limited to):

• When setting up and building the model; involvement of peer reviewers is strongly

recommended as, amongst other things, it will facilitate peer review following model

calibration, validation and sensitivity/uncertainty analysis.

• Following model calibration, validation and sensitivity/uncertainty analysis; it is most

important that peer review is undertaken at this stage for establishing model credibility

and fitness for purpose.

• During the process of exploring options; involvement of peer reviewers is

recommended to give added confidence in the credibility of model results, especially

where results could be contentious.

• During the process of selecting the preferred option, peer review may be required,

particularly if this involves use of analytical techniques not subject to peer review at

other steps (such as multi-criteria analysis; MCA).

There are a number of specific points peer review should address, as appropriate. These

include considerations such as the degree of aggregation of water users, approaches

adopted for disaggregating water usage and other data, and the approach adopted for

representing existing water sharing rules in the model, and are relevant for scenario

analyses as well as for model calibration and validation. These would be additional to

considerations such as overall model set up, choice of calibration metrics, the scope and

quality of calibration and validation, and choice of metrics and criteria for use in selecting the

preferred option.

All these considerations are also relevant to establishing the fitness for purpose of the

model, and peer review can therefore make a valuable contribution to this. A necessary


4

adjunct to establishing fitness for purpose is gaining stakeholder acceptance of the credibility

of the model and results it produces, and the importance of peer review in this context is

emphasised (ie the greatest value from the peer review process can be obtained if it is co-

ordinated within the framework of the stakeholder consultation process).

Stakeholder consultation

Stakeholder consultation is especially important when a project is socially or politically

sensitive, which is likely when modelling water sharing rules. Initiating consultation early in

the process will help allay possible fears of stakeholders and facilitate gaining their trust and

support for the modelling. It is important that stakeholder consultation occurs as follows:

• During the problem definition phase, particularly when developing the problem

statement, defining objectives and developing the conceptual model, and also when

deciding metrics and criteria, and decision variables.

• During the option modelling phase, particularly following model calibration, validation

and sensitivity/uncertainty analysis (and peer review) and during the development,

testing and exploring of options, but also when setting up and building the model.

• During the phase to select the preferred option, as stakeholder preferences could

have a major influence on the result of this phase.

Aims of stakeholder consultation should include developing confidence in the credibility of

the model and its impartiality (eg no hidden agendas in the model), the modelling approach

and the results with the objective of keeping technical aspects of the modelling separate

from politics. To achieve this there is a need for transparency: to be honest about the

limitations of model as well as its strengths, particularly about uncertainty, but without

unnecessarily undermining confidence in the modelling. This provides a path towards

gaining acceptance of the rules that are underpinned by the modelling. Hence, the necessity

for clear communication tailored to the audience, and for peer review.

However, while stakeholder engagement can have great benefits, the process of

engagement has to be carefully managed as it can be diverted by various interests. The

engagement process needs a strong educational component at the beginning and this has to

be continuing as stakeholder representatives may change during the process. Also, at

various stages there is a need for public engagement as sometimes information may not be

disseminated by the stakeholder representatives.

Workshops and other meetings with community representatives are potentially valuable

mechanisms for engaging with stakeholders. Walking stakeholders through the various

stages of the model setup, calibration and scenario running gives them greater

understanding of the model as well as an understanding of its accuracy and limitations.

These meetings may also be a useful source of data, as discussed in the section on Gather

and clean-up data below.

Discussions with stakeholders should emphasise that models are information tools

supporting water planners and only part of the process for arriving at a satisfactory solution.

Models provide the best information available at the time, but a process needs to be put in

place to allow model enhancements/improvements to better inform the process in the future.

Implementing water sharing rules should be an adaptive process.

Before any model is presented to stakeholders in the consultation process the model needs

to be stable and robust. Demonstrating and explaining baseline and scenario model results


5

to stakeholders may be difficult enough without having to explain that numbers are changing

due to model upgrades or bug fixes. The same stable model should preferably be used

throughout the whole consultation process to maintain stakeholder confidence in the model

and the predicted outcome. However, while the software should be as stable as possible, the

stakeholder consultation process could lead to the discovery of new information that either

warrants or necessitates improving or correcting the model. In this situation, the effects of

changes in the model on the results should be explained to stakeholders so they can clearly

see what they are.

While the model should be stable and robust, there will still be uncertainties in model results

and these should be acknowledged and explained in communications with stakeholders,

even if this might prove to be difficult to do. This applies particularly where socio-economic

impacts could be severe: convincing stakeholders of the need for changes in this type of

situation can be difficult when there is significant uncertainty in the model results. Even if

stakeholders are keen to know the uncertainty in the modelling results they may not know

how to deal with it. These difficulties can be minimised, if not overcome, by combining

uncertainty analysis with a risk assessment which shows what the consequences of the

uncertainty are expected to be. This is discussed further in the sections on Uncertainty and

Risk and on Sensitivity/Uncertainty Analysis, below.

During scenario modelling, regular meetings enable stakeholders to critique results and

propose new scenarios to be modelled. In the interests of achieving constructive outcomes,

the aim should be to keep discussions focussed on what the results are showing rather than

arguments about the credibility of the model and the results in themselves. It also needs to

be ensured stakeholders understand the time and effort required to produce results,

especially in circumstances where code changes or other major changes are involved, and

expectations of what can be delivered in a given time frame need to be managed.

If public meetings are held, needs for transparency, establishing the credibility of the

modelling, clear communication tailored to the audience and keeping the modelling apart

from the politics, become even more important. While presentation of information is very

important, finding ways to put the model results in a form that is easy to understand by the

non-specialist can be a big challenge and, if necessary, expert advice should be sought on

suitable approaches. Skilled communication is also needed, especially when communicating

with the public, and the task should be assigned to someone with the appropriate skills, not

necessarily the modeller.

2.2 Problem definition

Problem statement

In river systems where there are water sharing rules already in place, problems requiring

modelling input may arise due to reasons such as:

• development of a new water policy;

• adjusting an existing policy; or

• modifying the way an existing policy is implemented, perhaps because it is not

meeting agreed objectives.


6

The issues to be addressed usually revolve around questions of how to best manage these

systems within changing physical, social and economic constraints and the changing

demands of water users, including the environment.

Where new developments are being investigated, the problem could be a need to explore

options for water sharing rules to inform development of policies and procedures for

managing these new developments.

However, in either case, the specifics of the problem to be addressed from a modelling

perspective will almost certainly need to be discussed and confirmed with relevant

stakeholders, particularly the person or group commissioning the work.

Objectives

In existing regulated systems, and in unregulated systems where there are water sharing

rules already in place, the primary objective of a modelling study will very often be to define a

new set of water sharing rules or modifications to an existing set of water sharing rules.

Commonly, this will be driven by a new, or changed, water policy supporting an overall water

management objective, which might also be refined. However, this overall water

management objective will rarely be sufficient in itself as a modelling objective and, as a

general rule, it will be necessary to confirm the specific modelling objectives with relevant

stakeholders. The primary stakeholder will be the person or group commissioning the study.

However there may also be other interested or affected parties who will need to be consulted

during the process of defining the model objectives (eg river operators, water users,

environmental managers, etc).

When modelling new developments, modelling water sharing rules may not be the initial

objective and only a secondary consideration. Nevertheless, an appropriate objective or

range of objectives should be agreed with relevant stakeholders.

In either case, it might also be appropriate to consider possible secondary objectives. These

could include using the model to support a comparison of the performance of actual

operation of the rules with the expected performance, taking into account the effects of

climate variability during the assessment period. Other possibilities include using the model

to investigate potential climate change impacts and impacts due to other non-stationary

issues like changes in land use (farm dams etc), bushfires and groundwater.

In order to best define the modelling objective, an initial view of the form of the rules should

be developed. If there are existing rules, it may be appropriate to start from these and then

consider changes relative to the existing rules. The form of the rules should be linked to the

overall water management objective and cater for the needs of all water users, including the

needs of environmental assets, as appropriate. It also needs to be borne in mind that when

considering the form of the rules, this could also extend to water use accounting rules,

resource assessment procedures and trigger thresholds for restrictions, and possible

changes in these. This initial view will be the basis for developing modelling scenarios.

Understanding problem domain

This step involves identifying and agreeing with stakeholders, the range of disciplines that

needs to be considered to address the problem at hand. Potentially, the range of disciplines

relevant to modelling water sharing rules is quite wide.


7

For example, when there is a mix of consumptive and non-consumptive water users, and the

scope of work includes consideration of environmental flow rules as well as water supply for

human needs (which could include urban, hydro-power, industrial, agricultural and

aquaculture), it may be appropriate to consult specialists in various aspects of aquatic

ecology in addition to obtaining advice from specialists on supplying water for human needs.

Input from economists and specialists on social acceptance and adaptation may also be

needed if effective outcomes are to be obtained and understood.

A similar situation applies when modelling of water quality is involved, which may require

consideration of dilution flow rules, interception schemes or flushing flows. Expertise on

water quality criteria and receiving water dynamics may need to be consulted if effective

outcomes are to be obtained and understood.

When modelling an existing regulated system, input from dam operators is essential to

understanding the practicalities of implementing water sharing rules and representing the

rules realistically in the model. Input from operators responsible for collecting or maintaining

information on water usage and accounts may also be required. These aspects are

discussed further in the section on data gathering and clean-up.

System definition

From the point of view of modelling water sharing rules, there are a number of particular

aspects to consider in the system definition step in addition to the points in the generic

guidelines. These include:

• In addition to its biophysical features, the system could encompass water

management policies, which could be in strategies, plans or legislation; water usage

recording and accounting systems; operating rules and real time operating decisions

in existing regulated systems; sociological and behavioural aspects (eg water user

practices); and economics.

• How individual water users are to be represented; what degree of aggregation is

appropriate, having regard to data reliability and privacy issues, amongst others.

• How environmental water holders are to be represented: whether each holder of

environmental entitlements will be represented individually or be combined as a

group having regard to availability of long term environmental watering plans.

• Constraints on availability of data on water usage, water user behaviour and

practices, water use accounts, and operating rules and practices, in existing systems.

• Ambiguities and uncertainties in water management strategies, policies, plans and

legislation, which need to be clarified eg “by a process determined by the Minister”.

On occasion it may be necessary to seek legal advice to resolve these.

• Differences between water management rules and how these rules are

operationalised within the system.

Conceptual models

As water sharing rules typically comprise a number of components, it is important that these

components and how they interact are understood and agreed between stakeholders, in

conceptual terms at least. As discussed previously in relation to defining objectives, not only

do these include elements of water sharing rules themselves but they also include the


8

various elements of water use accounting rules, resource assessment procedures and

trigger thresholds for restrictions. Furthermore, as water sharing rules are rarely modelled in

isolation from other aspects of river system behaviour, it is important their conceptual model

is considered in the wider river system modelling context as well.

Metrics and criteria

The metrics and criteria adopted should be linked to the modelling objective, and should be

based on the initial view of the form of the rules developed when defining this objective. They

should also be linked to the overall water management objective and cater for the needs of

all water users, including the needs of environmental assets, as appropriate. Active

participation of stakeholders in deciding metrics and criteria should assist in gaining their

acceptance and support. Results from scenario modelling should be able to be compared

directly with the adopted metrics and criteria.

Many of the metrics used to demonstrate outcomes of the water sharing rules are presented

in terms of relative change (eg change relative to some base case). Given some of the

uncertainties (discussed in the section on Uncertainty and Risk, below) more confidence can

be placed in the relative difference in outcomes than the absolute numbers.

Relevant examples of metrics and criteria include reliability of water access, or supply, for

urban and irrigation water supplies. They also include performance requirements for

environmental flows such as the frequency and duration of inundation of wetland areas, and

durations of intervals between events.

When investigating changes in water sharing rules, relevant criteria might include allowable

changes in access to water by various categories of users, and deciding on these might

involve making trade-offs which some users are sensitive to. However, in some cases it may

not be possible to decide criteria such as these prior to modelling, and they may have to be

allowed to emerge during the process of exploring scenarios, which should enable the trade-

offs involved to be better understood.

When deciding on metrics and criteria, consideration should be given to the reliability of the

basis for these and sensitivity if they are not met to varying degrees. For example, if the

requirement is for a wetland to be inundated for 30 days every 5 years on average, but the

model results predict inundation occurring for only 29 days every 5 years on average, then

what are the consequences of this, and do they matter? However, if the duration is only 20

days, or 15 days, then how does the situation change? Similar considerations apply to urban

and irrigation water supplies.

Decision variables

Decision variables include any factors that stakeholders can adjust to influence the

performance of the system and, as such, are inputs to developing and modelling scenarios.

They are likely to be closely related to the metrics and criteria discussed in the preceding

section, which are evaluated after the model run. Agreeing on the values of decision

variables can be expected to involve making trade-offs, and the outcome could be expected

to affect the predicted performance of the system. Decision variables also impact on the

factors underlying the trade-off process, such as likely economic and amenity impacts due to

changes in reliability of urban and irrigation water supplies, and likely impacts on the health

of environmental assets due to changes in watering regime.


9

Initially, decision variables may not be able to be quantified except in a preliminary way.

Results from scenario modelling may provide the understanding needed to support making

the trade-offs necessary and enable realistic quantification (some may never be able to be

quantified and it may be inappropriate to try, but it may be possible to define them

qualitatively). If this process leads to an outcome that is not satisfactory then it may be

necessary to review the decision variables and try again.

Uncertainty and risk

Uncertainty

In the context of modelling water sharing rules there are a number of sources of uncertainty

that are particularly relevant. These include:

• Limitations in quality of water use data. This applies especially to data on irrigator

behaviour and their infrastructure, but may extend to data for other categories of

water user as well; noting that behaviour and infrastructure may change over time. It

also applies to data on water use and water accounts, as this normally has to be

disaggregated where it is available. These limitations have implications for the quality

of calibration achievable.

• Ability to represent rules, whether existing or proposed, in the model. In addition,

where there are existing rules, there may be differences between the rules as

specified and actual implementation practices, and differences in the ways rules are

implemented from time to time. In practice, operators may make decisions on

occasions which are difficult to describe in terms that are suitable for modelling and

also operational practices may be affected by maintenance of infrastructure at

various times, possibly for quite lengthy periods.

In addition to model and data related sources of uncertainty, there is uncertainty relating to

the rules themselves; for example, that environmental flow rules will deliver the

environmental outcomes sought; that changing the access to water, and perhaps reliability of

supply, for all water users will have economic and amenity impacts as predicted.

More detailed guidance on uncertainty analysis is provided in separate guidelines on this

subject (Lerat et al, in prep).

Risk

Note: the standard definition of risk in AS/NZS ISO31000:2009 is “the effect of uncertainty

on objectives”.

With respect to risk, it is important to consider both the likelihood and potential

consequences of successful implementation of new or changed rules and also of

unsuccessful implementation. This should include consideration of social impacts as well as

economic and environmental impacts. When evaluating scenarios and identifying the

preferred option, trade-offs may be necessary to achieve a reasonable balance between

social, economic and environmental impacts. The process in Phase 4 of the procedure in the

generic guidelines is designed to support making better informed trade-offs.


10

Preliminary assessment

The rationale for developing an initial view of the form of the required water sharing rules is

discussed earlier in the context of defining objectives. In addition, a preliminary assessment

of how these rules are expected to perform should be made, and this should be linked to the

agreed performance metrics and criteria.

2.3 Option modelling

Methodology development

Modelling water sharing rules is usually one activity within a wider project but, given this may

be the main reason the project is being undertaken, it may have a major influence on

decisions about methodology to be adopted for the project as a whole. However, this activity

cannot be considered in isolation and other factors, such as data constraints, may override

preferences for modelling water sharing rules. Where this occurs, compromises may have to

be made which will necessitate a less than desirable approach to modelling water sharing

rules. In this situation, it will be useful to document the implications of this for the expected

accuracy and reliability of results, if only in qualitative terms, so that stakeholders can clearly

see what these are. Assessment of uncertainty and sensitivity analysis (discussed in the

section on Sensitivity/uncertainty analysis) will be of help in this regard.

Gather and clean up data

The following discussion applies mainly to modelling water sharing rules in systems with

existing developments and rules. However, when modelling systems where there is currently

no development or water sharing rules, this discussion provides a potentially useful checklist

of points to consider for modelling water sharing rules.

Note that separate guidelines on data gathering and review in the context of Source are

proposed and these will give more comprehensive coverage of this topic.

Data types

Potential sources of data on water sharing rules include inter-jurisdictional agreements or

treaties, and water management strategies, plans, policies and legislation. These rules could

include the following, not necessarily independent, aspects (based on DERM, 2011):

• Water access rules and allocation policies;

• Flood control rules;

• Release rules: irrigation, industrial, urban, hydropower, environmental, recreational;

• Diversion rules;

• Water transfer rules;

• Security of water supply requirements for various categories of water users

(potentially, including the environment);

• Rules defining storage carryover volumes and reserves to be held;


11

• Allowances for transmission losses, such as evaporation and seepage losses, and

gains such as downstream tributary inflows;

• Restriction rules and supply capacity constraints;

• Rules for operation of fish passage devices; and

• Inter-jurisdictional agreements or treaties, and other strategies, plans, policies and

legislation.

Information on procedures used to implement water sharing rules in real time should also be

collected. In particular, information should be sought from dam operators as implementation

of rules in practice may differ at various times from the formal rules. Dam or system

operators are also valuable sources of information on how practical proposed changes in

rules, or new rules, are going to be operationalised.

Information on water user accounting systems, such as annual accounting, continuous

accounting, capacity sharing and continuous sharing is needed as accounting systems

affect, and are affected by, the rules. Information on release calculation methods is needed

as well.

For purposes of model calibration and validation particularly, data is needed on water use

volumes and patterns, how these are affected by water user decision making, and the

factors influencing these decisions. Relevant factors could include:

• Allocation announcements (where used);

• Allocation carryover rules and actual carryover volumes;

• Status of individual accounts and the rules for operating these accounts;

• Water ordering procedures;

• Whether off-allocation water is available or not;

• Temporary and permanent water trades;

• Water user licence conditions;

• Crop areas and crop types planted;

• Water user infrastructure;

• Antecedent conditions;

• Relationships between storage volume, seasonal forecasts, minimum inflows (where

used) and the announced allocation;

• Expectations (forecasts) of future water availability (amongst other things, these can

influence risk taking behaviour of water users), and

• Other potential drivers of risk taking behaviour of water users, including hydrological

characteristics such as seasonality and reliability of flow, level of security of supply

required, whether perennial or annual crop (in the case of irrigators).

Insight into how these might change when modelling scenarios is needed as well.

Other policies, plans, strategies and legislation not directly related to water management

may also have a bearing on water sharing rules, and these need to be sought out and their

implications understood. These could include national and social development plans, primary

industry policies, and biodiversity protection strategies.


12

Data sources

The main sources of data are usually the responsible water management and operating

agencies. Potential sources within these agencies include licensing databases, hydrographic

data databases, water use billing systems, records from water use accounting and water

ordering systems, plans and drawings of infrastructure, policy and other related documents

and files, policy staff, operating staff, and field staff such as metering inspectors and

advisers.

Other agencies and organisations may also be sources of data. These could include an

environmental management agency, which might have a compliance role and also have a

role in rule setting, such as for environmental flows or water quality management; this

agency may also be a source of data on aquatic environmental assets. Agriculture agencies

are another potential source of data, with respect to irrigation enterprises, as are local

government and water utilities, and agencies collecting national data (such as the Australian

Bureau of Statistics and the Australian Bureau of Meteorology).

Water users are valuable sources of information on their infrastructure and how they use it,

although some users may have privacy issues. Obtaining information on a confidential basis

may overcome these concerns, but this will depend on a level of trust being established with

the users. It may be easier for some water users to provide data if they can be assured they

will be grouped for modelling, and it will not be possible to identify individuals in the results.

Stakeholder consultation is one way of achieving the level of trust required (but not

necessarily the only way); individual meetings and questionnaires are among potentially

suitable techniques to use for obtaining the data.

Data quality issues

Some issues with data quality are:

• Practices associated with day-to-day operation of storages are often poorly

documented and can only be ascertained through direct communication with the

operators.

• River system models often require water usage and accounting data at smaller time

steps than has been collected. Therefore, the data has to be disaggregated, as

discussed below.

• Where water usage is metered, the data may be subject to large errors related to

meter type and age, and also due to variations in the head (water level) in the river.

For example, where diversion is by pumping, pumps become less efficient at low

heads (water levels) and, if metering is based on counting pump revolutions or

energy use, this can lead to usage being overestimated. Where sufficient data on

metering accuracy exists, water usage data should be adjusted accordingly in order

to ensure correct representation in the water balance. Where water usage is not

metered, surrogate data should be sought. For example, for irrigation water usage a

suitable surrogate may be crop area data together with a pro-rata allowance of water

per unit area of crop; while for urban water usage a per capita allowance may be

suitable, with adjustment for commercial and industrial purposes.

• There might be reliability issues with data obtained from water users, especially if it is

obtained by questionnaire, and it should be ground-truthed where possible. Remote

sensing can provide a useful means of independent verification of some data


13

supplied by water users, such as irrigated crop areas and on-farm storage

characteristics, although remote sensing techniques may fail to distinguish irrigated

crops from dryland crops and limitations in remote sensing imagery frequency and

resolution may constrain accuracy. Other data types, such as pump capacities, may

be able to be cross-checked with data held by the water agencies.

Data disaggregation approaches

Data sets relevant to modelling of water sharing rules which may require disaggregation

potentially include water usage data, water account data and water order data; used in

model calibration and validation. While these types of data might be collected on a

continuous basis (eg for major canal diversions), very often the data is collected at longer

intervals which may range from monthly, to 3 monthly and up to annual.

There are many techniques available for disaggregation. However, irrespective of the

technique chosen, it should be borne in mind that, as disaggregation typically involves

imposing an assumed pattern on the raw data to provide data at the required time interval,

model results may be sensitive to the disaggregated pattern adopted and this may affect the

quality of model calibration and validation achievable. If this is a concern then the sensitivity

could be checked by disaggregating using a different pattern and comparing results obtained

with the different patterns. Sensitivity to the disaggregation pattern can be a particular

problem when usage is driven by flow events (eg off allocation extraction into on farm

storages).

Care also needs to be taken to ensure the disaggregated pattern is not distorted by errors in

the data used to derive the pattern. For example, if residual flows between two gauging

stations are being used as the basis for deriving a daily pattern for water usage data

(recorded, say, monthly) then there are several potential sources of error that should be

considered. These include instrumentation errors, groundwater/surface water interactions (ie

transmission losses or gains) and residual catchment inflows.

One potentially useful approach to minimising the problem of errors is to calibrate the model

for pre-development conditions and use the results to evaluate unaccounted differences.

Allowances can then be made for these differences in the disaggregation methodology.

However, it needs to be recognised that the unaccounted differences may be subject to

change due to data non-stationarity issues, discussed below.

Data non-stationarity issues

Factors directly relevant to modelling water sharing rules (ie calibration and validation) which

can cause non-stationarity include:

• Pattern of diversions: eg growth in urban water demands, growth in areas developed

for irrigation, changes in farm infrastructure, such as growth in on farm storage

capacity which could especially impact on unregulated diversions, change in irrigation

efficiency, change in type of crop and season planted.

• Changes in farmer behaviour. Experience shows there are differences in farmers’ risk

behaviour after a drought compared to their behaviour after a series of wet years.

Planting behaviour is also liable to change when new varieties of a given crop are

introduced, in addition to changes due to a change in crop type.


14

• Changes in water accounting arrangements, and changes in water sharing rules

affecting dam release patterns.

• Consequent changes in transmission loss regime.

• Development / upgrading of major infrastructure such as dams.

• Geomorphological changes, such as changes in braided channel systems due to

floods, changes to constraints through structural works or land purchase, and

groundwater level changes affecting transmission losses and gains.

• Land management and use changes, including effects of forest growth, bushfires and

recovery from these, development of farm dams and other interception schemes in

catchments.

• Climate change, and medium term climate variability cycles (say, 30 years or longer

in duration).

• Instrumentation changes.

Growth in use, infrastructure and operational changes may be readily accommodated in

model calibration and validation, but adjusting for changes such as transmission losses may

require use of change detection techniques (Kundzewicz and Robson, 2004; Radziejewski

and Kundzewicz, 2004; WMO, 2009; Yue and Pilon, 2004).

Setting up and building a model

The important point to consider in this step is that the model should be able to simulate any

existing and past water sharing rules, needed for model calibration and validation, and also

be able to simulate anticipated scenarios for new rules, as far as these can be reasonably

foreseen. It should be possible to anticipate likely scenarios to at least some degree when

defining objectives and during the preliminary analysis step. However, it is acknowledged

that often unforeseen scenarios will emerge which, if they are to be analysed, will

unavoidably entail changes to the model set up and may possibly entail changes to the

model code, or new or changed plug-ins, as well.

The model structure needs to take into account requirements for modelling water users and

water use accounting systems as well as water sharing rules. Requirements for modelling

water users, such as how they are grouped, should come from the system definition and

methodology development steps but they may need to be revisited when setting up the

model. Other elements of water sharing rules, such as off-allocation systems, may have a

bearing on the approach adopted to grouping water users (here the term “off-allocation”

refers to access by water users to flood and other flows in excess of ordered water, and

deemed to be in excess of replenishment and environmental needs, which is additional to

their allocated water entitlement). In some cases, external factors, such as socio-political

considerations, may override technical considerations and a particular level of aggregation

which cannot be justified solely on technical grounds may have to be adopted.

In existing regulated systems, the model needs to adequately (realistically) represent current

operating practices, and operational characteristics (eg behaviour of re-regulating storages,

allowances for transmission losses, etc) and constraints (eg channel capacity); and also how

these might change in response to changes in water sharing rules. For modelling new

developments, potential operating constraints and how to represent these should be

considered. If the model is well set up then it should enable potential new rules to be

represented in a way that lends itself to practical implementation (and if proposed new rules


15

cannot be realistically represented in the model then this may raise questions as to whether

they can be implemented in practice).

Calibrate model

The extent of calibration needed or possible when modelling water sharing rules will depend

on whether there are rules already in existence or not; and if there are existing rules, to what

extent they are relevant to modelling proposed changes or new rules. Key points are:

• If the modelling software does not allow for aspects such as water sharing rules and

water supply infrastructure to vary within a model run, then calibration and validation

in the traditional sense (ie comparing modelled data against observed data for a

different period than used to calibrate the model) are possible only when rules have

been in place for some time; and this is typically rare.

• There may be differences between the way rules are operationalised and the way

they are specified in a policy or plan; in this situation, calibration and validation should

be based on operational practice. The rules as specified in a policy or plan could then

be modelled as a scenario and results compared, if desired.

• Data available for calibration and validation may not be representative of the range of

hydrologic and water availability conditions that could occur, which will limit the

quality of calibration and validation achievable, and the confidence that can be placed

in model results.

• Most of what is practicable relates to sensitivity analysis, rather than calibration and

validation, which is discussed in the section on Sensitivity/uncertainty analysis.

• Sensitivity analysis includes making checks to ensure rules in the model are

operating as intended; this applies whether calibration and validation are possible or

not, and applies to the modelling of new rules (scenarios) as well. The analysis

involves ensuring all aspects of the rules are activated which will necessitate pushing

the model to simulate both wet and dry extremes.

• When calibration is possible, it is likely to entail iterating between the Calibrate model,

Validate model and Sensitivity/uncertainty analysis steps.

Considerations relevant when calibration is possible are discussed below, mainly in the

context of calibration of the model as a whole.

Where there are no existing rules, or rules that are relevant to modelling proposed changes,

calibration for rules is not possible but the issues discussed below should be taken into

account when translating representations of rules in models to rules able to be applied in

practice.

When modelling a system with existing water sharing rules, and calibrating the modelling of

these is part of a larger calibration exercise for a river system model as a whole, it is likely to

take place late in the calibration process, after the calibration of physical characteristics such

as routing and the storage water balance (using observed historical inflow and release data).

Cumulative errors in the calibration results from these earlier steps may have an adverse

effect on the quality of calibration achievable for modelling water sharing rules. Adverse

effects can be minimised by adopting a staged calibration strategy, starting with calibrating

demands and water user behaviour using historical storage releases, operating

rules/practices and flow data, allocation/off-allocation data and water account data.


16

The next step would be storage behaviour calibration, with releases no longer forced, which

is the step where water sharing rule calibration typically occurs. Values of parameters used

to represent water sharing rules may need adjusting from initial estimates to enable storage

behaviour to be adequately reproduced.

To get the best results, advice should be sought from river operators and field operatives on

results obtained compared to actual storage behaviour and the reasons for, and significance

of, differences between them. If operating practices are significantly different from specified

rules then it may be necessary to calibrate using operating practices, and treat modelling of

the specified rules as a scenario, supported by sensitivity analysis as discussed in the

section on Sensitivity/uncertainty analysis below.

Other parameters requiring adjustment could include over order factors (used to represent

operator and water user behaviour: operators sometimes release more water than is strictly

necessary and water users sometimes inflate their orders as well, both aiming to ensure

there is no shortfall in deliveries), parameters used in representing re-regulating storage

behaviour and parameters used in representing water ordering, water allocations and off-

allocation periods (when water used is not debited from accounts). During storage behaviour

calibration, it may be convenient to initially force some of these to historical values

(particularly water allocations and off-allocation periods) and then progressively allow them

to be simulated, as a means of unpacking the influences of the various parameters on the

calibration. However, this approach will not avoid the necessity to revisit and fine tune the

calibration of other parameters initially calibrated while these ones were forced to historical

values.

Model responses should be evaluated for periods when there are no water orders as well as

for periods when there are water orders (DERM, 2011), where the presence or absence of

water orders could be a function of time of year (eg whether in the irrigation season or not) or

seasonal conditions (ie whether wet or dry), or a combination of these.

There may also be interaction with values of parameters representing aspects of water user

behaviour, such as risk taking. The approach adopted of aggregating water users may have

a bearing on the extent and nature of adjustments needed as well. Hence, further adjustment

of the demand calibration may prove to be necessary during storage calibration. However, it

is unlikely there will be a need to adjust values of parameters representing water use

accounting rules.

In unregulated systems, water sharing rule calibration will have to be undertaken at the same

time as demand calibration. Staged calibration is not possible as there are no regulating

storages involved.

When interpreting results it is especially important that modellers should:

• have a sound understanding of how the model is working;

• be able to identify any results that appear to be counter intuitive and understand

whether they are valid or not; and

• where counter intuitive results are valid, be able to explain and document the

reasons.

As with calibration and validation of all models, representative periods of record should be

chosen for water sharing rule modelling calibration and validation whenever possible.

However, when calibrating for water sharing rules, the choice is often constrained by data

availability limitations. In addition to data availability, points to consider include:


17

• Relevance as a baseline for modelling, which relates to the non-stationarity issue

discussed in the section on Gather and clean up data.

• Diversity in resource availability and total water usage

• Related to climatic variability, the model should be tested over periods with

varied water availability and water usage levels whenever possible.

• For example, during a wet period there may be plenty of water available but

water usage will be low if rainfall exceeds irrigation demand. High water usage

may occur early in a dry period, when there is still adequate water available, but

when resources become limited water usage will be low.

The goodness of fit between modelled and recorded data should be evaluated using a

number of comparative statistics (and definitely more than one), which could be the same as

usually used when calibrating other aspects of the model. Where the water sharing rule

calibration and validation is based on disaggregated data, the period used in the statistics

should be no shorter than the interval at which the data is collected. In other words, if water

usage data is collected at three monthly intervals, for example, then three monthly statistics

or statistics for a longer period (eg annual) should be used.

Validate model

As stated in the generic guidelines, it is important to recognise that validation is a test of

usefulness, not truth (CREM, 2008; Oreskes et al, 1994; Silberstein, 2006); ie “verification”,

which is a test of truth, is not logically possible.

Validating the modelling of water sharing rules using independent data is not often possible

or worthwhile, and validation will have to rely on sensitivity analysis. If the independent data

is for rules that are identical to the rules that applied during the calibration period, but the

hydrological conditions are different, then using this data for validation is likely to be

valuable. If the only independent data available is for a period when the rules are different

then this is unlikely to be helpful for validating the modelling of water sharing rules (although

it may well be useful for validating other aspects of the modelling) and validation may have to

rely on peer review and interaction with stakeholders to gain acceptance and assurance that

the calibrated model is working correctly. Peer review and interaction with stakeholders

should occur in any event.

Sensitivity/uncertainty analysis

As indicated in the section on Calibrate model, sensitivity analysis is essential for ensuring

water sharing rules, as represented in the model, are operating as intended. The important

aspect to check is how the rules as modelled perform under extreme conditions, especially

extreme dry conditions, but also under very wet conditions; whether modelled behaviour is in

accordance with expectations. Amongst other reasons, this can provide valuable insight into

how water management can be expected to work under these conditions; it also provides

insight into uncertainties in model results. Failing to check this behaviour can lead to

problems, particularly when modelling scenarios where the extremes may become more

common, such as when modelling climate change scenarios. Specific points include:

1 Rules as prescribed in relevant planning documents (such as water policies) and as

implemented in practice, and differences between them need to be captured; whether


18

both need to be modelled may be contingent on the extent of the differences and on

overall project objectives.

2 It should also be borne in mind there may be uncertainties in implementation in

practice due to factors such as scheduled and unscheduled maintenance of

infrastructure, and ad hoc operational decisions.

3 Performance should be evaluated in going from wet conditions to extended drought;

that is, performance as the system is modelled to run out of water needs to be

evaluated, even if an artificial input data time series has to be created to achieve this

(eg an extended period of zero inflows).

4 The priority order (hierarchy) with which supplies to different classes of water users

are restricted and then cease as the system dries off may need to be considered. For

example, the relative priority between general security and high security users is clear.

However, there may also need to be priorities set, and modelled, between different

classes of high security users (eg supply for critical human water needs is likely to

have higher priority than high security irrigation supplies).

5 Results may be sensitive to the distribution of resources between storages. For

example, where a system wide allocation is made considering resources in a number

of storages but supplies to part of the system are restricted by low levels in a particular

storage.

6 Results may be sensitive to changes in assumptions about initial conditions, especially

in storages, and performance for a range of starting conditions should be checked.

7 Alternative sequencing of extremes, especially dry extremes in the context of water

sharing, should be considered for a number of reasons, including:

a) Performance behaviour if there is a drier sequence than the worst on record

should be evaluated, if only to support planning for this eventuality. This might

include consideration of rights to water in dead storage even though storages

may not have reached such low levels historically.

b) Rules should not be fine tuned to a single sequence, as the future will almost

certainly be different.

c) Wet sequences should be evaluated to investigate whether there is a point

where the rules become inconsequential, and the potential consequences of

this.

d) Likely effects of seasonality changes can be investigated; particularly if the

system is at risk of changing from being perennial to ephemeral. Effects may

include changes in transmission losses, which might cause the loss allowance

to become inadequate, and changes in water delivery patterns for water users,

including the environment.

e) Security of supply criteria may be sensitive to factors such as the severity of

exceedance of a threshold (eg for water quality), durations of events (whether

high or low flow or water quality events) and intervals between events as well as

probabilities of occurrence of events, even if they are not couched in these

terms. For example, for a general security supply a requirement might be to

supply the full allocation in 60% of years, but the consequences for water users

if the supply fails in five consecutive years might be very different to the

consequences if the supply fails in five individual years, each separated by a

number of good years. In the case of a water quality threshold, exceeding the


19

threshold by 10% for a short period may be of little consequence, but if it is a

long term persisting exceedance the consequences might be severe; also the

intervals between short term exceedances might have a bearing on the nature

of the consequences.

f) Estimates of the probabilities of events will be more reliable if multiple

sequences are generated than if a single sequence is used, and the

uncertainties surrounding these estimates can also be evaluated. The

uncertainty results will provide information on the confidence, and confidence

intervals, associated with the probability estimates.

8 The above points provide the basis of a risk management approach to water

management.

9 When evaluating the results of these sensitivity analyses, the detail and intent of the

underlying planning document (or draft, as appropriate) should be checked for

consistency with those results; noting that planning documents often do not go into

details, especially in relation to management expectations under extreme conditions.

More detailed guidance on uncertainty analysis is provided in separate guidelines on this

subject (Lerat et al, in prep).

Develop, test and explore options

As indicated in the generic guidelines, this step should be undertaken in consultation with

stakeholders, as discussed in the section in these guidelines on Stakeholder interaction,

supported by appropriate levels of peer review as discussed in the section in these

guidelines on Peer review.

Scenario modelling should include evaluation of predicted performance under extreme

conditions as a matter of course, as discussed in the section on Sensitivity/uncertainty

analysis above.

Options investigated should be realistic, ie if adopted they can be implemented in practice.

Representation in the model needs to be realistic and options should not be too finely tuned

to the particular characteristics of the input data sets used, especially the time series data, in

terms of their modelled performance (sensitivity analysis should help avoid this). If realistic

representation of a scenario entails a change in the model then the calibration and validation

of the model as a whole should be checked; this is especially important if the model code is

changed.

Absolute values from scenario modelling results may have a high level of uncertainty but

results from one scenario relative to another, or to a baseline case, may be more reliable

and therefore meaningfully compared. Interpretation of results should be cognisant of this

and, where possible, quantitative analysis should be used to determine whether the model

has enough resolution to compare impacts of management scenarios. For example the

signal to noise ratio (Bormann, 2005) or comparison of confidence intervals could be used.

Reporting and communication of scenario results

A number of points relevant to reporting or otherwise communicating the results of scenario

analyses for water sharing rules are discussed in the section on Stakeholder Consultation,

above. More general points are discussed in the section on Report/communicate scenario

analyses in the generic guidelines.


20

The required outputs from scenario modelling will be application specific. Generic statistics

such as averages over the simulation period may be somewhat informative but, following on

from the points discussed in the section on Sensitivity/uncertainty analysis above, in many

cases a sufficient understanding of impacts or outcomes requires more detailed reporting of

model results.

For example, a useful way of presenting results for reliability of supply is by using a reliability

plot which shows water allocated as a percentage of entitlement versus percentage of water

years modelled this allocation is equalled or exceeded (eg see Figure 1). However, as this

plot uses ranked data, similar to a flow-duration curve, all information about sequencing is

lost and additional plots and tables are needed if information on sequencing is to be

provided. These might include time series plots and tables or charts of statistics of

exceedance events and intervals between events, amongst other things.

Figure 1: Sample plots for showing reliability of supply results

Much the same situation applies when exceedances of thresholds are of interest. Results

could be presented in the form of exceedance severity-frequency and duration plots (similar

to rainfall IFD plots), as illustrated in

Figure 2, with confidence limits added if required; box and whisker plots are another useful

means of presentation. However, as these also lose information about sequencing, they

would have to be supported by additional plots and tables or charts to provide this

information.


21

Sample curves for given threshold values Sample curves for given event durations

Figure 2: Sample plots for presenting threshold exceedance probability results

Further points to consider include:

• The hydrologic data sequence, or sequences, used as input to the modelling may

contain a number of predominantly wet spells and a number of predominantly dry

spells of various durations. If relative impacts vary markedly between wet and dry

spells the results for these spells should be reported separately in addition to the

overall results.

• Results of sensitivity analyses should be presented as well as results of scenario

analyses.

• Any apparently anomalous results should be presented and the reasons for them

explained. Apparent anomalies can occur for a variety of reasons when analysing

and comparing scenarios. For example the rules in one scenario might cause the

storage to be modelled as falling below a critical threshold or, conversely, to spill, at

one point in time whereas in another scenario this does not happen; but if the rules or

the storage behaviour are sensitive to this difference then the impacts may be

disproportionate.

Model acceptance

The generic guidelines state that model acceptance means gaining acceptance the model is

fit for purpose. Therefore, gaining model acceptance and understanding what the term “fit for

purpose” means in practice is essential for modelling water sharing rules and for minimising

the potential for controversy that may surround this activity.

Importantly, it may be the case that it is necessary to accept a model which is less than ideal

because limitations in data availability, or other constraints, prevent doing any better.

However, provided the model can be demonstrated to be useful within the constraints that

apply, and subject to caveats about accuracy, then it may still be seen to be fit for purpose.

This may be especially true if the model is demonstrably the best achievable under the

circumstances.


22

There are a number of criteria relevant to modelling water sharing rules that could be

assessed to determine whether a model is suitable for its intended purpose. Preferably the

metrics used should be quantitative, such as the comparative statistics for evaluating the

model calibration, but where this is not possible, qualitative measures should be used.

Directly assessing fitness for purpose for modelling water sharing rules will only be possible

when there are existing rules to test against. By using model results to evaluate how well

existing rules are operating as intended (discussed in the section on Sensitivity/uncertainty

analysis above) the fitness for purpose of the model can be established. As this involves

comparisons with policy or other water management documents, depending on the level of

detail in these documents, this assessment is likely to require use of qualitative measures

rather than quantitative metrics. When there are no existing rules to test against only indirect

assessment is possible. Sensitivity analysis results may assist in this situation, although

sensitivity itself is not an indication of model fitness.

Regardless of whether qualitative or quantitative metrics are used, they should not be used

in isolation to evaluate fitness for purpose (in isolation, failure to meet these may not

necessarily be an indication that the model is not fit for purpose, nor would success in

meeting them necessarily be an indication that the model is fit for purpose). The evaluation

should also consider criteria for assessing the fitness for purpose of the overall model. These

criteria include:

1 Does the model replicate historical data sufficiently well for important flow, storage or

water usage characteristics? The definition of sufficient needs to be considered with

respect to the model use and alternative sources of information. For example, if the

water sharing rules are highly sensitive to an absolute prediction then the highest

accuracy is required. A comparison of relative impacts between scenarios needs to

consider whether it is valid to compare expected values or if the prediction probability

density function needs to be compared. For the latter, it will be necessary to have

relatively low uncertainty (noise) compared to the signal due to management impact

being examined (Bormann, 2005).

2 What confidence can be placed on the model calibration, validation and

sensitivity/uncertainty analysis? Where these are based on a satisfactory length and

range of data (including extremes such as floods and extended droughts), and where

the data is of good quality, then the diagnostic results should be reliable.

3 Is prediction uncertainty evaluated, and are the methodology and results acceptable?

4 Where required for evaluation of management scenarios, does the model include

sufficient flexibility to be easily adjusted to simulate the required scenarios? Is the

required data and expertise appropriate?

5 Are the parameter values and internal fluxes physically realistic? Are they applicable

to scenarios anticipated to be modelled?

A possible approach to assessing fitness for purpose would be to express the performance

criteria as targets and use them to determine a performance class (where Class 1 is the

best). Each performance class would have reliability and accuracy caveats associated with

it, and the model would be assessed as being “fit for purpose subject to the caveats that

apply”, regardless of performance class (if the model did not meet the criteria for the lowest

performance class then this would be indication that the model is too unreliable to be useful

and therefore not fit for purpose). An example of such an approach has been adopted for

assessing the fitness for purpose of models under the Basin Salinity Management Strategy

for the Murray-Darling Basin (Murray-Darling Basin Commission, 2005; Appendix 2.5).


23

If a model fails on any one of the criteria then it would be downgraded to the next lower

performance class. However, specification of simple thresholds may be too strict, and it may

be appropriate to include a tolerance allowance. If one target is not met the model does not

get downgraded as long as the failure is not too severe but if more than one is not met then

the model is downgraded irrespective of the severity of the shortfall. What is meant by “not

too severe” would have to be defined. For example, the target value of the coefficient of

efficiency could be 0.8 or better, and the tolerance allowance could be 0.05. Where this

target is the only one not met, the model would not be downgraded unless the coefficient of

efficiency was less than, say, 0.75. The target could also be expressed in terms of a range of

values if required (eg for Class 1, the calibration period should not be less than 10 to 20

years).

This approach would have the advantage of accommodating the situation where data

availability limitations constrain the overall quality of the model achievable, while also making

it clear where the model is constrained and why. It also avoids the need to set criterion

weights and the consequent potential to disguise poor aspects of model performance.

2.4 Compare options and select the “best”

Information on this subject is provided in eWater’s Guidelines for water management

modelling (Black et al, 2011). Some additional points, relating to criteria on which the

selection of the preferred options could be based, are in the next section.

Option selection criteria

Many of the metrics and criteria relevant to modelling water sharing rules also have potential

application in a decision making process to select the preferred option. Quantitative

measures such as reliability of water access, or supply, criteria for urban and irrigation water

supplies, and criteria for environmental flows are potentially useful for defining objective

functions for optimisation. These measures may also be useful with MCA, either used

directly or converted into more qualitative measures (eg measures of environmental health –

good, fair or poor).

Ultimately, selecting the best or preferred water sharing rule option will come down to a trade

off process. There are very few water sharing rules that do not have some adverse impacts

on something or someone. The skill is to find a solution that all impacted parties are

prepared to accept and have implemented. Any implementation process needs to be

followed up with a monitoring and auditing process that can assess the benefits from the

water sharing rules and if needed fine tune the water sharing rules in the future.


24

3 References AS/NZS ISO31000:2009. Risk Management – Principles and Guidelines. Standards

Australia and Standards New Zealand. ISBN 0 7337 9289 8.

Black, D.C., Wallbrink, P.J., Jordan, P.W., Waters, D., Carroll, C., and Blackmore, J.M.

(2011). Guidelines for water management modelling: Towards best-practice

model application. eWater Cooperative Research Centre, Canberra, ACT.

September. ISBN 978-1-921543-46-3.

Bormann, H. (2005) Evaluation of hydrological models for scenario analyses: Signal-to-

noise-ratio between scenario effects and model uncertainty. Advances in

Geosciences, 5: 43–48. Available at: www.adv-geosci.net/5/43/2005/adgeo-5-

43-2005.pdf.

CREM (2008) Guidance on the Development, Evaluation and Application of Environmental

Models. Council for Regulatory Environmental Modeling, Office of the Science

Advisor, U.S. Environmental Protection Agency. Draft. August. Available at:

www.epa.gov/crem.

DERM (2011) River System Simulation Modelling in Queensland. Draft guidelines version 2,

April 2011. Queensland Department of Environment and Resource

Management.

Kundzewicz, Z.W. & Robson, A.J. (2004) Change detection in river flow records – Review of

methodology. Hydrological Sciences Journal, 49(1): 7–19. Available at:

www.iahs.info/hsj/hsjindex.htm.

Lerat, J., Peeters, L. and Shao, Q. (in prep) Uncertainty Analysis. eWater Cooperative

Research Centre.

Loucks, D P and van Beek, E (2005) Water resources systems planning and management:

an introduction to methods, models and applications. UNESCO, Paris and WL |

Delft Hydraulics, The Netherlands. 680 pp. ISBN 92-3-103998-9. Available at:

ecommons.library.cornell.edu/handle/1813/2799.

McMahon, T A and Adeloye, A J (2005) Water Resources Yield. Water Resources

Publications LLC, Highlands Ranch, Colorado, USA. ISBN 1-887201-38-6.

Murray-Darling Basin Commission (2005) Basin Salinity Management Strategy Operational

Protocols. Version 2.0. MDBC Publication No. 35/05. ISBN 1 9210386 67.

March. Available at:

www2.mdbc.gov.au/subs/dynamic_reports/bsms_op_protocols/HTML/index.htm

Oreskes, N., Shrader-Frechette, K. and Belitz, K. (1994) Verification, validation and

confirmation of numerical models in the earth sciences. Science, 263: 641–6. 4

February.

Radziejewski, M. & Kundzewicz, Z. W. (2004) Detectability of changes in hydrological

records. Hydrological Sciences Journal. 49(1): 39–51. Available at:

www.iahs.info/hsj/hsjindex.htm.

http://www.adv-geosci.net/5/43/2005/adgeo-5-43-2005.pdf

http://www.adv-geosci.net/5/43/2005/adgeo-5-43-2005.pdf

http://www.ewater.com.au/

http://www.iahs.info/hsj/hsjindex.htm

http://ecommons.library.cornell.edu/handle/1813/2799

http://www.ewater.com.au/



25

Silberstein, R.P. (2006) Hydrological models are so good, do we still need data?

Environmental Modelling and Software, 21(9): 1340–1352.

doi:10.1016/j.envsoft.2005.04.019.

Simonovic, S P (2009) Managing Water Resources: methods and tools for a systems

approach. UNESCO, Paris, and Earthscan, London. ISBN 978-92-3-104078-8.

Stedinger, J R and Taylor, M R (1982) Synthetic Streamflow Generation: 2. Effect of

Parameter Uncertainty. Water Resources Research, 18(4): 919-924, August.

doi:10.1029/WR018i004p00919.

WMO (2009) Guide to Hydrological Practices. Volume II: Management of Water Resources

and Application of Hydrological Practices. WMO No. 168. 6th ed. World

Meteorological Organisation, Geneva, Switzerland. ISBN 978-92-63-10168-6.

Yue, S. & Pilon, P. (2004) A comparison of the power of the t-test, Mann-Kendall and

bootstrap tests for trend-detection. Hydrological Sciences Journal. 49(1), 21–37.

Available at http://www.iahs.info/hsj/hsjindex.htm.


eWater Cooperative Research Centre eWater Limited ABN 47 115 422 903

UC Innovation Centre, Building 22

University Drive South Bruce, ACT, 2617, Australia

T: +61 2 6201 5168 [email protected]

www.ewater.com.au

© 2012 eWater Ltd

guidelines for modelling water sharing rules in ewater source

Environment

modelling water

option modelling

use of information

ewater sourcecopyright

national water commissionfor

problem definition

acknowledgmentsewater

noncommercial use