benefits assessment guidance - the national...

Benefits Assessment Guidance User Guide

Submitted to the Environment Agency for England and Wales

January2012

eftec 73-75 Mortimer Street London W1W 7SQ tel: 44(0)2075805383 fax: 44(0)2075805385 [email protected] www.eftec.co.uk

mailto:[email protected]

http://www.eftec.co.uk/

MBrierley

Typewritten Text

GEHO0312BWDZ-E-

Benefits Assessment Guidance – User Guide

eftec i January 2012

This document has been prepared by:

Economics for the Environment Consultancy Ltd (eftec)

73-75 Mortimer Street

London

W1W 7SQ

Study team (in alphabetical order):

Mr Lawrie Harper-Simmonds (eftec)

Dr Bruce Horton (eftec associate)

Ms Ece Ozdemiroglu (eftec)

Mr Allan Provins (eftec)

Acknowledgements

The study team would like to thank the members of the steering group have inputted to the

development of the User Guide: Polly Chancellor (Environment Agency), Harry Walton

(Environment Agency), Pauline Smith (Environment Agency), Stuart Sampson (Environment

Agency), David McGrath (Ofwat), Steve Moncaster (Anglian Water), Lesley Tait (Thames Water)

and Clair Dunlop (Yorkshire Water). Thanks also to Trevor Wade (Cascade Consulting) for

comments provide on the worked example. Apologies to any inadvertent omissions from this

list. As ever, any errors are the responsibility of the authors alone.

eftec offsets its carbon emissions through a biodiversity-friendly voluntary offset purchased from the WorldLand Trust (http://www.carbonbalanced.org) and only prints on 100% recycled paper.

http://www.carbonbalanced.org/


eftec January 2012 ii

CONTENTS

1. INTRODUCTION............................................................................... 1

1.1 PURPOSE OF THE USER GUIDE .............................................................. 1 1.2 SCOPE OF THE USER GUIDE ................................................................. 2 1.3 STRUCTURE OF GUIDANCE NOTE ............................................................ 3

2. OVERVIEW – VALUE TRANSFER ............................................................ 4

2.1 THE VALUE TRANSFER PROCESS ............................................................. 4 2.2 BASIC CONSIDERATIONS FOR THE USE OF VALUE TRANSFER .................................. 7

3. VALUING ENVIRONMENTAL AND SOCIAL IMPACTS OF WATER RESOURCE OPTIONS............................................................................................. 9

STEP 1 – REVIEW DECISION-MAKING CONTEXT ..............................................10 STEP 2 – DETERMINE ENVIRONMENTAL AND SOCIAL IMPACTS .............................11 STEP 3 – QUANTIFY ENVIRONMENTAL AND SOCIAL IMPACTS ...............................15 STEP 4 - SELECT VALUATION EVIDENCE ......................................................18 STEP 5 - ESTIMATE VALUE OF ENVIRONMENTAL AND SOCIAL IMPACTS ...................23 STEP 6 – AGGREGATION ........................................................................26 STEP 7 – SENSITIVITY ANALYSIS ...............................................................28 STEP 8 – REPORTING ............................................................................30

4. SUMMARY OF TASKS ...................................................................... 31

REFERENCES ...................................................................................... 35

GLOSSARY ........................................................................................ 37

ANNEX 1: WORKED EXAMPLE (SEPARATE DOCUMENT)


eftec January 2012 1

1. INTRODUCTION

1.1 Purpose of the user guide

Aim of the guidance

The purpose of this user guide is to clarify the process of options appraisal with respect to the

assessment of environmental and social costs and benefits in water resource planning in

England and Wales. It is intended to help water resource planners undertake a more complete

assessment of options using the existing Benefits Assessment Guidance (BAG) (Environment

Agency, 2003). The note supports both the Environment Agency (EA) and water companies in

fulfilling their statutory duties with regard to water resource management and planning.

The existing Benefits Assessment Guidance

The BAG was developed by the EA to support options appraisal for the 2004 Periodic Review of

prices limits (PR04). It comprises of a series of documents that set out a structured and

prescriptive approach for assessing and valuing a range of environmental and social impacts

associated with water resource and water quality schemes. Although it was designed for use by

both the EA and water companies, the BAG was primarily used by the EA to assess schemes

proposed under the National Environment Programme (NEP).

The water resource planning context

In the subsequent 2009 Periodic Review of price limits (PR09) the EA explicitly set out an

expectation that water companies should use the BAG to calculate average incremental social

costs (AISCs) of options as part of the analysis supporting their water resource management

plans (WRMPs). This led to an increase in the number of organisations and individuals that

sought to use and apply the BAG.

Many companies did not use BAG as a central part of their assessments at PR09. In part this was

because they used the results of their own customer preference surveys to assign values to

environmental and social impacts. In addition, companies tended to defer to requirements

established under the Strategic Environmental Assessment (SEA) Directive, which entailed

qualitative assessments of options that were consistent with SEA guidance (ODPM, 2005) and

impact categories. This note takes account of both of these issues. It also provides practical

guidance on overcoming other perceived limitations of the BAG, including the reliance on value

transfer (by reference to the latest relevant guidance in this area) and difficulties in

consistently specifying schemes and their impacts. Of course, some limitations remain and

therefore a more fundamental review of the BAG process is required in due course.



1.2 Scope of the user guide

This user guide does not replace the BAG. Instead, it is intended to be a linking document that

draws together the BAG and more recent value transfer guidance provided by Defra (2010),

which is explicitly focussed on the context of assessing environment and social costs and

benefits in water resource planning. It should be read and applied in conjunction with the

original BAG and other relevant methodologies and guidance, which are ‘signposted’ in Section

3.

The main areas covered by this user guide are:

Where and when to use the BAG and the level of detail needed for assessments;

Consideration of recently developed value transfer guidelines (Defra, 2010) to ensure that

application of the BAG meets good practice for value transfer;

Categorisation of environmental and social impacts associated with WRMP schemes;

Current UK Government guidance for valuing greenhouse gas emissions;

Relevant recent studies and results that might be appropriate for value transfer;

Updating the transfer values to a 2011 price base;

Current UK Government guidance with respect to discounting of future costs and benefits;

Focusing on most relevant parts of the BAG for water resources planning purposes.

Target audience

This user guide should be used by the EA and water company planners in conjunction with the

BAG documentation to develop options for water resource management planning. This is

expected to be principally in relation to the development of WRMPs and the Restoring

Sustainable Abstraction (RSA) programme. It is designed to complement other, broader

guidelines and methodologies, particularly:

The Water Resources Planning Guideline;

The Economics of Balancing Supply and Demand (EBSD); and

The Strategic Environmental Assessment Directive guidance.

Like the BAG, this user guide is aimed at non-economists tasked with the practical assessment

of environmental and social impacts of water resource schemes. It guides the user through the

assessment on a step-by-step basis.



1.3 Structure of guidance note

Following this introduction, the user guide is structured as follows:

Section 2: Overview of value transfer – this provides a non-technical outline of the basic

principles of the BAG and its use of value transfer, the practical steps for undertaking value

transfer, and some key considerations for those tasked with valuing environmental and

social impacts using this guidance.

Section 3: Valuing environmental and social impacts of water resource schemes – this

section provides the core step-by-step guidance. Along with consideration of issues related

to valuing the impacts of water resource planning schemes, it also ‘signposts’ where

further explanation or greater detail is required or available in existing documents, such as

the BAG (to avoid repetition).

Section 4: Summary of tasks – this is intended to provide a ‘quick reference’ guide for

users of the guidance, highlighting the key tasks in the step-by-step process for valuing

impacts of water resource schemes.

Glossary - definitions for key terms.

Annex 1: Worked example – this aids the exposition of the step-by-step guidance provided

in Section 3. The example presents a stylised representation of the impacts of a resource

development scheme (an effluent re-use scheme).



2. OVERVIEW – VALUE TRANSFER

2.1 The value transfer process

Background – basic principles of the BAG

The BAG is based on the principles of cost-benefit analysis (CBA), a ‘decision-making tool’ that

enables the consistent comparison of the costs and benefits of a proposed option in monetary

terms (Box 2.1). In the context of water resource planning, CBA enables:

1. An assessment of whether an option – either a resource development scheme, or demand

management option – is ‘worthwhile’, in terms of total benefits (e.g. positive

environmental and social impacts) outweighing total costs (e.g. CAPEX, OPEX and negative

environmental and social impacts); and

2. A prioritisation assessment / ranking of schemes to determine the optimal programme of

options, in terms of generating the greatest net benefit (i.e. total benefits of the

programme minus total costs).

In practice, options assessment in water resource planning is largely based on cost-

effectiveness analysis, rather than CBA. This centres on the estimation of average incremental

social costs (AISCs) and determination of a priority ranking of options1.

Box2.1: Are the basic principles of the BAG still relevant?

In short, yes. The principles that informed the development of the BAG and its methodological

approach still apply. For the most part the introductory text provided in the BAG – concerning

the use of CBA and the valuation of environmental costs and benefits [see BAG Part One,

Section 3] - still stands and provides an informative and useful account of key principles for the

interested reader. The principles that underlie the valuation of environmental and social costs

and benefits are well rehearsed, see for example: eftec (2006), Defra (2007), UK NEA (2011).

More caution is required in relation to the discussion of analytical issues in CBA (see BAG Part

One, Section 4). Here attention should be paid to newer studies and guidance (which are

highlighted in Section 3 of this note). A particular aspect to highlight is more recent

commentary on the valuation of biodiversity and non-use values, although it should be

recognised that this remains a challenging area for practical analyses.

1While the BAG is based on the principles of CBA there is no conflict with the cost-effectiveness basis of the WRMP process. In fact the requirement to calculate AISCs reflects CBA principles; particularly where they account for net environmental and social impacts (i.e. both positive and negative impacts).



Estimating environmental and social costs and benefits using value transfer

The approach to valuing environmental and social costs and benefits in the BAG relies on ‘value

transfer’ (note that in the BAG this is referred to as ‘benefits transfer’), which is the process of

estimating environmental and social impacts in monetary terms using readily available

information from existing studies (rather than undertaking new valuation studies).

Value transfer represents the most common way of monetising environmental and social

impacts across a broad range of public policy areas. Defra (2010) has provided ‘practical

guidelines’ for the use of value transfer, which are intended to support policy and project

appraisals across Central Government and Executive Agencies, such as the EA2. The steps in

these guidelines broadly reflect those in the BAG and are briefly described below.

Value transfer steps

The Defra guidelines set out an eight-step approach to value transfer, broken down into four

stages. This is illustrated in Figure 2.1, which highlights the series of tasks and inputs that are

required in the process of valuing environmental and social impacts of water resource planning

options. The process requires:

Stage One: an assessment of the decision-making context within which evidence on the

value of environmental and social costs and benefits is needed (e.g. WRMP, RSA), and

consideration of whether value transfer is an appropriate approach in this setting (Step 1).

This is covered in Part One of the BAG and elaborated in Section 1 of subsequent parts (e.g.

BAG Parts Two and Three).

Stage Two: identification of the baseline and a qualitative and quantitative assessment of

the environmental and social impacts of the water resource planning option, based on

environmental impact or strategic environmental assessments (Steps 2 & 3). In the BAG,

this is considered under each impact category.

Stage Three: appropriate selection and application of available evidence to estimate the

value of the identified environmental and social impacts of the water resource planning

option, including identification of key assumptions and corresponding sensitivity analysis

(Steps 4 to 7). In the BAG, this is again considered under each impact category.

Stage Four: reporting of results for decision-making, ensuring that all key assumptions and

limitations of the analysis are transparently documented (Step 8). Part Six of the BAG

covers this step.

In practice the value transfer process will not always follow a linear progression through the

eight steps. An iterative process can be taken through Steps 2-4, particularly in more detailed

assessments, where supporting data and information are refined as understanding of the

environmental and social impacts to be valued improves.

2Defra’s value transfer guidelines and supporting material are available here: http://www.defra.gov.uk/environment/natural/ecosystems-services/valuing-ecosystem-services/ [Accessed January 2012].

http://www.defra.gov.uk/environment/natural/ecosystems-services/valuing-ecosystem-services/



Figure 2.1: Value transfer steps

STEP 1 Review decision-making context

Key tasks

Is VT appropriate and possible.

What is the appropriate level of effort.

Resources

Defra VT guidelines [Step 1].

STEP 2 Determine

environmental and social impacts

STEP 3 Quantify

environmental and social impacts

Key tasks

Categorisation of impacts.

Define nature of impact and affected population.

Define the baseline conditions.

Describe the impacts qualitatively.

Assess the impacts quantitatively.

Resources

Defra VT guidelines [Step 2 and 3].

BAG Part One [Section 4.1] – baseline and time horizon of analysis.

BAG Part Two [Sections 2-9] and Three [Sections 2-6] – checklists and lookup tables for qualitative assessment; guidance on data sources, methods and tools for quantitative assessment.

Existing SEAs and EIAs.

Defra (2007) for principles of establishing impact pathways.

BAG appraisal summary table to develop Steps 2-7.

STEP 4 Select valuation evidence

STEP 5

Estimate value of impacts

STEP 6 Aggregation

STEP 7 Sensitivity analysis

Key tasks

Identify evidence that matches the impacts of interest.

Is the evidence of sufficiently high quality.

Which value transfer approach to use.

Calculate the transferred value.

Calculate the aggregated value of the impact.

Which key parameters affect the transferred value the most.

Resources

Defra VT guidelines [Steps 4 – 7].

DECC guidance for GHG emissions.

Defra/IGCB for emissions of air pollutants.

BAG Corrigenda [Section 1&2] – recreation values.

BAG Part One [Section 3.2] – adjustments to values.

BAG Part One [Section 10] – estimation of aggregate values.

HM Treasury Green Book (2003) – discount rates.

BAG Part Six [Sections 2-3] – calculation of NPV.

Key tasks

Complete appraisal summary table.

Summarise sensitivity analysis.

Summarise key parameters and caveats.

Resources

BAG Part Six – guidance on bringing together the AST and presenting the aggregated results for decision making.

STEP 8

Reporting



2.2 Basic considerations for the use of value transfer

To ensure ease of application by non-economists and consistency across different assessments,

the BAG guides the user through the value transfer process in a prescriptive and ‘mechanical’

manner. However, coupled with the broad scope of the BAG (including guidance on valuing

water quality improvements in both rivers and coastal areas), this makes for a large and

unwieldy set of documents for the user to navigate in order to find guidance relevant to water

resources planning.

In addition, the application of value transfer can be challenging, particularly given information

gaps and the consequent judgements and assumptions that are then needed. While water

resource planners may be comfortable in making assumptions concerning the operation of

schemes, they are typically less familiar with the finer details of economic analysis.

With these points in mind, it is useful to outline some basic considerations for those tasked

with valuing the environmental and social impacts of water resource planning options:

Determine the level of effort that is required in the analysis: As a general rule, more

scrutiny in decision-making will be placed on impacts that are judged to be significant

and/or valued highly in either absolute or relative terms. This is particularly the case

where the assessment of these impacts is sensitive to specific assumptions and judgements

that may not be agreed across stakeholders. More effort (e.g. more detailed assessments,

more sensitivity testing) will be required in these instances.

Recognise the limitations of value transfer: Users should approach assessments with a

view to ‘making the best’ of the information and data that is available. Fundamentally the

BAG is limited by its use of value transfer, where the selection and use of valuation

evidence from existing studies is reliant on: (i) the coverage and quality of studies that are

actually available and relevant; and (ii) the judgement and assumptions that are used to

apply the evidence in the analysis. The BAG guidance states that “The results of any

assessment can be assumed to provide only rough indicators of the benefits or disbenefits

[costs] that will be delivered by a single scheme” [BAG Part One, Section 1]. Overall, if

having completed an initial assessment of impacts using the BAG there is a view among

stakeholders that a more detailed analysis is required, then the initial assessment has

‘done its job’ and further effort should be expended or work commissioned (e.g. in-depth

environmental assessment; or potentially primary valuation work).

Assessments and the valuation of impacts should be transparent: even with explicit

recognition of the limitations of the analysis, there remains a risk that the credibility of

water resource planning decisions can be questioned, particularly if assumptions are made

that are not agreed across stakeholders, or where valuations of environmental and social

impacts are key to the options that are chosen. The practical response is to ensure that all

analyses are transparently reported and an appropriate amount of sensitivity analysis is

undertaken. While the BAG is a contribution to the range of information and evidence used

to make water resource planning decisions, the scrutiny of assessments based on it should

not be disproportionate to other elements that also contribute to the process.



Dealing with data gaps and uncertainty concerning environmental and social impacts:

all options appraisal is subject to data gaps and uncertainty. The key requirements for

planners are to: (i) make sure any gaps and assumptions are clearly highlighted; (ii) aim to

complete at least a qualitative assessment where impacts are expected to be significant;

and (iii) undertake appropriate sensitivity analysis on the most significant impacts and key

assumptions that could be open to challenge.

Above all, where significant challenges are faced – for example difficulty in quantifying and

valuing particular impacts – there is always the option to consult with wider stakeholders.

Advice can be sought from other water resource planning experts and, if assessments are likely

to be subject to significant scrutiny (e.g. a major resource development scheme such as a

reservoir), it may be appropriate to consult EA economists for guidance (for example to help

determine if value transfer evidence will be considered to be sufficiently robust for decision-

making). This will help ensure that expectations as to the level of analysis that is required are

explicitly recognised and appropriately managed.



3. VALUING ENVIRONMENTAL AND SOCIAL IMPACTS OF

WATER RESOURCE OPTIONS

This section is framed around the eight value transfer steps set out in Figure 2.1. The content

is focused on the task of assessing and valuing environmental and social impacts associated

with water resource planning options:

Where the original BAG is still applicable, the guidance is not repeated, but rather

‘signposted’, and users are referred to the relevant BAG document or section. For water

resource schemes, the relevant parts of the BAG are:

o Part One: Introduction o Part Two: Assessment of River and Groundwater Schemes

o Part Three: Assessment of Schemes Affecting Reservoirs, Lakes and Broads

o Part Six: Bringing the Results Together

o Corrigenda (note that this updates the approach to informal recreation and non-use

values in the BAG, primarily replacing Part Two Sections 2 and 9, respectively)

All of these documents are available on the EA’s website:

http://www.environment-agency.gov.uk/business/sectors/37305.aspx

[Accessed January 2012]

Where there is more recent guidance that should replace the original BAG, this is

highlighted and relevant instructions are provided. A number of references are made to

Defra’s Value Transfer Guidelines, which are available with supporting material here:

http://www.defra.gov.uk/environment/natural/ecosystems-services/valuing-ecosystem-

services/ [Accessed January 2012]

Other sources of information are referenced within Steps 1-8.

Where relevant, practical guidance is provided that is explicit to the challenges faced in

using value transfer in the context of water resource planning.

Shaded boxes are included to show the key considerations in each step. These should be

considered as particularly important aspects of the assessment.

The step-by-step guide is accompanied by worked example (see Annex 1) which provides a

practical demonstration of the use of the BAG.






STEP 1 – REVIEW DECISION-MAKING CONTEXT

The purpose of this step is to:

Judge if the BAG and value transfer approach is appropriate, given the water resource

planning decision to be made; and

Judge the level of effort required in the assessment of environmental and social impacts.

Key considerations

The Defra Value Transfer Guidelines provide further discussion on determining if value

transfer is appropriate and the degree of uncertainty in evidence that can be

accommodated in decision-making [Step 1: ‘Establish the policy good decision-context].

For both water companies and the EA, prior to undertaking any significant assessment effort it

is worth considering whether, and the extent to which, environmental and social impacts

should be assessed using value transfer (i.e. whether to move to Step 2). Questions include:

Has the SEA or other assessment highlighted significant environmental and social effects?

Are these effects amenable to quantification and value transfer, or would another

approach to gather evidence (e.g. a primary valuation study) be preferable and feasible?

Apart from valuation, what other inputs (e.g. scientific and technical and/or deliberative

and participatory) are likely to be important in the decision-making process?

Box 3.1 offers some general commentary on circumstances where use of the BAG may or may

not be appropriate.

Box 3.1: In what circumstances is the BAG approach appropriate?

In practice decision-making is never based on ‘perfect information’ - the BAG guidance notes

that assessments should be seen as ‘rough indicators’ - and a certain degree of uncertainty has

to be accommodated in the evidence that informs decisions.

In many cases the BAG approach will be sufficient for the valuation of environmental and social

impacts in relation to water resource planning. This is because:

It represents a consistent and transparent framework for assessing impacts;

It enables explicit identification of the most significant impacts in qualitative and

quantitative terms; and

Resources are not available to support the primary valuation of impacts.

The BAG is therefore suited best to a large scale ‘screening’ of options to develop plans (i.e.

options ranking/prioritisation). Where evidence is need to support decisions concerning

schemes with (financial, environmental and social) impacts that are expected to be significant,

or where there will be a high level of scrutiny of assessments (e.g. a contentious resource

development scheme), then there is a need to consider commissioning more detailed

assessments (both quantitative assessments and primary valuation) if time and resources allow.



STEP 2 – DETERMINE ENVIRONMENTAL AND SOCIAL IMPACTS


Identify the environmental and social impacts of the water resource option being assessed.

Identify the likely scale and significance of the impacts, both in terms of environmental

effects and the affected population.

Identifying environmental and social impacts

The BAG appraisal summary table (AST) format should be used to develop the assessment

of environmental and social impacts through Steps 2 – 7. [BAG Part Two, Section 1.3, Table

1.1].

In practice, the basis for establishing the range of impacts from a scheme will be an SEA or

similar assessment (e.g. environmental impact assessment, EIA). A challenge is that the outputs

of SEAs and other assessments of this type are ordinarily not intended to inform the

quantitative and value transfer assessments prescribed by the BAG. As a consequence:

Some work may be required to ‘link’ or ‘translate’ the output of an SEA into the BAG

assessment format (the importance of this task should not be under-estimated); and

This may require various assumptions to be made about the scale and significance of

impacts, particularly in view of location-specific issues (e.g. the environmental baseline).

Assumptions made in the task of developing BAG assessments from SEA outputs should be

documented and subject to sensitivity analysis (see Step 3 for further detail).

Categorisation of impacts

The primary ‘tool’ for developing assessments in the BAG is the appraisal summary table (AST).

This is intended to provide a transparent summary of analysis undertaken, structured in terms

of ‘impact categories’. The BAG impact categories typically differ from those assessed in an

SEA or an EIA (although there is also some commonality and overlap), or now included in

‘ecosystem services approach’ assessments (see Box 3.2).

The alignment of impact categories is largely a presentational issue and users should tailor the

AST to account for the impacts of relevance. The task is to ensure that an assessment identifies

the full range of impacts and takes account of these. Importantly though, the basis for

categorising impacts should be transparent, informed by technical assessments, and avoid

potential for double counting in the quantification and valuation of impacts. A view is also

required within the classification of impacts as to how this links to valuation evidence that can

be applied (in Steps 4 and 5).



Table 3.1 lists the type of water resource options that are typically considered in WRMPs along

with the impact categories in both the BAG and the SEA Directive3.

Table 3.1: Water resource options and BA/SEA impact categories

Water resource option BAG impact categories SEA Directive impact

categories

Resource development

Reservoirs

Water transfer

Artificial recharge

Effluent re-use

Aquifer storage and

recovery

Desalination

Groundwater abstraction

Surface water abstraction

New connections

Conservation value /

biodiversity

Heritage,

archaeology,

landscape

Informal recreation

Amenity

Boating / watersports

Canoeing

Swimming / bathing

Coarse / sea angling

Game angling (trout

and salmon)

Commercial fisheries

Shellfisheries

Impacts on abstractors

Land take

Odour

Noise

Global warming

potential

Biodiversity

Human health

Fauna & flora

Soil, water, air

Climate

Material assets

(e.g. housing)

Cultural and

archaeological

heritage

Landscape

Demand management

Leakage reduction

Pressure management

Water butts

Cistern devices

Metering

Water efficiency

Greywater recycling

Rainwater harvesting

Overall, it is important to consider which BAG impact categories are likely to be important for

the purposes of the option in question. This should include consideration of whether the BAG

categories miss any impacts that could be significant. For example, some of the positive

impacts of demand management options (e.g. water saving) are largely outside the scope of

the BAG impact categories, but should be considered within an assessment if significant (e.g.

enough to have beneficial environmental effects).

3SEA Directive Annex I (f) and footnote.



Box 3.2: What is the ecosystem services approach?

The past decade has seen the development of the ‘ecosystem services approach’ (ESA). The

increasing interest in the ESA – as demonstrated by the UK National Ecosystem Assessment (UK

NEA, 2011) - reflects wider recognition of multiple objectives, trade-offs and synergies that

exist in the realm of ‘natural environment’ policy (e.g. environmental protection, land-use

management, sustainable development). Its development has been the focus of a concerted

research effort and policy analyses that, like the BAG, highlight and attempt to measure the

contribution of ecosystems and the biological diversity contained within them to individual and

social wellbeing.

The UN Millennium Ecosystem Assessment (MEA, 2003; 2005) established the four main

categories of ecosystem services: provisioning services, regulating services, cultural services

and supporting services. Assessments of environmental and social impacts of water resource

options within an ESA require the same basic principles, inputs and tasks that are set out in the

BAG (i.e. identifying impacts, qualitative and quantitative assessments, identifying the

affected population, monetary valuation of impacts). In this sense, the ESA represents an

evolution rather than a revolution.

Defra (2007) ‘An Introductory Guide to Valuing Ecosystem Services’ provides a useful overview

of the ESA. Available here: http://www.defra.gov.uk/environment/natural/ecosystems-

services/valuing-ecosystem-services/ [Accessed January 2012]

The Defra value transfer guidelines also provide further discussion of the practical use of the

ESA for appraisal of costs and benefits [Step 2: ‘Define the policy good and affected

population’].

Key considerations

The Defra Value Transfer Guidelines provide a general overview of the main

considerations to address in establishing impacts to be valued and the affected population

[Step 2: ‘Define the policy good and affected population’].

Once the relevant impact categories have been defined, a more detailed examination of these

impacts is required in advance of their quantification (in Step 3). The purpose here is to

establish the key considerations that will need to be reflected in the analysis:

The nature of the impact

o Whether it is a ‘market’ or ‘non-market’ impact and the type of use and/or non-

use value derived [see BAG Part One, Sections 2.2 and 2.3];

o The location of the impact, the scale over which it occurs and the timing (e.g. due

to construction or operation).





The affected population:

o What user population is affected - Part One of the BAG [Section 3.4] includes a

useful discussion on defining the affected population for use values.

o Is a non-use population likely to be affected – the BAG provides an explicit

methodology for accounting for non-use values [see Part One, Section 3.3].

o The socio-economic characteristics of the affected user and likely non-user

populations.

Within these considerations it is also useful to assess where sensitivity analysis will be needed

to assess data gaps and information uncertainty. For example, initial assessments of

environmental effects of options may be ‘desk-based’ exercises that draw on more general

scheme descriptions and may not have detailed accounts of likely site specific effects. While

this assessment can be refined by more detailed EIA work, a practical response is to consider

the likely ‘worst’ and ‘best’ case outcomes as part of sensitivity testing. This will provide a

guide as to whether further technical studies are needed.



STEP 3 – QUANTIFY ENVIRONMENTAL AND SOCIAL IMPACTS


Determine and measure the baseline against which environmental and social impacts

should be assessed.

Produce a qualitative summary of the environmental and social impacts of the water

resource option being assessed.

Provide a quantitative assessment of the environmental and social impacts.

Assessing environmental and social impacts

In an economic assessment (i.e. based on the principles of CBA) the costs and benefits arising

from an option are measured relative to a baseline, which accounts for the outcomes that

would have occurred without the option. Establishing the baseline (and time horizon of the

option) is therefore a necessary starting point for the assessment.

Refer to BAG Part One [Section 4.1] for guidance on defining the baseline and time horizon

for the analysis.

In assessing the environmental and social impacts of a water resource option, the following is

required:

1. Define the baseline conditions (without the impact(s)) over which the impact will be

quantified:

o The economic baseline (e.g. the numbers of users)

o The environmental baseline (e.g. river flow)

2. Describe the impacts qualitatively in terms of:

o Scale - the change compared to the baseline.

o Nature - a change in ‘quality’ (e.g. biological river water quality indicator) or

‘quantity’ (e.g. river flow).

o Direction – an improvement or deterioration.

o Timing - immediate, gradual, limited period, in perpetuity.

o Location - in relation to all location/spatial effects of the impact.

BAG Part Two [Sections 2-9] and Three [Sections 2-6] include a number of checklists and

‘look-up’ tables to help the user identify and record the qualitative impact of an option

systematically.

Within the BAG impact categorisation (see Step 2), the impacts most likely to be relevant

to the construction and operation of water resource planning options include:

o Amenity

o Biodiversity

o Global warming potential (GHG emissions)



o Landscape

o Noise (which has also been taken to incorporate other forms of disturbance,

particularly from construction activities and streetworks)

o Recreation (i.e. boating/watersports, canoeing, swimming/bathing and angling)

3. Assess the impact(s) quantitatively in terms of:

o Units of change for ‘quantity’ impacts (e.g. reduction in tonnes of emissions,

reduction in the size of a habitat) and/or parameters for ‘quality’ impacts (e.g.

change in the biological oxygen demand (BOD)).

o The risk of a particular change occurring (e.g. flood risk).

o The change in the affected population (e.g. reduction in visitor numbers).

BAG Part Two [Sections 2-9] and Three [Sections 2-6] provide more detailed guidance on

data sources, methods and tools for developing quantitative information, for each impact

category, including the estimation of the affected population.

UKWIR provide guidance on quantifying emissions of operational and embodied carbon:

http://www.ukwir.org/site/web/news/carbon-accounting [Accessed January 2012]

A challenge can be faced where the quantification of environmental impacts does not

‘match’ the units that monetary valuations are presented in. For example, improvements

in BOD are not valued in their own right, but rather for the effect on the availability of fish

populations. This could attract use values through recreational fishing and angling, and

non-use values through existence, bequest or altruistic motives. In these cases it is

important to explicitly establish the ‘impact pathway’ through which changes in

environmental quality impact on use and non-use values. In some cases proxy measures

may be available (e.g. measuring biodiversity impacts in terms of the area of habitat

gain/loss).

The basic principles for establishing impact pathways are summarised in Defra (2007) An

Introductory Guide to Valuing Ecosystem Services (see Chapter 3).

4. Provide supporting data to help select the relevant economic valuation evidence:

Note this is unlikely to be required in ‘basic’ applications of the BAG. For more detailed

value transfer analyses (see Step 4 and 5) it may be necessary to collect supporting data to

aid the selection of appropriate valuation evidence and its practical application. In

particular, supporting data may be used to provide more accurate estimates of the relevant

population, or to adjust the transfer values themselves. Such data can include:

o Socio-economic and demographic characteristics of the affected population:

specific requirements will vary case to case but could include: household income

or GDP per capita; socio-economic group; education; occupation status; age

profile; household size; number of dependents.

o Patterns and frequency of use (e.g. number of visits to a site).

o Availability of substitute sites (e.g. alternative locations for angling)

Note that supporting data will often have to be derived from external sources, such as the

Census, visitor surveys, etc. In addition, collecting supporting data is a task that may need

http://www.ukwir.org/site/web/news/carbon-accounting



to be re-visited as the analysis develops (as noted in Section 2.1 there can be an iterative

process between Steps 2-4 in more detailed assessments).

The BAG provides some guidance on how supporting data might be derived and used as

part of the quantification of impacts [see for example Part Two, Section 2.3].

Practical illustrations of the range of data that can required in value transfer exercises

are available in the case studies that support the Defra Value Transfer Guidelines [see

value transfer case studies: http://www.defra.gov.uk/environment/natural/ecosystems-

services/research-and-case-studies/ [Accessed January 2012]

5. Identify the key parameters to be tested in sensitivity analysis:

These will vary from case-to-case and by different types of impact, but should be identified

as part of the process of working through 1-4 above. Depending on data gaps and

assumptions that are required, sensitivities to test may include (see also Step 7):

o Ranges – lower and upper bounds in addition to mid-point estimates for

quantitative data and relationships.

o Scenarios – that account for sensitivity in multiple parameters (e.g. a minimum

change and a maximum change scenario).

o Assigning probabilities to outcomes.

Key considerations

For a more detailed account of the key points in quantifying environmental and social

impacts as part of the value transfer process, see the Defra Value Transfer Guidelines

[Step 3: ‘Define and quantify the change in provision of the policy good’].

The task of quantifying environmental and social impacts of water resource schemes need not

be complex, but the assessment should be transparently reported. In effect the aim is to

produce a robust assessment that ‘stands by itself’ and can inform decision making in its own

right (i.e. in the absence of monetary values).

http://www.defra.gov.uk/environment/natural/ecosystems-services/research-and-case-studies/

http://www.defra.gov.uk/environment/natural/ecosystems-services/research-and-case-studies/



STEP 4 - SELECT VALUATION EVIDENCE


Match available valuation evidence to the environmental and social impacts of the water

resource option being assessed.

Identify the main sources of uncertainty that arise from the use of the available valuation

evidence.

Selecting valuation evidence

The task of selecting appropriate evidence from existing studies as part of a value transfer

exercise requires the following:

1. A review of existing studies to ensure that all evidence potentially relevant to the impacts

of interest is identified;

2. An assessment of the ‘match’ between the impact(s) of interest (as defined and quantified

in Steps 2 and 3) and the impacts that are valued in the available study/studies;

3. An assessment of the quality of the valuation evidence available to determine if it is

sufficiently robust to be used; and

4. The selection of appropriate valuation evidence to transfer.

Refer to the Defra Value Transfer Guidelines for a more detailed treatment of the key

issues in selecting appropriate valuation evidence [Step 4: ‘Identify and select monetary

valuation evidence’].

Understanding the BAG approach

The BAG simplifies the above requirements by guiding users to select values from a series of

studies that were available at the time of its development. In most cases more than one

valuation study is detailed for each impact category so that there is scope for sensitivity

analysis based on a range of values.

While the BAG approach is appealing from the perspective of ensuring consistency of

application across different users, it is subject to limitations, notably:

Studies and valuation evidence documented in the BAG are now dated. While changes in

general price levels can be accounted for (see Step 5), valuation methodologies have

moved on, and what may have been regarded as ‘leading edge’ or ‘good practice’ in

primary studies when the BAG was produced may have been superseded by more recent

developments.



The task of assessing the ‘match’ between the impact(s) to be valued and the available

valuation evidence is fundamental to the transparent and robust application of value

transfer, and should take account of a defined set of criteria (see Box 3.3). In reality these

criteria are stringent and in only a few instances would they all be met. Where significant

differences are evident between the impact(s) to be valued and the available evidence,

there is a need to determine if these can be controlled for in the analysis. This however

takes the assessment beyond the scope of the BAG and the analysis that should be

reasonably expected from a non-economist.

It is recommended that users continue with the process set out in the original BAG for selecting

values to estimate environmental and social benefits, subject to two conditions:

1. First, it must be recognised that the BAG approach may not be appropriate in all

circumstances. In cases of ‘high level’ screening of options and developing rankings based

on AISCs, it is likely to be sufficient. However, where impacts of a scheme or programme

are expected to be highly significant and contentious (e.g. a major resource development

scheme), the BAG approach may not be sufficiently robust to inform final decisions. In

these situations use of the BAG will serve to highlight that further assessment work is

required.

2. Second, for a number of impact categories, newer valuation evidence is available and

should be considered. In addition, more explicit guidance can be provided in relation to

some impact categories (e.g. biodiversity and non-use values, and landscape). These points

are highlighted below along with relevant updates to the valuation evidence.

Over the longer term the appropriate scope for the use of the BAG should be subject to a

fundamental review.

Box 3.3: What are the criteria for assessing the relevance of available valuation evidence?

The Defra value transfer guidelines detail criteria for ‘matching’ the impact(s) of interest in an

assessment to those for which valuation data are available from existing studies:

1. Similarity of impact definitions

2. Similarity of scale and magnitude of the impacts (including the baseline level)

3. Similarity of spatial scale and location

4. Similarity of affected populations;

5. Similarity of the number and quality of substitutes in relation to the impacts

6. Similarity of the ‘market constructs’ (e.g. cultural, institutional, economy, etc.)

Where there is a poor match across these criteria, value transfer should not be undertaken

unless it possible to control for differences between the available evidence and the context

within which it will be used. This requires the use of explicit ‘adjustments’ to account for

differences; e.g. weighting values by differences in average incomes in the affected

populations.



Updating the valuation evidence

Since the original BAG, a number of newer studies have become available that are relevant to

the various categories of environmental and social impacts, and can potentially be utilised in

the appraisal of water resource options. The main developments are summarised below:

Valuing greenhouse gas emissions:

The BAG includes the impact category ‘global warming potential’. Assessments in this

regard should now follow UK Government guidance for valuing greenhouse gas emissions.

DECC / HM Treasury (2011) provides comprehensive guidance.

Current guidance for valuing greenhouse gas emissions is available from DECC:

http://www.decc.gov.uk/en/content/cms/emissions/valuation/valuation.aspx


Valuing air quality impacts:

The BAG does not include a category for emissions of air pollutants such as nitrogen dioxide

(NO2) and sulphur dioxide (SO2). However some SEAs and technical studies may estimate

emissions of pollutants from energy generation (in addition to greenhouse gases) and

construction activities (e.g. vehicle movements). Guidance for valuing air pollution

impacts, primarily in terms of effects on human health, is provided by Defra and the Inter-

departmental Group on Costs and Benefits (IGCB).

See Defra/IGCB for current guidance for valuing emissions of air pollutants:

http://www.defra.gov.uk/environment/quality/air/air-quality/economic/


Valuing impacts on recreation:

The original BAG relies largely on a single study (the ‘Mimram study’) for ‘informal

recreation values’ (e.g. walking, picnicking) associated with water resource schemes (Box

3.4). While more recent studies have valued recreation benefits associated with rivers,

these have been undertaken in the context of water quality improvements from policy

drivers such as the Water Framework Directive and the National Environment Programme.

There have also been methodological developments in the way that recreation values can

be assessed, including the use of geographical information systems - see for example Sen et

al. (2011) as part of the UK NEA – but these studies have typically concentrated on other

types of sites (e.g. forest based recreation). For now, however, it is recommended that

users continue with the approach outlined in the BAG, primarily to maintain ease and

consistency of application.

Refer to the BAG Corrigenda, Section 1 and Section 2 for further detail on the approach

for valuing recreation impacts.

http://www.decc.gov.uk/en/content/cms/emissions/valuation/valuation.aspx

http://www.defra.gov.uk/environment/quality/air/air-quality/economic/



Box 3.4: What evidence is there for assessing the benefits of reducing low flows in rivers?

In the BAG, value estimates derived from the ‘Mimram study’ (Jacobs Gibb, 2002) were the

most widely used and applied by the EA at PR04. The purpose of this study was to identify and

assess the full range of benefits to users and non-users of returning low flows to a more natural

state in the River Mimram, a chalk stream in Hertfordshire. The study also provides a series of

adjustment factors for the subsequent use of the results via value transfer. It remains relevant

as it is the most comprehensive study to date to assess the benefits associated with reducing

low flows in rivers.

The study provides separate estimates for use and non-use values, including valuations specific

to informal recreation and in-stream recreation (although angling is specifically excluded).

These estimate the benefits of flow improvements in terms of the amount that households are

on average willing to pay each year for five years to improve the Mimram so that it runs dry

once every 20 years instead of once every four years.

The benefit of a partial recovery is also examined, where this corresponded to the Mimram

running dry once every 10 years instead of once every 20 years. These values vary only slightly

from full restoration (as a result their validity is questioned by the study).

Given the reliance on this one study at PR04, the use of its results has received a great deal of

attention. In the BAG corrigenda, the EA adjusted the Mimram values, presenting them as ‘per

km’ values and annualising at 3.5% over 25 years so they could be considered to be ‘annual

values’.

Valuing landscape impacts:

Valuations reported in the original BAG with respect to landscape impacts are particularly

dated (typically 15 plus years old). It is recommended that users do not attempt to value

landscape impacts of water resource schemes unless these are expected to be significant

and it can be demonstrated that the valuation of this impact does not overlap with

assessments that may be made in relation to recreation and/or amenity values. A risk here

is double-counting, since valuations can be applied over the same affected population (e.g.

visitors, local residents) and it is not necessarily clear that available values in the BAG are

additive in this regard.

Note that the recommendation not to value landscape impacts does not remove the need

for a qualitative assessment of the impact, based on either the approach outlined in the

BAG or following SEA or EIA outputs. In addition major infrastructure projects that have a

significant a landscape impact may require a primary valuation study and it will be

appropriate to consult with wider stakeholders to discuss the scope of such an assessment.

Valuing biodiversity impacts and non-use values:

With regards to water resource schemes and low flow alleviation, the BAG relies heavily on

the ‘Mimram study’ (Box 3.4). However in general the valuation of impacts on terrestrial

and aquatic biodiversity remains a challenge; for commentary, see for example UK NEA

(2011). Since the development of the BAG a number of newer studies have been

undertaken, including studies for Defra estimating the benefits of Biodiversity Action Plans



(BAP) (Christie et al. 2011a) and the management of SSSIs (Christie et al., 2011b). In

certain cases it may be appropriate to apply results from these studies, but this will require

careful consideration and application, and it is suggested that advice is initially sought

from an EA economist.

Where assessments include the valuation of impacts on biodiversity and estimation of non-

use values, it is recommended that this is undertaken as part of sensitivity testing. The

approach should be to assess the overall influence of the valuation of these impacts on

decision-criteria; i.e. does the inclusion of these values change the ranking of schemes or

the assessment of whether a scheme is worthwhile (the comparison of costs and benefits).

Water company ‘customer preference’ studies

As part of the PR09 business plan process, virtually all water companies in England and Wales

commissioned ‘customer preference’ surveys that estimated values for improvements in service

across a range of water (e.g. interruptions to supply, low pressure, security of supply),

wastewater (e.g. sewer flooding) and environmental service factors. For the most part the

focus of the environmental services aspect was generic river water and coastal/bathing water

quality improvements. As a consequence, valuations from these studies are not appropriate for

site-specific application and have not supported the assessment of environmental and social

impacts in development of WRMPs. However, they may be useful for demand management

options or other measures which lead to an overall, non-specific change in abstraction. In

addition, companies may, in future studies of this type, include more detailed service factors

that are relevant to water resource options, or design studies in such a way that values can be

derived for specific water resource schemes.

Key considerations

The task of selecting the valuation evidence that will be used to value environmental and

social impacts of water resource schemes requires the user’s judgement and the subjective

nature of this aspect of the analysis can give rise to much scrutiny:

Sensitivity analysis

o Assessments that examine the implications of using a range of valuations (i.e. low,

high, mid-point) will generally be regarded as more robust than those that rely on

single point estimates.

Transparency

o Reasons for selecting and not selecting potentially relevant studies and valuation

evidence should be documented, in order to demonstrate that a comprehensive

review of available evidence has taken place.

The BAG AST allows users to document reasons for selecting valuation evidence and

sensitivity ranges that are tested [BAG Part One, Section 4.4].



STEP 5 - ESTIMATE VALUE OF ENVIRONMENTAL AND SOCIAL

IMPACTS


Estimate the value the environmental and social impacts of the water resource option

being assessed.

Apply any adjustments to the valuations that are required.

Estimating the value of environmental and social impacts

There are two basic approaches to value transfer: (i) ‘unit value transfer’; and (ii) ‘value

function transfer’. Unit value transfer, which is the approach adopted in the BAG, represents

the basic approach. Valuation function transfer is typically a more involved process and is

largely beyond the scope of the use of the BAG.

Within the BAG, valuation evidence from available studies may be applied ‘unadjusted’ or

‘adjusted’. The BAG provides guidance on adjustments to control for particular factors that

may differ between different schemes (i.e. due to location specific conditions); for example

the size of the affected population (Box 3.5) or the impact of a scheme on flow (Box 3.6).

Refer to the BAG Part One, Section 3.2 for further detail on adjustments to values.

Box 3.5: How does the BAG recommend adjustments in values?

In relation to informal recreation, the BAG [see Corrigenda, Section 1] recommends that values

sourced from the ‘Mimram study’ are reduced according to the distance households live from

the river, using the following distance decay relationship:

Households living within 0.5 km of river: £0.34 per household per km per year;

Households living between 0.5 and 3 km of river: £0.25 per household per km per year;

Households living between 3 and 12 km of river: £0.07 per household per km per year; and

Households living between 12 and 60 km of river: £0.03 per household per km per year.

Whilst this ‘adjusted unit value transfer’ approach is not the most sophisticated available, it is

practical, since the more complex approach of ‘function transfer’ generally requires a far more

detailed level of analysis.



Box 3.6: Accounting for changes in flow

In relation to water resource options, the BAG includes an explicit approach to factoring flow

changes into calculations (Part Two, Section 1.7). The PHABSIM model (Physical HABitat

SIMulation) is based on the relationship between environmental impacts and flow changes. It

allows users to ‘mechanically’ adjust for the fact that some of the valuation studies in the BAG

(particularly in the recreation and non-use sections) are concerned with changes in river level,

and were not developed specifically to remedy or avoid low-flow situations. Using this model,

the benefit from changes in flow is limited to a maximum 50% of the appropriate transfer

value, and is generally around 30% of the original value.

It should be noted that this flow adjustment is not necessary for values derived from the

‘Mimram study’ (Box 3.4), since this study was specifically designed for value transfer in

relation to low-flow schemes.

Updating values to current prices

Economic assessments require that costs and benefits are estimated on a consistent basis. In

the BAG, all values that are reported from studies available at the time are presented in 2001

prices. Values that are sourced from the original BAG documents will need to be inflated to

current prices or to the base year for the analysis. Factors for inflating values can be sourced

from price indices or the GDP deflator.

Refer to the Defra Value Transfer Guidelines for further detail on updating values to

current prices [Step 5: ‘Transfer evidence and estimate value of the policy good’].

Further adjustments to values

Some consideration may be given to whether values reported in the BAG should also be

adjusted to account for changes in income levels since the guidance was originally developed.

In particular, values estimated in terms of individual’s willingness to pay are expected to be

influenced by their income; therefore if incomes rise, the willingness to pay for avoiding

environmental and social impacts may also be expected increase.

This point is most likely to be applicable to the older studies reported in the BAG (10 plus

years) but in practice it is questionable as to whether these studies remain relevant due to the

time that has passed since they were undertaken. In most instances it will be sufficient to

inflate values to current prices (see above) and further adjustments (which add further

assumptions that need to be justified), other than those already detailed in the BAG, should

ordinarily not be required. Where further adjustments are made, they should be subject to

sensitivity testing, to examine the effect on results with and without the adjustment.

Refer to the Defra Value Transfer Guidelines for further detail on adjusting values to

account for income differences [Step 5: ‘Transfer evidence and estimate value of the

policy good’].



Key considerations

In most instances adjustments that are made to values (other than inflating to current price

levels) should be transparently reported and subject to sensitivity testing; i.e. to understand

the implication the adjustment has on estimates of costs and benefits and how this may

influence decision criteria.

Along with the AST, the EA developed a spreadsheet for recording calculations in a

methodical and consistent way. This can be obtained from the EA although water

companies may choose to develop their own versions of both the AST and spreadsheet.



STEP 6 – AGGREGATION


Estimate the total value of environmental and social cost and benefits of the water

resource option being assessed across the time horizon for the analysis and/or the life of

the scheme.

Calculating aggregate costs and benefits

The task of estimating the aggregate (total) value of the environmental and social impacts of

water resource options involves:

1. Aggregating the estimated value for impacts (from Steps 4 and 5) over the quanitified

measure of the impacts (from Step 3); and/or

2. Aggregating over the affected population; then

3. Aggregating over the time horizon for the analysis or the lifetime of the scheme.

The principles set out in the BAG guidance for aggregation remain applicable since they refer

to the basic calculations that are required. The BAG also provides advice on assessing the

timing of costs and benefits (i.e. when they are judged to occur within the time horizon for the

analysis). Note also that these calculations should be recorded in the AST.

Refer to the BAG Part One, Section 10, for guidance on estimating aggregate values in

relation to river and groundwater schemes.

More recent studies – for example in relation to estimating recreation values – have developed

approaches for accounting for spatial variation in values across affected populations (e.g. local

residents). This includes the use of geographical information systems (GIS). Whilst this scale of

analysis is beyond the scope of assessments that can reasonably be expected for the screening

of water resource options, the approach detailed in the BAG with respect to the use of the

‘Mimram study’ results (Box 3.5) does provide a practical method for including spatial effects

via the distance decay of values. In general, where impacts are expected to be significant and

more detailed analysis and effort is required, more sophisticated approaches to accounting for

spatial variation in values should be adopted.

Refer to the Defra Value Transfer Guidelines for further discussion on accounting for

spatial variation in values [Step 6: ‘Aggregation’].

Updating discount factors

The original BAG was developed prior to the most recent version of the HM Treasury (2003)

Green Book, which sets out the discount factors that should be applied in calculating the

present value of future costs and benefits. Supplementary guidance has also been issued in

relation to discounting impacts over the very long term (more than 50 years).



Refer to the HM Treasury Green Book for guidance on discount rates:

http://www.hm-treasury.gov.uk/data_greenbook_supguidance.htm


Key considerations

As with all other aspects involved in estimating the value of environmental and social impacts,

assumptions that are applied in the aggregation process also represent factors that can be

tested via sensitivity analysis; for example assumptions in relation to the timing of impacts.

http://www.hm-treasury.gov.uk/data_greenbook_supguidance.htm



STEP 7 – SENSITIVITY ANALYSIS


Test the effect of different assumptions for key parameters in the calculation of the value

of environmental and social impacts of the water resource option being assessed.

Understand which assumptions have the greatest influence on decision-criteria (i.e. the

comparison of costs and benefits)

Basic approaches to sensitivity analysis

In practice the requirements for sensitivity testing will need to be determined by the user on a

case-by-case basis, since in different situations, scrutiny may fall on different aspects of the

analysis. However basic options that can be considered are:

Changing one parameter (e.g. quantified impact, estimated value, timing of impact, size of

affected population, etc.) at a time to see the effect on the resulting value estimate;

Using scenarios to account for sensitivity in multiple parameters (simultaneously changing

parameters such as the quantified impact and estimated value to assess ‘best’ and ‘worse’

cases);

Assigning probabilities to outcomes (e.g. to reflect technical assessments of the likelihood

of an impact occurring, such as a low flow);

Using Monte Carlo analysis to account for sensitivity in multiple parameters especially when

there are significant uncertainties; and

Switching values and benefits thresholds (see below).

In most cases, the testing of individual parameters should be sufficient for the appraisal of

water resource schemes, and this is largely the approach that is established in the BAG (Part

Six, Section 4) and to be reported in the AST.

Testing in terms of scenarios may also be effective in considering the potential impacts from a

programme of schemes. Note also that technical assessments may anyway include probabilistic

information in the estimation of quantified impacts, and this can be tested as an individual

parameter too.

Refer to the Defra Value Transfer Guidelines for further discussion of sensitivity testing

[Step 7: ‘Conduct sensitivity analysis’].

Switching values and thresholds

To help ‘establish’ the level of uncertainty in an appraisal, users should consider estimating

switching values and benefits thresholds. These can help address some of the concerns and

scrutiny that wider stakeholders may have with the results of value transfer analyses. The basic

premise is to establish how ‘wrong’ the value transfer has to be for the recommendation to

change – in other words (in the case of CBA) for the net present value (NPV) to switch from



positive (benefits outweigh costs) to negative (costs outweigh benefits) or vice versa, or for an

alternative scheme to be preferred.

Refer to the Defra Value Transfer Guidelines for further details for estimating switching

values or benefits thresholds [Step 7: ‘Conduct sensitivity analysis’].

Note the BAG, Part Six Sections 2-3 describes the process for calculating net present

values.

Key considerations

Based on the preceding Steps (1-6), key considerations for sensitivity testing in relation to

estimating the value of environmental and social impacts of water resource schemes include:

Step 2: assumptions required to link outputs of technical assessments and SEAs to the BAG

assessment format, such as matching impact categories and determining the scale and

significance of environmental and social effects.

Step 3: the quantitative assessment of impacts (i.e. a high – low measure or determining

the probability of outcomes) and the determination of the affected population (e.g.

number of visitors).

Step 4: use of a range of value estimates (i.e. low, high, mid-point) from more than one

relevant study (if available). Note that a particular amount of uncertainty concerns the

estimation of non-use values and biodiversity impacts and as a ‘default’ the assessment

should calculate aggregate values with and without the inclusion of these values.

Step 5: any adjustments that are made to values other than those prescribed in the BAG

and the task of inflating to current prices.

Step 6: assumptions related to the timing of impacts and the scale of the affected

population (note this links to Step 3 above).



STEP 8 – REPORTING


Provide a transparent record of the assessment of environmental and social impacts,

documenting all assumptions and data sources

Level of detail required in reporting

Transparent reporting is essential for informing decision-making of the likely accuracy of

evidence provided and this should be supported by documenting all assumptions and data

sources. Within the BAG the AST provides the principle ‘tool’ for reporting, particularly where

assessments are carried out across a large number of options.

Determining whether the AST alone represent a sufficient level of detail has to be judged on a

case by case basis by the user:

Initial screening of a large number of options using the AST approach is likely to be

sufficient although users may wish to adapt the format to include more detailed

commentary on individual assumptions or assumptions that are made across all schemes

and impacts. Additional material (e.g. supporting spreadsheets) should be made available

to provide a clear audit trail for calculations.

If more detailed assessments are made – for example in the case of schemes with

significant impacts – then a greater level of detail may be required in reporting to support

the results of the analysis (i.e. showing a greater range of sensitivity testing and

interpretation of findings).

Particular attention should be paid to non-quantified impacts that may be significant but

which have not been quantified for good reasons (e.g. lack of appropriate valuation study).

There is a danger that these impacts are ignored, even where they are cited in the AST.

Action taken to avoid double counting should be described. The nature of the BAG and

more recent approaches to value transfer should minimise the risk of double-counting (e.g.

by systematically assessing impacts in separate categories), but the risk remains inherent in

analyses of this type. The BAG (Part One, Section 3.5) provides more detail on this issue.

All decision rules considered and used should be set out and described. The focus of the

BAG (see Part Six, Section 3) is on NPV and benefit-cost ratio (BCR), but other rules are

available.

Refer to the BAG (Part Six) for guidance on bringing together the Appraisal Summary

Tables and presenting the aggregated results for decision-making.



4. SUMMARY OF TASKS

BASIC TASKS DETAILS Done () STEP 1-Review decision-making context

Is value transfer the appropriate approach to meet the information needs of a given decision-making context?

The phase in the policy or project decision-context

The scale of effects of the policy or project

The scale of investment/expenditure

Legal, political and stakeholder context

Is value transfer possible?

Is sufficient economic and scientific information available?

Are sufficient time and resources available?

If yes, what is the appropriate level of effort? Record answer:

If no, would primary valuation or an approach other than economic valuation would be better?

Record answer:

Key references and considerations Defra VT guidelines Step 1: ‘Establish the policy good decision-context’

STEP 2 –Determine environmental and social impacts

Categorisation of impacts Qualitative description

Link / translate outputs of SEA to the BAG

The nature of the impact

Market or non-market [BAG Part One, Section 2.2 and 2.3]

Location

Scale

Timing

The affected population What user population is affected? [BAG Part One, Section 3.4]

Is a non-users population likely to be affected? [BAG Part One, Section 3.3]

Socio-economic characteristics of the user / non-user

The parameters to be tested in sensitivity analysis Record answer:

Key references and considerations BAG appraisal summary table (AST), BAG Part Two, Section 2.5

Defra VT guidelines Step 2: ‘Define the policy good and affected population’



STEP 3 –Quantify environmental and social impacts

Define the baseline conditions [See BAG Part One, Section 4.1]

Economic baseline

Environmental baseline

Describe the impacts qualitatively [See BAG Part Two; BAG Part Three]

Scale of the impact

Nature of the impact

Direction of the impact

Timing of the impact

Location of the impact

Assess the impacts quantitatively [See BAG Part Two, Sections 2-9; BAG Part Three, Section 2-6; Defra (2007) Chapter 3]

Units of change for quantity impacts

The risk of a particular change

The change in affected population

Identify the parameters to be tested in sensitivity analysis Ranges for quantitative data

Scenarios that account for sensitivity in multiple parameters

Assigning probabilities to outcomes

Key references and considerations Defra VT guidelines, Step 3: ‘Define and quantify the change in provision of the policy good’

STEP 4 – Select valuation evidence

BAG valuation evidence Assess original evidence

Is there any evidence that matches the environmental and social impacts of interest?

Are valued impacts sufficiently similar?

Are the scale and magnitude of the impacts sufficiently similar?

Are the locations and spatial scale sufficiently similar?

Are the affected populations sufficiently similar?

Are the number and quality of substitutes sufficiently similar?

Are the market constructs sufficiently similar?

Is the evidence of sufficiently high quality? Are the data collection procedures sound?

For survey-based economic valuation methods is the sample representative?

Does the study follow the best practice?

Are the results consistent with the expectations based on the economic theory?

If not, can the discrepancy be explained?

New evidence Greenhouse gas valuation [See DECC / HM Treasury (2011)]

Air quality valuation [See Defra / IGCB (2011)]

Identify the parameters to be tested in sensitivity analysis Record answer:

Key references and considerations Defra VT guidelines, Step 4: ‘Identify and select monetary valuation evidence’



STEP 5 -Estimate value of environmental and social impacts

Which value transfer approach is to be used? Unit value

Adjusted unit value

Value function – collate supporting data

What is the transferred value for the impacts of interest Inflate to current prices

Report adjustment factors used or why none is used [See BAG Part Two; Part Three]


Key references and considerations Defra VT guidelines, Step 5: ‘Transfer evidence and estimate value of the policy good’

STEP 6 –Aggregation

What is the aggregated value of the impact [See BAG Part One, Section 10 for river and groundwater schemes]

Aggregate over the quantified measure of the impact

Aggregate over the affected population

Aggregate over the specified time horizon

List assumptions about aggregation in particular whether unit value is assumed to vary across space and time

What is the present value of the change? How do (i) the change in the provision, (ii) the annual value and (iii) the affected population change over time?

Aggregate over time

Report the time period and discount rate used [See HM Treasury (2003)]


Key references and considerations Defra VT guidelines, Step 6: ‘Aggregation’

STEP 7 – Sensitivity analysis

Which key parameters affect the transferred value the most?

Step 2 The type and size of the affected user and non-user population(s)

Step 3 The magnitude, direction, the timing and the spatial nature of the change Quantitative estimates of the change Uncertainties and gaps in supporting data (e.g. socio-economic characteristics of affected population, patterns of use, availability of substitutes)

Step 4 The selection of evidence from existing studies such as unit values, value functions and empirical relationships (e.g. distance decay)



Step 5 Best estimates and confidence intervals for unit value of change Best estimates and confidence intervals for value function coefficients Adjustment factors that are applied to unit value estimates or function coefficients Policy good values of explanatory variables in value functions

Step 6 Discount rate and time horizon

What is the nature and significance of the effects of key parameters on the results?

Record annual and/or present value estimates of value for each run of sensitivity analyses

What is the switching value? Record the switching value:

What is the benefit threshold? Record the benefit threshold:

Key references and considerations Defra VT guidelines, Step 7: ‘Conduct sensitivity analysis’

STEP 8 – Reporting

Appraisal summary table Complete

Sensitivity analysis

Summarise

Key parameters and caveats (Steps 2-6)

Summarise

Key references and considerations Record



REFERENCES

Christie, M., Hyde, T., Cooper, R., Fazey, I., Dennis, P., Warren, J., Colombo, S. and Hanley,

H. (2011a) Economic Valuation of the Benefits of Ecosystem Services delivered by the UK

Biodiversity Action Plan, Report to Department for Environment, Food and Rural Affairs.

Christie, M., Dzanja, J., Hyde, T., and Minter, R. (2011b) Economic Valuation of the Benefits of

SSSIs: Annex 3, Report to Department for Environment, Food and Rural Affairs.

DECC / HM Treasury (2011) Valuation of energy use and greenhouse gas emissions for appraisal

and evaluation.

Available from:

http://www.decc.gov.uk/en/content/cms/about/ec_social_res/iag_guidance/iag_guidance.as

px [Accessed January 2012]

Defra (2007) An Introductory Guide to Valuing Ecosystem Services.

Available from:

http://www.defra.gov.uk/environment/natural/ecosystems-services/valuing-ecosystem-

services/ [Accessed January 2012]

Defra (2010), Valuing Environmental Impacts: Practical Guidelines for the Use of Value

Transfer in Policy and Project Appraisal, Prepared by eftec for Department for Environment,

Food and Rural Affairs.

Defra / IGCB (2011) Air quality damage cost guidance.

Available from:

http://www.defra.gov.uk/environment/quality/air/air-quality/economic/damage/


eftec (2006) Valuing our natural environment, Report to Department for Environment, Food

and Rural Affairs.

Environment Agency (2003) Guidance: Assessment of Benefits for Water Quality and Water

Resources Schemes in the PR04 Environment Programme, Part One – Introduction, Version 1,

issued 25th March 2003.

Available from:



HM Treasury (2003), TheGreen Book: Appraisal and Evaluation in Central Government. The

Stationery Office, London.

Jacobs Gibb (2002) ‘River Mimram Low Flow Public Preferences Study. Final Report – Phase

1’.In association with – Warwick Business School, Corporate Citizenship Unit and Glasgow

University. Report to Environment Agency Thames Region.

Millennium Ecosystem Assessment (MEA) (2003), Ecosystems and human well-being: Synthesis.

Island Press, Washington, D.C., USA.

http://www.decc.gov.uk/en/content/cms/about/ec_social_res/iag_guidance/iag_guidance.aspx

http://www.decc.gov.uk/en/content/cms/about/ec_social_res/iag_guidance/iag_guidance.aspx



http://www.defra.gov.uk/environment/quality/air/air-quality/economic/damage/




Millennium Ecosystem Assessment (MEA) (2005), Ecosystems and human well-being: a

framework for assessment. Island Press, Washington, D.C., USA.

ODPM (2005) A Practical Guide to the Strategic Environmental Assessment Directive, Office of

the Deputy Prime Minister.

Available from:

http://www.communities.gov.uk/publications/planningandbuilding/practicalguidesea


UK NEA (2011), TheUK National Ecosystem Assessment: Synthesis of the Key Findings, UNEP-

WCMC, Cambridge.

Available from:

http://uknea.unep-wcmc.org/


http://www.communities.gov.uk/publications/planningandbuilding/practicalguidesea

http://uknea.unep-wcmc.org/



GLOSSARY Adjusted unit value transfer

Transfer of a mean average (or median) value estimate for a study good that is adjusted to account for some factor (or factors) to estimate the value of policy good.

Affected population The population of the users and non-users that are affected by the change in the provision of a market or non-market good or service.

Appraisal summary table (AST)

Table developed alongside the BAG to record the results of an appraisal, including qualitative description, quantitative and monetary assessment, key assumptions used and sensitivity analysis.

Average incremental cost (AIC)

Used in water resources planning as the basis for comparing the relative cost per unit of capacity, or output, of different options (excluding environmental and social costs).

Average incremental social cost (AISC)

Used in water resources planning as the basis for comparing the relative cost per unit of capacity, or output, of different options (including environmental and social costs).

Benefit – cost ratio (BCR)

The ratio of the benefits of project or policy relative to its costs, expressed in monetary terms. A value greater than 1 indicates that calculated benefits exceed calculated costs.

Biochemical oxygen demand (BOD)

The amount of dissolved oxygen needed by aerobic biological organisms in a body of water to break down organic material present in a given water sample at certain temperature over a specific time period.

Cost – benefit analysis (CBA)

A decision-making tool that compares costs and benefits of a proposed policy or project in monetary terms.

Cost-effectiveness analysis (CEA)

A decision-making tool that compares the cost of different options for achieving the same or similar outcomes.

Discounting The process of expressing future values in present value terms. This allows for the comparison of flows of costs and benefits over time, regardless of when they occur.

Distance decay Pattern of declining unit values for a non-market good or service as distance from it increases.

Economic valuation evidence (monetary valuation evidence)

Economic values, value functions and other empirical evidence available from existing (primary) studies that provides the source of evidence for value transfer. Previous value transfer analyses may also provide evidence for current applications.

http://en.wikipedia.org/wiki/Oxygenation_(environmental)



Economics of Balancing Supply and Demand (EBSD)

Guidance on economic appraisal of water resource planning options that provides the general context for option development and appraisal.

Environmental Impact Assessment (EIA)

Identification and assessment of environmental impacts resulting from a proposed policy or project.

Geographical information systems (GIS)

An information system that captures, stores, analyses, manages, and presents data that is linked to geographic location.

Market goods Goods and services traded in formal markets.

National Environment Programme (NEP)

Refers to the water companies’ five yearly environmental improvement programme.

Net present value (NPV) The difference between the present value of costs and the present value of benefits.

Non-market goods and services

Goods and services that are not traded in markets and are consequently ‘un-priced’ (e.g. environmental goods and services).

Present value (PV) A future value (cost or benefit) expressed in present day terms by means of discounting.

Restoring Sustainable Abstraction (RSA)

This is the Environment Agency’s programme for reducing the amount of water taken from the environment, and also includes modifications to abstraction licenses.

Sensitivity analysis Assesses how the uncertainty in the output of statistical model or decision-making tool can be attributed to different uncertainties in the inputs of the model.

Strategic Environmental Assessment (SEA)

A system of incorporating environmental considerations into policies, plans, and programmes. It may be conducted before a corresponding EIA is undertaken.

Unit value transfer Transfer of a mean average (or median) value estimate for a study good to estimate the value of policy good.

Value function transfer A statistical relationship between the value of a study good and a set of explanatory variables that is transferred to estimate the value of the policy good.

Value transfer Process by which readily available economic valuation evidence is applied in a new context for which valuation is required.

benefits assessment guidance - the national...

Documents