stream 2 of the regional nrm planning for climate change ...environment.gov.au › system › files...

Stream 2 of the

Regional NRM

Planning for Climate

Change Fund 2013 –

2016 Final Evaluation

(Final Report)

Prepared for the Department of the

Environment

15 July 2016

Clear Horizon Consulting

129 Chestnut Street, Cremorne VIC 3121

Tel: +61 3 9425 7777

www.clearhorizon.com.au

http://www.clearhorizon.com.au/

Stream 2 – Final Evaluation

Final Report Clear Horizon Consulting i

Contact details

Clear Horizon Contact Client contact

Stuart Raetz

Clear Horizon Consulting

(03) 9425 7777

[email protected]

Science Partnerships Section

Science Division

Department of the Environment

[email protected]

Project Director Lee-Anne Molony

Internal Contributors Stuart Raetz, Lee-Anne Molony, Jill Campbell, Thomas Hannon

External Contributors Patrick O’Connor

Last saved 1/07/2016 8:34 AM

Clear Horizon Reference Number CH12_166

Disclaimer

This document has been produced with information supplied to Clear Horizon by projects and

stakeholders involved in the Australian Government’s Stream 2 of the Regional NRM Planning

for Climate Change Fund 2013 – 2016 program. While we make every effort to ensure the

accuracy of the information contained in this report, any judgements as to suitability of the

information for the client’s purposes are the client’s responsibility. Clear Horizon extends no

warranties and assumes no responsibility as to the suitability of this information or for the

consequences of its use.

mailto:[email protected]

mailto:[email protected]


Final Report Clear Horizon Consulting 2

Contents

1. Executive Summary ............................................................................................................................. 5

1.1. Background .................................................................................................................................. 5

1.2. Stream 2 final evaluation .......................................................................................................... 6

1.3. Final evaluation findings ............................................................................................................ 6

1.4. Key learnings from Stream 2 .................................................................................................... 8

2. Introduction ........................................................................................................................................ 10

2.1. Background ................................................................................................................................ 10

2.2. Scope ........................................................................................................................................... 11

2.3. Purpose ....................................................................................................................................... 11

2.4. Audience ..................................................................................................................................... 11

2.5. Key evaluation questions ......................................................................................................... 12

3. Findings ............................................................................................................................................... 13

3.1. The effectiveness of Stream 2 ................................................................................................ 13

3.2. The relevance of Stream 2....................................................................................................... 31

3.3. The appropriateness of Stream 2 ........................................................................................... 40

3.4. The efficiency of Stream 2 ....................................................................................................... 43

3.5. The legacy and sustainability of Stream 2 ............................................................................ 45

Attachment One: Stream 2 Program Logic ........................................................................................... 51

Attachment Two: The Cluster Model ....................................................................................................... 52

Attachment Three: Results chart ............................................................................................................. 53

Attachment Four: Stream 1 Report 2013-14 – 2014-15 Synthesis ................................................. 63

Attachment Five: Methodology ................................................................................................................ 70

Approach ................................................................................................................................................. 70

Data collection ....................................................................................................................................... 70

Data analysis .......................................................................................................................................... 71

Data synthesis ........................................................................................................................................ 71

Limitations .............................................................................................................................................. 72

Attachment Six: Interview guides ............................................................................................................ 73

Attachment Seven: Output definitions ................................................................................................... 79

Attachment Eight: Glossary of terms ...................................................................................................... 81



List of figures

Figure 1. Summary of Stream 2 outputs and outcomes (2013 - 2016) ........................................... 15

Figure 2. Climate Change in Australia website users per month ....................................................... 18

Figure 3. National Projections help desk client support groups ......................................................... 18

Figure 4. The extent to which AdaptNRM modules met user needs ................................................. 37

Figure 5. Stream 2 Program Logic .......................................................................................................... 51

Figure 6. The Cluster Map showing the eight Stream 2 NRM clusters .............................................. 52

List of tables

Table 1. Evaluation audience and information requirements ............................................................ 11

Table 2. Summary of outputs delivered by Stream 2 (2013 – 2016) ............................................... 14

Table 3. Stream 2 products stored in Terra Nova (February 2016) ................................................... 17

Table 4. Terra Nova page views ............................................................................................................... 17

Table 5. Examples of use of Stream 2 product by NRM representatives ......................................... 21

Table 6. Modes of engagement used by Stream 2 projects ............................................................... 34

Table 7. Stream 2 Results Chart (2013 – 2015) .................................................................................. 53

Table 8. Uses of Stream 2 products by NRM regions as reported in Stream 1 reporting (2013-14

– 2014-15) .................................................................................................................................................. 63

Table 9.Glossary of terms ......................................................................................................................... 81



Acronyms

CMA

DCCEE

DEWNR

DIICCSRTE

EOPO

ERF

Catchment Management Authority

Australian Government Department of Climate Change and Energy Efficiency

South Australian Department of Natural Resources and Water

Australian Government Department of Industry, Innovation, Science, Research and Tertiary

Education

End of Program Outcome

Emissions Reduction Fund

GBCMA

IPCC

KEQ

Goulburn Broken Catchment Management Authority…

Intergovernmental Panel on Climate Change

Key evaluation question

LLS

M&E

Local Land Services

Monitoring and Evaluation

MCAS-S

MCMA

NACC

NARCLiM

NCCMA

NESP

NGO

NR AMLR

NRM

Multi-Criteria Analysis Shell for Spatial Decision Support

Murray Catchment Management Authority

Northern Agricultural Catchments Council

NSW / ACT Regional Climate Modelling

North Central Catchment Management Authority

National Environmental Science Program

Non-Government Organisation

Natural Resources Adelaide & Mount Lofty Ranges

Natural Resource Management

PWG Project working group

RI

SCARP

SCNRM

SWCC

USQ

Resources Inventory

Southern Slopes Climate Change Adaptation Research Partnership

South Coast Natural Resource Management

South West Catchments Council

University of Southern Queensland



1. Executive Summary

This report presents the final evaluation of the implementation of Stream 2 of the Regional

Natural Resource Management Planning for Climate Change Fund.

1.1. Background

The Australian Government’s Regional Natural Resource Management (NRM) Planning for

Climate Change Fund (NRM Fund) provided $43.9 million over five years to improve regional

planning for climate change and help guide the location of carbon and biodiversity activities.

The NRM Fund was administered by the Australian Government Department of the

Environment1 (the Department) and delivered through two streams: Stream 1 ($28.9 million) to

support regional NRM organisations to revise existing regional plans; and Stream 2 ($15

million) to produce regional level climate change information and provide guidance on the

integration of that information into regional NRM and land use planning.

The objective of Stream 2 of the NRM Fund was to improve the capacity of regional NRM

organisations to plan for climate change by improving the quality and accessibility of regionally

relevant information on climate change impacts and potential adaptation responses.

The Stream 2 program was delivered through eight regional projects and two national projects

(‘Stream 2 projects’):

a ‘National Projections’ project lead by the Commonwealth Scientific and Industrial

Research Organisation (CSIRO) and Bureau of Meteorology (BoM) to deliver regionalised

climate projections,

eight ‘climate impacts and adaptation’ projects (Monsoonal North, Murray Basin, Wet

Tropics, Central Slopes, Southern and South Western Flatlands, Southern Slopes, East

Coast and Rangelands) delivered to eight NRM ‘clusters’ based on existing regional NRM

boundaries, and

a national ‘impacts and adaptation’ project (the ‘AdaptNRM’ project).

A NRM Information Management Project with Griffith University has also been supported with

NRM funding to enhance the Terra Nova web portal as the repository of Stream 2 published

material. Clear Horizon Consulting was engaged by the Department to provide monitoring and

evaluation (M&E) support for the program and to undertake a mid-term evaluation (2014) and a

final evaluation (2016).

See Section 2 for more detail on the Stream 2 projects.

1 There have been a number of agencies responsible for the administration of the program since its inception due to

administrative arrangement order changes including the Department of Environment, the Department of Climate

Change and Energy Efficiency (DCCEE) and the Department of Industry, Innovation, Climate Change, Science,

Research and Tertiary Education (DIICCSRTE).



1.2. Stream 2 final evaluation

This final evaluation covers the entire program timeframe (from 2013 to 2016) and the

investment of $15 million. The purpose of this final evaluation is to demonstrate the

achievements of Stream 2, share learning, and contribute to the knowledge base for research

and investment in climate change science and NRM planning.

This final evaluation was prepared by drawing on multiple data sources including: a series of 36

semi-structured phone interviews2; annual project reporting (between 2013 and 2016), Stream

1 regional reporting (39 NRM regions); NRM planning documentation (16 NRM regions);

departmental records; and project documentation and resources. The evaluation findings are

presented below.

1.3. Final evaluation findings

Stream 2 has effectively met its overarching objective of delivering regionally-relevant climate

change information and support to NRM regions via eight cluster projects, the National

Projections project and the national impacts and adaption project (AdaptNRM). This was

supported by fit-for-purpose, regionally-specific engagement. The findings presented below

focus on what and how information has been delivered, the support given to enable planners to

integrate climate science into NRM planning and the relationships and linkages that have

developed as a result, and the legacy that is likely to remain from the program.

Delivery of information to assist climate adaptation by regional NRM bodies

NRM regions can now approach climate adaptation planning with more confidence – knowing

they have access to scientifically-credible climate information and resources.

At the end of Stream 2 in 2016 regionally-relevant, high quality climate change impacts and

adaptation information is more accessible to NRM regions than at the beginning of the

program. A total of 958 NRM climate research and planning products3 have been delivered by

the ten projects including 183 next generation climate change projections products. More than

268 engagement activities have been delivered across ten projects to support the use and

application of climate change research and science in the NRM planning process. Through web

access, up-to-date regionally relevant climate change information is more accessible to NRM

organisations and stakeholders than previously.

Integrating climate science into NRM planning

Most NRM regions are using the climate science from Stream 2 of the Regional Natural

Resource Management Planning for Climate Change Fund for their planning.

Uptake and application of Stream 2 products by NRM regions is widespread with around three

quarters of the NRM regions specifying examples of using Stream 2 outputs in the planning

process. Climate science outputs developed by Stream 2 have been used as technical inputs

into NRM planning (e.g. use of the Agricultural Production Systems Simulator [APSIM] by

Northern Tablelands in the Central Slopes cluster). Significantly, there was evidence that

Stream 2 science products have influenced the planning frameworks adopted and used by NRM

organisations in some regions to guide their thinking about how to plan for climate change. For

2 Interviews were undertaken with regional NRM representatives (n=20), Stream 2 project representatives (n=13)

and the Department of Environment representatives (n=3). 3 Including interim, draft and final products delivered by Stream 2.



instance, the Adaptation Pathways approach further developed by the Southern Slopes project

was used by the Corangamite CMA and in East Gippsland CMA in Victoria to guide spatial NRM

planning.

Stream 2 has had a marked influence on the NRM planning process as well as delivering

products and resources to support regional NRM planning. Where Stream 2 has had a positive

effect on planning capacity within NRM organisations, there tends to also be strong linkages,

ties and geographical proximity to the Stream 2 project teams.

Interactions between researchers and planners

The relationships between researchers and NRM planners – developed through many types of

interaction – led to enhanced capacity and understanding amongst both the science and NRM

communities.

A range of engagement and delivery approaches were used by Stream 2 projects to understand

and respond to the planning needs of NRM regions. For the most part, this has supported the

delivery of products and support to NRM regions that has been well received, relevant and fit-

for-purpose. Multiple modes of engagement were often necessary to ensure coverage across

socially diverse and geographically wide-spread cluster areas. User uptake of information was

supported in regions and clusters where there was a common and widely-accepted focus (e.g.

agriculture) and existing networks (e.g. where there were well-established industries and

linkages). The devolved funding agreements set up by the Department enabled Stream 2

projects to tailor their engagement activities and processes to the specific clusters with which

they were working.

By and large the cluster approach taken in Stream 2 has led to inter-regional learning and

connections across NRM groups that would not have otherwise been established. This was

strongest in clusters where: (i) inter-regional forums were established and brokered by the

project teams, and (ii) existing inter-regional forums existed (i.e. Victoria and South Western

Australia). While the cluster model has facilitated inter-regional learning in most clusters, in

others it has reinforced existing geographical and institutional divisions across state

boundaries. For instance, in the Southern and South-Western Flatlands cluster, the South

Australian NRM regions (Eyre Peninsula, Northern and Yorke, Adelaide and Mount Lofty Ranges

and Kangaroo Island) are geographically split from the Western Australian based project team

and NRM regions (Northern Agricultural, Wheatbelt, Perth, South West and South Coast). This

split was exacerbated by repeated staff turnover in the liaison point between the Perth-based

project team and the South Australian NRM planners. Ultimately, inter-regional learning in this

cluster was compromised as a result of these factors, which relate to both the design of the

cluster model as well as project implementation (staff turnover).

Stream 2 has not fully realised considerable opportunities for greater program coordination and

increased efficiencies across project and cluster boundaries. This is due largely to the fact that

the program did not mandate or build any formal structures for information sharing into the

program. Coordination across clusters could have been strengthened for example if the

Department had established a centralised program coordination role with clear responsibilities

for encouraging and supporting opportunities for information sharing.



The legacy of Stream 2

While there is uncertainty and considerable challenges ahead in regional planning for climate

change the outcomes of the program are likely to be enduring well beyond the timeframe of

delivery.

Reflecting the outcomes of the program, the legacy of Stream 2 includes significant advances

by regional bodies in their ability to adaptively plan for climate change and the embedding of

new information and knowledge into regional planning processes. As regional NRM

organisations adaptively plan for climate change they will have improved access to responsive,

adaptive tools and frameworks (such as the adaptation pathways approach), research outputs,

online resources and platforms (such as the Climate Change in Australia website and the Terra

Nova platform), as well as improved networks and relationships between researchers, planners

and NRM representatives.

While scientists, researchers and NRM managers and planners will continue to derive benefits

from the outcomes achieved by the program there are a range of challenges and constraints,

which will affect the legacy of the program. These include:

the cluster model was designed purely for Stream 2 and has no longevity beyond the

project timeframe (2013 – 2016), though it potentially has applicability for future

federal NRM investment and programming

lack of ongoing funding to support the application of the Climate Change in Australia

website beyond the possibility of short-term basic maintenance through a project under

the Australian Government National Environmental Science Programme (NESP)

staff turnover and organisational restructuring within regional bodies resulting in a loss

of corporate knowledge and capacity built through the program

low levels of awareness about Stream 2 within the Department and at a state level

among relevant agencies and departments

absence of formal structures or resources to sustain linkages between researchers and

planners.

Ultimately the responsibility for implementing NRM plans lies with a diverse range of state,

regional and local, community stakeholders who operate within the networks facilitated by

regional NRM bodies. As these stakeholders begin to implement NRM planning for climate

change, the influence that the program has had on NRM organisations’ ability to adaptively plan

and manage climate change impacts will become increasingly apparent.

1.4. Key learnings from Stream 2

Based on the findings of this evaluation, a number of learnings for any future Australian

Government investment in climate change impacts, adaptation research and NRM planning

have been identified. Applying the learnings will necessitate coordination and support from

other sectors such as regional NRM bodies and researchers.

The ‘impact’ of Stream 2 was largely on the NRM planning process. The update of NRM plans

was a benefit of Stream 2 but the lasting value of this program will be the influence on the

regional NRM planning process more generally. In this regard, the program’s impact is

influenced by state policy frameworks that determine the role of NRM plans. Enhanced linkages

and relationships between planners and researchers are critical in supporting the program’s

contribution to the regional NRM planning process.



Learnings for consideration in future investments are outlined below.

Recognise the role of engagement in supporting science translation. Investment in climate

adaptation at the science to policy and management interface must explicitly address the

need for engagement between the producers and users of science. This can be done by

simply requiring a percentage of funds or grants devoted to supporting science translation

to be apportioned specifically to engagement activities. Early and sustained engagement

between the producers and users of science is critical to support uptake.

Bridging the climate science and practitioner gap is necessarily challenging. Researchers

and NRM practitioners operate with different worldviews in different systems. The process

of bringing scientists and practitioners together is a dynamic process which requires

adaptive, flexible approaches in procurement, programming, project management and

evaluation.

Investment in science and NRM at a national scale requires sufficiently resourced

coordination. To fully realise the benefits from a program of this scale - such as cross-

cluster learning - it is necessary to adequately fund program coordination beyond a formal

administrative capacity.

Take institutional considerations into account in further applications of the cluster model.

Stream 2 has proven to be an effective trial of the cluster model for delivering NRM

investment nationally. There have been significant benefits in delivering the approach on

bio-climatic factors. However institutional factors, including state and institutional

boundaries, should be more fully considered to maximise the benefits of this approach.

This was most acutely demonstrated in a geographical split between South Australia and

Western Australia in the Southern and South-Western Flatlands cluster.

Provide sufficient flexibility in contracting arrangements in any future funding program for

climate change adaptation. This program has demonstrated that program design doesn’t

need to be prescriptive to effectively deliver outcomes. Indeed, it is arguable that, in order

to be effective, climate adaptation investment needs to be sufficiently flexible and

responsive to emergent needs, priorities and values.

Carefully consider the sequencing in designing future NRM science and planning

investment. The concurrent delivery of Streams 1 and 2 presented considerable challenges

as well as opportunities for projects in achieving the objectives of the NRM Fund. In some

regions, NRM plans were updated with Stream 1 funding prior to receiving updated science

and research information through Stream 2. This concurrent timing created difficulties but

also opportunities for the program as researchers and NRM representatives were required

to engage with one another in ways that would not otherwise have been possible.



2. Introduction

2.1. Background

The Regional Natural Resource Management (NRM) Planning for Climate Change Fund (NRM

Fund) was allocated $43.9 million over five financial years to improve regional planning for

climate change and help guide the location of carbon and biodiversity activities. The NRM Fund

was one of the Land Sector Measures under the Australian Government's Clean Energy Future

(CEF) Plan (now defunct).

The NRM Fund was provided in two streams:

Stream 1 has $28.9 million over four financial years to support regional NRM

organisations across Australia to revise existing regional NRM plans to help identify

where in the landscape adaptation and mitigation activities should be undertaken. This

stream was administered by the Australian Government Department of the Environment

(hereafter, ‘the Department’).

Stream 2 has $15 million over four financial years to produce regional level climate

change information and provide guidance on the integration of that information into

regional NRM and land use planning. This stream was also administered by the

Department.

The objective of Stream 2 of the NRM Fund was to improve the capacity of regional NRM

organisations to plan for climate change by improving the quality and accessibility of regionally

relevant information on climate change impacts and potential adaptation responses. Key

components of the approach to Stream 2 delivery include a focus on delivering to end-user

needs, with project teams required to work closely with the NRM clusters to identify and

prioritise the impacts information they need. A further overarching program focus was on

improving the way climate change information is delivered, managed and accessed by decision-

makers into the future.

Of the total $15 million funding some $13.63 million was allocated to ten major projects

(‘Stream 2 projects’) and an information management support project. Each Stream 2 project

has designated ‘Project Leads’. The projects include the ‘National Projections’ project

(supported by in-kind contribution from Commonwealth Scientific and Industrial Research

Organisation [CSIRO] and the Bureau of Meteorology [BoM]) and eight ‘Impacts and Adaptation’

cluster projects4 (Monsoonal North, Murray Basin, Wet Tropics, Central Slopes, Southern and

South Western Flatlands, Southern Slopes, East Coast and Rangelands) as well as one national

project focused on delivering regionally-relevant impacts and adaptation information at the

national level (the ‘AdaptNRM’ project). An additional project, the national information

management project, was added in June 2013 (implemented between 2013 and 2015).

The National Projections project started work in September 2012, while the 8 clusters and the

AdaptNRM projects started between the end of January and the end of April 2013.

4 In the original project design the National Projections project was referred to as ‘Element 1’, while the ‘Impacts and

Adaptation’ projects and ‘AdaptNRM’ were collectively referred to as ‘Element 2’.



2.2. Scope

This report presents the findings of the final Stream 2 evaluation. This evaluation covers the

entire program timeframe from 2013 to 2016 and investment ($15 million). The evaluation

focused on program impact, effectiveness of outcomes achieved, and legacy. The evaluation

has focused on the outcomes rather than the process of delivery which was considered in the

interim evaluation (conducted in 2014).

The purpose of the final evaluation is consistent with the original purpose outlined in the

Monitoring and Evaluation (M&E) Framework in 2013: to provide accountability, identify

learnings and to contribute to the knowledge base.

2.3. Purpose

The purpose of this final evaluation report is to:

Demonstrate progress towards program goals and objectives5

Share learnings between projects and clusters

Contribute to the knowledge of what works and why for future research and investment

decisions with regard to climate change science and NRM planning

Meet information requirements set by the Australian Government6.

This report specifically addresses these requirements by outlining the results of Stream 2

projects against the key evaluation questions (KEQs) of the Stream 2 M&E Framework.

2.4. Audience

The primary audiences for this final evaluation report are the Department and the Stream 2

project teams. The primary audience for the evaluation and their information needs are set out

below.

Table 1. Evaluation audience and information requirements

Who What do they need to know

Department of the

Environment

Effectiveness and value of the program outcomes

Appropriateness of the model

Impact and legacy

Learnings on research uptake and the model for connecting

research and planning

Stream 2 project teams Impact and legacy


Regional NRM organisation

representatives

Impact and legacy


Minister for the Environment Impact of Stream 2 on NRM planning: pathways to action

Legacy

Stream 1 program team Learnings from Stream 2 and its interactions with Stream 1

Legacy

Secondary users of M&E findings presented in this report include regional and cluster

representatives involved in NRM planning as well as a range of other external NRM

stakeholders including policy and research practitioners.

5 The goals and objectives for Stream 2 are outlined in Section 2.1. 6 Formerly stipulated by the Land Sector Carbon and Biodiversity Board (LSCBB).



2.5. Key evaluation questions

To address the purpose of M&E for Stream 2, a number of key evaluation questions (KEQs)

have been identified. The KEQs have been used to structure and inform all M&E activities at a

program level and cover the five categories of effectiveness, relevance, appropriateness,

efficiency and legacy.

The KEQs for Stream 2 of the NRM Fund are:

1. How effective was Stream 2 in achieving its intended outcomes?

2. To what extent was Stream 2 relevant to the context and needs?

3. To what extent was the implementation process appropriate?

4. How efficient was Stream 2?

5. How sustainable and enduring are the outcomes of Stream 2 likely to be?

A detailed methodology for undertaking this evaluation is provided in Attachment Five.



3. Findings

3.1. The effectiveness of Stream 2

This section will address key evaluation question (KEQ) one: How effective was Stream 2 in

achieving its intended outcomes? The intended outcomes of Stream 2 were to:

Improve the quality and accessibility of regionally relevant information on climate

change impacts and potential adaptation responses

Improve the capacity of regional NRM organisations to plan for climate change

Contribute towards government objectives of the Carbon Farming Initiative (CFI) and the

Biodiversity Fund in guiding the location of carbon and biodiversity activities in the

landscape.

Stream 2 has effectively met its overarching objective of delivering regionally-relevant climate

change information and support to NRM regions via eight cluster projects, the National

Projections project and the national impacts and adaption project (AdaptNRM).

This was supported by fit-for-purpose regionally-specific engagement. Within a broader shift

towards adaptive planning approaches more generally, the influence and value of Stream 2 was

highest in regions where this shift was already taking place. In these regions, e.g. Corangamite

(Southern Slopes), Goulburn Broken (Murray) and South West Catchments Council (South

Western Flatlands), Stream 2 has catalysed the transition towards adaptive planning

approaches, which recognise the dynamic, uncertain, inter-connected, trans-disciplinary nature

of NRM planning for climate change.

This section will address the overarching question of effectiveness by outlining the outputs

delivered by Stream 2 (3.1.1), the quality of the science delivered (3.1.2), the accessibility of the

climate change information provided (3.1.3), the use and application of Stream 2 products by

NRM regions (3.1.4), the influence of the program on regional NRM capability and capacity

(3.1.5) and research capability (3.1.6), as well as the contribution of the program towards

Government objectives under the CFI and the Biodiversity Fund (3.1.7).

3.1.1 Outputs delivered by Stream 2

Outputs delivered by Stream 2 consisted of knowledge products and engagement processes to

support the uptake of products, capacity development and ultimately improved NRM planning

for climate change (Figure 1). Between 2012 and 2016 fit-for-purpose product delivery and

capacity development were supported by a total of 268 engagement activities including

consultations, planning meetings and workshops to understand planner information needs.

A total of 958 information products and outputs were delivered by projects (including interim

and draft products), while a total of 183 next generation climate change projections products

were delivered to NRM regions and project teams. The National Projections were used by all

project clusters to develop communication products for NRM planners.



Table 2. Summary of outputs delivered by Stream 2 (2013 – 2016)

Indicator Activities

Outputs/products7

2013-2014 2014-2015 2015-2016 PROGRAM

TOTAL Sub-

total

Yearly

total

Sub-

total

Yearly

total

Sub-

total

Yearly

total

Research teams have

worked with regional NRM

organisations, through the

eight clusters, to identify

priority climate change

issues and the information

they will need to inform

planning (end year 1)

Consultations 98

268 268

Workshops 42

Presentations 42

Program

management

meetings

75

Other 118

Quality and timely climate

change information and

support is delivered to

regional NRM

organisations via the eight

clusters

Training 13

316

15

394

17

248 958

Products 1659 16510 5511

Consultations 67 60 6

Workshops 17 18 37

Presentations 31 48 45

Mentoring (e.g.

one-to-one

support)

15 66 6412

Peer reviews 8 22 5

Other - - 1913

Next generation climate

change projections are

delivered to regional NRM

organisation via the eight

clusters (one-off

deliverable) (year 2

through 4)

Training 1

16

4

138

2

29 183

Products 2 46 6

Consultations 10 37 6

Workshops 3 5 3

Presentations - 45 6

Technical

support - 1 6

N/A

Media articles 6

6

6

15

8

33 54 Academic

publications - 9 25

7 See Attachment Eight for a list of definitions for outputs and products delivered. 8 Central Slopes: reports; Rangelands: cluster meetings, presentations; Southern Slopes: newsletters 9 Summary reports 15; data sets 125; guidance documents 4; fact sheets 7; other 14. 10 Summary reports 29; datasets 15; guidance documents 3; fact sheets 38; webpages 33; other 47. 11 Summary reports 15; datasets 8; guidance documents 8; fact sheets 11; other 13. 12 One-to-one support: Adapt NRM approx. 45; Central Slopes 7; Murray Basin 10; Southern Slopes 1. The Wet Tropics

and Southern and South Western Flatlands clusters also reported providing many instances of phone and email

support to NRM planners. 13 Murray Basin: sub-project teleconferences 8; Southern Slopes: project management meetings 11.



2012-13

In the first reporting year (2012-13) Stream 2 activity primarily focussed on planning and

engagement. A number of engagement approaches were used to design, prioritise and plan

projects. Consultation was undertaken via a range of means including workshops, face to face

meetings, teleconferences and phone discussions. Workshops were the most common means

of consultation used by projects to identify user needs and requirements.

2013-14

In the second reporting year (2013-2014) impacts and adaptation projects broadened and

deepened engagement with NRM regions. At this stage, they also began delivering climate

change information and support to regional NRM organisations via the eight clusters and the

two national projects. During the year there were some 268 engagement activities conducted

by projects to identify priority issues for NRM planning; as well as the delivery of 319 outputs

during this period to deliver information to NRM regions.

2014-15

In the year 2014-2015 impacts and adaptation projects were primarily focused on delivering

climate change information and support to NRM regions via the eight clusters and the two

national projects. In 2014-2015 next generation climate change projections information were

delivered to regional NRM planning organisations, cluster projects and the National Projections

team. During the year there were: 141 outputs to deliver next generation climate change

projections, mostly through products, presentations and consultation; and 340 outputs to

deliver information to NRM regions, the majority of which were consultations, presentations and

summary reports14. An important output from the National Projections project was the launch

of the Climate Change in Australia website.

2015-16

In the final reporting year (2015-2016) projects primarily delivered advisory activities. As

expected, there were less products produced in the final year (55) most of which were summary

reports, guidance documents and factsheets. In the 2015-16 reporting year there were 64 one-

to-one mentoring activities for NRM planners and 45 presentations were delivered by project

teams. One-to-one mentoring and the presentations included support to develop content for

NRM plans, frame adaptation pathways and use climate change information.

Figure 1. Summary of Stream 2 outputs and outcomes (2013 - 2016)

14 See Attachment Eight for a list of definitions for outputs and products delivered.

http://www.climatechangeinaustralia.gov.au/en/



3.1.2 Quality of Science Delivered

Quality refers to the accuracy and degree of scientific certainty and rigour associated with

Stream 2 products as opposed to the useability and relevance of products (see Section 3.2).

Quality was assured, in part, through ongoing peer review, and editorial and quality assurance

processes undertaken in projects. It was also supported by the application and selection

process of projects undertaken by the Department, which took into account the scientific

credentials of proponents. By year two of the program a total of eight peer reviews had been

reported by four clusters to support the delivery of quality products. By year three, 22 peer

reviews had been reported from an additional two clusters. In addition to NRM planner reviews,

nine academic publications were reported for the 2014-15 reporting year. The number of peer

reviewed academic publications rose to 25 in the final year (2015-16) of the program.

3.1.3 Accessibility of climate change information

Stream 2 has reached a broad range of stakeholders with climate change information including

indigenous NRM groups and remote regional groups across Australia. Relevant information on

climate change impacts and adaptation was made more accessible to NRM organisations

between 2013 and 2016 through:

delivery of 958 outputs to support regional NRM organisations via the eight clusters and

the two national projects, including: information products (reports, data-sets, guidance

documents, fact sheets), training, mentoring support via workshops, presentations and

other delivery methods;

delivery of 183 outputs which translate next generation climate projections to NRM

planners via the eight clusters and the AdaptNRM project, including: information products

(web tools, journal papers, animations, spatial layers and data sets), presentations,

consultations and other delivery methods;

launch of the Climate Change in Australia website in January 2015 by the National

Projections project. By 22 June 2016 the web site had received some 153,746 unique

visitors with 1,011 registered users; while the AdaptNRM website has had 20,000 unique

page views (February 2016) since launching in June 2014.

Throughout the program, accessibility was also reported to have been supported by

engagement processes within clusters (see Section 3.2 for a discussion of engagement). At the

start of the program (2013-14), Stream 2 was considered an important source of information in

the majority of Stream 1 applications (39 out of 53 regions). This shows that many NRM

planners were aware of and prepared to engage with Stream 2 products at the beginning of the

program.

The NRM Information Management Project

Near the end of Stream 2 over 100 content items15 developed by Stream 2 projects were stored

in the Terra Nova16 repository with metadata descriptions to encourage their discovery. A

summary of the products stored in the Terra Nova web-platform is provided below in Table 3

below.

15 At the time of reporting (June 2016) the number of content items stored on the Terra Nova platform exceeds 100

products, which was the amount reported in the NRM Information Management final report in February 2016. 16 The Terra Nova web platform is an information hub established to store climate adaptation data. Terra Nova was

used to store information developed through the Stream 2 program. Website: https://www.terranova.org.au/

https://www.terranova.org.au/



Table 3. Stream 2 products stored in Terra Nova (February 2016)

Cluster Items Metadata quality17

Central Slopes 4 6/10

East Coast 32 10/10

Monsoonal North 8 7/10

AdaptNRM 6 6/10

Rangelands 7 folders 10/10

Southern Slopes 14 9/10

Southern and South Western Flatlands 6 8/10

Wet Tropics 7 7/10

3C Modelling (East Coast, Central Slopes and Murray Basin) 16 9/10

Murray Basin 6 6/10

(Source: NRM Information Management Project Final Report, February 2016)

Stream 2 products have been accessed widely through the Terra Nova platform. Table 4

provides a summary of the page views for project collection (per project) and individual content

(per item), which total 12,994 and 10,925 respectively.

While the Terra Nova database is one of many platforms for data storage and access in the

program, it was intended as the centralised repository for information. With open content and

an easily accessible web-platform, Terra Nova provides “high discoverability” as stated in the

NRM Information Management Project report. Because of the large number of draft and interim

products developed by Stream 2 projects, not all products developed through the program are

stored on the Terra Nova platform.

Table 4. Terra Nova page views

Cluster Collection page views Content page views

East Coast 2,989 2,562

Southern Slopes 2,313 1,940

3C Modelling (East Coast, Central Slopes and Murray Basin) 1,498 1,321

AdaptNRM 1,112 984

Central Slopes 1,085 833

Murray Basin 972 779

Rangelands 929 809

Wet Tropics 854 735

Southern and South Western Flatlands 657 556

Monsoonal North 585 406

Total views 12,994 10,925

(Source: NRM Information Management Project Final Report, February 2016)

The accessibility of National Projections

Since the launch in January 2015, the National Projections Climate Change in Australia website

has recorded 1,011 registered users and some 153,746 unique users, of which 22.6 per cent

were return visitors, who undertook 202,446 sessions averaging more than 12,100 sessions

per month (from 27 January 2015 until 22 June 2016). See Figure 2 for the total number of

users per month from the website launch in April 2015 to February 2016.

17 As rated by the NRM Information Management project team using criteria including: keywords and open licensing

marking.



Figure 2. Climate Change in Australia website users per month

(Source: National Projections Project)

A total of 122 client support requests were provided by the National Projections helpdesk to a

range of users from universities, NRM organisations and consultants being the main user

groups (see Figure 3). The helpdesk was supported by additional in-kind contribution from

CSIRO to support service delivery.

Figure 3. National Projections help desk client support groups

(Source: National Projections Project)

Although the types of requests fielded by the National Projections helpdesk varied widely,

assistance with NRM information was the main type of client service contact provided, with 41

per cent of requests relating to NRM. In a follow up survey of website users by the National

Projections team18 the survey found that nearly 60 per cent of users were “next-users; e.g. using

the information for impacts assessment”, while around 20 per cent were either “generators of

climate change information; e.g. climate researcher” or “end-user of climate change

information; e.g. planner and/or decision maker”.

18 Sent to over 800 registered users, the survey received 99 responses.



3.1.4 Use and application of Stream 2 products by NRM regions

At the end of the program in 2015-16, NRM planners are now applying and using a range of

Stream 2 products, tools and approaches in the NRM planning process. The extent of use and

uptake varies from region to region within each cluster. Factors which supported utilisation

include the timing of release and engagement, institutional capacity, planner capability,

linkages and ties to Stream 2 projects as well as peer support networks within and between

regional representatives. Due to the wide regional variation and national scale of Stream 2, the

actual extent of uptake has not been fully assessed19. However, based on Stream 1 reporting

and interviews with NRM representatives (n=20) it is estimated that around three quarters of

NRM regions have adopted, used and applied Stream 2 products in their planning processes.

Ultimately the use of products may not be manifest for some time to come, with several

planners stating an intention to use products, particularly the AdaptNRM Biodiversity Modules.

As would be expected, the use and uptake of Stream 2 products has largely taken place in the

final year of the program (2015-16) following an extensive process of engagement and support

in the preceding years (2013- 2015). At the mid-point of the program (2014), uptake of Stream

2 products was limited and primarily consisted of NRM representatives referencing and citing

products in NRM plan development. Product uptake and adoption tended to be stronger in

regions and clusters where there was a clear focus and strong social cohesion around a

particular issue (i.e. agriculture). By contrast, biodiversity uptake was less clear due to greater

uncertainty and lower levels of community support and institutional mandate.

Refer to the Northern Tablelands LLS case study below for an example where planning needs

were met through engagement by Stream 2’s Central Slopes.

19 Instances of uptake of Stream 2 products presented in this section have been opportunistically sourced through

interviews with a non-representative sample of NRM planners (n=20) and via a sample of Stream 1 reporting,

representing around three quarters of NRM regions.



Case Study: Product use – Northern Tablelands Local Land Services

The Stream 2 researchers from the Central Slopes Cluster Project engaged with Northern

Tablelands Local Land Services about research needs and developed products and outputs

relevant to core areas of planning and program activity. These included products for planning:

Determination of the impacts of climate change on future cropping (from APSIM

modelling)

Modelling of the impacts of climate change on native vegetation to identify priority

areas for revegetation and protection of native vegetation (3C modelling undertaken in

conjunction with AdaptNRM Project)

Development of a climate risk matrix for climate change adaptation planning. This

information was used to determine the impacts of climate change on livestock and

grazing (predominately). Information was integrated into state and transition models to

determine drivers of change and management actions required in response

Development of climate temporal analogues for use as a decision support tool to

investigate future climates and similar current climates

Extreme climate event information was integrated into state and transition models to

assist in identification of key drivers and key management actions

Information provided on the impacts of climate change on upland wetlands on the

Northern Tablelands. The information was integrated into state and transition models

The Reforestation Modelling Tool was used to estimate carbon sequestration amounts

in trees and debris pools. This information provided guidance on where the best

locations in the region are for sequestering carbon for mitigation.

The Central Slopes group (under Stream 2) have provided many opportunities for collaboration and

have provided support wherever possible. This list [above] is not indicative of how much support they

have given.

(NRM Planner20)

In the final year, use of Stream 2 products has matured and broadened with NRM

representatives using products for a range of purposes including background information, to

verify and confirm existing data, engagement and communication, NRM plan development and

spatial analysis. In some regions, there is evidence that Stream 2 products such as ‘Adaptation

Pathways’ are being used to guide thinking and strategic decision making for the management

of natural resources.

In many cases, NRM plans updated under Stream 1 were informed by Stream 2 products,

consistent with the intended objectives of the NRM fund.

Updating our strategy is the main work we have done with the information and products we received

[through Stream 2]. We have got the majority of the Stream 2 outputs into the strategy: [Stream1

plan, including] biosecurity, temperature and rainfall, and the aquatic species report have all been

incorporated. We are working on using what the community value combined with expert opinion and

then we put that through MCAS-S into our strategy.

(NRM Planner)

20 Where quotations or data references are included in the report, they are referenced as follows:

“NRM Planner” – A representative of an NRM organisation who has a role in planning, knowledge

management, etc. Most likely involved in Stream 1 activities

“Cluster Project Team” – Member of one of the project teams from one of the 8 clusters

“National Projections” – member of the National Projections team

“AdaptNRM” – member of the AdaptNRM Team

“The Department” – staff of the Department of the Environment



A total of 39 Stream 1 annual project reports were received for the reporting periods 2014-15

and 2015-16, of which 35 reported using Stream 2 products21. Out of the 35 regions who

reported using Stream 2 products a total of 85 tools and products22 were identified by regions.

Three quarters of the products and tools identified (63 out of 85) were used in the NRM

planning process as part of Stream 1. A variety of uses were reported by regions, ranging from

inputs into the planning process, as a tool for engagement and consultation to technical uses

such as vulnerability and impact assessment. Examples of the uses of Stream 2 products in

Stream 1 reporting are provided in Attachment Four.

A total of 51 regional representatives were contacted from 48 regions of which 16 regions

provided plans or planning documentation (i.e. technical reports). 12 of these 16 regions

provided NRM plans that had used Stream 2 products. Similar to the analysis of Stream 1

reporting above, the uses of Stream 2 products are summarised for selected projects in Table 5

below.

Table 5. Examples of use of Stream 2 product by NRM representatives

NRM Org. Cluster Stream 2 Products Use

Natural

Resources

Adelaide &

Mount

Lofty

Ranges23

Southern

and South

West

Flatlands

Implications of Climate Change for Biodiversity

(AdaptNRM), Helping Biodiversity Adapt

(AdaptNRM), Draft 3C modelling for biodiversity

management under future climate (Drielsma et

al, 2014), Southern and southwestern flatlands

climate change project: Data layers explained

(Ford et al, 2014)

Synthesis identifying

implications for NR

AMLR management

strategies

Eyre

Peninsula

Southern

and South

West

Flatlands

Hope P. et al. (2015) Southern and South-

Western Flatlands Cluster Report, Climate

Change in Australia Projections for Australia’s

NRM Regions: Cluster Reports, eds. Ekström, M.

et al., CSIRO and BoM Australia.

Global impacts of

climate change

Perth NRM Southern

and South

West

Flatlands

Neville, S. (2014). Planning for Climate Change -

MCAS-S Analysis Process, Perth Region NRM.

Consultant’s report for Perth Region NRM Inc.

Ecotones & Associates, Denmark., WA.

Joint MCAS-S

technical report

AW NRM Monsoonal

North /

Rangelands

Ninti One Ltd and CSIRO publications:

Davis J (2014), Scott JK (2014), Bastin G (2014),

Measham TG (2014), Pavey CR and Bastin G

(2014), Pavey CR (2014), Bastin G, Stokes C,

Green D and Forrest K (2014)

Fire risk impacts,

aquatic refugia,

vulnerability

assessment, invasive

species and ecology,

Buffel grass

management

CCMA24 Southern

Slopes

AdaptNRM Biodiversity Module Adaptation pathways

for NRM assets, Fire

risk impacts

In other regions planners reported that Stream 2 outputs had limited influence on their plans.

For instance, in the Murray Basin cluster a planner commented that there was “too much data”

and information that it was not possible to “interpret it in a way that is useful” at a regional

scale. The Southern and South-Western Flatlands Case Study below provides an illustration and

description of a synthesis report in the Adelaide and Mount Lofty Ranges (AMLR) region in

South Australia, which found that the products did not meet their needs.

21 Stream 1 reporting was provided by the Department for the years 2014-15 and 2015-16. Stream 1 reporting is not

representative of all regions because a limited amount of reporting was provided based on availability at the time of

reporting (March 2016). Reporting is also for different time periods (2014-15 and 2015-16) for different regions. 22 Some products identified included packages of information such as the AdaptNRM Modules (1 – 4). 23 See AMLR NRM Case Study for further detail. 24 See CCMA Case Study for further detail.



Case Study: Product applicability – Southern and South Western Flatlands

Natural Resources Adelaide and Mount Lofty Ranges (NR AMLR), South Australia

NR AMLR commissioned a review of the applicability of biodiversity climate change planning

products for on-ground NRM in the region1. The review was completed in October 2015 and

included review of three Stream 2 products: Product A (Ford et al, 2015) 2, Product B (Williams

et al, 2014)3 and Product C (Prober et al, 2015)4. The framework for review assessed the

degree to which each product’s findings could directly inform NR AMLR’s on-ground

management strategies and was substantially based on the perceived level of uncertainty in

each of the products, relative to the quantitative uncertainty associated with NR AMLR’s

management strategies. The review was very thorough and undertaken with the specific

region’s planning framework as a primary consideration in assessing the utility of any planning

product.

Products A and B may provide some guiding principles of relevance to the NR AMLR region’s

management strategies at the community and landscape levels because product uncertainty is

approximately equivalent to management decision risk. The review indicated that these

products ‘may’ have some usefulness for some aspects of regional management strategies but

are often limited by mismatch between the actual planning framework used in the region and

the planning framework assumed in the products, by the choice of data used in the models

(where the region believes more complete or superior data was available), and by the need for

additional information to be brought together with the product for further planning to be useful.

Product C was not considered to have direct utility for NR AMLR’s management strategies,

mainly due to uncertainty in the products findings and implications.

The mismatch between the Stream 2 products and the region’s planning approach and needs is

attributed to institutional barriers in knowledge brokering, some of which appear to arise from

the arrangements made to cluster Western Australian NRM regions together with non-

contiguous South Australian NRM regions.

…to make their (Stream 2) research useful for us it would have made sense to consult with us in the

process of designing their research methodologies (for example in selecting their data sources and

the parameters used for our region’s analysis), as we could have told them what would be more and

less useful for us from the outset.

(NRM Planner)

1Seed Consulting Services (2015) Assessing the applicability of biodiversity climate change planning products for on-

ground NRM in the AMLR region. A report prepared for Natural Resources Adelaide and Mt Lofty Ranges. Adelaide,

South Australia. 2Ford, B. & Cook, B. (2015). Southern and South-Western Flatlands climate change project: Data layers explained.

Report No CENRM 139. Centre of Excellence in Natural Resource Management, University of Western Australia. 3Williams KJ, Prober SM, Harwood TD, Doerr VAJ, Jeanneret T, Manion G, and Ferrier S (2014) Implications of climate

change for biodiversity: a community-level modelling approach, CSIRO Land and Water Flagship, Canberra. 4Prober SM, Williams KJ, Harwood TD, Doerr VAJ, Jeanneret T, Manion G, Ferrier S (2015) Helping Biodiversity Adapt:

Supporting climate-adaptation planning using a community-level modelling approach. CSIRO Land and Water

Flagship, Canberra.

Adoption of planning frameworks and approaches

Adaptation pathways is an example of an approach that was been adopted by a number of

regional bodies in the Stream 2 program. It is “an analytical approach to planning that explores

and sequences a set of possible actions that are based on external developments over time”

(Haasnoot et al. 2013:485 cited in Bosomworth et al, 2015). This evaluation has found that the

Adaptation Pathways approach has had an influence on NRM planning approaches within

regional bodies in the Southern Slopes, Murray and Wet Tropics clusters.



Case study: Application and uptake of the Adaptation Pathways approach

The adaptation pathways approach was utilised in the Southern Slopes cluster by a joint effort

between the research teams and the NRM regions in the cluster who were actively involved in

piloting the approach. In the Corangamite region, NRM representatives played a key role in

developing and refining the approach with the research team.

As a process now we won’t use anything else. It allows us to put climate change at the centre of our

decision making; it is through the whole planning process now.

(NRM Planner)

In the East Gippsland region in eastern Victoria, the adaptation pathways approach was used as

part of the planning process to inform community discussions over contested resources.

Adaptation pathways [enabled us to] focus on places the most contested landscapes in the region

and get people working together to flush out the issues where there would be a contest for

resources. We used it [adaptation pathways] in a workshop with stakeholders. They key was you

could draw it on a wall with issues on the vertical axis and time on the horizontal axis. By drawing

lines on a grid like this it became clear to people that there were multiple pathways to the same

result. For instance, you could jump off this track and get on that track as a result of changes in the

issues or timing. As a result of this process we got better thinking. This was useful in difficult

discussions in our public consultation.

(NRM Planner)

Meanwhile, in the Wet Tropics cluster the approach was influential in informing the structure

and “core” of the Cape York NRM plan:

We are now looking at the core of the [NRM] plan being those adaptation pathways. In everything

we’re planning for we’re adapting to change and identifying adaptation pathways. One we know of is

fire. The entire Cape York is burnt out each year. We’re moving towards a more targeted approach.

Once we reach a point it leads to change in the system. But if you burn correctly you can reduce the

impact it has on heat, etc.

(NRM Planner)

In the Murray Cluster a project team member referred to the approach as being part of a

broader “paradigm shift” in the way that NRM bodies approach and undertake regional NRM

planning (see 3.5 Legacy). While uptake of the adaptation pathways approach was relatively

concentrated to a minority of regional bodies in Southern Slopes, Murray Basin and Wet Tropics

clusters, there is no doubt that it was an influential approach across the Stream 2 program.

Application of AdaptNRM products by NRM representatives

This evaluation has found specific examples of application of AdaptNRM modules by NRM

planners. Interviews with NRM representatives (n=20) highlighted that there was variable

uptake of AdaptNRM products by NRM regions with some regions being able to provide

examples of how the AdaptNRM products were or could be used while others were not able to

do so. Unlike the National Projections project that targeted products at both cluster projects and

regional NRM representatives, the AdaptNRM project developed products specifically for use at

a regional level.

The AdaptNRM Biodiversity Module in particular was cited by many planners as a tool that they

had used or intended to use. A Greater Sydney LLS planner in particular stated that the



Biodiversity Module was “transformational” in its emphasis on change and continual adaptation

as a principle for biodiversity management.

The AdaptNRM module on biodiversity was transformational in my understanding of climate change.

What it emphasised for me was the need to change our thinking. AdaptNRM gave us a series of

modules about how we should be approaching future management. These modules are simple in

concept but complex in application. For example, one of the modules was about maintaining the

evolutionary character of biodiversity. These modules have an implied management principle: to

implement management needs to become more interventionist. This is a move away from how

things have been: we have been provoked to move from thinking in a stable to a moving world. We

need to plan not just for conservation but for change through time.

(NRM Planner)

As the case study below shows, in Corangamite the AdaptNRM Biodiversity Module was also

used as a tool for community discussion, which led to an elevated and more detailed

understanding of the fire risk to the Otways when ‘ground-truthed’ with community knowledge.

This demonstrates using the module as a tool for catalysing discussion as an input into the

draft regional Climate Change plan25. It also provides a useful example of the application of an

AdaptNRM module to regional NRM planning.

Case Study: Applying AdaptNRM in the Otways

AdaptNRM arrived at the right time in 2014-15 when there was no support or direction on

Biodiversity from the State Government [Victoria]. Throughout the project we have provided

feedback and input into AdaptNRM modules. All our feedback has been taken on board and has

been reflected in the revised product. AdaptNRM work has directly informed the region’s draft

climate change plan. AdaptNRM did a case study of the Otways, which was used directly in the

draft climate change plan. AdaptNRM provided us with a case study of temperate rainforest in

the Otways based on one of their modules. The study showed that temperate rainforest is

projected to constrict by 2060. We showed it to our experts and they said that taking into

account bushfire risk, with hotter, more intense fires a more likely scenario is that the rainforest

will disappear altogether by 2060 rather than constrict, which is what the AdaptNRM module

predicted. So what the module provided was a basis for discussion, from which we could make

a fuller assessment of the likely impacts on the rainforest by 2060. The study [undertaken by

AdaptNRM] is written up in the draft climate change plan taking into account both the module

and the expert review.

(NRM Planner)

Application and uptake of the National Projections

The National Projections were delivered to NRM planners through direct engagement via a

series of national workshops, consultations and presentations by the National Projections team.

In addition, NRM regions also received the projections work indirectly through the use of

projections in cluster project products and tools. Evidence collected for the evaluation suggests

that, due to the concurrent timing of delivery of the cluster projects with the National

Projections project, NRM planners were more likely to have received projections work directly

from the projections team directly rather than through project products and materials.

The most important tool I have used is the CSIRO projections. Without understanding what the

climate looks like we can’t do anything.

(NRM Planner)

25 As the Draft Climate Change Plan for Corangamite was still under review at the time writing it was not possible to

include an excerpt in this report.



Project reporting indicates that the National Projections have been used by all project clusters

to develop communication products for NRM planners. In total 183 next generation climate

change projections products have been delivered to regional NRM planners via the cluster

projects (or directly to planners from the National Projections team).

This is the first time [in Australia] that we have delivered a set of projections that provide much

better access to the relevant information for end users. There is evidence that it is being used by

next users… the information is being used in impact assessments by researchers to inform adaption

planning or update existing NRM plans.

(National Projections)

When interviewed the majority of project leads (6 out of 9) provided further details on how they

(AdaptNRM and the clusters) had used National Projections data. The use of the National

Projections varied within and between projects depending on the needs of the planners in their

regions. At one end of the spectrum summary information from the National Projections was

reported to be sufficient for cluster projects to meet planner’s needs (e.g. by developing

communication products and inputs to reports).

We actually produced an indigenous brochure for consultation – used in schools and public audience

as well [drawing on the projections].

(NRM Planner)

We used [National Projections] summary materials as inputs to some of our outputs (e.g. Info Portal

Report cited projections summary information)

(NRM Planner)

The National Projections were used by regional NRM representatives in the development of

NRM plans to provide context, background information and to validate existing climate data

(where it existed). Some regions (e.g. Eyre Peninsula) commented that the National Projections

outputs were the most useful information that they had received through the Stream 2

program. Other regions stated that they had used the projections to facilitate community

discussions such as in the Greater Sydney region and in the Alinytjara Wilurara case study

below.

We used the projections [in community workshops] to communicate what bushfires might look like

in the region going forward. The reason for the workshops was to look at where the community is

vulnerable and manage the risk to people and property. We came up with some interesting stuff

around balancing ecological values [why people live in vulnerable areas] and management actions

for safety [in a changing climate].

(NRM Planner)



Case study: Applying projections in the Alinytjara Wilurara NRM Region

We had a quick look at the communications products but they are not relevant for our region. It

is very remote; most people did not speak English and given people struggle with so many other

areas of their lives NRM was not the most important issue. We did not want to portray too much

of a dark situation [about climate change projections]. We had to be quite careful about how to

engage so we used a two-way approach. We used the information and simplified it in some

ways [pictorial representations] to trigger conversation. We didn’t use projections much [in the

plan] but rather for community consultation. If they raised an issue, then we would bring along

information on that. We were trying to see it from their angle; what they knew and then provide

more information as a result. It was being said by them and not us in the first instance.

(NRM Planner)

Several projects also cited using various modelling products (e.g. for carbon sequestration, sea

level rise, agricultural productivity, species distribution etc.) which were distributed to NRM

planners from the clusters using the projections data.

We find the projections extremely useful… In our project we used a lot of species distribution

modelling [which integrated the projections work].

(Cluster Project Team)

We used the projections work in the science to predict things like agricultural productivity and sea

level rise. We used their materials and they went straight to the NRM groups.


Where existing projections were available, such as in Tasmania, planners used National

Projections outputs to validate and confirm projections made in previous and current NRM

plans. While some regions and clusters used different climate data-sets such as NSW / ACT

Regional Climate Modelling (NARCLiM) and the existing Tasmanian climate projections

conducted by the CSIRO in 2007.

3.1.5 Planner capability and capacity

NRM planners report improved capability and capacity to use, apply and interpret climate

change information and tools as a result of their involvement in Stream 2. The influence on

planning capacity varies from region depending on the organisational context, level of turnover

within the organisations, the background, experience and expertise of the NRM representatives.

In addition, broader factors such as access to services, remoteness and proximity to high

population centres, where the research teams tend to be located also influenced capacity

development. Also, it is important to recognise that NRM organisations were operating from

different levels of capacity prior to the program, which has necessarily influenced the capacity

gains made by Stream 2 across the country.

The principle factor in the program that influenced the capability of NRM representatives was

the strength of relationships and ties to researchers. Where planners reported a good

relationship and direct access to researchers (i.e. being located close to researchers) they were

more likely to report improvements in their climate change knowledge, understanding and

ability to apply Stream 2 tools and approaches. In short, close proximity to research teams

enabled greater opportunities for face-to-face contact via informal meetings as well as



structured activities (i.e. workshops). By contrast, NRM representatives geographically isolated

from research teams were less likely to build strong connections.

All national and cluster projects report having increased planner capabilities to plan for climate

change. Project leads discussed having contributed to increasing two main capabilities for

planners. Firstly, increasing planning capability to incorporate uncertainty and adaptation

pathway approaches and resilience thinking into planning for climate change (AdaptNRM,

Murray Basin, Monsoonal North, Southern Slopes, Southern and South West-Flatlands).

The main outcome that I would identify is that they really get that this is about decision making

under uncertainty… At the start the NRM planners would have said that it was about data, they were

trying to make it a predictive, problem solving exercise. Now they are saying “how do we structure

our decisions in pathways and look at multiple futures?” They understand that adaption is about

decision making and how you use information.

(AdaptNRM)

In addition, project leads report that the program has empowered planners to communicate

climate science in their own regional and organisational contexts.

Those involved with the project feel a bit more confident to talk about climate change with people

because they have climate change information for the Rangelands. It’s much easier to speak about

what might and might not happen.


Planners also noted improvements in their ability to communicate climate change concepts

and issues with greater clarity and ease as a result of Stream 2. In some cases, capability to

communicate was enhanced by the confidence of being backed up and supported by perceived

“credible” science and national institutions.

Prior to my involvement I was a bit frightened about climate change. For instance, prior to the

project I didn’t know how to start a conversation with a farmer. Literally there were rooms that I

would enter and could not talk about climate change but now we can discuss it. Before it was like

climate change was off in the distance and all we could do was acknowledge it. There are so many

good examples that I can use to communicate now; i.e. grain growers in Victoria who have a reduced

harvest window etc., Tomato growers, Viticulture etc. There are some solid pieces of information that

are in my anecdotes that I can draw on.

(NRM Planner)

Regional representatives stated that they had learned about how to apply and interpret climate

data as a result of Stream 2 (for example see Murray CMA [MCMA] below). A Goulburn-Broken

CMA representative for example commented that “I’m much more confident with interpreting

climatic data and we are more aware of what uncertainty means in relation to Climate Change

as planning. [As a result] adaptation planning as a theoretical concept has trickled down

through the organisation: we’re getting better at it”. Likewise, the Mallee CMA representative

commented in relation to the influence of the National Projections.

It’s definitely been useful for me to get an understanding of how the projections might hit the

ground. But at the same time, learning about the limitations of the certainty of what we can and

can’t say [has] helped me to communicate to other punters out there about what climate change

might mean in the region.

(NRM Planner)



The majority of planners identified challenges in applying Stream 2 outputs. Some commented

more pointedly that they had difficulties in understanding and applying Stream 2 outputs.

The projections were not quite there and so we used small scale research on particular NRM issues

which were more useful than the projections themselves – like the impact of climate on agriculture

and pests. Applied science was more useful for a planning and communications tool than

projections which we only included in the technical reports.

(NRM Planner)

A minority of NRM representatives commented that Stream 2 has had limited influence on their

ability to plan. For instance, an Eyre Peninsula representative commented that Stream 2

outputs were “too technical” and beyond the capability of the organisation. In the case of Eyre

Peninsula, low uptake should be be seen in the context of turnover in the Southern and South-

Western Flatlands project team (within the South Australian Department of Environment, Water

and Natural Resources [DEWNR]) as well as relative geographic isolation from the project team

(see Section 3.3 for a discussion of cluster model).

3.1.6 Researcher capability

All project representatives interviewed (n=13) indicated that having an explicit focus on end

users throughout the life of the Stream 2 program had built their capability to deliver products

to meet the needs of NRM planners. Interviewees indicated that many researchers who have

been part of Stream 2 have gained and or enhanced their capabilities to: engage and

communicate with NRM planners, communicate climate change science to NRM planners, be

flexible and responsive to changing needs and contexts, manage expectations about what can

and cannot be delivered, and ultimately deliver products that respond to NRM representative’s

diverse needs and regional contexts.

Some people in our team [have gone from] fairly typical scientists to putting themselves in other

people’s shoes… They have developed a better understanding of how regions work, how planning is

done, the level of information that they need and the formats that they require it in. For some people

it has been part of a broader shift and a really significant broadening [of] skills and capacity.


Similar to planners, researchers also stated that they had developed their ability to

communicate as a result of Stream 2, as highlighted below in the East Coast and Central Slopes

clusters.

We all learned a lot about presentation, where we are pitching it, they generally don’t want too much

data, but we learned more about pitching and providing information. We learned about what

concerns them and their competing concerns. For instance, what we were doing is a small concern

[for them]. How complex the space is that they work in.


The main change has been that some scientists have developed more of an ability to speak in plain

English… They had to think about what NRM planners need to do their job. They got better at the

hard sell. It has come about through dialogue with the NRM planners, their context, constraints and

capacity…


Researchers, such as the Southern Slopes team, also noted that they had developed their

understanding of regional NRM issues and priorities.



We have a greater understanding of the current NRM situation in this region, though recognising that

the situation changes rapidly. Ability to work with NRM planners was enhanced, and some of the

team are continuing to explore these relationships through other projects


A highlight cited by all project leads was the strengthening of relationships between researchers

and NRM planners, particularly being able to develop a mutual understanding of each other’s

needs, priorities and working contexts. For the National Projections team, being focused on

planning needs with a clear purpose and mandate has increased their capability to work with

end users. More broadly has contributed to a shift in thinking within the CSIRO and the BoM

about the way climate projections research should be done.

At the start of the program we had a couple of researchers here who were open about not seeing the

value of end user input into our work... The NRM work [via Stream 2] has consolidated our end user

focus… getting to know the users is really valuable…


This focus on end users and engagement by the National Projections team has built on capacity

developments achieved from previous programs delivered by the CSIRO and the BoM in the

Pacific26, leveraging investments made by the Australian Government in climate change

science and adaptation.

I think that there is a greater understanding of the challenges associated with delivering climate

change information to user groups in an Australian context and we appreciate more the reality of

what it is like to be on the land… I don't think that the context of the research we undertake has

been this close to the user groups before in Australia; we have gained the capacity for a new focus

on supporting next users in Australian context. This kind of work has been undertaken in the Pacific

Island Nation context, applying it in the Australian context bought a new set of challenges but that

has led to new ideas and tools and a new understanding of the types of tools and products that are

used in NRM planning, we have learnt more about how to plug into their models.


AdaptNRM and cluster project leads reported that increases in researcher capability to meet

the needs of planners were underpinned by relationships that have strengthened over time

(East Coast, Wet Tropics, Rangelands, Southern Slopes).

The highlights have been the enhanced interaction between researchers and planners. We all got a

lot out of that. Some of us, as researchers, don’t have direct input into NRM, planners particularly;

we definitely got more educated in the process. We understood what their needs were. That was

quite illuminating.


It’s become much more collaborative… the researchers would have developed a much better

appreciation of how NRMs work, what types of information can be used, what constraints they have

to work with. I also think the corollary is also true that NRMs have developed a much better

appreciation for the constraints and potential the scientists are experiencing as well.


26 Via the Pacific Climate Change Science (PACCSAP) program.



3.1.7 Supporting government objectives

Ultimately the investment in Stream 2 aims to contribute to updating NRM plans that guide the

location of carbon and biodiversity activities that take into account climate change impacts and

adaptation responses. Because Stream 1 of the NRM fund was designed to directly update the

development of current NRM plans, Stream 2 necessarily interfaced with Stream 1 to achieve

this objective. With changes in the direction of Australian Government policy on climate change

during the period of delivery between 2013 and 2016, the Stream 2 program was required to

continue delivery of outputs despite uncertainty over the possibility of a carbon market and

economy.

Stream 2 has supported the achievement of government objectives by providing spatial

guidance for biodiversity and carbon offsets under the Carbon Farming Initiative. The link

between Stream 2 outputs and government objectives was more apparent in some clusters

than others due to the focus and emphasis of their work. For instance, the East Coast cluster

included a carbon farming sub-project.

The 2012-2013 annual review identified a clear link and alignment between Stream 2 end-of-

program outcomes and broader goals articulated in the Clean Energy Future (CEF) Plan and

Land Sector Package objectives. Stream 2 of the NRM Fund was designed by DCCEE (now the

Department of Environment) and established by the Australian Government with the aim of

contributing to the CEF Plan. More specifically, Stream 2 was designed to contribute CEF Land

Sector Measures including the Carbon Farming Initiative, the Biodiversity Fund and the

Indigenous Carbon Farming Fund.

In year two (2013-2014), some Stream 2 projects (3 out of 8) had made small, yet potentially

significant steps towards Land Sector Package objectives, including: consultation, project

development and spatial analysis being taken towards informing the location of carbon and

biodiversity activities. In years three (2014-2015) and four (2015-16) half (four out of eight) of

the projects had undertaken explicit activities to contribute towards government objectives of

guiding carbon and biodiversity activities. Of these projects, East Coast and South-Western

Flatlands provided clear examples of activities to contribute towards the Emissions Reduction

Fund (ERF), as detailed below.

East Coast: compiling biophysical, economic and other data for carbon farming

assessment; attending meetings with Commonwealth Government representatives

regarding new Emissions Reduction Fund framework and the implications for carbon

farming (2014-15). Initiating a project to develop a spatial analysis tool to identify the best

areas for soil and vegetation carbon sequestration with three NSW LLS’. Finalisation of the

Queensland Government Carbon Farming report. Development of a web-based tool to

visualise carbon farming opportunities from forest regrowth. Development of a joint-project

between consortium researchers to realise opportunities in the ERF (2015-16).

Southern and South-West Flatlands: use of the MCAS-S spatial planning tool by South

Coast NRM (SCNRM), South West Catchments Council (SWCC) and Northern Agricultural

Catchments Council (NACC) to identify sites for carbon planting (2015-16).

Taking into account the project design as well as interview feedback from project leads the

other clusters’ Impacts and Adaptation projects did not explicitly focus on contributing towards

the ERF largely because it wasn’t prioritised by the NRM regions.



3.2. The relevance of Stream 2

This Section addresses KEQ2: To what extent was Stream 2 relevant to the context and needs?

The relevance of Stream 2 will be addressed by considering the engagement approaches used

in the program (Section 3.2.1), the extent to which end-user needs were met by the program

(3.2.2), as well as mechanisms for integrating user feedback into the research process (3.2.3)

and providing support to targeted regional NRM organisations in the program (3.2.4).

The process of engaging and working with the users of products and stakeholders involved in

NRM planning (i.e. other researchers and NRM stakeholders such as NRM body CEOs),

supported the delivery of fit-for-purpose regionally relevant NRM planning products.

There were a wide and extensive range of engagement activities undertaken by Stream 2

projects to understand user-needs and to build the capacity of NRM organisations to plan for

climate change. The quality, accessibility and usefulness of the products and services delivered

through Stream 2 were strongly underpinned by the engagement activities which focused on

both understanding and responding to the needs of NRM planners.

3.2.1 Engagement

As outlined in Section 3.1 and the Stream 2 output summary more than 268 engagement

activities and processes were delivered by projects across eight clusters to support product

development, design and uptake between 2013 and 2016. A range of engagement processes

and approaches were undertaken by projects. The Stream 2 Interim Evaluation (2014) found

that there was a variety of engagement models being used by Stream 2 projects. In addition to

engaging with NRM bodies, cluster projects also worked with, and alongside, other cluster

projects, the National Projections project, the AdaptNRM project, and a range of other federal,

state, regional and local government agencies in addition to research institutions, non-

government organisations (NGOs) and industry organisations.

As projects progressed into year two (2013-14) and three (2014-15) engagement variously

became more targeted, and purposeful as project teams, intermediaries (next users of research

outputs whom project teams directly interface with to support uptake and use of Stream 2

products by NRM bodies) and end-users (NRM bodies) became more familiar with each other’s

needs and capabilities27. Where regions experienced turnover in end-users (i.e. planners) or

intermediaries (i.e. knowledge brokers or consortium partners), the process of uptake and

engagement was made much more difficult and challenging. Similarly, where institutional

changes took place external to the program (i.e. in the restructure of regional bodies in NSW),

these changes disrupted the continuity of engagement and in many cases undid the gains

made by project teams. During this period of restructuring, NRM representatives were either

absent, removed or distracted by the institutional changes taking place.

Another challenge in engagement was attaining sufficient coverage across wide, geographic

areas. This challenge was noted by both the National Projections team and the AdaptNRM

project who engaged across all regions and clusters. At the end of the program all project leads

interviewed (n=13) reiterated that the quality, accessibility and usefulness of the products and

services delivered through Stream 2 were strongly underpinned by the engagement activities

which focused on both understanding and responding to the needs of NRM planners. These

project leads indicated that the process of engaging and working with next users allowed them

to deliver products that have been integrated into NRM planning.

27 Refer to Attachment Eight for a glossary of terminology used in this report.



Tailored engagement

The diversity of needs across NRM organisations meant that engagement and consultation

needed to occur in a tailored way. This resulted in different engagement approaches for

different clusters, project teams and NRM groups. There were mixed results from different

engagement approaches, the common theme from which was that tailored approaches at the

regional body level were most effective. There were some overarching lessons to be drawn from

across the projects, reflected in the comment below:

Creating effective, collaborative relationships both among the Southern Slopes Climate Change

Adaptation Research Partnership (SCARP) team and with NRM agencies has required lots of time

and prompt responses, face-to-face contact, listening, good intentions, and a healthy dose of

humour. One size does not fit all.

(Southern Slopes, Annual Report 2013-14)

Largely, the implications of the diversity of NRM needs led to a recognition of the need for (a)

multiple scales of engagement and, (b) multiple modes of engagement (e.g. a mix of face-to-

face, telephone and web based interaction). Engagement needed to occur within project teams,

across multiple regional NRM organisations at the cluster scale and then, sometimes, at varying

scales within NRM bodies (i.e. at the executive as well as operational level/s). A common theme

across the project reports was the need and preference for some face-to-face interaction, in

combination with a range of other modes as suggested by AdaptNRM.

Multiple modes of engagement need to occur to cover the range of NRM planner needs. While, face-

to-face activities are generally much more preferred and effective, particularly for achieving

interactive, peer-to-peer learning and for larger groups (>10) of people who are unfamiliar with one

another, other forms of engagement such as online forums, tele/video-conferences, group emails

and individual consultations (phone/email) can be very effective for particular purposes.

(AdaptNRM, Annual Report 2014-15)

Models of engagement

Project self-reporting indicates that the variety of engagement approaches was effective in

engaging and supporting regional NRM organisations to plan, deliver and take up planning

products, resources and tools. A range of specific observations about effective engagement

were made by projects. A common theme in these observations was that the means of

engagement need to be targeted to the needs of intermediaries (cluster representatives,

researchers and other stakeholders) and end-users (NRM representatives)28. In some projects

this was facilitated by centralised, coordinating entities such as a Project Working Group (East

Coast) and a Brokering Hub (Wet Tropics). The variety of modes of engagement used by

Stream 2 projects is summarised in Table 6 below. While there was a wide variety of

engagement modes and mechanisms used by project teams, a common approach for projects

was to use one individual or lead as a focal point for engagement with NRM groups.

28 Refer to Attachment Eight for the definitions of terminology used to describe ‘intermediaries’ and ‘end-users’ etc.



Overall, this evaluation found that the following engagement strategies were useful in

supporting uptake of Stream 2 products and resources by NRM groups:

regular face to face workshops and meetings across the whole cluster to share

information and progress projects

provision of a knowledge broker or similar position that had a defined role in the

project for coordinating all interactions between NRM groups and researchers

one-on-one work with NRM groups to ensure the cluster products, tools and

resources were relevant, understood and accessible

co-design of products, tools and resources to ensure that they met the needs of local

users; i.e. where NRM groups were involved in the formative thinking and design of

products (e.g. providing input into early decision making in product development) as

well as in subsequent stages of development (e.g. through review)

direct provision of advice from researchers where NRM groups requested it

work across clusters via meetings, teleconferences and/or workshops involving

planning representatives from each NRM group.

The devolved approach to engagement taken across the program resulted in a range of

engagement approaches. A summary of the different engagement methods used by projects to

understand and respond to user needs is provided in Table 6 below.

Regardless of the engagement approach taken, interviews with NRM representatives and

project researchers showed that the real value and benefits of engagement often occurred

informally as individuals and teams began to get to know one another better and develop

rapport, trust and reciprocity, which in turn enabled a more productive working partnership

between research producers and users. This is not to downplay the importance of formal,

planned engagement, which typically provided a basis for developing, maintaining and

strengthening relationships.



Table 6. Modes of engagement used by Stream 2 projects

Project Engagement methods29

National

Projections

National Projections User Group

Co-development of products

1:1 project engagement

Presentation, feedback and consultation on outputs

Engagement with NRM regions via eight Cluster Coordinators

User workshops

AdaptNRM Module review process

User workshops

1:1 project engagement

1:1 NRM group engagement (not all NRM groups)

Central

Slopes

Joint project team comprised of researchers with NRM body representation

Engagement lead

1:1 NRM group engagement

User consultation and workshops

Monsoonal

North

Project steering committee with NRM body representation


Presentation and consultation

Rangelands Employment of an Engagement Officer to distribute information and share

outputs

Scientific advisory committee


User consultation and workshops

Southern

Slopes

Joint project development

Workshops, interviews, focus groups to identify needs

Project Steering Committee

Use of a live information portal to gain feedback


East Coast Planners Working Group (PWG)

Establishment of a “Community of Practice”


Southern and

South

Western

Flatlands

Workshops to identify needs and gain input from NRM representatives

Appointment of NRM group liaison point

Product review


Wet Tropics “Brokering Hub” established to facilitate engagement

Co-development of information tools

Engagement lead

Identification of shared goals


Consultation on products during review

Joint-publications


Murray Basin Engagement lead



Exit interviews with NRM groups

The Wet Tropics “Brokering Hub” provides an example (see Case Study below) of a formal

engagement mechanism established by the project team to connect researchers with planners

in a joint partnership throughout the project. The “Brokering Hub” comprised of a focal point for

engagement, which facilitated connections between researchers and NRM groups via series of

29 Note that because the methods presented are indicative because they are drawn from self-reporting and interview

data. As a result, the engagement methods listed may not be representative of the full range of approaches used by

projects.



structured engagement activities and forums as well as ongoing, regular contact with

individuals in NRM groups.

Case Study: The Wet Tropics “Brokering Hub”

It was through the brokering hub process that we collectively identified projects to be

developed... At first we were asked what products we wanted so the whole process was a little

back the front. We definitely had the opportunity in the beginning to provide a list but in the

initial year there was not a lot of interaction with Stream 2 so they didn’t meet our needs at all.

It has improved enormously [since then], with a more active brokering hub and discussing what

we do with the products. The engagement with the scientists and having a knowledge broker

has been critical in this process. The concept of an end-user is a silly one. There is no use being

part of it at the end. We are a partner and participated as a partner. I don’t think the products

would have been as good if we hadn’t been a partner.

(NRM Planner)

3.2.2 End-user needs

By purposively engaging with NRM representatives and providing targeted information to

address NRM planning for climate change needs, Stream 2 has met the needs of NRM groups

not previously addressed. However, the extent to which end-user needs have been met varies

across the regions and clusters. This is in part due to the complexity of NRM planning for

climate change and the fact that ‘needs’ are context dependent (i.e. may vary according the

individual and organisation involved), value-laden and often contested. In this dynamic context,

Stream 2 has made a concerted effort and clear gains in addressing planning needs.

Where a clear focus and objective for engagement was present around a resource of common

value, such as agriculture, end-user needs were more likely to be met. In addition, as noted in

Section 3.1, end-user needs were also more likely to be met in regions where there was strong

existing capacity and ability (i.e. sufficient resources, time, capability and expertise), a mandate

and support to engage, as well as linkages and proximity to the project team.

The ability to meet end-user needs was in some cases hindered by a lack of flexibility in

contracting arrangements, which committed project teams delivering work in groups and

‘consortia’ comprised of multiple partners before the needs of end-users were understood. For

instance, in the East Coast cluster a consortium of six project research partners was

established. Feedback from East Coast project representatives suggested that the consortia

approach hindered the responsiveness of the team due to the large number of partners

involved and because of the need to consult across all partners before acting on opportunities.

An inherent issue in Stream 2 was the tension between needs versus wants. For instance, what

was ‘wanted’ by an NRM region could in fact differ from what their ‘needs’ were depending on

what stakeholder values were at play (both within a cluster, project or region). This tension was

manifested in engagement activities, often early in the program, where project teams and

cluster representatives negotiated what would be delivered.

‘Impacts and adaptation’ cluster projects

All clusters commented in Annual Reporting (2013-14) that they had purposefully engaged with

NRM managers from the outset to ensure they would meet end-user needs. In 2014-15, three

clusters had also begun to develop an evidence base (through formal surveys and interviews) to



demonstrate that the products generated were meeting end-user needs, specifically citing NRM

feedback about “usefulness” and “useability” (Southern Slopes, Central Slopes and East Coast).

However, throughout the course of the program, all clusters reported difficulties in fully meeting

end-user needs.

A key issue reported throughout the funded period by Stream 2 projects was the timing of

Stream 1 and Stream 2 funding. Several project leaders commented during the period of

delivery that Stream 2 should have been implemented prior to Stream 1 rather both Streams

being implemented concurrently. They argued that sequential timing would have provided the

necessary data and research (Stream 2) to update plans (Stream 1). However, as highlighted in

Section 3.3.1 (see Case study: Co-learning between regions) and 3.2 (Support) there is a

counter-argument to this perspective that Streams 1 and 2 were mis-matched; that the

concurrent timing of Stream 1 and 2 in fact supported co-learning among researchers and

practitioners by forcing them to work together to address the planning and research challenges.

National Projections project

The National Projections team engaged with NRM groups, researchers and the Department to

determine user needs prior to starting research. This early engagement process involved

establishing a User Panel and eight cluster coordinators, undertaking regular communication

with cluster representatives and running nine user workshops between March and May 2013 in

eight regions. This facilitated a shared understanding of user needs, research capabilities and

available resources, and an opportunity to co-develop products and manage expectations from

the outset.

The National Projections were received very well by NRM representatives interviewed for this

evaluation who found the information highly relevant and at an appropriate scale. The National

Projections user survey also found that engagement and delivery approaches taken by the

project were considered effective by between 85-90% of respondents. Project representatives

also uniformly praised the quality of the work and scientific output produced by the CSIRO

projections team, regarded by cluster project representatives as “world-class”.

The National Projections were pitched at the right level. They provided various climate scenarios and

time periods; gave us key messages and generalities for planning underpinned by rigorous science.

It was a well written document; we didn’t have to apply the research.

(NRM Planner)

This positive response to the website from project teams was reiterated by the Projections team

who have received feedback from project teams.

All of the feedback on the website has been positive. People appreciated the unprecedented access

to data, information at a scale that was more relevant to their needs, support and guidance to use it,

the elements of access and guidance were not available in our previous release in 2007.


In the survey of Climate Change in Australia website by the Projections team the majority of

respondents rated the usefulness of the information highly in terms of meeting their needs;

with around 44 of 99 users rating content as 4 out of 5 and a further 27 of 99 respondents

rating the content as 5 out of 530.

30 Where 1 was ‘Strongly disagree’ and 5 was ‘Strongly agree’.



In interviews with project leads the extent to which the National Projections project met

different NRM planner needs varied. From project lead and NRM representative interviews it

was clear that the Projections Project provided well received communication of the science

underpinning the projections. Several project leads indicated that the National Projections team

had been responsive to NRM planners’ requests for general climate change information such as

clarifying the concept of scientific uncertainty in specific regional and cluster context/s. A major

challenge for the projections team was in meeting diverse planner needs.

Some people wanted learning materials to share with their stakeholders. Then there were technical

experts who wanted highly detailed products, through to people who knew almost nothing about

climate change… [Another factor was that] some people struggled in areas where there was climate

denialism [some clusters had a higher proportion of climate deniers in their communities and even

on some NRM boards]. In these cases it was a major source of frustration that we could not meet

their immediate needs [to address denialism] which was not in the remit of the project


In addition to meeting diverse NRM planning needs an additional challenge for the National

Projections team was in meeting broader needs from government (local and state) and

community stakeholders, particularly as the website gained attention through national media

exposure via a featured article on The Conversation (8 April 2015). This presented a challenge

for the National Projections team in balancing and prioritising their engagement time and

resources for the primary audience (NRM regions) as well as unanticipated secondary

audiences (the broader public).

AdaptNRM project

The AdaptNRM project final project evaluation conducted in March 2016 found that on average,

AdaptNRM modules largely met the needs of NRM representatives (n=30)31. These ratings were

supported by comments from NRM representatives at the AdaptNRM workshop as reflected in

Figure 4 below where a majority of respondents rated AdaptNRM modules 1 – 5 as meeting

their needs ‘well’, with the exception of Module 2, which a majority of participants rated as

‘Neutral’.

Figure 4. The extent to which AdaptNRM modules met user needs

31 On a scale including ‘very well’, ‘well’, ‘neutral’, ‘not well’ and ‘not at all’.

https://theconversation.com/a-new-website-shows-how-global-warming-could-change-your-town-39682



19

11 11

26 24

54

37

52

3744

23

48

33 3328

4 4 4 4 4

0

10

20

30

40

50

60

Module 1:

Adaptation planning ‘The NRM

Adaptation Checklist’

Module 2: Weedsand climate

change

Module 3:Implications of

climate change onbiodiversity

Module 4: Helpingbiodiversity adapt

Module 5: Sharedlearning

% r

esp

on

de

nts

AdaptNRM: How well AdaptNRM modules met user needs

Very well Well Neutral Not well Not at all

The National Projections team noted that engagement with AdaptNRM helped the Projections

project deliver information of relevance to end-users. This was because the AdaptNRM team

had engaged with NRM group representatives on their climate information needs in the early

stages of the project during 2013-14. The outcomes of these consultations were communicated

to the National Projections team by the AdaptNRM team.

3.2.3 Feedback

End-users provided feedback to project teams on the purpose, use and format of Stream 2

products and processes throughout the program via a range of mechanisms. The diversity of

engagement activities delivered by Stream 2 projects meant there was a range of opportunities

for stakeholders to provide feedback to the project teams throughout the life of the program.

The frequency of engagement, and hence interaction and feedback between the project teams,

intermediaries and end-users32 peaked at year two, where engagement was most intensively

delivered between projects. In this year (2013-14) some 268 engagements were conducted by

the ten projects to identify priority climate change issues and the information they needed to

inform planning.

User feedback was a critical factor in the engagement process which some projects

emphasised as being a highlight of the program. Feedback between users and suppliers of

research products was supported by adaptive, iterative approaches to project delivery. As the

quote from AdaptNRM below illustrates, the process of engagement was necessary to support

feedback in order to meet user needs.

The value is not in the product but the process of deep engagement. We shared really early drafts,

gained written feedback, organised discussion sessions and online forums. [As a result we were able

to] get them discussing together and get better feedback as a result…We worked with [NRM

planners] to make [the modules] specifically relevant to their region…One of our hallmarks has been

discussion based delivery, where anyone who wants to participate, has a chance to explore and

digest the products and then provide feedback. While the lead content scientists have been able to

look on, the main aim is talk with the NRM groups in an atmosphere where they can speak openly.

(AdaptNRM)

NRM planners and project teams both described a deliberative process at the beginning of the

project to determine and respond to user needs. For instance, in the projects which took a

32 Refer to Attachment Eight for definitions of terms used such as ‘intermediaries’ and ‘end-users’.



heavily consultative, participatory approach, such as SCARP, the project teams were trying to

understand user needs while the users were trying to understand what the projects could offer

them. Or, as it was characterised at the 2016 national workshop, projects would ask NRM

groups “what do you need?”, and NRM groups would respond “what can you offer us?” This

circular questioning was often described as confusing, challenging and difficult by planners and

researchers. However, this situation was also described as being “dynamic” by NRM and project

representatives interviewed for this evaluation as it led to a greater shared understanding of

one another’s needs.

User feedback in the ‘impacts and adaptation’ cluster projects

In the first year of the program (2012-13) three of the engagement plans (AdaptNRM, Wet

Tropics and Monsoonal North) clearly showed how avenues for feedback from stakeholders,

including NRM planners, would be provided. At the mid-point in the program (2014), while it

was apparent that many projects had implicit, informal processes for eliciting feedback from

cluster representatives and users, it was less clear how feedback from users would be used to

inform the development of products and modes of delivery. Despite this, the targeting of

engagement activities and early evidence that products and delivery mechanisms were relevant

to user needs would suggest that feedback between users and project teams was occurring

informally.

The SCARP team came to the project with their own perspectives. In the first couple of meetings

they brought some ideas which were quickly shot down by the group and they were put in their

place. Very quickly they learned that they need to listen to us and let us guide them as to what our

needs were. For instance, they were initially very focused on social and agricultural issues. We

[planners] told them that our needs were to do with NRM and so this changed their focus. They

responded to us.

(NRM Planner)

User feedback via the Climate Projections User Panel

In early 2013 the Climate Projections User Panel was established to provide an interface

between the Projections project, AdaptNRM and cluster projects to ensure that feedback

between projects was recognised and taken into account in the development of projections

research products. Hosted by the Projections project, the purpose of the panel was to “ensure

that the preparation and delivery of climate change projections information for Australia’s

natural resource management regions meets the needs of the user community” and to “create

a mechanism for representative feedback to the CSIRO projections team” (Climate Projection

User Panel TOR).

The panel provided projections users with an opportunity to provide feedback via email, video

and teleconferences. It was supported by a user panel engagement portal to provide

transparency in the user panel feedback and guidance process. In 2014-15 the Projections

project reported that AdaptNRM feedback on different versions of the brochures, reports and

website (including guidance material, tools and data) was taken into account during the review

process.

User feedback in the AdaptNRM project

The national AdaptNRM project reported that, through various engagement activities, it was

able to identify, understand and respond to the needs of NRM groups. Throughout the course of

the design, development and delivery of AdaptNRM NRM groups were provided with several

opportunities for input into AdaptNRM modules. For example, specific feedback from over 20



NRM managers was integrated into the development and draft review of delivery modules 3

and 4 (AdaptNRM, 2014-15 Annual Report).

3.2.4 Support

Peer support was a critical factor in supporting delivery between planners. Peer support

between NRM group representatives was a common enabler identified across a number of

regions.

One thing that was very good was they [Central Slopes] made sure we spent a lot of social time

together; we had a lot of chats together to become familiar across regions, but also with the

researchers. This was really setting the scene for collaboration. We weren’t shoved in a room and

told to come up with answers. They were very keen to find out our requirements and needs. Our

needs have evolved over this time and they have always stayed in touch and modified their

approach.

(NRM Planner)

Relationships between NRM organisation representatives, researchers and agency

representatives was a key factor in enabling the achievement of outcomes (see Section 3.1).

I don’t think the tools and report have been the most useful part of the project. For me it’s been the

relationships the have been built [between planners and researchers] that are the most useful:

having the ability to be on the phone with researchers as they are working and being able to follow

up with them and thrash ideas around have been very valuable. The actual documents or tools that

we get can be valuable but it’s working through together that is of higher value.

(NRM Planner)

The concurrent timing between Stream 1 and 2 also arguably further encouraged planners and

researchers to work together. In some regions and clusters this led to a deliberative process

where researchers and practitioners necessarily learned from one another (See Case study: Co-

learning between regions, Section 3.1).

3.3. The appropriateness of Stream 2

This section addresses KEQ3: To what extent was the implementation process appropriate?

Appropriateness primarily concerns the design of the Stream 2 cluster approach and model in

addressing the planning for climate change needs of NRM regions across Australia (see Section

3.3.1).

The cluster model has enabled the delivery of climate change information and support to 56

regions via eight clusters. It has facilitated cross-jurisdictional learning between NRM regions

and states in a way that would not have otherwise been possible.

Demarcated predominantly along bioclimatic considerations, the cluster model did not ‘work’ in

all cases, particularly where it required working across jurisdictional boundaries. In one

instance, the model reinforced the wide geographical distance between South Australian NRM

regions (Eyre Peninsula, Northern and Yorke, Adelaide and Mount Lofty Ranges and Kangaroo

Island) from the Perth based project team in the Southern and South-Western Flatlands cluster.

This split between these two NRM regions and the project teams was exacerbated by high levels



of staff turnover in the contact point between the project team based in Perth and the SA NRM

regions.

3.3.1 The ‘cluster’ approach and model

The decision to take a cluster approach was made by DCCEE with the intent to provide an

organised structure, efficiency in delivery and opportunities for shared learning across

jurisdictions. The approach aimed to identify a scale of delivery that balanced the need for

regionally relevant information with the requirement for national climate projections. This

model was developed by the DCCEE in 2012 in consultation with the (then) 56 NRM regions, the

CSIRO Adaptation Flagship, State Governments, and the Land Sector Carbon and Biodiversity

Board (LSCBB).

The aim of the cluster model, articulated by the DCCEE in the Program Guidelines (August

2012), was “to provide a structure to deliver regionally relevant information, while benefitting

from efficiencies in the joint delivery of relevant science information and opportunities for peer

learning”. According to this document “the grouping of NRM regions into eight clusters took into

account the nature of the projected change in climate, the range of adaptation options set by

biogeography, and the predominant land use”. In addition, the Guidelines note that “a number

of the clusters cross State/Territory boundaries, and therefore different jurisdictional

arrangements may apply across some clusters. The cluster design increases opportunities for

cross-learning and collaboration in similar geographical zones”.

In addition to the factors outlined above, when the clusters were demarcated DCCEE also took

into consideration factors including the number of NRM regions per cluster. While clusters were

based on biophysical and climatic factors, in some cases existing institutional structures were

taken into account; i.e. in the Rangelands cluster where an existing alliance was in place

between organisations. Following an internal process of approval DIICCSRTE delineated eight

NRM regional clusters: Monsoonal North, Murray Basin, Wet Tropics, Central Slopes, Southern

and Southern and South-Western Flatlands, Southern Slopes, East Coast and Rangelands (refer

to Attachment Two for the cluster map).



Inter-regional dialogue

The cluster approach facilitated inter-regional dialogue, learning and exchange between

regions. As noted above (see Section 3.2) connections between planners and peer support was

a critical enabler in the Stream 2 program. As a Sydney LLS planner noted: “Talking to my

colleagues from the other NRM regions was a major breakthrough”. This outcome was also

evident in Southern Slopes cluster where a joint working group between planners resulted in co-

learning across regional boundaries (see below).

Case study: Co-learning between regions

In 20 years of involvement with the CMA, never have I seen CMAs work so closely together as

they did on this project. Similarly, never have I seen the level of collaboration with researchers

and planners as on this project. The collaboration arose out of the need to work together. It was

clear from the beginning that, for the CMAs to derive any benefit, we would need to cooperate.

There was no competition like there is in other programs. SCARP realised this early on too. The

CMAs involved in SCARP have collaborated on a number of things now. Together we conducted

a vulnerability assessment for Southern Victoria. This arose out of the work that we were doing

together [in Stream 1 & 2]. It is always based on trust. The relationship between the projects –

SCARP and AdaptNRM – and us (CMAs). It was the opportunity to thrash things out that was

important; having all CMAs in the one room; it forced us all to work together.

(NRM Planner)

A related outcome was that the cluster model enabled research to reach a wider area beyond

state jurisdictions, than would have otherwise been possible, allowing greater efficiency of

output. For instance, in the Central Slopes project, carbon farming research outputs produced

by the Queensland Herbarium were distributed to NSW regional bodies, who would not have

otherwise received these outputs without the cluster model. The cluster approach also

facilitated learning at a national scale on the differences between state jurisdictions in the

regional NRM delivery model. See ‘Institutional boundaries’ below.

Institutional boundaries

As noted above, the Stream 2 cluster model boundaries were developed based on biophysical

rather than institutional considerations. In the Stream 2 program all clusters crossed multiple

state jurisdictions with the exception of Wet Tropics in Queensland. For instance, the

Rangelands cluster crosses Western Australia, South Australia, New South Wales, Northern

Territory and Queensland. Meanwhile, the Southern and South-Western Flatlands covers the

south west region of Western Australia and the coastal southern region of South Australia.

Projects/programs had to contend with variation between each state’s legislative and

institutional frameworks for NRM bodies and their varying degrees of state control. For

instance, in New South Wales the Local Land Services (LLS) now have a legislated role to

deliver NRM, amongst other services such as biosecurity, as an extension of the NSW State

Government. By contrast, in Tasmania the NRM bodies have no such legislated function.

In the Southern and South-Western Flatlands project there was a geographic and institutional

split between West Australian and South Australian jurisdictions (refer to cluster map,

Attachment Two). To address this split between the two states, the Southern and South-Western

Flatlands partnered with the South Australian Department of Environment, Water and Natural

Resources (DEWNR) to broker connections between the SA NRM regions and the Western

Australian based project. As the Case Study below shows, staff turnover in the SA DEWNR



contact point role during the course of the project reinforced a disconnect between the SA

planners and the project teams based in Perth.

Case Study: South West and South Western Flatlands – A split cluster

It was challenge to us [Stream 2]. There has been change of personnel in our contact point

[within SA DEWNR] with the SW project. Over the course of three years we have three people

who were leading the project from the SA branch. There wasn’t a constant source of contact. It

wasn’t until the third person who was able to show us what was available. All the support we

received was via email. This didn’t necessarily reflect the project but changes in the government

liaison between the project [SW Flatlands] and planners. We didn’t have much contact with WA

based at UWA either. We understand why they split up the clusters based on climates and land

uses. But the areas are very different. It is important to consider in the future. South Australia is

very different institutionally too. This has not really been taken into consideration.

(NRM Planner)

This quote highlights the range of factors that resulted in end-users being disconnected from

the project team including institutional arrangements, staff turnover and the engagement

approach taken by Southern and South-Western Flatlands, which outsourced the liaison to a

government department, whom made irregular contact with NRM representatives.

3.4. The efficiency of Stream 2

This section addresses KEQ4: How efficient was Stream 2? In particular, this section will cover

the efficiency of Stream 2 in terms of: (i) program coordination (Section 3.4.1), and; (ii)

administration of funds in accordance with contract agreements (Section 3.4.2).

The efficiency of Stream 2 delivery was constrained in terms of program coordination, primarily

with respect to cross-cluster coordination. The program funds were efficiently administered.

This was due largely to the fact that the program did not mandate or build any formal structures

for information sharing into the program such as explicit coordination activities across clusters.

It should be noted, however, that the oversight and coordination by the Department have

provided a sufficient level of consistency to ensure smooth delivery of the program. Meanwhile

in terms of funding disbursement, project and budget expenditure records from the Department

show that Stream 2 projects delivered outputs as per funding agreement milestones and

expenditure between 2013 and 2016.

3.4.1 Coordination

Although cross-cluster coordination and collaboration was a stated aim of Stream 2 there was

no formal requirement or expectation written into contract agreements for projects to work

together to achieve greater coordination and consistency. As a result, there were few examples

of cross-cluster coordination beyond the coordination functions administered by the

Department to support program delivery. One example of cross-cluster collaboration work that

stands out was the 3C Modelling jointly undertaken by Murray Basin, East Coast and Central

Slopes33 to evaluate the impacts of climate change on biodiversity across an area spanning one

quarter of the Australian continent over a timeframe to 2050. Meanwhile, the National

33 Drielsma M, Manion G, Love J, Williams K, Harwood T, (2014) Draft 3C MODELLING For Biodiversity Management

Under Future Climate

https://www.terranova.org.au/repository/3c-modelling-east-coast-central-slopes-and-murray-basin-nrm-collection/draft-report-3c-modelling-for-biodiversity-management-under-future-climate



Projections project coordinated a number of cross-cluster events including a side-event at the

Greenhouse 2013 conference, which brought all clusters together. However, in the context of

the whole program the joint cross-cluster 3C Modelling example is an isolated example.

As the lead agency responsible for administering the Stream 2 funding, the Department took

responsibility for coordinating Stream 2 across eight cluster projects, the two national projects

(‘National Projections’ and Impacts and Adaptation national project [‘AdaptNRM’]), in addition

to the Terra Nova information management project and the Monitoring and Evaluation

component of the program, which was contracted to Clear Horizon Consulting Pty Ltd.

Throughout the course of the project DIICCSRTE and then the Department undertook a number

of measures to increase coordination across projects and reduce duplication of project and

program outputs (as reflected in the program logic, see Appendix One). These measures

included:

Hosting a program workshop in 2013, 2014, 2015 and 2016 which brought together

representatives from all Stream 2 project teams, along with representatives from

government departments and regional NRM planning organisations. Among other aims,

these workshops sought to develop agreement among project teams on ways to coordinate

and communicate across the program and identify areas for possible collaboration

between project teams.

An activity calendar was developed and used in 2012-3 and 2013-4. An online program

calendar included key dates for workshops and meetings for each project, providing

opportunities for projects to coordinate joint activities and avoid duplicative consultation.

The appointment of NRM cluster representatives by DIICCSRTE in the early stages of the

program (2012-13 and 2013-14). Representatives were appointed to act as a point of

contact for DIICCSRTE and to coordinate input from their clusters during the application

process.

In 2013, following the Program Workshop in February, DIICCSRTE established a program

coordination group (PCG) to coordinate engagement across the Stream 2 projects.

There was limited evidence during the course of the program of clusters coordinating work

beyond these forums facilitated by the Department, possibly because there was no formal

requirement for projects to coordinate and work together across clusters. Feedback from

project representatives suggests that coordination across clusters was constrained by high

demands on the time of cluster representatives to engage both within as well as across clusters

boundaries.

Managing the complexity of NRM Clusters and number of NRM regions within clusters has also been

challenging. Establishing and maintaining cross-border collaborations and connections on a regular

basis is almost impossible as each project must concentrate on within Cluster delivery within the

agreed time frames.

(Murray Basin 2013-14 Annual Report)

In summary, there was a gap between the expectations of the Department and the funding and

design of the program given that coordination and shared learning across clusters was part of

the stated intent of Stream 2 (see 4.3 the Cluster model in ‘Appropriateness’). Like any

engagement activity, cluster and project coordination requires resources, time and inputs to

materialise benefits. For instance, in a recent science project funded by the Australian

Government, Securing Australia’s Future, an additional science synthesis project was

purposefully funded separately to integrate findings across a range of funded projects.

http://www.acola.org.au/index.php/projects/securing-australia-s-future



3.4.2 Delivery of milestones in accordance with funding agreements

Project records and budget expenditure show that Stream 2 projects mostly delivered outputs

on time against funding agreement milestones between the years 2012-13 and 2015-16. While

there were minor delays in projects communicating sub-project information to the Department,

the issues causing the delays were subsequently resolved. In some cases, this involved the

milestone payments not being fully expended by the milestone date with funds being

committed (e.g. contracted). Expenditure was therefore considered to be ‘on track’ by the

department.

3.5. The legacy and sustainability of Stream 2

This section considers KEQ5: How sustainable and enduring are the outcomes of Stream 2 likely

to be? In particular, this section will outline factors supporting the legacy of Stream 2 (Section

3.5.1) as well as challenges to the legacy of Stream 2 (3.5.2). The legacy and sustainability of

Stream 2 also concerns the capacity of NRM bodies to apply Stream 2 products and resources

(3.5.3), the extent to which information generated through the program is “embedded” within

regional NRM bodies (3.5.4), and the ongoing use of products and resources (3.5.5).

Significantly, the legacy of Stream 2 will also be influenced by the linkages and relationships

developed through the program (3.5.6).

The legacy of Stream 2 includes significant advances by regional bodies in their ability to

adaptively plan for climate change and the embedding of information and knowledge into

regional NRM planning processes.

These organisations now have the potential to influence other regions to increase their capacity

as well. As they adaptively plan for climate change they will be supported by responsive,

adaptive tools and frameworks (such as AdaptNRM ‘The NRM Adaptation Checklist’), online

resources and platforms (such as the Climate Change in Australia website and the Terra Nova

platform), the development of networks and relationships within and between researchers,

planners and NRM representatives and the publication of research outputs.

However, this legacy may be less effective over time for a number of reasons. The legacy will

likely be challenged and constrained by a lack of ongoing funding to support the application of

the climate projections; staff turnover and organisational restructuring; low levels of awareness

about Stream 2 within the Australian Government; and a lack of formal funding and federal,

state and regional support to sustain planner and researcher engagement with a climate

change focus in NRM.

3.5.1 Factors supporting the legacy of Stream 2

The legacy of Stream 2 was actively considered by Stream 2 projects throughout the period of

delivery. In years 2012-13 to 2015-16 the following preparations were undertaken to support

the longevity of Stream 2 and durability of the outcomes post-2016:

Developing responsive tools. Evidence indicates that information, tools and resources

delivered by Stream 2 were developed in response to changing climatic conditions as well

as the needs and priorities of end users

Developing online resources and platforms; i.e. the AdaptNRM website and the Climate

Change in Australia websites.

Implementing an information repository project (Terra Nova) led by Griffith University to

provide storage for all Stream 2 outputs in a centralised place, and where outputs are



searchable for the longer term, as opposed to being housed on a range of institutional

websites.

Supporting the development of networks and relationships within and between

researchers, planners and NRM representatives within and across cluster boundaries; i.e.

through the national Stream 2 workshop and ongoing, informal linkages and connections

between researchers, NRM representatives and stakeholders.

Publication of research outputs by research teams both in academic, peer-review scientific

journals and in a range of commissioned reports.

3.5.2 Challenges and constraints for the legacy of Stream 2

The legacy of Stream 2 will be affected by a range of issues, constraints and challenges, some

beyond the control and scope of Stream 2 including:

The cluster model was designed purely for Stream 2 and may have no longevity beyond the

program timeframe (2013 – 2016), although the lessons from the cluster approach may

be applicable to other forms of investment and programming federally.

The Climate Change in Australia website has no funding beyond possible basic

maintenance through the NESP program. The longevity of the website and its application

will depend on securing future funding, which at the end of the program (mid-2016) is

uncertain.

Staff turnover and organisational restructuring within regional bodies and to a lesser extent

within research institutions has resulted in a loss of corporate knowledge and capacity

developed through the program. Turnover is, however, a given in programs of this kind.

There was inconsistent uptake of resources and products across different NRM regions (see

3.1) with some regions achieving clear outcomes, while others have not achieved the same

results through the program. However, while capacity and uptake was variable across the

program “the history of regional NRM in Australia demonstrates that concepts permeate

the NRM system, moving from the early adopters and leading organisations to other

regions over time” (Cluster representative).

There are low levels of awareness about Stream 2 both federally34 and at a state level

among relevant agencies and departments. However, to a large degree the durability of

program outcomes is not dependent on federal and state government awareness of the

program.

Relationships and linkages developed through the program are not presently supported by

formal structures or resources to sustain ongoing connections with the explicit goal of

improving NRM planning to adapt to climate change.

Responsibility for implementing regional NRM plans typically lies outside the direct

influence of NRM bodies, which have the ability to facilitate and guide NRM change but not

the actual capacity to enact landscape and asset changes sought through planning, which

typically occur across a range of private and public land tenure arrangements. In this

context, NRM bodies are key enablers for change that must work with and through a

diverse range of state, regional and local, community stakeholders to deliver NRM

activities.

3.5.3 Long-term capacity for integrating NRM climate change planning

As detailed in Section 3.1, the Stream 2 program has had an inconsistent effect on capacity

across regional NRM bodies involved in the program. In short, while some regional bodies

developed capacity through the program and made significant advances in their ability to plan

for climate change, this positive influence was not widespread and tended to be concentrated

in regional bodies that were better equipped through existing capabilities, networks, resources

and mandated to plan adaptively. These regional NRM bodies who have embraced planning for

34 noting that in the funded period (2013 – 2016) the responsible agency went through three successive

changes in administrative arrangements.



an uncertain climate future may be seen as leaders and ‘champions’ with the ability to

influence other NRM regions. Given a reasonably high uptake of resources and outputs from

Stream 2 by NRM regions, it is reasonable to assume that the majority of NRM regions are well

placed to capitalise on the gains made in their capacity for NRM planning for climate change

from Stream 2. This was supported by stronger relationships and ties to research institutions

and the availability and accessibility of products, tools and processes to support NRM planning

for climate change (see below).

3.5.4 Embedding information and knowledge into planning processes

Adaptive planning approaches and systems

Several regions are moving towards adaptive planning approaches, which emphasise the

planning process rather than the plan itself. This change towards adaptive planning was

described as being part of a broader paradigm shift in the way that NRM regions approach and

undertake regional NRM planning by a Murray Basin team member:

My overwhelming reflection is how fundamentally different the conversations at these final

engagement workshops have been compared to workshops and meetings even 12-18 months ago.

There has been a paradigm shift in some regional planning teams in which critical climate

adaptation concepts such as multiple futures, dealing with uncertainty, flexibility and learning now

have primacy in the discussion. Previously these concepts were recognized but it was not fully

understood how central they are to climate adaptation planning.


In this context, NRM was viewed by researchers and planners as a complex, changing system.

Therefore, to be relevant, planning is necessarily a dynamic, emergent process, which must be

adaptive in order to respond to climate change.

One of the overall things is we have changed our approach from looking at regional and strategic

plan to taking a landscape approach taking into account resilience concepts looking at how climate

change will affect different areas differently. This has helped us understand the issues around

climate change and look at the issues in the future.

(NRM Planner)

Stream 2 has, in part, catalysed this shift towards adaptive planning by supporting and

strengthening adaptive planning approaches. NRM plans are, for instance, increasingly being

developed and updated as virtual, online “live documents” (see below).

3.5.5 Ongoing use of products, tools and information

The following online resources and platforms developed by Stream 2 will provide ongoing

accessibility to Stream 2 products and resources, dependent on ongoing funding and support:

the National Projections website Climate Change in Australia provides resources, links and

products developed by the National Projections project as well as links to the eight regional

impacts and adaptation cluster projects.

the Terra Nova website provides an information repository for all National Projections

products

the AdaptNRM website provides an online repository of AdaptNRM products and resources

the Department have committed to updating the Stream 2 web-page to provide links to

Stream 2 products.

http://www.climatechangeinaustralia.gov.au/en/

https://terranova.org.au/

http://adaptnrm.csiro.au/



Through a linkage with the National Environmental Science Programme’s Earth Systems and

Climate Change Hub, the National Projections team have potentially earmarked limited funding

for basic maintenance of the Climate Change in Australia website in the short term. In addition,

Stream 2 researchers, project teams and NRM organisations have produced a large number of

research outputs, which are being published in academic publications and in the wider online

environment, which will provide ongoing accessibility well beyond the program timeframe.

Online plans

Online plans and portals provide an opportunity for NRM and land use planners to develop,

maintain and update plans in an adaptive, responsive and flexible manner. While the exact

figure is unknown it estimated that around half of the regional bodies currently have online

plans (AdaptNRM). As the Rangelands NRM representative below states, online plans will

support NRM plan adaption and evolution.

That’s the intent of the online plan. It involves adaptive learning as we become better informed so

the plan continually evolves. It is quite a practical plan based on our delivery of projects but there is

the potential to link the plan to a possible knowledge hub. The regional plan hosts the most relevant

knowledge. This long term nature of [the plan online] puts it all into perspective.

(NRM Planner)

Online plans were provided by a number of regions for this evaluation including: SWCC, Territory

NRM, Rangelands WA NRM, CCMA, Reef Catchments and Terrain NRM. In addition, an online

forum was established during the Stream 2 period by a coalition of Victorian CMAs35.

3.5.6 Linkages and relationships supporting the legacy of Stream 2

Linkages within and between regions, researchers and agency representatives have been

developed and strengthened by Stream 2. Due to the national scale of the program and size of

the clusters there was necessarily a wide degree of differentiation in the quality of relationships

developed across clusters. Despite this there was evidence of ongoing partnerships between

researchers and end-users from all Stream 2 projects. Although most partnerships were

reported as informal, some Stream 2 projects are developing formal partnerships through

project work, which are expected to endure beyond the life of the project.

I think the personal interactions between researchers and NRM groups and the greater

understanding of what NRM is all about. This is the most lasting. This is a shared outcome, a lot

more sympathy both ways… we are more approachable. There are new projects that will come up

and we are more comfortable to work together with people in the local landscape; i.e. Carbon

farming.


In 2014-15 a number of projects reported that Stream 2 has contributed to a shift from a

situation where there was limited interaction between climate researchers and NRM planners in

Australia to a situation where strong relationships based on “trust” and a mutual understanding

of research capabilities and end-user needs had been forged (Projections project, AdaptNRM,

Monsoonal North, Southern Slopes, Wet Tropics, Central Slopes).

35 Funded by the Victorian State Government.

http://www.environment.gov.au/science/nesp/current-projects/earth-systems

http://www.environment.gov.au/science/nesp/current-projects/earth-systems



Ongoing relationships will be a critical factor in supporting and enabling outcomes achieved by

Stream 2.

Everyone is aware of the structural flaws [in Stream 2] but the relationships between regions and

with scientists have stimulated my community. It has created networks and it was agreed it would

be useful to maintain these into the future. Unfortunately, we recognise that any formal networking

needs financial support, so a less approach of staying in touch and sharing resources has been

maintained.

(NRM Planner)

At the time of the evaluation there was no formal mechanism to support ongoing relationships

between researchers and practitioners nationally within the Australian NRM sector. There are

forums where planners, researchers and agency representatives will support ongoing

connections, such as the Annual NRM Knowledge Conference. Ongoing relationships between

researchers and NRM practitioners are therefore likely to be largely based on informal,

incidental connections and opportunities as well as through existing departmental working

groups and committees.

Inter-regional networks

As noted in Sections 3.1 (Effectiveness) and 3.2 (Relevance) Stream 2 has provided an

opportunity for planners to form stronger linkages and ties across regions. At the end of the

program, inter-regional networks that provide a forum for planners to connect and provide peer

support to one another exist in a number of state jurisdictions as highlighted below. In these

jurisdictions, planners are working to apply Stream 2 products cross-regionally.

Victoria. The Victorian CMA NRM Planning for Climate Change; a forum comprised of

representatives from all nine Victorian CMAs. Initially funded through Stream 1 the forum

met monthly until January 2016 and now meets on a needs basis in person, via email or

phone.

Western Australia. South East NRM groups are currently coordinating the establishment of

a shared spatial dataset and website to support cross-regional planning.

Cross-regional linkages between planners are critical in supporting the legacy of Stream 2.

Engagement with other stakeholders

Several NRM representatives noted that the actual responsibility for implementing NRM plans

often lies outside of regional NRM bodies themselves. This of course varies across states

depending on the legislative framework for regional NRM in each state (refer to Section 3.3 for

more detail). Notwithstanding these institutional differences across states, NRM bodies typically

work with and through regional networks of industry, research and community entities to

deliver NRM strategies and plans. As a representative from NRM South in Tasmania

commented:

We are developing a strategy on behalf of the community. This isn’t NRM Souths’ Strategy; we (NRM

South) are not land managers. It is their strategy.

(NRM Planner)

In Victoria the CMAs and Victorian Catchment Management Council (VCMC) committed equal

amounts to fund a State-wide Climate Change Coordinator at 0.4FTE until December 2016. The

http://www.conference.nrmregionsaustralia.com.au/



position is presently hosted by the Goulburn Broken CMA. One responsibility of this position is to

look for opportunities to engage with the Victorian State Government to support ongoing

utilisation of Stream 1 and 2 outputs. For instance, a forum took place in May 2016 to discuss

the utilisation of Stream 2 outputs in the Victorian Government Climate Change Adaptation

Plan and the Victorian Biodiversity Strategy. The State-wide coordinator is also working with a

national Emissions Reduction Fund (ERF) group and is in discussions with the Victorian

Catchment Management Council about how the Stream 1 and 2 work can assist in the

development of Catchment Condition Reporting with regard to climate change36.

36 Correspondence with the State-wide Climate Change Coordinator.



Attachment One: Stream 2 Program Logic

Figure 5. Stream 2 Program Logic



Attachment Two: The Cluster Model

Figure 6. The Cluster Map showing the eight Stream 2 NRM clusters



Attachment Three: Results chart Table 7. Stream 2 Results Chart (2013 – 2015)

KEQ Summary of findings Evidence

1. How effective was Stream 2 in achieving its intended outcomes?

1.1. To what

extent have Stream

2 projects improved

the quality and

accessibility of

regionally relevant

information on

climate change

impacts and

potential adaptation

responses?

Stream 2 has effectively increased

the quality and accessibility of

regionally-relevant climate change

information and support by

delivering information and support

to NRM regions via eight cluster

projects, the National Projections

project and the national impacts

and adaption project (AdaptNRM).

Increased accessibility evidenced

by the development and delivery of

a total of 828 information products

by ten Stream 2 projects between

2013 and 2016; of which around

100 final products are stored on

the Terra Nova web-platform,

publically accessible.

Wide-spread uptake and adoption

of Stream 2 tools and resources

with at least three quarters of NRM

regions demonstrating tangible

examples of product use in the

planning process.

Accessibility: Between 2012-13 and 2015-16 improved accessibility to regionally relevant information on

climate change impacts and adaptation was supported by:

The delivery of 659 outputs to support regional NRM organisations to plan for climate change via the

eight clusters, including: information products (reports, data-sets, guidance documents, fact sheets),

training, mentoring support via workshops, presentations and other delivery methods (2013-2015)

The delivery of 157 outputs which translate next generation climate projections to NRM planners via the

eight clusters (2013-2015), including: information products (web tools, journal papers, aminations,

spatial layers and data sets), presentations, consultations and other delivery methods.

The delivery of the Adapt NRM outputs including: module guides, website resources, data access portal

resources (maps and datasets), module delivery sessions, module review sessions, conference

presentations and meetings and newsletter updates.

The launch of the Climate Change in Australia website by the Projections project From 27 January 2015

until 22 June 2016 the web site had 153,746 unique website users with 202,446 unique sessions

(averaging 12,100 sessions per month).

Specific examples of increased accessibility from the final round of project reporting (2015-16) include:

Murray Basin: during initial consultation in 2013 most regions identified other regions as the main

source of information; by contrast recent consultation regions cite a wider range of available sources of

information, and are aware of how to access that information. Consultation by Murray Basin

representatives in 2016 found a high level of familiarity in general with Stream 2 products.

Rangelands: 7 out of 11 survey respondents at the end of the project reported improved access to

regionally relevant climate change information as a result of the project.

SCARP: 8 out of 9 NRM regions report that the quality and accessibility of regionally relevant climate

change information improved as a result of SCARP, particularly through the information portal

established by SCARP. Also in terms of enabling regions to find and access such information

themselves. On average, regional bodies rated the quality and accessibility of information provided

through SCARP as 7.9 out of 10.




Examples of use of Stream 2 products in the NRM planning process from project reporting (2015-16) include:

National Projections: Use of the climate futures tool for scenario planning (ACT Environment and

Planning Directorate), to inform likely climate change impacts (SE NRM), to underpin second pass

impact and vulnerability assessment (Wimmera CMA & West Gippsland CMA), to inform development of

a Spatial Assessment Tool (GBCMA), to influence the draft NRM plan (Southern Gulf Catchments Ltd), to

confirm vulnerability assessments (Burnett Mary), to inform climate refugees mapping (SEQ

Catchments), to inform draft risk assessment process and spatial analysis (Hunter LLS), as a

“centrepiece” in the draft NRM Strategy informing the “Possible Futures” section (NRM South), to

develop sub-regional plans summarising future conditions (Eyre Peninsula), used “where appropriate”

(Kangaroo Island NRM), to inform program development and implementation as well as being used on

the Swan River Strategy website (Perth NRM).

Central Slopes: 3 out of 5 regional NRM plans (QMDC, NW LLS, Northern Tablelands LLS) draw on

Central Slopes resources, with remaining 2 regional bodies expected to provide outputs (April 2016)

East Coast: See spatial analysis for carbon soil and vegetation carbon sequestration (KEQ1.3 below). The

development of “website chapters” by GSR LLS to draw on AdaptNRM Biodiversity modules. The

development of an updated plan by SEQ Catchments whereby the project team were involved in review;

research included in the process of plan development rather than the end-product. Incorporation of the

projections into the background of the FBA plan.

Monsoonal North: NRM plans prepared for Northern Gulf NRM, Territory NRM, WA Rangelands, NQ Dry

Tropics and Cape York NRM utilised project outputs. Southern Gulf Plan (being finalised) will use project

outputs. Specific examples of utilisation include the use of materials were reportedly in Indigenous

schooling (further details not provided), use of the CLiMAS projections.

Murray Basin: Citation of Stream 2 sources including: North Central CMA climate adaptation plan cites

the Climate Change in Australia as a key information source; Wimmera & SE NRM similarly uses

projections in NRM planning documentation. Adoption of the Adaptation Pathways approach by SE

NRM, GBCMA and North Central Catchment Management Authority (NCCMA).

Rangelands: seven NRM regions all report that they “have or will use [Rangelands project] information in

their NRM plan”. Some sources were reported as being used while others were not by Rangelands NRM

regions. NRM plans updated with Rangelands outputs include: National Projections summaries,

inclusion of the “Rangelands Adaptation Table”, links to the AdaptNRM and Climate Change in Australia

websites (Territory NRM), incorporation of the National Projections as the “basis for supporting climate

change risk” (SW NRM), to inform the draft addendum of Strategic Plan with references to projections,

likely impacts and adaptation strategies (Western LLS), to inform the location of climate refugia (Desert

Channels Qld), to update the Climate Change addendum (Alinytjara Wilurara NRM), and to integrate

recommendations from cluster reports (SA Arid Lands NRM). In addition, Stream 2 outputs were used by

Territory NRM during the planning process where cluster projections were used during participatory

planning.

SCARP: wide adoption of the Adaptation Pathways framework with the approach being used “across

Victorian CMAs”. Victorian CMAs developed climate adaptation plans as adjuncts to existing plans,

Tasmanian and NSW regions incorporated SCARP products & processes into scheduled strategic plan

reviews.

East Coast: use of the climate impacts and adaptation information in the SEQ Catchments draft NRM




plan.

Southern and South-West Flatlands: uptake of the MCAS-S spatial planning tool by some regions

(SCNRM, SWCC & NACC) to guide carbon planting, and to prioritise areas for climate change adaptation

strategies (Perth NRM).

Wet Tropics: updated NRM plans containing information from Wet Tropics outputs and the National

Projections. NRM plans also influenced in their structure and approach by Wet Tropics; i.e. identification

of sectoral themes in the Terrain and Reef Catchments’ plans, and framing of adaptation pathways in

Cape York NRMs’ plan. Development of Climate Action Strategies by Traditional Owners to include in

Management Plans, co-presented at the AIATISIS Native Title Conference and at a workshop on

Transdisciplinary Science in 2015.

1.2. To what extent

have Stream 2

projects improved

the capacity of

regional NRM

organisations to

plan for climate

change?

Wide variability in the influence on

climate change planning capacity

within NRM regions, with some

regions reporting organisational-

level shifts in planning approaches

(i.e. adoption of the “Adaptation

Pathways” approach in CCMA),

while others report limited or no

influence on planning capacity.

Support for capacity development was provided by projects to NRM organisations:

Adapt NRM: has provided support through the provision of module guides and web-based resources,

interactive review and delivery sessions, and by providing opportunities for shared learning between

NRM groups

Monsoonal North: NRM planners have improved qualifications and training related to climate

knowledge.

Murray Basin: Increased awareness of regionally relevant information on climate impacts within most

NRM regions. Regions with staff continuity and stable institutional arrangements have “built personal

and organisational capacity around core concepts of adaptation planning, around the conceptual

difference between climate adaptation planning and other forms of planning”. Supported by self-

reported capacity development; adoption of concepts (i.e. NCCMA and GBCMA have both incorporated

climate adaptation pathways into their planning; examples of NRM regions actively engaging with and

questioning different approaches; “The amount learnt in just three years is huge! If you look at where we

were three years ago and where we are now, we are really ready to get serious about climate adaptation

planning and I think you will see that happen in the next round of catchment planning” (2015-16)

Southern and South Western Flatlands: capacity among NRM bodies is variable reflected in low uptake

and access of products. Capacity was increased in some regions to use the MCAS-S spatial planning

tool.

Wet Tropics – Increased understanding among NRM bodies of the process of climate change research.

Fostered through the Brokering Hub, which provided a forum for interactive, ongoing communication

between individual researchers and NRM planners.

Rangelands – NRM regions report “increased confidence and understanding of Climate Change

projections” as a result of engagement with the National Projections team. This in turn supported

“improved discussions with communities” and “inclusion [out outputs]” in NRM plans. NRM regions in

the Rangelands cluster also reported a “little improvement” in capacity as a result of Stream 2.

East Coast – NRM planners report “enhanced knowledge…access to tools and information for climate

adaptation planning”.

In addition to developing NRM organisation capacity, Stream 2 influenced the capacity of researchers to

engage with NRM organisations.




National Projections – development of knowledge brokering capability among CSIRO and BoM climate

scientists; “enhanced ability to translate technical information in a way that can be understood by

next/end-users” (2015-16 report).

1.3. How has

the program

contributed to

Government

objectives?

Limited direct influence and

contribution to Government

objectives of spatially guiding

carbon sequestration and

biodiversity fund activities.

Specific research outputs guiding

spatial prioritisation for carbon

sequestration and biodiversity fund

activities include the use of the

MCAS-S tool in Southern and South-

Western Flatlands, carbon farming

research undertaken in the East

Coast cluster and the 3C Modelling

jointly undertaken by Murray Basin,

East Coast and Central Slopes to

evaluate the impacts of climate

change on biodiversity up to 2050.

At year two (2013-2014), Stream 2 some projects (3 out of 8) were found to have made some small, yet

potentially significant steps towards Land Sector Package objectives, including: consultation, project

development and spatial analysis being taken towards informing the location of carbon and biodiversity

activities.

At year three (2014-2015) 4 out of 8 cluster reported specific examples of progress towards these

government objectives, including:

Use of MCAS-S spatial planning tool has informed the identification of sites for carbon sinks for tree

planting (Flatlands).

East Coast compiling biophysical, economic and other data for carbon farming assessment and have

attended meetings with Commonwealth Government representatives regarding new Emissions

Reduction Fund framework and the implications for carbon farming (East Coast).

At year four (2015-16) examples included:

East Coast: 3 NSW LLS have initiated a project to develop a spatial analysis tool to identify the best

areas for soil and vegetation carbon sequestration. Tool still being developed under contract by a

consultant at the time of evaluation. In addition, BMRG used the Carbon Farming report to meet Stream

1 requirements. Finalisation of the Qld Government Carbon Farming report. Development of a web-

based tool to visualise carbon farming opportunities from forest regrowth. Development of a joint-

project between consortium researchers to realise opportunities in the Emissions Reduction Fund (ERF).

Southern and South-West Flatlands: use of the MCAS-S spatial planning tool by SCNRM, SWCC and

NACC to identify sites for carbon planting.

The 3C Modelling jointly undertaken by Murray Basin, East Coast and Central Slopes to evaluate the

impacts of climate change on biodiversity up to 2050.

2. How relevant was Stream 2 to the context and needs?

2.1. How well

did the information

and support

delivered through

the program meet

end-user needs?

Stream 2 products and support

have largely met user-needs

balancing the needs for scientific

rigour with a user-focused research

approaches.

Over the life of the program (2015-16) the delivery of information to meet NRM planners needs was

supported by a number of engagement activities:

In the first reporting year (2012-13) Stream 2 projects undertook primarily planning and engagement

activities consistent with program expectations. To summarise, projects used a range of informal and

formal means to engage with clusters to design and plan for national projects.

The delivery of 268 engagement activities to identify priority needs and issues for NRM planners,

including: consultations, program management meetings, workshops, presentations and other delivery

methods.

https://www.terranova.org.au/repository/3c-modelling-east-coast-central-slopes-and-murray-basin-nrm-collection/draft-report-3c-modelling-for-biodiversity-management-under-future-climate




Feedback from NRM groups confirmed that information and support provided by Stream 2 met their needs:

Adapt NRM modules and resources were received well. For instance, the Final Adapt NRM workshop

evaluation found that the workshop “increased” participant “understanding of Adapt NRM modules”

moderately for 53% of participants and strongly for 30% of participants.

East Coast: 4 out of 4 planners ‘strongly agree[d]’ that the Biodiversity project was ‘relevant to NRM

planning and that they ‘will be able to use the results of the project’, while participants were ‘neutral’ or

‘agreed’ that the Planning Packages ‘were useful and relevant’; at the ‘Climate Change Adaptation for

NRM Planning’ Workshop (22-23 April 2015).

Rangelands: Survey respondents (2016) all (n=11) indicated that interactions between NRM regions and

scientists were valuable and that these “interactions contributed to the usefulness of the end product”.

SCARP: 8 out of 9 regional bodies state that the project has taken approach that has met their needs.

Face to face meetings in particular were cited as the main form of engagement that was most useful to

NRM bodies. On average, regional bodies rated the information and support provided by SCARP as 7.6

out of 10 in terms of meeting their expectations, while the effectiveness of engagement approaches

were rated as 7.7 out of 10 by end of project (2016) evaluation survey participants (n=9).

East Coast – the “Consortium” approach (see 2.2 below) of involving multiple (six) research partners in

project design and delivery was potentially “less flexible” in responding to changing user needs than

projects with fewer partners, and hence reduced consultation demands / transaction costs. Feedback

from NRM regions indicate that the PWG workshops were largely appreciated but they did not always

meet their needs. The PWG provided a source of focus for interactions between researchers and

planners during delivery. 1:1 engagement was highly valued by researchers and planners.

Wet Tropics – emphasis on co-development of research outputs in the project. Establishment of the

“Brokering Hub” mechanism to facilitate researcher-planner engagement (see 2.2 below). The research

model facilitated the identification of project goals and outcomes to be met by the development of

research products. Project rated as ‘Good’ in terms of providing “fit-for-purpose” outputs.

2.2. How

effective were the

various engagement

and delivery

approaches used by

the cluster projects?

A range of engagement and

delivery approaches were used by

Stream 2 projects to understand

and respond to the planning needs

of NRM regions. This engagement

process has supported, for the

most part, the delivery of products

and support to NRM regions that

have been well received, relevant

and fit-for-purpose.

On a project-by-project basis:

Central Slopes: the Project Team was comprised of regional NRM organisations, the university

researchers and government partners. The inclusion of NRM organisation representatives on the central

project team support targeted and relevant engagement by Central Slopes where “regional NRM

organisation participants expressed a strong desire to be active research partners, instead of ‘end-

users’” (Central Slopes, 2015-16)

Monsoonal North: NRM planners sat on a project steering committee, which provided direct feedback to

projects. Planners also provided a line of communication to the project teams on NRM plan

development timelines.

National Projections – climate projections User Panel, regional workshops and training courses – “The




information provided is excellent and extremely valuable to awareness raising and adaptation planning

at a broad level”

Rangelands – employment of an ‘Engagement Officer/Knowledge Broker’ who played a ‘key’ role in

distributing information and outputs to ‘broader users’

SCARP – Founding principle of project to “engage in responsive, demand-driven research practice”.

Relationships developed through “regular informal structured interactions including workshops,

interviews, focus groups and surveys”. Joint development of sub-projects on adaptation planning, spatial

prioritisation, impacts information synthesis and capacity building by researchers and NRM region

representatives. Co-delivery of outputs via an interactive, iterative approach to project implementation.

Use of a ‘container’ (a “live information portal”) as a virtual space where participants could provide and

receive information throughout the process.

East Coast – A consortium approach was taken whereby several research partners (six) were involved in

the project. Establishment of a Planners Working Group (PWG), consisting of a series of workshops

delivered to planners. Creation of a “Community of Practice” within and between the research and NRM

groups.

Southern and South-Western Flatlands – consultation and information provision delivered to NRM

planners via three workshops. Appointment of a representative from SA DEWNR to act as a conduit

between SA NRM regions and the project team.

Wet Tropics – establishment of a “Brokering Hub” mechanism in the project to facilitate engagement

between the research teams and NRM bodies. Identification of “Key Accountabilities” as well as shared

goals for Stream 2 by NRM representatives and project team member, in turn setting the direction for

project consultation, engagement, outputs and outcomes. Use of multiple modes of engagement

including workshops and through an interactive, consultative report review process. Engagement was

rated by NRM bodies as ‘Good’ to ‘Very good’ in the Final evaluation. One NRM region commented that

“the researchers have been largely engaging and tailoring the process to the needs of individual

researchers or NRM groups”. Some aspects of engagement were less well received; i.e. using a “list of

NRM concerns” to structure the first report. In addition, differing perspectives between researchers on

what constituted appropriate engagement; i.e. ‘client-provider’ model vs. ‘co-research’ models.

2.3. How well

were NRM planning

needs and feedback

integrated into

Program delivery?

Feedback was sought and used by

Stream 2 projects with varying

levels of success and tended to be

stronger in more collaborative

projects with a structured and

explicit approach to engagement.

Different mechanisms for providing

feedback were used by projects.

Feedback was explicitly sought,

gathered and used in a minority of

cluster projects (Wet Tropics and

Southern Slopes). In other cluster

Examples of feedback provided by projects in 2015-16 reporting include:

SCARP: User needs were “reportedly well-integrated into program delivery throughout iterative process

of engagement and sub-project development”. In an end of project evaluation survey, participants rated

user-feedback highly, with an average score of 8.7 out of 10.

Wet Tropics: Project rated on average as ‘Good’ to ‘Very good’ by NRM regional representatives in terms

of “adaptive learning and decision making” comprising incorporation of feedback. One region

commented that: “Researchers have been very adaptive to respond to the requests and feedback from

the NRM groups”.




projects feedback was sought and

used implicitly.

3. To what extent was the implementation process appropriate?

3.1. How

appropriate was the

‘cluster’ approach in

achieving its intent?

The cluster approach taken in

Stream 2 has led to inter-regional

learning and connections across

NRM groups that would not have

otherwise been established. This

was strongest in clusters where; (i)

inter-regional forums were

established and brokered by the

project teams, and; (ii) existing

inter-regional forums existed (i.e.

Victoria and South Western

Australia).

The cluster model has also

reinforced geographical and

institutional divisions across state

boundaries, which has (along with

other factors) effectively

disconnected NRM regions from

Stream 2 projects.

Refer to section 3.3

3.2. To what

extent did Stream 2

promote good

practice in the

integration of

climate change

information into

NRM plans?

Refer to Section 3.1 and KEQ1.1 Refer to Section 3.1 and KEQ1.1

4. How efficient was Stream 2?

4.1. To what

extent did projects

realise opportunities

to increase

efficiency through

coordination?

Limited efficiency in terms of

coordination at a program or cross-

project level despite largely

administrative coordination

functions due to the fact that

coordination was not budgeted for.

Refer to Section 3.4




4.2. Are

projects on track

against Funding

Agreement

milestones and

expenditure?

Stream 2 projects delivered

outputs as per funding agreement

between 2012-13 and 2015-16.

Refer to Section 3.4

5. How sustainable and enduring are the outcomes of Stream 2 likely to be?

5.1. How well

did the Program

enhance the longer-

term capacity for

integration of

climate change

information into

NRM plans?

Limited and variable influence on

long-term capacity due to relatively

high turnover in regional bodies

among NRM planners who received

the bulk of support provided.

Barriers to longer-term capacity include:

High turnover within NRM bodies in several clusters; i.e. in East Coast and Murray

Institutional restructuring; during the timeframe of delivery and following delivery, will potentially nullify

efforts and gains made.

The influence on longer-term capacity:

Rangelands – regional NRM representatives (n=11) were not “as confident in [capacity] improvement”

as they were in improved access (see 5.2 below). Some regions report improvement in longer-term

capacity, though on the whole regions report a “little improvement” in capacity.

SCARP – some regional bodies felt that the development of long-term capacity was outside the scope of

the SCARP project. Long-term capacity received a moderate rating in the SCARP NRM region survey in

2016 with an average rating of 6.9 out of 10.

5.2. What

evidence is there

that products, tools,

information and

knowledge

developed through

the Program have

been embedded

into NRM planning

processes?

Several examples of how products,

tools, processes and approaches

have been well integrated into the

planning cycle by NRM bodies.

Integration of products, tools and

approaches was supported by

consultative, collaborative

engagement processes to support

adoption.

Examples of integration of products, tools, information and knowledge into the planning process include:

Murray Basin – Clear intent among some regions to continue to integrate climate change adaptation

concepts and tools into planning at regional and sub-regional scale/s. See KEQ1 for examples of

adoption of planning approaches; i.e. adoption of the Adaptation Pathways approach in GBCMA, SE

NRM and NCCMA. Uptake was variable across regions in the Murray Cluster. Adoption by some regions

plays a stepping stone for wider dissemination of approaches. Supported by the Victorian Climate Forum

as well as the piloting of the Adaptation Pathways approach in the adjacent SCARP cluster.

SCARP – decision making frameworks and the Adaptation Pathways approach were regarded as useful

in future planning processes.

Southern and South-Western Flatlands: adoption of the MCAS-S spatial planning tool by NRM regions

(SCNRM, SWCC, NACC & Perth NRM)

Wet Tropics – co-development of research products in formats consistent with NRM group needs will

support ongoing use and adoption; i.e. “NRM partners have expressed their intention to use the fact

sheets series to initiate deeper conversations and develop scenarios with key stakeholder groups or

industry sectors. Cape York NRM have used the fact sheets (as well as the foundational reports

developed in this project) to formulate ‘memes’ that are published weekly on their website” (2015-16).




5.3. What

evidence is there

that product, tools,

information and

knowledge

developed through

the Program will be

used in an on-going

way, after the life of

the Program?

Tools and products developed by

Stream 2 will be accessible well

beyond the project time-frame.

Online information storage in a

range of platforms including the

centralised Terra Nova repository

will support ongoing accessibility.

The extent to which these products

are accessed and used beyond the

program timeframe will vary, with

evidence (see KEQ1) suggesting

that demand is relatively high and

capacity to use the products among

planners is sufficient.

Evidence of increased accessibility via online portals and platforms includes:

Terra Nova – centralised web-portal for all Stream 2 products.

National Projections – the Climate Change in Australia website provides a repository of information

developed through Stream 2. Website maintenance will be supported through the Earth System and

Climate Change Hub for the next three years.

SCARP – products available on a web-portal.

Southern and South-Western Flatlands: storage of products on regional NRM body web-pages.

5.4 How well

did the Stream 2

projects improve

links between

researchers and

end-users?

Stream 2 has effectively improved

links between researchers and

NRM organisations across all

projects.

Stream 2 has provided a forum for planners and NRM representatives to forge connections to support the

program legacy.

Adapt NRM – Final workshop evaluation (March 2016) found that the workshop facilitated connections

between planners; 67% ‘strongly agreed’ that workshop provided a forum for NRM planners to share

adaptation ideas with peers; while 67% also ‘strongly agreed’ that the workshop helped them identify

people that they could approach in the future.

Central Slopes – Communities of practice being established; to support capacity (2015-16)

Examples of improved connections between NRM bodies and research organisations include:

Central Slopes – Ongoing partnership between USQ and QMDC have a formal agreement for

supplementary activities; with intent to develop further agreements; interest in jointly developing a

regional economic model to cost the impacts of climate change and determine relative benefits from

different adaptation options (2015-16)

Monsoonal North – Enduring linkages between planners and researchers; i.e. projects have evolved into

new work funded by the NESP Northern Australia Environmental Resources Hub. Linkages are supported

by a deeper understanding among researchers of NRM planning needs, while NRM planners have a

greater understanding of available knowledge and how it can be applied to improve planning.

Murray Cluster – NRM regional staff and researchers were brought together to develop relationships and

build rapport and familiarity with one another’s perspective; i.e. one planner commented: “there was a

genuine effort by people in Stream 2 to engage and better understand the needs of the regions”.

Rangelands – Project participants (n=11) including NRM regions (n=7) report that the “most useful” part




of the project was enhanced connections between climate scientists and planners. Evidenced by

“Inclusion of regions in on-going water refugia monitoring ARC Linkage project application lead by Dr

Jenny Davis, and the interaction of CSIRO with RCP regarding its Climate Adaptation Pathways work and

the use of rangeland regions as case studies (WLLS, CWLLS, North Gulf (Nth Monsoon)”.

East Coast – establishment of a “Community of Practice” to support ongoing researcher-planner

engagement. Supported by ongoing relationships between researchers and planners.

Southern and South-Western Flatlands – increased interactions between researchers and end-users

resulting in collaboration and improved linkages between groups. Better linkages improved by requests

from groups to undertake further species modelling from project researchers.

Wet Tropics – enhanced linkages between researchers and NRM bodies developed via the Brokering

Hub reflected in co-authorship of a Science Synthesis report. NRM bodies have expressed a desire to

continue working together in the future.

SCARP – relationships developed through the program are now being formalised into new working

projects; “links between researchers and end-users were greatly improved across the board” (SCARP,

2015-16). SCARP NRM regions reported improved links with researchers as a result of the project with

an average rating of 8.5 out of 10 being given by participants in the end of project evaluation survey

(n=9); “Our links with researchers is now established as far as climate change research and continues to

develop and extend into other areas” (CCMA).



Attachment Four: Stream 1 Report 2013-14 – 2014-15 Synthesis Table 8. Uses of Stream 2 products by NRM regions as reported in Stream 1 reporting (2013-14 – 2014-15)

Organisation name* Reporting period Cluster What was used?** How was it used?**

ACT Environment and

Planning Directorate

1/01/2015 to

30/06/2015

Murray Basin Decision-making framework n/a

Adaptation Planning Resource Inventory n/a

Plain Language Information Packages Communicate the potential risks of climate change

Biodiversity Modelling under Multiple

Futures

Identify potential climate corridors and climate refugia.

Exploring Adaptation Pathways Explore how NRM decisions might be structured and

sequenced.

Australian Climate Futures Identify future climate scenarios for scenario planning. To

help community stakeholders translate climate scenarios.

SA MDB NRM 1/07/2015 to

31/12/2015

Murray Basin NRM Adaptation Checklist To ensure that as we collectively develop actions to address

NRM issues with our partners, to ensure our planning

approach is flexible.

South East NRM Board 1/07/2014 to

30/06/2015

Murray Basin Projections Informed understanding of likely climate change impacts in

SA regions

AdaptNRM Biodiversity analysis The Carbon and Environmental Planting Guideline for SA

were informed by biodiversity analysis

North Central CMA 1/07/2015 to

31/12/2015

Murray Basin 3C Used to support the connectivity analysis.

Adaptive Pathways Helped in completing 4 adaptive pathways workshops that

have supported the planning process.

Mallee CMA 1/07/2015 to

31/12/2015

Murray Basin AdaptNRM (Module 1 – 4)

Sub project concepts developed to address Murray Basin

Cluster specific information/data needs have been integrated

into MCMA project planning to further refine regional context

and data related activities.

Wimmera CMA 1/07/2015 to

31/12/2015

Murray Basin Projections Used to underpin the second pass impact and vulnerability

assessment.

SCARP Adaptation Pathways Playbook Used to help inform planning for the adaptation workshops

Goulburn Broken CMA 1/07/2015 to

31/12/2015

Murray Basin Projections Used in the development of the Spatial Assessment Tool that

informed the development of the Climate Change Adaptation

Plan.

AdaptNRM Used in the development of the Strategy. The NRM Planning

Module provided useful guidance and highlighted a

framework to be considered in this project (i.e. adaptive,

flexible, multiple futures etc.)




Adaptation Pathways Project Principles applied to the Goulburn Broken Adaptation

Pathways Trial. Several workshops took place to align the

concept with the Spatial Assessment Tool and its outputs.

The project manager has been sharing the information

generated by the stream 2 project across the organisation.

NQ Dry Tropics 1/07/2015 to

31/12/2015

Monsoonal North Climate Knowledge Synthesis (Griffith

University)

n/a

Indigenous Knowledge (CSIRO & Griffith

University)

Discussed with Traditional Owners and with planners in

surrounding regions to determine good ways of presenting

climate data for different audiences

Decision support process map Benchmark for coverage of issues

Modelling and online tools (Scaling

Biodiversity Data, JCU)

Inclusion of climate issues in regional biodiversity strategies

within plan

Draft Report: Guide for NRM Planners

(CSIRO & JCU)

Contextual information to base NRM strategies to build

resilience in rural communities

"Beef Industry Case Study" (JCU) Contextual information on socio-economic dynamics of

grazing industry

Southern Gulf

Catchments Ltd

1/07/2014 to

31/12/2014

Monsoonal North Regional Drivers of Change Decision Making and Planning for Natural Resource

Management

AdaptNRM web platform (Biodiversity) n/a

Social Resilience of Agricultural

Landscapes (Final report)

n/a

Projections To influence the draft NRM Plan

Northern Gulf

Resource

Management Group

Ltd

1/07/2015 to

31/12/2015

Monsoonal North n/a n/a

North Coast LLS 1/07/2014 to

30/06/2015

East Coast 3C We have used the 3C products in our project and some of the

theory out of Carbon Farming work also.

Riverina LLS 1/07/2014 to

30/06/2015

East Coast n/a Stream 2 resources have been included in all discussions

with reference panel as appropriate.




Burnett Mary Regional

Group for NRM Ltd

01/07/2014 to

30/06/2015

East Coast Projections To confirm assessments of asset vulnerability to climate

change and to develop a region-specific infographic for plan

incorporation and community engagement activities. To

inform Round 2 expert panel deliberations, circulate to

regional networks and incorporate into the plan.

Regrowth to remnant – veg regeneration

value and mapping

Info considered in target drafting and to be incorporated into

mapping for draft strategy

Burnett-Mary Natural Resource

Management Region: Horticultural Sector

Material provided useful general background information to

inform plan drafting

List of reference materials To check that we have covered off on our review of

adaptation planning processes and also to see if any other

information was available for our region that we were

unaware of.

Fitzroy Basin

Association Inc.

1/07/2015 to

31/12/2015

East Coast n/a n/a

SEQ Catchments

Limited

1/01/2015 to

30/06/2015

East Coast Dowdy, A. et al. 2015, East Coast Cluster

Report

Informed the ‘Changing Climatic Conditions into the Future’

section of the updated SEQ NRM Plan

Climate projections Climate projection data for the SEQ Region was sourced from

the USQ. Data includes rainfall gradients, stable temperature

zones, Cool zones & maximum temperature zones all which

informed our climate refuges mapping.

Hunter Local Land

Services

1/07/2014 to

31/12/2014

East Coast National Projections Informed the draft risk assessment process and given an

indication of the climate variable that need to be factored

into the risk assessment and spatial analysis aspects of the

project.

AdaptNRM Accessed and have informed various aspects of the project.

Central West LLS Feb 2015 to July

2015

Central Slopes n/a Data and speakers have been utilised at a number of field

days and workshops aimed primarily at landholders to

ensure that they have the most up-to-date information for

decision-making.

North West LLS January 2015 to

June 2015

Central Slopes AdaptNRM 3C products Incorporated into addendum once they were adapted to suit

regional boundaries and more easily explained products for

community consultation

APSIM - Climate Change Impacts on

Cropping

n/a

Grass-Gro Climate Change Impacts on

Grazing

n/a

SWAT n/a

CO2 Sequestration products

n/a




Queensland Murray-

Darling Committee Inc

July 2014 to

December 2014

Central Slopes n/a n/a

Condamine

Catchment NRM

Corporation Ltd

1/07/2015 to

30/12/2015

Central Slopes 3C 3C Project data for consideration of how to incorporate into

the wildlife section of the plan in the final stage.

Reef Catchments 1/07/2016 to

30/12/2015

Wet Tropics AdaptNRM Weeds Module A presentation was given to the Mackay Regional Pest

Management Group on the outputs (and where to find spatial

projections) on outputs from the AdaptNRM Weeds module.

Cape York NRM 1/07/2015 to

31/12/205

Wet Tropics n/a n/a

n/a

n/a

FNQ NRM Limited 1/07/2015 to

31/12/2015

Wet Tropics n/a This reporting period we have referred to all of the

documents and publications from the Wet Tropics Cluster in

preparing the draft plan.

Projections Used and referred to the Climate Change in Australia Website

published as part of the National Projections project.

Desert Channels

Queensland

31/07/2015 to

31/12/2015

Rangelands n/a Stream 2 products inserted into the community information

plan to show the current research into the rangelands and

climate change. Considered to inform the NRM plan.

South East LLS 1/07/2014 to

30/06/2015

SCARP n/a SCARP assisted in the development of material for

presentation at both steering committee and working group

meetings and Peat Leith attended both meetings and

presented on behalf of SCARP.

Adaptation Pathways Adaptation pathways continue to be examined in detail and

an Adaptation Pathways Playbook was published to assist

with the development of the plan.

Cradle Coast Authority 1/07/2015 to

31/12/2015

SCARP AdaptNRM website for Biodiversity n/a

NRM South 1/07/2015 to

31/12/2015

SCARP Projections Included in the final draft NRM Strategy for Southern

Tasmania as a centrepiece. To inform the development of the

‘Possible Futures’ sections and to inform Target setting and

Priority Actions.

SCARP Reports To inform the development of the ‘Possible Futures’ sections

of the draft strategy and to inform Target setting and Priority

Actions. Incorporated into the final draft NRM Strategy for

Southern Tasmania.

Southern Slopes Information Portal

Report

Particularly useful in articulating the ‘Possible Futures’

sections of the strategy and also guiding the threats and

opportunities to the natural resources in Southern Tasmania.




Corangamite CMA 1/07/2014 to

30/06/2015

SCARP SCARP Reports A valuable source of information and resource during this

reporting period to develop a planning process that is logical,

comprehensive and transparent.

Adaptation Pathways for NRM Embedded capability by the Corangamite CMA to identify and

use tools and frameworks in a consistent manner to inform

NRM planning for climate impacts and adaptation, and its

implementation.

East Gippsland CMA 1/07/2014 to

30/06/2015

SCARP SCARP Reports n/a

Projections n/a

AdaptNRM n/a

Glenelg Hopkins

Catchment CMA

1/07/2015 to

31/12/2015

SCARP SCARP Reports Incorporated into the development process and the draft

document. During this period the Southern Slopes projections

were further incorporated into the document.

PPW CMA 1/01/2015 to

30/06/2015

SCARP Four adaptation pathways planning

workshops with regional stakeholders

between February & June 2015.

Information from these workshops will be used to refine the

NRM & Climate Change plan.

Adaptation pathways Adaptation pathways method is consistent with the intent of

the Port Phillip & Western Port RCS and was used to shape

our project’s direction and develop content for the plan.

Information portal Accessing information portal for literature about: Impacts of

climate change to our region’s environmental assets;

Adaptation pathways; Carbon sequestration and climate

change mitigation.

West Gippsland CMA 1/07/2015 to

31/12/2015

SCARP Projections Used to underpin the second pass impact and vulnerability

assessment.

SCARP NRM Toolkit Used to help inform writing the Strategy.

SCARP Adaptation Pathways Playbook Used to help inform planning for the adaptation workshops.

Northern Tasmanian

NRM Authority

30/01/2015 to

1/01/2016

SCARP Projections Incorporation of climate projections information into the

consultation draft

SCARP Spatial Prioritisation report To inform prioritisation processes. Development of the

Environmental Strategic and Institutional Scans report

included consideration of available tools, resources and

learnings from SCARP and AdaptNRM.

AdaptNRM modules Informed planning principles for conservation management

and biosecurity. Development of the Environmental Strategic

and Institutional Scans report included consideration of

available tools resources and learnings from SCARP and

ADAPTNRM programs.




South West NRM Ltd 1/01/2015 to

30/06/2015

South West

Flatlands

Scott, John K. 2014, Cenchrus ciliaris

(buffel grass) and climate change, CSIRO

Ecosystem Sciences, Wembley, WA.

Used for physiological information and climate change

modelling distribution.

Scott, JK, Webber, BL, Murphy, H Ota, N,

Kriticos, DJ and Loechel, B 2014,

AdaptNRM Weeds and Climate Change:

supporting weed management

adaptation

Used for flagging management actions

Eyre Peninsula NRM

Board

1/07/2015 to

31/12/2015

South West

Flatlands

Climate Change in Australia - Projections

for NRM regions - Southern and South

Western flatlands cluster

Used to develop the five sub-regional plans by including a

summary of future conditions.

Kangaroo Island NRM

Board

1/07/2015 to

31/12/2015

South West

Flatlands

n/a Stream 2 products, including the regionalised projections and

AdaptNRM modules, continue to be used where possible.

Wheatbelt NRM Inc. 1/07/2015 to

30/12/2015

South West

Flatlands

CENRM Plant Refugia under Climate

Change (A2- 2080).

Mapped distribution data for threatened flora, birds, fauna;

revegetation species, coastal vegetation, economic

vegetation, vegetation types. Includes mapping of over 700

individual species.

NCCARF Biological Refugia under climate

change.

n/a

Projections n/a

AdaptNRM Weeds and Biodiversity

modules

n/a

South Coast Natural

Resource

Management Inc.

1/07/2015 to

30/12/2015

South West

Flatlands

Ford, B. & Cook, B. (2015). Southern and

South-Western Flatlands climate change

project: Data layers explained.

n/a

Plant Refugia under Climate Change (A2-

2080)

n/a

Species distribution modelling n/a

Mapped distribution data n/a

NCCARF: Biological Refugia under climate

change

n/a

Rogers DJ (2013) A framework for nature

conservation under future climates, to

inform climate adaptation planning in the

Adelaide & Mt Lofty Ranges NRM region

n/a

Projections n/a

AdaptNRM n/a




Perth NRM 1/07/2015 to

31/12/2015

South West

Flatlands

CENRM Species Distribution & Bioclimatic

Modelling

To inform PNRM program development through the Living

Landscapes and Living Wetlands programs. Used in MCAS-S

models.

Spatial data layers n/a

Projections Used to inform PNRM program and project development and

implementation. CSIRO/BOM has also been used on the

Swan Region Strategy website to provide an overview of the

latest climate change projections for this region. To update

information provided on the Swan Region Strategy website.

To inform a summary report of the findings of the PCC

project.

*A total of 39 Stream 1 annual project reports were received for the reporting periods 2014-15 and 2015-16 of which 35 reported using Stream 2 products.

**Responses to ‘What was used?’ and ‘How was it used?’ are extracted from: “During this reporting period, did you use any of the information and resources

developed under Stream 2 of the Regional NRM Planning for Climate Change Fund?” and “If YES, what information or resources did you use and how?”.



Attachment Five: Methodology

The overall methodology for Stream 2 M&E was designed by the M&E consultant team drawing

on consultation with the Department and project representatives. This methodology is

documented in an overarching program level M&E Framework. All M&E activities are

independently led and coordinated by Clear Horizon but regular input is provided by cluster

projects through annual project-level M&E reporting and participation in M&E activities. The

final evaluation was lead independently by the Clear Horizon team, who were also active in

coordinating project M&E over the time –frame of the program (2013 – 2016).

The evaluation has used mixed methods and a range of data sources to address the key

evaluation questions (KEQs).

Approach

To meet this evaluation purpose an evaluation methodology (outlined in Section 3) designed

and delivered as part of a broader suite of monitoring and evaluation (M&E) activities

undertaken by the evaluation team during the period of Stream 2 implementation. The

approach to this evaluation was participatory in that Stream 2 projects and departmental

representatives played an active role in setting the agenda and parameters for inquiry at the

planning stage (2013), in providing monitoring inputs through annual reporting, (2013- 2016)

and by assisting with the synthesis and interpretation of evidence at the final evaluation

reporting stage (2016). By explicitly recognising and responding to the needs of the

commissioners and users of this evaluation the evaluation team have taken a utilisation-

focused approach (Patton, 1997).

This evaluation has also taken a theory-based approach whereby a program logic model was

developed and used to articulate clarify the contribution of Stream 2 projects to the expected

goals and outcomes for the program. Finally, by articulating the mechanisms for change in the

program and seeking to situate outcomes achieved and case studies within the broader

regional and cluster context for delivery, this evaluation has also drawn on aspects of the

‘realist’ approach to evaluation (Pawson and Tilley, 2004). Finally, by drawing on a range of

qualitative and quantitative data sources this evaluation has taken a mixed methods approach

to addressing the information needs for Stream 2.

Data collection

Data was collected via multiple methods outlined below including desktop review, interviews

and case studies.

Desktop review

Documentation including the interim evaluation (2014), project reporting (2013 – 2016) and

project-specific evaluation/s (2016), as well as a sample of 16 NRM plans, a total of 39

regional Stream 1 reports submitted to the Department for the years 2014-15 and 2015-16

were reviewed.



These documents been synthesised against the KEQs to provide an evidence base from which

to develop evaluation findings. Where possible, multiple data sources and lines of evidence

have been used to triangulate, verify and support findings.

Interviews

A total of 20 Interviews were conducted with targeted ‘end-users’ of Stream 2 outputs NRM

planners. Planners – or actors involved in ‘planning’ – were selected purposively on a

cluster/project basis in accordance with their use of Stream 2 products and interaction with

Stream 2. Informants were identified in consultation with the projects. Interviews were

conducted face-to-face and via phone depending on the location and availability of informants.

In addition to ‘end-users‘, a total of 13 project representatives were interviewed to gather data

on program effectiveness, legacy and the appropriateness of the program model. Every project

was represented in this series of interviews.

To ensure privacy interviewees were assigned codes which were used for analysis and

verification. See Attachment Seven for the interview guides used.

Case studies

Case studies of specific instances of practice change have been developed to provide an in-

depth illustration of outcomes and the causal mechanisms that lead to the change. Case

studies and ‘episodes’ for investigation were identified from the descriptive interview data

provided by planners based around themes that were discussed and developed with the

Department prior to investigation. Case studies have also been chosen to represent different

regions and clusters in the program. They are presented in boxes throughout this report.

Data analysis

Qualitative and quantitative data was analysed using techniques appropriate to the data type

(numerical or descriptive) and collection method. Each data source collected was analysed

individually and ‘cleaned’ to ensure that the information clearly addressed the purpose of the

evaluation and KEQs. Quantitative (numerical) data were analysed descriptively to identify

frequency, range and distribution using appropriate statistical techniques. Qualitative data

sources such as interview transcripts were analysed and cross-analysed to identify emergent

themes in relation to the KEQs. Where possible multiple data sources were used to develop

findings.

Data synthesis

Data was brought together via desktop review, interview and case study in a ‘results chart’ to

collate, synthesise and develop findings. Evidence from interviews and analysis of regional NRM

plans were used to develop case studies demonstrating program outcome/s; the significance of

the outcome/s; and the contribution of the project/s and/or program. At the National Workshop

on 15-16 March 2016 the evaluation team facilitated a half-day session (3 hrs) to interpret the

results and develop interim findings. During this workshop, participants including project and

departmental representatives reviewed a range of data sources and through a facilitated

process began to develop likely findings and themes for the evaluation. Evaluative judgements



were made at the reporting stage drawing on indicators, rubrics and through an analysis of the

overall weight and value of the collected evidence against each KEQ.

Limitations

This report partly relies on self-reported data contained in the annual reports of the

national projects and regional clusters. These reports have not been independently

verified.

The outcomes and achievements reported in the national and cluster annual reports,

and synthesised in this report, have not been independently verified with reference to

original sources.

For some findings, the scale of change is not known. For example, the reports do not

always provide information about the total population of ‘NRM planners’ or indicate how

many planners have made changes. Similarly, with planner interviews, the extent of

change reported and presented may not be representative of the region or the NRM

group on which they are speaking on behalf of.



Attachment Six: Interview guides

Stream 2 Final Evaluation Interview guide: NRM Planners

We have been contracted by the Department of Environment (the Department) to evaluate the

program ‘Stream 2 of the Regional NRM Planning for Climate Change Fund 2013 – 2016’. This

program has been established to support better NRM climate change planning by NRM bodies.

Stream 2 has provided research and support to NRM decision makers through a series of

projects delivered nationally. Your [NRM ORGANISATION] has participated in this program

through the [PROJECT / CLUSTER]. As an NRM group planner and/or representative from [NRM

REGION] we would like to interview you to understand if the climate change information,

support and resources provided by the [PROJECT] have been useful and valuable for NRM

planning in your region.

The interview will take around 30 – 45 minutes of your time. Just to confirm is now a good time

to talk? Yes / No

We won’t identify you personally but in the report that we prepare for the Department we may

associate your comments with your NRM organisation if that is ok with you? Yes / No

1. What is your NRM organisation and position?

NRM body/organisation

Position

2. Which Stream 2 project/s have you been involved with?

Project/cluster (i.e. Rangelands)

Length/duration of involvement with [PROJECT]

3. How have you been involved?

Project activities

i. Consultation (i.e. workshops etc.)

ii. Products

National projections project activities

AdaptNRM project activities

Other forums/groups to support climate change in planning?

4. What products, tools, resources or information have you received from [PROJECT]?

5. How useful have you found these products, tools, resources or information in assisting

NRM planning for your region?

Format

Scale

Applicability

Accessibility



Understability/applyability (i.e. were you able to use it?)

Timing

6. Have you used these products, tools, resources or information? Yes / No

If so, for what?

Provide example/s (who/what/when/where etc.)

7. Do you intend to use these products, tools, resources or information?

If so, how?

If not/why not?

8. How well has [PROJECT] provided support to you?

To interpret and apply tools

To access advice

To deal with uncertainty and risk in planning

9. How has [PROJECT] supported you to develop your own skills and ability?

To deal with uncertainty and risk in planning

To interpret and apply tools

To make NRM decisions taking into account climate change science and

research

10. How has [PROJECT] supported [NRM ORGANISATION] to plan for NRM under climate

change?

Awareness and understanding of climate change in NRM planning

Partnerships and linkages (i.e. with researchers, other NRM groups etc.)

Conceptual approaches/frameworks (i.e. resilience thinking)

Planning processes and systems (i.e. for updating/renewing plans)

Adaptability/flexibility and ability to update the planning

11. Do you have any other comments about [PROJECT]



Stream 2 Final Evaluation Interview guide: Project Leaders (Impacts and Adaptation)

As you know we have been contracted by the Department of Environment to provide M&E

support to the ‘Stream 2 of the Regional NRM Planning for Climate Change Fund 2013 – 2016’

(Stream 2) program. We are now conducting the final evaluation for Stream 2 for the

Department. This evaluation is looking at a range of things including the effectiveness and

legacy of Stream 2. Your [PROJECT] has delivered climate change research and support to NRM

planners as part of Stream 2 from 2013 – 2016.

The interview will take around 30 – 45 minutes of your time. Just to confirm is now a good time

to talk? Yes / No


associate your comments with your Project if that is ok with you? Yes / No

1. Just to recap, can you tell me about your role in the project?

Project team

Cluster

Overview of activities

2. What have been the highlights of the project? Why?

3. What have been some of the challenges of the project? Why?

4. From some of the things that you have discussed, what so far has been the most

significant positive or negative change or outcome for you because of this project?

Note that this is about change not process (i.e. not about timing of delivery and/or how the

program has been delivered).

Capacity (Planner and/or researcher capacity; individual or organisational)

Availability / application of research

Relationships, linkages etc.

5. What, if any changes or outcomes have you noticed in terms of the capability of your

team to plan for NRM under climate change?

Ability to work with NRM planners

Application of tools and approaches at a regional level

Awareness of regional issues

Projections

Other

6. How did you use the National Projections work in your cluster?

As an input for products / tools etc.

Provided information to planners / NRM groups etc.

Didn’t / limited use

Other



7. How well did the National Projections meet your projects needs in terms of delivering

climate change information to NRM groups?

Quality of information

Relevance at cluster/region level/s

Ease of use, relevance, accessibility and application

8. How well did the National Projections project support your project to access and use the

National Projections?

Access

Use

9. How well did the national AdaptNRM project support NRM regions in your cluster to plan

for climate change?

Access

Use

10. Finally, what legacy has your project has left for ongoing benefits to be realised in the

[CLUSTER]? Are there any major challenges / barriers that may prevent ongoing

benefits from being realised?

Capacity; i.e. awareness of climate change in NRM planning

(planner/researcher)




Adaptability/flexibility and ability to update planning

11. Do you have any other comments?



Stream 2 Final Evaluation Interview guide: National Projections

As you know we have been contracted by the Department of Environment (‘the Department’) to

provide M&E support to the ‘Stream 2 of the Regional NRM Planning for Climate Change Fund

2013 – 2016’ (Stream 2) program. We are now conducting the final evaluation for Stream 2 for

the Department. This evaluation is looking at a range of things including the effectiveness and

legacy of Stream 2.Your [PROJECT] has delivered climate change projections to projects and

NRM planners as part of Stream 2 from 2013 – 2016.

The interview will take around 30 minutes of your time. Just to confirm is now a good time to

talk? Yes / No



1. Just to recap, can you tell me about your role in the project?

Project team

Overview of activities

2. What have been the highlights of the project? Why?

3. What have been some of the challenges of the project? Why?

4. From some of the things that you have discussed, what so far has been the most

significant positive or negative change or outcome for you because of this project?

Capacity (Planner and/or researcher capacity)

Availability of research

Application of research

Relationships, linkages etc.

5. What, if any changes or outcomes have you noticed in terms of the capability of the

Projections team to deliver research for NRM planning?

Ability to work with NRM planners

Awareness of regional issues etc.

6. Finally, what legacy has the National Projections left for ongoing benefits to be realised?

Are there any major challenges / barriers that may prevent ongoing benefits from being

realised?

Data; products; website etc.









Stream 2 Final Evaluation Interview guide: Department of Environment (Stream 1 and

2)

As you know we have been contracted by the Department of Environment (‘the Department’) to

provide M&E support to the ‘Stream 2 of the Regional NRM Planning for Climate Change Fund

2013 – 2016’ (Stream 2) program. We are now conducting the final evaluation for Stream 2 for

the Department. This evaluation is looking at a range of things including the effectiveness and

legacy of Stream 2. You have been identified as a key informant who can contribute to our

evaluation of Stream 2 from 2013 – 2016.

The interview will take around 30 minutes of your time. Just to confirm is now a good time to

talk? Yes / No



Time & date: Interviewer:

1. Just to recap, can you tell me about your role in Stream 2?

Stream 1

Stream 2

the Department

2. Stream 2 was designed to better connect research with planning. How well do you think

the cluster approach supported the program to meet this need?

What has worked well? What hasn’t worked well?

Which aspects of the cluster approach have suited the intent of the program?

Which aspects could be better suited?

3. Stream 2 consisted of two national projects and eight cluster projects to deliver climate

change science to planners. How well do you think this model supported the program to

fulfil this purpose?

What has worked well? What hasn’t worked well?

4. Finally, what legacy has Stream 2 left for ongoing benefits to be realised?

Data; products; website etc.



Partnerships and linkages (i.e. with researchers, NRM groups etc.)






Attachment Seven: Output definitions Table 9. Output definitions used by Stream 2 projects

Performance

Indicator Year Output type

Definition

Research teams have

worked with regional

NRM organisations,

through the eight

clusters, to identify

priority climate

change issues and

the information they

will need to inform

planning (end year

one of the program)*

2013

Consultation

Events or activities where feedback is

sought from NRM organisations.

Feedback can be sought via meeting,

teleconference, focus group, survey,

workshops etc.

Workshop

An event hosted by the project team

whereby project, cluster and regional

NRM representatives come together

to discuss, identify, prioritise issues

and information.

Presentation

A speech or a talk delivered by a

project team member where

information is presented to NRM

organisations for the purposes of

then prioritising issues or

information.

Program management

meetings

Internal project meetings for the

purposes of project management.

Products

Items produced and attributable to

the project. Can include a range of

outputs such as articles, newsletters,

module outlines, reports, datasets,

guidance documents and fact sheets.

These products are focus on

gathering, confirming, prioritising

climate change issues or information.

Training

A unit, module, course or exercise

delivered to provide information with

the intent of identifying and

prioritising NRM planning needs.

Other

Any other activities and outputs

delivered by your project to identify

and prioritise NRM planning needs.

Quality and timely

climate change

information and

support is delivered to

regional NRM

organisations via the

eight clusters

2014

2015

2016

Training


delivered to provide information or to

train people to use information.

Products





articles, guidance documents and

fact sheets. These products are

focused on delivering information or

data.

Workshops An event hosted by the project team





Performance

Indicator Year Output type

Definition

to discuss information and products

including data.

Presentations




organisations.

Mentoring (e.g. one-on-

one support)

Instances of tailored advice and

support provided to an individual or

group within an NRM organisations

around climate change information.

Peer reviews

Evaluation of scientific, academic

products developed by the project by

others working in the field.

Next generation

climate change

projections are

delivered to regional

NRM organisation via

the eight clusters

(once-off deliverable)

(years two through

four of the program)

2014

2015

2016

Training


delivered to deliver next generation

climate projections developed by

CSIRO Element One National Project.

Products





articles, guidance documents and

fact sheets. These products are

focussed on delivering next

generation climate change

projections.

Workshops

An event hosted by the project team



to discuss, next generation climate

change projections.

Presentations




organisations particularly on next

generation climate change

projections.

Mentoring

Instances of tailored advice and

support provided to an individual or

group within an NRM organisations

around climate change projections.

Other (Specify)

Any other activities and outputs

delivered by your project to deliver

next generation climate projections

developed by CSIRO Element One

National Project.



Attachment Eight: Glossary of terms Table 9.Glossary of terms

Term Definition

Activities Activities are conducted to bring a change in a situation or behaviour that is

expected to contribute to outcomes. For example, incentives scheme advertised,

workshops run, awareness raising.

Appropriateness A measure of whether an intervention is suitable in terms of achieving its desired

effect and working in its given context. Suitability may apply, for example, to

whether the intervention is of an appropriate type or style to meet the needs of

major stakeholder groups.

Broader goals Are long-term goals that the program outcomes are expected to contribute towards

(it is acknowledged that many other factors and programs are also contributing to

these broader goals).

Effectiveness The extent to which an initiative/project meets it intended outputs and/or

objectives.

Efficiency The extent to which activities, outputs and/or the desired outcomes are achieved

with the lowest possible use of resources.

End of Program

Outcome

The desired final result of the program (may be some years after the program has

finished).

End-user The ultimate persons that will use or benefit from the products and services

delivered by the program.

Evaluation Essentially, evaluation is about untangling the threads of “what happened”, “why”,

and “to what effect”. It can be used to assess the appropriateness, effectiveness

and efficiency of the intervention, as well as to inform decision making and

demonstrate accountability.

Immediate

outcomes

Any immediate changes or tangible products that are a direct result of the activities

– e.g. attendance workshop, awareness raised, or compliance.

Indicator A quantitative or qualitative factor or variable that provides a simple and reliable

means to measure achievement, to reflect the changes connected to an

intervention, and or to help assess the performance of a development action.

Intermediaries A person or group (such as a knowledge broker) who acts a link between program

stakeholders (such as researchers and end-users).

Intermediate

outcome

Medium term outcomes that occur as a result of the outputs and that are

necessary preconditions for the achievement of end-of-program outcomes.

Legacy The extent to which the benefits of a project/initiative extend beyond its

implementation.

Monitoring Monitoring is the on-going process of collecting routine data, usually internally, to

track progress with previously identified activities and outputs.

Outcomes The results or changes that can be attributed to the activities (eg. changes in

stakeholder’s knowledge and skills). Outcomes can be at the short, medium or

long-term scale.

Outputs Outputs are usually relatively immediate and are the direct results or products of

activities.

Primary audience The stakeholders who will receive the evaluation results directly, and who will use

the information for decision-making, such as program continuation or

improvement.



Program logic A visual depiction of the program theory and logic behind how activities lead to

outcomes. It is usually represented as a diagram that shows a series of causal

relationships between inputs, activities, outputs, outcomes and goals.

Qualitative Observations or information expressed using words rather than numbers.

Quantitative Information about quantities; that is, information that can be measured and written

down with numbers.

Relevance The extent to which the intended program outcomes addressed the program

context and end-user needs.

Rigour Rigorous (“trustworthy”) research that applies the appropriate research tools to

meet the stated objectives of the investigation. Rigorous research is both

transparent and explicit; in other words, researchers need to be able to describe

what they did (or plan to do) in clear, simple language.

Rubric An attempt to communicate expectations of quality around a task. In many cases,

scoring rubrics are used to define consistent criteria for grading or scoring. Rubrics

allow all stakeholders to see the evaluation criteria.

Scope A written description of the breadth of work for an evaluation (a description of what

is included).

Utilisation Utilisation-Focused Evaluation is an approach based on the principle that an

evaluation should be judged on its usefulness to its intended users. Therefore

evaluations should be planned and conducted in ways that enhance the likely use

of the findings to inform decisions and improve performance.

http://atlasti.com/research-tools/

stream 2 of the regional nrm planning for climate change ...environment.gov.au › system › files...

Documents