Monitoring, Evaluation, Reporting and Improvement Strategy –Caring for our Country and the Biodiversity FundApril 2013

Contents

1 Introduction 42 Purpose of this strategy

63 MERI principles and Key Evaluation Questions

74 Implementing MERI — funding recipients (project level MERI)

85 Implementing MERI — the Australian Government (program level MERI) 136 MERI Toolkit

157 Terminology

15

1 Introduction

To help understand how investments to improve our natural resource management (NRM) and biodiversity conservation are making a difference, information about where investments are made and what they are achieving is needed. This information can also be used to help understand how we can improve the design and delivery of projects and programs.

Monitoring, Evaluation, Reporting and Improvement (MERI) are simple concepts that, when applied, help us understand what is being achieved and help identify possible improvements, for projects and programs. The figure below provides a simple depiction of the MERI cycle.

Figure 1: The cycle on monitoring, evaluation, reporting and improvement

Funding recipients and the Australian Government have a role to play in ensuring that Caring for our Country and the Biodiversity Fund are adequately monitored, evaluated and reported to allow identification of achievements and integration of improvements.

Investments in NRM and biodiversity conservation are made through a number of different funding streams. Each of these funding streams uses specific documentation and processes. However, information from each of these streams will be used to understand progress towards the broader Caring for our Country and Biodiversity Fund

strategic objectives. Data and information from the following will be collected and reported as part of this MERI Strategy:

The Biodiversity Fund1

Caring for our Country, including: Regional Delivery Sustainable Environment stream Sustainable Agriculture stream

Implementation of this MERI Strategy will be supported by a range of other documentation that sets specific requirements for recipients under the relevant funding streams, including program guidelines, funding agreements, and planning and reporting templates (provided as part of the MERI Toolkit (see section 6). Further information on Caring for our Country and the Biodiversity Fund can be found at the Caring for our Country andBiodiversity Fund web pages.

This MERI Strategy differs from the previous Caring for our Country MERI strategy in that it:

Provides a consistent and integrated approach to monitoring evaluating, reporting and improvement for both Caring for our Country and the Biodiversity Fund.

Clarifies the role of funding recipients in providing project level data, and strengthens the processes which enable this.

Strengthens the Australian Government commitment to making as much information as possible public, to help maximise learning about how best to invest to address the natural resource management and biodiversity conservation challenges that Australia faces.

2 Purpose of this strategy

This document outlines the approach needed from funding recipients and the Australian Government, to ensure:

Funding recipients can track and report on project progress and achievements.

The Australian Government can track program progress and report on program achievements, including their impacts on the environment, natural resource base and community.

Information collected along the way is used effectively, including by funding recipients and the Australian Government to inform ongoing and future design and delivery of projects and programs.

Accountability of public expenditure.

To build a picture of how our NRM and biodiversity conservation investments are making a difference, information from a range of sources is needed. This MERI Strategy aims to support the collection and communication of information at three levels, as depicted in Figure 2.

1 Biodiversity is one component of the broader Land Sector Package, which also includes Regional Natural Resource Management Planning for Climate Change Fund and the Indigenous Carbon Farming Fund. These funds will implement separate though complimentary MERI processes to this strategy.

Figure 2: Information from three sources will be used tobuild a comprehensive picture of program achievements

3 MERI principles and Key Evaluation Questions

The following principles should be applied when designing and implementing MERI at both project and program level:

Fit for purpose: Tailored to the design, purpose and objectives of Caring for our Country and the Biodiversity Fund projects and programs.

Credible: Based on scientifically and administratively defensible methods and approaches.

Transparent: Clearly demonstrate how public money has been spent, the resulting outputs, and where possible outcomes.

Cost effective: The cost of MERI activities is within allowable limits, and where possible draws on and links with existing processes and information.

The following Key Evaluation Questions should help frame the broad approach to MERI at the project and program level. Based on the general Key Evaluation Questions below, project and programs may develop more specific project or program evaluation questions.

Effectiveness: Were the planned activities performed, and to the standard required?

Impact: What is the outcome, or result of having done the planned activities - i.e. what changes in the condition of environmental assets, changes in management practices and/or changes in institutions can be observed?

Appropriateness: Was this the best way to have addressed the problem? Did the activities and the way they were undertaken align with stakeholder needs and expectations. Were they consistent with relevant NRM policy and priorities?

Efficiency: Did the project / program achieve the desired results within budget? To what extent did the project / program attain the highest value from available resources – could the same outcomes be achieved more simply and for less cost next time?

Legacy: Will the project / program continue to have an impact after its completion?

4 Implementing MERI – funding recipients (project level MERI)

Funding recipients can allocate a portion of their total project budget to MERI activities, in accordance with the relevant funding stream guidelines.

In general, Caring for our Country and Biodiversity Fund funding recipients will be required to:

1. Develop a Project MERI Plan (see 4.1 below).

2. Undertake monitoring and evaluation activities to collect information as evidence of project progress and achievements (see 4.2 below).

3. Provide progress reports throughout the life of the project, and a final report upon completion of the project (see 4.3 below).

MERI requirements differ for the Biodiversity Fund and Caring for our Country. Biodiversity Fund projects are relatively similar and so monitoring and evaluation activities are largely predetermined. Caring for our Country will invest in a wide range of projects and so individual MERI approaches will need to be developed.

NOTE: Not all Caring for our Country funding programs refer to ‘Project MERI Plans’ specifically -specific MERI related requirements are detailed in relevant funding stream specific documentation, including guidelines, funding agreements and templates.

4.1 Project (MERI) Plans Caring for our Country and Biodiversity Fund recipients may be required to develop a plan (using the funding stream specific templates), at the commencement of their project, which in general aims to:

Document the detail and structure of the project. This helps identify which aspects or stages of the project should be monitored (some project templates will call for the development of a program logic — see text box on page 9)

Monitor and evaluate progress and collect evidence of achievements (see 4.2)

Provide detail of communication activities. Communication of project activities, experiences, and achievements amongst funding recipients, the community and the broader public is an important aspect of MERI.

The type and complexity of monitoring and evaluation required for Caring for our Country funded projects will depend on the type and size of the project. This is reflected in relevant templates, with smaller grant recipients required to develop a simple project MERI plan, while larger projects will be required to develop a more comprehensive plan. Further information on the MERI requirements for Caring for our Country is described in section 4.2.

What is program logic?

Program logic is an approach to project or program planning that illustrates simply, the logic behind how activities lead to outcomes. Program logic is usually represented as a diagram or matrix, which shows a series of expected consequences eventually resulting in the final outcome. This structure helps identify where monitoring activities should be integrated throughout the project, to show whether the project is progressing as planned and demonstrate achievements. The diagram below illustrates the types of ‘outcomes’ that occur at the different stages of a project or program. Example program logics are provided in the MERI Toolkit (see section 6).

Figure 3: Basic structure of a NRM program logic

4.2 Project monitoring and evaluation

Funding recipients are responsible for collecting and reporting project level data that demonstrates progress and achievement of project outcomes. The type of information that is collected will vary depending on the type and size of the project. It will also depend on the particular outcomes that are being monitored or evaluated. Generally speaking, funding recipients have less control and influence on longer term outcomes (see Figure 3, above) and so these outcomes can be harder and more costly to monitor and evaluate.

This section of the strategy provides an outline for funding recipients of both Caring for our Country and the Biodiversity Fund to guide monitoring and evaluation activities.

4.2.1 Caring for our Country

The way monitoring and evaluation is undertaken at the project level will vary across the broad variety of Caring for our Country projects.

When thinking about the type and purpose of monitoring and evaluation at the project level, funding recipients should consider:

The type of monitoring and evaluation activities that are feasible given the available MERI budget (set out in the funding agreement) and project team capacity.

Key elements (activities or outcomes) of the project that should be monitored and evaluated to provide evidence of progress and /or achievements — what is key in understanding the progress and success of the project?

Program outcomes and the types of monitoring at the project level that would help demonstrate the project’s contribution to the program outcomes.

Existing monitoring and evaluation approaches that can be used to ensure consistency across projects, within regions, across the program etc.

As part of its commitment to MERI, the Australian Government will advise on possible monitoring and evaluation approaches (including standard indicators) to promote a consistent approach to collecting and assessing project level data. This advice will be made available on the Caring for our Country website.

To fulfil monitoring and evaluation activities funding recipients can use their MERI budget to purchase third party specialised services. For example a project may choose to engage a third party to provide specialised monitoring, or to assess monitored data and information and provide an evaluation of their project. Intentions to engage a third party evaluator should be detailed in the Project MERI Plan.

Table 1 (below) indicates the types of monitoring approaches at different project levels and for different project capacities, and provides some example evaluation questions.

Project outcome level

Purpose of monitoring

Evidence need

Monitoring approach (e.g)

Evaluation Questions (e.g)

Longer term project outcome

To help understand the NRM or biodiversity outcome (or impact) of the project.

Evidence of asset condition change(e.g. enhanced ecosystem health)

• (existing or new) scientifically robust methodology applied.

• Use (existing or new) indicators of change

• What was the impact of the project?

• Were there unintended impacts

• Were there external factors which impacted on the project.

Intermediate project outcome

To help understand whether the project (effectively) achieved the management or conservation activities it set out to.

Evidence of project deliverable (e.g. reduction in density of weeds / pests)Evidence of better practice adopted.

• Baseline and post activity pest numbers on project site

• Photo point (GPS linked) documentation

• Follow-up survey with participants

• Recognised ‘management’ actions recorded

• Did the project achieve what it set out to and to the standard expected, and within budget?

• What could have been done differently / better?

Immediate project activities and outcomes

To understand whether a specific project activity occurred and had the intended result (was appropriate) in order for the project to progress.

Evidence of revegetation activities completedEvidence of workshop attendance

• Photographic documentation• Registration of attendees

and attendee satisfaction with workshop

• Was the activity completed and did it have the intended result?

• Was the activity appropriate, or would a different approach be better?

4.2.2 Biodiversity Fund

All Biodiversity Fund projects, with the exception of Round 1 projects, with approved Funds greater than or equal to $500,000 (GST exclusive) are required to collect ecological monitoring data and report the data collected to the department. This data can be collected using one of the following recognised methods, the department may add additional methodologies to this list:

Habitat Hectares BioMetric: Terrestrial Biodiversity

Tool BioCondition TasVeg: Tasmanian Vegetation

Condition Assessment Method Bushland Condition Monitoring Native Vegetation Condition

Assessment & Monitoring for WA

If you do not intend to use one of these listed monitoring methods, you must then collect and report ecological monitoring data to the department in accordance with the department’s Biodiversity Fund Ecological Monitoring Guide.

All Biodiversity Fund funding recipients will be required to answer the evaluations questions provided by the Australian Government within the relevant reporting templates. This will require providing simple data about the project and some observations about how the project is progressing. This data and information will be used to help understand how Biodiversity Fund projects are contributing towards the program objectives.

The reporting templates and the Biodiversity Fund Ecological Monitoring Guide are available at the Biodiversity Fund website.

4.3 Project reporting Funding recipients will be required to report periodically to the Australian Government on the progress of their project, and provide a final report on project completion, using the relevant reporting templates. Information from these reports will be used by the Australian Government to communicate publicly the location, objectives and other details of projects. The Australian Government will also consolidate project level reports to track progress and communicate program outcomes to stakeholders and the broader community.

Periodic and final reporting will generally involve:

Summarising delivery of project activities in the reporting period including a summary of monitoring results.

Reporting progress towards project outcomes, including using available standard measures (provided in reporting templates).

Identifying lessons learnt and necessary changes to the project design or delivery

Providing any other relevant information including maps, photos, case studies, raw data etc.

Financial reporting.

5 Implementing MERI – the Australian Government (program level MERI)

The Australian Government is committed to accounting for, and demonstrating outcomes from, its NRM and biodiversity conservation investments.

The Australian Government will be responsible for program level MERI for Caring for our Country and the Biodiversity Fund, including:

Providing advice and developing processes, systems and MERI documentation to support funding recipients undertake their MERI responsibilities.

Collating, analysing and reporting on consolidated project data.

Storing and making publicly available, relevant project and program information.

Supporting program level monitoring, research and evaluation activities across themes, landscapes or regions.

Providing timely updates on program progress and achievements.

Separate program level MERI plans for the Caring for our Country Sustainable Environment and Sustainable Agriculture streams, and Biodiversity Fund have been developed to outline the strategic objectives, outcomes and investment approaches for monitoring and evaluation at the program level. These will be available on the Caring for our Country and Biodiversity Fund websites.

Where applicable, Program MERI Plans will also outline relevant project level monitoring approaches (including standard indicators), which align with the program monitoring approach. These plans will be updated from time to time, and will be available on the Caring for our Country and Biodiversity Fund websites.

5.1 Program monitoringThe Caring for our Country and Biodiversity Fund program monitoring will be underpinned by three types of monitoring data (also see Figure 2):

1. Project information — information provided by funding recipients to the Australian Government will be collated, analysed and reported as it becomes available. Project information will be presented for individual projects and also help provide snap shots on program progress.

2. Targeted monitoring — The Australian Government will support monitoring activities, which complement information provided at the project level, to enable an understanding of how the program is progressing towards its objectives and impacting on NRM and biodiversity assets. Monitoring activities may focus on particular geographic areas (including project sites), themes, and assets. Results from monitoring activities will inform evaluation processes (see 5.2).

3. Landscape scale, research and other data — Information from existing sources will be drawn on to provide further evidence of changes to the natural resource base, and as context for understanding the programs impact. This may include remotely sensed data (such as extent of native vegetation), or supporting relevant scientific research.

5.2 Program evaluation Specific theme or asset based evaluations may be undertaken, for Caring for our Country and the Biodiversity Fund to provide further depth and analysis of the achievement and / or to inform future program design and delivery.

Evaluations for Caring for our Country and the Biodiversity Fund will aim to assess progress towards program objectives, as well program administration and delivery. They will take into

account input from funding recipients and other stakeholders, and will assist with program improvements and any future program design. They will also focus on the achievements and NRM and biodiversity conservation outcomes from Caring for our Country and the Biodiversity Fund.

5.3 Program reportingAs a way of sharing information and demonstrating results as they occur, the Australian Government will make project and program information publicly available. To date this has occurred through annual departmental and program progress reports. The Australian Government intends to move towards a more detailed and interactive reporting approach, publicly reporting project and program level data, as it becomes available. Information will continue to be communicated through the following regular reports:

Annual departmental reports to demonstrate accountability and report on progress to Parliament and the public. For the Biodiversity Fund, this will also include the annual reports of the Land Sector Carbon and Biodiversity Board.

Program progress updates will provide a platform to provide updates on program progress, highlight particular achievements and share information and case studies from the project level.

Monitoring and evaluation reports generated through the Australian Governments targeted scientific monitoring approach will be published and made publicly available upon completion.

6 MERI Toolkit

The following documents will directly support the implementation of this MERI Strategy, and may be updated from time to time:

Available on the Caring for our Country website http://www.nrm.gov.au/funding/meri

Caring for our Country Project MERI Plan template (under $80,000)

Caring for our Country Project MERI Plan template (over $80,000)

MERI FAQ

Example Project MERI Plan for Caring for our Country

Example Program Logic

Sustainable Environment Stream Program MERI Plan

Sustainable Agriculture Program MERI Plan

The following are useful reference materials for the development and implementation of MERI:

Australian Government Natural Resource Management Monitoring Evaluation Reporting and Improvement Framework.

Developing and Using Program Logic in Natural Resource Management

Available on the Department of Sustainability, Environment, Water, Population and Communities, Land Sector Package, Biodiversity Fund Website http://www.environment.gov.au/cleanenergyfuture/biodiversity-fund/meri

Biodiversity Fund Project MERI Plan template

Biodiversity Fund Reporting Templates

Biodiversity Fund Ecological Monitoring Guide and guidance video

Example project MERI plan for Biodiversity Fund

7 Terminology Achievement For the purpose of this MERI Strategy ‘achievement’ is a general term

referring to the realisation of particular activities and outcomes.

Baseline The state or condition of an asset (environmental, social etc) prior to an intervention (project or program)

Evaluation A systematic process of information collation and analysis to assess the impact, appropriateness, effectiveness, efficiency or legacy of a project or program.

Monitoring The (preferably regular) collection of data / information, usually to assist timely decision making, and demonstrate changes over time.

Objective A desired result of an activity, project or program

Program level Refers to Australian Government MERI responsibilities and activities at whole ofCaring for our Country program and the Biodiversity Fund program.

Program Outcome The Strategic Objectives or 5 Year outcomes of Caring for our Country.

Project Outcomes The results of project activities. These may be immediate, intermediate or longer term outcomes.

Project level Refers to funding recipient MERI responsibilities and activities

Funding recipient Person or organisation that receives funding from the Caring for our Country or Biodiversity Fund program(s).

Australian Government The Australian Government as represented by the Department of Agriculture, Fisheries and Forestry and /or the Department of Sustainability, Environment, Water, Population and Communities.