nifa reporting web conference december 10, 2009. start the recording…

NIFA Reporting Web Conference

December 10, 2009

Start the Recording…

Bart HewittBart is the Accountability and Reporting Leader for the Office of Planning and Accountability. He leads the national effort for the State Plan of Work and Annual Report process under the Agricultural Research, Extension and Education Reform Act; manages the Multistate Research Fund Project Proposal approvals for NIFA; and develops performance information for the PART and NIFA Budget.

(202) 270-0747

bhewitt@nifa.usda.gov

www.nifa.usda.gov/opa

E-mail questions to rwc@nifa.usda.gov

User Support• (202) 690-2910 or

servicedesk@nifa.usda.gov • Do not contact Texas A&M support• FAQs and other information on the

CSREES Reporting Web Conference web page at www.nifa.usda.gov/rwc

E-mail questions to rwc@nifa.usda.gov

Format and Logistics• E-mail questions to

rwc@nifa.usda.gov • E-mail topic suggestions to

rwc@nifa.usda.gov • Conferences are recorded and will be

available on the Reporting Web Conference web page at www.nifa.usda.gov/rwc

E-mail questions to rwc@nifa.usda.gov

To Receive AnnouncementsAn RWC e-mail list will notify interested parties on news, schedules, and other issues relating to the series. To subscribe:

– Send an e-mail to lyris@lyris.nifa.usda.gov. – Skip your subject line and in the body of

your message type: subscribe reportingwc. 

– Be sure you receive an e-mail confirming your subscription.

E-mail questions to rwc@nifa.usda.gov

Introduction to Plan of Work

December 2009

Plan of Work – A Historical Perspective• 1984 – 1991 NARS• 1992 – 1995 PPARS• 1994 – 1996 PPARS GPRA Pilot• 1996 and 1997 PPARS Transition• 1998 and 1999 GPRA Planning and

Reporting System• 2000 - Present - AREERA

E-mail questions to rwc@nifa.usda.gov

Plan of WorkA Historical Perspective

• NARS

–Fully Narrative

–Structured

–Attempted to cover the total Extension Portfolio

E-mail questions to rwc@nifa.usda.gov

Plan of WorkA Historical Perspective

• PPARS

–Focused on Targeted Programs

–Narrative and Numeric Data

–DOS Based Computer Template• Relational Database

E-mail questions to rwc@nifa.usda.gov

Plan of WorkA Historical Perspective

• PPARS – GPRA Pilot– 1994 – 1996

– Five States

– Selected Targeted Programs

– Targets for Each Year and a Baseline

E-mail questions to rwc@nifa.usda.gov

PPARS Components

• State Program Overview

• Targeted Programs

• Civil Rights

E-mail questions to rwc@nifa.usda.gov

State Program Overview• Narrative

• Described the total Extension Program in the State

• Focused on Base Programs

E-mail questions to rwc@nifa.usda.gov

State Accomplishment Overview• Narrative Executive Summary

required each year

• Stressed program results and documented benefits

• Reflected significant achievements of the Base Programs

E-mail questions to rwc@nifa.usda.gov

Targeted Programs

• National Initiatives– Food Safety and Quality– Plight of Young Children– Communities in Economic

Transition– Sustainable Agriculture– Waste Management– Youth at Risk– Decisions for Health

• 3(d) and Specially Funded Items– EFNEP– Farm Safety– Integrated Pest Management– Pesticide Applicator Training– Pesticide Impact Assessment– Renewable Resource Extension

Act

Other Programs Dislocated Farmers Assistance Grants Urban Gardening Program Conservation and Forestry Titles of the 1990

Farm Bill Environmental Education Limited Resource/Low Income Individuals and

Families Multicultural Diversity Parenting and Family-Youth Programs Sea Grant/Marine Extension Programs Volunteer Development and Management

E-mail questions to rwc@nifa.usda.gov

Targeted Programs – POW

• Situation Statement• Research and Empirical Base• Objectives and Indicators

– Data Collection Methodology• Target Audience• Implementation Plan

– Staff Training– Internal and External Linkages– Volunteers– Key Components– Reaching Target Audience(s)– Mileposts

• Estimated Program Cost• Estimated FTEs• Number of Volunteers• Plan of Work Contact

E-mail questions to rwc@nifa.usda.gov

Targeted ProgramsAnnual Report

• Narrative Overview

• Success Stories

• Exemplary Programs

• Objectives and Indicators

• Estimated Program Costs

• Estimated FTEs

• Number of Volunteers

• State Contacts

E-mail questions to rwc@nifa.usda.gov

PPARS - Sample IndicatorPOW Proj

Number of Participants

Number Adopted

1992

1993

1994

1995

Total

E-mail questions to rwc@nifa.usda.gov

PPARS – GPRA PilotYear # of Participants

Instructed# of Participants who became pregnant

# who became pregnant in a comparable population

Baseline 1993

Target Actual Target Actual Actual

1994

1995

1996

E-mail questions to rwc@nifa.usda.gov

Civil Rights Plan• Goals and Procedures

– Equal Employment Opportunity– Program Delivery– Public Notification– Civil Rights Training– On-site Compliance Reviews

• Population and Clientele Projections• Reporting Options

– Total (100%) Data Collection– Rotational Reporting– Independent Survey

E-mail questions to rwc@nifa.usda.gov

Civil RightsAnnual Report• Narrative Accomplishment Summary• Success Stories• Goals and Procedures

– Equal Employment Opportunity– Program Delivery– Public Notification– Civil Rights Training– On-site Compliance Reviews

• Population and Clientele Projections

E-mail questions to rwc@nifa.usda.gov

GPRA Planning and Reporting System• Based on PPARS GPRA Pilot

database system• Used 5 GPRA Goals instead of

Targeted Programs• Used GPRA Objectives• Used Targeted Indicators which could

be aggregated across the nation

E-mail questions to rwc@nifa.usda.gov

AREERA• Agricultural Research, Extension, and

Education Reform Act of 1998• Plan of Work• Annual Report of Accomplishments

and Results• Multistate Extension• Integrated Research and Extension• Multistate Research name

E-mail questions to rwc@nifa.usda.gov

2000 – 2004 Plan of Work• Arranged around the 5 GPRA goals

established by USDA when the 5 year Plan of Work began

• Unstructured text driven by the states’ perceived need for maximum flexibility and an aversion to a database driven template

• Annual Reports began using Key Themes as a means to try to gain some order to the unstructured text data

E-mail questions to rwc@nifa.usda.gov

2005 – 2006 POW Update• Submitted to Old CSREES by April 1,

2004

• Extended the current Plan of Work Cycle to include FY 2005 & 2006

• 5 – 10 page document – States could add any relevant information

not included in 2000 – 2004 Plan of Work– Outlined changes in focus over two year

period from the original Plan

E-mail questions to rwc@nifa.usda.gov

Original Plans for New POW

• 2007 – 2011 (Rolling 5 Year Plans)

• New plan was due June 1, 2006

• Proposed written guidance published POW Cycle in June 2005 in Federal Register

• 30 Day Comment Period

• Final Guidelines published January 2006

E-mail questions to rwc@nifa.usda.gov

Initial Plan of Work Software• Web-based and database driven• Pilot testing in three phases

– September– October

• Released on November 16, 2005– “Sandbox” version

• Production Version Released in January 2006

E-mail questions to rwc@nifa.usda.gov

Structure of 5-Year POW1. Plan Overview Section

2. Planned Programs Section

3. Multistate Extension and Integrated Research and Extension (Sections 105 & 204 of AREERA)

E-mail questions to rwc@nifa.usda.gov

Structure of 5-Year POW1. Plan Overview Section

• Plan Overview Text • Overall FTEs/SYs• Stakeholder Input• Merit Review (Program Review)• Evaluation of the Multis & Joint

Programs

E-mail questions to rwc@nifa.usda.gov

Structure of 5-Year POW

2. Planned Programs Section• Name of State Defined Program Unit

• Knowledge Area Classification

• Program Duration

• Basic Logic Model Format

• Evaluation Strategy

E-mail questions to rwc@nifa.usda.gov

Structure of 5-Year POW3. Multistate Extension and Integrated

Research and Extension (Sections 105 & 204 of AREERA)

• Base Percentage• Target Percentage for All Future Years• Plan Target Dollars for Duration of Plan

E-mail questions to rwc@nifa.usda.gov

Plan of Work – A Key Component of the One Solution

• How does the Plan of Work relate to the One Solution Initiative?– Part of the larger One Solution

– Web-based system for Formula Funds

– Same goals as One Solution

E-mail questions to rwc@nifa.usda.gov

One Solution Initiative• Goals of One Solution

– Simplify Reporting– Reduce burden on Partners and NIFA– Improve quality of accountability data– Meet increasing performance & budget

reporting expectations

E-mail questions to rwc@nifa.usda.gov

Plan of Work Goals

Why do we have a new Plan of Work Electronic System?

– Responds to AREERA– Opportunity to reduce reporting burden over time– Maximize usefulness of information– Improve Plan of Work accountability– Improve Quality of the data received from States– Meet increasing performance & budget reporting expectations

• Agency Program management• OMB and USDA• Congress

E-mail questions to rwc@nifa.usda.gov

Plan of Work – What is Covered?• What State work must be included in

the Plan of Work?

– NIFA Formula and required Matching Funds

• Federal Funds: – Hatch– Smith-Lever 3(b) and (c), [but NOT 3(d)]– 1890 Extension– Evans-Allen

• All required non-federal matching funds

E-mail questions to rwc@nifa.usda.gov

Plan of Work – Requirements

• Approved Plan of Work is an integral requirement for release of funds to institutions

• A completed plan needs to include:– Stakeholder Input Documentation

– Peer Review for Research

– Merit Review for Extension

– Multi-state Extension and Integrated Research and Extension Financial Data

– Planned Programs

E-mail questions to rwc@nifa.usda.gov

Logic Model

• Why organize Planned Program data around the Logic Model?– Conceptual depiction of key elements of any program

or project– Endorsed by GAO– Commonly used by:

• Federal Research Agencies• Private Research Organizations• Cooperative Extension Services

– Useful for planning, describing & implementing programs, and designing evaluations

E-mail questions to rwc@nifa.usda.gov

Planned Programs Structure• Each Planned Program in the Plan of Work contains these key components

– Situation• What problems, needs, or opportunities will the program address?

– Assumptions• Explains why particular activities to address situation were chosen• Validated by research, literature and experience

– External Factors• That can help or hinder results, and alternative explanations

– Inputs• What assets and resources will be applied?

– Activities/Outputs• What program activities do you plan to do, who will you serve?• What are some of the products produced?

– Outcomes• What changes in Knowledge, Action, and/or Condition are expected.• What changes in Knowledge, Action, and/or Condition will be measured?

– Evaluation• What Types of Evaluation is Planned?• What methods will be used?

E-mail questions to rwc@nifa.usda.gov

Occur when a societal condition is improved due to a participant’s action taken in the previous column.

For example, specific contributions to:

- Increased market opportunities overseas and greater economic competitiveness

- Better and less expensive animal health

- Vibrant & competitive agricultural workforce

- Higher productivity in food provision

- Better quality-of-life for youth & adults in rural communities

- Safer food supply

- Reduced obesity and improved nutrition & health

- Higher water quality and a cleaner environment

Generic Logic Model for NIFA Reporting NIFA – Office of Planning & Accountability

(This model is intended to be illustrative guide for reporting on NIFA-funded research, education and extension activities. It is not a comprehensive inventory of our programs.)

Outcomes

Actions

InputsSituation Activities

Knowledge

What we invest:

- Faculty

- Staff

- Students

- Infrastructure

- Federal, state and private funds

- Time

- Knowledge

- The collection of stakeholder opinions

Occurs when there is a change in knowledge or the participants actually learn:

- New fundamental or applied knowledge

- Improved skills

- How technology is applied

- About new plant & animal varieties

- Increased knowledge of decision-making, life skills, and positive life choices among youth & adults

- Policy knowledge - New improved methods

Description of challenge or opportunity

- Farmers face increasing challenges from globalization

- Opportunity to improve animal health through genetic engineering

- Insufficient # of trained & diverse professionals entering agricultural fields

- Youth at risk

- Invasive species is becoming an increasing problem

- Bioterrorism

- Obesity crisis

- Impaired water quality

External Factors - A brief discussion of what variables have an effect on the portfolio, program or project, but which cannot be changed by managers of the portfolio, program, or project. For example, a plant breeding program’s success may depend on the variability of the weather...etc.

Occur when there is a change in behavior or the participant’s act upon what they’ve learned and:

- Apply improved fundamental or applied knowledge

- Adopt new improved skills

- Directly apply information from publications

- Adopt and use new methods or improved technology

- Use new plant & animal varieties

- Increased skill by youth & adults in making informed life choices

- Actively apply practical policy and decision-making knowledge

Conditions

Assumptions - These are the premises based on theory, research, evaluation knowledge etc. that support the relationships of the elements shown above, and upon which the success of the portfolio, program, or project rests. For example, finding animal gene markers for particular diseases will lead to better animal therapies.

What we do (Activities): - Design and conduct research- Publish scientific articles- Develop research methods and procedures- Teach students- Conduct non-formal education- Provide counseling- Develop products, curriculum & resources

Who we reach (Participation):

- Other scientists- Extension Faculty- Teaching Faculty- Students- Federal, state & private funders- Scientific journal, industry & popular magazine editors- Agencies- Policy and decision- makers- Agricultural, environmental, life & human science industries- Public

Outputs

Version 1.2

- New fundamental or applied knowledge

- Scientific publications

- Patents

- New methods & technology

- Plant & animal varieties

- Practical knowledge for policy and decision-makers

- Information, skills & technology for individuals, communities and programs

- Participants reached

- Students graduated in agricultural sciences

Plan of Work – Benefits

– Reduced reporting burden on institutions

– Easier for Partners to review what other states are doing • POWs are Published in REEIS• Selected Data are published in the Leadership Management

Dashboard

– Easier for both Partners and NIFA to identify performance measures & track progress with structured format

– States will receive better and more timely feedback on their Plan of Work and Annual Report from NIFA

E-mail questions to rwc@nifa.usda.gov

POW– Benefits for NIFA

– More efficient & timely plan reviews by our NPLs

– Database queries make it easier to locate & analyze data

– Annual Reports will Provide supporting documentation for Portfolio reviews, PART, budget submission, other external requirements

– More efficient and accurate linkages to USDA Strategic Plan (Goals and Objectives) by linking Knowledge Areas to portfolios

E-mail questions to rwc@nifa.usda.gov

Plan of Work Data Use• How does NIFA use the information from the Plan of

Work for planning and accountability?– NPL State Liaison Communication– Portfolio Reviews– OMB PART process– Budget Performance Integration– GAO and OIG inquiries– Answer Congressional & Departmental inquiries

• Brings greater visibility of successes of Formula Funded Programs

E-mail questions to rwc@nifa.usda.gov

Evaluating Research & Education Efforts: Portfolio Approach

– OMB demands led to development of the portfolio assessment tool and measures;

– Portfolio analysis used to assess progress toward goals; guide RFAs/POWs

– Tracking papers, citations, patents, products, educational efforts, adoption of products/ practices

– Uses OMB R&D criteria (relevance, quality, performance)

E-mail questions to rwc@nifa.usda.gov

Evaluating Research & Education Efforts: Portfolio Approach

– Knowledge Areas (KA’s) for each Planned Programs

– KAs for each Outcome in Annual Report

– KAs are the link between Plan of Work and Portfolio Reviews

E-mail questions to rwc@nifa.usda.gov

Cascading Alignment

Portfolio

Knowledge Area code

Goal

Mission

Strategic Objective

Planned Program/POW

Plan of WorkEvidentiary materials for Portfolios

Strategic and evaluation plans

Tracks papers, citations, patents, products, educational efforts, adoption of products/ practices

Evaluation studies and special analyses

Input tables to show portfolio priorities and emphasis

E-mail questions to rwc@nifa.usda.gov

Hints for the 2009 Annual Report

Outputs and Outcomes• Limit Outputs and Outcomes

– Report Outcomes that stand out

• Most useful for our reports– Qualitative Outcome Statements in

conjunction with Quantitative Measure

E-mail questions to rwc@nifa.usda.gov

What does a Quality Outcome Statement Look Like?• Brief• Effective Impact Statements:

– Provide quantifiable evidence of change or difference the program made. (It really is all about the money. Bucks are the gold standard. Audiences want to know the return of investment.)

– Provide only enough detail to be easily understood.

– Highlight public benefits, outcomes, payoffs.

E-mail questions to rwc@nifa.usda.gov

Sample Qualitative Outcome• Issue: Producers, seedsmen, grain

merchandisers, processors, crop consultants, plant breeders, and extension staff are interested in new cultivars that bring them increased revenue.

E-mail questions to rwc@nifa.usda.gov

Sample Qualitative Outcome• What has been done: The new

varieties Faller Wheat, Lariat and Stampede pinto beans, Sheyenne non-transgenic soybean, RG7008RR soybean, and Pinnacle two-row barley were released for use.

E-mail questions to rwc@nifa.usda.gov

Sample Qualitative Outcome• Results: The estimated dollar value to

producers, seedsmen, grain merchandisers, processors, crop consultants, and plant breeders on these new varieties is $290,600,000 for 2007. Moreover, because of best management practices developed by research and extension, wheat and barley producers reduced economic losses by $40,000,000 through use of better varieties of wheat and through use of fungicides.

E-mail questions to rwc@nifa.usda.gov

Types of OutcomesChange in Knowledge• Occur when there is a change in

knowledge or the participants actually learn:– New fundamental or applied knowledge– Improved skills– How technology is applied– About new plant & animal varieties– Increased knowledge of decision-making, life

skills, and positive life choices among youth & adults

– Policy knowledge– New improved methods

E-mail questions to rwc@nifa.usda.gov

Types of OutcomesChange in Action or Behavior

• Occur when there is a change in behavior or the participant’s act upon what they’ve learned and:– Apply improved fundamental or applied knowledge– Adopt new improved skills– Directly apply information from publications– Adopt and use new methods or improved

technology– Use new plant & animal varieties– Increased skill by youth & adults in making

informed life choices– Actively apply practical policy and decision-making

knowledge

E-mail questions to rwc@nifa.usda.gov

Types of OutcomesChange in Condition• Occur when a societal condition is improved

due to a participant’s action taken (Change in Action).For example, specific contributions to:– Increased market opportunities overseas and

greater economic competitiveness– Better and less expensive animal health– Vibrant & competitive agricultural workforce– Higher productivity in food provision– Better quality-of-life for youth & adults in rural

communities– Safer food supply– Reduced obesity and improved nutrition & health– Higher water quality and a cleaner environment

E-mail questions to rwc@nifa.usda.gov

Types of Outcomes Needed• Outcomes related to USDA Priorities:

– Global Food Security and Hunger– Climate Change– Sustainable Energy– Childhood Obesity– Food Safety

Brevity and Conciseness in the Executive Summary• The Executive Summary is a brief

overview narrative of your total program.

• Copy and paste text into this field from your current state’s yearly brief publication.

• Two to five pages should suffice. • Highlights of your State program• Let the Planned Programs section

attend to detail. E-mail questions to rwc@nifa.usda.gov

Future of the Plan of Work?• Farm Bill Requirements• Expert Panel Meets May 2010

– Chosen through regional executive directors– Two panelists from each region (Total 11 chosen)– 5 panelists from NIFA– Make recommendations for further streamlining

Plan of Work Additional Information

– NIFA Plan of Work Web Page• http://www.nifa.usda.gov/business/reporting/planrept/plansofwork.html

– Plan of Work Training Page• http://www.nifa.usda.gov/business/reporting/planrept/training_fy0711.html

– Contact Information• Bart Hewitt, Accountability and Reporting Leader• pow@nifa.usda.gov• 202-720-0747

E-mail questions to rwc@nifa.usda.gov

Questions?

E-mail questions to rwc@nifa.usda.gov

See you in February!• Next NIFA Reporting Web Conference is

scheduled for Thursday, February 11 from 2-4 pm (Eastern)

• E-mail topic suggestions for February to rwc@nifa.usda.gov

• Visit the conference web site at www.nifa.usda.gov/rwc for:– The recording of this conference– The slides from this conference– Announcements

E-mail questions to rwc@nifa.usda.gov