Program Performance Accountability& Measurable Results

Kevin Keaney, ChiefPesticide Worker Safety Programs

U. S. EPA2005

OPP

Outline

Performance Measurement: Context, Value, Types

Accountability Drivers & Problems

GPRA & PART

Elements of a Good Measures Program

Where Accountability Fits StrategicallyProgram Success is built from critical components.

CLEAR GOALSPublic Health & Ecological

Protection Outcomes

GOOD PLANNINGStrategic Objectives

EFFECTIVE IMPLEMENTATIONMultiple Levels

ADEQUATE RESOURCESAcquisition & Allocation

MEANINGFUL ACCOUNTABILITYMeasures & Tracking

OPP

Value of Performance/Results Data

MULTIPLE USES Guiding Program Management: Decisions;

Priorities; Resources

Performance Accountability Tracking program success and progress

Communication

General Public Interested Stakeholders Decision Makers (Congress, Admin.)

Meaningful Accountability = Good Management = Good Government

5

OPP

Types of Performance Measures

Performance measures vary depending on level of organization & need.

Strategic GoalsBroader/Long-TermBroader/Long-Term

Specific/Short-TermSpecific/Short-Term

Mid-RangeObjectives

AnnualCommitments

IndividualPerformance Stds

OPP

Accountability Drivers / Mechanisms

Statutes

EPA 5 Year Strategic Plan

3 Year NPM Commitments

Budget Formulation

Annual Work Plans

Budget Allocation

Tracking Systems/Reporting

Audits

Performance Standards

OPP

Problems w/ Current Systems

Components:…are complex,…are created at different times,…by different people,…for different purposes,…little thought about connections or alignment among the parts.

Inevitably creates:…conflicts,

…confusion, …unnecessary burdens, &

…mixed messages.

Doesn’t work: …no accountability …no consequences.

OPP

Accountability Overload

FIFRA!

GPRA!

REGIONALINITIATIVES!

OECA!LAWSUITS!

STRATEGICPLAN!

OMB!PART!

BUDGET!

SPECIALPROJECTS!

MEDIACOVERAGE!

FLEXIBILITY!FOCUS! POLITICAL

PRESSURE!

OPP

Accountability Drivers/Mechanisms

Statutes

EPA 5 Year Strategic Plan

3 Year NPM Commitments

Budget Formulation

Annual Work Plans

Budget Allocation

Tracking Systems/Reporting

Audits

Performance Standards

OPP

Accountability Drivers/Mechanisms

Statutes

EPA 5 Year Strategic Plan

3 Year NPM Commitments

Budget Formulation

Annual Work Plans

Budget Allocation

Tracking Systems/Reporting

Audits

Performance Standards

OPP

GPRA

GPRA = Government Performance and Results Act of 1993

GPRA is a statute aimed at increasing accountability by requiring measurement of program results & linking those results to resource allocation.

Provides a statutory framework for strategic planning and reporting that requires federal agencies to:

Establish accountability measures to assess the outcomes of each program activity, and

Prepare performance plans that establish annual performance goals.

OPP

GPRA Auditing

Program Assessment Rating Tool (PART)

Office of Management & Budget assumed the role of evaluating progress against GPRA.

OMB is committed to audit all federally funded programs against GPRA requirements on a 5 year cycle. Size of individual programs receiving detailed audit varies.

This is the third year. Several Pesticide programs have been audited.

OPP

GPRA Audits

OMB

Fire is invented

OPP

GPRA Auditing

Program Assessment Rating Tool (PART)A matrix of questions with a numeric value & requiring

succinct answers.Principle focus has been on the existence of Outcome

Measures satisfying OMB.OMB has not been bound by pre-existing measures

(EPA’s Strategic Plan; NPM Guidance).Moving target of changing requirements.Short turn-around times.Average initial scores of programs have been very low

(40 - 60 out of 100) and many classified as “Results Not Demonstrated”.

“re-PARTing” is the norm.Results are published each year with the federal budget.Programs not demonstrating results for successive

years are subject to budget cuts or elimination.

OPP

PART Budget Consequences

EPA

OMB

OPP

Criteria for Good Measures

VALUEHIGH

OutcomesClear

Wide use

LOW

BeansComplex

Narrow use

MeaningfulUnderstandable

Useful

BURDENLIGHT

FewSimple

LowUser-friendly

HEAVY

ManyHardHigh

Technical

# of MeasuresDifficulty of Collection

Data System CostsExpertise Required

Performance Measures Quadrant Analysis

High Value

Low Value

LightBurden

HeavyBurden

OUTCOMENIRVANA

OUTPUTHELL

1. High Value/Light Burden 2. High Value/Heavy Burden

4. Low Value/Heavy Burden3. Low Value/Light Burden

Measures Comparison Matrix

VALUE BURDEN

Meaningful Understandable UsefulPossible Measures

Measures Criteria

OPP

Process for Measures Improvement

Specific Problem Formulation Baseline data collection: Current measures; current

data systems, etc. Scope of project? Strategic measures only or a

universal results ethic (measures for all projects). Participants? OPP; OECA; Regions; States/Tribes;

industry; NGOs; etc. Outputs? Measures for Strategic Plan/NPM/Annual

Workplans/PART; Public Reports; Change risk assessment/management practices; Change or consolidate databases.

Timeline/Deadlines? 6 months (PART); 2 years (next 5 Year Strategic Plan).

Process Structure

Senior Level Steering CommitteeSet Direction; Manage Process; Resolve conflicts / issues

Program Specific WorkgroupsGather Data; Coordinate; Develop Measures + Data Systems

Regis. WPC&TESARegis. Review

SAI

Coordinate w/ Partners & Stakeholders

Regions USDAOW,ORD,

OECAIndustry

States,Tribes

NGOs

Databases & Information SystemsEvaluate existing sources; Opportunities for cost savings

OPP

Take Away Message

All government funded activities will be held to higher standards of performance accountability to justify budgets.

Cooperative work is required to establish meaningful accountability measures and performance targets.

Well documented progress toward quantifiable public health and environmental protection goals will help ensure continued program operation and funding.

OPP

Program Performance Accountability& Measurable Results

QUESTIONS ?