The Lumina CenterThe Lumina CenterGrantseeking Workshop SeriesGrantseeking Workshop Series

PresentsPresents

Outcomes & Evaluations April 20, 2006

Valparaiso UniversityValparaiso University

Today’s Agenda Today’s Agenda

Project components Defining outcomes Outcomes based planning Outcomes vs. outputs Identifying your audience Setting outcomes to drive

planning Making outcomes measurable

Traditional Project ComponentsTraditional Project Components

InputsInputs(Money, staff, volunteers, facilities, (Money, staff, volunteers, facilities,

equipment, supplies)equipment, supplies)

ActivitiesActivities(Strategies, techniques, types of (Strategies, techniques, types of

treatment)treatment)

OutputsOutputs(direct products, classes taught, (direct products, classes taught,

educational materials distributed)educational materials distributed)

Added Project ComponentsAdded Project Components

InputsInputs ActivitiesActivities OutputsOutputs OutcomesOutcomes

(including indicators and data sources)(including indicators and data sources)

Why Measure Outcomes?Why Measure Outcomes?

The “So What” factorThe “So What” factor

As funding becomes more scarce, service As funding becomes more scarce, service providers, governments, other funders, providers, governments, other funders, and the public are calling for clearer and the public are calling for clearer evidence that the resources they expend evidence that the resources they expend actually produce benefits for people. actually produce benefits for people.

Funders want greater accountability for the Funders want greater accountability for the use of their resources.use of their resources.

Defining OutcomesDefining Outcomes

Outcomes are the BENEFITS realized Outcomes are the BENEFITS realized as a result of the project. as a result of the project.

They represent the IMPACT of the They represent the IMPACT of the project on its participants.project on its participants.

They typically represent a CHANGE in They typically represent a CHANGE in the behavior, attitude, skills, status, the behavior, attitude, skills, status, knowledge, or life condition of the knowledge, or life condition of the project participants.project participants.

Outcome Based PlanningOutcome Based Planning

Outcome based Outcome based planningplanning uses uses audience needs and hoped-for audience needs and hoped-for results as the foundation for results as the foundation for programs and design decisionsprograms and design decisions

Outcome Based EvaluationOutcome Based Evaluation

Outcome based Outcome based evaluationevaluation is a systematic is a systematic way to assess the extent to which a way to assess the extent to which a program has achieved its intended results.program has achieved its intended results.

It focuses on the key questionsIt focuses on the key questions ““How has my program made a How has my program made a

difference?”difference?” ““How are the lives of the program How are the lives of the program

participants better as a result of my participants better as a result of my program?”program?”

Benefits of Outcome Based Benefits of Outcome Based Planning and EvaluationPlanning and Evaluation

Provides a logical framework for Provides a logical framework for program developmentprogram development

Increases program effectivenessIncreases program effectiveness Communicates program valueCommunicates program value Generates information for decision-Generates information for decision-

makingmaking

Limitations of Outcome Based Limitations of Outcome Based Planning and EvaluationPlanning and Evaluation

Not identical to formal researchNot identical to formal research Suggest cause and effect, it doesn’t Suggest cause and effect, it doesn’t

intend to prove itintend to prove it Shows contribution, not attributionShows contribution, not attribution

Outcomes ChecklistOutcomes Checklist

Are the outcomes related to the “core Are the outcomes related to the “core business” of your program?business” of your program?

Is it within your control to influence the Is it within your control to influence the outcomes?outcomes?

Are your outcomes realistic and Are your outcomes realistic and attainable? attainable?

Are your outcomes achievable within Are your outcomes achievable within the funding and reporting periods?the funding and reporting periods?

Outcomes ChecklistOutcomes Checklist

Are your outcomes written as change Are your outcomes written as change statements—will things increase, statements—will things increase, decrease, or stay the same?decrease, or stay the same?

Have you moved beyond client Have you moved beyond client satisfaction in your outcomes?satisfaction in your outcomes?

Is there a logical sequence among your Is there a logical sequence among your short-term, intermediate, and long-term short-term, intermediate, and long-term outcomes?outcomes?

Are there any big “leaps” in your Are there any big “leaps” in your outcomes (i.e. gaps in the progression of outcomes (i.e. gaps in the progression of impacts)?impacts)?

Develop a ProgramDevelop a Program

Activities and services leading Activities and services leading towards intended outcomestowards intended outcomes

Have a definite beginning and Have a definite beginning and endend

Design it to change attitudes, Design it to change attitudes, behaviors, knowledge, or behaviors, knowledge, or increase skills and abilities increase skills and abilities based on assumed needbased on assumed need

AudienceAudience

Programs are developed as a result Programs are developed as a result of assumptions about people’s of assumptions about people’s needsneeds

Information can be drawn from:Information can be drawn from: Your experiencesYour experiences A program partner’s experiencesA program partner’s experiences Formal or informal researchFormal or informal research

Audience NeedsAudience Needs

NEED: NEED: A condition, want, deficit that is A condition, want, deficit that is common to a group of individualscommon to a group of individuals

SOLUTION: SOLUTION: A program that will change A program that will change or improve behaviors, knowledge, or improve behaviors, knowledge, skills, attitudes, life condition, or skills, attitudes, life condition, or statusstatus

DESIRED RESULTS: DESIRED RESULTS: The change or The change or improvement you expect to achieveimprovement you expect to achieve

ProcessProcess

NEEDS SOLUTIONS RESULTS

Getting to evaluation of outcomes:•What are the indicators?•What are the data sources?

Outcome EvaluationOutcome Evaluation

OutcomeOutcome IndicatorIndicator Data Data SourceSource

Students know key Students know key principles of principles of archival selectionarchival selection

The # and % of students who The # and % of students who can list and define at least 5 can list and define at least 5 valid selection criteriavalid selection criteria

Course Course assignmentassignment

Students can apply Students can apply basic concepts of basic concepts of archival archival arrangementarrangement

The # and % of students who The # and % of students who can correctly arrange a folder can correctly arrange a folder of 15 unsorted documents of 15 unsorted documents according to provenance and according to provenance and original orderoriginal order

Course Course assignmentassignment

Students can Students can create basic create basic archival recordsarchival records

The # and % of students who The # and % of students who can generate a basic USMARC can generate a basic USMARC record for 3 test documentsrecord for 3 test documents

End of course End of course examexam

ResultsResults

Based on audience successBased on audience success That they know something or will be That they know something or will be

able to do somethingable to do something They can be measured at various They can be measured at various

times (i.e. Initial, Intermediate, Long times (i.e. Initial, Intermediate, Long Term)Term)

Targets can be established for each Targets can be established for each periodperiod

ReportsReports

Reports summarize the results of Reports summarize the results of outcome data and include:outcome data and include:

Participant characteristicsParticipant characteristics Inputs, activities & services, outputs and Inputs, activities & services, outputs and

outcomesoutcomes Elements requested by fundersElements requested by funders Comparisons of previous periodsComparisons of previous periods Interpretation of dataInterpretation of data

Evaluate the SystemEvaluate the System

After a reasonable period of operation, After a reasonable period of operation, evaluate the effectiveness of the evaluate the effectiveness of the outcomes system:outcomes system:

Have audiences been sufficiently identified?Have audiences been sufficiently identified? Are outcomes clearly written?Are outcomes clearly written? Are outcomes sufficient to describe what Are outcomes sufficient to describe what

you hope will happen?you hope will happen? Are data collection methods cost efficient?Are data collection methods cost efficient?

Coming AttractionsComing Attractions

September 21, 2006: “Introduction to September 21, 2006: “Introduction to Grantseeking on Campus”Grantseeking on Campus” (repeat) (repeat)

October 19, 2006: “Federal October 19, 2006: “Federal Grantseeking”Grantseeking”

November 16, 2006: “Collecting Data November 16, 2006: “Collecting Data for Grantseeking”for Grantseeking”

Questions and CommentsQuestions and Comments