v 2 region 5 evaluation session 11 16 11
TRANSCRIPT
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
1/26
Region 5 RPTAC
Region 5 Evaluation
Work GroupNovember 16, 2011
Region 5 RPTAC1
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
2/26
Region 5 RPTAC
Barbara Buswell: Region 5 RPTAC Director
Nora Thompson: Region 6 RPTAC Director
Presented by:
2
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
3/26
Region 5 RPTAC
1. Explore evaluation sections of RFA
2. Increase understanding of evaluation approaches and
terms
3. Consider alignment of evaluation methods with logic
model, goals & activities, and OSEP Performance
Measures
4. Increase confidence in tackling evaluation dimensionsin project implementation and proposal development
3
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
4/26
Region 5 RPTAC
The Accountability Era
4
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
5/26
Region 5 RPTAC
What Does An Evaluation Do?
Answers one or more of four main types of questions?
What occurred ; what did we do?
How well did it go; how well did we do it?
So what? What difference did our efforts make? What
changed?
What should we change in future projects?
5
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
6/26
Region 5 RPTAC
Are we doing what we promised in our proposal in the
timeline established?
In what ways and to what extent is our project as a wholeeffective?
How effective are our specific approaches and activities?
How can we be sure that we are meeting our outcomes?
6
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
7/26
Region 5 RPTAC
Selected Sections from 2011 RFA
Sample Evaluation Table
7
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
8/26
Region 5 RPTAC
Key Sections directly related to evaluation in 2011
Request for Application
Management Section (section total of 20 points) (e) (ii)
Quality of Project Evaluation Sections (section total of 15points)
(f) (i)
(f) (ii)
(f) (iii)
8
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
9/26
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
10/26
Region 5 RPTAC
What evaluation methods do you use in your PTI or
CPRC project?
10
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
11/26
Region 5 RPTAC
Methods of Data CollectionSources of Information Existing data
- Program records,
attendance logs, etc- Pictures, charts, maps,
pictorial records
Program participants
Others: key informants,
nonparticipants, proponents,
critics, staff, collaborators,
funders, etc.
Data Collection Methods
Survey
Interview
Test Observation
Group techniques
Case study
Photography Document review
Expert or peer review
11
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
12/26
Region 5 RPTAC
(f) Quality of the Project Evaluation
(f) (i). The extent to which the methods of
evaluation are thorough, feasible, and
appropriate to the goals, objectives, and
outcomes of the proposed project
12
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
13/26
Region 5 RPTAC
(f) Quality of the Project Evaluation
(f) (ii). The extent to which the methods ofevaluation are appropriate to the context
within which the project operates
13
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
14/26
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
15/26
Region 5 RPTAC
Some projects include an evaluation chart
that articulates for each goal:
The Goal
Evaluation Question
Methodology to Be Used
Criteria
Timeline
15
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
16/26
Region 5 RPTAC16
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
17/26
Region 5 RPTAC
Use a variety of approaches to ensure that you
are using both formative and summative
approaches.
Consider the appropriateness of the method to
the activity.
Consider the yield of various approaches.
17
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
18/26
Region 5 RPTAC
Remember to be sure that your evaluation methods andplan align with
Your grant goals
Your management plan
Your logic model
The parameters in the RFA
OSEP Performance Measures (high quality, relevant, useful)
18
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
19/26
Region 5 RPTAC
Formative
Summative
Qualitative
Quantitative
High Quality
Relevant Useful
Others?
19
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
20/26
Region 5 RPTAC
Formative = Process/Implementation
Conducted during the development and implementation
of a program
Primary purpose is providing information for program
improvement.
20
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
21/26
Region 5 RPTAC
Summative = Outcome/Impact
Evaluation conducted after completion of a program (or
a phase of the program) to determine program
effectiveness and worth
21
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
22/26
Region 5 RPTAC
Qualitative
Effective, relevant, useful
Depth and detail
Generally narrative
Quantitative
How many, how much
Numbers of workshops, numbers of families trained
22
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
23/26
Region 5 RPTAC
Take Turkey Match quiz within 1 week
Begin outlining evaluation section
Check out Lingo to Go to sound sophisticated
Attend second workgroup session on Wednesday, November
30 at 12:00 pm MDT
Call Individual TA for specific questions or clarity
23
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
24/26
Region 5 RPTAC
Question: What about having a consultant or friend write
the grant (or evaluation section) for me? Pitfalls & benefits?
Can someone from TA write a section for me?
Can TA review and critique my proposal?
Do we really have to do all this work?
24
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
25/26
Region 5 RPTAC
What Have We Learned?
25
-
8/3/2019 V 2 Region 5 Evaluation Session 11 16 11
26/26
Region 5 RPTAC
Thank you!
26