project evaluations a framework for your own project
Post on 21-Dec-2015
215 views
TRANSCRIPT
![Page 1: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/1.jpg)
project evaluationsproject evaluations
a framework for your own project
![Page 2: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/2.jpg)
who needs a project evaluation anyway?
• If your thesis/dissertation involves the implementation of a project or device and you want to prove its effectiveness
• If you are conducting an experiment these might be helpful guidelines
• If you are conducting a workshop or a teaching seminar• Federally funded projects must be evaluated (Government
Performance and Results Act, GPRA, 1993)– If you have an NSF grant to do a project
![Page 3: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/3.jpg)
why do project evaluations?
• Federally funded projects must be evaluated (Government Performance and Results Act, GPRA, 1993)– If you have an NSF grant to do a project
• proves causal connections • provides information on how to improve a project• allows you to understand if goals were met• can account for unanticipated consequences • can better communicate to stakeholders
![Page 4: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/4.jpg)
types of evaluations see also visual on
board
1) Formative evaluation: begins during project and continues throughout project life
a) Implementation evaluation: is the project going as planned?
b) Progress evaluation: is the program meeting its goals? (most important in the beginning stages of the project)
2) Summative evaluation: did the project meet its stated goals? Similar to progress evaluation, but at the end of project
“When the cook tastes the soup, that’s formative.When the guests taste the soup, that’s summative.”
![Page 5: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/5.jpg)
the big picture
1) Develop a conceptual model of the project* 2) Identify project goals*3) Define evaluation questions and measurable
outcomes* 4) Develop evaluation design* 5) Collect data6) Analyze data7) Provide information to interested audiences *the rest of the presentation only focuses on 1-4
![Page 6: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/6.jpg)
1)Develop a conceptual model and 2) identify project goals
The logic modelInputs Activities Short-Term Long-Term
• Fill in the model• Set up a timeframe for project• Sometimes helpful to work backwards (start with long-term
outcomes)
![Page 7: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/7.jpg)
1) my conceptual model and project goals
• Inputs: time to plan, help from other TAs and Donna, research guidance from Ken
• Activity: in recitation sections, TAs choose one of three activities and do the activity with the class to discuss either a) ecotourism, b) participatory development, or c) the causes of poverty
• Short-term outcomes: students actively participate in the activities, explore issue a, b, or c and discuss/reflect on the issues with other students
• Long-term outcomes: students are more aware of international development issues and will continue to learn about them outside of class
![Page 8: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/8.jpg)
3) Define measurable outcomes and evaluation questions
four steps1) Identify key stakeholders
(see pg 7 the NSF Guide)2) Formulate evaluation questions that stakeholders might ask
-may produce more than you’ll actually use, but that’s OK 3) Define outcomes in measurable terms
-use the questions generated in step two to fill out number three in the Goals & Objectives worksheet (see pg 9)
4) Prioritize & eliminate questions -consider your most important stakeholders first -what is success? Statistical significance, etc -consider resources: time, money, staff capability (general rule: 5-10% of cost for evalutions
![Page 9: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/9.jpg)
3) measurable outcomes and evaluation questions
• Example of stakeholder questions • Measurable outcomes: Example of goals
worksheet
• Prioritize: example of survey
![Page 10: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/10.jpg)
4) Develop an evaluation design
• Select methodological approach and data collection instruments – Data collection matrix (see pg 15)
• Determine who will be studied and when– randomized samples are best– Comparison groups if possible– If not, maybe levels of implementation?
![Page 11: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/11.jpg)
evaluation design
• student surveys (450-500)• TA surveys (8)• professor survey (1)• any possible follow-up interviews with any of the
above three• must acknowledge the parameters of my samples
![Page 12: Project evaluations a framework for your own project](https://reader036.vdocuments.net/reader036/viewer/2022062516/56649d5a5503460f94a3a210/html5/thumbnails/12.jpg)
references
• NSF 2002: The 2002 User-Friendly Handbook for Project Evaluation
• NSF 1997: The User-Friendly Handbook for Mixed Method Evaluations
• Stake, R. 1957: Program Evaluation, Particularly Responsive Evaluation
• Yin, R.K. 1989: Case Study Research: Design and Method