gwynn mettetal. discuss different ways to assess outcomes help you decide which methods would be...
TRANSCRIPT
![Page 1: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/1.jpg)
Gwynn Mettetal
![Page 2: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/2.jpg)
Discuss different ways to assess outcomes Help you decide which methods would be
best for your project
![Page 3: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/3.jpg)
Who are you? What sort of Vision 2020 project are you
planning? (the two sentence version)
![Page 4: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/4.jpg)
Assessment—evidence that your project is making a difference
You MUST assess the effectiveness of your Vision 2020 grant to get continued funding!
Lots of strategies possible Depends on your goals Depends on your situation
![Page 5: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/5.jpg)
![Page 6: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/6.jpg)
Quantitative (numbers) Grades, attendance, ratings on a scale, retention rate
Qualitative (words) Interviews, essays, open ended survey questions
Both are fine, just different
![Page 7: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/7.jpg)
Existing data (easiest, already there) ◦ Student records◦ Archival data◦ Student work in course
Conventional sources (easy, but must generate)◦ Behavioral data—journals, library usage◦ Perceptual data—surveys, focus groups, interviews
Inventive sources (difficult)◦ Products or performances
![Page 8: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/8.jpg)
![Page 9: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/9.jpg)
Must treat students respectfully Must protect privacy Must “do no harm”
Collecting new data (not coursework) from your own students?◦ Have someone else collect and hold until grades are in◦ Can’t force them to participate◦ Can’t take up too much instruction time
Institutional Review Board (IRB)◦ If planning to publish
![Page 10: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/10.jpg)
Add power--compare groups!◦ Before and after◦ Different course units◦ This semester and last◦ Two sections with different methods◦ Your class to that of another instructor
Be realistic--start small
![Page 11: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/11.jpg)
![Page 12: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/12.jpg)
Validity—does your evidence (data) mean what you think it means?
Example test scores = deep learning? What if just rote memory? What if students cheated?
![Page 13: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/13.jpg)
Reliability—would you get the same evidence if you collected it again? Or was this just a fluke?
Example: Test scores = deep learning? What if you gave again next week and scores were very
different?
![Page 14: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/14.jpg)
In general, hard to have both.
Real life is messy (valid, not as reliable) Experiments are controlled (reliable, not as
valid)
Solution is . . .
![Page 15: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/15.jpg)
Get several different types of data Different sources:
◦ Instructors, students, advisors, records Different methods:
◦ Surveys, observations, student work samples Different times:
◦ Start and end of semester, two different classes, two different semesters
![Page 16: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/16.jpg)
Course evaluations
final project rubric
Comparison to last semester’s class
![Page 17: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/17.jpg)
What data could YOU collect?
![Page 18: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/18.jpg)
![Page 19: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/19.jpg)
Qualitative analyses: look for themes in words and behaviors
Theme 1: Students understood more abstract concepts after group discussion. (Follow with quotes from student exams, other evidence.)
![Page 20: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/20.jpg)
Quantitative analyses: simple graphs, tables Simple statistics: means, correlations, t-tests
![Page 21: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/21.jpg)
Focus on practical significance, more than statistical significance
![Page 22: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/22.jpg)
What would convince YOU?
![Page 23: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/23.jpg)
![Page 24: Gwynn Mettetal. Discuss different ways to assess outcomes Help you decide which methods would be best for your project](https://reader030.vdocuments.net/reader030/viewer/2022032612/56649ebe5503460f94bc9000/html5/thumbnails/24.jpg)
If evidence was good, keep your old strategy If evidence was weak, tinker to improve your strategy Plan to assess again, after working with a new group of
students You will need to show how you used your data to get
continued Vision 2020 funding!