surviving and thriving in programmatic self-assessment...
TRANSCRIPT
![Page 1: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/1.jpg)
Surviving and Thriving in Programmatic Self-Assessment:
Faculty and Administrative Perspectives
Presented at the 2015 HLC Annual Meeting By:
David G. Brailow, Ph.D., Vice President for Academic Affairs and Dean, Franklin College
Allison Fetter-Harrott, Ph.D., Assistant Professor of Political Science, Franklin College
Randall D. Smith, Ph.D., Assistant Professor of Political Science, Franklin College
![Page 2: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/2.jpg)
INTRODUCTION AND INSTITUTIONAL BACKGROUND
![Page 3: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/3.jpg)
By the Numbers
• ENROLLMENT: Just over 1000 full-time students.
• ACADEMICS: 55 majors, 41 minors, and 11 pre-professional programs
• FACULTY: 74 full-time professors; Average class size = 15, Student-to-Faculty ratio = 12:1.
• STUDENT CHARACTERISTICS: 96% receive financial aid; 35% participate in NCAA Division III sports; 30% first generation.
• OUTCOMES: 95% graduates employed/attending graduate school within six months of graduation.
![Page 4: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/4.jpg)
Franklin College and HLC
• Higher Learning Commission reaffirmed Franklin College accreditation in 2012.
• 2012 Systems Appraisal Feedback Report reinforced opportunity for more systematic program review vis-à-vis SLOs.
• Transitioned to Open Pathway. • Program assessment planning AY 2012-13,
executed AY 2013-14. • Administration invites Dr. Tami Eggleston,
McKendree University, for faculty-wide training on mission statement formation and SLO setting.
And the HLC
![Page 5: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/5.jpg)
Program Review Protocol
• Departments assess at least one student learning outcome per year
• Departments submit assessment reports every three years
• Departments undergo full program review every seven years
• VPAA and Associate VPAA review and discuss results with each department
Program Review Protocol
![Page 6: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/6.jpg)
Franklin College Political Science
• Political Science pilots program assessment at Franklin College.
• Review Process permitted: – Clarification of department mission and vision,
– Linking SLOs and Dept. Goals to mission and vision,
– Seeking empirical evidence to measure achievement of SLOs and Goals.
• Results were mixed – Some clear success,
– Empirical evidence supports curricular revision, etc. to better fulfill mission and vision.
Political Science
![Page 7: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/7.jpg)
SEVEN STRATEGIES FOR SUCCESSFUL ASSESSMENT
![Page 8: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/8.jpg)
1. Don’t Panic!
• Faculty should pursue their vision for their program.
• “Skillful teachers” naturally critically reflect on their work (Brookfield, 2006). Programmatic assessment formalizes practice.
• Embrace assessment that fulfills your goals. • Resistance to assessment may be
rational/expected (Bloxman & Boyd, 2008; Knight & Yorke, 2003), but should not obstruct.
![Page 9: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/9.jpg)
2. Let Vision Guide You
• Begin with mission and vision (Eubanks, 2010).
• Then, identify key student learning outcomes (SLOs) and programmatic goals (Bloxham & Boyd, 2008; Eubanks, 2010).
• Brainstorm about how SLOs and goals are, and might be, met.
• The assessment world is in your hands
2. Let Vision Guide You
![Page 10: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/10.jpg)
3. Don’t Reinvent the Wheel (KISS)
• REDUCE: Do not assess everything, only progress toward SLOs/goals (Bloxham & Boyd, 2008). Example: See Supplement.
• We put a spin on Eubanks (2010) and others:
• REUSE: Existing classes and assignments may fit SLOs/goals. Example: POL 110 Pre-Post.
• RECYCLE: Rubrics currently in use (slightly altered and/or AAC&U’s rubrics) may assist in assessment Example: POL 360 and Critical Thinking.
3. Don’t Reinvent the Wheel
![Page 11: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/11.jpg)
4. Assessment Doesn’t Have to Be Quantitative
• Employ a diverse range of measures (Bloxham & Boyd, 2008; Shipman, et. al., 2003).
4. Assessment Doesn’t have to be Quantitative
• Example: Success via qualitative exit interviews and focus group with our graduating seniors (using mostly open-ended questions).
![Page 12: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/12.jpg)
5. If you are going to use numbers, don’t over-complicate
• Colleagues can help.
• Do not be put-off by a “small n.” Take the first step. The “n” will come with time. Even a “small n” result may be useful.
5. If you are Going to use Numbers, Don’t Overcomplicate
• Avoid elaborate/cumbersome systems (Eubanks, 2010).
• Much may be learned from t-tests and z-scores.
![Page 13: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/13.jpg)
6. Value to Students
• Build around your best learning experiences.
• Identify best part of program, and see if students agree (Eggleston, 2013; Bain, 2004).
6. Value to Students
![Page 14: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/14.jpg)
7. Intrinsic Value to the Craft
• Use results to inform teaching and innovate, becoming more efficient and effective (Asand, Amidon & Huffman, 2010; Eubanks, 2010; Knight & Yorke, 2003; Shipman, et. al., 2003 ).
• If you don’t assess yourself; someone else might!
• Example: Results indicated possible changes and methods to improve student performance on comps.
7. Intrinsic Value to the Craft
![Page 15: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/15.jpg)
QUESTIONS?
![Page 16: Surviving and Thriving in Programmatic Self-Assessment ...webmedia.jcu.edu/assessment/files/2015/05/G-TUE-1000-c-Smith... · (Brookfield, 2006). ... Stephen D. The Skillful Teacher:](https://reader031.vdocuments.net/reader031/viewer/2022030411/5b455cf37f8b9a202f8b9624/html5/thumbnails/16.jpg)
American Association of Colleges and Universities. "VALUE (Valid Assessment of Learning in Undergraduate Education)." Association of American Colleges & Universities. N.p., 07 Oct. 2014. Web. 11 Feb. 2015. <http://www.aacu.org/value>.
Asand, Hardin L., Stevens Amidon, and Debrah Huffman. "Slouching Toward Assessment: One Department's Journey Toward Accountability Bliss." Coming to Terms With Student Outcomes Assessment: Faculty and Administrators' Journeys to Integrating Assessment in Their Work and Institutional Culture. Ed. Peggy L. Maki. Steerling, VA: Stylus, 2010. 107-22. Print.
Bloxham, Sue, and Pete Boyd. Developing Effective Assessment in Higher Education: A Practical Guide. Maidenhead: Open UP, 2007. Print.
Bain, Ken. What the Best College Teachers Do. Cambridge, MA: Harvard UP, 2004. Print.
Brookfield, Stephen D. The Skillful Teacher: On Technique, Trust, and Responsiveness in the Classroom. 2nd ed. San Francisco, CA: Jossey-Bass, 2006. Print.
Eggleston, Tami. "Outcomes, Rubrics, Assessment Workshop I." Assessment Workshop. Franklin College, Franklin, IN. 2-3 Jan. 2013. Lecture.
Eggleston, Tami. "Outcomes, Rubrics, Assessment Workshop II." Assessment Workshop. Franklin College, Franklin, IN. 20 Jun. 2013. Lecture.
Eubanks, David A. “Some Uncertainties Exist." Coming to Terms With Student Outcomes Assessment: Faculty and Administrators' Journeys to Integrating Assessment in Their Work and Institutional Culture. Ed. Peggy L. Maki. Steerling, VA: Stylus, 2010. 47-58. Print.
Knight, Peter, and Mantz Yorke. Assessment, Learning and Employability. Maidenhead: Society for Research into Higher Education & Open UP, 2003. Print.
Rhodes, Terrel L., ed. Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics. Washington DC: Association of American Colleges and Universities, 2010. Print.
Shipman, Donald, Susan L. Aloi, and Elizabeth A. Jones. "Addressing Key Challenges in Higher Education Assessment." Journal of General Education 52.4 (2003): 335-46. JSTOR. Web. 13 Feb. 2015. <http://www.jstor.org/stable/27797968>.
Bibliography: