derek herrmann & ryan smith illinois state university university assessment services
DESCRIPTION
IT TAKES TWO: A Systematic, Effective, & Practical Process for Integrating Assessment & Program Review. DEREK HERRMANN & RYAN SMITH Illinois State University University Assessment Services Association for Institutional Research, New Orleans, LA, 6-5-2012. About ISU. DRU - PowerPoint PPT PresentationTRANSCRIPT
IT TAKES TWO:A SYSTEMATIC, EFFECTIVE, & PRACTICAL PROCESS FOR INTEGRATING ASSESSMENT & PROGRAM REVIEW
DEREK HERRMANN & RYAN SMITHIllinois State University
University Assessment Services
Association for Institutional Research, New Orleans, LA, 6-5-2012
ABOUT ISU
• DRU• Total enrollment: 21,080• 35.5% of new students were
transfer students• 82.7% of degrees granted were
baccalaureate degrees
ABOUT ISU
• Retention rate: 85.1%• Graduation rate: 69.8%• Student to Faculty ratio: 19:1• Organizational structure
– UAS is part of Academic Affairs– IR is part of Finance and Planning
ABOUT ISU
• Illinois Board of Higher Education (IBHE)– All academic degree programs
required to have assessment plans on file
– All academic programs required to undergo a periodic and systematic program review
THE PROBLEM
1. Shifting views of higher education
“Higher education is one of the greatest hopes for intellectual and civic progress in this country. Yet, it is seen by many as part of the problem rather than the solution.”
- C. Palomba & T. Banta, Assessment Essentials, 1999
THE PROBLEM
2. Increasing Accountability
Higher Learning Commission
In the 11 pages of Criteria in the proposed pathways model, assessment is mentioned 19 times, with Teaching and Assessing for Learning as its own criteria.
IBHE
- C. Palomba & T. Banta, Assessment Essentials, 1999
THE PROBLEM3. Changing
World
THE PROBLEM
4. Assumptions about evaluation and assessment
“Assessment projects are simply getting in the way of good teaching… We spend far too much time and energy in trying to assess learning, an unattainable goal….I love teaching, however. It is a great pleasure to share my enthusiasm for (my discipline) and I absolutely love the chance to pick the minds of my bright students.”
- B. Moore, Washington State Board of Community & Technical Colleges, Using Assessment to Improve Instruction (video),
http://www.youtube.com/watch?v=BZ3USs16J3Y
THE PROBLEM
HOW COLLEGES ARE ORGANIZED
Board
President
VP
Dean
Director
Staff
VP
Dean
Director
Staff
VP
Dean
Director
Staff
HOW THEY ARE IN REALITY
Random People
Random People
Board
President
Union Leaders
Planning Person
Political Hire
Students
VP
Academic Dean
I.R. Person
Faculty
Provost
Assess. Dean
Director
Adjunct Faculty
BASIC ASSESSMENT MODEL
Establish Learning
Outcomes
Gather Evidence
Analysis
Make a Decision/Change
Mission/Goals
Accreditation, IBHE/ICCB, State of Illinois, Strategic plans, Tech. plans, enrollment mgmt. plans, facilities plans, financial plans, environmental scan data, IR data
Governance, Budgets, Resources, Boards, Politics, Org. culture, good and bad bosses, etc.
Education, training, & experience, life: health, spouses/partners, kids, family, friends, pets, etc.
Establish learning goal
Students will be able to explain the
impact of leadership on their
development
Provide learning opportunities
Mentorship program with
university leaders
Assess learning (gather evidence)Students keep a reflective journal
Coded and analyzed by leadership staff
Use the results (decision)
Students report a lack of contact with
mentors
Mission
Process is the outcome
Failure of decision-making or making it meaningful
FAILURE OF DECISION-MAKING
ADVANTAGES
THE PROBLEM
How does one design processes that…
- Engage faculty and staff?- Can be used for true improvement and
change in today’s society?- Make information meaningful for decision-
making?- Meets accountability standards?- Are genuine, not just transparent?
DISCLAIMER
AT ISU
• 8-Year cycle that includes– Process for the Review of Academic
Assessment Plans (PRAAP)– Self-study– Program Review– Annual Update
• Many across campus involved in these processes
AT ISU
• Year 1 – PRAAP– Chairs/program directors asked to
provide current assessment plans– Assessment Advisory Council
(AAC) reviews them using a rubric and completes a feedback form
THE PRAAP RUBRICUndeveloped Developing Established Exemplary
Program goals and intended student learning outcomes
Systematic assessment of student learning
Feedback from key stakeholders
Use of results/”Closing the Loop”
AT ISU
• Year 1 – PRAAP– Chairs asked to provide current
assessment plans– Assessment Advisory Council
(AAC) reviews them using a rubric and completes a feedback form
– UAS staff meets with chairs/program directors to discuss the feedback
AT ISU
• Year 2 – Self-Study– Faculty and staff complete self-
study for program review– Assessment plan is used to collect
data
AT ISU
• Year 3 – Program Review– Academic Planning Committee
examines Program Review documents
– Often programs submit follow-up reports based on assessment plans
AT ISU
• Years 4-8 – Annual Update– Chairs asked to complete a short
form concerning their assessment activities
– Questions based on Higher Learning Commission’s (HLC) Fundamental Questions for Conversations Student Learning
THE ANNUAL UPDATE FORM• Please provide a brief summary
of the previous year’s data which was collected related to your outcomes.
• What efforts were used during the past year to involve faculty and constituencies in assessment processes?
THE ANNUAL UPDATE FORM• Please outline what action plans
have been developed based upon the assessment process.
• Please provide any additional comments or information.
• Have changes or improvements been made to the Academic Assessment Plan in the previous year?
AT ISU
• Years 4-8 – Annual Update– If applicable, an updated
assessment plan is sent to UAS– Collection of forms provided to
programs to be used in the next cycle
ADVANTAGES
1. Assessment is a separate from program review, but still leads the process.
- A culture of inquiry over a culture of evidence.- One is about performance, the other about
learning.
ADVANTAGES
What does it mean?
Where do we go?
How well are we doing?
- Jane Chapman, Heartland Community College, It’s Eleven O’Clock: Do You Know Where Your Students Are?, 15th Annual Comm. College Assessment Fair, Bloomington, IL, 2011
A focus on what results mean, instead of what they are.
ADVANTAGES
2. Value people’s time, resources, energy, history, thoughts, and opinions.
- Attitudes, beliefs, and engagement with assessment are just as important as processes.
- Faculty have control through the self-study and review process.
- Assessment & program review aren’t used to micro-manage or fix a problem that is not perceived to exist.
- Ideas, people, and dialog lead – technology follows.
ADVANTAGES
3. There is consensus and agreement about the process, and it is used to make decisions.
- It’s more important to be genuine, rather than transparent.
- Assessment and program review are never goals – they’re processes.
CONTINUOUS IMPROVEMENT
Nothing is ever perfect, but it can be healthy.
Incremental changes to rubric.
QUESTIONS?COMMENTS?
FOR MORE INFORMATIONhttp://assessment.illinoisstate.edu/program/resources.shtml
http://provost.illinoisstate.edu/academic/program-review.shtml