assessment of academic advising 2011 nacada summer institute charlie nutt the presenters acknowledge...

Assessmentof

Academic Advising2011 NACADA Summer Institute

Charlie Nutt

The presenters acknowledge and appreciate the contributions of NACADA colleagues Susan

Campbell, Rich Robbins, Mike Kirk-Kuwaye, Lynn Higa, Tom Grites, and Eric White in preparation of

materials for this presentation

NACADA Executive OfficeKansas State University

2323 Anderson Ave, Suite 225Manhattan, KS 66502-2912

Phone: (785) 532-5717 Fax: (785) 532-7732

e-mail: [email protected]

© 2011 National Academic Advising Association

The contents of all material in this presentation are copyrighted by the National Academic Advising Association, unless otherwise indicated. Copyright is not claimed as to any part of an original work prepared by a U.S. or state government officer or employee as part of that person's official duties. All rights are reserved by NACADA, and content may not be reproduced, downloaded, disseminated, published, or transferred in any form or by any means, except with the prior written permission of NACADA, or as indicated below. Members of NACADA may download pages or other content for their own use, consistent with the mission and purpose of NACADA. However, no part of such content may be otherwise or subsequently be reproduced, downloaded, disseminated, published, or transferred, in any form or by any means, except with the prior written permission of, and with express attribution to NACADA. Copyright infringement is a violation of federal law and is subject to criminal and civil penalties. NACADA and National Academic Advising Association are service marks of the National Academic Advising Association.

The Global Community for Academic Advising

mailto:[email protected]

• Review of assessment cycle

• Identifying stakeholders

• Identifying student learning outcomes

• Mapping the students’ learning experiences

• Identifying measures

• Using the CAS standards

• Assessment and research

Overview


The Assessment Cycle (Maki, 2002, 2004)

Gather

Evidence

Interpret

Evidence

Identify

Outcomes

Implement Change

Mission/PurposesEducationalObjectives


The Assessment Flowchart (adapted from Darling, 2005)

Student Learning OutcomesCognitive, Psychomotor, AffectiveProcess/Delivery Outcomes

Mapping the ExperienceWhat experiences?When or by when?

Gathering EvidenceWhen gathered?Where gathered?

How often gathered?From whom gathered?

How gathered?Minimum performance criteria for success?

ValuesVision

MissionGoals

Programmatic Outcomes

Sharing and Acting Upon the ResultsInterpret how results inform practice

How and with whom to share interpretationFollow up on implemented changes

Start the process all over again!


Mapping of Outcomes

The Assessment Matrix/Table

Institutional Mission Statement

Local Mission Statement

Specific Goal or Objective

Specific Process/Delivery Outcome or Student Learning Outcome

Where Outcome Occurs

When or By When Outcome Occurs

Outcome Measure

Minimum Performance Criteria for Success

Data Instruments

Action(s) Based on Outcome Data


Identifying Key Stakeholders:Who Should Be Involved?

• Colleagues, faculty, administrators, institutional researchers, staff, students, and institutional community

• Decide how the assessment team will interact, overlap, and/or support other institutional efforts

• Encourage stakeholders on and off campus

• Continuous communication and feedback is a must!


• Building of a shared trust

• Building of a shared motivation

• Building of a shared language

• Building of support for academic advising institutionally-wide

• The result is a shared ownership and belief in the process

Benefits of a Collective and Collaborative Process


Exercise 1:Identify Your Stakeholders


Key Terms• Process/Delivery Outcomes – Articulate the expectations

for how academic advising is delivered and what information should be delivered through the academic advising experience

• Student Learning Outcomes – Articulate what students are expected to know, do, and appreciate as a result of involvement in the academic advising experience

• Mapping – The process of determining when, where and how the outcomes for advising will be accomplished over the students’ academic career and beyond


Process/Delivery Outcomes• Typically what is evaluated via student

satisfaction surveys• e.g., “the advisor provided accurate information”• e.g., “the setting for the advising session was

appropriate”• etc.

• Do not assess student learning, but evaluate what processes occurred during the advising interaction


Focus onStudent Learning Outcomes


It’s All About Student LearningIt’s All About Student Learning

Understanding what, how and when students learn through academic advising


Advising is Teaching


Advisor as Teacher

• facilitator of communication

• coordinator of learning experiences

• referral agent who connects students with all of the institution’s resources and co-curricular opportunities that can help them be successful


Teaching

Knowledge of subject matter

Preparation - planning and organization of course material

Advising

Knowledge of institutional policies, procedures, programs, resources

Preparing for advising meetings


Teaching

Engaging students in participation of their learning

Giving students feedback on their progress

Helping students learn to analyze and problem solve

Advising

Guiding students to be self-directed and autonomous

Working with student to regularly evaluate the student’s goals and progress

Assisting students in decision-making skills


Teaching

Clarity of presentation of subject matter

Establishment of dialogue with students in the classroom

Demonstration of excellent listening skills

Advising

Sharing information in a clear manner

Leading students to question and interact with the advisor

Listening to what advisees are saying verbally and non-verbally


Teaching

Assigning out-of class work

Working from a developmental perspective

Providing a learner-centered environment

Advising

Assigning out-of-session tasks

Working from a developmental perspective

Providing a learner-centered environment


Teaching

Identification and communication of student learning outcomes for the curriculum or course

Use of a course syllabus

Advising

Identification and communication of student learning outcomes for the advising process/interaction

Use of an advising syllabus


Focus on the Learner/Advisee

As a result of academic advising, what do we want students to demonstrate that they…

• Know (cognitive student learning outcome)

• Are able to do (behavioral student learning outcome)

• Value and appreciate (affective student learning outcome)


Mapping of Outcomes








Outcome Measure


Data Instruments



Cognitive SLOs

What do we want students to KNOW as a result of participating in academic advising?

Know general education requirements

Know about academic support services

Know how to use the student information system to register

Know how to use the catalog

Etc…….


Exercise 2:Identify Three Things You Want Students to Know as a Result of

Academic Advising


Tips on Developing SLOs

• avoid compound SLOs, e.g., “students understand and value the purpose of a liberal arts education,” “students understand and utilize their degree progress reports”

• as you are developing each SLO, think about where and when the opportunities for student learning may occur for that desired SLO

• as you are developing each SLO, think about how you might measure whether or not the SLO has been achieved


Behavioral/Psychomotor SLOs

What do we want students toDo as a result of participating in academic advising?

Generate their degree audit

Make advising appointments

Keep advising appointments

Ask for help

Access course descriptions and degree requirements using the online catalog

Etc….


Exercise 3:Identify Three Things You Want Students to Do as a Result of

Academic Advising


Affective SLOs

What do we want students to Value or Appreciate

as a result of participating in academic advising?

Value/Appreciate general education

Value/Appreciate the advising relationship

Value/Appreciate the process of learning

Etc….


Exercise 4:Identify Three Things You Want

Students to Value or Appreciate as a Result of Academic Advising


Clear and measurable student learning outcomes for academic advising may also be derived from

• the NACADA Concept of Academic Advising

• the NACADA Core Values

• the CAS Standards for Advising

All are included in your Summer Institute Guide


Using the CAS Standards in Program Assessment


The Council for the Advancement of Standards in Higher Education (CAS)

has developed roughlythree dozen different

“self assessment” guidesavailable in all areas of

Student Services. They rangefrom assessment of advisingto housing to help desks…


The CAS Standards for Academic Advising can easily be utilized as outcome statements for your advising program


It is not necessary to attempt to use all of the CAS Standards as your outcome statements…

…for those you may choose to use, be sure to develop related student learning outcomes as well as process/delivery outcomes


Mapping the Learning Experience• What should be learned: e.g., student will know the

components of the institution’s General Education requirements

• Where it should be learned: e.g., orientation workshops, advising sessions, personal reading of catalog or curriculum guide

• When or by when it should be learned: e.g., prior to first year (orientation); by end of first semester; by end of first year; before declaring a major, etc.


Mapping of Outcomes








Outcome Measure


Data Instruments



Exercise 5:Mapping Your Desired Outcomes:

Identifying Where/When the Process/Delivery Occurs

ORWhere/When the Student Learning

Opportunities Occur


Once the desired process/delivery and student learning outcomes have been identified, as well as when and where they will occur, the next step is to determine who or what will be measured and how the data will be gathered…

…using multiple measures of varying types


Mapping of Outcomes








Outcome Measure


Data Instruments



Types of Measurement and Data

• Qualitative

• Quantitative

• Direct

• Indirect

• Multiple measures!!!


Measures can (and should) include existing institutional data

• Information from Institutional Research, Admissions, Registrar, etc. can provide tracking data, GPAs, retention rates, and other information you can utilize as assessment data

– this can be a source of some of the multiple measures utilized (in addition to formal instruments, satisfaction inventories, and others)


For both process/delivery and student learning outcomes, you need to identify the minimum criteria for success of the outcome measure, e. g.,• number of students exhibiting a specific learning performance• percentage of students exhibiting a specific learning performance• advisor rating of student performance• student rating of specific aspect of advising process• advisor rating of specific aspect of advising process• etc.


Exercise 6:Identifying Multiple Measures

Select one process/delivery or one student learning outcome, and identify three ways to measure whether or not that specific outcome has been met


So I Have The Data – Now What?

• Sharing and Acting Upon Results– Interpret results regarding how they inform the

advising process/delivery, student learning, and decision-making

– Determine with whom and how the results are reported

– Decide how you will implement changes based on the results

– Start assessment cycle again…


Determine How and With Whom Results Are Shared

• Administration: President, Provost, various committees

– via annual report, strategic plan, white paper, Web sites, etc.

• Faculty: all faculty, curricular committees, faculty advisors

– via performance reviews, annual reports, strategic plans, Web sites, etc.

• Students: all students, student advisees, student senate, student groups

– via newsletters, annual reports, Web sites, etc.

• Budgeting entities

– via annual reports, budget requests, Web sites, etc.

• Accreditors

– via self-studies, accreditation reports, Web sites, etc.


• Revise pedagogy or curriculum

• Develop/revise advisor training programs

• Design more effective programming– advising, orientation, mentoring, etc.

• Increase out-of-class learning opportunities

• Shape institutional decision making– planning, resource allocation

• Other…

Interpret How Results Will Inform Decision Making


Decide How You WillFollow-up on Implemented

Changes• Timetable to implement changes

– implement all or specific components on a schedule

• Assessment of implemented changes– repeat assessment cycle again

• Continuous assessment– assessment is on-going


Final Exercise:

Complete One of Your Outcome Matrices


THANK YOU!

2011 NACADA Summer Institute

Assessment of Academic AdvisingDr. Rich Robbins

Bucknell University


assessment of academic advising 2011 nacada summer institute charlie nutt the presenters acknowledge...

Documents

purpose of nacada

members of nacada

assessment team

institutional researchers

institutional communitydecide

institutional effortse

copyright infringement

state government officer