ASSESSMENT OF MISSION AT CHRISTIAN COLLEGES AND

UNIVERSITIES

HOW DO WE KNOW WE ARE ACHIEVING OUR MISSIONS?

ASBURY COLLEGE

GAY H. HOLCOMB, PH.D.Director of Institutional Research and Assessment

QUALITY ENHANCEMENT PLAN (QEP)

Recent requirement of SACS Focus on a learning outcome central to the

mission of the institution Become intentional about a critical

element of its mission. Broad-based involvement & support Commit significant resources, able to

sustain Topic that institution becomes known for

ASBURY’S QEP ON SPIRITUAL VITALITY

Academic Excellence & Spiritual Vitality

Cornerstone Project Holiness Scripture Stewardship Mission

STUDENT LEARNING OUTCOMES1. Understand Cornerstones2. Core Curriculum 3. Personally Value Cornerstones 4. Creation Care5. Service Learning6. Christian Mission & Stewardship7. Increase Student Initiative

MEASURES1. Capstone Course Assignments2. Furnishing the Soul Inventory by Alidade 3. Case Studies of Exemplars4. Chapel Survey5. Alumni Surveys6. Co-Curricular Involvement Survey &

Participation Logs7. Waste Audits8. Mix of qualitative/quantitative & direct/indirect

AT THE HEART OF WHO ASBURY IS Enthusiastic buy-in

Each Cornerstone tied to Mission Each Stu Learning Outcome, Measure,

Achievement Target tied explicitly to Strategic Vision

Impacts everything we do Cornerstone Project approved by SACS

in Dec Initial phase – no data yet

Fully re-accredited by SACS in December 2009

BETHEL UNIVERSITY

JOEL FREDERICKSON, PH.D.Professor and Chair, Psychology Department

MINING EXISTING DATA Institutions generally participate in

national surveys (indirect assessment) and standardized tests (direct assessment)

Use this existing data to assess your institution’s mission and core values

EXAMPLES OF SURVEYS National Survey of Student Engagement

(NSSE) Cooperating Institutions Research Project

(CIRP) First-year survey (from HERI @ UCLA) College Senior Survey (CSS) Follow-up to CIRP College Student Beliefs & Values Inventory

(CSBV) from HERI Student Satisfaction Inventory (SSI)

Additional CCCU items are added to this survey.

EXAMPLES OF DIRECT ASSESSMENTS OF LEARNING

Collegiate Learning Assessment (CLA) Collegiate Assessment of Academic

Proficiency (CAAP), from ACT Major Field Tests, from ETS Defining Issues Test (DIT): Measure of

Moral Thinking California Critical Thinking Skills Test

(CCTST)

LINKING ASSESSMENTS TO MISSION Examine your institution’s mission,

vision, and, if applicable, core values. Do an audit of the various assessments

your institution is engaged in. Link the two together.

NSSE items: Bethel’s Core Value of Reconciliation 1e. Included diverse perspectives (different races, religions,

genders, political beliefs, etc.) in class discussions or writing assignments

1u. Had serious conversations with students of a different race or ethnicity than your own

1v. Had serious conversations with students who are very different from you in terms of their religious beliefs, political opinions, or personal values

10c. Institutional Emphasis: Encouraging contact among students from different economic, social, and racial or ethnic backgrounds

11l. Educational & Personal Growth: Understanding people of other racial and ethnic backgrounds

HAD SERIOUS CONVERSATIONS WITH STUDENTS OF A DIFFERENT RACE OR ETHNICITY THAN YOUR OWN

2005 2006 2007 2008 20092.1

2.2

2.3

2.4

2.5

2.6

2.7

2.8

BethelCCCUCarnegie

ENCOURAGING CONTACT AMONG STUDENTS FROM DIFFERENT ECONOMIC, SOCIAL, AND RACIAL OR ETHNIC BACKGROUNDS

2005 2006 2007 2008 20092.1

2.2

2.3

2.4

2.5

2.6

2.7

2.8

2.9

3

BethelCCCUCarnegie

STUDENT SATISFACTION INVENTORY: BETHEL’S CORE VALUE OF CHRIST-FOLLOWERS

74. Being on this campus is contributing to my spiritual growth.

75.My understanding of God is being strengthened by classroom and/or campus experiences.

76.Faculty, administrators, and/or staff are helpful to me in processing issues related to my faith.

78.This campus provides adequate opportunities for involvement in ministry.

79.Given where I am spiritually right now, this campus is a good "fit" for me.

BEING ON THIS CAMPUS IS CONTRIBUTING TO MY SPIRITUAL GROWTH

Bethel CCCU5.2

5.4

5.6

5.8

6

6.2

6.4

6.6

ImportanceSatisfaction

CSS: ADDRESSING THE LEADERSHIP COMPONENT OF OUR MISSION

Students “very satisfied” or “satisfied” with their college in: Leadership opportunities

Since entering college you have: Participated in leadership training

Students reporting “much stronger” abilities and skills compared to when they first entered college: Leadership abilities

Compared with the average person of his/her age, student rated self “above average” or “highest 10%” in Leadership ability

Considerations noted as “essential” or “very important” in thinking about student’s career path: Leadership potential

Student objectives noted as “essential” or “very important”: Becoming a Community Leader

CSS ANALYSES Comparisons with other 4-year religious

colleges & all 4-year private colleges Longitudinal comparisons for:

Becoming a Community Leader Leadership ability

BECOMING A COMMUNITY LEADERLONGITUDINAL CHANGES

First-year Senior Change

Bethel 35.5 44.1 8.6

Religious Colleges 31.8 41.1 9.3

All Private Colleges 34.0 40.8 6.8

CORE VALUES: LEARNERS & TRUTH-SEEKERS

Combining data from direct and indirect assessments. CCTST indicates good growth in Critical

Thinking CLA results indicate poor growth in

integrative writing NSSE & CSS data indicate that students

perceive that they are growing in critical thinking

SUMMARY Don’t get caught up in precise

definitions (the “what really is critical thinking?” conversations)

Focus on using a variety of exisiting measures to assess mission.

Look for a convergence of evidence

CORNERSTONE UNIVERSITY

TIMOTHY DETWILER, Ph.D. Associate Provost for Traditional Programming

CORNERSTONE UNIVERSITY 2650 Students

1200 Traditional - 1000 Adults 250 Seminary - 200 Seminary (Asia)

Began in 1941 A faith-based institution Grand Rapids, St. Joseph, Kalamazoo,

Detroit, Holland Worldview, information literacy, civitas

and spiritual formation mark the programs

Mission Statement

University Assessment Plan

Campus-wide Learning Objectives

Core Curriculum Design

A Specific Core Course

CORNERSTONE UNIVERSITY

Mission Statement

To empower men and women to excel as influencers in our world for Christ by offering a student-focused learning community where Jesus Christ is central.

MISSION STATEMENTS One of the main purposes of a mission

statement is to guide decisions about organizational priorities in a systematic manner. Once an organization has identified what it wants to accomplish, it has to determine how to accomplish those things and how to measure the success of those activities.

(Nancy Kidd – NCA Director)

Effective organizations are the momentary results of constant reassessment and redesign. When the effectiveness of organizations is not tested, they rapidly tend to become unresponsive and inefficient.

Rodney Stark

ACADEMICS/SPIRITUAL FORMATION A public frame for the assessment

efforts Resonate deeply with the cultural

memory A stimulus to initiate change

Referring to a mason’s level A leveling instrument or a plumb line Placing our “stones” in order and

keeping them in line

MISHQELET PROJECT

“CIVITAS”DIRECT ASSESSMENT METHODS

Essays (pre/post experiences) Exams Experiential Aspects (self

reporting/observational) Electronic Discussion Board (rubric) Journal (rubric) Capstone Project Portfolio

CIVITASINDIRECT METHODS OF ASSESSMENT

Student Satisfaction Survey (EFS, SSI, CSS)

Experiential Reflective Journals Alumni Surveys Course Evaluation Forms Professional Evaluation of the Program

ENG 212 WRITING IN CULTURE

Link to mission Link to core curriculum learning

objectives Link to the Humanities Division ENG 212

THE KEY TO IT ALL

Institutional leaders have the major role in determining whether assessment results are important in making improvements. Unless top-level administrators believe that planning and assessment data are linked and important in strategic and logistic decisions, the assessment program in general will be brought into a dead-end. (IACBE)

MISSION DRIVEN ASSESSMENT

The end of virtually all assessment activities (except for some accountability initiatives) is the improvement of student learning, campus programming and institutional enhancement – in other words “organizational development”.

(Nichols & Nichols)

ORGANIZATIONAL DEVELOPMENT

Assessment is a strategy for organizational

Accountability Distinctiveness Accreditation Effectiveness Reflection Development

ORGANIZATIONAL DEVELOPMENT

MORE IMPORTANTLY, assessment is a strategy for understanding, confirming

and improving student learning and educational quality based upon an

institution’s mission.

Your Context

Your Mission, History, Culture,

Vision, Goals

Assessment is a matter of

institutional responsibility,

integrity,and mission.

MOUNT VERNON NAZARENE UNIVERSITY

DAVID W. KALE, Ph.D.Director of Assessment and Professor of

Communication

ASSESSING STUDENT LEADERSHIP MVNU Vision Statement: An academic

community of faith preparing Christ-like leaders for life long service and global impact.

The purpose of this assessment project was to help students assess their own leadership development.

In the process we wanted to see the degree to which institutional vision was reflected in student thinking about leadership.

ASSESSING STUDENT LEADERSHIP To achieve this, I worked with student

leaders to develop a rubric with which they could assess their own leadership ability.

I started by having all leaders list aspects of leadership they would like to use to assess themselves.

I then compiled a master list with all the leadership aspects included in their lists.

ASSESSING STUDENT LEADERSHIP I then presented the total list to them

and asked them to vote for five on the list they wanted to use to assess themselves.

Based on their voting, I then constructed a rubric using their top five dimensions.

You have a copy of the rubric I constructed in the materials which were distributed to you.

ASSESSING STUDENT LEADERSHIP As I have found in using this method of

rubric construction in other settings, the method itself is an important assessment procedure.

What aspects of leadership do our students consider so important that they want to use them in assessing their own leadership abilities?

ASSESSING STUDENT LEADERSHIP1. Compassion2. Empowering3. Communication – listening4. Honesty5. Empathy

ASSESSING STUDENT LEADERSHIP We were particularly interested and pleased

to see compassion as the #1 leadership ability chosen by our students.

This certainly ties in to the vision statement’s emphasis on developing Christ-like leaders.

One of our staff members who had done his doctoral studies in leadership indicated that he had never seen a list of leadership qualities that included compassion.

ASSESSING STUDENT LEADERSHIP While we were very pleased with the

orientation of the list toward empowering and meeting the needs of others, we also had some concerns with the list.

Leadership characteristics such as problem solving, ability to motivate others and creativity came in between 15th and 20th on the list. This gave our student development staff some information about the kinds of training student leaders needed.

ASSESSING STUDENT LEADERSHIP Another concern we had with the rubric

is that when the student leaders used it to assess themselves, they gave themselves very high marks right from the start.

We therefore had a ceiling effect which negated our plan to have students assess themselves at the beginning and end of the academic year to chart their development during that time.

ASSESSING STUDENT LEADERSHIP This was a surprise to me because in

using this approach to rubric construction for student self assessment in their moral and ethical reasoning ability, almost none of them gave themselves high marks.

ASSESSING STUDENT LEADERSHIP In summary of this strategy, we found the rubric

construction process to be very informative in terms of giving us feedback on the degree to which institutional vision was reflected in the leadership qualities students selected to assess their leadership abilities.

It was also helpful in providing information to student development as to areas of leadership development they might focus on in training student leaders.

We did not find this strategy helpful in developing a method of student self assessment that would permit them to identify areas in which they needed to improve or to track their development over an academic year.

GENERAL EDUCATION OBJECTIVE We have a general education objective

associated with our mission statement which addresses developing students’ abilities in moral and ethical reasoning.

SENIOR REQUIRED GENERAL EDUCATION COURSE IN APPLIED CHRISTIAN ETHICS

Section sizes kept to 24 students Debate national health care, drilling for

oil in ANWR, affirmative action, waterboarding, etc.

Course text: Stassen and Gushee, Kingdom Ethics, Intervarsity Press

DEFINING ISSUES TEST Based on the concepts of Lawrence

Kohlberg Essentially a critical thinking test in the

area of ethics Students get scores in Personal

Interest, Maintaining Norms, Post conventional or Principled Reasoning , etc.

We administer the test to first year students, juniors and seniors.

Mean S. D. N Sig.MVNU Seniors

23.06 10.62 76

MVNU Juniors

26.94 11.43 60 T= -2.04P= <.05D = .35

Nat’l Sample All Seniors

24.8 12.53 2441 ns

Nat’l Sample All Politically conserv.Seniors

23.48 12.01 881 ns

DEFINING ISSUES TEST – 2PERSONAL INTEREST SCORE

MVNU SENIORS 2006

MODEL FOR ANALYZING MORAL AND ETHICAL ARGUMENTS

Powers and Authorities

Social change Threats

Openness

Rules and Practices

Principles and Virtues

LoyaltiesInterestsPassionsModels

BASIC CONVICTIONSGod and Human

NatureForgiveness

Christlikeness and Justice

Mission of the Church

USE OF THE MODEL IN THE TEXT Students use the model in class in

groups to analyze a case of euthanasia from 20/20 on television.

Students use the model individually on an outside of class assignment to analyze a case provided by the instructor.

For mid-term exam, students use the model on a case they have not seen before exam period.

Mean S. D. N Sig.MVNU Seniors ( 2006)

23.06 10.62 76

MVNU Seniors (2009)

21.17 11.28 44

National Sample – All Seniors

24.8 12.9953 2441 D = .347T = -2.01P < .05

MVNU Juniors (2006)

26.56 11.82 164 D= .49T= -4.04P< .05

DEFINING ISSUES TEST – 2PERSONAL INTEREST SCORE

MVNU SENIORS, 2006 &2009

SUMMARY Now that there is evidence that I am

making progress in reducing students’ use of personal interest in making moral and ethical decisions, I would like to see even more progress on their parts in using internalized biblical principles as opposed to relying so much on external sources.