nacada regional 2012 handout-1

Advising Learning Outcomes: A Multi-institutional Study

Region 8 NACADA ConferencePre-Conference Workshop

March 18, 2012

Cathleen L. Smith Professor Emerita of Psychology

Portland State University

[email protected] Janine M. Allen Professor Emerita of Education

Portland State University

[email protected]

Agenda Describe the advising curriculum and what we

would expect students to learn from advising encounters

Summarize research on how community colleges and universities differ

Present a study that examines advising learning in community college and university students

Discuss future directions for advising learning research and assessment

Advising Learning: A New Emphasis

New emphasis: What students should learn in academic advising encounters

NACADA Concept of Academic Advising: – Advising is “integral to fulfilling the teaching

and learning mission of higher education”– And, as such, has its own curriculum,

pedagogy, and student learning outcomes (NACADA, 2006)

Learning-Centered Advising:State of the Literature

More conceptual than empirical

Focused on: Identifying the advising curriculum (e.g., Hemwall &

Trachte; Lowenstein) and learning outcomes advising should produce in students Distinguishing between learning-centered advising and more traditional approaches (i.e., prescriptive and developmental advising)


Focused on: Speculating about the mechanisms by which learning takes place in advising encountersDifferentiating learning outcomes from other aspects of advising (e.g., student responsibilities) Advocating for the adoption and use of a learning-centered advising paradigm


A logical next step in the evolution of this new advising paradigm is to gather empirical data on the learning outcomes that are thought to arise from participation in advising encounters

Deriving our Advising Learning Outcomes

In formulating our learning outcomes, we began with our conception of quality academic advising as a multi-dimensional process encompassing five domains

– Integration– Referral– Information– Individuation– Shared responsibility


Integration of the student’s academic, career, and life goals with each other and with aspects of the curriculum and co-curriculum

Referral to campus resources for academic and non-academic problems

Provision of information about degree requirements and how the university works with regard to policies and procedures


Individuation, or consideration of students’ individual characteristics, interests, and skills

Shared responsibility, or encouraging students to assume responsibility for their education by providing them with opportunities to develop and practice planning, problem-solving, and decision-making skills

Advising Curriculum

Integration, Referral, Information: Advising Content

Individuation, Shared Responsibility: Advising Pedagogy

Advising Content: Integration

Connected learning: one of the primary goals of liberal education

Connected learning: central to developmental advising

Connected learning: considered by students as especially influential

Integration: Learning OutcomeUnderstands Connections

“I understand how my academic choices at name of institution connect to my career and life goals”

How does learning on this outcome facilitate student success?– Students who are purposeful in their

educational decisions are more likely to succeed

Advising Content: Referral

Advising is conduit through which the student becomes aware of resources at the institution that assist with – Academic problems (e.g., writing, test

anxiety, tutoring) – Non-academic problems (e.g., child care,

financial, physical and mental health)

Referral: Learning Outcome Knows Resources

“When I have a problem, I know where at name of institution I can go to get help”

How does learning on this outcome facilitate student success?– Students who use support services available

to them tend to perform better in college

Advising Content: Information

Our past research has shown the primary importance to students of the information domain; thus it was represented by two learning outcomes

Advising Content: Information

First, advising should assist students in understanding the multitude of requirements they face in order to successfully complete their program of study

Second, advising involves helping students navigate their complex institution by assisting them in understanding how things work with regard to its timelines, policies and procedures

Information: 1st Learning Outcome Knows Requirements

Community college students: “I know what requirements (e.g., prerequisites, general education, transfer requirements) I must fulfill at name of community college in order to meet my educational goals”

or

University students: “I know what requirements (e.g., major, general education, other university requirements) I must fulfill in order to earn my degree”

Information: 1st Learning OutcomeKnows Requirements

How does learning on this outcome facilitate student success?

– Knowledge of degree requirements is the sine qua non for student success (the essential condition without which students cannot obtain a degree).

Information: 2nd Learning Outcome Understands How Things Work

“I understand how things work at name of institution (timelines, policies, and procedures with regard to registration, financial aid, grading, graduation, petition and appeals, etc.)”

How does learning on this outcome facilitate student success?– Students must develop a cognitive map of the

bureaucratic geography of their institution in order to successfully navigate it

Advising Learning Outcomes = Retention Predictors

Having a plan to achieve one’s educational goalsHaving a significant relationship with faculty or staff on campus

Retention-Related Learning Outcome:Has Educational Plan

“I have a plan to achieve my educational goals” How does learning on this outcome facilitate

student success?– Having a plan to achieve one’s educational

goals is a measure of goal commitment– Students without plans may take longer to

graduate and are at risk of dropping out of college

Retention-Related Learning Outcome:Has Significant Relationship

“I have had at least one relationship with a faculty or staff member at name of institution that has had a significant and positive influence on me”

How does learning on this outcome facilitate student success?– Students who can identify mentors on campus have

developed significant relationships with faculty or staff members

– Research has shown that students with these relationships are likely to persist

Affective Learning Outcomes

We wanted to measure not only what students know and can do, but also what they might appreciate or value, as a result of participation in advising

We wanted outcomes that might reflect that students had received quality academic advising, benefited from it, and thought others might too

Affective Learning Outcomes

Values Advisor/Advisee Relationship– “It is important to develop an advisor-advisee

relationship with someone on campus”

Supports Mandatory Advising– “There should be mandatory academic advising for

students”

How does learning on these affective outcomes facilitate student success?– No evidence yet

What are These Advising Learning Outcomes Measuring?

All 8 learning outcomes are measures of students’ meta-cognition

Meta-cognition: What students know about their own knowledge and values

Need for Empirical Study of Advising Learning Outcomes

Present a study that used these measures of advising learning

To examine differences between two- and four-year institutions

Differences in Community College & University Environments

Academic Environment

Social Environment

Physical Environment

Support Environment

Differences in Advising at Community Colleges & Universities

Our own research showed that there are differences in students’ advising experiences

Students at community colleges – are more satisfied– have more sources of advising and support – experience closer relationships with

advisors

Differences in Advising Learning at Community Colleges & Universities?

Community colleges and universities provide students with different advising environments

Are there also differences in advising learning at these two types of educational environments

Research Questions

Given the differences in students’ advising experiences at community colleges and universities, do students at the two types of institutions :– differ on our eight advising learning

outcomes? – agree or disagree on the parts of the

advising curriculum that are more difficult?

Multi-Institutional Study:Nine study institutions in Oregon

Community Colleges – Chemeketa Community College – Portland Community College

Private Universities– Concordia University– University of Portland

Public Universities– Eastern Oregon University – Oregon State University – Portland State University – University of Oregon– Western Oregon University

Method

Online administration of the Inventory of Academic Advising Functions – Student Version

Administered in 2010 or 2011 Students invited to participate:

– Universities: All fully admitted students – Community colleges: All students enrolled in

credit-bearing classes

Method

To ensure that all students in the study had similar educational goals– We selected students at the two

community colleges who indicated that their main reason for attending the college was to earn credit toward a bachelor’s (4-year) degree

Research SampleInstitution Number of

ParticipantsParticipation Rate

Community Colleges 7172

Chemeketa Community College 1159 33.7

Portland Community College 6013 21.1

Four-Year Institutions 15156

Private Universities

Concordia University 437 43.1

University of Portland 1599 52.5

Public Universities

Eastern Oregon University 1206 38.3

Oregon State University 4026 22.1

Portland State University 2746 15.5

University of Oregon 3647 21.1

Western Oregon University 1495 32.7

Method

To ensure that all students in the study were at a similar educational level

– We selected only lower division students at the universities (freshmen and sophomores)

Reducing the sample to 12,003 students– 4831 university students– 7172 community college students

Respondent DemographicsGender and Age

Gender Community College Students

n (%)*

University Students

n (%)*

Totaln (%)*

Female 4552 (64.2%) 3129 (64.8%) 7681 (64.4%)

Male 2543 (35.8%) 1700 (35.2%) 4243 (35.6%)

Unknown 77 2 79

Mean Age 27.9 years 20.6 years

* Percent of those with known gender

Respondent DemographicsEthnicity

Ethnicity Community College Students

n (%)*

University Students

n (%)*

Totaln (%)*

Asian American 533 ( 9.1%) 442 ( 10.1%) 975 ( 9.6%)

African American 367 ( 6.3%) 60 ( 1.4%) 427 ( 4.2%)

Hispanic 640 (11.0%) 330 ( 7.6%) 970 (9.5%)

Native American 102 ( 1.7%) 52 ( 1.2%) 154 ( 1.5%)

White 4048 (69.4%) 3340 (76.7%) 7388 (72.5%)

Multi-Ethnic 124 ( 2.1%) 95 ( 2.2%) 219 ( 2.1%)

Pacific Islander 23 ( 0.3%) 36 ( 0.8%) 59 ( 0.6%)

Unknown 1333 476 1809

* Percent of those with known ethnicity

Measures of Advising Learning Outcomes

8 advising learning outcomes, each measured by a 6 point Likert-type scale

1 = Strongly Disagree 6 = Strongly Agree

First Research Question

Do students at community colleges and universities differ on the eight advising learning outcomes?

– ANCOVA– Controlling for age

Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)

1=strongly disagree 6=strongly agree***p<.001

Advising Learning Outcome Community College Studentsn = 7170

University Studentsn = 4831

I understand how my academic choices at name of institution connect to my career and life goals

5.10 (1.09)*** 4.81 (1.14)



Advising Learning Outcome Community College Students

n = 7170


When I have a problem, I know where at name of institution I can go to get help

4.44 (1.48)*** 4.27 (1.37)




n = 7170


I know what requirements I must fulfill in order to meet my educational goals / earn my degree

4.81 (1.32) 4.76 (1.21)




n = 7170


I understand how things work at name of institution

4.65 (1.29)*** 4.28 (1.24)




n = 7170


I have a plan to achieve my educational goals

5.43 (0.93)*** 5.29 (0.95)




n = 7170


I have had at least one relationship with a faculty or staff member at name of institution that has had a significant and positive influence on me

4.30 (1.67)*** 4.10 (1.54)




n = 7170


It is important to develop an advisor-advisee relationship with someone on campus

4.84 (1.26) 5.07 (1.05)***

There should be mandatory academic advising for students

4.16 (1.64) 4.43 (1.44)***




n = 7170


It is important to develop an advisor-advisee relationship with someone on campus

4.84 (1.26) 5.07 (1.05)***

There should be mandatory academic advising for students

4.16 (1.64) 4.43 (1.44)***

I have had at least one relationship with a faculty or staff member . . .

4.30 (1.67) 4.10 (1.54)





Understands Connections 5.10 (1.09)*** 4.81 (1.14)

Knows Resources 4.44 (1.48)*** 4.27 (1.37)

Knows Requirements 4.81 (1.32) 4.76 (1.21)

Understands How Things Work 4.65 (1.29)*** 4.28 (1.24)

Has Educational Plan5.43 (0.93)*** 5.29 (0.95)

Has Significant Relationship4.30 (1.67) 4.10 (1.54)

Values Advisor-Advisee

Relationship

4.84 (1.26) 5.07 (1.05)***

Supports Mandatory Advising4.16 (1.64) 4.43 (1.44)***

Second Research Question

Do students at community colleges and universities agree or disagree on the parts of the advising curriculum that are more difficult?

– Within subjects ANOVA– Compared the results of the two groups

Do Students Report More Learning on Some Outcomes than Others?

1=strongly disagree 6=strongly agree



Understands Connections 5.10b 4.81c

Knows Resources 4.44e 4.27e

Knows Requirements 4.81c 4.76c

Understands How Things Work 4.64d 4.28e

Has Educational Plan5.43a 5.29a

Has Significant Relationship4.30f 4.10f

Values Advisor-Advisee

Relationship

4.84c 5.07b

Supports Mandatory Advising4.16g 4.43d

Summary of Findings

Students at both types of institutions report extensive advising learning

But student learning varies as a function of educational environment

Summary of Findings

Community college students are more likely to – Understand connections– Know resources – Understand how things work– Have an educational plan

Summary of Findings

University students are more likely to – Value the advisor/advisee relationship– Support mandatory advising

Summary of Findings

Some aspects of the advising curriculum are easier for students to master, evidenced by higher scores on these outcomes

Other aspects are more difficult For the most part, these patterns were

the same for the two groups of students

Summary of Findings Community college and university

students

– Most likely to report they had an educational plan

– Least likely to report they had a significant relationship

Only exception: Supports mandatory advising

Implications for Practice

Universities need to strengthen their delivery of the advising curriculum

Implications for Practice Both types of institutions need to make

concerted efforts to ensure that– Students know where to go to get help with

problems– Students are hooked up to a caring and helpful

person at the institution– Students know how things work at their

institution with regard to timelines, policies, and procedures

Next Steps: Research on Advising Learning

Is advising learning associated with whether and how often students receive advising?

Is advising learning associated with where students get their information about classes to take to meet requirements?

Does advising learning predict retention?

Next Steps: Assessment of Advising Learning

Electronic advising portfolio– Facilitated by the advisor – Maintained by the student

Reflection scaffolding: Shared responsibility in action

Chronicles advising history Evolves as the student learns and develops

Assessment of Advising Learning:Advising e-Portfolio

Includes exercises and activities – designed by the advisor– completed by the student– aided by embedded links to resources and

tools Promotes advising learning Provides opportunities for students to

document their learning


Advising learning involvesSelf-assessment Identifying and connecting academic, career, and life goalsDeveloping plans to achieve goals


Challenges to implementation and useTime

– Students– Advisors

Who assesses the portfolio?How is it assessed?

Discussion

Questions

Comments

Implications