Download - Nacada regional 2012 handout-1
Advising Learning Outcomes: A Multi-institutional Study
Region 8 NACADA ConferencePre-Conference Workshop
March 18, 2012
Cathleen L. Smith Professor Emerita of Psychology
Portland State University
[email protected] Janine M. Allen Professor Emerita of Education
Portland State University
Agenda Describe the advising curriculum and what we
would expect students to learn from advising encounters
Summarize research on how community colleges and universities differ
Present a study that examines advising learning in community college and university students
Discuss future directions for advising learning research and assessment
Advising Learning: A New Emphasis
New emphasis: What students should learn in academic advising encounters
NACADA Concept of Academic Advising: – Advising is “integral to fulfilling the teaching
and learning mission of higher education”– And, as such, has its own curriculum,
pedagogy, and student learning outcomes (NACADA, 2006)
Learning-Centered Advising:State of the Literature
More conceptual than empirical
Focused on: Identifying the advising curriculum (e.g., Hemwall &
Trachte; Lowenstein) and learning outcomes advising should produce in students Distinguishing between learning-centered advising and more traditional approaches (i.e., prescriptive and developmental advising)
Learning-Centered Advising:State of the Literature
Focused on: Speculating about the mechanisms by which learning takes place in advising encountersDifferentiating learning outcomes from other aspects of advising (e.g., student responsibilities) Advocating for the adoption and use of a learning-centered advising paradigm
Learning-Centered Advising:State of the Literature
A logical next step in the evolution of this new advising paradigm is to gather empirical data on the learning outcomes that are thought to arise from participation in advising encounters
Deriving our Advising Learning Outcomes
In formulating our learning outcomes, we began with our conception of quality academic advising as a multi-dimensional process encompassing five domains
– Integration– Referral– Information– Individuation– Shared responsibility
Deriving our Advising Learning Outcomes
Integration of the student’s academic, career, and life goals with each other and with aspects of the curriculum and co-curriculum
Referral to campus resources for academic and non-academic problems
Provision of information about degree requirements and how the university works with regard to policies and procedures
Deriving our Advising Learning Outcomes
Individuation, or consideration of students’ individual characteristics, interests, and skills
Shared responsibility, or encouraging students to assume responsibility for their education by providing them with opportunities to develop and practice planning, problem-solving, and decision-making skills
Advising Curriculum
Integration, Referral, Information: Advising Content
Individuation, Shared Responsibility: Advising Pedagogy
Advising Content: Integration
Connected learning: one of the primary goals of liberal education
Connected learning: central to developmental advising
Connected learning: considered by students as especially influential
Integration: Learning OutcomeUnderstands Connections
“I understand how my academic choices at name of institution connect to my career and life goals”
How does learning on this outcome facilitate student success?– Students who are purposeful in their
educational decisions are more likely to succeed
Advising Content: Referral
Advising is conduit through which the student becomes aware of resources at the institution that assist with – Academic problems (e.g., writing, test
anxiety, tutoring) – Non-academic problems (e.g., child care,
financial, physical and mental health)
Referral: Learning Outcome Knows Resources
“When I have a problem, I know where at name of institution I can go to get help”
How does learning on this outcome facilitate student success?– Students who use support services available
to them tend to perform better in college
Advising Content: Information
Our past research has shown the primary importance to students of the information domain; thus it was represented by two learning outcomes
Advising Content: Information
First, advising should assist students in understanding the multitude of requirements they face in order to successfully complete their program of study
Second, advising involves helping students navigate their complex institution by assisting them in understanding how things work with regard to its timelines, policies and procedures
Information: 1st Learning Outcome Knows Requirements
Community college students: “I know what requirements (e.g., prerequisites, general education, transfer requirements) I must fulfill at name of community college in order to meet my educational goals”
or
University students: “I know what requirements (e.g., major, general education, other university requirements) I must fulfill in order to earn my degree”
Information: 1st Learning OutcomeKnows Requirements
How does learning on this outcome facilitate student success?
– Knowledge of degree requirements is the sine qua non for student success (the essential condition without which students cannot obtain a degree).
Information: 2nd Learning Outcome Understands How Things Work
“I understand how things work at name of institution (timelines, policies, and procedures with regard to registration, financial aid, grading, graduation, petition and appeals, etc.)”
How does learning on this outcome facilitate student success?– Students must develop a cognitive map of the
bureaucratic geography of their institution in order to successfully navigate it
Advising Learning Outcomes = Retention Predictors
Having a plan to achieve one’s educational goalsHaving a significant relationship with faculty or staff on campus
Retention-Related Learning Outcome:Has Educational Plan
“I have a plan to achieve my educational goals” How does learning on this outcome facilitate
student success?– Having a plan to achieve one’s educational
goals is a measure of goal commitment– Students without plans may take longer to
graduate and are at risk of dropping out of college
Retention-Related Learning Outcome:Has Significant Relationship
“I have had at least one relationship with a faculty or staff member at name of institution that has had a significant and positive influence on me”
How does learning on this outcome facilitate student success?– Students who can identify mentors on campus have
developed significant relationships with faculty or staff members
– Research has shown that students with these relationships are likely to persist
Affective Learning Outcomes
We wanted to measure not only what students know and can do, but also what they might appreciate or value, as a result of participation in advising
We wanted outcomes that might reflect that students had received quality academic advising, benefited from it, and thought others might too
Affective Learning Outcomes
Values Advisor/Advisee Relationship– “It is important to develop an advisor-advisee
relationship with someone on campus”
Supports Mandatory Advising– “There should be mandatory academic advising for
students”
How does learning on these affective outcomes facilitate student success?– No evidence yet
What are These Advising Learning Outcomes Measuring?
All 8 learning outcomes are measures of students’ meta-cognition
Meta-cognition: What students know about their own knowledge and values
Need for Empirical Study of Advising Learning Outcomes
Present a study that used these measures of advising learning
To examine differences between two- and four-year institutions
Differences in Community College & University Environments
Academic Environment
Social Environment
Physical Environment
Support Environment
Differences in Advising at Community Colleges & Universities
Our own research showed that there are differences in students’ advising experiences
Students at community colleges – are more satisfied– have more sources of advising and support – experience closer relationships with
advisors
Differences in Advising Learning at Community Colleges & Universities?
Community colleges and universities provide students with different advising environments
Are there also differences in advising learning at these two types of educational environments
Research Questions
Given the differences in students’ advising experiences at community colleges and universities, do students at the two types of institutions :– differ on our eight advising learning
outcomes? – agree or disagree on the parts of the
advising curriculum that are more difficult?
Multi-Institutional Study:Nine study institutions in Oregon
Community Colleges – Chemeketa Community College – Portland Community College
Private Universities– Concordia University– University of Portland
Public Universities– Eastern Oregon University – Oregon State University – Portland State University – University of Oregon– Western Oregon University
Method
Online administration of the Inventory of Academic Advising Functions – Student Version
Administered in 2010 or 2011 Students invited to participate:
– Universities: All fully admitted students – Community colleges: All students enrolled in
credit-bearing classes
Method
To ensure that all students in the study had similar educational goals– We selected students at the two
community colleges who indicated that their main reason for attending the college was to earn credit toward a bachelor’s (4-year) degree
Research SampleInstitution Number of
ParticipantsParticipation Rate
Community Colleges 7172
Chemeketa Community College 1159 33.7
Portland Community College 6013 21.1
Four-Year Institutions 15156
Private Universities
Concordia University 437 43.1
University of Portland 1599 52.5
Public Universities
Eastern Oregon University 1206 38.3
Oregon State University 4026 22.1
Portland State University 2746 15.5
University of Oregon 3647 21.1
Western Oregon University 1495 32.7
Method
To ensure that all students in the study were at a similar educational level
– We selected only lower division students at the universities (freshmen and sophomores)
Reducing the sample to 12,003 students– 4831 university students– 7172 community college students
Respondent DemographicsGender and Age
Gender Community College Students
n (%)*
University Students
n (%)*
Totaln (%)*
Female 4552 (64.2%) 3129 (64.8%) 7681 (64.4%)
Male 2543 (35.8%) 1700 (35.2%) 4243 (35.6%)
Unknown 77 2 79
Mean Age 27.9 years 20.6 years
* Percent of those with known gender
Respondent DemographicsEthnicity
Ethnicity Community College Students
n (%)*
University Students
n (%)*
Totaln (%)*
Asian American 533 ( 9.1%) 442 ( 10.1%) 975 ( 9.6%)
African American 367 ( 6.3%) 60 ( 1.4%) 427 ( 4.2%)
Hispanic 640 (11.0%) 330 ( 7.6%) 970 (9.5%)
Native American 102 ( 1.7%) 52 ( 1.2%) 154 ( 1.5%)
White 4048 (69.4%) 3340 (76.7%) 7388 (72.5%)
Multi-Ethnic 124 ( 2.1%) 95 ( 2.2%) 219 ( 2.1%)
Pacific Islander 23 ( 0.3%) 36 ( 0.8%) 59 ( 0.6%)
Unknown 1333 476 1809
* Percent of those with known ethnicity
Measures of Advising Learning Outcomes
8 advising learning outcomes, each measured by a 6 point Likert-type scale
1 = Strongly Disagree 6 = Strongly Agree
First Research Question
Do students at community colleges and universities differ on the eight advising learning outcomes?
– ANCOVA– Controlling for age
Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)
1=strongly disagree 6=strongly agree***p<.001
Advising Learning Outcome Community College Studentsn = 7170
University Studentsn = 4831
I understand how my academic choices at name of institution connect to my career and life goals
5.10 (1.09)*** 4.81 (1.14)
Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)
1=strongly disagree 6=strongly agree***p<.001
Advising Learning Outcome Community College Students
n = 7170
University Studentsn = 4831
When I have a problem, I know where at name of institution I can go to get help
4.44 (1.48)*** 4.27 (1.37)
Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)
1=strongly disagree 6=strongly agree***p<.001
Advising Learning Outcome Community College Students
n = 7170
University Studentsn = 4831
I know what requirements I must fulfill in order to meet my educational goals / earn my degree
4.81 (1.32) 4.76 (1.21)
Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)
1=strongly disagree 6=strongly agree***p<.001
Advising Learning Outcome Community College Students
n = 7170
University Studentsn = 4831
I understand how things work at name of institution
4.65 (1.29)*** 4.28 (1.24)
Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)
1=strongly disagree 6=strongly agree***p<.001
Advising Learning Outcome Community College Students
n = 7170
University Studentsn = 4831
I have a plan to achieve my educational goals
5.43 (0.93)*** 5.29 (0.95)
Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)
1=strongly disagree 6=strongly agree***p<.001
Advising Learning Outcome Community College Students
n = 7170
University Studentsn = 4831
I have had at least one relationship with a faculty or staff member at name of institution that has had a significant and positive influence on me
4.30 (1.67)*** 4.10 (1.54)
Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)
1=strongly disagree 6=strongly agree***p<.001
Advising Learning Outcome Community College Students
n = 7170
University Studentsn = 4831
It is important to develop an advisor-advisee relationship with someone on campus
4.84 (1.26) 5.07 (1.05)***
There should be mandatory academic advising for students
4.16 (1.64) 4.43 (1.44)***
Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)
1=strongly disagree 6=strongly agree***p<.001
Advising Learning Outcome Community College Students
n = 7170
University Studentsn = 4831
It is important to develop an advisor-advisee relationship with someone on campus
4.84 (1.26) 5.07 (1.05)***
There should be mandatory academic advising for students
4.16 (1.64) 4.43 (1.44)***
I have had at least one relationship with a faculty or staff member . . .
4.30 (1.67) 4.10 (1.54)
Mean Ratings on Advising Learning Outcomes (Standard Deviations in Parentheses)
1=strongly disagree 6=strongly agree***p<.001
Advising Learning Outcome Community College Studentsn = 7170
University Studentsn = 4831
Understands Connections 5.10 (1.09)*** 4.81 (1.14)
Knows Resources 4.44 (1.48)*** 4.27 (1.37)
Knows Requirements 4.81 (1.32) 4.76 (1.21)
Understands How Things Work 4.65 (1.29)*** 4.28 (1.24)
Has Educational Plan5.43 (0.93)*** 5.29 (0.95)
Has Significant Relationship4.30 (1.67) 4.10 (1.54)
Values Advisor-Advisee
Relationship
4.84 (1.26) 5.07 (1.05)***
Supports Mandatory Advising4.16 (1.64) 4.43 (1.44)***
Second Research Question
Do students at community colleges and universities agree or disagree on the parts of the advising curriculum that are more difficult?
– Within subjects ANOVA– Compared the results of the two groups
Do Students Report More Learning on Some Outcomes than Others?
1=strongly disagree 6=strongly agree
Advising Learning Outcome Community College Studentsn = 7170
University Studentsn = 4831
Understands Connections 5.10b 4.81c
Knows Resources 4.44e 4.27e
Knows Requirements 4.81c 4.76c
Understands How Things Work 4.64d 4.28e
Has Educational Plan5.43a 5.29a
Has Significant Relationship4.30f 4.10f
Values Advisor-Advisee
Relationship
4.84c 5.07b
Supports Mandatory Advising4.16g 4.43d
Summary of Findings
Students at both types of institutions report extensive advising learning
But student learning varies as a function of educational environment
Summary of Findings
Community college students are more likely to – Understand connections– Know resources – Understand how things work– Have an educational plan
Summary of Findings
University students are more likely to – Value the advisor/advisee relationship– Support mandatory advising
Summary of Findings
Some aspects of the advising curriculum are easier for students to master, evidenced by higher scores on these outcomes
Other aspects are more difficult For the most part, these patterns were
the same for the two groups of students
Summary of Findings Community college and university
students
– Most likely to report they had an educational plan
– Least likely to report they had a significant relationship
Only exception: Supports mandatory advising
Implications for Practice
Universities need to strengthen their delivery of the advising curriculum
Implications for Practice Both types of institutions need to make
concerted efforts to ensure that– Students know where to go to get help with
problems– Students are hooked up to a caring and helpful
person at the institution– Students know how things work at their
institution with regard to timelines, policies, and procedures
Next Steps: Research on Advising Learning
Is advising learning associated with whether and how often students receive advising?
Is advising learning associated with where students get their information about classes to take to meet requirements?
Does advising learning predict retention?
Next Steps: Assessment of Advising Learning
Electronic advising portfolio– Facilitated by the advisor – Maintained by the student
Reflection scaffolding: Shared responsibility in action
Chronicles advising history Evolves as the student learns and develops
Assessment of Advising Learning:Advising e-Portfolio
Includes exercises and activities – designed by the advisor– completed by the student– aided by embedded links to resources and
tools Promotes advising learning Provides opportunities for students to
document their learning
Assessment of Advising Learning:Advising e-Portfolio
Advising learning involvesSelf-assessment Identifying and connecting academic, career, and life goalsDeveloping plans to achieve goals
Assessment of Advising Learning:Advising e-Portfolio
Challenges to implementation and useTime
– Students– Advisors
Who assesses the portfolio?How is it assessed?
Discussion
Questions
Comments
Implications