1
Tracking College Students Over Time: Tracking College Students Over Time: Using CIRP & CSS Data for Longitudinal Using CIRP & CSS Data for Longitudinal AssessmentAssessment
Victor Sáenz (UCLA) Victor Sáenz (UCLA) John Pryor (UCLA) & John Pryor (UCLA) & Gavin Henning (Univ. of New Hampshire)Gavin Henning (Univ. of New Hampshire)
Wednesday, May 17th, 2006Wednesday, May 17th, 2006AIR ForumAIR ForumChicago, ILChicago, IL
2
Presentation GoalsPresentation Goals
Overview of CIRP Freshman Survey
Introduce the CSS survey as a longitudinal follow-up to the CIRP Freshman Survey
• Discuss how institutions can utilize HERI longitudinal data to assess various aspects of the first-year experience
• Share descriptive findings from the 2005 CSS national aggregate data
• Institutional perspective on using HERI longitudinal data
3
Higher Education Research Institute
CIRPCooperative Institutional
Research Program
FreshmanSurvey YFCY CSS
Faculty Survey
Funded Research
•Atlantic Philanthropies•Templeton Foundation•National Institutes
of Health
4
CIRP, CSS, CIRP, CSS, & the I-E-O Model& the I-E-O Model
InputsCIRP Freshman Survey
(e.g., academic performancein high school, financial concerns
prior to college entry, expectations for college,
degree aspirations,self-concept in high school)
Environments CSS
(e.g., place of residence during college, interactions
with peers and faculty, curricularand co-curricular experiences)
OutcomesCSS
(e.g., satisfaction with college, retention, post-college plans)
5
Using CIRP & CSS Data to Enhance Campus Assessment Efforts: Methodologies
•Descriptive analyses with your campus data
•Comparative analyses
•Measures of association
•Longitudinal analyses
•Factor analyses
•Multivariate analyses
6
Using CIRP & CSS Data to Enhance Campus Assessment Efforts: Purposes
•Self-study reports
•Retention studies
•Recruitment issues
•Examining group differences among students
•Strategic planning
•Presentations to various constituents
•Creation of a student information system
•Measuring student development & institutional impact
7
CIRP Freshman SurveyCIRP Freshman Survey
Largest and longest-running national study of American college students…2006 is 40th Anniversary
Initiated in 1966 at the American Council on Education; Housed at HERI (UCLA) since 1973
Administered annually to over 400,000 incoming freshmen at more than 600 colleges and universities nationwide
Since 1966: 12 million students; 1,800 institutions
8
CIRPCIRPFreshman Survey ItemsFreshman Survey Items
Demographic Characteristics Expectations for College High School Experiences Degree Aspirations & Career Plans College Finances Attitudes, Values, & Life Goals Reasons for Attending College
FRESHMAN TRENDS (1966 – present)
9
CIRP FS Trends: CIRP FS Trends: Entering College FreshmenEntering College Freshmen
Value: Keeping Up to Date with Political Affairs
0
10
20
30
40
50
60
70
Perc
ent
"very
im
port
ant"
or
"ess
enti
al"
10
CIRP FS Trends: CIRP FS Trends: Entering College FreshmenEntering College Freshmen
Grade Inflation
0
10
20
30
40
50
Per
cent
A- orhigher
C+ orlower
11
CIRP FS Trends: CIRP FS Trends: Entering College FreshmenEntering College Freshmen
Contrasting Values
0
20
40
60
80
Per
cent
Develop a meaningful philosophy of life
Be very well off financially
12
CIRP FS Trends: CIRP FS Trends: Entering College FreshmenEntering College Freshmen
Reason for Attending College:"To Be Able To Make More Money"
0
10
20
30
40
50
60
70
80
1976 1979 1982 1985 1988 1991 1994 1997 2000 2003
Perc
ent
"very
im
port
ant"
Men
Women
13
CIRP Freshman TrendsCIRP Freshman Trends
Socio-historical context
Consistent trends over time
Changing student profile over the last 40 years (e.g, values, reasons for going to college, etc.)
What about students’ college experiences?
14
CSSCSS
HERI follow ups since 1967 Sponsored research
College Student Survey Launched in 1993 Allowed institutions to participate on their own timetable
Most use CSS as an exit survey for seniors
15
CSS Major ThemesCSS Major Themes
Academic and social adjustment Sources of satisfaction and dissatisfaction Academic, residential, and employment
experiences Plans for the next academic year Patterns of behavior Life goals Self-concept and feelings of personal success
16
2005 CSS*
*Data are unweighted.
116 Colleges and Universities 17,929 students
85% took the CIRP Freshman Survey in 2001
62% Female 38% Male
17
Results: 2005 CSS
Higher Satisfaction 4 out of 5 would make the same choice over
again to attend their college Class size (89.3%) Quality of Instruction (88.0%) Major courses (86.5%) Sense of community (76.9%) Ability to find faculty/staff mentor (67.5%)
18
Results: 2005 CSS
Higher Satisfaction Internet access (81.2%) Computer facilities (73.3%) Opportunities for Community Service (59.2%) Leadership opportunities (59.6%)
19
Results: 2005 CSS
Lower Satisfaction Student Housing (54.0%) Campus Health Services (43.7%) Job Placement (36.4%)
20
CSS (by racial group):Overall Satisfaction w/College Experience
Overall Satisfaction w/College Experience
77.1% 76.2% 74.0%68.1% 66.8% 64.8%
0%10%20%30%40%50%60%70%80%90%
100%
White AmericanIndian
Latina/o Other Race Asian/PacificIslander
AfricanAmerican
Perc
ent
dissatisfied/very dissatisfied
satisfied/very satisfied
21
Students felt successful in…
36%
42%
49%
52%
67%
69%
73%
0% 20% 40% 60% 80% 100%
Utilize campus resources
Get to know faculty
Manage time
Develop academic skills
Adjust to academic demads
Understand prof expectations
Develop close friends
23
Longitudinal Expectations/Reality: CIRP Freshman Survey and CSS
18%
46%
44%
28%
57%
61%
0% 20% 40% 60% 80%
Join Frat/Sor
Change Major
ChangeCareer
FreshmanCSS
24
Longitudinal Expectations/Reality: CIRP Freshman Survey and CSS
21%
95%
33%
97%
0% 20% 40% 60% 80% 100% 120%
SeekPersonal
Counseling
Socializeanother race
FreshmanCSS
25
Longitudinal Results: CIRP Freshman Survey & CSS
Change in Students' Self-Concept
Percent rating themselves "above average" or "highest 10% relative to their peers : At college entry At the end of college Change
Computer skills 30.5 44.7 14.2
Self-understanding 56.5 67.2 10.7
Writing ability 52.7 62.7 10.0
Self-confidence (social) 48.2 57.4 9.2
Public speaking ability 40.2 48.9 8.7
Self-confidence (intellectual) 62.5 67.9 5.4
Leadership ability 64.0 67.1 3.1
Understanding of others 68.6 71.6 3.0
Popularity 39.5 42.0 2.5
Artistic ability 28.9 31.0 2.1
Creativity 55.2 57.0 1.8
Competitiveness 57.6 59.1 1.5
Academic ability 79.1 79.9 0.8
26
Longitudinal Results: CIRP Freshman Survey & CSS
Change in Students' Self-Concept
Percent rating themselves "above average" or "highest 10% relative to their peers : At college entry At the end of college Change
Drive to achieve 78.5 78.3 -0.2
Emotional health 57.4 56.9 -0.5
Spirituality 44.3 39.9 -4.4
Physical health 59.6 53.7 -5.9
Religiousness 39.3 32.0 -7.3
Mathematical ability 51.7 40.2 -11.5
28
Tradition of Using CIRP Freshman Survey at UNH
UNH was pilot school in 1966 Data for 16 years going back to 1966 Have administered biennially since 1999 Have used data to better understand
incoming students Haven’t been using data for any type of
evaluation or outcomes assessmento UNH is lacking specific outcomes
29
College Student Survey at UNH
In 2001 UNH began to look more at student outcomeso Focus on learning, not just satisfaction and
general programmatic outcomes such as attendance
Viewed CSS as a tool for that assessment
30
College Student Survey at UNH
Compared CSS to NSSE and chose CIRP/CSSo High response rate for CIRPo Control of pre-college factors with CIRPo Great deal of question overlap between CIRP/CSS and
NSSEo CIRP/CSS was cheaper for UNHo Ability to pre-/post-test with same panel of students, not
just sample from same cohorts Need SSNs
o 30 extra questions on CIRP/CSS to customize for our campus
31
College Student Survey at UNH
CSS administration challengeso Inadequate mail/email addresses in 2001o Tried to have departments administer in capstone
experiences and other activities where seniors were together Only three departments participated
o Lack of intentional purpose for using data once collected UNH academic plan is general not specific enough to
evaluate against We are in the process of identifying institutional student
outcomes
32
College Student Survey at UNH
CSS administration spring 2005o Pilot test with 789 students who took CIRP in
2001 and that provided SSNs for linkageo Able to reach 763 of 789 students from this
sampling frame 97% coverage rate
o 278 of 763 students responded 36% response rate
33
Using CIRP/CSS Data at UNH
Understanding student characteristics Needs assessment Satisfaction survey of various campus services Outcomes assessment Benchmarking Program evaluation Strategic planning
34
Using CIRP/CSS Data at UNH
Understanding student characteristicso How they spent their time, behaviors, values,
etc. and how this has changed over their college career CIRP 2006: 27, 32, 33, 36, 38 CSS 2006: 7, 8, 13, 15, 21
o Educational activities engaged in at college CSS 2006: 7, 9
35
Using CIRP/CSS Data at UNH
Needs assessmento Areas of self-identified weaknesso CIRP item 28, 39
Satisfaction survey of various campus serviceso CSS 2006: 10, 18
36
Using CIRP/CSS Data at UNH
Outcomes assessmento Achievement of UNH student outcomes
Can use extra 30 questions Can investigate relationships between variables to
identify possible predictors for outcomes while controlling for pre-college inputs
o Examining how students’ perceived skill level/traits has changed during college career CIRP 2006: 31/CSS 2006: 23 CSS 2006: 19
37
Using CIRP/CSS Data at UNH
Outcomes assessment (continued)o Understanding the perceived impact of faculty
CSS 2006: 24
38
Using CIRP/CSS Data at UNH
Benchmarking against other institutions on any variety of variables
Program evaluationo Can compare students in certain programs
based on CSS variables, or change between CIRP and CSS on particular variables Academic program Co-curricular program
39
Using CIRP/CSS Data at UNH
Strategic planning Helping to assess selected goals and
strategies of the strategic plan Increase opportunities for student engagement
CSS 2006: 7, 9, 13, 19 Strengthen and facilitate sense of community
CSS 2006: 10, 18, 22, 24, 26
40
Using CIRP/CSS Data at UNH
Strategic planning (continued) Helping to assess selected goals and
strategies of the strategic plan Actively develop multicultural competence and
support diversity Establish and maintain a healthy/safe environment
for students and staff CIRP 2006: 26/CSS 2006: 15 CSS 2006: 26
41
Fostering use of CIRP/CSS at UNH
Commitment to assessment and evidence-based planning and decision-making
Concrete goals, strategies, and outcomes that can be evaluated
Marketing how the data can be used Putting the data into practice for assessment,
planning, and decision-making
42
For More Information:For More Information:
Victor Saenz [email protected]
John Pryor [email protected]
Gavin Henning [email protected]
http://www.gseis.ucla.edu/heri/heri.html
(310) 825-1925
http://www.gseis.ucla.edu/heri/cirp.htm
http://www.gseis.ucla.edu/heri/css.html