direct assessment methods - university of arizona assessment methods ... design of the education ......
TRANSCRIPT
Direct Assessment Methods Measuring UA Student Employee Experiences
Lucas SchalewskiRebecca ClaasenMegan Forecki
Student Employee Development & Appreciation Committee
Student Employment as a Promising High-impact Practice
• 80% of students work while enrolled in college • Higher-persistence rates, increased sense of belonging,
and peer relationships• Considered a valuable practice employers look for • Most students understand work as experiential
How do we assess student employee learning and development?
Citi & Seventeen Magazine. (2013). College student pulse.King, J. E. (2002). Crucial choices: How students’ financial decisions affect their academic success. American Council on EducationKuh, G.D. (2008). High-impact educational practices. AAC&U.Cheng, D. X. (2003). How families of low-and middle-income undergraduates pay for college: Full-time dependent students in 1999-00.
Identify Goals & Objectives
Outcomes
Design Assessment
Method
Collect & Analyze Data
Communicate Findings
Refine & Change
Direct Methods
Process employed to gather data which requires students to display student knowledge, behavior, or thought processes.
Direct evidence of student learning
Provides evidence of student learning
Indirect Methods
Process employed to gather data which asks students to reflect upon their knowledge, behaviors, or thought processes
Infers student learning
Provides evidence of students’ attitudes, perceptions, and experiences
Indirect Methods
Examples• Student satisfaction• Job placement rates• Focus groups• Retention rates• Exit interviews• Self-reported data
Advantages• Easy to administer• Clues to what should be
assessed directly• Great for values and beliefs
Challenges• Provide only impressions and
opinions, not hard evidence• May tell you what they think
you want to hear
Direct Methods
Examples• Pre and post-tests• Oral Presentation• Quizzes• Check-lists• Rubrics• Capstone project• Reflections
Advantages• Requires students to
demonstrate knowledge• Provides data that directly
measures outcomes• Supports formative assessment
Challenges• Time consuming• Students may not prefer this
method
SLO: Building managers will be able to explain all policies and procedures following training.
Indirect: Please rate your level of agreement with the following…I can explain the policies and procedures building managers implement when closing down the building.
Strongly agreeAgreeNeutralDisagreeStrongly Disagree
Direct:Summarize the polices and procedures building managers implement when closing down the building.
[Open text box]
Pre-post TestPlease state your level of agreement for the following:Strongly agree, Agree, Neutral, Disagree, Strongly Disagree
I prioritize the needs of thecommunity over my own.
I shop at places based on a company’s ethics.
What are the top 3 reasons you will participate in service in the future?
To understand social issuesPre-Test: 3
Post-test: 14
Because it makes me feel goodPre-test: 4Post-test:0
Communicate Findings
DenverCompare the healthcare
system in the US to a less
developed nation.
San Francisco What are the root causes of
hunger?:
TexasWhat are the benefits of
wildlife conservation,
and how do the centers benefit
their communities?
Las VegasHow can we improve the design of the
education system?
Open-ended questions:
Successes/Challenges
Office of Scholarships and Financial Aid
Content AssessmentHands-On Practice
Shadowing RubricSelf-Reflection
Supervisor PlacementExit Interview
Direct Assessment
0
10
20
30
40
50
60
70
80
90
100
Week 1 Week 2 Week 3 Week 4 Week 5
D2L Content Assessment
Attempt 1 Attempt 2
Content Assessment
Student Employee - Customer Service
SLO: Student desk representatives will be able to demonstrate accomplished customer service skills.
Advantages• Mixed-method and direct measures• Quantify behaviors, thoughts, and performances• Help students understand what is expected of them • Provide descriptive feedback to students for improvement• Promotes more accurate, unbiased, and consistent scoring
Rubrics
Student Employee Outcomes:Providing customer serviceSolving problemsActing professionally
Communicating Managing tasksInteracting with others
Summary & Recommendations
Summary• Student employment is a high-impact practice. • Direct and indirect assessments have strengths and
weaknesses, we need both• Rubrics are an accessible and useful tool to use for
direct assessment of student employees
Recommendations:• Have well-written outcomes for student employees• Use the best method(s) to assess outcomes• Involve students in the process, use the findings
• Seek out consultation from A&R if needed