SURVEYS, OBSERVATIONS, AND RUBRICS OH MY! ASSESSING CAREER SERVICES

Jessica M. Turosjmturos@bgsu.edu Bowling Green State UniversityCareer Center

OUTLINEBrief review of assessment/evaluation literature and the assessment cycle

Discussion of assessment practices used including successes and challenges

Ways we can use assessment to inform our work

Question and Answer

LEARNING OUTCOMES

Decipher pertinent assessment/evaluation literature

Identify the assessment cycle and potential successes and challenges

Determine quantitative and qualitative methods used to conduct assessment/evaluation

LEARNING OUTCOMES CONT.Recognize potential assessment practices used in career services

Distinguish pros and cons of each method of assessment/evaluation

Name potential assessment/evaluation practices and methods

ASSESSMENT/EVALUATIONGathering information – measurement

Using information for individual and institutional improvement – evaluation

Student learning – assessment (Astin, 1993)

ASSESSMENT/EVALUATIONAssessment - “any effort to gather, analyze, and interpret evidence which describes institutional, divisional, or agency effectiveness” (Schuh, Upcraft, & Associates,

2001)Comprehensive processCulture of assessment

ASSESSMENT LITERATURE Accountability

SummativeJustification Satisfaction

Improvement FormativeLearning

Accountability and improvement are not mutually exclusive

AUTHENTIC ASSESSMENT Involves real-world information and

skillsUses critical thinking skills to

explore unstructured problemsPerformed in situations as similar to

the context as possibleAllows for feedback, practice, and

second chancesEmpowers students

TYPES OF AUTHENTIC ASSESSMENTReal-world projectsSimulated environmentCase studiesService-learning projectsPerformancesEvaluation based on observation of

workPortfolios

ASSESSMENT CYCLE (SUSKIE, 2004)

ASSESSMENT METHODS

IndirectQuestionnaireEvaluationsSurveysInterviews Focus groups

DirectExamsPerformance

measuresProjectsPresentations

Case studiesPortfolios

ASSESSMENT METHODSQuantitative – learning and

satisfaction EvaluationsSurveysQuestionnairesTestsRubrics

ASSESSMENT METHODSQualitative – learning and

satisfaction ObservationsInterviewsFocus groupsRubricsPortfoliosMixed Methods – learning and

satisfaction

ASSESSMENT PRACTICES USEDSurveys/Evaluations

Paper versus electronicResponse rateData entry

Examples Feedback formJob and Internship Fair

Student and EmployerWorkshops and TrainingsCourses

ASSESSMENT PRACTICES USEDObservations

Formal versus informalParticipation/Buy-inCollaboration

Examples Job and Internship Fair Observation

FormStaff and Students

Student appointments Case Consultation Meetings

ASSESSMENT PRACTICES USEDRubrics

Usage – staff and students Portfolios

ProductProcess

ExamplesMinimum Resume RequirementsResume RubricsTeacher Portfolio ReviewE-Portfolios

ASSESSMENT PRACTICES USEDFocus groups

ParticipationTiming

Interviews ParticipationTiming

ExamplesWorkNet Usability Perfect Interview Learning

WAYS ASSESSMENT CAN INFORM OUR WORKCase Studies Discussion Case 1 – Your office is implementing a

workshop series and you want to know if it is effective in terms of students learning key concepts and students feeling engaged with the material. How do you assess the series?

Case 2 – Your office is interested in learning the impact of your services (drop-in critiques and individual meetings) and resources (web content and printed resources) on students’ resumes. How do you assess the impact?

WAYS ASSESSMENT CAN INFORM OUR WORKExamples from offices

Successes Challenges Suggestions

QUESTION AND ANSWER

REFERENCES American College Personnel Association. (1996). The

student learning imperative: Implications for student affairs. Retrieved July 10, 2009

from http://www.acpa.nche.edu/sli/sli.htm  Astin, A. W. (1993). Assessment for excellence: The

philosophy and practice of assessment andevaluation in higher education. New York:

American Council on Education and the Oryx Press. Burley, H., & Price, M. (2003). What works with

authentic assessment. Educational Horizons, 81(4), 193 – 196.

Gulikers, J., Bastiaens, T., & Kirschner, P. (2006). Authentic assessment, student and teacher perceptions: The practical values of the five-dimensional framework. Journal of Vocational Education and Training, 58(3), 337-357.

REFERENCES CONTINUED Keeling, R. P. (Ed.). (2004). Learning reconsidered: A

campus-wide focus on the student experience. The National Association of Student Personnel Administrators and the American College Personnel Association.

La Lopa, J. M. (2005). Developing a student-based evaluation tool for authentic assessment. New Directions for Teaching and Learning, 100, 31-36.

Lombardi, J. (2008). To portfolio or not portfolio: Helpful or hyped? College Teaching, 56(1), 7-10.

 Palomba, C. A., & Banta, T. W. (1999). Assessment essentials: Planning, implementing, and improving assessment in higher education. San Francisco: Jossey-

Bass. Schuh, J. H., Upcraft, M. L., & Associates (2001).

Assessment practice in student affairs: An applications manual. San Francisco: Jossey-Bass.

REFERENCES CONTINUED

 Shavelson, R. J. (2007). A brief history of student learning assessment: How we got where we are and a proposal for where to go next. Washington, DC:

Association of American Colleges and Universities.

Suskie, L. (2004). Assessing student learning: A common sense guide. Bolton, MA: Anker.

Svinicki, M. D. (2005). Authentic assessment: Testing in reality. New Directions for Teaching and Learning, 100, 23-29.

 U.S. Department of Education. (2006). A test of leadership: Charting the future of U.S. higher education. A report of the commission appointed by Secretary of Education M. Spellings. Washington, D.C. Pre- Publication copy, September 2006. Retrieved from: http://www.ed.gov/about/bdscomm/list/hiedfuture/reports/pre-pub-report.pdf