why grades don't 'make the grade': explaining how learning outcomes assessment can...

Why Grades Don’t ‘Make the Grade’

Explaining How Learning Outcomes Assessment Can Improve

Student Learning

Workshop ObjectivesBy the end of this workshop, participants will be able to:

– 1. Explain the differences and relationships between grades, grading, scoring, and learning outcomes assessment.

– 2. Discuss circumstances during which grades, or learning outcomes assessment data, could be used to demonstrate different facets of student achievement.

– 3. Articulate strategies for promoting learning outcomes assessment as a process for collecting specific diagnostic information for the purpose of improving student learning.

Institutional ProfileDel Mar College

– Founded in 1935– two campuses + two centers in Corpus Christi, TX– Enrolls 22,000 students annually– Hispanic-Serving Institution– 600 faculty (285 FT / 348 PT)– 596 staff – 3 nationally accredited fine arts programs: music, art,

drama– CTE make >$56K annually ($6K over state avg)– Retains 58.4% of students

Myth-conceptions

Agenda

Activity 1: Compare and Contrast

See Page 1 in Worksheets

Definition of “Grade”• Complex global evaluations that

represent the overall proficiency of students.

See Page 2 in Worksheets

Definition of Assessment

Assessment is the systematic and

ongoing method of gathering, analyzing

and using information from measured

outcomes to improve student learning.

Learning Outcomes Assessment

Three Main Components:1. Articulate the goals for student learning.2. Gather information about how well students

are achieving goals. 3. Use the information for improvement.

Student learning outcomesclearly state

the expected knowledge, skills, attitudes, competencies, and habits of mind that students are expected to acquire

when they complete: • a course• a program, or• a certificate or degree

Grades

Agenda

Three Reasons1. Grades Mean Different Things to Different

People2. Grades and Learning Outcomes

Assessment Have Different Purposes3. Grades and Learning Outcomes

Assessment Provide Different Information

Why can’t I just use grades to document assessment for my

course?

1. Grades Mean Different Things to Different People

ACTIVITY 2• Varying Components of a Course

Grade – THINK – PAIR – SHARE – COMPARE – What different components make up YOUR

students’ final course grade? – What different components make up the

final course grade for students in your colleague’s courses?

– Assignment grades? – Exam grades?

See Page 2 in Worksheets

Why exam, assignment, and/or course grades may not necessarily measure ability to meet a learning outcome:

• inconsistent evaluations within course, between courses, semesters, campuses;

• grades can depend on instructor-specific factors: weight for assignments, importance of additional topics;

• grades can depend on student behavior (missed or late assignments)

2. Grades and Learning Outcomes Assessment Have Different Purposes

Grading Grading serves multiple purposes:

• Evaluation• Communication• Motivation• Organization• Faculty & Student

Reflection(Walvoord & Anderson, 2010)

Photo Credit: Terenzini, P. T. (Nov/Dec 1989). Assessment With Open Eyes: Pitfalls in Studying Student Outcomes.” Journal of Higher Education, Vol. 60, No. 6, pp. 644-664.

Learning Outcomes Assessment

3. Grades and Learning Outcomes Assessment Provide Different Information

ACTIVITY 3: Exam & Assignment Grades vs. SLO Assessment

• Talk to your colleagues and discuss:– What trends do you notice?– What might you conclude about the

students’ grades versus meeting the student learning outcomes 1 and 2?

See Page 3 in Worksheets

Professionalism Rubric

ACTIVITY 3: Exam & Assignment Grades vs. SLO Assessment

 Skill/Outcome

Identifycomponents of professionalism

(Exam 1)15pt

Identifycomponents of professionalism

(HW1)20pt

Demonstrate Professionalis

m(Clinical 1)

10 pt

Demonstrate Professionalis

m(Clinical 2)

10 pt

TotalIdentify

components35 pt

Totaldemonstrate

professionalism20 pt

TotalScore/Grade55 pts

Std 1 10 15 6 9 25 15 40=C

Std 2 14 18 10 8 32 18 50=B

Std 3 14 17 5 5 31 10 41=C

Class Average 12.67 16.67 7 7.33 29.34 14.33

Professionalism SLOs: 1) Identify components of professionalism in the workplace. 2) Demonstrate professionalism in the workplace.

See Professionalism Rubric

“Aren’t Grades Enough?” Summary

Data from learning outcomes assessment CAN provide:

• specific information about students’ performance on discrete tasks

• meaningful data across course sections or time • objective student data which can be used for

improvement of student learning or recognition of student achievement

Example 2: SLO – Communication

SLO: Explain and defend ideas orally and in writing.

Grading Criteria(10 points each)

Student#1 #2 #3 #4 #5 #6 #7 #8 #9 #10 AVG

1. Clearly state the purpose. 9 8 7 9 7 9 10 8 9 9 8.3

2. Clearly understand the audiences’ values, attitudes, goals, and needs. 7 8 7 7 5 6 7 6 7 9 6.9

3. Evaluates evidence (data) for accuracy and relevance. 8 9 8 9 6 7 8 7 8 9 7.9

4. Use vocabulary appropriate to the subject and purpose(s) 7 8 9 9 5 9 8 7 8 10 7.6

5. Use correct reference forms. 7 6 7 6 5 6 7 8 9 9 6.7

6. Use correct grammar, syntax (word order), punctuation, and spelling. 8 9 7 8 6 7 8 9 8 10 8

7. Present accurate information. 9 7 8 9 7 6 8 9 9 8 8

8. Develop patterns or organization for ideas. 6 7 8 9 7 8 9 7 9 9 7.9

9. Identify strengths and weaknesses of an argument, situation, or action. 7 6 7 9 5 7 7 8 7 9 7

10. Summarize the main ideas clearly. 8 7 8 9 7 6 7 8 10 10 7.9

TOTAL GRADE: 76 75 76 84 60 71 79 77 84 92

Agenda

Linking Teaching, Learning, Grading, & Life

• Authentic Assessment• Test Blueprinting• All-In-One Grading Rubrics

"...Engaging and worthy problems or questions of importance, in which students must use knowledge to fashion performances effectively and creatively. The tasks are either replicas of or analogous to the kinds of problems faced by adult citizens and consumers or professionals in the field."

-Grant Wiggins

What’s “authentic” about authentic assessment?

How Does Authentic Assessment Differ?

Traditional -------------------------------------- Authentic

Selecting a Response ------------------------ Performing a Task

Contrived (for the purposes of assessment) --------------------- Real-life

Recall/Recognition -------------------- Construction/Application

Teacher-centered-------------------------------Learner-centered

Indirect Evidence ------------------------------- Direct Evidence

Test Blueprinting

• An outline of the test that lists the learning goals that students are to demonstrate (Suskie, 2009)

• The process of linking tests to learning goals (Walvoord & Anderson, 2010)

Are you testing what you think is most important?

All-In-One Rubrics

https://www.web.virginia.edu/iaas/assess/tools/rubrics.shtm

excerpt

NOW – How would you answer these questions?

Final Tips & Take-Aways• be patient...remember most higher education

faculty are content experts, not education or assessment experts

• use the term “learning outcomes assessment” instead of “assessment”

• show them, don’t tell them– use concrete examples and exercises

• encourage the use of rubrics to link grades & LO assessment

• encourage authentic assessment• use test blue-printing for exams • repetition, repetition, repetition• be patient

Contact Information

Jean Downs

Director of Assessment

Del Mar College-Corpus

Christi, TX

jdowns2@Delmar.edu

Lucy JamesFaculty Coordinator of AssessmentAssociate Professor of ASL & InterpretingDel Mar College–Corpus Christi, TXljames@Delmar.edu

• Walvoord, B. E. (2010). Assessment Clear & Simple: A Practical Guide for Institutions, Departments, and General Education (2nd ed). San Francisco: Jossey-Bass.

• Carnegie Mellon University (n.d.). Grading vs. Assessment of Learning Outcomes: What’s the difference? LINK

• Downs, J. (2014). Training: Designing Effective Methods to Measure a Student Learning Outcome in Your Course. Created for ED5083 - Processes in Instructional Design. http://jeandowns.weebly.com/

References & Resources

• National Institute of Learning Outcomes Assessment (NILOA): – LINK

• Introduction to Rubrics – DePaul Teaching Commons – LINK

• Creating Rubrics – a five-part tutorial on rubric creation and implementation: LINK

• American Association of Colleges & Universities (AAC&U) VALUE Rubrics – LINK

• Richman, W.A., & Ariovich, L. (2013). All-in-one: Combining Grading, Course, Program, and General Education Outcomes Assessment. (Occasional Paper No.19). Urbana, IL: University of Illinois and Indiana University, NILOA LINK

References & Resources

• Bengiamin, N. N., & Leimer, C. (2012). SLO-Based Grading Makes Assessment an Integral Part of Teaching. Assessment Update, 24(5), 1-16.

• Walvoord, B. E. & Anderson, V. J. (2010). Effective Grading: A Tool for Learning and Assessment in College / Edition 2

• Stallbaumer-Beishline, L. M. (2012). Outcomes Assessment Essentials: Test Blueprinting, A Course-Embedded Tool. LINK

• Terenzini, P. T. (Nov/Dec 1989). Assessment With Open Eyes: Pitfalls in Studying Student Outcomes. Journal of Higher Education, Vol. 60, No. 6, pp. 644-664.

References & Resources