how to create rubrics - university of north texas at dallas · student’s oral presentation...

HOW TO CREATE ANALYTIC RUBRICSELIZABETH O. GIDDENS, PH.D.

WHAT IS A RUBRIC?

A rubric is a scoring tool that lays out the specific expectations of an assignment or goal. Rubrics break up the

assignment/goal into smaller parts, and provide a detailed description of what constitutes acceptable and

unacceptable levels of performance for each part (Stevens and Levi, 2013).

WHAT DOES A RUBRIC LOOK LIKE?

An analytic rubric is a table/matrix of criteria and

their descriptors (UC Denver and UConn):

The left side of a rubric matrix lists

the dimensions. The dimensions are the

components/parts of an assignment or project.

Across the top of the rubric matrix is the rating

scale. The values could be “above” “meets” and

“below.” They also could be, and often are,

numerical (3, 2, 1).

Descriptors under the rating scale and to the

right of the dimensions provide written

descriptions for each level of performance.

1 Below 2 Meets 3 Above

Grammar The author makes

numerous errors in

grammar, mechanics,

and/or spelling, and it

interferes with the

reader’s understanding.

The author makes

some errors in

grammar, mechanics,

and/or spelling, but it

does not interfere with

the reader’s

understanding.

The author makes no

errors in grammar,

mechanics, or spelling.

Example of a dimension of a rubric about a paper

REMINDER OF WHO DOES WHAT TYPE OF ASSESSMENT…

Educational Assessment Student-Service Assessment Administrative Assessment

• Faculty • Experiential Learning

• Distance Learning

• Library

• Learning Commons

• Academic Advising

• Admissions and Orientation

• Financial Aid

• Leadership

Development/Multicultural

• Disability Services Office

• Career Services

• Veteran’s Success Center

• Counseling and Wellness

• Residence Life

• Police and Parking

• Development and Foundation

• Sponsored Projects

• Marketing and Communication

• President, Provost, and Deans

• Registrar

• Facilities

• Budget and Planning

• Financial Services/Accounting

• Strategic Analysis/Reporting

• Human Resources

• Risk Management

• Information Technology

• Athletics

WHY SHOULD YOU USE A RUBRIC?

As an instructor, rubrics can help you communicate your expectations to students and assess student work

consistency and efficiently. Rubrics can also provide students with informative feedback on their strengths and

weaknesses, and prompt students to reflect on their own work. They can also help you design your assignments

(because they give you a chance to really think about what you want from the students).

As a student-service staff member, rubrics can help you determine the effectiveness of your program and services.

They can also help you measure your alignment to CAS Standards (more on this later).

As a administrative staff member, rubrics can help you judge the quality of a product or project. They can also help

you prioritize your needs.

FOR EXAMPLE (FACULTY)

This rubric was created to assess a

student’s research paper (Carnegie

Mellon).

ANOTHER EXAMPLE (FACULTY)

This rubric was created to assess a

student’s oral presentation

(Northern College).

FOR EXAMPLE (STUDENT-SERVICE STAFF)

This rubric was created to assess a

student’s intercultural awareness,

perhaps after intensive programming

(UNC).

ANOTHER EXAMPLE (STUDENT-SERVICE STAFF)

This rubric was created to assess a

student’s career-planning skills and

study-success skills, perhaps after a

1:1 meeting (Paul Gordon Brown).

FOR EXAMPLE (ADMINISTRATIVE STAFF)

This rubric was created to

determine which software they

wanted to purchase and

implement (Western University).

ANOTHER EXAMPLE (ADMINISTRATIVE STAFF)

This rubric was created to

determine priorities for

deferred maintenance (MTU).

ANOTHER EXAMPLE (STUDENT-SERVICE STAFF) – CAS STANDARDS

This rubric was created to see if a

department aligned with the CAS

Standard about “Ethics” (CAS).

HOW TO MAKE A RUBRIC

1. Decide on a manageable number of important dimensions of the assignment/goal.

2. Decide how many performance levels you would like to include. Decide on the labels for each performance level. Examples:

Excellent, competent, needs work (3 levels)

Accomplished, proficient, developing, beginning (4 levels)

Excellent, good, adequate, needs work, incomplete (5 levels)

3. For each dimension, describe the highest performance. Be as specific as possible.

4. Define the remaining performance levels (“good”, “adequate”, “needs work”, etc.) for each dimension. Be as specific as possible. Describe each level as completely as you can. For example, make sure to specify how “good” is different from “adequate” or “excellent” (Laney).

5. Optional Step:

A. Using rubrics for grades. See video here starting at 4:03: https://www.youtube.com/watch?v=jSwbR1oDXLY

B. Adding weights to each dimension so some are scored more heavily than others toward the final grade. See article here: https://www.teachervision.com/how-weight-rubrics-part-four-five-part-series

LET’S DESIGN ONE FOR “MAKING BROWNIES”

I made brownies! To do this, I…

1. Washed hands to ensure hygiene

2. Gathered and measured ingredients

3. Followed recipe for collating ingredients,

including stirring and baking

4. Cut, presented, and served completed dish

5. Tasted the final product

See handout. These five steps became my dimensions; you

can choose different ones if you want. Also, I already

included a 1-3 rating scale. In groups, fill out the descriptors.

TIPS FOR WRITING DESCRIPTIONS…

No rubric is perfect, but let’s try to make it as

perfect as we can. Here’s how:

Don’t write too much. Don’t write too little.

Ensure each level is different from the other so a

student cannot overlap into two scores.

Avoid vague terms.

Be consistent. Use parallel sentence construction to

describe the same content across different

performance levels for each dimension. For example,

see to the right (Megan Oak Leaf).

USING PARALLEL SENTENCE STRUCTURE

PARALLEL EXAMPLE

NOT PARALLEL EXAMPLE

IN GROUPS, WORK ON YOUR RUBRIC FOR 10-15 MINUTES

IS THE RUBRIC YOU MADE ANY GOOD?

To the right is a quality control rubric that

can be used to judge the quality of the

rubric you made (Stevens and Levi, 2013)

– handout provided.

NO RUBRIC IS PERFECT, BUT THIS IS HOW WE CAN WORK

TOWARDS IT…

Reliability

Reliability is the extent to which an instrument (your rubric) yields consistent results when used repeatedly under the same conditions.

When two different graders use the reliable rubric on the same performance, they'll give similar scores. This is called inter-rater reliability.

Your rubric should also yield consistent results if one grader uses the same rubric to judge the same performance at different times. This is called intra-rater reliability. (UC Denver)

Validity

Content Validity refers to whether a content expert would agree with the criteria/dimensions chosen in the rubric. For example, if your rubric is assessing a student's ability to write a thematic essay, are your dimensions composed of the essential elements of a thematic essay?

Criterion Validity relates to the predictability of student performance. If a student gets a high score using your rubric as the measurement tool, can you also infer that the student will perform that task well on another similar test?

Face Validity refers to whether the rubric appears to be valid to its users. Students/others have a stake in the quality of your rubric, so it's important that the rubric seems right to them. (UC Denver)

SOURCES

Steven and Levi, 2013, “Introduction to Rubrics,” 2nd Edition

https://studentaffairs.unc.edu/faculty-staff/student-affairs-assessment/resources/survey-items-and-rubrics

http://www.northernc.on.ca/leid/docs/ja_createrubric.pdf

https://laney.edu/assessment/how-to-create-rubrics-and-examples/

http://www.ucdenver.edu/faculty_staff/faculty/center-for-faculty-development/Documents/Tutorials/Rubrics/1_what_is/index.htm

https://assessment.uconn.edu/wp-content/uploads/sites/1804/2016/06/How_to_Create_Rubrics.pdf

https://www.cmu.edu/teaching/assessment/assesslearning/rubrics.html

https://www.mtu.edu/facilities/pdfs/2011-mtu-facility-assessment-report-2011-10-13-update.pdf

https://www.cas.edu/Files/Presentations/CAS_Basics_082015.pdf

http://meganoakleaf.info/writingrubricsright.pdf

https://teaching.uwo.ca/pdf/elearning/Rubric-for-eLearning-Tool-Evaluation.pdf

https://www.youtube.com/watch?v=jSwbR1oDXLY

https://www.teachervision.com/how-weight-rubrics-part-four-five-part-series

https://www.thoughtco.com/essay-rubric-2081367

THANK YOU! QUESTIONS?