grading: assessment techniques to enhance learninggrading or marking systems there are many systems...

Grading:

Assessment Techniques to Enhance Learning KURT F. GEISINGER, PH.D.

AUGUST, 2015, KING FAHD UNIVERSITY

My History in Grading My own history studying grading…as a psychometrician

◦ Published several studies on grade inflation in late 1970s and early 1980s

◦ Developed scales that measure faculty members orientations to grading

◦ Wrote monograph that became “Marking Systems” entry in the Encyclopedia of Educational Research

My work as a dean and vice president for academic affairs/ provost where I worry far more about student learning

The Essence of Grading Grades matter!

◦ To Students

◦ To Programs

◦ To the institutions

Grades are specific to given courses

Students take grades very seriously

Faculty do too, although grading is an onerous task

Grading is deeply embedded in higher education

Grading or Marking Systems There are many systems in place:

◦ Letter grades are most common (A-F is most typical)

◦ Numerical grades also exist in some setting (1-100) ◦ Sometimes schools align numerical grades with letter grades

◦ PASS-FAIL grading also exists ◦ At some schools, all courses are pass-fail

◦ Most schools offer limited numbers of pass-fail classes

◦ A few schools only offer descriptive grading; at one school I know, every student in every class gets a brief letter written to him/her telling him/her how they have done in the class, what skills they have shown, and so forth. It is a small liberal arts school with very small classes.

Grading at King Fahd University

Letter Grades Points Descriptive Grades

A+ 4.00 Exceptional

A 3.75 Excellent

B+ 3.50 Superior

B 3.00 Very Good

C+ 2.50 Above Average

C 2.00 Good

D+ 1.50 High-Pass

D 1.00 Pass

F 0.00 Fail

IP - In Progress

IC - Incomplete

DN 0.00 Denial

NP - No grade-Pass

NF - No grade-Fail

W - Withdrawn

WP - Withdrawn with Pass

WF 0.00 Withdrawn with Fail

AU - Audit

Purposes of Grading To provide information to students about their performance

Administrative institutional purposes

Sorting and selecting students

Motivating students

Research tools

Preparation for life

General Principles

Context ◦ Consider what you plan to do with data collected, scores or grades?

◦ Classroom Assessment

◦ Formative Assessment

◦ Summative Assessment

◦ Qualitative vs Quantitative

◦ Curricular representation ◦ Achievement & Ability

◦ Preassigned weights or proportional values

Assessments ◦ Plan ahead & be clear and specific

◦ Create Test Blueprints

◦ Tasks / item responses will generate relevant evidence?

◦ Make items that are construct relevant ◦ Create scoring rubrics

Assigning Grades ◦ Qualitative Approach ◦ Quantitative Approaches ◦ Defensibility, Validity & Fairness

Context

◦ Consider Content ◦ Unit

◦ Curriculum / Course Outline

◦ Consider what you plan to do with data collected, scores or grades?

◦ Classroom Assessment

◦ Formative Assessment

◦ Summative Assessment

◦ Qualitative vs Quantitative

◦ Curricular representation ◦ Achievement & Ability

◦ Preassigned weights or proportional values

What is the main purpose of assessment in relation to curricula?

• End of program or exit criteria

• End of semester

• End of unit

• In progress

Clarify: ◦ Who will write, administer, and score each assessment?

◦ What decisions will be made using the results (high/low stakes)?

◦ When will the assessment be administered? (timing & frequency)

◦ Where will the test be administered? (environment and resources)

◦ Why focus on various levels of learning?

◦ How will the scores be reported to the students?

What Kinds of Things Should be Included in Grades?

Affective values? ◦ Students’ interest in the course

Classroom behavior? ◦ Speaking up in class

◦ Participating actively in class, seeming interested

Attendance?

Student growth—how much an individual student learns in a class relative to what they knew at the beginning of the course

Student learning!

Grade Inflation One big concern that began in the 1970s

Average grades at institutions increased on average from about 2.20-2.30 (on a four-point scale) to 2.70-2.80

Popular press and some higher education theorists argued that this probably happened due to the US military draft and the Vietnam War

◦ Limited evidence to support this hypothesis

Changes in grading policies have much to do with this (non-punitive grading standards)

◦ Pass-fail grading

◦ The ability to drop courses late in a semester

Changes in the approach/attitudes of faculty members

For Grading to Be Meaningful Faculty must identify the most meaningful aspects of learning in a class

Faculty must assign weights to different components of grading that represent the learning in a class

◦ E.G., if tests and projects are both included, then they should be weighted in a manner consistent with their importance in terms of student learning

Faculty must construct tests and assignments that assess learning in accordance with the more meaningful aspects of learning

Faculty should set reasonable standards

Faculty should use assignments to guide student learning

Grading as a Motivator Without question, grades affect student motivation

◦ “What’s going to be on the test?”

◦ “Will this be on the test?”

In psychology, we talk about Intrinsic and Extrinsic Motivation

It’s sometimes hard as a faculty member to try to motivate students in terms of their intrinsic motivation

It is also easy to fall into the pattern of telling students or implying to students what is likely to be on the test

Assessment as Part of Grading In this age of accountability, accrediting bodies insist on outcomes assessment. Requirements include:

◦ Assessing student learning with both direct and indirect measures. ◦ An indirect measure might be an evaluation of a graduate’s work performance.

◦ Connecting assessments to the mission, goals, and objectives of the program.

◦ Attending to reliability and validity of measures.

◦ Demonstrating widespread faculty involvement

◦ Using assessment information for improvements.

◦ Integrating assessment and planning and budgeting.

Intrinsic Motivation Intrinsic motivation refers to motivation that is driven by an interest or enjoyment in the task itself, and exists within the individual rather than relying on external pressures or a desire for reward.

Intrinsic motivation is a natural motivational tendency and is a critical element in cognitive, social, and physical development. Students who are intrinsically motivated are more likely to engage in the task willingly as well as work to improve their skills, which will increase their capabilities.

Students are likely to be intrinsically motivated if they: ◦ attribute their educational results to factors under their own control, also known as

autonomy

◦ believe they have the skills to be effective agents in reaching their desired goals, also known as self-efficacy beliefs

◦ are interested in mastering a topic, not just in achieving good grades

Extrinsic Motivation Extrinsic motivation refers to the performance of an activity in order to attain an outcome, whether or not that activity is also intrinsically motivated.

Extrinsic motivation comes from outside of the individual. ◦ Rewards (for example, money or grades) for showing the desired behavior

◦ Punishment following misbehavior.

◦ Competition, because it encourages the performer to win and to beat others.

◦ Praise and recognition, a cheering crowd, or the desire to win a trophy are also extrinsic incentives.

Are grades important to everyone?

Classroom Assessment Techniques

Another presentation, but a big part of this one too

Keeping faculty knowledgeable about student learning through frequent, short, ungraded assessments in class

◦ One minute quiz where students describe the one thing that they did not understand well from that day’s class

In this presentation, I am more focused upon “graded” assignments, assignments where students perform and faculty members assign marks to their work.

Three False Hopes to be Dismissed Grading can be totally objective

Faculty can reach 100% consensus on grading

Grading is the ideal way to motivate students

Twelve Principles of Good Grading: 1/12

1. Appreciate the complexity of grading; Use it to help students learn

◦ Survey by Milton, Pollio and Eison demonstrated that among 6,000 students and faculty, there were many different perceptions of grading and of the roles of grading

◦ Students perceptions of grading differed according to their previous experience, motivation and so on

◦ Evaluation is part of a system that is based on the beliefs of those who participate in the system

◦ We need a system that meets our needs

◦ There is no perfect evaluation system

◦ The four primary roles of grades: evaluation, communication, motivation, and organization


2. Substitute judgment for objectivity

There is no perfect standard

There is no way to assess all students with perfect reliability using any standard

◦ Reasons for questionable objectivity ◦ Even with objective question, one cannot ask them all

Job of faculty member: to render a judgment in as informed and professional a manner as possible

In a world where faculty do not wish to stress memory to the exclusion of critical thinking and problem solving, judgment is important to higher order thinking and learning


3. Use your grading time effectively

Make the best judgment you can and then move on

You have other work to do.


4. Be open to change

Grade inflation happened in the United States.

At one time a “C” was an average grade and GPA averages throughout the United States were about 2.20

Today at most schools they are about 2.70

At some schools, including Stanford University, they are about 3.50

Imagine moving to another university that has a vastly different approach to grading


5. Listen and Observe ◦ Dressel’s advice to young faculty members in Handbook of Academic Evaluation

It is the meaning that students attach to grades that most affects student learning

Observe students and listen to them

Grades are not part of the laws of the Universe

Rather, they carry specific meanings to specific individuals

If we are trying to use grades to improve student learning, we should listen to students and hear how they interpret various grades


6. Communicate and collaborate with students

Grading can be antagonistic. No one wants that.

The fair and helpful evaluation of students is what students both seek and need.

Student-faculty relationships and student-peer relationships are critically important for students.

Try to find ways to collaborate with students on goals.

Seek to get students actively involved in the learning process.

Ray Bucko’s approach to having students evaluate the work of each other as a learning activity.

Students should know on the syllabus exactly how they will be evaluated, with rubrics and percentages, if possible


7. Integrate Grading with Other Key Processes

Grading should be integrated with teaching

Go over tests so that students understand what they did not learn effectively initially

Have testing and other evaluative components of your class integrated with planning, teaching, and other class activities


8. S e i z e t h e t e a c h a b l e m o m e n t

Informal feedback can be tremendously important to students

Remember how influential some of your own faculty were to you!

Think about the values what you wish to communicate to students and try to embody them

When students are emotional about the grades they have received or your evaluation of them, try to stay calm and determine what you wish to communicate to them

What about plagiarism?


9. Make Student Learning the Primary Goal

Grades are sometimes important for external reasons: potential employers wish to see the grades, scholarship and graduate admissions committees want to see them, and so on.

Nevertheless, directing student learning should be the primary use of grades and grading

Grading can influence learning if learning is the primary goal and grading is used to enhance it

If grades are the goal rather than learning, there is less of a chance that learning will be enhanced, and intrinsic motivation certainly will not be


10. Be a teacher first and a grader second

Grades serve as part of the gatekeeper function in higher education

Sorting should occur only after every student has had an equal opportunity to learn material

We do serve as gatekeepers, but only at the end of the educational process; we are teachers until that time

Throughout a semester, we should remain focused upon helping students to learn, determining what they need to learn, and advising them


11. Encourage learner-centered motivation

If one holds grades over students’ heads, that discourages student-centered learning and motivation

Students can become more self-motivated if it is due to their engagement with the instructor in the learning process

We should encourage students that hard work does have positive affects and that they can largely control their own futures

Students’ attitudes toward grades can negatively affect their motivation to learn


12. Emphasize Student Involvement

Whatever we do, we should attempt to seek students to invest their time and energy in the academic process

Astin’s Conditions for Excellence in Grading

Astin, summarizing 100s of studies of undergraduate grading and learning cited 3 conditions of excellence in grading. The three are:

◦ Student involvement in learning (most important)—relating to the amount of time and energy students’ invest

◦ High expectations by the faculty member

◦ Assessment and feedback—students need to improve and the only way that they can do so is if they get feedback—think about the comments that you make on student papers and examinations

Six Suggestions for Making Assignments Part of the Grading Process

1. Begin by considering what you want students to learn

2. Select tests and assignments that both teach and test the learning you value most.

3. Construct a course outline that shows the nature and sequence of major tests and assignments.

4. Check that the tests and assignments fit your learning goals and are feasible in terms of workload.

5. Collaborate with your students to set and achieve goals.

6. Give students explicit directions for their assignments.

Considering What One Wants Students to Learn

Faculty need to be able to say, “At the end of this course, I want students to be able to …”

Verbs to use include “define, argue, solve, create”

Reduction in the need to recall/remember facts (in many cases)

A good exercise is to sit and do this for courses that you are presently teaching and list out these student learning outcomes

Selecting Assignments and Tests that Measure What the Instructor Values Most

This relates to using your time most effectively

Faculty report that they are most interested in teaching students to analyze, synthesize and think critically.

Reviews of assignments, however, indicate that they stress students to acquire knowledge, comprehend basic concepts and ideas and terms, and to apply such knowledge

Evaluate your own examinations to check what you are measuring

Even essay tests can simply assess students abilities to expound upon learned facts, theories and so on

Choose assignments that are interesting and provide students with the opportunity to learn new skills

Permit peer collaboration when it is appropriate

Constructing a Course Outline Start with a very basic skeleton of the course to be taught

Ask yourself if the workload is manageable

Identify assignments that cause students to use their higher-order thinking skills

Ask “What should students learn?” rather than “What content should I cover?”

Draft assignments to maximize the learning you desire your students to achieve

Check Tests and Assignments for Fit and Feasibility

Ask yourself two questions: ◦ Do my tests and assignments fit the kind of learning I want for students? (Fit)

◦ Is the workload I am planning for the students (and myself) reasonable and sustainable? (Feasibility)

For example, are the tests that you are building the kinds on which students can succeed if they stay up all night the night before the examination?

Can you maintain the workload for grading if your class goes from 15 students to 45?

Collaborating with Student to Set and Achieve Goals

Some faculty meet with students before finalizing their goals

They discuss with students what the students want from the class in terms of learning and skills

Even a discussion of these goals could be quite useful

Try to reach agreement

Have students write their own personal goals in the class

Sometimes this is difficult if courses are in a sequence

Making Assignments and Tests Clear to Students

Give the students ample opportunity to ask questions in advance of assignments and tests

Be as explicit as possible

If you are asking students to run an experiment, for example, ◦ Tell them exactly what they are asked to do, that is, how they should conduct

the experiment

◦ What analyses of the results should be

◦ What the resultant report should look like

Example of an Assignment to Design and Report on an Experiment

Your written report should demonstrate that you have formulated a hypothesis, designed a solid experiment, controlled variables, operationally defined terms, and interpreted data appropriately. You should also demonstrate that you understand the score and sequence of the scientific report format and the importance of quantification to scientific writing.

Workshop Activity: Assignments

Rubrics, Etc.

What obligations do we have at the undergraduate level for teaching communications skills?

Rubrics, Etc.

What obligations do we have at the undergraduate level for teaching communications skills? Plenty…

Rubrics, Etc.

Modern View: • Get students ready for grad school • Write APA style papers Reality Check: How many students will write APA papers after graduation?

Rubrics, Etc.

Traditional Writing Communication Modes: • Experimental Paper • Theoretical Review • Posters • Reaction Papers • Journal Entries

Rubrics, Etc.

Professional Writing Communication Modes: • Case Notes • Grant Proposals • Business Memo/Letters • Project Reports • Web Sites

Rubrics, Etc.

Speaking Communication Modes: • Formal Presentation • Brief Presentation • Debate • Panel Presentation • Paired Presentation • Interviewing

Rubrics, Etc.

SANITY-SAVING STRATEGY: The Common Rubric

Get department to converge on good writing quality

Simplify assessment tracking to 3 categories ◦ Strength

◦ Acceptable

◦ Needs development

CRITERIA

Clarity of purpose

Situational context

Argument organization

Conclusion quality

Language management

Quality of evidence

Disciplinary style

Delivery/format

AREA CRITERIA 2 1 0 PointContribution/Notes:

Fostering Motivation A first principle is student involvement

The more active student learning is, the better

Research by Milton, Pollio, and Eison categorized students as grade-oriented or learning-oriented

Some students also see themselves as helpless, and that their efforts will not pay off

Other students do feel that they have control of their lives, over what happens to them, and that they can earn success

Grade-Oriented Students I think that without regularly scheduled examinations I would not learn and remember much

I do not find studying at home to be interesting

I will withdraw from an interesting class rather than risk a low grade

I get irritated by students who ask questions that go beyond what we need to know on examinations

How can we best motivate such students?

Should we attempt to help them to become more learning-oriented?

Learning-Oriented Students I find the process of learning new material fun

I enjoy classes in which the instructor attempts to relate material to concerns beyond the classroom

I often discuss interesting material that I have learned in class with friends and family

I try to make time for outside reading despite the demanding coursework

Faculty are Mostly Learning-Oriented

To get to where we are, we tend to enjoy learning

Now, we read things in our field for curiosity sake

We often read or get into conversations on intellectual topics outside of our areas of expertise

In US, faculty are very likely to read newspapers…students not

Two best ways to motivate students emerge from academic faculty-student contact and student-peer contact

It helps to tell students that you know that they can succeed

If you can get students to read material before class, then you can get them to discuss the material in class, to argue it, to debate it, and the like

At Some Level, Does it Matter?

Establishing Criteria and Standards

If one has clear criteria:

It saves grading time

The process becomes fairer and more consistent

Helps students know what to expect

Help students get feedback about their learning and evaluate their own learning

Help professors teaching sequenced courses communicate with each other

Help peers provide feedback to each other

Can explain grading more efficiently and accurately

Example: scoring master’s comprehensive essay questions

Rubrics and Clear Criteria Having a rubric moves one from unstated criteria to highly explicit criteria

Moves from Norm-referenced grading to more of a Criterion-referenced perspective

Identify exactly what you wish to measure in any given test or assignment

Work Activity: Developing a Rubric

Example Rubric for a Science Report

5 ◦ Is appropriate in tone and structure

for a science journal

◦ Contains all necessary names

◦ Identifies the method of experimentation

◦ Allows reader to understand the design

4 ◦ Is appropriate in tone and structure

for a science journal

◦ Contains most descriptors

◦ May lack some names

◦ Identifies the function of the experimentation

◦ Suggests the design

Example Rubric for a Science Report

3 ◦ Identifies function and names but

does not allow the reader to anticipate the design

2 ◦ Identifies function or name, but

not both

◦ Lacks design information or is misleading

1 ◦ Is patterned after a non-scientific

discipline or missing

Notice that all of the descriptors begin with a verb, mostly active verbs

One can also override the scale score if one disagrees with it

What about Multiple-Choice Tests (Or Other Objectively Scored Tests)

Several methods

One method begins by categorizing items as to whether they assess higher-order thinking or not

Then one identifies the percentage of questions in the higher-order and lower-order questions one would expect students at various grade levels to get correct

Can also use certain students about whom one has an idea of their likely grade and see how they do

Or both of the above and then adjust

Why Develop a Rubric? Makes grading more consistent and fair

Saves time in the actual grading process

Helps to diagnosis student strengths and weaknesses

Permits teacher to suggest readings for students to deal with their weaknesses

Can track changes in students individually and classes over time

Makes it easier for graduate assistants to do the grading

Helps to make grading more consistent, professor-to-professor

Can help later with programmatic outcomes assessments

Grading Using a Rubric Can also assign grades using a rubric in a manner similar to the previous slide

List out the characteristics of an A+ student, an A, a B, and so on…

What Kind of Grader Are You? Norm-referenced grader

Criterion-referenced grader

Self-referenced grader

Each of these is considered a frame of reference

They have implications for the grades you assign and how high they are

There was probably a change in philosophy happening in the 1970s and 1980s

Are You Norm-Referenced? Grading “on the cure” is the best way to evaluate student performance.

It is an obligation of the university to grade students in a comparative manner to aid prospective employers in locating the best candidate for jobs.

A final examination on which almost all students score quite high is probably too easy, no matter how well the students know the material.

Are You Criterion-Referenced? A student should be able to know in advance how well he must do to receive a high grade.

The extent to which a student has met the requirements of a course should determine his grade.

Before a course begins, a professor should have already determined the criteria that he plans to use to grade students.

Inspecting grades awarded in a particular course should indicate which students have an acceptable mastery of the subject matter and which students do not.

Are You Self-Referenced? A professor should take a student’s effort and ability into account when grading.

A student should be graded in relation to his potential to succeed in a class.

A student who is utilizing all of his ability should receive a high grade, regardless of his overall ability.

What kinds of assessments and assignments do you utilize?

Essay tests?

Multiple-choice tests?

Other objective tests? (e.g., fill-in the blanks, true-false)

Term papers?

Projects?

Do you test primarily for memory or problem solving?

The implications in terms of grading are…

Why?

Differences across Disciplines There are large differences across disciplines

Sciences tend to grade more stringently than other disciplines

Education is the highest

These tend to be stable and consistent across institutions

They may relate to the attitudinal and frame of reference issues

Conclusions Grading should be used to help students learn

This approach requires that students are involved in their learning, that they are active learners, and they wish to have communication with faculty members and peers

Motivating students with grades has traditionally been tied to extrinsic motivation; we want students to have intrinsic motivation

Following the principles enclosed in this presentation should help increase student learning

grading: assessment techniques to enhance learninggrading or marking systems there are many systems...

Documents