1 testing and grading ensuring that we meet the objectives

1

Testing and Grading

Ensuring that we meet the objectives

2

Desired Learning Levels

TeachingPhilosophy

Students Abilities Course Level and Scope

Write Course and Topic Objectives

Select Effective Teaching Methods

How Students Learn

BA 9200 Course Overview

The Lecture

Active LearningTesting and Grading

3

Testing Overview

A. Preparing Exams 1. Type of Tests2. Number of Tests3. Questions per Test, and4. Table of Specification

B. Writing Questions 1. Effective Stems and Distracters

2. Structured Essay Questions and Answers

C. Grading 1. Scoring Key and PartialCredit for Multiple Choice

D. Improving Exams 1. Survey and Difficulty and Discrimination Indices

4

A. Preparing Exams

1. Type of Tests

Norm Based Criterion Based

Goal Sort and Classify Students Assess Student Competency

Scores Must Have Large Spread in Scores

Spread in Scores Not Relevant

Difficulty Questions Must Range from Easy to Difficult

Questions Must Reflect Demands of Profession

Timing All Students May Not Finish Exam

All (or Most) Should Finish Exam

5

A. Preparing Exams

2. Number of Tests

Not Effective: One Exam at the end Effective: Unit Tests/Quizzes followed by a final exam.

6

A. Preparing Exams

3. Criterion-Based Tests: Number of Questions per Test

RoteMultiple choiceShort Answer

2 minutes4 minutes

MI LevelMultiple ChoiceShort Answer

4 minutes8-10 minutes

Critical ThinkingMultiple ChoiceShort Answer

10-15 minutes15-30 minutes

Rough guideline for time per question at each level.

7

A. Preparing Exams 4. Table of Specification

Rote Exercise MI

Measures of Center

15% 25%

Measures of Spread

10% 15% 10%

Shape of Distribution

10% 15%

8

Testing Overview






9

B. Writing Questions

1. Multiple Choice Questions

Essay Questionsa. Require the Student to Remember Information.b. Are Scored More Reliably.c. Are More Difficult to Construct.d. Are Easier to Construct.

The Median isa. the Average Value.b. the 25% Percentile Value.c. the 75% Percentile Value.d. that Number such that Approximately 50% of the Numbers Are Higher and 50% are Lower than the Median Value.

10

B. Writing Questions Multiple Choice Questions

The Term Used in Connection with Sound that Corresponds to Color with Reference to Light is a. Pitch.b. Hue.

c. Timbre. d. Intensity.

One Basic Advantage of Using Short Answer Essay Questions as Opposed to Long Answer Ones is that Short Answer Essay Itemsa. Generating Questions is Easier.b. Allow for a Broader Sample of Questions.c. Students Prefer Them.d. They Relate More Closely to the Course Objectives.

11

B. Writing Questions Writing Effective Stems Ideal Stem Answerable Without Seeing Choices. Questions Address Important Concepts. Relate Each Question to One or More Topic Objectives.

Question Stem Followed by Cognitive Objectives that Question Addresses{O12, O15}

Cite Authority in Stem where applicaple

12

B. Writing Questions Writing Effective Stems

Solve Math Problems to Ensure All Data in Stem. State Stem in Positive Language. Same Phrase Appears in Each Answer, Shift It to Stem. Use Complete Sentence Format.

13


Writing Effective Distractors Use Best-Answer Format. Write Best or Correct Answer First. For Concept Question

Correct Answer to Similar Concept. Have Correct Elements, But Also Incorrect Elements From Students' Answers to Essay Version of Question.

14


Writing Effective Distractors

For Math Question Reflect Mistakes That Students Make. Use "Not Sufficient Information to Solve Problem"

Option. Avoid “None of Above” for Math Problems

15


Writing Effective Distracters

Correct Answer Should Not Be Longest Answer. All Distractors Should Form Complete and Grammatically-Correct

Sentences With Stem. Avoid Using Never or Always. Avoid an “All of the Above” Option. Avoid Repeating Words From Stem. Vary Correct Answer Among Four Possible Answer Positions.

16


2. Writing Short-Answer Questions

Base on Objectives. Use “Structured Essay” Format.

Uses Precise Verbs. Helps Students Frame Answer.

Use Iterative Procedure. Write Ideal Answer. Could Students Achieve Ideal Given

Question Wording? Questions Address Important Concepts. Solve Math Problems.

17

Compare and Contrast “Amadeus” and “The Shawshank Redemption”.

John Locke: The Key to David Hume? Discuss.

Is the Central Limit Theorem Critical to Constructing a Confidence

Interval on a Population Mean? Discuss and Defend.


Unstructured Short-Answer Questions

18


The All-Bluff Answer and the Need for the Structured

FormatIs Central Limit Theorem Critical to Constructing a Confidence Interval on a Population Mean? Dr. Deming Noted that the Central Limit Theorem is Essential.

As You Said in Class, “The Central Limit Theorem is a Critical Topic.”

The Central Limit Theorem Says that ……..

Finally, Hypothesis Testing is Similar to Confidence Intervals in that ...

19

B. Writing Questions Structured Answer Format

Compare and Contrast “Amadeus” and “The Shawshank Redemption”.

Basis 1 is _________ Basis 1 is Important Because _________

Basis 2 is __________ Basis 2 is Important Because __________

20


Structured Answer Format Students then Complete the Table.

Bases Amadeus Shawshank 1 2 3 4

21

Can the Bias in a forecast ever be greater than the MAD value? [O15] 10 Points

Select One Response Below and Complete.

Yes Because

No Because


22

Suppose the Central Limit Theorem was not true.Would It be Possible to Construct a Confidence Intervalon a Population Mean? [O23] [15 Points]

Select One of the Responses Below and Complete.Would Never be Possible Because

Would Always be Possible Because

Would Only be Possible When


23

Suggest a modification to the analogy method that would increase the likelihood that a Sensing-dominant manager would use the Method. [O45, O56] [25 Points]

Describe the Modification.

This Modification Increases the Likelihood that aSensing-Dominant Manager Would Use theAnalogy Method Because …


24

Testing Overview






25

C. Grading

Grading Issues for All Types of Exams

Written Comments Are Highly Motivating. “Excellent Improvement” “Sorry Did So Poorly. Please See Me.”

Return Within One Class (MC) or One Week (Short Answer).

Provide Key After Quiz, if Possible.

26

C. Grading

Grading Issues on Multiple Choice Exams

Partial Credit for Multiple Choice?

On Numerical Problems: Provide Optional White Space and Only Check if Wrong Option Selected.

Defend Concept Answer to Complex Question: Provide Optional White Space and Only Check if Wrong Option Selected.

27

C. Grading

Grading Issues on Short Answer Exams

For Small Classes, Provide Audio Feedback. Students Hand In Audio Cassette to Instructor. Write Annotated Comments on Exam.

Use a Detailed Scoring Key. Grade One Question at a Time. Shuffle Papers After Each Question.

28

Testing Overview






29

D. Improving Exams Use Test Assessment Instrument During Term.1. Have exams reflected topic objectives?2. Have exams reflected important subject content?3. Has there been sufficient time to complete exams?4. Has instructor reacted positively to student concerns on questions,

grading, etc.? __________________________________________1 2 3 4 5

Not at All Moderately Very Much

5. How could the exams be improved?

30

Additional Readings

Bloom, B., Hastings, T., and Madaus, G. Handbook of Formative and Summative Evaluation for Student Learning. New York: McGraw Hill, 1971.

Jacobs, L and Chase, C. Developing and Using Tests Effectively: A Guide for Faculty. San Francisco, CA.: Jossey-Bass, 1992.

31

Desired Learning Levels

TeachingPhilosophy

Students Abilities Course Level and Scope

Write Course and Topic Objectives

Select Effective Teaching Methods

How Students Learn

BA 9200 Course Overview

The Lecture

Active LearningTesting and Grading

1 testing and grading ensuring that we meet the objectives

Documents

number of questions

short answer essay questions

question stem

writing questions1

broader sample of questions

course objectives

state stem

multiple choiced