Keith Thiede, College of EducationAndy Goodman, Center for Teaching & Learning

Introductions Goals

◦ Agree on higher order thinking skills◦ Consider iterative process of test construction◦ Consider role of item analysis◦ Explore means of preparing students for the test◦ Examine 2 recipes for multiple choice test

construction◦ Write at least one multiple choice test question

for one of your courses

Bloom’s taxonomy (see handouts) Differences in higher and lower order

thinking skills Grid game

◦ information presented is factual or conceptual and represent different levels of thinking

◦ Place the objective in the appropriate block◦ Not all spaces will be filled

Easier to write higher order thinking skills when original objective was higher order

Which one of the following objectives requires students to engage in higher-level cognitive behavior?

A. Recite the alphabet B. Compare two books by the same author C. Identify the parts of the body D. List the three basic types of rocks

Stem

AlternativesDistractor

Distractor

Distractor

Answer

Expect to revise items several times. Simple item analysis can inform this

process:◦ Examine performance on individual items (90/10

rule)◦ Look at how well distractors worked◦ Examine performance of top and bottom students

Students expectations about tests affect many things:◦ How they prepare for a test◦ Their perceptions of the fairness of a test

Recipe 1 Consider your learning objective (e.g.,

application) Align instruction with objective (give

students examples and practice applying concept)

Align assessment with objective (provide a new situation to assess whether students can apply what they have learned anew)◦ Scenario◦ Graph/Data

Mrs. Liu’s students completed research papers on life in colonial times. The students worked on their papers for several weeks and turned in their final versions at the end of the marking period. Mrs. Liu evaluated the papers and used the scores she assigned to help determine the students’ report card grades for that marking period.

This is an example of what type of assessment? A. formative

B. objectiveC. summativeD. standardized

Directions: Below is a list of flaws often found in the stems of multiple-choice items. For each of the multiple-choice items that follow, decide whether the item stem has any flaws. Use the following symbols to indicate your response: A if the stem contains no flaws B if the stem doesn’t suf ficiently set the problem C if the stem contains a clue to the right answer D if the stem is negatively stated, and the negative is not highlighted E if the stem contains unnecessary, irrelevant words Each of these symbols can be used more than one time, or not at all. Write your response on the blank line before the item. Each correctly answered item is worth one point.

Another Example.

____ 1. I n Africa A. mining diamonds is an important industry. (keyed as the correct

response) B. there are many tropical rain forests. C. camel caravans carry trade goods.

Recipe 2 – Turning essay questions into multiple choice questions

Consider what you want your students to be able to describe/do

What are the essential elements you want to see in the answer

Turn at least one of those essential elements into the correct answer

Consider less or non-essential elements that might be part of an essay response

Turn those into the distractors

There were several Generative Approach suggestions presented in class for teaching melody (using Bruner’s learning theory). A)What are some of the important aspects to employ when planning a melody lesson for 1st graders?

B) Melody has many sub-concepts. Which aspects of melody do you attempt to teach first, second, etc.? Why?

Jerome Bruner’s learning theory could best be applied to a music setting in which of the following scenarios?

A)Students begin to develop an understanding of pitch by using sol-fege hand signs. Ks learn only sol and mi, 1st grade adds la, 2nd grade adds low do, etc.

B) Students begin to develop an understanding of rhythm by walking to and tapping a steady beat in K, using bars that show different lengths in 1st & 2nd grades, and transitioning to standard notation in 3rd.

C) Students learn pitch & matching by hearing & echoing a teacher. This begins in K and continues until all children can match pitch accurately. Around 3rd grade they learn a song by rote, then by note.

Check out this website for test construction: http://jonathan.mueller.faculty.noctrl.edu/toolbox/tests/morethanfacts.htm

Take 10 minutes to write one or two higher order multiple choice test questions and answers

Share with tablemates◦ Your questions◦ Thought process you used to create the

question/responses

Testing isn’t easy Application or analysis are the easier higher

order thinking skills to assess in multiple choice

Goal of testing is to figure out what students know or can do