creating tests that assess higher order thinking skills dr. jennifer hurd innovative educators...
TRANSCRIPT
Creating Tests That Assess Higher Order Thinking Skills
Dr. Jennifer HurdInnovative Educators
ObjectivesWhen you complete this workshop, you
should be able to:• Write different types of questions that will assess more
than knowledge and understanding on the Bloom’s Taxonomy Scale.
• Analyze test items to determine if these test items assess higher order thinking skills.
• Avoid signaling the correct answer by knowing common errors in teacher-developed exams.
Bloom’s Taxonomy
Developing Objectives Using Bloom’s Taxonomy
• Objectives must be measurable
• Objectives must be attainable
• Objectives must have a limiting factor
Objectives must:
• Contain the same key words that we will discuss for developing questions.
• Map to the mission statement or goals for the entire course or department
• Demonstrate the skills that have been attained for each level of Bloom’s Taxonomy
Example of an Objective
According to the course material, 85% of the students will be able to design a business plan for a new
business of their choice.
Example of an Application Objective
According to the course content, 90% of the students in this course will be able to complete a mock interview
with three of the instructors
Learning Outcomes
• Are the primary building blocks of good curriculum design.
• Support the goal or learning outcome in that each is a small step to arriving at what the learner is supposed to know or be able to do.
• Are carefully worded with qualifiers to restrict the conditions and terms under which the learning outcomes are met.
• Are measurable!
Learning outcomes must contain verbs that describe
observable, measurable actions.
Learning Outcomes should:
Provide the basis to select or design instructional materials, content or assessments
Provide a framework within which learners can organize their efforts to complete the learning task
Planning a test
• Determine the method of assessment– Demonstrating knowledge and understanding– Assessing critical thinking skills– Assessing problem solving skills– Assessing performance of procedures– Assessing application skills– Assessing collaborative learning skills– Assessing research skills
Planning: Specifying What is to be Assessed
• Clarify instructional learning outcomes• Define the purpose of the test
– Pre-instructional (pre-test)– Tests and quizzes– Final exams
• Specify what will be tested and at what level
• Develop a test blueprint
Blueprint for a TestContent/
Skill Area
Knowledge/Understanding
Critical Thinking
Skills
Skill Comprehension
Application
Critical Thinking Problem Solving
Performanceof
Procedures/Demonstrati
ng Techniques
Collaborative Learning
Skills
Integrate Learning
Into Professional
Practice
AssessingResearch
Skills
Basic Tenants of Test Item Writing
• Reading Level• Write items as simply as possible• Test for important details• Avoid direct quotes or verbatim
statements• Identify any opinions or authority that
are used in test items
Anatomy of an Exam Question
• Should have three parts:– A key word that tells the students what they are
expected to do– An object that tells them what general
information to use to perform their task– A limiting factor or factors that tells what
specific information to include in the answer
Bloom’s Taxonomy
Let’s look at each level of Bloom’s Taxonomy to see how we can develop questions for each level.
Knowledge Questions
• remembering;
• memorizing;
• recognizing;
• recalling identification and
• recall of information
• Who, what, when, where, how ...?
• Describe
An example of a knowledge question
According to the instructor, define the key words for knowledge questions.
An example of a knowledge question
According to the instructor, list the key words for knowledge questions.
Can you change this question into a M/C question?
Writing a question
According to the instructor, which of the following are the key words for knowledge
questions?
A. list, synthesize, evaluate, define, write
B. List, define, dramatize, solve
C. List, define, duplicate, memorize
D. None of the above
E. A and B only
Application Question Key Words
• Imagine• Pretend• Make Believe• Visualize• Assume
• Put yourself in the situation of
• Use your knowledge to
• Suppose
In groups of three or four write a three-part application question on any topic or subject area using one of the key words listed above. Be prepared to share your question.
Imagination Activity
• Imagine that you are creating your own company• You can hire 5 key people to help you run your
company• These 5 people can be anyone from the past or
present• Who would you hire and why?• You have 3 minutes to decide, then you will
discuss your response with your table partners. One person from each table will share with the group.
Evaluation Question Key Words
• Rank• Order• Judge• Grade• Measure• Determine• Convince• persuade
• Decide • Support• Explain• Critique• Criticize• Recommend• Suggest
Sample Three-part Evaluation Question
• According to the number of safety-related features each contains, determine which compact car is safest to drive.
Key Word: Determine
Object: to find out which compact car is safest to drive
Limiting Factor: cars must be compact
Only safety-related features to be examined
A/B VisitArgue Workshop
• This workshop allows students to persuade, convince, determine, decide, recommend, suggest, support, and explain parts of course content.
• Applying course content to other parts of the course or to their own life makes it even more meaningful and worthwhile.
A/B VisitArgue Workshop
• Set up the room for A/B visits, or have students get into pairs.
• Your topic is to argue whether or not there should be a student success course for college credit on your campus.
• The people on the outside (A’s) will take the pro stance.
• The people on the inside (B’s) will take the con stance.
Synthesis Question Key Words
• Combine• Integrate• Modify• Revise• Improve• Rearrange• Reconstruct• Hypothesize
• Substitute• Create• Generate• Devise• Design• Compose• Plan• Predict
Synthesis Sample Question
• Rearrange the amendments to the U.S. Constitution in order from least to most important in everyday life. As well, devise any new amendments that you think would improve everyday life. Justify your amendments according to your text or your notes.
Rearrange the amendments to the U.S. Constitution in order from least to most important in everyday life. As well, devise any new ammendments that you think would improve everyday life.
• Key Words: rearrange and devise• Object: To discover which constitutional
amendments you consider most important and which you would like to add to improve everyday life
• Limiting Factor: deal only with amendments to the U.S. Constitution.
Other Synthesis Assessments
• Ask students to create mnemonic devices to remember course content.
• Students can create crossword puzzles that center on a particular part of course content
• Create a visual product to show course content.
• Example: Create a company logo for XXX company.
Analyzing Test Items to Determine if they Assess Higher
Order Thinking Skills
In the next three questions, use the chat feature to tell me whether the
questions assess higher order thinking skills. Why/why not?
According to the text, good money management
skills include
A. Planning, budgeting, and living within your means
B. Living within your means and using credit cards to help.
C. Setting goals, budgeting, living within your means, and using credit cards wisely
D. Living on a budget and using credit cards when necessary
E. Living on a budget without credit cards.
Use your knowledge from the text and the class discussions of Hitler’s strategies to suggest how he lost World War II.
According to the text and class discussions, which of the following would be a reaction of an insecure/avoidant baby to the strange situation research technique?
A. The baby would show great upset when the parent departed and would cling to the parent upon his/her return.
B. The baby may or may not show great upset when the parent departed, but would cling to the parent upon his/her return.
C. The baby would show great upset when the parent departed, but would ignore the parent upon his/her return.
D. The baby may or may not show upset when the parent departed, but would ignore the parent upon his/her return.
Common Errors Made by Teachers in Constructing Test
Items
Don’t give the answer away!
Avoid these common errors.
Test Construction Errors to Avoid
• Don’t use “All of the above” or “None of the above” only when they are the correct answer.
• Avoid distractors that are a joke or an insult.
Test Construction Errors to Avoid
• Make sure the stem of the question and the distractors/answer are grammatically correct
• Avoid having one choice that is extremely longer than the other choices.
Test Construction Errors to Avoid
• If all the distractors are numbers, 90% of the time the correct answer is not the highest number or the lowest number.
• Avoid using absolute determiners in the answer choices.
Test Construction Errors to Avoid
• Avoid using more inclusive options.
• Avoid answer options that state a reason.
Learning how to play the piano while singing occurs in the
A. Affective domain
B. Motor domain
C. Cognitive domain
D. Motor and cognitive domains
E. None of the above
According to the text, those who believe in the work ethic believe that
A. All people will do good work.
B. Hard work is always rewarded.
C. Work is good and important.
D. Only good workers should receive pay.
The normal adult temperature of 98.6 degrees F is equivalent to
A. 18 degrees C
B. 27 degrees C
C. 37 degrees C
D. 42 degrees C
Summary
• Use three-part questions to make sure you are asking questions that cover all of Bloom’s Taxonomy
• Write objectives to measure all aspects of Bloom’s Taxonomy
• Have students create questions and grade questions
• Use other methods instead of just paper/pencil tests to assess students in higher order thinking skills
Resources
Parker, J. (2006). Workshops for Active Learning. Boston: Houghton Mifflin Company.
This book provides the anatomy of an exam question process that is used in this presentation.
Resources
http://www.acu.edu/academics/adamscenter/course_design/syllabus/verbs.html
• This link contains a list of learning outcomes verbs that relate to Bloom’s Taxonomy
Resources
I included a handout with this presentation titled Checklist for Evaluating Test Items.
This handout can help you evaluate the different types of test items to avoid signaling the answers as we discussed in this webinar.
Resources
http://www.schreyerinstitute.psu.edu/MultipleChoiceItems/
• This link contains more suggestions for developing higher level multiple choice test items.
Resources
How to Assess Higher-Order Thinking Skills in Your Classroom
By Susan M. Brookhart
You can order this book at this website:
Rhttp://www.ascd.org/publications/books/109111.aspx
