test item writing and analysis
TRANSCRIPT
![Page 1: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/1.jpg)
![Page 2: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/2.jpg)
September 10, 2011UIC
TEST ITEM WRITING and ITEM ANALYSIS
![Page 3: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/3.jpg)
The Systematic Planning Process:
Identify Instructional Goals
Analyze Learners
Identify Objectives
Plan instructional activities
Choose Instructional Media
Design Assessment Tools
Implement Instruction
Revise Instruction
![Page 4: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/4.jpg)
Steps in Test Construction
Identify Objectives Develop TOS Write Items
![Page 5: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/5.jpg)
Illustrations on Sampling
Population(Universe of
Characteristics)
Sample( Best estimates of the
Population)
Test
![Page 6: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/6.jpg)
Illustrations on Sampling
Universe of Characteristics
A Better Model
Best Model
A Poor Model
![Page 7: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/7.jpg)
Illustrations on SamplingInvalid and Unreliable Test•Too many errors
Valid but Unreliable Test• Errors causes test scores to be unstable and inconsistent
Valid and Reliable Test• Observed errors are not significant (or negligible)
![Page 8: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/8.jpg)
THREE-STAGE MODEL OF CLASSROOM MEASUREMENT
CONTENT VALIDITY
Stage 1 Stage 2 Stage 3
OBJECTIVES ACTIVITIES TEST
Test items must validly measure instructional
objectives
![Page 9: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/9.jpg)
A. Research Designs 1. Differentiate Quantitative and Qualitative reseach.2. Classify the various research designs3. Identify common research designs4. Contruct research problems in each research designs.
total
Topics / Objectives K C A AN S E total
THINKING SKILLS
1,2
3,4,5,67,8,9,1011,12
Item Placement
2 2
2 2 41 2 1 4
2 2
60
Hint:# of items = # of days per topic / total # of days X total # of items
1 6
2 121 61 6
10
DAYS IDEAL
SAMPLE TABLE OF SPECIFICATION
![Page 10: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/10.jpg)
WORKSHOP 1
You need to have:a. Sample syllabusb. Test Questionnairec. Paper and Pen
![Page 11: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/11.jpg)
Instructions:Choose a syllabus and a
questionnaire from your compilation
Match the items in the questionnaire with the competencies measured in the syllabus
Make an evaluation about the content validity of the test – whether it measures what it is suppose to measure.
Volunteers will share their summary of evaluation
![Page 12: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/12.jpg)
Matching Requirements:
Item versus objectiveThinking Skill versus
objectiveProportion of items per
objective versus the total number of items, total number of objectives, and the time element.
![Page 13: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/13.jpg)
TemplateObjectiv
esList of Items
Skills Time Frame
Total
A. Solve the ….
5, 3, 5
ok 3
![Page 14: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/14.jpg)
![Page 15: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/15.jpg)
What do you see in the picture?
![Page 16: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/16.jpg)
How many legs does this elephant have?
![Page 17: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/17.jpg)
![Page 18: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/18.jpg)
Multiple Choice Type of
Test
![Page 19: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/19.jpg)
AdvantagesA large number of ideas can
be addressed in a short period of response time.
These questions are easily and quickly scored.
Questions can elicit responses from all cognitive levels, from knowledge to evaluation.
Questions can be improved over time by analyzing them in light of student performance.
![Page 20: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/20.jpg)
DisadvantagesIt is time-consuming to write good items, especially those at higher cognitive levels.
Test-wise and English fluent students tend to be favored.
![Page 21: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/21.jpg)
varieties of multiple-choice items
A. The correct answer variety
Who invented the sewing machine?
a. Fulton*b. Howec. Singerd. Whitee. Whitney
![Page 22: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/22.jpg)
B. The best answer variety
What was the basic purpose of a test?
a.Measure teacher’s abilityb.determine students’ achievementc. Assess learning outcomesd. Identify students’ weaknesses
![Page 23: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/23.jpg)
The multiple response variety
What factors are principally responsible for the clotting of blood?
a.contact of blood with a foreign substance*b. contact of blood with injured tissuec. oxidation of hemoglobin*d. presence of unchanged pro thrombin
![Page 24: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/24.jpg)
Use of Analogies
Man is to woman as boy is to
a.Fatherb.Motherc.Girld.Lady
![Page 25: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/25.jpg)
The incomplete statement variety
In 2008, millions of pesos worth of corn, rice, and wheat are destroyed in the Philippines by
a. ratsb. grasshoppersc. birds*d. pests
![Page 26: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/26.jpg)
The negative variety
Which of these is NOT true of viruses?
a. Viruses live only in plants and animals.b. Viruses reproduce themselves.*c. Viruses are composed of very large living cells.d. Viruses can cause diseases.
![Page 27: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/27.jpg)
The substitution varietyPassages to be readSurely the forces of education should be fully utilized to acquaint youth with the real nature of the dangers to democracy, (1) for no other place offers (2) as good or better opportunities than the school for; a (3) rational consideration of the problems involved.
Items to be answered1. *a. , for
b. . Forc. —ford. no punctuation needed
2. a. As good or better opportunities thanb. as good opportunities or better thanc. as good opportunities as or better than*d. better opportunities than
3. *a. rationalb. radicalc. reasonabled. realistic
![Page 28: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/28.jpg)
The combined response variety
In what order should these sentences be written in order to make a coherent paragraph?A. A sharp distinction must be drawn between table manners and sporting manners.B. This kind of handling of a spoon at the table, however, is likely to produce nothing more than an angry protest against squirting grapefruit juice about.C. Thus, for example, a fly ball caught by an outfielder in baseball or a completed pass in football is a subject for applause.D. Similarly, the dexterous handling of a spoon in golf to release a ball from a sand trap may win a championship match.E. But a biscuit or a muffin tossed and caught at the table produces scorn and reproach.a. A. B, C. D. E*b. A, C. E, D. Bc. A, E. C. D, Bd. B, E, D. C. A
![Page 29: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/29.jpg)
Basic Guidelines
![Page 30: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/30.jpg)
Adolescents obtain information about marriage
A. in order to reduce the possibility of pregnancyB. from parentsC. from same-sex friends*D. in a haphazard fashion
![Page 31: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/31.jpg)
PRESENT A CLEAR PROBLEM IN THE STEM
Most adolescents obtain information about marriage from
A. same sex friends*B. parentsC. health service personnelD. teachers
![Page 32: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/32.jpg)
Before the Civil War, the South's _____was one of the major reasons manufacturing developed more slowly than it did in the North.
A. emphasis on staple-crop production*B. lack of a suitable supply of raw materialsC. short supply of personnel capable of operating the necessary machinery
![Page 33: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/33.jpg)
PUT ALTERNATIVES AT THE END OF THE QUESTION
Before the Civil War, manufacturing developed more slowly in the South than in the North. One of the major causes of this was the South's___
A. emphasis on staple-crop production*B. lack of a suitable supply of raw materialsC. short supply of personnel capable of operating the necessary machinery
![Page 34: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/34.jpg)
In objective testing, the term objective
A. refers to the method of identifying the learning outcomesB. refers to the method of selecting the test contentC. refers to the method of presenting the problemD. refers to the method of scoring the answers*
![Page 35: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/35.jpg)
PUT MOST OF THE WORDING IN THE STEM
In objective testing, the term objective refers to the method of
A. identifying the learning outcomesB. selecting the test contentC. presenting the problemD. scoring the answers*
![Page 36: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/36.jpg)
For almost a century, the Rhine river has been used by Europeans for a variety of purposes. However, in recent years, the increased river traffic has resulted in increased levels of diesel pollution in the waterway. Which of thefollowing would be the most dramatic result if, because of the pollution, the Council of Ministers of the European Community decided to close the Rhine to all shipping?
A. closure of the busy river Rhine ports of rotterdam, Marseilles and GenoaB. increased prices for Ruhr products*C. reduced competitiveness of the French Aerospace IndustryD. shortage of water for Italian industries
![Page 37: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/37.jpg)
AVOID UNNECESSARY WORDINESS
Which of the following would be the most dramatic result if, because of diesel pollution from ships, the river Rhine were closed to all shipping?
A.closure of the busy river Rhine ports of Rotterdam, Marseilles and GenoaB. increased prices for Ruhr products*C. reduced competitiveness of the French Aerospace IndustryD. shortage of water for Italian industries
![Page 38: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/38.jpg)
Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, which of the following should not happen?
A.Children should not believe all of their behavior is bad.B. Children should understand the reason(s) why they are being punished.C. Children should understand that the teacher, not the children, controls when the punishment will end.*
![Page 39: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/39.jpg)
AVOID NEGATIVELY WORDED STEMS
Sometimes a teacher finds it necessary to use a mild form of punishment. When this occurs, it is important that the children understand:
A.that it may be a long time before happy times return to the classroomB.the reason(s) why they are being punished*C. that the teacher, not the children, controls when the punishment will end
![Page 40: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/40.jpg)
Good
Thurstone's 7 primary mental abilities include all of the following EXCEPT:
A. word fluencyB. reasoningC. social interaction*D. perceptual speed
![Page 41: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/41.jpg)
Which of the following men contributed most towards he defeat of Hitler‘s Germany in World War II?
A. Winston ChurchillB. Josef StalinC. Franklin D. RooseveltD. George Patton
![Page 42: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/42.jpg)
AVOID REQUIRING PERSONAL OPINION
![Page 43: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/43.jpg)
What is the area of the rectangle described in Question 1?
A. 1050 sq. cm.B. 7396 sq. cm.C. 7654 sq. cm.*D. 8188 sq. cm.
![Page 44: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/44.jpg)
AVOID LINKED OR CLUED ITEMS
![Page 45: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/45.jpg)
What is the official state bird of Pennsylvania?
A.mountain laurelB.B. PhiladelphiaC. ruffed grouse*D. ibex
![Page 46: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/46.jpg)
ALL OPTIONS SHOULD BE HOMOGENEOUS
What is the official state bird of Pennsylvania?
A. goldfinchB. robinC. ruffed grouse*D. wild turkey
![Page 47: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/47.jpg)
Who succeeded Giscard d'Estaing as President of France in l981?
A. Georges PompidouB. Charles De GaulleC. Francois Mitterand*D. Mick Jagger
![Page 48: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/48.jpg)
ALL OPTIONS SHOULD BE PLAUSIBLE
![Page 49: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/49.jpg)
Which of the following is the best definition of a seismograph ?
A. An apparatus for measuring sound waves.B. An apparatus for measuring heat waves.C. An apparatus for measuring earthquake waves.*D. An apparatus for measuring ocean waves.
![Page 50: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/50.jpg)
PUT REPEATED WORDS IN THE STEM, NOT THE
OPTIONSA seismograph is an apparatus for measuring:
A. earthquake waves*B. heat wavesC. ocean wavesD. sound waves
What type of waves does a seismograph measure?
A. earthquake*B. heatC. oceanD. sound
![Page 51: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/51.jpg)
An angle of 90o is called a
A.acute angleB.obtuse anglesC.right angle*
![Page 52: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/52.jpg)
MAKE ALL OPTIONS GRAMMATICALLY
CONSISTENT WITH STEM
Angles of 90o are called
A.acute anglesB.obtuse anglesC.right angles*
![Page 53: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/53.jpg)
The average difference between males and females in the attainment of pubertyis:
A. 1 yearB. 2 years*C. 3 yearsD. no difference
![Page 54: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/54.jpg)
ORDER OPTIONS LOGICALLY
![Page 55: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/55.jpg)
RULES FOR WRITING MULTIPLE-CHOICE ITEMS
Design each item to measure an important learning outcome.
Present a single clearly formulated problem in the stem of the item.
Put the alternatives at the end of the question, not in the middle nor at the beginning.
Put as much of the wording as possible in the stem.
Eliminate unnecessary wordiness
![Page 56: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/56.jpg)
Avoid negatively worded stems. "Which of the following is not..........“
Avoid requiring personal opinion. Other item types are more suitable for this.
Avoid textbook wording. Do not have linked or clued items. All options should be homogeneous.
![Page 57: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/57.jpg)
All options should be plausible. Put repeated words in the stem, not in the options
Punctuation should be consistent. Make all options grammatically consistent with the stem of the item.
List options vertically.
![Page 58: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/58.jpg)
Order options logically. Use the option "all of the above" sparingly.
Use the option "none of the above" sparingly.
Make All distracters the same length
![Page 59: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/59.jpg)
Exercise:Joseph Estrada was an
ActorFormer PresidentDriverScientist
![Page 60: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/60.jpg)
In what year did humans first set foot on the moon?
1975195719691963
![Page 61: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/61.jpg)
Some test items Are too difficultAre objectiveAre poorly constructedHave multiple defensible answers
![Page 62: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/62.jpg)
Ferdinand Magellan came to the Philippines in a
CarBoatAirplaneballoon
![Page 63: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/63.jpg)
WORKSHOP 2
You need to have:a. Sample syllabusb. Test Questionnairec. Paper and Pen
![Page 64: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/64.jpg)
Instructions: Evaluate your items using the guidelines in item writing as your criteria.
Revise poor items.Identify the objective/competence in every revised item.
Build an item bank
![Page 65: Test item writing and analysis](https://reader036.vdocuments.net/reader036/viewer/2022062900/58f352ae1a28ab7d2a8b45af/html5/thumbnails/65.jpg)