analysis of test formats, poor and good items
Post on 24-Jun-2015
981 Views
Preview:
TRANSCRIPT
Analysis of Test formats
Authors:Hector Rojas
Daniel BlancoYuly Caraballo
Matching
STRENGTHS
Objective Easy to design Easy to include a
representative example
WEAKNESSES
Easy to guess Only for grammar,
vocabulary, pronunciation
Not good for testing the skills.
Students need to decide on what answers accepted as correct.
Gap-filling
STRENGTHS Objective Easy to correct Provides clear contexts It can be used to
correct different strategies.
WEAKNESSES Extremely difficult to
design an item with only one possible answer (generally solved by providing a list of terms)
Need to think about possible answers (if a list is not provided)
Cloze
STRENGTHS
Easy for the teacher to create
Easy for the students to understand
Easy to correct and design .
WEAKNESSES
Unexpected correct answers are possible
Mainly for vocabulary It can be irritating for students
Completion
STRENGTHS It is easy to construct Requires simple
recognition of information (eliminatting the guessing factor)
It is good to evaluate listening for specific information
WEAKNESSES Depend on knowledge
of language Difficult to grade
depending on evaluation criteria
It can be irritating for students
Multiple Choice
STRENGTHS Easy to correct and grade Scores are less affected
by guessing Good for checking
understanding Usually it requires less
time for test takers to answer
WEAKNESSES Difficult to construct It does not allowed test
takers to demonstrate knowledge beyond the options provided
Emphasize in recognition and guessing rather than in production
True-False
STRENGTHS Easy to write Can cover a wide
range of difficulty levels
Usually it requires less time for test takers to answer
Usually it is easy to grade and score
WEAKNESSES Guessing element The score can not be
reliable It does not allow test
takers to demonstrate broad range of knowledge
Transformations
STRENGTHS
Good for testing some structures
Easy to write and administer
Specifically tackle one grammatical objective
Allow recognition of a connection between “grammar” and “meaning”
WEAKNESSES
Very artificial It can be more than
one possible transformation in some cases
Questions
STRENGTHS
It is good for checking if students comprehend texts.
It is useful to test the student's ability to analyze a concept.
WEAKNESSES
It is not easy to score It takes more time to
correct and answer.
Recommendations for item design
FORMAT RECOMMENDATIONS
Matching • Give more options than questions.• Each answer should have distracters.• Try meaningful items.• Make sure there’s only one possible selection.• Do not ask to draw lines to match.
Gap filling • No blank at the beginning.• Give enough context.• Avoid clues.• Only one answer • Give line with the same length
Cloze Don’t use nth word, for example : 5th , 3rd
Completion • Avoid the statement becomes ambiguous.• Use blanks of the same length throughout the test so that
the length is not a clue.• If possible, put the blank at the end of a statement rather
than at the beginning. Asking for a response before the student understands the intent of the statement can be confusing and may require more reading time.
Multiple choice
• Be clear and concise in your word and phrase choices.• Make sure that there is only one clearly correct answer
from the options given to the student.• Randomly distribute the correct answer options i.e. A, B, C,
D etc so that there is not a clear pattern that becomes obvious to the student.
True-False • Make sure that the answer is clear and that it could not be either or.• Try not to use negative questions such as “this novel was
not written by....”• Use a random order of true and false responses with your
test questions to avoid creating a pattern.• Use more false questions than true questions as they have
been proven to cater towards higher cognitive level students.
Questions • It has to be easy to understand.• Check how you will judge answers.
Poor & Good items
Multiple choice items
All words should belong to the same grammatical category.
She had to help the ______ old man up the stairs.
A. weak B. slowly C. try D. wisdom
A. weak B. strong C. energetic D. athletic
Adverb Verb Noun
Poor & Good items ii
Grammatical clues make the answer obvius.
She needs to get up earlier so she’s buying an ______ clock.
A. time B. alarm C. watch D. bell
She needs to get up earlier so she’s buying ______ clock.
A. a time B. an alarm C. a watch D. a bell
Deleting the indefinite article would make this item better in terms of objectivity.
Poor & Good items iii
The level of difficulty of the distractors highlights the correct alternative.
They needed lots of training to operate such ______ equipment.
A. easy B. sophisticated C. blue D. wise
“Sophisticated” clearly contrasts with the rest of the options, so students might select it right away.
Poor & Good items iv
Options that collocate with the verb in the sentence. (Differentiators)
She sent the _______ yesterday.
A. letter B. gift C. food D. books
She wrote a _______ yesterday.
A. letter B. gift C. friend D. books
“Post”, “friend” and “courage” are good replacements for “gift”, “food” and “books” that are items that can also be sent or mailed.
Poor & Good items v
Completion
The absence of context will lead to the acceptance of unexpected correct answers.An advanced learner could get to the conclusion of
completing with the letter “y” to form the adjective “nervy”, which means “bold” or “offensive”.
He was a very nerv_____ person.
He was always alert. He was a ________ person.
In this case, given the context, the correct option is “nervous”.
Poor & Good items vi
Words that may take two opposite suffixes lead to ambiguity.
That was a care______ answer.
Enough context will permit the student know which alternative is the most appropiate one.
Yesterday he got on the wrong bus. So today, he was care_______ to find the right one.
Poor & Good items vii
True-FalseColumbus discovered America.
Columbus sailed to the new world in 1942.
The use of trivial information must be avoided . Given the fact that they are more likely to be true than ambiguous information.
If used, they must be presented in a more objective way.
Poor & Good items viii
Since penguins can’t fly, they cannot swim very well either.
Since penguins are unable to fly, they swim badly.
Avoiding the use of negative statements will prevent confusion and contradiction.
Poor & Good items ix
Rattlesnakes live in North America, have fangs, feed on small rodents and crush their prey.
The first three clauses are true, however the last one is not.
The entire sentence must be either true or false, otherwise it will create confusion.
Rattlesnakes crush their preys.
Poor & Good items x
Matching
___1. Sit-and-reach___2. 50-yard dash___3. Pull-up___4. Shuttle run___5. Balke treadmill___6. Dyer backboard volley___7. Disch putting___8. Biopsy
c. Muscle fibersh. Golff.Tennisa.Vo₂maxe.Agilityg.Arm strengthd.Speedb.Flexibility
Items are too heterogeneus, making the answers too obvious.
Contain the same number of items, so the last item could be answered by elimination.
Match column 1 and comlumn 2:
The instructions do not clarify how many times items in column 2 may be used or a basis for matching.
Poor & Good items xi
TransformationChange the sentences to the future
simple:
Doug studied English yesterday.Doug went to the movies with his family.He eats a hamburger.They enjoy the day in the mall.
The activity lacks context
The sentences should be provided in the same tense, to prevent confusion.This exercise requieres more contextualized
instructions, such as:
This is what Doug usually does on the weekend, write about what he is going to do next the weekened.
Item and test analysis.
Item analysis: measure the performance of each item in a test and help us to revise and improve both, items and test as a whole.
Use: -To build reliablity and validity into a test.- More diagnosis information on students.
Features of item analysis
Item difficulty: deals with the number of students who answer correctly. The best items are that are neither too difficult or too hard.
IF = # of students answering the item correctly
Total of students responding that item.
Features of item analysis
Item discrimination : differenciates those test takers with high - abilities from those with low - abilities.
Features of item analysis
Distractor efficiency: deals with the way a distractor lures test takers, especially those with low abilities.
References
Brown, Douglas (2004) Language assessment: Principles and Classroom prractices. Longman, London.
Cunningham, George (1998) Assessment in the Classroom: Constructing and interpreting Tests. Routledge, London.
Morrow, James and others (2010)Measurement and Evaluation in Human Performance. Human kinetics, Illinois.
Madsen, Harold (1983) Techniques in testing. Oxford University Press, New York.
top related