WRITING TEST ITEMSFOR

LITERATURE IN ENGLISH

A Paper Presented at the Copperbelt LATAZ Conference Held From 3rd – 4th February, 2015 at Kansenshi Secondary School.

Renny Kapembwa Saini BA.ED (UNZA) Examiner/Item-Writer (E.C.Z - Literature in English)

WHAT IS ITEM WRITING? Item writing is often regarded as a process of

constructing and/or developing test questions. The key is to remember the end goal: that each

specific test item should focus on a piece of knowledge (or skill, ability, trait) and differentiate between candidates with high and low levels of knowledge.

Test item development therefore constitutes a conscious process of formulating, reviewing, editing and perfecting test items and questions.

Because assessment is a critical component of instruction, if properly used, it can aid in accomplishing key curricular goals.

TOOLS FOR ITEM WRITING

Teaching syllabus Examination syllabus Recent past question papers Test specification/grid

The teaching syllabus outlines the content to be covered in a subject. The Examination syllabus on the other hand outlines the rubric or set of directions to be followed when setting the test such as maximum marks allowed, whether essay type, short answers, multiple choice, etc., total marks allocated to each response or to a question and time allowed for each paper.

Recent past question papers are intended to guide the setter on the expected quality or standard. Recent past papers also serve as reference to a setter to guide him/her to avoid repeating questions that had appeared in recent examinations. Test specification or grid on the other hand is intended to balance the test so that the test covers a broad spectrum of levels of skills and not just one level.

CONSTRUCTING TEST ITEMS

A good test calls for a knowledgeable and skillful item-writer. A good test also should have; VALIDITY, RELIABILITY and DISCRIMINATION.

- Validity.A good test should posses validity. The test should measure and predict accurately what it is supposed measure, and nothing else. If for example a test is to measure achievement, it should do only that. - Reliability.A test cannot be valid unless it is reliable. If a test gives different results when it is administered a second time, then the test is not reliable. The assumption is that a good test should be consistent. It should give consistent results.

- DiscriminationExamination results show the differences in performance of the students that take the examination. The examination results show the achievement of each student in respect of the skills and subject areas that were learnt and examined.

BLOOMS TAXONOMY

Bloom's taxonomy refers to a classification of the different objectives that educators set for students (learning objectives).

It divides educational objectives into three "domains": cognitive, affective, and psychomotor (sometimes loosely described as "knowing/head", "feeling/heart" and "doing/hands" respectively).

Within the domains, learning at the higher levels is dependent on having attained prerequisite knowledge and skills at lower levels.

The goal of Bloom's taxonomy is to motivate educators to focus on all three domains, creating a more holistic form of education.

In Item Writing, one needs to have a good working understanding of Bloom’s Taxonomy.

This is to ensure that the set questions have Validity, Reliability and Discrimination.

A good test should therefore, be all rounded, i.e., it should be able to test across all the cognitive domains and not just the lower order thinking skills.

To ensure discrimination, a good test should combine the simple questions (that can be answered by almost everyone), the more challenging questions (that cannot be answered by the weak candidates) and the difficult questions (that can only be answered by the gifted candidates).

CONTENT MAPPING This is a process by which individual learning outcomes

or objectives are plotted across Bloom’s Taxonomy in order to guide the decision to assigning appropriate number or proportions of questions to appropriate levels of the curriculum under the listed content areas.

Content can be mapped by using the following action verbs as clues when aligning the specific outcomes across Bloom’s taxonomy:

Knowledge: (Remembering or retrieving previously learned material)- know - identify - relate - list- define - recall - memorize - repeat- record - name - recognise - acquire

Comprehension: (The ability to grasp or construct meaning from material)

- restate - locate - identify - discuss

- illustrate - interpret - recognise - explain

- report - express - describe - review

- infer - draw - represent - differentiate

Application: (The ability to use learned or implement material in new and concrete situations)

- apply - relate - develop - translate 

- use  - operate - organize - employ

- restructure - interpret - demonstrate - illustrate

- practice - calculate - show - exhibit

- dramatize

Analysis: (The ability to break down or distinguish the parts of material into its components so that its organizational structure may be better understood) - analyze - compare - probe - inquire - examine - contrast - categorize- differentiate - investigate - detect - survey - classify - deduce - experiment - scrutinize - discover - inspect - dissect - discriminate - separate

Synthesis: (The ability to put parts together to form a coherent or unique new whole) - compose - produce - design - assemble - create - prepare - predict - modify - tell - plan - invent - formulate - collect - set up - generalize - document - combine - relate - propose - develop - arrange - construct - organize - originate - derive - write - propose

Evaluation: (The ability to judge, check, and even critique the value of material for a given purpose)

- judge - assess - compare - evaluate

- conclude - measure - deduce - argue

- decide - choose - rate - estimate

- validate - consider - appraise - value

- criticize - infer

TABLE OF SPECIFICATIONS FOR LITERATURE IN ENGLISH (OLD SYLLABUS)

As earlier alluded to, a Table of Specifications is a grid that specifies the total marks for a given test item/question and how the marks are broken down according to blooms taxonomy.

This is in order to balance the test so that the test covers a broad spectrum of levels of skills and not just one level.

The 2013-2015 Literature in English Exam Syllabus Test Specification/Grid is as below;

Content Knowledge Comprehension Application Analysis Synthesis Evaluation TOTAL

(Context)

Play/Novel 8 5 3 2 - 2 20

(Essay)

Play/Novel - - - - - 20 20

(Essay)

Novel - - - - - 20 20

(Essay)

Novel - - - - 20 - 20

(Essay)

Novel - - - - 20 - 20

TOTAL 8 5 3 2 40 42 100

THE IMPORTANCE OF TEST SPECIFICATIONS

The consistency of yearly examinations can be guaranteed by specifying in advance the distribution of questions over a number of aspects.

The table of specification can guarantee consistency and comparability.

For instance, one should not pass an exam simply because they wrote in one given year, and not the other.

Similarly, one should not pass a test simply because it was prepared by such and such a teacher, and not the other.

In short, the table of specifications sets the standard.

TYPES OF QUESTIONS IN LITERATURE.

Context Questions. Essay Questions.

WHAT ARE CONTEXT QUESTIONS?

Context questions are those based on an excerpt or extract from the text of study. Unlike for comprehension questions in English Language, candidates attempting context questions in Literature are expected to draw their answers from their extended knowledge of a given text, and NOT necessarily or directly from the given excerpt.

AN EXCERPT FROM

SOYINKA: KONGI’S HARVEST

DENDE: I got no co-operation at all

From the blacksmith. It was the best

I found in the blacksmith’s foundry.

DANLOLA: Some soup-pot foundry. Find me

Such another ladle and I’ll

Shove it up your mother’s fundaments.

DAODU: [storms in] stops short as he sees signs of activity.]:

I was told you would not take part in today’s

procession.

DANLOLA: The ostrich also sports plumes but I’ve yet to see

That wise bird.

Leave the ground.

DAODU: But all this preparation…

DANLOLA: When the dog hides a bone does he not

Throw up sand? A little sand in the eye

Of His Immortality will not deceive

His clever Organising Secretary. We need to

Burry him in shovelfulls.

[re-enter Dende]

You horse manure. Is this a trip

To gather mangoes for the hawker’s tray?

Tell me, did I ask for a basket fit

To support your father’s goiter? I thought

I specially designed a copper salver.

DENDE: The smith had done nothing at all about it.

DANLOLA: The smith! The Smith! All I hear

Is some furnace blower called

The smith!

DAODU: [sharply, to Dende.] Send for the smith.

EXAMPLES OF QUESTIONS BASED ON THE EXCERPT(i) What is Dende referring to when he says, “It was the best I found in the

blacksmith’s foundry?” Mention the many ways that Danlola describes it.

(ii) Just before Dende says he “got no co-operation from the blacksmith,” what does Danlola complain of and what is happening at the palace?

(iii) In response to Daodu’s statement, explain what Danlola means in the figure of speech he uses by saying “The ostrich also… ground.”

ESSAY QUESTIONS Essay questions fall under the subject question

type. Subjective or essay items allow the learners

freedom to organize and present an original answer. According to (Stalnaker: 1951, p.495), “an essay

question is a test item which requires a response composed by the examinee or pupil, usually in the form of one or more sentences of a nature that no single response or pattern of responses can be listed as correct, and the accuracy and quality of which can be judged subjectively only by one skilled or well informed in the subject.”

Therefore, we can say that there are no specified right answers to essay type questions.

PURPOSES FOR USING SUBJECTIVE QUESTIONS There are two major purposes for using essay

questions that addresses different learning outcomes.

(i) To assess the pupils’ understanding of the subject matter content. (as in literature)

(ii) To assess the pupils’ writing abilities. (as in language)

These two purposes are so different in nature that it is best to treat them separately. Therefore, an essay item should meet the following criteria: Requires pupils to compose rather than select their

response. Elicits pupil responses that must comprise more

than one or two sentences.

The accuracy and the quality of pupils’ responses to essays must be judged subjectively by a competent specialist in the subject such as a trained teacher.

This is important because the nature of essay questions is such that only competent teachers in the subject can judge to what degree pupil responses to the essay question are complete, accurate, correct and free from extraneous information.

Ineffective essay items allow pupils to generalize in their responses without being specific and thoughtful about the content matter.

Effective essay questions, on the other hand, elicit depth of thought from pupils that can only be judged or marked by someone with appropriate experience and expertise in the subject matter.

Thus, content expertise is essential for both writing and grading of essay questions.

ABILITIES MEASURED BY ESSAY TEST ITEMS• In addition, essay questions can be used to measure attainment of a

variety of objectives. Listed below are 14 types of abilities that can be measured by essay items:– Comparisons between two or more things– The development and defense of an opinion– Questions of cause and effect– Explanation of meanings– Summarizing of information in a designated area– Analysis– Knowledge of relationships– Illustrations of rules, principles, procedures and applications– Application of rules, laws and principles to new situations– Criticism of the adequacy, relevance or correctness of a concept,

idea or information– Reorganization of facts – Formulating solutions to problems– Discriminations between objects, concepts or events– Inferential thinking.

• Essay questions provide an effective way of assessing complex learning outcomes that cannot be assessed by objective questions.

• They are an effective way to measure higher order cognitive objectives such as analysis, synthesis and evaluation.

• They are unique in measuring pupils’ ability to select content, organize and integrate it and then present it in logical prose.

• They have an effect on pupils’ learning. Pupils do not memorize facts but try to get a broad understanding of complex ideas and to see relationships. Essays also allow pupils to present their answers to a question by allowing them to arrive at their own conclusions.

• Essay questions also allow teachers to gain insights into pupils’ way of viewing and solving problems. With such insights, teachers are able to detect problems pupils may have with their reasoning process and help them overcome them.

DOES THE USE OF ESSAY QUESTIONS ELIMINATES GUESSING?

Although this problem of guess work is higher in objective questions, it still exists in subjective questions as well.

The type of guessing which is popular in essay responses is “bluffing” or “waffling”.

Some pupils are adept at using various forms of bluffing such as vagueness, generalizations, padding (irrelevant information) ambiguities, tautology, etc.

Examiners and teachers should be wary of such “pedestrian” type of answers to avoid giving credit where it is not due.

CONSTRUCTION OF ESSAY QUESTIONS

Is it true that Essay questions are easy to construct?

 It is true to a large extent that essay items are easier to construct as compared to objective items.

However, this should not be taken for granted because for any teacher to construct any good test item, time and effort must be invested.

Essay questions that are constructed hastily without much thought and review usually perform poorly.

Since few essay questions can be given regularly on any test, each item must be a good item. And because most subjective questions are designed to measure higher - level cognitive skills, we must be sure that the item does, in fact, tap these skills.

To accomplish these goals, certain guidelines should be followed when preparing essay items:

(i) Define the intended learning outcome to be

assessed by the item. 

Knowing the intended outcome or goal is crucial for designing essay questions. If the outcome to be assessed is not clear, it is likely that the question will assess for some skill, ability other than the one intended.

(ii) Define the task and shape the problem clearly

  Essay questions have two variable elements - the degree to which the task is structured and focused.

Remember essay questions pose a problem; so ensure to clearly state it so that it can elicit a solution from the pupils by giving them a task which needs to be performed in a stipulated time.

In composing essay questions, one must avoid over-generalised or leading questions. The latter clearly paves way to a particular answer.

It is important to consider the directive verbs. These are verbs that best describe the desired ability to be assessed.

The term ‘directive verb” refers to verbs that are thoughtfully and carefully selected to provide pupils with clear instructions and directions for their responses.

Therefore, a task should consist of a directive verb and the object of that verb.

Below is a diagram indicating some examples of directive verbs that teachers can utilize when preparing essay questions.

From the diagram above, the intersection is a clear indication that some verbs can subjective or objective or both.

- Avoid Complexity in Essay QuestionsUse language that is simple for Grade 12s sothat all the candidates understand the questionin order to give appropriate responses.Example: Literature -2011in ‘Things Fall Apart’

Discuss the theme existentialism vis-à-vis Okonkwo.

• Even though subjective items demand less time to prepare than objective items, a great deal of thought and care should go into the writing or preparation of an essay item.

•  Writing good essay items takes a lot of practice. It is unrealistic to think that you will write very good ones at first but with time and practice makes perfect. Experience will teach to think through each item that you write.

• It will also teach you to try to imagine all the possible things that pupils or examinees might think about as they respond to your item.

THE END