08 assessing reading and writing

ID7756 Evaluación del Aprendizaje de Idiomas

Enero-Marzo 2014

Prof. Yris Casart

[email protected]

A framework for assessing readingRecommendations for reading

assessment

• Step 1: Write test specifications

• Step 2: Select an appropriate text

• Step 3: Modify the text

• Step 4: Write test questions

• Step 5: Decide on weighting

• Step 6: Validate and re-validate

Lloyd & Davidson (2009) en Coombe, Folse & Hubley (2007)

• Step 7: Ensure standardized test

administration

• Step 8: Ensure reliable rating/marking

• Step 9: Rescale (if necessary)

• Step 10: Provide students with diagnostic

feedback

• Step 11: Evaluate your test

• Step 12: Recycle your test

Lloyd & Davidson (2009) en Coombe, Folse & Hubley (2007)

Coombe, Folse & Hubley (2007)

1. Use specificationsfor even coverage of curricular outcomes

to promote content validity

2. Avoid skill contamination by limiting writing

3. Choose text to fit purpose of assessment

and to match students’ level.

4. Make items less difficult than passage

level

5. Test items in the order of the passage.

6. Sample skills with range of formats.7. Keep a file of authentic material from

newspapers, magazines, etc.8. Avoid texts with controversial or biased

material.9. For objective formats, try to make all

statements positive.10. Rephrase material using synonyms to

avoid students looking for verbatim matches.


indirect vs. direct assessmentfree writing vs. guided writing

writing assessment scalesresponding to student writing

10 things to remember

Indirect writing assessment

• It measures correct usage in sentence level constructions and focuses on spelling and punctuation via objective formats like MCQs and cloze tests.

• Measures determine the S’s knowledge of writing sub-skills such as grammar and sentence construction which are assumed to constitute components of writing ability

• Measures are largely concerned with accuracy rather than communication.


Direct writing assessment

• It measures the S’s ability to communicate through the written mode based on the production of written texts.

• It requires Ss to come up with the content, find a way to organize the ideas and use appropriate vocabulary, grammatical conventions and syntax.

• Measures integrate all elements of writing.


free writing guided writing

requires Ss to read a

prompt that poses a

situation and write a

planned response

based on a combination

of background

knowledge and

knowledge learned from

the course.

requires Ss to

manipulate content that

is provided in the

prompt, usually in the

form of a chart or

diagram. It’s a bridge

between objective and

subjective formats.


• Be clear about expected form/length of response (1 paragraph, 250 words, a letter)

• Clearly specify what you want in the prompt(3 causes & effects, two supporting details)

• Specify discourse patterns students are expected to use(i.e. compare/contrast, cause/effect, description etc.)

• Ask Ss to provide something beyond the prompt (i.e., opinion, inference, prediction.


• Considerations for GW are useful for free writing.

• Use of multiple raters. Agree on grading criteria in

advance & calibrate before grading).

• Decide on which scale to use beforehand.

• Acquaint Ss with marking scheme in advance by

using it for teaching, grading homework and

providing feedback.

• Provide Ss with enough space for an outline, a draft

and the finished product.

• Avoid issues that might offend or disadvantage Ss.Coombe, Folse & Hubley (2007)

Holistic marking Analytical marking

It is based on marker's

total impression of the

essay. It is variously

termed impressionistic,

global or integrative

marking.

It requires Ts to give

separate ratings for the

different components of

language ability. It has

been termed discrete

point marking.


Advantages Disadvantages

• Reliable if done under no time constraints & if Ts receive training.

• Generally perceived to be quicker than other types of marking

• Since overall writing ability is assessed, Ss are not disadvantaged by one lower component, i.e. poor grammar.

• Scores emphasize the writer’s strengths

• Unreliable if marking is done under short time constraints & with untrained Ts.

• Longer essays often tend to receive higher marks

• Tendency for marker to overlook sub-skills that make up writing.

• It does not provide a profile of the student's writing ability and to provide feedback.

• Difficult to interpret composite score


Advantages Disadvantages

• Generally more effective with inexperienced teachers.

• provide Ts with a "profile" of their Ss' strengths & weaknesses

• Training raters is easier because the scales are more explicit and detailed.

• Perceived to be more time consuming

• A set of specific criteria has to be written and markers need to be trained: attend frequent calibration sessions.

• Because Ts look at specific areas in a given essay, the most common being content, organization, grammar, mechanics and vocabulary, marks are often lower


• Feedback provides opportunities for Ss to learn & make improvements

• Most common type of written teacher feedback is handwritten comments, usually at the end of the paper or in the margins

• Some Ts like to use correction codes to provide formative feedback and to facilitate marking

• Electronic feedback is particularly valuable because it can be used to give a combination of handwritten comments and correction codes.


sp Spelling

vt Verb tense

ww Wrong word

wv Wrong verb

☺ Nice idea/content!

₪ Switch placement

¶ New paragraph

? I don’t understand

Coom

be,

Fols

e&

Huble

y(2

007)

1. Give students multiple writing assessment opportunities.

2. Test a variety of writing skills and create tasks of varying lengths.

3. Develop prompts that are appropriate for the Ss.

4. Evaluate all answers to one question before going on to the next.

5. Mark only what the student has written.


6. Have a systematic approach for dealing with marking discrepancies: average of the two raters for a small discrepancy and to utilize a third rater if there is a big discrepancy.

7. Get students involved. 8. Provide students with diagnostic

feedback.9. Practice blind or double blind marking. 10. Calibrate and recalibrate. The best way to achieve inter-rater reliability is to practice.


Coombe, C., Folse. K, & Hubley, N. (2007) A Practical

Guide to Assessing English Language Learners. Ann

Arbor, MI: University of Michigan Press.

08 assessing reading and writing

Education