Assessing Writing

Contents

Micro-skills vs. macro-skillsTypes of writingScoring writing

Holistic scoringAnalytic scoringObjective scoring

Genres of written language

1. Academic writing

2. job-related writing

3. Personal writing

Theses, shopping list, phone messages email, diaries, advertisements, compositions

Micro-skills vs. Macro-skills Micro-skills:

Phoneme-grapheme correspondences Mechanics such as spelling and punctuation Word order patterns, grammatical systems, & rules Cohesive devices in written discourse

Macro-skills: Rhetorical modes, communicative functions & purposes Main/supporting ideas, new/given info.,

generalization/exemplification Literal & implied meanings Culturally specific references Prewriting devices, paraphrases, synonyms, drafting,

feedback, revising, and editing

Types of Writing (1) Imitative (micro-skills):

Mechanics of writing: fundamental, basic tasks of writing letters, words, punctuation, and short sentences

Focus on form.Handwriting letters, words, and punctuation

• Copying• Listening cloze selection task

Spelling tasks and detecting phoneme• Spelling/ multiple-choice reading-writing spelling tasks• Matching phonetic symbols (p. 225)

Types of Writing

Intensive/controlled (micro-skills):Sentence production: proper vocabula

ry, collocations, idioms, grammarStill focus on form, with some concern

about meaning and context

Intensive Writing Assessment Dictation and Dicto-com Grammatical Transformation Tasks (p. 226)

Change the tenses in a paragraph Change the statement to yes/no questions

Picture-cued tasks Short sentences Picture description Picture sequence description

Vocabulary Assessment Tasks Reordering words in a sentence Short-answer and sentence completion tasks

Dictocomp A variation on the traditional dictation

procedureT reads while Ss listen carefullySs write a summary (not word for word as

in dictation)Decide scoring criteria (Bailey 151)

Rationale:Meaning-orientedConveying & capturing meaning = key

component of communication

Types of Writing (2)

Responsive (macro-skills): Paragraph writing: narratives, descriptions, short reports, lab

reports, summaries, responses to reading, interpretations of charts/graphs

Guided question and answer Paragraph construction tasks Emphasis on context and meaning

Extensive (macro-skills): All-purpose essay writing

• Term paper, research project report, thesis Process of multiple drafts

Scoring Writing

Holistic scoring Analytical scoring Objective scoring

Holistic Scoring (1)

A single scale is used to describe different levels of writing performance.

Not to think about the individual components of the writing skill or to count the number of errors in the writing.

The rater reacts to the writing as a whole. A single score is given.

• E.g., on a scale of 1 ~ 10. Focuses more on communication; overall

impression; e.g.,TOEFL’s TWE (Brown 239; Bailey 187)

Holistic Scoring (2)

Advantages• Fast• Higher inter-rater reliability• Easily understood by teachers and students alike• Widely applicable to writing across different disci

plines• Emphasizes the writer’s strengths

Holistic Scoring (3)

Disadvantages (Brown 242)

One score masks differences across individual composition or the subskills within each score.

No diagnostic info. Is available (no washback).The scale may not apply equally well to all genre

s of writing.Raters need to be extensively trained to use the

scale accurately.

Training Procedure

Norming process: Familiarization with the scale Read several benchmark papers & discussing h

ow they relate to the scale Raters independently read a mixed set of paper

s and score them. Comparison of scores awarded by all raters Discuss discrepancies Another set of papers is read & scored.

Primary Trait Scores

Focuses on “how well students can write within a narrowly defined range of discourse”

A four-point scale ranging from zero to 4. The accuracy of the account of the original

(summary) The clarity of the steps of the procedure and

the final result (lab report) The description of the main features of the

graph (the graph description) The expression of the writer’s opinion

(response to an article)

Analytic Scoring (1) “Embodies hypotheses about the underlying

constructs that comprise a given skill,” or evaluate separately various components

Assessing Ss’ performances on a variety of categories E.g., 5 components:

• Content 13 ~30 points• Organization 7 ~ 20 points• Vocabulary 7 ~20 points• Language use 5 ~25 points• Mechanics 2 ~5 points

Analytic Scoring (2)

Why called “analytical scoring?”Because scale designers have analyzed

the hypothesized components of the writing skill, and it’s these components that make up the categories used in scoring

Analytic Scoring (3)

Evaluating separately various components:Points-off, or weighted score, or split gradeScoring profile: based on five components

• Content• Organization• Vocabulary• Language use• Mechanics