assessing grammar christinealicestaceyjennyann test evaluation summary teaching summary teaching...

43
Assessing Grammar Assessing Grammar Christine Christine Alice Alice Stacey Stacey Jenny Jenny Ann Ann

Post on 20-Dec-2015

227 views

Category:

Documents


0 download

TRANSCRIPT

Assessing GrammarAssessing Grammar

ChristineChristineAliceAlice

StaceyStaceyJennyJennyAnnAnn

Test EvaluationTest Evaluation

Summary TeachingSummary Teaching Evaluation of an Existing Evaluation of an Existing

TestTest

Five characteristics to measure Five characteristics to measure communicative grammarcommunicative grammar

1.1. The test must provide more context than The test must provide more context than only a single sentence.only a single sentence.

2.2. The test taker should understand what The test taker should understand what the communicative purpose of the task is.the communicative purpose of the task is.

3.3. He or she should also know who the He or she should also know who the intended audience is.intended audience is.

4.4. He or she must have to focus on meaning He or she must have to focus on meaning and not form to answer correctly.and not form to answer correctly.

5.5. Recognize is not sufficient. The test taker Recognize is not sufficient. The test taker must be able “to produce grammatical must be able “to produce grammatical responses.”responses.”

By: P.R. Dickins

Therefore…Therefore…

Have the test taker say or write Have the test taker say or write something (5) of discourse length (1) something (5) of discourse length (1) in order to perform some in order to perform some communicative task (2) for a known communicative task (2) for a known audience (3), and what is said or audience (3), and what is said or written must make sense (4).written must make sense (4).

Assessing grammar can only be Assessing grammar can only be tested as part of a test of writing or tested as part of a test of writing or speaking?speaking?

Important Principle: What are we trying to Important Principle: What are we trying to Measure?Measure?

““English grammar is chiefly a system of syntax that English grammar is chiefly a system of syntax that decides the order and patterns in which words are decides the order and patterns in which words are arranged into sentences.” – Close 1982arranged into sentences.” – Close 1982

Separated but closely related to assessing writing Separated but closely related to assessing writing and assessing speaking.and assessing speaking.• Oral performanceOral performance• Writing samplesWriting samples

Accuracy and fluencyAccuracy and fluency Count the errors and rate their severityCount the errors and rate their severity

Focus on Grammar onlyFocus on Grammar only• ‘‘Meaning’ is a separated matterMeaning’ is a separated matter• Integrated testsIntegrated tests• Discrete point testsDiscrete point tests

Traditional Paper and Pencil TestTraditional Paper and Pencil Test

NRT: TOEFLNRT: TOEFL CRT: Classroom testsCRT: Classroom tests Types of Questions:Types of Questions:

• Complete sentencesComplete sentences• Sentence combiningSentence combining• Paragraph editingParagraph editing• Correct topic sentence taskCorrect topic sentence task• Multiple ChoiceMultiple Choice• Fill in the blankFill in the blank• True/FalseTrue/False• CombinationCombination

Alternative AssessmentAlternative Assessment

GamesGames Class observationClass observation Online activitiesOnline activities PortfolioPortfolio

GamesGames

Design to restrict the language needDesign to restrict the language need Usually these games are defined by Usually these games are defined by

their focus ontheir focus on• The Use of LanguageThe Use of Language• Limited options for communicationLimited options for communication• AccuracyAccuracy

Split SentencesSplit Sentences

Write out some sentences, and then Write out some sentences, and then cut each sentence in half. Hand out cut each sentence in half. Hand out these pieces to the students, and ask these pieces to the students, and ask them to find the matching half them to find the matching half among the other students. among the other students.

Students’ abilities in combing Students’ abilities in combing sentencessentences

Grammar knowledge to all forms of Grammar knowledge to all forms of the sentencesthe sentences

ExamplesExamples

If you eat thatIf you eat that If you touch the dogIf you touch the dog If you steal my If you steal my

boyfriendboyfriend If you go out nowIf you go out now If you don’t leaveIf you don’t leave If you don’t book a If you don’t book a

ticketticket

you’ll be sickyou’ll be sick it’ll bite youit’ll bite you I’ll never speak I’ll never speak

to you againto you again you’ll get you’ll get

soakedsoaked I’ll call the policeI’ll call the police you’ll be lucky you’ll be lucky

to get a seatto get a seat

Sentences from picturesSentences from pictures

Hand out a sheet of pictures. Ask Hand out a sheet of pictures. Ask students to come out sentences from students to come out sentences from the pictures. Or ask students to tell the pictures. Or ask students to tell a stories.a stories.

Enough grammar knowledge to make Enough grammar knowledge to make sentences by students. sentences by students.

AccuracyAccuracy

Memory TestMemory Test

Give limited time for students to see Give limited time for students to see the picture. Ask them to describe the the picture. Ask them to describe the picture without seeing it. picture without seeing it.

Testing students’ ability of making Testing students’ ability of making sentences.(enough grammar sentences.(enough grammar knowledge)knowledge)

AccuracyAccuracy The transformation of students’ The transformation of students’

sentencessentences

Picture DictationPicture Dictation

A student try to describe the picture A student try to describe the picture to other students who haven’t seen to other students who haven’t seen it.it.

Testing if a student can make Testing if a student can make sentences which can help him/her sentences which can help him/her communicate well.communicate well.

AccuracyAccuracy

Strip StoryStrip Story

Make students into groups and Make students into groups and assign them a text from a strip story. assign them a text from a strip story. Ask them to find the order and Ask them to find the order and resolve the problem from the story.resolve the problem from the story.

See if students can find the time See if students can find the time order from the tenses of the order from the tenses of the sentences.sentences.

AccuracyAccuracy

Miming an actionMiming an action

Ask students to role play in a limited Ask students to role play in a limited situation.situation.

Through acting, it can test students’ Through acting, it can test students’ ability of thinking of lines. (which ability of thinking of lines. (which may contains different tenses.)may contains different tenses.)

AccuracyAccuracy

Growing StoriesGrowing Stories

Story-building activitiesStory-building activities Students’ ability of making sentences Students’ ability of making sentences

with different tenseswith different tenses

QuestionnairesQuestionnaires

Turn what have taught in class into a Turn what have taught in class into a questionnaire. Get students to questionnaire. Get students to survey each other. (does not contain survey each other. (does not contain fully-written-out questions.)fully-written-out questions.)

It can see if students’ totally It can see if students’ totally understand what have learned in understand what have learned in class. Also it can see if students’ can class. Also it can see if students’ can use it well.use it well.

Classroom ObservationClassroom Observation

Classroom activitiesClassroom activities Classroom communication: Elicited Classroom communication: Elicited

dialoguesdialogues Written exerciseWritten exercise

Online ActivitiesOnline Activities In multimedia teaching times, teachers can use In multimedia teaching times, teachers can use

Internet as a way of assessing students’ ability of Internet as a way of assessing students’ ability of grammar grammar

On-line quizzes help students to have more On-line quizzes help students to have more chances of practiceschances of practices

On-line tests also provide clear explanation after On-line tests also provide clear explanation after finishing the quizzesfinishing the quizzes

Students can surf on these grammar website to Students can surf on these grammar website to do practicedo practice

Teachers can ask students to do these exercise Teachers can ask students to do these exercise and to keep record for themselves. Finally report and to keep record for themselves. Finally report back for one of the reference of assessmentback for one of the reference of assessment

Online activity : websitesOnline activity : websites

http://http://www.ccc.commnet.edu/grammar/index.htmwww.ccc.commnet.edu/grammar/index.htm Ex1. Irregular verbs quiz Ex1. Irregular verbs quiz

http://www.ccc.commnet.edu/cgi-shl/par2_quiz.pl/irrhttp://www.ccc.commnet.edu/cgi-shl/par2_quiz.pl/irregular_quiz.htmegular_quiz.htm

Grammar Assessment Grammar Assessment http://http://www.prenhall.com/grammarassessmentwww.prenhall.com/grammarassessment//

Self-study grammar quizzes Self-study grammar quizzes http://a4esl.org/q/h/grammar.htmlhttp://a4esl.org/q/h/grammar.html

ESL Quizzes Center ESL Quizzes Center http://www.pacificnet.net/%7Esperling/quiz/grammahttp://www.pacificnet.net/%7Esperling/quiz/grammarr

PortfolioPortfolio

Definition:a purposeful collection of student Definition:a purposeful collection of student work that tells the viewer about the studentwork that tells the viewer about the student

Works should be organized by chronological Works should be organized by chronological order and category– teacher can evaluate order and category– teacher can evaluate the child’s achievementsthe child’s achievements

Works including: Works including: 1. Traditional paper and pencil tests1. Traditional paper and pencil tests 2. Classroom observation2. Classroom observation 3. Online activities3. Online activities 4. 4. Written assignmentsWritten assignments

Written assignmentsWritten assignments

Give students some assignments to Give students some assignments to practice at homepractice at home

Ask students using learned grammar Ask students using learned grammar in class to write journal in class to write journal

Term paper throughout one semester Term paper throughout one semester including several drafts– we can see including several drafts– we can see student’s progress in grammar.student’s progress in grammar.

Advantages of portfolioAdvantages of portfolio

Why portfolio is the best way to Why portfolio is the best way to assess?assess?

Strengthening students’ learning Strengthening students’ learning Enhancing the teacher’s roleEnhancing the teacher’s role Improving processes Improving processes

Test Evaluation of An Existing TestTest Evaluation of An Existing Test

Section 2 of Section 2 of the Computer-Based Test the Computer-Based Test of TOEFL of TOEFL

General Background InformationGeneral Background Information

1.1. Title: Section 2 of TOEFL-Structure and Written Title: Section 2 of TOEFL-Structure and Written

ExpressionExpression

2.2. Author: Education Testing Service (ETS)Author: Education Testing Service (ETS)

Test Evaluation of An Existing TestTest Evaluation of An Existing Test

3. Purpose of decision 3. Purpose of decision ::

SectionSection 2 measures the ability to recognize 2 measures the ability to recognize

language that is appropriate for standard language that is appropriate for standard

written English. written English.

Test Evaluation of An Existing TestTest Evaluation of An Existing Test

4. Date of Application:4. Date of Application:

a. 1962, the formation of a National a. 1962, the formation of a National

Council on the TOEFLCouncil on the TOEFL

b. July 1998 (Most parts of the world)b. July 1998 (Most parts of the world)

c. October 2000 (Asia, including c. October 2000 (Asia, including

Taiwan)Taiwan)

Practical OrientationPractical Orientation

1.1. Ease of administration: Ease of administration: a.a. Computers neededComputers needed

b.b. ConvenienceConvenience

2.2. Price: expensive ($130) Price: expensive ($130)

3.3. Quality of items: clarityQuality of items: clarity

Practical OrientationPractical Orientation

4.4. Ease of scoring: Ease of scoring: a.a. Photo score reportingPhoto score reporting

b.b. Computerized scoring (0~30) Computerized scoring (0~30)

c.c. Cumulative scoringCumulative scoring

5.5. Ease of interpretation: transmission Ease of interpretation: transmission

speed. speed.

Practical OrientationPractical Orientation

6.6. Test construction: Test construction: a.a. Computer-adaptive TestingComputer-adaptive Testingb.b. Complete an incomplete sentenceComplete an incomplete sentencec.c. Identify one unacceptable answer form four uIdentify one unacceptable answer form four u

nderlined words. nderlined words.

7.7. Timing: 20-25 questions/ 15-20 minutes (includiTiming: 20-25 questions/ 15-20 minutes (including “pretest questions”)ng “pretest questions”)

““Description of the Computer-Based TOEFL Test” Description of the Computer-Based TOEFL Test” www.ets.orgwww.ets.org

Test Evaluation of An Existing TestTest Evaluation of An Existing Test

Model Test Model Test

Analysis of TOEFL Grammar TestAnalysis of TOEFL Grammar Test

8 sentence patterns8 sentence patterns• Basic Sentence & Compound SentenceBasic Sentence & Compound Sentence• Adjective ClausesAdjective Clauses• Participial Phrases Participial Phrases • AppositiveAppositive• Noun ClausesNoun Clauses• Adverbs ClausesAdverbs Clauses• Prepositional PhrasesPrepositional Phrases• ExpletivesExpletives方有毅 方有毅 托福文法速成托福文法速成 來欣留學英語中心 來欣留學英語中心

Analysis of TOEFL Grammar TestAnalysis of TOEFL Grammar Test

15 Answering Technique15 Answering Techniquess

1.1. Word Form Word Form ………………........20 %20 %2.2. Word Choice Word Choice ………….......14 %14 %3.3. Verbs ………………Verbs ………………........11 %11 %

4.4. Parallelism ………Parallelism ………............9 %9 %

5.5. Pronouns ……………Pronouns ……………......7 %7 %

6.6. Singular/Plural Nouns …Singular/Plural Nouns …......…………......…………....6 %6 %

7.7. Verbal (Infinitives, Gerunds, and Verbal (Infinitives, Gerunds, and Participles) ……Participles) ……....……......6 %6 %

8.8. Prepositions ……Prepositions ……....……....6%6%

9.9. Articles ……………………….5 %Articles ……………………….5 %10.10. Comparatives andComparatives and Superlative sentence……4 %Superlative sentence……4 %11.11. Conjunctions ……………… 4 %Conjunctions ……………… 4 %12.12. Redundancy………………….1 %Redundancy………………….1 %13.13. Negative Words ……………2 %Negative Words ……………2 %14.14. Word Order ………………….4 %Word Order ………………….4 %15.15. Misplaced Modifiers ….…..1 %Misplaced Modifiers ….…..1 %

Total ……………………….100 %Total ……………………….100 %方有毅 方有毅 托福文法速成托福文法速成 來欣留學英語中心 來欣留學英語中心

Analysis of TOEFL Grammar TestAnalysis of TOEFL Grammar Test Reliabilities: Reliabilities: describes the tendency of a set of scores to

be ordered identically on two or more tests, and it can be estimated by a variety of statistical procedures.

Table 1. Reliabilities and Standard Errors of Measurement (SEM)

July 1998 – June 1999

Reliability SEM

Listening 0.98 2.76

Structure/Writing 0.88 4.89

Reading 0.88 2.73

Total Score 0.95 10.8

““Description of the Computer-Based TOEFL Test” Description of the Computer-Based TOEFL Test” www.ets.orgwww.ets.org

Analysis of TOEFL Grammar TestAnalysis of TOEFL Grammar Test ValidityValidity1.1. ------pure lead, the lead ore is mined, then melted, and finally re------pure lead, the lead ore is mined, then melted, and finally re

fined. fined. (A) Obtaining (B) Being Obtained(A) Obtaining (B) Being Obtained (C) To obtain (D) It is obtained(C) To obtain (D) It is obtained

2.2. Louisa May Alcott is Louisa May Alcott is mainlymainly noted for noted for herher very popular very popular A B A B novel, novel, Little WomenLittle Women, , whomwhom she she publishedpublished in 1868. in 1868. C DC D3.3. SomeSome hangars, buildings used to hold large aircraft, are hangars, buildings used to hold large aircraft, are AA veryvery tall that rain occasionally falls tall that rain occasionally falls fromfrom clouds that form clouds that form B CB C alongalong the ceilings. ( the ceilings. ( 聶群 聶群 精修文法精修文法 來欣留學英語中心 來欣留學英語中心 )) DD

Analysis of TOEFL Grammar TestAnalysis of TOEFL Grammar Test

Strength: Strength: Accurate and Accurate and

objective scoresobjective scores Careful monitoring Careful monitoring

of testof test Commitment to Commitment to

international international educationeducation

Expert advisory Expert advisory committeescommittees

Highly skilled test develHighly skilled test developersopers

Longstanding reliability Longstanding reliability and consistencyand consistency

Ongoing test improvemOngoing test improvementsents

Standardized delivery prStandardized delivery proceduresocedures

““Description of the Computer-Based TOEFL Description of the Computer-Based TOEFL Test” Test” www.ets.orgwww.ets.org

Analysis of TOEFL Grammar TestAnalysis of TOEFL Grammar Test

ConclusionConclusion

No other test in the world is as reliable as a staNo other test in the world is as reliable as a sta

ndard for measuring a nonnative speaker’s andard for measuring a nonnative speaker’s a

bility to use English at the university level.bility to use English at the university level.““Description of the Computer-Based TOEFL Test” Description of the Computer-Based TOEFL Test” www.ets.orgwww.ets.org

ENDEND