293_performance assessment, bordin chinda

8/4/2019 293_Performance Assessment, Bordin Chinda

1/16

7/23/2011

1

PerformancebasedAssessment

Rajamangala UniversityofTechnologyLanna

24July2011

BordinChinda,PhD

EnglishDivision,DepartmentofWesternLanguages

ChiangMaiUniversity

Performanceassessment

drivingtest

Olympicdiving

anartistsportfolio

asurgeons1sttriplebypassoperation

use

of

a

language

(e.g.,

English)


Wecannotjudgehowwellapersoncanuse

thelanguageunlesswemakethemperform

thelanguage

Wecannotjudgehowwelltheyperformifwecannotspecifythetask


2/16

7/23/2011

2


atestinwhichtheabilityofcandidatesto

performparticulartasksisassessed Tasks

designedtomeasurelearnersproductive

languageskillsthroughperformances

allowlearnerstoexhibitthekindsoflanguageskillsthatmayberequiredinarealworldcontext


Tasksforthesamepurposeandpopulationshouldhavethesamespecificationatasktypespecification

Everynewtaskshouldbeunique

Everytaskwiththesamespecificationshould

perform

the

same

for

the

same

people,

for

a

parallelgroupofpeopleonadifferentoccasion,andregardlessofwhodoesthescoring


Eachnewtaskneedstobevalidatedbeforebeingusedinalivesituation

Eachperformanceassessment needstohaveanongoingqualityassuranceprocess


3/16

7/23/2011

3


Characteristics

thereisaperformancebythecandidate

theperformanceisjudgedusinganagreed

setofcriteria

thereisadegreeofauthenticity ofthe

assessmenttasks


Alanguagetestissaidtobeauthentic whenitmirrorsasexactlyaspossiblethecontentandskillsundertest,e.g.

conversationtesttotestforconversationskills

anessaytesttotestforwritingessayskills atestofmedicalEnglishfornonEnglish

speakingdoctorswishingtoworkinEnglishmedium

PerformanceassessmentRATER

RATING(SCORE)

SCALE(Ratingcriteria/ratingrubrics)

PERFORMANCE

INSTRUMENT

CANDIDATE


4/16

7/23/2011

4


therater needstousearating

scaleinratingaperformanceto

arriveatascoreforthat

performance


teachersinclassrooms(raters),arerequiredto

makecomplicatedjudgements

morethantherightwrongdecisionsinMC

wherethestudentsresponsescanbemarked

aseithercorrectorincorrect


WritingGoalsandObjectives

Beforeaperformanceassessmentiswritten,

theteachershouldclearlyidentifygoalsand

objectivesoftheactivity.

Goals arebroadstatementsofexpected

studentoutcomes.

Objectives dividethegoalsintoobservable

behaviours.


5/16

7/23/2011

5


WritingGoalsandObjectives

Questions:

WhatdoIhopetolearnaboutmystudents'

knowledgeorskills?

Whatcontent,skillsandknowledgeshouldthe

activitybedesignedtoassess?

WhatevidencedoIneedtoevaluatethe

appropriateskillsandknowledge?


Recommendationsforwritinggoalsandobjectives: Thestatementofgoalsandaccompanying

objectivesshouldprovideaclearfocusforbothinstructionandassessment.

Bothgoalsandobjectivesshouldreflectknowledgeandinformationthatisworthwhilefor

studentstolearn.

Therelationshipbetweenagivengoalandtheobjectivesthatdescribethatgoalshouldbeapparent.


Recommendations(Cont.): Alloftheimportantaspectsofthegivengoal

shouldbereflectedthroughtheobjectives.

Objectivesshoulddescribemeasurablestudent

outcomes.

Goalsandobjectivesshouldbeusedtoguidethe

selectionofanappropriateassessmentactivity.


6/16

7/23/2011

6


Recommendationsfordevelopingperformanceassessments:

Theselectedperformanceshouldreflectavalued

activity.

Thecompletionofperformanceassessments

shouldprovideavaluablelearningexperience.

Thestatementofgoalsandobjectivesshouldbe

clearlyalignedwiththemeasurableoutcomesof

theperformanceactivity.


Recommendations(Cont.): Thetaskshouldnotexamineextraneousor

unintendedvariables.

Performanceassessmentsshouldbefairandfree

frombias.


Recommendationsforadministeringperformanceassessments:

Bothwrittenandoralexplanationsoftasksshouldbe

clearandconciseandpresentedinlanguagethatthe

studentsunderstand.

Scoringrubricsshouldbediscussedwiththe

studentsbeforetheycompletetheassessment

activity.


7/16

7/23/2011

7

Assessmentcriteria

Aratingscale(orrubrics inAm.Engl.)isa

scaleforthedescriptionoflanguage

proficiencyconsistingofaseriesof

constructedlevelsagainstwhichalanguage

learnersperformanceisjudged.

Assessmentcriteria

Thelevelsorbandsarecommonly

characterisedintermsofwhatsubjectscando

withthelanguageandtheirmasteryof

linguisticfeatures.

Assessmentcriteria

Ratingscalesalsorepresentthemostconcrete

statementoftheconstructbeingmeasuredor

descriptors.

Thedescriptorsdescribethelevelof

performancerequiredofcandidatesateach

pointonaproficiencyscale.


8/16

7/23/2011

8

Assessmentcriteria

Recommendationsfordevelopingscoringrubrics: Thecriteriasetforthwithinascoringrubric

shouldbeclearlyalignedwiththerequirements

ofthetaskandthestatedgoalsandobjectives.

Thecriteriasetforthinscoringrubricsshouldbe

expressedintermsofobservablebehavioursor

productcharacteristics.

Scoringrubricsshouldbewritteninspecificand

clearlanguagethatthestudentsunderstand.

Assessmentcriteria

Recommendations(Cont.): Thenumberofpointsthatareusedinthescoring

rubricshouldmakesense.

Theseparationbetweenscorelevelsshouldbeclear.

Thestatementofthecriteriashouldbefairandfree

frombias.

Analyticvs.

holisticratingscales

Withananalyticscale,ratersareaskedto

judgeseveralcomponentsofaperformance

separatelyastraits,criteria,ordimensionsof

performance.

Thesecomponentsaredividedsothatthey

canbejudgedseparatelyratherthangivinga

singlescorefortheentireperformance.


9/16

7/23/2011

9

Analyticvs.


Analyticscalesareusedwhen

planninginstructiontoshowrelativestrengths

andweaknessesofaperformance

teachingstudentsthenatureofaquality

performance

givingdetailedfeedback,

knowinghowtopreciselydescribequalityis

moreimportantthanspeed

Analyticvs.


withaholisticscale,raters areaskedtogivea

judgementonacandidatesperformanceasa

whole,orinotherwords,asinglescoreforan

entireperformancebasedonanoverall

impressionofacandidateswork

Analyticvs.


Holisticscalesareusedwhen

speedofscoringismoreimportantthan

knowingpreciselyhowtodescribequality,

theperformancesaresimple

whenaquicksnapshotofoverallachievement

istheobjective


10/16


11/16

7/23/2011

11

Stepsindesigning

analyticratingscales

Step4:Writethoroughnarrativedescriptions

forexcellentworkandpoorworkforeachindividualattribute.

Describethehighestandlowestlevelsof

performanceusingthedescriptorsforeach

attributeseparately.

Stepsindesigning


Step5:Completetherubricbydescribing

otherlevelsonthecontinuumthatranges

fromexcellenttopoorworkforthecollectiveattributes.

Writedescriptionsforallintermediatelevels

ofperformance.

Stepsindesigning


Step6:Collectsamplesofstudentworkthat

exemplifyeachlevel.

Thesewillhelpyouscoreinthefutureby

servingasbenchmarks.


12/16

7/23/2011

12

Stepsindesigning


Step7:Revisethecriteria,asnecessary.Be

preparedtoreflectontheeffectivenessofthe

criteriaandreviseitpriortoitsnext

implementation.

Ratingscales

Beginning

1

Developing

2

Accomplished

3

Exemplary

4Score

Criteria1 Description

reflecting

beginning

levelof

performance

Description

reflecting

movement

toward

masterylevel

of

performance

Description

reflecting

achievement

ofmastery

levelof

performance

Description

reflecting

highestlevel

of

performance

Criteria2

Criteria3

Ratingscales

Eachcriterionstatementshouldbemodifiedto

describeeachleveloftheperformances

attribute(s).

Thechoiceofwordsthatdescribethechanging

valuesoftheattributeusuallydependsonverbalqualifiers.


13/16

7/23/2011

13

Ratingscales

Threeverbalqualifierscalesarecommonly

used:

amount(e.g.,few,some,all)

frequency(e.g.,rarely,sometimes,often),and

intensity(e.g.little,some,very).

Ratingscales

Meaningfuldescriptionsofperformances

criteria

levels(standards)

keepthefocusonthelanguage

Reliability

Areliable assessmentisonethatelicitssimilarperformancesallthetime:

fromdifferentpeoplewiththesamebehaviour

ondifferentoccasions

onsametasktypebutdifferenttasks

whendifferentpeoplerate


14/16

7/23/2011

14

Validity

Avalid assessmentstartsbyasking

whatdowewantlearnerstoknow? whatdowewantthemtobeabletodo?

howwellshouldtheybeabletodoit?

whatdoesitlooklike?

Avalid assessmenttellstheanswerstothesequestionstoeveryoneinvolved

Sample1

Ratingcriteriaforan essay

4 Entriesarecreativelywritten.Proceduresand

resultsareclearlyexplained.Reportiswell

organized.

3 Entriescontainmostoftherequiredelements

andareclearlywritten.

2 Entriesareincomplete.Theremaybesome

spellingorgrammarerrors.

1 Writingismessyandentriescontainspelling

errors.Pagesareoutoforderormissing.

Problem(consistency)

Theattributes/categories arenotexplicit

Atlevel1,focusareonaccuracyofspelling

andorganization

Atlevel2,organizationisdropped(accuracy

remains)

Atlevel3,clarityisadded

Atlevel4,vaguerequirementofcreativityisadded


15/16

7/23/2011

15

Suggestions

Requiredele men ts Orga niz at io n Cla ri ty

43

2

1

Suggestions

Requiredelements

4 All oftherequiredelementsarepresent

3 Most oftherequiredelementsarepresent

2 Some oftherequiredelementsarepresent

1 Few oftherequiredelementsarepresent

Suggestions

Organization

4 Thereportis extremely organized.

3 Thereportis mainly organized.

2 Thereportis moderately organized.

1 Thereportis slightly organized.


16/16

7/23/2011

Suggestions

Clarity

4 Thereportis extremely clear.3 Thereportis mainly clear.

2 Thereportis moderately clear.

1 Thereportis slightly clear.

Task

Pronunciation&

Delivery

Communication

Strategies

Vocabulary&

Language

Patterns

Ideas&

Organization

6

5

4

3

2

1

0

AssessmentCriteriaforGroupInteraction

References

Brown,J.D.(2005).Testinginlanguageprogram:acomprehensiveguidetoEnglishlanguageassessment.NewYork:McGrawHill.

Davies,A.,Brown,A.,Elder,C.,Hill,K.,Lumley,T.andMcNamara,T.(1999).Dictionaryoflanguagetesting.Cambridge:CambridgeUniversityPress.

Davison,C.(2007).Viewsfromthechalkface: EnglishlanguageschoolbasedassessmentinHongKong.

LanguageAssessmentQuarterly, 4(1),3768. Fulcher,G.andDavidson,F.(2007).Languagetestingandassessment:anadvancedresourcebook.Oxon:

Routledge.

HampLyons,L.(2007).Theimpactoftestingpracticesonteaching:Ideologiesandalternative.InJ.

Cummins&C.Davison(Eds.),InternationalhandbookofEnglishlanguageteaching(Vol.PartI,pp.487504).NewYork:Springer.

Hughes,A.(2003).Testingforlanguageteachers.Cambridge:CambridgeUniversityPress. Mertler, CraigA.(2001).Designingscoringrubricsforyourclassroom.PracticalAssessment,Research&

Evaluation,7(25).RetrievedMay5,2011fromhttp://PAREonline.net/getvn.asp?v=7&n=25. McNamara,T.(2000).Languagetesting.Oxford:OxfordUniversityPress. Moskal,BarbaraM.(2003).Recommendationsfordevelopingclassroomperformanceassessmentsand

scoringrubrics.PracticalAssessment,Research &Evaluation,8(14).RetrievedJuly23,2011fromhttp://PAREonline.net/getvn.asp?v=8&n=14.

293_performance assessment, bordin chinda

Documents