AutomaticTextAnalysisinReadingResearchanditsuseinReadingAssessment

AylinÜnaldı

IATEFLTEASIGConferenceLondon,1-2November

RIPProf.CyrilWeir

Issues

•  Validityinreadingassessmentfacetsofcontextualvalidity

•  Conceptoftextualcomplexity•  Howautomatictoolsareusedindeterminingtextualcomplexity

•  Howtheinformationfromautomatictoolscanguideusintheassessmentandvalidationprocedures

2

Textselection

•  ImaginethatyouarepreparingareadingtaskforaB2levelEAPexamforyourstudents.

•  Wouldthefollowingexcerptsbesuitableforthatpurpose?Why/Whynot?

3

Whatmakestextsdifficult?•  Text1)ThreeusesoftheInternetwillbebrieflyoutlinedhere.Thefirstcanbe

foundinusingtheInternettogainstraightforwardaccesstodataonallmannerofworldwideissues.Byusingasearchengine(suchasGoogle)andtypinginkeywords,youwillsoongainaccesstoworldmaps,librarycatalogues,archivednewspapersandjournals,officialgovernmentdocumentsandallmannerofculturalphenomena.

•  Text2)Aquitedifferentformofpastoralism,oftencallednomadicpastoralism,alsobasedontheherdingofcattleandotherlivestock,cametoflourishinothersettings,notablyinthesteppelandsnorthoftheagriculturalzoneofsouthernEurasia.Althoughthereissubstantiveuncertaintyovertheextenttowhichclimaticcriseshelpedtheriseofnomadicpastoralismasalifestyle,itisclearthatitwasmorewidespreadincertaingeographicalregionswherethelowrainfallmadefarmingimpractical.

•  Text3)Ifwemakeapromise,forexample,weputourselvesinasituationinwhichthedutytokeeppromisesisamoralconsideration.Ithaspresumptiveforce;andifnoconflictingapparentdutyisrelevant,thenthedutytokeepourpromisesautomaticallybecomesanactualduty.Whataboutsituationsofconflict?Fromonepointofview,anadequatemoralsystemcanneverproducemoralconflict,norcanabasicmoralprinciplebeoverriddenbyanothermoralprinciple.

4

Constructvalidity(Weir,2005;KhalifaandWeir,2009)

5

Task+Response

ContextualParameters

ContextValidity(Weir,2005)

6

Intestpreparation:Whatarethecontextualfeaturesweneedtotakeintoconsiderationwhenwejudgethesuitabilityoftextsforassessmentpurposes?Howdowesetthelevelsofcomplexityintextstargetedatdifferentproficiencylevels?

Circulardefinitionsofcomplexity(1)  “[complexityisthe]useofmorechallenginganddiffcult

language...Complexityistheextenttowhichlearnersproduceelaboratedlanguage”(R.Ellis&Barkhuizen2005:139)

(2)  “Grammaticalandlexicalcomplexitymeanthatawidevarietyofbothbasicandsophisticatedstructuresandwordsareavailabletothelearner”(Wolfe-Quintero,Inagaki,&Kim1998:69,101)

(3)  “Complexityrefersto...thecomplexityoftheunderlyinginterlanguagesystemdeveloped”(Skehan2003:8)

(BulteandHousen,2012)7

ComplexityTaxonomy(Bulte&Housen,2012)

8

«…complexityreferstoapropertyorqualityofaphenomenonorentityintermsof(1)thenumberandthenatureofthediscretecomponentsthattheentityconsistsof,and(2)thenumberandthenatureoftherelationshipsbetweentheconstituentcomponents.»

Inventoryoflinguisticcomplexitymeasures(Bulte&Housen,2012)A.GRAMMATICALCOMPLEXITY

a.Syntactic•  i.Overall•  MeanlengthofT-unit•  Meanlengthofc-unit•  Meanlengthofturn•  MeanlengthofAS-unit•  Meanlengthofutterance•  S-nodes/T-unit•  S-nodes/AS-unit•  ii.Sentential–Coordination•  Coordinatedclauses/clauses•  iii.Sentential–Subordination

–  Clauses/AS-unit–  Clauses/c-unit–  Clauses/T-unit–  Dependentclauses/clause–  NumberofSubordinate

clauses–  Subordinateclauses/clauses–  Subordinateclauses/

dependentclauses–  Subordinateclauses/T-unit–  Relativeclauses/T-unit–  Verbphrases/T-unit

•  iv.Subsentential(Clausal+Phrasal)

•  18.Meanlengthofclause•  19.S-nodes/clause•  v.Clausal•  Syntacticarguments/clause•  vi.Phrasal•  Dependents/(noun,verb)

phrase•  vii.Other(±syntactic

sophistication)•  20.Frequencyofpassiveforms

21.Frequencyofinfinitivalphrases

•  22.Frequencyofconjoinedclauses

•  23.FrequencyofWh-clauses24.Frequencyofimperatives25.Frequencyofauxiliaries26.Frequencyofcomparatives27.Frequencyofconditionals

•  b.Morphological•  i.Infectional•  Frequencyoftensedforms•  Frequencyofmodals

•  Numberofdierentverbforms•  Varietyofpasttenseforms

•  ii.Derivational•  Measureofaffixation

•  B.LEXICALCOMPLEXITY•  a.Diversity•  Numberofwordtypes•  TTR•  MeansegmentalTTR•  GuiraudIndex•  (Wordtypes)2/words•  D•  b.Density•  Lexicalwords/Functionwords•  Lexicalwords/Totalwords•  c.Sophistication•  40.Lessfrequentwords/Total

words

9

Syntacticcomplexitymeasures(Lu,2010)•  Lengthofproduction

1.  Meanlengthofclause(MLC)2.  Meanlengthofsentence(MLS)3.  MeanlengthofT-unit(MLT)

•  Sentencecomplexity4.  Meannumberofclausespersentence(C/S)

•  Subordination5.  MeannumberofclausesperT-unit(C/T)6.  MeannumberofcomplexT-unitsperT-unit(CT/T)7.  Meannumberofdependentclausesperclause(DC/C)8.  MeannumberofdependentclausesperT-unit(DC/T)

•  Coordination9.  Meannumberofcoordinatephrasesperclause(CP/C)10. MeannumberofcoordinatephrasesperT-unit(CP/T)11. MeannumberofT-unitspersentence(T/S)

•  Particulargrammaticalstructures12. Meannumberofcomplexnominalsperclause(CN/C)13. MeannumberofcomplexnominalsperT-unit(CN/T)14. MeannumberofverbphrasesperT-unit(VP/T)

10

Cohmetrixindices:McNamara,Graesser,McCarthyandCai(2014)http://tool.cohmetrix.com/• Number Label LabelV2.x Text Text2 Fulldescription

• 1 DESPC READNP 1 1 Paragraphcount,numberofparagraphs• 2 DESSC READNS1 11 8 Sentencecount,numberofsentences• 3 DESWC READNW 191 206 Wordcount,numberofwords• 4 DESPL READAPL 11 8 Paragraphlength,numberofsentencesinaparagraph,mean• 5 DESPLd 0 0 Paragraphlength,numberofsentencesinaparagraph,standarddeviation• 6 DESSL READASL 17.364 25.75 Sentencelength,numberofwords,mean• 7 DESSLd n/a 9.470 8.246 Sentencelength,numberofwords,standarddeviation• 8 DESWLsyREADASW 1.571 1.718 Wordlength,numberofsyllables,mean• 9 DESWLsyd 0.810 1.040 Wordlength,numberofsyllables,standarddeviation• 10 DESWLlt 4.780 5.505 Wordlength,numberofletters,mean• 11 DESWLltd 2.560 2.921 Wordlength,numberofletters,standarddeviation

• 12 PCNARz n/a -1.154 -1.042 TextEasabilityPCNarrativity,zscore• 13 PCNARpn/a 12.510 14.920 TextEasabilityPCNarrativity,percentile• 14 PCSYNz n/a 0.779 -0.334 TextEasabilityPCSyntacticsimplicity,zscore• 15 PCSYNp n/a 77.940 37.070 TextEasabilityPCSyntacticsimplicity,percentile• 16 PCCNCz n/a -0.649 1.697 TextEasabilityPCWordconcreteness,zscore• 17 PCCNCpn/a 26.110 95.450 TextEasabilityPCWordconcreteness,percentile• 18 PCREFz n/a -1.872 -0.416 TextEasabilityPCReferentialcohesion,zscore• 19 PCREFp n/a 3.070 34.090 TextEasabilityPCReferentialcohesion,percentile• 20 PCDCz n/a -0.015 -0.036 TextEasabilityPCDeepcohesion,zscore• 21 PCDCp n/a 49.600 48.800 TextEasabilityPCDeepcohesion,percentile• 22 PCVERBzn/a -0.279 -2.506 TextEasabilityPCVerbcohesion,zscore• 23 PCVERBp 38.970 0.620 TextEasabilityPCVerbcohesion,percentile• 24 PCCONNz -3.123 -4.501 TextEasabilityPCConnectivity,zscore• 25 PCCONNp 0.090 0 TextEasabilityPCConnectivity,percentile• 26 PCTEMPz -2.841 0.952 TextEasabilityPCTemporality,zscore• 27 PCTEMPp 0.230 82.890 TextEasabilityPCTemporality,percentile

11

descriptives

TextEasibility

Cohmetrixindices• 28 CRFNO1 CRFBN1um 0.100 0.429 Nounoverlap,adjacentsentences,binary,mean• 29 CRFAO1 CRFBA1um0.100 0.100 0.571 Argumentoverlap,adjacentsentences,binary,mean• 30 CRFSO1 CRFBS1um 0.5 0.714 Stemoverlap,adjacentsentences,binary,mean• 31 CRFNOa CRFBNaum 0.164 0.321 Nounoverlap,allsentences,binary,mean• 32 CRFAOa CRFBAaum 0.255 0.464 Argumentoverlap,allsentences,binary,mean• 33 CRFSOa CRFBSaum 0.309 0.070 Stemoverlap,allsentences,binary,mean• 34 CRFCWO1 CRFPC1um 0.050 0.079 Contentwordoverlap,adjacentsentences,proportional,mean• 35 CRFCWO1d 0.073 0.079 Contentwordoverlap,adjacentsentences,proportional,standarddeviation• 36 CRFCWOa CRFPCaum 0.040 0.056 Contentwordoverlap,allsentences,proportional,mean• 37 CRFCWOad 0.072 0.066 Contentwordoverlap,allsentences,proportional,standarddeviation

• 38 LSASS1 LSAassa 0.140 0.428 LSAoverlap,adjacentsentences,mean• 39 LSASS1d LSAassd 0.156 0.134 LSAoverlap,adjacentsentences,standarddeviation• 40 LSASSp LSApssa 0.115 0.406 LSAoverlap,allsentencesinparagraph,mean• 41 LSASSpd LSApssd 0.107 0.126 LSAoverlap,allsentencesinparagraph,standarddeviation• 42 LSAPP1 LSAppa 0 0 LSAoverlap,adjacentparagraphs,mean• 43 LSAPP1d LSAppd 0 0 LSAoverlap,adjacentparagraphs,standarddeviation• 44 LSAGN LSAGN 0.229 0.363 LSAgiven/new,sentences,mean• 45 LSAGNd 0.125 0.167 LSAgiven/new,sentences,standarddeviation

• 46 LDTTRc TYPTOKc 0.841 0.823 Lexicaldiversity,type-tokenratio,contentwordlemma• 47 LDTTRa n/a 0.635 0.644 Lexicaldiversity,type-tokenratio,allword• 48 LDMTLD LEXDIVTD 103.284 117.494 Lexicaldiversity,MTLD,allword• 49 LDVOCD LEXDIVVD 100.030 101.993 Lexicaldiversity,VOCD,allword

• 50 CNCAll CONi 94.241 111.650 Allconnectivesincident• 51 CNCCaus CONCAUSi 20.942 14.563 Causalconnectivesincident• 52 CNCLogic CONLOGi 36.649 33.981 Logicalconnectivesincident• 53 CNCADC CONADVCONi 10.471 14.563 Adversativeandcontrastiveconnectivesincident• 54 CNCTemp CONTEMPi 20.942 29.126 Temporalconnectivesincident• 55 CNCTempxCONTEMPEXi 10.471 9.709 Expandedtemporalconnectivesincident• 56 CNCAdd CONADDi 62.827 77.670 Additiveconnectivesincident• 57 CNCPos n/a 0 0 0 Positiveconnectivesincident• 58 CNCNeg n/a 0 0 0 Negativeconnectivesincident• 59 SMCAUSv CAUSV 31.414 24.272 Causalverbincident• 60 SMCAUSvpCAUSVP 36.649 324.272 Causalverbsandcausalparticlesincident• 61 SMINTEp INTEi 20.942 19.417 Intentionalverbsincident• 62 SMCAUSr CAUSC 0.143 0 Ratioofcasualparticlestocausalverb• 63 SMINTEr INTEC 0.800 0.600 Ratioofintentionalparticlestointentionalverb• 64 SMCAUSlsaCAUSLSA 0.124 0.044 LSAverboverlap• 65 SMCAUSwnCAUSWN 0.410 0.466 WordNetverboverlap• 66 SMTEMP TEMPta 0.550 0.929 Temporalcohesion,tenseandaspectrepetition,mean

12

wordoverlapacrosstext

LatentSemanticAnalysis:meaningoverlapsacrosstext

Lexicaldiversity

repetitionofitemsthatcanindicatethetypeoftext/argument:temporalelements(narration),causalelements(cause&effect)…

Cohmetrixindicesofthetwotexts67 SYNLE SYNLE 5 4.75 Leftembeddedness,wordsbeforemainverb,mean68  SYNNPSYNNP 1 1 Numberofmodifierspernounphrase,mean

69  SYNMEDpos 0.715 0.584 MinimalEditDistance,partofspeech70 SYNMEDwrd 0.902 0.877 MinimalEditDistance,allwords71 SYNMEDlem 0.878 0.860 MinimalEditDistance,lemmas72  SYNSTRUTa 0.120 0.076 Sentencesyntaxsimilarity,adjacentsentences,mean73  SYNSTRUTt 0.117 0.106 Sentencesyntaxsimilarity,allcombinations,acrossparagraphs,mean

74 DRNP 350.785 393.204 Nounphrasedensity,incidence75 DRVP 225.131 155.340 Verbphrasedensity,incidence76 DRAP 47.120 43.689 Adverbialphrasedensity,incidence77 DRPP 115.183 150.485 Prepositionphrasedensity,incidence78 DRPVALAGLSPSVi. 15.707 9.709 Agentlesspassivevoicedensity,incidence79 DRNEG DENNEGi 0 4.854 Negationdensity,incidence80 DRGERUND 26.178 24.272 Gerunddensity,incidence81  DRINF INFi 20.942 9.709 Infinitivedensity,incidence

82 WRDNOUN 287.958 320.388 Nounincidence83 WRDVERB 115.184 126.214 Verbincidence84 WRDADJ 99.476 106.796 Adjectiveincidence85 WRDADV 78.534 77.670 Adverbincidence86 WRDPRO 26.178 19.417 2Pronounincidence87 WRDPRP1s 0 0 Firstpersonsingularpronounincidence88 WRDPRP1p 0 0 Firstpersonpluralpronounincidence89 WRDPRP2 20.942 0 Secondpersonpronounincidence90 WRDPRP3s 0 0 Thirdpersonsingularpronounincidence91  WRDPRP3p 0 19.417 Thirdpersonpluralpronounincidence

13

subjectandNPcomplexity

syntacticsimilarity

phrasecomplexity

phraseincidence

Cohmetrixindicesofthetwotexts

92 WRDFRQc 2.060 1.901 CELEXwordfrequencyforcontentwords,mean93 WRDFRQa 2.858 2.803 CELEXLogfrequencyforallwords,mean94  WRDFRQmc 0.885 0 CELEXLogminimumfrequencyforcontentwords,mean

95  WRDAOAc 384.833 326.188 Ageofacquisitionforcontentwords,mean96  WRDFAMc 564.783 542.324 Familiarityforcontentwords,mean97 WRDCNCc 359.018 430.732 Concretenessforcontentwords,mean98 WRDIMGc 393.367 450.784 Imagabilityforcontentwords,mean99 WRDMEAc 404.784 423.431 Meaningfulness,Coloradonorms,contentwords,mean100 WRDPOLc 3.616 3.272 Polysemyforcontentwords,mean101 WRDHYPn 7.385 5.711 Hypernymyfornouns,mean102 WRDHYPv 1.827 1.948 Hypernymyforverbs,mean103 WRDHYPnv 2.273 1.923 Hypernymyfornounsandverbs,mean

104 RDFRE 56.304 35.356 FleschReadingEase105 RDFKGL 9.720 14.725 Flesch-KincaidGradelevel106 RDL2 10.687 5.5 Coh-MetrixL2Readability

14

wordfrequency

otherworddifficultyproperties

readability

USEFULINDICES

•  SomeindicesfromCoh-metrix•  Lexile•  Vocabprofilerv tochecksimilaritiesanddifferencesbetweentextsweuseforoneparticularassessmentpurposeoracrossdifferentpurposeswithconsistency

v todevelopteachingmaterialsv todetermineanchorpointsalongthecurriculum(A2-B1texts…)

v toadapttextsfromdifferentsourcesforreadingexams

15

Shallwetry?1.Type:“typesofliteracyassessment”onGoogle:2.Openthedocumenttitled:«TypesofLiteracyAssessment:Principles,Procedures,andApplications»

3.Copytwoparagraphsunderthesubtitle:«LiteracyAssessment»(YoucanpastetheparagraphsonaWorddocument)4.Type«Vocabprofile»onGoogle

16

TypesofLiteracyAssessment:Principles,Procedures,andApplications…LiteracyAssessment

Tounderstandliteracyassessment,wefirstneedtothinkabouttheterm“literacy,”whichisdiscussedthroughoutthechaptersinthistextbook.Literacyhastraditionallybeenregardedashavingtodowiththeabilitytoreadandwrite.Morerecently,literacyhasevolvedtoencompassmultidimensionalabilitiessuchaslistening,speaking,viewing,andperforming(NGA&CCSSO,2010),alongwithculturalandsocietalfactors(Snow,2002)thatcanfacilitateorconstrainliteracydevelopment.Thismultidimensionaldefinitionofliteracyrequireseducatorsandpolicymakerstoconceptualizeliteracyincomplexways.Controversiesarisewhentherichnessofliteracyisoverlysimplifiedbyassessmentsthatarenotmultidimensionalorauthentic,suchastheoveruseofmultiple-choicequestions.Educatorsmayfindthelackofauthenticityoftheseassessmentsfrustratingwhenresultsdonotappeartorepresentwhattheirstudentsknowandcando.Ontheotherhand,moreauthenticassessmentmethods,suchasobservingstudentswhoaredeliberatingthemeaningoftextsduringgroupdiscussions,donotpreciselymeasureliteracyskills,whichcanlimitthekindsofdecisionsthatcanbemade.Eventhoughtheassessmentofliteracyusingmultiplechoiceitemsversusmoreauthenticproceduresseemslikeopposites,theydohaveanimportantfeatureincommon:theybothcanprovideanswerstoeducationalquestions.Whetheroneapproachismorevaluablethantheother,orwhetherbothareneeded,dependsentirelyonthekindofquestionsbeingasked.Soifsomeoneasksyouifamultiplechoicetestisagoodtestorifobservingastudent’sreadingisabetterassessmentprocedure,youranswerwilldependonmanydifferentfactors,suchasthepurposeoftheassessment,alongwiththequalityoftheassessmenttool,theskillsofthepersonwhoisusingit,andtheeducationaldecisionsneedingtobemade.Thischapterwillhelpyoulearnmoreabouthowtomakedecisionsaboutusingliteracyassessmentsandhowtousethemtoimproveteachingandlearning.

KristenA.Munger:https://courses.lumenlearning.com/literacypractice/chapter/5-types-of-literacy-assessment-principles-procedures-and-applications/

17

18

•  Let’slookatLexile(youneedtoregisterforthat)•  Type“freelexileanalyser”onGoogle.•  Tologinnow:Username:aunaldi@boun.edu.trPassword:happyconference1

19

20

•  Now1.  Type“Cohmetrix”onGoogle2.  ChooseCoh-MetrixWebTool3.  Pastethetwoparagraphs4.  Checkreadabilitystatisticsatthebottomofthe

list

21

22

•  Then1.  Type“Cohmetrix”onGoogle2.ChooseCoh-MetrixTextEasabilityAssessor3.Username:aylinunaldi4.PastethetwoparagraphsandAnalyze

23

24

Acomparison«WhatisSociety:MeaningandDefinitionofSociety»Downloadedfrom:http://www.studylecturenotes.com/social-sciences/sociology/133-what-is-society

WhatisSocietyIndividualisthebasiccomponentofsociety.Theinteractionofindividualswitheachothergivesbirthtogroup.Thesocialgroupsinteractwitheachotheranddeveloprelationshipswitheachother,leadstoasociety.Theplayersinfootballorothergamescametogetherisnotasociety,butjustanaggregateofpeople.Withinthesocietytherearepatternsandgroupingsonthebasisoflikenessanddifferences."Likeness"createsachainofrelationsamongtheindividualshavingsimilarityinoneormoreconditions'likesameprofession,sameresidence,samecaste,familyandkinship,college,age,sexetc."Consciousnessofkindisdevelopedandthepeopleofsimilarinterestsarejoinedtogetherrustlingintheformationofvariousgroupsandcategories.Withoutdifferenceinculturalconditionsofasocietythehumanlifewouldhavebeenmonotonousandprobablylimitedinwhichlittlechangeispredictable.Thesystemofgiveandtakerelationshipcreatesreciprocalrolesinhumanlife.Thesedifferencesleadtovarietyofhumanbehaviorsandsocialdivisionoflabor;theprocessofspecializationisdeveloped.Manisdependentonsocietyforbasicneedssatisfactioni.e.food,protection,education,etc.Therearesocietiesonlocalaswellasonnationallevels.

25

LiteracyAssessment•  K1: 76.31%•  K1+K2: 81.54%•  AWL: 10.46%•  Type-tokenratio: 0.54•  Lexilelevel: 1440•  FleschReadingEase: 29•  Flesch-KincaidGradeLevel:16•  Coh-metrixReadability: 11.76•  Narrativity: 38%•  SyntacticSimplicity: 14%•  WordConcreteness: 14%•  ReferentialCohesion: 69%•  DeepCohesion: 83%

WhatisSociety?•  K1: 81.19%•  K1+K2: 86.14%•  AWL: 9.41%•  Type-tokenratio:

0.59•  Lexilelevel: 1000•  FleschReadingEase: 37.55•  Flesch-KincaidGradeLevel:12•  Coh-metrixReadability: 14.47•  Narrativity: 3%•  SyntacticSimplicity: 53%•  WordConcreteness: 35%•  ReferentialCohesion: 32%•  DeepCohesion: 17%

26

Indices•  K1:•  K1+K2:•  AWL:•  Type-tokenratio:•  Lexilelevel:•  FleschReadingEase:•  Flesch-KincaidGradeLevel:•  Coh-metrixReadability:•  Narrativity:•  SyntacticSimilarity:•  WordConcreteness:•  ReferentialCohesion:•  DeepCohesion:

27

How do we set the standards?

•  Wecanformourowncorpusbasedon:coursematerials;readingtexts,coursebookstargetlevelmaterials(universitycoursebooks,gradedreaders…)alreadysetcriteria(research,CEFRequivalence…)

28

•  McNamaraetal(2014):AutomatedEvaluationofTextandDiscoursewithCoh-Metrix:Cambridge,CUP

29

CEFRLevels(?)From:https://linguapress.com/teachers/flesch-kincaid.htm

30

FleshReadingEase CEFRLevel

50+ C2(HigherEducation)

50-60 C1

60-70 B2

70-80 B1

80-90 A2

90-100 A1

BogaziciUniversityEnglishProficiencyTestSpecifications

SearchReadingPurpose:Searchreading:ResponseMethod:Itemcharacteristics:Numberofitems:Weighting:Discoursemode:(Genreandrhetoricalstructure):Natureofinformation:Contentknowledge:CulturalknowledgeLexicalandstructuralproperties:Textlength: 2100–2200wordsSentencelength: 18–20wordsWordlength: 5–5,2characters%Passive: 20–25%FleschReadingEase:40–42%FKGradeLevel: 12CohmetrixRead.: 10-11LexileLevel: 1150–1250K1: 75–78%K1+K2: 80-85%AWL: 10%Narrativity: <30%(butvaries,i.e.historicaltexts)Syntacticsimplicity: 60–65%Wordconcreteness: variesaccordingtosubjectmatter(i.e.highinhistoricaltexts)Referentialcohesion: 30–40%Deepcohesion: 75–85% Testadministrationprocedure: Totaldurationofthetest:

CarefulReadingPurpose:Carefulreadingatgloballevel:ResponseMethod:Itemcharacteristics:Numberofitems:Weighting:Discoursemode(Genreandrhetoricalstructure):NatureofinformationContentknowledge:Culturalknowledge:Lexicalandstructuralproperties:Textlength: 1900-1930wordsSentencelength: 20-21wordsWordlength: 5-5,2characters%Passive: 20-25%FleschReadingEase: 40-42FKGradeLevel: 12-13CohmetrixRead.: 12-13LexileLevel: 1200-1300K1: 75-80%K1+K2: 80-82%AWL: 10-12%Narrativity: <30%(canchange)Syntacticsimplicity: 50-55%Wordconcreteness: variesaccordingtosubjectmatterReferentialcohesion: 30-40%Deepcohesion: 75-85% Testadministrationprocedure: Totaldurationofthetest:

31

THANKYOUAssist.Prof.AylinÜnaldı

BoğaziçiUniversityDepartmentofForeignLanguageEducation

aunaldi@boun.edu.tr