significant increase across proficiency levels (ω2=.33). · web view2020. 6. 8. · # clauses...

Table 1: Part-of-SpeechSub-category Measure Notes on operationalisation Study Findings

Verbs Verbs # all verbs (tagged with the Biber tagger)

Grant & Ginther 2000*a Increases with essay grade.

Mean frequency of all verbs (tagged with the ICE tagger)

Espada-Gustilo 2011a Significant increase with proficiency level.

Verb forms % verb form types from the total # words

Llanes, Tragant & Serrano 2015

Significant increase over 3-week study abroad programme (d=.09).

Modal verbs # modal verbs (tagged with the ICE tagger)

Espada-Gustilo 2011a No clear increases across proficiency levels. N.B ANOVA is reported as significant but increase is not linear across the 3 proficiency levels.

# modal verbs (tagged with the Biber tagger)


reference to Biber 1988 implies counts are per 1,000 words

Ferris 1994a No significant differences across proficiency levels.

Auxiliary modal verbs # auxiliary modal verbs Torras and Celaya 2001*a Increases over time.

Primary verb types # primary verb types Torras and Celaya 2001*a Increases over time.

Lexical verb types # lexical verb types Torras and Celaya 2001*a Increases over time.

Tense/aspect # present tense verbs(reference to Biber 1988 implies counts are per 1,000 words)

Ferris 1994a Significantly more used by higher proficiency group.

# present tense verbs (tagged with the Biber


tagger)# simple present verb phrases

Verspoor, Schmid & Xu 2012a

Significant negative correlation with essay score (r=-.73).

% present tense verbs Santos, Nerbonne & Verspoor 2012

Amongst the strongest correlates of essay score in pooled model (Ranked 5th out of 8th in strongest correlates).

# simple past verb phrases Verspoor, Schmid & Xu 2012a

Significant positive correlation with essay score (r=.73).

Frequency counts of past-tense verbs

Guo, Crossley & McNamara 2013

Significant negative correlation with holistic essay scores for integrated essays (r=-.17) but no correlation reported for independent essays.

Frequency counts of verbs in non- third person singular present form


Significant negative correlation with holistic essay scores for integrated essays (r=-.34) and independent essays (r=-.44).

Frequency counts of verbs in third person singular present form


Significant positive correlation with holistic essay scores for integrated essays (r=.19) but no correlation reported for independent essays.

# progressive verb phrases Verspoor, Schmid & Xu 2012a

Non-significant positive correlation with essay score (r=.05).

# past tense/perfect aspect Ferris 1994a Significantly more verbs

verbs(reference to Biber 1988 implies counts are per 1,000 words)

produced by higher proficiency group.

# perfect aspect verbs (tagged with the Biber tagger)

Grant & Ginther 2000*a No clear increase across grades.

# present perfect verb phrases


Non-significant negative correlation with essay score (r=-.00).

# past perfect verb phrases Verspoor, Schmid & Xu 2012a


Verbs in base form Frequency counts of verbs in base form


Significant negative correlation with holistic essay score for integrated essays (r=-.40) and independent essays (r=-.28).

# Non-finite verbs # non-finite verb types Torras & Celaya 2001*a Increases over time.

Non-finite verb ratio # non-finite verbs ÷ words Hou, Verspoor & Loerts 2016

Significant increase over time for higher proficiency university group.

Finite verb ratio # finite verbs ÷ words Hou, Verspoor & Loerts 2016

No significant change over time for higher proficiency university group.

Verspoor, Lowie, Chan & Vahtrick (2017, study 1)

Significant positive correlation with holistic writing score (r=.59).

Past participle verbs Frequency counts of past participle verbs


Significant positive correlation with holistic essay scores for integrated

essays (r=.44) and independent essays (r=.46).

Gerund or present participle verbs

Frequency counts of gerunds or present participle verbs.


Significant positive correlation with holistic essay scores for integrated essays (r=.19) but no correlation reported for independent essays.

Nouns # Nouns # nouns (tagged with the ICE tagger)


# noun types Torras & Celaya 2001*a Increases over time.


Ferris 1994a No significant differences across proficiency levels.

adjectives # Adjectives Mean frequency of adjectives (tagged with the ICE tagger)


# adjective types Torras & Celaya 2001*a Increases over time.

# Participials (as part of combined variable)



Lexical density # Adjectives + nouns +verbs/all words

total # lexical words/total # orthographic words x 100

Nihalani 1981 No significant differences between proficiency levels.

total # lexical words/total # words in essay

Engber 1995 Non-significant positive correlation with writing scores (r=.23).

total # content words / total # words per essay

Becker 2010 Significant increase across proficiency levels (ω2=.33).

N.B. No post-hoc differences between the 3 proficiency levels are made clear or reported.

total # content words / total # words per essay

Vidakovic & Barker 2010 No clear differences between CEFR levels.


Kim 2014 Significant positive correlation with proficiency level (η²=.21).


Lorenzo & Rodriguez 2014 No significant difference across school years.

NB. We are assuming that this is the measure used, though the description in the article appears to confuse density and diversity; specifically with type-token ratio.

Gregori-Signes & Clave-Arroitia 2015*

Decreases across time and proficiency level.

Total # lexical words/total # words

Hou, Verspoor & Loerts 2016

Significant increase over time for low proficiency university group and a reported non-significant difference for the high school group and the high proficiency university group.

Content word ratio Zheng 2016 No significant difference over time.

Weighted lexical densityContent words/functions words, with high-frequency items given half the weight

Banerjee, Franceschina & Smith 2007*

Increases across proficiency levels [NB. Though the authors make this claim, it's not clear that the data really

of low-frequency items support it. Significant differences across band levels go down as well as up].

articles #Articles # of articles (tagged with the ICE tagger)


# articles per 100 words Castro 2004 Non-significant increase across proficiency levels.

#Singular definite and indefinite articles

Normed frequency counts per 10,000 words

Taguchi, Crawford & Wetzel 2013*

More frequent in higher rated essays.

# Indefinite articles # indefinite articles (tagged with the Biber tagger)


# Definite articles # definite articles (tagged with the Biber tagger)


# definite articles (tagged with the Biber tagger)


# Singular or plural determiners



More determiners used in lower rated essays.

# Demonstrative determiners



More determiners used in lower rated essays.

# Post-determiners Normed frequency counts per 10,000 words


More determiners used in higher rated essays.

# Pre-qualifiers Normed frequency counts per 10,000 words


More pre-qualifiers used in higher rated essays.

# Pre-quantifiers Normed frequency counts per 10,000 words


More pre-quantifiers used in higher rated essays.

adverbs #Adverbs # Intensifying adverbs Perez-Paredes & Diez-Bedmar 2012a

Significant differences in the use of “very” between adjacent grades only for 7 vs 8, with grades 8 vs 9 or 9 vs

10.Significant differences in the use of “very” between non-adjacent grades 7 vs 9 and grades 7 vs 10

# adverbs (tagged with the Biber tagger)


Mean frequency of (general) adverbs (tagged with the ICE tagger)


# adverb types Torras & Celaya 2001*a Increases over time.

Types of connectors # Connectors Normed counts per 200 words

Shaw & Liu 1998 Significant increase over time.

# Conjunctions # conjunctions per 100 words

Castro 2004 Non-significant decrease across proficiency levels.

Incidence of and Coh-metrix index Crossley, Kyle & McNamara 2016

Significant positive correlation with organisation (r=.13) and overall score (r=.17).

Incidence of intentional connectives

Coh-metrix index Crossley, Kyle & McNamara 2016

Significant negative correlation with overall score (r=-.15).

Incidence of causal connectives


Significant negative correlation with organisation (r=-.16) and overall score (r=-.16).

# Positive connectives Coh-metrix index Crossley, Kyle & McNamara Significant positive

2016 correlation with organisation (r=.24) and with overall score (r=.33).

Significant increase over time (n2=.20)

# Positive intentional connectives


Significant negative correlation with organisation score (r=-.12) but no reporting for overall score.

# Positive causal connectives Coh-metrix index Crossley, Kyle & McNamara 2016

Significant positive correlation with organisation score (r=.21) and overall score (r=.30).

# Negative connectives Coh-metrix index Crossley, Kyle & McNamara 2016

Significant negative correlation with overall score (r=-.11) but no result reported for organisation scores.

pronouns # All pronouns # all pronouns (tagged with the ICE tagger)


# pronouns per 100 words Castro 2004 No clear pattern across proficiency levels.

# Personal pronouns Frequency counts of personal pronouns(not explicitly normalized for text length)


Significant negative correlation with holistic essay scores for integrated essays (r=-.32) and independent essays (r=-.30).

# 1st/2nd person pronouns # 1st/2nd person pronouns(tagged with the Biber tagger)


Grant & Ginther 2000*a Increases with essay grade.# 1st person pronouns (singular)

Normed counts per 200 words

Shaw & Liu 1998 Significant decrease over time.

# 1st person pronouns (plural)

Normed counts per 200 words

Shaw & Liu 1998 No significant decrease over time.

# 3rd person pronouns # 3rd person pronouns(tagged with the Biber tagger)

Ferris 1994a No significant difference across proficiency levels.

Grant & Ginther 2000*a Increases with essay grade.# Impersonal pronouns # impersonal pronouns

(tagged with the Biber tagger)


Pronoun: noun ratio Coh-metrix index Crossley, Kyle & McNamara 2016

Significant negative correlation with overall score (-.15) but no result reported for correlation with organisation score.

Prepositions # Prepositions # all prepositions(tagged with the Biber tagger)


# all prepositions (tagged with the ICE tagger)


# Prepositional phrases Normed per 1,000 words Mazgutova & Kormos 2015 No significant change over time for either the high school or undergraduate group.

Coh-metrix index Guo, Crossley & McNamara 2013

Significant positive correlation with holistic essay scores for integrated

essays (r=.17) but no correlation reported for independent essays.

Number of conjunctions Coh-metrix index Normed incidences of conjunctions

Crossley, Kyle & McNamara 2016

Significant positive correlation with organisation (r=.18) and overall scores (r=.24).

Number of subordinating conjunctions

# Subordinating conjunctions



More conjunctions used in lower rated essays

Normed incidences of simple subordinators (Coh-metrix index)

Crossley, Kyle & McNamara 2016

Significant positive correlation with overall scores (r=.11).

Pooled POS measures Pooled POS measure Pooled POS measure consists of measures from Lu’s (2011) Syntactic Complexity Analyser and count based measures for: nouns (various types), pronouns (various types), adjectives, conjunctions, interjections, determiners, prepositions, verbs (various types)

Vajjala 2018 Significant positive correlation with proficiency level for TOEFL essays (r=.74) and FCE texts (r=.49).

*denotes a study that has no inferential statistics a = denotes a study where features are not explicitly normalised for text length

Table 2: Phrase structure2a: Noun Phrase structure

Measure Notes on operationalisation Study FindingsMean length of noun phrase # words per noun phrase

(manual counts)Bulté & Housen 2014 Significant increase with time (Cohen’s d=.41).

Significant positive correlation with overall essay score (r=.36) and language use score

(r=.37).Lahuerta-Martinez

2018 Significant positive correlation with holistic

essay score (r=.63). Significantly longer noun phrases produced in

year 4 than in year 3 (p=<0.001).Verspoor, Lowie, Chan

& Vahtrick (2017, study 1)

Significant positive correlation with holistic essay score (r=.47).

Number of Modifiers per noun phrase

Coh-metrix index Crossley & McNamara 2014

Significant increase over time (n2=.12). Significant positive correlation with language

use score (r=.21) and combined essay score (r=.21).


Significant positive correlation with holistic essay scores for integrated essays (r=.26) and independent essays (r=.34).

Mazgutova & Kormos 2015

Significant increase across time for the undergraduate group (r=-.48) but no change for the high school group.

Dependents per nominal Average dependents per nominal Kyle & Crossley 2018 Significant positive correlation with holistic essay grade (r=.31)

Number of complex nominals # Complex nominals (SCA index) Lorenzo & Rodriguez 2014

Significant decrease from 10th-11th grade (Hedge's g = .40)

No significant differences between 9th-10th or 11th-12th grades.

Number of complex nominals per clause

# Complex nominals ÷ # clauses (SCA index)


Significant increase over time for senior high school group.

Significant increase over time for lower proficiency university group.

No significant change over time for higher proficiency university group.

Kim 2014 Significant increase with proficiency level.

Kim & Crossley 2018 Non-significant positive correlation with source-based essay scores (r=.11).Significant positive correlation with independent essay scores (rs=.22).

Kyle & Crossley 2017 Significant correlation with essay grade but removed from analysis due to multicollinearity.

Kyle & Crossley 2018 Significant correlation with essay grade but removed from analysis due to multicollinearity with mean length of clause.

Lorenzo & Rodriguez 2014

Significant decrease from grade 10-11 (Hedge’s g=-.44) and grade 11-12 (hedge’s g=-.86).

Non-significant increase from grade 9-10 (Hedge’s g=.28).

Lu 2011 Significant increase from years: 1-2; 2-3; 1-3; 2-4; 1-4.

Paquot 2018 No significant differences across proficiency levels.

Shih & Ma 2012 Significant differences between grades 7-11, 8-11, 9-11 and 7-12.

Yang, Lu & Weigle 2015

Significant positive correlation with writing quality scores for ‘appearance’ topic (r=.12) and ‘future’ topic (r=.20).

Yau 1991 Higher ESL proficiency group produce significantly more complex nominals per clause than lower ESL proficiency group.No significant increase in complex nominals per clause between higher ESL proficiency group and L1 native speaker group.

Yoon & Polio 2016 No significant increase over time for narrative and argumentative essays.

Yoon 2017 Significant differences between proficiency levels (η2=0.03).

Post-hoc tests reveal non-significant differences between adjacent proficiency levels and non-adjacent B1.1-B2 levels.

Significant differences between non-adjacent levels CEFR A2-B1.2 and A2-B2.

Number of complex nominals per t-unit

# Complex nominals ÷ # t-units (SCA index)



Significant increase over time for lower proficiency university group.

Significant decrease over time for higher proficiency university group.

Kim 2014 Significant increase with proficiency level (η2 =.23)Kim & Crossley 2018 Non-significant positive correlation with source-based

essay scores (rs=.09) and independent essay scores (rs=.10).

Kyle & Crossley 2018 Removed from analysis as it either violated normality assumption or failed to reach a small effect size for correlation with essay score.


Significant decrease from grade 10-11 (Hedge’s g=-.33) and grade 11-12 (Hedge’s g=-.72).Non-significant increase from grade 9-10 (Hedge’s g=.16).

Lu 2011 Significant increase from years: 1-2; 1-3; 1-4Paquot 2018 No significant differences across proficiency levelsYoon & Polio 2016 No significant increase over time for narrative and

argumentative essays.# Complex nominals ÷ # t-units (manual counts)

Shih & Ma 2012 Significant differences between grades 7-10, 8=10, 7-11, 8-11, 9-11, 7-12 and 8-12.

Yau 1991 Higher ESL proficiency group produce significantly more complex nominals per t-unit than lower ESL proficiency group.

No significant increase in complex nominals per t-unit between higher ESL proficiency group and L1 native speaker group.

Number of ‘Of phrases’ as post-modifiers (concrete/locative meaning)

# per noun phrase Parkinson & Musgrave 2014

No significant difference between MA and EAP groups.

Number of ‘Of phrases’ as post-modifiers (abstract meaning)



Number of participial pre-modifiers


Significantly more pre-modifiers used by MA group than EAP group.

Number of appositive noun phrases as post-modifiers


Significantly more noun phrases used by MA level than EAP group.

To clauses as post-modifiers # per noun phrase Parkinson & Musgrave 2014


Simple postmodifiers NPs with one postmodifier only. Normed per 1,000 words


No significant change across time.

Complex postmodifiers NPs with more than one postmodifier only. Normed per 1,000 words


Significant increase over time for undergraduate group (r=-.37) but no significant change for high school group.

Number of complex nominals in subject position

Mean # complex nominals in subject position


No significant change over time for either the undergraduate or high school groups.

Dependents per object of the preposition

Average dependents per object of the preposition

Kyle & Crossley 2018 Significant positive correlation with holistic essay grade (r=.30).

Prepositions per nominal Prepositions per nominal Kyle & Crossley 2018 Significant positive correlation with holistic essay grade (r=.29).

Prepositions per object of preposition

Prepositions per object of preposition


Adjectival modifiers per object of Adjectival modifiers per object of Kyle & Crossley 2018 Significant positive correlation with holistic essay

preposition preposition grade (r=.26).Dependents per object per preposition

Dependents per object per preposition


Dependents per nominal (SD) Standard deviation of dependents per nominal


Dependents per direct object (SD) Standard deviation of dependents per direct object


Dependents per direct object Average dependents per direct object


Dependents per nominal subject Dependents per nominal subject Kyle & Crossley 2018 Significant positive correlation with holistic essay grade (r=.21).

Determiners per nominal Determiners per nominal Kyle & Crossley 2018 Significant positive correlation with holistic essay grade (r=.15).

Adjectival modifiers per direct object

Adjectival modifiers per direct object


Number of attributive adjectives # Attributive adjectives (tagged with the Biber tagger)

Grant & Ginther 2000*a

Increases with essay grade.

Normed per 100,000 words Staples & Reppen 2016

Significant positive correlation with language score for L1 Chinese (r=.24) but not L1 Arabic (r=.16) writers.


Significantly more adjectives used by EAP group than MA group.



More used in high rated essays.

Number of common attributive adjectives


Significantly more attributive adjectives used by EAP group than MA group.

Number of less common attributive adjectives


Significantly more less common adjectives used by EAP group than MA group.

Number of Pre-modifying nouns # pre-modifying nouns Grant & Ginther 2000*a


Normed per 100,000 words Staples & Reppen 2016

Significant positive correlation with language score for L1 Arabic (r=.23) but not L1 Chinese (r=-.10) writers.


Significantly more pre-modifying nouns used by MA group than EAP group.

Possessive nouns as pre-modifiers # per noun phrase Parkinson & Musgrave 2014

Significantly more possessive nouns used by MA group than EAP group.


2b: Verb phrase/clause structureMeasure Notes on

operationalisationStudy Findings

Number of verb phrases # Verb phrases Crossley & McNamara 2014

Significant decrease with time (η2=.10). Significant negative correlation with language use score

(r=-.23) and combined essay score (r=-.24).

# Verb phrases (SCA index)


Significant decrease from grade 9-10 (Hedge’s g=-.40). Non-significant decrease from grade 10-11 (Hedge’s g=

-.10).Significant increase from grade 11-12 (Hedge’s g=.51).

Number of conditional verb phrases

# Conditional verb phrases



Number of passive verb phrases

# Passive verb phrases Verspoor, Schmid & Xu 2012a


Verb phrase density Incidence of verb phrase scores

Aryadoust 2016 Predicts language scores only at pre-course essay stage. Reported as having greatest score predictive power at pre

and post course levels.

Verb argument construction: Delta P

TAASSC index Kyle & Crossley 2017 Significant positive correlation with holistic essay score (r=.25).

Verb argument construction: faith score

TAASSC index Kyle & Crossley 2017 Significant positive correlation with holistic essay score (r=.17).

Average verb lemma construction frequency

TAASSC index Kyle & Crossley 2017 Significant negative correlation with holistic essay score (r= -.23).


2: AdverbialsMeasure Notes on operationalisation Study Findings

# Adverbials # adverbials (tagged with the Biber tagger)

Ferris 1994a No significant differences across proficiency levels

# Adverb wh- # adverbs (wh adverbs) (tagged with the ICE tagger)

Espada-Gustilo 2011a Significant increase across proficiency levels

# Adverbial clauses # of adverbial clauses (manual counts)

Kameen 1979 Non-significant differences between ‘good’ and ‘poor’ writers.

Ratio of adverbial finite clauses

% of adverbial finite clauses over all clause types. Adverbial finite clause is taken to be a clause with a finite verb functioning as an adverbial


Non-significant negative correlation with holistic essay score (r=-.29).

Adverbial subordination # adverbials that indicate subordination (e.g. ‘because she was late’)(tagged with the Biber tagger)


Ratio of adverbial clauses to secondary clauses

# adverbial clauses ÷ # secondary clauses (manual counts)


# Causative adverbial clauses

# causative adverbial clauses normed per 100,000 words

Staples & Reppen 2016 No significant correlations with language ratings for L1 Arabic or L1 Chinese groups

# Conditional adverbial clauses

# conditional adverbial clauses normed per 100,000

Staples & Reppen 2016 Significant positive correlation with language

words rating for L1 Chinese (r=.24) but not L1 Arabic (r=-.08) writers.

normed per 1,000 words Mazgutova & Kormos 2015 Significant increase over time for undergraduate group (r=-.44) but no significant change for high school group.

# if clauses Normed count per 200 words

Shaw & Liu 1998 No significant increase over time.

# Other adverbial clauses # other adverbial clauses normed per 100,000 words

Staples & Reppen 2016 Significant positive correlation with language rating for L1 Chinese (r=.24) but not L1 Arabic (r=-.04) writers.

# Finite adverbial clauses # finite adverbials Verspoor, Schmid & Xu 2012a

Significant positive correlation with essay score (r=.38)

2d: Prepositional phrase structureMeasure Notes on operationalisation Study Findings

Number of prepositional phrases # Prepositional phrases (Coh-metrix index)

Crossley & McNamara 2014

Non-significant increase over time (η2 =.06).

Non-significant positive correlation with language use score (r=.18) and combined essay score (r=.23).

# Prepositional phrases (tagged with the Biber tagger)

Ferris 1994a Significantly more prepositional phrases used by higher proficiency group.

Number of total prepositional phrases with a locative/concrete meaning


No significant difference between MA group and EAP group.

Number of total prepositional phrases with abstract meaning


Significantly more prepositional phrases used by MA group than EAP group.

Number of post noun modifying prepositional phrase



More prepositional phrases used in higher rated essays.

Number of prepositional phrases with prepositions other than ‘of’ (concrete/locative meaning)


No significant difference between MA group and EAP group.

Number of prepositional phrases with prepositions other than ‘of’ (abstract meaning)


Significantly more prepositional phrases used by MA group than EAP group.


Table 3: Clause structureSub-category Measure Notes on operationalisation Study Findings

Number of words before the main verb

# Words before main verb Coh-metrix index Crossley & McNamara 2014 Significant increase over a semester ¿¿=.09).

Non-significant positive correlation with language use scores (r=.12) and combined essay score (r=.17).

Diversity of clauses # Different clauses # different clauses Casanave 1994 Non-linear increase over time.

All complement clauses # Complement clauses # that complement and infinitive complements(features tagged with the ICE tagger)

Espada-Gustilo 2011a Significant increase across proficiency levels.

Noun clauses # Noun clauses # noun clauses (manual counts)


Nominal clause ratio Ratio of nominal clauses to all secondary clauses (manual counts)


# Finite nominals clauses # finite nominal clauses Verspoor, Schmid & Xu 2012a


Ratio of finite nominal clauses

% finite nominal clauses over all clause types. A finite nominal clause is taken to be a clause with a finite verb functioning as a nominal


Non-significant positive correlation with holistic essay score (r=.21).

# Nominal subjects per clause

Kyle & Crossley 2018 Significant negative correlation with holistic essay score (r=-.17).

Relative clauses # Relative clauses # relative clauses (tagged with the Biber tagger)

Ferris 1994a Significantly more relative clauses used by higher proficiency group.

normed per 1,000 words Mazgutova & Kormos 2015 Significant increase over time for undergraduate group (r=-.37) but no significant change for high school group.

# per noun phrase Parkinson & Musgrave 2014 Significantly more relative clauses used by MA group than at EAP group.



# relative clauses (tagged with the ICE tagger)

Espada-Gustilo 2011a Significant increase across proficiency levels.

Ratio of relative clauses to all clause types.



# Subject, object and preposition relative clauses

# relative clause types (tagged with the Biber tagger)


# That relative clauses Normed frequency counts per 10,000 words


More relative clauses used in lower rated essays.

Normed frequency counts per 200 words


# Wh relative clauses Normed frequency counts per 10,000 words


Slightly more relative clauses used in lower rated essays.

Normed frequency counts per 200 words


# Subject relative clauses Coh-metrix index Crossley & McNamara 2014 Non-significant increase over time (η2 =.04).

Non-significant positive correlation with language use score (r=.08) and combined essay score (r=.10).

Finite complement clauses # Noun + that clauses per noun phrase Parkinson & Musgrave 2014 No significant difference between MA level and EAP groups.

Normed # per 100,000 words

Staples & Reppen 2016 No significant correlation with language rating for L1 Chinese or L1 Arabic writers.



Slightly more clauses used in higher rated essays.

Normed count per 200 words

Shaw & Liu 1998 Significant increase over time.

# ‘Of + ing’ noun complement clauses

# per noun phrase Parkinson & Musgrave 2014 No significant difference between MA level and EAP groups.

# Verb + that clauses Normed # per 100,000 words

Staples & Reppen 2016 No significant correlation with language rating for L1 Chinese or L1 Arabic writers



More clauses used in higher rated essays.

That clause adjective complements

# That clause adjective complements (manual counts)



More clauses used in lower rated essays.

Adjectival clauses = relative clauses)

# Adjectival clauses (manual counts)

# adjectival clauses (manual counts)


ratio of adjectival clauses to secondary clauses (manual counts)


Non-finite clauses # Non-finite clauses # non-finite clauses Verspoor, Schmid & Xu 2012a


# Non-finite elements per clause

# non-finite elements ÷ # clauses

Yang, Lu & Weigle 2015 Non-significant positive correlation with writing quality scores for ‘appearance’ topic (r=.03) and significant positive correlation with writing quality score for ‘future’ topic (r=.20).

# Non-finite constructions Manual frequency counts Verspoor, Lowie, Chan & Vahtrick (2017, study 1)

Non-significant positive correlation with holistic

essay score (r=.13)Participial clauses -‘ed’ participial clauses as

post-modifiers# per noun phrase Parkinson & Musgrave 2014 Significantly more clauses

used by MA group than EAP group.

‘ing’ participial clauses as post-modifiers

# per noun phrase Parkinson & Musgrave 2014 No significant difference between MA and EAP group.


Table 4: CoordinationSub-category Measure Notes on operationalisation Study Findings

Indices of coordination

Coordination index # of independent clauses ÷ # of combined clauses

Bardovi-Harlig 1992*

Decreases as proficiency level increases.

Casanave 1994 Non-linear increase over time

Gene-Gil, Juan-Garau & Salazar-Noguera 2015

Significant increase for CLIL group (p=0.01) but not for the non-CLIL group over 3 school years.

All Coordination

Implies coordination combines phrasal and clausal coordination

# clauses and phrases that indicate coordination (tagged with the Biber tagger)

Ferris 1994a No significant differences across proficiency levels

# clauses and phrases that indicate coordination (tagged with the ICE tagger)

Espada-Gustilo, 2011a

Significant increase across proficiency levels

Sentence coordination

Ratio of sentences containing coordination

Ratio of sentences beginning with ‘and’, ‘but’, ‘or’ to all sentences (manual counts)


# Sentences containing coordination

# sentences containing coordination (manual counts)


Phrasal coordination

# Coordinate phrases per clause

SCA index Hou, Verspoor & Loerts 2016


No significant change over time for lower proficiency university group.


Kim 2014 No significant difference across proficiency levels.

Kim & Crossley 2018

Non-significant positive correlation with source-based essay scores (rs=.03).

Significant positive correlation with independent essay scores (rs=.19).

Kyle & Crossley 2017

Significant positive correlation with holistic essay score (r=.19)



Lu 2011 Significant increase from year: 1-3; 1-4 Paquot 2018 No significant differences across proficiency levels


Non-significant positive correlation with essay scores for ‘appearance’ topic (r=.03) and significant correlation with essay scores for ‘future’ topic (r=.21).

Yoon & Polio No significant increase over a semester for

2016 narrative and argumentative essays.

Yoon 2017 Significant differences between proficiency levels (η2=0.02)

# coordinate phrases per clause (manual counts)

Shih & Ma 2012 Significant differences between grades 7-10, 8-7 and 10-7.

# Coordinate phrases per T-unit

# coordinate phrases ÷ # t-units (SCA index)





Lu 2011 Significant increase from years: 1-3.Kim 2014 Significant increase with proficiency level.Kim & Crossley 2018

Non-significant positive correlation with independent essay scores (rs =.12).Non-significant negative correlation with source-based essay scores (rs= -.03).


Measure either did not meet normality assumption or its effect size did not meet the minimum ‘small’ effect size required for further analysis.


Measure either did not meet normality assumption or its effect size did not meet the minimum ‘small’ effect size required for further analysis.

Paquot 2018 No significant differences across proficiency levels

Yoon & Polio 2016

No significant increase over time for narrative and argumentative essays.

# coordinate phrases ÷ # t-units (manual counts)

Shih & Ma 2012 Significant differences between grades 7-10, 8-7 and 10-7.

# Coordinate phrases # coordinate phrases (SCA index) Lorenzo & Non-significant decrease from grade 9-10

Rodriguez 2014 (Hedge’s g=.-.09). Non-significant decrease from grade 10-11

(Hedge’s g=.-.10). Significant increase from grade 11-12

(Hedge’s g=.37).Clausal coordination

# Coordinate clauses # coordinate clauses Torras and Celaya 2001*

Increases over time.

coordinate clause ratio # of coordinate clauses ÷ # sentences

Bulté & Housen 2014

Significant increase over a semester (Cohen’s d=.31).

Non-significant positive correlation with overall essay score (r=.11) and language use score (r=.09).

Lahuerta-Martinez 2018

Significant positive correlation with holistic essay scores (r=.38).

Significantly more clausal coordination in year 4 than year 3 texts (p=<0.001).


Table 5: SubordinationMeasure Notes on operationalisation Study Findings

Subordinate clause ratios

# Dependent clauses ÷ # clauses (manual counts)

Banerjee, Franceschina & Smith, 2007*

Non-linear increase with proficiency level.

Santos, Nerbonne & Verspoor 2012

Amongst the strongest correlates of essay score (Ranked 4th out of 8 strongest correlates in pooled prediction model).

Bulté & Housen 2014 Non-significant increase over time (Cohen’s d=.04) Significant positive correlation with holistic essay score

(r=.24) and language use score (r=.29).

Hirano 1991 Significant positive correlation with test scores: texts with specified audience (r=.48); texts with no specified audience (r=.32).

Homburg 1984 Overall significant increase across levels but individual level comparisons are not significant.

Kameen 1979 Non-significant differences between ‘good’ and ‘poor’ writers.Knoch, Rouhshad, Oon & Storch 2015

No significant change over time.



Significantly more dependent clauses in year 4 texts than year 3 texts (p=<0.001).

Shih & Ma 2012 Significant differences between grades 7-10, 8-10, 9-10, 7-11, 8-11, 7-12 and 8-12.

Storch & Hill 2008 Significant increase over time.Storch 2009 No significant change over time.Struc & Wood 2011 Significant increase over time for narrative essays.

No change over time for argumentative essays.SCA index Hou, Verspoor &

Loerts 2016 No significant change over time for senior high school

group or lower proficiency university group. Significant decrease over time for higher proficiency

university group.Kim 2014 Significant increase with proficiency level. Lorenzo & Rodriguez 2014

Non-significant decrease from grade 9-10 (Hedge’s g=-.13).


Significant decrease from grade 11-12 (Hedge’s g=-.47).

Lu 2011 Significant decrease from years 2-4.Paquot 2018 No significant differences across proficiency levels.Yoon & Polio 2016 No significant increase over a semester for narrative and

argumentative essays.# Dependent clauses ÷ # t-units (SCA index)


No significant change over time for senior high school group or lower proficiency university group.


Kim 2014 Significant increase with proficiency level.Lorenzo & Rodriguez 2014




Lu 2011 Significant decrease from years 2-4.Mazgutova & Kormos 2015

No significant change across time for either the undergraduate or high school groups.



Significant positive correlation with essay score for ‘appearance’ topic (r=.14) and ‘future’ topic (r=.08).

Yoon & Polio 2016 No significant increase over time for narrative and argumentative essays.

# Dependent clauses ÷ # t-units Shih & Ma 2012 Significant differences between grades 7-10, 8-10, 9-10, 7-11, 8-

(manual counts) 11, 7-12 and 8-12.Struc & Wood 2011 Non-significant increase over time for narrative and

argumentative essays.

Overall subordination

# Complement clauses, relative clauses and adverbial subordination



Embedded clauses Coh-metrix index Guo, Crossley & McNamara 2013

Significant negative correlation with holistic essay scores for independent essays (r=-.34) but no correlation reported for integrated essays.

Number of subordinate clauses

# Subordinate clauses Torras & Celaya 2001a Increases over time for late-starter group only.Espada-Gustilo 2011a Significant increase across proficiency levels.

S-bars Incidences of S-bars Crossley & McNamara 2014

Non-significant development over time ¿¿=.00). Non-significant negative correlation with language use

score (r=-.09) and combined essay score (r=-.13).Infinitives Incidences of infinitives Crossley & McNamara

2014 Non-significant increase over time (η2=.00). Significant positive correlation with language use score

(r=.23) and combined essay score (r=.32).Infinitive clause per 1,000 words Mazgutova & Kormos

2015Significant decrease over time for the high school group but no significant change over time for undergraduate group.


Table 6: Information structureSub-category Measure Notes on operationalisation Study Findings

Passive measures

# Passive sentences # sentences with one or more passives

Kameen 1979 Significantly more passives used by 'good' writers

Kawata 1992 Significant differences between proficiency levels

# Passives per sentence # passives ÷ # sentences Kameen 1979 Significantly more passives used by 'good' writers

# Sentences with at least 1 passive # sentences with at least 1 passive Kameen 1979 Significantly more passives used by 'good' writers.

# Passives # by and agentless passives(tagged with the Biber tagger)

Ferris 1994a Significantly more passives used by higher proficiency group.



# by and agentless passives(tagged with the ICE tagger)

Espada-Gustilo, 2011a

Significant increase across proficiency levels.

# passives (manual counts) Verspoor, Schmid & Xu 2012a

Increases across proficiency groups 1-4 (dropping from 4-5).

Normed counts per 200 words Shaw & Liu 1998 Significant increase over time.# Dynamic passives # dynamic passives (manual counts) Kameen 1979 Significantly more passives

used by 'good' writers.# Passives per clause # of passives per clause (manual

counts)Kameen 1979 Significantly more passives

used by 'good' writers.# Passives in main clause # passives in main clauses (manual Kameen 1979 Significantly more passives

counts) used by 'good' writers.# Passives in secondary clauses # passives in secondary clauses

(manual counts)Kameen 1979 Significantly more passives

used by 'good' writers.# Passives per sentence # of passives per sentences (manual


used by 'good' writers.# Passives per T-unit # of passives per T-unit (manual


used by 'good' writers.Active measures

# Actives in secondary clauses # actives in secondary clauses (manual counts)

Kameen 1979 Non-significant differences between ‘good’ and ‘poor’ proficiency writers

# Actives in main clauses # actives in main clauses (manual counts)

Kameen 1979 Non-significant differences between ‘good’ and ‘poor’ proficiency writers

nominalisations Frequency counts of words ending in ‘-ment’, ‘-ity’, ‘-ness’.

Frequency counts (tagged with the Biber tagger)



Mean frequency of nominalisations Frequency counts (tagged with the ICE tagger)

Espada-Gustilo 2011a

Significant increase with proficiency level.


Table 7a: Generic indices: T-unit-based measures

Measure Notes on operationalisation Study FindingsLength in words # Words per T-unit (manual counts) Arnaud 1992 Significant positive correlation with essay

scores (r=.32).

Arthur (1979, study 1) Non-significant decrease between 12 timed essays over time.

Becker 2010 Significant increase between proficiency levels (ω2=.60).

Significant increases between proficiency levels 1-2, 1-3 and levels 2-3.

Bulté & Housen 2014 Significant increase over a semester (cohen’s d=.47).

Significant positive correlation with overall essay score (r=.40) and language use score (r=.43).

Casanave 1994 Increases over time.

Cumming, Kantor, Baba, Erdosy,

Eouanzoui, & James 2005

Significant increase across proficiency levels (η2=.11).

Flahive & Snow 1980 Significantly increases across proficiency levels.Gene-Gil, Juan-Garau

& Salazar-Noguera 2015

For the CLIL group there is a significant increase in t-unit length (p=0.04) and a non-significant increase for the non-CLIL group.

Gyllstad, Granfeldt, Bernardini, & Kallkvist

2014

Significant positive correlation with proficiency level (Kendall's tau=.48)

Hirano 1991 Significant positive correlation with test scores: texts with specified audience (r=.60), texts with

no specified audience (r=.58).Homburg 1984 Significant increase across proficiency levels.

Significant from level 5 to level 6 onlyHo-Peng 1983 Significantly increases across proficiency groups

for both writing tasks.Intaraprawat &

Steffensen 1995*Increase in “good” essays

Ishikawa 1995 No significant change over time.

Kameen 1979 Significantly longer in ‘good’ writers' texts.

Kawata 1992 Significant increase across proficiency levels.Knoch, Rouhshad, Oon

& Storch 2015Significant increase over time (p=0.05).

Knoch, Rouhshad & Storch 2014

Non-significant increase over time.

Larsen-Freeman & Strom 1977

Linear increase across proficiency level. However, none of the increases are statistically significant.

Larsen-Freeman 1978 Significant linear increase across proficiency levels.

Larsen-Freeman 1983 Study 1: Significant increase across proficiency levelsStudy 2: Significant increase across time

Larsen-Freeman 2006 T-unit length increases over time (reported as a group average of 1+ standard deviation over time)

Nihalani 1981 Non-significant increase across proficiency levels.

Perkins 1980 Non-significant differences between

proficiency levels. Non-significant positive correlation

with writing score (r=.33).

Plakans, Gebril and Bilki 2016

Significant positive correlation with integrated essay scores (r=.32).

Shih & Ma 2012 Significant difference between grades 7-10, 8-10, 9-10, 7-11, 8-11 and 9-11.

Storch 2009 No significant change over timeStorch & Tapper 2009 No significant change over time

Struc & Wood 2011 Significant increase over time for narrative essays.

Non-significant increase over time for argumentative essays.

Tedick 1990 Significant increase across proficiency levels for general and field specific topics.

Xudong, Cheng, Varaprasad & Leng

2010


Yau 1991 Higher ESL proficiency group produce significantly longer t-units than lower ESL proficiency group.

No significant increase in mean length of t-unit between higher ESL proficiency group and L1 native speaker group.

Yau & Belanger 1984 Significant increase with proficiency level for narrative and expository essays.

Youn 2014 Significant positive correlation with proficiency level (r s=.42) and pragmatic performance score (r s=.44).

Zyad 2012* Increase over time. Longer t-units used by more proficiency writers

# Words per T-unit (SCA index) Hou, Verspoor & Loerts 2016


No significant change over time for lower proficiency university group


Kim 2014 Significant increase with proficiency level (η2=.24)

Kim & Crossley 2018 Significant positive correlation with source-based essay scores (rs=.16).

Non-significant correlation with independent essay scores (rs =.08).

Lu 2011 Significant increase from years: 1-2; 1-3; 1-4


No significant change across time for either the undergraduate or high school groups.



Significant positive correlation with writing quality for ‘appearance’ topic (r=.21) and ‘future’ topic (r=.22).

Yoon & Polio 2016 Significant increase over a semester for narratives essays (ηρ2=.10) but not argumentative essays.


Post-hoc tests reveal non-significant differences between adjacent proficiency levels.

Significant differences between CEFR A2-B2 and B1.1-B2 levels.

# Long t-units (21 words or more) (manual counts) Kameen 1979 Significantly more long t-units used by good writers

Ratio of long t-units to all t-units (manual counts) Kameen 1979 Significantly higher ratio used by good writersRatio of short t-units to all t-units (short t-units are 1-8 words long) (manual counts)

Kameen 1979 Significantly lower ratio used by good writers

# Short t-units (manual counts) Kameen 1979 Significantly lower number of short t-units used by good writers

# Medium t-units (9-20 words)(manual counts) Kameen 1979 Non-significant differences between ‘poor’ and ‘good’ proficiency writers.

Length in clauses # Clauses ÷ # t-units Banerjee, Franceschina & Smith,

2007*

Non-linear increase with proficiency level

Bardovi-Harlig & Bofman 1989

No significant difference across proficiency groups

Becker 2010 Significant increase across proficiency levels (ω2=.37)

Significant increase between levels 1-2 and 1-3.

Casanave 1994 Non-linear increase over timeCumming, Kantor,

Baba, Erdosy, Eouanzoui, & James

2005

No significant difference across proficiency levels.

Flahive & Snow 1980 Significantly increases across proficiency levelsGene-Gil, Juan-Garau

& Salazar-Noguera 2015

Significant increase over time for CLIL group (p=0.01) although the non-CLIL group also have a non-significant increase in subordination.

Gyllstad, Grenfeldt, Bernadini & Kalkvist

Significant positive correlation with proficiency (Kendall's tau=.46)

2014Hirano 1991 Significant positive correlation with test scores:

texts with specified audience (r=.46), texts with no specified audience (r=.30).


No significant change for senior high school students or lower proficiency university group.


Ishikawa 1995 No significant change over timeKameen 1979 Non-significant differences between ‘good’ and

‘poor’ writers.Kim 2014 Significant increase with proficiency level.

Knoch, Rouhshad, Oon & Storch 2015

No significant change over time.



Larsen-Freeman 2006 Increase in clauses per t-unit over time (reported as a group average of 1+ standard deviation over time)

Llanes, Tragant & Serrano 2015

No significant increase over time on abroad programme.

Lu 2011 No significant differences across year groups.Paquot 2018 No significant differences across proficiency

levelsPerkins 1980 Non-significant differences between

proficiency levels. Non-significant positive correlation

with writing score (r=.10)

Qin & Uccelli 2016 Non-significant negative correlation

with writing quality scores for argumentative essays (r=-.13).

Non-significant positive correlation with writing quality for narrative essays (r=.09).

Shih & Ma 2012 Significant difference between grades 7-10, 8-10, 8-11.

Storch & Hill 2008 Significant increase over timeStorch 2009 No significant change over time

Struc & Wood 2011 Non-significant increase over time for argumentative and narrative essays.

Yau & Belanger 1984 Significant increase over proficiency levels for narratives but not for expository essays.

Youn 2014 Significant positive correlation with proficiency level (r s=.25) and pragmatic performance score (r s=.30).

Yoon 2017 Non-significant differences between proficiency levels

Yoon & Polio 2016 Significant increase over a semester for narrative essays (ηρ2=.18) but not argumentative essays.

Zyad 2012* Increases over time. Longer length in clauses in more proficient texts.

# Verb phrases ÷ # t-units Hou, Verspoor & Loerts 2016

Significant increase over time for high school group.




Non-significant decrease from grade 9-10 (Hedge’s g=-.09)



Lu 2011 No significant development across year groups.Yoon & Polo 2016 Non-significant increase over a semester for

narrative and argumentative essays (ηρ2=.00).Kim 2014 Significant increase with proficiency level.

Kyle & Crossley 2018 Measure either did not meet normality assumption or did not reach small effect size and was therefore removed from analysis.

Lu 2011 No discrimination ability for any of the proficiency levels.


Shih & Ma 2012 Significant differences between grades 7-10, 8-10, 9-10, 10-12, 7-11, 8-11 and 9-11.

Complexity index Sum of weighted scores divided by number of T-units. Weighted score = score of weighted structures plus number of words in t-unit divided by number of words in t-unit:1 = derivational morphemes and adjectives; T-units with no embedded or complex morphological forms2 = relatives, passives, embedded clauses, possessives, comparatives3 = adverbial and noun clauses

Perkins 1980 Non-significant correlation with essay score (r=.19).

Flahive & Snow 1980 Significant increase across proficiency levels.

Complex t-unit ratio (CT/T)

# Complex T-units ÷ # t-units (SCA index) Hou, Verspoor & Loerts 2016

No significant change over time for senior high school group or the lower proficiency university group.

Significant decrease over time for higher proficiency university group

(p=0.001)

Kim 2014 Significant increase with proficiency level.Lu 2011 No significant development across year groups.


Number of complex t-units

% Complex t-units Casanave 1994 Non-linear increase over time # Complex t-units (SCA index) Lorenzo & Rodriguez

2014 Non-significant decrease from grade 9-

10 (Hedge’s g= -.23). Non-significant decrease from 10-11

(Hedge’s g=-.07). Non-significant increase from grade 11-

12 (Hedge’s g=.26).Number of t-units with more than one level of embedding

# t-units with more than one level of embedding Casanave 1994 Non-linear increase over time

Number of t-units joined by (,)

# t-units joined by (,) (manual counts) Kameen 1979 Non-significant differences between ‘good’ and ‘poor’ writers.

Ratio of t-units joined by (,) to all t - units

# t-units joined by (,) ÷ # t-units (manual counts) Kameen 1979 Non-significant differences between ‘good’ and ‘poor’ writers.

Number of t-units joined by ‘and’, ‘but’, ‘or’

# t-units joined by ‘and’, ‘but’, ‘or’ (manual counts) Kameen 1979 Non-significant differences between ‘good’ and ‘poor’ writers.

Ratio of number of t-units joined by ‘and’, ‘but’, ‘or’ to all t-units

Ratio of t-units joined by ‘and’, ‘but’, ‘or’ to all t-units (manual counts)



Table 7b: Generic indices: : clause-based measuresMeasure Notes on

operationalisationStudy Findings

Mean length of clause

# Words ÷ # clauses Casanave 1994 Non-linear increase over timeGyllstad, Grenfeldt, Bernadini & Kalkvist

2014

Significant positive correlation with proficiency (Kendall’s tau=.31)

Hirano 1991 Significant positive correlation with test scores: texts with specified audience (r=.40), texts with no specified audience (r=.38).


Significant increase over time for high school group. Significant increase over time for lower proficiency university group No change over time for higher proficiency university group.

Ishikawa 1995 No significant change over timeKameen 1979 Significantly longer clauses used by good poor writers

Kim 2014 Significant increase with proficiency levelKim & Crossley 2018 Significant positive correlation with source-based essays (r=.14) and independent

essays (r=.25).Knoch, Rouhshad, Oon

& Storch 2015No significant change over time.



Kyle & Crossley 2017 Significant positive correlation with holistic essay score (r=.24).Lorenzo & Rodriguez

2014 Non-significant increase from grade 9-10 (Hedge’s g=.19). Significant decrease from grade 10-11 (Hedge’s g=-.43). Significant decrease from grade 11-12 (Hedge’s g=.-.70).

Lu 2011 Significant increase from years: 1-2; 1-3; 2-4; 1-4.

Paquot 2018 Significant increase across proficiency levels.

Qin & Uccelli 2016 Significant positive correlation with essay scores for argumentative essays (r=.32) and narrative essays (r=.19)

Shih & Ma 2012 Significant difference between grades 9 & 10, and 9-11.

Struc & Wood 2011 Non-significant increase over time for narrative and argumentative essays.

Yang, Lu & Weigle 2015 Non-significant positive correlation with writing quality score for ‘appearance’ topic (r=.13) and significant positive correlation with writing quality score for ‘future’ topic (r=.23).

Yau & Belanger 1984 Significant increase with proficiency level for expository essays but not for narratives.

Yoon & Polio 2016 No significant increase over a semester for narrative and argumentative essays. Yoon 2017 Significant differences between proficiency levels (η2=0.03).

Post-hoc tests reveal non-significant differences between adjacent proficiency levels and non-adjacent CEFR A2-B1.2.

Significant differences between CEFR B1.1-B2 and A2-B2 levels.Youn 2014 Significant positive correlation with proficiency level (r s=.33) and pragmatic

performance score (r s=.30).Zyad 2012* Increases over time. Longer clauses used by more proficiency writers.

Mean length of finite clause

Mean length of finite clause

Bulté & Housen 2014 Significant increase over a semester (Cohen’s d=.50). Significant positive correlation with overall essay score (r=.48) and

language use (r=.49).


Table 7c: Generic indices : sentence-based measuresMeasure Notes on operationalisation Study FindingsMean length of sentence

# Words per sentence Bulté & Housen 2014

Significant increase over time (Cohen’s d=.44).


Espada-Gustilo 2011a

Significant increase with proficiency level.

Homburg 1984 Significant increase from level 5 to level 6 but not from levels 6 to 7.


Significant increase over time for senior high school student group.

No significant change over time for the lower proficiency university group.

Significant decrease over time for higher proficiency university group

Ishikawa 1995 No significant change over time.Kawata 1992 No significant differences between

proficiency levels.Kim & Crossley

2018 Significant positive correlation

with source-based essays (rs =.14).

Non-significant positive correlation with independent essays (rs=.05).

Kim 2014 Significant increase with proficiency

level.Kobayashi & Rinnert 2013

Decreases over time.



Significantly longer sentences produced in year 4 than year 3 (p=<0.001).


Non-significant increase from grade 9-10 (Hedge’s g=-.28).

Significant decrease from grade 10-11 (Hedge’s g=.-46).

Non-significant decrease from grade 11-12 (Hedge’s g=.-14).

Lu 2011 Significant increase from years: 1-2; 1-3; 1-4.


Shih & Ma 2012 Significant difference between grades 8-10 only.

Torras & Celaya 2001*




Vidakovic & Barker 2010

Increases with proficiency level.


Significant correlation with writing quality for ‘appearance’ topic (r=.27)

and ‘future’ topic (r=.25).Yoon & Polio 2016 No significant increase over time for

both narrative and argumentative essays.


t-units per sentence

# T-units per sentence Hou, Verspoor & Loerts 2016

No significant change over time reported for any of the groups.

Homburg 1984 No significant differences across proficiency levels.

Ishikawa 1995 No significant change over time for class 1. Significant increase for class 2.

Lu 2011 No significant difference across year groups.


Shih & Ma 2012 No significant differences between any of the grade levels.

Struc & Wood 2011

Non-significance increase over time for narrative and argumentative essays.


Non-significant positive correlation with writing quality for ‘appearance’ topic (r=.15) and ‘future’ topic (r=.11).

Yoon 2017 Non-significant differences between proficiency levels (η2=0.00).

Yoon & Polio 2016 No significant increases over time for narrative and argumentative essays.

clauses per sentence

# Clauses per orthographic sentence Struc & Wood 2011

Non-significant increase over time for narrative and argumentative essays.

SCA index Hou, Verspoor & Loerts 2016

No significant change over time for the senior high

school or lower proficiency university group.

Significant decrease over time for higher proficiency group.

Ishikawa 1995 No significant change over time for class 1. Significant increase for class 2.

Kim 2014 Significant increase with proficiency level.



Non-significant decrease from grade 10-11 (Hedge’s g= -.26).


Lu 2011 Significant decrease from years: 1-4.Paquot 2018 No significant differences across

proficiency levels.Length of orthographic sentence

Mean length of orthographic sentence Struc & Wood 2011

Significant increase over time for narrative and argumentative essays.

Length of restricted sentence

Mean length of restricted sentence Struc & Wood 2011

Significant increase over time for narrative essays.

Non-significant increase over time for argumentative essays.

Simple sentences

# Simple sentences/sentences Struc & Wood 2011

Increase over time for narrative essays.Decrease over time for argumentative essays.


Significant decrease over a semester (Cohen’s d=.43).

Significant negative

correlation with overall essay score (r=-.43) and language use score (r=-.45).



Year 3 students produce non-significantly more simple sentences than year 4 students (p=0.93).

Complex sentences

# Complex sentences/sentences Struc & Wood 2011

Increase over time for narrative and argumentative essays


Non-significant increase over a semester.




Significantly more complex sentences produced by year 4 students than year 3 students (p=<0.001).

# Complex sentences Cahyono, Mukminatien &

Amrina 2016

Significant positive correlation with writing quality scores (r=.32).

Compound sentences

# Compound sentences/sentences Struc & Wood 2011

Increase over time for narrative and argumentative essays


Significant increase over time (Cohen's d=.39).

No significant correlation with

overall essay score (r=.11) or language use score (r=.05).



Significantly more compound sentence produced by year 4 students than year 3 students (p=<0.001).

Compound-complex sentences

# Compound-complex sentences/sentences Struc & Wood 2011

Increase over time for narrative essays.

Decrease over time for argumentative essays.


Non-significant increase over a semester.

Non-significant correlation with overall essay score (r=.18).

Significant positive correlation with language use score (r=.24).



Year 4 students produce non-significantly more compound-complex sentences than year 3 students (p=0.16).

Sentence variety index

Variety index is a measure of sentence variety which ranges from 0-100 with 0 meaning sentences are all one type.

Struc & Wood 2011

Increase over time for narrative and argumentative essays.

Simple sentence

% Simple sentences over all sentence types. A simple sentence is taken to be a sentence with one main clause (including non-finite dependent

Verspoor, Lowie, Chan & Vahtrick

Non-significant positive correlation with holistic essay

ratio clauses) with subject and finite verb (2017, study 1) score (r=.35).Compound sentence ratio

% Compound sentences over all sentence types. A compound sentence is taken to be two main clauses each with its own subject and finite verbs


Significant negative correlation with holistic essay score (r=-.04).

Complex sentence ratio

% Complex sentences over all sentence types. A complex sentence is taken to be one main clause and one or more finite dependent clauses



Compound complex ratio

% Compound complex sentences over all sentence types. A compound complex sentence is taken to be a sentence containing both subordinate and coordinate elements (each with a finite verb)



Sentence complexity measure from Botel, Darwins and Granowsky

Sentence complexity is rated with simple sentences scored as a 0; SVIODO, SVOC, prepositional modifiers added to 0-count structures, noun modifiers, adverbial modifiers and modals are scored as 1; passive sentences, paired constructions, comparatives, dependent clauses and conjunctive adverbs are scored as 2; clauses used as subjects and absolute constructions are scored as 3. The final sentence index is the averaged score of all sentences.

Perkins 1980 Non-significant differences between proficiency levels.

Significant positive correlation with writing score (r=.44).

Sentences containing dependent clauses

# Sentences containing a dependent clause Verspoor, Schmid & Xu 2012a


Number of nodes per sentence

# nodes per sentence (manual calculation) Torras & Celaya 2001*



significant increase across proficiency levels (ω2=.33). · web view2020. 6. 8. · # clauses...

Documents