bei
DESCRIPTION
TRANSCRIPT
Task-internal and task-external readiness:
A report of the effects of topic familiarity and strategic planning on task performance by L2 learners of
different proficiency levels
Gavin Bei Xiaoyue
The Chinese University of Hong Kong
3rd Biennial International Conference on Task-Based Language Teaching, Lancaster University, Sept 15, 2009
Part 1
Research background
Contextualizing
• Task-based instruction research looks at:• 1. Task characteristics: • Subjective or objective, structural or non-structural,
familiar or unfamiliar…• 2. Task conditions:• Monologic or interactive, Pre/Post task activities,
planning or non-planning…• 3. Participants:• Gender, motivation, learning style, proficiency…
Research background 1: topic familiarity
• 1. Comprehension (many)• facilitative• e.g., Shimioda, 1993; Barry and Lazarte, 1995; Bügel &
Buunk, 1996; Chen and Donin, 1997; Johnson, 1982; Lee, 1986; and Chang, 2006
• no effect on comprehension • e.g., Hammadou, 1991; Peretz & Shoham, 1990; and Carrell
(1983)
• 2. Production (few)
• Mostly in L1 research by psychologists
• Higher fluency, but inconsistent in accuracy or complexity
Research Background 2: planning types
• Two macro and four micro types of planning (Ellis, 2005)• 1. pre-task• 1) rehearsal 2) strategic planning• 2. within-task• 1) pressured 2) unpressured
• Or simply three micro types (Ellis, in press)• 1) rehearsal 2) strategic planning 3) within-task planning
Research background 3: strategic planning
• Ample studies (e.g., Skehan, Foster, Ellis, Crookes, Wigglesworth, etc.) with quite some consistent results.
• Planning raises: Fluency + Complexity• (sometimes, but usually not) Accuracy• Skehan: Trade-off of between Comp. and Accu. • Robinson: Planning does not lead to Comp, no
trade-off.• Is proficiency important here?
Research background 4: Proficiency and Familiarity
• Hudson (1982):
• In Reading: Familiarity > Proficiency.
• Schmidt-Rinehart (1994):
• In Listening: Familiarity > Proficiency.
• Carrell (1983):• In Reading: Proficiency > Familiarity (NS:NNS)
• Chern (1993):
• In Reading: Proficiency > Familiarity.
Research background 5: proficiency and planning
• Wigglesworth (1997): low proficiency did not benefit from planning.
• Tavokoli and Skehan (2005): planning drove high and low learners for better performance.
• Kawauchi (2005): more Flu. and Comp. for higher learners, more Accu. for the lower. The advanced gained the least.
• Most other studies did not consider proficiency.
Part 2
The study design
and methodology
1.Participants and proficiency test
• Participants: 80 HK Cantonese-speaking undergraduates volunteers to participate.
• A C-test as proficiency test to group participants
------- borrowed from Dornyei and Katona (1992).--------The validity and reliability are good in the
literature and in the present context. See appendix 1
2. Tasks
• Topic 1: Natural Viruses.
• Topic 2: Computer Viruses.
Topic 1 Topic 2
Medicine Majors (N=40)
+ familiar – familiar
Computer Majors (N=40)
– familiar + familiar
3. Independent Variables
• 1. Topic familiarity (within): 2 levels
• familiar VS unfamiliar task
• 2. Planning (between): 2 levels
• non-planning VS 10-min planning
• 3. Proficiency (between): 2 levels
• intermediate VS high
Study design
Planning(between)
Proficiency(between)
Topic familiarity (within)
Familiar Unfamiliar
Planners
High 20 20
Intermediate 20 20
Non-planners
High 20 20
Intermediate 20 20
Each cell consists of 10 computer majors and 10 medicine majors as counterbalancing to rule out the topic effect.
4. Dependent Variables
• Fluency: pausing, speech rate, MLR, phonation time, repairs, etc.• Accuracy: error-free clauses ratio, length of correct clause, and
errors per 100 words.• Complexity: Clauses per AS unit, AS unit length, and clause
length• Lexis: lexical diversity, lexical sophistication, and lexical density.• Formality: F-score, DB-score• Totally 21 measure were employed. See Appendix 2 for a detailed
description.
• P value: the significance level to tell whether there is an effect.• Cohen’s D value: the effect size to tell how big the effect is.
5. Statistical procedures
• A repeated measures analysis of variance (ANOVA) was employed.
Part 3
Results
Breakdown fluency main effects 1
T. Familiarity Planning Proficiency
p D p D p D
Speech rate .000 .26 .000 .58 ns /
Phonation time .000 .35 .000 .62 ns /
Mean length of run .016 .17 .046 .32 ns /
No. Mid-clause pauses .000 .38 .001 .54 ns /
No. End-of-clause pauses ns / ns / .027 .48
Table 1. p and Cohen’s D value
Means omitted due to the space limit. The means show that the familiar topics and the planning time improve fluency. Same directions below unless there is a note.
Breakdown fluency main effects 2
T. Familiarity Planning Proficiency
p D p D p D
Mid-clause silence total .000 .38 .000 .61 ns /
End-of-clause silence ttl. .014 .19 .007 .59 ns /
Av. mid-clause pause .019 .28 .000 .71 ns /
Av. End-of-clause pause ns / .004 .64 ns /
No. filled pauses .085 / ns / ns /
Table 2. p and Cohen’s D value
• T. Familiarity has significant interactions with Planning in:
• 1) speech rate• 2) phonation time• 3) No. Mid-clause pauses• 4) Mid-clause silence total (per 100 words)• 5) End-of-clause silence total (per 100 words)
Breakdown fluency: interactions
All showing one pattern: planning compensates for the unfamiliar topics.
Measures: speech rate and mid-clause silence total
Breakdown fluency: a summary
1) T. Familiarity affects fluency in a strikingly similar way as Planning does.
2) Approximately, the effect sizes of T. Familiarity is half as big as those of Planning.
3) Planning mitigate the difference between familiar and unfamiliar topics.
4) The effects of Proficiency is marginal, and probably overridden by T. F. and Planning.
Repair fluency
T. Familiarity Planning Proficiency
p D p D p D
False starts ns / .000 1.02 ns /
Reformulations .088 / .001 .53 ns /
Replacements .077 / .008 .43 ns /
Repetitions .001 .40 .000 .75 ns /
Table 3. p and Cohen’s D value
Note: planning induced more replacements, though reducing others.
Accuracy
Table 4. p and Cohen’s D value
T. Familiarity Planning Proficiency
p D p D p D
Error-free clauses ratio
.02 .22 ns / .000 .69
70% accuracy clause length
ns / ns / .000 .57
Errors per 100 words
.000 .38 ns / .000 .77
Complexity
Table 5. p and Cohen’s D value
T. Familiarity Planning Proficiency
p D p D p D
Clauses per AS unit
ns / .018 .39 .067 /
Words per AS unit ns / .000 .81 .000 .52
Words per clause ns / ns / ns /
Lexis
Table 6. p and Cohen’s D value
T. Familiarity Planning Proficiency
p D p D p D
Lexical diversity .018 .29 ns / ns /
Lexical sophistication .000 .41 ns / ns /
Lexical density ns / .008 .43 .031 .39
Formality
Table 7. p and Cohen’s D value
T. Familiarity Planning Proficiency
p D p D p D
F-score .000 .49 .003 .48 ns /
DB-score ns / .019 .38 ns /
Part 4
Some conclusions
Conclusions 1
• 1. Planning is more powerful in driving fluency than T. Familiarity. It can reduce the differences between familiar and unfamiliar topics in breakdown fluency.
• 2. Topic familiarity and planning seem to be more concerned with meaning expression (similar).
• 3. T. familiarity and planning affect different syntactic areas (different).
• 4. Proficiency affects mostly forms, esp. accuracy, but not so much meaning expression (fluency and lexis).
• 5. Higher proficiency does not appear to remove the trade-off effects. So L2 learners are L2 learners!
Conclusion 2
• 6. Factor analyses of all measures show:• 1) there’s probably an end-of-clause fluency different
from breakdown and repair fluency.• (Av. Pause and total silence at the end of a clause, and
phonation time. )• 2) there’s probably a noun-phrase complexity as
compared to the syntactic complexity. • (words per clause, F-score, DB-score, and Lambda)
• 6. A broader perspective on planning stems from the similarities and differences between T. familiarity and strategic planning in this study, in which I argue that T.F. can be regarded as a kind of implicit planning (see next page).
A general framework of task-readiness
Macro-dimension
Micro-dimension Sample studies
Learnerreadiness
fora task
Task-internal readiness (implicit planning)
Topic familiarity (prior domain knowledge)
This study
Schematic familiarity (story structure)
Skehan and Foster (1999)
Task familiarity (task types)
Bygate (2001)
Task-external readiness (explicit planning)
Rehearsal ( content repetition)
Bygate (1996)
Strategic (pre-task) planning
Foster and Skehan (1996)
Within-task (on-line) planning
Yuan and Ellis (2003)
Thank you!
Q and A
Appendix 1: reliability and validity of C-test
Dornyei and Katona (1992) found that the C-test is reliable (the internal consistency coefficients are very consistent, .75 and .77 respectively, for university English majors and secondary students) and valid (C-test is significantly highly correlated with different other proficiency tests like the General Language Proficiency and TOEIC). Cronbach’s alpha reached .84 in Daller and Phelan (2006). Klein-Braley and Raatz (1984), Klein-Braley (1985), Cohen, Segal and Bar-Siman-Tov (1984), Klei-Braley (1997), and Grotjahn, 1995 generally supported such a claim on written tasks. More importantly here, the C-test was reported to be highly correlated with oral tasks as well in recent studies (e.g., r=.64 in Arras, Eckes and Grotjahn, 2002, and also in oral lexical performance in Daller and Xue, 2007). More recently Dai (this conference) reported in Chinese context, Cronbach’s Alpha=.770, Concurrent validity r= .633, p<0.01 (correlated with CET-4). In this study, the Cronbach Alpha is=.64 in the pilot study, but =.74 in the main study.
back
Appendix 2: description of dependent variables
General Category Variable Name Description
Fluency
Pausing The number of pauses and the amount of silence. In the present study it is operationalized as any break of 0.4 second or longer.
Repair Fluency This measure is orthogonal to breakdown fluency and should be treated separtely. In the present study it is operationalized as the total number of repetitions, replacements, false starts and reformulations.
Speech Rate A pruned speech rate is investigated here because it shows the ‘real’ speed of the speaker. It is operationalized as the total words per minute after deletion of reformulations, replacements, false starts, repetitions, pauses and silence total.
Mean Length of Run The number of words uttered before any breakdown or repair fluency is encountered.
Phonation time The ratio of voicing time to the total time of utterance.
Accuracy
Error-free Ratio The ratio of error-free clauses to all clauses.
Errors per 100 Words The number of errors in every pruned one hundred words.
Length Accuracy The length of a clause with 50% of all clauses of the same length correct is set as the cut-off point beyond which the participant cannot produce correct clause at 50% level.
Complexity
Subordination Ratio The ratio of subordinate clauses per AS unit.
Words Per AS Unit The average word number in all AS units.
Words Per Clause The Average word number in all clauses
Lexis
Lexical diversity: the D value
Corrected Type-token ratio, an index of the extent to which the speaker avoid returning to the same set of words.
Lexical sophistication: the Lambda value
The extent to which speech contains difficult or rare words.
Lexical Density The ratio of content words to the total words.
Formality F-score (Noun frequency+adjective freq.+preposition freq.+article freq.-pron. Freq. –verb freq.-adverb freq. – interjection freq. +100) / 2
From F. Heylighen and J. Dewaele (1999).
DB-score The ‘involved’ style words in Biber, Conrad and Reppen (1998).
back
ReferencesBarry, S. & Lazarte, A. (1995). Embedded clause effects on recall: Does high prior knowledge of
content domain overcome syntactic complexity in students of Spanish? Modern Language Journal, 79, 491–504.
Biber, D., Conrad, S. & Reppen, R. (1998). Corpus Linguistics: Investigating Language Structure and Use. Cambridge: Cambridge University Press.
Bügel, K. & Buunk, B. (1996). Sex differences in foreign language text comprehension: The role of interests and prior knowledge. Modern Language Journal, 80, 15–31.
Bygate, M. (1996). Effects of task repetition: appraising the developing language of learners. In Jane Willis & Dave Willis (Eds). Challenge and change in language teaching. Oxford : Heinemann.
Bygate, M. 2001. Effects of task repetition on the structure and control of oral language [A]. In M. Bygate, P. Skehan & M. Swain (eds). Researching Pedagogical Tasks: Second Language Learning, Teaching and Testing [C]. Harlow, England: Longman.
Carrell, P. L. (1983). Three components of background knowledge in reading comprehension. Language Learning, 33, 183-207.
Chang, C. (2006). Effects of topic familiarity and linguistic difficulty on the reading strategies and mental representations of non-native readers of Chinese. Journal of Language and Learning, 4, 172-198.
Chen, Q. & Donin, J. (1997). Discourse processing of first and second language biology texts: Effects of language proficiency and domain-specific knowledge. Modern Language Journal, 81, 209–227.
Chern, C. (1993). Chinese students’ word-solving strategies in reading in English. In T. Huckin, M. Haynes, & J. Coady (Eds). Second language reading and vocabulary learning. Pp. 67–82. Westport, CT: Ablex.
Crookes, G. (1989). Planning and interlanguage variation. SSLA, 11, 367-383.Ellis, R. (2005). Planning and task-based performance: Theory and research. In R. Ellis. (Ed).
Planning and task performance in a second language. Amsterdam: Johan Benjamins.
Ellis, R. (in press). The Differential Effects of Three Types of-Task Planning on the Fluency, Complexity and Accuracy in L2 Oral Production. Applied Linguistics.
Hudson, T. (1982). The effects of induced schemata on the “short-circuit” in L2 reading: non-decoding factors in L2 reading performance. Language Learning, 32/1, 1-31.
Hammadou, J. (1991). Interrelationships among prior knowledge, inference, and language proficiency in foreign language reading. The Modern Language Journal,75: 27-39.
Johnson, P. (1982). Effects on reading comprehension of language complexity and cultural background of text. TESOL Quarterly, 16, 169–181.
Kawauchi, C. (2005). The effects of strategic planning on the oral narratives of learners with low and high intermediate proficiency. In R. Ellis. (Ed). Planning and task performance in a second language. Amsterdam: Johan Benjamins.
Lee, J. F. (1986). Background knowledge and L2 reading. Modern Language Journal, 70, 350–354.Peretz, A., & Shoham, M. (1990). Testing reading comprehension in LSP: Does topic familiarity
affect assessed difficulty and actual performance? Reading in a Foreign Language, 7, 447–455.Shimoda, T. A. (1993). The effects of interesting examples and topic familiarity on text
comprehension, attention, and reading speed. Journal of Experimental Education, 61, 93-103.Schmidt-Rinehart, B. C. (1994). The effects of topic familiarity on second language listening
comprehension. The Modern Language Journal, 78, 179-189.Skehan, P. & Foster, P. (1999). The influence of task structure and processing conditions in narrative
retellings. Language Learning, 49 (1): 93-120.Tavakoli, P. & Skehan, P. (2005). Strategic planning, task structure, and performance testing. In R.
Ellis. (Ed). Planning and task performance in a second language. Amsterdam: Johan Benjamins. Foster, P. & Skehan, P. (1996). The influence of planning and task type in second language
performance. SSLA, 18, 299-323.Wigglesworth, G. (1997). An investigation of planning time and proficiency level on oral test
discourse. Language Testing, 14 (1): 85-106.Yuan, F. & Ellis, R. (2003). The effects of pre-task and on-line planning on fluency, complexity and
accuracy in L2 monologic oral production. Applied Linguistics 24(1): 1–27.