is rosetta stone the future of language learning?
DESCRIPTION
The present study is among the first to empirically examine the learning outcomes associated with the Rosetta Stone program as a textbook in a class or instead of altogether. Although initial results of basic proficiency and fluency revealed no significant difference between groups, continued linguistic analysis of individual oral and written data has revealed differences in terms of basic lexical and morphosyntactic knowledge as well as proficiency. This session presents the analysis of individual and group data in order to make the case for why programs such as Rosetta Stone cannot replace language classes.TRANSCRIPT
Gillian Lord University of Florida
Is Rose%a Stone the future of language
learning?
Why this study? • Omnipresent and powerful marke9ng*
makes Rose<a Stone … • En7cing to the average language learner • Temp7ng to administrators • Appealing to educators? • The best known language program on the market
*“RoseBa Stone … spent $98.5 million on adver7sing in 2011, up from $70.5 million in 2010, according to Kantar Media”
www.ny7mes.com/2012/06/20/business/media/roseBa-‐stone-‐ads-‐emphasize-‐fun-‐not-‐efficiency.html
A(n important) side note… • The study reported here was conducted with Rose8a Stone’s knowledge and technical support. • They were not involved in the design, data collec>on or analysis. • The Rose8a Stone licenses were purchased at regular price.
What do reviews of Rosetta Stone say? • Lafford, Lafford & Sykes (2010) • Evaluate if programs provide the tools necessary for effec7ve language learning, based on features that research has shown to be important (interac7on, relevant contextualiza7on of language, etc.)
• “… these products do not incorporate a number of the [necessary] research-‐based insights (e.g., the need for culturally authen7c, task-‐based ac7vi7es) that informed SLA scholars might have given them.”
• Santos (2011) • Lack of context • General inability to respond to spontaneous student speech • What Rose8a Stone calls interac7on is “a rather poor and limited version of what one would encounter in a real-‐life conversa7on”
• DeWaard (2013) • “Not a viable replacement of current instruc7on at the postsecondary level”
• Based on personal experience, professional reac7ons
What do academic reviews of Rosetta Stone say?
What do empirical studies show? • Vesselinov (2009) – commissioned by RoseBa Stone; RS
users who knew nothing prior to using the program demonstrated increased knowledge of the language afer a period of use.
• Nielson (2011) – self-‐study programs in workplace; some success but remarkable aBri7on; lack of community (e.g., Rovai, 2002) • Stevenson & Liu (2010) – lack of ability to engage learners in true interac7on; users do not take advantage of Web 2.0 tools to network. • This study – Phase 1 of analysis indicated that first-‐semester gains in some areas are comparable between RS users and a classroom control…
Participants • Par7cipants were University of Florida students enrolled in Beginning Spanish 1 (avg. age = 20) • L1 English • No other L2 proficiency (beyond h.s. requirement) • No prior Spanish instruc>on
• Par7cipants belonged to one of 3 environments: • Classroom (C): N=4 • Rose8a Stone (RS): N=4 • Classroom+Rose8a Stone (RS+C): N=4
Original popula7on had 20-‐25 par7cipants in each of the three groups.
Participants Control (Classroom) group (C) • In-‐tact sec7on of Beginning Spanish • Followed regular syllabus with standard materials • Carried out standard classroom assessment materials • Met with researcher 3x during semester
Participants RoseBa Stone group (RS) • Self-‐selected (required by IRB) • Not required to aBend any regular class • Used Rose8a Stone package (“Conversa7onal Spanish”):
• 16-‐week course designed to cover material comparable to a face-‐to-‐face beginning class
• 6 units of Rose8a Stone® Version 4 TOTALe® Spanish, each has 4 lessons [Level 1, half of Level 2]
• Minimum of 6 RoseBa StudioTM sessions • Minimum of 8 hours in RoseBa WorldTM • Monitoring of program access and 7me on task
• Followed predetermined deadlines in progressing through the material • Met with researcher 3x during semester
Participants Classroom + RoseBa Stone group (RS+C) • In-‐tact sec7on of Beginning Spanish class • Same instructor as control group • Used Rose8a Stone materials as their textbook (including all features described for RS group) • Met with researcher 3x during semester
Data collected • General oral and wri<en proficiency and skills • January, March, May
• Par9al CLEP test (30 items) • May
• Versant Automated proficiency test • May
• Assessment of aYtudes • January, May
• Discussion of experiences • January, March, May
Phase 1 Results: CLEP test Average scores (converted to %)
38.65 39.17 47.50
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
100.00
Classroom RoseBa Stone RS+class
p = 0.165
Phase 1 Results: Versant test Average scores (converted to %)
27.08 26.25 20.00
0.00
10.00
20.00
30.00
40.00
50.00
60.00
70.00
80.00
90.00
100.00
Classroom RoseBa Stone RS + Class
p = 0.615
PHASE 2 ANALYSIS
Final interview -‐ Classroom INTERVIEWER: Cuéntame, ¿qué te gusta hacer en tu 7empo libre, o los fines de semana? SL: Repitas, please. INTERVIEWER: ¿Qué te gusta hacer? SL: Qué te gusta hacer… INTERVIEWER: ¿Te gusta ir a películas? ¿Te gusta escuchar música? SL: Uh, ¿fin de semana? INTERVIEWER: Sí. SL: Uh, sí. En fin de semana, yo… yo estudio, uh, mucho. INTERVIEWER: ¿Sí? SL: Uh, para mis exámenes. Sí. Yo tengo muchos examines en química orgánica, biología, y laboratorio. Uh, sí. Mucho, uh… no, muy ocupado. So, no películas, no, uh, deportes. INTERVIEWER: ¿Cuál fue la úl7ma película que viste? SL: Cuál te… INTERVIEWER: La úl7ma vez, the last 7me, que viste una película. SL: Phew… Hmm. Let’s see… dos menses. INTERVIEWER: Meses, mhm. SL: Meses. Ago. ¿Cómo se dice “ago”? INTERVIEWER: Hace. Hace dos meses. SL: Hace, sí. INTERVIEWER: Wow. SL: Yo no… yo no veo muchas películas en Gainesville. INTERVIEWER: ¿Qué película fue esa, hace dos meses? ¿Cómo se llamaba? SL: Uh, el pelí—la película… ¿cómo se dice “was”? INTERVIEWER: Era, o fue. SL: Era. La película era… INTERVIEWER: ¿No te acuerdas? SL: Yo no… sí.
Final interview – Rosetta Stone INTERVIEWER: Mhm, ¿y qué haces en Gainesville? SH: Um… you’re going to have to forgive me, my mind’s like blown… Um, yo estoy estudiar. INTERVIEWER: ¿Tú estudias? ¿Y qué más? SH: Yo trabajo en un restaurante de Dragonfly. INTERVIEWER: Y, ¿con mucha frecuencia, vas de compras? SH: Yo no entendí, repe7rlo, por favor. INTERVIEWER: ¿Con mucha frecuencia, vas de compras? “Ir de compras” significa go shopping. SH: All right, say that one more 7me, please. INTERVIEWER: ¿Con mucha frecuencia, vas de compras? SH: Uh, no, uh, no voy a… what did you, how did you say “to go shopping”? INTERVIEWER: Ir de compras. SH: No voy de compras. INTERVIEWER: Y, ¿qué vas a hacer este verano? SH: Este verano, yo voy a visitar Brazil. INTERVIEWER: Vas a visitar Brazil, y ¿vas a estudiar en Brazil? SH: No, um, yo voy a trabajar en Brazil. INTERVIEWER: Y, em, ¿qué más a hacer en Brazil? ¿Vas a leer, vas a jugar deportes? SH: What am I going to do in Brazil? I thought I just answered that. INTERVIEWER: ¿Solo trabajar? SH: I don’t know, I’m going on a missions trip, I don’t know how to express that in Spanish, but… INTERVIEWER: Pues, buena suerte, muchas gracias.
Fluency analysis • All interviews • 3 groups x 4 par7cipants = 12 par7cipants x 3 interviews = 36 • Transcribed and analyzed for fluency measures
• “Fluency” • Total number of words spoken • Number of Spanish words; Number of English words • Number of dysfluencies • Lexical density (number of unique Spanish words) • Number of fillers/non-‐lexical items
General observations
GROUP
Total # words
# Span
ish words
# En
glish words
# Fillers
# Clarifica9o
n requ
ests in Spa
nish
# Clarifica9o
n requ
ests in English
Repe
99on
s/false
starts
#Uniqu
e words
Classroom Average 147.78 109.97 21.67 16.14 2.06 1.03 2.81 50.22
RS+C Average 90.61 47.53 34.75 8.33 0.11 1.75 2.14 27.22
RS Average 131.21 88.53 39.91 8.72 0.61 3.24 5.54 45.32
Ratio of L1/L2 words
0.26
0.83
0.68
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
1.00
Ra9o of English-‐to-‐Spanish words used, by group
Control Average
RS + class Average
RoseBa Stone Average
Classroom Average
0 = no English words produced 1 = 1 English word produced for every Spanish word
Assistance requests
2.06
1.03
0.11
1.75
0.61
3.24
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
# Clarifica7on requests in Spanish # Clarifica7on requests in English
Average # of clarifica9on / assistance requests by group
Control Average RS + class Average RoseBa Stone Average
Assistance requests
2.06
1.03
0.11
1.75
0.61
3.24
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
# Clarifica7on requests in Spanish # Clarifica7on requests in English
Average # of clarifica9on / assistance requests by group
Control Average RS + class Average RoseBa Stone Average
Other considerations (1) Language learning motivation • All learners indicated that their primary mo7va7on was to fulfill their language requirement • One third (4/12) reported that they chose Spanish, specifically, as a language that would be useful in their par7cular careers • RS volunteers par7cipated out of curiosity or convenience, but shared same degree of language learning mo7va7on.
Motivation – Classroom Group Group High school language
background Why Spanish?
Classroom No Spanish French 3
I am required to take a language for my major.
Classroom No Spanish La7n 3
Language requirement and for myself since I feel Spanish is a good asset as a physician.
Classroom No Spanish Fulfill [college requirement].
Classroom No Spanish La7n AP
I am going to Panama on a service trip. I believe formal classes would help me gain a beBer grasp of the language than picking it up on my own.
Motivation – RS+C Group Group High school language
background Why Spanish?
Rose<a Stone + Class No Spanish I am taking Spanish because I feel like it will be beneficial later on in life.
Rose<a Stone + Class No Spanish As a requirement and to benefit my future jobwise.
Rose<a Stone + Class No Spanish French 2
I need two semesters of a foreign language to graduate.
Rose<a Stone + Class No Spanish French 4
College requirement.
Motivation – RS Group Group High school
language background Why Spanish?
Why volunteer for RS?
Rose<a Stone No Spanish Foreign language requirement.
Heard a lot of good things about RoseBa Stone so decided to try it.
Rose<a Stone No Spanish La7n 3
CLAS requirement Can beBer manage my 7me and schedule and move more at my own pace without dealing with class.
Rose<a Stone No Spanish French 2
Required for major.
Sounded beneficial.
Rose<a Stone No Spanish ASL 3
Spanish is useful in my state/needed FL requirement.
I was going to use my own to supplement educa7on anyway.
Other considerations (2) Time on task
GROUP Comple9on Rate
Average Score
Total Course Usage (hours)
Total Class Time (hours)
Classroom 96.99% 90.77% 70.00 39.00
RS+C 93.67% 98.63% 32.81 37.25
RS 97.67% 95.88% 30.69 NA
Other considerations (3) Attitude survey • Few changes (pre-‐post) in any group • Significant changes (RS+C, C) on item #3: • “I am enjoying my Spanish-‐learning experience this semester.”
• Significant changes (RS) on item #11: • “Interac>ng via chat or telephone is comparable to interac>ng face-‐to-‐face.”
• Changes (RS, RS+C) on item #19: • “I would prefer to learn a language on my own >me and at my own pace than in a group or classroom seGng. “
Conclusions • Across environments… • Comparable outcomes on some measures • Different development of conversa7onal skills, discourse strategies • Skep7cism towards program is jus7fied
• More research (always!) needed • Larger, more varied sample size • Different proficiency levels • Broader student group
• More sensi7ve/appropriate tes7ng measures • Asess cultural awareness and competence
Thank you. glord@u[l.edu
• Special thanks to: • UF College of Liberal Arts and Sciences
• UF Humani7es Scholarship Enhancement fund
• Carlos Enrique Ibarra (sta7s7cs)
• Caroline Reist, Keegan Storrs, Diana Wade (RA)
• Laura Bradley (RoseBa Stone)
QR code here
Works Cited Bley-‐Vroman, R. (1988). “The fundamental character of foreign language learning.” In W. Rutherford & M. Sharwood Smith (Eds.), Grammar and second language teaching (pp. 19-‐30). Rowley, MA: Newbury House. Bley-‐Vroman, R. (2009). “The evolving context of the Fundamental Difference Hypothesis.” Studies in Second Language Acquisi>on 31(2), 175-‐198. DeWaard, L. (2013). “Is RoseBa Stone a viable op7on for L2 learning?” Forthcoming in ADFL Bulle>n. Godwin-‐Jones, R. (2007). “Emerging technologies; Tools and trends in self-‐paced language instruc7on. Language Learning and Technology,” 11(2), 10-‐17. Retrieved 26 September 2012 from hBp://llt.msu.edu/vol11num2/emerging/ Godwin-‐Jones, R. (2009). “Emerging technologies: Speech tools and technologies. Language Learning and Technology,” 13(3), 4-‐11. Retrieved 26 September 2012 from hBp://llt.msu.edu/vol13num3/emerging.pdf Krashen, S. D. & Terrell, T. D. (1983). The Natural Approach: Language acquisi>on in the classroom. Hayward, CA: Alemany Press. Lafford, B., Lafford, P. & Sykes, J. (2007). “Entre dicho y hecho …: An assessment of the applica7on of research from second language acquisi7on and related fields to the crea7on of Spanish CALL materials for lexical acquisi7on.” CALICO Journal, 24(3), 427-‐529. Nielson, K. B. (2011). “Self-‐study with language learning sofware in the workplace.” Language Learning and Technology, 15(3), 110-‐129. Retrieved 26 September 2012 from hBp://llt.msu.edu/issues/october2011/nielson.pdf Rovai, A. P. (2002). “Development of an instrument to measure classroom community.” The Internet and Higher Educa>on, 5, 197-‐211. Santos, V. (2011). “Review of Rose8a Stone Portuguese (Brazil) levels 1, 2, & 3.”CALICO Journal, 29(1), 177-‐194. Stevenson, M. P. & Liu, M. (2010). “Learning a language with web 2.0: Exploring the use of social networking features of foreign language learning websites.” CALICO Journal, 27(2), 233-‐259 Vesselinov, Roumen. Measuring the Effec>veness of RoseBa Stone. hBp://resources.roseBastone.com/CDN/us/pdfs/Measuring_the_Effec7veness_RS-‐5.pdf.
Rosetta Stone interface
Rosetta Stone interface (vocabulary)
Rosetta Stone interface (grammar)
Rosetta Stone interface (pronunciation)
Rosetta Stone interface (World – “play”)
Rosetta Stone interface (World – “talk”)
Rosetta Stone interface (World – “explore”)
Rosetta Stone interface (Studio)