speaking proficiency under learner- and shared-control ... › ... · speaking proficiency under...

102
Evaluating the Impact of an Online English Language Tool’s Ability to Improve Users’ Speaking Proficiency under Learner- and Shared-control Conditions by Shane Dixon A Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy Approved April 2015 by the Graduate Supervisory Committee: Robert Atkinson, Chair Wilhelmina Savenye Brian Nelson ARIZONA STATE UNIVERSITY May 2015

Upload: others

Post on 26-Jun-2020

21 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

Evaluating the Impact of an Online English Language Tool’s Ability to Improve Users’

Speaking Proficiency under Learner- and Shared-control Conditions

by

Shane Dixon

A Dissertation Presented in Partial Fulfillment

of the Requirements for the Degree

Doctor of Philosophy

Approved April 2015 by the

Graduate Supervisory Committee:

Robert Atkinson, Chair

Wilhelmina Savenye

Brian Nelson

ARIZONA STATE UNIVERSITY

May 2015

Page 2: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

i

ABSTRACT

This study aims to uncover whether EnglishCentral, an online English as a

Second Language tool, improves speaking proficiency for undergraduate students with

developing English skills. Eighty-three advanced English language learners from the

American English and Culture Program at Arizona State University were randomly

assigned to one of three conditions: the use of EnglishCentral with a learner-control,

shared-control, and a no-treatment condition. The two treatment groups were assigned

approximately 14.7 hours of online instruction. The relative impact of each of the three

conditions was assessed using two measures. First, the Pearson Versant Test

(www.versanttest.com), a well-established English-as-a-second-language speaking test,

was administered to all of the participants as a pre- and post-test measure. Second,

students were given a post-treatment questionnaire that measured their motivation in

using online instruction in general, and EnglishCentral specifically. Since a significant

teacher effect was found, teachers involved in this study were also interviewed in order to

ascertain their attitude toward EnglishCentral as a homework tool.

Learner outcomes were significantly different between the shared and learner

conditions. Student motivation was predictive of learning outcomes. Subjects in the

shared condition outperformed those in the learner condition. Furthermore, those in the

shared condition scored higher than the control condition; however, this result did not

reach statistical significance. Results of the follow-up teacher survey revealed that while

a teacher’s view of the tool (positive or negative), was not a predictor of student success,

teacher presentation of the tool may lead to a significant impact on student learning

outcomes.

Page 3: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

ii

ACKNOWLEDGEMENTS

I want to thank Dr. Robert Atkinson for believing in this project, and for Mark Rentz,

Mary Chang, and Lois Malone for helping to sustain it. I also want to thank my wife for

putting up with me for every time this study demanded that I hold her a conversational

hostage

Page 4: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

iii

TABLE OF CONTENTS

Page

ABSTRACT ......................................................................................................................... i

LIST OF TABLES .............................................................................................................. v

LIST OF FIGURES ........................................................................................................... vi

CHAPTER 1--INTRODUCTION AND LITERATURE REVIEW................................... 1

Statement of the Problem ................................................................................................ 2

Purpose of the Study ....................................................................................................... 6

Review of Literature........................................................................................................ 6

Language Acquisition: Speaking ................................................................................ 7

Student Motivation.................................................................................................... 10

Learner Control ......................................................................................................... 12

Shared Control .......................................................................................................... 18

Research Questions ....................................................................................................... 20

CHAPTER 2—METHODOLOGY .................................................................................. 21

Participants and Design ................................................................................................. 21

Design of Learner Control ........................................................................................ 22

Design of Shared Control.......................................................................................... 24

Design Similarities in Both Conditions .................................................................... 26

English Central Online Learning Tool ...................................................................... 28

Control Group ............................................................................................................... 35

Materials ........................................................................................................................ 36

Measures........................................................................................................................ 36

Procedure ....................................................................................................................... 40

Scoring .......................................................................................................................... 44

CHAPTER 3—RESULTS ................................................................................................ 46

Assumptions .................................................................................................................. 46

Primary Analysis ........................................................................................................... 47

Page 5: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

iv

Supplemental Analysis .................................................................................................. 49

CHAPTER 4: DISCUSSION AND IMPLICATIONS ..................................................... 53

Shared-Control Superior to Learner-Control ................................................................ 53

Supplemental Findings .................................................................................................. 58

Finding 2: The Tech Awareness Effect .................................................................... 58

Finding 3: The Teacher Effect .................................................................................. 59

Limitations .................................................................................................................... 60

Conclusions ................................................................................................................... 61

Future Directions ........................................................................................................... 62

REFERENCES ................................................................................................................. 64

APPENDIX ....................................................................................................................... 69

ENGLISHCENTRAL VIDEOS IN SHARED CONTROL MODEL ......................... 69

ENGLISHCENTRAL SURVEY FOR LEARNER AND SHARED CONTROL

PARTICIPANTS AND ENGLISH CENTRAL SURVEY FOR TEACHERS ............ 74

PEARSON VERSANT TEST SAMPLE ITEMS ......................................................... 79

ENGLISHCENTRAL ASSIGNMENT INSTRUCTIONS FOR BOTH SHARED

AND LEARNER CONTROL CONDITIONS ............................................................. 82

SPRING 2014 TECH SURVEY AT THE AMERICAN ENGLISH AND CULTURE

PROGRAM ................................................................................................................... 87

CONSENT FORM FOR SHARED AND LEARNER CONTROL CONDITIONS .... 89

PEARSON VERSANT TEST INSTRUCTIONS (PHONE FORMAT) ...................... 93

Page 6: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

v

LIST OF TABLES

Page

Table 2.1: Learner Control Elements in Each Format of the Present Study ..................... 27

Table 2.2 Usability Questions ........................................................................................... 37

Table 2.3 Motivation Questions ........................................................................................ 38

Table 2.4 Technology in General Questions..................................................................... 38

Table 2.5: Schedule of Events .......................................................................................... 41

Table 3.1: A Priori Orthogonal Contrasts ......................................................................... 48

Table 3.2 Mean Amount of Improved Speaking Fluency for Learner, Shared, and Control

Conditions ......................................................................................................................... 48

Table 3.5 One-Way Analysis of Variance Summary Table for the Effects of Gender on

Speaking Scores ................................................................................................................ 51

Table 3.6 One-Way Analysis of Variance Summary Table for the Effects of Language on

Speaking Scores ................................................................................................................ 51

Table 3.7 Teacher Motivation and Student Speaking Scores by Learner and Shared

Condition........................................................................................................................... 52

Page 7: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

vi

LIST OF FIGURES

Page

Figure 1.1: Kay’s Learner Control Model ........................................................................ 15

Figure 2.1: Video Player ................................................................................................... 27

Figure 2.2: Watch Feature (in both treatment conditions) ................................................ 29

Figure 2.3: Learn Feature (in both treatment conditions) ................................................. 30

Figure 2.4: “My Words” Vocabulary Learning Feature ................................................... 30

Figure 2.5: “My Words” Vocabulary Learning Feature ................................................... 31

Figure 2.6: Speak Feature ................................................................................................. 31

Figure 2.7: Pronunciation Toolbar .................................................................................... 32

Figure 2.8: Pronunciation Learning Feature ..................................................................... 33

Figure 2.9: Shared-Control Design: The Advanced 1 Listening and Speaking Channel . 34

Figure 2.10: Shared-Control Design: The Advanced 1 Listening and Speaking Channel 34

Figure 2.11: Shared-Control Design: The Advanced 1 Listening and Speaking Channel 35

Figure 3.1: Histograms of learner, system, and control groups ........................................ 46

Page 8: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

1

CHAPTER 1--INTRODUCTION AND LITERATURE REVIEW

In pursuit of the second language education market, a market which until now has

generally found its home in traditional brick and mortar classrooms, very few online products

have been as well-received as EnglishCentral, which recently reached over two hundred million

lines of spoken text (“EnglishCentral,” 2013). The popularity of this online tool can probably be

traced to several features, including the familiarity of the user interface. This interface shares

features with video-sharing websites Youtube and Vimeo in that it shows thousands of real life

videos, all searchable by popularity and topic. Furthermore, part of its acceptance, especially in

the marketplace, may be traced to the flexibility with which it presents its product. Individual

learners can purchase a subscription fee and, by selecting from a wide variety of options, take

control of their own learning. Teachers and administrators, on the other hand, can create virtual

classrooms students can join. Within these classrooms, teachers can monitor the videos watched,

the number of lines spoken, and can listen in on any line of text the students have spoken.

Finally, due to advanced speech recognition software, EnglishCentral identifies language sounds

by making accommodations for a user’s gender, language of origin, and probable phonemic

patterns, and then provides feedback that allows students to work on individual phonemes.

With all this said, there are a number of questions surrounding the EnglishCentral

environment. First, while internal data from EnglishCentral suggests discrete sounds may

improve over time, it is unknown whether or not the environment improves overall speaking

fluency (personal communication, June 4, 2014). Thus, a study that measures overall speaking

proficiency would help to evaluate whether or not EnglishCentral has a larger, more holistic

effect.

Page 9: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

2

A second educational concern does not ask if English Central is effective, but rather how

EnglishCentral might be most effective. This particular question surfaces when attention is paid

to the open interface (wherein a student is in charge of their own video selection), or a more

closed design (referred to as a “channel”) wherein teachers and/or administrators choose videos

for learners. While a teacher can monitor students in either design, it is unknown whether or not

the selection of videos should be put into the hands of teachers or students. In this light, a study

that measures the relative impact of each of these settings seems relevant and valuable.

A final concern in designing this study comes from the need to measure student attitude

toward the EnglishCentral environment. While in certain environments online learning

engagement has been shown to exceed that of face-to-face classroom instruction (Robinson &

Hullinger, 2008), online learning designs can be susceptible to variations in student enjoyment

based on demographic information such as gender and first language. This study will also seek

to uncover any variations accounted for by a number of such student factors.

Statement of the Problem

Online learning continues to grow in nearly all academic disciplines. A survey by Allen

and Seaman demonstrate this point by showing that multiple disciplines, from computer and

information science to history and liberal arts, are moving toward online learning and showing

equal amounts of market penetration (2008). This trend is likely the case because online learning

is perceived to have unique advantages. For example, online learning supporters tout the ability

to have asynchronous instruction, meaning that a teacher need not always be present for learning

to occur, and that a student may enter a discussion thread, an online lesson, or watch a video at

any time (Parsad, Lewis, & Tice, 2008). Secondly, online education often boasts the ability to do

assessment, thus relieving teachers from the burden of doing an unnecessary amount of

Page 10: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

3

correction (Collis, De-Boer & Slotman, 2001). Online learning is also seen as possibly more

motivating, since students are allowed to navigate and explore online spaces in ways that may

encourage autonomy and self-directed learning (Cordova & Lepper, 1996). Finally, there is an

economic factor that suggests that online learning may represent a method of increasing

profitability by reducing the number of teachers and classrooms, thus expanding the number of

students in a single classroom. Certainly, the recent proliferation of MOOC’s (massive open

online courses) demonstrates this point.

While interest in online education has never been stronger, much of what passes as online

learning has gone untested, especially when trying to make a case of its utility in comparison

with traditional classrooms. This makes the claims of online advocates somewhat difficult to

assess. Opponents of online education cite this lack of rigorous testing as a result of the novelty

of these new tools, and that many of the claims made by online advocates consist more of

commercial sound bites than academically sound research (Anderson, 2008).

This lack of testing but obvious expansion is apparent in the ESL online learning market.

English-as-a-second-language professionals are incorporating technology in the classroom, and

the use of internet websites and online learning tools has never been greater. For example, in a

survey of tools used among the 100 faculty members at Arizona State University’s American

English and Culture Program, over 140 online learning tools are currently being used (see

Appendix E for a full list). Since online tools are not mandated by administration (with few

exceptions), this suggests that educators are, of their own accord, anxious to use, experiment

with, and improve their classrooms with the use of online learning tools.

One particular way in which teachers are attempting to use online tools is by assigning

homework that has been previously created in an online format. Online quizzes, readings,

Page 11: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

4

videos, and other assignments give teachers a chance to search for lesson activities that pair with

the themes or goals of the classroom. Websites specific to ESL lesson planning such as

Randall’s Cyber Listening Lab, Voice of America, Breaking News English, Busy Teacher, ESL

Galaxy, and Dave’s ESL Café offer lesson plan tips, ideas, worksheets, or other online helps.

Other websites give teachers the facility to engage students in real world English not

specifically tied to ESL or ESL objectives. Teachers often send students links to particular

articles or activities from these websites. These might include but are not limited to the

California Distance Learning Project, NPR, Ted Talks, CNN Video, Ohio U Reads, and Time

Magazine.

Additionally, teachers often create their own materials using online tools that provide

templates for quizzes, flashcards, reading activities, and so forth. Quizlet, Zap Reader, and

Rubistar are examples of these kinds of online tools.

At this point, however, we must concede that while some teachers are actively engaged in

using these online tools, and some in very creative ways, there are a number of teachers that use

very little technology. Some teachers and students experience a certain amount of technophobia,

which has led the administration at the American English and Culture Program to make the

decision to not require the use of specific online tools, with one exception. Since Blackboard is

the official learning management system (LMS) at Arizona State University, it is required of all

AECP teachers as the proper form to communicate syllabi, grades, and other assignments.

Blackboard includes a suite of tools such as discussion boards, voice boards, the attachment of

Word, Excel, and other files, the ability to embed video and audio files, and even the ability to

create online quizzes. (Note: more recently, the AECP has made EnglishCentral a required

element for all Advanced 2 students.)

Page 12: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

5

All in all, many of the websites referred to above, especially those specifically catering to

ESL audiences, have their limitations. Many of the tools referred to were developed by

teachers, meaning that often the programming or look of the website leaves something to be

desired. Some of the websites, such as Quizlet and Rubistar, allow teachers to manage and

develop the curriculum itself, but this inevitably creates a vetting or quality control problem and,

as a result, a great number of errors in the shared templates are common. Furthermore, teachers

themselves must constantly match the vast amount of content to the content of their own classes,

making online use unwieldy and inconsistent. Finally, as was mentioned earlier, many of these

online tools have not undergone the scrutiny of empirical testing, thus making any claims on

their usefulness almost impossible.

As a website and a learning tool, EnglishCentral has some clear advantages for ESL

teachers and administrators. The EnglishCentral platform allows for asynchronous learning,

which makes it a possible candidate for homework given outside of the classroom. Furthermore,

since it provides progress benchmarks, a teacher can assign a certain amount of work and quickly

ascertain whether goals were reached or not. Since EnglishCentral has its roots in video players

like Youtube and Vimeo, EnglishCentral may represent an immediately understood product,

thus reducing cognitive load and improving an end user’s experience. EnglishCentral’s

similarity to Youtube and/or Vimeo is that, like these ubiquitous online tools, it contains a

repository of videos that can be played, stopped, and reviewed, and even liked or disliked. Since

the repository contains clips found by famous celebrities, interviews, and shows, students receive

authentic language, which may prove to be a motivator compared to pre-packaged language

lessons. Thus, because of this already understood delivery system, it is posited that it may

motivate students to move beyond the required learning amounts and perform better in

Page 13: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

6

comparison to those in a classroom given traditional homework assignments. It is one of the

goals of the current study to ascertain this possibility.

As was noted before, it is not entirely clear whether EnglishCentral’s open platform,

which allows ESL students to choose any video, or the more-controlled platform wherein

teachers choose videos (the “channel”), will allow for better gains in learning and motivation.

Thus, it is also a goal to ascertain which delivery system in EnglishCentral might be most

effective.

Purpose of the Study

This study aims to inform various parties including administrators, teachers, and

developers. First of all, administrators must ask questions as to the utility of the products

teachers are using within the classroom, especially if a decision is made to invest in or mandate

such a program. This study will inform administrators on the general utility of the English

Central tool as a program-wide tool, and give recommendations as to how it might best be used.

Second, it can also inform teachers whether or not to invest time in learning the EnglishCentral

tool, and whether introduction of this tool is likely to bring learner results. Finally, online

learning developers such as those at EnglishCentral will be interested to know the strengths and

weaknesses of this learning format, and might inform the entire field as to the direction online

language learning might take.

Review of Literature

The subsequent section provides an overview of the literature relevant to this study.

First, a discussion of the prominent theories on speaking proficiency is presented. Second, a

discussion on student motivation, and in particular how it relates to online learning, is also

Page 14: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

7

discussed. Third, an overview of learner, system, and shared control will help demonstrate some

of the principles behind this study’s design.

Language Acquisition: Speaking

Teaching non-native English learners how to speak English can be difficult. According to

Bailey and Savage, speaking in a second or foreign language has long been viewed as “the most

demanding of the four skills” (1995, p. vii). Brown strengthens that argument by listing the

multiple challenges of speaking in a non-native tongue: reduced forms, contractions, vowel

reduction, elision, use of slang, stress, rhythm, intonation, interaction, monitoring while

speaking, and modifying one’s speech according to the effect of the listener (2001, p. 270-1).

Additionally, the number of hours required to be a good speaker can be extremely limited in a

classroom setting since students must share speaking time with other students. Instructors often

compensate by providing group and pair activities, discussions, and individual or group

presentations (Brown, 2001). However, by so doing, speaking becomes difficult for teachers to

assess. Instructors cannot easily watch and/or record all learners and, thus, often miss out on

providing adequate feedback.

Furthermore, speaking is also difficult to assess since teachers are generally required to

create rubrics that contain some of the many possible categories that speaking entails. Teachers

must decide: Does speaking include the use of body language? Does speaking include the use of

visual aids? These and similar questions demonstrate the difficulty of assigning precisely what

speaking, and especially competent speaking, might be.

Early theorists focused almost solely on grammatical accuracy, making classroom

instruction somewhat limited in scope but perhaps easier to manage. Theorists in the 1980’s

rightly shifted that thinking to include other elements that were previously ignored. For

Page 15: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

8

example, Canale and Swain (1980) suggested a theory of communicative competence that,

besides grammatical competence, comprises three additional areas: sociolinguistic, discourse,

and strategic competence. These three areas highlighted a need for teachers to move beyond just

grammatical forms by including a look at overall communication and fluency. Riggenbach and

Lazaraton (2001) explain that because of wide adoption of this theory (widely referred to as the

“communicative approach”) almost all teachers now use materials that reflect the need to “mirror

the authentic communication that occurs in the real world.” Thus, activities that reflect real

language rather than “one characterized mainly by input that has been modified or simplified”

provides more opportunities for learning since speakers focus on the immediate task of

communication itself in “real time” (p. 125-6). In practice, today’s modern language classroom

now typically includes much more than grammatical elements. This leads to an interesting

dilemma: while it may seem reasonable to include all the features that truly comprise speaking,

this often creates an overwhelming burden on classroom instruction that is already limited by

time.

Another early theorist, Stephen Krashen, posits a theory that influences communicative

competence as well, and may indeed provide a shortcut. He suggests that the improvement of

speaking comes not so much by speaking itself as by receiving comprehensible input in the form

of watching and listening to others speak, either through daily interactions, videos, movies, or

any other form (see Ellis, 2012, p. 246-252 for a full discussion). Years later, after a number of

studies, Krashen’s theory still boasts a number of supporters. In a review of dozens of studies

for and against his and competing theories, he states that “evidence for the Input Hypothesis

remains very strong. In second and foreign language acquisition method comparisons, students in

classes containing more comprehensible input consistently outperform those containing less”

Page 16: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

9

(Krashen, 1994, p. 46-7). In other words, students who receive real, authentic, and level-

appropriate input tend to perform better than students in other classrooms.

Krashen (1994) asserts that for input to be comprehensible, it must be suited to the level

of the individual. In this light, EnglishCentral might be seen as a distributor of the kind of input

that learners might need in order to become better speakers themselves. Features that seem well

suited to helping students receive comprehensible input include videos ranked by degree of

difficulty, the ability to repeat and slow down phrases, and a dictionary guide that hyperlinks

every word to its definition.

In light of this discussion, one criticism of Krashen’s theory, however, deserves some

attention. White argues that “certain types of generalizations which learners make cannot be

unlearnt simply by understanding input” (Ellis, 2013, p. 251-2). White goes on to explain that

what is needed in these cases is negative evidence (also referred to as corrective feedback). This

means that for learners to acquire language, especially when they have overlooked or

overgeneralized a rule, they must receive correction. On the contrary, students who receive no

correction may never notice certain elements in the input they are receiving, and thus never attain

target-language proficiency.

EnglishCentral may boast some features that might help students in this regard. Since

EnglishCentral provides vocabulary and pronunciation feedback, it is likely that students will

better monitor their own progress and know where they are likely to make mistakes. With these

tools, students may better adjust their learning. While not the sole focus of this study, some

attention will be paid in assessing the effectiveness and use of this feedback.

Page 17: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

10

Student Motivation

Most ESL teachers readily accept the idea that motivation is key to learning a language.

One prominent theorist, R.C. Gardner, states that language learners must have “the combination

of effort plus desire to achieve the goal of learning the language” (Ellis, 2001, p. 509).

In second language acquisition, theories of “integrative motivation,” first put forth by

Gardner and Macyntire (1991) suggest that learners who have a positive attitude toward the host

culture and language tend to outperform those who do not. Integrative motivation also suggests

that those with positive attitudes toward the learning environment should outperform those who

do not (see Gardner, 1980; Gardner & Lalonde, 1985).

While testing this construct initially yielded mixed results, this is likely because of the

difficulty of measuring and defining the concepts themselves (see Au 1988 for a brief review).

More recently, Gardner, has reasserted the position that integrative motivation is a primary factor

for L2 achievement, and thorough results from a 2003 study showed that regardless of the

learning environment (whether learning English in an English speaking country or learning

English as a Foreign Language), there is a strong correlation between the two (Ellis, 2012).

Motivation in Online Learning Environments. Online learning may sometimes create greater

gains in motivation. As was discussed previously, Robinson and Hullinger (2008) demonstrated

that online learners reported deeper levels of engagement than their on-campus peers. On four

measures of engagement, including academic rigor, active and collaborative learning, enriching

educational experience, and student-to-faculty interaction, online students reported higher levels

than both freshman and senior on-campus students.

Furthermore, there may be a particular kind of motivation that is more prevalent in online

learning. Maehr (1976) introduces the term “continuing motivation” to reflect the concept that

Page 18: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

11

learners with this kind of motivation “reflect an ongoing willingness to learn” (p. 443). While

teachers in traditional classrooms can report this kind of motivation when, for example, students

‘stay after class,’ time constraints in the traditional classroom often restrict this kind of

motivation. Since online learning is often free from such time constraints, it may prove

advantageous. Learners, in this sense, are put in control of their own learning, and with fewer

time constraints, continue with online lessons well beyond a teacher’s requirements.

This concept is tied into a larger argument about learner control. While it will be argued

later in this discussion that too much learner control is negative, learner control can indeed create

better gains in motivation. As Kinzie and Sullivan (1989) explain, by giving learners more

control over the learning process; competence, self-determination, intrinsic interest, and even

greater personal meaning may result. In their study, they demonstrate that, even though pre-and

post-test performance between subjects remained the same, subjects who received a learner-

control treatment were more favorable toward the learning program. They state that the results

show a highly significant difference in continuing motivation favoring learner versus program

control.

While online learning may prove advantageous in some studies, it is important to concede

that other studies demonstrate that online learning can be detrimental to learning outcomes.

Online learning is particularly susceptible to student criticism, especially when the technology

fails or the user misunderstands how to use the technology and, therefore uses “poor approaches

to learning with online resources” (Ellis, Marcus, & Taylor, 2005, p. 239). Often students do not

perceive the online learning tools intuitively, and as a result, cannot recognize any given tool’s

usefulness. Other problems include the fact that institutions often have a wide variety of teacher

Page 19: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

12

experience with technology (Wilson & Stacey, 2003), and that training requires economic

considerations as well (Borko, Whitcomb, & Liston, 2009, p. 4).

Furthermore, teachers now have additional competencies to master, which, according to

some proponents, includes new roles such as “content facilitator,” “technologist,”

“manager/administrator,” adviser/counselor” and so forth. Teachers often feel overburdened by

the many roles they face in the classroom, and often resent being given these additional roles

(Goodyear et al., 2001).

In the current design, the question remains as to how to properly motivate students in the

EnglishCentral environment. On the one hand, giving students charge of their own learning, in

some studies, has proved beneficial. However, it is also possible that the use of online

instruction is demotivating in a classroom setting, especially if teachers and students see it as an

imposed instrument that takes away teacher and student autonomy. This study intends to provide

information as to what system (learner control or system) provides more motivation to students.

This will be discussed in more detail in chapter two.

Learner Control

Learner-control (LC) online instruction has often been assumed as superior to instruction

that is system-control (SC). Many studies confirm this assumption, demonstrating the

effectiveness of computer-based instruction in various forms including adaptive hypermedia

(Brusilovsky, Kobsa, & Nedjl, 2007), adaptive multimedia learning (Kalyuga, 2008), and

intelligent tutoring systems (Anderson, Corbett, Koedinger, & Pelletier, 1995; Mitrovic,

Koedinger, & Martin, 2003; VanLehn et al., 2005). While the superiority of LC instruction is

often presumed, there are some studies that suggest there is no particular advantage (Corbalan,

Kester, & van Merrienboer, 2009; Swaak & De Jong, 2001) and perhaps even demotivating

Page 20: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

13

(Katz & Assor, 2007). Some researchers have gone on to suggest that the entire field of study for

learner control is flawed because the definitions of learner control are too varied, and the ability

to study such diverse abstractions are impossible (Reeves, 1993). However, a clear picture about

learner control studies has emerged in regards to learner expertise. While students often enjoy

learner control, whether or not learners actually receive benefit from such control depends upon

their level of expertise. Thus, designers are encouraged to balance between learner and system

control based on this simple dynamic.

As Corbalan et al. (2009) explain, learners must have free cognitive resources because

LC posits an additional load on cognitive processing. Researchers conceptualize this need for

free resources as prior knowledge of the material presented. This greater skill or knowledge may

have a dual effect. First, since the more experienced or skilled group tends to have less cognitive

load, additional LC features are not as intrusive or disruptive (Williams, 1996). Second, since

the more experienced group is unlikely to want to listen to already understood material, system-

control environments can be very de-motivating, whereas LC environments can increase

motivation and create better learning outcomes (Cordova & Lepper, 1996). Thus, the ability to

skip past already understood material becomes a tool that increases productivity (Clark, Nguyen,

& Sweller, 2006). Mayer and Chandler (2001) also demonstrated positive results from a learner

control model, showing that properly segmented instruction can help to lower cognitive load and

create better learning outcomes.

However, this leads to one of the most critical queries for this study. Are advanced

language learners experts in online education or not? One might posit that because they are

advanced language learners that they might enjoy and learn more in a learner control model.

However, it may also be that advanced language learners, because they are still not completely in

Page 21: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

14

command of language processes, struggle to understand a website that is not provided in their

primary language. Thus, controlling certain aspects of the process might increase both

enjoyment and learning.

To complicate matters, language level may not be the only variable that increases or

decreases cognitive load for this particular group of students. ESL students who report greater

degrees of comfort in online environments and have more experience with online learning may

indeed outperform those who report less comfort with the EnglishCentral environment, thus

demonstrating that expertise in online learning environments matters as much if not more than

language level itself.

Since it is unclear how much control to provide advanced English-as-a-second-language

learners, this study demonstrates two differing amounts of learner control. What follows is a

review of learner and system/shared control principles that undergird the current study’s basic

design. First, a review of the advantages and disadvantages of learner control is given, and then

a discussion of system and shared control will help inform the current design.

The basic argument for learner control touts that LC environments give learners the

ability to find information when and how they want it. Judy Kay (2001) suggests defining

learner control as an instructional technique for learners to “control the learning process” and

“actively [construct] their own understanding of a learning domain” (p. 114). Under this

definition, learner control might be seen in terms of an environment wherein learners are free to

select what information they see and in what sequence (Lawless & Brown, 1997). Such tools

might include the ability to stop and start, skip, review, speed up, and slow down the material as

it is presented. These “control” tools, in turn, theoretically give rise to better learning gains.

Page 22: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

15

In an attempt to demonstrate how LC tools interact with the learner, Kay (2001) put forth

a simple diagram. In a condensed version of her diagram below (Figure 1), a learner is shown

interacting with a number of tools (domain and generic), and an accompanying support system

(teachers, simulated peers, and peers).

Figure 1.1: Kay’s Learner Control Model

The varying strength of the lines (some dotted, some bolded) are meant to suggest how

student use of any one particular tool varies, and highlights the concept that in an LC system,

learners can ignore, use, or obsess over any of the multiple opportunities available to them (for a

more detailed description, see Kay 2001).

Learner Control and the ESL student. Several concerns about learner control appear in the

literature with regard to English-as-a-second-language students. Some researchers in cross-

cultural studies suggest that Asian students, such as Japanese or Chinese students, may not wish

to engage in the kind of do-it-yourself learning that comes from largely Western perspectives

(Heinle, 2003; Iyengar & DeVoe, 2003; Oishi, 2000). If true, learner control might be seen as a

poor fit for those who do not believe strongly in self-directed learning behaviors. However, in

two studies on Chinese learners, researchers found the opposite. They note that, among these

Page 23: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

16

Asian students, “autonomous study motivation positively predicts adaptive learning attitudes and

academic success…whereas controlled motivation was associated with higher drop-out rates

[and] maladaptive learning attitudes” (Vansteenkiste et al. 2005, p. 468).

Other possible problems of learner control for ESL learners include disorientation,

referring to the inability of students to navigate an online learning system, distraction by the

environment (Scheiter & Gerjets, 2007), and lack of instructor knowledge to act as technical

support (Peng, 2010). Above all, what these issues demonstrate is a need to clearly design LC

environments, and to ensure that learners and instructors are aware of and able to manage the

navigational tools given them.

Other Issues with Learner Control. One important aspect that was touched on briefly in the

discussion on expertise is whether or not learners perceive the opportunities that the controls

represent (Corbalan, Kester, & Merrienboer, 2009). As one might conclude from the concept of

increased cognitive load, a technological novice might encounter a number of useful features that

could truly increase understanding, yet that learner may perceive the helps negatively because

they are seen as distracting or overwhelming. To state the point generally, learners might see

certain features but fail to see them as advantageous simply because they do not understand or

perceive the benefit associated with the feature. Thus, learners fail to recognize the opportunities

given to them (Perkins, 1985). This inability to recognize opportunities may be exacerbated by

the fact that learners in online environments tend to rush to completion of a task rather than

consider the task as a time to explore. If learners see learner control as empowering, important,

and useful, than the chance for the tools to be used increases. In fact, even when the controls

given are not associated with the learning outcome, learners often perceive the program more

positively and, in turn, perform better as a result (Cordova & Lepper, 1996; Katz & Assor, 2007;

Page 24: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

17

Kinzie, 1990). This may suggest that focusing on superficial features of control rather than

actual content choices may indeed create an illusion or feeling of control that, regardless of the

amount, may benefit students. Such features might include access and control over personal

information (what should my user name be? What picture do I want for an avatar?), access to the

user model (for example, the ability to access what is being measured and why), and detail views

that demonstrate what a learner has achieved (Kay, 2001).

Finally, in actual practice LC systems can be difficult to design. One reason for this is

simply due to the lack of clear separation between learner and system control. While it might be

obvious in certain contexts that activating or deactivating certain features give learners more or

less control (for example, a hyperlinked dictionary), there are always a number of controlled

features in any one system. Thus, the dichotomy between LC and SC environments is more of a

theoretical construct for researchers than a reality (Reeves, 1993). In fact, most researchers

would agree that the point is not whether LC is better than SC environments, but how much

control a particular group should be given in a particular online environment (Clarebout &

Vandewaetere, 2011). To this end, some studies are designed to show which LC features are

useful and which represent too great a cognitive load (e.g. Hasler, Kersten, & Sweller, 2007).

Ultimately, most researchers studying online learner control have identified the importance of

striking a pedagogical balance between these two. In the current study, two competing models

were developed that closely mirror the models most often used in the English Central

environment. These should be rightly called a learner control model (which gives students

access to choose from thousands of videos) and a shared control model (which limits the ability

to choose content, but still allows certain amounts of learner control).

Page 25: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

18

Shared Control

The argument for shared control stems from the idea that too many learner controls can

be ineffective or even detrimental (Corbalan, Kester, & van Merrienboer, 2009; Katz & Assor,

2007; Swaak & De Jong, 2000). In addition, shared control offers designers the chance to

carefully construct the material and pacing of an environment. While learner control can be seen

as beneficial to learners with certain characteristics, it holds true that system control may be

beneficial to those students who do not have those characteristics. As was discussed earlier,

expertise is a characteristic of those who both enjoy and perform well in LC systems (Corbalan

et al, 2009). In most of these same studies, what is being suggested is that those who are not

experts may not enjoy or perform as well as their expert counterparts (Williams, 1996). In this

line of thinking, it may hold that ESL learners, as non-experts in English, may find open

platforms designed in English for English too unstructured and frustrating.

The case for controlling certain number of variables in the English language environment

is heavily made by ESL theorists. ESL professionals have long suggested that classrooms be

controlled language labs, with guided content to help learners engage in material in an organized,

systematic way. Such variables that are often controlled involve language level, content and

context, high interest, and vocabulary.

Language Level. Perhaps of principle concern for an ESL environment is the need to control the

language level of the environment. Stephen Krashen, whose comprehensible input theory was

discussed earlier, proposes that a learner must engage in language that is just beyond the

learner’s current language system (1985). In this hypothesis, he claims that acquisition results

from comprehensible input, meaning that without linguistic simplification and the help of

“contextual and extra-linguistic clues,” a learner is not likely to understand the information

Page 26: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

19

given. Thus, adhering to the language that fits the level of the learner becomes a primary goal of

any curriculum (Ellis, 2013, p. 246).

Content, Context, and High Interest. Another principle concern is the need to control for

content. Some theorists suggest carefully building language curriculum around relevant themes.

For example, concrete lists for themes have been created with linguistically diverse students in

mind (see Kucer et al, 1995; Whitmore & Crowel 1994). Freeman and Freeman suggest that by

carefully choosing content, English learners more easily develop academic language because

certain terms repeat naturally during the theme study (1998). Furthermore, pre-selecting content

that matches an ESL student’s background is often seen as key to both motivation and learning.

By adjusting the purposes for learning English to the kinds, types, and genres of English,

students are more likely to advance in their own areas of need and expertise. As Brown (2001)

points out, many of the current titles in ESL, especially when learners move beyond basic levels,

offer theme-based courses. English for Special Purposes (ESP) is a growing field that addresses

that precise belief, with hundreds of offerings such as nursing, aviation, business, and academic

coursework.

Vocabulary. Another area that is often controlled is vocabulary. Whereas the English language

contains well over 100,000 words, a number of textbooks try to engage students in the more

frequently used vocabulary such as the AWL (Academic Word List). Several studies have

demonstrated the need to introduce vocabulary that learners are likely to encounter at their

current language level and context, and to introduce those words using recycling (Nation, 2001;

Singleton, 1999). Recycling is a term that refers to the strategy of demonstrating vocabulary

words in more than one context, with the belief that earners who are provided chances for

Page 27: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

20

repeated use of the same word will more likely retain those words and use them correctly in

context (Murcia, 2001) (see Gass et al., 1999).

Research Questions

The research questions for this study are as follows:

1. In addition to 168 hours of classroom instruction, does the use of EnglishCentral

as a homework tool create better learning gains in speaking fluency for the

treatment groups (learner and shared) vs. the control condition?

2. Is the shared-control or learner-control system in the EnglishCentral environment

better at achieving learner gains in speaking proficiency?

3. Is student attitude, operationalized as the combination of motivation, ease of use,

and feelings about technology, affected by the learner control and shared control

models? Do other variables such as age, gender, first language, and teacher

affect learning outcomes?

Page 28: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

21

CHAPTER 2—METHODOLOGY

Participants and Design

The American English and Culture Program is an intensive English program dedicated to

providing academic, cultural, and English language immersion for aspiring collegiate students. In

2012, 1,172 students came from fifty-three different countries. Top ten countries include Saudi

Arabia (471), China (143), South Korea (97), Japan (81), Kuwait (71), Taiwan (59), United Arab

Emirates (41), Indonesia (39), Qatar (37), Brazil (13), and Mexico (13). Most students have

student visas (F-1), although forty-one students came on work (J-1) or other types of visas.

Advanced 1 Students represent one of the largest portions of the program population

(about 15-25%). These students have attained a high level of English, often equating to around a

450-500 on the TOEFL exam. Such students typically have desires to study business,

engineering, or the hard sciences. While there is a great diversity among them, these students

also tend to come from backgrounds wherein English study was a requirement in their home

country’s educational environment. Students are placed in two core classes (Reading and

Writing, Listening and Speaking) and choose from an array of electives to round out their study.

Students are required by visa regulations to complete at least twenty-one hours of study.

Students at the Advanced 1 level are placed into classes with a class size averaging fifteen

students.

As previously discussed, the current design consists of a single independent variable with

three levels. These levels correspond to learner-control, shared-control treatment, and a no

treatment control. Prior to the study, participants were placed into a total of twelve classrooms by

a school administrator, who made concessions for gender and country of origin (seeking to have

equal numbers for each class). Afterwards, each classroom was randomly assigned to a

Page 29: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

22

condition. Instructors for each classroom were then informed of the subjects’ conditions within

their own classrooms. All instructors in the study were ESL professionals with at least five

years of experience and a master’s degree in TESOL or a related field.

Design of Learner Control

In the learner control model, learners were given open access to the entire repository of

videos available on EnglishCentral. Through a teacher script (see Appendix D), they were

instructed on how to select videos, but were explicitly given a charge to find any video they

wished. To ensure approximately equal amounts of time spent learning English online each

week, learners were given benchmarks set by an internal English Central system. On average,

English Central’s platform has determined that for four videos watched, students receive

approximately 14.7 hours of instruction. Thus, participants were each encouraged to watch four

videos, each at least two minutes in length. Additional benchmarks are also included to ensure

that students are using the speaking component. Again, EnglishCentral’s platform has derived

an average of 1,000 speak points per two-minute video.

Speak points are assessed by grading how similar a speaker sounds to a database of

native speech segments. EnglishCentral developed and uses a system they refer to as the

“Speech Assessment System.” It uses speech recognition software based on over one hundred

million utterances to “provide users a consistent measure of pronunciation quality” (English

Central, 2015).

EnglishCentral reports the following:

Students are graded by how similar they sound to native speech. When the student's

speech differs significantly from that of a native speaker, the grade is reduced. The

places on the line that differ are highlighted by the word turning yellow. The student

Page 30: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

23

should click on yellow words, and compare their speech to that of a native speaker. The

system will help the student identify differences by highlighting the locations in the word

that differ. The student should try to imitate the pronunciation of the native speaker.

Students can gain up to a maximum number of points for each line spoken. These

maximum possible points are different for each line, and are set by the difficulty of the

line. Every time the pronunciation of a student differs significantly from that of a native

speaker, points are deducted from those awarded. Perfect native speech will earn the

maximum possible points. Speech that differs from native will earn less. As the dialog

proceeds, the maximum and awarded points accumulate. The student is graded by the

ratio of the earned points and the maximum possible points. This grade is displayed after

every line. After the final line, the student is given a grade, and earned points for the

dialog. (EnglishCentral, 2015)

Additionally, participants were required to learn twenty vocabulary words every week. This

number was derived from the fact that each video automatically highlights five vocabulary words

and creates interactive exercises to recycle (a linguistic term that means to periodically reinforce

words in order to enhance recall). EnglishCentral explains briefly the process of choosing

words based on the videos in its database:

Scientific Approach to Word Choice. The EnglishCentral vocabulary learning system is

based on an analysis of over 400 million words. The EnglishCentral vocabulary learning

system uses this analysis to identify and rank the most important vocabulary words by

difficulty level, and then tracks every word the learner sees, speaks, and studies over time

to track and accelerate learning. These words can be sorted and learned by frequency, by

correlation to test scores such as TOEFL or TOEIC, by goals such as learning business

Page 31: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

24

English or English for movies, or by studying the specific words contained in

particular videos. Research shows that spaced repetition is the most efficient way to

learn new words quickly. This learning technique is prominently used in flashcard based

system. EnglishCentral goes farther by providing spaced-repetition learning in the

context of authentic videos. Providing authentic context increases retention and improves

the experience. EnglishCentral’s computer-adaptive learning system teaches words in the

authentic contexts in which they occur, allowing learners to develop definitional and

collocational knowledge of the words they are studying in a wide range of contexts.

(EnglishCentral, 2015).

Design of Shared Control

In the shared control environment, subjects were given videos carefully selected by a

team of four ESL veterans. This team of curriculum designers had all taught the Advanced 1

Listening/Speaking course multiple times each, and all held the highest rank in the teacher

organization (lecturer). In addition, they held positions of authority including curriculum and

computer coordinator. All of these teachers hold a master’s in TESOL or a related field and have

at least five years of ESL experience. This team was selected by the director of the program, who

expressed interest in creating a channel for the Advanced 1 level. This team selected a total of

forty eight possible videos corresponding to twelve themes (four videos per theme) for students

to watch over the eight-week period (see Appendix A). A serious vetting process to select

videos included a look at the level of each video as well as content, high-interest, and

vocabulary.

Content. Videos were selected ahead of time to mirror those themes that are discussed in

the classroom on a week-to-week basis. Themes were as follows: Communication Studies

Page 32: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

25

Child Psychology, Sociology, Business and Global Marketing, Cognitive Psychology,

Anthropology/Biology, Astronomy, Political Science, Linguistics, Economics,

Biology, and Sociology. Teachers generally chose a new theme each week, and with twelve

themes to choose from in seven full weeks of instruction, this means that a total of twenty-eight

videos were required for students to watch (which matches precisely with the number of videos

required for the learner control condition). With a total of twelve possible themes, some

variability existed for teachers to choose which theme would best match their students’ interest,

and this was deemed an important choice: to provide teachers with the ability to select themes

that best represented what they wanted taught in class.

Level. While videos that matched the theme and content were of primary interest, the

four ESL veterans were asked to look mostly for advanced level videos of a consistent length.

Each unit of four videos contained academic language that mirrored the language used in

advanced level classrooms. Of the forty-eight videos, thirty four were identified by English

Central as being an advanced level, and an additional twelve were identified as intermediate

level. Two videos were identified as beginning videos, but were ultimately deemed acceptable

because the content of those videos matched the curriculum precisely.

High-interest. During the selection process, videos were also chosen based on high

interest. To do so, the four ESL veterans, after selecting a number of videos, were asked to

remove any videos that would not be considered “high interest.” With this selection of videos,

the scores were further reviewed by two student workers with backgrounds in TESOL. These

workers reviewed each video and assigned a score between one and ten (one meaning low

interest, and ten high). Finally, the researcher also gave high interest scores based on his

Page 33: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

26

experience in the Advanced 1 classroom. The mean score of the final videos given was 6.94, and

no video selected scored under five.

Vocabulary. Finally, the four veteran ESL instructors selected vocabulary within the

videos as well. The veterans reviewed each video and selected the vocabulary most pertinent to

each theme. These vocabulary words were shared with English Central designers, and were

consequently included in the vocabulary quiz section of the English Central player. Thus, in the

shared control environment, all students shared similar vocabulary input as their classmates.

While the 48 videos were chosen to reflect the chapters of the textbook used in class, a

concession should be made that instructors in the study (not the 4 ESL veterans, but the 12

instructors) are allowed to choose which chapters to focus on in any one class. This follows

natural policy within our organization to have teachers have semi-autonomy over their own

curriculum. Thus, while the shared control model gave instructors 48 videos to choose from (4

videos for 12 chapters), instructors choose 28 videos (7 chapters) that they deemed appropriate

for their classroom.

Design Similarities in Both Conditions

The video player (see Figure 2.1) is a constant feature regardless of condition, and was thus

given to both the treatment groups. This gave a certain amount of learner control to ALL

participants, such as the ability to:

1. Pause, stop, and go back to previous segments during the watching of a video

2. Slow down videos (slow down the rate of speech)

3. Review previously watched videos

4. Click on any particular word for a hyperlinked dictionary

Page 34: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

27

The video player follows a tripartite structure (watch, learn, and speak), meaning that all

students must first watch the video uninterrupted, then learn vocabulary words in a gap fill

activity, and then mimic the speech from the video in order to practice pronunciation.

Figure 2.1: Video Player (in both treatment conditions)

All learners in the treatment groups also had access to their personal information, which

includes up-to-the-minute reports detailing overall course completion, watched videos, learned

words, speak points, spoken videos, pronunciation feedback, lines recorded, and overall grade

(based on the internal benchmarks). A brief overview indicating overall amount of control for

both treatment groups is provided in Table 2.1.

Table 2.1: Learner Control Elements in Each Format of the Present Study

Control Element Learner-controlled

Format Shared-control Format

1. Personal information (user name

and password)Yes Yes

2. Pause, stop, slow down, review Yes Yes

3. Access hyperlinked dictionary Yes Yes

Page 35: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

28

4. Access reports Yes Yes

5. Review top scores in your class Yes Yes

6. Vocabulary Learning Feature

(databank of words learned

called “My Words”)

Yes Yes

7. Select content (videos) Yes No (selected by ESL

professionals)

8. Select vocabulary Yes No (Selected by ESL

professionals)

9. Vocabulary Learning Feature Yes Yes

10. Pronunciation Learning Feature Yes Yes

11. Select amount of content No No

12. Select language level No No

English Central Online Learning Tool

EnglishCentral’s basic video player format involves a tripartite structure called

Watch/Learn/Speak. This format allows for students to interact three times or more with a video.

In all three parts of the format, closed captioning appears below the video screen. As previously

indicated in chapter 2 (Table 2.1), both treatment groups will be given access to all features of

this video player.

Watch Feature. In this format (see Figure 2.2), a video is first watched in its entirety on

a video player with certain navigational controls. Students are able to speed up, slow down, and

replay each video and/or portion of the video in the watch format. In watch mode, under the

video itself closed captioning appears and key words are underlined. The text appears just above

a green bar, which demonstrates the length of each segment of the video. Videos are typically

between two to ten minutes in length, and vary in language level, which is indicated below the

player to the left. In the channel format, however, items such as length and level can be more

easily controlled. The popularity of a video is indicated below the player to the right, which can

Page 36: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

29

be gauged by both number of views or Facebook likes. Finally, a download button is also

available just to the right of the popularity measure.

Figure 2.2: Watch Feature (in both treatment conditions)

Learn Feature. The format that follows (Figure 2.3) allows students to listen to the video

again, but with an additional task. They are invited to type in key words as they hear them. The

word may be listened to in the context of the phrase as many times as the learner chooses. Each

space represents a single character in the word, and words must be spelled correctly. A

misspelling is indicated after a word is spelled by highlighting in red the characters that were

incorrect, with the dictionary showing correct spelling (as well as the definition and

pronunciation).

Page 37: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

30

Figure 2.3: Learn Feature (in both treatment conditions)

There is also a databank (see Figure 2.4) which will record all words reviewed in the learn

feature so that students can continue using the words in context even after they watch the video.

This quiz feature allows students constant review with the vocabulary that is targeted by the

curriculum/channel.

Figure 2.4: “My Words” Vocabulary Learning Feature (in both treatment conditions)

Page 38: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

31

This feature can be found while on any screen within the website on a tab just below the

main header called “My Words” (see Figure 2.5). A red dot to the side of the “My Words” link

indicates that there are words yet to be mastered within the My Words space, and the number of

words in the databank is indicated just to the right. Students can review words learned at any

time. An additional quiz feature is provided in the shared control environment, in that a quiz

feature link appears after each week’s series of videos for learners to review words from that

week.

Figure 2.5: “My Words” Vocabulary Learning Feature (in both treatment conditions)

Speak Feature. The last feature (found in Figure 2.6) of the video player is perhaps the

most technologically sophisticated . Here learners are required to speak or mimic the phrases of

the video. Voice recognition software is sensitive to accent and gender, and calculates a score

and provides feedback on the sounds that are in need of improvement.

Figure 2.6: Speak Feature (in both treatment conditions)

Page 39: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

32

In association with this feature, a learner can also attend to a databank of sounds that are

identified as “the most difficult” by looking at the pronunciation toolbar found in the header

(Figure 2.7) to the right of “My Words.”

Figure 2.7: Pronunciation Toolbar

Sounds, indicated by the phonemic alphabet, are shown in green (if mastered) or red (if

un-mastered). By clicking on this pronunciation tool, a learner will go to a databank (Figure 3.8)

that gives access to resources for a learner to work on specific sounds based on individual needs.

A learner can select a discrete sound which will attach to a series of videos for improvement on

that sound.

Page 40: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

33

Figure 2.8: Pronunciation Learning Feature

Shared Control. A shared-control model on EnglishCentral involves the careful

selection of videos by a school into a “channel.” This model emphasizes the teacher’s ability to

choose from EnglishCentral’s repository of videos and select specific videos based on interest,

topic, and level. Often this model is used to create a course that follows a particular course (for

example, by following the chapter themes in a textbook). In the example found in Figure 2.9, the

American English and Culture Program designed a course called Advanced 2 Listening/Speaking

Channel, with units that correspond to chapters of a textbook covered in class. Units may be

composed of as many videos as the curriculum dictates. In this particular example, each unit

contained four videos each, which, on the whole and based on video length, corresponded to 2.1

hours of study.

Page 41: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

34

Figure 2.9: Shared-Control Design: The Advanced 1 Listening and Speaking Channel

Learner Control. Finally, rather than creating a channel, learners themselves can be

given access to choose videos of interest themselves. In this model, learners were given goals or

benchmarks (Figure 2.10). Notice that the amount of hours matches precisely with the amount

of hours in the shared-control design (2.1 hours).

Figure 2.10: Shared-Control Design: The Advanced 1 Listening and Speaking Channel

Page 42: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

35

After setting goals, teachers can invite students to look at the large repertoire of videos

found in the “Browse” feature of EnglishCentral (Figure 2.11). EnglishCentral has a wide

variety of videos that students can look for according to topic, key word search, channels, and

pronunciation.

Figure 2.11: Shared-Control Design: The Advanced 1 Listening and Speaking Channel

Control Group. The control group also received homework assignments, but not using

the EnglishCentral system. Teachers reported using speaking assignments such as interviews

and audio discussion boards. Teachers also reported listening assignments such as watching

videos from Tedtalks.com and other online sources. Often, note taking was used to assess

students’ fidelity in following the out-of-class assignments. The average amount of homework

assigned was approximately 2.3 hours, meaning there was slightly more treatment in the control

group.

Page 43: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

36

Materials

At the start of the class instruction, instructors in the treatment condition all received

access codes to either the shared-control or learner-control version of the EnglishCentral

platform. Within the first week, access codes were assigned to subjects through teacher-student

email. A link sent to each student allowed the student to enter into an online classroom wherein

the teacher could monitor student activity. Since the use of headphones with a microphone was a

materials requirement, subjects that may have been without a built-in microphone on their

computer were given headphones.

Measures

Versant Test. The Pearson Versant English Test is a well-known test in the English as a

Second Language field. Its use varies from corporations (to evaluate the speaking proficiency of

employees) to academic institutions (to evaluate subjects). Versant boasts a high degree of

reliability and validity, and at the overall score level, the test scores “are virtually

indistinguishable from scoring that is done by careful human transcriptions and repeated

independent human judgments. The correlation between the two is 0.97” (Versant website).

The Versant test is a 62-item test conducted over fifteen minutes and is composed of

reading, repeats, short questions, sentence builds, story retelling, and open questions. It has a

range of scores from 20-80 and reports sub scores on sentence mastery, vocabulary, fluency, and

pronunciation. It is a randomized item test and can be delivered over phone or computer. It is

comprised of six sections (A-F). For purposes of this study, the Versant was taken over the

phone (since some students at the beginning had limited access to headphones with

microphones). Students are given an access code with instructions on a sheet of paper, and are

Page 44: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

37

then instructed to take the test within a certain time frame. The test may be taken on any phone,

although a stable connection is highly recommended.

For full sample test items, please see Appendix C.

The Student Attitudinal Survey. The attitudinal survey is a three-part thirty-item survey

with twenty-seven questions in a Likert scale format. The Likert scale has a range of five, and is

scaled from one (strongly disagree) to five (strongly agree). Questions in Parts 1 and 2 were

adapted from the System Usability Scale (Albert & Tullis, 2008). Sauro reports the SUS to have

a high degree of reliability, with a Cronbach’s alpha of .92 (2011). The survey includes nine

questions about usability (2, 3, 4, 5, 6, 7, 10, 12, 13), all with the intent to demonstrate student

feelings about how usable the platform was. Questions 2-7 asked students to reflect on the

complexity or simplicity of the tool and whether or not it was perceived as user friendly.

Question 10, 12, and 13 were all quality questions. Question 10 asked about the quality of

videos themselves. Question 12 asked about the quality of customer support, and question 13

asked if the time for completing the EnglishCentral assignment was sufficient. Questions were

deliberately randomized.

Table 2.2 Usability Questions

2. I found this website unnecessarily complex.

3. I think that I would need help to be able to use this website.

4. I found this website very awkward/confusing to use.

5. I would imagine most people would learn to use this

website very quickly.

6. I feel very confident using this website.

7. I needed to learn a lot of things before I could begin

with this website.

10. The quality of the videos were very good.

12. I was satisfied with customer service at English Central.

13. The amount of time given to complete English Central assignments was enough.

Page 45: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

38

Ten questions about motivation were also included (1, 8, 9, 11, 15, 16, 17, 20, 22, 23).

These questions aimed to understand student engagement and enjoyment of the tool, and whether

the tool (and subsequent parts) was perceived as one that was useful outside of the classroom.

Table 2.3 Motivation Questions

1. I think that I would like to use this website frequently.

2. I enjoyed using this website.

9. I would recommend this website to a friend,

classmate, or family member.

11. I will keep studying with English Central after my

class ends.

15. The watch feature helped me learn English.

16. The learn feature helped me learn English.

17. The speak feature helped me learn English.

20. I enjoyed this class (course).

22. This course helps me improve outside of class.

23. This course will help me for a long time.

Five questions were also given about technology in general (21, 24, 25, 26, 27). These

questions. These questions were created to understand student views of technology (websites,

online learner tools) in general.

Table 2.4 Technology in General Questions

21. The technology in this course was good.

24. I like courses with technology.

25. I have used websites to learn English in the past.

26. I am comfortable using English Central Technology

27. I wish I knew more about how to use technology.

Question 28, 29, and 30 asked for demographic information (age, gender, and first

language, respectively).

Page 46: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

39

Teacher Attitudinal Survey. Teachers were given a revised version of the student

attitudinal survey. There were two parts. The first part asked questions based on usability and

motivation and excluded general questions on comfort with technology. The technology

questions were excluded since the purpose of this portion of the survey was to ascertain if

teachers’ feelings toward the EnglishCentral platform. It was posited that teachers’ feelings

might have an influence on student performance, and the survey was used to determine any

possible teacher effect caused by teacher attitudes.

Table 2.5 Teacher Attitudinal Questions (Motivation and Usability)

1. I think that I will use this website frequently. (motivation question)

2. I find this website unnecessarily complex. (usability question)

3. I think that I would need help to continue using this website. (usability)

4. I find this website very awkward/confusing to use. (usability)

5. I imagine most students would learn to use this website very quickly. (usability)

6. I feel very confident using this website. (usability)

7. I needed to learn a lot of things before I could begin with this website. (usability)

8. I enjoy using this website. (motivation)

9. I would recommend this website to a colleague. (motivation)

10. The quality of the videos are very good. (usability)

11. I will keep using EnglishCentral. (usability)

12. I was satisfied with customer service at EnglishCentral. (usability)

13. The amount of time given to complete EnglishCentral assignments was sufficient.(usability)

14. English Central helped me use my textbooks. (usability)

The second portion of the teacher survey asked three additional questions. The first

question asked teachers how many hours of homework they provided each week. The purpose of

this question was to determine equality of treatment. The second question asked for a positive

Page 47: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

40

view of EnglishCentral, and was phrased to elicit open-ended responses: How is EnglishCentral

particularly suited to English language learning. The second question asked for a negative view

of EnglishCentral, and was also phrased to elicit open-ended responses: How could English

Central be improved to better English language learning.

Semi-structured Interviews. In addition to the surveys, students and teachers in the

study were given a chance to participate in semi-structured interviews. All students, after and

during the study, were given open access to the researcher. Several students contacted the

researcher throughout the study via email. All students who contacted the researcher were given

the opportunity to share their opinions both via email and in person. All teachers in the study

were given a chance to share their idea about the platform in semi-formal interviews and emails.

To avoid creating bias, the researcher did not solicit responses from students or teachers,

although the follow up question “What else did you think about EnglishCentral?” was given.

Teachers were also given an additional question after the study, which was: “What do you think

might have contributed to your students’ success or lack of success using EnglishCentral?”

Procedure

The study ran in spring 2014, with follow-up procedures and data collection through

May. Initially, each student was assigned to one of multiple sections of a listening/speaking

course, and each course was randomly assigned to a condition. In all, twelve sections were

created, and class sizes varied from fifteen to eighteen participants, with each course meeting

daily at the same time for every section.

All participants received a Pearson Versant pre- and post-test (before and after

treatment). The two experimental groups were assigned a minimum of 14.7 hours of English

Central treatment. As explained previously, Group A received the teacher-created channel

Page 48: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

41

(shared-control) consisting of twenty eight videos while Group B received the open platform

(learner-control) and were given benchmarks that match the length of time. The control group

(Group C) received no EnglishCentral treatment but received approximate equal time through

out-of-class homework. Participants in Groups A and B, upon completion of the course,

received the motivational questionnaire (Appendix B) detailing their experience with the English

Central tool. Each teacher received a schedule identical to the one given in Table 3.11.

Table 2.5: Schedule of Events

Week 0 (Before Classes Begin)

Training of A1 L/S Teachers (1 hour)

Overview of English Central

Overview of Versant

Random Selection

Week 1

Set up English Central Accounts

Pre-Test for Groups A, B, and C

(Versant Phone Test)

Start Treatment Period (Group A and B)

Week 2

Treatment Period for Group A and B

Week 3

Week 4

Week 5

Week 6

Week 7

Finals Week

Post-Test for Groups A, B, and C

(Versant Phone Test)

English Central Survey

(Attitudinal Survey)

Post-Study Events Possible Interviews

Week 0. Before classes began, instructors were present for a training meeting on the English

Central study.

Training in week 0 was three-fold. Teachers were trained on:

Page 49: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

42

1. Consent forms. All instructors were given consent forms for their students (see

Appendix F). They were informed that while the study was voluntary, participation

in EnglishCentral would be a required and graded part of each student’s grade.

Instructors were informed to read the consent form on the first day of classes word for

word. Any questions about consent not answered in the consent form itself would be

relayed to and answered by the researcher.

2. An overview of EnglishCentral. Instructors were given an overview of the English

Central website. They were told that they had already been given access (through an

email link previously sent to each instructor), and instructors were told how subjects

would sign up for that access (instructors were required to send a link and “invite”

students to their classrooms). Instructors were also told that all subjects in the

treatment groups would receive training during the first week with the researcher and

the instructors in a computer-mediated space, thus ensuring that each student was able

to register and begin practicing on the EnglishCentral platform, and also to ensure

that each group received equal training. Teachers were also informed that all subjects

would receive access to EnglishCentral free of charge.

3. An overview of the Pearson Versant. Instructors were given separate codes (up to 20

each) for every member of their classroom. They were also instructed to inform

subjects of the deadline for completing the pre-test. Instruction guides were provided

on every code sheet (see Appendix G). Teachers were told to assign codes to each

student present in the first three days of class, and that subjects would be given their

scores as an added benefit for participation in the study. A sign-up list was provided

for each teacher to keep track of assigned codes.

Page 50: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

43

Week 1. Subjects were given consent forms upon their entry into the class. Teachers gave each

student the access code and phone number for the Versant Test, and directions were read by the

instructor on how to take the test via phone. Directions (with the phone number) were also

printed on each access code. The Versant Test was then administered over a forty-eight hour

time period.

On the last day of week one, subjects from Groups A and B visited a computer-mediated

classroom with the researcher and a student worker. Since all Advanced 1 Listening/Speaking

classes met at the same time period throughout Arizona State University’s campus, teachers

brought their students to the classroom space and each treatment group met simultaneously.

Advanced 1 Listening/Speaking classes are a total of two hours (120 minutes), so treatment

group A was given training in the first hour, and treatment group B was given training in the

second hour.

Subjects were given instructions (see Appendix D) on how to sign up to English Central,

and were also given a tutorial on how to use the EnglishCentral tool. Two sets of instructions

were provided depending on condition. In the shared control group (Group A), subjects were

instructed to choose from the videos assigned to their classroom based on the themes the teacher

would assign, and that if they completed the videos correctly, they would be able to meet the

four weekly goals found in the benchmarking tool (videos watched, words learned, videos

spoken, and speak points). In the learner control group (Group B), subjects were instructed to

choose any video. They were also told to pay attention to the benchmarking tool (found under

the curriculum tab) in order to achieve an equivalent amount of time as Group A. Both groups

were instructed that they could do more than the weekly goal, but that full marks for

participation were given upon completion of each of the four numbers.

Page 51: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

44

Week 2-7. Subjects for both treatment conditions were given a deadline at 4:59 pm each week to

complete their assignment. The first assignment due date on the first day of the second week,

and the last assignment was due on the first day of the 8th week (during finals). Subject results

were monitored by teachers, and additional reports on usage were provided by the researcher to

the instructors at week four and seven. An open-door policy was instituted for subjects and

instructors on any technical or other issues with EnglishCentral during this period. Since

EnglishCentral automatically tabulates hours, words learned, videos watched, and speak points,

teachers did not need to score student results, but were guided to understand the teacher tools and

various reports EnglishCentral provides. Students were all required to participate fully in

English Central as part of a homework grade, which was given weekly. Teachers accessed

reports in both conditions (at least) weekly to ascertain whether the weekly benchmark was

achieved.

Week 8. Subjects for all conditions were given the post-test during week 8. Any results past the

deadline were discarded. Several students again complained of faulty connections to the phone

number, and an attempt was made to provide additional access codes to a few. A report of these

students is made in chapter 4.

Scoring

The access codes provided by Pearson Versant served as identification numbers and

were used in both pre- and post-test environments, and tests were automatically scored (within

minutes) by the computer system. Scoring occurs based on different rhythms and pronunciation

patterns. The system identifies words used (content) and the pace, fluency, and pronunciation of

the words used (manner). Statistical modeling built from previously captured and rated native

and non-native speakers is then used in order to give a base measurement to the subject’s speech.

Page 52: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

45

This base measurement is further divided into four diagnostic sub scores: sentence mastery,

vocabulary, fluency, and pronunciation. Sub scores are then weighted and calculated in order to

produce an overall score. The course instructors provided the scores for the Pearson Versant to

all participants.

Scores for the motivational questionnaire were also tabulated for each individual student.

A student received scores for each of the twenty-seven Likert scale questions (one= low, five=

high) based on the System Usability Scale (Albert & Tullis, 2008). These scores were tabulated

to produce overall scores in one of three categories. Motivation, consisting of ten questions, was

tabulated as the total of all ten questions (low score=ten, high=fifty). Usability, consisting of nine

questions, was tabulated also (low score=nine, high=forty-five). General comfort with

technology was tabulated as the total of five questions (low score=five, high=twenty-five). After

controlling for gender, age, and language of origin, these three variables, overall usability

(USETOTAL), overall motivation (MOTTOTAL), and overall comfort with technology

(TECHCOMF) was used to report any shared variance with overall speaking proficiency.

An additional survey measured teacher’s response toward the online learning

environment. This survey contained fourteen Likert Scale (1 = low, 5 = high) questions also

based on the System Usability Scale (Albert & Tullis, 2008), and asked an additional four

follow-up questions to ascertain teacher’s future recommendations and overall feelings about the

EnglishCentral environment.

Page 53: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

46

CHAPTER 3—RESULTS

Assumptions

Tests to ensure equal variance were performed. First, variances of the learner-control

(29.19), shared-control (18.52), and the control group (28.26) were compared using a ratio test.

The largest group variance divided by the smallest group variance yields a result of 1.58 and

demonstrated roughly equal variances. The Levene statistic confirmed this simple ratio test with

a nonsignificant result of .791 (p=.457). Tests to determine normal distribution were also

performed. The histograms (shown in Figure 3.1) demonstrate fairly normal distributions.

Figure 3.1: Histograms of learner, system, and control groups

Tests of normality (Kolmogorov-Smirnov and Shapiro-Wilk) also showed generally

normal distributions save for one. The shared control score D(25) = .187, p = .025 does

significantly deviate from normality. This corroborates the report from the histogram, which

visually demonstrates a slight skew toward positive scores.

A look at the skewness corroborates the assumption of normality for the control group

(-.287), and the learner group (-.512). Again, the shared control group is slightly, but not

problematically, skewed (.969).

A final look at normality was performed by examining kurtosis. In this test, mesokurtic

(a score of zero) indicates a normal distribution, whereas platykurtic (flat) scores would be

Page 54: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

47

indicated by positive scores, and leptokurtic (peaked) by negative scores. Scores were as follows:

learner control: .278, shared control: 1.836, and control: 1.7. None of the scores were in excess

of two, so the distributions are within normal parameters.

Finally, several precautions were made to ensure that no violations of the assumption of

independence were made. First of all, EnglishCentral records each and every line spoken by the

student, thus ensuring that any use of EnglishCentral was performed by the subject. In addition,

each student had an individual account that recorded their progress, and these accounts were

created using passwords only the subjects themselves created and knew. Finally, since tests on

the Pearson Versant were given through the use of access codes, each test was independently

distributed. Furthermore, all tests were recorded in full, allowing a researcher to listen in on the

conversation and ensure that the subject in question took the test. The researcher cross-

referenced each pre-test score with a post-test score (164 tests in total) to ensure that subjects

voices were the same.

Primary Analysis

In the first analysis, two a priori orthogonal contrasts were used to compare the means of

the learner, shared, and control group scores in speaking proficiency (see Table 3.1 on the next

page). Effect size (Cohen’s d) was calculated for any statistically significant differences. A

description of the two contrasts follows:

Contrast 1: The first of two orthogonal pairwise comparisons compared the treatment

groups to the control group. This corresponds to the first research question.

Contrast 2: The second orthogonal pairwise comparison compared the differences in

means of the treatment groups themselves. This corresponds to the second research question.

Page 55: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

48

Table 3.1: A Priori Orthogonal Contrasts

Contrast

Learner

Control

Group (LC)

Shared

Control

Group

(SC)

Control

Group

(CG)

Contrast 1

(LC +SC vs

CG)

½ ½ -1

Contrast 2

(LC vs SC) 1 -1 0

As per design requirement for a priori orthogonal contrast coding, only k-1 contrasts

(two) were used and both these contrasts sum to zero.

The shared, learner, and control groups demonstrated differing means but approximate

standard deviations, as can be seen in the table below (Table 3.2). No significant difference was

found for Contrast 1 between the treatment groups and the control on the learning measure, t (80)

= 0.745, p = .459. For Contrast 2, there was a statistically significant difference between the

shared- and learner control groups, t (80) =2.017, p = .047, Cohen’s d = .56.

Table 3.2 Mean Amount of Improved Speaking Fluency for Learner, Shared, and Control

Conditions

Speaking Score

M SD

Shared Control 2.76 4.3

Learner Control .031 5.40

Control .500 5.31

Page 56: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

49

Supplemental Analysis

Linear regressions, one-way ANOVA’s, and independent sample t-tests were conducted

to determine other factors that may contribute to student success on the EnglishCentral platform.

Finally, follow-up data was collected to ascertain the presence of a teacher effect. Each of these

analyses is now presented.

Linear Regression 1. The purpose of this regression was to determine whether, in

addition to variance caused by the learner/system models, other possible variables in the English

Central Platform could account for learning outcomes. These included total hours spent on

EnglishCentral, the number of videos watched, amount of vocabulary learned, and the speaking

points acquired.

In this regression, the first model (learner/shared control) was a statistically significant

predictor of speaking improvement F(1,50) =4.12, p=.047, ∆R2 = .071. In the second model,

inclusion of total hours accounted for additional variance F(1,50)= 3.75, p=.059, ∆R2= .061, as

did the third F(1,50)=2.009, p=.162, ∆R2 = .032, fourth F(1,50)=1.8, p= .186, ∆R2 = .028, and

fifth models F(1,50)=.518, p=.475, ∆R2 = .008. The final model was statistically significant F(5,

50) = 2.769, p =.041, R2= .201, accounting for 20 % of the total variance.

In addition to the regression, a one-way analysis of variance (ANOVA) was calculated

with total hours spent as a dependent measure, and learner outcomes as the independent measure.

The analysis was significant, F(1, 57)= 2.364, p = .012.

Another one-way analysis of variance (ANOVA) was calculated with number of videos

watched as a dependent measure and learner outcomes as the independent measure. The analysis

was also significant F(1,56) = 3.83, p < .001.

Page 57: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

50

Linear Regression 2 and One-way ANOVA. A second regression sought to determine

the amount of variance accounted for by the motivational survey. The survey was divided into

three sections, with subsections on ease of use, motivation, and experience in technology.

Student attitude was operationalized as the sum total of all three of these subsections.

In the first model, regression of usability (ease of use) was a statistically significant

predictor of speaking improvement F(1,50) =7.465, p=.009, ∆R2 = .130. In the second model,

inclusion of motivation accounted for additional variance, but was not statistically significant

F(1,50) = .966, p=.331, ∆R2= .017. In the third model, the addition of tech experience also

accounted for additional variance but was not statistically significant, F(1,50)=.491, p=.487, ∆R2

= .009. The final model was statistically significant F(3, 48) = 2.943, p=.042, R2= .155,

accounting for 15% of the total variance.

In addition to a regression, a one way ANOVA with student attitude as a dependent

measure and learner/shared control conditions as the independent measure was performed.

While not reaching statistical significance (p = .059), there was a difference in means (learner

control 71.4, shared control = 80.9).

Gender, First Language, and Teacher Effect. Additional ANOVAS were conducted to

identify any possibly variance resulting from gender (Table 3.5) or first language (Table 3.6).

Neither ANOVA yielded a statistically significant result. Gender was dummy coded (1= female,

0 = male), and first language was dummy coded as well (Arabic = 0, Chinese = 1, Japanese = 2,

Korean = 3, Portuguese = 4, Spanish = 5).

Page 58: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

51

Table 3.5 One-Way Analysis of Variance Summary Table for the Effects of Gender on Speaking

Scores

Table 3.6 One-Way Analysis of Variance Summary Table for the Effects of Language on

Speaking Scores

One final ANOVA (Table 3.7), demonstrated a significant result. An analysis of variance

showed that the teacher assigned to each student group had a significant effect on student scores

F(11, 71) = 2.681, MSE = 1533.10, p = .006.

Due to this positive result, an additional inquiry was made to see if high levels of

motivation among teachers might have resulted in high performance for students. Mean scores

from the teacher motivation survey for teachers in shared and learner conditions were M= 52.5,

55.25, SD=7.04, 11.5. This data demonstrated that teachers in the learner control treatment had

had nearly identical scores compared to their shared control counterparts. This runs counter to

the assumption that high levels of motivation among teachers, as operationalized by the

motivational survey, should yield better learning scores for students.

Source Df SS MS F P

Between-group 1 3.031 3.031 .113 .737

Within-group 81 2166.969 26.753

Total 82 2170.00

Source Df SS MS F P

Between-group 5 116.044 23.209 .870 .505

Within-group 77 2053.956 26.675

Total 82 2170.00

Page 59: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

52

Further inquiry into this data (see Table 3.7) shows that the student learning scores with

the most positive teachers on the survey, are nearly identical to the scores of the least positive

teachers (M= 1.478, 1.449). Based on these scores, no further statistical analysis was performed

related to learning outcome and teacher motivation.

Table 3.7 Teacher Motivation and Student Speaking Scores by Learner and Shared Condition

Condition Teacher ID Student Speaking Score Teacher Motivation Score

Learner

6 -2 57

3 1.25 45

2 -1.58 58

7 4.429 61

Shared

1 .667 65

4 6.833 46

5 2.4 59

12 -.286 40

Page 60: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

53

CHAPTER 4: DISCUSSION AND IMPLICATIONS

Shared-Control Superior to Learner-Control

The most major finding of this study demonstrated that a shared control system was

superior to that of a learner control system. The shared control system gave students a clear

advantage in learning (2.76 point gain in shared control, compared to a .031 point gain in learner

control, and .5 point gain in the control group). While this may initially appear to run counter

from other studies such as Mayer and Chandler’s (2001) experiment on the superiority of learner

control, a cautious look would reveal otherwise. Mayer and Chandler’s study demonstrated that

learner control segments were more efficacious than a continuous presentation, thus giving rise

to the concept of “segmenting” as a multimedia principle. In our particular study, both the

learner control and shared control models gave students this type of control. The video

presentations were each segmented and students in both conditions were allowed to pause,

review, and slow down these segments. Thus, the segmenting principle was properly used in

both treatment groups. However, the additional control of choosing the videos themselves was

given only to the learner control group, and appears to have negated the positive effects present

by a segmenting principle. Thus, while control of the video player was an advantage for those in

the shared control group, that advantage disappeared and reversed when participants were given

an additional control, thus making it clear that control over video selection proved too much for

this student population.

In light of these findings, it would be accurate to state that video selection had a negative

effect on student scores in speaking proficiency. This tends to agree with recent research on

some of the negative impacts of choice in general. In the book, Gamification by Design,

Zichermann and Cunningham (2010) state, “In brief, enough choice is good—too much choice is

Page 61: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

54

bad…Give the [learner] just enough choice to engage him without overwhelming him” (p. 23).

Thus, as some studies have confirmed (Corbalan, Kester, & van Merrienboer, 2009; Katz &

Assor, 2007; Swaak & De Jong, 2000), a shared control model is often superior to a learner

control model because it limits the amount of choice that students have. Iyengar makes the same

conclusion, that while choice appears superficially good to most participants (see Ausburn 2011),

too much choice confounds participants’ abilities to feel comfortable about the choices made.

Thus, this current study serves as a warning to online designers and online learners: more choice

does not always lead to better results. Iyengar shares a story about the problems of too much

choice at a grocery store near Stanford University, and this story serves as a perfect metaphor for

online choice:

We decided to do a little experiment, and we picked jam for our experiment…We set up

a little tasting booth right near the entrance of the store. We there put out six different

flavors of jam or 24 different flavors of jam, and we looked at two things: First, in which

case were people more likely to stop, sample some jam? More people stopped when there

were 24, about 60 percent, than when there were six, about 40 percent. The next thing we

looked at is in which case were people more likely to buy a jar of jam. Now we see the

opposite effect. Of the people who stopped when there were 24, only three percent of

them actually bought a jar of jam. Of the people who stopped when there were six, well

now we saw that 30 percent of them actually bought a jar of jam. Now if you do the

math, people were at least six times more likely to buy a jar of jam if they encountered

six than if they encountered 24 (2011, pg. 2).

While this story is presumably about consumer choices, Iyengar goes on to suggest that

this applies to online choice in general. While online learners may feel that more choice is

Page 62: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

55

always better, if you present too many choices to online participants, they are less likely to fully

participate and are likely less motivated to make a selection. In the current study, it should be

noted that while not statistically significant at the p = .05 level, there was a 9.49 point difference

on the motivational survey (p = .059).

This goes back to the concept introduced by Katz and Assor (2007) that for choice to feel

motivating, students must feel that they are autonomous learners. Thinking in those terms, it

seems likely that students in the context of the current study may not have seen themselves as

autonomous.

So why didn’t these particular students find the additional autonomy liberating? In the

semi-structured interviews, several students commented on the post-survey that the English

Central website was too large, and that finding a video that was of an appropriate length, topic,

and interest was not always easy. Some students also complained that while selecting videos on

their own, they didn’t realize the video was not a sufficient length until it was too late, and thus,

they would have to watch an additional video in order to complete the required benchmarks.

This corroborates studies that show that disorientation ensues because of the inability to navigate

an online learning system (Scheiter & Gerjets, 2007).

In addition, it may be that without pairing the homework assignments to actual in-class

assignments, the use of EnglishCentral felt superfluous—outside of the confines of the

classroom itself. One student commented the following, “Why are we doing this? It has nothing

to do with the class.” Regardless of the fact that the class itself was called “Listening/Speaking”

and the tool itself was a listening/speaking tool, students had a hard time grasping how the two

coincided because the videos they were selecting were unrelated to classroom topics.

Page 63: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

56

Students in the learner control also had a tendency to choose videos that appeared in the

top row of the selection grid, which corroborates the idea that students often rush to completion

of the task rather than take advantage of the controls available to them (Cordova & Lepper,

1996).

In contrast, the shared control system provided carefully selected videos that tied to the

topics of the classroom itself. In that light, students may have felt that the curriculum was

especially designed for them, and gave them additional practice in vocabulary, content, and

language level. This corroborates the view of ESL theorists such as Stephen Krashen (1985) that

input must be modified to the level of the learner, and also strongly correlates with the view of

many ESL practitioners who have long suggested that content itself should be structured in a

way to increase interest and limit the number of linguistic forms (Kucer et al, 1995; Whitmore &

Crowel, 1994). Furthermore, Brown (2001) demonstrates that in the field of ESL, ESL

textbooks and courses are generally thematically presented, and thus, students who are used to

having themes chosen for them may have found the idea of choosing their own themes

unusual—even burdensome. In this sense, the pre-selection of content is more likely seen as the

responsibility of the teachers, not the students.

In addition, some of the areas in which learner control is seen as superior to system

control were negated by the fact that the shared control system allowed a number of items often

understood as LC. For example, Clark, Nguyen, and Sweller (2006) shared the fact that LC

becomes a tool that increases productivity when experienced learners are able to skip past

already understood material. Since both the LC and SC models gave learners control over the

video player, any positive learning gains caused by control over the video player would have

occurred in both conditions.

Page 64: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

57

One student, for example, stated, “I love English Central. I got to play with things. I

could watch a video many times and I could slow things down. I also worked on a sound or

vocabulary item as much as I wanted. If I had many problems, I could get help with the

dictionary.” Here the student focused not only on the various features of the player itself (the

ability to slow down the rate of speech, the ability to rewatch a video, or listen to each phoneme

in a particularly difficult word), she also emphasized the idea of “play.” Here it can be seen how

the concept of being able to navigate, move outside the lines so to speak, gave her a sense of

autonomy that was motivating. This seems to corroborate the view that Judy Kay (2001)

suggests for defining learner control as one for students to “actively construct their own

understanding of a learning domain” (p. 114).

Some studies have suggested that ESL students may not be good candidates for

autonomous learning in general, whether shared or learner (Heinle, 2003; Iyengar & DeVoe,

2003; Oishi, 2000). This study, and the current trend in U.S. ESL students, is heavily influenced

by the influx of Middle Eastern students. Thus, this study was an opportunity to demonstrate

how current populations react to both shared and learner control conditions. What was interesting

to note is that a number of Middle Eastern men, in particular, showed resistance to the online

tool, and in fact, several conversations ensued that demonstrated a feeling that online instruction

was something that “could be done at home.” In this light, it appears that there was a general

preference for face-to-face instruction, and a belief that a teacher should assume control of the

students’ learning. This belief--that a teacher should have control--is corroborated by Hofstede

(2015) who suggests that Middle Eastern culture often assumes that power is distributed

unequally, and that “a collectivist culture [such as Syria, Iraq, and the UAE], fosters strong

relationships where everyone takes responsibility for fellow members of a group” (1). In this

Page 65: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

58

light, it may be seen by some Middle Eastern students that teachers are relinquishing their role,

and thus, the English Central tool might be seen by students as a way for a teacher to abdicate

responsibility.

A final note is necessary to elaborate on this finding. While a 3-point gain on an 80-point

test may be rightly understood as a small change, it is also important to concede that as students

become more and more advanced in their language learning, smaller gains become possible.

Thus, while very large gains might be made in basic and intermediate levels of English, smaller

gains are typical among advanced learners.

Supplemental Findings

Finding 2: The Tech Awareness Effect

A minor finding of this study demonstrated that certain students outperformed others

based on the tech savviness they likely came into the study with previously. The second linear

regression showed a clear learning advantage among those learners who reported high scores on

usability (ease of use) of the EnglishCentral platform. This indicates that students who had

comfort with the online format and tools presented by EnglishCentral improved their speaking

more than those who may have found the website less user friendly. Thus, language skills

improved as a result of higher online and technology skills, whereas gender, language, and even

students who worked hard (based on total number of hours, total speak points, total vocabulary

learned), received no statistical advantage.

Thus, it seems clear that a successful student in the EnglishCentral environment is one

that can intuitively understand the EnglishCentral platform, and is able to navigate it with more

available cognitive resources than his or her fellow counterparts. This also corroborates past

research, which suggests that disorientation can occur when students do not intuitively

Page 66: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

59

understand an online learning system, and that available cognitive resources can be exhausted

when learners are unfamiliar with online learning (Clarebout & Vandewaetere, 2011, Hasler,

Kersten, & Sweller, 2007; Scheiter & Gerjets, 2007).

Finding 3: The Teacher Effect

One last effect worthy of discussion is the fact that there was a statistically significant

difference among student scores by teacher. It was at first posited that this statistically

significant result might be tied to a teacher’s positive or negative view of English Central itself.

In other words, if the teacher really loved English Central, perhaps it bled into how the students

performed.

However, an additional inquiry proved that teachers’ views, in both conditions, were

higher in the learner condition, and those students fared worse. And further inquiry showed that

of the eight teachers in both learner and shared conditions, the average learning gain of students

with the most positive four teachers on the survey are nearly identical to those of the four least

positive (M = 1.478, 1.449). This strengthens the argument that shared control is superior to

learner control, because it demonstrates that such gains are independent of a teacher’s perception

of the tool.

However, if teacher perceptions of EnglishCentral didn’t create the difference, then what

else might have created this teacher effect? Interviews were conducted post-survey to see if any

other factors might have proven beneficial or harmful to student scores.

First, a look at the numbers revealed that student scores were highly variable from group

to group, and that a single learner score might change each class considerably. Thus, while

variability across each condition was equally distributed, that variability was not as equally

Page 67: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

60

distributed across teacher groups. So the teacher effect, in this view, is most likely caused by

this fact.

Other possibilities might include what teachers actually did or did not do within the

confines of their own classroom. However, since the school is heavily controlled by the same

textbooks and objectives, this does not appear to be the likely factor. Other considerations such

as years of teaching also did not prove to show any significant result.

Limitations

The student population at the American English and Culture Program is fairly

representative of the current U.S. ESL population, and as one of the largest ESL schools in the

country, the robust numbers certainly help to ensure sound sampling practices. However, it is

true that throughout the United States in the last few years, there has been a large influx in the

Middle Eastern population. This is reflected in the current study with fifty-five of the eighty-

three participants representing the Middle East. Since international student populations have

traditionally changed over the years, it may be difficult to generalize the results of this study to

future populations, although certainly it should generalize well to current U.S. trends. Another

trend, especially among the Middle Eastern populations, is to send a disproportionate amount of

men to women. This is also reflected in the- study, with twenty-one women represented overall,

and only five of those coming from the Middle East.

Another limitation is the amount of time spent studying in the program. This program is

particularly intensive compared to other programs, lasting only eight weeks. This may have

limited the ability to find a stronger learning trend, and find a trend between LC and SC

conditions compared to the control group.

Page 68: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

61

Conclusions

Overall, the use of EnglishCentral was seen as being as effective at increasing student

scores as a control group given equal timed treatment, and the shared control condition was

clearly superior to the learner control group. Based on the fact that EnglishCentral provides a

larger amount of practice than a teacher can logistically provide, this may be an attractive

offering for administrators, teachers, and students who wish to improve speaking proficiency.

Furthermore, EnglishCentral provides more feedback and instant feedback on language

instruction, which is one of the fundamental difficulties of teaching speaking. Finally, since this

feedback becomes available to a teacher, the EnglishCentral platform also serves as a guide for

teachers to instruct and alter curriculum according to student needs.

A cautious note should follow. This research should not be seen as an open endorsement

of EnglishCentral, and it should be noted that this research demonstrated the need to consider

the following precautions before using the EnglishCentral platform. First of all, EnglishCentral

has a cost. This research was not designed to provide any information on whether or not the

EnglishCentral platform is cost effective compared to other, perhaps less expensive methods. In

the experience of the language school in question (American English and Culture Program), the

cost of EnglishCentral, in general, has been deemed no more than a single textbook in a class

and was a properly priced tool for consideration.

Another consideration is the fact that technology can and does often fail. While most

students were successful in using the platform, some students were unfamiliar or unequipped to

solve simple tech issues, such as the process of allowing EnglishCentral access to the internal

microphone/speaker on each laptop. Since a number of laptops or computers have differing

requirements, and since many students were unfamiliar with the system, caution should be made

Page 69: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

62

in ensuring that students are well equipped with handling online difficulties. Since teachers were

often the de facto technical support for students, this also may cause problems.

Some teachers reported not wanting to help students understand the tool, especially since

the teachers themselves were not familiar with some of the more technical aspects of the

platform. This corroborates the idea found by Peng (2010) that there is often a lack of instructor

knowledge to act as technical support. In this study there was a tendency for teachers to “hand

over” any difficulties a student was having to the researcher or EnglishCentral support team,

even though the difficulties were well within the training they received. This apparent

impatience toward the system might be seen in view of the fact that teachers don’t perceive

themselves as technical experts, and that anything having to do with such issues is met with a

certain amount of resistance.

Future Directions

No doubt with the proliferation of online learning tools, researchers will continue to

verify or dispute their use in educational circles. Future directions of this research would do well

to consider how to best use EnglishCentral and other such tools in different contexts than the

one presented here.

One useful direction would be to ascertain whether or not the tool might be best used

within the classroom itself. While this study’s focus was on the use of EnglishCentral as a

homework tool, it may prove beneficial or more beneficial to consider it as an in-class

assignment. Since some students in the learner control condition in this study suggested that the

homework tool felt superfluous, it may be that a focus of EnglishCentral within the class would

prove more beneficial. Furthermore, it would be interesting to examine if the same kind of

Page 70: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

63

disorientation that occurred with students using a tool at home would occur with a teacher and

other learners present.

Another useful direction would be to determine if the use of EnglishCentral improved

general English proficiency. This particular study demonstrated that there was improvement in

speaking proficiency, as operationalized by the Pearson Versant, but did not generalize to overall

English proficiency. Several common ESL tests such as the TOEFL and IELTS tests would

serve as good measures for a future study. Does EnglishCentral have an ability to increase

scores on general English proficiency tests (and their respective subsections)?

Also, this study underrepresented Asian and Latin communities. A study that was more

robust might be able to demonstrate differing results based on these communities. Since learner

autonomy seems to be one of the key aspects of whether or not learner and/or shared conditions

show learning gains, it would be interesting to see if such differences exist across cultures. In

addition, since ease of use (usability) also indicated a learning gain, it would be interesting to

note which cultural groups might be more tech savvy as well. An alternate study, in fact, might

simply look at capturing students with general proficiency in online learning tools, and link

familiarity of those tools to the ability to perform well on EnglishCentral. Finally, while certain

results here did not attain statistical significance, it appears that some were trending toward

significance. A replication study with greater power would help to ascertain whether the trends

toward significance are ultimately met.

Page 71: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

64

REFERENCES

Allen, E., & Seaman, J. (2008). Staying the course: online education in the United States

2008. The Sloan Consortium. Babson Survey Research Group. Retrieved June 26, 2014,

from http://www.sloan-c.org/publications/survey/pdf/staying_the_course.pdf

Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors:

Lessons learned. The Journal of the Learning Sciences, 4(2), 167–207.

Anderson, T. (2008). Theory and practice of online learning. Edmonton: AU Press.

Au, S. (1988). A critical appraisal of Gardner’s social-psychological theory of second language

(L2) learning. Language Learning, 38: 75-100.

Ausburn, L.J. (2004). Course elements most valued by adult learners in blended online education

environments: An American perspective. Educational Medial, 41 (4), pp. 327-337.

Bailey, K.M., & Savage, L. (1995). New ways in teaching speaking. Alexandria, VA: TESOL.

Borko, H., Whitcomb, J., & Liston, D. (2009). Wicked problems and other thoughts on issues of

technology and teacher learning. Journal of Teacher Education, 60: 3-7.

Brown, H. D. (2001). Principles of language learning and teaching (2nd ed.). White Plains:

Addison Wesley Longman, Inc.

Brusilovsky, P., Kobsa, A., & Nejdl, W. (2007). The adaptive web: Methods and strategies for

web personalization. Heidelberg: Springer-Verlag.

Canale, M. & Swain, M. (1980). Theoretical bases of communicative approaches to second

language testing and teaching. Applied Linguistics, 1(1): 1-47.

Celce-Murcia, M. (Ed.) (2001). Teaching English as a second or foreign language (3rd ed.)

Boston: Heinle & Heinle. Clarebout, G. & Vendewaitere, M. (2011). Can instruction as such affect learning? The case

of learner control. Computers & Education, 57, 2322-2332.2

Clark, R.C., Nguyen, F., and Sweller, J. (2006). Efficiency in learning: evidence-based

guidelines to manage cognitive load. San Francisco: Pfeiffer.

Collis, B., De-Boer, W. & Slotman, K. (2001). Feedback for web-based assignments. Journal of

Computer Assisted Learning, 17, 306–313.

Corbalan, G., Kester, L., & van Merriënboer, J. J. G. (2009). Combining shared control with

variability over surface features: Effects on transfer test performance and task

involvement. Computers in Human Behavior, 25, 290–298.

Page 72: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

65

Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation and the process of learning:

Beneficial effects of contextualization, personalization, and choice. Journal of

Educational Psychology, 88, 715–730.

Cucchiarini, C., Strik, H., & Boves, L. (1997). Using speech recognition technology to assess

foreign speakers' pronunciation of Dutch. In J. Leather & A. James (Eds.), New Sounds

97: Proceedings of the Third International Symposium on the Acquisition of Second-

Language Speech, (61-68): Klagenfurt: University of Klagenfurt.

Derwing, T. M., & Munro, M. J. (2005). Second language accent and pronunciation teaching: A

research-based approach. TESOL Quarterly, 39, 379-398.

Ellis, R. (2001). The study of second language acquisition. Oxford: Oxford

University Press.

Ellis, R. (2012). The study of second language acquisition (2nd ed). Oxford: Oxford

University Press.

Ellis, R. A., Marcus, G. G., & Taylor, R. R. (2005). Learning through inquiry: Student

difficulties with online course-based material. Journal of Computer Assisted Learning,

21(4), 239-252.

EnglishCentral (2013). 100 millionth line spoken on English Central! English

Central Official Blog. Retrieved December 2, 2013, from http://blog.englishcentral.com/.

EnglishCentral (2015). What is the pedagogy behind English Central? English

Central. Retrieved January 5, 2015, from

http://pt.englishcentral.com/help?forum=365248&article=20641697.

Engwall, O., & Balter, O. (2007). Pronunciation feedback from real and virtual language

teachers. Computer Assisted Language Learning, 20(3), 235-262.

Freeman, D.E., Freeman Y.S. (1998) Essential Linguistics. Portsmouth, NH: Heinemann.

Gardner (1980). On the validity of affective variables in second language acquisition:

Conceptual, contextual, and statistical considerations. Language Learning. 30: 255-270.

Gardner, R., & Lalonde, R. (1985). Social factors in second language attrition. Canadian Journal

of Psychology, 2:1-15.

Gardner, R., & MacIntyre, P. (1991). An instrumental motivation in language study: Who says it

isn’t effective? Studies in Second Language Acquisition, 13: 57-72.

Gass, S.M., A. Mackey, M.J. Alvarez-Torres, and M. Fernandez-Garcia. (1999). The effects of

task repetition on linguistic output. Language Learning, 49 (4): 549-581.

Page 73: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

66

Goodyear, P., Salmon, G., Spector, J. M., Steeples, C. & Tickner, S. (2001). Competencies for

online teaching: A special report. Educational Technology Research and Development,

49(1), 65-72.

Hasler, B. S., Kersten, B., & Sweller, J. (2007). Learner control, cognitive load and instructional

animation. Applied Cognitive Psychology, 21, 713–729.

Heffernon, N. & Johnson, A. (2006). The short readings project: A CALL reading activity

utilizing vocabulary recycling. Computer Assisted Language Learning, 19(1), 63-77.

Hofstede, G. (2015). The Hofstede Center: Country Comparison. Retrieved from http://geert-

hofstede.com/syria.html.

Iyengar, S. (2011, Nov) Sheena Iyengar: How to make choosing easier [Video file] Retrieved

from http://www.ted.com/talks/sheena_iyengar_choosing_what_to_choose?language=en

Kalyuga, S. (2008). Implications of expertise reversal effect for adaptive multimedia learning.

In J. Luca, & E. Weippl (Eds.), Proceedings of World Conference on educational

multimedia, hypermedia and Telecommunications, pp. 4167–4174. Chesapeake, VA:

AACE.

Katz, I., & Assor, A. (2007). When choice motivates and when it does not. Educational

Psychology Review, 19, 429–442.

Kay, J. (2001). Learner control. User Modeling and User-Adapted Interaction, 11, 111–127.

Kinzie, M. B. (1990). Requirements and benefits of effective interactive instruction: Learner

control, self-regulation, and continuing motivation. Educational Technology

Research and Development, 38, 5–21.

Kinzie, M. B. and Sullivan, H.J. (1989) Continuing motivation, learner control and CAI.

Educational Technology Research and Development, 37(2), 5–14.

Krashen, S.D. (1994). The input hypothesis and its rivals. Implicit and Explicit Learning of

Languages, Academic Press. London: Ellis, N, pp. 45–77.

Kucer, S. B., Silva, C., Delgado-Larocco, E. (1995). Curricular conversations: Themes in

multilingual and monolingual classrooms. York, ME: Stenhouse.

Lawless, K.A., Brown, S.W. (1997) Multimedia learning environments: Issues of learner control

and navigation. Instructional Science, 25, 117–131.

Lee, S. and Lee, Y. H. K. (1991). Effects of learner-control versus program-control strategies

on computer-aided learning of chemistry problems: For acquisition of review?

Journal of Educational Psychology, 83(4): 491-498.

Page 74: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

67

Li, X. & Soh, L. (2003). A literature review on learner control strategies in software

tutoring systems. University of Nebraska at Kearney.

Maehr, M. L. (1976). Continuing motivation: An analysis of a seldom considered educational

outcome. Review of Educational Research, 46, 443-462.

Mitrovic, A., Koedinger, K.R., Martin, B. (2003). A comparative analysis of cognitive tutoring

and constraint-based modeling. Proceedings of the ninth international conference on user

modeling, Springer-Verlag, pp. 313–322.

Nation, I.S.P. (1990). Teaching and learning vocabulary. New York: Heinle & Heinle.

Neri, A., Cucchiarini, C., & Strik, H. (2008). The effectiveness of computer-based speech

corrective feedback for improving segmental quality in L2 Dutch. ReCALL, 20(2), 225-

243.

Parsad, B., Lewis, L., & Tice, P. (2008). Distance Education at Degree-Granting

Postsecondary Institutions: 2006–07. National Center of Education Statistics.

Retrieved June 26, 2014, from http://nces.ed.gov/pubs2009/2009044.pdf

Peng, S. (2010). The implementation of online learning for ESL programs: Factors and

perspectives. Dissertation, University of Southern California.

Perkins, D. N. (1985). The fingertip effect: How information-processing technology shapes thinking. Educational Researcher, 14, 11–17.

Precoda, K., Halverson, C. A., & Franco, H. (2000). Effects of speech recognition-based

pronunciation feedback on second-language pronunciation ability. Paper presented at

the InSTILL. Retrieved from

http://www.speech.sri.com/people/hef/papers/InstillUserStudy.pdf

Reeves, T. C. (1993). Pseudoscience in instructional technology: The case of learner control

research. Proceedings of Selected Research and Development Presentations at the

Convention of the Association for Educational Communications and Technology, 15(1),

820-831.

Riggenbach and Lazaraton (2001). In Celce-Murcia, M. (Ed.). Teaching English as a second or

foreign language (3rd ed.) Boston: Heinle & Heinle.

Robinson, C. C., & Hullinger, H. (2008). New benchmarks in higher education: Student

engagement in online learning. Journal of Education for Business, 84(2), 101-108.

Sauro, J., and Lewis, J.R. (2012). When designing usability questionnaires, does it hurt to be

positive? In Proc. CHI 2011, 2215-2223.

Scheiter, K. & Gerjets P. (2007). Learner control in hypermedia environments

Page 75: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

68

Educational Psychology Review, 19, pp. 285–307.

Seferoglu, G. (2005). Improving students' pronunciation through accent reduction software.

British Journal of Educational Technology, 36(2), 303-316.

Steinberg, E. R. (1989). Cognition and learner control: A literature review, 1977-1988. Journal

of Computer-Based Instruction, 16(4): 117-121.

Swaak, J., de Jong, T. (2001). Learner vs. system control in using online support for simulation-

based discovery learning. Learning Environments Research, 4, 217–241.

VanLehn, K., Lynch, C., Schultz, K., Shapiro, J.A., Shelby,, R.H., Taylor, L. (2005) The Andes

physics tutoring system: Lessons learned. International Journal of Artificial Intelligence

in Education, 15(3), 147–204

Vansteenkiste, M., Zhou, M., Lens, W., & Soenens, B. (2005). Experiences of autonomy and

control among Chinese learners: Vitalizing or immobilizing? Journal of Educational

Psychology, 97, 468–483.

Whitmore, K.F., & Crowel, C.G. (1994) Inventing a classroom: Life in a bilingual whole

language learning community. York, ME: Stenhouse.

Williams, M. D. (1996). Learner-control and instructional technologies. In D. H. Jonassen (Ed.),

Handbook of research for educational communications and technology (pp. 957–982).

New York: Simon & Schuster Macmillan.

Wilson, G. and Stacey, E. (2003). Online interaction impacts on learning: Teaching the teachers

to teach online. In G. Crisp, D. Thiele, I. Scholten, S. Barker and J. Baron (Eds), Interact,

Integrate, Impact. Proceedings 20th Annual Conference of the Australasian Society for

Computers in Learning in Tertiary Education, 541-551.

Yarbus, A. L. (1967), Eye movements and vision, New York: Plenum.

Zichermann, G., Cunningham, C. (2011). Gamification by Design: Implementing Game

Mechanics in Web and Mobile Apps, Canada: O’Reilly Media.

Page 76: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

69

APPENDIX A

ENGLISH CENTRAL VIDEOS IN SHARED CONTROL MODEL

Page 77: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

70

EnglishCentral Websites for Advanced 1 L/S

CONTEMPORARY TOPICS

Unit 1: Communication Studies (Slang/Language change) (TOTAL= 9:00min)

o http://www.englishcentral.com/video/13064/the-history-of-english-chapter-9-

internet-english (How internet produced language change Chapter 9)

o http://www.englishcentral.com/video/11464/voice-activated-elevator-they-dont-

do-scottish-acc (Trying to use different accents to operate a voice detecting

elevator)

o http://www.englishcentral.com/video/11178/phrasal-verbs-and-idioms-for-

negotiating (Business Idioms)

o http://www.englishcentral.com/video/15590/successful-negotiation-tone-

language-and-word-choi (learning how to choose words wisely when negotiating)

Unit 2: Child Psychology (Children w/special talents) (TOTAL= 10:01min)

o http://www.englishcentral.com/video/10988/the-baby-map-genius (Child good at

maps)

o http://www.englishcentral.com/video/16994/students-create-flight-simulator

(High School students create flight simulator)

o http://www.englishcentral.com/video/17023/pain-is-life (Shaolin monk trained as

a child to not feel pain)

o http://www.englishcentral.com/video/12619/showing-off-hidden-talent (Everyone

is unique)

o http://www.englishcentral.com/video/15417/the-linguistic-genius-of-babies-3 (A

study about how babies learn so quickly)

o http://www.englishcentral.com/video/19207/bill-gates-10000-hours-and-a-

lifetime-of-fanaticis (Bill Gates talking about Gladwell’s theory

Unit 3: Sociology (Social Status/Lifestyles) (TOTAL= 9:82min)

o http://www.englishcentral.com/video/13413/can-i-be-your-friend (Socializing as

if on Facebook and twitter )

o http://www.englishcentral.com/video/17111/the-power-of-words (Bullying)

o http://www.englishcentral.com/video/12491/give-me-the-noblesse-not-the-oblige

(Noblesse not Oblige)

o http://www.englishcentral.com/video/11846/diet (Celebrities being thin)

o http://www.englishcentral.com/video/15683/dorobo-people-one-womans-choice

(Woman living in rural Tanzania misses the city life)

o http://www.englishcentral.com/video/16214/life-among-the-dead (Living in a

Cemetery)

Unit 4: Business (Global Marketing) (TOTAL=8:77 min)

o http://www.englishcentral.com/video/19583/smart-toys-a-new-global-market-

trend (Smart toys)

o http://www.englishcentral.com/video/10402/presentations-marketing (Presenting)

o http://www.englishcentral.com/video/16946/pepsi-commercial-a-funny-job-

interview (Pepsi advertisement)

o http://www.englishcentral.com/video/10218/iphone-check (iphone advertisement)

o http://www.englishcentral.com/video/10543/what-is-sixthsense (portable surface)

Page 78: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

71

o http://www.englishcentral.com/video/16105/us-travel-demand-for-chinese-

tourists (Chinese hotels in U.S)

o http://www.englishcentral.com/video/11917/communication (communication)

o http://www.englishcentral.com/video/10274/starbucks-business-model

(Starbuck’s business model)

Unit 5: Cognitive Psychology (Memory Improvement) (TOTAL= 8.86min)

o http://www.englishcentral.com/video/14766/crime-investigations-making-faces

(Facial recognition)

o http://www.englishcentral.com/video/10699/ginkgo-biloba-ineffective-in-

preventing-cognitive- (Pill that improve Memory in elderly)

o http://www.englishcentral.com/video/11232/remembering-memories (Different

types of memories)

o http://www.englishcentral.com/video/20409/clenching-fists-boosts-memory

(Clenching your fist & smelling rosemary improves your memory)

o http://www.englishcentral.com/video/19213/memories-arent-lost-theyre-

somewhere (Man uses facebook to regain memories)

Unit 6: Anthropology/Biology (Love in Film) (TOTAL= 9.21min)

o http://www.englishcentral.com/video/20261/the-meaning-of-love (Love in

advertising)

o http://www.englishcentral.com/video/11870/your-ideal-type (What is your ideal

type?)

o http://www.englishcentral.com/video/11619/true-love-in-romantic-movies (True

Love in Romantic Films)

o http://www.englishcentral.com/video/17051/toxic-attraction-versus-real-love

(difference between love and toxic attraction)

o http://www.englishcentral.com/video/19818/love-knows-no-distance (Love in

advertising)

o http://www.englishcentral.com/video/16017/kids-talk-about-true-love (Kids Talk

about True love)

Unit 7: Astronomy (Space missions) (TOTAL= 8.78min)

o http://www.englishcentral.com/video/11974/space-travel (Astronaut

requirements)

o http://www.englishcentral.com/video/12377/vacation-in-space (Vacation in

Space)

o http://www.englishcentral.com/video/10483/the-big-bang-theory (Big Bang

Theory)

o http://www.englishcentral.com/video/10532/new-images-from-deep-space

(images of Space)

o http://www.englishcentral.com/video/18173/neil-armstrong-a-symbol-of-

accomplishment (Neil Armstrong bio)

o http://www.englishcentral.com/video/10714/the-internet-how-it-all-started (How

space missions helped create the internet)

o http://www.englishcentral.com/video/20641/chinese-astronaut-to-give-lecture-

from-space

Unit 8: Political Science (Surveillance jobs) (TOTAL= 8.94min)

Page 79: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

72

o http://www.englishcentral.com/video/11749/trespasser-calls-911-on-himself (Man

breaks in house and calls 911 on himself)

o http://www.englishcentral.com/video/12289/meet-the-facebook-safety-team

(Facebook Safety)

o http://www.englishcentral.com/video/17089/security-breaches-in-london-focus-

attention-on-oly (Security at the Olympics)

o http://www.englishcentral.com/video/12423/workplace-privacy-forget-about-it

(Surveillance videos at work)

o http://www.englishcentral.com/video/20452/voa-60-usa-newspapers-name-the-

source-of-nsa-leaks (Edward Snowden reveals how the U.S. government spies on

your phone calls)

o http://www.englishcentral.com/video/11954/mistakes (Accidentally stealing)

o http://www.englishcentral.com/video/11027/dont-trust-me (People are too

trusting)

Unit 9: Linguistics (Animal communication) (TOTAL= 9:30min)

o http://www.englishcentral.com/video/18536/monkey-waiters (Monkey waiters)

o http://www.englishcentral.com/video/10951/babies-pick-up-language-melodies-

in-the-womb (Learning language begins in the womb)

o http://www.englishcentral.com/video/10978/hitch (Body language)

o http://www.englishcentral.com/video/18566/linguistics-as-a-scientific-study

(Branches of linguistics)

o http://www.englishcentral.com/video/11838/guide-dogs (what guide dogs do)

o http://www.englishcentral.com/video/11937/the-ultimate-dog-tease (silly viral

video of a man talking to his dog)

o http://www.englishcentral.com/video/18516/never-underestimate-cats

(commercial about cats’ love of milk)

o http://www.englishcentral.com/video/11799/pets (pet psychics)

Unit 10: Economics (Money/Speculating on how money will change) (TOTAL=

8.65min)

o http://www.englishcentral.com/video/18952/money-can-change-everything (Is

money good or bad?)

o http://www.englishcentral.com/video/10231/do-avatars-make-money (Need to

work to make money)

o http://www.englishcentral.com/video/15419/eurozone-economic-crisis (The Euro)

o http://www.englishcentral.com/video/12322/talking-about-the-economy (Money

Fluctuation)

o http://www.englishcentral.com/video/13358/apple-has-more-money-than-the-us-

government (Apple has more money than the government)

o http://www.englishcentral.com/video/19196/saving-money-by-using-coupons

(Saving money with coupons)

o http://www.englishcentral.com/video/14602/inflation-1 (Inflation Part 1)

o http://www.englishcentral.com/video/14623/inflation-2 (Inflation Part 2)

Unit 11: Biology (Extending human life) (TOTAL= 8:88min)

o http://www.englishcentral.com/video/12445/judgment-day (Quotes on the fear of

death)

Page 80: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

73

o http://www.englishcentral.com/video/12173/the-power-of-smiling (Living longer

by smiling)

o http://www.englishcentral.com/video/12575/die-another-day-longevity (Positives

and Negative of living longer)

o http://www.englishcentral.com/video/11849/plastic-surgery (Plastic Surgery isn’t

bad)

o http://www.englishcentral.com/video/11182/smart-exercise (Exercising improves

brain development)

o http://www.englishcentral.com/video/19035/if-kids-had-more-time (humans life

expectancy has gone down 5 years)

Unit 12: Sociology (Marriage) (TOTAL= 11:06min)

o http://www.englishcentral.com/video/11815/inside-the-royal-wedding-how-kate-

met-william (Prince William and Kate)

o http://www.englishcentral.com/video/11536/a-couple-for-63-years-recalls-first-

date (Couple married 63 years – not love at first site)

o http://www.englishcentral.com/video/10756/wedding-dress (Low marriage

statistics in France)

o http://www.englishcentral.com/video/11794/relationships (Give and Take in a

marriage)

o http://www.englishcentral.com/video/17872/how-to-pull-off-a-surprise-marriage-

proposal (shows how to be creative in proposing)

o http://www.englishcentral.com/video/13472/first-gay-marriages-held-in-new-york

(Gay Marriage)

o http://www.englishcentral.com/video/16646/what-you-dont-know-about-

marriage-3 (Things that can ruin your marriage)

Page 81: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

74

APPENDIX B

ENGLISHCENTRAL SURVEY FOR LEARNER AND SHARED CONTROL

PARTICIPANTS AND ENGLISH CENTRAL SURVEY FOR TEACHERS

Page 82: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

75

AECP’s EnglishCentral Survey Name: ______________________________

(Your answers will not be shared with your teachers)

PART 1: For each of the following statements, mark one box that best describes your

reaction to the website today.

Strongly Disagree Strongly Agree

1. I think that I would like to use this website frequently. □ □ □ □ □

2. I found this website unnecessarily complex. □ □ □ □ □

3. I think that I would need help to be able to use □ □ □ □ □ this website.

4. I found this website very awkward/confusing to use. □ □ □ □ □

5. I would imagine most people would learn to use this □ □ □ □ □ website very quickly.

6. I feel very confident using this website. □ □ □ □ □

7. I needed to learn a lot of things before I could begin □ □ □ □ □ with this website.

8. I enjoyed using this website. □ □ □ □ □

9. I would recommend this website to a friend, □ □ □ □ □ classmate, or family member.

10. The quality of the videos were very good. □ □ □ □ □

11. □ □ □ □ □ I will keep studying with EnglishCentral after my class ends.

12. I was satisfied with customer service at EnglishCentral.□ □ □ □ □

13. The amount of time given to complete EnglishCentral □ □ □ □ □ assignments was enough.

14. EnglishCentral helped me understand my textbooks. □ □ □ □ □

Part 2 (Please see other side)

Page 83: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

76

PART 2: For these next questions, refer to the picture below.

Strongly Disagree Strongly Agree

15. The watch feature helped me learn English. □ □ □ □ □

16. The learn feature helped me learn English. □ □ □ □ □

17. The speak feature helped me learn English. □ □ □ □ □

18. I liked to choose videos that were interesting to me. □ □ □ □ □

19. I often chose videos that the teacher did not require.□ □ □ □ □

20. I enjoyed this class (course). □ □ □ □ □

21. The technology in this course was good. □ □ □ □ □

22. This course helps me improve outside of class. □ □ □ □ □

23. This course will help me for a long time. □ □ □ □ □

24. I like courses with technology. □ □ □ □ □

25. I have used websites to learn English in the past. □ □ □ □ □

26. I am comfortable using EnglishCentral Technology□ □ □ □ □

27. I wish I knew more about how to use technology. □ □ □ □ □

PART 3: Demographics

1. What is your age?

2. What is your first language?

3. What is your gender? (male or female)

Explain your experience using English Central (you may use the back of this paper).

Watch Feature

Speak Feature

Learn Feature

Page 84: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

77

AECP’s EnglishCentral Teacher Survey

Section Taught: ____________

How many times have you used English Central in the class prior to the study? __________

PART 1: For each of the following statements, mark one box that best describes your

reaction to the website.

Strongly Disagree Strongly Agree

I think that I will use this website frequently. □ □ □ □ □

I find this website unnecessarily complex. □ □ □ □ □

I think that I would need help to continue using □ □ □ □ □

this website.

I find this website very awkward/confusing to use. □ □ □ □ □

I imagine most students would learn to use this □ □ □ □ □

website very quickly.

I feel very confident using this website. □ □ □ □ □

I needed to learn a lot of things before I could begin □ □ □ □ □

with this website.

I enjoy using this website. □ □ □ □ □

I would recommend this website to a colleague □ □ □ □ □

The quality of the videos are very good. □ □ □ □ □

I will keep using English Central. □ □ □ □ □

I was satisfied with customer service at English Central. □ □ □ □ □

The amount of time given to complete English Central. □ □ □ □ □

assignments was sufficient.

English Central helped me use my textbooks. □ □ □ □ □

Part 2 (Please see other side)

Page 85: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

78

PART 2: Write down your responses in the space provided.

15. In addition to EnglishCentral homework, how many hours of homework did

you give each week?

16. How is EnglishCentral particularly suited to English language learning?

17. How could EnglishCentral be improved to better English language learning?

Page 86: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

79

APPENDIX C

PEARSON VERSANT TEST SAMPLE ITEMS

Page 87: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

80

Versant Sample Items

Part A. Reading. Please read the following sentences as you are instructed.

(audio) Please read sentence 6.

(sample response) They played loud music all night while she was trying to sleep.

(audio) Now read sentence 8.

(sample response) He wants to move out of the neighborhood.

Part B. Repeat. Please repeat each sentence that you hear.

(audio) When it is cold, I don’t go out.

(sample response) When it is cold, I don’t go out.

(audio) It is going to rain tomorrow, isn’t it?

(sample response) It is going to rain tomorrow, isn’t it?

Part C. Questions. Now please just give a simple answer to the questions.

(audio) Would you get water from a bottle or a newspaper?

(sample response) A bottle.

(audio) Are oranges or bananas more like fruits or vegetables?

(sample response) Fruits.

Part D. Sentence Builds. Now please rearrange the word groups into a sentence.

(audio) Was reading. My mother. Her favorite magazine.

(sample response) My mother was reading her favorite magazine.

(audio) We didn’t. The movie. Enjoy.

(sample response) We didn’t enjoy the movie.

(audio) Your books. Leave. At home.

(sample response) Leave at home the book.

Page 88: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

81

Part E. Story Retelling. You will hear three brief stories. Each story will be spoken once

followed by a beep. When you hear the beep, you will have thirty seconds to retell the story in

English. Try to retell as much of the story as you can including the situation, characters, actions,

and ending. You will hear another beep at the end of the thirty seconds.

(audio) Bill wanted to watch TV when he got home. His father said that first he had to

walk the dog. Then he had to clean his room. After he finished his chores, he could

watch TV.

(sample response) Bill is going home and he…and the dog…and his room is cleaning.

Part F. Open Questions. You will hear two questions about family life or personal choices. Each

question will be spoken twice, followed by a beep. When you hear the beep, you will have forty

seconds to answer the question. You will hear another beep at the end of the forty seconds.

(audio) Given a choice, would you like to live in a large city or a small town. Please

explain why.

(sample response) I want to live in a large city because you have much more shopping

centers and you have much more people in there. I live in a big city and it is very fine.

In a small town, I don’t know, I don’t live in a small town.

Page 89: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

82

APPENDIX D

ENGLISH CENTRAL ASSIGNMENT INSTRUCTIONS FOR BOTH SHARED

AND LEARNER CONTROL CONDITIONS

Page 90: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

83

EnglishCentral Assignment: Group A (Shared Control)

1) After logging in, click on My Class at the top of the screen. This is the Advanced 1

Listening Speaking channel.

You will complete one unit each week. There are four videos per unit. Your teacher will

tell you which unit you should complete within the week.

2) For each video, complete all three tabs for that video: watch, learn, and speak.

3) Make sure you take the quiz (video number 5) at the end of each unit.

Page 91: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

84

There are 4 goals for you to complete by every Sunday before 5pm:

watch 4 videos

learn 20 words (You may take a quiz called “My Words” located just below the

word “Browse” at the top of the page)

speak 4 videos

earn at least 4,000 speak points (based on pronunciation accuracy)

The 4:59 pm deadline is very important.

If your speak points are low, you need to redo (speak again) the videos to get more

points. Your grades are based on the percentage of the goals you complete.

*When you click on the ‘speak’ tab, make sure the box next to “focused speak mode” is NOT

checked (also found under the settings icon that looks like a wheel). If there is a checkmark

in this box, you will only record part of the video and it will be impossible for you to reach your

speak points goal.

*Use the tools in EnglishCentral to improve. Next to the English Central logo at the top of the

web page, there is a tab called “My English”. Click on it to get feedback on your specific

pronunciation strengths and weakness. Use the videos to practice and improve the sounds that

you have trouble with.

If you wish to watch videos outside of the class channel, click on Browse. Then, under ‘Filters’

and choose “Advanced.” Finally, look for videos either using the menus on the left (topic, skills,

or channels) or by using the search box to type in anything you might be interested in.

Page 92: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

85

EnglishCentral Assignment: Group B (Learner Control)

1) After logging in, click on the “Browse” tab on the far right.

2) Under “Filters” choose “Advanced”

3) Choose a video that you think looks interesting and click on it. Generally, it must be over 90

seconds long to give you a good number of points.

4) Complete all three tabs for that video: watch, learn, and speak.

5) Repeat with 3 other videos

Page 93: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

86

There are 4 goals for you to complete by every Sunday before 5pm:

watch 4 videos

learn 20 words (You may take a quiz called “My Words” located just below the

word “Browse” at the top of the page)

speak 4 videos

earn at least 4,000 speak points (based on pronunciation accuracy)

The 4:59 pm deadline is very important.

If your speak points are low, you need to redo (speak again) the videos to get more points

or browse for additional videos. Your grades are based on the percentage of the goals

you complete.

*When you click on the ‘speak’ tab, make sure the box next to “focused speak mode” is NOT

checked (also found in the settings icon that looks like a wheel). If there is a checkmark in

this box, you will only record part of the video and it will be impossible for you to reach your

speak points goal.

*Use the tools in English Central to improve. Next to the English Central logo at the top of the

web page, there is a tab called “My English”. Click on it to get feedback on your specific

pronunciation strengths and weakness. Use the videos to practice and improve the sounds that

you have trouble with.

Page 94: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

87

APPENDIX E

SPRING 2014 TECH SURVEY AT THE AMERICAN ENGLISH AND

CULTURE PROGRAM

Page 95: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

88

Online and Tech Tools Used at the American English and Culture Program

ABC Adobe Acrobat

Amazon Instant Video

ASU library ASU website

Audacity

Authorstream.com AzarGrammar / Azar’s blog

AZ Central

BBC / BBC video Blackboard

Blogger

Boggle's world Breaking News English

Bubbl.us

Busyteacher.org California Distance Learning Program

Cambridge Phrasal Verb Machine Canvas

Cengage

Citation machine Citation producer

Citefast

CNN COM-1 aviation listening

Corpus of Contemporary American

English Curriculet

Cyber ESL listening

Dave's Dictionary.com

Discoverengineering.org

Discovery Dropbox

Ease.ly

Easyprompter.com

ELLLO

English Central

English for everyone english4u

englishclub.com

englishpage.com englishpronunciationlessons.com

englishvocabularyexercises.com

Enneagraminstitute.com esl-fast

esl-lab.com

eslgalaxy eslvideo.com

eslwriting.org

Evernote Excel

Facebook

Flickr Ginger software

goanimate4schools.com

Google calendar Google clip art

Google docs

Google images Google search

Google/Google Scholar

GoogleVoice Great Writing

Great Paragraphs

History.COM Internet TESL journal

iCloud imbd

iMovie

Instagram Intermediate Listening Comprehension

iTunes

Jing jobing.com

justtheword.com

Kayak Learnboost

learnersdictionary.com

Linguist Toolbox Linked in

listenandwrite.com

ListeningLab Many Things

math.com Meme generator

Micrograde

MP3 Audio generator for iphone and droid

MW dictionary online

MyELT National Geographic

NBC

Netflix Notability (iPad)

NPR

Office Online stopwatch

OWL (Purdue University)

Paper.li Paraphrasing

Pathways

PBS Learning

Pearson Versant

Peoplesoft

PIcktochart pjalien.wordpress

Podcasts

Powerpoint Prezi

QR codes

QuickGrader Quizlet

Randall's ESL Lab

Rcampus (rubrics) realenglish.com

Refworks

Rewordify Rubistar

SalesForce

Santa Monica Chemistry Dept. Scholastic

Science channel

SeaMonkey Socrative

Son of Citation Machine

Sound Forge by Sony Spelling City

Spreeder

SPSS Standford Core NLP

Storycorps.org

Study zone Survey Monkey

Teachertube.com Teamwork PM

TED

Ted-ed Tell Me More

Ten fast fingers

Tesol.org Testmoz

Thesaurus.com

Today’s Meet TopNotch Active Book

Townsend Online

Travelocity Turning Point Technologies (clickers)

Udemy.org

UNESCO U of Iowa

U of Minn Urban dictionary

Vine

Visuwords VOA

Voiceboard

Voice recorders Voicethread

Voki

WeChat Weather.com

Wikipedia

Windows Movie Maker WP Carey academic integrity quiz

www.elcivics.com

Yahoo You Tube

Zap reader

Zite

Zotero

Page 96: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

88

Number of teachers who use selected technologies

Number of teachers who use certain functions in Blackboard

Page 97: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

89

APPENDIX F

CONSENT FORM FOR SHARED AND LEARNER CONTROL CONDITIONS

Page 98: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

90

CONSENT FORM: Groups A and B ENGLISHCENTRAL STUDY

RESEARCHERS Dr. Robert Atkinson, an Associate Professor at Arizona State University’s School of Computing, Informatics, and Decision Systems Engineering and Shane Dixon, a PhD candidate in Educational Technology, have invited your participation in this research study. STUDY PURPOSE The purpose of the research is to find out the effectiveness of English Central.

DESCRIPTION OF RESEARCH STUDY There are two parts to this study. PART 1 The first part of the study is required of all Advanced 1 Listening/Speaking students. During these eight weeks, you will watch a series of videos provided by English Central, and perform tasks to improve your English. Following your participation, you will take a survey asking about the experience. This is a class assignment and your teachers will give you a grade for your participation. You may also be given a think aloud assignment that will last about an hour. It will not be graded. I am asking for your permission to use these assignments as data for this research. PART 2 The second part of the study is not required coursework. There will be two tests (the Pearson Versant Speaking Test) that will be given to you, one at the beginning of the session (week 1) and one at the end (week 8). The tests are 30 minutes long and will NOT affect your grade in the course. You will be given the results of your test if you so choose. Filling out the tests will be considered your consent for this part of the research. Your participation will last from one to two sessions at the American English and Culture Program (8-16 weeks). Approximately 150 subjects will be participating in this study.

RISKS There are no known risks from taking part in this study, but in any research, there is some possibility that you may be subject to risks that have not yet been identified.

BENEFITS English Central is a language tool that can help improve your pronunciation, especially by focusing on specific sounds called phonemes. English Central is also a tool that helps in listening comprehension and speaking proficiency.

CONFIDENTIALITY All information obtained in this study is strictly confidential. The results of this research study may be used in reports, presentations, and publications, but the researchers will not identify you. In order to maintain confidentiality of your records, Robert Atkinson will not collect or share your name.

WITHDRAWAL PRIVILEGE Your participation is voluntary. You may withdraw at any time without penalty. Participation will not affect your grade.

VOLUNTARY CONSENT Any questions you have concerning the research study or your participation in the study, before or after your consent, will be answered by Robert Atkinson at [email protected] or Shane Dixon at [email protected]. If you have questions about your rights as a subject/participant in this research, or if you feel you have been placed at risk; you can contact the Chair of the Human Subjects Institutional Review Board, through the ASU Office of Research Integrity and Assurance, at 480-965 6788.

By signing below you are agreeing to have your classwork used as data in this research project.

___________________________ _________________________ ____________ Subject's Signature Printed Name Date

Page 99: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

91

CONSENT FORM: Group C ENGLISH CENTRAL STUDY

RESEARCHERS Dr. Robert Atkinson, an Associate Professor at Arizona State University’s School of Computing, Informatics, and Decision Systems Engineering and Shane Dixon, a PhD candidate in Educational Technology, have invited your participation in this research study. STUDY PURPOSE The purpose of the research is to find out the effectiveness of an English Language

Tool.

DESCRIPTION OF RESEARCH STUDY

There will be two tests (the Pearson Versant Speaking Test) that will be given to you, one at the beginning of the session (week 1) and one at the end (week 8). The tests are 30 minutes long and will NOT affect your grade in the course. You will be given the results of your test if you so choose. Filling out the tests will be considered your consent for this part of the research. Your participation will last from one to two sessions at the American English and Culture Program (8 weeks). Approximately 150 subjects will be participating in this study.

RISKS There are no known risks from taking part in this study, but in any research, there is some possibility that you may be subject to risks that have not yet been identified.

BENEFITS English Central is a language tool that can help improve your pronunciation, especially by focusing on specific sounds called phonemes. EnglishCentral is also a tool that helps in listening comprehension and speaking proficiency.

CONFIDENTIALITY All information obtained in this study is strictly confidential. The results of this research study may be used in reports, presentations, and publications, but the researchers will not identify you. In order to maintain confidentiality of your records, Robert Atkinson will not collect or share your name.

WITHDRAWAL PRIVILEGE Your participation is voluntary. You may withdraw at any time without penalty. Participation will not affect your grade.

VOLUNTARY CONSENT Any questions you have concerning the research study or your participation in the study, before or after your consent, will be answered by Robert Atkinson at [email protected] or Shane Dixon at [email protected].

Page 100: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

92

If you have questions about your rights as a subject/participant in this research, or if you feel you have been placed at risk; you can contact the Chair of the Human Subjects Institutional Review Board, through the ASU Office of Research Integrity and Assurance, at 480-965 6788.

By signing below you are agreeing to have your classwork used as data in this research project.

___________________________ _____________________________________ Subject's Signature Printed Name

Page 101: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

93

APPENDIX G

PEARSON VERSANT TEST INSTRUCTIONS (PHONE FORMAT)

Page 102: Speaking Proficiency under Learner- and Shared-control ... › ... · Speaking Proficiency under Learner- and Shared-control Conditions by ... Randall’s Cyber Listening Lab, Voice

94