A Comparative Study of Distributed Learning Environments on Learning Outcomes

Download A Comparative Study of Distributed Learning Environments on Learning Outcomes

Post on 08-Apr-2017




0 download

Embed Size (px)


<ul><li><p>This article was downloaded by: [] On: 04 December 2014, At: 22:39Publisher: Institute for Operations Research and the Management Sciences (INFORMS)INFORMS is located in Maryland, USA</p><p>Information Systems Research</p><p>Publication details, including instructions for authors and subscription information:http://pubsonline.informs.org</p><p>A Comparative Study of Distributed LearningEnvironments on Learning OutcomesMaryam Alavi, George M. Marakas, Youngjin Yoo,</p><p>To cite this article:Maryam Alavi, George M. Marakas, Youngjin Yoo, (2002) A Comparative Study of Distributed Learning Environments onLearning Outcomes. Information Systems Research 13(4):404-415. http://dx.doi.org/10.1287/isre.13.4.404.72</p><p>Full terms and conditions of use: http://pubsonline.informs.org/page/terms-and-conditions</p><p>This article may be used only for the purposes of research, teaching, and/or private study. Commercial useor systematic downloading (by robots or other automatic processes) is prohibited without explicit Publisherapproval, unless otherwise noted. For more information, contact permissions@informs.org.</p><p>The Publisher does not warrant or guarantee the articles accuracy, completeness, merchantability, fitnessfor a particular purpose, or non-infringement. Descriptions of, or references to, products or publications, orinclusion of an advertisement in this article, neither constitutes nor implies a guarantee, endorsement, orsupport of claims made of that product, publication, or service.</p><p> 2002 INFORMS</p><p>Please scroll down for articleit is on subsequent pages</p><p>INFORMS is the largest professional society in the world for professionals in the fields of operations research, managementscience, and analytics.For more information on INFORMS, its publications, membership, or meetings visit http://www.informs.org</p><p>http://pubsonline.informs.orghttp://dx.doi.org/10.1287/isre.13.4.404.72http://pubsonline.informs.org/page/terms-and-conditionshttp://www.informs.org</p></li><li><p>Information Systems Research, 2002 INFORMSVol. 13, No. 4, December 2002, pp. 404415</p><p>1047-7047/02/1304/0404$05.001526-5536 electronic ISSN</p><p>A Comparative Study of DistributedLearning Environments on</p><p>Learning Outcomes</p><p>Maryam Alavi George M. Marakas Youngjin YooGoizueta Business School, Emory University, Atlanta, Georgia 30322</p><p>Kelley School of Business, Indiana University, Bloomington, Indiana 47405Weatherhead School of Management, Case Western Reserve University, Cleveland, Ohio 44106</p><p>maryam_alavi@bus.emory.edu gmarakas@indiana.edu yyoo@po.cwru.edu</p><p>Advances in information and communication technologies have fueled rapid growth inthe popularity of technology-supported distributed learning (DL). Many educationalinstitutions, both academic and corporate, have undertaken initiatives that leverage themyriadof available DL technologies. Despite their rapid growth in popularity, however, alternativetechnologies for DL are seldom systematically evaluated for learning efficacy. Considering theincreasing range of information and communication technologies available for the develop-ment of DL environments, we believe it is paramount for studies to compare the relativelearning outcomes of various technologies.In this research, we employed a quasi-experimental field study approach to investigate the</p><p>relative learning effectiveness of two collaborative DL environments in the context of an ex-ecutive development program. We also adopted a framework of hierarchical characteristics ofgroup support system (GSS) technologies, outlined by DeSanctis and Gallupe (1987), as thebasis for characterizing the two DL environments.One DL environment employed a simple e-mail and listserv capability while the other used</p><p>a sophisticated GSS (herein referred to as Beta system). Interestingly, the learning outcome ofthe e-mail environment was higher than the learning outcome of the more sophisticated GSSenvironment. The post-hoc analysis of the electronic messages indicated that the students ingroups using the e-mail system exchanged a higher percentage of messages related to thelearning task. The Beta system users exchanged a higher level of technology sense-makingmessages. No significant difference was observed in the students satisfactionwith the learningprocess under the two DL environments.(Technology-Supported Learning; Distributed Learning; Learning Assessment; Learning Models)</p><p>IntroductionDespite interest and rapid growth in the variety, range,reach, and options of distributed learning programs,little systemic research into the effectiveness of distrib-uted learning technologies exists. Several authors (e.g.,Alavi et al. 1995, Storck and Sproull 1995) have en-couraged and called for an increase in research studies</p><p>to guide the design of effective and cost-efficient dis-tributed learning environments.As a step toward this goal, we conducted an empir-</p><p>ical assessment of two distributed learning ap-proaches, which we describe using the framework ofhierarchical characteristics of group support system(GSS) technologies as outlined by DeSanctis and</p><p>Dow</p><p>nloa</p><p>ded </p><p>from</p><p> info</p><p>rms.</p><p>org </p><p>by [</p><p>155.</p><p>33.1</p><p>20.1</p><p>67] </p><p>on 0</p><p>4 D</p><p>ecem</p><p>ber </p><p>2014</p><p>, at 2</p><p>2:39</p><p> . Fo</p><p>r pe</p><p>rson</p><p>al u</p><p>se o</p><p>nly,</p><p> all </p><p>righ</p><p>ts r</p><p>eser</p><p>ved.</p></li><li><p>ALAVI, MARAKAS, AND YOOA Comparative Study of Distributed Learning Environments</p><p>Information Systems ResearchVol. 13, No. 4, December 2002 405</p><p>Gallupe (1987) in the context of an executive devel-opment program. The first environment was based onelementary electronic mail and listserv technology,while the second was based on a more complex andsophisticated GSS technology.In this paper, we adopt a cognitive perspective on</p><p>learning. Learning is defined as changes in an individ-uals mental models or knowledge representations(Shuell 1986). According to this definition, learning in-volves acquisition of knowledge and change in knowl-edge structures rather than a behavior (performance)per se (Greeno 1974). An important implication of thisdefinition is that one learns and acquires knowledgewhile behavior (performance) is a possible outcome ofknowledge acquisition (Shuell 1981, Stevenson 1983).According to Ausubel (1968), successful performancerequires other abilities including perseverance, flexi-bility, improvisation, problem sensitivity, and tacticalastuteness, in addition to knowledge acquisition(learning). Thus, learning may not always be reflectedin behavior or performance. Following from these ob-servations, we examine the relative changes in partic-ipants mental models (as represented in their knowl-edge structures), rather than examining changes inbehavior, as a measure of their learning in two differ-ent distributed learning environments.</p><p>Background: CollaborativeDistributed LearningThe variety and flexibility of modern information andcommunication technologies provide a wide array ofpossibilities for the development of various forms ofdistributed learning approaches (Alavi and Leidner2001). The study contained herein focuses only on onetype of distributed learning environment, the collabo-rative distributed learning model, for its conceptualbase.In the collaborative distributed learning model, stu-</p><p>dents acquire knowledge and understandingprimarilythrough social interactions across time and/or geo-graphic distance and do so by using information andcommunication technologies. Collaborative distrib-uted learning as defined herein accords with the sociallearning theory originally identified by Vygotsky</p><p>(1929), which emphasizes learnings social genesis. Ac-cording to this theory, learning involves the socialcreation of knowledge through an instructional strat-egy employing a small-group problem-solving ap-proach by students (Johnson et al. 1991). Learningarises from the opportunity for the group members tomonitor each others thinking, opinions, and beliefs,while also obtaining and providing feedback for clar-ification and enhancement of comprehension. An in-dividuals exposure to the group members points ofviewmay challenge his/her initial understanding and,thus, further motivate learning (Glaser and Bassok1989). In other words, using a collaborative distributedlearning approach, learning occurs through commu-nication and collaborative interactions: in this case,technology-mediated interactions and communication.The primary role of technology applications in col-</p><p>laborative distributed learning is to enable flexibility,reach, timeliness, and increased frequency of group in-teraction and communication processes. Examplesrange from simple e-mail to various functional levelsof group support systems that enable anytime, any-place interactions among group members.Henceforth, the term distributed learning (DL) is</p><p>used to refer to the collaborative distributed modelcharacterized in this section.</p><p>Research FrameworkIn most empirical studies focusing on the effective-ness of DL, the comparison is made between learningoutcomes (students grades and/or perceived learn-ing) in the DL environment and learning outcomesobtained in a traditional face-to-face environment(e.g., Alavi 1994, Alavi et al. 1995, Storck and Sproull1995, Webster and Hackley 1997, Hiltz and Wellman1997). The effectiveness of DL is then established byshowing either no significant difference in the learn-ing outcomes of the two environments or by showinghigher learning outcomes associated with the DLenvironment.According to Turoff and Hiltz (1995), the objective</p><p>of distributed learning is not merely to duplicate thefeatures and effectiveness of a face-to-face environ-ment, but rather to use the powers of technology to</p><p>Dow</p><p>nloa</p><p>ded </p><p>from</p><p> info</p><p>rms.</p><p>org </p><p>by [</p><p>155.</p><p>33.1</p><p>20.1</p><p>67] </p><p>on 0</p><p>4 D</p><p>ecem</p><p>ber </p><p>2014</p><p>, at 2</p><p>2:39</p><p> . Fo</p><p>r pe</p><p>rson</p><p>al u</p><p>se o</p><p>nly,</p><p> all </p><p>righ</p><p>ts r</p><p>eser</p><p>ved.</p></li><li><p>ALAVI, MARAKAS, AND YOOA Comparative Study of Distributed Learning Environments</p><p>Information Systems Research406 Vol. 13, No. 4, December 2002</p><p>create a more effective learning environment. There-fore, we chose to employ what we believe to be a moreaccurate measure of learning to gauge the comparativeeffectiveness of the DL environments. Our decisionwas to measure the outcomes of DL in terms of bothcognitive and perceived learning. As defined previ-ously, cognitive learning involves changes in an individ-uals mental models; i.e., internal representations ofknowledge elements comprising a domain as well asinterrelationships among those knowledge elements(Ansari and Simon 1979, Neches 1987, Siegler 1986).Perceived learning is defined as changes in the learnersperceptions of skill and knowledge levels before andafter the learning experience. Approaches to the mea-surement of these two learning outcomes are describedlater in the paper. Given our definition and character-ization of DL, students in this environment learn byworking cooperatively in small technology-supporteddistributed groups on problem-solving tasks designedto promote learning. By definition, however, thegroups collaborative learning interactions and com-munications are distributed in both time and spaceand, therefore, must be mediated and supported bysome application or suite of communication and infor-mation technologies.The flexibility of modern information and commu-</p><p>nication technologies implies that alternative configu-rations, each with different features, functionality, andcomplexity, could be employed to support distributedlearning. One general classification of such amediatingtechnology is the typical group support system (GSS).Within the GSS literature, DeSanctis and Gallupe(1987) have identified three levels of increasing func-tionality and complexity in a GSS. Level-1 systemsprovide simple message exchange capability (i.e., ca-pability for creation, transmission, and storage of mes-sages), which is aimed primarily at facilitating funda-mental communication and information exchangeamong group members. Level-2 systems extend beyondsimple messaging capabilities and include tools forcommunication and task structuring (e.g., meetingagenda setting or group interaction models) and infor-mation manipulation, management, filtering, and sort-ing. Unlike a Level-1 system, Level 2 can support au-tomated decision support and modeling tools as well</p><p>as functions intended to facilitate multiparticipant col-laboration. Finally, Level-3 systems expand their capa-bilities beyond systems at Levels 1 and 2 and providemachine-induced group interaction patterns and guid-ance to prescribe communication rules for the group.Considering the variety of functionality of informa-</p><p>tion technologies, studies comparing the relative effi-cacy of alternative technology configurations for sup-port of DL should inform both the academic andpractitioner communities. Adopting the DeSanctis andGallupe (1987) GSS taxonomy as our guide, we inves-tigated the relative learning efficacy of two DL envi-ronments: one mirroring a Level-1 GSS and one con-figured as a Level-2 GSS. The Level-1 GSS consisted ofa simple text-based messaging system. This systemprovided capabilities for sending and receiving e-mailmessages as well as a simple listserv capability for sup-port of one-to-many messages. The Level-2 system,referred to as Beta system, provided sophisticatedmes-saging and e-mail capabilities, document and infor-mation management, workflow structuring and coor-dination, information search and filtering, and groupcollaboration and threaded discussion capabilities. Theacquisition, training, and operating costs of the Betasystem were much higher than for the Level-1 e-mailsystem. The Beta system not only enabled fluent inter-actions among participants, but also provided the ca-pability to see the overall structure of the task (all sub-components of the task and the ordered steps to betaken to accomplish them) through the informationmanagement and workflow structuring features of theGSS. These features were tightly integrated and wereprovided to the user through a unified multimediainterface.</p><p>HypothesesLearning emerges from the interaction of a stimulus(information) and the mind of the learner. Cognitivelearning theories identify a number of cognitive sub-processes involved in learning. These subprocesses canbe divided broadly into two categories: reception andstructuring. Reception (Ausubel 1968) involves the per-ception of information in the learners short-termmemory. Structuring (Norman 1982) consists of pro-cessing this information and connecting it to appro-priate prerequisite concepts retrieved from the long-term memory to form new (or modified) knowledge</p><p>Dow</p><p>nloa</p><p>ded </p><p>from</p><p> info</p><p>rms.</p><p>org </p><p>by [</p><p>155.</p><p>33.1</p><p>20.1</p><p>67] </p><p>on 0</p><p>4 D</p><p>ecem</p><p>ber </p><p>2014</p><p>, at 2</p><p>2:39</p><p> . Fo</p><p>r pe</p><p>rson</p><p>al u</p><p>se o</p><p>nly,</p><p> all </p><p>righ</p><p>ts r</p><p>eser</p><p>ved.</p></li><li><p>ALAVI, MARAKAS, AND YOOA Comparative Study of Distributed Learning Environments</p><p>Information Systems ResearchVol. 13, No. 4, December 2002 407</p><p>Figure 1 An Overview of the Executive Development ProgramFocus of this Study</p><p>structures. In the context of collaborative learning,where external stimuli to individual learners arise pri-marily from social interactions with others (Vygotsky1929, Pia...</p></li></ul>


View more >