facilitating teachers' reuse of mobile assisted language learning resources using educational...

Facilitating Teachers’ Reuse of Mobile AssistedLanguage Learning Resources Using

Educational MetadataPanagiotis Zervas and Demetrios G. Sampson, Senior Member, IEEE

Abstract—Mobile assisted language learning (MALL) and open access repositories for language learning resources are both topics

that have attracted the interest of researchers and practitioners in technology enhanced learning (TeL). Yet, there is limited

experimental evidence about possible factors that can influence and potentially enhance reuse of MALL resources stored in open

access repositories. In this paper the Mobile2Learn Framework is proposed, which aims at supporting sharing and reuse of MALL

resources within the context of MALL courses development. Moreover, a quantitative analysis of the reuse of MALL resources

developed with the proposed Mobile2Learn Framework is conducted. The results of this analysis provided with evidence that

completeness of metadata fields related with educational objectives based on Common European Framework of Reference for

Languages (CEFR) can strongly influence reuse taking place among Mobile2Learn Framework users.

Index Terms—Learning objects, mobile and personal devices, E-learning standards

Ç

1 INTRODUCTION

LANGUAGE learning has been a primary field of appli-cation of mobile learning, which is defined as the

process of learning and teaching that occurs with theuse of mobile devices providing flexible on-demandaccess (without time and device constraints) to learningresources, experts, peers and learning services from anyplace [1], [2]. This has led to the development of a newapproach for technology-enhanced language learning(TELL) which is commonly referred to as mobile assistedlanguage learning (MALL). MALL is typically defined as“an approach to language learning that is assisted orenhanced through the use of a handheld mobile device” [3].As a result, a number of MALL systems have been pro-posed such as CAMLES [4], TAMALLEþ [5], JAPELAS2[6] and PALLAS [7], aiming to investigate the potentialadvantages of using mobile devices in language learning.

On the other hand, the emerging open educationalresources (OERs) initiatives have enabled teachers toorganize, classify and store digital educational resourcesand their associated metadata in web-based repositoriestowards facilitating their sharing and reuse by other teach-ers [8], [9]. These initiatives have also influenced the field ofTELL and some web-based open access repositories havebeen recently developed towards supporting open access,sharing and reuse of digital language learning resources.However, these repositories do not include digital languagelearning resources that can be delivered to mobile devices

for supporting MALL. Additionally, these repositories donot put emphasis on the reuse of digital language learningresources and there is limited evidence about the factorsthat could influence and possibly enhance reuse of educa-tional resources in the field of Language Learning, as wellas in the field of MALL. As a result, it is worthy to investi-gate technology-supported solutions that can support openaccess and reuse of MALL resources.

Within this context, in this paper we present the Mobile2-Learn Framework, which aims at providing the technologi-cal means to facilitate open access and reuse of MALLresources from the perspective of MALL courses design anddevelopment. Moreover, a quantitative analysis of the reuseof MALL resources developed with the proposed Mobile2-Learn Framework is conducted. In particular, from theresults of this analysis it was identified that completeness ofMALL resources metadata records can strongly influencereuse taking place among Mobile2Learn Framework users(namely, foreign language teachers), whereas completenessof metadata fields related with educational objectives basedon Common European Framework of Reference for Lan-guages: Learning, Teaching, Assessment (CEFR) canstrongly influence reuse taking place among Mobile2LearnFramework users with different countries of origin.

The paper is organized as follows: Following this intro-duction, in Section 2 we discuss existing efforts in the OERsarea for supporting open access and reuse of MALL resour-ces and we identify the limitations of current practices. Sec-tion 3 describes our proposal, namely, the Mobile2Learnframework for facilitating open access and reuse to MALLresources within the context of MALL courses design anddevelopment and we present the tools of the proposedframework with emphasis on the educational metadataaspects of the framework. In Section 4, we conduct a quanti-tative analysis of the reuse of MALL resources withinMALL courses developed with the proposed Mobile2Learn

� The authors are with the Department of Digital Systems, University ofPiraeus, Piraeus, GR-18534, Greece, and with the Information TechnologiesInstitute, Centre for Research and Technology Hellas, Greece.E-mail: {pzervas, sampson}@iti.gr.

Manuscript received 7 Nov. 2012; revised 23 July 2013; accepted 2 Nov. 2013;date of publication 22 Dec. 2013; date of current version 6 May 2014.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference the Digital Object Identifier below.Digital Object Identifier no. 10.1109/TLT.2013.39

6 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 7, NO. 1, JANUARY-MARCH 2014

1939-1382� 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Framework Tools and we discuss the results of our study.Finally, we present our concluding suggestions and ourideas for future work in this field.

2 OPEN ACCESS AND REUSE OF MOBILE

ASSISTED LANGUAGE LEARNING RESOURCES

Over the past years a number of web-based open accessrepositories with digital language learning resources havebeen developed such as:

� The FLORE Repository (http://flore.uvic.ca/), whichwas developed by the “French Learning ObjectRepository for Education” project and providesopen access to digital resources for teaching Frenchas second language [10].

� The Tutela Repository (Tutela, http://tutela.ca/),which has been funded by Citizenship and Immigra-tion Canada and provides open access to digitalresources for teaching Canadian English and Frenchas second language.

� The Languages Open Resources Online Repository(LORO, http://loro.open.ac.uk/), which was devel-oped by Department of Languages at the UK OpenUniversity and provides open access to digitalresources for teaching a variety of languages.

� The Language Box (http://languagebox.ac.uk), whichwas developed by the Faroes project and providesopen access to digital resources for a wide variety oflanguages at various levels [11].

� The SPEAKAPPS Repository (http://oer.speakapps.org/), which was developed by the EU-fundedSPEAKAPPS Project and provides open access todigital resources for a wide variety of languages atvarious levels [12].

These repositories are mainly used by foreign languageteachers for: (a) searching and reusing digital languagelearning resources for their teaching activities and (b)sharing their digital language learning resources withother foreign language teachers. An important factor, inorder to facilitate foreign language teachers in the processof searching, retrieving and reusing digital languagelearning resources, is the existence of educational meta-data for these resources. The dominant metadata standardfor characterizing educational resources with metadata isthe IEEE Learning Object Metadata (IEEE LOM) Standard[13]. FLORE Repository and Tutela Repository adopt theIEEE LOM standard for characterizing their languagelearning resources, whereas LORO Repository adoptsDublin Core Metadata Element Set [14] and Language BoxRepository and SPEAKAPPS Repository adopt their ownmetadata model. However, the majority of the examinedrepositories do not put emphasis on specific metadata fordescribing the language learning characteristics of theirlanguage learning resources. More specifically, languagelearning resources stored in most of these repositories can-not be searched and retrieved based on the particular lan-guage learning objectives they address. Moreover,although all examined repositories support open access tolanguage learning resources, these resources have notbeen designed by following the W3C Mobile Web Best

Practices [15]. As a result these resources will not have anoptimum performance when delivered to mobile devicesin terms of page layout and content, navigation and links,as well as user input.

On the other hand, the issue of whether language learn-ing resources are reused and how, seems to be an importantone for the existing language learning repositories. Morespecifically, recent studies by Beaven [16], by Pulker andCalvi [17], as well as by Comas-Quinn et al. [18] have inves-tigated, focusing on the LORO Repository, the type ofchanges made to language learning resources when reusedand the reasons for these changes. Nevertheless, these stud-ies do not provide metrics for measuring reuse of languagelearning resources stored in LORO Repository or in othersexisting language learning repositories, in general. Conse-quently, there is no experimental evidence about the possi-ble factors that can influence and possibly enhance reuse ofeducational resources in the field of Language Learning, aswell as in the field of MALL.

Next, we address these issues by introducing theMobile2Learn Framework, which aims to support openaccess and reuse to MALL resources within the contextof MALL courses design and development. Finally, it isworth mentioning that there is an increasing interest incurrent MALL research that focuses on MALL systemsthat aim to engage learners in the process of sharingand reusing their own resources and there are severalstudies that investigate these issues as reported in [19],[20], [21], [22]. Nevertheless, Mobile2Learn Frameworkfocuses only on sharing and reuse of MALL resourcesthat are generated by foreign language teachers. Thus,aspects related to the sharing and reuse of learner-generated MALL resources are beyond the scope of theproposed Mobile2Learn Framework.

3 THE MOBILE2LEARN FRAMEWORK

3.1 Mobile2Learn User Roles

The Mobile2Learn Framework targets foreign languageteachers. However, within the Mobile2Learn Frameworkthere are three main user roles that foreign language teach-ers could undertake, namely:

� MALL Content Suppliers, that is, the role responsiblefor designing and developing MALL resources inthe form of “Learning Objects (LOs)” [23]. For thepurpose of our work, a Learning Object is defined as“any type of digital resource that can be reused to supportlearning” [24]. The MALL Content Suppliers need tobe able to (a) convert their existing digital languagelearning resources and/or create new digital lan-guage learning resources that meet mobile deliveryrequirements and (b) characterize these resourceswith metadata that are meaningful in relation to theMALL characteristics of the resources. Thus, theMobile2Learn Framework provides them with aset of guidelines (based on the W3C Mobile WebBest Practices 1.0) and the technological means(namely, a metadata authoring toolkit, described inSection 3.2.2) for developing MALL resources andtagging them with appropriate educational metadata

ZERVAS AND SAMPSON: FACILITATING TEACHERS’ REUSE OF MOBILE ASSISTED LANGUAGE LEARNING RESOURCES USING... 7

based on an appropriate designed LOM applicationprofile (described in Section 3.2.2).

� MALL Courses Suppliers, that is, the role responsiblefor designing MALL courses based on a pre-definedsequence of learning activities (in our work, referredto as course template) which represents the adoptedlanguage teaching practice as a workflow. Therefore,in our work a MALL course is defined as: a sequenceof learning activities populated with MALL resour-ces [25] conducted entirely via a mobile device, tar-geting specific educational objectives and withduration of 8 to 16 teaching hours in total. Moreover,a MALL course template is defined as: a sequence ofgeneric learning activities representing a specificMALL teaching practice [26], which potentially canbe populated with different MALL resources fordeveloping different MALL courses. Thus, the Mobi-le2Learn Framework provides the MALL CoursesSuppliers with a methodology and the technologicalmeans (namely a course authoring toolkit describedin Section 3.2.3) for defining their MALL teachingpractices and for representing them in a commonmachine understandable format following the IMSLearning Design (LD) specification [27]. Further-more, the Mobile2Learn Framework provides themwith a set of indicative examples of MALL coursetemplates representing specific MALL teaching prac-tices, which they can use and modify. Finally, itoffers them access to a web-based repository ofMALL Resources (in the form of LOs characterizedwith appropriate educational metadata) to facilitatethem in the design and the development of theirMALL courses.

� MALL Services Providers, that is, the role responsiblefor designing MALL programs as a synthesis ofMALL courses and delivering them to their students.The Mobile2Learn Framework provides them with(a) access to a repository of MALL courses (repre-sented in the form of IMS Learning Designs) whichthey can use to search and retrieve MALL courses

and (b) the technological means (namely, a coursedelivery tool) for delivering MALL courses to theirstudents via mobile devices.

Fig. 1 presents the identified user roles, their interconnec-tions, as well as, their needs and the tools that the Mobile2-Learn Framework offers them to support these needs.

3.2 Mobile2Learn Tools

The Mobile2Learn Framework provides the user roles iden-tified in Section 3.1 with a set of key tools that are describednext in detail.

3.2.1 Mobile2Learn MALL Courses Authoring Toolkit

for Designing MALL Course Templates and MALL

Courses

This is a software tool that enables the MALL Courses Sup-pliers (a) to express their MALL teaching practices, in theform of MALL course templates, using a common machineunderstandable way, and (b) to design and develop MALLcourses using a reference set of pre-defined MALL coursetemplates. As a result, a set of MALL course templates,which represent different MALL teaching practices, can bedesigned to facilitate the development of MALL coursesthat adopt these practices. Fig. 2 presents a snapshot of theMobile2Learn MALL Courses Authoring Toolkit, whichprovides MALL Courses Suppliers with a graphical inter-face for creating MALL courses conformant with the IMSLD Specification and packaging them along with theirrelated MALL resources.

3.2.2 Mobile2Learn Metadata Authoring Toolkit

This is a software tool that allows the MALL ContentSuppliers and the MALL Courses Suppliers to authoreducational metadata for their MALL resources andcourses, as well as, to organize and offer MALL resourcesand courses through the Mobile2Learn Web Repository.Educational metadata describe the different characteris-tics and attributes of a MALL resource or course, e.g.,title, description, keywords, target user group or subjectdomain. They are made up of data items that are associ-ated with a MALL resource or course, which are calledmetadata elements. Each MALL resource or course is

Fig. 1. The Mobile2Learn framework overview.

Fig. 2. Creating a new MALL course based on a pre-defined MALLcourse template.


associated with a metadata record composed by metadataelements with specific values. The more complete a meta-data record is, better informed decisions can be taken byMALL Courses Suppliers and MALL Services Providers,when searching to (re)use MALL resources and MALLcourses correspondingly.

The Mobile2Learn Metadata Authoring Toolkit offers anauthoring wizard for describing MALL resources andcourses with educational metadata conformant with IEEELOM standard. However, it is beyond the scope of IEEELOM to directly support the description of characteristicsrelated with MALL. As a result, in order to handle the spe-cific characteristics of the MALL resources and courses,extensions have been implemented to the value space of theIEEE LOM Classification Category (Nr. 9) through a LOMApplication Profile, proposed in Zervas and Sampson [28].More specifically, two controlled vocabularies have beenintroduced for the sub-element “Taxon Path.Taxon.Entry(Nr. 9.2.2.2)” based on the different values that the sub-element “Purpose (Nr. 9.1)” takes, as described below:

� When the value is “educational objective” then thepurpose of the Classification Element (Nr. 9) is todefine the educational objectives that a learningobject is targeting. As a result, in the sub-elementTaxon Path.Source we can use the “CEFR Levels”value to state that the educational objectives arederived from those defined in Common EuropeanFramework of Reference for Languages: Learning,Teaching, Assessment which is a framework used todescribe achievements of learners in foreign lan-guages across Europe [29]. CEFR provides six refer-ence levels, which are becoming widely accepted asthe European standard for grading an individual’slanguage proficiency, adopted also from the Euro-pass Language Portfolio [30]. These six levels are[29]: (a) Basic User: A1 and A2, (b) IndependentUser: B1 and B2 (c) Proficient User: C1 and C2. As aresult, the sub-element “Taxon Path.Taxon.Entry”can take these values. Fig. 3 presents the process ofcharacterizing a MALL resource or course based onthe CEFR educational objectives addressed.

� When the value is “accessibility restrictions” then thepurpose of the Classification Element (Nr. 9) is to

define the accessibility restrictions, which need to befollowed so that the learning object can be properlydelivered through a specific mobile device. As aresult, in the sub-element Taxon Path.Source we canuse the “Screen Resolution” value to state the acces-sibility requirements for the screen resolution of themobile device to be used for the proper delivery ofthe learning object and in the sub-element “TaxonPath.Taxon.Entry” we can use the values of differentmobile devices screen resolutions. Fig. 4 presents theprocess of characterizing a MALL resource or courseaccording to the screen resolution of the mobiledevice that it is going to be delivered in.

3.2.3 Mobile2Learn Web Repository

This is a web-based platform enabling the MALL ContentSuppliers and the MALL Courses Suppliers to share theirMALL resources and courses. Moreover, the Mobile2LearnWeb Repository (http://www.mobile2learn.eu/) providesthe MALL Services Providers with the possibility to searchand retrieve MALL courses that can be integrated to theireducational offers.

The functionalities of the Mobile2Learn Web Repositorycan be summarized as follows:

� Submit and Store. MALL Content Suppliers andMALL Courses Suppliers are able to submit andstore MALL resources and courses to the Mobile2-Learn Web Repository along with their related edu-cational metadata, which has been previouslydeveloped by using the Mobile2Learn MetadataAuthoring Toolkit

� Search and Retrieve. All user roles of the Mobile2-Learn Web Repository are able to search andretrieve MALL resources and courses by usingsearching criteria, which match with the educa-tional metadata of these resources and courses (seeFig. 5). More specifically, the search form includessearching elements according to the CEFR levelsand the screen resolution of the mobile device (asdescribed in Section 3.2.2), as well as other search-ing elements, which are mapped to metadata ele-ments provided by the IEEE LOM standard.

Fig. 3. Authoring Metadata related with CEFR educational objectives. Fig. 4. Authoring Metadata related with Mobile Devices Characteristics.


� Rate/Comment. All user roles of the Mobile2LearnWeb Repository are able to provide their ratings andcomments for the MALL resources and coursesstored in the Mobile2Learn Web Repository. Theseratings and comments are typically related with theimpressions of the users who have used a specificMALL resource/course.

3.2.4 Mobile2Learn MALL Courses Delivery Tool

This is a software tool suitable for mobile devices withWindows mobile or Android operating systems that facil-itates MALL Services Providers to deliver to their stu-dents MALL courses that have been retrieved from theMobile2Learn Web Repository. Figs. 6 and 7 presentsnapshots of the Mobile2Learn MALL Courses DeliveryTool, a platform for delivering MALL courses, which areconformant with the IMS LD Specification [31]. Further-more, the Mobile2Learn MALL Courses Delivery Toolenables enrolment of multiple roles/actors (individuallearners, groups of learners and teachers), as well as ren-dering of HTML-based content and flash files.

3.3 Mobile2Learn Aspects of Sharing and Reuse

As we can notice from the description of the Mobile2LearnFramework, the MALL resources and courses produced bythe tools of the framework follow the current LearningTechnology and Web Mobile Content Specifications andStandards, namely the IEEE LOM Standard, the IMS LDSpecification and the W3C Mobile Web Best Practices 1.0.This fact facilitates the development of an underlying infra-structure for sharing MALL resources though the Mobile2-Learn Repository and potentially reusing them in differentMALL courses that are developed with the Mobile2LearnMALL Courses Authoring Toolkit and delivered with theMobile2Learn MALL Courses Delivery Tool.

However, in order to evaluate whether the proposedMobile2Learn Framework can contribute towards the reuseof MALL resources within different MALL courses, we con-duct a quantitative analysis of MALL resources reuse. Thisanalysis can provide with evidence about the factors thatcould influence and possibly enhance MALL resourcesreuse within different MALL courses developed with Mobi-le2Learn Framework Tools.

4 QUANTITATIVE ANALYSIS OF MALL RESOURCES

REUSE

In this section, we present a quantitative analysis of thereuse of MALL resources within the Mobile2Learn Frame-work. First, related work is introduced regarding similarstudies focused on LOs reuse. Then, the research questionsand the adopted research method are described. Finally, theresults are outlined.

4.1 Related Work

Within the TeL literature, there are existing works that havestudied the issue of measuring LOs reuse for different datasets [32]. Koper [33] has defined three levels of LOs reuse,as follows:

� First level reuse. The creator of the LO reuses it to con-struct another LO of higher granularity.Fig. 6. Selecting a role for participating to a MALL Course.

Fig. 7. Rendering MALL Resources of a MALL Course.

Fig. 5. Searching Mechanism for MALL Resources/Courses.


� Second level reuse.Amember of a community reuses aLO created by someone else within the samecommunity.

� Third level reuse. A member of a community reusesa LO created by someone who is not a member ofthis community.

Ochoa [34] has conducted a quantitative analysis ofLOs reuse in ARIADNE Repository (http://ariadne. cs.kuleuven.be/finder/ariadne/). Within this study, thereuse was considered to take place at second level, asdefined by Koper [33]. The total reuse percentage wascalculated around 22 percent across learning objects ofdifferent granularity. This percentage was calculated asthe number of LOs that have been reused by any userwithin LOs of higher granularity compared to the totalnumber of LOs in the repository. Additionally, withinthis study it was analyzed whether LOs popularity(regarding how many times a LO has been accessed) caninfluence the LOs reuse. The analysis was based on cal-culating the Kendall’s tau correlation coefficient betweenthe rank of the LO in the reuse and its popularity scale.The results of the study revealed that there was no corre-lation between the popularity of a LO and the number oftimes that it has been reused.

Other similar studies have been conducted by Petrideset al. [35] and Duncan [36], who have also studied LOsreuse in Connexions Repository (http://cnx.org/). Withinboth studies, the reuse was considered taking place at sec-ond level, as defined by Koper [33]. A similar approach toOchoa [34] was adopted and a reuse percentage was calcu-lated around 20.50 percent across learning objects of differ-ent granularity. Furthermore, Duncan [36] analyzedwhether the age of the LOs and the number of keywordsavailable in the metadata of the LOs can influence the LOsreuse. The analysis was based on the Pearson’s correlationcoefficient between the rank of the LO in the reuse and itsage, as well as the number of keywords assigned to it. Theresults of the study showed that there was no significantcorrelation between LOs reuse and their age, as well astheir number of keywords.

Finally, Vuorikari and Koper [32] conducted a similarstudy and examined LOs reuse in Learning ResourceExchange (LRE) Repository (http://lreforschools. eun.org/) and LeMill (http://lemill.net/) Repository. Thereuse was considered as taking place at the second andthird levels, as defined by Koper [33]. More specifically,they reported (a) a second level reuse rate of approxi-mately 19 and 22 percent for LRE Repository and LeMillRepository respectively and (b) third level reuse rate ofapproximately 12 and 7 percent for LRE Repository andLeMill Repository respectively. The third level reuse wascalculated across communities with users of different spo-ken languages or different countries of origin. Neverthe-less, no evidence was provided within this study aboutpossible factors that could influence the second and/orthird levels of LOs reuse.

As a result, it appears, from the aforementioned stud-ies related with LOs reuse in existing repositories, thatsecond level reuse percentage varies from 19 to 22 per-cent, whereas third level reuse percentage varies from 7to 12 percent. Nevertheless, there is limited evidence

about the factors that can influence LOs reuse and achievereuse percentages higher than previously reported ones.Thus, the main purpose of our study is to measure thereuse of MALL resources within different MALL coursesdeveloped with Mobile2Learn Framework Tools andidentify empirical evidence about the factors that influ-ence the reuse within Mobile2Learn Framework.

4.2 Research Questions

The primary research question that we aim to answerwith this study is: “What are the main factors that influenceMALL resources reuse within different MALL courses devel-oped with Mobile2Learn Framework Tools?” More precisesub-questions related to the primary research question,that could be answered include the following:

1. What is the percentage of MALL resources reuse at first,second and third level within different MALL courses pro-duced by the Mobile2Learn Framework? We shouldmention here that we consider:

- Second level reuse as taking place among allMobile2Learn Framework, users (namely, for-eign language teachers). This is a key hypothesisin similar studies from the literature [32], [34],[35], [36], [37].

- Third level reuse as taking place amongMobile2Learn Framework users with differentcountries of origin. The reason for investigat-ing reuse among users of different country oforigin is that cross-country reuse of MALLresources has been a key recent hypothesis insimilar studies in the field of technology-enhanced learning [32] and we consideredthat it will be worthy to investigate this alsoin the field of MALL.

2. Is there a relation between MALL resources reuse at first,second and third level and the level of completeness ofMALL resources metadata records?

3. Is there a relation between MALL resources reuse at first,second and third level and the number of different meta-data values related with CEFR educational objectivesadded for the Classification metadata element?

The answers to these questions could facilitate us tocompare MALL resources reuse with similar studies (asdiscussed in Section 4.1) and identify differences or simi-larities. Moreover, the study of metadata records com-pleteness versus MALL resources reuse could provide uswith evidence whether the information added via meta-data to MALL resources can influence their reuse. Addi-tionally, the study of the number of CEFR educationalobjectives added for the Classification metadata elementversus MALL resources reuse could provide us evidenceabout the validity of our approach for enhancing MALLresources metadata with language learning educationalobjectives related with CEFR levels (as presented in Sec-tion 3.2.2) towards increasing MALL resources reuse.Finally, we should clarify at this point that only educa-tional metadata records were analyzed, whereas ratingsand comments added by the users of the Mobile2LearnRepository were not considered in this study.


4.3 Research Method

4.3.1 Participants

The participants who used the Mobile2Learn Frameworktools were English language teachers and they were selectedbased on their previous experience in using ICT tools forforeign language teaching. The participants’ average teach-ing experience with ICT tools was 4.7 years, so they can becharacterized as experienced teachers. The participantswere engaged in specially designed five-day workshops,which were held in four Vocational Education and TrainingOrganizations (VET) located in four European countries,namely Greece, Czech Republic, Netherlands and Spain.The procedure that was followed was the following:

� During the workshops. The participants were trained inthe process of using the Mobile2Learn Frameworktools. Next, they assumed the role of MALL contentsuppliers and developed MALL resources, whichwere characterized with educational metadata (byfollowing the LOM application profile presented inSection 3.2.2) and uploaded to the Mobile2LearnRepository. The participants chose to create theseMALL resources in flash format. The MALL resour-ces were also tailored to meet the specific screen reso-lution value of the mobile devices handed to them fortesting purposes. During this phase, the participantswere supervised by the workshops’ tutors, who alsoprovided with face to face assistance to the partici-pants for using theMobile2Learn Framework Tools.

� After the workshops. The participants were allowed athree-month period to undertake the role of MALLCourses suppliers and develop MALL courses byusing or re-using the MALL resources that weredeveloped by all participants during the four differ-ent workshops and were available in the Mobi2LearnRepository. During this process, the participantswere also able to develop new MALL resourceswhen existing MALL resources were not suitable tobe used in the context of the MALL courses that theywere developing. It should be noted that during thisphase the participants were not supervised and theywere asked to develop MALL courses on their own.However, they were able to request online technicalsupport by the workshop tutors in case of difficultieswith Mobile2Learn Framework Tools.

4.3.2 Data Set

Table 1 presents the snapshot of the Mobile2Learn Reposi-tory which was used for performing our study.

As we can notice from Table 1, the total sample of MALLcontent suppliers and MALL courses suppliers consists ofN ¼ 112 participants. The countries of origin of the partici-pants were Greece ðN ¼ 33Þ, Netherlands ðN ¼ 27Þ, SpainðN ¼ 26Þ and Czech Republic ðN ¼ 26Þ.

The total number of MALL resources developed was719 and the total number of MALL courses developedwas 132. It is worth mentioning that 582 (80.94 percent)MALL resources were developed in total during theworkshops, whereas 137 (19.06 percent) MALL resourceswere developed after the workshops period. Regardingthe MALL courses, all of them were developed duringthe three-month period after the workshops. Finally,each participant developed an average of six MALLResources ðSD ¼ 0:96Þ and 1 MALL Course ðSD ¼ 0:49Þ.The fact that the calculated standard deviation is ratherlow means that the number of MALL resources andMALL courses developed by each participant was almostevenly distributed.

4.3.3 Methodology

In order to address the primary research question, as wellas the additional sub-questions (presented in Section 4.2),we adopt a similar methodology for a quantitative analysisof LOs reuse proposed by Ochoa [34] and we adopt thethree levels of reuse proposed by Koper [33]. More specifi-cally, our methodology includes the following steps:

1. Amount of reuse. (a) Quantitatively analyze MALLresources reuse within different MALL courses byfollowing the metrics for measuring LOs reuse atfirst, second and third level, as proposed by Koper[33] and adapted in the context of our study (seeTable 2) and (b) compare reuse percentages withsimilar studies from the literature and identify dif-ferences or similarities (addressing sub-question 1).

TABLE 1Description of Mobile2Learn Repository Data Set

TABLE 2Metrics for MALL Resources Reuse


2. Reuse versus Metadata Completeness. (a) Calculate thecompleteness of MALL resources metadata recordsby using the following formula:

QComp ¼PN

i¼1 P ðiÞN

; (1)

where P ðiÞ is 1 if the ith metadata field has a no-nullvalue or 0 otherwise. N is the number of metadatafields defined in the Mobile2Learn LOM applicationprofile used for describing the MALL resources and(b) calculate Kendall’s tau correlation coefficientbetween MALL resources reuse (at first, second andthird level) and completeness of MALL resourcesmetadata records (addressing sub-question 2).

3. Reuse versus number of CEFR educational objectivesaddressed. Calculate Kendall’s tau correlation coeffi-cient between MALL resources reuse (at first, secondand third level) and the number of different meta-data values related with CEFR educational objectivesadded for the Classification metadata element(addressing sub-question 3).

4.4 Results

4.4.1 Amount of Reuse

In order to measure the reuse at first, second and third level,we applied the reuse metrics presented in Table 2 to thedata set of the Mobile2Learn Repository presented inTable 1. The results of the reuse metrics at each level is pre-sented in Table 3.

As we can notice from Table 3, Mobile2Learn Frameworknoticeably facilitates reuse at first, second and third level.More specifically, reuse at first level is 19.88 percent butalthough the amount of reuse at first level is promisingthere were not any previous studies, so as to compare with.Additionally, at second level, Mobile2Learn Frameworkgoes beyond (35.04 percent) the general trend of 22 percentreuse, which has been reported from similar studies. Wecan also notice that second level reuse is higher than firstlevel reuse. This means that MALL resources creators actedmainly as MALL content suppliers and they were not alsoinvolved in the process of developing MALL courses.Finally, Mobile2Learn Framework outperformed also atthird level reuse (13.49 percent), where the reported reusefrom similar studies was 7 to 12 percent. These results pro-vided us with evidence that Mobile2Learn Frameworkcould (a) support reuse for the creators of the MALL resour-ces and (b) improve reuse among users of the Mobile2LearnFramework, as well as across country boundaries (that is,among users with different countries of origin).

4.4.2 Reuse versus Metadata Completeness

Table 4 presents the calculated Kendall’s tau correlationcoefficient between number of times of MALL resourcesreuse and the completeness of their metadata records, so asto identify if a statistically significant correlation betweenthese two variables existed. We should also mention at thispoint that the average number of completeness of MALLresources metadata records was 0.7387 ðSD ¼ 0:1102Þ.

As we can notice from Table 4, there was no correlationbetween the number of times of reuse at first level and themetadata completeness. This means that the completenessof metadata records does not affect reuse when this is takingplace at first level (that is by the creator). This was expectedsince the MALL resource creator does not need to beinformed about the metadata of a MALL resource that he/she has created, so as to decide whether to reuse a MALLresource or not. On the other hand, there was a significant,positive correlation ðt ¼ 0:898; p < 0:05Þ between the num-ber of times of reuse at second level and the metadata com-pleteness. As a result, we can identify that metadatacompleteness is an important factor that influence reusewithin the Mobile2Learn Framework, when reuse is takingplace among all users of the Mobile2Learn Framework (sec-ond level reuse). This provides with evidence that the LOMapplication profile used for characterizing the MALLresources of the Mobile2Learn repository (as presented inSection 3.2.2) includes meaningful metadata elements,which can enhance MALL resources reuse when they havebeen completed by the MALL resources’ creators. Finally,there was also a positive correlation ðt ¼ 0:467; p < 0:05Þbetween the number of times of reuse and the metadatacompleteness for third level reuse but this correlation wasweaker than the calculated correlation for second levelreuse. This means that reuse across country boundaries(third level) is less strongly linked with completeness of theMALL resources metadata records. This could be explainedby the fact that reuse taking place across country boundariescould be linked with the completeness of only specific meta-data elements of MALL resources metadata records relatedwith the facilitation of cross-country reuse. This is furtherinvestigated and discussed in the next section.

4.4.3 Reuse versus Number of CEFR Educational

Objectives Addressed

Table 5 presents the calculated Kendall’s tau correlationcoefficient between the number of times of MALL resourcesreuse and the number of different metadata values relatedwith CEFR educational objectives added for the Classifica-tion metadata element.

TABLE 3Amount of Reuse

TABLE 4Correlation between Number of Times of Reuse

and Metadata Records Completeness


As we can notice from Table 5, there was no correlationbetween the number of times of reuse at first level and thenumber of CEFR educational objectives addressed for eachMALL resource. This means that number of educationalobjectives addressed for eachMALL resource does not affectits reuse when this is taking place at first level (that is by thecreator). This was expected, since theMALL resource creatoris able to reuse aMALL resource created by him/herwithoutbeing informed about the educational objectives that thisMALL resource is targeting. On the other hand, there was apositive correlation ðt ¼ 0:768; p < 0:01Þ between the num-ber of times of reuse at second level and the number of CEFReducational objectives addressed. As a result, we can identifythat the number of CEFR educational objectives addressedfor each MALL resource is a notable factor that influencereuse within the Mobile2Learn Framework, when reuse istaking place all users of the Mobile2Learn Framework (sec-ond level reuse). Finally, there was also a significant positivecorrelation ðt ¼ 0:945; p < 0:05Þ between the number oftimes of reuse and the number of CEFR educational objec-tives addressed for third level reuse. This means that reuseacross country boundaries (third level) is noticeably linkedwith the number of CEFR educational objectives addressedfor each MALL resource. The positive correlation at secondand third level reuse with the number of educational objec-tives addressed by the MALL resources can be explained bythe fact that the educational objectives are expressed by theCEFR levels (as presented in Section 3.2.2). CEFR levels arewidely accepted across Europe for describing achievementsof learners of foreign languages and they are important infor-mation to be exploited when reuse is taking place at secondand third level. This also provides us with evidence that ourproposal for enhancing MALL resources metadata with lan-guage learning educational objectives relatedwith CEFR lev-els (as presented in Section 3.2.2) was a valid approach forenhancingMALL resources reuse.

5 CONCLUSIONS AND FUTURE WORK

In this paper, we presented the Mobile2Learn Framework,which adopts the current Learning Technology specifica-tions andWebMobile Content Specifications, aiming to sup-port open access and reuse to MALL resources within thecontext of MALL courses design and development. In thisframework, we identified the main user roles and we pre-sented the key tools which empower them in the process ofthe design and development of MALL resources andcourses. Within the proposed Mobile2Learn Framework, weconducted a quantitative analysis of MALL resources reuse,so as to measure the reuse percentage of MALL resourceswithin different MALL courses developed by Mobile2Learn

Framework, as well as to identify empirical evidence aboutthe factors that influence the reusewithin this framework.

The results of this analysis provided us with indicationsthat:

� The proposed Mobile2Learn Framework can sig-nificantly (a) facilitate reuse taking place by thecreators of the MALL resources (first level) and(b) enhance reuse among all users of the Mobile2-Learn Framework (second level), as well as acrossusers of the Mobile2Learn Framework with differ-ent countries of origin (third level). The proposedMobile2learn Framework resulted in better secondand third level reuse results compared with simi-lar studies from the literature.

� Completeness of metadata records, as well as thenumber of educational objectives addressed for eachMALL resource does not appear to influence the firstlevel reuse. This was expected and can be explainedby the fact that MALL resource creator is able toreuse a MALL resource created by him/her withoutbeing informed about the MALL resource metadataor educational objectives that the MALL resource istargeting. On the other hand, second level reuse isinfluenced mainly by the completeness of metadatarecords and less strongly by the number of educa-tional objectives addressed for each MALL resource,which are derived from the CEFR levels (asdescribed in Section 3.2.2). This could be explainedby the fact that users, who performed second levelreuse, need to be informed about all metadata ele-ments of a MALL resource before reusing it. Addi-tionally, third level reuse is influenced significantlyby the number of CEFR educational objectivesaddressed for each MALL resource and less stronglyby the completeness of MALL resources metadatarecords. This could be explained by the fact thatusers, who performed third level reuse, need to beinformed about suitable metadata elements (thatcould facilitate cross-country reuse) before reusing aMALL resource. Finally, this provided us with evi-dence that our proposal for enhancing MALL resour-ces metadata with language learning educationalobjectives related with CEFR levels was a validapproach for enhancing MALL resources reuse.

The aforementioned indications could also facili-tate developers of MALL repositories during the pro-cess of developing new repositories or enhancingexistingMALL repositories towards achieving higherreuse results of MALL resources. More specifically,developers ofMALL repositories should consider:

� Empowering their end-users with appropriate anduser-friendly metadata authoring tools, so as to moti-vate them to provide complete metadata descrip-tions that will eventually facilitate and enhancesecond level reuse.

� Enhancing the metadata model that is used todescribe MALL resources with language learningeducational objectives related to existing commonlyaccepted frameworks such as CEFR. This can even-tually facilitate and enhance third level reuse.

TABLE 5Correlation between Number of Times of Reuse And Number

of CEFR Educational Objectives Addressed


Finally, it should be also noted that there could be otherfactors that may have affected reuse of MALL resourcesexcept from the metadata that were used for characterizingthem. For example, the profile of the participants, who wereICT experienced English language teachers, could haveaffected the reuse results. Moreover, our experimentaldesign, with the first phase including specially designedfive-day training workshops, could have motivated the par-ticipants to further reuse MALL resources. These factorscould be further exploited in future experiments with theproposed framework. Additionally, future work regardingthe proposed Mobile2Learn Framework includes its exten-sion to consider, also, the learner user role and identifyaspects that are related with sharing and reuse of learner-generated MALL resources. This will enable us to designand execute experiments with language learners, so as toidentify metadata elements that are required and/or usefulin promoting open access and reuse of learner-generatedMALL resources.

ACKNOWLEDGMENTS

The work presented in this paper was supported by theMW-TELL Project (http://www.Mobile2Learn.eu) fundedby European Commission under the Leonardo da Vinci(LdV) Sectoral Programme of the Lifelong Learning Pro-gramme. Contract No: 2008–1-GR1-LEO05–00693.

REFERENCES

[1] M. Sharples and J. Roschelle, “Guest Editorial: Special Issue onMobile and Ubiquitous Technologies for Learning,” IEEETrans. Learning Technologies, vol. 3, no. 1, pp. 4-5, Jan./Mar.2010.

[2] J. Traxler, “Current State of Mobile Learning,” Mobile Learning:Transforming the Delivery of Education and Training, M. Ally, ed.,pp. 9-24, Athabasca Univ. Press, 2009.

[3] K.E. Valarmathi, “Mobile Assisted Language Learning,” J. Tech-nology for ELT, vol. 2, no. 2, 2011.

[4] V.A. Nguyen and V.C. Pham, “CAMLES: An Adaptive MobileLearning System to Assist Student in Language Learning,” Proc.IEEE Seventh Int’l Conf. Wireless, and Ubiquitous Technology in Edu-cation (WMUTE ‘12), pp. 72-76, 2012.

[5] S. Fallahkhair, “Development of Location-Based Mobile LanguageLearning System to Support Geolearners,” Ubiquitous Learning: AnInt’l J., vol. 4, no. 1, pp. 1-10, 2012.

[6] C. Yin, H. Ogata, Y. Tabata, and Y. Yano, “JAPELAS2: Supportingthe Acquisition of Japanese Polite Expressions in Context-AwareUbiquitous Learning, Mobile and Ubiquitous Technologies forLanguage Learning,” Int’l J. of Mobile Learning and Organisation,vol. 4, no. 2, pp. 214-234, 2010.

[7] S.A. Petersen and J.K. Markiewicz, “PALLAS: PersonalisedLanguage Learning on Mobile Devices,” Proc. Fifth IEEE Int’lConf. Wireless, Mobile, and Ubiquitous Technology in Education,2008.

[8] N. Friesen, “Open Educational Resources: New Possibilities forChange and Sustainability,” Int’l Rev. of Research in Open and Dis-tance Learning, vol. 10, no. 5, 2009.

[9] A. Lane and P. McAndrew, “Are Open Educational ResourcesSystematic or Systemic Change Agents for Teaching Practice?”British J. Educational Technology, vol. 41, no. 6, pp. 952-962,2010.

[10] C. Caws, N. Friesen, and M. Beaudoin, “A New Learning ObjectRepository for Language Learning: Methods and Possible Out-comes,” Interdisciplinary J. Knowledge and Learning Objects,vol. 2006, no. 2, pp. 112-124, 2006.

[11] K. Borthwick, M. Arrebola, D. Millard, and Y. Howard, “Learningto Share in the Language Box: A Community Approach to Devel-oping an Open Content Repository for Teachers and Learners,”Proc. EuroCALL Conf., 2009.

[12] C. Appel, F. Santanach, and S. Jager, “SPEAKAPPS: New Toolsand Activities for the Development of Speaking Skills in a SecondLanguage,” Proc. Fourth Int’l Conf. Education and New LearningTechnologies. 2012.

[13] IEEE LTSC, Final Draft Standard for Learning Object Metadata,http://ltsc.ieee.org/wg12/, 2005.

[14] Dublin Core Metadata Initiative, “Dublin Core Metadata Ele-ment Set 1.1: Reference Description,” http://dublincore.org/documents/dces/, 2005.

[15] J. Rabin and C. McCathieNevile, Mobile Web Best Practices 1.0,http://www.w3.org/TR/mobile-bp/, 2008.

[16] T. Beaven, “Use and Reuse of OER: Professional Conversationswith Language Teachers,” J. e-Learning and Knowledge Soc., vol. 9,no. 1, pp. 59-71, 2013.

[17] H. Pulker and A. Calvi, “The Evaluation and Re-Use of Open Edu-cational Resources in Language Teaching—A Case Study,” Proc.OER13: Creating a Virtuous Circle, 2013.

[18] A. Comas-Quinn, J. Wild, and J. Carter, “Leveraging Passion forOpen Practice,” Proc. OER13: Creating a Virtuous Circle, 2013.

[19] L. Pemberton, M. Winter, and S. Fallahkhair, “A User CreatedContent Approach to Mobile Knowledge Sharing for AdvancedLanguage Learners,” Proc. Eighth World Conf. Mobile and Contex-tual Learning (mLearn ’09), 2009.

[20] T. Sweeney, R. Pemberton, and M. Sharples, “Toponimo: A Geo-social Pervasive Game for English Second Language Learning,”Proc. 10th World Conf. Mobile and Contextual Learning (mLearn ’11),2011.

[21] H. Ogata, M. Li, B. Hou, N. Uosaki, M.M. El-Bishouty, and Y.Yano, “Scroll: Supporting to Share and Reuse Ubiquitous Learn-ing Log in the Context of Language Learning,” Research and Prac-tice in Technology Enhanced Learning, vol. 6, no. 2, pp. 69-82, 2011.

[22] L. Gjedde and M. Bo-Kristensen, “Workplace Mobile-AssistedSecond Language Learning: Designing for Learner GeneratedAuthenticity,” Left to My Own Devices: Learner Autonomy andMobile-Assisted Language Learning (Innovation and Leadership inEnglish Language Teaching, J.E. D�ıaz-Vera, ed., vol. 6, pp. 183-195,Emerald Group Publishing Limited, 2012.

[23] R. McGreal, Online Education Using Learning Objects. Falmer Press,2004.

[24] D.A. Wiley, The Instructional Use of Learning Objects. Assoc. forEducational Comm. and Technology, 2002.

[25] F. Alonso, G. Lopez, D. Manriques, and J.M. Vines, “An Instruc-tional Model for Web-Based E-Learning Education with a BlendedLearning Process Approach,” British J. Educational Technology,vol. 36, no. 2, pp. 217-235, 2005.

[26] I. McAlpine and B. Allen, “Designing for Active Learning Onlinewith Learning Design Templates,” Proc. Ascilite ICT: ProvidingChoices for Learners and Learning, 2007.

[27] IMS Global Learning Consortium (GLC), IMS Learning DesignSpecification Version 1.0, http://www.imsglobal.org/learningde-sign/, 2003.

[28] P. Zervas and D. Sampson, “Enhancing Educational Metadatawith Mobile Assisted Language Learning Information,” Proc. 10thIEEE Int’l Conf. Advanced Learning Technologies (ICALT ’10), 2010.

[29] Council of Europe, A Common European Framework of Reference forLearning, Teaching and Assessment. Cambridge Univ. Press, 2001.

[30] D. Little, “The European Language Portfolio: Structure, Origins,Implementation and Challenges,” Language Teaching, vol. 35,pp. 182-189, 2002.

[31] D. Sampson, K. G€otze, and P. Zervas, “Delivering IMS LearningDesign Activities via Mobile Devices,” Proc. Seventh IEEE Int’lConf. Advanced Learning Technologies (ICALT ’07), 2007.

[32] R. Vuorikari and R. Koper, “Evidence of Cross-Boundary Use andReuse of Digital Educational Resources,” Int’l J. Emerging Technol-ogies in Learning, vol. 4, no. 4, pp. 51-56, 2009.

[33] R. Koper, “Combining Reusable Learning Resources and Serviceswith Pedagogical Purposeful Units of Learning,” Reusing OnlineResources: A Sustainable Approach to E-Learning, pp. 12-19, KoganPage, Limited, 2003.

[34] X. Ochoa, “Learnometrics: Metrics for Learning Objects,” Doctoraldissertation, KU Leuven, 2008.

[35] L. Petrides, L. Nguyen, C. Jimes, and A. Karaglani, “Open Educa-tional Resources: Inquiring into Author Use and Reuse,” Int’l J.Technology Enhanced Learning, vol. 1, no. 1-2, pp. 98-117, 2008.

[36] S.M. Duncan, “Patterns of Learning Object Reuse in the Connex-ions Repository,” All Graduate Theses and Dissertations, Paper423, http://digitalcommons.usu.edu/etd/423, 2009.


[37] A. Margaryan and A. Littlejohn, “Repositories and Communitiesat Cross-Purposes: Issues in Sharing and Reuse of Digital Learn-ing Resources,” J. Computer Assisted Learning, vol. 24, no. 4,pp. 333-347, 2008.

Panagiotis Zervas received the diploma in elec-tronics and computer engineering from the Tech-nical University of Crete, Greece, in 2002, themaster’s degree in computational science fromthe Department of Informatics and Telecommuni-cations of the National and Kapodistrian Univer-sity of Athens, Greece, in 2004. He is currentlyworking toward the PhD degree in the Depart-ment of Digital Systems, University of Piraeus,Greece. His research interests focus on context-aware mobile learning systems and digital sys-

tems for open access to educational resources and practices. He has10-year professional experience in the development and managementof International, European and National projects. He is the co-author ofmore than 65 scientific publications with at least 83 known citations. Hehas received four times Best Paper Award in International Conferenceson Advanced Learning Technologies. He is a member of the ExecutiveBoard of the IEEE Technical Committee on Learning Technology andthe technical manager of Educational Technology and Society.

Demetrios G. Sampson (SM’04) receivedthe diploma in electrical engineering from theDemocritus University of Thrace, Greece, in1989, and the PhD degree in electronic systemsengineering from the University of Essex, UnitedKingdom, in 1995. He is a professor of digital sys-tems for learning and education in the Depart-ment of Digital Systems, University of Piraeus,Greece, and a research fellow at the InformationTechnologies Institute (ITI), Centre of Researchand Technology Hellas (CERTH). He has been

the founder and director of the Advanced Digital Systems and Servicesfor Education and Learning (ASK) since 1999. His main scientific inter-ests are in the area of learning technologies. He is the co-author of morethan 306 publications in scientific books, journals, and conferences withat least 1,365 known citations (h-index: 20). He has received six BestPaper Awards in International Conferences on Advanced LearningTechnologies. He is a senior and golden core member of the IEEE andhe was the elected chair of the IEEE Computer Society Technical Com-mittee on Learning Technologies (2008–2011). He is the co-editor-in-chief of Educational Technology and Society (impact factor 1.171,2012). He is also a member of the steering committe of the IEEE Trans-actions on Learning Technologies, member of the advisory board of theJournal of King Saud University—Computer and Information Sciences,member of the editorial board of 20 International/National journals, anda guest co-editor of 26 special issues of International journals. His partic-ipation in the organization of scientific conferences involves: generaland/or program committee chair of 35 International conferences, pro-gram committee member of 330 International/National scientific confer-ences. He has been a keynote/invited speaker at 48 International/National conferences. He has been project director, principle investiga-tor, and/or consultant of 65 R&D projects with external funding at therange of 14 Million € (1991–2016). He is the recipient of the IEEE Com-puter Society Distinguished Service Award (July 2012).


facilitating teachers' reuse of mobile assisted language learning resources using educational...

Education

reuse of mall resources

eld of language learning

eld of mall

inuence reuse

introduction language

learning services

process of learning

learning tel