augmenting reality and formality of informal and non-formal settings to enhance blended learning

Augmenting Reality and Formalityof Informal and Non-Formal Settings

to Enhance Blended LearningMar P�erez-Sanagust�ın, Davinia Hern�andez-Leo, Patricia Santos,

Carlos Delgado Kloos, Senior Member, IEEE, and Josep Blat

Abstract—Visits to museums and city tours have been part of higher and secondary education curriculum activities for many years.

However these activities are typically considered “less formal” when compared to those carried out in the classroom, mainly because

they take place in informal or non-formal settings. Augmented Reality (AR) technologies and smartphones can transform such informal

and non-formal settings into digitally augmented learning settings by superimposing “digital” layers of information over physical objects

or spaces. At the same time, the formality of these settings increases when connected to formal settings through these digital layers.

The right combination of AR and mobile technologies with computer-based educational tools such as Learning Management Systems

(LMSs) drives this digital connection, leading to articulated blended learning activities across formal, non-formal and informal settings.

This paper contributes to the TEL field with: (1) three blended learning activities illustrating the idea of augmented informal/non-formal

settings; (2) results from the cross-analysis of these activities that evidence the impact of technology to enhance blended learning; and

(3) a set of lessons learned about the possibilities of NFC/GPS AR technologies and LMSs for blended learning. This work provides

insights for the design and implementation of similar technology-enhanced blended learning activities.

Index Terms—Virtual and augmented reality, blended learning, smartphones, learning management systems

Ç

1 INTRODUCTION

BORN within graphics, augmented reality (AR) was ini-tially a technologically challenging topic and an interac-

tion paradigm alternative (or complementary) to virtualreality [14]. The wide adoption of smartphones and avail-ability of AR software has made AR widespread. Novelapplications propose forms of interactions that superimposelayers of ‘digital’ contextualized information over ‘physical’settings offering new opportunities for learning experiences[33]. For instance, the iPhone application, Starmap, allowslearning about the constellations on a map displayed on itsscreen “superimposed” onto the real stars being viewed inthe sky. Within this context, we refer to AR as the technolo-gies that enable the superimposing of layers of ‘digitally’

contextualized information over ‘physical’ settings forenriching or augmenting the real world.

Many researchers seize new opportunities for learning inplaces other than the classroom using smartphones and ARtechnologies to bring these digital layers to informal andnon-formal settings [15], [16], [50]. On one hand, the any-where and anytime capabilities of mobile technologies andthe different types of sensors that they incorporate (e.g.,cameras, GPS) offer the opportunity of augmenting any set-ting or object with interactive digital information. Informalsettings such as forests or cities, or non-formal such asmuseums, become digitally augmented spaces with contex-tual information that scaffolds and supports learning [33].On the other hand, smartphones have been used to helplearners find connections between their daily life and educa-tion, bridging the gap between learning in formal, non-formal or informal settings [6], [23], [49]. The use of smart-phones in education has typically been associated withexploratory, informal-type activities because they takeplace in informal settings and do not follow an organizedcurricular structure [39]. However this work explores howusing combinations of smartphones and AR technologiesalong with learning management systems (LMS) informaland non-formal settings become augmented settings,enhancing blended learning (BL). In this context, BL isdefined as learning through combinations of activitiesacross formal, non-formal and informal settings that aremixed and integrated into the same learning flow or processusing combinations of technologies; that is understandingblend in a broad sense: blend of activities (including ele-ments of different learning theories or pedagogicalapproaches), settings and technologies [31].

� M. P�erez-Sanagust�ın is with the Departamento de Ingenier�ıa. Telem�atica,Universidad Carlos III de Madrid, Avd. Universidad 30, E-28911.E-mail: [email protected].

� D. Hern�andez-Leo is with the Departament de la Informaci�o i les Comuni-cacions, Universitat Pompeu Fabra, Barcelona, Roc Boronat 138, E-08018.E-mail: [email protected].

� P. Santos is with the BRILLE research centre, Education Department,West of England University, Frenchay Campus, Block S, BRISTOL BS161QY. E-mail: [email protected].

� C. Delgado Kloos is with the Departamento de Ingenier�ıa. Telem�atica, Uni-versidad Carlos III de Madrid, Avd. Universidad 30, E-28911.E-mail: [email protected].

� J. Blat is with the Departament de la Informaci�o i les Comunicacions, Uni-versitat Pompeu Fabra, Barcelona, Roc Boronat 138, E-08018.E-mail: [email protected].

Manuscript received 14 Mar. 2013; revised 12 Feb. 2014; accepted 10 Mar.2014. Date of publication 19 Mar. 2014; date of current version 2 July 2014.For information on obtaining reprints of this article, please send e-mail to:[email protected], and reference the Digital Object Identifier below.Digital Object Identifier no. 10.1109/TLT.2014.2312719

118 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 7, NO. 2, APRIL-JUNE 2014

1939-1382� 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

To address this challenge, we present three contribu-tions. First, we illustrate the concepts of augmented realityand augmented formality of informal and non-formal set-tings by giving an overview of three real blended learningactivities: Discovering the Campus 2009 [30], Discovering theCampus 2010 [7] and Discovering Barcelona [31]. Second, wecontribute with a cross-analysis of these three learningblended learning activities. All these activities weredesigned, implemented and evaluated in authentic learningsituations in previous work. However, a systematic cross-analysis of the three activities, which are similar interven-tions in several contexts, facilitates the identification of con-trasted evidences about the learning benefits of augmentingreality and formality of informal and non-formal settings.Third, a set of lessons learned about the possibilities of com-bining mobile, AR technologies and LMSs in blended learn-ing activities/contexts, discussing its advantages andlimitations, is presented. Altogether, this paper providesinsights about both the design (technological perspective)and application (educational perspective) of AR technolo-gies in informal and non-formal settings with formal learn-ing purposes.

The remainder of the paper is structured as follows.Section 2 reviews the literature framing the concepts ofaugmenting reality and formality of informal and non-for-mal settings. Then, Section 3 presents the three blendedlearning activities. Section 4 introduces the multicase thatstructures the cross-analysis of the three activities and themethodological approach of the analysis. Section 5 followspresenting the findings of the separated cases, the resultsof the cross analysis and other complementary findingsemerging from the analysis. Section 6 describes a list of les-sons learned from the analysis. Finally, Section 7 summa-rizes the main contributions of this paper and highlightsits main conclusions.

2 BACKGROUND

This section presents an overview of the literature in mobilelearning that inspires and sustains the concepts of“augmented reality” and “augmented formality” to framethe terminology adopted throughout this paper.

2.1 Augmented Reality: Layers of ‘Digital’Information to Support Learning

The concept of augmented reality has been associated withseveral meanings. Some definitions focus on the technologi-cal perspective and define AR as “the technology of addingvirtual objects to real scenes” [13]. However, the concept ofAR has evolved towards another (probably more natural)perspective. These new definitions focus on the idea of“blending” digital with real world information [22] forenriching and supplementing real settings and creating dig-itally augmented physical settings [33] or (sometimescalled) “blended spaces” [2]. As Dunleavy et al. (2009) [11]state: “AR interfaces enable ubiquitous computing’ modelsin which students carrying mobile wireless devices throughreal world contexts engage with virtual information super-imposed on physical landscapes (. . .). This type of mediatedimmersion infuses digital resources throughout the realworld, augmenting students’ experiences and interactions”.

Accordingly, we define AR as “the technologies that enablesuperimposing layers of ‘digital’ contextualized informationover ‘physical’ settings for enriching or augmenting realworld interactions”.

Many researchers have experimented with combinationsof smartphones and AR technologies for supporting learn-ing. FitzGerald et al. [15] and Frohberg et al. [16] offer inter-esting reviews of the state of art in this field. Most of theseexperiments propose activities based on socio-constructivisttheories, including elements mostly influenced by (some-how interrelated) learning approaches such as: (1) collabora-tive learning, which are “situations in which two or morepeople learn or attempt to learn something together” [9]; (2)mobile learning, “the use of wireless mobile technology toaccess information and learning materials from anywhereand anytime” [1] or any of its other acceptation [24]; (3) situ-ated learning [3], where learning is the product of the activityin a particular context and culture in which it is developed;or (4) inquiry-based learning, a “student-centred active learn-ing approach focused on questioning, critical thinking andproblem solving” [38]. For the purposes of this article, ourliterature review focuses only on activity-types based on theabove-mentioned pedagogies and using tag-based andGPS-based technologies as the means to interact with thecontext and to support learning. We classified these activitytypes into three categories.

1. Augmented outdoor guided trips. Use location-basedtechnologies such as GPS to relate media resourceswith a specific location. These resources automati-cally pop up to the user depending on their position.Typically, these are geo-learning experiences thatoffer information to guide learners through the activ-ity [40], [36], offering contextualized resources abouta particular setting/object [35] or recommendingmaterials that best match the students’ locations [12].

2. Augmented indoor guided tours. Use tag-based technol-ogies such as NFC, QR codes or AR markers toextend indoor elements/objects with digital infor-mation. Usually, these technologies are means forsupporting active learning and learning about partic-ular objects in indoor settings such as museums [17],classrooms [42], [34] or mixed areas with low GPSsignal [30], although AR markers have also beenused in open areas for triggering information aboutprojects or videos [25].

3. Educaching. Educaching can use both tag-based andGPS-based technologies to link physical surround-ings with digital contents [10]. The main particular-ity of these activities is that they introduce a stronggamification component.

2.2 Augmented Formality: Connecting DigitalLayers to Bridge Formal and Informal Settings

Mocker and Spear [27] review formal, non-formal and infor-mal learning and argue that two elements articulate thesedefinitions: (1) the setting where the learning takes place;and (2) the general approach to instruction. Accordingly,they define: (1) formal learning, occurring in traditional set-tings and whose learning objectives and the means to reachthem are decided by someone else besides the learner;

P�EREZ-SANAGUST�IN ET AL.: AUGMENTING REALITY AND FORMALITY OF INFORMAL AND NON-FORMAL SETTINGS TO ENHANCE BLENDED... 119

(2) non-formal learning, occurring in lifelong learning set-tings and whose objectives are decided by the learners, butthe means are proposed by others; and (3) informal learn-ing, where learners have little or no control at all over thechoice of learning objectives, but control the means that canresult into learning. In a more recent paper, Sefton-Green[39] defines informal learning in contrast to formal learningtaking into account the incorporation of ICT for learningpurposes as “two kinds of continuum”: organization (a cur-riculum, or how learning is structured) and setting (wherelearning takes place). That is, formal learning is structuredand organized, while informal learning does not have aclear structure, and is casual or accidental. Setting rangefrom the more formal (such as schools), through intermedi-ate or non-formal (museums or galleries), right to socialstructures (families and communities).

We adopt a combination of the arguments and defini-tions by both authors and adapt them to the context of thiswork. Specifically, we define formal, non-formal and infor-mal learning as a “continuum” between the way objectivesare defined and achieved and the setting where the activitytakes place. Accordingly we contend that in formal learningthe learning objectives are defined by someone else besidesthe learner and the means to achieve them is determined bysomeone other than the learner. In non-formal learning, thelearner controls what he wants to learn, but does not controlthe means to achieve this learning. In informal learningobjectives are not defined because learning is casual, but thelearner controls the means that can result into learning. Inaddition, we consider that settings range from formal (class-room), non-formal (museum type) or informal (those thatnot belong to any educational institution). Consequently,and according to these definitions, we understand that it ispossible to find formal learning situations occurring in for-mal, informal and non-formal settings, informal learningsituations taking place in formal, informal and non-formalsettings or non-formal situations taking place in formal,non-formal or settings.

Researchers in mobile learning assume that learningflows across locations, time, topics and technologies ratherthan occurring within a fixed location [40], [41]. This mobil-ity across contexts and spaces facilitates and supportsexploration and conversation, the fundamental processesby which meaning is sought and attained. Explorationinvolves physical movements through a setting and conver-sations are the bridge that connects learning across such set-tings [40], [49]. Both the interaction with the environmentand the mobility stimulate and promote active forms oflearning [22], augmenting and expanding the learning bene-fits of traditional activities.

Several studies introduce some of the aspects that sup-port this idea. These approaches benefit from the capabili-ties of mobile technologies to combine tasks conductedindoors and outdoors including elements of different learn-ing theories into blended learning activities that supportlearning in context, fostering students’ reflection and moti-vation [43], [23], [50].

In this frame, this paper focuses on informal and non-formal settings and on how to augment them for explicitlysupporting formal activities. Concretely, this work exploreshow smartphones combined with AR and educational

technologies such as LMSs can bridge formal, non-formaland informal settings to build up articulated blended learn-ing activities across them. We argue that technology sup-ports the integration of activities across formal, non-formaland informal settings and at the same time helps augmentformality of informal and non-formal settings.

3 THREE BLENDED LEARNING ACTIVITIES

This section presents three BL activities to illustrate howcombinations of technologies were used to augment realityand formality of informal/non-formal settings for enhanc-ing blended learning. These activities are Discovering theCampus 2009 and 2010 editions and Discovering Barcelona.All the activities were designed to include elements of learn-ing approaches such as collaborative learning, situated-based learning and mobile learning. Each activity employeddifferent technologies to augment reality and formality ofthe informal/non-formal setting, offering a different sup-port for social interactions and interactions within the set-ting. Discovering the Campus 2009 and 2010 used RadioFrequency Identification/Near Field Communication(RFID/NFC) technologies for augmenting a university cam-pus. Discovering Barcelona employed an ad hoc softwarebased on GPS technology for enriching tours of the city ofBarcelona with digital information. Discovering the Campus2009 and Discovering Barcelona used Moodle LMS for bridg-ing the connections between formal, non-formal informalsettings, whereas Discovering the Campus 2010 combined .LRN LMS with the use of the IMS Learning Design (IMSLD) specification. All the activities are the result of a partici-patory design process with practitioners, in which the selec-tion of technologies was driven by their particulareducational needs and the limitations imposed by the tech-nological infrastructure available in each educational insti-tution. All activities were enacted with real students andteachers and evaluated in previous research work [7], [30],[31]. In this work, we cross-analyse these activities from theperspective of how reality and formality of informal/non-formal settings were augmented in each activity to identifycontrasted evidences about its learning benefits.

3.1 Discovering the Campus 2009

Discovering the Campus 2009was included in the compulsorysubject “Introduction to Communication and InformationTechnologies” (ICIT) of the engineering degrees of the Uni-versitat Pompeu Fabra in 2009. The objective of this experi-ence was to facilitate students’ first contact with theuniversity campus: its services and community, while at thesame time meeting other freshmen. The experience wasstructured into three phases.

1. Discovering the campus. Students had to freely explorefive selected areas of the campus. For the explora-tion, students had three different options: (1) access-ing the university website; (2) walking around thecampus, reading posters fixed on key areas of thecampus and asking other students; or (3) participat-ing in an exploratory activity using smartphones.The students could freely choose one out of the threeoptions, or combine two or the three options; no spe-cific option was mandatory.


The campus was digitally augmented with 46interactive RFID/NFC tags distributed in keyphysical areas of campus buildings. Students usingthe third option were provided with NFC-enabledsmartphones to interact with the tags and accessthe videos, pictures and sounds that augmentedthe information about specific physical areas. Thisinformation was extracted from the universitywebsite. In this way, students choosing option 1, 2or 3 had the same information. Students had 20-30 minutes for exploring the campus. The streamof tags accessed by each student and the corre-sponding timestamps were stored in a log file inthe mobile phones. At the end of this phase, allstudents (in option 1, 2 or 3) filled in a web-basedquestionnaire in Google Forms about the differentareas visited during the exploration and their pre-ferred buildings and services.

2. Explaining the campus. The teacher placed Studentsinto groups of four or five, and each group wasassigned as an expert in one of the five areas of thecampus. Teachers used the log files obtained fromthe mobile phones via Bluetooth to identify the“building areas” expertise of the students accordingto the tags they accessed in their interactive visit tocampus. For those students performing the activityusing the other options (without NFC mobilephones), the information about their campus areasexpertise was extracted from the answers to the finalquestionnaire. The list of groups was published inthe institutional Moodle LMS. At home, each grouphad to prepare a presentation about their assignedarea and upload it to Moodle.

3. Reflecting about the campus. This activity was con-ducted individually. Each student reviewed all thepresentations available in Moodle LMS, to fill in aquestionnaire of 20 questions about the five areas ofthe campus.

The activity lasted two weeks with the participa-tion of 241 students and three teachers. For theexploratory activity 74 of 241 students voluntarilychose to perform the augmented exploratory activity(option 3), and the remaining 167 students one of theother options (108 the university website and 59chose walking around the campus, reading postersfixed on key areas of the campus and asking otherstudents).

In this activity the university campus was the non-formalsetting, which was augmented using RFID/NFC tags. Boththe log files registered in the NFC-enabled smartphonesand the answers to the questionnaires were employed toorganize the groups from one phase to the other to connectactivities across the campus, the classroom and home.

3.2 Discovering the Campus 2010

Discovering the Campus 2010 shares with the first edition ofthe activity (2009) the educational context, the learningobjectives and the use of NFC/RFID technologies to aug-ment the campus. The differences with the previous activ-ity stems on the technological system used to integratethe different learning settings. In the 2010 edition, IMS

Learning Design specification was used for supportingthe connections across settings. Specifically, a Unit ofLearning (UoL) in IMS LD was created to computation-ally represent the sequence of activities. The UoL wasinterpreted by the .LRN LMS and complemented with ageneric system integration (GSI) system [8]. The GSI auto-matically presented the activities in a sequence to the stu-dents and the required resources according to theirgroup. Also, the system facilitated the semi-automaticgroup formation according to the information gatheredfrom the visit.

As in the previous edition (2009), the activity combinedindividual and collaborative activities. However, the activ-ity was deployed into a two-hour session for 25 students toallow all the students to have a smartphone for the campusexploration activity. Students followed the flow of activitiesguided by the computer (.LRN system) instead of receivingthe instructions from the teacher or via Moodle. Although,both the campus exploration with the mobile phones andthe web were the same than in the 2009 edition, the flow ofactivities changed slightly with respect to the first version ofthe experience.

1. Discovering the campus. The flow of activities in IMSLD was presented to the students using the .LRNplatform. Learners were divided into two groups.While one group performed the exploration of thedigitally augmented campus with the mobilephones, the other group explored the campus via theuniversity website. The groups swapped tasks after20 minutes. The answers to the questionnaire andthe activity log files were automatically analysed bythe system, producing .csv file with a summaryof the events generated by each student. This .csv filewas shown as a spreadsheet to the teacher contain-ing: (1) the number of tags accessed per building;and (2) the building expertise (the building with themaximum number of tags accessed).

2. Explaining the campus. The actions of the students inthe previous phase were considered for the distribu-tion of the students in expert groups. This was asemi-automatic process, carried out with the formu-lae of the spreadsheet, which collected all the infor-mation about the exploratory activity. The teachercould manipulate this final distribution directly overthe spreadsheet. Once the grouping phase finishedand no additional group changes were expected, theteacher marked the activity as finished in the .LRN.This action synchronized the flow with the informa-tion in the spreadsheets in a way that .LRN automat-ically showed each student their expert group andthe specific activity they had to complete (i.e., elabo-rate a presentation of the assigned campus building).All the members of each group had to work togetherin the presentation and upload it into the .LRNsystem.

3. Reflecting on the campus. In this final phase the teacherhad to press a button in .LRN to automatically sendthe delivery of the submitted presentations to all thegroups. Students had to review all the presentationsand to access to the final assessment task.


Thirty one students and four teachers participated in thisactivity. As in the previous edition, the campus was aug-mented with RFID/NFC tags. However, in this case, a sys-tem based on IMS LD and .LRN was in charge ofautomatically integrating the outcome of the activitiesacross locations.

3.3 Discovering Barcelona

High school teachers designed Discovering Barcelona to helptheir students reflect about the urbanism and socio-geo-graphical characteristics of Barcelona city. The experiencecombined an exploratory activity in the city and a reflectiveactivity in the classroom. For the exploration, the 32 stu-dents, in six groups of five to six people and equipped witha smartphone, visited one of the six districts of the city. Thedistricts were augmented previously with questions thatteachers created and geo-positioned using QuesTInSitu[36]. QuesTInSitu is a web-based application for generatingsequences of questions associated to a geographical coordi-nate. At the time of this experience, QuesTInSitu did notintegrate a module for detecting the actual position of thestudents in real time, and was complemented with Media-Scapes, maps with the position of the questions that wereinstalled in the mobile devices for the activity. MediaScapes[45] allows showing users media files associated to a geo-graphical coordinate when passing by this location. Theactivity was divided into four phases:

1. Assigning districts. The students were groupedaccording to their knowledge and preferences aboutthe districts of Barcelona collected from an individ-ual survey that they answered at home.

2. Discovering the district. Each group, equipped with aGPS enabled smartphone with Internet connection,simultaneously explored the district to which theywere assigned. During the visit groups had toanswer the geo-positioned questions, which had

associated feedback that guided the students to thenext point and gave them hints about the urban andsocial characteristics of the area. In addition, stu-dents collected pictures and notes about the district.

3. Reflect about your district. Students prepared a presen-tation about the district they visited using the mate-rial collected and uploaded to the QuesTInSituapplication. Afterwards, students made their presen-tations to the rest of their classmates via the institu-tional Moodle of the high school. All the groupspresented the conclusions of their visit in class twoweeks after the visit to Barcelona.

In this case, the city of Barcelona was the informal settingaugmented with questions generated with QuesTInSitu andpositioned in MediaScapes maps. Then, the outcomes gen-erated by the students during their visits were the inputsfor preparing the presentation in the classroom using Moo-dle, so as to integrate both activities into a continuous learn-ing flow.

4 CROSS-ANALYSIS OF THE THREE BL ACTIVITIES

To study the learning benefits of augmenting reality andformality of informal settings, we cross-analyze the findingsof the three blended learning experiences in a multicasestudy adjusted to our research purposes. Multicase study isa methodology typically employed by educational research-ers to study experiences of cases in real situations [44].However, multicase studies have been successfully appliedin engineering-oriented studies as an instrument for study-ing the effects of the technology in context [20] or in otherdisciplines when the evaluation involves human-relatedreal experiences [5].

This paper adapts the multicase methodology accord-ing to the proposal by Hern�andez-Leo et al. [20] to facili-tate the cross-analysis of their findings (See Fig. 1). Thestrength of using this methodology relies on enriching the

Fig. 1. Schema of the multicase study. Research aim, research questions, and issues of the two cases comprising the multicase.


understanding of the main research question and provid-ing multiple perspectives of the same proposition for astronger validation.

As shown in Fig. 1, we structured the multicase startingfrom the main research aim, which corresponds to what“we seek to understand”: Exploring the blended learning bene-fits of using smartphones combined with AR technologies andLearning Management Systems for augmenting reality and for-mality of informal and non-formal settings. The research aim isthe umbrella for defining the two research questions thatwill guide the evaluation and cross-analysis of the cases:(1) What are the learning benefits of using smartphones and ARTechnologies to transform informal/non-formal settings into digi-tally augmented learning settings?; and (2) What are the learningbenefits of using smartphones, AR Technologies combined withLMSs to augment formality of informal/non-formal settings?

We organized the three blended learning activitiesunder study according to the information they provideabout the main research aim. Specifically, we structure theexperiences into two case studies depending on: (1) thetechnology employed to augment the informal/non-formal setting; (2) the learning objectives of the activity;and (3) the informal/non-formal setting where the activitytakes place. The two cases are: CASE 1: Discovering theCampus and CASE 2: Discovering Barcelona. As shown inFig. 1, the case Discovering the Campus comprises the twolearning activities Discovering the Campus 2009 and 2010.The objective of this case is to understand the learningbenefits of using smartphones and tag-based technologies(RFID/NFC) with Moodle or .LRN and IMS LD and toaugment reality and formality of the University Campus(a non-formal setting). The case Discovering Barcelonaonly comprises the activity Discovering Barcelona. In thiscase the objective is to understand the learning benefits ofusing smartphones and position-based applications (Ques-TInSitu and MScape) with Moodle to augment the realityand (learning) formality of the city of Barcelona (an infor-mal setting). The issues under study in each case (as for-mulated in Fig. 1) are the particularization of the researchquestions (of the multicase) for each specific case.

To answer the research questions of the multicase, weanalysed the data of the different cases from the perspectiveof the issues. The data were collected using mixed methodscombining quantitative and qualitative data gathering tech-niques [21], [46]. The quantitative data are useful for show-ing trends, while the qualitative provide an in-depthunderstanding of the learning experience enacted [19]. Thequantitative sources of information were closed questionsin questionnaires for students and teachers as well as finalcourse grades. The qualitative data were open questionsand observations taken by researchers during the experi-ments. Combining these techniques allow taking intoaccount the contextual issues of each case (characteristics ofstudents and practitioners, achievement of the educationalbenefits. . .), particularly important when analysing techno-logically supported learning practices as authentic situa-tions [26], [46]. Then, for analytically contrasting all thesedata we triangulate qualitative and quantitative data (ques-tionnaires, observations, logs. . .) to have several confirma-tions and perspectives of the general tendencies in the useof a technology [18].

5 THE CROSS-ANALYSIS

The first step to carry out a cross-analysis consists in analy-sing the data of Cases 1 and 2 separately, guided by itsissues, and extracting the findings for each case. In a secondstep, the findings of each case are organized according tothe two research questions to which they provide answers.The findings of a case give the perspective of the researchquestions from a particular activity and context. Treating allfindings together allows extracting contrasted results aboutthe research questions based on evidences, which corre-spond to the data behind each finding.

5.1 Findings of the Separated Cases in theMulticase

Tables 1, 2, 3 and 4 show the findings of the two cases foreach issue under analysis. Tables 1 and 2 correspond to thefindings of the Case 1: Discovering the Campus and Tables 3and 4 to those related with the Case 2: Discovering Barcelona.While Tables 1 and 3 show the findings around the issuesrelated to augmenting reality of informal/non-formal set-tings (particularization of research question 1 for the tech-nologies applied in each case), Tables 2 and 4 focus on thefindings derived from the issues on augmenting the formal-ity of informal/non-formal settings via its integration withactivities in formal settings (particularization of researchquestion 2 for each of the cases). The information in thesetables is organized as follows.

The first column shows the findings of the case for theissue indicated in the caption of the Table. Each finding isidentified with a code written in bold that indicates thenumber of the finding and the case to which it belongs:FCampX for findings related with Discovering the campusand FBCNX for those related with Discovering Barcelona(where X is the number of finding). We use these codes inthe remainder of the paper to refer to the findings. The sec-ond column shows the partial results that support each ofthe findings, which were extracted from the analysis of therow data of each experiment according to the issues in themulticase (see Fig. 1). The third column refers to supportdata selected for exemplifying the type of information thatsupports the partial results. Each selected support datapoints to specific sections/pages of previous publicationswhere additional data are included. The on-line supplemen-tary material of this paper (see Sup. Mat.1) provides thecomprehensive list of partial results and data sets of thethree blended learning activities analysed in the multicase.

5.2 Results of the Cross-Analysis

Table 5 shows the results of the cross-analysis and indicatesthe findings from the two cases that support them. The find-ings are referred in the table using the same codesemployed in Tables 1, 2, 3, 4 highlighted in bold.

First column in Table 5 shows the results regarding theresearch question 1 What are the learning benefits of usingsmartphones and AR technologies to transform informal/non-formal settings into digitally augmented learning set-tings? The results in this column evidence that NFC/GPS

1. http://193.145.50.210:8080/TLT/TLT-ComplementaryData/Index.html.


TABLE 1Findings Case Discovering the Campus—Issue 1—What Are the Learning Benefits of Using Smartphones and AR

Tagging-Based Technologies (NFC) to Digitally Augment the Campus Space?

TABLE 2Findings Case Discovering the Campus—Issue 2: What Are the Learning Benefits of Using Smartphones,AR Tagging-Based Technologies (NFC) Combined with Moodle LMS or IMS LD and .LRN to Integrate

Activities Across form., Non-form and Infor. Settings?


enabled smartphones and AR technologies are a goodmechanism for digitally augmenting an informal/non-formal setting with educational purposes to support activi-ties with learning benefits.

The first result 1.I indicates that combining NFC/GPSenabled smartphones with AR technologies such as NFCtags or QuesTInSitu for creating geo-positioned routes ofquestions supports situated learning. Findings FCamp1 andFBCN1 show that the students learn better about a particu-lar object, area or location in situ. Adding digital informa-tion over the reality facilitates students’ ability tocontextualize information from different sources (Internet,

school books. . .) and focus on concrete aspects, obtaining abetter understanding of the object/area under study. Find-ing FCamp1 of the Case 1: Discovering the Campus especiallysupports this result. Partial results of this finding indicatethat students voluntarily participating in the activity sup-ported by smartphones and AR tags performed better com-pared with those who did not (doing the activity online orexploring the campus without technological support), dem-onstrating a more accurate and profound knowledge aboutthe campus’ services and resources in the related assess-ment task. Besides, partial results of FBCN1 of the Case 1:Discovering Barcelona point out that using GPS enabled

TABLE 3Findings Case Discovering Barcelona Issue 1: What Are the Learning Benefits of Using GPS Enabled Smartphones and

AR Technologies to Digitally Augment the City?

TABLE 4Findings Case Discovering Barcelona Issue 2: What Are the Learning Benefits of Using GPS Enabled Smartphones,

AR Technologies with Moodle LMS to Integrate Activities Across Formal, Non-Formal and Informal Settings?


smartphones as the mediating artefact to answer questionsabout the city in situ help students focus their attention onthe task and retain the contents and details of the areasunder study.

The result 1.II indicates that although NFC/GPS enabledsmartphones interactively guide the students along thespace where the activity takes place, this guidance is notenough. Partial results of finding FBCN3 of the Case 2 Dis-covering Barcelona support this statement, indicating thatother complementary elements are needed to supportstudents’ exploration. On the one hand, students feel com-fortable with the guidance provided by the smartphonesand the GPS-based technologies, although they also find ituseful to complement the activity with a physical map. Inaddition, findings FCamp2 and FBCN1 indicate that, whilesmartphones allow students to move freely, the informationfacilitated through the NFC tags (about the different pointin the campus) or GPS applications (showing the situationof the student all the time) provide the necessary guidelineto locate themselves, supporting learners in the practice oforientation skills. Partial results of FBCN1 indicates thatusing smartphones and AR GPS-based technologies pro-motes students’ autonomy and active learning, whileenabling students to practice their orientation and techno-logical skills and to understand the urban space and theirelements. In addition, partial results of FCamp2 show thatboth teachers and students perceived the combination ofSmartphones and AR NFC-based technologies as a helpfuland useful approach for learning about how the campusspace is organized, locating the different buildings andservices.

On the other hand, teachers highlighted the importanceof the feedback as a way to structure the activity. Feedbackcan be composed by hints (designed by the teachers) toguide students along the learning flow; and help them focuson the relevant aspects of the activity in situ (where they donot have the support of the teacher) (FBCN3).

The second column in Table 5 shows the results regard-ing the research question What are the learning benefits ofusing smartphones and AR technologies combined withLMSs to augment the formality of informal/non-formal set-tings? The results related with this research question indi-cate that the use of NFC/GPS enabled smartphones and ARtechnologies combined with LMSs is a good mechanism tobuild up digital connecting layers across formal and

informal/non-formal settings. These digital layers integratethe activities taking place in formal and informal/non-formal settings into a continuous and articulated learningflow. A learning flow in which settings of informal/non-formal nature augment their formality, since they supportformal activities that generate outcomes which have a directimpact on follow-up in-class activities.

Finding 2.I in Table 5 indicates that these combinations oftechnologies support students to apply the contents learnedin class to other contexts. This is supported by the partialresults of finding FBCN4, FCamp3 and FCamp4. First, partialresults of FBCN4 show that in activities in which studentshave been physically in contact with the element understudy, reflection is promoted by supporting with ICT thecollection evidences to bring them back to the classroom. Inthe case “Discovering Barcelona”, the outcomes from thevisit were collected with QuesTInSitu question/answer sys-tem in combination with Moodle, to compile and share themain outcomes and discuss them in class. Teachers valuethe outdoor technology-supported activity because studentsreinforce concepts and ideas worked in class. Besides, find-ing FCamp4 shows that a variety of technologies also facili-tate the inclusion of a variety of media that enriches thestudents’ learning experience. For the teachers participatingin the 2009 edition of Discovering the Campus, using differenttechnologies also means having contact with several techno-logical environments and types of content. For the students,this variety enriches the learning experience, while helpsthem settle down the concepts learned in the differentcontexts.

However, and as indicated by partial results of FCamp3,using smartphones and AR technologies for interactivelycollecting information is not enough to have an articulatedand integrated activity across settings. This integrationrequires technologies driving the connections along digitallayers for capturing the interactions in one setting and tosend them to the other. In the case of Discovering the Cam-pus this was done capturing the students’ intentionalityduring the campus exploratory activity into log files.Then, this information was sent with Moodle in the firstedition and .LRN/ IMS LD for the second one, and theinformation was used to form groups for the next activi-ties. These grouping policy fostered students’ collabora-tion. Similar results have been observed in other studiessuch as Myartspace [48], in which evidences collected in a

TABLE 5Results of the Cross-Analysis


museum were the input for a further reflective activity inthe classroom.

Finally, result 2.II indicates that complementing NFC/GPS smartphones with LMSs and Log files systems do notonly scaffold students along the blended learning activity,but also support teachers in its orchestration across settings.This is especially noticeable since it is supported by multi-ple partial results of findings FCamp5, FCamp6 and FBCN4.FCamp5 indicate that combining smartphones, AR NFCtechnologies and IMSLD/.LRN systems teachers and stu-dents perceived the activities taking place in the campus,home and the classroom as a continuous articulated learn-ing flow that enriches the learning experience because ofthe variety of contents and technology-enhanced tasks.

Complementary to these results, partial results of find-ing FCamp6 indicate that this combination of technologiesproposed supported teachers in defining the groupingpolicies taking into account the students’ performances inthe different settings. RFID/NFC and smartphones tech-nologies were in the case Discovering the Campus themeans for capturing students’ interaction into log filesand defining digital representations of their personalexperience with the campus. These representations werethe input for defining roles or profiles for further activi-ties. Also, partial results of this finding point out that theteachers successfully followed students’ activity acrosssettings. It is noticeable that both FCamp5 and FCamp6 arefindings related with the 2010 edition of Discovering theCampus, in which the learning flow was structured usingan integrated set of technologies in which most of theorchestration processes along the learning flow wereautomatic. Besides, partial results of finding FBCN4points out that teachers highlight the importance of usinga technological bridge connecting formal and informalsettings for facilitating a more comprehensive evaluationof the activity that took place in the informal setting.

5.3 Other Complementary Findings

In addition to the results in Section 5.2, other relevant issueswere detected during the cross analysis.

Data in both the cases of Discovering the Campus and ofDiscovering Barcelona indicate that students enjoyed andvalued the initiative of incorporating technologies as alearning support. Students’ comments highlight that theyappreciated the interactive and mobility possibilitiesoffered by the smartphones because they felt having the“control” and being the main actors of the activity. More-over, students appreciate working in groups because thisis something helpful and different compared with othersimilar activities. For instance, one student from the caseDiscovering Barcelona said: “This activity is better and morefun compared to other activities (such as going to a museum).Moreover, this activity allows us to work in groups in a veryfun way” [Q-st-final] ([30] p. 460). Further, in both casesstudents’ and teachers’ comments describe the activityusing adjectives such as “innovative”, “dynamic”,“interesting”, “fun” and “enriching”.

Data in the case Discovering Barcelona indicate that theteachers attribute partly the students’ motivation as intrinsicto the use of ICT in school activities. In order to exclude theHawthorne effect [29], we investigated other aspects as

identified in Pintrich’s framework [32]. Pintrich’ frameworkidentifies several aspects that should be taken into accountwhen designing activities for increasing students’ motiva-tion: efficacy, control, interest, values and goals. Accordingto Pintrich’s principles, we identified the following aspectsinfluencing motivation in the activities under study: (1) thevariety of contents and technology-enhanced tasks(FCamp4 & FCamp5, FBCN4); (2) the value of the activity tolearn about aspects worked in class and needed by the stu-dents (FCamp1, FCamp2, FBCN4); (3) the collaborativecomponent of the activities (FCamp4); (4) the feeling ofchoice and control (FCamp5, FBCN3); and (5) the self-effi-cacy and competence components based on feedback andadaptation of the activities (FCamp5, FCamp6, FBCN3).

All these results indicate that, although both teachers andstudents are not used to these technology-based activities,they quickly adopted the technology as a support for learn-ing, highlighting its benefits and advantages over othermore traditional situations.

6 LESSONS LEARNED

For a deeper understanding and discussion of the results,we study and compare the technological combinationemployed in each case and its implications. The comparisonresults on a list of lessons learned that offers the researchcommunity some insights about how to augment realityand formality of informal/non-formal settings for enhanc-ing blended learning.

1. Use tag-based technologies for digitally augment-ing closed non-formal settings such as museums orinformal open settings such as a city in which theobjects/areas under study are close to each other tosupport situated and active learning [1.I in Table 5].In most mobile devices, GPS technologies do notwork and lose precision in short distances smallerthan 2 meters. Besides, GPS cannot be used indoors.In these cases, tag-based technologies are the mostappropriate solution. Several studies demonstratethe suitability of NFC Technologies for controllingthe students going to the class in an educationalcentre [42], transforming the classroom into an inter-active setting. These technologies can, for example,support scenarios in which the sound of the mobilephones is set off when students enter the class orin which promoting active learning is achievedwhen students are asked to interact with a NFC-augmented blackboard for answering questions [34].This is also the case of Discovering the Campus, inwhich the campus mixed closed areas (the library)and opened areas (campus courtyard) where theGPS connectivity and precision was very low.

2. Capture students’ interests and profiles in non-guided exploratory learning situations [1.II inTable 5]. Tag-based technologies require the activeparticipation of the students to get the digital infor-mation superimposed to an augmented object. Regis-tering students’ deliberate interaction with an objectis a mirror of their interests in such object or area. Ina learning situation with a high amount of informa-tion available but a limited time, the interactions


with the tags represent the students’ preferencestowards the particular environment. This is the strat-egy used in the case Discovering the Campus in whichthe aim was to capture the students’ interests regard-ing particular campus areas and consider this infor-mation to shape the following activity (in this case asthe parameter for the group formation policy). WithGPS-based technologies it would be possible to havea similar effect if, when displaying content to the stu-dents at a concrete position, they could chose to reador not read such information. Then, the profile of thestudents would be defined according to those infor-mation points accessed.

3. Use GPS-based technologies in guided exploratorylearning situations in which students have to fol-low determined routes [1.II in Table 5]. GPS technol-ogies allow positioning digital information into aparticular geographical coordinate. Applicationssuch as QuesTInSitu use this GPS functionality totrigger automatically contextual questions to the stu-dents when passing by a concrete location. In thiscase, receiving contextual information is not a volun-tary action. Students will always receive the informa-tion and then choose if they want to read it or rejectit. For this reason, these technologies are interestingfor promoting serendipitous learning [47]. In addi-tion, GPS-based technologies are useful when guid-ing the students along a particular route. In theDiscovering Barcelona case, the GPS helped studentsadvance in their exploration along the city.

4. Combine tag-based and GPS-based applicationswith paper maps or appropriate feedback systemsto structure and guide exploratory activities [1.II inTable 5]. Using paper maps provide students withan overview of the complete augmented learning set-ting. If using tag-based technological approaches themaps will help students to find the interactive infor-mation points. If using GPS-based applications, themaps will be especially useful in case of low GPS sig-nal. Moreover, similar experiments combining bothtypes of maps show that paper maps provide the stu-dents with a global view of the whole area to beexplored, while digital maps are used to be focus ina particular sub-area [37].

Offering feedback to students in particular loca-tions can support the guidance too, at the same timethat the feedback allows highlighting importantlearning aspects of the activity. Moreover, providingfeedback (clarifying the result of the question andindicating what the correct answer is) can serve asreward to the students, having positive effects intheir motivation [32], [35].

5. Integration of technologies for augmenting formal-ity [2.I., 2.II., 2.III in Table 5]. Creating augmentedBL activities requires a seamless and articulated inte-gration between the data generated in learning activ-ities taking place in formal, non-formal and informalsettings. As stated by Vavoula et al. in [49] “A suc-cessful learning activity should be integrated withother learning events, building on them and contrib-uting to their outcomes”.

Currently, there are several educational tools specificallydeveloped to drive blended learning activities data flowsacross settings. For example, the nQuire [28] toolkit for sup-porting inquiry based learning between settings such ashome and the classroom, or Myartspace systems [48], whichprovides both Web and mobile applications to supportinquiry learning between classrooms and museums. Bothproposals provide a technological solution to send the datagenerated in one setting to influence a further activity takingplace at another different setting. Usually, these technolo-gies are ad-hoc monolithic solutions designed in collabora-tion with technologist and educational experts to supportthis data flow between settings for a particular learningpurpose/scenario.

However, not all the educational institutions or teacherscan afford these ad hoc developments. In these scenarios,educators should select from existing technologies for sup-porting BL across settings (e.g., mobile applications forinformal settings, learning management systems for formalsettings) and integrate the data generated in each to articu-late the learning flow. Selecting these technologies has to bedriven by both the educational design of the activity and bythe technologies available in their institutions (see [49] for aguideline on what to consider for selecting the appropriatetechnology in each activity). But, in BL activities that requireintegrating data generated in different settings, the selectedtechnologies have also an implication on how the data flowacross settings is produced [31].

There are BL situations across settings in which it issufficiently satisfactory to combine the use of severalexisting technologies suitable for each setting to get anarticulated data flow between activities. This is what hap-pened in the case of Discovering Barcelona, in which thecombination of QuesTInSitu and Moodle articulated theflow of the data generated in the city to the classroom.The data collected during the trip in the informal settingwas the input for preparing the final presentations in theclassroom. Both teachers and students were in charge ofmoving the data collected during the trip with QuesTIn-Situ (pictures and other evidences) to Moodle for sharingit with the rest of their colleagues. This process was donemanually. Manual integration is an affordable and easy-to-adopt solution for augmenting formality for thoseinstitutions that do not have resources to develop a tool-ing ad hoc. Many educational institutions already useLMSs that practitioners could combine with other tech-nologies such as Layar to articulate the data flowsbetween formal, non-formal and informal settings, aug-menting their formality.

However, this type of integration entails some limita-tions. First, technologies available for the teachers have alimited set of functionalities that may not match with theteachers’ interest and needs, forcing practitioners to re-define their activity according to what the tool offers. Sec-ond, manually recovering the data generated in one settingto use it in another setting requires a control from theteacher side, which can be unfeasible in complex activitydesigns or in situations with massive number of students.In these situations, this type of integration can be very ineffi-cient, hindering the adoption of these BL activities as dailyschool practices.


This is what happened in the 2009 edition of Discoveringthe Campus. In this edition of the activity, teachers manuallyprocessed the log files of the students’ interaction with thecampus to organize them in groups. Teachers used Moodleto create the different groups and assign them their corre-sponding tasks according to their profile. But adapting thegroup formation, supporting transitions between activitiesand artefacts across locations using diverse technologies,displaying the appropriate tools to students depending ontheir group, or assigning the correct task to each group wasvery demanding and complex, entailing lot of difficultiesfor the practitioners.

To alleviate this complexity we proposed for the 2010edition an integration of technologies based on the use of alearning technology specification for automatizing thesetasks. Concretely, we proposed using a unit of learning cod-ified in an extended IMS LD and running in the .LRN LMSto structure the learning flow. In this case, the .LRN LMSand a complementary system compliant with the IMS LDspecification [7], [8] were in charge of interpreting the con-ditions computationally represented in the learning flow toautomatically generate the groups of students according tothe information collected in the log files and to automati-cally show to each group its corresponding activity. Theresult was an articulated blended learning flow that teach-ers could easily orchestrate and monitor, having an over-view of how the data generated in one setting flowed to theother setting.

Therefore, although in both 2009 and 2010 editions theactivity was the same, the results of the cross-analysisshows that using computational representations of thelearning flow facilitated the integration of data generatedacross settings, making the activity more feasible and easy-to-adopt by the teachers. Moreover, using automatic sys-tems for transferring data from one setting to the other alle-viates some orchestration tasks, facilitating the adoption ofthese BL activities in real educational contexts.

7 CONCLUSIONS AND FUTURE WORK

This paper shows that smartphones combined with the rightAR technologies and educational tooling such as LMSsenable augmenting informal/non-formal settings forincreasing the natural continuity between learning across set-tings. The contribution of the paper provides insights bothabout the design (technological perspective) and application(educational perspective) of AR technologies in informal/non-formal settings with formal learning purposes.

A cross-analysis of three authentic blended learningactivities organized in a multicase study involving twocases, Discovering the Campus and Discovering Barcelona illus-trates this idea. Each case proposes a combination of tech-nologies that result in an integration of formal withinformal/non-formal settings that allows transferring thedata from one to the other enhancing BL activities. First,using NFC/GPS enabled smartphones and AR technologiesto augment an informal/non-formal setting is a good mech-anism to support learning in context, transform traditionaltrip field activities into interactive and structured activitiesand foster students and teachers’ motivation and interest intechnology. Second, when combining these technologies

with LMSs and log files or other ad hoc software for captur-ing students’ interaction within the informal setting fosterstudents’ reflection, enrich experiences combining differentmedia and help teachers orchestrate the activity. The find-ings supporting these results evidence that using technol-ogy in blended learning activities facilitates the data flowbetween formal, non-formal and informal settings, produc-ing a stronger connection between activities taking place inthese settings and leading to an augment of formality.Finally, these results are sketched as a set of lessons learnedabout the possibilities of these technologies in two differentactivities/contexts.

The results and the lessons learned in this work do notonly point out the encouraging possibilities of specific com-binations of technologies, but they also identify directionsfor advancing the technology and design of activities forinformal and non-formal settings to enhance blended learn-ing. Within these directions, we identified several researchavenues that could be pursued in future work.

From a technological perspective, one of these directionsis the development of tools or services incorporating the ele-ments that have shown benefits to enhance BL activitiesacross formal and informal settings. For example, givingfeedback to guide the activity, providing monitoring orlearning analytics features to see how students’ progress inthe activity or to automate concrete workflow aspectsrelated with task distribution among students and acrosssettings. In this line, it would be interesting to explore thelearning benefits of combining feedback functionalities withgamification techniques, from two perspectives, as a way offostering their interest in the activity and as a means sup-porting students in advancing along the activity flow.

From a more pedagogical perspective, another line forfuture work would be to develop authoring features provid-ing BL gamified activity patterns, such as the augmentedtreasure hunts type games or guided trips identified in theliterature. This functionality would support practitioners inthe design and deployment of BL activities based on GPStechnologies or tag-based technologies, promoting theiradoption in real scenarios. Tools such as the “QR TreasureHunt Generator,”2 which provides an automatic solutionfor generating treasure hunting activities based on QR codeswith multiple choice questions, is a first approach towardsthis line.

Finally, derived from the complementary findingsobtained from the cross-analysis, it would also be of interestto explore, experiment and evaluate how to incorporate inthese activity patterns factors that can potentially enhancestudents’ motivation, such as those defined by Pintrich andGroot [32] or other related motivational models or toolssuch as the IMI and EMI models used in the work byBuckworth et al. [4].

ACKNOWLEDGMENTS

This work was partially funded by the Spanish Ministry ofScience and Innovation with the EEE Projects (TIN2011-28308-C03-01, TIN2011-28308-C03-03), by the eMadrid Proj-ect (S2009/TIC-1650) funded by the Regional Government

2. http://www.classtools.net/QR/.


of Madrid and with the support from Alliance four Univer-sities. The authors would also especially like to thank PilarSanagust�ın Viu, Francesc Santolaria, T�onia Boqu�e from theIES Duc the Montblanc.

REFERENCES

[1] M. Ally, Mobile Learning: Transforming the Delivery of Education andTraining. Edmonton, AB, Canada: Athabasca Univ. Press, 2009.

[2] D. Benyon, O. Mival, and S. Ayan, “Designing blended spaces,” inProc. 26th Annu. BCS Interaction Spec. Group, 2012, pp. 398–403.

[3] J. S. Brown, A. Collins, and P. Duguid, “Situated cognition andthe culture of learning,” Edu. Researcher, vol. 18, no. 1, pp. 32–42, 1989.

[4] J. Buckworth, R. E. Lee, R. Gail, L. K. Schneider, and C. C.DiClemente, “Decomposing intrinsic and extrinsic motivationfor exercise: Application to stages of motivational readiness,”Psychology Sport Exercise, vol. 8, no. 4, pp. 441–461, 2007.

[5] P. Cairns and A. L. Cox, Research Methods For Human-ComputerInteraction. Cambridge, U.K.: Cambridge Univ. Press, 2008.

[6] J. Cook, N. Pachler, and C. Bradley, “Bridging the gap? Mobilephones at the interface between informal and formal learning,” J.Res. Center Educational Technol., vol. 4, no. 1, pp. 3–18, 2009.

[7] L. de-la-Fuente-Valent�ın, M. P�erez-Sanagust�ın, P. Santos, D.Hern�andez-Leo, A. Pardo, C. Delgado Kloos, and J. Blat,“Technological support for the enactment of collaborative scriptedlearning activities across multiple spatial locations,” Future Gener-ation Comput. Syst., vol. 31, pp. 223–237, 2014.

[8] L. de-la-Fuente-Valent�ın, A. Pardo, and C. Delgado Kloos,“Generic service integration in adaptive learning experiencesusing IMS learning design,” Comput. Edu., vol. 57, no. 1, pp. 1160–1170, 2011.

[9] P. Dillenbourg, “What do you mean by collaborativelearning?,” in Collaborative-Learning: Cognitive and ComputationalApproaches, P. Dillenbourg, ed., Oxford, U.K.: Elsevier, 1999,pp. 1–19.

[10] S. M. Dobyns, M. S. Dobyns, M. S. and E. E. Connell, (2013), EDU-CACHING: Capturing the spirit of the hunt for learning, D. C.:Nat. Assoc. Gifted Children. [Online]. Availablehttp://www.nagc.org/index.aspx?id=1844

[11] M. Dunleavy, C. Dede, and R. Mitchell, “Affordances and limita-tions of inmersive participatory augmented reality simulations forteaching and learning,” J. Sci. Edu. Technol., vol. 18, pp. 7–22, 2009.

[12] M. M. El-Bishouty, H. Ogata, and Y. Yano, “PERKAM: Personal-ized knowledge awareness map for computer supported ubiqui-tous learning,” Edu. Technol. Soc., vol. 10, no. 3, pp. 122–134, 2007.

[13] N. A. M. El Sayed, H. H. Zayed, and M. I. Sharawy, “ARSC: Aug-mented reality student card,” Comput. Edu., vol. 56, no. 4,pp. 1045–1061, 2011.

[14] S. Feiner, B. Macintyre, and D. Seligmann, “Knowledge-basedaugmented reality,” Commun. ACM, vol. 36, no. 7, pp. 53–62, 1993.

[15] E. FitzGerald, A. Adams, R. Ferguson, M. Gaved, Y. Mor, and T.Rhodri, “Augmented reality and mobile learning: the state of theart,” in Proc. World Conf. Mobile Contextual Learn., Oct. 2012,pp. 62–69.

[16] D. Frohberg, C. Goth, and G. Schwabe, “Mobile learning projects:A critical analysis of the state of the art,” J. Comput. Assisted Learn.,vol. 25, no. 4, pp. 307–331, 2009.

[17] G. Ghiani, F. Patern�o, C. Santoro, and L. D. Spano, “UbiCicero: Alocation-aware, multi-device museum guide,” Interacting withComput., vol. 21, no. 4, pp. 288–303, 2009.

[18] E. G. Guba, “Criteria for assessing the trustworthiness of natural-istic inquiries,” Educ. Technol. Res. Develop., vol. 29, no. 2, pp. 75–91, 1981.

[19] E. G. Guba and Y. S. Lincoln, “Competing paradigms in qualita-tive research,” in Handbook of Qualitative Research, N. K. Denzinand Y. S. Lincoln, eds., New Delhi, India: Sage, 1994, pp. 163–194.

[20] D. Hern�andez-Leo, I. Jorr�ın-Abell�an, E. D. Villasclaras-Fern�andez,J. I. Asensio-P�erez, and Y. Dimitriadis, “A multicase study for theevaluation of a pattern-based visual design process for collabora-tive learning,” J. Vis. Languages Comput., vol. 21, no. 6, pp. 313–331, 2010.

[21] R. B. Johnson, A. J. Onwuegbuzie, and L. A. Turner, “Toward adefinition of mixed methods research,” J. Mixed Methods Res.,vol. 1, no. 2, pp. 112–133, 2007.

[22] L. Johnson, R. Smith, H. Willis, A. Levine, and K. Haywood, The2011 Horizon Report. The New Media Consortium, Austin, TX,2011.

[23] A. Kurti, D. Spikol, and M. Milrad, “Bridging outdoors andindoors educational activities in schools with the support ofmobile and positioning technologies,” Int. J. Mobile Learn. Organi-sation, vol. 2, no. 2, pp. 166–186, 2008.

[24] Y. Laouris and N. Eteokleous, “We need an educationally relevantdefinition of mobile learning,” in Proc. 4th Mobile Learning, 2005,vol. 2005, pp. 290–294.

[25] T. Y. Liu, T. H. Tan, and Y. L. Chu, “2D barcode and augmentedreality supported English learning system,” in Proc. IEEE/ACISInt. Conf. Comput. Inform. Sci., Jul. 2007, pp. 5–10.

[26] A. Mart�ınez-Mon�es, Y. Dimitriadis, E. G�omez-S�anchez, B.Rubia-Avi, I. Jorr�ın-Abell�an, and J. A. Marcos, “Studying par-ticipation networks in collaboration using mixed methods,”Int. J. Comput.-Supported Collaborative Learning, vol. 1, no. 3,pp. 383–408, 2006.

[27] D. W. Mocker and G. E. Spear, Lifelong Learning: Formal, Non-formal, Informal, and Self-directed. The ERIC Publications Clearing-house on Adult, Career, and Vocational Education, Columbus,Ohio, USA, 1982, no. 241.

[28] P. Mulholland, S. Anastopoulou, T. Collins, M. Feisst, M. Gaved,L. Kerawalla, and M. Wright, “nQuire: Technological support forpersonal inquiry learning,” IEEE Trans. Learning Technol., vol. 5,no. 2, pp. 157–169, Apr.–Jun. 2012.

[29] R. Gillespie, Manufacturing knowledge: A history of the Hawthorneexperiments. Cambridge, U.K.: Cambridge Univ. Press, 1993.

[30] M. P�erez-Sanagust�ın, G. Ram�ırez-Gonzalez, D. Hern�andez-Leo,M. Mu~noz-Organero, P. Santos, J. Blat, and C. Delgado Kloos,“Discovering the campus together: A mobile and computer-basedlearning experience,” J. Netw. Comput. Appl., vol. 35, no. 1,pp. 176–188, 2011.

[31] M. P�erez-Sanagust�ın, P. Santos, D. Hern�andez-Leo, and J. Blat,“4SPPIces: A case study of factors in a scripted collaborative-learning blended course across spatial locations,” Int. J. Comput.Supported Collaborative Learn., vol. 7, no. 3, pp. 443–465, 2012.

[32] P. R. Pintrich and E. V. De Groot, “Motivational and self-regulatedlearning components of classroom academic performance,” J. Edu.Psychol., vol. 82, no. 1, pp. 33–40, 1990.

[33] S. Price and Y. Rogers, “Let’s get physical: The learning benefits ofinteracting in digitally augmented physical spaces,” Comput. Edu.,vol. 43, no. 1–2, pp. 137–151, 2004.

[34] G. Ram�ırez, M. Mu~noz, and C. Delgado Kloos, “Exploring NFCfor supporting mobility in learning scenarios,” in Proc. IADIS Int.Conf. Mobile Learn., Apr. 2008, pp. 11–13.

[35] P. Santos, M. Balestrini, V. Righi, J. Blat, and D. Hern�andez-Leo,“Not interested in ICT? A case study to explore how a meaningfulm-learning activity fosters engagement among older users,” in Proc.Eur. Conf. Technol. Enhanced Learn., 2013, vol. 8095, pp. 328–342.

[36] P. Santos, M. P�erez-Sanagust�ın, D. Hern�andez-Leo, and J. Blat,“QuesTInSitu: From tests to routes for assessment in situactivities,” Comput. Edu., vol. 57, no. 4, pp. 2517–2534, 2011.

[37] P. Santos, D. Hern�andez-Leo, and J. Blat, “To be or not to be in situoutdoors, and other implications for design and implementation,in geolocated mobile learning,” Pervasive Mobile Comput., 2013, inpress, doi: http://dx.doi.org/10.1016/j.pmcj.2013.09.001

[38] J. R. Savery, “Overview of problem-based learning: Definitionsand distinctions,” Interdisciplinary J. Problem-Based Learn., vol. 1,no. 1, pp. 9–20, 2006.

[39] J. Sefton-Green, Literature Review in Informal Learning: With Tech-nology Outside School (Futurelab Series), NESTA Bristol, UK, 2004,pp. 1–39.

[40] M. Sharples, I. Arnedillo-Sanchez, M. Milrad, G. Vavoula, andG. Balacheff, “MobileLearning: Small devices, big issues,” inTechnology Enhanced Learning: Principles and Products, Ludvig-sen, N., Jong, S., Lazonder, T., Barnes, S., Eds., New York, NY,USA, Springer-Verlag, 2009, pp. 2–14.

[41] M. Sharples, J. Taylor, and G. Vavoula, “A theory of learningfor the mobile age. Learning through conversation and explo-ration across contexts,” in Medienbildungin neuen Kulturr€aumen,Bachmair and B. , eds., New York, NY, USA: Springer-Verlag,2010 Ed.

[42] C. W. Shen, Y. C. J. Wu, and T. C. Lee, “Developing a NFC-equipped smart classroom: Effects on attitudes toward computerscience,” Comput. Human Behavior, vol. 30, pp. 731–738, 2013.


[43] D. Spikol and M. Milrad, “Combining physical activities andmobile games to promote novel learning practices,” in Proc. IEEEInt. Conf. Wireless, Mobile, Ubiquitous Technol. Edu., Mar. 2008,pp. 31–38.

[44] R. Stake, Multiple Case Study Analysis. New York, NY, USA: TheGuilford Press, 2006.

[45] S. P. Stenton, R. Hull, P. M. Goddi, J. Reid, B. Clayton, and T.Melamed, “Mediascapes: Context-aware multimedia experi-ences,” IEEE Multimedia, vol. 14, no. 3, pp. 98–105, Jul.–Sep. 2007.

[46] A. Tashakkori and C. Teddlie, Handbook of Mixed Methods in Socialand Behavioral Research, Thousand Oaks, CA, USA: Sage, 2003,vol. 1, pp. 241–272.

[47] N. Vavoula and M. Sharples, “KLeOS: A personal, mobile, knowl-edge and learning organization system,” in Proc. IEEE Int. Work-shop Wireless Mobile Technol. Edu., Aug. 2002, pp. 152–156.

[48] N. Vavoula, M. Sharples, P. Rudman, J. Meek, and P. Lonsdale,“MyartSpace: Design and evaluation of support for learning withmultimedia phones between classrooms and museums,” Comput.Edu., vol. 53, no. 4, pp. 286–299, 2009.

[49] G. Vavoula, M. Sharples, P. Rudman, P. Lonsdale, and J. Meek,“Learning bridges: A role for mobile technologies in education,”Edu. Technol., vol. 47, pp. 33–36, 2007.

[50] N. Winters, “What is mobile learning?,” in Big Issues In MobileLearning: Report of a Workshop by the Kaleidoscope Network of Excel-lence Mobile Learning Initiative, M. Sharples, ed., Nottingham, U.K.:University of Nottingham, 2006, pp. 7–12.

Mar P�erez-Sanagust�ın received the MS andPhD degrees from the Universitat Pompeu Fabra(UPF), Spain, in 2006 and 2011, respectively.She is a researcher at the Gradient Lab of theGradient at the Universidad Carlos III de Madrid(UC3M). Her main research interests include:TEL, CSCL, m-learning, MOOCs, crowd-learn-ing, orchestration across spaces, smart citieslearning, and authoring tools for designing tag-based mobile learning activities.

Davinia Hern�andez-Leo received the MS andPhD degrees in telecommunications engineeringfrom the University of Valladolid, Spain, in 2003and 2007, respectively. She is currently an assis-tant professor at the Universitat Pompeu Fabra -Barcelona, where she coordinates the learningtechnologies research section of the GTI group.She is the vice-director of the Polytechnic Schooland leads its Teaching Quality and InnovationSupport Unit. She has been honoured with sev-eral awards in the field of Technology Enhanced

Learning and has authored more than 85 publications. Her researchinterests include CSCL, ICT-mediated orchestration of activities acrossspaces, educational modelling techniques, and (co-)design processes.

Patricia Santos received the MS and PhDdegrees from the Universitat Pompeu Fabra(UPF) in Barcelona, Spain, in 2007 and 2011,respectively. Since 2007 to 2013, she hasbeen collaborating with the Interactive Tech-nology Group (UPF), and currently she is aresearch assistant at the University of theWest of England (UWE), in the Bristol Centrefor Research in Lifelong Learning and Educa-tion (BRILLE). Her research interests includecomputing-based testing, m-learning, ubiqui-

tous learning, and learning in situ.

Carlos Delgado Kloos (M’05-SM’06) is afull professor of telematics engineering atUniversidad Carlos III de Madrid. At this univer-sity, he is also a vice-chancellor for infrastruc-tures and environment. His interests includetechnology-enhanced learning. He has beenand is presently involved in many projects withEuropean or Spanish funding. He has publishedmore than 200 articles in national and interna-tional conferences and journals. He has furtherwritten a book and co-edited another six. He is

a senior member of the IEEE.

Josep Blat received the graduation degree inmaths from the University of Val�encia, the PhDdegree from Heriot-Watt, Edinburgh, and a post-doc degree at Universit�e Paris-Dauphine. Hewas head of the Maths and Computer ScienceDepartment at the Universitat de les Illes Balearsfor several years and is currently a professor ofcomputer science at Universitat Pompeu Fabra.He leads the research group in interactive tech-nologies with activity in human-computer interac-tion, graphics, and learning technologies.


augmenting reality and formality of informal and non-formal settings to enhance blended learning

Documents