Open Research OnlineThe Open University’s repository of research publicationsand other research outputs

Beyond prototypes: Enabling innovation intechnology-enhanced learningOtherHow to cite:

Scanlon, Eileen; Sharples, Mike; Fenton-O’Creevy, Mark; Fleck, James; Cooban, Caroline; Ferguson, Rebecca;Cross, Simon and Waterhouse, Peter (2013). Beyond prototypes: Enabling innovation in technology-enhancedlearning. Open University, Milton Keynes.

For guidance on citations see FAQs.

c© 2013 The Authors

Version: Version of Record

Link(s) to article on publisher’s website:http://tel.ioe.ac.uk/wp-content/uploads/2013/11/BeyondPrototypes.pdf

Copyright and Moral Rights for the articles on this site are retained by the individual authors and/or other copyrightowners. For more information on Open Research Online’s data policy on reuse of materials please consult the policiespage.

oro.open.ac.uk

BEYOND PROTOTYPESEnabling innovation in technology-enhanced learning

TEL Technology Enhanced Learning Research Programme

tel.ac.uk

TEL Technology Enhanced Learning Research Programme tel.ac.uk

beyondprototypes.com

tel.ac.uk 3

Executive summary 05

1 Introduction 09

2 Methods 10

3 What is TEL? 12

Case study: Microelectronics Education Programme 13 3.1 Understanding education 14 3.2 Education as a super-stable system 15

4 Achievements underpinned by TEL 17

UK centres of excellence in TEL 18 4.1 Success story: mobile learning 19 4.2 Success story: Scratch 19 4.3 Success story: xDelia 20

5 TEL innovation: challenges and misconceptions 22

5.1 Shift the focus from technology to pedagogy and practice 23 5.2 Look beyond the formal education sector 23 5.3 Widen the TEL community 24 5.4 Connect TEL research and practice 25 5.5 Find new ways to assess the contribution of TEL 25 5.6 TEL success is not necessarily commercial success 26 Case study: Cognitive Tutor software 27

6 The process of TEL innovation 28

6.1 The TEL Complex 28 Case study: Yoza Cellphone Stories 30 6.2 Configurational innovation and bricolage 30 6.2.1 Configurational innovation 30 6.2.2 Social practices as part of the TEL complex 31 6.2.3 Bricolage 31 6.2.4 Bricolage and ‘persistent intent’ within the TEL space 32 Case study: iZone Driver Performance 32 6.3 The TEL Innovation Process 33 6.3.1 Vision of educational change 33 6.3.2 Pedagogical research and expertise 33 6.3.3 Developing practices in parallel to formal education 34 6.3.4 Co-constructing new practices 34 6.3.5 Methodology of Design-Based Research 35

7 Implications for research 36

7.1 Recommendations for researchers 37

8 Implications for policy and for funding 38

8.1 Recommendations for funders and policy makers 39

9 Challenges and opportunities 40

Bibliography 42

Acknowledgments 46Research team

CONTENTS

4 Beyond Prototypes: Enabling innovation in technology-enhanced learning

The TEL research programme, which ran from 2007 to 2013, has generated some substantial gains in our understanding of how to design and deploy technologies for learning.

These findings, together with the growing field of technolog y-enhancedlearning internationally, are witnessing the growth of TEL research into a vibrant academic field, extending throughout the UK and beyond. Yet there is a surprising failure to translate the findings, prototypes and outputsof projects into commercial products and ser vices that individually andcollectively achieve radical change in the quality of teaching and learning.

This difficulty seems part of a general problem of translating innovation in the laboratory (or classroom or school) into commercial gain:

A key recurring issue that has been raised in the Science and TechnologyCommittee’s previous inquiries is the difficulty of translating research intocommercial application, particularly the lack of funding—the so-called “valleyof death”. (Commons Select Cttee, 20111).

The field of Technology Enhanced Learning, despite some notable exceptions,is rife with results that never made it across the valley of death. In the TELresearch programme, there were some exciting and innovative examples ofworking prototypes that solved significant research problems. Yet few of theseprojects have successfully taken their prototypes to market. Three of the eightfunded TEL projects achieved success in gaining follow-on funding from theESRC specifically earmarked for the achievement of “impact”, although it istoo soon to know if and how such impact will be achieved, and more generally,the relationship between impact and the commercial exploitation of projects’outputs. In general, despite the fact that all projects successfully designedand built effective prototypes of systems: the question is how to move fromprototype to product.

This report addresses this issue head-on from an interdisciplinary perspectivethat brings together experts in diverse relevant fields including educationaltechnology, organizational behavior, innovation dynamics. I am grateful to theauthors and all those who gave their time to help clarify these difficult andimportant issues.

Prof Richard NossDirector, Technology Enhanced Learning Research Programme.London Knowledge Lab | Institute of Education | University of London

1http://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/news/111215-new-inquiry---bridging /

THE TEL RESEARCH PROGRAMME (TEL)

tel.ac.uk 5

EXECUTIVE SUMMARY

It sets out what can be done toimprove the process of moving fromacademic research and innovativeprototypes to effective andsustainable products and practices. Indoing so, it shows that technologicaldevelopment is only a small part ofthe picture. Significant and lastingTEL innovation requires long-termshifts in practice. These shifts are not confined to the classroom ortraining environment; they requirealterations to many differentelements of the education system. Inorder to make these shifts, differentcommunities and groups need towork creatively together over time, so policymakers and funders shouldplan for engagement with teamsable to initiate, implement, scale andsustain long-term innovation.

An expert multidisciplinary teamcarried out the research underpinningthis report. Initial analysis of thefield of TEL research, developmentand policy was used to select keyexamples of TEL innovation fordetailed study. Innovation was takento be the practical implementationof new ideas and technologies withthe intention of having an observableimpact on teaching and/or learning.The initial phase included systematicanalysis of data collected from in-depth interviews with key figures fromresearch and industry. Each member ofthe research team brought substantialpersonal expertise to the researchprocess, enabling them to set thefindings within a broader context. Thiswas a strength of the study, allowingteam members to link their analysisnot only to the field of educationaltechnology but also to understandingsdeveloped in the fields of organisationbehaviour and innovation dynamics.

This executive summary introducesthe four key insights described

in the report, links each withrecommendations to enablesuccessful TEL innovation and, finally,outlines the structure of the report.

Key insight 1: The TEL ComplexTechnology-enhanced learningconsists of much more than a setof research-informed products. Itis a complex system, which includescommunities, technologies andpractices that are informed bypedagogy (the theory and practice of teaching, learning and assessment).The many elements of the ‘TELComplex’ must all be taken intoaccount as an innovation is designed,developed and embedded. At theheart of the TEL Complex is a vision ofhow learning may be enhanced by theuse of technology. This vision requiresthe development or adaptation oftechnology over time in order tosupport a pedagogical approach.

Concurrent implementation ofpedagogy and technology requiresconsideration of current practices,including the activities andexpectations of learners and teachers.Less obviously, implementation mustalso take into account practices relatedto areas as diverse as local and nationalassessment criteria, health and safety,staff training, administration andprovision of technical support. In orderto understand these, researchers needto consider the ecology and technicalcontext into which an innovation isto be introduced. To do this, theymust be able to work effectivelywith diverse other communities,particularly learners, teachers andtechnology developers.

This innovation work takes placein a wider setting that includes thelocal, national and internationalenvironment. Research and education

The Beyond Prototypes report provides an in-depth examination of the processes of innovation in technology-enhanced learning (TEL).

6 Beyond Prototypes: Enabling innovation in technology-enhanced learning

are carried out within a context thatis shaped by regional and nationalpolicy, by funding constraints, andby the need to generate sufficientrevenue to ensure that an innovationis sustainable even when short-termfunding has run out. A successfulprocess of innovation pays attentionto these high-level issues as well asto issues that can be shaped andcontrolled locally.

Summary: TEL involves a complexsystem of technologies and practices.In order to embed significant TELinnovation successfully, it is necessaryto look beyond product developmentand pay close attention to the entireprocess of implementation.

Recommendations: Policy andfunding should support changes in pedagog y and practice, as well asthe technological developments thatwill support these. Project teamsshould be encouraged to identifythe elements that must be taken intoaccount in order to enable sustainableimplementation of an idea orprototype in the context of a vision of the enhancement of learning. Inorder to do this, researchers needto engage with the individuals andcommunities that will play a role in the implementation process.

Key insight 2: Persistent intentThe diverse nature of the TELComplex means that a linear model ofinnovation in which research findingsare applied and then adopted is rarelyappropriate within the educationsystem. Success in TEL is associatedwith ‘persistent intent’ – efforts by agroup of vision-enactors to developinspirational ideas and turn them

into products and practices overan extended period of time. Thisrequires both long-term commitmentand focused action. Teams ofresearchers need persistent intentin order to develop their work overtime with a shared educational goalin mind. Many different academic andbusiness-based research projects maybe aligned in order to work towards the same educational goal.

Persistent intent motivatesresearchers to work closely with thecommunities that will be involved in implementation, developing ashared vision that is owned not onlyby the project team, but also bythose who will take it forward oncethe research programme is completeand the development team has left.To carry out this work successfully,researchers need opportunities todevelop the skills that will enablethem to bridge the gaps betweenthose different groups.

Summary: Significant innovationsare developed and embedded overperiods of years rather than months.Sustainable change is not a simplematter of product development,testing and roll-out.

Recommendations: Policy andfunding should take into accountthe need for extended development.There should be capacity to supportindividuals and teams to engagein long-term projects capable ofturning inspirational ideas into fullyembedded products and practices.Researchers and developersshould be encouraged to plan forsustainability. The implementationand success of plans for sustainabilityshould be evaluated.

Key insight 3: BricolageThe work involved in successful TEL innovation can be characterisedas ‘bricolage’. This is a productiveand creative innovation process

Look beyond product development and pay close attention to the entire process of implementation.

tel.ac.uk 7

Summary: TEL innovation is aprocess of bricolage. This processincludes informed and directedexploration of the technologiesand practices required to achievean educational goal. It involvesexperimentation to generate freshinsights, and a creative use ofavailable resources. It also requiresengagement with a range ofcommunities and practices.

Recommendations: Policyand funding should encouragethe development of skilled,multidisciplinary teams that are ableto complete the TEL innovationprocess. Policy and funding shouldalso support the experimentationthat is necessary to generate freshinsights and achievable visions ofeducational developments.

Key insight 4: EvidenceDevelopment and implementationof new approaches to teaching andlearning must be trialed and testedso that widespread adoption of TELinnovation is based on evidence andnot on theor y alone. This evidencecould come from comparativetrials in classrooms or in trainingenvironments, judging success byimprovement in test scores. It couldcome from newly emerging sources,such as the learning analytics andcomparative (A/B) testing that areenabled by the increasing amountsof data generated by virtual learningenvironments, MOOCs and othereducational technologies. Orit could arise from educationaltransformations, opportunities to teach or learn in ways thatare simply not possible withouttechnolog y (such as distanceeducation providers offering degree

courses through online learning topeople who are unable to study at a conventional university).

We need new ways of judgingwhether an innovation is successfulin enhancing learning. Traditionalscientific methods developed forthe laboratory, such as randomisedcontrolled trials, have an importantrole to play in revealing the changesproduced by altering individualvariables, but are not adequate to dealwith situations in which everythinghas changed because a process ofbricolage has engaged the entire TELComplex. Quantitative measures cansignal whether learners know morethings, but are less useful in assessingwhether their understanding hasbeen deepened or whether they haveacquired the skills to learn more or

to work independently in the future.Without valid methods of identifyingsuccess, evidence-based innovationcannot take place.

Summary: Successful implementationof TEL innovation requires evidencethat the projected educational goalhas been achieved. Reliable evaluationsmust be carried out; their findingsmust be disseminated and acted on.Methods of evaluation are requiredthat can be applied to processes ofinnovation and to institutional change,as well as those that can be applied toshifts in technology usage.

Recommendations: Policyand funding should require theevaluation of TEL innovations in terms of their educational impact. The findings of theseevaluations should be available toother researchers and developers,

Methods of evaluation are required that can be applied to processes of innovation and to institutional change.

that involves bringing togetherand adapting technologies andpedagogies, experimentation togenerate further insights and awillingness to engage with localcommunities and practices.Bricoleurs do not start a projectand then consider which tools andmaterials will be required to achievetheir goals. Instead, they reviewtheir available tools and resourcesand work out how to use them toachieve their goal or somethingclose to their goal.

Bricolage is a practical process ofinnovation. It may be informed by a deep understanding of theory andunderpinning research, but does notrely solely on a theoretical model of what should work. It engages with relevant communities to ensurethat innovation works in practice and in context. Bricoleurs payattention to the restrictions andconstraints of a situation, and takesteps to overcome or compensatefor them. Through creativereinterpretation and arrangement of local practices and resources, they can enable new possibilities.

Successful TEL innovators, bothin academia and in business, arebricoleurs who achieve educationalgoals by bringing together pedagogicapproaches, diverse technologicalelements, frameworks and socialpractices. If TEL innovation is to beembedded successfully, bricoleursneed to be able to understand andtake into account the perspectives of different stakeholders, and tobuild links between the experiencesand knowledge of differentcommunities. Inter-disciplinar ycollaboration will involve educationtheorists, policy makers, software and technolog y developers, teachers,learners and other practitioners, all with a shared intent to move ideas across boundaries and toexplore new approaches to learningand teaching.

8 Beyond Prototypes: Enabling innovation in technology-enhanced learning

including those without access touniversity libraries. New methods of evaluation, such as the use oflearning analytics or comparative A/B testing where appropriate,should be developed and put into practice.

The key insights of this reportindicate that changes are necessaryto the ways in which TEL innovation isplanned for, funded and implemented.The continuing need is not forabstract grand challenges or short-term initiatives, but for a sustainedbuilding of capacity in technology-enhanced learning, through graduateprogrammes and investment innational hubs of expertise that sharetalent and facilities.

Organisation of the reportThe key insights and recommendationsset out in this Executive Summary aredeveloped in more detail in the mainbody of the report. This is structuredas follows.

Following the Executive Summary and Introduction:

Section 2 describes the study onwhich this report is based. In thefirst phase, close to 100 projects,products and programmes wereconsidered. Desk research, case studyand interviews were used to examinea selection of these in depth. In thesecond research phase, project teammembers, with a range of expertisein different fields, worked together toanalyse the data, develop insights andwrite the report.

Section 3 defines technology-enhanced learning, tracing itsorigins, identifying its main areas offocus, linking it to related fields andidentifying key past policy initiatives.The section goes on to exploredifferent conceptions of education,introducing the idea that educationis a super-stable system within

which successful innovation requiresattention to a range of differentpressures and practices.

Section 4 provides an overview ofTEL successes, pointing to the UK’srole as a world leader in this area and identifying TEL’s role indevelopments as diverse as theWorld Wide Web and the iPad. Threeareas of success – the field of mobilelearning, the development of theScratch educational programminglanguage, and the xDelia project thatdeveloped learning applications forfinancial traders – are consideredin detail. The section also identifiesreasons why TEL successes may gounnoticed, including the significanttimescales involved.

Section 5 deals with challenges toTEL research and innovation. Sixmisconceptions about these areexamined, and recommendationsthat will increase the potential of TEL to achieve widespread impactare identified. These show thatfunders, researchers and policymakers all have a role to play inachieving that impact.

Section 6 sets out key contributionsof this report. TEL should beconsidered as a technology complex,made up of a series of interconnectedelements that cannot be changedin isolation. A model of the TELComplex is set out, centred on avision of educational change The TELInnovation Process is also modelled,and bricolage is set at its heart.Design-based research is identifiedas a core methodology to supportevidence-driven innovation.

Section 7 examines the implicationsof this report for research. Persistentintent, engagement over time and theuse of an appropriate methodologyare identified as priorities. Successfulresearch also requires engagingwith the practices and stakeholdersthat must be taken into accountif research-informed innovation isto be embedded in practice. Thesection ends by considering theimplications for researchers of therecommendations identified in thisExecutive Summary.

Section 8 examines the implicationsof this report for policy andfor funding, identifying currentproblems, particularly in relationto sustainability, and proposing

solutions. The section ends byconsidering the implications forfunders and for policy makers of therecommendations identified in thisExecutive Summary.

Section 9 focuses on the way forwardfor TEL research. It identifies acontinuing need for sustained buildingof capacity in TEL, through graduateprogrammes and investment innational hubs of expertise that sharetalent and facilities.

Boxed case studies provide examplesof TEL innovation in practice.●

The continuing need is not for abstract grand challenges or short-term initiatives, but for a sustained building of capacity in technology-enhanced learning.

tel.ac.uk 9

1 INTRODUCTION

That programme supported eightlarge interdisciplinary projects todemonstrate how learning can betransformed through innovative useof digital technology, ranging fromdeveloping the social skills of autisticchildren to training dentists throughhaptic (sense-of-touch) simulators.Three of these projects have receivedfurther funding to embed the findingsand technology into educationalpractice.

As a final initiative of the TELResearch Programme, this reportwas commissioned to explorethe broader issues of translatinginnovation from early prototypes tosustained impact. Such impact maycome through widespread adoptionof new technology-enabled methodswithin the education sector, throughsuccessful educational products,or through new activities in homes,museums and outdoor environmentsthat involve learning with technology.The remit of the report is to indicatethe barriers to impact of innovationsin technology-enhanced learning andto propose new routes to achievinglarge-scale sustained transformationsin teaching, learning and assessmentthat benefit society.

The report addresses the question:

What should researchers, fundersand policy makers do to improvethe translation from innovativeprototypes to effective andsustainable products and practices?

In order to provide answers, thestudy on which this report is basedconsidered six facets of the generaltheme of moving beyond prototypes.These can be summarised as: (1)Challenges of commercialisation

within UK and international contexts,(2) Perspectives from differentsectors, (3) Audit of UK examplescommercialised internationally, (4)Assistance to market, (5) Examples ofsuccess and (6) Relationship betweenimpact and commercial exploitation.

One puzzle to be explored is thatsuccessive decades have seenmajor innovations in TEL that havepromised to transform education.These innovations have includededucational television in the 1960s,language labs in the 1970s, computer-based instruction in the 1980s,integrated learning systems in the1990s, virtual worlds for learning inthe 2000s, and Massive Open OnlineCourses (MOOCs) in the 2010s. Someof these have received extensive presscoverage. Yet for none of these hasthe initial roll out been underpinnedby extensive research, nor have anybeen subject to systematic trials ofthe kind that might be carried outbefore the introduction of a new druginto medical practice.

Most of these innovations have fadedfrom national attention. Some, suchas language labs and virtual worlds,have been adopted on a small scalewithin formal educational settings.MOOCs are still in the headlines; feware underpinned by research, andproviders are currently looking forways of assessing the educationalimpact of various formats. Meanwhile,schools, universities, businesses andindividual learners have graduallyadopted digital technologies for awide range of educational purposes,ranging from note taking to onlineassessment. TEL researchers areoften left playing catch-up, assessingthe effectiveness of technologies thathave already been rolled out at scale.

In 2007, the Economic and Social Research Council and the Engineering and Physical Sciences Research Council jointly funded a national Technology Enhanced Learning (TEL) Research Programme.

In order to address this problem, this report examines effective waysof establishing connections betweenresearch and successful adoptionof TEL. It proposes a model of TELthat extends beyond an initial periodof research and development and itconnects this model to a process ofTEL innovation (see Section 6).●

10 Beyond Prototypes: Enabling innovation in technology-enhanced learning

2 METHODS

The findings reported here are based on a systematic analysis of data collected from in-depthinter views with key opinion leaders,plus selected exemplar cases,published reports and commentar y.To this process was brought thesubstantial personal expertise ofindividual members of the projectteam. The team included expertswith extensive experience inpedagogy and technology gainedboth inside and outside the universitysector. These included professors of organisational behaviour andinnovation dynamics as well asprofessors of open education and educational technology. Thiscombination meant that the analysiscould draw on and combine insightsfrom these different fields. The reportis therefore not narrowly focused on educational technology, but places this in the wider context ofunderstandings of the different typesof innovation that are employed inmarket-oriented organisations.

An additional strength of this studyhas been the sustained involvementof project team experts at all stages,from research design to data analysis.This has made it possible to reviewthe data critically through multipleinterpretative lenses and to adopt a constructivist grounded theoryapproach [1] to identifying andunderstanding emergent themes.

The first phase of the researchdetermined an initial list ofinterviewees and cases, and definedresearch criteria for the interview andcase exemplars. Of particular interestwere the impacts, success factors andissues encountered in implementingTEL innovation; the degree to which

each innovation was considereddisruptive or in accord with currenteducational practice; and possibleactions to improve impact of TELresearch and development.

During the course of the study, close to 100 projects, products and programmes in the area of TELwere identified. The project expertsassessed each of these potentialcase studies in terms of how well it aligned with the six facets of theproject. They were also assessed in terms of the required level, nature and ease of access to relevant data. A full list of casesconsidered in the selection andresearch process is available on the project website (see Projectwebsite box). Approximately half the identified cases were selectedfor further consideration by initialdesk research and, in the light of this work, eight were selected forfocused case study. This approachcreated a tapestry of data, at varyinglevels of granularity, relating to a wide selection of UK andinternational TEL initiatives.

A series of interviews complementedthe desk research. These helped toidentify and explore issues in a morenuanced way than would otherwisehave been possible, and providedinsight from a range of seniorperspectives. Inter viewees includedinternational applied researchers as well as directors or seniormanagers of research institutes,public organisations and privatecompanies. In all, 14 semi-structuredinter views were carried out, eachbased on a protocol developed jointlyby the project team. Inter viewees’consent was obtained to record and

Section 2 describes the study on which this report is based. Close to 100 projects, products and programmes were considered. Desk research, case studies and interviews were used to examine a selection of these in depth. Project team members, with a range of expertise in different fields, worked together to analyse the data, develop insights and write the report.

tel.ac.uk 11

2 METHODS

use their data in the research and,where possible, inter viewees wereprovided with a sample list ofquestions prior to inter view. Theproject website gives details of theinter viewees and of the protocolused. Interviews were recorded andtranscribed, and the inter viewerchecked these transcriptions. In addition, previously publishedpresentations, commentaries andmedia inter views were used toprovide contextual information.

The second phase of the studyinvolved an iterative weekly cycle of data collection and analysis thatspanned a two-month period. Thisallowed theoretical sampling to beused to augment the list of cases and inter viewees as themes began to emerge. Each week, every memberof the project team was required tocode and identify key themes within a selection of in-depth inter viewtranscripts and data from caseresearch [2, 3]. After the weeklycoding had taken place, the team met to compare the coded data,agree on emerging themes, andcompare their analysis with that of previous weeks. This processprovided scope for the expert teammembers to analyse the data fromthe perspective of their field ofexpertise. Emerging themes wererecorded as memos on the rawinterview transcripts and case studydocumentation, as well as in theminutes of weekly meetings, and inpost-analysis inter views conductedwith each team member. In addition,narrative thematic analysis [4] wasundertaken on a selection of TELresearch linked to the UK Teachingand Learning Research Programmethat gave rise to this report. Wherepossible, project impact reports wereexamined, in order to explore themeasures of impact reported by each project.

During analysis, it was found thatpublicly available documentation

often lacked sufficient informationabout non-academic projectimplementation and about the impact and exploitation of the workduring the lifetime of the project and after its formal conclusion. Insome cases, it was difficult to findlinks to final or impact reportsproduced by funded projects. Deskresearch therefore highlighted theneed for more information aboutthe actual, and often challenging,practice and experience ofimplementing research and achievingimpact. It also drew attention to the lack of clarity demonstrated by many projects about plans forexploitation and, specifically, thereasoning and evidence for claimssuch as expecting ‘large-scale use of project results’. This initial findinghas been taken into account in thisreport’s recommendations related to evidence.

The theor y development phase of the analysis initially focused oncreating a broad reporting narrativethrough which to present thethemes identified. It then went on todevelop a model for the innovationprocess. As with earlier phases,existing theor y and concepts were considered together with thethemes emerging from the research.The breadth of the study’s scopemeant that theoretical saturationwas not expected across categories.Nevertheless, a high degree ofconfluence was obser ved acrossinter views, indicating thatappropriate emergent themes had been identified. These themes,which are explored in Section 6,were the TEL Complex, persistentintent, bricolage and evidence.●

Project website

Full details of the Beyond Protoypes study are available on the project website http://beyondprototypes.com/, including the methodologies for data collection and analysis, an overview of case studies considered, and some in-depth case studies

12 Beyond Prototypes: Enabling innovation in technology-enhanced learning

3 WHAT IS TEL?

Technology-enhanced learning(TEL) research focuses on howtechnologies can add value tolearning and teaching processes.Today’s learners have access toincreasingly powerful and affordablehandheld computing devices,including smartphones, gamesconsoles and tablet computers. They can share, interact and immersethemselves online with othersthrough the use of social networksand virtual worlds. They can alsocreate identities and user-generatedresources that potentially have avirtual worldwide audience enabledby the Internet. Learners’ activitiescan be captured in real time andfeedback processes automatedwith increasing precision throughlearning analytics. Technologies thatallow users to post material andmessages online have the potentialto support learner inquiry, to offernew modes of representation andexpression requiring new formsof literacy, to support innovativethinking and problem solvingthrough collaboration, and to allowpublication of work to an authenticexternal audience [5].

TEL is able to make use of differentforms and formats of technology inthe pursuit of more engaging andbeneficial forms of teaching, learning,pedagogy and assessment. As thisreport highlights, good pedagogicintentions lie behind some of thisdevelopment but, ‘many importantTEL developments have often comefrom innovating with technologiesdeveloped for other purposes’ [6].

Technology-enhanced learning hasemerged as a preferred term of

reference for the research communityworking in this area. The term ismore generous and encompassingof new practices than the widerange of related labels, including‘educational technology’, ‘computer-aided learning’, ‘Information andCommunication Technologies’ (or‘ICT’, as they are often referred to inthe schools sector), and ‘e-learning’,to name but a few. ‘Technology-enhanced learning’ stresses that thetechnology is employed in service of the learning, and that it is not just adopted, but is expected todeliver improvement.

References to TEL, in relation tosupport and training, began toemerge in the 1990s and the firstTEL conference appears to havetaken place at the end of thatdecade. TEL-isphere 1999, held inBarbados, focused ‘on the use ofcommunications and informationtechnologies, and their potentialto enhance learning experiencesthrough helping students becomeactive participants in the educationalprocess’ [7].

Even before the emergence ofTEL as a named field of research,UK government policy was drivinginnovation in this area. The Beyond Prototypes website includes anoverview of relevant policies in theUK over the past decades, and aselection of those from Scotland,which has developed distinctivepolicies of its own. The website alsocontains a detailed version of theboxed case study included here,which deals with the implementationof the Microelectronics EducationProject, providing an early example

Section 3 defines technology-enhanced learning, tracing its origins, identifying its main areas of focus, linking it to related fields and identifying key past policy initiatives. The section goes on to explore different conceptions of education, introducing the idea that education is a super-stable system in which successful innovation requires attention to a range of different pressures and practices.

tel.ac.uk 13

3 WHAT IS TEL?

of government policy driving a well-structured process of TEL innovation.

In 2001, the final report of theEuropean Commission’s OpenConsultation Process on ‘NewResearch Challenges for TechnologySupported Learning’ clearly set out aseries of recommendations designedto further the research agenda fortechnology-enhanced learning. Thesewere to be carried out as part of theSixth Framework Programme (FP6)[10]. The report offered a response tothe fragmented nature of intellectual,disciplinary and research communityagendas related to technology andeducation at the time [11]. A diversityof TEL research effort had resultedfrom the different cultures, traditionsand trajectories associated with thevarious national educational systems,and their highly differentiatedexperience of and ambitions for usingtechnology in support of learning.

Successive programmes fundedby the European Union (EU) haveser ved to aid the integration ofresearch and the emergence of newresearch and doctoral communitiesin relation to technolog y-enhancedlearning. These have included thePROLEARN Network of Excellencethat dealt with technolog y-enhancedprofessional learning (2004-2008)and the Kaleidoscope Network ofExcellence focused on technolog y-enhanced learning and access tocultural heritage (2004-2008). These were followed by STELLAR,the European Network of Excellencein TEL (2009-2012). The STELLARwebsite specified that the networkrepresented ‘the effort of theleading institutions and projects inEuropean Technolog y-EnhancedLearning (TEL) to unify our diversecommunity’. Each of these networksbrought together broad teams ofresearchers working in diverse fields,including computer-supportedcollaborative learning (CSCL),blended and informal learning,

Case study: Microelectronics Education Programme

The £23 million Microelectronics Education Programme (MEP) for England, Wales and Northern Ireland was established by the Government in November 1980 and ran for six years. The aim was to support schools in preparing children ‘for life in a society in which devices and systems based on microelectronics are commonplace and pervasive’. To complement this work, the Department of Industry made £16 million available to help local education authorities purchase computers for schools.

MEP took into account areas as diverse as curriculum development, teacher training, resource organisation and support. It promoted change at national, regional and local levels, encouraging collaboration and cross-fertilisation of ideas. Plans for evaluation and field studies were incorporated from the start. Although there was relatively little emphasis on pedagogy, the programme did note the potential to ‘add new and rewarding dimensions to the relationship between teacher and class or teacher and pupil’ [8].

A report by Her Majesty’s Inspectorate in 1987 found that, while the MEP was only one of the agents of change and innovation in the field, ‘work with IT in schools and associated staff development owed much directly, and even more indirectly, to the programme.’ It noted the challenges associated with attributing direct impact to the programme because there was much activity in schools around microelectronics at the time and MEP resources were often channeled indirectly to schools. However, the report found that the strategy adopted by MEP considerably strengthened the number of well-informed teachers and trainers, that those involved often showed a resilience that allowed them to overcome difficulties and uncertainties which was necessary for successful implementation, that an impressive range of materials was developed, that closer links were formed between training and curriculum development in IT, and that the cascade principle of teacher training worked well where opportunities were offered [9].

as well as authoring systems andimmersive environments.

STELLAR played an important role in unifying research agendas andsetting mid-term challenges forthe research community, whilerecognising the complexity within and between the educationalsystems of partner countries [12].TEL research communities nowseek to adopt more collaborativeand inherently interdisciplinaryapproaches that bring togethereducational, learning and computersciences, as well as related disciplines.

Despite these moves towards unity,there still exist many closely relatedand overlapping research areas thatwould not describe themselves asprimarily TEL communities. Theseinclude, for example, those focusedon educational data mining, artificialintelligence in education, networkedlearning and learning analytics, as well as those identified in theparagraph above.

Within the UK, the work of the ESRCTechnology Enhanced LearningResearch Programme echoed thatof the European Commission by

14 Beyond Prototypes: Enabling innovation in technology-enhanced learning

3

promoting unity within the researchfield. The initiative represented asubstantial programme of work that was funded equally by two ofthe UK’s Research Councils, the ESRCand the EPSRC, from 2007 to 2012.The core aims of the TEL ResearchProgramme were to design andevaluate systems that would advanceunderstanding of learning andteaching in a technological context,and that would also improve teachingand learning practice.

In some ways, the TEL label canplace a bind on researchers. Fromthe perspective of practitionersand policy makers, ‘technology-enhanced learning’ captures the needto realise more from the potentialof technology to assist learningprocesses. However, the term canalso obscure the need for teachersupport and development that isoften required to ensure positiveimpact in the classroom or othereducational context.

TEL is not limited to technology-enhanced education; the world oflearning goes on outside the formallearning settings of the classroomand the lecture hall. Lifelong learnersalso engage in non-formal learning,including vocational and workplacetraining, which is not accredited bya traditional academic institution.At other times, they will be involvedin informal learning, in settingswhere they choose their ownmethods, define their own goals,or work towards shifting goals [13].As the case studies in this reportdemonstrate, arenas for informallearning, non-formal learning, lifelong

and professional learning are verymuch part of the TEL researchagenda. This engagement withdifferent forms of learning adds layers of complexity to an alreadybusy research arena.

While members of the TEL researchcommunity have sought to defineand reach agreement on the keyquestions and the ‘Grand Challenges’that can drive innovation ineducational and learning systems, it isworth considering the nature of thechallenge faced by TEL. Everyone hasexperienced education and will havea view about the role of technologyin supporting that experience.While there may be openness tothe integration of technology intoother aspects of everyday life, thereis a variable but understandableresistance to innovations that tamperwith the dominant educational ortraining practices, unless a particularinnovation is occupying a new niche.

3.1 Understanding educationIn reflecting upon the general themeof moving beyond prototypes, it isimportant to understand differentways in which the role of educationmay be characterised.

Education as a service: In someinstances, education has beenconsidered as a service, in the sensethat learning resources are providedfor learners and teachers. The BBCBitesize and OpenLearn websites are among the current examples of good quality educationalresources available online, many of which are available free of charge.Due to the wealth of resourcesavailable, learners and teachersneed to be strategic in determiningwhich resources to access and howto use them to build on currentunderstandings. Understandingeducation as a ser vice promptsa focus on innovation relating tobrand (the distinctive sets of values

Arenas for informal learning, non-formal learning, lifelong and professional learning are very much part of the TEL research agenda.

tel.ac.uk 15

You look at the [US] educational establishment, and ifthere’s any change it’s very slow. I don’t think the educationalestablishment has really embraced these ideas. And, to theextent they embrace them, I think a lot of times it’s surfacerhetoric and the reality underneath hasn’t changed. Businessleaders will say, ‘We need a different type of workforce in thefuture. We need different types of learners. We need peoplewho are creative and collaborative.’ But then you see whatthey oftentimes recommend for the schools, as just a smallincremental change from the way school has always been. So the rhetoric of what people say is needed doesn’t getmatched with what they really call for in schools.

Mitch ResnickHead of the Lifelong Kindergarten group at the MIT Media Lab

3

associated with an organisation),channel (the means for deliveringresources to learners and teachers),product system (the overall bundleof ser vices) and ser vice (interactionsproviding value for users) [14].These product-focused typesof innovation can influence howefficiently teaching is delivered, andthe quality of the resources availableto learners, but they are concernedwith only limited areas of teachingand learning and do not require animproved pedagogic approach.

Education as media production and presentation: Some Massive OpenOnline Courses (MOOCs) presenteducation as a process of mediaproduction and presentation. Thesecourses, sometimes referred to as‘xMOOCs’ or ‘behaviourist MOOCs’,are distinctive in that they are focusedon interaction with content. Theyhave a traditional linear structure,with targeted activities punctuatingthe acquisition of new knowledge[15]. High enrolment rates on thesecourses show that this approachcan be very successful in attractingpotential learners but, at the sametime, high drop-out rates point to thedisadvantages of treating educationsimply as a process of productionand presentation. Such MOOCs mayadopt an innovative profit model, aninnovative structure, an innovativeprocess or an innovative channelfor course delivery [14]. Once again,although innovation in these settingsmay be significant, it does not requirean improved pedagogic approachor imply an increase in educationalimpact on individual learners.

Education as a conversation:Theorists from Dewey to Laurillardsee education as a conversationduring which we adjust to eachother’s perspectives [16, 17].Learning is a process of comingto know through meaning makingand conversation. Online toolsallow extensive and extended

conversations about learning totake place both asynchronouslyand synchronously, but concernsabout focus, privacy and e-safety,as well as the logistics involved inproviding the necessary technologicalinfrastructure and support, all workagainst adoption of the use ofthese tools in the classroom. Thisunderstanding of education opensthe way to significant TEL innovation,focused on changes in practice andpedagogy that, enabled by the use oftechnology, can produce measurableeducational impact.

The three paragraphs above set outdifferent ways in which education can be understood. The first twofocus on resources and deliver y and are associated with processes of innovation and development that are already well understood[14]. The third, education as aconversation, is the one that ismost reliant on the process of TELinnovation rather than on innovationin areas familiar from business,such as brand and profit model. Inorder to engage successfully in TELinnovation, it is important to look at education not only as a processbut also as a system.

3.2 Education as a super-stable systemA system typically combines a set ofinterdependent components. In theeducational system, innovators whoare aiming for educational impactnot only have to consider aspects ofteaching and learning, but also howthe change will interact with otheraspects of the system’s operation.

Unless an innovation is well alignedwith the system into which it isintroduced, the chances are that it willbe resisted or ignored by the system.Unless the necessary developmentand support systems are available,teachers will rarely have the time orthe inclination to test and adopt newpractices. This means that educationis a super-stable system, and theclassroom is a challenging spacewithin which to innovate. In order tounderstand these challenges, whichare inherent in the TEL innovationprocess, it is necessary to understandwhy education, as a super-stablesystem, is resistant to change.

An important theme in organisationtheory research concernsexplanations of stability andhomogeneity in organisational

16 Beyond Prototypes: Enabling innovation in technology-enhanced learning

configurations and practices. A strongand overlapping set of explanationscome from open systems theory [18]and from institutional theory [19].Both emphasise the role of multipleoverlapping and mutually reinforcingsystemic pressures in maintainingstability in organisational practices.These pressures also play a role inbringing about homogeneity betweenpractices in organisations facingsimilar environment pressures.

Institutional theory emphasisesthe role of three forms of pressurein determining and legitimatingparticular practices. These pressuresare coercive (laws and regulation),normative (social values andexpectations), and cultural-cognitivemindset (culturally determinedand reinforced understandings ofthe world). Where organisationalpractices are strongly institutionalised,in that they are underpinned bysuch multiple reinforcing sets ofdeterminants, they can be very hardto disturb. For example, researchshows that management innovationsin a parent company may be hardto transmit to subsidiaries in adifferent country where practices areunderpinned by different institutionalpressures [20, 21].

Other research suggests thatchanging a single system within anorganisation often fails to produceintended changes without changesto associated systems. For example,a large study of middle managers inprivate sector firms showed trainingin new management approachesto be ineffective in bringing aboutdesired change without attention tothe role of pay systems, promotionsystems, allocation of resources andsymbolic communication from seniormanagers in reinforcing existingbehaviours [22].

Education systems are, in the senseabove, strongly institutionalised.Educational practices are

3

underpinned and reinforced bymultiple overlapping social forcesand intertwined with other mutuallyreinforcing practices. Coercivepressures include not only centralgovernment legislation and regulationbut also national examination systemsand the bodies that control these.Normative pressures include publicand political expectations that aremediated and amplified by the media, the role of professional bodies and employer organisations;cultural cognitive mindsets arereinforced and transmitted by thestrong role of apprenticeship inteacher development.

Within education systems, differentpractices play a mutually reinforcing

role. Pedagogic practices interactwith and are constrained bytimetabling practices, budgetingpractices, safeguarding practices,inspection practices, data capturepractices, governance practices,assessment practices and manyothers. Initiatives that seek to changejust one practice component areunlikely to achieve traction unlessattention is paid to other elements of the system.

Ball and colleagues [23]documented the difficulties ofpolicy implementation in schools.Their findings mirror some ofthe conclusions presented in thisreport. They found that policies aretranslated, interpreted and absorbedinto existing ways of doing things,often being markedly changed or de-natured in the process. As one of thedeputy heads they interviewed stated,‘I think we know what we want todo with our school, we know exactly

what is needed [...] and we’re takingthe school in that direction. Policycomes at us and we’ll sort of harnessit to continue going in that direction’[23, p.51].

Accounts of successful changewithin such institutionalised systemsemphasise the importance of bridgingand brokering across organisationalboundaries and understanding thatpractices have to be recreated innew contexts to function withinthe ecology of practices into whichthey are being introduced [21, 24,25]. Section 4 provides an overviewof some of the significant successesachieved by TEL, demonstratingsome of the ways in which boundarycrossing takes place.●

Changing a single system within an organisation often fails to produce intended changes without changes to associated systems.

tel.ac.uk 17

The UK is currently a world leaderin the area of TEL. BETT – formerlyknown as the British EducationalTraining and Technology Show –has been running since 1985 and isthe largest educational technologyevent in the world, bringing togetherover 35,000 educators and learningprofessionals. A recent list of the top20 e-learning companies in Europein terms of their innovation, scale,market impact and revenue growthincluded 10 UK firms [26].

Successful TEL interventions havebeen identified and catalogued indifferent reports [6, 27]. In System Upgrade: Realising the Vision for UK Education, Noss highlighted onerecent set of examples:

The Technology Enhanced Learning research programme has spent more than four years developing systems and software that, for example, use artificial intelligence to teach teenagers algebra and help autistic children with their social skills. We have created virtual islands where young people acquire the confidence to tackle some of life’s bigger challenges. We have exploited the potential of giant touch-screen tables to encourage young children to work together. We have taken sense-of-touch technology – the sort that makes that gaming controllers vibrate – and used it to train dentists cheaply and effectively [6]

International reviews suggest thatTEL interventions lead to outcomes

that have impact on a similar scale tothat produced by other educationalinterventions [see, for example, 28, 29, 30]. Inter ventions that haveclaimed a larger impact, such asCognitive Tutors (see boxed casestudy), have taken decades to develop for a limited range ofcurriculum topics. As Borgman and her colleagues comment,

New technologies follow complex trajectories often supported or thwarted by other technologies, infrastructural issues, competing standards, social systems, political decisions, and customer demands [31, p17]

The extended period of developmentthat precedes successfulimplementation means that the scaleof TEL success may go unnoticedby observers who are expecting fastresults or by those who are lookingfor a new product or procedure andthus do not notice the emergenceof an entire field, such as mobilelearning, from TEL research. It maygo unnoticed by observers whoexpect the results of TEL researchto be confined to the field ofeducation and so do not make theconnection between TEL innovationand life-changing developmentssuch as the World Wide Web. Itmay also go unnoticed by thosewho are expecting a linear modelthat proceeds from idea to pilot tofull-scale roll out. The ecologicalmodel of TEL points to the ways thatdifferent components combine andintermingle; one vision diffuses andinspires others.

Section 4 provides an overview of TEL successes, pointing to the UK’s role as a world leader in this area and identifying TEL’s role in developments as diverse as World Wide Web and the iPad. Three areas of success – the field of mobile learning, the development of the Scratch educational programming language, and the xDelia project that developed learning applications for financial traders – are considered in detail. The section also identifies reasons why TEL successes may go unnoticed, including the significant timescales involved.

4 ACHIEVEMENTS UNDERPINNED BY TEL

18 Beyond Prototypes: Enabling innovation in technology-enhanced learning

UK centres of excellence in TEL

UK centres of excellence in TEL research have been prominent in national, European and international networks. The largest assembly of TEL researchers in the UK is at The Open University, with some 30 academics in the Institute of Educational Technology and a similar number of people with a core research interest in TEL spread across the Knowledge Media Institute and university faculties.

The London Knowledge Lab, a partnership between the Institute of Education and Birkbeck, has around 50 academics engaged with research into digital technologies, new media and knowledge. At the University of Nottingham, the Learning Sciences Research Institute is a centre of excellence for research in the learning sciences and technological innovation, engaging 12 core academics. These three institutes have formed CTEL – a collaboration aimed at sharing research and innovation. This includes running a series of ‘What the research says…’ events for industry, practitioners and policy makers.

Other centres of research excellence include the Centre for Innovation in Technologies and Education at Southampton; the Centre for Learning, Knowing and Interactive Technologies at Bristol; the Serious Games Institute at Coventry; the Learning and New Technologies Research Group at Oxford; the Educational Technology Research Group at Warwick; the Centre for Studies in Advanced Learning Technology at Lancaster; the Children and Technology Lab at Sussex; the Centre for Applied Research in Educational Technologies at Cambridge; the Institute of Learning Innovation at Leicester; the Caledonian Academy research centre at Glasgow Caledonian, and the Technology Enhanced Learning Group at Durham.

A recent national initiative has been the formation of the Future Learning Academic Network. The network brings together academics from Birmingham, Edinburgh, Exeter, Glasgow, Leeds, Loughborough, Nottingham, The Open University, Reading, Sheffield, Southampton, Strathclyde and the University of East Anglia, all of whom are engaged with research related to FutureLearn and innovations in learning with technology.

Educational technology has been theinspiration or catalyst for many otheractivities. TEL is an endeavour thatattracts people from many differentfields. The theme of ‘enhancingeducation through technology’has captured the imagination andefforts of innovators in computing,technology and psychology. This canbe seen as far back at the 1950s, whenSkinner introduced the concept of ‘teaching machines’ [32], and the1960s, when Engelbart developed

a conceptual framework foraugmenting the human intellect [33].

Researchers working to developtechnologies for educationalenhancement have generally foundthat creating successful educationaltechnology is harder than they hadanticipated, that it takes longer thanthey had planned, and that the routefrom vision to implementation is morecircuitous than they had expected.The products of their innovation

sometimes prove easier to marketoutside formal education. Thishas been true of Alan Kay, whosework on the handheld Dynabooklearning device in the 1970s ledto the development of the iPad[34]; Nicholas Negroponte, whoseOne Laptop Per Child project alsodeveloped technology for widerconsumer use [35]; Seymour Papert,whose work on constructionistlearning led to collaboration withLEGO in creating the Mindstormsrobot-building kits and to millionsof children worldwide learning toprogram computers [36]; and TimBerners Lee, whose work on managingknowledge led directly to the creationof the World Wide Web [37].

The Web began as a project toprovide physicists at CERN withfacilities that could support learningby information sharing and dataexchange. These physicists formeda widely dispersed and computer-literate group, using differentcomputers. Tim Berners Lee wrotea simple hypertext program calledEnquire to keep track of people andprograms; enabling mail and fileinterchange between different typesof computer system and differentnetworks. Developing the Enquirecode led him to something muchlarger, ‘a vision encompassing thedecentralized, organic growth ofideas, technology and society’ [37:1].This was not a smooth progression.

The Web arose as the answer to an open challenge, through the swirling together of influences, ideas, and realizations from many sides, until, by the wondrous offices of the human mind, a new concept jelled. It was a process of accretion, not the linear solving of one well-defined problem after another. [37:3]

TEL research has led to thedevelopment of major innovationsthat have taken root across the world.

4

tel.ac.uk 19

It has also had important successesboth within and beyond the field of education, ranging from broadareas of development to morefocused achievements. The followingsections consider three ver ydifferent examples of TEL success. In each case, innovation is anextended process, in which anunderstanding of the ecologies andcommunities within which innovationwill be embedded is developed overtime. Although these innovationsrequire the use of technologies, they are not dependent upon asingle technolog y, the equipmentthat is employed changes over time. This is particularly clear with the first example, mobile learning.

4.1 Success story: mobile learningThe development of mobile learningcan be traced back to its origins inTEL research.

In the earlier half of the last decade, sophisticated mobile technology was scarce, fragile, expensive and difficult, and was the prerogative of institutions. In the second half of the decade, mobile technology became universal, robust, cheap, diverse and easy. [38]

Researchers began to studythe potential of mobile devicesto enhance fieldwork, allowingstudents to learn when and wherethey needed to do so [39]. Thefindings of small-scale studies ledto a vision of mobile devices ableto support lifelong learning, devicesthat would be portable, availableanywhere, adaptable and intuitiveto use [40]. Since then, mobilephones have become smart phones,and generations of personal digitalassistants (PDAs) and other mobiletechnologies have come and gonebut researchers have continuedto work towards that vision ofeducational innovation.

In the early 21st century, with cheapand robust mobile technologies widely available, the European Unionbegan to fund major multinationalpartnership projects, includingMOBIlearn and M-learning. Thissupported the development of mobile learning from a small-scaleresearch interest to an internationalphenomenon [41]. More than 26million people in Bangladesh have now accessed the BBC Janala language learning ser vice on mobile phones [42]. Alongsidedevelopments in Europe and Asia,the USA has also awoken to thepossibilities of mobile learning, and the spread of mobile devices.

Following mobile technolog yresearch in education at thebeginning of the centur y, such as the Palm Pioneers project [43], therehas been a stream of innovation in the US around 1:1 learning withhandheld devices in classrooms and lecture theatres.

Sheer weight of numbers providessome indication of the success ofthese innovations, but it is difficultto use scientific methods developedfor laborator y use to assess a shift in learning behaviour on this scale.It is clear that people are benefitingfrom opportunities to learn incontexts that were never possible in the past, but there is stillmuch work to be done to enableconsistent educational impact.While some schools have adoptedinnovative approaches to developingmobile learning between homeand school: there are schools thatbuy the latest tablet computer orhandheld gadget and only thenconsider how to use it.

Based on his experience of mobilelearning projects, Steve Vosloo, seniorproject officer in mobile learning atUNESCO, highlights the challenge ofassessing the new skills that youngpeople are learning through TEL:

There aren’t really recognized measures for these. What you find doesn’t fit neatly into the standard assessment system. So actually conducting the assessment, when the standard formal assessment does not necessarily recognise all of the learning that’s taking place, was definitely a challenge. That’s probably something that comes up in TEL quite a lot. [Steve Vosloo]

Providing evidence-based researchthat can support learners andteachers to make informed decisionsabout engagement with mobilelearning is a continuing challenge.

4.2 Success story: ScratchA different form of TEL success isrepresented by Scratch. This is avisual programming environmentfor children to create and shareinteractive stories, games andanimations and to think creatively,reason systematically, and workcollaboratively [44]. Scratch is aproject of the Lifelong KindergartenGroup at the MIT Media Lab, headedby Mitch Resnick, and has receivedfunding from the US governmentNational Science Foundation, privatecompanies and foundations. Firstlaunched in 2007, and provided freeof charge, approximately 4 millionScratch projects have already beenuploaded and shared by users. A spin-off online community for educatorscalled ScratchEd was launched in July

4

There are schools that buy the latest tablet computer or handheld gadget and only then consider how to use it.

20 Beyond Prototypes: Enabling innovation in technology-enhanced learning

4

2009, highlighting the value of theprogramming environment for formalas well as informal learning.

The project had its origins in morethan 40 years of development atMIT of research on children learningthrough programming. This wasstrongly influenced by SeymourPapert’s work from the late 1960sonwards, including teaching childrento control a programmable ‘turtle’using the Logo programminglanguage [45, 46]. Papert’s research,in turn, drew on the work publishedby developmental psychologist JeanPiaget from the 1930s to the 1970s.

In working with Seymour Papert and being involved with the Logo community, I saw a lot of what drove it and what limited it. I saw both the strengths and the weaknesses of the Logo Community. We went on to start our own network of after-school centres, the Computer ClubHouses. The initial inspiration for Scratch came from our work at ClubHouses, where we saw what kids were looking for and realised that there were no good tools available. [Mitch Resnick]

Scratch emerged not only asa consequence of researchers’educational vision and experiencebut also as a consequence of thesustained engagement the team hadwith young learners through theClubHouses. The first, and flagship,Intel Computer ClubHouse in theNetwork was opened in collaborationwith the MIT Media Lab in 1993 at theComputer Museum and later moved

to the Museum of Science, Boston.Sponsors have since included Adobe,Autodesk, Hewlett-Packard and LEGOsystems and today there are over 200ClubHouses around the world.

This long history of developmentand of community building overa period of more than 40 years,provided the inspiration and basisfor the development of Scratch. Theinfluence of this work is still increasingand is becoming increasingly evidentin Europe. In the UK, governmentpolicy now places increasedemphasis on programming withinthe curriculum. This is linked with agrowth in after-school programmingclubs and in teacher training. Theorganisation Code Club, for example,has over 1200 clubs nationwide. Bothin school and out of school, Scratchis used as a gateway to programming.Elsewhere in Europe, there issimilar interest in programmingand computing environments. Forexample, the Portuguese governmenthas developed a partnershipbetween the government, a highereducation institution, and internetprovider SAPO that seeks to providecontinuing professional developmentand that has a focus on Scratch.

In 2013, the first European Scratchconference was held in Spain and‘Scratch Day’ was marked by 184events in 47 countries.

Scratch is an example of an innovationthat has succeeded by runningalongside the education system in its use in informal settings outside theclassroom and in the developmentof a network for sharing the creative

products produced by children. Itsinnovative features come from theconfiguration of software and socialnetworking elements. Like muchTEL innovation, it has involved theassembly of existing skills, ideas andresources in a new and productiveway. It has also involved extensivework in building and engaging with the communities involved.

4.3 Success story: xDeliaExtended work with stakeholdercommunities was key to the successof xDelia (www.xDelia.org). This wasa three-year project that combinedresearch into the decision-making,learning and trading practices ofprofessional and private financialtraders with the development oflearning applications. The projectexplored the role of emotions andemotion regulation in financialdecision-making and how learninginter ventions that focused onimproved emotion regulation mightimprove financial decision-making. Inits later stages, the project evaluateda series of learning inter ventionsthat exploited a combinationof physiological sensors andserious games approaches. Theseinterventions were combined in anoverall pedagogic approach foundedon an understanding of the role ofemotions in trading, practice-basedapproaches to learning and a closeunderstanding of traders’ trading and learning practices.

The project was funded by the EU as part of the SeventhFramework Programme (FP7) andwas carried out by a consortiumof academic institutions and acommercial partner. The initial stage of the project took the formof in-depth studies of the behaviourof traders and the ways in whichtheir emotional state influencestheir decision making. The projectfocused on a particular emotion-driven trading bias as a proof of

The initial inspiration for Scratch came from our work at ClubHouses, where we saw what kids were looking for and realised that there were no good tools available.

tel.ac.uk 21

4

concept: the tendency to hold loss-making assets longer than assets at a gain (the disposition effect).

Novel use was made of physiologicalsensors to provide feedback onemotion regulation capabilities,and games were developed thatmimic important aspects of tradingand financial decision making. Theproject successfully demonstratedlinks between engagement in thelearning inter ventions and real-world behaviour; it also built onwork demonstrating links betweentraders’ effective emotion regulationand performance. Outputs from thiswork are being exploited in severaldifferent contexts and Saxo Bank ’sonline investor education platformfor its clients has developed anapproach to developing clients’understanding of the role ofemotions in their trading based on outputs from the project.

In comparison with the other examplesdiscussed in this section, xDelia is a good example of a time-boundedproject that, in three years, was ableto move from initial concept to a fullyembedded concept with demonstrableeducational impact. This process,though, was based on extendedengagement with the communitiesinvolved. Jeffrey Lins, head ofadvanced research and innovation atSaxo Bank, was a project partner. Fromhis perspective, the establishmentand success of the consortium wererooted in earlier work

The academic lead on xDelia has a remarkable understanding about how it works inside investment banks, not only because he’s studied it academically but, having interacted with these kind of organisations, he understands them [Jeffrey Lins]

He also attributed the success ofthe project to exploratory studiesand to reviews of previous work

that were carried out at the start.These provided the consortiumwith a detailed understanding of theenvironments in which their workwould be implemented.

The work also extends forward,beyond the period of projectfunding. Early work has now begun,in collaboration with commercialorganisations, to explore the potentialof the approaches developed by xDeliain other fast-paced environmentssuch as training racing car drivers andair traffic control. A vision of usinglearning interventions to improveemotion regulation in high-pressuresituations links this work over time. Aswith the other examples in this section,the core of the work is not a singletechnology, but a willingness to makeuse of technology to achieve a visionof enhancing learning.

xDelia can therefore be understoodas one element of a large body ofwork that extends over time. However,it was also successful in delivering asignificant innovation in a relativelyshort time frame. Key features of theproject enabled this success.• Research team members had a

close understanding of the ecologyof practices in the areas where theinnovation would be embedded

• There was close collaboration with acommercial partner from the outset,which played a significant role in theresearch and development process.

• Explorator y studies investigatedthe nature of existing informal and formal learning practices inthese environments.

• The target audience was involved in early studies and trials.

• It engaged a highly cross-disciplinary team, with expertise

in TEL, sensors, the psychologyof decision-making and emotionregulation, behavioural economics,neuroeconomics, and financialtrading and markets.

• There was a strong commitmentto dissemination by engagementrather than by broadcast.

These features are shared with theother TEL successes considered in this section. A close attention to theinnovation process by consortiummembers, and an understanding of theimportance of previous developmentsand cross-community engagement,enabled the xDelia team to workthrough the innovation processconsciously and relatively quickly. InSection 6, these features are broughttogether in a model of the TELInnovation Process. This process alsotakes into account the misconceptionsand challenges that may beencountered by researchers. These are considered in Section 5, togetherwith ways of addressing them.●

xDelia was able to move from initial concept to a fully embedded concept with demonstrable educational impact.

22 Beyond Prototypes: Enabling innovation in technology-enhanced learning

5 TEL INNOVATION: CHALLENGES AND MISCONCEPTIONS

The TEL research community hashad undoubted success in extendingthe vision and reach of innovationsthat have reached millions of peopleworldwide, notably mobile learning[47, 48] and MOOCs [49]. TEL hasprovided theoretical underpinningfor technologies that are used tosupport learning, and policy briefingsadvising on how these technologiescan be used to achieve educationalimpact. This work has included criticalappraisals of tools that are widelyused in schools and universities,including integrated learning systems[50], interactive whiteboards [51]and virtual learning environments[52]. The enduring success of TheOpen University is closely allied withresearch into innovations in TEL[53, 54]; this research drives thecontinued adoption of new methodsof distance learning and assessment.

Despite its achievements, the TELresearch community has neitherthe coherence nor the scale ofother scientific communities suchas particle physicists or climatescientists. For example, the WorldClimate Research Programme‘organizes large-scale observationaland modelling projects and providesthe international forum to alignefforts of thousands of climatescientists to provide the best possibleclimate information’ [55]. By contrast,the STELLAR European Network ofExcellence in TEL [11], funded by theEuropean Commission, integrated15 leading research organisations inTEL between 2009 and 2012. It wassuccessful in coordinating research,informing governments of TELinnovations, and supporting initiativessuch as the European Conference inTechnology Enhanced Learning, but

it would not claim to have alignedthe efforts of thousands of learningscientists.

Grand challenges of TEL such as‘Make use and sense of data toimprove teaching and learning’[12] have the potential for moreimmediate social impact than thehunt for the Higgs Boson, buthave never captured the publicimagination. Why is this? One reasonis the complexity of TEL, whichwill be examined in Section 6. Themost straightforward approach toinnovation is to focus on marketinga technology, rather than on thecomplexities involved in using thattechnology to achieve significanteducational impact. However, thetechnology by itself is not theinnovation, the importance of thetechnology lies in the ways in which it can enhance learning by supportingor transforming a particularpedagogy or practice. Without theseunderpinning elements, it is likely tobe reduced to an expensive way ofdoing something that was done morecheaply in the past.

To give one example, the originalconception of interactive whiteboardsas a TEL innovation brought together acomplex of technology and practices.These included practices related topedagog y, classroom organisation,teacher-pupil relationships, staffdevelopment and technical support.Changes to all these practices wererequired in order to realise the boards’potential for fostering interaction,creativity and collaboration. However,a study of their use in schoolsnoted that ‘the tools of educationaltechnology have no magical power in themselves; only by being embedded

Section 5 deals with challenges to TEL research and innovation. Six common misconceptions are examined and recommendations are identified that will increase the potential of TEL to achieve widespread impact. These show that funders, researchers and policy makers all have a role to play in achieving impact.

tel.ac.uk 23

5 TEL INNOVATION: CHALLENGES AND MISCONCEPTIONS

in the practices of teachers andlearners do their mediational meanscome into play’ [51].

Without training and technicalsupport, sometimes without reliablenetwork connection, careful set-up,or budget for replacement parts andrepairs, it is difficult for teachers touse interactive whiteboards in waysthat support learner engagement andinteraction. Nevertheless, the boardshave often been purchased withoutattention to these practices. It hasproved easier to present the boardspurely as an innovative technology thatcan be bought, installed and used. Thisapproach has reduced opportunitiesfor significant pedagogical impactbecause it treats the new technologyas a direct substitution for theprevious blackboard or whiteboard,and pays little or no attention to thesupport and resources required inorder for teachers and learners togain extra benefits.

In such cases, the failure to achieveeducational impact is associatedwith a failure to recognise that TELinnovation consists of a processof implementation rather than atechnology. The sub-sections belowset out a series of misconceptionsthat can limit the success of TELinnovation, and identify ways ofaddressing these and thus improvingthe potential of the TEL researchcommunity to achieve widespreadpublic or commercial impact.

5.1 Shift the focus from technology to pedagogy and practice

MISCONCEPTION: The technologyis the innovation. Innovation thereforefollows a linear path from idea toprototype, deployment and evaluation.

As set out above, the temptation,particularly in time-bounded, grant-funded projects, is to focus on a

technological innovation. A visibletechnology such as One Laptop PerChild (OLPC) has an obvious appealfor governments and media, but itsdevelopment has been criticised fordivorcing the provision of technologyfrom its content, training and use [56].

Technology should not be the primary driver of educational activity; it should support it. Technology such as the XO-1 [the computer model developed for the OLPC project] should only ever be considered as supportive of educational practice, never as core to it. [56:244]

Starting with an educational challengeis more likely to produce successfuleducational transformation thanstarting with a technology. A reviewof the use of interactive whiteboardsin schools in 2011 concluded that

there is a need to reassess the use of computer technology from an educational, rather than a technological, perspective; and develop a more sophisticated conceptual model of how ICT can facilitate teaching and learning in the classroom. [51:362]

While the new ‘app economy’ thatmarkets software applications couldtheoretically produce educationalsoftware that can be transported intoclassrooms and shown to improvelearning outcomes, this is unlikelygiven the past failure of individualtechnological resources or tools tohave a major influence on education.We need to look beyond the linearmodel of TEL innovation, in orderto see how new technology-enabledmethods of teaching, learning, and

assessment could have a sustainedeffect on the practice of education.Learning through social networking,the use of mobile devices to supportlifelong learning, the use of analyticsto improve learning design – allthese visions of the enhancement oflearning by technology involve thecreation and implementation of newsystems rather than specific pieces of software.

RECOMMENDATION: Policyand funding should support andencourage changes in pedagogy andpractice, as well as the technologicaldevelopments that will support these.

5.2 Look beyond the formal education sector

MISCONCEPTION: TEL innovationshould be focused on formaleducation.

Historically, government policyrelated to TEL has focused onformal education. In the 1980s,the Microelectronics EducationProgramme (see the boxed casestudy in Section 3) aimed to ensurethat school leavers would be familiarwith computers and their potentialapplications. In the 1990s, theTeaching and Learning TechnologyProgramme provided impetus foradoption of TEL across the universitysector. In 2005, the Departmentfor Education and Skills report on‘Harnessing technology: transforminglearning and children’s ser vices’included references to lifelonglearning electronic portfolios andoccupational training, but focusedmost of its attention on the formalsector [57]. In the same decade,the ‘Strateg y for e-learning’ and

Technology should not be the primary driver of educational activity; it should support it.

24 Beyond Prototypes: Enabling innovation in technology-enhanced learning

its successor ‘Enhancing learningand teaching through the use oftechnology’ published by the HigherEducation Council for England(HEFCE) focused, as would beexpected from HEFCE, on the highereducation sector [58, 59].

Attention to informal learning has notbeen entirely absent. The ComputerLiteracy Project of the 1980spromoted public understandingof microelectronics technologiesthrough the medium of television.Earlier this year, the Departmentfor Business Innovation and Skillspublished a research paper on‘The maturing of the MOOC’ thatmade recommendations regardingfuture research [60]. Nevertheless,most policy in this area has beenconcerned with the teaching andlearning that takes place in schools,colleges and universities.

A related issue is that access to TELresearch publications remains limitedfor those outside the universitysector. Researchers working inbusiness and industry have restrictedaccess to research findings, and aretherefore hampered in their abilityto engage in evidence-informedinnovation. While working on theBeyond Prototypes research study,a team member who works in thecommercial sector was unable toaccess all the literature cited inthe bibliography that appears at

5

the end of the report. Even someteam members who had access touniversity library facilities foundit difficult to locate some end-of-project reports.

During the past decade, the informallearning sector has emergedas an important area for large-scale commercial TEL innovation.Individuals are adopting openeducational resources and mobilelearning software on a massive scale.To take the example of just oneprovider, from 2008 to 2013 therewere over 64 million downloads byaround 9 million unique visitors of the open educational resourcesshared on iTunes by The OpenUniversity. Worldwide, there isan increasing demand for lifelonglearning, for staff development andfor on-the-job training opportunities.

Entrepreneurial universities, offeringnew methods of informal learning ona worldwide scale, have fuelled therecent explosive growth of MOOCs.In 2010, 200 students enrolled onStanford’s Introduction to ArtificialIntelligence course. The followingyear, when it was made freelyavailable online, 160,000 studentsfrom 190 countries enrolled [61].

Developments in the informal sectorsupport developments in the formalsector. Universities are currentlyinvestigating how they can convert

informal learners into formal learnersby providing pathways to enrollmentand to qualifications. The successof Scratch is closely associated withthe way in which socially supportedinformal learning acts as a bridge into formal learning.

RECOMMENDATIONS: In order toaddress the growing importance ofthe informal learning sector, policyand funding should support theexperimentation that is necessar y to generate fresh insights andachievable visions of educationaldevelopments. Research findings in all areas of TEL should be available to researchers both inside andoutside the university sector.

5.3 Widen the TEL community

MISCONCEPTION: Specialized areasof expertise drive TEL innovation.

Research and development in learningwith technolog y are fragmented,with separate communities forTEL, e-learning and computer-based training. Specialist researchconferences have been establishedin many areas, including Computer-assisted Language Learning, MobileLearning, Computer SupportedCollaborative Learning, NetworkedLearning, Serious Games, OpenLearning, Artificial Intelligence andEducation, and Educational Media.Despite the work of the EuropeanNetworks of Excellence in engaging a broad range of researchers, the TELcommunity is not able to speak withone strong voice.

A focus on establishing specialisedcommunities of researchers meansthat less attention is paid to buildinglinks with learners, teachers, policymakers and industry. This means it isdifficult to complete the innovationprocess because experience,expertise and visions of educationalchange are not widely shared, and

What’s quite unique about Coventry is that they have thisvery large Business Development Support Office […] What’sgood about that is that they know how to behave aroundindustry and they know how to deal with industry and theyknow how to connect things together. And that’s a hugeadvantage for Coventry because we have this very longtradition of working with industry and I think that’s reallymade a big impact.

Sara de FreitasFounder of the Serious Games Institute at Coventry University

tel.ac.uk 25

5

there are limited opportunitiesfor understanding the ecology ofpractices within which innovationswill be deployed.

Various initiatives suggest ways ofbroadening the TEL communityand making links between research,teaching and industry. The BETTeducation show is currentlydeveloping a research strand. Inaddition, it encourages the sharing ofbest practice and teaching innovationby connecting with practitioner-runTeachMeet gatherings and hostinga large-scale TeachMeet eachyear. The Association for LearningTechnology (ALT) works to improvepractice, to promote research and toinfluence policy, providing a forumfor researchers and practitionersin further and higher education.The series of ‘What ResearchHas to Say’ events organised bythe London Knowledge Lab, theUniversity of Nottingham, and TheOpen University has communicatednew developments in TEL researchdirectly to companies and policymakers. Other opportunities need tobe found to integrate the disparateresearch and practitioner groupsand the emphasis needs to be ondissemination by engagement withlearning taking place by all parties,rather on dissemination by broadcast.

RECOMMENDATIONS: In orderto widen the TEL community,researchers need to engage with theindividuals and communities that willplay a role in the implementation ofinnovations. Policy and funding shouldencourage the development of skilled,multidisciplinary teams that are able tocomplete the TEL innovation process.

5.4 Connect TEL research and practice

MISCONCEPTION: Most of the TEL innovation process takes placewithin universities.

TEL research largely operates atan elevated level and focuses onmedium- to long-term innovation. It is focused on universities, thoughthis is hardly surprising, given that the funding for UK research comesmainly through the Research Councilsand the European Commission. The focus in the forthcoming ECHorizon 2020 programme on smallcompanies may help to shift thatemphasis, but TEL is not a centraltheme in that programme.

The Mobile Learning Network(MoLeNET) programme of capitalfunding for further education (FE)institutions to embed learning withmobile technologies into FE was agood example of TEL research beingembedded directly into practice.Training practitioners to becomeTEL researchers is not the solutionbecause practitioners are alreadyfully occupied – rather there is aneed to form enduring partnershipsbetween academic TEL researchers,practitioners in schools, colleges and workplaces, and innovativee-learning companies.

In order for TEL innovations to havelong-term educational impact, theymust be embedded successfully.In some cases, they are embeddedwithin universities but, in most cases,they are deployed elsewhere. It isimportant to pay attention to andplan for this element of the process.

RECOMMENDATIONS: Policy andfunding should take into accountthe importance of this stage of theinnovation process and the need forextended development. There should

be capacity to support individuals andteams to engage in long-term projectscapable of turning inspirationalideas into fully embedded productsand practices. Researchers anddevelopers should be encouragedto plan for sustainability and toidentify the elements that must betaken into account in order to enablesustainable implementation of an ideaor prototype, in the context of a visionof the enhancement of learning. Theimplementation and success of plansfor sustainability should be evaluated.

5.5 Find new ways to assess the contribution of TEL

MISCONCEPTION: Scientificmethods developed for laboratoryresearch are the best way of evaluatingthe impact of TEL innovations.

‘No significant difference’ is anissue that has dogged TEL fromits inception. It has been difficultto demonstrate a significantpositive impact associated with theintroduction of TEL into a classroom.

A major study in the form of asecond-order meta-analysis of theimpact of technology on classroomlearning (a synthesis of the findings ofmeta-analyses, encompassing 1,055research studies) found an effect sizeof 0.35 [62]. To put this in context, itis below an effect size of 0.4, whichis the level at which the effects ofinnovation enhance achievement insuch a way that real-world differencescan be observed [62] and lowerthan the effect size of some othereducational innovations, including

MISCONCEPTION: Scientific methods developed for laboratory research are the best way of evaluating the impact of TEL innovations.

26 Beyond Prototypes: Enabling innovation in technology-enhanced learning

5

reciprocal teaching (0.74) andmastery learning methods (0.58).

The authors of the study indicatethat simply measuring the effectof introducing technology missesthe important point that it is howthe technology is used that makesa difference. For example, effectsizes from computer technologyused as ‘support for cognition’were significantly greater thanthose related to computer usefor ‘presentation of content’ [62].The implication is that it is morehelpful to examine the influenceof a combination of innovativepedagogy and technology ratherthan technology alone. The way inwhich the innovation is introduced,including management support andteacher development, is also likely toinfluence the outcomes substantially.

Evidence-based practice is crucialfor TEL because it ensures thatlearning is enhanced by technolog yin practice, as well as in theor y.However, the methods for assessingeducational impact have to becarefully chosen and appropriateto the innovation. Goldacresuggests that ‘randomised trials aregenerally the most reliable tool wehave for finding out which of twointer ventions works best’ [63]. There are certainly cases in which this method can be used reliably.The large numbers of participantsengaged in MOOCs allowcomparative testing, in whichrandomly selected groups assignedto one of two conditions and theoutcomes assessed quickly. Thisdiffers from randomized controltrials, in being part of a processof rapid testing and development.In other situations, though, it canprove impossible to alter one ortwo variables while other factorsremain constant. This is particularlytrue when TEL innovationinvolves changes to a series ofinterconnected practices.

Kirkwood and Price investigatedwhat enhancement of learningmeans in the context of TEL[64]. They note that it is difficultto attribute causality whenindependent variables are not heldconstant; the comparative studymethod is only appropriate whereother elements of teaching arereplicated. A learning enhancementthat is associated with the provisionof additional resources or tools forone group of learners may simply beattributable to extra time spent onthe task or extra teacher attention. If a study does succeed in replicatingall elements of the learningexperience other than the one beingassessed, it is not necessarily clearwhat has been enhanced. Differentevaluations of the same innovationcan give different results, and thesame results can be interpreted or presented in different ways as the Cognitive Tutor Software casestudy (see box) shows.

The comparative approach isassociated with behaviouristviews of learning, and makes theassumption that enhancement will beassociated with quantitative change– improvement in test scores – ratherthan qualitative changes that aremore difficult to measure, such as aricher or deeper understanding. Aninnovation that is tailored to meetthe requirements of a randomisedcontrolled trial may have to limit itsscope, thus reducing its impact, inorder to do this.

RECOMMENDATIONS: Thereis an urgent need to ensure thatTEL innovation is evidence basedand has demonstrable impact. Inorder to do this, policy and fundingshould require the evaluation ofTEL innovations in terms of theireducational impact. New methodsof evaluation, such as the use oflearning analytics or A/B testingwhere appropriate, should bedeveloped and put into practice.

5.6 TEL success is not necessarily commercial success

MISCONCEPTION: Effective andsustainable TEL products andpractices have commercial value and commercialisation shouldtherefore always form part of the innovation process.

The assumption that innovationshould be associated withcommercialisation was built intothe Beyond Prototypes studyfrom the start. Each example ofTEL innovation was consideredin the context of challenges tocommercialisation, assistance tomarket and relationship betweenimpact and commercial exploitation.This analysis showed that, althoughver y important in some contexts andfor some projects, commercialisationis not an essential part of the TELinnovation process.

Innovation is not synonymous withinvention and it is rare that TELresearch produces an individual item of exploitable technolog y in the short term. University-ledresearch is more likely to be focusedon long-term educational impactthan on commercial success.Academic researchers are recruitedand trained to research the theor yand science of learning. They arewell placed to test and evaluate TELinnovations, because education isthe core business of a university.However, they are not encouragedto create educational enterprises,which might compete with theiruniversity for students. Nor arethey encouraged or resourced toreplicate the role of an R&D unitwithin a large company.

The belief by many academics inopen research and free access tolearning does not necessarily fitwell with a commercial imperativefor profitability and return oninvestment. Publicly funded

tel.ac.uk 27

Case study: Cognitive Tutor software

Carnegie Learning has been a provider of innovative, research-based mathematics curricula for middle and high school students in the USA for over a decade and in 2012 was announced as the winner of the Software & Information Industry Association (SIIA) CODiE for Best Mathematics Instructional Solution. The company was founded in the 1990s by staff at Carnegie Mellon University working with practising teachers. In 2011 the Apollo Group acquired the company for $75 million, with an additional $21.5 million payable to the university for related technology.

Throughout this period there has been sustained debate around the issue of how adaptive learning software may enhance student learning. One focus has been on student test scores with some research noting little or no statistically significant change, other studies identifying significant impact, and some reporting mixed, anecdotal or selective data [65-69]. What is clear, however, is that a favourable perception can be bolstered by research and that this can influence purchasing decisions. However, it was found that the effectiveness of software was dependent on teachers’ ability to use it [70]. This required consideration of the complex of TEL elements, resulting in the development of materials such as textbooks, a recommended programme of teacher training, and pedagogy with greater emphasis on individual learning.

A recent comprehensive report by RAND on Cognitive Tutor has found that the cumulative effect can be that ‘treatment group teachers reported less implementation of traditional practices such as lecturing with students taking notes and greater implementation of more progressive practices such as facilitating student work or assigning students to work in groups and give presentations’ [68].

5

universities have a strong motivationto share their research anddevelopment with the public thatpaid for it. Funders may reinforcethis impetus towards openness by requiring that the results of research are made freely and openlyavailable. However, tr ying to makeever ything open and free makes ithard for industr y partners to buildon and extend the work in orderto make money. Some companies,such as Google and Facebook, haveachieved commercial success basedon ‘freemium’ models that providefree ser vices or content and makemoney in other areas. However,many industry partners do not wishto take on the difficulties and risksinherent in the implementation of such models. There is a need for sustainable funding that cansupport both open research andcommercial exploitation.

Academic TEL research has aneffective role to play in contributingto the fundamental research andevaluation of design-based researchpartnerships that are put in place to develop new learning-technologysystems. This is a larger-scaleenterprise than developing individualpieces of software or carr ying outsmall-scale evaluations. It requirescoordination across institutions,involving academic, practitioner andcommercial partners. This shouldnot be an afterthought, but shouldbe planned from the start of theproject. Diana Laurillard, Professor of Learning with Digital Technologies,notes that

early association between academicsand a company wanting to makea certain product is critical. Thecompany needs to feel this isimportant for its future portfolio of products.

A partnership role, with theuniversity as an investor orinnovation partner (as with

FutureLearn, currently partneredwith 26 universities worldwideas well as other institutions) or along-term relationship between auniversity and companies (as withthe Serious Games Institute) islikely to have more successful andsustainable outcomes than workingon the assumption that universitiesare suppliers of TEL inventions.Firms need access to universityexpertise, but are unlikely to expectthe key contribution of universityresearchers to be the production of new technolog y.

RECOMMENDATIONS: Wherecommercialisation is an issue, itshould be taken into account whenimplementing the recommendations

made earlier. When project teamsidentify the elements that must betaken into account in order to enablesustainable implementation of anidea or prototype, in the contextof a vision of the enhancementof learning, they should take thepossibility of commercialisation intoaccount. This requires engagementwith the individuals and communitiesthat will take responsibility forcommercialisation. Policy and fundingshould encourage the developmentof skilled, multidisciplinary teamsthat are able to complete the TELinnovation process. If a commercialoutcome is required, this shouldbe specified from the start andthe project team should includecommercial expertise.●

28 Beyond Prototypes: Enabling innovation in technology-enhanced learning

6 THE PROCESS OF TEL INNOVATION

6.1 The TEL ComplexAlthough, as Section 5 showed, TELis rarely simply a product, a commontendency is to focus solely upon thetechnology element and its transferinto practice. This implicit assumptionof a linear model of innovation oftenunderlies the expectations of policymakers. It is typically assumed thatprocesses of research, developmentand diffusion follow sequentially.Sometimes a phase during whichopportunities or gaps in the marketare identified may precede theresearch. Rothwell [71] provides auseful summary of different modelsof innovation, identifying two linearmodels: technology push and marketpull. The merits and deficiencies ofthe linear model formed the focus ofsession held at a Nobel symposium in 2002 [72].

A study of 2000 cases of innovationby Keeley and his colleagues [14]provided a counterpoint to the simple‘kit’ view of innovation. The studyidentified ten areas of innovation,including the processes involvedin providing ser vices, the ser vicesthat provide value for customers,the profit model, the organisationalstructure, the product performance,the channel by which the ser vicesare delivered and the process ofcustomer engagement. Whileinnovation may take place in any of these areas, it often includesseveral working together. Forexample, an innovative form ofcustomer engagement may require anew form of organisational structure.Although these areas of innovationare framed in relation to market-oriented business organisations

rather than TEL, together theycapture the sense that innovation isgenerally complex and its successfulachievement may involve changes tomany different elements in a mannerthat is multiple rather than linear.

The Beyond Prototypes casestudies show that TEL should beunderstood as a ‘complex’ comprisinga series of components that needto be addressed together. A generic‘technology complex’ includes a widerange of elements, including purpose,materials, procedures, knowledge,organisational structure, industrystructure, location, social relationsand culture [73].

In the case of the TEL complex,several key elements must be takeninto account in order to develop andrealise a vision of innovation. These areset out in the model in Figure 1, andconsidered in the paragraphs below.

Pedagogy is a crucially importantcomponent of successful TELinnovation and goes well beyond thetechnical elements used to supportit. In terms of the areas of innovationidentified by Keeley and his colleagues[14], pedagog y comprises anextremely complex and distinctiveprocess which involves both studentand teacher engagement, delivering aset of educational services by meansof specific channels.

Technical components are the mostvisible components when consideringinnovation within the TEL Complex.They are the technological elementsthat are used to support thepedagogy with the aim of achieving

Section 6 shows that TEL should be considered as a technology complex, made up of a series of interconnected elements that cannot be changed in isolation. A model of the TEL Complex is set out, centred on a vision of educational change. The TEL Innovation Process is also modelled and bricolage is set at its heart. TEL innovation should be evidence driven, and a core methodology is identified as design-based research.

tel.ac.uk 29

6 THE PROCESS OF TEL INNOVATION

a vision that is concerned withenhancing learning in a specified way.

The ecology of practices and technical context must be taken into account, because any TELinnovation will be implemented in a specific ecolog y of practices. For example, the development ofYoza Cellphone Stories (see box)took into account the limited accessto books experienced by somecommunities in South Africa as wellas local practices associated with the use of mobile phones.

Current practices are not easilyaltered; they are at the core of super-stability in the overalleducational system. In the TELComplex, practices include explicitaspects of teachers’ practice as well as the tacit knowledge acquiredthrough extensive apprenticeship,training programmes and longexperience. Students’ practicesare also crucial, and systematicallyrelate to those of teachers. Howstudents learn, both formallythrough structured teaching andlearning programmes, and informally,through social and peer interaction,is important for the effectiveoperation of TEL innovation.

In stable systems, innovation thatinvolves changes throughout theentire ecolog y is characterised as ‘system innovation’. Manydifferent sub-components haveto work together, with eachsubcomponent subject to theconstraints of the overall system.These complex interdependenciesmake it difficult to get anyone element to work or makea difference by itself withoutconsideration of the whole [74].Moreover, different componentscan combine and recombine in manydifferent configurations. Ultimatesuccess depends on the totality ofthe configuration or bundle, ratherthan on any single component.

Communities involved in the TEL Complex include students,teachers, researchers and thoseengaged in technical development. In literature dealing generically with innovation, these would becharacterised as suppliers andcustomers. These four communitieshave others associated with them,including the parents of young

learners, the families of maturestudents, the managers ofeducational institutions and thepeople responsible for teachertraining and technical support. Thecommunities associated with thesedifferent sets of stakeholders oftenhave different values, perspectives,

objectives and above all, expertise.This strong community presencewithin the TEL Complex constitutes a major challenge for TEL innovation,and in many cases exhibits super-stability, meaning that change isextremely difficult to achieve. Inparticular, current expectations of teachers and students affect the adoption of TEL innovations.

The wider context (including policy, funding and revenue generation) Although TEL is not typically a conventional market-oriented business example,there is always a need for sourcesof funding to initiate, sustain andsupport the processes of innovation

POLICY CONTEXT

ENVIRO

NM

ENT

FUN

DIN

G

REVENUE GENERATION

VISION

STUDENT COMMUNITY

PEDAGO

GIC

RES COM

MUN

ITY

TEACH

ER C

OM

MU

NIT

Y

TECHNICAL COMMUNITIES

ECOLOGY OF PRACTICES

TECH CONTEXT

PEDAGOGY

TECH NICAL

Figure 1: The Beyond Prototypes model of the TEL Complex

Any TEL innovation will be implemented in a specific ecology of practices.

30 Beyond Prototypes: Enabling innovation in technology-enhanced learning

Case study: Yoza Cellphone Stories

The aim of the Shuttleworth Foundation funded Yoza Cellphone Stories project (Yoza), formally entitled m4Lit, was to promote leisure reading by the distribution of m-novels to mobile phones in South Africa – a country where less than 10% of public schools have functional libraries but 70% of urban youth have internet-enabled mobile phones. The project began in 2009, taking inspiration from work done in Japan, using an existing mobile chat platform to release content and advertise, and publishing in local languages, including Afrikaans and isiXhosa, as well as English. Yoza considers the key innovation in this process of bricolage not to be the use of phones, but the provision of really engaging stories (some published in episodes), available easily and affordably, with readers able to comment and see others’ comments in near real time.

In early 2013, Yoza won the Netexplo Award in Paris and had a catalogue of over 50 openly licensed m-novels, poems and plays, some of which deal with difficult subjects such as living with HIV. Use of the service has been strong, with over half a million completed reads and 50,000 user comments recorded in the 17 months to December 2012. Securing further funding has proved challenging. However, content has been reused elsewhere, including by Young Africa Live, and the model has helped pave the way for other initiatives in South Africa such as the FunDza Literacy Trust.

6

The model of the TEL Complexillustrated in Figure 1 shows that, in order for innovators to developand achieve a vision of TEL, itis necessar y to engage with allthese elements: pedagog y andtechnology, current practices andcommunities, the local ecology andthe wider complex. This is a processof configurational innovation thatrequires research teams to engage in ‘bricolage’.

6.2 Configurational innovation and bricolageA central theme in the BeyondPrototypes case studies andinter views is that successfulinnovations in TEL are often not new inventions. They more ofteninvolve assembly of technologicalelements and practices, most of whichalready exist, into novel configurations,applied in new settings.

6.2.1 Configurational innovationTechnological innovations, like TELinnovations, often do not rest onnew technological componentsbut rather on the ways in whichpre-existing and well-understoodtechnologies are configured to meetnew challenges. Fleck introduced theidea of ‘configurational technologies’to describe and analyse the ways inwhich technical systems are createdand configured to conform to thecontingencies of specific applications.Local contingencies crystallise to formtechnological configurations [78].

Peine built on this work and used the case of Smart Home systems to show how learning and innovationdevelop in the application ofconfigurational technologies [79].The configurational nature of SmartHome technologies is inherent in the wide range of technologicalsystems and expertise that must bebrought together to create smarthomes. It is also inherent in the need for these homes to work in the context of the local socialpractices and ever yday routines of homeowners.

Both Fleck and Peine emphasise that configurational innovation arises from ‘learning by trying’, bywhich they mean active engagementin design and local experimentationin response to local practices. Thisplaces the stress on innovation as implementation.

There are important parallels herefor TEL innovation, but this reportgoes further in stressing the role ofpractices as part of TEL innovations,not just as part of their context.TEL innovations are most readilyunderstood as configurations, notjust of technological componentsbut also of social practices. As shownby the example of the interactivewhiteboard in Section 5, relevantpractices include, but are notlimited to, pedagogical practices.

and development. This need forsustainable funding has recently been identified in the innovationliterature as an associated ‘businessmodel’ [75] which incorporatesan ‘earning logic’ [76] or ‘revenuemechanism’ [77]. It involves, at the very least: (a) the provision ofvalue, (b) the effective utilisation of assets or resources in providingthat value, as well as (c) the securing of sustainable support through some form of revenue generation. In the case of TEL, if policy dictatesthat funding for TEL is subsumedwithin general educational budgets or within special project funding, then competition with regulardemands or the time-limited nature of project funding can workagainst long-term sustainabilityand adequacy of support. Thisis important, because complexinnovations typically require decades for effective diffusion.

tel.ac.uk 31

6

Although TEL research producesnovel technologies and pedagogies,such work is only a small part ofTEL innovation and should be seenas just one component of broaderconfigurational work.

6.2.2 Social practices as part of the TEL complexThe Beyond Prototypes case studiesand inter views foreground the rolethat is played by social practices, notjust as a context for TEL innovationbut as important elements in theconfiguration of TEL innovations. Forexample, in Mitch Resnick’s account ofthe development of Scratch quotedin Section 4.1, the importance ofprior work on related technologiessuch as Logo is evident. However,a pivotal element of this case is theengagement with and understandingof children’s informal learningpractices in programming clubhouses,and the later appropriation of socialnetworking practices to supportsocial learning. Indeed, the principalinnovation of Scratch lies less in thenature of the programming languagethan in the configuring of Scratchand a social platform to engagewith the informal learning and socialnetworking practices of young peoplethat have emerged in relation tocomputer games.

Similarly, in the case of iZone(see boxed case study), existingtechnologies from motor racing, flightsimulation, eye-tracking and sportsscience have been brought togetherwith a particular set of coachingpractices and a deep understanding of the practices of racing drivers.

There’s as much effort goes into training in this boardroom as

happens on the simulator. You know, when we first start, the hardest thing is to get people to accept that to be successful they’ve got to change their lifestyle. [Alex Hawkridge, chairman of iZone Driver Performance]

The technologies afford newdevelopments in coaching practicesand these afford new ways ofconfiguring the technologies.

The TEL innovation process thusinvolves many different stakeholders,all of whom are embedded indistinct communities with differentexpectations and understandingsof TEL and of learning and teaching.Selected technical elements, specificpedagogic ideas and desired practiceshave to be pulled together intoeffectively working bundles, drawingon contributions from the disparatestakeholder communities. All thesebundles of distinct elements have tobe addressed in order for innovationto take place. This process can becharacterised as bricolage.

6.2.3 BricolageThe anthropologist Lévi-Strausscoined the word ‘bricoleur’ todescribe someone who makes dowith whatever is at hand. Bricoleursdo not typically start a project

and then consider which tools andmaterials will be required to achievetheir goals. Rather, they review theiravailable materials and tools and workout how to use them to achieve theirgoal or something close to their goal[80]. Above all, bricolage is rootedin engagement with the concreteproperties of a situation and theavailable materials, rather than withan abstract model of how they willbehave. For Lévi-Strauss, bricolagedoes not only apply to the materialbut also to the realm of ideas andsocial practices.

The ‘bricoleur’ is adept at performing a large number of diverse tasks; but, unlike the engineer, he does not subordinate each of them to the availability of raw materials and tools conceived and procured or the purpose of the project. His universe of instruments is closed and the rules of his game are always to make do with ‘whatever is at hand’, that is to say with a set of tools and materials which is always finite and is also heterogeneous because what it contains bears no relation to the current project, or indeed to any particular project, but is the contingent result of all the occasions there have been to renew or enrich the stock or to maintain it with the remains of previous constructions or destructions. [81:11]

Lévi-Strauss described bricolage ascharacteristic of primitive societiesand contrasted it with a scientific

The ‘bricoleur’ is adept at performing a large number of diverse tasks; but, unlike the engineer, he does not subordinate each of them to the availability of raw materials and tools conceived and procured or the purpose of the project.

Although TEL research produces novel technologies and pedagogies, such work is only a small part of TEL innovation.

32 Beyond Prototypes: Enabling innovation in technology-enhanced learning

Case study: iZone Driver Performance

iZone was set up in 2009 to address a change in Fédération Internationale de l’Automobile (FIA) regulations, which reduced racing teams’ testing time. While test equipment and simulators for the testing of cars and components were already used, nothing was available that could replace track time for drivers. Sophisticated simulators with video screens had been developed over the previous 35 years, but much more complex systems, able to provide physical feedback such as g-forces, were required for the development of elite drivers.

iZone addressed this problem by interlinking physiological systems and electromechanical systems. It uses eye-tracking technology to enable coaches to analyse drivers’ performance and assess their control during the simulation. This technology was developed by the company’s simulator designer, John Reid, who was inspired by an article about the use of eye-tracking systems in helicopter gunships.

iZone has links with Cranfield Aerospace that stretch back to the 1980s, when company chairman Alex Hawkridge used the wind tunnel at Cranfield to develop the aerodynamics of Toleman F1 cars. The company now uses the g-force technology from Cranfield’s helicopter trainer and also has PhD students from Cranfield working with the company on aspects of the project. A similar long-term relationship with the Department of Electrical Engineering at the University of Sheffield has also helped with the development of the simulator.

Based on work with racing drivers prior to setting up iZone, the team has created a training regime developed by sports scientists and sport psychologists to offer a complete driver development programme that includes the use of the simulators. The sport psychology input came from Dave Collins, who had developed a name for coaching and mentoring in athletics and football as well as in motorsports.

Most technology businesses are concerned with the protection of intellectual property (IP), but Alex Hawkridge’s view is that, ‘the things that are patentable, we don’t think it would be wise to patent, because you then tell people exactly what you’re doing.’ He considers that the most important way to protect the business’s IP is to keep developing the simulator business. The potential for iZone to run a similar operation at every major racetrack in the world is a real opportunity; a future way forward might include franchising the model in order to maintain its speed of development.

6

approach in modern societies.However, subsequent authorshave built on his work to show theimportance of bricolage to areasof current society as diverse asscientific endeavour [82], productdesign, entrepreneurship [83 ,84], social entrepreneurship [85],financial economics [86] and theenactment of social change [25].These streams of research havedrawn on the notion of bricolage inorder to understand how innovationand change come about throughthe creative reinterpretation andarrangement of existing socialpractices and resources, and howthis process enables new possibilitieswithin the constraints of existingsocial systems and institutions.

6.2.4 Bricolage and ‘persistent intent’ within the TEL spaceBricolage is a core theme thathas emerged from the BeyondPrototypes study. The narrativesproduced by case studies andinterviews reveal successful TELinnovators not simply as inventorsor as scientists proposing andtesting hypotheses but also asbricoleurs who achieve educationalgoals by bringing togetherdiverse technological elements,frameworks and social practices.The ultimate success of TEL lies inthe implementation of the entire‘TEL Complex’ modelled in Figure 1.However, much innovative work failsto be implemented, because it doesnot take the complex into account.

A research group is not equipped to take on all the other factors that are required in order to scale, to move from a research project or pilot to a scaled one […] They are lulled into thinking that when they have a successful pilot the next step will be easy. The next step is the hardest step of all. When they go to schools with their piece of kit and their wonderful technology [they fail because] other factors such as curriculum, professional development, sustainability and appropriateness are not taken into consideration [Elliot Soloway, founder of the Center for Highly Interactive Computing in Education]

tel.ac.uk 33

EXPLO

RATI

ON

AN

D N

ETW

ORK

ING

Vision ofeducational

change

Co-constructingnew practices in

formal education

Brokering,covening, influencing

and taking what thesystem will give

Developing practicesin parallel to

format education

Technologydevelopments in

wider society

Technology R&D ineducational field

Understandingecology of practices

Practical researchand expertise

BRICOLAGE

PERSISTENT IN

TENT

Figure 2: Beyond Prototypes model of the TEL Innovation Process

6Each element of the complex requiresexplicit and careful considerationin order to avoid failure andmaximise chances of success. Thisprocess of TEL bricolage does nottake place at a single point in theprocess of innovation process butextends throughout the process.TEL innovation, as is typical forany complex example, can requiredecades for full diffusion and duringthat time researchers engage inbricolage as they work towards theirevolving vision of the developmentof learning and teaching. Thisinvolves not only the combination ofresources but also the developmentand assembly of a stock of resources,in addition to the development of a close understanding of the natureand affordances of what is at hand[87]. This process lies at the heart of the TEL Innovation Process.

6.3 The TEL Innovation ProcessLiterature based on extensiveexperience in various business settingsillustrates that multiple factors andissues have to be attended to in orderfor a new technology or practiceto be employed effectively within acomplex. Many authors have observedthis fundamental characteristic of theimplementation of innovation, yetit is nearly always overlooked whennew technologies are developed [See,for example, 14, 73, 88, 89-91]. In thecontext of innovation in educationalmultimedia, Van Lieshout and hiscolleagues have identified innovationas a process of ‘social learning’ [92].

Figure 2 presents TEL innovation asa process of bricolage that involvesthe assembly of technologicalelements and social practicesto inform a complex process ofinnovation that has the aim ofachieving educational goals. Asnoted above, while the inventionof new technological elements orpedagogic approaches may be acomponent of such innovation it is

by no means a necessary condition.Some elements from that process areexpanded below, along with a designmethodology that encompasses theentire TEL Innovation Process.

6.3.1 Vision of educational changeGenerating change in educationalpractices that is more than local

and temporary is difficult to do anddemands persistent intent over time.This, in turn, requires a clear vision ofwhat could and should be achieved.However, the Beyond Prototypescase studies and interviews show thata clear vision is rarely the startingpoint for the innovation process.

Instead the vision often emergesand evolves through exploration,through networking and throughactive engagement in research,development and educationalpractice. Visions of educationalchange are co-created throughengagement with different aspects of the TEL Complex.

6.3.2 Pedagogical research and expertiseEngagement in research intoeducational technologies andpedagog y has an important roleto play in TEL innovation. Thedirect products of this researchare important but so too are the

Vision often emerges and evolves through exploration, through networking and through active engagement in research, development and educational practice.

34 Beyond Prototypes: Enabling innovation in technology-enhanced learning

6

connections and expertise that are created during the researchprocess. The Beyond Prototypesstudy also highlights another crucialform of research. This is researchthat is aimed at understanding theecolog y of practices with whicha particular TEL innovation mustengage. Examples from the casestudies included in this reportinclude young people’s relationshipsto stor ytelling practices in theYoza case, financial traders’ tradingpractices and learning practices in relation to xDelia, and children’ssocial networking practices inrelation to Scratch.

6.3.3 Developing practices in parallel to formal educationBecause it is difficult to achieve rapidand significant innovation withinformal education sectors, successfulinnovations may impact first oninformal learning practices. In somecases, this can provide a platform for translating the innovation intoformal education.

To take one of the TEL successesdescribed in Section 4 as an example,Scratch is a complex made up ofsoftware, hardware platform(s),informal learning practices, sociallearning practices, as well as manyother elements. This TEL complexis enacted within an ecolog y ofpractices that includes elementssuch as software security practicesand the friendship networks (onlineand offline) of the target audiencein its initial informal learninginstantiation. In this case, it wasinitially enacted within the contextof a specific ecolog y of informal

learning practices. Increasing publicand political concern about schoolleavers’ lack of programming skills has created opportunities totranslate the use of Scratch intoformal learning contexts. In theprocess it becomes something new. Scratch as a TEL complexis different when enacted in aninformal learning setting to itsenactment in a formal learningsetting and the process oftranslating the complex from onesetting to another is non-trivial.

6.3.4 Co-constructing new practicesResearchers bring models andtheories of learning to theinnovation process. These arerefined through working withteachers and learners, togetherdeveloping an understanding ofhow the ecolog y of the educationalsetting impacts upon these modelsand theories. This co-construction

means that researchers andpractitioners can come, over time, to mutual understanding and respect for the ways in whichtheor y-informed TEL can be enacted in real classrooms and other educational settings.

A notable consequence of thecomplexity of TEL, representing amajor developmental opportunity, is that there is scope for ‘user-driven’contributions from both teachersand students. Making use of thesecontributions requires engagementwith users and a willingness to acceptinitial proposals that are sufficientlyunfinished or unpolished to allow foreffective intervention and ownershipto take place. Such engagement mayalso be a necessar y condition toproperly understand the ecolog y of practices that will be the contextfor any particular TEL innovation.

In order to ensure that TELinnovation is evidence driven, it isimportant that this TEL Innovationprocess is aligned with a researchmodel that supports evaluation of what has been achieved, andthat can build on previous findings.A core methodolog y is thereforedesign-based research.

Table 1: Comparison of design-based research with experimental studies

Experimental studies Design-based studies

Laboratory studies Real-world situations that contain limitations, complexities and dynamics

Aimed at testing hypotheses Aimed at designing new interventions and generating hypotheses

Usually single dependent variable

Multiple dependent variables (though not all are investigated)

Control of variables, through specification of fixed procedures

Iterative and flexible revisions of the research design

Typically isolated from the social world

Typically involve social interactions

Researchers are the decision makers

Partners contribute to the decision making

It is important that this TEL Innovation process is aligned with a research model that supports evaluation of what has been achieved, and that can build on previous findings.

tel.ac.uk 35

6

6.3.5 Methodology of Design-Based ResearchThis methodology has been developedover the past two decades andstems from ground-breaking work byCollins [93] and Brown [94]. Theseresearchers developed the idea ofdesign experiments when they foundthat traditional laboratory experimentswere not sufficient to address thequestions of interest to them.

The Design-based Research Collectiveasserts that ‘design-based research,which blends empirical educationalresearch with the theory-drivendesign of learning environments,is an important methodology forunderstanding how, when, and whyeducational innovations work inpractice’ [95]. Barab and Squiresexplain that

design-based research […] was introduced with the expectation that researchers would systemically adjust various aspects of the designed context so that each adjustment served as a type of experimentation that allowed the researchers to test and generate theory in naturalistic context [96:3].

Wang and Hannafin describe theapproach in more detail as

A systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories [97:6].

In many projects this involves iterativecycles of designing pedagog y andtechnolog y, running an inquir y, andthen carr ying out evaluation andanalysis that feed into the next cycle. In this way, some of the key findings

of the research are embedded withinthe system: not only in the designof the software but also in how it isused by a growing and developingcommunity of practice.

Schoenfeld writes of the way in whichdesign experiments work.

Properly construed, a design experiment consists of the creation of an instructional intervention on the basis of a local theory regarding the development of particular understandings. The intervention is then examined with regard to the accuracy of the underlying local theory and the power of the intervention, with an eye

toward refining both. Doing so thus calls for having a solid theoretical perspective and for possessing design skills, two talents rarely found in one individual. This raises the issue of design teams as part of the research endeavor [98].

Table 1 compares design-basedstudies with more conventional,laborator y-based experimentalstudies. The experimental modelworks well when researchers areable to control the process of theresearch and isolate individualvariables. The design-basedapproach aligns better with themodel of the TEL innovation processthat is set out in this report.●

A design experiment consists of the creation of an instructional intervention on the basis of a local theory regarding the development of particular understandings.

36 Beyond Prototypes: Enabling innovation in technology-enhanced learning

7 IMPLICATIONS FOR RESEARCH

Previous sections have set out howthe TEL Innovation Process takesplace, key elements of this process andrecommendations for the future. Thissection examines the implications ofthese elements for researchers.

As significant TEL innovation takesplace over an extended period of time,persistent intent on the part of anyresearch team has been identified ascrucial. Imogen Casebourne, Directorof Learning at the EducationalProgram Innovations Center, tracedthe evolution of mobile learning frominitial research in the 1990s, throughthe time when her company startedworking in the area, to the present day:

We started creating mobile learning about 10 years ago ourselves, using PDAs [personal digital assistants]. It was only a few unusually forward-thinking clients for a very long time, who were interested in exploring it. Whereas, obviously, in more recent years it has really taken off.

The same extended developmentprocess has taken place withcomputer-supported collaborativelearning. Seb Schmoller, Chief Executiveof the Association for LearningTechnology between 2003 and 2012,provided an overview of the growthof this area, illustrating that the TELinnovation process can take decades.Significantly, he suggested that thetimescale for this process can be soextended that original research isforgotten or under-utilised, indicatinga need for persistent engagement andintent over extended periods of time:

15 years ago, there was a research field, though ‘field’ is perhaps too wide a term for it, which was

concerned with computer-supported collaborative working, CSCW. I think a lot of the thinking and findings of CSCW research are very relevant now, because the tools and systems to support computer-supported collaborative work are now ubiquitous, which they weren’t when the research was being done. When the research was being done, a lot of effort had to be put into designing and sustaining the tools and services in order for the research just to happen. Now the communication tools and systems are everywhere, but the kind of ideas that CSCW research threw up, I think to some extent have lain dormant, and are not properly utilised. Because lots of people coming to this technology enhanced learning afresh now don’t realise that there’s a back story of work that is very important.

The eventual line of development, andthe vision of innovation, is not alwaysclear at the start. Early researchmay explore the affordances of aset of technologies – such as mobiledevices or online conferences – ormay investigate how technologiescan be used to support a particularpedagogic approach. When SeymourPapert worked in the 1970s and 80son ‘creating the conditions underwhich intellectual models will takeroot’ [36], there was no technolog yavailable that could enable thedevelopment of a programminglanguage to be used collaborativelyand synchronously by childrenacross the world. Nevertheless, hiswork on constructionism and onprogramming with children deeplyinfluenced Mitch Resnick, and hasinformed his team’s development ofScratch at MIT during the last decade.

Section 7 examines the implications of this report for research. Persistent intent, engagement over time and the use of an appropriate methodology are identified as priorities. Successful research also requires engaging with the practices and stakeholders that must be taken into account if research-informed innovation is to be embedded in practice.

tel.ac.uk 37

7 IMPLICATIONS FOR RESEARCH

Another way in which TEL innovationextends over time is in the tensionbetween creative and practicalapproaches. Early research mayemploy ‘catwalk technologies’ thatdemonstrate an exciting new conceptbut that are also costly, difficult tomaintain and often impractical forextended use [99]. Successful massexploitation requires a ‘ready-to-wear’ system that can be used off theshelf without problems. Researcherstherefore need to take into accounthow this shift will take place, who willcarry it out, and how it will be funded.

Engaging with user communitiesfrom the start has the potential tomake them genuine stakeholders innew knowledge. The Epic LearningGroup, a global provider of learningtechnologies, takes a consultativeapproach; staff work as partnersor advisers with customers inorder to develop appropriate andaffordable solutions. Jeffrey Lins,head of research and innovationat Saxo Bank and a member of thexDelia consortium, stresses that it isimportant for companies to take somedegree of ownership of research anddevelopment, so that these are notdisconnected from implementation.

We designed a project [XDelia] that was aimed at a particular ecosystem. We knew it existed; we didn’t theorise that it would exist sometime somewhere, or did exist somewhere. We knew it existed, we knew basically what that ecosystem was about and I think we had a clear vision for how we connected into that ecosystem. And that was a lot of the power of the project. [Jeffrey Lins, head of research and innovation at Saxo Bank]

Continuing engagement helpsresearchers to gain an understandingof the environment in which theirwork will be implemented, and to beclear what has to happen before aninnovation can be applied in practice.

Diana Laurillard, Professor of Learningwith Digital Technologies (LKL):

You improve the impact of an innovation by looking at what are the drivers in education, what makes people sit up and worry, and it’s funding flows, it’s curriculum requirements, it’s assessment and it’s quality.

This work on understanding theenvironment is crucial. Seb Schmollerof ALT noted that, ‘If your organisationwants to do things differently, you needto do them differently in a way that willwork within the rules and frameworksthat govern the ‘ecosystem’ you arein. It’s easy to try to do things thatultimately just won’t work because theydo not conform to those rules andframeworks’. A technology is unlikelyto result in significant change unless itis connected with shifts in pedagogyand practice. Steve Vosloo, seniorproject officer in mobile learning atUNESCO, observed

I have seen many cases where computers are put into a school or into a computer lab and the teaching, the learning, the whole paradigm, has not changed at all. The only thing that’s different is that it’s being done through a keyboard, and not on pen and paper.

An important role for researcherswithin the TEL innovation process isto ensure that research is evidencebased. This means employing amethodolog y that is appropriateto the process. Section 6.3 .5identified design-based researchas a key approach. However, othernew possibilities are opening upas technolog y-enhanced learningexpands its scope. Learning analytics,‘the measurement, collection, analysisand reporting of data about learnersand their contexts, for purposes ofunderstanding and optimizing learningand the environments in which itoccurs’ [100], provide actionable

intelligence that can provoke orencourage practical action [101].These analytics help educators andlearners ‘to increase the degreeto which our choices are based onevidence rather than myth, prejudiceor anecdote’ [102]. The large numbersof participants in MOOCs allow rapidcycles of A/B testing in which usersare randomly exposed to one oftwo variations of a TEL innovation– control A or treatment B – withchanges in outcome explained bythis assignment, leading to insightsfor further development [103]. Thedevelopment of one TEL innovationmay require the use of many methods.

7.1 Recommendations for researchers• Research teams should identify, at

an early stage, the steps requiredto enable scalable and sustainableimplementation beyond prototypes,so as to enhance learning.

• Researchers need to engagefully with the individuals andcommunities that will play a role in the implementation process.

• Research teams should consideradopting Design-Based Researchas a systematic but flexiblemethodology for research-ledinnovation, based on collaborationamong researchers andpractitioners in real-world settings.

• The interim and final results fromdesign-based studies should besystematically shared with otherresearchers so that the processof innovation can be compared,expanded, and continued overtime. They should also be widelydisseminated to policy makers andpractitioners, through events suchas ‘what research says’ meetings.

• Research institutes should setup long-lasting collaborationsand consortia, involving schools,museums and other educationalsettings as test-beds, to supportlarge-scale comparative and cross-cultural investigations.●

38 Beyond Prototypes: Enabling innovation in technology-enhanced learning

8 IMPLICATIONS FOR POLICY AND FOR FUNDING

Some past policy initiatives havesupported the TEL innovationprocess well. The MicroelectronicsEducation Programme, establishedin 1980, ran for six years with theaim of preparing children for a worldin which microelectronics wouldbe commonplace and pervasive.Coupled with an initiative that mademoney available for schools to buycomputers, this policy drove a well-structured process of TEL innovationthat extended over time and tookinto account the changes necessarythroughout the TEL complex. Duringthe 1990s, the Teaching and LearningTechnology Programme providedimpetus for the adoption of TELacross the university sector. As withthe MEP, this programme took intoaccount the different elements ofthe TEL complex and was thereforeable to provoke change across theeducational system.

No such system-wide initiativeis currently in place; the closestequivalent is the move to introduceprogramming within schools throughchanges to the curriculum. The currentresearch funding system within the UKis not aligned with the TEL innovationprocess, though it is well suited tosupporting short- and medium-term projects capable of producingresults and academic publicationsthat help universities to build a strongsubmission for government audits suchas the Research Excellence Framework(REF). In some cases, as with mobile

learning and computer-supportedcollaborative learning, such projectscan build over a period of years into abody of work that is used to transformlearning and teaching. However, this isan uncertain process of developmentthat is not ideal. Impact outsideacademia would be supported bychanges to current funding modelsin order to support long-termengagement and sustainability.

Seb Schmoller of ALT identified asignificant problem with externallyfunded projects:

I will stand more chance of successfully innovating if I try to innovate within the general constraints and parameters under which I and the organisation are expected to operate, rather than by making use of some temporary external funding that can be used to stimulate activity; because once the funding dries up the stimulus is removed and the activity ceases.

A project makes a short-termdifference but then the funding runsout. The research and developmentteam disbands and moves on to otherfunded projects. Without supportand maintenance, the successfulinnovation begins to wither. Atthis stage, it is not clear how thework should be taken forward. Oneoption would be to move towardscommercialisation, but this presentsseveral problems. As Section5.6 explained, TEL success is notnecessarily commercial success.

At the most basic level, there may benothing to commercialise. A changein pedagogy or practice is unlikelyto be a marketable commodity; anew technology without a changein pedagogy or practice is unlikely

Section 8 examines the implications of this report for policy and for funding, identifying current problems, particularly in relation to sustainability, and proposing solutions.

A project makes a short-term difference but then the funding runs out. Without support and maintenance, the successful innovation begins to wither.

tel.ac.uk 39

8 IMPLICATIONS FOR POLICY AND FOR FUNDING

to have a significant impact. If theresearch project does result in apotentially marketable innovation, it isunclear who should take that forward.The primary role of universities is notto market products. Researchers arenot trained in marketing and are notrecruited for their entrepreneurialvision and spirit. As individuals,they may not have the skills, theinterest, or the legal right to take aninnovation developed by a team anyfurther. They may also be limited intheir choice of business model byfunders’ requirements that their workshould be freely and openly available.

A better option is to ensure that theinnovative development is sustainable.In order for this to happen, theproject must have engaged peoplewho are willing and able to supportits continued success. The successof the Scratch project, for example,is strongly connected with itsdevelopment of online and face-to-face user communities. On a smallerscale, organisations and individualsneed the sense of ownership thatcomes from working closely withresearchers in order to develop aninnovation that works in context.This may not be an artefact. As SteveVosloo, senior project officer in mobilelearning at UNESCO, notes:

You can look at other facets of the broader learning experience. There is some administration that needs to happen, there is some assessment that needs to happen, there is content to be delivered. Perhaps technology can help speed up the assessment process or the administration process. This is not learning, per se. But it creates a more efficient whole experience that could allow the teacher or the learner more time to actually teach and learn. So it’s a kind of system strengthening or efficiency-improving measure. But if you only think of the learner and the learning experience, you don’t get that.

A funding package that focusessolely on the development of an artefact cannot adequatelysupport this process. Focused,persistent intent is needed in orderto encourage teams of researchersto extend and develop their workover time, with a shared goal in mind.Persistent intent has the potentialto focus researchers’ attention onthe context in which their work takesplace, encouraging them to developthe skills necessar y to work withpeople in different contexts andbridge the gaps between them.

Knowledge transfer partnershipshave a role to play here. However,they currently stress ‘the transferof knowledge, technology and skillsto which the company partnercurrently has no access’ [104]. Thereis less emphasis on the non-financialbenefits gained by the universitypartner. Jeffrey Lins, head of research and innovation at Saxo Bank, commented that

the boundaries have to be blurred, and that comes from respect on both sides. Commercial entities need to realise that universities actually do understand that things cost money, and how the business world works and what customers are like and these kind of things, because they do, and they model these things and they are intensely interested in understanding them. On the other hand, universities have to understand that there’s a lot of competent research, researchers and research capabilities outside their walls.

The commercial partner is crucial inproviding the ability to contextualisea problem and to understand itsecology. At the same time, universitiesoffer ways of reframing problemsand identifying new perspectives.There is knowledge on both sidesof the partnership that needs to betranslated and transformed.

8.1 Recommendations for funders and policy makers• Policy and funding should

support innovations in pedagogyand practice, as well as thetechnological developments thatwill support these. This shouldrecognize the need to fundprofessional development ofpractitioners and evaluation of the innovation in practice.

• Policy and funding shouldrecognize the importance ofextended development and providesupport for scaling and sustainingof innovations, beyond prototypesinto educational transformations.

• Policy and funding shouldencourage the developmentof skilled, multidisciplinar yteams that are able to completethe TEL innovation process.Recognition and support shouldbe given to visionary thinking andexperimentation, to generate freshinsights and achievable visions ofeducational developments.

• There is a need to build researchcapacity in TEL within the UK. Theresearch councils should give aclear indication as to where TELproposals should be submitted, andensure that proposals are reviewedby people with appropriateexpertise in TEL research.Evaluation criteria should includesuccessful implementation of plansfor scaling and sustainability.

• Funders should provide support to research teams to evaluate their innovations for educationalimpact and transformation, throughappropriate qualitative and/orquantitative methods. New methodsof evaluation, such as learninganalytics should be encouraged.

• Findings must be made available toother researchers and developers,including those without accessto university libraries, so thatthe results of research anddevelopment can be used tocontinue and complete theinnovation process.●

40 Beyond Prototypes: Enabling innovation in technology-enhanced learning

9 CHALLENGES AND OPPORTUNITIES

Education is a major export for the UK economy, estimated to be worth £17.5 billion in 2011 [105]. Technolog y for learning now forms an integral part of that educational export market,including direct income frompublishing of e-books, onlinelearning, and educational software, plus indirect benefits from competitive advantage in MOOCs, educational analytics, online learning resources, andblended and mobile learning.

This innovation needs to becontinually refreshed in order tomaintain a competitive edge in the combination of technolog y and pedagog y. A year ago there was no major UK investment inmassive open online learning. Now, 23 UK universities have made a substantial strategic commitmentto the FutureLearn company. Theyare offering free education tohundreds of thousands of peopleworldwide, in part as a means ofattracting overseas students toregister for UK degree courses. The pace of change may bequickening, as education entersa similar period of disruptiveinnovation to that faced by theentertainment and banking sectorsten years ago, and as the nationdevelops its understanding of howto develop effective learning withtechnolog y at large scale.

As this report has shown, researchassociated with technology andlearning has influenced and beeninfluenced by other sectors, resultingin some surprising benefits to the UKeconomy. Arm Holdings, the majorBritish semiconductor and softwaredesign company grew out of the workin the 1980s by Acorn computers tobuild the BBC Microcomputer as partof the Microelectronics EducationProgramme. During the same period,thousands of teenagers learnedto play and to program games on

microcomputers such as the SinclairSpectrum, developing talent thatinitiated the UK computer gamesindustry. At the same time, TheOpen University was developing itsdistinctive approach to supportedonline learning that combinescomputer-delivered materials withhuman tutorial support.

Now there is an opportunity forsimilar confluences of research-ledinnovation in learning and technology,around massive online learning,mobile learning, haptic technologiesfor learning, learning design, learninganalytics, technology-based science,maths and computing education,interactive e-books and multimediaeducational publishing. Some of these are altering traditional sectorssuch as publishing and universities;others are opening new businessopportunities in educational software

Section 9 focuses on the way forward for TEL research. It identifies a continuing need for sustained building of capacity in TEL, through graduate programmes and investment in national hubs of expertise that share talent and facilities.

Research associated with technology and learning has influenced and been influenced by other sectors, resulting in some surprising benefits to the UK economy.

tel.ac.uk 41

9 CHALLENGES AND OPPORTUNITIES

applications, and technologies suchas haptic simulators.

These developments arefundamentally interdisciplinar y. They can only take place through the combined efforts of topicspecialists, technologists, andexperts in teaching, learning andassessment. The UK still lacksexpertise in the linking disciplineof educational technolog y. Thecontinuing need is for neitherabstract grand challenges nor short-term initiatives, but for a sustainedbuilding of capacity in technolog y-enhanced learning, through graduateprogrammes and investment innational hubs of expertise that share talent and facilities.

To compete with other national TELcentres such as SRI Internationalin the USA, Nanyang TechnologicalUniversity Singapore, NationalCentral University Taiwan, and ÉcolePolytechnique Fédérale de Lausannein Switzerland, the UK needs to poolresources across universities activein TEL research, involving innovativecompanies and partner schools,

The continuing need is for neither abstract grand challengesnor short-term initiatives, but for a sustained building ofcapacity in technology-enhanced learning, through graduateprogrammes and investment in national hubs of expertise thatshare talent and facilities.

colleges and museums. The prize will be a sustained ability to do‘big R&D’ that develops substantialeducational systems over a prolongedperiod and evaluates them with arange of learners in informal andformal settings.

The focus for future TELresearch should be on effectivetransformation of educationalpractices, rather than smallincremental improvements, and on how these transformationscan be scaled and sustained. We need to design new forms of learningthat people (teachers, students andinformal learners) want to adoptand use. The evidence presentedin this report shows that research-led innovation in TEL is a complexprocess; the big successes cannotbe predicted but they can benurtured through a supportiveenvironment that co-constructslearning and technolog y, supportsthe persistent intent of visionaries,subjects educational innovations tosystematic evaluation, and partnerswith innovators in the educationsector and in creative industries.●

The focus for future TEL research should be on effective transformation of educational practices, rather than small incremental improvements.

42 Beyond Prototypes: Enabling innovation in technology-enhanced learning

BIBLIOGRAPHY

[17] Laurillard, D. Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies. RoutledgeFalmer, New York, 2002.

[18] Katz, D. and Kahn, R. L. The Social Psychology of Organizations. John Wiley & Sons, 1978.

[19] Scott, W. R. Institutions and Organizations (2nd ed.). Sage,Thousand Oaks, CA, 2001.

[20] Fenton-O’Creevy, M., Gooderham, P. and Nordhaug, O.Human resource management in US subsidiaries in Europeand Australia: centralisation or autonomy? Journal of International Business Studies, 39, 151-166, 2008.

[21] Fenton-O’Creevy, M., Gooderham, P., Cerdin, J.-L. andRonning, R. Bridging roles, social skill, and embeddedknowing in multinational organisations. In Politics and Power in the Multinational Corporation: The Role of Institutions, Interests and Identities, edited by C.Dorrenbacher and M. Geppert. Cambridge UniversityPress, Cambridge, 2011.

[22] Fenton-O’Creevy, M. Middle management resistance to strategic change initiatives: saboteurs or scapegoats?In Managing Strategy Implementation : An Organizational Behaviour Perspective, edited by P. C. Flood, T.Dromgoole, T. Carroll and L. Gorman. Blackwell Business,Oxford, UK, 2000.

[23] Ball, S. J., Maguire, M. and Braun, A. How Schools Do Policy: Policy Enactments in Secondary Schools.Routledge, 2012.

[24] Creed, W. E. D., Scully, M. A. and Austin, J. R. Clothes makethe person? The tailoring of legitimating accounts and thesocial construction of identity. Organization Science, 13 , 5,475-496, 2002.

[25] Fligstein, N. Social skill and the theor y of fields.Sociological Theory, 19, 2, 105-125, 2001.

[26] Edxus Group Europe’s 20 Fastest Growing and MostInnovative e-Learning Companies Named http://edxusgroup.com/europes-20-fastest-growing-and-most-innovative-e-learning-companies-named-2/ 2013.

[27] Mayes, T., Morrison, D., Mellar, H., Bullen, P. and Oliver,M. Transforming higher education through Technology Enhanced Learning The Higher Education Academy,York, http://www.heacademy.ac.uk/assets/documents/learningandtech/Transforming.pdf 2009.

[28] Li, Q. and Ma, M. A meta-analysis of the effects of computertechnology on school students’ mathematics learning.Educational Psychology Review, 22, 3 , 215-243 , 2010.

[29] Sharples, M., Arnedillo-Sánchez, I., Milrad, M. and Vavoula,G. Mobile learning: small devices, big issue. In Technology-Enhanced Learning: Principles and Products, edited by N.Balacheff, S. Ludvisgsen, T. de Jong, A. Lazonder and S.Barnes. Springer, 2009.

[1] Charmaz, K. Constructing Grounded Theory: A Practical Guide through Qualitative Analysis Sage, London, 2006.

[2] Strauss, A. and Corbin, J. M. Basics of Qualitative Research: Grounded Theory Procedures and Techniques Sage,Newbury Park, 1990.

[3] Ryan, G. W. and Bernard, H. R. Techniques to identifythemes. Field Methods, 15, 85-109, 2003.

[4] Riessman, C. K. Narrative Analysis Sage, Newbury Park, 1993.

[5] Crook, C. and Harrison, C. Web 2.0 Technologies for Learning at Key Stages 3 and 4: Summary Report. Becta,Coventry, http://www.becta.org.uk 2008.

[6] Noss, R., Cox, R., Laurillard, D., Luckin, R., Plowman, L.,Scanlon, E. and Sharples, M. System Upgrade: Realising the Vision for UK Education. Available at http://telit.org.uk/,http://telit.org.uk/ 2012.

[7] Gao, N. TEL-isphere 1999: The Caribbean & Technolog y-Enhanced Learning. Humanities and Social Sciences Net.http://h-net.msu.edu/cgi-bin/logbrowse.pl?trx=vx&list=EDTECH&month=9904&week=c&msg=TmqV6TPCYpeMPw/UHTlrKw 1999.

[8] Fothergill, R. The Microelectronic Education Programme Strategy (April ) http://www.naec.org.uk/organisations/the-microelectronics-education-programme/the-microelectronics-education-programme-strategy. 1981.

[9] Her Majesty ’s Inspectorate The work of theMicroelectronics Education Programme. Studies in Design Education Craft and Technology, 19, 3 , 160-163 , 1987.

[10] European Commission. Open Consultation: New Research Challenges for Technology Supported Learning. Availableat http://cordis.europa.eu/ist/telearn/consultation.htm 2001.

[11] STELLARnet Concept and Project Objectives. http://www.stellarnet.eu/about/ 2013.

[12] Sutherland, R., Eagle, S. and Joubert, M. A Vision and Strategy for Technology Enhanced Learning: Report from the STELLAR Network of Excellence. STELLAR Network ofExcellence 2012.

[13] Vavoula, G. KLeOS: A Knowledge and Learning Organisation System in Support of Lifelong Learning. PhD,University of Birmingham, Birmingham, UK, 2004.

[14] Keeley, L., Pikkel, R., Quinn, B. and Walters, H. Ten Types of Innovation: The Discipline of Building Breakthroughs.Wiley, Hoboken, NJ, 2013.

[15] O’Toole, R. Pedagogical strategies and technologies for peerassessment in Massively Open Online Courses (MOOCs).Discussion Paper. University of Warwick, Coventry, 2013.

[16] Dewey, J. Experience and Education. Macmillan, London &New York, 1938.

tel.ac.uk 43

BIBLIOGRAPHY

[30] Pilkington, R. M. Measuring the impact of informationtechnology on students’ learning. In International Handbook of Information Technology in Primary and Secondary Education, edited by J. Voogt and G. Knezek.Springer, New York, 2008.

[31] Borgman, C. L., Abelson, H., Dirks, L., Johnson, R.,Koedinger, K. R., Linn, M. C., Lynch, C. A., Oblinger, D. G.,Pea, R. D., Salen, K., Smith, M. S. and Szalay, A. Fostering Learning in the Networked World: The Cyberlearning Opportunity and Challenge. NSF Taskforce onCyberlearning, http://www.nsf.gov/pubs/2008/nsf08204/nsf08204.pdf 2008.

[32] Skinner, B. F. Teaching machines. Science, 128, 969-977, 1958.

[33] Engelbart, D. C. Augmenting Human Intellect: A Conceptual Framework. Contract AF49(638)-1024 ,Stanford Research Institute: for the Air Force Office of Scientific Research, Washington, Menlo Park, California 1962.

[34] Kay, A. and Goldberg, A. Personal Dynamic Media.Computer, 10, 3 , 31-41, 1977.

[35] Negroponte, N. The $100 laptop. In Proceedings ofGlobalization and Education: Joint Working Group, thePontifical Academy of Sciences, the Pontifical Academy ofSocial Sciences 16-17 November, Walter de Gruyter, 2007.

[36] Papert, S. Mindstorms: Children, Computers and Powerful Ideas. Basic Books, Cambridge, MA, 1980.

[37] Berners-Lee, T., Fischetti, M. and Dertouzos, M. L. Weavingthe Web: The Original Design and Ultimate Destiny of the World Wide Web by Its Inventor. Harper Information,2000.

[38] Traxler, J. Mobile Learning – Confronting Reality InProceedings of the ALT-C Manchester, UK http://altc2012.alt.ac.uk/talks/28124 , 11-13 September, 2012.

[39] Rieger, R. and Gay, G. Using mobile computing to enhancefield study. In Proceedings of CSCL ‘97 Proceedings of the2nd international conference on Computer support forcollaborative learning 1997.

[40] Sharples, M. The design of personal mobile technologiesfor lifelong learning. Computers & Education, 34 , 177-193 ,2000.

[41] Kukulska-Hulme, A., Sharples, M., Milrad, M., Arnedillo-Sánchez, I. and Vavoula, G. Innovation in mobile learning: aEuropean perspective. International Journal of Mobile and Blended Learning, 1, 13-35, 2009.

[42] World Innovation Summit for Education BBC Janala.http://www.wise-qatar.org /content/bbc-janala 2011.

[43] Wichita State University Palm Pioneers. http://www.education.wichita.edu/m3/help/kellogg2_files/frame.htm2003.

[44] Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk,N., Eastmond, E., Brennan, K., Millner, A., Rosenbaum,E., SIlver, J., SIlverman, B. and Kafai, Y. B. Scratch:Programming for All. Communications of the ACM, 52, 11, 60-67, 2009.

[45] Papert, S. and Solomon, C. NIM: A Game-PlayingProgram. AI Memos (1959 - 2004), http://18.7.29.232/handle/1721.1/6199?show=full, 1970.

[46] Papert, S. Teaching Children To Be Mathematicians vs.Teaching about Mathematics. AI Memos (1959 - 2004)http://dspace.mit.edu/bitstream/handle/1721.1/5837/AIM-249.pdf?sequence, 1971.

[47] Kukulska-Hulme, A., Sharples, M., Milrad, M., Arnedillo-Sánchez, I. and Vavoula, G. Innovation in mobile learning: a European perspective. International Journal of Mobile and Blended Learning, 1, 1, 13-35, 2009.

[48] Walsh, C. and Shaheen, R. English in Action (EIA): mobile phones as an agent of change for large-scaleteacher professional development and English languagelearning in Bangladesh. In Proceedings of AmericanEducational Research Association Annual ConferenceSan Francisco, CA., 27 April – 1 May, San Francisco, CA., 2013 .

[49] Liyanagunawardena, T. R., Adams, A. A. and Williams, S. A.MOOCs: A Systematic Study of the Published Literature2008-2012. 2013.

[50] Van Dusen, L. M. and Worthen, B. R. The impact ofintegrated learning system implementation on studentoutcomes: implications for research and evaluation.International Journal of Educational Research, 21, 1, 13-24.

[51] Mercer, N., Gillen, J., Kleine Staarman, J., Littleton, K. and Twiner, A. Interactive whiteboards: does newtechnology transform teaching? In Learning across Sites: New Tools, Infrastructures and Practices, edited by S. R.Ludvigsen, A. Lund, I. Rasmussen and R. Säljö. Routledge,London, 2011.

[52] Dillenbourg, P., Schneider, D. and Synteta, P. VirtualLearning Environments. In Proceedings of Third HellenicConference ‘Information & Communication Technologiesin Education’ Greece, Greece, 2002.

[53] Sharples, M., McAndrew, P., Weller, M., Ferguson, R.,FitzGerald, E., Hirst, T., Mor, Y., Gaved, M. and Whitelock,D. Innovating Pedagogy 2012: Open University Innovation Report No. 1. The Open University, Milton Keynes 2012.

[54] Sharples, M., McAndrew, P., Weller, M., Ferguson, R.,FitzGerald, E., Hirst, T. and Gaved, M. Innovating Pedagogy 2013: Open University Innovation Report 2. The OpenUniversity, Milton Keynes 2013.

[55] World Climate Research Programme WCRP Strategy.http://www.wcrp-climate.org /index.php/about-wcrp/about-strategy 2013.

44 Beyond Prototypes: Enabling innovation in technology-enhanced learning

[68] Pane, J. F., Griffin, B. A., McCaffrey, D. F. and Karam, R.Effectiveness of Cognitive Tutor Alegbra I at Scale. RANDWorking Paper. Available at http://www.rand.org /pubs/working_papers/WR984.html 2013.

[69] WWC. Carnegie Learning Curricula and Cognitive Tutor Software. WWC Intervention Report. Available at http://ies.ed.gov/ncee/wwc/pdf/intervention_reports/wwc_cogtutor_083110.pdf 2010.

[70] Daugherty, L., Phillips, A., Pane, J. F. and Karam, R. Analysisof Costs in an Algebra I Curriculum Effectiveness Study.RAND, Available at http://www.rand.org /pubs/technical_reports/TR1171-1.html, 2012.

[71] Rothwell, R. Towards the fifth-generation innovationprocess. International Marketing Review, 11, 1, 7-31, 1994.

[72] Grandin, K., Wormbs, N. and Widmaim, S. (ed.) TheScience-Industry Nexus: History, Policy, Implications.(Nobel Symposium 123). Science History Publications/USA,Sagamore Beach MA., 2004.

[73] Fleck, J. and Howells, J. Technology, the technology complexand the paradox of technological determinism. Technology Analysis and Strategic Management, 13 , 4 , 523-531, 2001.

[74] Nelson, R. R. and Winter, S. G. In search of useful theory of innovation. Research Policy, 6, 1, 36-76, 1977.

[75] Teece, D. J. Business models, business strategy andinnovation. Long Range Planning, 43 , 172-194 , 2010.

[76] Nenonen, S. and Storbacka, K. Business model design:conceptualizing networked value co-creation. International Journal of Quality and Service Sciences, 2, 1, 43-59, 2010.

[77] Chesbrough, H. Business model innovation: opportunitiesand barriers. Long Range Planning, 43 , 354-363 , 2010.

[78] Fleck, J. Configurations: crystallizing contingency.International Journal of Human Factors in Manufacturing,3 , 1, 15-36, 1993.

[79] Peine, A. Understanding the dynamics of technologicalconfigurations: A conceptual framework and the caseof Smart Homes. Technological Forecasting and Social Change, 76, 3 , 396-409, 2009.

[80] Lévi-Strauss, C. La Pensée Sauvage. Librarie Plon, Paris,France, 1962.

[81] Lévi-Strauss, C. Savage Mind. University of Chicago,Chicago, 1962.

[82] Turkle, S. and Papert, S. Epistemological pluralism: stylesand voices within the computer culture. Signs, 16, 1, 128-157, 1990.

[83] Baker, T., Miner, A. S. and Eesley, D. T. Improvising firms:bricolage, account giving and improvisational competenciesin the founding process. Research Policy, 32, 2, 255-276, 2003.

[56] Leaning, M. The One Laptop Per Child project and theproblems of technology-led educational development. InHigh-tech Tots: Childhood in a Digital World, edited by I. R.Berson and M. J. Berson. Information Age Publishing, 2010.

[57] DfES. Harnessing Technology: Transforming Learning and Children’s Services. Department for Education and Skills,London 2005.

[58] HEFCE. HEFCE Strategy for E-learning. Higher EducationFunding Council for England, Joint Information SystemsCommittee & Higher Education Academy London, http://management-class.co.uk/readings/05 _12.pdf 2005.

[59] HEFCE. Enhancing Learning and Teaching through the Use of Technology. Higher Education Funding Councilfor England, http://www.hefce.ac.uk/media/hefce1/pubs/hefce/2009/0912/09_12.pdf 2009.

[60] Haggard, S. The Maturing of the MOOC. Department forBusiness Innovation and Skills, London, https://http://www.gov.uk/government/uploads/system/uploads/attachment_data/file/240193/13-1173-maturing-of-the-mooc.pdf 2013.

[61] Leckhart, S. and Cheshire, T. University just got flipped:how online video is opening up knowledge to the world.Wired, 5: May 2012, 2012.

[62] Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C.and Schmid, R. F. What forty years of research says aboutthe impact of technology on learning a second-ordermeta-analysis and validation study. Review of Educational Research, 81, 1, 4-28, 2011.

[63] Goldacre, B. Building Evidence into Education. Departmentfor Education, London, https://http://www.gov.uk/government/uploads/system/uploads/attachment_data/file/193913/Building_evidence_into_education.pdf 2013.

[64] Kirkwood, A. and Price, L. Technolog y-enhancedlearning and teaching in higher education: what is‘enhanced’ and how do we know? A critical literaturereview. Learning, Media and Technology. DOI 10.1080/17439884.2013.770404 , 2013 .

[65] Dynarski, M. and Means, B. Effectiveness of Reading and Mathematics Software Products: Findings from the First Student Cohort. US Department of Education, Available athttp://ies.ed.gov/ncee/pdf/20074005.pdf 2007.

[66] Empirical Education. Comparative Effectiveness of Carnegie Learning’s Cognitive Tutor Bridge to Algebra Curricilum: A Report of a Randomised Experiment in the Maui School District. Empirical Education ResearchSummary. Available at http://www.empiricaleducation.com/pdfs/ES-CognitiveTutor2.pdf 2007.

[67] Morgan, P. and Ritter, S. An Experimental Study of the Effects of Cognitive Tutor Algebra I on Student Knowledge and Attitude. Carnegie Learning. Available at. http://www.carnegielearning.com/web_docs/morgan_ritter_2002.pdf 2002.

tel.ac.uk 45

[84] Garud, R. and Karnøe, P. Bricolage versus breakthrough:distributed and embedded agency in technologyentrepreneurship. Research Policy, 32, 2, 277-300, 2003.

[85] Di Domenico, M., Haugh, H. and Tracey, P. Socialbricolage: theorizing social value creation in socialenterprises. Entrepreneurship Theory and Practice, 34 , 4 , 681-703 , 2010.

[86] MacKenzie, D. An equation and its worlds. Bricolage,exemplars, disunity and per formativity in financialeconomics. Social Studies of Science, 33 , 6, 831-868,2003 .

[87] Duymedjian, R. and Rüling, C.-C. Towards a foundationof bricolage in organization and management theory.Organization Studies, 31, 2, 133-151, 2010.

[88] Utterback, J. M. Mastering the Dynamics of Innovation.Harvard Business School Press, Boston, MA, 1994.

[89] Swanson, E. B. Information System Implementation: Bridging the Gap between Design and Utilisation. Irwin,Homewood, Illinois, 1988.

[90] Rhodes, E. and Wield, D. Implementing New Technologies: Innovation and the Management of Technology. Blackwell,Oxford, 1985.

[91] Leonard-Barton, D. Wellsprings of Knowledge. HarvardBusiness School Press, Boston, 1985.

[92] Van Lieshout, M., Egyedi, T. M. and Bijker, W. E. Social Learning Technologies: The Introduction of Multimedia in Education. Ashgate, 2001.

[93] Collins, A. Toward a design science of education. In New Directions in Educational Technology edited by E. Scanlonand T. O’Shea. Springer-Verlag, New York, 1992.

[94] Brown, A. Design experiments: theoretical andmethodological challenges in creating complexinter ventions in classroom settings. The Journal of the Learning Sciences, 2, 2, 141-178, 1992.

[95] Design Based Research Collective Design-based research:an emerging paradigm for educational inquiry. Educational Researcher, 32, 1, 5-8, 2002.

[96] Barab, S. A. and Squire, M. Design based research: puttinga stake in the ground. The Journal of the Learning Sciences, 13 , 1, 1-14 , 2004.

[97] Wang, F. and Hannafin, M. J. Design-based researchand technolog y-enhanced learning environments.Educational Technology Research and Development, 53 , 4 , 5-23 , 2005 .

[98] Schoenfeld, A. H. Bridging the cultures of educationalresearch and design. Educational Designer: Journal of the International Society for Design and Development in Education, 1, 2, 2009.

[99] Adams, A., FitzGerald, E. and Priestnall, G. Of catwalktechnologies and boundary creatures. ACM Transactions of Computer-Human Interaction, 20, 3 , 15-40, 2013.

[100] Siemens, G., Gašević, D., Haythornthwaite, C., Dawson,S., Buckingham Shum, S., Ferguson, R., Duval, E., Verbert,K. and Baker, R. S. J. d. Open Learning Analytics: An Integrated and Modularized Platform (Concept Paper).SOLAR 2011.

[101] Ferguson, R. Learning Analytics for Open and Distance Education. Commonwealth Educational Media Centrefor Asia (CEMCA), New Delhi, India, http://cemca.org.in/ckfinder/userfiles/files/EdTech Notes_LA _Rebecca_15 May.pdf 2013.

[102] Cooper, A. What Is Analytics? Definition and Essential Characteristics. JISC CETIS, Bolton, http://publications.cetis.ac.uk/2012/521 2012.

[103] Kohavi, R., Henne, R. M. and Sommerfield, D. Practicalguide to controlled experiments on the web: listen toyour customers not to the hippo. In Proceedings of 13thACM SIGKDD International Conference on KnowledgeDiscovery and Data Mining, ACM, 2007.

[104] Technology Strategy Board Knowledge TransferPartnerships: FAQs. http://www.ktponline.org.uk/faqs/ 2013.

[105] Department for Business Innovation and Skills.International Education: Global Growth and Prosperity.Available from http://www.gov.uk/bis 2013.

46 Beyond Prototypes: Enabling innovation in technology-enhanced learning

ACKNOWLEDGMENTS

Professor Eileen Scanlon is Regius Professor of Open Education andAssociate Director of Research and Innovation in the Institute of EducationalTechnology at The Open University. She has extensive research experienceon educational technology projects in open education, science learning andevaluation.

Professor Mike Sharples is Professor of Educational Technology at TheOpen University. He has over 30 years of experience in bridging researchand exploitation of TEL, through projects with British Telecom, Racal, Kodak,Microsoft, Nokia, Sharp and FutureLearn.

Professor Mark Fenton-O’Creevy is Professor of OrganisationalBehaviour with the Open University Business School. He is a senior fellowof the HEA and a National Teaching Fellow, with a recent track record ofsuccessful commercialisation of a major TEL project, the EU FP7-fundedxDelia project.

Professor James Fleck is Professor of Innovation Dynamics at the Open University Business School and has researched innovation (mostrecently in e-learning) for 30 years, focusing specifically on issues of practicalimplementation. He was a member of the Academic Panel for the DTI/HTMInnovation Policy Review chaired by Lord Sainsbury.

Dr Caroline Cooban is Development Director at The Mighty Creatives, a creative development agency for children and young people, workingacross formal and informal settings. She also has over 15 years of experienceof university research business development and knowledge transfer in thefield of educational and learning technologies.

Dr Rebecca Ferguson is a lecturer in the Institute of EducationalTechnolog y. She has worked as research lead on educational technolog yprojects including SocialLearn and EnquiryBlogger. Previously, she worked as editor and journalist on a range of publications, including the Radio Times.

Dr Simon Cross works in the Learning and Teaching Development Teamin the Institute of Educational Technology. He has worked in research andcoordination roles on external and internal projects with a focus on the openonline learning space, learning design implementation and practice, badgesand assessment.

Peter Waterhouse has worked in a variety of educational and businessroles with particular experience in TEL, within both academia and industry.He has graduate qualifications in marketing as well as teaching and learning.

Many thanks to our interviewees: Alex Hawkridge, Diana Laurillard, Domic Savage, Elliot Soloway, Imogen Casebourne, Jeffrey Lins, Jeremy Roschelle, Mitch Resnick, Peter Scott, Roger Blamire, Sara de Freitas, Seb Schmoller, Steve Vosloo and Thomas C Reeves.

Research team

ACKNOWLEDGMENTS

beyondprototypes.com

TECHNOLOGY ENHANCED LEARNING RESEARCH PROGRAMME

London Knowledge Lab Institute of Education University of London 23-29 Emerald Street London, WC1N 3QS

YouTube youtube.com/tlrptelTwitter @TLRPTELEmail tlrptel@gmail.comPhone +44 (0)20 7911 5577tel.ac.uk