learning technology research

Learning Technology ResearchPeter Goodyear

Quality Research: fitness for purpose(Re)framing relations between technology and learning

Capacity-building, collaboration, etc

http://www.slideshare.net/petergoodyear90/health-elearning-rg-may-2014

http://www.slideshare.net/petergoodyear90/health-elearning-rg-may-2014

Educational research quality does not have a strong reputation

“While there was world leading research in most subfields, at the lower end there was insufficient building upon other work, including testing hypotheses/research questions in new contexts. The contrast between the innovative and the pedestrian was especially acute in research in HE, assessment, early years, adult education, and in subject-discipline focused work, including ICT.

Small scale work sometimes reflected inward referencing sub-communities with insufficient awareness of the generic mainstream or of relevant international developments.”

UK RAE 2008 Subject Overview Report of the Education Panel, p3http://www.rae.ac.uk/pubs/2009/ov/

http://www.rae.ac.uk/pubs/2009/ov/

Educational research quality application-focussed R&D has extra problems

1) Institutions can be risk averse when selecting research outputs to be assessed for quality (eg RAE, peer-review in ERA); user-focused outputs under-represented in UK 2008 RAE

2) Best user-oriented outputs: “sophisticated and complex ideas and data were vividly and eloquently communicated by their authors” Weaker outputs: ”valuable material was expressed in an unnecessarily arcane and convoluted way”

3) “user audiences tend to be impressed by work that consciously builds on what has been done before – the systematic review and research synthesis being prime examples of cumulative work” – much of the weaker work made insufficient reference to recent, relevant research; missed opportunities to integrate knowledge

4) “given that many researchers in education come to research from a teaching career or an academic career outside the discipline of education, we think that it might be advisable in any future exercise to make an even clearer distinction between pedagogical research in higher education and descriptive or anecdotal accounts of teaching developments and evaluations

NO YES

YES Pure basic research (Niels Bohr)

Use-inspired basic research

(Louis Pasteur)

NO ‘Bird spotting’(Gilbert White)

Pure applied research (Thomas Edison)

Considerations of use (practical application etc)?

Quest for fundamental understanding (theoretical/ scientific advances, etc)?

Resolving the tension between applied and pure researchResearch in ‘Pasteur’s Quadrant’ (Stokes, 1997)

Alan SchoenfeldUC Berkeley (1999)

Improving the quality of educational research(in Pasteur’s Quadrant)

Quality (fitness for purpose)



Unpacking ‘quality’


Use by whom?For what purposes?How do they work?What kinds of knowledge can make a difference to their work?Resolvable.


What needs explaining?What does an adequate explanation look like?What kinds of facts, principles, models, theories, etc are suitable for our domain(s) of study?Widely (and wildly) contested – make an explicit, considered choice & justify it.

Lack of attention to these issues has undermined general perceptions of the usefulness of educational research

Unpacking ‘quality’


Use by whom?For what purposes?How do they work?What kinds of knowledge can make a difference to their work?Empirically resolvable.

Lack of attention to these issues has undermined general perceptions of the usefulness of educational research

Research users are local: key actors in an evolving ecosystem – research informs iterative improvements to course, curriculum, pedagogy, environment, tools & resources etc; actionable knowledge, contextually sensitive; cf formative evaluation, participatory design

Research users are distant/global: the research & (re)design loop is mediated by publication in journal articles etc; generalisation; principled/declarative knowledge;

Types of Educational Research (IES/NSF)

Foundational (Early Stage / Exploratory)1. Foundational2. Early-stage/exploratory3. Design & development research

Impact research (What works)4. Efficacy research5. Effectiveness research6. Scale-up research

Source: this and next two slides adapted from National Academy of Engineering and National Research Council (2014 ) STEM integration in K-12 education: Status, prospects, and an agenda for research, Washington DC, The National Academies Press. pp138-40. Much more extensive version in IES/NSF (2013) Common guidelines for education research and development. Washington DC. http://ies.ed.gov/pdf/CommonGuidelines.pdf

‘Gold Standard’ Methods for #5 & #6 (such as RCTs) are inappropriate for foundational/early stage research.

Bench -> BedsideLab -> Learnplace

Translation processes are researchable too

Foundational/Early-Stage

Contributes to core knowledge in education (eg basic understandings of teaching and learning, cognition, competence, functioning of educational systems and learning environments, etc)

1. Foundational: provides fundamental knowledge that may be relevant to improving learning outcomes; theories of learning, cognitive processes etc

2. Early-stage/exploratory: Finding relationships/patterns in educational variables; exploring logic of relationships in ways that may inform subsequent interventions

3. Design & development research: Develops and tests candidate solutions to educational problems; may be as much for refinement of theory or improved understanding the nature of the problem as for demonstrating the value of the solution

Impact research

Contributes to evidence of impact; reliable estimates of the ability of a fully-developed intervention (etc) to achieve intended outcomes; methodological imperatives to precisely specify the intervention, the intended outcome measure, reduce bias)

4. Efficacy: testing under ideal/enhanced conditions (eg with high level of support from researchers etc who developed the intervention)

5. Effectiveness: examines effectiveness in ‘normal’ circumstances (minimal developer involvement)

6. Scale-up: examines effectiveness across a wide range of populations, contexts, cicumstances etc

(Re)framing relations between technology and learning

From gadget-based acceptance testing to the orderly evolution of learning systems

Activity-centered analytical framework

(Emergent) Activity Tasks

Artifacts, tools, texts, etc

Dyads, groups, teams; roles; divisions of labor

Physically Situated

Socially Situated

Outcomes

(Goodyear & Carvalho, 2014, p. 59)

(Emergent) Activity Tasks

Artifacts, tools, texts, etc

Dyads, groups, teams; roles; divisions of labor

Physically Situated

Socially Situated

Outcomes

(Goodyear & Carvalho, 2014, p. 59)

Goal-directed action (intentions formed in the mind precede and direct selection of tools, actions etc)

Embodied cognition, extended mind, intermingling of mind-body-world (tool-using action brings forth intention)


Technology-aided learning comes naturally

“Human-machine symbiosis, I believe, is simply what comes naturally. It lies on a direct continuum with clothes, cooking (‘external, artificial digestion’), bricklaying and writing. The capacity to creatively distribute labour across biology and the designed environment is the very signature of our species, and it implies no real loss of control on our part. For who we are is in large part a function of the webs of surrounding structure in which the conscious mind exercises at best a kind of gentle, indirect control. ”

(Andy Clark, 2003, p174, emphasis added)

Metaphors for learning

• Learning as acquisition• Learning as participation– Learning through participation (e.g. to acquire ...)– Learning to participate (e.g. in a valued, ongoing

practice)• Learning as knowledge creation• Learning as creating epistemic

tools/environments

Metaphors for learning

Learning to participate involves understanding the properties of tools (etc); incorporating them into your activities (instrumental genesis); using them to create new tools


Outcomese.g. cognitive; psychomotor; affective

identity, enculturation, engagement in valued practices, etc

Activitye.g. mental; physical

problem-solving; reflectivewith/without others; with/without tools/artefacts

heavily situated or abstracted, etc

Peoplee.g. friends, workmatesindividually; collectively

dyads, groups, teams, communities

Tools, artefactsand other resources in the material-digital world

e.g. books; wikis, email; LMS/VLEs; seminar rooms, etc

Study situationIntrinsic feedback

from self-monitoring; over various timescales

Extrinsic feedbackvia others;

over various timescales

Maintenance Products

TasksMacro-meso-micro in scale

e.g. dissertation project; essay; lab report; answering a questionembedded meso-scale assessed tasks as key devices

Activity-centered analysis

Outcomes

Activity

People Tools, artefacts

Tasks

Activity-centered analysis

The nature of valued learning outcomes

Reliable links between activities and outcomes

Factors affecting the translation of tasks into activities (deep, surface & strategic learning…)

The ergonomics of learning environments

Effects of grouping strategies

Shifting focus from the affordances of single tools/artefacts to analysing and influencing (? as a participant)

complex evolving meshworks of people and things

IISME: Innovations in Science & Maths EducationCoCo: Researching combinations of new pedagogy & new technologyCHAI: Human-Computer Interaction ResearchLatte: Learning and Affective Technologies?CPC Science of Learning Science node: methods for researching learning in biomedical sci

STL: Sciences & Technologies of Learning SyReNs network focus of researching in STEM education & professional learninghttp://sydney.edu.au/research/stl/ http://blogs.usyd.edu.au/stl/

“CRISTL”: Research Capacity Building & Research-Informed Educational Innovationnot an overarching structure; facilitating networking

Support & Collaboration

[email protected]

http://sydney.edu.au/research/stl/

http://blogs.usyd.edu.au/stl/

Recommended readingSfard, A. (1998) On two metaphors for learning and the dangers of just choosing one. Educational Researcher, 27, 4-12.

Paavola, S., Lipponen, L. & Hakkarainen, K. (2004) Models of innovative knowledge communities and three metaphors of learning. Review of Educational Research, 74, 557-576.

Overdijk, M., Diggelen, W., Kirschner, P. & Baker, M. (2012) Connecting agents and artifacts in CSCL: Towards a rationale of mutual shaping. International Journal of Computer-Supported Collaborative Learning, 7, 193-210.

Affordances; Structuration; Instrumental genesis

Lonchamp, J. (2012) An instrumental perspective on CSCL systems. International Journal of Computer-Supported Collaborative Learning, 7, 211-237.

Malafouris, L. & Renfrew, C. (Eds.) (2010) The cognitive life of things: Recasting the boundaries of the mind, Cambridge, McDonald Institute for Archaeological Research, University of Cambridge.

Further reading: connecting physical setting to human activity

Malafouris, L. & Renfrew, C. (Eds.) (2010) The cognitive life of things: Recasting the boundaries of the mind, Cambridge, McDonald Institute for Archaeological Research, University of Cambridge.Malafouris, L. (2013) How things shape the mind: A theory of material engagement, Cambridge, MA, MIT Press.Ingold, T. (2011) Being alive: Essays on movement, knowledge and description, Abingdon, Routledge.Clark, A. (2008) Supersizing the mind: Embodiment, action, and cognitive extension, Oxford, Oxford University Press.Clark, A. (2003) Natural-born cyborgs: Minds, technologies, and the future of human intelligence, Oxford, Oxford University Press.Sterelny, K. (2012) The evolved apprentice: How evolution made humans unique, Cambridge MA, MIT Press.Sterelny, K. (2003) Thought in a hostile world: The evolution of human cognition, Oxford, Blackwell.Kirsh, D. (2013) Embodied cognition and the magical future of interaction design. ACM Trans. Comput.-Hum. Interact. 20, 1-30.

Probably won’t use slides after this point

except if needed for discussion

Follow up: Goodyear, P. & Carvalho, L. (2013) The analysis of complex learning environments. In Beetham, H. & Sharpe, R. (Eds.) Rethinking pedagogy for a digital age: Designing and delivering e-learning. RoutledgeFalmer.

Clark: self-engineering“We do not just self-engineer better worlds to think in. We self-engineer ourselves to think and perform better in the worlds we find ourselves in. We self-engineer worlds in which to build better worlds to think in. We build better tools to think with and use these very tools to discover still better tools to think with. We tune the way we use these tools by building educational practices to train ourselves to use our best cognitive tools better. We even tune the way we tune the way we use our best cognitive tools by devising environments that help build better environments for educating ourselves in the use of our own cognitive tools (e.g. environments geared toward teacher education and training). Our mature mental routines are not merely self-engineered: they are massively, overwhelming, almost unimaginably self-engineered.” Clark, A. (2008) Supersizing the mind: Embodiment, action, and cognitive extension, Oxford, Oxford University Press, pp59-60.

1 way communication

2 way communication

position and route finding

traversing steep ground

checking pulse

administering painkillers

bandaging

evacuating casualty

visual search

worrying about damaging iPad

“…knowledge generation … [is] … a joint exercise of relational strategies within networks that are spread across space and time, and performed through inanimate (e.g. books, mobile phones, measuring instruments, projection screens, boxes, locks) as well as animate beings in precarious arrangements.

Learning and knowing are performed in the processes of assembling and maintaining these networks, as well as in the negotiations that occur at various nodes comprising a network… Things – not just humans, but the parts that make up humans and non-humans – persuade, coerce, seduce, resist and compromise each other as they come together.”

Tara Fenwick et al., (2011, 10) Emerging approaches to educational research: tracing the sociomaterial

Analysis and design of complex learning environments: sociomaterial perspectives

Successful participation in the exercise involves:

1. learning to use each tool, at least with sufficient fluency to be able to act according to the established protocols, but ideally with a level of automaticity that binds tool and action in a smooth flow

2. integrating the use of the tools into a web of activity, involving smooth effective action, co ordination with others, focus on the priority goals, ‐etc.

3. turning the individual and aggregate experiences of the exercise into learning that lasts

The point of the exercise is not to master the individual tools but to participate in the construction of a co ordinated web of activity that can ‐result in a successful rescue, minimising danger to participants, and leaving traces (in some kinds of memory) that mean doing something like this again will not feel entirely new….