the academy research observatory: development and community engagement martin oliver london...
TRANSCRIPT
The Academy Research Observatory:development and community engagement
Martin Oliver
London Knowledge Lab
& Higher Education Academy
Overview
Context: what’s motivated this?
Evidence, practice, policy and e-learning
Projects at the IoE
The Academy’s Research Observatory
The landscaping study
Early pilots
Current developments
Future plans
Discussions
Your uses of evidence; the Observatory; etc
Some background
Familiar account of the rise of New Managerialism
Monitoring, accountability and performance indicators
Private, professional judgements (e.g. around pedagogy) made into strategic, manageable issues
“What gets measured gets done”
Rational and pragmatic
Evidence-based practice (a caricature)
Draws from a positivist tradition of enquiry
Evolution in medicine
The hierarchy of evidence – reliability as key
Summarise and aggregate studies appropriately (and often, mathematically)
However…
Medical model doesn’t necessarily work in e-learning
Qualitative research acceptable or favoured
Education as a theory-building field (whilst evidence-based practice movement eschews theory for pragmatics)
Practical (and political) issue of de-centred field (no Cochrane database… what fields should we cover?)
Evidence of benefits from involving practitioners in research, rather than differentiating researchers from practitioners (the studied)
What alternatives might there be…?
And more generally…
Clegg’s critique of systematic reviewAppeal to evidence as part of wider erosion of professional judgement
‘Consumer’ represented in the data, but only as constituted and spoken for by the researchers
Reviews undertaken to shore up policy, not inform it
‘Black box’ of favoured methods fails to explain what works – which is important given complexity of teaching and learning
Consequent gap between reviews and practical application
A theorisation
Wenger’s communities of practice model
Practice and Reification as necessary but fundamentally different (experience and abstraction)
Reifications received by communities and meaning negotiated in relation to practice
A social account of acceptable (policed) interpretation
This has implications for how we make sense of research
An aside about legitimation
Lyotard’s account of knowledge in post-modernity
Separation of ‘knowledge’ from the knower
Knowledge as a commodity, valorised through use (consumption)
Value to researchers? To practitioners? To policy makers?
Scepticism about grand narrativesLanguage games as the means of legitimising claims
(…somehow, this should all connect…)
So…
More general interest in ‘evidence informed’ practice and policy
Problems perceived to exist at various stages:• Finding research
• Making sense of it
• Acting on it
Not strongly visible in UK policies• cf. No Child Left Behind: “scientifically-based research”
…but evident in policy discourse
“We need social scientists to help determine what worked and why, and what types of policy initiative are likely to be most effective”
Blunkett, 2000
Harnessing TechnologyA practice-based research environment
64. Our best understanding of how to improve practice comes through learning from others – from experimentation, collaboration, and dissemination. We need an active, practice-oriented R&D forum to bring together teachers, lecturers researchers, and industry to develop the leading edge applications. It would combine the practitioner knowledge of teachers and lecturers with the specialist knowledge of learning technologists and industry suppliers. It should help to build the evidence-base for the value and impact of e-learning.
An interesting start• A social account of knowledge-building• Explicitly linked to practice
Although…• Political imperative: unifying practice, support and the
market• Evidence “for” (not to change?) value and impact
Research into the ‘teaching-research nexus’
This research indicates that staff who see their research as tentative and as part of a wider debate in the discipline, and see their teaching as supporting student conceptual change, are more likely to bring their teaching and research together.
By contrast staff who see their research as atomistic investigations and their teaching as concentrated on teacher-focused transmission of information are less likely to experience strong connections between teaching and research.
At departmental level teaching and research are now often organised separately, and in many cases limited thought is given to, and few explicit policies determine, how they might be linked. Indeed in some cases it may well be that the pressures for research selectivity, such as those of the RAE, are causing increased fissures between teaching and research.
The ‘nexus’ isn’t one thing
Differentiation between:
• Research tutored• Research resources central to curiculum
• Research based• Inquiry-based learning
• Research led• Oriented to subject content
• Research oriented• Focus on process of knowledge construction
Meanwhile, back at the ranch…
IoE participation in Academy-funded e-Learning Benchmarking work (Mellar et al)• A structured programme of self-assessment• But…
“Whilst the changes to strategy are important, it is clear that these in themselves will not be able to bring about the changes needed. The rationale for participation thus became more one of what we could learn generally from other participants’ experiences: about the meaning of e-learning for them, and the way in which they organised and managed e-learning in their institutions.”
(Benchmarking blog entry)
PREEL
From Pedagogic Research to Embedded e-learningBuilding on the Benchmarking project
Funded through the Academy Pathfinder funding
Within our institution (and we suspect in many others) there are pockets of excellent practice in e-learning, and several strong research communities (as well as specific work undertaken by individuals) but there is a lack of coordination, and a recognition that this wealth of experience is not coming together to provide a unified e-learning experience for all our students. This project sets out to bridge this gap between e-learning research and e-learning practice within the IoE, and to demonstrate how this was done in such a way as to be of value to other HE institutions.
PREEL
Led from the LTU
Projects to redevelop curricula, leading to publications
A network built around this work
Workshops to share existing research
Report summarising research, tools, resources
A real person to talk to! (A key element)
A focus on involving people in research
PREEL2
Building on the work in Quality Assurance at a national level
e-Learning QA/QE Special Interest Group
A community-led initiative, involving QAA
London Pedagogy Planner
Laurillard et al – building on existing research into curricula and technology
Development of a “prototype for a collaborative online planning and design tool that supports lecturers in developing, analysing and sharing learning designs”
A formal, computable representation of course-related decisions
Developing strategies for translating e-learning research into practice
Building on PREEL, LPP, etcMellar, Laurillard & Hadjithoma-Garstka
PREEL process: “practitioners could not see the links between the research and their particular needs”
LDSE (Learning Design Support Environment): “It will be a space for teachers to get support in designing their lessons with integrated technology, share their practices of technology enhanced learning and make it easier for other to embed TEL in their practices.”
Action research as a way forwards?
Meanwhile, somewhere in York…
The Observatory
A service promoting and exploring the use of practice- and research-based evidence to influence policy and practice in teaching and learning in Higher Education
Tools to enable access to evidence and syntheses of this evidence (e.g. prototype repository and syndicated search, wiki)
Spaces (real and virtual) to help communities explore evidence-based practice and its implications for students' learning
A work in progress
Initial proposal to HEFCE for an e-Learning Research Observatory
Landscaping studyhttps://mw.brookes.ac.uk/display/hearoc
Judged to have wider relevanceA research observatory for Higher Educatione-Learning, Widening Participation, Employer EngagementOther ‘strands’ may be added - although the strands may not be explicit represented in the final structure
Overview of development
• August 07 – July 08• Scoping the Observatory• Landscaping report for e-learning (exemplar area)• Generation of pilot resources and services• Proof of concept piloted at Academy conference
• July 08 – July 09• Development phase• Pilots focusing on community engagement• Wider consultation• Promoted at Academy conference July 09
Landscaping evidence use in e-learning
Series of exploratory studies within e-learning Beetham, Sharpe & Benfield
‘Landscaping’ consultationInterviews with key informantsSurvey (116 responses)Follow-up interviewshttps://mw.brookes.ac.uk/display/hearoc/
Yes, but…
Keen on single point of access; research reviewed, evaluated and synthesised
After that, great variability
“The impossibility of categorising respondents as users, producers, policy makers or intermediaries for research is in itself is an important outcome.”
HB: What kind of research or evidence should a research observatory focus on?
I: Evidence that e-learning really works.
HB: What would that evidence look like?
I: It would need to show real improvements to learning outcomes, across a large number of students. It would have to have credibility and rigour.
HB: Can you think of an example of research evidence of that kind?
I: Not off the top of my head, no.
HB: So does this research really exist?
I: No, the observatory would have to fund it. OR Yes, it is out there, the observatory will have to work really hard to find it.
HB: Can you think of a situation when evidence like this has really changed people’s practice or understanding, in your experience?
I: Well, the cynics always ask for evidence that e-learning really works.
HB: Do you think that evidence, if you had it, would lead them to change their minds?
I: No, they would find arguments against it from their own discipline perspective.
HB: So what about people who are actually open to change?
I: They never ask for evidence. They ask for examples, especially from their own subject area, and practical ideas. They are really responsive to other people in their discipline who have tried something and made it work.
Identified strategic choices, approaches and risksStrong persuader or neutral observer? (Setting agendas)Funder?Push or pull communications?Quality assured, selective research or evidence and examples?Audience: researchers, intermediaries, practitioners or policy makers?Building authority or democratic knowledge? (Who can write?)Central or local knowledge management?Face-to-face or technology supported networks?
Building from this: e-Learning pilot
How do communities produce, share and use evidence?QA/QE in e-learning SIGExpert review about e-portfoliosDiscussion about professional role (M25 Learning Technologists group)Open peer commentary on the national development programmes
Review processes, to identify approaches that may have wider valueFeed these back to inform the development of the whole observatory
Echoed conclusions from IoE projects
Resources provided for people were not necessarily (ever?) taken up
Communities rise and fall
Expert review uninviting to others (useful, but not an invitation to contribute)
Open peer commentary engaged invitees then stopped
Pilot site for preliminary consultation
Building from this pt2: wider consultation
Inviting contributions from wider groups
Employee learning and widening participation communities
TLRP conference workshops
ELESIG meeting (Thursday)
Case studies with communities (Heads of e-Learning Forum)
Input into specifying the Observatory; documented cases of evidence generation and use
Building from this pt3: e-Learning projects
Small grants for research
Projects awarded, visited once, conclude by generating reports
New model
Projects awarded, brought together, given technical infrastructure (wiki, Ning)
Will be visited, encouraged to use Web2.0, brought together mid-project and again at end
Wiki for Academy-funded projects
However…
Continued visibility of central resource
Yes, it’s a repository… but it’s not just a repository
If it’s just a website it’ll be pointless
Invitation to engage does not guarantee engagement
And what next?
New round of projects (provisionally)
Smaller-scale reviews
Special Interest Groups
Possibly projects about technology to support evidence-informed practice
Emphasis on engagement (I hope)Wiki-based reviews for open engagement
Social networks – finding people, not just research outputs
Conclusions?
Scepticism about centralised determination of evidence and method Authoritative or authoritarian?
Interested in community production and use of evidence
Community-owned tools and spaces as a worthwhile experiment
Some possible starting points for discussion
A personal bias towards a social account of negotiating the meaning of encounters with research evidence
How credible is the approach represented here?
Is this approach to evidence the right one?
How should the choices outlined earlier be responded to?
Will this be able to help change practice?
How does this relate to you and your practice?
What kinds of decisions do you draw on explicit evidence for?
How could these be supported?
How could you share your conclusions?
What kinds of synthesis or structuring would be most useful and credible?
Emails - [email protected]
More information at:http://www.heacademy.ac.uk/ourwork/research/observatory
Piloting site:http://academy-research-observatory.pbwiki.com/