implementing quality elearning in higher education: change efforts, tensions and contradictions

IMPLEMENTING QUALITY ELEARNING IN HIGHER

EDUCATION: CHANGE EFFORTS, TENSIONS

AND CONTRADICTIONS

Patrizia Ghislandi – Juliana RaffaghelliUniversity of Trento

5th International Conference of Education, Research and Innovation

Madrid, 19-21 November 2012

Rationale

A) eLearning and change in higher education: tensions and contradictions

ELearning evolution and its contribution to Higher Education change What is eLearning quality?

Quality as a complex object

B) The PRIN project case: “Evaluation for the improvement of educational contexts”

The Methodological Approach

First Findings from an exploratory phase

C) Conclusions

Mediated quality. deep understanding, reflection and contextualized design

3

Web 2.0 and Social Media

Are Universities fully using the power of technologies to rethink pedagogical practices?

4

Dimensions of change

The initial media revolution (1.0)

2.0 Future

Teachers Sage on the Stage

Guide on the Side

The orchestra director: harmonizing the generation of content

Pedagogical Practices & Learning

Individual assignments on given contents

Collaboration Open Knowledge and flexible social networking: networked learning

Institution campus Interinstitutional Cooperation

Beyond local and institutional barriers

Students Passive role Initial sense of being part of the educational process

Co-creation of contents

University: from tradition to new pedagogical horizons

eLearning , Key piece of Pedagogical Change

We all agree but… (O’ Hearn, 2000; Holley, 2000; Volery, 2000; Rosenblitt, 2006; )

the differential infrastructure and readiness of different types of higher education institutions to utilize the technologies’ potential ;

the extent to which the ‘old’ distance education technologies and the new technologies replace teaching/learning practices in classrooms ;

the role of real problems, barriers and obstacles in applying new technologies ;

the impact of the new technologies on different student clienteles ;

information acquisition vs knowledge construction in higher education ;

cost considerations ; the human capacity to adapt to new learning

styles in face of the rapid development of the technologies ; and

the organizational cultures of academic and corporate worlds.

Unprepared Institutions

Organizational Cultures

Pedagogical Beliefs

Infrasatructure

Digital Skills

The reality today: Open questions…

How can we conceive and introduce QUALITY Elearning in Higher Education (QeLHE)?

Access and prevention of drop outs with more people following studies at University level;

• Transparency and usability of teaching contents;

• Open relationships with the society and the world of work to improve young students’ transitions,

• Wider access through the use of eLearning;

Building on Laurillard, 2002

The problem of defining quality

Diverse Cultures of Quality are underpinned by diverse values: Exceptional/Original: the

value is on the uniqueness

Distinctiveness: not for all

Excellence: The highest levels of performance

Fitness for Purpose: doing what has been planned

Inclusive: all people can participate 7

8

Studies on Quality of eLearning in HE

Q UNESCO Quality for all

EFQUEL –UE-

SLOAN-C MODEL –USA-

CENTRO VIRTUAL PARA EL DESARROLLO DE ESTÁNDARES DE CALIDAD PARA LA EDUCACIÓN SUPERIOR A DISTANCIA EN AMÉRICA LATINA Y EL CARIBE

ISO/IEC 19796

Sistematic ApproachesAccess?Excellence?InnovationInclusiveness?

9

Quality of eL in HE is a Complex issue

Elements Dimensions

Multiperspective The teacher – the student – the institution, the evaluators

Diverse Methods of Analysis

Benchmarking – guidelines – standards – quantitative or qualitative approaches

Diverse Time In itinere – ex ante – ex post

Diverse Meanings Pedagogical – Organizational – Technological – Economical

Diverse Levels of Analysis

Individual – Group – Institutional – Socio-cultural

Reconceptualizing Quality of eLearning

Quality is not an intrinsic, universal value

It is very much about the

methodology of evaluation, And the substantial epistemological principles and values underlying the process of evaluation

Evaluation process and underlying values

The selection of qualitative methods, a phenomenological approach based on narrative self-evaluation, peer-

evaluation and meta-evaluation, emphasizes the interest on processes and on the

empowerment of participants AS COMMITTED EVALUATED This logic studies the topic within its context, uses an emerging design

that accounts for reality as subjective and multiple, lessen the distance between “official” evaluators and participants (Denzin & Lincoln, 2011).

As a result, the evaluation process encompasses a

transformational (participatory/innovative) opportunity for the engaged individuals/

institutions.(Creswell, 2007; Mertens, 2009).

A transformational perspective

The transformational perspective is the kernel of a quality learning culture:

a human group that take part of a learning experience as a deep, reflective experience, connected to the own professional/ personal identity ▪ For which purposes do I learn? What can I do with this

learning? not just for accomplishing activities, recalling

information, and obtaining credentials (course diploma).

Teachers and students should become insiders of the culture of quality.

The PRIN project case: “Evaluation for the improvement of educational contexts”

PRIN (Projects of National Relevance, Ministry of University, Education and Research of Italian Republic) project 2009 “Evaluation for the improvement of educational

contexts. A research involving University and local communities in the participatory development of innovative assessment models”

PRIN first exploratory phase: Analysis of the institutional culture (values, meanings,

beliefs) about quality in HE; Analysis the stakeholders approach to quality of HE and

in this context, to quality of eLearning in HE.

• Understanding QeLHE at different levels

Preliminary Analysis

• Conflicts and contraddictions in the discourses about QeLHE within the University

Exploratory fieldwork • Pedagogical

Innovation, participatory evaluation

Design & Interventio

n

• Joint analysis on the educational impact of the quality model

Reflection & Feedback

14

• Methodological Approach: Case Study, Participatory Action Research• Main sources of data: documents, interviews, forum analysis,

observations, design workshops, use of tools for design (Conole, 2012)• Method for data analysis: • Exploratory phase: Discourse analysis, Semio-pragmatic Analysis• Transformative phase: design based research (DBR)

PRIN research design

• Understanding QeLHE at different levels

Preliminary Analysis

• Conflicts and contraddictions in the discourses about QeLHE within the University

Exploratory fieldwork • Pedagogical

Innovation, participatory evaluation

Design & Intervention

• Joint analysis on the educational impact of the quality model

Reflection & Feedback

15

PRIN research design

Subphase 1 (A1-2 / A8-9)

Subphase 2 (A3-A6 / A10)

Exploratory PhaseDocumental/Web Analysis Interviews / Observation

3 Courses A.A. 2011/2012 6 StudentsMethod: Case Study/PAR Near 500 students 6 Academics 3 Academics 1 NVA 1 Support to Didactics 4 Instructional Designers 2 eTutorsTransformative Phase Planning Interventions InterveningMethod: dbr 4 "Learning Design Workshops" 3 Courses A.A. 2012/2013 Near 500 students 3 Academics 2 eTutors

The PRIN project case – First Exploratory phase “Evaluation for the improvement of educational contexts”

Focus on the critical tensions and contradictions within the institution between the several stakeholders to implement concrete practices linked to the own vision of educational quality

Methodological approach Documental Analysis Semio-pragmatic Analysis Interviews Triangulation and Member-checking

Results: Tensions and Contraddictions regarding the perspectives on QeLHE

Perspectives on eLearning inside an academic course Students

More than having access to materials: I didn't think they (materials) are very useful, I had other files from other classes and other friends, they are giving me some other stuffs to help me learn [S3]

Something very complementary: Due to my way of learning, I like to follow the teacher’s lessons, then I organize my study. I don’t like technologies very much. It is ok if I can just receive communications, or have access to materials (…) If I can choose, I take the FTF course. [S1]

Better possibilities of communicating among students and the teaching staff: the teacher has to generate a sense of continuity between the FTF activities and the online. I like that. Clearly a teacher with 100 students cannot do this very well. But this year there was another assistant (the eTutor) that accompanied us and it was very good [S2]


Perspectives on eLearning inside an academic course Teachers

A way of following the institution approach to learning: Our university is not outside of the times, and eLearning came to stay. So it is better to tackle the issue and be prompt to do what is our duty.[T2]

Facilitating the access to materials: well, I don’t use eLearning in an advanced way; I have to recognize that it has facilitated the delivery of materials, but I never adopted collaborative ways, for I like to work FTF if I can. So eLearning helps me to qualify my course in this sense [T4]

Opening to continuing pedagogical innovation: eLearning is like a Trojan horse…you introduce the technological frame, then you start to rethink all your teaching practices and in the end the nature of the knowledge that you teach. This should be a never ending process [T1]


Perspectives on eLearning inside an academic course

Instructional Designers

Technologies are not all in the implementation and quality of eLearning: in our initial phases of implementation of a project to introduce eLearning at academic level, we adopted different technologies. The eLearning platform adopted till today represented an important frame to support teachers in their way of working with eLearning. But this is not all, this is (and continues to be) the excuse to rethink the way of teaching (the pedagogical approach, our comment) [ID1]

An invisible role: our role must be invisible, must be a base and a springboard for the teacher that wants to adopt eLearning (…) but it would be better if it was better recognized (…) no teacher likes to be told how to teach. The problem is that in eLearning, the deep knowledge of your subject do not necessarily take to the good delivery of online activities (…) sometimes we are seen as the “text editors” [ID2]

Institutional context matters: the political context in the institution clearly addresses what we can do or not in order to promote eLearning and the renew of teaching methods [ID1]


Perspectives on eLearning inside an academic course Academic Secretariat

eLearning is the last concern in a process of quality evaluation in HEI: I never really care about eLearning, even when I understand it importance. I see the importance of technologies in what I do every day with students, but to me (…) there are other important issues to solve. To be part of the Bologna process, with the Dublin indicators (…) we have to change the way we evaluate students (…). I think that an important position to implement this process is that of the coordination of academic courses, but now the role is overwhelmed of bureaucracy and the academic in charge cannot dedicate too much attention to institutional change.


Perspectives on eLearning inside an academic course External Evaluator

An excellent researcher is a good teacher. If you do real research and you are an excellent researcher, you are able of being an excellent teacher.

The recognition of the teaching activity in HEIs: academics are not really recognized by their teaching activity. Research counts, not teaching, and teaching is a heavy work that they are not always open to focus properly if it takes time from research.

Technologies can help the communication of your research field into your teaching. I’m not an expert of eLearning, of course I recognize the value. The technologies in my field of teaching are important to show concepts/practices that in today’s crowded universities you cannot always present.

Results: Contradictions

eLearning has a very different and rather contradictory status among the interviewees.

The main contradictions regard the dimensions that matter for a overall quality culture in HEI: some stakeholders concern is on the policy

context and institutional change other claim for the recognition of eLearning as

field of practice that is evidence based Concern on innovation vs. concern on tradition

Conclusions: Quality of eLearning is implemented depending on personal positionings

Personal positioning

Approach Interest on eLearning

Practices Quality values

Outsiders of QeLHE (1)

Sense of duty with regard to a model that it is being implemented generally at the University

Secondary issueNot aware

Implementing only official programmes (*);

TraditionOutcomes

Outsiders of QeLHE (2)

Solving specific problems on current practices

eLearning as support of what we already do

Delivery of content facilitated by eLearning platforms

TraditionOutcomes

Insiders of QeLHE

Clear personal conviction on innovation, evidence based driven

eLearning can be a mean to transform pedagogy

Experimenting with eLearning

TransformationProcess

(*) In Italy there are only very few regulations regarding eLearning

Conclusions: Mediated quality. deep understanding, reflection and contextualized design

Raise awareness on the context as well as specificities of pedagogical innovation within higher education could lead to the harmonization of quality discourses.

We call this operation mediation of quality: quality. From a socio-constructivist approach: means offering

tools that would support the processes of negotiating the many values lying behing a quality culture

Tools mediate learning of stakeholders to pass from a position as outsiders of quality to a position as insiders, or active agents of change.

For a Quality approach based on design: making visible the invisible

On Learning Design

making the design process more explicit and shareable (…) help learners to make more sense of their educational provision and associated learning pathways. (Grainne Conole 2011)

On the incommensurability of Quality

Quality….you know what it is, yet you don’t know what it is. But that’s self-contradictory. But some things are better than others, that is, they have more Quality. But when you try to say what Quality is, apart from the things that have it, it all goes poof!. There’s nothing to talk about. But if you can’t say what Quality is, how do you know that it even exists? If no one knows what it is, then for all practical purposes it doesn’t exist at all. But for all practical purposes it really does exist (Robert Pirsig. 1974)

Thank you for your interest!

For communications:

[email protected]