methodological issues in research on moocs - ecel2014
DESCRIPTION
The raise of attention to the relatively new phenomenon of MOOCs has put them at the cutting edge of the debate on networked teaching and learning. Research on MOOCs seems to have overcome the exploratory phase, and is approaching a consolidation of themes and objectives. However, little attention has been paid to methodological issues in MOOCs research. Furthermore, the methodological approaches most widely adopted in this area could cast out conclusions, which should be reconsidered either from a critical theoretical point of view, or from studies of empirical replication. In this paper the authors have reviewed fifty-seven journal articles on MOOCs in order to analyze the methodological approaches most commonly adopted in this field of research. The results have been initially grouped, taking into consideration the traditional methodological classification: quantitative, qualitative, mixed-methods, design-based research, literature review, theoretical contribution. Furthermore, the methods adopted within the above mentioned approaches have been considered. In order to deepen on the understanding about the methodological approaches, the conceptual model “full cycle of educational research”, with its seven phases has been adopted to classify the several articles reviewed. On these basis, the authors analyze the “methodological trends” within the field of MOOCs. The purpose is to show gaps and criticalities as well as to suggest future directions for selecting methodological approaches in the field of MOOCs research. 13th European Conference on e-Learning ECEL-2014 Aalborg University, Copenhagen, Denmark 30-31 October 2014TRANSCRIPT
Methodological Issues in
Research on MOOCsJuliana Raffaghelli and Donatella Persico
Institute for Educational Technologies - National Research Council of Italy
ECEL201413th European Conference on e-Learning
30-31 October, Copenhagen, Denmark
Research assumptions and the development of a research field……are based on the «methodological lens» adopted
From: Mr. Tekhan, 2007 «Optika» - CC-BY-NC-SA 2.0 - https://www.flickr.com/photos/theklan/474695210/
What can we learn taking a look at the
methodological issues in a research field?
Which are the main research
problems relating to a vision of
the «universe»
Which are the values that are
assumed as the base for quality
research
Which are the instruments
selected to explore that
universe
research aims
research methods
methodological
approaches
Research on MOOCs
A hype (Young, 2013)
Beyond the exploratory phase
Consolidation of themes and objectives (Liyanagunawardena et al. 2013)
Two phases of Scholarship(Ebben & Murphy, 2013)
2009-2011 «Experimentation, Enthusiasm, cMOOC»
2012-2013 «Critical approach, xMOOC, Learning Analytics»
Little attention has beenpaid to the methodologicalapproaches and methodsin MOOCs research
From: Mathiew Plourde, 2013 «MOOC Poster» - CC-BY 2.0 - https://www.flickr.com/photos/mathplourde/8620174342/
A literature
review
Distribution of papers analyzed per year
Specialized databases consulted
Results I
Research Aims 2009-2013
8
17
7
8
1
4
7
2
3
0 2 4 6 8 10 12 14 16 18
(DLM) Design for Learning in MOOCs
(LPM) Learning Processes in MOOCs
(CMET) Contribution of MOOCs to educational theory
(MP) MOOCs' pedagogy
(TM) Technological tools for MOOCs
(TPM) Teaching processes in MOOCs
(MID) MOOCs for institutional development
(LR) Literature Review
(MASM) Methodological approaches to study MOOCs
Results II
Methodological Approaches 2009-13
6
12
2
7
6
11
4
9
0 2 4 6 8 10 12 14
Theoretical
Conceptual
Literature Review
Quantitative (exploratory, experimental, etc.)
Qualitative (Discourse analysis, narrative analysis,Digital ethnography, etc.)
Mixed Methods
Design Based Research
Not clear
(23)
Empirical
research
(18)
Non
empirical
Learning analytics, survey
and digital ethnography
Results III
Methodological Approaches 2009-13
0 0 0
2
15
2
1
0 0
2
7
6
0
1
0 0
2
4
0 0
2
0
5
3
0 0 0 0
3
1
0 0 0 0
1 1
0 0 0 0
2
0
2009 2010 2011 2012 2013 2014
Theoretical-Conceptual Descriptive-Exploratory/ QUAN-QUAL Descriptive-Case Study
Intervention-Case Study Intervention-Experimental/Design Experiment Intervention-Experimental/ Random Experiment
Literature Review
Results III
Methods 2009-13
8
3
20
2
6
3 3
16 16
42
43
2
20
31
2 2
0
5
10
15
20
25
Learn
ing A
naly
tics
Inte
rvie
ws
Surv
ey
Focus
Gro
up
Ele
ctr
onic
corp
ora
(fo
ra,
tweets
,so
cia
l netw
ork
s post
ing,
etc
.)
Mult
imodal Learn
ing a
rtif
acts
(ass
ignm
ents
, pre
senta
tions,
data
repre
senta
tion u
sed b
y learn
ers
,…
Experi
menta
l G
roups
Conceptu
al
Desc
ripti
on o
f dim
ensi
ons
Vir
tual eth
nogra
phy
Them
ati
c a
naly
sis
Dis
cours
e a
naly
sis
Socio
-pra
gm
ati
c /
eth
nogra
phic
/m
ult
imodal analy
sis
for
mult
imodal art
ifacts
Conte
nt
analy
sis
Desc
ripti
ve S
tati
stic
s
Sta
tist
ical corr
ela
tion
Facto
rial analy
sis
Infe
rencia
l Sta
tist
ics
Netw
ork
analy
sis
Data Collection Data analysis
Discussion
Little variation with regard to the methodological approaches and methods
adopted along the period 2009-2014.
The methodological debate is missed and a profusion of methods are used
jointly
Understanding the MOOCs research evolution from Gorard’s “Full cycle of
educational research”
11, 20%
17, 30%
5, 9%
17, 30%
4, 7%
2, 4% 0, 0%1) Evidence analysis
2) Development of idea or artifact
3) Feasibility Study
4) Prototyping and trialing
5) Field Studies and design stage
6) Definitive testing
7) Dissemination Impact and monitoring"
Conclusions Taking into consideration the
methodological issues, research on MOOCs is mostly collocated at the level of the first four phases of the “full cycle of educational research”.
Two challenges for the future:
“Methodological Tuning”
Evolution of the methodological approaches under the pressure of the specific needs of this research field
Other topics that should be analysedwithin the methodological debate:
Ethics (big data are available)
Internal consistence (connections between the research problem and the methods adopted)
Generalization (QUAN)
Educative/ontological/strategic value (QUAL)
Nullius in verba…
Time for discussion!
From: Duncan Hull, 2005 «Question Everything» - CC-BY 2.0 -
https://www.flickr.com/photos/dullhunk/202872717/