Using design-based research (DBR) in the mobile learning

context

Agnieszka (Aga) Palalas, Ed.D.

apalalas@georgebrown.ca

1March 2013

1. DBR overview

2. MELLES case study

3. DBR implications, limitations and recommendations

4. Conclusions

5. Discussion

Agenda

2

3

DBR methodology

“Design experiments” > design-based research = design research = DBR = EDR = researching innovative educational designs in their naturalistic settings

(Brown, 1992; Collins, 1992)

A systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories.

(Wang & Hannafin, 1999, p. 7)

4

ILDF model: Integrative Learning Design Framework (Bannan, 2009)

Informed Exploration

Enactment Evaluation:

Local ImpactEvaluation:

Broader Impact

Substantive knowledge + procedural knowledge

5

Design principles

If you want to design intervention X for the purpose/function Y in context Z,

then you are best advised to give that intervention the characteristics A, B, and C [substantive emphasis], and

to do that via procedures K, L, and M [procedural emphasis], because of arguments P, Q, and R.*

(Van den Akker, 1999, p. 9)

Multi-cycle process of DBR ⇨ design principles

What are the characteristics of an effective,

pedagogically-sound MELLES for students’ mobile

devices, through which adult ESP students in a community

college enhance listening skills, while expanding their

learning outside the classroom?

6

Research question

What are the characteristics of an <intervention X> for the purpose/outcome Y

(Y1, Y2, …, Yn) in context Z*

(Plomp, 2009, p. 19)

Practice research

Drawing on engineering and technological research

Focusing on design, construction, implementation and adoption of learning solutions

Process focused and iterative

Interventionist: applied solutions to real educational problems

Contextual: real people, context, and cultural background, in-situ investigation/evaluation

“Action Research on steroids”

DBR

Applied

Participatory

Collaborative

Utility oriented

Theory driven

Unifying theory and practice

Evolving

Systemic

DBR

Rich feedback Responsive Flexible

Agile

BUT

Multileveled and multifaceted

Complex

Messy

DBR

Co-developing theory and design in-situ

Testing in real-world learning situations involving all actors/end users

With people for people

End-users as partners (active co-creators)

Combination of motivation and efforts

Students as researchers

Diverse evaluation perspectives

Participatory 1/2

“We are moving away from a passive information age towards an active participation age” (Farmer & Gruba, 2006, p. 149)

Diverse expertise of practitioners and researchers

Collaboration and support

Network of colleagues

◦ Flexible dialogue from whenever

◦ Online platform and tools (e.g., Wiggio, Collaborate)

Dynamic feedback from actors

◦ at milestones and agile

Participatory 2/2

Context is king

Evolution of theory

Evolution of practice

Benefits of DBR (m-learning)

Build foundational understanding of m-learning trends and needs

Gain insight into learner m-learning behaviours

Generate inspiration and ideas for appropriate design

Define what functionality is most critical

Improve the usability of infrastructure

Inform future design solutions

Provide professional development

Raise awareness and understanding of m-learning

More benefits of DBR (m-learning)

14

Mobile-Enabled Language Learning Eco-system (MELLES)

What are the characteristics of an effective, pedagogically-sound MELLES for students’

mobile devices, through which adult ESP students in a community college enhance

listening skills, while expanding their learning outside the classroom?

Evolution of theory• MELLES design principles• Ecological Constructivism

Evolution of practice• MELLES prototype• Model for replication

DBR application for educational context(Palalas, 2012)

15

Statement of the problem - Evolution

Benchmarking Study (2005-2007)

M-learning Exploratory Study (2007-2009)

MALL Design Pilot (Jun-Aug 2009)

DBR Pilot (Sep 2009–Jan 2010)

MELLES DBR Study (Jun 2010–Dec 2011)

2005

2007

2011

Social Constructivism --> SCT -->

Ecological

Constructivism

2009

2010

16

Inadequate aural skills instruction - college ESP students

Purpose: MELL educational intervention to enhance effectiveness and appeal of ESP◦ augment in-class learning◦ out-of-class language practice◦ students’ own mobile devices◦ replicable and reusable design principles

Statement of the problem - Final

What are the characteristics of an effective,

pedagogically-sound learning object MELLES for

students’ mobile devices, through which adult ESP

students in a community college enhance listening

skills, while expanding their learning outside the

classroom?

Research question

17

18

Methodology: DBR

• Bannan, B. (2009)• Barab, S., & Squire, K. (2004)• Brown, A. (1992)• Dede, C. (2004)• Herrington, J., McKenney, S., Reeves, T., &

Oliver, R. (2007)• Kelly, A. (2009)• Plomp, T. (2009)• Reeves, T. (2006)• Van den Akker et al (2006)• Wang, F., & Hannafin, M. J. (2005)

19

Informed Exploration

20

Critical Elements of Effective Design:

Pedagogy PEDAGOGIC PROCEDURE - How CONTENT - What CONTEXT - When and Where ACTORS - Who

Technology FUNCTIONALITY - How TECH SOLUTION – What TECH CONTEXT - When and Where

Findings: Emerging Themes

21

Findings: Emerging Theoretical Framework

Ecological Constructivism

• Social Constructivism + Sociocultural Theory + Ecological Linguistics

• “the recent metaphor of ecology attempts to capture the interconnectedness of psychological, social, and environmental process in SLA” (Lam & Kramsch, 2003, p.144)

• Affordance: “ … a particular property of the environment that is relevant – for good or ill – to an active, perceiving organism in that environment” (van Lier, 2000, p. 252)

22

Enactment

23

Results

24

Evaluation: Local impact

DBR: Implications

The intervention “embod[ied] specific theoretical claims about teaching and learning, and reflect[ed] a commitment to understanding the relationships among theory, designed artifacts, and practice. […] research on specific interventions can contribute to theories of learning and teaching.”

(Design-Based Research Collective, 2003, p.6)

(Palalas & Hoven, 2013)

26

DBR: Limitations The scope of DBR

• complexity of the system – breadth• coordination of research and design activities• no objective measure of learning• amount of data• consensus-reaching and interaction among actors • intensity

The role of the researcher• multifaceted• conflicting roles• threats to validity

Transferability of findings

DBR: Recommendations Rigorous data collection/analysis procedures Regular communication Findings documentation and reporting Sound conceptual framework Clear deadlines and deliverables “Flexible” (academic schedules) Solid project management Collaborative DBR research team

“If a researcher is intimately involved in the conceptualization, design, development, implementation, and researching of a pedagogical approach, then ensuring that researchers can make credible and trustworthy assertions is a challenge”

(Barab & Squire, 2004, p.10)

Project context

Objectives

Audience

Location

Methods

Schedule

Outcomes

Plan well

Thank you!

Q&A

29

apalalas@georgebrown.cahttp://mobi-learning.com/ Twitter: @agaiza LinkedIn: http://www.linkedin.com/in/apalalas Presentations: http://www.slideshare.net/agaiza Publications: http://georgebrown.academia.edu/apalalas

Bannan, B. (2009). The Integrative Learning Design Framework: An illustrated example from the domain of instructional technology. In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp. 53-73). SLO: Netherlands Institute for Curriculum Development.

Barab, S., & Squire, K. (2004). Design-Based Research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1-14. doi:10.1207/s15327809jls1301_1

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of the Learning Sciences, 2(2), 141–178.

Collins, A. (1992). Towards a design science of education. In E. Scanlon and T. O’Shea (Eds.), New directions in educational technology (pp. 15–22). Berlin: Springer.

Dede, C. (2004). If Design-Based Research is the answer, what is the question? The Journal of the Instructional Sciences, 13 (1).

Design-Based Research Collective. (2003). Design-Based Research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5-8. Retrieved from http://www.designbasedresearch.org/reppubs/DBRC2003.pdf

Farmer, R., & Gruba, P. (2006). Towards model-driven end-user development in CALL. Computer Assisted Language Learning, 19(2 & 3), 149-191.

Palalas, A. (2012). Design guidelines for a Mobile-Enabled Language Learning system supporting the development of ESP listening skills (Doctoral dissertation, Athabasca University). Retrieved from http://hdl.handle.net/10791/17

Palalas, A., & Hoven, D. (2013). Implications of using DBR to investigate the iterative design of a mobile-enabled language learning system. CALICO

Plomp, T. (2009). Educational design research: An introduction. In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp. 9-36). SLO: Netherlands Institute for Curriculum Development.

Reeves, T. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.), Educational design research: The design, development and evaluation of programs, processes and products (pp. 52-66). New York: Routledge.

Van den Akker, J. (1999). Principles and Methods of Development Research. In J. van den Akker, R.M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1-14). Boston: Kluwer Academic.

Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and Development, 53(4), 5-23.

References

30