evaluating a mobile toolkit for designing mobile learning activities

Dr David Parsons | CITRENZ 2017

Evaluating a Mobile Toolkit for Designing Mobile Learning

Activities

Kathryn MacCallum and David Parsons

Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017

● Development and testing of Mobile Learning Activity Design Analyser

● First cycle of 6 stage design science process from motivation through to evaluation

● Key features of the analysis tool● Data gathered from use by a small group of educators● Relationship between learning theory and mobile

learning activity design

Agenda


● Mobile Learning Activity Design Analyser● Analyses a learning activity design by responding to a

set of statements about its features● Output is a visualised and aggregated graphic

describing how the learning activity leverages key aspects of specific learning theories

● Alongside this graphic there are some suggestions on how the overall design of this activity might be improved and developed further

The Tool


● Behaviourism● Constructivism● Experiential learning● Situated cognition● Communities of practice● Connectivism

Six Learning Theories


The phase of work described here is the first cycle through a 6 stage design science process 1. Problem identification and motivation2. Define the objectives for a solution3. Design and development4. Demonstration5. Evaluation6. Communication

Peffers, K, Tuunanen, T, Rothenberger, M. & Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of management information systems, 24(3), 45-77.

Methodology


1. Problem + 2. Solution Objectives

1. ProblemSupport is needed to integrate learning theories into mobile learning activity design

2. Solution objectivesCreate a simple software tool to provide some guidance for how learning theories can be integrated into mobile learning activities


3. Design and Development

Design requirements● Targeted feedback to the user based on inputs● Should run on mobile devices (modelling)

Development● HTML5 + JavaScript● JQuery UI● Slider Pips● ChartJS


4. Demonstration

● The app itself is partly covered in our IAmLearn eBook chapter

● This presentation is about the evaluation rather than the app itself

● However we will run a quick demonstration

tinyurl.com/manalyser

http://tinyurl.com/manalyser


Participants

● 15 educators, experienced in delivering mobile learning activities, were invited to participate voluntarily and anonymously in the study

● 7 agreed to participate● 5 were able to complete all four stages of the study


4 Part Process

Part 1: Participants were asked to outline a mobile learning activity (targeted learners, learning objective(s), description, tasks, mobile features, types of learning)Part 2: Participants used the Mobile Learning Activity Design Analyser toolPart 3: Participants used the feedback from the tool to modify their activity designsPart 4: Participants used the tool again, this time to reflect on their design modifications


5 Learning Design Activities

1. Does 'fitbit' monitoring increase user motivation?Identify motivators in the FitBit phone app

1. Place-based learningCritical reflections of place using video

1. Introduction using geo-tagged videoNew distance students sharing videos

1. Maths problem solving via site tripTravel to the zoo - logistics analysis

1. The returning soldierANZAC memorial sculpture trail


Integration of learning theories into the initial designs (left) and the final designs (right)

5. Evaluation (4 complete data sets)


Perceived Usefulness


+ Feedback● “Think it is a really useful tool.”

● "The real 'gain' I got from it was from the first cycle

where it suggested where I could increase certain

learning theory content/processes. This allowed me to

reflect on the 'feedback' score and ask myself 'did I

really want the activity to go this way?' So, by getting

the first feedback I could see areas where I genuinely

thought I could go back and add some more detail.”


- Feedback● “I suppose it 'assumes' that an activity should have

equal portions of each learning theory? I am not sure I

agree with that...”

● "I almost needed to do this a third time."

● “need more support on actual mobile learning design.”

● “would need a bit more info about some of the

meanings of the terms.”


Conclusion

● A way to more authentically engage in evaluating a mobile activity

● Dynamic nature of the tool enables a more visual and interactive engagement

● Makes the educator more aware of the design of the application

● Enables them to critically evaluate and provide suggestions to improve the design (based on appropriate learning theory)

evaluating a mobile toolkit for designing mobile learning activities

Education