where technology enables knowledge the national ict research centre 1 m-learning – evaluating the...
TRANSCRIPT
1
Where technology enables knowledge
The National ICT Research Centre
m-learning – Evaluating the Effectiveness and the Cost
John Traxler
National ICT Research Centre
2Where technology enables knowledge
The National ICT Research Centre 2
Other presentationsDeveloper’s sessions:• Geoff Stead + Jo Colley Monday 12:00
Combined session on content and technology
Research Papers:• John Traxler Monday 14:30
Evaluating effectiveness and cost
• Alice Mitchell + Kris Popat Monday 16:00
The potential of games
• Jill Attewell + Carol Savill-Smith Tuesday 14:45
Focus on learners and learning
3Where technology enables knowledge
The National ICT Research Centre 3
Understanding Cost is Important• Profitability, Return on Investment
• for universities, colleges, trainers, schools
• The transition from print-based learning• perceived economies of scale, possible large up-front costs,
increased risks
• globalisation; competition; industrialisation
• increased training and staff development
• learning delivered by tools and teams
• quality, accountability, visibility
• changes in working practices
4Where technology enables knowledge
The National ICT Research Centre 4
Outline of Talk
• Predicting the Costs of Software• Predicting the Costs of Educational
Multimedia• Predicting the Costs of m-learning• Matching the Costs of m-learning to the
Benefits• The Real Difficulties
5Where technology enables knowledge
The National ICT Research Centre 5
Software Cost-Estimation
• Many different approaches, most dependent on local technology, history, environment; modest successes
• Attempts to calculate effort sometime before implementation and delivery
• Often based on some measure of program size e.g. KLOC or Function Points
6Where technology enables knowledge
The National ICT Research Centre 6
Basic COCOMO
• Basic version: effort depends of program size
• E = ab(KLOC) exp(bb)
• D= cb(KLOC) exp(db)
• N, number of people, =E/D
7Where technology enables knowledge
The National ICT Research Centre 7
Project ab bb cb db
Type
Organic 2.4 1.05 2.5 0.38
Semi 3.0 1.12 2.5 0.35detached
Embedded 3.6 1.20 2.5 0.32
Basic COCOMO
8Where technology enables knowledge
The National ICT Research Centre 8
Intermediate COCOMO• Intermediate version : effort depends
on program size and “cost-drivers”, each on 6-point scale from “very low” to “extra high“– product attributes
• reliability complexity etc
– hardware attributes• performance memory etc
– personnel attributes• experience capability etc
– project attributes• tools methods
etc
9Where technology enables knowledge
The National ICT Research Centre 9
Educational Software Costs - “rules of thumb”
• Nothing useful will happen in the first three to six months after you decide to go with[courseware]. It doesn’t make any difference whether you go with a vendor or start producing your own in-house (Lee & Zemke 1987)
• The first course produced by a new [courseware] development group will be a collection of mistakes. Throw it away. (Lee and Zemke 1987)
• Where team members are not used to working together or are geographically apart, add 10-15% [to the total effort] (Casey et al 1988)
• Analysis and design comprise 50% of total effort (Casey et al. 1988)• Even a skilled Instructional Design author will revise plus or minus
half the material after first or second draft (and then 20-25% in the third draft) (Casey et al. 1988) -cited by Marshall
10Where technology enables knowledge
The National ICT Research Centre 10
Some Industry Figures
11Where technology enables knowledge
The National ICT Research Centre 11
Multimedia Educational Software Cost-Estimation - Marshall
• COCOMO used on 14 projects in 1990’s• mainly 1 hour learner time (cf. KLOC)• 4 potential cost-drivers (24 sub-heads)
– course difficulty– development environment– subject expertise– interactivity
» Ian Marshall, Abertay University
12Where technology enables knowledge
The National ICT Research Centre 12
Multimedia Educational Software Cost-Estimation - Marshall
• Significant cost-drivers so far– development environment
• instructional design method (+ 4 more)
– course difficulty• number of objectives
• level of objectives
• existing course material
13Where technology enables knowledge
The National ICT Research Centre 13
Gagne’s Media-Mix
14Where technology enables knowledge
The National ICT Research Centre 14
15Where technology enables knowledge
The National ICT Research Centre 15
Laurillard’s Model or Framework
teacher’s world
delivery
discussion
adaptationreflection
descriptions(theory)
actions(practice) interaction
16Where technology enables knowledge
The National ICT Research Centre 16
print, lecture, Web pages
teacher’s world
theory
practice
delivery
17Where technology enables knowledge
The National ICT Research Centre 17
seminar, conferencing
teacher’s world
discussion
theory
practice
18Where technology enables knowledge
The National ICT Research Centre 18
laboratories, fieldwork
teacher’s world
theory
practice
interaction
reflection?
19Where technology enables knowledge
The National ICT Research Centre 19
teaching package
teacher’s world
delivery
discussion ?
adaptation?
theory
practice
Where technology enables knowledge
The National ICT Research Centre 20
20
VLE courses
workshops
theory
practice
course, noticeboard
group folder,assignments,chat, forum, e-mail
set tasks
agree goals, do tasks
feedback, experience
chat
discusstasks
apply theory
Where technology enables knowledge
The National ICT Research Centre 21
21
m-learning courses
exercises
theory
practice
iPAQ
mPortal, SMS
develop exercises
iPAQ
SMS, mPortal
mPortal
discusstasks
apply theory
22Where technology enables knowledge
The National ICT Research Centre 22
Practical Strategies
• Course Resource Appraisal Model– implemented as Excel spreadsheet
– Open University course resource planning and management tool
– based on Laurillard’s work
– looks at student workload and author workload across all media options
• Media Advisor– also based on Laurillard’s work but simplified
– developed at UNL by Martin Oliver & Grainne Connole
– aimed at individual lecturers to be used iteratively
– in public domain, on CD
23Where technology enables knowledge
The National ICT Research Centre 23
24Where technology enables knowledge
The National ICT Research Centre 24
Media Advisor
25Where technology enables knowledge
The National ICT Research Centre 25
Media Advisor
26
Where technology enables knowledge
The National ICT Research Centre
m-learning in an Imperfect World
Some of the Real and Hidden Costs
27Where technology enables knowledge
The National ICT Research Centre 27
Constraints to m-learning• Hands-on - computing, music-making, workshop
• In vivo - medicine/dentistry/nursing/veterinary, field trips (perhaps)
• Interpersonal - interview skills, presentations
• Social - team-work (perhaps), business, marketing
• Expressive - ballet, dance
• Using Tools and Machines - engineering
• Laboratory Use - science (perhaps)• ( and exams, assessments, vivas etc)
28Where technology enables knowledge
The National ICT Research Centre 28
Costs to m-learning Students
• 75% of (undergraduate) students own PC, 29% on internet, nearly 100% have ‘mobiles, almost none have PDAs
• time seen as main cost by (undergraduate) students (but is this location-dependent?)
29Where technology enables knowledge
The National ICT Research Centre 29
Costs to Students
• Online learning(!) material was often printed– Text of practicals 33%
– Discussion of practicals 31%
– Web pages 45%
– Conference messages 54%» PLUM Report No. 122
• Would m-learning students want to do the same or can they read from PDAs?
30Where technology enables knowledge
The National ICT Research Centre 30
Students’ Preferences
“Which of these learning tools do you prefer to use?”
– books 67%
– lectures 36%
– videos 36%
– computers 19%• Campaign for Learning, 1996 - predates PDAs!
Why these preferences?
What will be preferences of m-learning students?
– audio-tapes 11%
– internet7%
– none/no pref. 3%
– none/don’t want to learn 3%
31Where technology enables knowledge
The National ICT Research Centre 31
Student Use of PC-based CMC
• Limited active participation– participate : lurk
» 40: 60
» 30: 70
» 10: 90
• varied strategies to improve these– based on postings to “teaching online” mailbase, 1999
– see Gilly Salmon’s work @OU
• Can m-learning turn lurkers into participants?
32Where technology enables knowledge
The National ICT Research Centre 32
Costs to m-learning Students
• Forms of disadvantage• dyslexia; visual impairment
• not confident with IT or English
• less affluent
• “wrong” cognitive or learner style
• New forms of disadvantage• bandwidth poverty
• interface poverty (wrt PCs)
33Where technology enables knowledge
The National ICT Research Centre 33
Thanks for your time!
My email [email protected]
Samples are at www.ctad.co.uk/m-learning
m-learning project is at www.m-learning.org