mobile phones in language education
DESCRIPTION
Presentation of Anglia Pilot on Mobile Phones in Language Education for GloCALL 2010 conferenceTRANSCRIPT
Using mobile phones for the assessment of and for oral skills development in
secondary education.
Ton Koenraad
Hogeschool Utrecht, University of Applied Sciences, Faculty of Education
TELLConsult
Overview
Project context & goals
Pre-Pilot: set-up & research questions
Data collection
Results & conclusions
Next phase
English for Kids Foundation
“Voice for children”
A, not -for-profit organisation
" Voice for children "
Every kid has a right to communicate.
English is a good tool to communicate internationally.
The English for Kids Foundation wants to promote the use of English
to communicate internationally.
We want to give children 'a voice'.
Projects are to meet the
following requirements:
> long term target (3-5 yrs)
> aimed at children
> close co-operation with
local organisations in the
project countries so that
activities can be embedded
and be allowed to proceed
EFKF projects:
India
Nicaragua Very Young
Learners
South Africa Elandsdoorn
Kenia Teacher support
Surinam EDS
Gobabis Namibia
Cambodia
www.efkf.org
www.englishforkidsfoundation.org
Anglia Network Europe www.anglianetwork.eu
15 European countries
International context
Step by step
- 28 January - 15 April - 20 May - 24 June - Speaking Tests
Examinations
Why this Project?
Anglia: flexibility / assessment of - sharing innovative speaking practice materials - delivery of formal language assessments
English for Kids: mobile as infrastructure, & schools assessment for: washback effect of testing
Issues to be researched
general organisation
task and test design,
teacher competences
face & content validity aspects
system usability
Research design / instrument development
Why use a phone?
Natural interface. Widely available. Familiar to students. Technology used as a tool – not “accessorizing education”. Also available on iPod Touch & Skype.
20 20
How it works
Teachers:
Personal online workspace to set spoken
questions/tasks.
Questions are easy to set online by using a
microphone or uploading audio files.
Audio player allows teachers to review spoken
work and leave feedback.
Over time teachers establish a digital portfolio
of student work.
21
How it works
Students:
Connect using:
Mobile phones
iPod Touch
Skype
Landline
Computer
Access spoken exercises & leave voice responses.
Connect with other students for role play.
Personal online workspace to store work, listen & receive
feedback.
Listen to exemplar and sample questions posted by teachers.
22
Pilot
2 secondary ed. EFL teachers
2 Anglia member schools
Volunteer students (n= 20)
Assessment: asynchronous, interview format
Oral presentation skills
Data Collection
Instruments Pupils Teachers
Pre-Questionnaire
+ +
Recordings + Test scores
+ +
Post-Questionnaire
+
Structured Interview
N=2 +
Reflections Developers
Pre-Questionnaire: Learner Profiles (1)
Aspect Group A Group B
Group size 12 8
Girls 5 4
Boys 7 4
Avg. Age 13.5 13.5
Years of English 3 3
Attitude to Learning English
Fairly positive
Positive
Average score at Secondary
6.37
7.6
Pre-Questionnaire: Learner Profiles (2)
Aspect Group A Group B
Speaking Skills: (Self reported)
Fairly good Good
Like speaking in class
So, so Definitely
Actual speaking hardly Very frequently
Telecollaboration at school
n/a Slightly more than once
Tel. Experience in projects
n/a
Very occasionally
Tel. Experience IRL
seldom seldom
Pre-Questionnaire: Learner perceptions: L2 in class & IRL
Aspect Group A Group B
L2 in lessons Once in 3 lessons
Practically every lesson
Answers /Discussion Only now and then
(very) frequently
Pairwork Hardly ever sometimes
Use of English IRL
1. Chat in games
2. Holidays 3. Skype
1. Holidays, 2. Chat in
games 3. Txt chat
Assessment of Oral skills
Aspect Group A Group B
Assessed Tasks in 2010
All: 1 All: 3.5
School reports: Oral skills included?
35% ? 70% ?
Expectations: Is tele-testing valid?
Yes: 35% ?: 55%
Yes: 60% ?: 40 %
Post: Valid Yes: 60% 40%
Topics
Introduction/warming up
Your holiday this year
A good school
Social networks
A million euros
The climate
Post-Questionnaire (1)
Aspect Group A Group B
Technically OK? Yes So, so
Read Instructions
Yes Sure
Different from expectation
Yes Yes
Questions: complexity,
speed, loudness,
Hard to remember,
Speed bit fast, Not loud enough
Idem, but Speed OK
Post-Questionnaire (2)
Aspect Group A Group B
Answer time left Yes Yes
Expected Mark Just sufficient O.K
Problem Topics Networks; 1M Euros Good School; Climate
Test Location School School + home
O.K. to do Yes, quite Yes
More pleasant without teacher
No No
Pupils’ Comments
Time constraint is unnatural
Was interrupted: new session needed
Retries: worries about costs
Questions could be louder
Questions: peer voice is more inviting
Teacher Perceptions
System usability -System: fairly user-friendly
Topics - More alignment with pupils‟ interests might be needed
Validity -Content measured in time is less suitable as criterion when no interaction is possible - computer-based testing, as such, not perceived as unusual or unfriendly. - Retry option?
Reviewing work online
34
Teacher Perceptions
Teacher competencies: - Knowledge of CEFR -> difficult, training / practice needed - Evaluation categories (content, accuracy, complexity, fluency.) useful; scoring doable in one session. But …would prefer a grading scale that results in a CEF-level: better match to Dutch current grade system
Teacher perceptions
Implementation - Use as practice material and preparation for speaking test. Actual testing: rather face-to-face - Gives students the opportunity to practice outside the classroom, extra practice - Chances for providing individualised feedback - May help reduce anxiety of insecure & shy students:
Conclusions / next steps
Improve briefing (demo, online tutorial) + raise awareness implications of re-tries
Redesign questions (granularity)
Try-out alternatives: - system access (landlines, computers) - content aligned to syllabus / textbook
Conclusions
Pupils, teachers & management have concerns about costs
Also found in other projects: […] cost to the end user is a major consideration and can be a barrier to successful uptake when using mobile devices (Kukulska-Hulme & Shield, 2007)
Thank you for your attention.
Comments, Questions?
www.koenraad.info
Annex
Literature Selection Research data
Learnosity
Literature selection
Collins, T. (2005). „English Class on the air: Mobile Language Learning with CellPhones‟, Proceedings of the Fifth IEEE International Conference on Advanced Learning Technologies (ICALT‟05).
http://csdl2.computer.org/comp/proceedings/icalt/2005/2338/00/23380402.pdf
Fallahkair, S., Pemberton, L. & Griffiths, R. 2007. „Development of a cross-platform ubiquitous language learning service via mobile phone and interactive television‟. Journal of Computer Assisted Learning, 23 (4), 312-325.
Kukulska-Hulme, Agnes; Sharples, Mike; Milrad, Marcelo; Arnedillo-Sanchez, Inmaculada and Vavoula, Giasemi (2009). Innovation in Mobile Learning: A European Perspective. International Journal of Mobile and Blended Learning, 1(1), pp. 13–35.
Literature selection (2)
Naismith, L., Lonsdale, P., Vavoula, G. & Sharples, M. (2004). „Literature Review in Mobile Technologies and Learning‟. FutureLab Report 11. http://www.futurelab.org.uk/resources/documents/lit_reviews/Mobile_Review.pdf.
Sharples, M. (Ed.). (2006). Big issues in mobile learning. Report of a workshop by the Kaleidoscope Network of Excellence Mobile Learning Initiative, University of Nottingham, UK.
Shield, Lesley and Kukulska-Hulme, Agnes (2008). Special issue of ReCALL on Mobile Assisted Language Learning. Cambridge University Press.
Thornton, P. & Houser, C. (2005). „Using mobile phones in English education in Japan‟. Journal of Computer Assisted Learning, 21, (3): 217-228.
Pre-Questionnaire: Learner Profiles (1)
Aspect Group A Group B
Group size 12 8
Girls 5 4
Boys 7 4
Years of English 3 3
Positive Attitude to Learning English
2.42 STD: 0.51
3.11 STD: 0.33
Average score at Secondary
6.37 STD: 0.9
7.6 STD: 1.7
Speaking Skills: (Self reported)
6.58 STD: 0.9
7.55 STD 0.68
Pre-Questionnaire: Learner Profiles (2)
Aspect Group A Group B
Like speaking in class
2.67 STD: 0.78
3.78 0.44
Actual speaking 1.5 STD: 0.52
3.56 STD: 0.53
Telecollaboration at school
n/a 2.0 0.0
Tel. Experience in projects
n/a
1.38 STD: 1.0
Tel. Experience IRL
1.83 STD: 1.19
1.44 STD: 0.73
Pre-Questionnaire: Learner perceptions: Oral L2 in class
Aspect Group A Group B
L2 in lessons 2.1 STD: 0.50
5.0 0.0
Answers /Discussion 2.2 / 1.6 3.9 / 3.2
Pairwork 1.6 1.9
English IRL
1. Chat in games
2. Holidays 3. Skype
1. Holidays, 2. Chat in
games 3. Txt chat
Assessment of Oral skills
Aspect Group A Group B
Assessed Tasks in 2010
All: 1 All: 3.5
Formal reports: Oral skills included?
35% ? 70% ?
Is tele-testing valid?
Yes: 35% ? : 55%
Yes: 60% ?: 40 %
Post: Valid Yes: 60% 40%
Post-Questionnaire (1)
Aspect Group A Group B
Likert scale Disagree 1 Agree 4
Technically OK? 2.43 STD: 1.13
1.8 0.84
Read Instructions
2.8 STD: 0.7
3.0 0.7
Different from expectation
2.7 STD: 0.76
2.8 1.3
Questions: complex, speed,
loudness,
Not loud enough Hard to
remember
idem Speed OK
Post-Questionnaire (2)
Aspect Group A Group B
Likert scale Disagree 1 Agree 4
Answer time left 3.4 STD: 0.5
3.2 1.1
Expected Mark Just sufficient STD: 1.4
O.K 1.1
Problem Topics Networks; 1M Euros Good School; Climate
Fun to do 2.8 STD: 0.4
2.4 0.9
More pleasant without teacher
2.07 STD: 0.6
2.0 1.0
www.learnosity.com
Twitter @learnosity
Speak. Listen. Learn.