TEXT

Evaluating the impact of ‘blended’ or ‘flexible learning’ Evaluating the impact of ‘blended’ or ‘flexible learning’

in a single M.Sc. module in a single M.Sc. module

in Alcohol and Drug Studies, in Alcohol and Drug Studies,

University of Paisley University of Paisley

2001-2004.2001-2004.

Iain McPheeIain McPhee

Centre for Alcohol and drugs StudiesCentre for Alcohol and drugs Studies

Observations made on evaluating a ‘blended learning’ environment.

1.An overview of the impact of blended learning on recruitment and performance

2.The student experiences in a blended learning environment

3.An understanding of workload implications and the Tutor experience

Alcohol and Drug Studies at the University of Paisley

• M.Sc. in alcohol and drug studies first validated the 1979.

• 2005: offer flexible modes of study in a virtual learning environment (VLE) using the web based learning platform, Blackboard.

Alcohol and Drug Studies at the University of Paisley

3 modes of delivery from 2001 to 2004 for 1 level 11 module.

• Study mode 1 on campus study (full and part time)

• Study mode 2 online study (part time only)

• Study mode 3 online study with face-to-face tutorial (part time only)

Selected themes from literature

• (Diaz & Cartnal 1999)-Institutions often assume that module delivery is appropriate for any setting

• Harasim et al (1995) note that volume of communications increase markedly with online teaching;

• Shaw and Young (2003) for online teaching to be at least as effective as on campus teaching requires concerted effort from tutors, in terms of flexibility in approach to working hours.

Overview of the evaluation

• The assessment results of this study largely support previous research;

• summative grades do not differ greatly between online and on campus cohorts.

Why? • students are not disadvantaged by the isolation of online

study (regularly reported by distance learners),

• Tutors increase effort to achieve parity in the learning experiences of all 3 modes of teaching delivery.

Learning and support for the 3 modes of learning

• All students receive a set of interactive core reading materials

• all students are registered on VLE blackboard.

• All students are regularly contacted both in real-time (synchronous) via the VLE web chat facility, and offline (asynchronous) via the discussion boards and the University and blackboard e-mail facility.

Student numbers NURSM001

2001-02 2002-03 2003-04 2004-05

On campus (1) 55 34 47 24

Online (2) 32 32 19 32

Blended (3) 0 10 10 8

Males (%) 35 32 32 38

Females (%) 65 68 68 62

Total 8787 7676 7676 6464

•This is a core module and it is essential that all students study this module.

•Previous to 2001, part time flexible online study was not an option.

Student numbers by year and study modality

Table 1

Sex ratio of postgraduate students for module nursm001Who completed the module in one semester (the numbers differ from students registered with the University)

2001 2002 2003 2004

Male Female Male Female Male Female Male Female

On Campus (Group 1) 19 30 10 37 6 22 11 20

Online(Group 2)

9 23 4 13 0 6 5 10

Blended(Group 3)

0 0 2 8 0 5 0 6

TOTAL (%) 3535 6565 2222 7878 1515 8585 3131 6969

Student numbers NURSM001

2001-02 2002-03 2003-04 2004-05

On campus (1) 55 34 47 24

Online (2) 32 32 19 32

Blended (3) 0 10 10 8

Males (%) 30 32 32 38

Females (%) 54 68 68 63

Total 87 76 76 64

Sex ratio and completion rates

• Students who did not complete the assessments in one semester were omitted from this analysis.

• Part time students (groups 2 & 3) take longer to enrol on the course due to all paperwork having to be posted out, sent back for signing, before registration can take place.

• They are usually in work, & are older than ’traditional’ postgraduate students.

On campus teachingOn campus teaching

• power point slides, flip chart and whiteboard.

• short activities focus on the development of specific cognitive and affective skills required for this module.

• This power point presentation is uploaded to the VLE to be accessed by all students.

Online teachingOnline teaching

• follows on campus learners although modified for dissemination via the VLE.

• All lecture content; core reading and additional reading are all made available via the VLE as flash files.

• discussions take place both in real time (synchronous) via the fortnightly web chats, and in the weekly discussion boards (asynchronous).

• discussion topics are based on the 5 unit learning outcomes.

Online teaching

Synchronous via the web chat facility in the VLE

• Live chats every 2 weeks & take 1 hour to deliver.

• topics driven by the learning outcomes - participants share their views by keyboard conferencing.

• virtual chat allows social interaction

• live chat promotes a social matrix -students begin to use each other as a learning resource.

Online teachingOnline teaching

Asynchronous contact and the discussion board

• In the module under scrutiny - 5 learning units

• generate at least 2 questions per unit, which are uploaded to the VLE discussion boards.

• maintaining contact and motivating students requires constant monitoring to be effective.

• the discussion is summarised weekly and the summary can be archived as an educational resource.

Tutors experienceTutors experience• On campus cohort (group 1)1.5 hours direct teaching contact. not including drop in and informal discussions out with scheduled

teaching.

• Online cohort (group 2)5 hours 30 minutes includes all e-mail and discussion forum contact and 1 hour per week

in virtual synchronous teaching.

• Blended cohort (group 3)6 hours and 30 minutes as group 2 plus1 hour per week direct teaching (accounting for 2 days

face to face tutorials and seminars).

Tutors experienceTutors experience

• This increased teaching input and contact for online and blended learning students is thought to be essential

• balances the regular contact that on campus students experience.

• The attempt to create parity and treat the students as one cohort does increase workload.

• Weekend and evening work is now essential.

AssessmentsAssessments• All TMA’s are delivered to and from the ‘assignments’ facility within the

VLE.

• The Tutur downloads the essays from the VLE, marks them, and delivered back to the student via the VLE, grades can be accessed via the grade book facility in BB.

This module uses a 2-stage assessment

• TMA1-a 1000 word essay or an online multiple choice test (not discussed here).

• formative feedback is intended to help students with the end of term essay called TMA2 consisting of 3500 words.

• Feedback for TMA’s is structured to comply with a marking frame and standardised feedback sheet gives the students clear directions on what they should do to prepare for the end of term assignment.

Mean grades per year at completion of module NURSM001

2001 2002 2003 2004-05

On Campus (Group 1) - 56.26 (09.13) 65.30 (6.30) 64.03 (7.42)

Online Group 2) 54.33 (10.53) 65.10 (3.72) 63.76 (5.55)

Blended (Group 3) - 55.85 (10.55) 67.40 (5.22) 64.00 (7.34)

Mean (all groups) 56.87 54.45 50.37 59.28

•a small but significant difference in mean scores between study modes 2 and 3.

•This suggests that the blended mode of study (group 3) results in a higher average pass rate.

•It could just as likely be a cohort effect.

table 2

Assessment resultsAssessment results

Value Label N

Group 1.00 2002 - On campus

38

4.00 2003 - On campus

26

7.00 2004 - On campus

30

ResultsResults

ANOVA OF ON CAMPUS BY YEAR AND BY MODE OF DELIVERY

Source

Type III Sum of

Squares df

Mean Squar

e F Sig.

Corrected Model 1605.499(a) 2 802.749 12.861 .000

Intercept 351123.033 1 351123.033 5625.512 .000

GROUP 1605.499 2 802.749 12.861 .000

Error 5679.874 91 62.416

Total 359871.000 94

Corrected Total 7285.372 93

as a significant difference was found (.000)

I decided to run a T test to compare the groups by year.

Dependent Variable: Scores a R Squared = .220 (Adjusted R Squared = .203)

T-test by group & year

• For all study modes

• Significant differences 02-03; 02 – 04

• No differences 2003-2004

• The mean score is 10 points less overall for all groups in 2002. This may be due to global changes in TMA 1 assessment from 2002. A multiple choice online test using ‘questionmark perception’.

1 way ANOVA by group & year

• 2002 – no significance between groups (.785)

• 2003 – no significance between groups (.729)

• 2004 – no significance between groups (.993)

Section A Within teaching time…  

Strongly agree

Agree Neutral Disagree Stronglydisagree

A1 Content & learning outcomes were clear 50% 38% 13% 0% 0%

A2 Place & time of classes were clear 38% 50% 13% 0% 0%

A3 Assessment requirements were clear 56% 31% 13% 0% 0%

A4 Cope with workload   38% 50% 13% 0% 0%

A5 Teaching methods assisted learning 50% 44% 6% 0% 0%

A6 Useful support & feedback by staff 38% 31% 25% 6% 0%

A7 Ask questions & seek advice from staff 50% 31% 13% 6% 0%

A8 Confident of achieving learning outcomes 31% 44% 25% 0% 0%

Section AWhat is worth noting is that in general these student cohorts have had an overall positive experience on the module.

1 student (6%) did not consider the feedback on assignments useful (A6) nor did they seek advice from the Tutor.

Student experiencesStudent experiences

Section B Your LTC's questions…

 Strongly

agree

Agree Neutral DisagreeStrongly

disagree

B1 Key issues and debates   44% 44% 13% 0% 0%

B2 Contribution to critical thinking 44% 56% 0% 0% 0%

B3 Module was intellectually stimulating 50% 44% 6% 0% 0%

B4 Engaged in 6-8 hours independent study 25% 50% 25% 0% 0%

B5 Seminars / workshops were effective 31% 56% 6% 0% 6%

B6 Coursework was an effective learning aid 25% 69% 6% 0% 0%

Section BThe respondents considered that the teaching contributed to their critical thinking and was intellectually stimulating.

One student (6%) thought that the seminars were not effective (B5).

Student experiencesStudent experiences

Section C Outwith teaching time…

 Strongly

agree

Agree Neutral Disagree Strongly disagree

C1 Prepared well by previous study / exper. 25% 31% 38% 6% 0%

C2 Devote necessary time   13% 25% 44% 19% 0%

C3 Access to computing facilities 13% 56% 13% 13% 6%

C4 Library provided good support 25% 19% 31% 25% 0%

C5 Needed help from teaching staff 0% 6% 44% 13% 38%

Section CA significant number of respondents considered that the library did not provide them with good support. This issue should be addressed.

One student (6%) did not have access to computing facilities (C3) during this module.

Student experiencesStudent experiences

Conclusion

• In summary while flexible teaching and delivery has increased the numbers of part time and distance learning students,

• resulted in a marked increases in Tutor time allocated to online contact with these students.

• Any planned increased in student numbers will necessitate a dedicated tutor to maintain and support online students.

Conclusion• The following conclusions my be drawn from this

evaluation, drawing on both staff perceptions and the student experiences as indicated in the questionnaire returns and qualitative discussions.

• At this level for this type of learning the intellectual & social dynamic of face-to-face learning plays a key role, which may not adequately be replaced by a VLE, without major workload implications for the Tutor and for the Institution.

• The most effective learning environment may be one, which combines elements of both

Conclusion• Online and virtual learning technologies have definitive advantages – especially in

their flexibility.

• They do not however offer any magic solutions to the problems encountered in teaching students in other contexts.

• The technology and the reading materials available online and in paper format do not in and of themselves generate motivation and engagement.

• students on the module all face many demands on their time, and many who may need additional online support, do not always seek it out. Persuading students to engage with each other and the materials requires constant encouragement of students by the tutor to participate.

• A VLE requires a different style of approaching teaching and learning, and thus entails a considerable amount of time and effort before, during and after taught inputs. Therefore any value of adopting this flexibility in approach must be balanced against the opportunity costs entailed in implementation.

Selected referencesSelected references

Harasim L et al (1995) Learning Networks: A Field Guide to Teaching and Learning On-line. Cambridge MA: MIT.

 Diaz, D. P., & Cartnal, R. B. (1999). Students' learning

styles in two classes: Online distance learning and equivalent on-campus. College Teaching 47(4), 130-135.accessed at:http://www.collegedegreeguide.com/articles-fr/styles.htm

Shaw, D. Young S., ‘Costs to instructors in delivering equated online and on-campus courses’ (2003) The journal of Interactive Online Teaching, volume 1, number 4, Spring 2003. accessed at: http://www.ncolr.org/

Contact detailsContact details

Paisley university website for centre for alcohol and drugs studies:

http://www.paisley.ac.uk/socialsciences/cads/

Email for iain mcphee:Iain.mcphee@paisley.ac.uk

Web page:http://www.paisley.ac.uk/socialsciences/cads/staff/iain-mcphee.asp