IARPInformatics Audio Recording Pilot

Dr Apurba KunduAssociate Dean (Learning & Teaching)

School of Informatics

University of Bradford

2

IARP, 2006-07Informatics Audio Recording Pilot (IARP), semesters 1 and 2, 2006-07.

Using digital devices to • record lectures, seminars and/or presentations, • edit the recordings, and then • upload them onto the University of Bradford’s VLE (Blackboard) for students

to access.

IARP located in the University of Bradford School of Informatics:• Department of Electronic Imaging and Media Communications (EIMC)• Department of Computing

IARP co-ordinated by Dr Apurba Kundu, Associate Dean (Learning & Teaching).

3

Origins of the Pilot, 2005-06“I teach the module ‘Telecommunications and

Networking to about 70% of the EIMC final year…

This year [2005-06]… I had a large number of students with disabilities, including one who was completely blind, and I concluded that it would be wise to record the lectures. This… turned out to be extremely successful, bringing potential benefits to far more than just the disabled.

I bought an Olympus [digital recorder]... Recording the lectures was quite easy… [and] I… became comfortable with the system… and established a routine of editing and saving the files immediately after the end of each lecture… I also recorded my seminar sessions…

Once the recording had been converted to MP3 format I then uploaded it to the Blackboard site for this module, simply in a form that the students could download to their preferred device.

It was… a considerable surprise…to see the results… where the average was very…high (somewhat over 70%)… On the other hand, last year the average had been the lowest for all of the students concerned (55%) and, while a few minor factors might be identified, the only substantial change from 2005 to 2006 was the use of the audio recording.

I asked the students on several occasions whether the audio recordings were useful and they said that they were. The recordings were… strongly praised as being very helpful. Discussions with one of the students with a degree of disability revealed that she had listened to the recordings four times over. I recall other students saying that they had transferred the recordings to their iPods and listened to them on their journey to the University etc. I believe that the German students (of whom there was a significant number) also found it particularly useful…

-- Professor Peter Excell, July 2006

4

IARP, 2006-07

At the start, IARP involved:• 13 lecturer volunteers (i.e., no workload credit!)• A mix of 22 modules

– taught at the undergraduate and postgraduate levels,– containing a wide mix of assessments from essays to group

presentations, and – varying in class size from 3 to 120 students.

• Over 500 students were involved.

Was IARP successful? By what measure?

5

Quantitative Measures

Level Module Dept Assessments Stud

1 Audio Visual Fundamentals EIMC 75% Exam, 25% Project 109

1 Creating Virtual Reality Comp 50% Exam, 50% Gp project 3

1 Creating Virtual Reality (2nd instance) Comp 50% Exam, 50% Gp project 10

1 Introduction to 3D Animation (Stage 1) EIMC 100% Exam 77

1 Media Audiences EIMC 50% Presentation, 50% Paper 12

1 Media Institutions EIMC 75% Essay, 25% Presentation 39

2 Cyberpsychology Comp 100% Essay 43

2 Introduction to 3D Animation (Stage 2) EIMC 100% Exam 64

2 Robotics 4 Comp 30% Coursework, 70% Exam 28

2 Soundscapes EIMC 50% Ind presentation, 50% Gp project 120

2 Specialist Project 2 EIMC 15% Gp proj, 35% Logbook, 50% Product 94

3 Interactive Systems and Video Game Design EIMC 25% Ind proj&report, 75% Gp proj&report 23

3 Telecommunications and Networking EIMC 100% Exam 52

M Computer Animation and Special Effects EIMC 25% Report, 75% Coursework 20

M Design for Mobile Content EIMC 60% 2x Projs, 40% 2x Props&Presentations 16

M Mobile Applications EIMC 30% WAP, 30% Proj, 40% 2x Reports 16

Ordered by Level (with Department, Assessments and Student Numbers)

6

Quantitative Results

Level Module 2005-06 Mean Change in Mean 2006-07 Mean

1 Media Institutions 52.1 + 6.0 58.1

1 Creating Virtual Reality (2nd instance) 44.8 + 5.4 50.3

2 Cyberpsychology 57.1 + 4.5 61.6

1 Audio Visual Fundamentals 47.7 + 2.1 49.8

M Design for Mobile Content 57.0 + 1.4 58.4

1 Media Audiences NA NA 60.2

2 Robotics 4 63.7 - 0.8 62.9

1 Introduction to 3D Animation (Stage 1) 52.2 - 0.9 51.3

1 Creating Virtual Reality 44.8 - 1.1 43.7

2 Soundscapes 53.9 - 1.8 52.1

2 Introduction to 3D Animation (Stage 2) 56.2 - 2.0 54.2

3 Telecommunications and Networking 70.3 - 2.3 68.0

2 Specialist Project 2 60.1 - 5.3 54.8

M Mobile Applications 59.9 - 5.8 54.1

3 Interactive Systems and Video Game Design 64.2 - 8.6 55.6

M Computer Animation and Special Effects 70.9 - 9.0 61.9

Ordered by Change in Mean

7

Quantitative Results

Level Module 2005-06 Std Dev Change in Std Dev 2006-07 Std Dev

M Mobile Applications 8.5 + 10.0 18.5

M Computer Animation and Special Effects 4.5 + 8.1 12.6

3 Telecommunications and Networking 15.8 + 5.5 21.3

1 Creating Virtual Reality (1st instance) 13.5 + 5.0 18.0

M Design for Mobile Content 13.3 + 2.0 15.3

2 Introduction to 3D Animation (Stage 2) 9.3 + 1.9 11.1

1 Media Institutions 5.8 + 1.6 7.4

2 Soundscapes 12.6 + 1.0 13.6

2 Specialist Project 2 12.6 + 1.0 13.6

1 Media Audiences NA NA 13.3

1 Introduction to 3D Animation (Stage 1) 11.1 - 0.8 10.3

1 Audio Visual Fundamentals 14.3 - 0.4 13.9

2 Robotics 4 19.3 - 3.3 16.0

1 Creating Virtual Reality (2nd instance) 13.5 - 4.6 8.9

2 Cyberpsychology 15.8 - 7.3 8.5

3 Interactive Systems and Video Game Design 15.1 - 8.8 6.3

Ordered by Change in Standard Deviation

8

Quantitative Results

Level Module Mean Std Dev Students

1 Media Institutions + 6.0 + 1.6 39

1 Creating Virtual Reality (2nd instance) + 5.4 - 4.6 10

2 Cyberpsychology + 4.5 - 7.3 43

1 Audio Visual Fundamentals + 2.1 - 0.4 109

M Design for Mobile Content + 1.4 + 2.0 16

1 Media Audiences NA NA 12

2 Robotics 4 - 0.8 - 3.3 28

1 Introduction to 3D Animation (Stage 1) - 0.9 - 0.8 77

1 Creating Virtual Reality (1st instance) - 1.1 + 5.0 3

2 Soundscapes - 1.8 + 1.0 120

2 Introduction to 3D Animation (Stage 2) - 2.0 + 1.9 64

3 Telecommunications and Networking - 2.3 + 5.5 52

2 Specialist Project 2 - 5.3 + 1.0 94

M Mobile Applications - 5.8 + 10.0 16

3 Interactive Systems and Video Game Design - 8.6 - 8.8 23

M Computer Animation and Special Effects - 9.0 + 8.1 20

Ordered by Mean, then Standard Deviation, then Student Numbers

9

Student Comments

Various modules:

• “Audio being available was great”.• “It was great that podcasts were available”.• “Audio recordings of labs [was] very useful”.• “Yes I used the podcasts and found them

useful”.• “[A major strength of the module was the]

recorded lectures, so we were able to go back to things we forgot”.

• “[A major strength of the module was the] audio of lectures on Blackboard”.

• “Lecture audio via Blackboard is very helpful”. • “Just wanted to say thank you for providing such

a fantastic module. I feel I got a lot from the audio part of the module… The MP3s were a great help and I found they work on a lot of levels when it comes to reinforcing all the material.”

• “I used the podcasts and they were useful for refreshing my memory of the lectures”.

Robotics 4: student responses

Yes%

No%

Did you use the audio recordings?

50 50

Did you find them easy to locate on Blackboard?

100 0

Did you find them useful?

100 0

Did you use them for lectures you attended?

33 67

Did you use them for lectures you missed?

83 17

Do you think recording lectures is a worthwhile use of lecturer's time?

75 25

Lecturer Comments: NegativeThree of the 13 original lecturer

volunteers (with 4 modules between them) ended up not participating in IARP due to:

• Late start (recorded only 2 lectures).

• Insufficient time to manipulate and upload audio files: “I barely got time to download the audio files to my PC let alone edit them and put them as podcast”.

• Audio unsuited to nature of material being taught.

Another lecturer did not use one module intended for IARP due to:

• Technological difficulties: “Due to [a] lamentable cock-up, I accidentally deleted several recordings in the process of putting them on Blackboard, so I abandoned it for this this module”.

Of those who participated, difficulties cited similarly include:

• Insufficient time: “I found it time consuming and a pain to do the following: Spend 20 minutes transferring data to my machine. Spend an hour or so editing out pauses and partially recorded class-room interactions. Compression to MP3 and upload a further 30 minutes”; “To be honest, I found it a pain and would probably try to avoid it because it took too much time”.

• Audio unsuited to nature of material: “I think it’s more appropriate to subjects where memorising content from the lectures is likely to directly impact on marks or student learning (where the dictated content of the lectures makes up the key content of what the student needs to know to pass)”.

• Pointless exercise: “In my view the pilot in this case did not improve on the 'traditional' lecture format other than to edit and refine the actual 'real' presentation. I can't see any reason to replace the lecture with a recording”.

• Pedagogic concerns: “I am against dictating what type of learning resources lecturers must provide. I reluctantly accept that PowerPoint is now a default requirement. I am worried that students are losing the ability to take notes (a key skill for future meetings etc) and that they don’t bother turning up because they don’t feel they have to – thereby losing out on interaction”.

Lecturer Comments: Positive

Lecturers acknowledged that their students found IARP useful:

• Verbal feedback from the trial was overwhelmingly positive. A number of students indicated that they were using the podcasts and some had subscribed to the podcast RSS feed using itunes.

Audio recordings were not just for lectures:• “The main use that may have differed from other folk is that I

recorded the group meetings from Specialist Project 2 and then put these on line. This had two effects. Firstly it allowed the group to share the discussion with those who couldn't make the meeting and, secondly, it provided a record of who attended. The students said they liked being able to listen to the feedback and guidance later”.

• “We also recorded the individual project proposals… where the students pitch their own project ideas in small groups to… [two another lecturer] and myself. Recording them meant that our feedback was then available to the project supervisors (allocated after all the presentations had taken place) and subsequently to the students. This meant that there was little chance of anything being missed in that feedback as opposed to written notes being used as the sole record”.

Attendance did not seem to suffer:

• I was “surprised by how many listened and found it useful, even those who attended”.

• I expected attendance to fall as a result but… even after students… knew podcasts were available they continued to attend well.

In sum,• “In general I was fairly

happy with the podcasts on this module”.

• “In all it was a positive experience for me, and it seemed to be the same for the students!”

• I enjoyed participating in the pilot and would definitely podcast lectures again.

Conclusions & Future Studies

THE STUDENT EXPERIENCE• In line with expectations, the

student experience of IARP was almost uniformly positive.

• Contrary to prior expectations, IARP did not negatively affect attendance.

THE LECTURER EXPERIENCE• There are more imaginative

uses for audio recordings other than simply recording lectures (e.g., recording group discussions and/or marker’s feedback).

• The technology used—particularly the software—to record, edit and upload files to the VLE was onerous.

• However, also contrary to expectations, IARP improved the mean of less than half (5 of 16) of those participating modules that had been taught in 2005-06.

A WIDER TRIAL?• “I am not sure about making podcasts compulsory; I

would have thought we would need a more prolonged trial for that…”.

DISCIPLINE SPECIFIC APPLICATIONS?• IARP involved modules are taught in the respective

departments of EIMC and Computing where both students and staff are expected to be comfortable adapting to new technologies.

• Would students and staff in, for instance, arts & humanities and/or social sciences have the same experience of audio recording technology?

• Perhaps the nature of delivery in the above are more suited to audio recordings?

A LISTENING UNIVERSITY AND ITS AUDIENCE?• “A point that is rarely raised in discussion of podcasts

is the question of the University's relationship with the wider community. Some US universities make their lectures publicly available and in my opinion this is something… [we] should consider; the question is not just benefits to students directly but also public benefit ("Knowledge Transfer")”.

IARPInformatics Audio Recording Pilot

END

Please direct comments and questions toDr Apurba Kundu at a.kundu@bradford.ac.uk