Quality schemes in cooperative learning environments

Morten Flate PaulsenProfessor of Online Education

A 50-minutes presentation at the International Scientific Conference on ICT and Education

Montreal, May 2012

1

http://twitter.com/MFPaulsen

http://nettstudier.blogspot.com http://home.nki.no/morten

www.slideshare.net/MortenFP

www.facebook.com/mfpaulsen www.eden-online.org/blog/

Abstract

The presentation will reflect on systematic and continuous quality schemes in large-scale, online learning environments.

It will focus on the experiences with quality barometers and other quality enhancement tools and services at NKI Nettstudier –Scandinavia’s largest online education provider.

2

About me

• Worked with online education since 1986

• Professor of Online Education at NKI Nettstudier , Norway

• President of European Distance and E-learning Network (EDEN)

• Doctor of Education from the Pennsylvania State University, working for the American Center for the Study of Distance Education (1990-1992)

• Adjunct Professor Athabasca University (1999 to 2005)

• Professor associado Universidade Aberta, Portugal (2009-2010)

3

• 20th anniversary in 2011, 8 in secretariat, 11 in EC• EDEN members from 400 institutions in 56 countries:

– 20th anniversary

– 200 institutional members

– 2000 potential members of NAP – Network of Academics and Professionals

• Conferences– Open Classroom conference ,Athens October 2011

– Annual conference, Porto June 2012

– Research Workshop, Leuven October 2012

• EU-projects with 168 institutions

• European Journal of Open, Distance and E-learning www.eurodl.org

www.eden-online.org

EDEN-members in Canada• Athabasca University

• Canadian Network for Innovation in Education – CNIE

• Société de Formation a distance des Commissions Scolairs du Quebec - SOFAD

5

Annual Conferences

21. Oporto 201220. Dublin, 201119. Valencia, 201018. Gdansk, 200917. Lisbon, 200816. Naples, 200715. Vienna, 200614. Helsinki, 200513. Budapest, 200412. Rhodes, 200311. Granada, 200210. Stockholm, 20018. Moscow, 19997. Bologna, 19986. Budapest, 19975. Poitiers, 19964. Birmingham, 19953. Tallinn, 19942. Berlin, 19931. Krakow, 1992

Open Classroom9. Athens, 20118. Oporto, 20097. Stockholm, 20076. Poitiers, 20055. Copenhagen, 20024. Barcelona, 20003. Balatonfüred, 19992. Crete, 1997

1. Oslo, 1995

6

Research Workshops

7. Leuven, 20126. Budapest, 20105. Paris, 20084. Barcelona, 20063. Oldenburg, 20042. Hildesheim, 20021. Prague, 2000

Planning for Oslo EDEN’s Annual Conference, June 2013

8

NKI Nettstudier was established as a correspondence school in 1910

• One of Europe’s largest providers of online distance education• About 150 000 enrolments in online courses since 1987• Broad range of subjects from secondary to master level• About 400 distance education courses online (all in Norwegian)• About 12 000 online students in about 40 countries• About 70 % women• Individual progress plans/individual follow-up• Thousands of potential learning partners• Global student catalogue with 1200 profiles• Always room for more students• Exams at local schools and embassies

November 2011

www.nki.no

College, vocational- and secondary school

9

Picture from: http://youtu.be/NXBNVauG9bE

400 online courses

10

Picture from: http://youtu.be/NXBNVauG9bE

Focus on individual progress plans

Example of individual progress plan

12

NKI: A pioneer in Online Education1986: First LMS – specifically developed for distance learning1987: First online distance education course1992: Mandatory course for new NKI tutors on the Internet1996: First web-based courses2002: First m-learning courses2003: Speech synthesis in all courses2004: Individual planning system and individual follow up according to the student’s plan2004: Continuous tracking of turn-around time2006: Learning partner system2008: Global student profiles2010: Peer counselling system for tutors2011: Quality barometer for tutors and course materials

13

14

The most interesting pedagogical challenge in our lifetime is to provide online education that combines individual freedom with meaningful cooperation.I have struggled with this challenge since I first introduced my Theory of Cooperative Freedom in 1992.

www.ednews.org/articles/10626/1/An-Interview-with-Morten-Flate-Paulsen-Focusing-on-His-Theory-of-Cooperative-Freedom-in-Online-Education/Page1.html

http://issuu.com/mfpaulsen

15

Cooperative Freedom is discussed in my book Online Education which is available from

More resources on Cooperative Freedom• Cooperative online education

www.youtube.com/user/nki2008#p/a/u/2/HVXtbLg5ycE

• Online education barometers www.youtube.com/watch?v=gyoe7dGIwkE&feature=relmfu

• Transparency in Cooperative Online Education www.irrodl.org/index.php/irrodl/article/view/671/1267

• E-viva em Portugal http://nettstudier.blogspot.com/• Cooperative Online Education http://seminar.net/index.php/volume-4-issue-2-2008-

previousissuesmeny-124/100-cooperative-online-education • Cooperative Freedom and Transparency in Online Education

www.slideshare.net/MortenFP/futuralia-3400339• Visualizing student profiles

www.slideshare.net/MortenFP/visualizing-student-profiles-through-nkis-online-catalogue-and-student-network

• Transparency for Cooperationhttp://toonlet.com/archive?m=s&i=11245

Six Dimensions of FreedomCompilation of resources made by my students

at Universidade Aberta in Portugal

It is difficult to combine individual flexibility and cooperationhttp://ppel4.pbworks.com/

18

One may say that:

Individual learning is conducted alone

Collaborative learning depends on groups

Cooperative learning takes place in networks

Connectivism?

19

NKI provides cooperative online educationIndependence with the help of others

Cooperative learning

• Cooperative learning seeks to develop virtual learning environments that allow students to have optimal individual freedom within online learning communities.

• Cooperative learning is based on voluntary participation in a learning community

• Cooperative learning benefits from transparency

NKI’s Philosophy: We facilitate individual freedom within a learning community in which online students serve as mutual resources without being dependent on each other.

20

Tools and strategies to enhance quality in a virtual learning environment with:

• 12.500 students• 400 courses• 150 part-time teachers• 60 full-time employees

should be systematic, continuous and cost-effective

21www.nki.no

Strategies and tools to enhance quality

1. Large scale online education needs systematic, continuous and cost-effective quality measures

2. Transparency improves quality

3. Teacher Support Services

4. Quality Barometers

22

Transparencyor

privacy?23

24

Transparency improves quality

25

…of system data, course content, course work etc….

Audio Journalism: Berghs School of Communication (Sweden)Video Journalism: UPDATE – Danish School of Media and Journalism

Text Journalism: NKI Distance Education (Norway)

Transparency promotes cooperation

27

Dalsgaard, C. and Paulsen, M.F. (2009) Transparency in Cooperative Online Education. IRRODL, Vol 10, No 3.

Three degrees of transparency

1. Low: Private e-mail

2. Medium: Class forum

3. High: Global web2.0 services

Transparency provides better quality and more cooperation, but both students and teachers should be comfortable with the chosen transparency level.

28

Online teachers deserve decent support serviceshttp://nettstudier.blogspot.com/2011/08/online-teachers-deserve-decent-support.html

29

1992: Online course for online teachers2004: Response Barometer

2010: Peer counseling and self-evaluation2011: Quality Barometer for Teachers

30

Response Barometer for Teachers

• Introduced in May 2004• Resulted in much discussion in the

teachers’ online forum• A few teachers voiced strong criticism,

doubts and reservations

The average response time for all150 teachers in 2010 was 1.77 days.

In 2008 NKI introduced an automatic e-mail

reminder to the tutors who had one ore more

assignments waiting for 7 days or more.

Barometer Questions

1. What to measure?

2. Which scale?

3. Transparent for whom?

4. Possible to show trends?

5. What about Evaluation Moment, Frequency and Fatigue?

31

32Transparent feedback to the right people

Online Education Barometersshould have individual readings

cooperative readings, aims,, warning levelsand transparent feedback loops

Ideal for Individual Startand Progression

NKI’s Transparent:

Quality Barometerfor Online Teachers

Evaluation Icon

As soon as a teacher provides submission feedback and enters a grade, the far right evaluation icon shows up in the planning tool at the student’s start page.

Click the box to open it.

Evaluation Button

The student has to evaluate the teacher to embed the grade.

Click the evaluation button to open the teacher evaluation window.

Teacher Evaluation Window

Choose the teacher satisfaction level. The options from left to right are: Very dissatisfied, Dissatisfied, Neutral, Satisfied and Very Satisfied. Then enter some optional comments on the teacher’s work.

The chosen satisfaction level influences the Teacher Quality Barometer as soon as it is submitted.

After a random time delay for anonymity reasons, the teachers can see the comments in their Details Report.

Teacher Quality Barometer

Percentages shown in the teacher’s individual barometer and the cooperative barometer for all 140 teachers, are updated continuously as

additional evaluations are submitted.

The impressive barometer readings show the percentages of Very satisfied and Satisfied evaluations.

After a random anonymity delay, the student’s comments are shown in the Teacher Details Report.

Teacher Details Report

38

Example of detailed quality report for a teacher including anonymous verbal comments.

NKI’s transparent Teacher Quality Barometer was launched in 2011 for further quality improvements at Scandinavia’s leading online education provider.

On September 7, NKI’s 12.000 online students started evaluating their teachers.

On October12, NKI’s 140 online teachers could access the online evaluation results.

At the launch, 2000 evaluations were accrued and 88% were either Very satisfied or Satisfied.

Barometer Reading after 4017 Evaluations

40

2011Tutor quality barometer

Peer Evaluation of Teachers

41

In 2010, NKI introduced an online course structure for systematic peer evaluation among all the teachers. Random feedback the teachers have given on student submissions are sent to a colleague who evaluate it and give advice on possible improvements.

Professor Torstein Rekkedal has conducted a survey among the first participants and concluded that they perceive it as motivating and informative. They also reported that they learned at least as much from giving feedback as they did from receiving feedback. They will therefore both give and receive feedback on each other's work in the future.

Teacher Self-evaluation

Last school year, the majority of the teachers also delivered self-evaluation of one of the feedbacks they gave on student assignments. The experiences were so positive that future self-evaluation will be included in an online course structure in which the teachers will use the online submission system to evaluate their personal presentations, activity in forums and feedback on student submissions.

42

Star teachers deserve Golden Apples along with other stars who are awarded with gold medals, Oscars and Grammies. Therefore, NKI has introduced a new, and hopefully sustainable, scheme for nominating and awarding NKI-teachers of the year.

43

Thank youI care, so I share:

http://nettstudier.blogspot.com

http://home.nki.no/morten

https://twitter.com/MFPaulsen

www.slideshare.net/MortenFP

www.facebook.com/mfpaulsen

44

www.eden-online.org/blog