learning about learning 2.0: evaluating the new south wales public library learning 2.0 program

Download Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program

Post on 30-May-2018




0 download

Embed Size (px)


  • 8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program



    Ellen Forsyth, Mylee Joseph and Leanne Perry

    Ellen Forsyth, State Library of New South Wales, Sydney 2000. E-mail:eforsyth@sl.nsw.gov.au. Mylee Joseph, State Library of New South Wales, Sydney2000. E-mail: mjoseph@sl.nsw.gov.au. Leanne Perry, State Library of New SouthWales, Sydney 2000. E-mail: lperry@sl.nsw.gov.au.

    Presented at Research Applications in Information and Library StudiesConference 5 (RAILS5) University of Technology, Sydney 23 January, 2009.

    AbstractThe New South Wales Learning 2.0 training program was made available to

    the entire public library work force of over 2300 people in 2008 - 2009. Thispaper is a snapshot of the impact the Learning 2.0 course at September 2008,five months after the course was launched. The paper explores how thetraining has impacted on the level of skill, knowledge, and confidence relatingto web 2.0 tools.


    The aim of this evaluation of the New South Wales public library Learning 2.0program is to measure the impact of self paced web 2.0 training on individualsworking in New South Wales (NSW) public libraries. In this paper the focus is

    on how the training has impacted on the level of skill, knowledge, andconfidence relating to web 2.0 tools.

    Background to the Learning 2.0 Program

    2,344 staff are employed in 363 local government public libraries and onboard 22 mobile libraries throughout New South Wales1. From discussionsand anecdotal evidence, it became apparent to Public Library Services staff atthe State Library that a significant proportion of public library staff did notpossess the knowledge or technical skills required to fully participate in therapidly changing library service environment, including engagement with web2.0 technologies, which is emerging to meet changing client expectations inan increasingly interactive online environment.

    The New South Wales public library Learning 2.0program (Learning 2.0)2 isan online self-paced training program to allow public library staff across NewSouth Wales to learn more about emerging technologies on the web. Theprogram was written by Ellen Forsyth and Mylee Joseph. They work in thePublic Library Services branch at the State Library of New South Wales. Thisteam is responsible for promotion and development of public library servicesin New South Wales. The course was based on Learning 2.0 programs from

    Orange County Library System in Florida


    and King County Library System inWashington4. Learning 2.0 is a twelve week online self-discovery training

  • 8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program


    program designed to encourage library staff to explore new web-basedtechnologies like blogs, wikis and podcasts. Each week there is a module towork through which introduces participants to new web 2.0 tools. There willoften be a video to explain the tool, participants will see examples of goodpractice library and museum implementations, and there will be an exercise to

    try tools for themselves. The course included setting up a blog, exploringFlickr, tagging and Creative Commons; RSS and RSS aggregators, Wikipediaand other wikis; online video including Youtube and Google video,folksonomies, delicious and LibraryThing, answer boards, social searching,podcasts and audio, mashups, online applications and social networks.

    Each participant maintains a learning blog where they write about what theyhave discovered each week. Participants are encouraged to comment oneach others blogs. The original program was designed by Helene Blowers forthe Public Library of Charlotte and Mecklenburg County5. It has a creativecommons licence which has allowed other programs to be developed,

    including the programs at Orange County Library System in Florida6 and KingCounty Library System in Washington and the State Library of New SouthWales. The training program is completely built on web 2.0 technologies thatare freely available on the Internet. These sites include: Blogger, Flickr,YouTube, PBWiki & Bloglines.

    The New South Wales public library Learning 2.0 program was publicisedextensively through e-mail and meetings. All public library staff wereencouraged to participate. It was presented as a course in which everyonecould participate. There were no barriers to participation from the StateLibrary. Barriers to participation occurred due to local council policies onaccess to some sites (particularly YouTube and online e-mail), speed ofconnections (broadband is not everywhere) and the level of interest ofparticipants.Course material is accessed via the internet allowing staff in rural and regionalcentres equal access to the program. As there is no prior knowledgerequired, staff with any skill level can participate. The program is self paced,allowing flexibility as to when and where it is undertaken, and how long aperson takes to complete the course. The program is built using the sameweb 2.0 tools included in the course. It is published by the State Library of

    New South Wales with a Creative Commons licence allowing it to be easilyshared with other agencies.

    Background on the learning 2.0 evaluation what is happeningelsewhereWhile there are more than 250 individual 23 things or Learning 2.0programs7 either completed or currently underway in almost 500 participatinglibraries worldwide, there is limited information available around the evaluationand outcomes of these programs. Where information is available it is clearthat the evaluation of the success of programs has been largely measured vianumbers of completions and feedback received via participants through blog


  • 8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program


    Some programs have actively sought feedback via surveys at the end of theprogram, typically an online evaluation in the last week of the program forexample the Minnesota Multitypes 23 Things on a Stick program.8Generallysurveys have focussed on the participant assessment of the program,including reactions to the program, both initially and at the end of the program,

    favourite and least favourite parts of the program, what was most challengingabout the program, possible improvements for the program and ways web 2.0applications could be used in the participants library service.

    Recently there has been a move to establish clear evidence of the impact ofthe Learning 2.0 programs on library services. Helene Blowers, the designerof the original Learning 2.0 program, 23 things9 has initiated a survey forLearning 2.0 program co-ordinators. The survey seeks to evaluate howsuccessful the program has been on a global level and what are the keyfactors for success including issues such as organisational support andincentives. The survey also attempts to measure some of the broad impacts

    of the program rather than just the program itself, by including questions onthe outcomes for staff and the development of new services as a result of theprogram. Blowers survey of more than 700 course coordinators around theworld was released in August 200810 and the survey results were released inJuly 2009.11 This was a very high level survey which mainly asked aboutmotivation for participation, openness of participation, use of incentives andmanagement support

    The NSW Public Library Services Learning 2.0evaluation included a range ofstrategies which sought to measure both satisfaction of participants with theprogram and the impact of the program on knowledge and use of web 2.0tools at an individual and library service level. As with other Learning 2.0evaluations feedback received via program and participant blogs were a keysource of information for evaluating the success of the program. However,unlike most other evaluations we undertook to measure the change in skilllevel of participants in web 2.0 technologies over the course of the programand at nine month post program completion. Although Mayo Clinic LibrariesLearning 2.0 program have also undertaken some evaluation of the impact oftheir program on individual participants skill levels there is little evidence thatthis level of evaluation has been undertaken in a public library Learning 2.0program context.12

    Research methodologyThere are six components to the research methodologyCompletion ratesProgress of participants through the program was monitored and recorded bythe project team leaders. This information included the number of peoplestarting, the number who reached mid point and the number who completed.

    Self assessment of skills and knowledgeEach participant completed an anonymous self assessment survey at thebeginning of the training program. These were deliberately anonymous as it

    was important for people to be able to answer honestly. The survey wasdesigned to give a realistic snapshot of participant skills and understanding. It

  • 8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program


    was considered more important to have a state wide overview rather thantracking changes to individuals.

    Mid point surveyThe mid point survey was included in the activities for week six to check how

    participants were managing online learning. The survey was designed toidentify issues with the program and online learning, what was working andwhat wasnt. It was also des