Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program

Download Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program

Post on 30-May-2018




0 download

Embed Size (px)


<ul><li><p>8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program</p><p> 1/23</p><p>LEARNING ABOUT LEARNING 2.0: EVALUATING THE NEW SOUTHWALES PUBLIC LIBRARY LEARNING 2.0PROGRAM</p><p>Ellen Forsyth, Mylee Joseph and Leanne Perry</p><p>Ellen Forsyth, State Library of New South Wales, Sydney 2000. E-mail:eforsyth@sl.nsw.gov.au. Mylee Joseph, State Library of New South Wales, Sydney2000. E-mail: mjoseph@sl.nsw.gov.au. Leanne Perry, State Library of New SouthWales, Sydney 2000. E-mail: lperry@sl.nsw.gov.au.</p><p>Presented at Research Applications in Information and Library StudiesConference 5 (RAILS5) University of Technology, Sydney 23 January, 2009.</p><p>AbstractThe New South Wales Learning 2.0 training program was made available to</p><p>the entire public library work force of over 2300 people in 2008 - 2009. Thispaper is a snapshot of the impact the Learning 2.0 course at September 2008,five months after the course was launched. The paper explores how thetraining has impacted on the level of skill, knowledge, and confidence relatingto web 2.0 tools.</p><p>Aim</p><p>The aim of this evaluation of the New South Wales public library Learning 2.0program is to measure the impact of self paced web 2.0 training on individualsworking in New South Wales (NSW) public libraries. In this paper the focus is</p><p>on how the training has impacted on the level of skill, knowledge, andconfidence relating to web 2.0 tools.</p><p>Background to the Learning 2.0 Program</p><p>2,344 staff are employed in 363 local government public libraries and onboard 22 mobile libraries throughout New South Wales1. From discussionsand anecdotal evidence, it became apparent to Public Library Services staff atthe State Library that a significant proportion of public library staff did notpossess the knowledge or technical skills required to fully participate in therapidly changing library service environment, including engagement with web2.0 technologies, which is emerging to meet changing client expectations inan increasingly interactive online environment.</p><p>The New South Wales public library Learning 2.0program (Learning 2.0)2 isan online self-paced training program to allow public library staff across NewSouth Wales to learn more about emerging technologies on the web. Theprogram was written by Ellen Forsyth and Mylee Joseph. They work in thePublic Library Services branch at the State Library of New South Wales. Thisteam is responsible for promotion and development of public library servicesin New South Wales. The course was based on Learning 2.0 programs from</p><p>Orange County Library System in Florida</p><p>3</p><p>and King County Library System inWashington4. Learning 2.0 is a twelve week online self-discovery training</p>mailto:eforsyth@sl.nsw.gov.aumailto:mjoseph@sl.nsw.gov.aumailto:mjoseph@sl.nsw.gov.aumailto:lperry@sl.nsw.gov.aumailto:eforsyth@sl.nsw.gov.aumailto:mjoseph@sl.nsw.gov.aumailto:lperry@sl.nsw.gov.au</li><li><p>8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program</p><p> 2/23</p><p>program designed to encourage library staff to explore new web-basedtechnologies like blogs, wikis and podcasts. Each week there is a module towork through which introduces participants to new web 2.0 tools. There willoften be a video to explain the tool, participants will see examples of goodpractice library and museum implementations, and there will be an exercise to</p><p>try tools for themselves. The course included setting up a blog, exploringFlickr, tagging and Creative Commons; RSS and RSS aggregators, Wikipediaand other wikis; online video including Youtube and Google video,folksonomies, delicious and LibraryThing, answer boards, social searching,podcasts and audio, mashups, online applications and social networks.</p><p>Each participant maintains a learning blog where they write about what theyhave discovered each week. Participants are encouraged to comment oneach others blogs. The original program was designed by Helene Blowers forthe Public Library of Charlotte and Mecklenburg County5. It has a creativecommons licence which has allowed other programs to be developed,</p><p>including the programs at Orange County Library System in Florida6 and KingCounty Library System in Washington and the State Library of New SouthWales. The training program is completely built on web 2.0 technologies thatare freely available on the Internet. These sites include: Blogger, Flickr,YouTube, PBWiki &amp; Bloglines.</p><p>The New South Wales public library Learning 2.0 program was publicisedextensively through e-mail and meetings. All public library staff wereencouraged to participate. It was presented as a course in which everyonecould participate. There were no barriers to participation from the StateLibrary. Barriers to participation occurred due to local council policies onaccess to some sites (particularly YouTube and online e-mail), speed ofconnections (broadband is not everywhere) and the level of interest ofparticipants.Course material is accessed via the internet allowing staff in rural and regionalcentres equal access to the program. As there is no prior knowledgerequired, staff with any skill level can participate. The program is self paced,allowing flexibility as to when and where it is undertaken, and how long aperson takes to complete the course. The program is built using the sameweb 2.0 tools included in the course. It is published by the State Library of</p><p>New South Wales with a Creative Commons licence allowing it to be easilyshared with other agencies.</p><p>Background on the learning 2.0 evaluation what is happeningelsewhereWhile there are more than 250 individual 23 things or Learning 2.0programs7 either completed or currently underway in almost 500 participatinglibraries worldwide, there is limited information available around the evaluationand outcomes of these programs. Where information is available it is clearthat the evaluation of the success of programs has been largely measured vianumbers of completions and feedback received via participants through blog</p><p>entries.</p></li><li><p>8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program</p><p> 3/23</p><p>Some programs have actively sought feedback via surveys at the end of theprogram, typically an online evaluation in the last week of the program forexample the Minnesota Multitypes 23 Things on a Stick program.8Generallysurveys have focussed on the participant assessment of the program,including reactions to the program, both initially and at the end of the program,</p><p>favourite and least favourite parts of the program, what was most challengingabout the program, possible improvements for the program and ways web 2.0applications could be used in the participants library service.</p><p>Recently there has been a move to establish clear evidence of the impact ofthe Learning 2.0 programs on library services. Helene Blowers, the designerof the original Learning 2.0 program, 23 things9 has initiated a survey forLearning 2.0 program co-ordinators. The survey seeks to evaluate howsuccessful the program has been on a global level and what are the keyfactors for success including issues such as organisational support andincentives. The survey also attempts to measure some of the broad impacts</p><p>of the program rather than just the program itself, by including questions onthe outcomes for staff and the development of new services as a result of theprogram. Blowers survey of more than 700 course coordinators around theworld was released in August 200810 and the survey results were released inJuly 2009.11 This was a very high level survey which mainly asked aboutmotivation for participation, openness of participation, use of incentives andmanagement support</p><p>The NSW Public Library Services Learning 2.0evaluation included a range ofstrategies which sought to measure both satisfaction of participants with theprogram and the impact of the program on knowledge and use of web 2.0tools at an individual and library service level. As with other Learning 2.0evaluations feedback received via program and participant blogs were a keysource of information for evaluating the success of the program. However,unlike most other evaluations we undertook to measure the change in skilllevel of participants in web 2.0 technologies over the course of the programand at nine month post program completion. Although Mayo Clinic LibrariesLearning 2.0 program have also undertaken some evaluation of the impact oftheir program on individual participants skill levels there is little evidence thatthis level of evaluation has been undertaken in a public library Learning 2.0program context.12</p><p>Research methodologyThere are six components to the research methodologyCompletion ratesProgress of participants through the program was monitored and recorded bythe project team leaders. This information included the number of peoplestarting, the number who reached mid point and the number who completed.</p><p>Self assessment of skills and knowledgeEach participant completed an anonymous self assessment survey at thebeginning of the training program. These were deliberately anonymous as it</p><p>was important for people to be able to answer honestly. The survey wasdesigned to give a realistic snapshot of participant skills and understanding. It</p></li><li><p>8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program</p><p> 4/23</p><p>was considered more important to have a state wide overview rather thantracking changes to individuals.</p><p>Mid point surveyThe mid point survey was included in the activities for week six to check how</p><p>participants were managing online learning. The survey was designed toidentify issues with the program and online learning, what was working andwhat wasnt. It was also designed to find if the current level of support beingprovided was effective.</p><p>Completion surveyThe survey, at the end of the program, repeated the questions from the midpoint survey and the self assessment. The purpose of this survey was toidentify the changes in participants perceptions of their skills and knowledgeof web 2.0 tools, and to comment on the content of, and support for, theprogram. This survey may be skewed by the high achievers completing and</p><p>others falling by the wayside and not completing the survey.</p><p>Nine month post completion surveyA nine month completion survey will be undertaken with all participants whocomplete the program. This survey will explore the longer impact of theLearning 2.0 program on library services. The survey will include looking atthe tools people are using in their libraries, what policies have changed andhow skill levels have been maintained or enhanced. This survey wasundertaken in March and the results are being analysed.</p><p>Feedback on blogsThe program was designed to elicit comments and feedback on both theprocess and progress of learning about web 2.0 tools. All participants wererequired to establish a learning diary blog which was publicly visible, linked tothe course blog roll but did not disclose their identity or library. The progressof individuals was monitored by comments and feedback posted on theirlearning blogs, and comments posted on the administrators blog.</p><p>Evaluation results</p><p>1. Program participants, progress and completion rates</p><p>More than 10% of the public library workforce in New South Wales havesuccessfully completed the program to date. While this is reasonable, initialexpectations of completion rates were much higher. Over one third of theNew South Wales public library workforce started this program. This was thelargest number of people to ever to participate in training provided by theState Library. A snapshot of the variety of roles is useful in assessingparticipation across the profile of the NSW public library workforce.</p><p>Table 1 Role of people undertaking the program</p><p>Roles No. enrolled</p><p>Library Manager 52Team leader / coordinator 91</p></li><li><p>8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program</p><p> 5/23</p><p>Branch librarian / Branchlibrary officer</p><p>96</p><p>Specialist [eg. children's,reference, local studiesetc.]</p><p>176</p><p>Customer service officer 17Mobile library driver 4</p><p>Library officer / technician 165</p><p>Library assistant 240</p><p>Other 76</p><p>Total 917</p><p>The following table shows the numbers of people undertaking the program, bytype of public library at the start, midpoint and completion of the program.</p><p>Table 2 Completion rates</p><p>Start Mid point Completion</p><p>Branch Library 331 107 87</p><p>Central Library 446 149 110</p><p>Regional headquarters 67 29 18</p><p>Mobile library 6 4 2</p><p>Joint use library 7 1 3</p><p>Other 36 6 6</p><p>Total 893 296 226</p><p>Note the different totals in the tables are because not everyone chose to</p><p>provide information about the kind of library they work in.</p><p>The mix of library workplaces was useful in assessing the accessibility of thecourse and any barriers to participation and completion. The results indicatedthat staff working at a branch library were just as likely to complete the courseas those working in central and headquarters locations, ie. there were norelevant geographical barriers and staff at branch locations were equally selfmotivated to complete the course. This is useful information for planningfuture professional development activities for the NSW public library network.</p><p>2. Self assessment of skills and knowledge</p><p>The self assessment undertaken by participants at the mid point andcompletion of the program provides a clear indication of an increase inparticipant skill and knowledge level of the web 2.0 tools. As illustrated inTables 3 and 4, a clear shift in skill level was indicated in all web 2.0 tools.The shift followed a consistent pattern of moving up one stage in terms of themajority of participants skill levels. In all cases where the majority ofparticipants indicated that they were unfamiliarwith a web 2.0 tool prior to thecommencement of the course the majority of participants moved to the newuser skill and knowledge levels. Similarly where the majority of participants</p><p>indicated that they were new users prior to the commencement of the courseat the completion of the course the majority of these participants had moved</p></li><li><p>8/14/2019 Learning about learning 2.0: evaluating the New South Wales public library Learning 2.0 program</p><p> 6/23</p><p>up one level to userskill and knowledge levels. By the completion of thecourse at least 82% of participants indicated they were at new user level orabove for all web 2.0 tools.</p><p>Table 3.Self assessment prior to commencing the Learning 2.0 program</p><p>0%</p><p>10%</p><p>20%</p><p>30%40%</p><p>50%</p><p>60%</p><p>70%</p><p>80%</p><p>90%</p><p>100%</p><p>Blogs</p><p>ollaboration</p><p>oftware[eg.</p><p>agging[eg.</p><p>el.icio.us,</p><p>Online</p><p>project</p><p>Graphic</p><p>organizer</p><p>RSS</p><p>Podcasts</p><p>and</p><p>Wikisand</p><p>Wikipedia</p><p>Social</p><p>networking</p><p>Instant</p><p>messaging</p><p>nlinevideo</p><p>[eg.</p><p>igitalonline</p><p>usic[eg.</p><p>Digital</p><p>ictures[eg.</p><p>Mashups</p><p>Answer</p><p>boards[eg.</p><p>Internet</p><p>safety</p><p>Power U</p><p>User</p><p>New Use</p><p>Unfamilia</p><p>Table 4. Self assessment at completion of the Library 2.0 program</p><p>0%</p><p>10%</p><p>20%</p><p>30%</p><p>40%</p><p>50%</p><p>60%</p><p>70%</p><p>80%</p><p>90%</p><p>100%</p><p>Blogs</p><p>ollaboration</p><p>oftware[eg.</p><p>agging[eg.</p><p>del.icio.us,</p><p>Online</p><p>project</p><p>Graphic</p><p>organizer</p><p>RSS</p><p>Podcasts</p><p>and</p><p>Wi...</p></li></ul>