PublicServiceforLarge‐ScaleDigitalCollec5onsJohnMarkOckerbloom(Penn)

LeslieJohnston(LibraryofCongress)ChrisPowellandJeremyYork(Michigan)

DLFforumworkingsessionOctober31,2011

It’sallaboutcollabora5on•  Wecollaboratewithusers

•  Wecollaboratewitheachother

•  Adis5nc5vequalityoflibraries

FromaphotobyColleenMcMahon,CC‐BYhQp://www.flickr.com/photos/gatz125/5134393346/

Issueswe’lldiscuss

•  Whatdigitalpublicserviceaccomplishes•  Implemen5ngdigitalpublicservices•  Evalua5ngandscalingupdigitalpublicservices

•  We’llpresentquicktakesontheseissues,aswe’vedealtwiththeminourcollec5ons,inthefirsthalfofthissession

•  Thesecondhalfisyourturntodiscuss,andaskandanswerques5ons

Differentcollec5ons,differentscales

•  MLibraryDigitalCollec5ons– PresentedbyChrisPowell

•  HathiTrustDigitalLibrary– PresentedbyJeremyYork

•  Lib.ofCongressDigitalCollec5ons&Services– PresentedbyLeslieJohnston

•  TheOnlineBooksPage– PresentedbyJohnMarkOckerbloom

Qs:Goalsofdigitalpublicservice

•  Howdoyoufindoutwhoyourusersare,andwhattheymostneed?

•  What’sthemostusefulthinguserscandoforyou,andhowcanyoubestgetthemtodoit?

•  Howdoyouensureyou’repayingaQen5ontothepeopleandissuesyoushould?

•  Howdoesthetypeofcollec5on,interface,oraudienceaffectyourinterac5ons?

Qs:Implemen5ngdigitalpublicservice

•  Doyouhaveservicequalitygoalsandbenchmarks?Howdoyouevaluatethem?

•  Whodoesthework?Howdoyouallocatelabor,anddotriage?

•  Whatkindsoftechnologiesorimplementa5onsdoyoufindmostuseful?

•  Hasuserfeedbackpromptedyoutochangeyourdesignorwhatyoudo?How?

Qs:Evalua5ngandscalingupdigitalpublicservice

•  Doyouusepublicservicedatatojus5fycollec5onsandservicework?Ifso,whatdatadoyoufindmosteffec5ve?

•  Howcandifferentservicesandins5tu5onsmosteffec5velycollaboratewitheachother?

•  Whatelseisgoodtoknowaboutsuppor5nglarge‐scaledigitalcollec5onservices?

UniversityofMichiganDigitalLibraryProduc5onService

ChrisPowell

Lotsofstuff

•  Variousformats:– Con5nuoustoneimagecollec5ons

– Full‐textcollec5ons:encodedXMLandpageimages

– Non‐MARCmetadatacollec5ons– EAD‐encodedfindingaidcollec5ons

•  Varioussources•  Variousaccessmodels

Lotsofques5ons

•  Viaemailfrom1996‐2004•  MovedtoFootprintsinJune2004

•  Hardtoquan5fytheearlydays– Noformalprotocol– NoaQempttopreservetheques5onsoranswers–  IntermiQentlyincludedinoveralllibraryreferencesta5s5cs

Footprints

•  Centralizeemailfromusersofouronlineresources

•  Replytopeoplewhoprovidecontactinforma5on

•  Actonthosethatwecan,evenwithoutcontactinforma5on

•  Deletethosewecannotfollowupon•  Assignedtothoseresponsibleforthecontent

Afewnumbers

•  Justover5000“5ckets”inthesystem,openandclosed

•  25,0005cketshavebeendeleted– Spam– Anonymousandnotac5onable

•  Roughly650ofthe50005cketsarecategorizedasMBooksorHathiTrust

•  MovedHathiTrusttoitsownsysteminMarch

Whoareweserving?

•  UniversityofMichiganaffiliates•  Affiliatesofuniversi5eswhosubscribetomaterialswehost

•  Otherlibrariansandacademics

•  Otherstudents•  Thegeneralpublic

Whataretheyasking?

•  Topicsconsistentregardlessofcollec5on/site– Didyouknowthiswasbroken/wrong?– NowthatI’vefoundthis,howcanIgetaccess?– CanIusethisimageinmybook/project?–  Ihavethisoldbook–isitvaluable?– Canyouhelpmewiththisverycomplexquery?

•  ThepercentagesvaryinHathiTrust– Moreaboutcontenterrorsandaccessdesires

WhataretheynotaskingaboutHathiTrust?

•  Ques5onsrelatedtothecontent•  Manyrepeatedques5onsinsomeofourcollec5ons–  WhatisApocrypha?WhyisEcclesiates9showingasQoh.9?Whyisn’t

thisquoteinyouronlineCollectedWorksofAbrahamLincoln?Whycan’tIfindtheverse9:11aboutthewrathoftheeagleinyouronlineKoran?

•  Alsogeneralques5ons–  Thiscensusfrom1901listspeopleborninthistown,butnoneofthe

gazeQeersoratlasesshowatownofthisname.ThePhiladelphiaCentennialexhibiFonhistorysaysthis,asouvenirmedallionfromarelaFvesayssomethingelse.Whichisright?Isitfake?AreyousurethesepoemsinCatholicWorldarebyWilliamGibson?

Observa5ons

•  HathiTrustfeedbackformsguideyoutoerror/accessrepor5ng,notsubjectinquiries

•  Single5tlesites(Bibles,Koran,Lincoln)getthemostrepeatedcontentques5ons

•  MoAgetsmanycontentques5ons,buthasaunifying(ifgeneral)theme

•  Affilia5onwithMLibrarymaymakecontentques5onsseemmorewelcome

•  Dataskewedbysomuchpastprac5ce

HathiTrustUserSupport

JeremyYork

Timeline

•  October2008–HathiTrustlaunched– hathitrust‐info@umich.edu

•  February2010–star5ngregularsta5s5cs•  August2010–Chrisjoinedhathitrust‐info•  UsersupportgroupsinceApril2011

WhatDigitalPublicServiceAccomplishes(1)

•  Howdoyoufindoutwhoyourusersare,andwhattheymostneed?–  Userfeedbackviathefeedbacklink–  Frequentdiscussionswithreference&instruc5onlibrarianswhosharetheir"front‐line"encounterswithusers

–  HathiTrustUXAdvisoryGroup&UX‐SIGforinterestedstaffatpartnerins5tu5onstodiscussandbringissuestothetable•  Developingsetofpersonasandscenarios

– Watchblogs&twiQerforunsolicitedfeedback–  Somewhatregularuserresearch(surveys)

Usesta5s5cs

Visits %newvisits Pageviews Pages/visit Timeonsite

Jan‐Jun2010 253,129 69% 2,154,385 11.6 6.3min

Jul‐Dec2010 907,524 75% 8,443,692 9.3 5.3min

Jan‐Jun2011 2,154,385 83% 14,945,119 6.9 4min

Jul‐Oct2010 2,274,468 84% 12,072,991 5.3 3.3min

WhatDigitalPublicServiceAccomplishes(1)

•  What’sthemostusefulthinguserscandoforyou,andhowcanyoubestgetthemtodoit?– Letusknowofproblems,issues,desires(contentandservices)

– Peoplewhoareinterestedletusknow

UserSupportIssues

0

50

100

150

200

250

300

350

400

450

Apr‐Jun2011 Jul‐11 Aug‐11 Sep‐11

Content

Cataloging

AccessandUse

WebApplica5ons

PartnerIngest

General

IssueType AugustIssues SeptemberIssues

Content 110 171Quality 96 154Non‐partnerDigitalDeposit 3 2Collections 8 4

Cataloging 26 25AccessandUse 111 127

Copyright 58 73Permissions 23 12Takedown 2 3PrintonDemand 6 17Inter‐libraryloan 0 5Full‐PDFore‐copyrequests 14 24Datasets 1 1DataAvailabilityandAPIs 1 7Reuseofcontent 7 5

Webapplications 27 22Functionalityproblems 5 5ProblemswithloginspeciNically

1 0

GeneralQuestionsaboutlogin 3 2Partnerssettinguplogin 4 5Usabilityissues 11 6Featurerequests 7 2

PartnerIngest 2 0General 59 65

Partnership 13 12Infrastructure 1 0Miscellaneous 45 53

hQp://www.hathitrust.org/wg_user_support_issue_types

WhatDigitalPublicServiceAccomplishes(2)

•  Howdoyouensureyou’repayingaQen5ontothepeopleandissuesyoushould?– Ensurethatwelookatmanydifferentsourcestogetthelargestvarietypossible.

– Wereceive,find,andsolicitfeedbackinwaysnotedabove

WhatDigitalPublicServiceAccomplishes(3)

•  Howdoesthetypeofcollec5on,interface,oraudienceaffectyourinterac5ons?– WearepreQyuniform

Implemen5ngDigitalPublicServices(1)

•  Doyouhaveservicegoalsandbenchmarks?Howdoyouevaluatethem?– Respondtofeedbackwithin1businessday– Handleissuessuchastake‐downno5cesandcri5calsystemsproblemsimmediately

– Nogoalsforresolvingotherproblems,butcorrec5onshappenpreQyquickly

– Usersupportasoutreachmechanism•  Importanttohowweareseenbythecommunity

Implemen5ngDigitalPublicServices(2)

•  Howdoyouallocatestaff,anddotriage?– Usersupportgroup– Rota5onof24‐hour“on‐call”periods

Implemen5ngDigitalPublicServices(3)

•  Whatkindsoftechnologiesorimplementa5onsdoyoufindmostuseful?

Users(partnerandnon)

JIRA

UserSupportWorkingGroup

UniversityofMichiganContacts•  Copyright•  Quality•  PrintonDemand

AllHathiTrustContacts

Documenta5on

Implemen5ngDigitalPublicServices(4)

•  Haveyouchangedyourdesignorwhatyoudobasedonuserfeedback?How?– PageTurnerimprovements•  BookReaderintegra5onandaccompanyinginterfacereorganiza5on;full‐screenmechanism•  Advancedsearchfeaturesforfull‐textsearch•  LabelingforPDFdownload

Evalua5ngandScalingUpDigitalPublicService(1)

•  Doyouusepublicservicedatatojus5fycollec5onsandservicework?Ifso,whatdoyoufindmosteffec5ve?– No,butkeyfactorinexpandingtoworkinggroupandhavingpartnersystemwascommunica5on

•  Howcandifferentservicesandins5tu5onsmosteffec5velycollaboratewitheachother?– Commonly‐usablesystem– Definedroles(contacts,etc.)– Definedworkflowsandprocedures

•  Whatelseisgoodtoknowaboutsuppor5nglarge‐scaledigitalcollec5onservices?– Factoritin,don’tunderes5mate5meneeded

Evalua5ngandScalingUpDigitalPublicService(2)

LibraryofCongressDigitalCollec5ons

LeslieJohnston

SoMuchStuff,It’sUsedasaUnitofMeasure

•  Whatcons5tutesa“LibraryofCongress”worthofdigitalcontentchangesallthe5me.

•  Ahugevarietyofformats:FullText,PageImages,ImageCollec5ons,FindingAids,ElectronicSerials,Video,Audio,Legisla5on,WebArchives.– Anes5mateof24.6millionfilesinthemainLCwebpresenceattheendof2010.

•  Avarietyofacquisi5onmethods,includingthroughtheUSCopyrightOffice

•  Avarietyofaccessrules(includedclassified)andaccessmethodsandsystems.

Ques5onsareAlwaysComingIn

•  Viaemail•  Via“AskaLibrarian”OnlineReference•  InPerson•  In2010,Librarystaffansweredover191,000emailoronlinereferenceques5ons.

Whoareweserving?

•  Congress•  Librariansatotherins5tu5ons•  AcademicResearchers

•  TheGeneralPublic

WhatDoTheyWanttoKnow?

•  DoyouhavethisthingthatIcouldn’tfind?Youmusthaveit,becausetheLibraryofCongresshaseverything.

•  Whyisn’teverythingfulltext?•  HowdoIgettherightstouseyourcollec5ons,andget

permissionstousetheminmyresearchorinapublica5on?•  HowcanIgetcopiesofyourdigitalfiles?•  Willyoudigi5zesomethinginyourcollec5onsforme?•  HowdoIciteyouronlinecollec5ons?•  Pleasefixthiserrorinyourwebsite,orinyour

bibliographicorauthorityrecords.•  CanIdonatedigitalcollec5onstotheLibrary?•  WilltheLibraryhelpmedigi5zemycollec5on?•  Whatstandardsdoyouusefordigi5zing?

WhatDigitalPublicServiceAccomplishes

•  Howwelearnsomethingaboutwhooutusersare,andwhattheyneedmost:– Interac5onswithDigitalReferencelibrarians– UserfeedbackviatheContactForm– CommentsontheLibrary’sblogs– ShareToolandWeblogsta5s5cs• Over580millionpage‐viewsoftheLibrary’swebsitein2010.

WhatDigitalPublicServiceAccomplishes

• Whatwelearn:– Whatarewedoingwrong?OneissueisthecomplexityoftheLibrary’swebpresence.Wenowhaveanini5a5vetomakeiteasiertofinddigitalcollec5onswithoutknowingabouttheminthefirstplace.

– Whatarewedoingright?Wegetmoreposi5vereac5onsandthanksfordoingwhatwedothannega5ve.

WhatDigitalPublicServiceAccomplishes

•  Howdoyouensureyou’repayingaQen5ontothepeopleandissuesyoushould?– Wereadthemessagesthatweget.– Weholdourselvestoaveryhighstandardofrespondingtoallreferenceinquiries.

WhatDigitalPublicServiceAccomplishes

•  Howdoesthetypeofcollec5on,interface,oraudienceaffectyourinterac5ons?– OurfeedbackfromCongressisdifferentfromourfeedbackfromthepublic•  Congressmakesdirectdigitalcollec5onbuildingandservicesdependingonthetasksathand•  Researchersandthepublicarerela5velyuniform

Implemen5ngDigitalPublicServices

•  Doyouhaveservicegoalsandbenchmarks?Howdoyouevaluatethem?– Respondtoques5onsandfeedbackwithin5businessdays

– 4‐6week5meframefordigitalreproduc5onrequests

– Nobenchmarksforresolvingotherissues

Implemen5ngDigitalPublicServices

•  Howdoyouallocatestaff,anddotriage?– Variesbycustodialunitandcollec5on– DigitalReferenceTeamshaspostedchathours,andarota5ngdutyroster

Implemen5ngDigitalPublicServices

• Whatkindsoftechnologiesorimplementa5onsdoyoufindmostuseful?– OnlineChat– Contact/FeedbackForms– OnlineFAQs– Documentedonlineguidelinesforuseofthecollec5ons

– Documentedstandardsonline

Implemen5ngDigitalPublicServices

•  Haveyouchangedyourdesignorwhatyoudobasedonuserfeedback?How?– Ofpublicservices?Notreally.– Ofouroverallwebpresence?Thatishappeningnow,tohelpusersbeQerdiscoverourcollec5ons.

Evalua5ngandScalingUpDigitalPublicService

•  Doyouusepublicservicedatatojus5fycollec5onsandservicework?Ifso,whatdoyoufindmosteffec5ve?– No,wedonot.Butthatdoesn’tmeanthatwedon’treportonthem.

•  Howcandifferentservicesandins5tu5onsmosteffec5velycollaboratewitheachother?–  Sharemoreinforma5onaboutwhatcollec5onsanddigi5zedcollec5onswehave•  Explicitlydocumentrightsformediafilesandmetadata•  Makethemavailableaslinkedopendatawherewecan

–  Common‐ishstandards,oratleastwell‐documentedstandards

–  Consistent,documentedworkflows•  Whatelseisgoodtoknowaboutsuppor5nglarge‐scaledigitalcollec5onservices?–  Thescalereallydoesmakeadifference.Ourresponse5mesarealwaysgoingtobeslower.

Evalua5ngandScalingUpDigitalPublicService

TheOnlineBooksPage

JohnMarkOckerbloom

•  1personworkingpart‐5mefallsbehind–  Bothincatalog,andinrespondingtopublic

•  Scalingupthecatalog– Metadataautoma5callydownloadedfromHathiTrust,othersources

–  Icon5nuetoaddnewentries,“curate”auto‐loadedentries,onmyownandatuserrequest

•  Scalingupuserservice–  Invitepeopletostandardforms– Makeefficientbackendtodealwithrequests

•  Ul5mately,nosubs5tuteforhumaninvestment

Tes5ngthelimitsofalow‐resourcecatalogproject,since1993

Feedbacklinksinresults

Requestcura5on

Userstellmewhattheywant

Andcanhearsomethingback

WhatIseeonmyend

EnsuringIgettoeveryone

•  Numberofonlinebookss5lltogrowlots–  EspeciallyasIaddmoreautoma5callyloadedsources

•  Staff,budgetnotlikelytogrow–  Userfeedbackhelpsmedeterminewhereeffortbestspent

•  Enhancedback‐endinterfacemaymakemul5plemaintainersfeasible

•  Maybeusefulcollabora5onswithinterns,volunteers–  Awaytogethands‐onexperiencewithlibrarianship

•  Sharingdatahelpsotherservicesbuildonmywork–  Enhancing“regular”librarycollec5onsmayenablesupport

Lookingahead:Collabora5on

Discuss:Goalsofdigitalpublicservice

•  Howdoyoufindoutwhoyourusersare,andwhattheymostneed?

•  What’sthemostusefulthinguserscandoforyou,andhowcanyoubestgetthemtodoit?

•  Howdoyouensureyou’repayingaQen5ontothepeopleandissuesyoushould?

•  Howdoesthetypeofcollec5on,interface,oraudienceaffectyourinterac5ons?

Discuss:Implemen5ngdigitalpublicservice

•  Doyouhaveservicequalitygoalsandbenchmarks?Howdoyouevaluatethem?

•  Whodoesthework?Howdoyouallocatelabor,anddotriage?

•  Whatkindsoftechnologiesorimplementa5onsdoyoufindmostuseful?

•  Hasuserfeedbackpromptedyoutochangeyourdesignorwhatyoudo?How?

Discuss:Evalua5ngandscalingupdigitalpublicservice

•  Doyouusepublicservicedatatojus5fycollec5onsandservicework?Ifso,whatdatadoyoufindmosteffec5ve?

•  Howcandifferentservicesandins5tu5onsmosteffec5velycollaboratewitheachother?

•  Whatelseisgoodtoknowaboutsuppor5nglarge‐scaledigitalcollec5onservices?– Whathaveyoulearnedthatyoudidn’texpect?

Collabora5on

PhotobyMaryMarkOckerbloom