Prepared for the Charleston Conference

Karen CalhounJanet Hawk

Online catalogs: What users and librarians wantA review of market research data

Online catalogs: What users and librarians wantA review of market research data

7 November 2008

With thanks to

Joanne Cantrell,

OCLC Market Research Analyst

Photo by allw3ndy

http://flickr.com/photos/allw3ndy/2757149584/

What did catalog quality mean in 1989?

What did catalog quality mean in 1989?Davis, Carol C. 1989. “Results of a survey on record quality

in the OCLC database.” Technical Services Quarterly. 7 (2):43-53.

Duplicate records

Bad name headings

Bad subject headings

The perception of “quality”: The eye of the beholderThe perception of “quality”: The eye of the beholder

Specialist’s view:

• Conformance to specifications

(rules)

• Priorities: Fullness and detail

Pragmatist’s view:

• Make as many materials as

possible available as quickly as

possible

• Priorities: speed and efficiency

End-user’s view:

• Easy and convenient

30-second summary of online catalog user studies30-second summary of online catalog user studies

Keyword searching reigns

The default search is chosen most often

Number of terms in a query: 1 to 3

Search failure rate (zero hits) is very high: 20 to 40 percent

The latest study:Moulaison, Heather L. 2008. “OPAC queries at a medium-sized academic library: a Transaction log analysis.” LRTS 52 (4): 230-237.

Will Google Books usurp the library catalog?Will Google Books usurp the library catalog?

Ludwig, Mark J. and Margaret R. Wells. “Google Books vs. BISON.” LibraryJournal, July 15, 2008. http://www.libraryjournal.com/article/CA6566451.html

0

5

10

15

20

25

Num

ber o

f Uniq

ue W

ord

s

Unique words contributed successivelyby subject-rich fields: records with TOC and summary notes

LCSH

Title

TOC and Summaries

Series

Markey, Karen and Karen Calhoun.1987. “Unique words contributed by MARC records with summary and/or contents notes.” Proceedings of the 50th ASIS Annual Meeting (Medford NJ: Learned Information), p. 153–162.

LCSH: from 3 to 7 wordsper record

Assumptions and mindsets: Where do subject-rich index terms come from?

Assumptions and mindsets: What is “full”?Assumptions and mindsets: What is “full”?

+ 3 more screens

Productdescriptionand purchaseinformation;‘More like this’

Editorialreviews andauthor info

‘Inside thebook’ tags, ratings,customerreviews,lists and more

With thanks to David Lankes: http://quartz.syr.edu/rdlankes/Presentations/2007/ALCTS.pdf

Bibliographicinformation

Library holdingsDetailsSubjectsEditionsReviews

Bibliographicinformation

Australianlibrary holdings

What Is online catalog “quality”?What Is online catalog “quality”?

“A persistent shortcoming in the

decision-making process [about

library database quality] that needs

to be addressed is the lack of

serious research into user needs

and benefits, and the actual impact

on users of database quality

decisions.”

—Janet Swan Hill

Hill, Janet Swan. 2008. “Is it worth it? Management decisions related to database

quality.” CCQ 46 (1): 5-26.

“You need more book descriptions. Telling me the author name and book title does not tell

me what a book is about.” -High school student-

“I would like to preview actual pages from the books. This would greatly help me educate myself on the

subject matter presented and get a

sense of what the book actually offers.” -College student-

“Please link me to the item i'm searching

for.”

-Graduate student-

Objectives of our metadata quality researchObjectives of our metadata quality research

•Start over with a blank page

•Identify and compare metadata expectations

• End users

• Librarians

•Compare expectations of types of librarians

•Determine end-user satisfaction with WorldCat.org

•Define a new WorldCat quality program

•Considering the perspectives of all constituencies of WorldCat

• End users (and subgroups of end users)

• Librarians (and subgroups of librarians)

How did we conduct the research?

Research methodologiesHow did we conduct the research?

Research methodologies

• Focus groups

• Conducted by Blue Bear, LLC

• Pop-up survey on WorldCat.org

• Conducted by ForeSee Results

• Librarian survey

• Conducted by Marketing Backup

End-user focus groupsEnd-user focus groups

• Focus groups:

• College students, ages 18–24

• General public, ages 25–59

• Scholars, including academic faculty and graduate students

• Format:

• Individual usability tests: captured comments on-screen

• Facilitator-led, group discussion

What did we learn?

End-user focus group resultsWhat did we learn?

End-user focus group results

Key observations:

• Delivery is as important, if not more important, than discovery.

• Seamless, easy flow from discovery through delivery is critical.

• Improved search relevance is necessary.

Pop-up surveyPop-up survey

• Live on WorldCat.org: May 12

• 11,000+ responses through July 10

• Evaluates the metadata most helpful in identifying a needed item

Who responded to the survey?Who responded to the survey?

Students: 19%

Teacher/professor: 15%

Other general searchers: 34%

Librarians/other library staff: 32%

End-user country:

USA: 56%

Canada: 4%

Mexico: 3%

United Kingdom: 3%

End-user language:

•English: 84%

•Spanish: 8%

•Other: 3%

•French: 2%End-user age:•18 & younger: 5%•19–30: 24%•31-40:17%•41-50:20%•51-60: 20%•61+: 13%

What did we learn?

Pop-up survey resultsWhat did we learn?

Pop-up survey results

Information most essential in identifying the item needed?End users (n=7535)

5%

7%

7%

12%

14%

24%

5%

0% 10% 20% 30% 40%

Summary/abstract

Citations

Links to onlinecontent/full text

Item details

Author

Ability to see what isimmediately available

List of libraries that ownit

Discovery

Discovery

Discovery

Discovery

Delivery

Delivery

Delivery

Delivery

Delivery

Delivery

Discovery

Discovery

Discovery

Discovery

Discovery

Discovery

Discovery

Discovery

Discovery

Discovery

Discovery

Discovery

Discovery

Discovery

What did we learn?

Pop-up survey suggestionsWhat did we learn?

Pop-up survey suggestions

Changes to help identify an item?

End users (n=7535)

‘Item details’ in WorldCat.org‘Item details’ in WorldCat.org

The World Is Flat

Lots of detail

Not Quite the DiplomatNot much detail

‘Subject information’ in WorldCat.org‘Subject information’ in WorldCat.org

6 subject-rich words:

Barack

Obama

Travel

Africa

Presidential

Candidates

End-user recommendationsEnd-user recommendations

• Improve search relevance

• Add more links to online full text (and make linking easy)

• Add more summaries/abstracts: Make summaries more prominent

• Add more details in the search results (e.g., cover art and summaries)

Librarian surveyLibrarian survey

• Currently in the field beginning September 2008 (U.S. and non U.S.)

• Preliminary data: 1,138 responses; North America (844) and 171 international as of 10/24/08

• Evaluates:

• The metadata most helpful in identifying a needed item

• Attributes liked most about WorldCat

• Recommended enhancements to WorldCat

Librarian surveyLibrarian survey

Acquisitions: 28%

Cataloging : 65%

Collection development or selection: 32%

Interlibrary loan: 25%

Reference/public service: 46%

Library director/administration: 20%

Current areas of responsibility

What did we learn?Librarian survey results: Reactions to WorldCat.org—compared to end users

What did we learn?Librarian survey results: Reactions to WorldCat.org—compared to end users

DISCOVERYMost essential information

What did we learn?Librarian survey results compared to end-user results

What did we learn?Librarian survey results compared to end-user results

DISCOVERYRecommended enhancements

What did we learn?End-user survey data compared to librarian survey data

What did we learn?End-user survey data compared to librarian survey data

DISCOVERYRecommended enhancements to WorldCat

What did we learn?

Librarian survey resultsWhat did we learn?

Librarian survey results

TOP recommended enhancements to WorldCat

Top 5 total librarian responses

What did we learn?

Librarian survey results

What did we learn?

Librarian survey results

Recommended enhancements to WorldCat Top 5 acquisition librarian responses

What did we learn?

Librarian survey results

What did we learn?

Librarian survey results

Recommended enhancements to WorldCat Top 5 cataloging librarian responses

What did we learn?

Librarian survey results

What did we learn?

Librarian survey results

Recommended enhancements to WorldCat Top 5 library director responses

What did we learn?

Librarian survey resultsWhat did we learn?

Librarian survey results

TOP enhancements for WorldCatTOP 5 responses by academic librarians

What did we learn?

Librarian survey resultsWhat did we learn?

Librarian survey results

TOP enhancements for WorldCatTOP 5 responses by public librarians

What did we learn?

Librarian survey resultsWhat did we learn?

Librarian survey resultsTOP enhancements for WorldCat

TOP 5 responses among international librarians

What did we learn?

Pop-up survey suggestionsWhat did we learn?

Pop-up survey suggestions

Changes to help identify an item?

End users (n=7535) – Bottom 8 mentions

2%

2%

2%

4%

5%

5%

6%

6%

0% 10% 20% 30% 40%

Add editorial reviews

Add recommendations

Add ratings

More publisher information

More format/type information

Increase accuracy

Remove duplicate records

More selection of non-English items

Information Not Sufficient

Recommendations from librarian survey (so far)Recommendations from librarian survey (so far)

• Merge duplicates

• Make it easier to make corrections to records (fix typos; do upgrades); “social cataloging” experiment—Wikipedia

• More emphasis on accuracy/currency of library holdings

• Enrichment—TOCs, summaries, cover art—work with content suppliers, use APIs, etc.

• Education about what users say they want

A few ideas to discussA few ideas to discuss

• Catalogs have many audiences, inside and outside the library

• With respect to metadata “quality,” librarians’ and end users’ definitions generally differ

• Different groups of end users have different priorities, but there are some commonalities across groups:

• The end user’s delivery experience is as important, if not more important than the discovery experience

• Most important for analog materials: summaries, TOCs, etc.

• Most important for licensed e- and digital materials: the ability to link easily and conveniently to the online content itself

Different groups of librarians have different priorities, but there are some commonalities across groups:

• Merge duplicate records

• Add TOCs