against instinct: choosing discovery & making it work

Post on 06-Dec-2014

103 Views

Category:

Technology

2 Downloads

Preview:

Click to see full reader

DESCRIPTION

Three universities will discuss assessment of discovery systems. Texas State University-San Marcos believed a discovery tool wouldn’t offer value beyond existing cross-platform searching tools. After two years of testing, we changed our minds and learned many lessons. We will present strategies and recommendations for conducting usability testing and surveys, testing and customizing interfaces, and post-implementation management. Florida International University and the University of North Florida have both implemented or trialed discovery systems in the past year and have been working on strategies to assess these systems. We will share what we have done so far, and then would like to open the floor to see what assessments attendees have planned or performed.

TRANSCRIPT

AGAINST INSTINCT

Choosing Discovery & Making it Work

Liane Taylor, Continuing Resources LibrarianArlene Salazar, Reference/Instruction Librarian

Lisa Ancelet, Head, Reference & Instructional Services Albert B. Alkek Library

Texas State University – San MarcosMarch 18, 2013

Electronic Resources & Libraries 2013

ABOUT US:TEXAS STATE UNIVERSITY-SAN MARCOS

5th Largest University in Texas (…but you’ve never heard of

us.)

34,000 FTE High Undergraduate (~30k FTE) 18 Master’s Programs (~3600 FTE) 11 Doctoral Programs (~400 FTE)

Main Library + Music Library + Satellite Campus Library

http://www.txstate.edu/about/history-traditions.html

THE SEARCH FOR THE PERFECT SEARCH TOOL THAT WILL SOLVE ALL OF OUR PROBLEMS FOREVERJuly 2009 - August 2011

THE TEAM: TRANSFORMATION & COMPOSITION

Federated Discovery Search Search Team Team

The Perfect Mix (Balance is key)• Reference/Instruction +

Acquisitions/Cataloging• For “One-Search” +

Against “One-Search”

BFFs• Met weekly for two years

(some more than 2 yrs)

http://www.flickr.com/photos/1000photosofnewyorkcity/6018490089/

WHEREFORE EBSCO DISCOVERY SERVICE?

We considered comparing EBSCO Discovery Service vs Summon, however…

• EBSCO Link Resolver• EBSCOHost-centric (75% of databases)• Clear Gateway to Subject-Specific

Databases

• December 2009: EBSCO Discovery Service Beta Test Site

EDS VS EBSCO POWERSEARCH: WOULD STUDENTS EVEN NOTICE THE DIFFERENCE?

PowerSearch combines most of our databases Transferred most db to EBSCO, except for several where

native interfaces were clearly stronger (i.e., EngVillage)

…but PowerSearch does not include Catalog, IR, other vendors

PowerSearch is FREE, EDS is NOT.

What does EDS give us that we don’t already have?

NOTE: Image not to scale

WHAT DOES SUCCESS LOOK LIKE?

(ORIGINAL GOALS VS VALUE TEST)

Provide enhancements to the library’s catalog such as facets and tags (discovery tools)

Provide the option for patrons to search multiple resources simultaneously (federated search)

Improve patron awareness of the many and varied electronic resources to which the library provides access

Provide customized search options for various disciplines or academic departments

Original Goals EDS Value Test

?EDS is valuable if a searcher can consistently type an unsophisticated, non-boolean search query, and as a default, with no sophisticated strategy, retrieve non-EBSCO results, including books from the Catalog (if not an article-only search), within the top 50 results.

THE LONG AND WINDING ROAD

Customizing, Testing, Tweaking Search Mode Parameters Facets Relevancy Ranking Tweaks

Waiting, waiting (Dec 2009-April 2011)

Listservs + Webinar

THE WINNER: OPTION 1 (EDS) 137 Respondents

37 faculty members (27%) 26 preferred Option 1 (EDS)

67 students (undergrad + grad) (48.9%) 46 preferred EDS

PROBLEM: Most were relying on number of hits or differences in the interface of one over the other.

 Why Option 1? After taking out interface differences that could be fixed,

results that include books from the catalog and journals articles.

Given the feedback comments, it was clear that either of the options were better than the existing search option.

FOCUS GROUPS

13 Participants (mix of undergrad, grad, fac)

Led group through 5 searches total, comparing: EDS to PowerSearch EDS to Catalog

Post-Search Feedback Questions, examples: Hard was this to do?

If “Hard” or “Very Hard” – Why? Were the results same -> very different? Which do you prefer?

AS CLEAR AS…For most of the questions, all interfaces were rated “About the Same” in terms of preference & performance

BUT!“Overall, which option did you prefer when searching for articles?”5=The Same 5=EDS 2=Neither 1=PowerSearch* (*likely mistake!)

READING (PROJECTING?) THE FINE PRINT

“I was impressed with option 1 and its one stop for everything”;

“Option 1 is more connected with the library catalog which helps especially if you want one place to find everything.”

“I like being able to immediately see the cover of whatever book im looking at. Both had equally relevant articles, but I prefer books to PDF's”;

“When looking for books, option 1 was better because it had the book plus reviews on the book when it came up”.

WHERE DO WE PUT IT?

Development of new homepage search widget If your site has a tabbed search widget, we

looked at it.

WHAT DO WE CALL IT?

“FINAL” SEARCH BOX

IT AIN’T OVER ‘TIL…WELL, EVER.

Implementation & Promotion

Advisory Group

Continued Customization

IMPLEMENTATION | INSTRUCTION

http://libguides.txstate.edu/startyourresearchatalkek

ADVISORY GROUP

Members for an advisory group were brought together to continue to shape EDS.

AGAINST INSTINCT, MAKING IT WORK

Our guts tell us two things: IT MUST BE PERFECT. IT MUST BE COMPLETE.

Your discovery tool is neither.

Manage Expectations! It’s a STARTING POINT, not an END-ALL, BE-ALL,

SEARCH TOOL FOREVERYONE

It can’t have everything in it! Many databases aren’t well-suited to be

included!

top related