metasearching: the problem, promise, principles, possibilities & perils roy tennant california...

Metasearching: The Problem, Promise,

Principles, Possibilities & Perils

Roy TennantCalifornia Digital Library

NISO Metasearching Meeting, October 2003

Purpose

• To provide an overview, to set the stage

• To help you begin thinking both about the possibilities, as well as current issues and problems

• To hopefully leave you with the impression that despite the problems, it’s still very worth doing


The Problem

• Most users do not care where the information they need comes from, or who provides it…nor should they have to

• But our systems presently require them to know:– How to select one or more databases– How to get to them– How to use the unique search options for each

• How can we create systems that minimize what the user needs to know to get what they want?

RemoteCatalogs

LocalCatalog Local

WebSite

RemoteWebSites

VendorDbs

RemoteDigitalContent

LocalDigitalContent


The Promise

• The “Holy Grail” of resource discovery: one-stop shopping

• The simplification of a formerly complex activity (put the complexity in the back end, not the front)

• Allows the user to focus on evaluating results, not figuring out where to search


Before we talk specifics…

…let’s start with some principles…


Principles

Only librarians like to search, everyone else likes to find


Principles

All things being equal, one place to search is better than two or more.


Principles

“Good enough” is often just that.


Principles

Our ability to create effective one-stop searching is dependent on our ability to appropriately target user needs.


Principles

The size of the result set doesn’t matter as much as how the results are presented.


Principles

Services should be placed as close to the user as possible.


Possibilities: Software Solutions

• Special “metasearching” software optimized for:– Creating and managing target and resource

profiles– Managing authorization and authentication– Parsing search strings and result sets– Merging, deduping, and ranking results

• Stand-alone or integrated with an ILS• Many vendors (although a number are

based on the same underlying software)


Possibilities: Implementation Scenarios

• One search box for all resources (only for small libraries w/few targets)

• An interface that encourages/requires the user to select a subject or task group of DBs

• Tailored portals for particular needs and user communities or purposes (e.g., “a few good things”)

• Search boxes in other systems (e.g., courseware)

Slide from Greg Van Essen, Endeavor


Perils

• Software Provider Concerns• Database Provider Concerns• Library Concerns• User Concerns• Note: the following lists are not

comprehensive, but illustrative


Software Provider Concerns

• Access management• Search mapping• Unreliability of targets• Systems that don’t support an API (that

must be screen-scraped)• Inadequate result data for good:

– Deduping– Ranking


Database Provider Concerns

• Access control (robust authentication and authorization)

• Load• Inappropriate searches (searching

databases that don’t apply)• Branding and “unfair” deduping


Library Concerns

• Selecting the right system• Cost (both upfront and ongoing)• System design and implementation• System maintenance

– Ability to add new resources/targets– Ease of interface changes– Ease of upgrades


User Concerns

• What must I go through before hitting the search button?

• How difficult is it to review results?• Are results ranked by relevance? (that will be my

assumption)• Will I get buried? (too many sources, too many

results?)• Do I have methods to easily focus in on what I want?• Once I find what I want, can I get to the full-text with

a click?• Can I copy a citation and put it in my paper?


Possible Futures

• More standards for this type of interaction (i.e., support NISO’s efforts)

• Effective results ranking• Learning from other systems (e.g.,

RedLightGreen.com’s citation displays)• Dynamic selection of sources based on

user query


Recap

• “One-stop shopping” for resource discovery is the Holy Grail of librarianship

• Metasearch services offer the best opportunity we’ve yet seen to attain this goal

• It ain’t perfect, but it’s well worth pursuing• If we can present solid resources next to

Google hits — and win back our users in the process — what’s not to like?

metasearching: the problem, promise, principles, possibilities & perils roy tennant california...

Documents

niso metasearching meeting

courseware slide

illustrative slide

underlying software

deduping ranking slide

unfair deduping slide

principles good

user needs