1 designing information architecture for search marti hearst university of california, berkeley...

1

Designing Information Architecturefor Search

Marti Hearst

University of California, Berkeleywww.sims.berkeley.edu/~hearst

NSF CAREER Grant, NSF9984741

Tutorial: SIGIR 2001

2

Outline

1. Motivation2. Search Interfaces:

1. Web search vs Site Search2. Search UIs: What works; what doesn’t

3. Methodology1. Information Architecture Defined2. Faceted Metadata3. Integrating Search into IA via Faceted Metadata

4. Results of Usability Studies5. Tools6. Conclusions

3

Contributors to the Research

• Dr. Rashmi Sinha• Graduate Students

– Ame Elliott– Jennifer English– Kirsten Swearington– Ping Yee

• Research funded by – NSF CAREER Grant, NSF9984741

4

Motivation and Background

5

Claims

• Web Search is OK– Gets people to the right starting

points

• Web SITE search is NOT ok• The best way to improve site

search is– NOT to make new fancy algorithms– Instead …

6

The best way to improve search:

Improve the User Interface

7

Recent Study by Vividence Research

• Spring 2001, 69 web sites– 70% eCommerce– 31% Service– 21% Content– 2% Community

• The most common problems:53% had poorly organized search results32% had poor information architecture32% had slow performance27% had cluttered home pages25% had confusing labels15% invasive registration13% inconsistent navigation

8

Vividence findings: effects on users

• Poorly organized search results– Frustration and wasted time

• Poor information architecture– Confusion– Dead ends– "back and forthing"– Forced to search

9

Vividence findings: effects on users

• Cluttered home pages– Creates disinterest– Wastes time– No contrast: everything has equal weight– Don’t know where to start– Failure to engage– No call to action– Failure to establish navigation– Layout reflects company organization chart– Investor centeredness

10

Vividence findings: characteristics

• Inconsistent Navigation– Primary navigation bar is, in fact, really

secondary– Un-scalable designs– Poor transitions between company divisions– "Junk Drawer" navigation bars– Random links– Shoe-horned functions– Heavy need to hit the "back-button"

11

Vividence Study

• Breakdown of most common search problems– 41% - of searches encountered no problems– 20% - had search problems not named below– 14% - of searches were not “advanced” enough– 12% - did not organize results well– 10% - of searches yielded inaccurate/unrelated results– 9% - were too slow– 8% - of searches had insufficient instructions– 7% - engine was too difficult to locate– 7% - of searches produced too few results– 7% - of searches were too limiting– 3% - of searches produced an error message– 3% - were too difficult to use

12

Other Relevant Studies

• Commercial studies (are not usually scientific, do not supply full details)– CreativeGood.com Holiday 2000 ecommerce

report– UIE, and Jared Spool’s talks:

http://world.std.com/~uieweb• Scientific studies (often less relevant to real web situations)

– Many papers from the CHI proceedings http://www.acm.org/dl/

– Papers from Human Factors and the Web http://www.optavia.com/hfweb/

– See the extensive bibliography from my textbook chapter (in this package).

13

The Philosophy

• Information architecture should be designed to integrate search throughout

• Search results should reflect the information architecture.

• This supports an interplay between navigation and search

• This supports the most common human search strategies.

14

The Approach

• Assign faceted metadata to content items

• Allow users to navigate through the faceted metadata in a flexible manner

• Organize search results according to the faceted metadata so navigation looks similar throughout

• Give previews of next choices• Allow access to previous choices

15

Advantages of the Approach

• Supports different task types– Highly constrained known-item

searches use one interface– Open-ended, browsing tasks use

another interface– Both types of interface use the same

underlying structure– Can easily switch from one interface

type to the other midstream

16


• Honors many of the most important usability design goals– User control– Provides context for results– Reduces short term memory load– Allows easy reversal of actions– Provides consistent view

17


• Allows different people to add content without breaking things

• Can make use of standard technology

18

Web Search vs. Site Search

19

Web Search is Working!

Survey finds high user satisfactionStudy by npd group

http://www.searchenginewatch.com/reports/npd.html

20

Why is Web Search Working?

• Web Search is Successful at Finding Good Starting Points (home pages)

• Evidence: – Search engines using

• Link analysis• Page popularity• Interwoven categories

– These all find dominant home pages

23

Organizing Search Results:What works, What Doesn’t

• There is a lot of prior work on this– Cha-Cha (Chen et al. 1999)

– Scatter-Gather clustering (Cutting et al. 93, Hearst et al. 1996)

• Becoming more prevalent in web search too.– Teoma– Vivisimo– Northern Light

24

Putting Results into Clusters

25

Drilldown – what does it mean?

26

Vivisimo – same idea

28

Yahoo lists category matches

29

Web Search Results Grouping

• Drill down one category • Cannot mix and match categories• Not clear if it is useful or not

– Can help differentiate different meanings of the same word.

• But …what about site search?

30

If Web search engines are providing source selection …

… what happens when the user gets to the site?

Follow Links … or …

Search

31

Following Hyperlinks

• Works great when it is clear where to go next

• Frustrating when the desired directions are undetectable or unavailable

Site SearchIs not getting good reviews

32

An Analogy

text searchhypertext

33

Analogy

• Hypertext: – A fixed number of choices of where to go next; – A glance at the map tells you where you are;– But may not go where you want to go.

• To get from Topeka to Santa Fe, may have to go through Frostbite Falls

• Site Search:– Can go anywhere;– But may get stuck, disoriented, in a crevasse!

34

Goal: An All-Tertrain Vehicle

• The best of both techniques– A vehicle that magically lays down

track to suggest choices of where you want to go next based on what you’ve done so far and what you are trying to do

– The tracks follow the lay of the land and go everywhere, but cross over the crevasses

– The tracks allow you to back up easily

35

Organizing Search ResultsWhat works; what doesn’t

36

What works, what doesn’t

• There is negative evidence for– Clustering– Fancy visualizations

• There is positive evidence for– Grouping into meaningful, consistent

categories– Relevance feedback

• Depends how you do it– Showing similar items

37

Koh

on

en F

eatu

re M

aps

on

Text

(fro

m C

hen e

t al.,

JAS

IS 4

9(7

))

UWMS Data Mining Workshop

Study of Kohonen Feature Maps

• H. Chen, A. Houston, R. Sewell, and B. Schatz, JASIS 49(7)

• Comparison: Kohonen Map and Yahoo• Task:

– “Window shop” for interesting home page– Repeat with other interface

• Results:– Starting with map could repeat in Yahoo

(8/11)– Starting with Yahoo unable to repeat in map

(2/14)


Study (cont.)

• Participants liked:– Correspondence of region size to #

documents– Overview (but also wanted zoom)– Ease of jumping from one topic to

another – Multiple routes to topics– Use of category and subcategory

labels


Study (cont.)

• Participants wanted:– hierarchical organization– other ordering of concepts (alphabetical)– integration of browsing and search– corresponce of color to meaning – more meaningful labels– labels at same level of abstraction– fit more labels in the given space– combined keyword and category search– multiple category assignment (sports+entertain)

41

Visualization of Clusters

– Huge 2D maps may be inappropriate focus for information retrieval • Can’t see what documents are about• Documents forced into one position in

semantic space• Space is difficult to use for IR purposes• Hard to view titles

– Perhaps more suited for pattern discovery• problem: often only one view on the

space

42

Summary: Clustering(Based on other studies as well)

• Advantages:– Get an overview of main themes– Domain independent

• Disadvantages:– Many of the ways documents could group

together are not shown– Not always easy to understand what they mean– Different levels of granularity

• Probably best for scientists only• Take heart – there is good evidence for

organizing via categories!

43

The DynaCat System

• Decide on important question types in an advance– What are the adverse effects of drug D?– What is the prognosis for treatment T?

• Make use of MeSH categories• Retain only those types of categories

known to be useful for this type of query.

Pratt, W., Hearst, M, and Fagan, L. A Knowledge-Based Approach to Organizing Retrieved Documents. AAAI-99: Proceedings of the Sixteenth National Conference on Artificial Intelligence, Orlando, Florida, 1999.

44

DynaCat

45

DynaCat Study

• Design– Three queries– 24 cancer patients– Compared three interfaces

• ranked list, clusters, categories

• Results– Participants strongly preferred categories– Participants found more answers using

categories– Participants took same amount of time with

all three interfaces

46

Cha-C

ha (

intr

anet

searc

h)

Cha-Cha: A System for Organizing Intranet Search Results, by Chen, Hearst, Hong, and Lin, Proceedings of 2nd USENIX Symposium on Internet Systems, Boulder, CO, Oct 1999. cha-cha.berkeley.edu

47

Cha-C

ha (

intr

anet

searc

h)

48

How People Search

49

The Standard Model

• Assumptions:– Maximizing precision and recall

simultaneously– The information need remains static– The value is in the resulting

document set

50

“Berry-Picking” as an Information Seeking Strategy (Bates 90)

• Berry-picking model– Interesting information is scattered

like berries among bushes– The user learns as they progress, thus – The query is continually shifting

51

A sketch of a searcher… “moving through many actions towards a general goal of satisfactory completion of research related to an information need.” (after Bates 89)

Q0

Q1

Q2

Q3

Q4

Q5

52

Search Tactics and Strategies

– Marcia J. Bates, Information Search Tactics, Journal of the AmericanSociety for Information Science, 30, 4, 1979

– Marcia J. Bates, Where should the person stop and the informationsearch interfaces start?, Information Processing \& Management, 26,

5,1990

Marcia J. Bates, The Berry-Picking Search: User Interface Design, UserInterface Design, Harold Thimbleby, Addison-Wesley, 1990

– Marcia J. Bates, The design of browsing and berrypicking techniquesfor the on-line search interface, Online Review, 1989, 13, 5,407—431.

– Vicki L. O'Day and Robin Jeffries, Orienteering in an informationlandscape: how information seekers get from here to there,

Proceedings of ACM INTERCHI '93, April, Amsterdam, 1993

53

Tactics vs. Strategies

• Tactic: short term goals and maneuvers– operators, actions

• Strategy: overall planning– link a sequence of operators together

to achieve some end

54

An Important Strategy

• Do a simple, general search– Gets results in the generally correct area

• Look around in the local space of those results

• If that space looks wrong, start over– Akin to Shneiderman’s overview + details

• Our approach supports this strategy– Integrate navigation with search

55

Term Tactics

• Move around a thesaurus– Look at category labels– Look at related terms– Look at parent terms– Look at child terms

• In older literature, refers to navigating the thesaurus itself, as opposed to the items themselves.

56

Source-level Tactics

• “Bibble”:– look for a pre-defined result set – e.g., a good link page on web

• Survey:– look ahead, review available options– e.g., don’t simply use the first term or first

source that comes to mind

• Cut:– eliminate large proportion of search domain– e.g., search on rarest term first

57

Source-level Tactics (cont.)

• Stretch– use source in unintended way– e.g., use patents to find addresses

• Scaffold– take an indirect route to goal– e.g., when looking for references to

obscure poet, look up contemporaries

58

Monitoring Strategies

• Check– compare original goal with current state

• Weigh– make a cost/benefit analysis of current or

anticipated actions

• Pattern– recognize common strategies

• Correct Errors• Record

– keep track of (incomplete) paths

59

Additional Considerations(Bates 79)

• Need a Sort tactic• When to stop?

– How to judge when enough information has been gathered?

– How to decide when to give up an unsuccesful search?

– When to stop searching in one source and move to another?

60

Information Architecture

61

A Taxonomy of WebSites

low

low

high

high

Complexity of Applications

Complexity of Data

From: The (Short) Araneus Guide to Website development, by Mecca, et al, Proceedings of WebDB’99, http://www-rocq.inria.fr/~cluet/WEBDB/procwebdb99.html

Catalog Sites

Web-based Information

Systems

Web-Presence

Sites

Service-Oriented

Sites

62

A View of Website Design

• Information design– structure, categories

of information

• Navigation design– interaction with

information structure

• Graphic design– visual presentation of

information and navigation (color, typography, etc.)Information Architecture

From Sitemaps, Storyboards, and Specifications: A Sketch of Web Site Design Practice as Manifested Through Artifacts. M.W. Newman and J.A. Landay. In proceedings of Designing Interactive Systems: DIS '00. August 17-19, 2000.

63

A View of Information Architecture

• Content Items +• Information Structure +• Navigation Structure +• Layout

64

Content Items

– The information items that the site is designed to show the user.

– Individual content items can be considered leaves in a tree, or base-level items.

– Aggregates of individual (base-level) items can be considered to be content items.

– This definition is especially relevant for catalog-style sites, for example:

• Image collection• Product selling• Collection of articles on some topic (medical, legal)• Collection of information about some entity (IRS, Park

Service)

65

Information Structure

• Independent of the website.• A set of descriptors which are used to

characterize the content of a website.• Consists primarly of a category structure

and a set of textual labels.• The categories can have flat, hierarchical,

faceted or network structure.• The textual labels include alternative

ways of expressing the same concepts (synonyms).

66

Navigation Structure

• Defined in terms of the website.• Site level:

– The paths connecting content items throughout the site.

• Page level: – The link from one page to others.

67

Exam

ple

fro

m W

alm

art

.com

68

Content

Navigation Structure

69

Related Items

• Often are content items• Related to the target by some

shared information structure• The particular related items that

are shown are revealed through the navigation structure

70

The Information Structure

• Consists of a set of descriptors for the content items

• Can’t really see it directly, since it is independent of web site description

• Can see parts of it in the navigation structure

71


Content Items


Start with an information structure (categoriesand labels) and a set of content items.

72


Content Items


Each content item is assigned some descriptors from the information structure.

Prod: CampingBrand: REIMaterial: NylonSize: 4-person

73

Navigation structure links items or groups of items.

74

Navigation Structure Differs from Information Structure

• Example:– Part of the info structure is the

product hierarchy.– Some products are assigned more

than one spot in the hierarchy (e.g., sports and games), thus forming a tree structure

– Navigation structure shows a progressive disclosure of the hierarchical structure only.

75


• Example:– Main navigation structure is the

product hierarchy.– However, “lateral” links are shown

from product leaf nodes to other nodes• (e.g., from a tent to a flashlight and a

sleeping bag)

76


• The differences can be much more profound

• Examples:– Show only main product categories at

top levels– After a search, show links according

to brands of items, but only those brands that make sense for the items retrieved by the search.

77

“Breadcrumbs”

• A navigation technique for showing either history or contextualizing hierarchy via hyperlinks.

• Two main types:– Hierarchy without history:

• Search results at walmart.com

– History across facets (without hierarchy):• Epicurious path recording.

78

An Important IA Trend• Generating web pages from databases• Implications:

– Web sites can adapt to user actions– Web sites can be instrumented – “An essential feature of a design

environment is to give authors the possibility of evaluating the current network against the final adaptive system.”

» Petrelli, Baggio, & Pezzulo, Adaptive Hypertext Design Environments: Putting Principles into Practice, AH 2000

79

Faceted Metadata

80

Metadata: data about dataFacets: orthogonal categories

Time/Date Topic RoleGeoRegion

81

Faceted Metadata: Biomedical MeSH (Medical Subject Headings)www.nlm.nih.org/mesh

82

Mesh Facets (one level expanded)

83

Using Mesh Facets

• Some stats:– >18,000 labels– avg depth: 4.5, max depth 9– ~8 labels/article on average

• How to go from the information structure to the navigation structure?

84

Using faceted metadata incorrectly

• Yahoo uses faceted metadata poorly in both their search results and in their top-level directory

• They combine region + other hierarchical facets in awkward ways

85

Yahoo’s use of facets

86


87


88


Where is Berkeley? College and University > Colleges and Universities >United States > U > University of California > Campuses > Berkeley

U.S. States > California > Cities >Berkeley > Education > College and University > Public > UC Berkeley

89

However, Yahoo does use some metadata well• Yahoo restaurant guide combines:

– Region – Topic (restaurants) – Related Information

• Other attributes (cuisines)• Other topics related in place and time

(movies)

90

Green: restaurants & attributes

Red: related in place & time

Yellow: geographic region

91

Combining Information Types

• Region– State

• City

• A & E– Film– Theatre– Music– Restaurants

• California• Eclectic• Indian• French

Assumed task: looking for evening entertainment

92

Other Possible Combinations

• Region + A&E• City + Restaurant + Movies• City + Weather• City + Education: Schools• Restaurants + Schools• …

93

Bookstore preview combinations

• topic + related topics• topic + publications by same author• topic + books of same type but related

topic

94

Problems with Metadata Usage

• Standard approaches– Paths are hand-edited, predefined– Not well-integrated with search– Not tailored to task as it develops– Not personalized– Not dynamic

95

Questions we are trying to answer

• How many facets are allowable?• Should facets be mixed and

matched?• How much is too much?• Should hierarchies be progressively

revealed, tabbed, some combination?

• How should free-text search be integrated?

96

Recipe Collection Examples

97

From soar.berkeley.edu (a poor example)

99

From www.epicurious.com (a good example)

103

Epicurious Metadata Usage

• Advantages– Creates combinations of metadata on the fly– Different metadata choices show the same

information in different ways– Previews show how many recipes will result– Easy to back up– Supports several task types

• ``Help me find a summer pasta,'' (ingredient type with event type), • ``How can I use an avocado in a salad?'' (ingredient type with dish

type), • ``How can I bake sea-bass'' (preparation type and ingredient type)

104

Metadata usage in Epicurious

PrepareCuisineIngredient Dish

Recipe

105



PrepareCuisineDishISelect

106



I >

Group by

PrepareCuisineDish

107



PrepareCuisineDishI >

Group by

108



PrepareCuisineDishI >

Group by

PrepareCuisineISelect

109

Recipe Information Architecture

• Information design– Recipes have five types of metadata

categories• Cuisine, Preparation, Ingredients, Dish,

Occasion• Each category has one level of

subcategories

110

Recipe Information Architecture

• Navigation design– Home page:

• show top level of all categories

– Other pages:• A link on an attribute ANDS that attribute

to the current query; results are shown according to a category that is not yet part of the query

• A change-view link does not change the query, but does change which category’s metadata organizes the results

111

Metadata Usage in Epicurious

• Can choose category types in any order• But categories never more than one

level deep• And can never use more than one

instance of a category– Even though items may be assigned more

than one of each category type

• Items (recipes) are dead-ends– Don’t link to “more like this”

• Not fully integrated with search

112

Epicurious Basic Search

Lacks integration with metadata

114

Information previews

• Use the metadata to show where to go next– More flexible than canned hyperlinks– Less complex than full search

• Help users see and return to what happened previously

• Reduces mental work– Recognition over recall– Suggest alternatives

115

The Importance of Information Previews

• Jared Spool’s studies (www.uie.com)

• More clicks are ok if– The “scent” of the target does not

weaken– If users feel they are going towards,

rather than away, from their target.

116

Problem with Metadata Previews as Currently Used

– Hand edited, predefined– Not tailored to task as it develops– Not personalized– Often not systematically integrated

with search, or within the information architecture in general

117

Putting it Together

118

Desiderata for Objects inInformation-Seeking Workspaces

• Structured• Fractal• Queriable• Navigable• Historical• Similarity Engine Compatible• Contextualized• Other

From Furnas, G., and Rauch, S., Considerations for information environments and the NaviQue workspace. In Proceedings of DL 98. Pittsburgh,PA, June, 1998.

Search Usability Design Goals

1. Strive for Consistency2. Provide Shortcuts3. Offer Informative Feedback4. Design for Closure5. Provide Simple Error Handling6. Permit Easy Reversal of Actions7. Support User Control8. Reduce Short-term Memory Load

From Shneiderman, Byrd, & Croft, Clarifying Search, DLIB Magazine, Jan 1997. www.dlib.org

120

Analogy: Chess

• Chess is characterized by a few simple rules that disguise an infinitely complex game

• Another intriguing characteristic: the three-part structure– Openings: many strategies, new ones all the

time, many books on this– Endgame: well-defined, well-understood– Middlegame: nebulous, hard to describe

• Our thought: search is similar and the middlegame is critically underserved.

121

Chess-based view of Info Architecture

• The Opening:– Usually exposes top-level hierarchy or

top-level facets (or both)– Usually also has a search component– This is also the place to expose the

main tasks that can be accomplished on the site

122

The Opening

123

The Opening

124

The Opening

125

The Opening

126

The Opening

127


• The Endgame:– Has become rather well-established in

shopping sites– Penultimate page: shows a list of

items– Leaf node:

• Shows one content item in detail• Lateral links

– To similar items (same facet)– To other items that go with it (other facets)

128

The Endgame – Penultimate Pages

129

The Endgame – Penultimate Pages

130

The Endgame – Leaf Nodes

131


• The Middlegame:– Hardest to describe/understand– The “berry-picking” part of supporting

search– Issues:

• How to progressively expose hierarchies?• How to show multiple facet choices?• How to integrate with search results?• How to show history / retain context?

132

Sophisticated Middlegames

zdnet.comContinued next slide

133


134


Walmart.comContinued next slide

135


136


137

Online Grocery Shopping Examples

• In each case, note– Chess analogy

• What is the opening?• What is the endgame?• How is the middlegame handled?

– How are search results integrated?– How is hierarchical drill-down

revealed?– Are multiple facets allowed?

138

Grocery shopping example

Homerun.com

139


Homerun.com

140


Homerun.com

141


peapod.com

142


peapod.com

143


peapod.com

144


webvan.com

145


webvan.com

146

Summary: Grocery Shopping Examples

• A good opening seems to make a big difference

• Familiar metadata helps make the task easier

• Middlegame hierarchy exposure– One uses cascading menus– Two use webpage-based drilldown

• Two use metadata to organize search results– But don’t use metadata creatively– Could organize by recipe, etc.

147

Medical Text Example

– Allow user to select metadata in any order

– At each step, show different types of relevant metadata, • based on prior steps and personal

history, • include # of documents

– Previews restricted to only those metadata types that might be helpful

148

Ecommerce Examples

• E-commerce sites are farther ahead than information collection sites

• However, their problem is usually easier– Single facet often works fine– Categories are familiar to users– Collections are often much smaller

• How to move this to large sites containing more abstract information?– Image collections?– Text collections

149

Current Search Approach

Can a metadata preview approach do better?

150

Asthma > Steroids

1. A steroid-induced acute psychosis in a child with athsma.2. Management of steroid-dependent asthma with methotrexate.

1. A steroid-induced acute psychosis in a child with athsma.2. Management of steroid-dependent asthma with methotrexate.

Steroids•Pregnanes• Pregnadienes (5)• Prednisone (5)• Pregnenes• Budesonide (4)• Corticosterone (3)

Other Views• Admin & Dosage (50)• Drug Effects (20• Therapeutic Use (25)• Risk Factors (4)• More …

User Preferred• Musculoskeletal (4)•Drug Resistance (6)

•All Categories (99)

99 Documents: [Sort by author] [Sort by popularity] [Sort by Steroids] [Cluster]

1. Effect of short-course budesonide on the bone turnover of asthmatic children.2. Effect of prednisone on response to influenza virus vaccine in asthmatic children.…

1. Effect of short-course budesonide on the bone turnover of asthmatic children.2. Effect of prednisone on response to influenza virus vaccine in asthmatic children.…

151

Asthma > Steroids > Admin & Dosage

1. Dosage levels for asthmatic steroids: A survey.1. Dosage levels for asthmatic steroids: A survey.

Steroids•Pregnanes• Pregnadienes (3)• Prednisone (5)

Related Categories•Inhalators (40)•Emotional Effects (25)•Preferred Suppliers (30)

User Preferred• Musculoskeletal (0)•Drug Resistance (2)

•All Categories (50)

50 Documents: [Sort by author] [Sort by popularity] [Sort by Dosage] [Cluster]

1. Optimal dosage levels for prednisone in the treatment of childhood asthma.2. …

1. Optimal dosage levels for prednisone in the treatment of childhood asthma.2. …

152

Other paths: back up and go forward

Asthma > Steroids > Budesonide > Huang

Asthma > Huang > Budesonide

Asthma > Steroids

Asthma > Steroids > Budesonide

153

Advantages of the Methodology

• Supports different types of information seeking tasks

• Uses interface idioms known to be usable for general users

• Flexible content entry and update– Allows for non-experts to add new content

independently– Makes use of standard DBMS technology

154

Advantages of the Methodology• Systematically integrates search:

– search results reflect the structure of the info architecture

– search results retain the context of previous interactions

– search results preview next choices

• Gives user control– Over order of metadata use– Over when to navigate vs. when to search

• Allows integration with advanced methods– Collaborative filtering, predicting users

preferences

155

Advantages

• Users have a feeling of control• Users can predict what will happen

– Not true of statistical ranking or clustering

• Adding new items to the system changes the behavior in understandable ways

• Users have flexibility – In ordering of operations– In combining of operations

156

Usability Study: epicurious

157

Epicurious Usability Study

• 9 participants so far– Independent Variables:

• 1) Epicurious Interface (Basic vs. Enhanced vs. Browse) • 2) Task type (known-item search vs. browsing for inspiration) • 3) Degree of constraint of query • 4) Number of results required (1 vs. many)

– Dependent Variables: • 1) Time to find satisfactory recipe(s)• 2) Navigation path (backtracking, starting over, revising

queries)• 3) Satisfaction with results of search • 4) Satisfaction with individual system features (e.g.

breadcrumbs, query previews, refine by hyperlinks)• 5) Likelihood of using each interface in the future.

158

Epicurious Usability Study

• Participants were asked to:– Do 3 pre-specified searches in advance– In the lab:

• Specify a cooking scenario of interest to them

– Search for 3 recipes for this recipe– Search for each recipe using each of the interfaces

• Complete several structured tasks

– Along the way, answer questions about• Getting closer or farther away from goal• Satisfaction with search results• Satisfaction with the interace

159

Usability Study: Preliminary Results, Preference Data

Site Basic Enhanced BrowseTotal "Very Likely" to Use: 7 2 4 7

Total "Likely" to Use: 0 1 1 0Total "Not Likely" to Use: 2 6 4 2

PERCENTAGES Basic Enhanced BrowseVery Satisfied 32 43 35

Satisfied 50 43 52Middle 9 4 4

Dissatisfied 9 9 9

160

Usability Study: Preliminary Results, Preference Data

FavoriteSubject_JG: EnhancedSubject_NS: EnhancedSubject_SP: Browse

Subject_RM: Browse

Subject_LA: Enhanced

Subject_MC: BrowseSubject_MW: BrowseSubject_NM: EnhancedSubject_CG: Browse

Query previews and navigation. Options to refine by course or season. Choose how you view the results

Searching within made all the difference. I could see how many results I was getting in each Very specific. I can choose more than 1 detail with search for recipe I'm looking for.Likes the way it narrows things down. And it gives you the numbers.

Found it simpler, more readable. Helped you hone in on the season.Liked the kid friendly, low fat optionWhy?

Can narrow down when you're stuck. You can always refine [your search].

Allowed me to make specific selections. I liked Browse too. Gave lots to choose from. Depends on what you’re looking for that day

Can limit and unlimit and limit again in a different way. Prioritize your criteria--change the first thing I clicked and go in a different direction. Easy to back up.

161

Usability StudyPreliminary Results: Feature Preference

Subject_SP Subject_RM Subject_LA Subject_MC Subject_MW Subject_NM Subject_CGQuery previews

Having a complete list of ingredients (enhanced and browse)

enhanced - 0, browse - +1

Search within resultsRefine using hyperlinks (browse)Set all criteria from one screen (enhanced)"May include" and "must include" options"All words," "any words," "exact phrase," and "Boolean" options

KEY+1 - Helpful0 - Not helpful-1 - InterferedDN - Didn't notice

162

Usability StudyPreliminary Results: Quantitative

Search Metrics Basic Enhanced Browse

No Facets / Keywords per search

Total per search 4.3 7.7 ~2.11

During Step 1 of Search 2.3 always 1

For Each Step of Search 2.2 3.5 ~1

Ave. times 0 Results

During Step 1 0.0% 15.8% 0.0%

Overall 12.2% 31.4% 0.0%

Ave. Time ( in seconds)

Time per Search 98.6 130.3 108.11

Time per Step 50.9 60.6 41.38

Ave. No Steps 1.86 2.1 2.81

Median No Results

Overall (excluding 0 results) 186 17 704

in Step 1 13 10 579

In Final Step 32 14 13

163

Usability StudyPreliminary Results: Constraint-based Preferences

# of Results High LowEnhanced (LA) Browse (LA)Enhanced (MC) Browse (MC)Browse (MW) Browse (MW)Enhanced (NM) Enhanced (NM)Basic (CG) Browse (CG)Enhanced (LA) Browse (LA)Enhanced (MC) Browse (MC)Enhanced (MW) Browse (MW)Enhanced (NM) Enhanced (NM)Enhanced (CG) Browse (CG)

Constraint

1 result needed

Many results needed

164

Observed patterns of use of epicurious metadata browse interface

• choosefacet• refine• refine• back• scan focus

• choosefacet• refine• back refine• scan focus

• choosefacet• refine• refine• scan focus

• choosefacet• refine• refine• back refine• scan focus

• choosefacet• refine• refine• searchword

• choosefacet• searchword• scan searchword• back refine• scan focus

• choosefacet• refine• back back refine• refine• refine

• choosefacet• refine• refine• searchword• scan

• choosefacet• refine• refine• back refine

• choosefacet• scan focus

165

Usability Study Results: Summary

• People liked the browsing-style metadata-based search and found it helpful

• People sometimes preferred the metadata search when the task was more constrained – But zero results are frustrating– This can be alleviated with query previews

• People dis-prefer the standard simple search

• More study needed!

166

Application to Image Search

167

Image Search: What is the task?

• Illustrate my slides? – “Find a crevasse”– Keyword match works

pretty well

• Find inspiration for an architectural design? – Needs richer search

support

168

Faceted Metadata for Image Collection

Planalto Palace Parti Communiste Francais Pantheon

Oscar Neimeyer Oscar Neimeyer Jacques-Gabriel Soufflot

20th Century 20th Century 17th & 18th C.

Brasilia Paris Paris

Stone Curvilinear Stone

Image:

Architect:

Period:

Location:

Concept:

169

Planalto Palace Parti Communiste Francais Pantheon

Oscar Neimeyer Oscar Neimeyer Jaques-Gabriel Soufflot

20th Century 20th Century 17th & 18th C.

Brasilia Paris Paris

Stone Curvilinear Stone

Image:

Architect:

Period:

Location:

Concept:

Faceted Metadata for Image Collection

170

SPIRO Query Form (Original)

171SPIR

O q

uery

on

Su

bje

ct:

chu

rch

172

Pilot Study

• Architecture task:– Emphasize images over text– Use hypertext-style interface as a

reasonable baseline for comparison– Find out how much choice is too

much– Find out whether explicit metadata is

better than implicit more-like-this

173

Evaluation Methodology

• Solicit feedback from architects to determine if faceted metadata is helpful and how to present it

• Informal evaluation of paper prototype• Informal study of a crude live version

– 1 hour one-on-one with 9 architects /grad students, 2 tasks (audio recorded) and a survey

174

Results of a pilot study with Archictects: Metadata is Helpful

• Very positive feedback about the general approach– All 9 participants named the

metadata in the search results area as their favorite aspect of Flamenco

• Metadata was successful at giving hints about where to go next– Perceived as useful “These are places

I can go from here.”

175

Results: More Metadata Please• Participants asked for more metadata

– Although there were complaints about the contents of the metadata, users still wanted more

• Longer lists of options (more hints)• Users wanted more control to make very specific searches

• Half the participants requested the ability to control order of results with metadata– Juxtapose visible images 2 different ways:

• Overview (one image from each project) vs. like together ( all images of a project next to each other)

– Different than ranking for text retrieval (precision, recall), but ordering does matter

176

Results: Complaints

• The UI was not successful at clarifying searching within results vs. starting a new search– Only 2 of the 9 participants understood the

distinction without discussion – but they want to do both

• The 1/3 of the participants who couldn’t find a treasure hunt image felt that Flamenco was slow– Corroborates findings that perceived system

speed is about finding what you want (Spool ‘00)

177

New Developments

• A new, sophisticated implementation• Richer, hierarchical, cleaned up

metadata• Usability Study contrasting four

versions:– Single search form– Multiple facet search form– Yahoo-style directory-based – Faceted interface with query previews

• Results TBA

181

Tools

• Our system (all open-source)– Mysql (has a text search component)– Python 2.2– Python-mysql– Webware (python application server)

• Earlier attempt– Cold fusion – not flexible enough, not

enough of a programming language

182

A new term: “Parametric Search”

• From an XML glossary– "A search request submitted to a

search or database engine delivered with consideration for the metadata of the underlying dataset.”

– www.sla.org/chapter/ctor/courier/v37/v37n1.pdf

183

Commercial Tools

• This list is NOT comprehensive• These are NOT recommendations• General Search

– Inktomi Search/site (formerly Infoseek ultra)

• Specializing in Online Catalogs– Dieselpoint– Requisite– Saqqara

• Question-answering– Askjeeves– Primus (formerly Answerlogic)

184

“Parametric” Search• A survey of sites using parametric search:

– http://www.amp.com/search/default.asp (see product family search)– http://ebiz.zilog.com/– http://www.sears.com (Dieselpoint)– http://dieselpoint.com/flashlink.htm (for Dieselpoint 2.0 demo)– http://www.findmro.com (Requisite's BugsEye)– http://www.cypress.com (Saqqara's one step)– http://infineon-tech.sacosnet.de/search/index.htm– http://www.idt.com/tools/parametric.html– http://www.ti.com/sc/docs/psheets/parms/uarts.htm#parms– http://www.gensemi.com/search/productsearch.htm– http://www.usa.samsungsemi.com/search/– http://www.gearfinder.com– http://www.mysimon.com/category/index.jhtml?c=babydiaperingbathing

Site list courtesy Mark Detweiler

185

“Parametric” Search Usage• Goal is to focus on product group for comparison

shopping.• Common Procedure

– Begin with a list of product "families" or groups. – User selects a category, and is prompted to

• 1) select a sub-category from a list of hyperlinks or • 2) select search parameters using a form

– If the number of results is too big, the system may prompt the user to refine the search further.

– When an acceptable number of results is returned, the user sees a list of products which can be:

• 1) sorted by various criteria • 2) selected for display in a comparison table • 3) viewed individually with more detail.

186

“Parametric” Search as used on these Sites

• Observations:– Only one facet (appropriate for

products?)– No query previews– Breadcrumbs rare– Many allow sorting by attribute to

facilitate comparison– “Others like this” simply moves up

the hierarchy

187

Summary and Conclusions

188

Summary• Web site search needs improvement

– Users want more organized results– Our approach: integrate navigation with search

• Metadata is being mixed and matched in interesting ways, but there are no guidelines on what works– We are investigating how to design websites

containing large sets of items

• Preliminary results indicate that metadata organization is useful in some situations– Depends on the type of search need

189

Advantages of the Methodology

• Supports different types of information seeking tasks

• Uses interface idioms known to be usable for general users

• Flexible content entry and update• Systematically integrates navigation &

search• Gives user control• Allows integration with advanced

methods

190

Summary

Our research goals– Systematically determine what works, with the

following emphases: • Task-centric• Integrate metadata with search• Dynamic previews• Easily retrace steps

– Develop recommendations that reflect both the task structure and the richness of the information structure

– In future: integrate with more sophisticated displays

191

Some Unanswered Questions

• How best show combinations of facets that consist of large hierarchies?

• How to use faceted metadata to expand (as opposed to refine)?

• How to integrate with relevance feedback (more like this)?

• How to incorporate user preferences and past behavior?

• How to combine facets to reflect tasks?

192

Thank you!

bailando.sims.berkeley.edu/flamenco.html

For more information:

1 designing information architecture for search marti hearst university of california, berkeley...

Documents

search uis

search interfaces

web search vs site search

search results sh

claims web search

common search problems

nsf9984741 slide

background slide