visual search interfaces for online digital repositories dissertation defense edward clarkson may 6,...

63
Visual Search Visual Search Interfaces for Online Interfaces for Online Digital Repositories Digital Repositories Dissertation Defense Dissertation Defense Edward Clarkson Edward Clarkson May 6, 2009 May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd, Georgia Tech Gary Marchionini, UNC – CH Colin Potts, Georgia Tech John Stasko, Georgia Tech

Upload: winfred-clarence-fields

Post on 17-Jan-2016

217 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

Visual Search Interfaces for Visual Search Interfaces for Online Digital Repositories Online Digital Repositories

Dissertation DefenseDissertation DefenseEdward ClarksonEdward Clarkson

May 6, 2009May 6, 2009Committee

Jim Foley, Georgia Tech

Gregory Abowd, Georgia TechGary Marchionini, UNC – CH

Colin Potts, Georgia TechJohn Stasko, Georgia Tech

Page 2: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

22

IntroductionIntroductionProblem SpaceProblem Space Environment of interest: online digital Environment of interest: online digital

repositories.repositories.

Spectrum of directed to exploratory Spectrum of directed to exploratory information-seeking tasksinformation-seeking tasks Directed: specific end goalDirected: specific end goal Exploratory: unspecified goal (browsing)Exploratory: unspecified goal (browsing)

Page 3: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

33

IntroductionIntroductionCurrent ApproachesCurrent Approaches Keyword search engines Keyword search engines

and search engine and search engine result pages (SERPs)result pages (SERPs)

Faceted classification Faceted classification and navigationand navigation

Problems:Problems: Contextualization Contextualization Vocabulary visibilityVocabulary visibility Detecting relationships Detecting relationships

within datawithin data

Page 4: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

44

IntroductionIntroductionMy Approach: ResultMapsMy Approach: ResultMaps Goal: provide context for search resultsGoal: provide context for search results

Approach: Pair familiar text listings with Approach: Pair familiar text listings with alternative visual representation alternative visual representation Leverage hierarchical metadata (expose more of Leverage hierarchical metadata (expose more of

its structure)its structure) Outlier, cluster, relationship detection more Outlier, cluster, relationship detection more

apparentapparent

Page 5: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

[Johnson 1991][Johnson 1991] 55

IntroductionIntroductionMy Approach: ResultMapsMy Approach: ResultMaps

Encode total repository Encode total repository within treemap according within treemap according to hierarchical metadatato hierarchical metadata Recursive division of 2-D area Recursive division of 2-D area

according to tree dataaccording to tree data

Visually accentuate Visually accentuate matched results within matched results within overall spaceoverall space

Interactively link treemap Interactively link treemap with text result listingswith text result listings Treemaps preserve contextTreemaps preserve context Listings preserve familiarityListings preserve familiarity

Item Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum

Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum

Item Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum

Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum

Item Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum

Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum Lorem Ipsum

n-Item Listing

ResultMap 1

ResultMap m

Page 6: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

[Lamping 1995], [Plaisant 2002][Lamping 1995], [Plaisant 2002] 66

IntroductionIntroductionMy Approach: ResultMapsMy Approach: ResultMaps

AdvantagesAdvantages Space-constrained display.Space-constrained display. Leaf node emphasis.Leaf node emphasis. Well-suited for grid-based web.Well-suited for grid-based web.

DisadvantagesDisadvantages Actual hierarchical structure not evident.Actual hierarchical structure not evident. Learning effects for novice users.Learning effects for novice users.

vs.vs.

Page 7: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

77

IntroductionIntroductionOverview and ContributionsOverview and Contributions0.0. IntroductionIntroduction

Thesis and ContributionsThesis and Contributions

1.1. SERP ResultMapsSERP ResultMaps

2.2. Faceted ResultMapsFaceted ResultMaps Modeling Faceted MetadataModeling Faceted Metadata

3.3. Design ImplicationsDesign Implications System DesignSystem Design Evaluation DesignEvaluation Design

Page 8: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

88

Research ApproachResearch ApproachThesisThesis

““ResultMaps constitute a lightweight visualization ResultMaps constitute a lightweight visualization mechanism for digital repository search systems. They mechanism for digital repository search systems. They provide a means for contextualizing repository content, provide a means for contextualizing repository content, providing several prospective benefits, while not providing several prospective benefits, while not impairing usage for uninterested users. Empirical impairing usage for uninterested users. Empirical studies of their usage, along with models of faceted studies of their usage, along with models of faceted environments, suggest a set of implications for design environments, suggest a set of implications for design of future systems in this space and their evaluation.”of future systems in this space and their evaluation.”

Page 9: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

99

Research ApproachResearch ApproachThesisThesis

““ResultMaps constitute a lightweight visualization ResultMaps constitute a lightweight visualization mechanism for digital repository search systems. They mechanism for digital repository search systems. They provide a means for provide a means for contextualizing repository contentcontextualizing repository content, , providing providing several prospective benefitsseveral prospective benefits, while , while not not impairing usage for uninterested usersimpairing usage for uninterested users. Empirical . Empirical studies of their usage, along with models of faceted studies of their usage, along with models of faceted environments, suggest a set of implications for design environments, suggest a set of implications for design of future systems in this space and their evaluation.”of future systems in this space and their evaluation.”

Page 10: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

1010

Research ApproachResearch ApproachThesisThesis

““ResultMaps constitute a lightweight visualization ResultMaps constitute a lightweight visualization mechanism for digital repository search systems. They mechanism for digital repository search systems. They provide a means for contextualizing repository content, provide a means for contextualizing repository content, providing several prospective benefits, while not providing several prospective benefits, while not impairing usage for uninterested users. impairing usage for uninterested users. Empirical Empirical studies of their usagestudies of their usage, along with models of faceted , along with models of faceted environments, suggest a set of implications for design environments, suggest a set of implications for design of future systems in this space and their evaluation.”of future systems in this space and their evaluation.”

Page 11: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

1111

Research ApproachResearch ApproachThesisThesis

““ResultMaps constitute a lightweight visualization ResultMaps constitute a lightweight visualization mechanism for digital repository search systems. They mechanism for digital repository search systems. They provide a means for contextualizing repository content, provide a means for contextualizing repository content, providing several prospective benefits, while not providing several prospective benefits, while not impairing usage for uninterested usersimpairing usage for uninterested users. Empirical . Empirical studies of their usage, along with models of faceted studies of their usage, along with models of faceted environments, suggest a set of implications for design environments, suggest a set of implications for design of future systems in this space and their evaluation.”of future systems in this space and their evaluation.”

Page 12: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

1212

Research ApproachResearch ApproachThesisThesis

““ResultMaps constitute a lightweight visualization ResultMaps constitute a lightweight visualization mechanism for digital repository search systems. They mechanism for digital repository search systems. They provide a means for contextualizing repository content, provide a means for contextualizing repository content, providing several prospective benefits, while not providing several prospective benefits, while not impairing usage for uninterested users. Empirical impairing usage for uninterested users. Empirical studies of their usage, along with studies of their usage, along with models of faceted models of faceted environmentsenvironments, suggest a set of implications for design , suggest a set of implications for design of future systems in this space and their evaluation.”of future systems in this space and their evaluation.”

Page 13: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

1313

Research ApproachResearch ApproachThesisThesis

““ResultMaps constitute a lightweight visualization ResultMaps constitute a lightweight visualization mechanism for digital repository search systems. They mechanism for digital repository search systems. They provide a means for contextualizing repository content, provide a means for contextualizing repository content, providing several prospective benefits, while not providing several prospective benefits, while not impairing usage for uninterested users. Empirical impairing usage for uninterested users. Empirical studies of their usage, along with models of faceted studies of their usage, along with models of faceted environments, suggest a set of environments, suggest a set of implications for design implications for design of future systems in this space and their evaluationof future systems in this space and their evaluation.”.”

Page 14: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

1414

Research ApproachResearch ApproachQuestionsQuestions1.1. How does adding SERP RMs affect How does adding SERP RMs affect user performanceuser performance on DL search on DL search

tasks?tasks?2.2. How does adding RMs affect How does adding RMs affect subjective impressionssubjective impressions—such as —such as

satisfaction and engagementsatisfaction and engagement—of DL interfaces?—of DL interfaces?3.3. Do RMs yield a greater level of Do RMs yield a greater level of knowledgeknowledge about the overall content in about the overall content in

digital library digital library as an ancillary effectas an ancillary effect of normal usage? of normal usage?4.4. How do RMs affect query string characteristics over sequences of How do RMs affect query string characteristics over sequences of

queries and other types of user behavior?queries and other types of user behavior?

5.5. How do faceted ResultMaps affect How do faceted ResultMaps affect subjective impressionssubjective impressions of faceted DL of faceted DL interfaces?interfaces?

6.6. How do faceted ResultMaps affect the incidence of How do faceted ResultMaps affect the incidence of data insightsdata insights (e.g., (e.g., identifying data relationships such as correlations between facets)?identifying data relationships such as correlations between facets)?

Page 15: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

1515

Research ApproachResearch ApproachExperimental StudiesExperimental Studies

Name Implementation Purpose Technique Duration Site ResearchQuestions

R1 SERPFormative/Summative

Experimental 1 hour Lab 1,2,3

R2 SERP Summative Experimental 1 hour Lab 1,2,3,4

R3 SERP Summative Log Analysis 6 months Field 4

F1 Faceted Formative Think-aloud 1-2 hours Lab 5,6

F2 FacetedFormative/Summative

Quasi-experimental

1 hour Field 5

F3 Faceted SummativeQuasi-

experimental6 weeks Field 5,6

Page 16: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

1616

SERP ResultMapsSERP ResultMapsDemoDemo

Page 17: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

[Lewis 1995], [Ghani 1991], [Cap[Lewis 1995], [Ghani 1991], [Capra 2007]ra 2007]

1717

SERP EvaluationsSERP EvaluationsStudy R2 DesignStudy R2 Design SummativeSummative

RQs: performance, subjective RQs: performance, subjective effects, ancillary effects, effects, ancillary effects, query constructionquery construction

Split-plot designSplit-plot design RM vs. control (between); RM vs. control (between);

repository size (within)repository size (within)

MeasuresMeasures Task time/accuracy, CSUQ, Task time/accuracy, CSUQ,

engagement, enjoymentengagement, enjoyment

Page 18: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

1818

SERP EvaluationsSERP EvaluationsStudy R2 ProcedureStudy R2 Procedure DatasetsDatasets

HCC EDL (~500 nodes)HCC EDL (~500 nodes) Intute (~5000 nodes)Intute (~5000 nodes)

6 tasks (1 practice)6 tasks (1 practice) Range of directed to open-Range of directed to open-

endedended

36 volunteers from intro 36 volunteers from intro HCI, PSYC coursesHCI, PSYC courses

Page 19: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

1919

SERP EvaluationsSERP EvaluationsStudy R2 ResultsStudy R2 Results

(n=19)(n=19)ResultMapResultMap

(n=17)(n=17)ControlControl

SmallSmall 587 nodes587 nodes

CSUQ: 5.17, CSUQ: 5.17, =0.69=0.69*ENJ: 5.08, *ENJ: 5.08, =1.10=1.10ENG: 5.33, ENG: 5.33, =0.88=0.88

CSUQ: 4.84, CSUQ: 4.84, =0.83=0.83*ENJ: 4.22, *ENJ: 4.22, =0.97=0.97ENG: 5.00, ENG: 5.00, =0.95=0.95

LargeLarge 5,518 nodes5,518 nodes

CSUQ: 4.61, CSUQ: 4.61, =0.94=0.94*ENJ: 4.91, *ENJ: 4.91, =1.04=1.04ENG: 5.28, ENG: 5.28, =1.09=1.09

CSUQ: 4.44, CSUQ: 4.44, =0.79=0.79*ENJ: 3.94, *ENJ: 3.94, =0.90=0.90ENG: 5.02, ENG: 5.02, =0.77=0.77

RM users fast, more accurate, better subjective results…RM users fast, more accurate, better subjective results… But only enjoyment difference significantBut only enjoyment difference significant

No differences in query characteristics between interfacesNo differences in query characteristics between interfaces Large repository had more, longer, more varied queriesLarge repository had more, longer, more varied queries

(n=19)(n=19)ResultMapResultMap

(n=17)(n=17)ControlControl

SmallSmall 587 nodes587 nodes

163s, 163s, =58s=58s70.5%, 70.5%, =14.4%=14.4%

183s, 183s, =71s=71s59.4%, 59.4%, =23.8%=23.8%

LargeLarge 5,518 nodes5,518 nodes

198s, 198s, = 49s = 49s 72.3%, 72.3%, =17.8%=17.8%

212s, 212s, = 64s = 64s 66.5%, 66.5%, =24.5%=24.5%

Page 20: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

2020

SERP EvaluationsSERP EvaluationsStudy R2 ResultsStudy R2 Results Positive RM ratingsPositive RM ratings

Anecdotal commentsAnecdotal comments ““Visual representation of Visual representation of

materials…is useful”materials…is useful”

No negative results, and No negative results, and many near-significant many near-significant resultsresults

StudyStudy MeasureMeasure PP

R1R1Outlier Task Outlier Task

AccuracyAccuracy0.070.07

R2R2 Task AccuracyTask Accuracy 0.110.11

R2R2Rep. Knowledge Rep. Knowledge

(Categories)(Categories)0.060.06

R2R2Design Impact Design Impact on Difficultyon Difficulty

0.120.12

R2R2Design Impact Design Impact

on Timeon Time0.090.09

Page 21: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

2121

IntroductionIntroductionOverview and ContributionsOverview and Contributions0.0. IntroductionIntroduction

1.1. SERP ResultMapsSERP ResultMaps

2.2. Faceted ResultMapsFaceted ResultMaps Modeling Faceted MetadataModeling Faceted Metadata

3.3. Design ImplicationsDesign Implications System DesignSystem Design Evaluation DesignEvaluation Design

Page 22: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

[Yee 02], [Capra 07], [Huynh 09], [Yee 02], [Capra 07], [Huynh 09], [Lee 09], [schraefel 05][Lee 09], [schraefel 05]

2222

Faceted NavigationFaceted NavigationIntroductionIntroduction Classify items in multiple Classify items in multiple

independent independent categorizations (facets)categorizations (facets) Ex.: architectural works Ex.: architectural works

(focus)(focus)

High recent uptake in High recent uptake in research, e-commerceresearch, e-commerce Amazon, eBay, etc.Amazon, eBay, etc. Flamenco, Relation Browser, Flamenco, Relation Browser,

mSpace, etc.mSpace, etc.

Items

Categorization

Categorization Categorization

Categorization

FocusFocus

FacetsFacets

Page 23: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

[Yee 02], [Capra 07], [Huynh 09], [Yee 02], [Capra 07], [Huynh 09], [Lee 09], [schraefel 05][Lee 09], [schraefel 05]

2323

Faceted NavigationFaceted NavigationIntroductionIntroduction Classify items in multiple Classify items in multiple

independent independent categorizations (facets)categorizations (facets) Ex.: architectural works Ex.: architectural works

(focus)(focus)

High recent uptake in High recent uptake in research, e-commerceresearch, e-commerce Amazon, eBay, etc.Amazon, eBay, etc. Flamenco, Relation Browser, Flamenco, Relation Browser,

mSpace, etc.mSpace, etc.

Arch. Works

Location

Materials Architect

Arch. Gender

FocusFocus

FacetsFacets

Page 24: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

2424

Faceted NavigationFaceted NavigationIntroductionIntroduction

Focus Area

Facet Area

.·˙

Arch. Works

Location

Materials Architect

Arch. Gender

Page 25: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

2525

Faceted NavigationFaceted NavigationIntroductionIntroduction

Facets contain relevant Facets contain relevant facet valuesfacet values Represent available Represent available

constraintsconstraints

Selections trigger UI updateSelections trigger UI update Matching focus itemsMatching focus items Relevant facet valuesRelevant facet values

Differences from SERP env.Differences from SERP env.

Page 26: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

2626

Faceted ResultMapsFaceted ResultMaps

Scalability factorsScalability factors Aggregating nodesAggregating nodes Ordered layoutsOrdered layouts

ImplementationsImplementations Flamenco-basedFlamenco-based SwivelSwivel

Page 27: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

2727

Faceted ResultMapsFaceted ResultMapsFlamenco DemoFlamenco Demo

Page 28: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

2828

)2.2.(

1.2.1.2.1..

)2.2.(

1.2.1.2.1..

)..(

)()(

)()(

)()()()(

)(

.:

,,1,,

,1,2,1,1,

,1,11,1,1

,11,12,11,11,1

,1,

,1,

,11,11

,1,,11,1

11

111

11

1̀1

1

n

nn

p

nn

n

jnknnkn

knknnnn

jkk

kk

jpp

knmnkm

kk

knnk

cIdecIde

IdeIdeIdeIdeIdeIdt

cIdecIde

IdeIdeIdeIdeIdeIdt

cparenttcparentt

eFACETeFACET

eFACETeFACET

eeee

tINTEREST

IdtQ

Modeling Faceted Metadata Modeling Faceted Metadata Modeling Faceted MetadataModeling Faceted Metadata

Focus

Facet

Facet

Facet

Ind. Facet

Ind. Facet

Ind. Facet

Page 29: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

2929

Modeling Faceted Metadata Modeling Faceted Metadata ImplicationsImplications Non-reductive selections could be detrimental to Non-reductive selections could be detrimental to

performanceperformance For M facets, N selections: M+1 queries that are N+1-way joinsFor M facets, N selections: M+1 queries that are N+1-way joins

Asynchronous preview dataAsynchronous preview data For M hierarchical facets, mean of B sub-values and N selections: For M hierarchical facets, mean of B sub-values and N selections:

tooltip previews require factor of B+1 more queriestooltip previews require factor of B+1 more queries

Only one extra query per ResultMapOnly one extra query per ResultMap ResultMaps require less restrictive version of query modelResultMaps require less restrictive version of query model

Page 30: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3030

Faceted EvaluationsFaceted EvaluationsStudy F2 DesignStudy F2 Design Summative/FormativeSummative/Formative Quasi-experimental between-subjectsQuasi-experimental between-subjects

7 ARCH 2111 precepts7 ARCH 2111 precepts 1 pilot, 4 test, 2 aborted (n=23 control, n=15 RM)1 pilot, 4 test, 2 aborted (n=23 control, n=15 RM)

Flamenco ResultMaps vs. controlFlamenco ResultMaps vs. control

Measures: tasks, CSUQ, engagement/enjoymentMeasures: tasks, CSUQ, engagement/enjoyment

Page 31: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3131

Faceted EvaluationsFaceted EvaluationsStudy F2 ProcedureStudy F2 Procedure Archivision data: 16K+ images, 11 facets, 6 RMsArchivision data: 16K+ images, 11 facets, 6 RMs

Preview performance poor; disabled for test.Preview performance poor; disabled for test.

In-class assignment during 50 minute class meetingIn-class assignment during 50 minute class meeting Ex.Ex.: Search for and list differences between American and Italian civic buildings: Search for and list differences between American and Italian civic buildings Task grading according to TA template/inputTask grading according to TA template/input

Execution problemsExecution problems Network outageNetwork outage Concurrent performanceConcurrent performance Warm-up/practice constraintWarm-up/practice constraint

Page 32: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3232

Faceted EvaluationsFaceted EvaluationsStudy F2 ResultsStudy F2 Results

Task 1aTask 1a Task 1bTask 1b Task 2Task 2

ResultMapResultMap 1.361.36 1.141.14 1.361.36

ControlControl 1.001.00 0.860.86 1.171.17

pp 0.130.13 0.370.37 0.440.44

Dominated by problems (performance, etc.)Dominated by problems (performance, etc.)

RM users: explicit positive commentsRM users: explicit positive comments Control users: explicit references to RM-like featuresControl users: explicit references to RM-like features

[It is] difficult to draw relationships with other categories [and] difficult to [It is] difficult to draw relationships with other categories [and] difficult to cross-reference material outside the searchcross-reference material outside the search

Side by side comparison of…categories would be useful; more graphics might Side by side comparison of…categories would be useful; more graphics might be nicer.be nicer.

Page 33: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3333

Faceted EvaluationsFaceted EvaluationsStudy F2 ImplicationsStudy F2 Implications SimplificationSimplification

No key facetNo key facet No differentiation between on/off screenNo differentiation between on/off screen Increase RM sizeIncrease RM size

Improve performanceImprove performance

Page 34: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3434

Faceted ResultMapsFaceted ResultMapsDesign IterationDesign Iteration SwivelSwivel

Modern tech., guided Modern tech., guided by model workby model work

RM versionRM version No key facetNo key facet Limit depthLimit depth

Stacked bar versionStacked bar version

Page 35: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3535

Faceted ResultMapsFaceted ResultMapsSwivel DemoSwivel Demo

Page 36: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3636

Faceted EvaluationsFaceted EvaluationsStudy F3 DesignStudy F3 Design SummativeSummative Longitudinal, within-Longitudinal, within-

subjects, quasi-subjects, quasi-experimentalexperimental 4 ARCH 2112 precepts 4 ARCH 2112 precepts

(N=66; 1/3 from F1 (N=66; 1/3 from F1 classes)classes)

Measures: CSUQ, Measures: CSUQ, engagement/enjoymentengagement/enjoyment

Page 37: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3737

Faceted EvaluationsFaceted EvaluationsStudy F3 ProcedureStudy F3 Procedure Longitudinal: 2 weeks x 3 conditionsLongitudinal: 2 weeks x 3 conditions

Same Archivision data from F1Same Archivision data from F1 Reduced facets: 8 facets, 4 RMsReduced facets: 8 facets, 4 RMs

Experimental variances/threats to validity:Experimental variances/threats to validity: Exam after period 1Exam after period 1 Cancelled classesCancelled classes

Page 38: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3838

Faceted EvaluationsFaceted EvaluationsStudy F3 ResultsStudy F3 Results Analysis problems: Analysis problems:

response rate and response rate and ‘useful’ responses ‘useful’ responses

27% returned all 3 27% returned all 3 surveys (9% all useful)surveys (9% all useful)

42 students returned at 42 students returned at least one useful surveyleast one useful survey

Survey Survey 11

Survey Survey 22

Survey Survey 33

Precept 1Precept 1 83%83% 50%50% 42%42%

Precept 2Precept 2 27%27% 33%33% 56%56%

Precept 3Precept 3 72%72% 83%83% 78%78%

Precept 4Precept 4 78%78% 66%66% 33%33%

TotalTotal 63%63% 59%59% 53%53%

Page 39: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

3939

Faceted EvaluationsFaceted EvaluationsStudy F3 ResultsStudy F3 Results Sig. pairwise CSUQ corr.:Sig. pairwise CSUQ corr.:

Bar/RM:Bar/RM: rr22=0.762; p < 0.01; n=14 =0.762; p < 0.01; n=14

RM/control:RM/control: rr22==--0.839; p<0.02; n=70.839; p<0.02; n=7

Bar/ControlBar/Control rr22==--0.513; p=0.09; n=120.513; p=0.09; n=12

Behavioral logs (page views)Behavioral logs (page views) RM/Bar longer sessions, more RM/Bar longer sessions, more

usage of facets than search usage of facets than search queriesqueries

ControlControl BarBar RMRM

LandingLanding 6262 135135 9696

Non-LandingNon-Landing 6565 170170 138138

Facet | SearchFacet | Search 28 | 3928 | 39 103103 | | 8181 8080 | 75 | 75

13

3

12

6

15

7

910

9

Rank 1 Rank 2 Rank 3

Fre

qu

ency

RM

Bar

Control

Page 40: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

4040

Faceted EvaluationsFaceted EvaluationsStudy F3 ImplicationsStudy F3 Implications Link to learning style?Link to learning style?

““The Map was more of a visual aid and helped me The Map was more of a visual aid and helped me organize my thoughts”organize my thoughts”

““Map is the most visual and most appealing of the three. Map is the most visual and most appealing of the three. This is good, especially when dealing with architects”This is good, especially when dealing with architects”

““I am a visual learner so the map version seemed to be my I am a visual learner so the map version seemed to be my favorite.”favorite.”

Power of defaults:Power of defaults: 33 direct usages (clicks) of top RM, 9 of next, none of 33 direct usages (clicks) of top RM, 9 of next, none of

others. others.

Page 41: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

4141

Design ImplicationsDesign ImplicationsSystem Design (from Evaluation)System Design (from Evaluation) Power of defaults (F2, F3)Power of defaults (F2, F3)

Ordering, progressive disclosure (trade-off with discoverability, insight)Ordering, progressive disclosure (trade-off with discoverability, insight)

UI overload (R2, F2, F3) UI overload (R2, F2, F3) Infovis additions less likely to overload on simpler base systems (mitigated by Infovis additions less likely to overload on simpler base systems (mitigated by

user experience, learning style).user experience, learning style).

Factor Scale Description

UserExperience Novice Expert

The skill level a user has with both a specific system design or with similar features from other systems.

LearningStyle

Visual Non-visualThe inherent level of comfort a user has with visual representations of information.

Interface

Base Complexity

Simple ComplexThe intricacy of the text-based portion of an online search system.

Visual Complexity

Simple ComplexThe intricacy of the visualization portion of an online search system.

Page 42: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

4242

Design ImplicationsDesign ImplicationsSystem Design (from Model)System Design (from Model) Extensions to faceted navigation toolsExtensions to faceted navigation tools

Probabilistic categorizationProbabilistic categorization Indirect selectionsIndirect selections

Only asynchronous previews are scalableOnly asynchronous previews are scalable

Page 43: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

4343

Design ImplicationsDesign ImplicationsSystem EvaluationSystem Evaluation User interest (R1,R2,F1,F2,F3)User interest (R1,R2,F1,F2,F3)

Direct vs. analytic data interestDirect vs. analytic data interest Pedagogical evaluation linkPedagogical evaluation link

Facilitate insight reporting (F2,F3)Facilitate insight reporting (F2,F3) Analytic user interestAnalytic user interest Low-cost reporting mechanisms (annotation, bookmarking) Low-cost reporting mechanisms (annotation, bookmarking) Evaluation incentivesEvaluation incentives

Task comparison: measure of isomorphism (R1,R2,F1)Task comparison: measure of isomorphism (R1,R2,F1) Similarity in local tree structures (tree alignment)Similarity in local tree structures (tree alignment) Similarity in target itemsSimilarity in target items

Page 44: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

4444

Conclusions and Future WorkConclusions and Future Work

Faceted navigationFaceted navigation Extending data, query capabilities of toolsExtending data, query capabilities of tools Complexity analysis of interface features, data limits based Complexity analysis of interface features, data limits based

on data/query modelson data/query models

Similarity of HCIR and infovis problemsSimilarity of HCIR and infovis problems Data integration/reformulation: (semi) automatic toolsData integration/reformulation: (semi) automatic tools Quantitative task isomorphism metricsQuantitative task isomorphism metrics

Page 45: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

4545

AcknowledgementsAcknowledgements

PersonalPersonal

ProfessionalProfessional Jim and committeeJim and committee Sham NavatheSham Navathe Sabir Khan, Benjy Flowers, Myung Seok Hyun, Carina Antunez, Sabir Khan, Benjy Flowers, Myung Seok Hyun, Carina Antunez,

Marietta Monaghan Marietta Monaghan

FinancialFinancial Stephen Fleming ChairStephen Fleming Chair DHS, NVAC, RVACDHS, NVAC, RVAC

Page 46: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

4646

Questions?Questions?

Page 47: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

BackupsBackups

Page 48: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

4848

SERP EvaluationsSERP EvaluationsStudy R3Study R3 HCC EDL log analysis, April-Oct. 2008HCC EDL log analysis, April-Oct. 2008

Users randomly shown either RM or control Users randomly shown either RM or control version of SERP based on IP address.version of SERP based on IP address.

22,867 requests22,867 requests 516 search requests (272 RM; 244 control)516 search requests (272 RM; 244 control)

……but no interesting resultsbut no interesting results

Page 49: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

4949

SERP ResultMapsSERP ResultMaps

Page 50: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5050

SERP ResultMapsSERP ResultMaps

Page 51: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5151

SERP ResultMapsSERP ResultMaps

Page 52: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5252

SERP ResultMapsSERP ResultMaps

Page 53: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5353

SERP ResultMapsSERP ResultMaps

Page 54: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5454

SERP ResultMapsSERP ResultMaps

Page 55: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5555

Faceted ResultMapsFaceted ResultMapsFlamencoFlamenco Larger, dimmer color

area proportional to number of matching

items on all result pages.

Smaller, brighter area proportional to

number of matching items on current result

page (if any).

Brushing a facet value outlines ResultMap nodes that have matching items; other nodes dim to help

distinguish them.

Page 56: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5656

Faceted ResultMapsFaceted ResultMapsFlamencoFlamenco

Page 57: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5757

Faceted ResultMapsFaceted ResultMapsFlamencoFlamenco

Page 58: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5858

Faceted ResultMapsFaceted ResultMapsSwivelSwivel

Page 59: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

5959

Faceted ResultMapsFaceted ResultMapsSwivelSwivel

Page 60: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

6060

Swivel BarsSwivel Bars

Page 61: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

6161

Items

Categorization

Categorization Categorization

Categorization

Modeling Faceted MetadataModeling Faceted Metadata Basic ER Data ModelBasic ER Data Model

Focus

Facet

Facet

Facet

Facet

FocusFocus

FacetsFacets

RelationshipsRelationships

Page 62: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

6262

Modeling Faceted MetadataModeling Faceted Metadata Extended ER Data ModelExtended ER Data Model

Facet

Ind. Facet

Focus

Ind. Facet

Facet

Ind. Facet

Facet

Ind. Facet

Focus

Facet

Facet

Facet

Ind. Facet

Ind. Facet

Ind. Facet

Facet

Page 63: Visual Search Interfaces for Online Digital Repositories Dissertation Defense Edward Clarkson May 6, 2009 Committee Jim Foley, Georgia Tech Gregory Abowd,

6363

…for each facet selection

Modeling Faceted Metadata Modeling Faceted Metadata Extended Query ModelExtended Query Model

Focus

Facet

Facet

Facet

Ind. Facet

Ind. Facet

Ind. Facet

)2.2.(

1.2.1.2.1..

)2.2.(

1.2.1.2.1..

)..(

)()(

)()(

)()()()(

)(

.:

,,1,,

,1,2,1,1,

,1,11,1,1

,11,12,11,11,1

,1,

,1,

,11,11

,1,,11,1

11

111

11

1̀1

1

n

nn

p

nn

n

jnknnkn

knknnnn

jkk

kk

jpp

knmnkm

kk

knnk

cIdecIde

IdeIdeIdeIdeIdeIdt

cIdecIde

IdeIdeIdeIdeIdeIdt

cparenttcparentt

eFACETeFACET

eFACETeFACET

eeee

tINTEREST

IdtQ

Hierarchical constraint

Name the facets that have selections

Join facet with selection to interest relationFacet selection

constraint