revisitation patterns and disorientation eelco herder session on empirical studies

Revisitation Patternsand Disorientation

Eelco Herder

Session on Empirical StudiesABIS Workshop 2003

Karlsruhe

Research Partners

IntroductionPrediction of DisorientationDescription of Pilot Study

Experimental ResultsDiscussion

Revisitation Patterns and Disorientation

Presentation Overview

Becoming Disoriented in a Web Site

Introduction

Where am IHow did I come hereWhere can I go to

Due to•non-linear structure of web sites•lack of context information•unfamiliarity with site or domain

Symptoms

Context Information Prevents Disorientation

Contextual links•e.g. menus, indexes and site maps•define users’ spatial or temporal context•are needed to understand the site structure•differ from site to site

Associative links•embedded in the content•interlink semantically related concepts•do not expose the site structure

Contextual Navigation Support

IntroductionPrediction of Disorientation

Description of Pilot StudyExperimental Results

Discussion

Revisitation Patterns and Disorientation

Presentation Overview

Task Efficiency and DisorientationUsers may be exploring the site or looking for specific information.

Users that are exploring a site take some disorientation for granted, similar to tourists in a city center.

We want to find out when people perceive their sense of disorientation as a problem.

Navigation Patterns

Goal Directed Exploring

Site Graph and Navigation Graph

Prediction of Disorientation

Web sites are commonly modeled as directed graphs

User navigation paths can be seen as an overlay of the site graph

1

2

5

3

4

6

Characteristics of User NavigationNavigation graphs describe browsing behavior

Path length and linearityRevisitation (cycles and clustering)

Analysis of navigation patterns helps in answering questions:

Is the user exploring or is he lost?Does the user understand the site structure?

Prediction of Disorientation

Revisitation and LostnessContradictory experimental results found in literature

P.A. SmithUsers who cannot find desired information are lost.Lostness measure based on number of revisits:the more revisits, the more likely the user is lost

J.E. McEneaneyWell-performing users return often to navigational landmarks.The more linear the path, the worse the performance

Prediction of Disorientation

IntroductionPrediction of Disorientation

Description of Pilot StudyExperimental Results on Revisitation

Discussion

Revisitation Patterns and Disorientation

Presentation Overview

Experimental Design (1)ObjectivesFind relations betweennavigation patterns and user satisfaction

Subjects30 randomly selected Dutch students(15 from Utrecht, 15 from Twente)

Session LengthComplete session lasted 2.5 hourNavigation session lasted only 40 minutes

Description of Pilot Study

Experimental Design (2)Data Gathering Methods

Questionnaires(motivational factors, expertise, evaluation)

Performance tests(working memory, spatial capacity, episodic memory)

Observation and interview

Thinking aloud protocols (recorded on tape)

Navigation data capturing (Scone)

Description of Pilot Study

Web Sites Used and Example TasksWhat are the definitions ofpersonal budget and financial goalas they are presentedwithin this website?

Calculate how much the real valueof £ 7.826 will decrease in 5 yearsgiven an annual inflation of 3%.

Description of Pilot Study

Access Tracking Software

Analysis software Graph Package

path visualizationssite metrics

navigation metricsinference methods Description of Pilot Study

A Navigation Graph

Description of Pilot Study

IntroductionPrediction of DisorientationDescription of Pilot StudyExperimental Results

Discussion

Revisitation Patterns and Disorientation

Presentation Overview

Experimental Results

Revisitation Patterns

Percentage of RevisitsAbout 40% for personal finance tasks,about 25% for browsing on Amazon

(Tauscher: revisitation rate of 58% on long term)

Use of Back Button•On average 9,2 % of navigation actions.•Large differences between sites (3,3 – 15,8 - 7,8)•User tendencies are comparable

Experimental Results

Revisitation: ‘I will find my way back later’Average Connected Distance (ACD)The expected length of the path betweenany connected two pages x and y in the navigation graph.

What Does It Mean?A higher ACD indicates that users will visit more pages before they return to a certain pageby a direct link, instead of by multiple clicks on the back button

ACD and working memory positively correlated

Experimental Results

ACD vs Number of Revisits

CONNECT

161412108642

RE

VISI

TS

60

50

40

30

20 Rsq = 0,2399

‘I will find my way back later’ means less revisitation

Experimental Results

Return RateReturn rateThe average number of times that a user will revisit a page.

Calculated by averaging the number of visits on all pages that have been visited at least twice.

What Does It Mean?A higher return rate means that users returnto a specific set of pages, most likely with a specific goal in mind.

Experimental Results

ACD, Return Rate and Disorientation

People with high Average Connected Distance have low revisitation rates – and low back button rates

They also make more intensive use of landmarks (once a page is revisited once, they come back more often)

Intensive use of landmarks is negatively correlated with disorientation.

Experimental Results

Average Time Spent on PagesMedian View TimeThe average time a user spends on a page while browsing.

In contrast, the average view time is overly influenced by a small set of ‘high content’ pages.

Our FindingsUsers who feel disoriented, spend more time on pages while browsing.The correlation is not very strong, though.

IntroductionPrediction of DisorientationDescription of Pilot Study

Experimental ResultsDiscussion

Revisitation Patterns and Disorientation

Presentation Overview

Revisitation Patterns and Disorientation

Combined measures on user navigation help in predicting users becoming disoriented.

Revisitation patterns are more accurate thanthe amount of revisitation.

We are still far away from perfection.Most likely uncertainty cannot be eliminated completely.

Discussion

Leveraging LostnessEven with slightly imperfect prediction of disorientation it is still possible to offer adaptive navigation support.

Examples of adaptive contextual navigation aids:•personalized site maps•visualization of navigation history•direct guidance•link annotation

In order to do that, we must be able to distinguish between the various kinds of disorientation.

Discussion

Limitations of this Study

The site structure has not been taken into account, due to its complexity.We have evidence that the structure is an important factor.

We haven’t yet been able to analyze the relations between the various measures in detail. This will involve time-consuming visual analysis, combined with more specific tasks and more detailed evaluation.

Discussion

Revisitation Patterns and Disorientation

Time for Questions

email: herder@cs.utwente.nl

Research Partners