benchmarking your web site

A centre of expertise in digital information management

www.ukoln.ac.uk

Benchmarking Your Web Site

Brian KellyUKOLNUniversity of BathBath, BA2 7AY

[email protected]://www.ukoln.ac.uk/

UKOLN is supported by:


www.ukoln.ac.uk

2

Contents• Introduction• Why Benchmark?• What Is Benchmarking?• Tools For Testing Web Sites• From Testing To Benchmarking• Case Studies:

WebWatch Surveys Of UK University Sector• Commercial Approaches• Discussion


www.ukoln.ac.uk

3

What Do You Think Benchmarking is?Could you turn to your neighbour and ask:

• What do you think benchmarking is?

• How you think benchmarking can help you?


www.ukoln.ac.uk

4

Benchmarking: A Definition

Benchmarking is about identifying and measuring best practice processes that work elsewhere and then emulating them.

The aim is to reduce duplication by learning from others who have already found the solution.

It is about:

• Understanding your weaknesses

• Comparison with your peers

Note that best practices are constantly evolving.


www.ukoln.ac.uk

5

Aims Of This Talk

By the end of the session you should:• Be able to benchmark your Web site in

relation to other sites in your community• Have seen examples of use of auditing

and evaluating tools• Have considered other types of

benchmarking activity available• Be in a position to decide whether to

adopt this methodology in your organisation


www.ukoln.ac.uk

6

Approaches To Benchmarking

There are a number of approaches to benchmarking of Web sites:

Manual Benchmarking• Use of manual techniques such as questionnaires,

usability studies, etc.

Automated Benchmarking• Automated benchmarking makes use of software

tools to support benchmarking • Approaches can include:

• Dedicated benchmarking software products, typically running on desktop PC

• Use of benchmarking services available on the Web


www.ukoln.ac.uk

7

Benchmarking Approaches

Manual: End users involved in process Can receive feedback which cannot be obtained

using software Can be time-consuming and expensive

Automated: Can be less expensive Can scale to many thousands of resources Restricted to aspects which software can

process

This talk deals mainly with automated benchmarking, using Web-based tools

This talk deals mainly with automated benchmarking, using Web-based tools


www.ukoln.ac.uk

8

Web Testing Services

Many Web-based services are available for reporting on various aspects of Web sites, including:

• HTML compliance• Browser compatibility• Broken Links• Load Time• Accessibility• Link popularity• …


www.ukoln.ac.uk

9

Example - NetMechanic

NetMechanic:• A Web-based

service for checking Web sites

• Various functions available for free

• Additional functions such as more comprehensive testing are licensed

http://www.netmechanic.com/http://www.netmechanic.com/


www.ukoln.ac.uk

10

Example – Doctor HTML

Doctor HTML:• A Web page

analysis tool• Free for

testing individual pages

• A licensed version can be installed locally for checking entire Web sites

http://www2.imagiware.com/RxHTML/http://www2.imagiware.com/RxHTML/


www.ukoln.ac.uk

11

Example - Bobby

Bobby:• A Web-based

accessibility checker

• Can test individual pages

• A licensed downloadable version can check entire Web sites

http://bobby.watchfire.com/http://bobby.watchfire.com/


www.ukoln.ac.uk

12

Example – Link Popularity

Link popularity:• An indication of

the popularity of a Web site

• Can be obtained by analysing search engines such as AltaVista and Google

http://www.linkpopularity.com/http://www.linkpopularity.com/


www.ukoln.ac.uk

13

Example – Server Analysis

Netcraft’s server analysis provides details of:

• Web server software

• Operating system environment

• Server availability (limited)

• Nos. of servers in domain

http://www.netcraft.com/http://www.netcraft.com/


www.ukoln.ac.uk

14

Example – Server AnalysisThe University of Dundee provide a HTTP analysis tool:

• Analyses HTTP headers

http://www.somis.dundee.ac.uk/general/wizards/fetchhead.html

http://www.somis.dundee.ac.uk/general/wizards/fetchhead.html


www.ukoln.ac.uk

15

Usability

Usability normally requires manual testingHowever automated support tools are also available such as WebSAT

http://zing.ncsl.nist.gov/WebTools/WebSAT/operation.html

http://zing.ncsl.nist.gov/WebTools/WebSAT/operation.html


www.ukoln.ac.uk

16

From Testing To Benchmarking

These tools are typically used for testing individual pages on one’s own Web siteHowever applying tools across other Web sites allows:

• Comparisons to be made with competitors and collaborators

• Examples of best practices to be found and lessons learnt from

• Examples of problems to be found and mistakes avoided

• Trends to be monitored by repeat surveys• Limitations of tools to be found by testing across

wide range of Web site environments


www.ukoln.ac.uk

17

UK HE Case Studies

Every 3 months a WebWatch survey is published in the Ariadne e-journal:

• Surveys include:• Accessibility of entry points• Nos. of Web servers• Nos. of links to organisation• Size of entry points• Web server software• Relationships

• Together with manual surveys of:• Search engine software• 404 error pages

• See: <http://www.ukoln.ac.uk/web-focus/webwatch/articles/#latest>

Cas

e S

tud

ies


www.ukoln.ac.uk

18

Accessibility

In September 2002 Bobby was used to analyse the entry points the UK University Web sites.

http://www.ariadne.ac.uk/issue33/web-watch/http://www.ariadne.ac.uk/issue33/web-watch/ FindingsOnly 4 pages appeared to comply with Bobby AA guidelinesFurther analysis revealed that only 3 complied

Cas

e S

tud

ies


www.ukoln.ac.uk

19

Size Of Home Page

The size of UK University entry points was analysed in 1998 and repeated in June 2001

http://www.ariadne.ac.uk/issue28/web-watch/http://www.ariadne.ac.uk/issue28/web-watch/

The reasons for the large entry points was reviewed.

Cas

e S

tud

ies


www.ukoln.ac.uk

20

Numbers Of Web Servers

A survey of the numbers of Web servers was carried out in 2000 and repeated in 2002


Most institutions have a small number of Web servers but a few have over 100

Cas

e S

tud

ies


www.ukoln.ac.uk

21

What’s Related?

Netscape’s What’s Related tool was used to record:

• Popularity• Nos. of pages

indexed• Nos. of links to site

http://www.ariadne.ac.uk/issue27/web-watch/

http://www.ariadne.ac.uk/issue27/web-watch/


www.ukoln.ac.uk

22

Links To University Web Sites

A survey of the number of links to UK University Web sites was published in 2000

The survey used the AltaVista and Infoseek search engines



www.ukoln.ac.uk

23

Search Engines

A manual survey of search engines used on UK University Web sites was carried out in 1997 and has been repeated every 6 months in order to monitor trends



www.ukoln.ac.uk

24

404 Error Pages

A (manual ) survey of 404 error pages on UK University Web sites was carried out in 1999 and repeated in 2002.

WebWatch: 404s - What's Missing? June 1999<http://www.ariadne.ac.uk/issue20/404/>

Revisiting 404 Error Pages In UK University Web Sites, June 2002, <http://www.ariadne.ac.uk/issue32/web-watch/>

WebWatch: 404s - What's Missing? June 1999<http://www.ariadne.ac.uk/issue20/404/>

Revisiting 404 Error Pages In UK University Web Sites, June 2002, <http://www.ariadne.ac.uk/issue32/web-watch/>

The original survey and article helped to raise the importance of well-designed 404 pages as an important navigation feature

Cas

e S

tud

ies


www.ukoln.ac.uk

25

404 Error PagesSignificant changes have been made since the findings of the first survey was published

Apache/1.3.6 Server at www.shef.ac.uk Port 80


www.ukoln.ac.uk

26

Limitations Of This Approach

What limitations do you think this approach may have?

Lim

itat

ion

s


www.ukoln.ac.uk

27

Limitations

Limitations

Reliance on third party tools Inconsistencies

across tools

Unusual aspects of Web sites Inadequacies of

automated tools

Can’t handle Intranets

Performance implications

Legal and ethical issues

Personalision, cookies, etc.

Lim

itat

ion

s


www.ukoln.ac.uk

28

Reliance On Third Party Tools

This approach relies on use of third party software:

• Company may go out of business• Company may introduce charging or change

conditions of use• Companies may change format of its output• Company may change algorithms (possibly

without notification)

Example:• The Bobby accessibility checker withdrew

a summary of the file size of resources.• CAST sold Bobby: the new company

introduced limitations to use of Bobby

Example:• The Bobby accessibility checker withdrew

a summary of the file size of resources.• CAST sold Bobby: the new company

introduced limitations to use of Bobby

Lim

itat

ion

s


www.ukoln.ac.uk

29

Intranets, etc.

Use of public third party Web sites for testing:• May not work with Intranets or Web sites which

require a username and password to access

Possible Solution Some testing services allow you to give a username and password

If you do this, you are trusting the service not to steal the username and password!

Lim

itat

ion

s


www.ukoln.ac.uk

30

Inconsistencies

Different tools may have inconsistent approaches

Lim

itat

ion

s

ExampleNetMechanic and Bobby (previous version) reported on the file size of analysed pages.Bobby only analysed the HTML page and inline images.NetMechanic also included external JavaScript and stylesheet files.NetMechanic respected the Standard for Robot Exclusion (SRE) and would not analyse images if the SRE banned access.Bobby ignored the SRE.

ExampleNetMechanic and Bobby (previous version) reported on the file size of analysed pages.Bobby only analysed the HTML page and inline images.NetMechanic also included external JavaScript and stylesheet files.NetMechanic respected the Standard for Robot Exclusion (SRE) and would not analyse images if the SRE banned access.Bobby ignored the SRE.


www.ukoln.ac.uk

31

Inadequacies

Automated tools:• Are not suitable for testing all aspects of a Web

site• Need to be complemented by manual testing• Reliance on automated results without warning

notices can cause confusion

Lim

itat

ion

s

Accessibility TestingAutomated tools such as Bobby can report on missing ALT tags

However automated tools cannot report that a meaningful ALT tag is given

<img src=“important-graph”>

<img src=“important-graph”> alt=“”>


www.ukoln.ac.uk

32

Performance Implications

Automated tools:• Could degrade the performance of Web

sites if poorly designedLim

itat

ion

s

Case StudyA HTML validation tool was used to check A Web site.Shortly after it was used, it was found that it repeatedly sent HTTP requests to a Web site, which felt that this was a denial-of-service attack.

Case StudyA HTML validation tool was used to check A Web site.Shortly after it was used, it was found that it repeatedly sent HTTP requests to a Web site, which felt that this was a denial-of-service attack.


www.ukoln.ac.uk

33

Legal And Ethical Issues

If your survey findings:• Give a negative impression of a Web site• Are flawed, and give an mistaken negative

impression of a Web site

could you be sued?

Lim

itat

ion

s

Case StudyWebWatch surveys seek to highlight examples of best practices.Care is taken in the language used when problems are reported.

Case StudyWebWatch surveys seek to highlight examples of best practices.Care is taken in the language used when problems are reported.


www.ukoln.ac.uk

34

Personalisation

How should testing tools behave if Web sites provide personalised interfaces:

• Make use of username details to personalise context

• Personalise the interface based on the user’s browser

• Personalise the interface based on other environment factors (e.g. time, referer page, language setting, etc.)

Lim

itat

ion

s


www.ukoln.ac.uk

35

Unusual Aspects

How should testing tools deal with other unusual aspects of a Web site such as:

• Web page redirects Splash screens• Pop-up windows Frames• etc.

Lim

itat

ion

s

ExampleWhen you give a URL the Web page is redirected to another URL.The new page displays a splash screen for 5 seconds and then moves to a new page with contains frames.In addition a pop-up window is displayed.How many pages are there?

ExampleWhen you give a URL the Web page is redirected to another URL.The new page displays a splash screen for 5 seconds and then moves to a new page with contains frames.In addition a pop-up window is displayed.How many pages are there?


www.ukoln.ac.uk

36

Benchmarking And QA

The benchmarking approach may be used to:• Ensure that Web sites comply with standards and

best practices• May be of interest to funding bodies• UKOLN involved in work in this area to ensure

that projects comply with standards and best practices in order that they will be interoperable

See <http://www.ukoln.ac.uk/qa-focus/surveys/>

See <http://www.ukoln.ac.uk/qa-focus/surveys/>


www.ukoln.ac.uk

37

Who Else Is Doing This?

Who else may be carrying out benchmarking surveys?

We will use a Google search for:• “accessibility surveys”, “Web site benchmark”,

“HTML compliance surveys”, etc.

In order to explore other approaches including:• Commercial approaches• Non-commercial approaches


www.ukoln.ac.uk

38

US Accessibility Surveys

Axel Schmetzke has carried out surveys of the accessibility of selected US University Web sites

http://library.uwsp.edu/aschmetz/Accessible/websurveys.htm

http://library.uwsp.edu/aschmetz/Accessible/websurveys.htm


www.ukoln.ac.uk

39

Try It For Yourself

The methodology which has been described can be used by yourself across your community

Benefits:• You will get an idea of how your compare with

your peers• For national bodies, funders, etc. you can gain a

profile of your community• There may be opportunities for describing your

community at conferences, etc.


www.ukoln.ac.uk

40

Implementing A Benchmark Survey

To implement your own benchmark across a community you can simply examine WebWatch articles and adapt the HTML for your own use.

Further details at <http://www.ariadne.ac.

uk/issue29/web-watch/>

http://bobby.cast.org/bobby/bobbyServlet? URL=http%3A%2F%2Fwww2.brent.gov.uk%2F&output=Submit&gl=wcag1-aaa

Technique Used• Use the Web service on a site• Copy URL into template• Determine URL structure• Use as basis for use with other URLs

www.ukoln.ac.uk

41

Implementation (2)

Simple technique:• Copy URLs into a

template

Better technique:• Create HTML file using

a server-side script

Better technique:• Use a backend

database so resources can be more easily managed

<a href=“tool?url”>Try it</a> …



Do for all websites<a href=“query_string?$website”>Try it</a>

…


www.ukoln.ac.uk

42

Next Generation Tools

We can expect to see further development in testing tools:Why?

• Compliance with, say, e-Government guidelines• Ensure Web sites work e.g. e-commerce

How:• Tools which provide richer functionality (e.g.

dealing with personalised Web sites)• Development of “Web services” for testing• Agreement on standards e.g. what is a Web

“page”• Development of XML standards for interchange of

results e.g. EARL


www.ukoln.ac.uk

43

Resources For You To Use

A series of exercises on Web site benchmarking is available, which contains details of a number of benchmarking tools

http://www.ukoln.ac.uk/web-focus/events/conferences/ucisa-tlig-2002/benchmarking/

http://www.ukoln.ac.uk/web-focus/events/conferences/ucisa-tlig-2002/benchmarking/


www.ukoln.ac.uk

44

Questions

Any questions?

benchmarking your web site

Documents

benchmarking of web

testing web sites

entire web sites http

webbased tools

web testing services

uk university web sites

webbased services

numbers of web servers