usability with project lecture 10 – 19/03/10

© Simeon Keates 2010

Usability with ProjectLecture 10 – 19/03/10Dr. Simeon Keates


Exercise – part 1

Consider sending an SMS or e-mail Look at one of your mobile phones … And a laptop, etc.

Perform exclusion calculations on each product using the data on:• http://www.eng.cam.ac.uk/inclusivedesign/


Exercise – part 2

Identify the common methods of interacting with the product

Identify which of the 7 DFS capability scales are involved in the interaction

Based on the DFS scales, estimate the limiting capability demand for each scale


Exercise – part 3

Report the number and %age of people excluded by each capability demand• For 16+ and 75+

Report the total number and %age of people excluded by the product• For 16+ and 75+

Prepare a 5 minute presentation to discuss:• Your exclusion calculation assumptions• Your exclusion calculation results• What were the principal causes of exclusion?• What do you think should be done to reduce the exclusion for each product?


Assessment of DTT STBs...


A systems overview

User

Remotecontrol

Service provider

Television

Remotecontrol

Set top box (STB)


Motivations for study

Commissioned by UK Department of Trade and Industry Wanted to find out who could not access DTT Original focus on ‘the disabled’ Definition broadened...

Why?


Typical assessment methods used in this research

Expert assessment

Exclusion analysis

User observation

Questionnaires

Interviews

Focus Groups

Assessment of STBs

Customer expectations


Methodology - Choice of STBs

2 STBs chosen for study STB1 - marketed as “easy to use” STB2 - market leader

1 digital satellite system chose as comparison STB3 – developed by content provider


Expert assessment

4 assessors 2 with DTV experience, 2 without STB protocol only Aims to identify most likely sources of problems• Define protocol for following assessments


Methodology - Analogue TV protocol

6 activities• Switch on• Change channel• Change volume• Teletext (find local weather)• Subtitles (on/off)• Switch off


Methodology - STB protocol

8 activities• Installation• Switch on• Change channels (direct + EPG)• Change volume• Teletext (find local weather)• Subtitles (on/off)• BBCi (find local weather)• Switch off


Expert Assessment - Results

13 major sources of difficulty found 4 – Installation and set-up• e.g. instruction manual, initial tuning

5 – Operation • e.g. multiple modes, subtitles

4 – Remote controls• e.g. labelling, layout


Exclusion analysis

Systematic analysis Combined with data from Office of National Statistics• Population data 1996/7 UK Disability Follow-Up Survey

Aims to calculate how many people have the difficulties highlighted by expert assessment• How many people in the user observation should have those difficulties?


Example – breakdown of installationHigh-level activities Medium-level activitiesUnpack STB cut through outer packaging, open box, pick

up contents, put down contents, find and recognise instructions

Follow instructions open and hold instructions, read instructions, understand instructions

Plan cable connections (what goes where) match box contents to instructions, identify AV components (e.g. VCR, TV, amplifier, speakers) and cables, plan signal flow, identify cables to be moved

Move components to get access to cable slots grip components, pull forward

Disconnect cables to be moved identify cables, grip and pull them

Place STB near AV components pick up and carry STB, put it down

Connect cables in correct order identify cables, identify cable sockets, pick up cables, connect them

Replace AV components to usual position grip components, push back


Exclusion analysis - Results

25

20

15

10

5

0Analogue DTV Analogue DTV

16+ 75+

Pop

ulat

ion

(,000

s)


Exclusion analysis - Results

25

20

15

10

5

0

25

20

15

10

5

0Analogue DTV Analogue DTV

16+ 75+

Pop

ulat

ion

(,000

s)P

opulation(%

age)


User observations - Overview

13 users - 12 aged 60+, 1 aged 24• 9 no DTV experience• 2 owned STBs• 1 owned satellite box• 1 owned iDTV• 7 PC users, 6 non-users

All ‘independent’ living


User observations - Methodology

2 hour sessions comprising: 30 minutes briefing 60-75 minutes with equipment• 15-20 minutes analogue• 40-60 minutes DTV• 2 STBs

15 minutes debriefing


User observations - Set-up


User observations - Example visual problems

Finding buttons on r/c • Especially POWER

Switching between r/c and screen• Different pairs of glasses

Reading on-screen font • No zoom facility

Reading instruction manual• Small print

New difficulty


User observations - Example motor problems

Pressing buttons on r/c

• Size and shape

Time-outs

• e.g. on EPG (040 -> 004)

Arrow button overshoot

• Oscillating cursor

New difficulty

New difficulty


STB2 remote control


STB1 remote control


User observations - Example ‘cognitive’ problems

Use of OK/SELECT Inconsistent language (OK=SELECT?) Which r/c to use / which mode am I in? How to start/navigate BBCi/Teletext? How to call up/navigate on-screen menus? How to operate/navigate the EPG? Inconsistent layout • e.g. LHS on screen, RHS on r/c


User observations - Summary of results


Summary

Cognitive/experience issues most important

Many of the problems easily avoidable


Origins of the problems for older users

New language / terminology • Jargon

New input paradigms• Part TV, part PC

New interaction concept• Interacting with STB, not TV

Inadequate explanationClassic case of

“designers designing for themselves”


Implications of prevalence of cognitive difficulties

What does this mean for assessment methods?• Single assessment methods vs. multiple?• In what order should they be used?

What does this mean for designers?• How to design for different experience?


What is “reasonable accommodation”?


Defining “reasonable accommodation”

Must offer “reasonable accommodation”• BUT what is reasonable?

Not defined explicitly• Companies left guessing

Will be defined in courts• Major risk/headache for companies


Attitudes to “reasonable accommodation”

EQUITABLE ACCESS

MINIMUM(compliance)

Access to functionality

IDEALAccess to

functionalityin same time

EQUITABLE ACCESS

MINIMUM(compliance)

Access to functionality

IDEALAccess to

functionalityin same timeIDEOLOGICAL

DIVIDE

Prag

mat

ists

Idea

lists


Interesting questions for companies

Is the equitable access ideal possible?• Is the equitable access minimum possible?• “ Equal, but different ” problem

Users with functional impairments => longer times

Can technology always make up the difference in user capabilities?

3 case studies…


Case study 1: The personal information point


The information point accessibility assessment

Sensory assessment: Screen too high and not adjustable Audio output not duplicated Visual output not duplicated

Motor assessment: Need to stand Reaching and dexterity demands

53% of target users excluded

Is this “reasonable”?


Case study 2 – Cursor assistance for motor-impaired users

Symptoms that can affect cursor control:

Tremor Spasm Restricted motion Reduced strength Poor co-ordination


User group behaviours

0

0.5

1

1.5

2

2.5

3

3.5

Group OA Group P Group Y Group A

0

20

40

60

80

100

120


Target activation times

Peak velocities

No. of incorrect clicks

0

1

2

3

4

5



Summarising the differences

Younger adults (IBM interns)• Shortest (1), fastest (1), more errors (3) - slapdash

• “I can fix it”• Games culture?

Adults (IBM regulars)• Shorter (2), faster (2), fewest errors (1)

• Best compromise between speed and accuracy? Parkinson’s users• Longer (3), slowest (4), fewer errors (2)

• Slow, but sure Older adults• Longest (4), slower (3), most errors (4)

• Vision difficulties?• Lack of experience


A method of cursor assistance

Haptic gravity wells:

Target

Gravity well

Attractive force


Experimental set-up


The effect of gravity wells

Target


Motor impairment in practice…


Results - Throughput

0

2

4

6

8

10

MI AB


Case study 2 summary

Haptic gravity wells are clearly very helpful MI users “with” on similar level to AB users “without”

BUT: AB users also improve “with” Is this “equal” time? Is this “reasonable”???


Case study 3 – Paperless office

AN Other wants to move to a paperless office• Currently receives 3.5 million pages per day

Paper documents are stored as TIFFs

Section 508 accessibility requirements• Sight-impaired• Low vision

Current solution – employ readers• “ Equal, but different. ”• Is this reasonable?


The study documents

Almost fully unconstrained Content:• Unconstrained vocabulary

Text:• Typed• Handwritten• Annotated• Stamps

Graphical content:• Diagrams• Charts • Graphs


Examples of the study documents


Examples of the study documents (cont.)


Readability metrics (text)

Translation rates:• Character-by-character• Word-by-word

Number and %ages of errors:• Level 1 - Minor• Level 2 - Moderate• Level 3 - Serious


TIFF file

OCR – The scanning process..................................11111.........11...11.......11.....11......11.....11.............11..........11111........111..11.......11....11......11.....11......11.....11......11....111......111..1.11.1.....1111..111..................

111

1

1

1

11 11

11

1

11

1111

11

1

111 1111

11 1

11

1

1 111

11

111 1

11 1

1

111111 1

1 1

1

1


OCR – Possible sources of scanning errors

Data LOSS NOISE


Comparing three OCR engines

“…also develop the skills to invert containers to get objects *inside. He should begin to find small details in a favorite picture book (a bird in a *tree, a small fish in the *ocean). His understanding of familiar objects should…”

FineReader:

“…*also *develop *the *skills *to *irxvert *containers *to *get *ob^ects *inside. *?e *should *begin *to *Znd *small *details *i? *a favorite *picture *baa? *?a *bird *in *a *true, *a *small *ash *in *the *ocean}. *his *understanding *of *familiar *ob^ects *should…”

OmniPage:

“…*also *de???op *the *s?il?s *ta *ivart?an#ainer?to *e?ob??cts?n?id?. *?e *shau?ti *b?ta *Znd *srnali *details *i?a *favarita *picture *baa??bi?rd *in *a *tra?,a *srr?a????in *tk?e *o?ean}. *?is *und?rt?a?af *fa.?i?iar *ob?ects *hau?d *co??i?u?ta *de?eiap *d?i?houi d…”

Recognita:


OCR results – Calculating the error rates

Record the document properties• # of words, characters• Font types (e.g. typed, handwritten) and sizes

Count instances of error types• Redaction errors• Spaces +ed, -ed• Format errors (e.g. wrong case, incorrect text positioning)• Extraction errors (i.e. incorrect translation)

• By character• By word

Classify severity• Level 1 – minor• Level 2 – moderate• Level 3 – severe

Calculate %age error rates

Note: classification for sighted users


OCR results – Overall word error % rates

Typed page 6.50 %(1 word in 15)

Typed page with notes

8.12 %(1 word in 12)

Faxes 14.45 %(1 word in 7)

Pictures and graphs

23.45 %(1 word in 4)

Handwritten reports

36.35 %(1 word in 3)

EKGs 49.72 %(1 word in 2)

A “typical sentence” contains 7 words.

An extraction error rate of 6.5% equates to 1 word error every 2 sentences.


Conclusions of OCR investigation

“ Current OCR technology is not capable of providing an acceptable level of text extraction from medical evidence as it is now received. ”

“ Technology cannot provide equitable access in this case. Alternative methods are required. ”

“ Equal, but different. ”


Overall summary

Some products clearly not “reasonable”• Case study 1

Technology cannot always make up for lack of user capability• Case study 2• Even when it does – the goalposts move!!!


Conclusion

What is needed is a framework for evaluating “reasonableness”

Based on quantifiable metrics

Reliable, repeatable, consistent, robust


Designing for cognitive support


Background

International Symposium• Held in October 2005• Venue: IBM TJ Watson Research Center• Sponsored by: IBM HA & AC• 10 invited experts

Data presented here from that Symposium


Defining C&LDs

US medical community definition• “A cognitive disability arises from an impairment that affects IQ”• i.e. effects capability to acquire, process and utilise knowledge• Arises from how medical conditions are diagnosed and treated

Educators focus on learning difficulties• Often focused on educational attainment

Do behavioural difficulties count?• Some say yes, some say no

No single, clear definition. Mostly focused on what users cannot do


Defining C&LDs for UI design

Alternative definition needed for UI design• Need to understand what users are capable of• Not what they cannot do

Need functional descriptions of capabilities Descriptions tailored to design project


Prevalence of C&LDs - US statistics

When asked • “Have you had a physical, mental or emotional condition lasting 6 months or

more that made it difficult to learn, remember or concentrate” 5.2% of all US adults agreed• (Source: 1999-2004 American Community Survey)


Prevalence of C&LDs - GB statistics

5.6% of GB adults self-report some form of cognitive impairment• (Source: 1996 Disability Follow-Up Survey)

“Cognitive impairment” comprised of:• Communication

• E.g. “Has some difficulty understanding what other people say or what they mean”• Intellectual Functioning

• E.g. “Often forgets what was supposed to be doing in the middle of something”

Figures from both surveys are most likely conservative


Role of cognition in interaction

Model Human Processor (Card, Moran and Newell)

F(t) = xp + yc + zm

where

• p = perceptual time unit

• c = cognitive time unit

• m = motor function time unit• x, y, z are integers


Role of cognition (cont.) - Simplex II

“Cognition” comprises:• Working memory• Emotions and drives• Perception• Output• Feedback• Complex output sequences• Cognitive models• Long-term memory• Executive functions(Adams)


Why C&LDs are not commonly addressed in UA…

They are often difficult to diagnose• Perception issues are easy to diagnose, cognitive ones are not

Many do not have a “universal” diagnosis• E.g. what exactly is dyslexia?

Many people with them are reluctant to admit this• The “invisible” impairment - stigma

Designing for them is not easy• No clear description of C&LD makes design more difficult

Can be assisted by “design best practices”


1 - Involve users in design process

“Standard” UA advice Can act as design partners• Working with designers

User study participants• Issues regarding ethical approval

The more designers know about their users, the better their designs become


2 - Find required set of demands to complete the task

Every task has associated capability demands• Writing a letter, using an ATM, etc.

Every IT system has associated capability demands• Navigation, interaction, etc.

Designers should ensure that IT system demands are no more than the task demands


3 - Use scaffolding techniques

New concepts need to be supported in multiple ways

New

concept

Existing

concept

New

concept

Existing

concept

Existing

concept

Existing

concept

The more references to existing knowledge, the more stable the new knowledge


4 - Use positive reinforcement

Many people with C&LD have low self-esteem• Perceived stigma• Often branded as “failures” or “thick” as children• Are used to being “judged”

IT is attractive because it is not so quick to “label”• Designers need to build on this perception• Avoid negative comments• Provide positive support and reinforcement


5 - Design for user’s learning strengths

Not everyone has uniform C&LDs Need to design to user strengths, e.g. …

Older adults • often have poor “dynamic” memory• often have good “crystalline” memory

People with dyslexia• written instructions hard to follow• visual / auditory instructions easy to follow

Relate new concepts to familiar ones

Use multiple modalities


6 - Design for flexibility

E.g.: Offer sliding scales of complexity

Save Print

File Edit Format

File Edit View Insert Format Tools

Basic word processor

Mid-level word processor

High-level word processor

Allow users to proceed at their own pace


6 - Design for flexibility

E.g.: Offer sliding scales of complexity


7 - Chunk information

Difficult:• “The 2007 Jeep Grand Cherokee Overland has a 5.7L Hemi engine that

develops 330 BHP. It’s a willing a responsive engine. Prices for this model start at around $37,000 and go up to $44,000 for a fully specced up version.”

Better:• Model: Jeep Grand Cherokee Overland• Engine: 5.7L Hemi• Price: $37,000 - $44,000

Small bits of knowledge at a time


8 - Keep hierarchy depths to a minimum

As a general rule, depths should be <3

is better than:

Breadth, not depth

Potential conflict with motor/sensory issues


9 - Recognise the use of coping strategies

Try to complement, support or augment these


Other best practices - I

Use clear language• Avoid jargon

Provide appropriate “help” functionality• Concrete, repeatable, focused and consistent

Present assistance options carefully• Make sure to avoid “stigma”


Other best practices - II

Avoid “feature creep”• Difficult for older users who rely on crystalline memory

Try to be consistent• Consistent methods of exiting, for example


Summary

Cognitive and learning difficulties are the “invisible” impairment They are widespread (c. 5% of the population) They affect how well users can interact with UIs But receive comparatively little research and design attention

Need more attention!


For further details

ACM SIGACCESS Newsletter:• September 2005 Number 83• http://www.acm.org/sigaccess/newsletter/sept05.php

International Journal on UA in the Information Society• Volume 5, Number 4, April, 2007


Exercise


Exercise – part 1

Finish making changes to your site

usability with project lecture 10 – 19/03/10

Documents