presenter: anuj tewari course: cs260 instructor: john canny based on work by indrani medhi

Presenter: Anuj TewariCourse: CS260

Instructor: John CannyBased on work by Indrani Medhi

Outline: Text-Free User Interfaces

Introduction

Initial ethnography and iterative rapid prototyping

Initial evaluation

Optimal audio-visual representation

Methodology

Discussion

Illiteracy

1-2 billion illiterate population in the world.

98% live in developing countries.

India’s rate of literacy estimated at ~60%.

Many labeled “literate” are semi-literate with low functional literacy.

Photo: Indrani Medhi

Two residents of a slum in Jayanagar, Bangalore

Target Users

Women from Bangalore slums

Informal sector domestic workers

Income range: INR 800-2500 (US$20-60) per month

Illiterate or semi-literate

Most have never seen a PC (those who have seen have never touched)


initial explorations before text-free UI work

NGO Partner

Stree Jagruti Samiti

Head: Geeta Menon Small NGO Works with several

Bangalore slums Focus on children’s

rights, women’s empowerment, fair wages for domestic laborers

Geeta Menon leading a SaturdayStree Jagruti Samiti meeting


How can an application UI be converted into one that is usable by non-literate users?

The Problem


Introduction


Initial evaluation


Methodology

Discussion

Initial EthnographyTVs, radios, CD players common; some mobile phones; no PCs

Established slums – some with 30-year history in city; concrete housing

Subjects most concerned about…

Jobs Healthcare Education

Nimmi interviewing a subject


Text-Free UI?

Design principles:

Pen or touch interface

Liberal use of imagery○ No text○ Semi-abstracted

cartoons

Voice annotation

Aggressive use of mouse-over functionality

Consistent help iconNaukri.com for domestic labourers?

Nouns vs. VerbsGoal is to achieve exact association between image and word.

Without action cues drawings tend to be interpreted as nouns.

With action cues, drawings tend to be interpreted as verbs.

Phenomenon well-understood by cartoonists and diagram designers.

“Kitchen sink” or “washing dishes”?

Text-Free UI Details

“Pot” or “cooking”?

Cultural Differences

Goal is to achieve as global a representation as possible.

Abstracted representations are very dependent on culture.

Minor misrepresentations can result in major confusion.

An urban family of three?


Religious Differences

Goal is still to achieve as global a representation as possible.

Religious differences can be the strongest of cultural differences.

There is no “neutral” culture or religion.

Increasingly general representations for time indication


From when to when?

Quirks of Non-Literacy

Various degrees of literacy

Total illiteracy Numbers, but not alphabets Alphabets, but not words Words, but not sentences Semi-literate, but not fluent

Many who are non-literate can read numbers

Accustomed to currency Able to do basic arithmetic Not necessarily 100%

accurate

Illiteracy doesn’t necessarily mean inability to read numbers.


Text-Free UI?

Original design Revised design

Text-Free UI

Design principles:

– Pen or touch interface

– Liberal use of imagery• No text (but numbers OK)• Semi-abstracted cartoons

– Aggressive use of mouse-over functionality

– Voice feedback

– Consistent help icon

Screenshot of text-free job search

Other Domains

Principles transfer to other applications:

Text-free healthcare informationText-free maps


Introduction


Initial evaluation


Methodology

Discussion

Questions

1. Can non-literate users use traditional text-based user interfaces at all?

2. Do the proposed design principles for text-free user interfaces allow non-literate users to use computers, and to what extent?

3. Which of the design principles make the most difference for a text-free UI?

Initial Evaluation

Experimental Set-UpThree versions:

Text-based Text-free without help

icon Text-free with help

icon

Subjects: Illiterate or semi-

literate○ (no literate subjects)

No PC experience

Subject grouping: Five-women group x 2 Individual x 4

Total time up to one hour per session

Task: For a friend who is unemployed, find the best-paying job in her neighborhood.

Measured: Task completion

(yes/no) Time required (second) Number of prompts

required (n)

All comments recorded; some trials video-recorded.

Initial Evaluation

Quantitative Results

Text-based UIs were completely unusable.

Obvious, but first formal such test in literature

With text-free UI, 30% were able to complete task.

(However, not enough participants for statistical significance.)

Tabulation of initial results

Initial Evaluation

Qualitative ResultsStrong preference for text-free principles, in particular…

Voice annotation Help icon

Collaborative use more successful

Subjects eager to engage, once comfortable with set up

Various barriers to technology: fear, lack of awareness, lack of cognitive model

Initial Evaluation

Indrani and a subject during initial evaluation

Questions

1. Can non-literate users use traditional text-based user interfaces at all?

2. Do the proposed design principles for text-free user interfaces allow non-literate users to use computers, and to what extent?

3. Which of the design principles make the most difference for a text-free UI?

Initial Evaluation

NO

Yes, but onlypartially; morework required

Hypothesis:- Voice feedback- Imagery - Help


Introduction


Initial evaluation


Methodology

Discussion

Text-Free UI

Design principles:

– Pen or touch interface

– Liberal use of imagery• No text (but numbers OK)• Semi-abstracted cartoons

– Voice feedback

– Aggressive use of mouse-over functionality

– Consistent help icon

From initial round of research…

Screenshot of text-free job search

Goal of StudyDetermine what types of icon

representations are best for UIs for non-literate users.

Audio: ○ With voice annotation○ Without voice annotation

Visual:○ Photorealistic video○ Hand-drawn animation○ Photograph○ Hand-drawn image○ Text

Domain of Testing

Desired characteristics of domain:

Admits to visual representations, Consists of information which subjects are not exposed to on a

daily basis, Has universally recognized meaning, Offers a range of cognitive complexity, Offers a range of visual complexity, and Can be applied to a practical purpose.

Healthcare / medical imagery 13 medical symptoms: multimodal in nature, involving elements of

color, temperature, and some requiring motion spanning

DrawingPhotograph AnimationVideo

Ensure consistency across representations…

Creation of Representations

Tested Representations

DrawingPhotograph AnimationVideo Text

Each with and without voice annotation (total 10 representations),

for 13 medical symptoms

Experimental Set-UpRepresentations:

One of 10 types Evenly distributed

among subjects All 13 symptoms per

subject

Subjects: Illiterate or semi-literate

○ (no literate subjects) No PC experience

Size: Individual; n = 200 20 for each

representation

Total time up to one hour per session

Threefold task: Speak phrase that first

comes to mind when shown a representation.

Explain symptom representation is intended to convey

Enact or depict the intended symptom (as explained by tester), for analysis

Measured: Accuracy (correct,

close, wrong) Response time

(seconds)

All comments recorded

Voice annotation critical, with dominant effect

• 30% quicker response time with audio

Among cases with audio, hand-drawn representations (both animation and static imagery) most accurately recognized

• But, effect not dramatic

Among cases without audio, animation most accurately identified

• 23% more accurate than static drawing

ACCURATE RESPONSES (out of 20)

5.955.45

7.35

5.9

2.65

9.759.3 9.65 9.3 9.45

0

2

4

6

8

10

12

STATIC PHOTO ANIMATION VIDEO TEXT

AVERAGE RESPONSE Without AUDIO AVERAGE RESPONSE With AUDIO

Quantitative Results

Number of accurate responses (out of 20)for various representations

Qualitative Results

Test / text anxiety of subjects Subjects uncomfortable with test-like environment of studies

Short-term category conditioning Priming for “aches” over other types of symptoms when they

appeared in sequence

Richer information is not necessarily better understood overall Photorealism contains clutter; associated with actual instance

Unrelated Context-laying activity changes value of static versus dynamic imagery

Socio-economic conditions correlated with cognition Those with less formal education had difficulty processing

visuals and audio at the same time; confused by multi-modal interface


Introduction


Initial evaluation


Methodology

Discussion

Initial Ethnography

Open-ended interviews Various combinations: focus

groups and individuals Topic: everything and anything Follow leads around hardships

Structured interviews Focus on themes from open-

ended interviews Prepare questions Be willing to deviate

Goal: Information, intuition, rapport

An open-ended interview in a subject’s home

Methodology

Time and attention critical!


Iterative PrototypingBuild initial prototype, based on intuition from interviews.

Iterate prototype and test. Maintain informality Work with small groups

○ 2-4 people

Prompt whenever necessary Record every comment

○ Video-record, if possible (but, high procedural overhead)

Listen to every comment Modify prototype

Refine until steady state.Testing a prototype

Methodology


Participative Design

Modify standard methods for non-literate partners.

Encourage storytelling.

Use aids that can be physically manipulated.

Be prepared for long, counseling-like sessions.

Methodology

Eliciting descriptions of medical symptomswith physical design aids


Subject enacting medical symptoms

Formal EvaluationBase on techniques from courses, textbooks

Plan according to standard practices

Avoid direct counterfactuals Place task in a story Use friend or neighbor as

protagonist, but not subject Chavan’s “Bollywood method”

Be flexible on… Location (avoid labs) Formal instructions Etc.

Methodology

Video: Archana Prasad

Text-Free UI History

Initial principles

Optimal AVrepresenta-

tion

Full-contextvideo

Text-Free UIfor mobiles

Psychology of non-literate

users

Text-Freehealthcareinformation

Online site(babajob.com)

Practical Systems

UIResearch

OtherResearch

Completed Ongoing FutureKey:

UIauthoring

tool

Semi-automatedcartooning

Discussion questions

Is it possible to replicate the same study on a mobile? How would the process/guidelines for design be different?

What might be some other areas where text-free UI could have potential? How?

What are some of the best tricks to eliminate designer biases from the design process?

presenter: anuj tewari course: cs260 instructor: john canny based on work by indrani medhi

Documents

textfree ui detailspot

indrani medhitextfree

application ui

indrani medhioutline

indrani medhihow

nonliterate users

labeled literate

indrani medhitwo residents