slides by yevgen borodin (slides adapted for psych 384, 3/3/09) department of computer science,...
TRANSCRIPT
Slides byYevgen Borodin
(slides adapted for Psych 384, 3/3/09)
Department of Computer Science, Stony Brook University
A Vision for a Universally Accessible Web
The web is designed for those who can filter out irrelevant information
Non-Visual Web BrowsingJaws, Windows Eyes, Hal
Serial audio interface
Shortcut-driven navigation in HTML DOM-tree
Navigation between links, headers, lines, etc.
Inaccessible images, links, multimedia, etc.
How blind people browse the webLandmarksSearchesList of linksHeadingsStructureSpeed of speechStatic sites [Bigham, et. al., ASSETS’07]
What is HearSay?Started:
Motivated by work in AIInformation extraction from web pagesProcess modelingAdded an audio interface
Now :Working with HKSB and HKNCCollaborate with IBM, UWSeveral faculty members, Ph.D., MS.,
undergrad
HearSay 3 Free!!!Multi-platformFocused on Web browsingFlexible multimodal interfaceSupports text-to-speech enginesSupports voice recognition engines
Improving navigation would make web browsing more efficient.Segment pagesIdentify patternsAdd 2D navigationSummarize content
All this helps, but...
Project Goals
Filter out irrelevant informationDiscover relevant informationProvide quick access to relevant content
Evaluate the usability of HearSayCompare HearSay to other screen readersDistribute a stable version of the program for
free
ScenariosRelevancy in Ad-hoc Web Browsing
Relevancy when Web content changes
Relevancy in Online Transactions (e.g. shopping, paying bills)
But what is relevant?
Manual Annotations of Content (the user could tell us what is relevant)
Beginning of Main Content
Search Button
Needed for Manual Labeling:
Provide an interface for creating annotations
Store annotations in a database
Query the database when the page loads
Apply the metadata to the page
Provide an interface for reviewing the annotations
Collaborative authoring of accessibility metadata[Takagi et al, 2008]
Social network connecting end-users and volunteers [http://socialaccessibility.alphaworks.ibm.com/]
Accessibility Commons (AC) DB to store metadata[Kawanaka, Borodin et al, 2008]
Web-based infrastructure for sharing metadata
Benefits of Social Accessibility
Shortens the time for accessibility renovations
Supported: headings, ALT tags, and titles
Workshop at UW – formed a consortiumDefined the Accessibility Commons DB schemaIdentified the object addressing methods: XPath,
MD5, URI
Automatic labeling of content to support web transactions
Labeling contentNon-visual web transactions are difficult
Consider all problems with non-visual browsingNeed to locate relevant concepts (buttons and
links)
Relevant concepts are similar across websitesSome variations, e.g. “add to cart”, “add to bag”Different labels, e.g. “Search”, “Go”, “Find”Evolution of relevance and form over time
Or instead of annotating elements, the whole process could be automated. Ex: AT&T Log-in Page
AT&T: Account Overview Page
AT&T: Make a Payment Page
AT&T: Confirm Payment Details
Macro Recording InterfaceCreate a recording (non-)visuallySave recording with a descriptionVoice interface to replay the macro-recordingPage (in)dependenceCustomizing what is readSpecifying variables
Context-Directed Browsing
Something has Just Changed…?
Dynamic Web ContentDynamic content:
Our actions often cause changeWe pay attention to changes of contentNew information is often in the changesAffects relevancy of information
Types of updates:Page refresh, redirect, JavaScript and AJAX
updates
Source of updates:User-invoked and Timer-based
Another example:
Dynamic Content Paradigm
Treat any content changes as “updates”:AJAX, JavaScript, refresh, redirectNavigation by following linksUsing back and forward buttons
Analyze and diff the updated Web contentProvide interface for reviewing the changes
Page Refresh
Filtering Repeated Content
User-Centric GoalsDiscover and present relevant information
firstMinimize access-time to relevant informationKeep users focused on tasks and informationFacilitate multi-tasking and refocusingEnable automation of repetitive tasksKeep the context of user actionsMinimize system distractions
HearSay for the SightedBrowsing on handhelds
Browsing over the phone
Browsing on-the-go
Other services
Web Accessibility in Handhelds
Mobile Browsing ProblemsData Transfer Cost is High
Connection is Slow
Small Screens
Lots of Scrolling
Context-driven Browsing
External CollaboratorsAccessibility Group at IBM JapanAccessibility Group at Google
HKSB and HKNCArizona State University
Conferences: ASSETS, W4A, CSUN
ConclusionWeb Accessibility is an important problemGlimpse of Interesting ApproachesMuch remains to be done:
E.g. Integration, Robustness (Specification and Verification)
SonificationOther modalities – touch, pen, ..Extensive end user studies to probe mental models
to drive technology development (feedback)Other Disabilities – cognitive, motor impairment,
etc.36
Questions?Comments?
Concerns?Suggestions?
www.cs.sunysb.edu/~hearsay
mailto:[email protected]
www.cs.sunysb.edu/~borodin