Web Browser Accessibilityusing Open-Source Software

http://amico.sourceforge.net

Željko Obrenović Jacco van Ossenbruggen

Semantic Media InterfacesCWI, Amsterdam

zeljko.obrenovic@cwi.nlhttp://www.cwi.nl/~obrenovi/

2 / 11

Introduction

• Accessible Web Browser requiresnovel interaction modalities

– Speech input and output for blind users– Gestures, head movement for spinal cord injured– Multilingual translation…– Combination of modalities

• Web is more and more interactive– Web 2.0 – promises much more interactivity… – But it is hard to go beyond modalities supported

by the browser (screen, keyboard, mouse)

3 / 11

Some Existing Solutions

• Browser Extensions– Easy to build by most of developers– Problem of integration of complex code

• Specially Designed Browsers– Less functional, not very popular– Label users as "disabled“

• Adapted Open Source Browsers– Complexity of development and maintenance of code

• Firefox 1.5.0.2: 10,970 files, 2,172,520 lines of code

– Limited by technologies used by browser

4 / 11

Potential of Open Sourceand Free Software• Available OSS components, however,

may support novel interaction modalities– Computer vision: OpenCV, HandVu…

– Speech: FreeTTS, MaryTTS, Sphinx-4…

– Language tools: WordNet, ConceptNet…

– Translation services…

• Can we better exploit OSS solutions,and reuse them with Web (2.0) technologies?

• Example: Speech control of Google Maps:

5 / 11

All you do is put it together?

– Many prog. languages– Many APIs– Semantic and temporal differences

• High-level Web XML-based APIs• Low-level APIs of the device world

– Fault tolerance– Flexible integration

• Why it is so hard to use OSS?

6 / 11

Our Solution: AMICO:WEB http://amico.sourceforge.net/• Service-based loosely-coupled integration

– Encapsulation OSS componentsas local or remote services

– Solves the language issue,many OSS already provide service interfaces

• Run OSS service as a process, functionality through open API

• Two parts:+ AMICO: Adaptable Multi-Interface COmmunicator+ Web Integration Interfaces

Web Browser

WebIntegrationInterfaces

AMICOOSS

Components

7 / 11

AMICOAdaptable Multi-Interface COmmunicator

• Publish-subscribe communication service:–share data & exchange messages–run components as services

• More APIs (TCP, UDP, XML-RPC, OSC, SOAP, SQL…)• Fault tolerance by loosely coupled integration• Temporal and data bridging

–Low-level and high-level variables and transformations

• Declarative XML abstraction, rapid prototyping–Flexible integration and reuse of components

Web Browser

WebIntegrationInterfaces

AMICOOSS

Components

8 / 11

AMICO:WEB –Browser Integration• Browser extensions (Firefox)

– Based on SIMILE Java Firefox extension– Use AMICO TCP and UDP interfaces

• AJAX– Use AMICO HTTP Interaface

(XMLHttpRequest)

• Applets and scripts– TCP connection

Also in combination

with extension such as Mozilla GreaseMonkey

}

9 / 11

AMICO Example 1 / 2

• Translation service– BabelFish service Text-to-speech

– Problems of combining (legacy) Web services,local components and browser components:

• BabelFish Web Service• 3 TTS engines: FreeTTS (English), Festival (Dutch),

Mary (German, Tibetan)• Local database (MySQL, MS Access)• Browser toolbar as an interface

10 / 11

AMICO Example 2 / 2

• Camera-based modalities– interaction with 3D (VRML) Web content

• Problem of temporal mapping– Fast low-level face detectors (simple numbers, 0.1s)– High-level and slower Javascript VRML browser API

11 / 11

"Under the hood"

WordNetdatabase

AMICOCore

TC

P

UDP

WordNet

HTTP

App

licat

ion

spec

ific

XML-RPCSQ

L

Firefox toolbar extension

Userdictionary

English speech

recognizer

English TTSengine

Dutch TTSengine

German TTSengine

ConceptNetYahoo BabelFishtranslation service

Face/gesture/motion

detection

AJAX XMLHttpRequest

Python

Java

Java

C++

C Java

MS AccessMySQL

MonetDB

Web (HTML)Service

JavascriptJava

Javascript

12 / 11

Conclusion• AMICO:WEB – A Platform for Rapid prototyping of

accessibility solutions in mainstream Web browsers– Exploiting potential of OSS and free software

– Platform for future work and Web accessibility research

– Relation with W3C Rich Web Application Backplane

• Applications– Improve accessibility of Web multimedia and e-learning systems

– Education: building accessibility solutions by undergraduate students

• Future work:– Solving open issues: security, installation…

– Internet Explorer plugins, Google Web Toolkit…

– Proxy and Server Integration Mechanisms