Web Browser Accessibilityusing Open-Source Software
http://amico.sourceforge.net
Željko Obrenović Jacco van Ossenbruggen
Semantic Media InterfacesCWI, Amsterdam
[email protected]://www.cwi.nl/~obrenovi/
2 / 11
Introduction
• Accessible Web Browser requiresnovel interaction modalities
– Speech input and output for blind users– Gestures, head movement for spinal cord injured– Multilingual translation…– Combination of modalities
• Web is more and more interactive– Web 2.0 – promises much more interactivity… – But it is hard to go beyond modalities supported
by the browser (screen, keyboard, mouse)
3 / 11
Some Existing Solutions
• Browser Extensions– Easy to build by most of developers– Problem of integration of complex code
• Specially Designed Browsers– Less functional, not very popular– Label users as "disabled“
• Adapted Open Source Browsers– Complexity of development and maintenance of code
• Firefox 1.5.0.2: 10,970 files, 2,172,520 lines of code
– Limited by technologies used by browser
4 / 11
Potential of Open Sourceand Free Software• Available OSS components, however,
may support novel interaction modalities– Computer vision: OpenCV, HandVu…
– Speech: FreeTTS, MaryTTS, Sphinx-4…
– Language tools: WordNet, ConceptNet…
– Translation services…
• Can we better exploit OSS solutions,and reuse them with Web (2.0) technologies?
• Example: Speech control of Google Maps:
5 / 11
All you do is put it together?
– Many prog. languages– Many APIs– Semantic and temporal differences
• High-level Web XML-based APIs• Low-level APIs of the device world
– Fault tolerance– Flexible integration
• Why it is so hard to use OSS?
6 / 11
Our Solution: AMICO:WEB http://amico.sourceforge.net/• Service-based loosely-coupled integration
– Encapsulation OSS componentsas local or remote services
– Solves the language issue,many OSS already provide service interfaces
• Run OSS service as a process, functionality through open API
• Two parts:+ AMICO: Adaptable Multi-Interface COmmunicator+ Web Integration Interfaces
Web Browser
WebIntegrationInterfaces
AMICOOSS
Components
7 / 11
AMICOAdaptable Multi-Interface COmmunicator
• Publish-subscribe communication service:–share data & exchange messages–run components as services
• More APIs (TCP, UDP, XML-RPC, OSC, SOAP, SQL…)• Fault tolerance by loosely coupled integration• Temporal and data bridging
–Low-level and high-level variables and transformations
• Declarative XML abstraction, rapid prototyping–Flexible integration and reuse of components
Web Browser
WebIntegrationInterfaces
AMICOOSS
Components
8 / 11
AMICO:WEB –Browser Integration• Browser extensions (Firefox)
– Based on SIMILE Java Firefox extension– Use AMICO TCP and UDP interfaces
• AJAX– Use AMICO HTTP Interaface
(XMLHttpRequest)
• Applets and scripts– TCP connection
Also in combination
with extension such as Mozilla GreaseMonkey
}
9 / 11
AMICO Example 1 / 2
• Translation service– BabelFish service Text-to-speech
– Problems of combining (legacy) Web services,local components and browser components:
• BabelFish Web Service• 3 TTS engines: FreeTTS (English), Festival (Dutch),
Mary (German, Tibetan)• Local database (MySQL, MS Access)• Browser toolbar as an interface
10 / 11
AMICO Example 2 / 2
• Camera-based modalities– interaction with 3D (VRML) Web content
• Problem of temporal mapping– Fast low-level face detectors (simple numbers, 0.1s)– High-level and slower Javascript VRML browser API
11 / 11
"Under the hood"
WordNetdatabase
AMICOCore
TC
P
UDP
WordNet
HTTP
App
licat
ion
spec
ific
XML-RPCSQ
L
Firefox toolbar extension
Userdictionary
English speech
recognizer
English TTSengine
Dutch TTSengine
German TTSengine
ConceptNetYahoo BabelFishtranslation service
Face/gesture/motion
detection
AJAX XMLHttpRequest
Python
Java
Java
C++
C Java
MS AccessMySQL
MonetDB
Web (HTML)Service
JavascriptJava
Javascript
12 / 11
Conclusion• AMICO:WEB – A Platform for Rapid prototyping of
accessibility solutions in mainstream Web browsers– Exploiting potential of OSS and free software
– Platform for future work and Web accessibility research
– Relation with W3C Rich Web Application Backplane
• Applications– Improve accessibility of Web multimedia and e-learning systems
– Education: building accessibility solutions by undergraduate students
• Future work:– Solving open issues: security, installation…
– Internet Explorer plugins, Google Web Toolkit…
– Proxy and Server Integration Mechanisms