A Mobile Interactive Maps Application for a Visually Impaired Audience

Presenter: Gottfried Zimmermann

Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

Authors: Nikolaos Kaklanis, Konstantinos Votis and Dimitrios Tzovaras

The problem By accessing web mapping applications like the

OpenStreetMapOpenStreetMap through a mobile device, a sighted user can virtually explore a map area

before actually go to the specific location

using GPS & a suitable application, user can explore the map area

around his/her current location real-time

However, users with severe visual impairments often meet great obstacles in accessing such information, as the interaction with this kind of applications is performed mainly through the visual channel.

OR

State of the art PocketNavigator (Pielot et al., 2010)

Android application uses vibration patterns to provide information regarding the position of user’s final destination

ahead/left/right/behind – relative to user’s current position not fully accessible for visually impaired users

Haptic GeoWand (Jacob et al., 2012) Android application location, orientation & distance information using vibration patterns not fully accessible for visually impaired users

TouchOver Map (Poppinga et al., 2011) Android application vibration and speech feedback focused on the visually impaired but not fully accessible

Ariadne GPS (Ciaffoni, 2012) iOS application text-to-speech, vibration, sound accessible for visually impaired users

What is missing?

An Android application fully accessible for the visually impaired users

The proposed solution – “Open Touch/Sound Maps” Enables access to OpenStreetMap data for the visually impaired and blind

users using a common mobile device (e.g. smart phone, tablet) that runs on Android.

Multilingual Currently: English & Greek

Modalities Vibration feedback

when finger is on a road/crossroad/point-of-interest (POI) Sonification

The pitch of a single-frequency sound is changing according to the distance to the next crossroad (as distance becomes smaller, sound frequency becomes higher)

Helps user realize the distance to the next crossroad Music chord – when user’s finger is on a crossroad

Text-to-speech Name of current road/crossroad/POI

Visual feedback (for the partially-sighted users)

“Open Touch/Sound Maps” - Architecture

Android device

OpenStreetMapweb service

Open Touch/Sound Maps

OSM retrieval module

OSM retrieval module

coordinates

coordinatesOSM

OSM

coordinates

distance to the next crossroad

current road/POI name

vibration feedback

visual feedback

audio cues

speech cues

Multimodal map

Multimodal map

search criterion

Google Mapsweb service

GPS

coordinates

search criterion

TTS moduleTTS module

Visual rendering moduleVisual rendering module

Sonification moduleSonification modulePosition identification

module

Position identification

module

Vibration moduleVibration module

OSM

“Open Touch/Sound Maps” –Interface of the application

Menu navigation Typing a search criterion Map exploration

“Open Touch/Sound Maps” –Map exploration using a tablet

“Open Touch/Sound Maps” – Innovation/Advantages Alternative modalities in accessing map

information vibration, sonification, text-to-speech

Enables the exploration of any map area around the world not predefined map areas

No special equipment is needed only a common mobile device running on Android

Intuitive map exploration user simply moves his/her finger over the touchscreen

Future work

Access to data coming from more web mapping frameworks e.g., Google Maps, etc.

Vibration patterns providing user with different positional cues e.g., vibrating 2 times per half a second may

probably mean that user gets closer to a POI, etc.

Thank you!

Questions

Video presentation: http://www.youtube.com/watch?v=wHBLKTWTvmg