“EyeMouse”: An interaction device for severely motor-

disabled people

Carlo Colombo, Massimiliano Corsini

Media Integration and Communication CenterUniversity of Florence

• Human-machine interaction system replacing the mouse with eye movements

EyeMouse

• Designed to exploit the residual mobility of severely motor-disabled people (e.g. multiple sclerosis)

Interaction

User

CVInterpreter

Screen

Live camera images

Interface commands

feedback

action

• User eye movements are captured through computer vision…

• …and then transformed into commands for the on-screen PC interface

• External eye and iris are captured by elastic template matching (snakes)

Eye capture

Snake template

Iris (circle)external eye

(ellipse)

Eye remapping

Image plane Screen plane

• The iris position in the image is remapped onto the screen plane

• The image-to-screen map is calibrated at startup

• Navigation: eye movements

• Selection: eye persistence, eye blinks

Mouse functionalities

• Interface feedback compensates for slight remapping errors

“X” movement

“O” movement

Current work

• Accommodation of head motion

• Automatic recovery of tracking errors for prolongated interaction

Contacts

Prof. Carlo Colombo

Dipartimento Sistemi e Informatica Via S. Marta 3 – Firenze

Centro di Eccellenza MIUR per la Comunicazione e Integrazione dei Media

Sede RAI, Largo A. de Gasperi 1 - Firenze

www.dsi.unifi.it/users/colombocolombo@dsi.unifi.it