em.doc
TRANSCRIPT
An Eye mouse Control System based on FPGA Technology
1Yih-Ran Sheu
1Juing-Shian Chiou
2Cheng-Hsin Chuang
1Nguyen Van Binh
1Wen-Li Shen
1Yao-Wei Tseng
1Department of Electrical Engineering, 2Department of Mechanical Engineering,
Southern Taiwan University of Science and Technology,
Tainan, TAIWAN
increasing the processing speed of the system. The eye
Abstract
The objective of this paper present a set of techniques integrated into an eye-mouse system using a field programmable gate array (FPGA). Our system requires a camera (TRDB-D5M), FPGA-DE2 board and a personal computer. A field programmable gate array (FPGA) is used to control the CCD camera and execute the image processing operations and control eye-mouse of computer via RS232 protocol. The coordinate of eye-mouse in our system is determined by the system based on FPGA imaging analysis. No computer is necessary in image processing. Good efficiency can be seen using FPGA hardware approach to determine the eye-mouse coordinate per image frame. The color information of eye is used to estimate the direction of pupil movement and then uses the direction information to control the mouse of computer.
Keywords: FPGA, eye mouse, direction of pupil
I. INTRODUCTION
Nowadays, eye tracking and eye mouse system more and more are widely. There are many eye tracking system using image processing by computer supported by many image processing tools such as Matlab, Visual C, Visual Basicetc, but these systems reduce the speed of the computer and sometimes cause unexpected situations. Thus, processing image to detect eye movements using FPGA hardware bring an alternative solution. In this paper, we practically design and develop the improved eye-mouse device with TRD-D5M camera, FPGA hardware, and computer. An eye tracking systems is a system that can track the movements of the user's eyes. Eye mouse system is a very good approach for people who do not have the ability to perform the movements of common control on computer such as mouse drag and mouse clicketc. In this paper, we perform image processing to detect pupil movement in FPGA hardware then sends control information to the computer to control a cursor on the screen. There are several different ways to track the directions of eye movements. Each approach has its own advantages, disadvantages, and limitations. We use information about eye color to detect the movement of the eye. There are many color spaces such as RGB, HSI, YUV, YIQ ... We choose the YUV color space because it has many advantages and simple. The image processing in FPGA is not as good as image processing in personal computer but this process is totally independent, thus
mouse system is used to access computers via eye movements.
II. SYSTEM DESIGN
The eye-mouse system consists of two modules. The first module including second -stage algorithm is proposed to implement image processing and pupil movement detection on FPGA hardware. An efficient pupil detection based on the pupil color information is employed to find the pupil and corner of users eye. After this, some algorithms is used to detect the direction information of pupil/corner of eye then transmit to computer via RS232 communication protocol. The second module is software on computer including cursor control block, user interface block, and application. The computed direction information of pupil movements is used to control the mouse of computer. Through the graphic user interface, users can implement some application.
The component of eye mouse system is introduced as follows.
Fig.1 The component of eye mouse system
II-1. HARWARE SPECCIFICATIONS The system consists of a personal computer, FPGA
hardware with TRD-D5M camera and RS232 cable,
FPGA hardware with a video capture, zoom buttons and adjust contrast. A real-time eye-tracking system requires an image capturing device operating at an extremely high frame rate to be able to effectively track the pupil in the presence of saccadic motion. Therefore, a high-frame rate image capture device is necessary in order to build a general eye tracker system for various human-computer interaction applications [1]. The TRDB_D5M Kit satisfies most of the above requirements to develop a 5 Mega Pixel Digital Camera on the Altera DE2-70 / DE2 boards.
Fig.2 The algorithm diagram for the proposed eye tracking system.
II-2 Image processing of an eye-mouse system The image input through the camera is transformed by FPGA hardware to obtain the image data of the plane coordinate. Every point (i,j) on the coordinate plane has its own image data. With this image information, the system is able to identify the pupil of eye from the image[5].Our methods based on color information about the pupil of eye to determine the pixel according to the definition given. The first problem is to represent images in the appropriate color space that allows to distinguish two components: composition luminance, luminous intensity and color components. RGB color space generally inappropriate because it mixes the two components. Although there are other color models such as CMY, YIQ and HSI, in this paper YUV color model is chose because of its advantages. In YUV color model, the luminance information is represented with Y component while the color information is represented by the U and V components. The main idea of using color-space information is to increase the information available about an image. The pupil feature detection system based on eigenfeature approach is used for gray-level Y and U, V color information of image. YUV color components can be obtained by using the RGB color coefficients as follows [6].
Y 0.299 0.587 0.114R
a. The definition of thresholding
255, f(i, j) T
g(i,j) = (2)
0, f(i, j) T
In accordance with the above definition, we can find a pupil value. A set of dark spots where the gray level is lower than the pupil value is exactly the pupil position we are searching for. If the binary threshold is not appropriate, the search result will show a dark spot which is unexpected [5].
a. Original image
b. Intensity image histogram
Fig.3 a. Original image b. Intensity image histogram
a. Y image
b. Gray image of pupil tracking process
Fig.4 a. Y image b. Y image of pupil tracking process
b. Calculation for the coordinate of the pupil center:
We use equation to calculate the coordinate of the pupil center searched by system.
U - 0.148 - 0.289
0.437G(1)
KK
1 1
V 0.615 - 0.515 0.100B
The firt thing to do is the thresholding.
XC,YC X,
K 1
K
Y (3)
1
Where K is the total number of the dark spot at which the gray level is lower than the threshold after dynamic binarizing of the eye image. X and Y are coordinates of the black pixel. [5]
Fig.5 The result of calculation for the coordinate of the pupil center.
II-2-2 Eye movement detection
After we have located the pupil center point from the eye we can proceed to extract the direction information from the pupil movements. The directions of the pupil movements are quantized into three directions: right, left, and center. The third different position of pupil center point corresponding to the third region on the picture with size 320x240 pixel is illustrated as follows.
Fig.6 The 3 different regions of pupil movement corresponding to the 3 directions of mouse.
In short, we can determine the directions of eye movements by pupil center position. Example: The pupils center position is at P(156,99), so pupil is center position. The pupils center position is at P(192,118), so pupil is right movement. The pupils center position is at P(75,111), so pupil is left movement. After the direction of the eye movement has been computed, we can use the direction information to move the cursor on the screen.
II-2-3 Communication between FPGA hardware and
computer
The direction information of pupil is transmitted to computer by RS232 or USB protocol communication. This paper we use RS232 protocol communication. Visual Basic software is used to receive the direction information of the eye movements from FPGA hardware. Although the baud rate of RS232 port is lower than USB, this speed is enough to satisfy our requirement.
III. EXPERIMENTAL RESULTS
III-1 Eye Tracking Experiment
In this paper, we only perform a simple mouse eye system. The obtained results are quite accurate because only three
regions of the window is active. For other complicated applications which require the pupil of the eye moving to more precise location of the window, only information detected from the color of the eye of people is not enough. In addition, changes in light conditions also reduce the accuracy of the system. Therefore, in these cases we need to combine information about color and other methods as detect edges, Hough Transform to upgrade the system. Some exact results are shown in the picture.
We can see that the pupil center point has Xc =156 so it is
center.
We can see that the pupil center point has XC=75 so it is
left movement.
We can see that the pupil center point has
ATTENTION!TRIAL LIMITATION - ONLY 3 SELECTED PAGES MAY BE CONVERTED PER CONVERSION.
PURCHASING A LICENSE REMOVES THIS LIMITATION. TO DO SO, PLEASE CLICK ON THE FOLLOWING LINK:
http://www.investintech.com/order_a2e_pro.htm