Electronic Device Control ByElectrooculography(EOG) Gesture Recognition

B.E. Project Report

by

Krishna JajodiaAmit GudekarAmey Kadam

under the guidance of

Prof. Y.S.RaoDepartment of Electronics and Telecommunication

Sardar Patel Institute of Technology, Andheri, MumbaiNovember 2010

Abstract

This paper describes an eye-control method based on electro oculography (EOG)to develop a system for assisted mobility .One of its most important features is itsmodularity, making it adaptable to the particular needs of each user according tothe type and degree of handicap involved .This paper describes the development ofa neural networks gesture recognition system whereby one can control a electronicdevice by using the components of his brain wave bio-potentials .Such a systemmay be used as a control device through human eye-movements ,facial muscleand brain wave bio-potentials .Neural networks are trained to classify EOG datainto one of two classes corresponding to two cognitive tasks performed by eighttraining segments.The operator’s forehead bio-potentials can be acquired andprocessed as electronic device control signals . Neural networks analyze user’sEOG signal in order to discern for the presence of a signal and then decide whetherit corresponds to a valid command .The trained neural network can effectivelyrecognize user intention, left or right based on EOG signal .The experimentalresults suggest that a electronic device can be operated by human brain wavebio-potentials with neural networks .This technique could be useful in multipleapplication such as mobility and communication aid for handicapped persons.

Contents

1 Introduction 1

2 Literature 4

3 EOG Electrodes 63.1 Ag-AgCl 4 MM TP Electrode - EL254 . . . . . . . . . . . . . . . 73.2 AG-AGCL EL258S . . . . . . . . . . . . . . . . . . . . . . . . . . 83.3 AG-AGCL+HOLE 8MM TP Electrode - EL258H . . . . . . . . . 93.4 Ag-AgCL+Hole 8MM TP Electrode EL258RT . . . . . . . . . . . 103.5 The ML317 EOG Pod . . . . . . . . . . . . . . . . . . . . . . . . 11

4 EOG gel 12

5 Design 135.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . 135.2 Requirement Specification . . . . . . . . . . . . . . . . . . . . . . 13

5.2.1 signal pick-up and amplification . . . . . . . . . . . . . . . 13

6 Electrooculography (EOG) 14

7 Implementation 187.1 Hardware Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 187.2 Principles of EOG Bio-potential Measurement . . . . . . . . . . . 19

8 Future Scope and Applications 228.1 Eye Tracking Computer User Interference . . . . . . . . . . . . . . 228.2 EOG Based Eye Blink Detection System . . . . . . . . . . . . . . 238.3 Hospital Alarm System . . . . . . . . . . . . . . . . . . . . . . . . 248.4 Automatic Sleep Stage Classification . . . . . . . . . . . . . . . . 248.5 Tracking Facial muscle and Eye Motion For Computer . . . . . . 258.6 portable Clinical EOG . . . . . . . . . . . . . . . . . . . . . . . . 27

References 30

i

List of Figures

3.1 EOG electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.2 EL254 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73.3 EL258S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83.4 EL258H . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93.5 EL258H . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.6 ML 317 EOG POD . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.1 EEEG/ECG/EMG/EOG PREP GEL 114 G - ELPREPEG . . . 12

6.1 EOG electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146.2 placement of electrodes . . . . . . . . . . . . . . . . . . . . . . . . 156.3 cornea-retinal potential . . . . . . . . . . . . . . . . . . . . . . . . 156.4 Noise Reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166.5 Eye movements by visually inspecting the spikes . . . . . . . . . . 17

7.1 eye movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

8.1 EOG measurement system . . . . . . . . . . . . . . . . . . . . . . 228.2 Blink detection method . . . . . . . . . . . . . . . . . . . . . . . . 238.3 Tracking facial muscle and eye motion for computer graphics ani-

mation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

ii

Chapter 1

Introduction

Eye movements are the most frequent of all human movements. Eye movementresearch is of great interest in the study of neuroscience. Since eye movementcan be controlled to some degree and track by modern technology with greatspeed and precision , they can now be used as a powerful input device and havemany practical applications in human computer interactions. The EOG is one ofthe very few methods of recording eye movements that does not require a directattachment to the eye itself. It is now accepted that the generated electricalpotential arises due to the permanent potential difference 10 to 30mV that existsbetween the cornea and ocular fundus. An electrical field is set up in the tissuessurrounding the eye and the rotation of the eye causes corresponding rotationof field vector. For this reason, it is possible to detect eye movement with theappropriate placement of electrodes on the skin surrounding the eyes.

Information, though conservative, shows that there are 70 million disabled peo-ple in India. One in every ten children or 3of inter-disciplinary research projects.Despite the recent technological improvements, eye trackers remain very mucha high cost research and academic tool requiring specially trained personnel toset-up and operate the systems.

In the last years, there has been a significant increase in the development of as-sistive technology for people with disabilities, improving the traditional systems.Also, the growing use of the computer, both in work and leisure, has led to thedevelopment of PC-associated handling applications, mainly using graphic inter-faces. This way, the traditional methods of control or communication betweenhumans and machines (joystick, mouse, or keyboard), that require a certain con-trol motor on the part of the users, they are supplemented with others that allowtheir use for people with severe disabilities.

1

Electro-occulography is recording technique that allows the standing potentialbetween the cornea and the posterior pole of the eye tube recorded. Becausethis biological signal is proportional to the rotational angle of eye this techniquepermits a wide range of rotational angles to be recorded, allows documenting ofeye movements when the eyelids are closed and unable the recording in the darkplaces. The EOG signal has the high signal to noise (S/N) ratio, therefore noelectromagnetic shielding is required. The advantage of EOG eye gaze interfaceis its simple configuration : in recording, a pair of electrodes is placed on theright and left temples and potential is amplified using the DC or AC amplifier.

Electro-occulography is widely used in ophthalmic research and clinical lab-oratories because it provides the method for recording with full range of eyemovements. Electro-occulograph can be used in ophthalmology for diagnosis andprognosis of several eye ailments.The basic idea used in EOG is to determinethe Arden-index of human eye from the deflections corresponding to the peakvalue and the dark trough value caused by changes of the resting potentials ofthe eyes. The resting potential is changed as the eye is moved and the move-ment of the eye is translated into electrical change of potential which is called anElectro-oculoghram.

One of the most developing researches in engineering that utilizes the exten-sive research in medicine is Biomedical engineering. This area seeks to help andimprove our everyday life by applying engineering and medical knowledge withthe growing power of computers. The area of this project can be applied notonly for helping disabled people but also in commercial use. Another area thatwill gain from Human-Machine interface is interactive computer games, test-ing subject’s responses and attention in simulators for training military and lawenforcers. The system will get input from human tested subject and will actaccording to it. The human input is the electronic signals produced by movingeyes. There are many different ways to measure this signal and we will use theelectro-occulography(EOG) to collect them. Generated electrical potential arisedue to the permanent potential difference of between 10 to 30mV that exist be-tween the cornea and ocular fundus. This is commonly referred to as cornearetinal potential, with the cornea being positive. An electric field is setup in tis-sue surrounding the eye and rotation of the eye causes a corresponding rotationof the field vector. For this reason it is possible to detect eye movement with theappropriate placement of the electrode on the skin surrounding the eyes. TheEOG is one of the very few methods for recording eye movements that does notrequire a direct attachment to the eye itself. For this reason, the EOG techniqueis preferred for recording eye movements.

2

We propose to design and build and electro-occulography(EOG) bio-potentialamplifier in order to obtain psychological signal due to eye movements and touse this signal to show directional discrimination. Our design can also be usedas a model for future advancement in human computer interaction. The EOGbio-potenial amplifier should be capable of detecting frequencies between DC10 Hz, the range at which most ocular movements operate. The EOG signalis in the micro volt range (50 to 3500 micro volt). Therefore, when the DCoffset is removed, it will be challenging to obtain a strong, useable signal giventhe minute nature of the recorded signal. Our choice of an EOG over otherpossible methods was selected based on ease of usage and low cost of production.The electro-occulography(EOG) is a measurement a bio-potentials produced bychanges in eye position. The fact that electrical activity could be recorded byplacing electrodes on the surface of the skin in the eye region was discovered in the1920s.It was realized that the electrical potential; induced corresponded to eyemovement. Originally it was thought that the induced electrical activity causedby eye movement corresponded to action potentials in the above mentio0ned pairsof muscles. It is now accepted that the generated electrical potentials arises dueto the permanent potential difference of between 10 to 30 mV that exists betweenthe cornea-occular fundus. The recording of the eye movement in EOG does notrequire any direct attachment to the eye itself. For this reason the EOG signal ispreferred for recording eye movements in sleep and dream research. Recently thistechnique has become popular for evaluating reading ability and visual fatigue ofsubjects.

3

Chapter 2

Literature

The expression the literature typically refer to publish in books, journals andconference proceeding that relate to the failed of investigation within which astudents project lies. Such literature also includes unpublished theses and disser-tation. The idea of our project emerged when we saw human computer interactionfor disabled and for more luxurious living. It enables the user to perform anyactivities without moving from there place. This application might be confusedwith the home automatic system but it is far most sophisticated then the usualhome automation systems. This application is implied upon several disabled peo-ple who are paralyzed and even for thoughts who want to control various devicesby the movement of their eyes alone without moving from their couches. Thisdevice is based upon human computer interface as in we tried to control a devicefrom eye gesture. we have used simple neural analysis related to eye gesturesand came to the conclusion that eye movements can produce signals oh certainmicrovolt which can be amplified to make an electronic device to work normally.

Various websites and journals where enough to provide sufficint and the nec-essary details of the project related the literature and also helped enough tomake correct implementation of the project. We also rendered help from var-ious medical journals and from some of the medical practitioners . electro-occulography(EOG) is the technique for measuring the resting potential of theretina. The resulting signal is called the electro-occulogram. The main applica-tions are in ophthalmological diagnosis and in recording eye movements. Unlikethe electro-retinogram, the does not represent the response to individual visualstimuli. Eye movement measurement: usually ,pairs of electrodes are placed ei-ther above and below the eye or to the left and the right of the eye. If the eyemoved from the centre position towards one position, this electrode ”sees” thepositive side of the retina and the opposite electrode ”sees” the negative side ofthe retina. Consequently potential difference occurs between the electrodes. As-suming that the resting potential is constant, the recorded potential is measurefor the eye position.

4

Ophthalmological diagnosis: The EOG is used to access the function of thepigment epithelium. During dark adaptation, the resting potential decreasesslightly and reaches minimum after several minutes. When light is switchedon , a substantial increase of the resting potential occurs which drops off aftera few minute when the retina adapts to the light. Te ratio of the voltages isknown as the Arden ratio. In practice, the measurement is similar to the eyemovement recording. The EOG signal is derived from the polarization potential,also known as the cornea- retinal potential(CRP), generated within the eyeballmetabolically active retinal epithelium. The EOG signal is acquired though abi-channel signal acquisitions systems namely, the horizontal and the verticalchannels. Electrodes placed on the either side of the eyes or above and belowthem pickup the potential generated by the motion of the eyeballs. This potentialvaries approximately proportional to the displacement of the eyeballs within theconductive environment of skull. Saccades inherent in eye motion as well as theblinking of the eyelids can produce changes in EOG signals. The strength of thesignal is 10 to 100 microvolt and the useful frequency component is DC 10 Hz.This necessitates the careful selection of the bio-potential amplifier.

The recording of the EOG signal has traditionally been associated with sev-eral problems. The signal is seldom deterministic, even for the same person fordifferent experiments. It is a result of the number of the factors including eyeball rotation and movements, eyelid movement. For this reason it is extremelyessential to eliminate the shifting resting potential( mean DC value), because thisvalue varies continuously.

5

Chapter 3

EOG Electrodes

Silver-silver chloride (Ag-AgCl) electrodes provide accurate and clear transmis-sion of surface biopotentials. EL250 series reusable lead electrodes are suitablefor most applications (ECG, EEG, EGG, EMG, EOG, and ERS recordings.)Use EL258 (8 mm recording diameter) for most applications. Use EL254 (4mm recording diameter) when closely spaced biopotentials are required. EL250series reusable electrodes are permanently connected to robust and pliable lead-wires (1 mm OD). The leadwires terminate in standard Touchproof connectors.Unshielded electrodes terminate in a single Touchproof connector. Shielded elec-trodes terminate in two Touchproof connectors, one connects to the Ag-AgCl diskand the other connects to the leadwire shield. The EL254 is unshielded and theEL254S is shielded. For best signal performance use shielded electrodes (EL254Sor EL258S) as recording electrodes and unshielded electrodes (EL254 or EL258)as ground or reference electrodes. Generally, for each Biopotential amplifier mod-ule, two EL254S or EL258S and one EL254 or EL258 are required. Use reusableelectrodes with: GEL103 conductive adhesive gel or see Electrode Acessories foradhesive collars and other gels.

6

Figure 3.1: EOG electrodes

3.1 Ag-AgCl 4 MM TP Electrode - EL254

Figure 3.2: EL254

Specifications:

• Dimensions: 7.2 mm outer diameter, 4 mm recording diameter, 6 mm high.

• Leadwire OD: 1 mm.

• Leadwire length: 1 meter.

7

3.2 AG-AGCL EL258S

Figure 3.3: EL258S

Specifications:

• Dimensions: 12.5mm outer diameter, 8mm recording diameter, 6mm high[EL258H-4mm high].

• Lead length: EL258, EL258S and EL258H - 1 meter, EL258RT - 1.5 meter.

8

3.3 AG-AGCL+HOLE 8MM TP Electrode - EL258H

Figure 3.4: EL258H

Specifications:

• Dimensions: 12.5 mm outer diameter, 8 mm recording diameter, 6 mm high[EL258H-4 mm high].

• ead length: 1 m.

9

3.4 Ag-AgCL+Hole 8MM TP Electrode EL258RT

Figure 3.5: EL258H

Specifications:

• Construction: Carbon fiber leadwire with integral Ag/AgCl electrode.

• Leadwire Length: 1.5 m.

• Leadwire Diameter: 1.0 mm.

• Leadwire Resistance: 174 Ohms/m.

• Dimensions: 7.2 mm outer diameter, 4 mm recording diameter, 6 mm high.

• MRI compatible: Yes.

• Radiotranslucent: Yes.

10

3.5 The ML317 EOG Pod

Figure 3.6: ML 317 EOG POD

Specifications:

• Amplification Range 2 mV, 1 mV, 500 V, 200 V, 100 V.

• CMRR common mode greater than 80 dB.

• Cable Length Metric 1.5 m.

• Certifications CE.

• Filtering Low Pass Filter 500 (fixed) 2nd order Butterworth.

• Frequency Response DC to 500 Hz.

• Front Panel Control offset knob for initial zeroing of device.

• Gain x1000.

• Gain Error 5

• IMRR isolation mode greater than 110 dB.

• Input Connection Type 3 shielded lead wire connectors.

• Input Impedance less than 100 M?.

• Model ML317.

• Temperature Drift 3 mV/C.

• Weight Metric 200 g.

• Weight Metric 200 g.

11

Chapter 4

EOG gel

EEEG/ECG/EMG/EOG PREP GEL 114 G - ELPREPEG/ECG/EMG/EOGPREP GEL 114 G - ELPREP Prepare skin and apply small amount to appropri-ate electrode site by squeezing near tube opening. Apply small amount to discelectrode and press into the paste that has been applied to the head. Clean withwarm water.

Figure 4.1: EEEG/ECG/EMG/EOG PREP GEL 114 G - ELPREPEG

Net weight: 114 g (4 oz)

12

Chapter 5

Design

5.1 Problem Statement

To implement such a device which would perform certain predefine actionsaccording to the gesture patterns obtained by the actions performed by the humaneye. The main hurdle of our project is to acquire the signals from the eye gesturesor the eye movements. The next hurdle could be to make use of this signal andimplement or involve them in a certain logic which would be predefine and makethe electronic device (computer interface) to produce the result according to thelogic defined by these gesture patterns.

5.2 Requirement Specification

In order to fulfil the above requirement there was a need to peak up the signalsusing sensors, filter these signals in the desired range and then amplify this signalsso that these signals can be applied to the device being used(computer interface).Thus the hardware used for this project is as follows:

5.2.1 signal pick-up and amplification

• Signal pick-up and amplification:

• Electrode used: Ag-AgCl electrodes

• Lowpasssignal components: R-C network,

• Fixed gain amplifier

• Variable gain amplifier

13

Chapter 6

Electrooculography (EOG)

EOG is a method for sensing eye movement and is based on recording thestanding corneal-retinal potential arising from hyper polarizations and depolar-ization existing between the cornea and the retina; this is commonly known as anelectrooculogram . This potential can be considered as a steady electrical dipolewith a negative pole at the fundus and a positive pole at the cornea as shownbelow.

Figure 6.1: EOG electrodes

The standing potential in the eye can thus be estimated by measuring thevoltage induced across a system of electrodes placed around the eyes as the eyegaze changes, thus obtaining the EOG (measurement of the electric signal of theocular dipole). The EOG value varies from 50 to 3500 V with a frequency range ofabout dc-100 Hz. Its behavior is practically linear for gaze angles of . It shouldbe pointed out here that the variables measured in the human body (any biopotential) are rarely deterministic. Its magnitude varies with time, even when allpossible variables are controlled. Most of these bio-potentials vary widely amongnormal patients, even under similar measurement conditions.

14

Figure 6.2: placement of electrodes

This means that the variability of the Electro-oculogram reading depends onmany factors that are difficult to determine: perturbations caused by other biopo-tentials such as EEG (electroencephalogram), EMG (electromiogram),in turnbrought about by the acquisition system, plus those due to the positioning oftheelectrodes,skin-electrode contacts, lighting conditions, head movements, blink-ing, etc. In various studies were made of the accuracy and precision of the EOGin tracking the eye gaze. To eliminate or minimize these defects, therefore, aconsiderable effort had to be made in the signal acquisition stage to make sure itis captured with the minimum possible perturbations and then during the studyand processing thereof to obtain the best possible results.

Figure 6.3: cornea-retinal potential

The fact that electrical activity could be recorded by placing electrodes onthe surface of the skin in the eye region was discovered in the 1920s. It wasrealised that the electrical potentials induced corresponded (almost linearly) toeye movement. Initially, it was believed that the induced electrical activity causedby eye movement. Nowadays, it is accepted that the induced electrical potentials

15

arise due to the permanent potential difference of between 10 and 30mV thatexists between the cornea and the ocular fundus (left) as shown in the Fig below.This is commonly referred to as the cornea-retinal potential (CRP) with thecornea being positive.

1. The Noise Reduction.

The eye movement signals are band limited due to the fact that thereis a speed limit on eye movements. Thus, a low pass filter with 20Hzcutoff could remove most of the high frequency noises. The largest noisewe observed was the 60Hz noise from the power line. We compared thestandard deviation of the signal in order to discriminate meaningful signalsfrom noise (see figure below). It turned out that the whole signal is within2 times of the deviation, and the base level noise was mostly within thedeviation. This led to our calibration strategy to choose the thresholdparameter to be above the deviation.

Figure 6.4: Noise Reduction

2. Characteristics Of Eye Movements.

In this experiment, we tried to characterize each type of eye movementsby visually inspecting the spikes. The result is summarized in the followingdecision table. ’+’/’-’ indicates positive/negative peak, ’0’ means belowcertain level,and N/A means does not matter. Blink is characterized aseither a consecutive ’+’ and ’-’, or ’+’, ’0’, ’-’.

Each pair of graphs is a result of aligning and overlaying 10 trials for eachmovement. The left and right graph corresponds to horizontal and verticalbipolar measurement, respectively. You can see the trials are overlapping

16

Figure 6.5: Eye movements by visually inspecting the spikes

strongly, which means that the shape and strength of signals are station-ary.When the eye is moving fast towards left or right, we get a strong peakin the horizontal bipolar measurement, and in the case of up or down move-ment, the peak was strong in the vertical bipolar measurement, as expectedby physiological insights. Note that for the blink, a positive peak followedby a slight negative peak on the vertical bipolar measurement is observed.

3. Calibration.

Since the EOG signal varies depending on several uncontrollable factors,such as placement and conductance of the electrodes, and also the ampli-tude pattern which differs across subjects, it is essential to have a processto calibrate the parameters of the detection and recognition system. Wehad a semi-automatic procedure for calibration.This is a typical calibrationgraph we generated. There are all five movements in the time frame. Thesubject was asked to move his or her eyes to the left, center, up, center, andthen blink in one second after the cue was given. Both vertical and hori-zontal signals were plotted in one figure along with the calculated standarddeviation. The deviations are used as a guideline for choosing threshold;it should be at least larger than the deviation. After a couple of stablecalibration graph is obtained, we decided the parameters for correct androbust discrimination of each eye movement.

17

Chapter 7

Implementation

As performance requirements increase, the implementation of control elementsin embedded applications is moving from 8 bits to 32 bits. At the same time,the implementation vehicle of choice for embeddedapplications is changing fromASICs to FPGAs due to cost and time to market pressures. This paradigmshiftis causing significant change in the choices designers are making for the executionof embeddedapplications. This is most evident in consumer, industrial, and auto-motive applications, which are thefastest growing segments of the FPGA market.The ARM7 processor is widely used in these segments.Implementing the ARM7processor in Actel Flash-based devices allows users to take maximum advantageofthis industry standard processor as well as the changes that are occurring in theembedded market.The ARM7 processor is an industry standardarchitecture witha huge ecosystem of tools, support, and embedded designer knowledge. It is themostwidely implemented 32-bit processor, with billions in use.

The ARM7TDMI-S is a general purpose 32-bit microprocessor, which offershigh performance and very low power consumption.The ARM architecture isbased on Reduced Instruction Set Computer (RISC) principles This simplicityresults in a high instruction throughput and impressive real-time interrupt re-sponse from a small and cost-effective processor core

7.1 Hardware Design

The electrodes which can be placed around the eyes will give certain signalswhich would pass through the low pass filters giving output signals of range 0 to10 Hz. These signals thus filtered and then passed on to the pre amplifier whichwould make the signals much powerful so that they become more prominentanalogue signals and are directly digitized by an A/D converter. Thus thesedigitized signals can be used directly for signal processing and are given to thedevice so that the device can perform the desired task.

18

Figure 7.1: eye movement

7.2 Principles of EOG Bio-potential Measure-

ment

The unifying principles of any bio-potential recording mainly consists of elec-trode design and attachments suited to the application, amplifier circuit designfor suitable amplification of the signal ang rejection of noise and interference andfinally good measurement practices to mitigate artifacts, noise and interference.

• Ag-AgCl Electrodes and Electrolytic Gel.

Electrodes for bio potential recordings are design to obtain the signal ofinterest selectivity while reducing the tendency to pick-up artifacts. Thedesign should be pragmatic to reduce cost and allow for good manufacturingand reliable long term use. These practice consideration determine whetherhigh quality but reusable electrodes made of silver or gold, or chipper dis-posable electrodes are to be used. Ag-AgCl Electrodes have been usedwhich produce low level of junction potential, motion artifacts and driftin the DC signal. Additionally, an an electrolytic gel based on AgCl wasapplied to the skin since the upper layers of skins are poor conductor of theelectricity. A gel concentration in the order of 0.1 molar concentration re-sults in good conductivity and low junction potential without causing skinirritation.

19

• Minimizing Noise,Artifacts and Inter-Channel Interference.

The frequency component of the EOG signal is very close to DC and hencethe separation of Dc drifts from the useful signal content is a difficult task.Dual channel acquisition of the EOG signal has been employed and consid-erable inter channel interference was observed. This interference arises dueto two factors ,namely, small deviation of the eyeball position towards theother channel during motion and improper alignment of the electrode pairs.In our system appropriate threshold hav been set to counter this effect andaid in correct determination of the eyeball postion.

• The Pre-Amplifier.

The design consideration of preamplifier ought to include proper ampli-fication and bandwidth, high input impedance , high CMRR, low noise,voltage fluctuation and stability against temperature. An instrumentationamplifier used to meet these requirements. The required low frequency re-sponse might make the amplifier susceptible shifts in junction potential atthe kin electrode interface. A drift cancelation may be necessary if requiredby the application.

• Signal Acquisition.

An instrumentatiomn amplifier has been used since it reduced the effectof common mode signals like power line interference, electrodes movementand skin muscles artifacts, which affect the electrodes the pairs, almostequally.

• Filtering and Amplification.

The pre amplifier After the EOG signal has been acquired and ampli-fied, the next stage is the passive band pass filter and the second stage ofamplification. The useful EOG signal contents varies between DC-10Hz. Abandpass filter with passband of 0.1 to 10 Hz is used to pass the reverentsignal contain and attenuate the DC offset. The noise, as well as the powersupply interference are also suppressed. A second stage of amplificationfollows a bandpass filter, since the gain of amplifier is not sufficient to am-plify te EOG signal; to usable level. This is achieved by a non-inverting mpwith an amplification of approximate 400. the attenuation provided by thefirst stage of high pass filtering is insufficient . hence we require a secondstage of offset removal which is provided by a first stage passive high passfilter with a cut off frequency 0.1Hz. The inputs from Ag-AgCl electrodes

20

are applied to a passive low pass filter consisting of the R-C network asshown. The 10K-470K and 0.01 micro farad from a passive filter networkthen its output is applied to the active filter network formed by the op-ampOP-07.Circuit consists of three stages :

– The filter circuit using OP-07.

– Fixed gain amplifier with gain =100.

– Variable gain amplifier with maximum gain=1.

The filter circuit consists of active and passive filter to pass the de-sire frequency i.e 0 to 10Hz and attenuates the unwanted frequencyand thus increasing the signal to noise ratio. The fixed gain ampli-fier amplifies the signal output from the filter circuitry and gives theamplification of 100 . The amplifier is used in inverting mode configu-ration and thus the gain of the amplifier is given by - R f / R1 . thusthe gain is given is given by -100k/ 1k i.e 100. In order to enhance thetotal gain of the circuit and to get the output as required for the signalfor further processing, the output from the fixed amplifier is appliedto variable gain amplifier with maximum gain of 100. Thus the themaximum overall gain is 100*100=10000.

21

Chapter 8

Future Scope and Applications

8.1 Eye Tracking Computer User Interference

Our main goal is to develop an inexpensive hardware software system for use inthe most challenging cases the estimated 150,000 disable persons able to controlonly the muscles of their eyes. This encompasses the construction of an EOGeye tracking hardware and its fine tuning in software, as well as the definition ofacknowledgeable eye behavior and the establishment of basic protocols governingon screen object selection and manipulation, such device can also be used formany virtual reality systems and video games.

Figure 8.1: EOG measurement system

Electro-oculography depends on the corneoretinal potential that creates anelectrical field in the front of the head. This field changes in orientation as theeyeballs rotate. The electrical changes can be detected by electrodes placed on theskin near the eyes. In clinical practice, the detected voltage changes are amplifiedand used to drive a plotting device, whereby a tracing of eye position is obtained.It is position is obtained is possible to obtain independent measure ments from the

22

two eyes. However, the two eyes move in conjunction in the vertical direction.Hence, it is sufficient to measure the vertical motion of only one eye and thehorizontal motion of both eyes. If the orientation of the eyes is measured , itis possible to locate the 3D position of a fixated target object by triangulation.Recognizing blinks as legitimate actions distinct from eye movement also allowtheir use for rapid invocation of important global commands, such as calling anattendant, and in each module as context sensitive command shortcuts. The EOGsystem can potentially recognize ”eye gestures,” such as left and right winkingand blinking, or any combination there of he eye gesture command languagecould even be extensible and programmable by the user himself. For example,during text entry or while scanning read-only text, a left blink rapidly followedby a right blink could be a page-up command; right followed by a left would bea page-down, etc.

8.2 EOG Based Eye Blink Detection System

The electrooculogram represents the electrical activity of muscles that controlmovements of eyes. The eye blinking is a natural protection system which defendsthe eye from environmental exposure. The spontaneous eye blink is consideredto be a suitable indicator for fatigue diagnostics during many, different tasks ofhuman being activity.

Figure 8.2: Blink detection method

The action of eye blinks covers a specific range of frequency and for that reasonit is possible to construct a function which processes the signal and generates anartificial peak when blink occurs. This function is called the detection function.This function is used to detect the spontaneous eye blink action. Nonlinear andlinear signal processing methods are applied to obtain the detection functionwaveform. On this base the position of an eye blink is estimated. The results

23

demonstrate that the measurement of an eye blink parameter provides reliableinformation for eye-controlled systems from human-machine interface.

8.3 Hospital Alarm System

we proposed an eye-movement tracking system. Based on electro- occulogra-phy (EOG) technology we detected the signal with different directions in eye-movements and then analyzed to understand what they represented about (e.g.horizontal direction or vertical direction). We converted the analog signal todigital signal and then used as the control signals for human computer interface(HCI). In order to make the system robust, several applications with EOG-basedHCI had been designed. Our preliminary results revealed more than 90

8.4 Automatic Sleep Stage Classification

Sleep is a natural periodic state of rest for the body, in which the eyes are usuallyclosed and consciousness is completely or partially lost. In this investigation weused the EOG and EMG signals acquired from 10 patients undergoing overnightpolysomnography with their sleep stages determined by expert sleep specialistsbased on RK rules. Differentiation between Stage 1, Awake and REM stageschallenged a well trained neural network classifier to distinguish between classeswhen only EEG-derived signal features were used. To meet this challenge andimprove the classification rate, extra features extracted from EOG and EMGsignals were fed to the classifier. In this study, two simple feature extractionalgorithms were applied to EOG and EMG signals. The statistics of the resultswere calculated and displayed in an easy to visualize fashion to observe tendenciesfor each sleep stage. Inclusion of these features show a great promise to improvethe classification rate towards the target rate of 100

Standard sleep stage classification is based on visual analysis of central EEG,EOG and EMG signals. Automatic analysis with a reduced number of sensorshas been studied as an easy alternative to the standard. In this study, a single-channel electro-oculography (EOG) algorithm was developed for separation ofwakefulness, SREM, light sleep (S1, S2) and slow wave sleep (S3, S4). Thealgorithm was developed and tested with 296 subjects. Additional validation wasperformed on 16 subjects using a low weight single-channel Alive Monitor. In thevalidation study, subjects attached the disposable EOG electrodes themselves athome. In separating the four stages total agreement (and Cohen’s Kappa) in thetraining data set was 74

24

8.5 Tracking Facial muscle and Eye Motion For

Computer

A motion tracking system enables faithful capture of subtle facial and eye mo-tion using a surface electromyography (EMG) detection method to detect mus-cle movements and an electrooculogram (EOG) detection method to detect eyemovements. An embodiment of the motion tracking animation system comprisesa plurality of pairs of EOG electrodes adapted to be affixed to the skin surfaceof the performer at locations adjacent to the performer’s eyes. The EOG datacomprises electrical signals corresponding to eye movements of a performer dur-ing a performance. Programming instructions further provide processing of theEOG data and mapping of processed EOG data onto an animated character. Asa result, the animated character will exhibit he same muscle and eye movementsas the performer.

The present invention providing a motion tracking system that enables faithfulcapture of subtle facial and eye motion. The invention uses a surface electromyo-graphy (EMG) detection method to detectmuscle movements, and an electroocu-logram (EOG) detection method to detect eye movements. Signals correspondingto the detected muscle and eye movements are used to control an animated char-acter to exhibit the same movements performed by a performer.

More particularly, an embodiment of the motion tracking animation systemcomprises a plurality of pairs of electromyography (EMG) electrodes adapted tobe affixed to a skin surface of a performer at plural locations corresponding torespectivemuscles, and a processor operatively coupled to the plurality of pairsof EMG electrodes. The processor includes programming instructions to per-form the functions of acquiring EMG data from the plurality of pairs of EMGelectrodes. The EMG datacomprises electrical signals corresponding to musclemovements of the performer during a performance. The programming instruc-tion further include processing the EMG data to provide a digital model of themuscle movements, and mapping the digital modelonto an animated character.As a result, the animated character will exhibit the same muscle movements asthe performer.

In an embodiment of the invention, a plurality of pairs of electrooculogram(EOG) electrodes are adapted to be affixed to the skin surface of the performerat locations adjacent to the performer’s eyes. The processor is operatively cou-pled tothe plurality of pairs of EOG electrodes and further includes programminginstructions to perform the functions of acquiring EOG data from the plurality ofpairs of EOG electrodes. The EOG data comprises electrical signals correspond-ing to eye movementsof the performer during a performance. The programming

25

Figure 8.3: Tracking facial muscle and eye motion for computer graphics anima-tion

instructions further provide processing of the EOG data and mapping of theprocessed EOG data onto the animated character. This permits the animatedcharacter to exhibit the same eye movements asthe performer.

A more complete understanding of the motion tracking system that enablescapture of facial and eye motion of a performer for use in producing a computergraphics animation will be afforded to those skilled in the art, as well as a re-alization ofadditional advantages and objects thereof, by a consideration of thefollowing detailed description of the preferred embodiment. Reference will bemade to the appended sheets of drawings which will first be described briefly.

Above is a block diagram illustrating a motion tracking system in accordancewith an embodiment of the present invention; a block diagram illustrates a motiontracking system 100 in accordance with an embodiment of the present invention.The motion tracking system includes a motion tracking processor adapted tocommunicate with aplurality of facial muscular electrode pairs and a plurality ofeye motion electrode pairs through a suitable electrode interface . The motiontracking processor 108 may further comprise a programmable computer havinga data storage device 106adapted to enable the storage of associated data files.As known in the art, one or more computer workstations may be coupled to themotion tracking processor 108 using a network to enable multiple graphic artiststo work with the stored data files inthe process of creating a computer graphicsanimation. The motion tracking processor 108 may further include a computergraphics animation system such as provided by a commercial software packagethat enables the creation of 3D graphics and animationfor the entertainment in-dustry, such as the Maya.RTM. software product line sold by Alias—Wavefront orother like products. It should be understood that the computer graphics anima-tion system may comprise an entirely separate computer hardware andsoftware

26

system from the motion tracking processor 108, or alternatively, may be incor-porated with the motion tracking processor 108 (e.g., as a ”plug-in”) as part ofa common hardware and software system.

8.6 portable Clinical EOG

It is very important to study the EOG system in eye movement for both cliniciansand basic scientists due to the abundant neuropathological information. How-ever, the present commercial stimulation instrument only provides very few andfixed patterns. Therefore, techniques of computer animation, data acquisition,data analysis and database management are applied to implement an intelligentinstrument that can stimulate and diagnose EOG system with patterns freely setby the doctors or the researchers, so that vision and nerve system illnesses canbe studied efficiently. Instead of the expensive and huge commercial stimulationinstruments in the present market, in this paper a personal computer is used dueto its cheapness, popularity, multifunction and fast speed.

A lot of interesting stimulation patterns can easily be created in shape, timesequence, and color under Windows 95. It is hoped that this invention cancontribute to clinical diagnosis and basic medical science research for EOG.

Monitoring eye movements is clinically important in diagnosis of diseases ofthe central nervous system. Electrooculography (EOG) is one method of ob-taining such records which usesskin electrodes, and utilizes the anterior posteriorpolarization of the eye. A new EOG diagnostic system has been developed thatutilizes two off-the-shelf portable notebook computers, one projector and simpleelectronic hardware. It can be operated under Windows 95, 98, NT, and hassignificant advantages over any other similar equipment, including programma-bility, portability, improved safety and low cost. Especially, portability of theinstrument is extremely important for acutely ill or handicapped patients. Thepurpose of this paper is to introduce the techniques of computer animation, dataacquisition, real time analysis of measured data, and database management to im-plement a portable, programmable and inexpensive contacting EOG instrument.It is very convenient to replace the present expensive, inflexible and large-sizedcommercially available EOG instruments.

A lot of interesting stimulation patterns for clinical application can be createdeasily in different shape, time sequence, and colour by programming in Delphilanguage. With the help of Winstar (a software package that is used to controlI/O and interrupt functions of the computer under Windows 95, 98, NT), theI/O communication between two notebook computers and A/D interface modulecan be effectively programmed. In addition, the new EOG diagnostic system is

27

battery operated and it has the advantages of low noise as well as isolation fromelectricity. Two kinds of EOG tests, pursuit and saccade, were performed on 20normal subjects with this new portable and programmable instrument. Based onthe test result, the performance of the new instrument is superior to the othercommercially available instruments. In conclusion, we hope that it will be moreconvenient for doctors and researchers to do the clinical EOG diagnosis and basicmedical science research by using this new creation.

28

Acknowledgment

We would like to express our deep sense of gratitude to Prof.Y.S.Rao for hisinvaluable help and guidance during the course of project. We are highly in-debted to him for constantly encouraging us by giving critics on our work. Weare grateful for having given us the support and confidence.

Krishna JajodiaAmit GudekarAmey KadamNovember 2010Sardar Patel Institute of Technology

29

Bibliography

[1] Hashimoto,Takahashi,Shimada ”Wheelchair control using an EOG- andEMG-based gesture interface”.

[2] By Harun H., Mansor W ”EOG signal detection for home appliances acti-vation”.

[3] K. Schilling, H. Roth, R. Lieb, and H. Sttzle ”Sensors to improve the safetyfor wheelchair users”.

[4] Rafeal Barea, Luciano Boquete, Manuel Mazo”System for Assisted MobilityUsing Eye Movements Based on Electroculography”.

[5] D. G. Evans, R. Drew, and P. Blenkhorn ”Controlling mouse pointer posi-tion using a infrared head-operated joystick”.

[6] R. Barea, L. Boquete ”Accurate interaction with computer by eye movementtracking ”.

30