electro-oculography (eog) measurement systemvision.technion.ac.il/projects/pspl/2004s34/book/eog...

30
1 Electro-Oculography (EOG) Measurement System מגיש ים: אסיה אוקון מקסים שמוקלר מנחה: קליימן אלדד ארז יוחנן אוקטובר2004 הטכניון לישראל טכנולוגי מכון חשמל להנדסת הפקולטה התמונה ומדעי הראיה לחקר המעבדה

Upload: trinhkien

Post on 15-Apr-2018

224 views

Category:

Documents


0 download

TRANSCRIPT

1

Electro-Oculography (EOG) Measurement System

:יםמגיש

אוקון אסיה

שמוקלר מקסים

:מנחה

אלדד קליימן יוחנן ארז

2004אוקטובר

מכון טכנולוגי לישראל–הטכניון הפקולטה להנדסת חשמל

המעבדה לחקר הראיה ומדעי התמונה

2

Table of Contents:

1. Acronym and Diagrams List..………………………………………………………………….3 2. Abstract………………………………………………………………………………………...4

3. Introduction…………………………………………………………………………………….5

4. Chapter 1: EOG measurement…………………………………………………………………7

5. Chapter 2: The hardware in use……………………………………………………………….11

6. Chapter 3: Hardware deficiency……………………………………………………………....12

7. Chapter 4: The software solution……………………………………………………………..16

8. Chapter 5: Implementation of the software…………………………………………………...22

9. Chapter 6: Final results………………………………………………………………………..24

10. Summary……………………………………………………………………………………....28

11. Appendix……………………………………………………………………………………....30

3

Acronym and Diagrams List

EOG Electro Oculogram HET Horizontal Eye Tracker LPF Low Pass Filter ENG Electronystagmograhy EMG Electromyogram - graphic record of the electrical activity of muscles EEG Electroencephalogram - graphic record of the electrical activity of the brain ECG Electrocardiogram - graphic record of electrical pulses given off by the heart Block diagram of the system, page 5 Figure 1.1: Horizontal section of the right human eye seen from above, page 7. Figure 1.2: An illustration of the electro-oculogram (EOG), page 8. Figure 1.3: Placement of Transducer Pickups to Measure Eye Movements, page 9. Figure 2.1: PC-ECG 1200S – used for cardiology imaging systems, page 11. Figure 3.1: Initial output signal from the eyes, page 12. Figure 3.2: Initial output signal from the signal generator, page 12. Figure 3.4: PC-ECG Impulse response, page 13. Figure 3.5: Frequency response of PC-ECG for 1=α , page 14. Figure 3.6: Impulse response of appropriate system for EOG measurement, page 14. Figure 3.7: Frequency response of appropriate system for EOG measurement, page 15. Figure 4.1: The correction process done by the software, page 16. Figure 4.2: Discrete Inverse system, page 16. Figure 4.3: Inverse system in Matlab, page 17. Figure 4.5: First drift phenomenon, page 18. Figure 4.6: Second drift phenomenon, page 19. Figure 4.7: Correcting the first drift phenomenon, page 20. Figure 4.8: Correcting the second drift phenomenon, page 21. Figure 5.1: Horizontal Eye Tracker Main Form, page 22. Figure 6.1: Good generated signal ("sigen") recording, page 24. Figure 6.2: Bad generated signal ("sigen") recording, page 25. Figure 6.3: Improved "sigen" recording, page 26. Figure 6.4: First successful recording of Eye movements, page 27. Figure 6.5: Second successful recording of Eye movements, page 27. Picture 1: Proposed ML317 EOG Pod device, page, 28. Picture 2: Data obtained with proposed hardware device, page 29. Picture 3: Improving advertisements with eye-movement research, page 30. Picture 4: Visual scan paths on instruments/dashboards – studies for the improvement of

human-computer interfaces, page 30.

4

Abstract

This project proposes to build human-computer interface based on eye movement detection. As

the tested subject eyes move on a horizontal scale the marker in the implemented software will follow his observation point, showing the current point on which the subject gazes. The proposed system includes two parts. The first is the hardware available in our lab the PC-EOG 1200 by Norav Medical L.T.D with which the eye signals (EOG) are obtained. The second is the implemented software in our project, Horizontal Eye Tracker (HET) which makes the detection and the visual presentation of the current observation point. Finally this project can be implemented in research of eye movement for smart advertisement, medical use, interactive computer games and military purposes.

5

Introduction

One of the most developing researches in Engineering that utilizes the extensive research in medicine is Biomedical Engineering. This area seeks to help and improve our everyday life by applying engineering and medical knowledge with the growing power of computers. The computers are efficient, straightforward and never get tired or sick, while humans though are smart and creative, become sick, weak and limited. Communication between humans seem usually much simple than the one involves humans and machines. This difficulty increases when a person is disabled. However, especially this kind of people has more to gain by assisting a machine in their everyday life.

The area of this project can be applied not only for helping disabled people but also in commercial use. Advertisement has developed greatly in the last decades. The fact it's being directed towards people makes it important to understand how people see and detect commercials (See Appendix for examples).

Another area that will gain from Human-Machine interface is Interactive computer games, testing subject's responses and attention in simulators for training military and law enforcers. The goal of this project is to build an inexpensive detecting system for those purposes. The system will get input from the human tested subject and will act according to it. The human input is the electronic signals produced by moving eyes. The nature and the source of these signals will be described in Chapter 1. There are many different ways to measure these signals and we will use the Electro-Oculography (EOG) to collect them. The recording of EOG will be done using an existing hardware in our lab. Then, this data will be processed digitally by the software implemented by us. The mission of this software is to detect and to distinguish between horizontal and vertical eye movements. In addition, the software has to determine the actual place on the computer screen on which the tested subject gazes. The detection point determined by the software will be presented immediately on the scale drawn on the main form.

In order to achieve satisfactory results by using this technique for detecting correctly the current observations point of the tested subject, several assumptions and restrictions are made on the eye movements and on the distance between the tested subject and the monitor of the computer. From our observations and experiments we found that this distance should be approximately 0.5 [m].

Block Diagram of the system

Digital Signal, 12bits / 2 [kHz]

Analog Signal, Amplitude ~ 100 [uV]

6

At first, in order to simplify the task, the software that we have implemented receives input

originated from only horizontal eye movements and makes the distinction between three observation points. As we have continued and advanced in our work and study of the hardware available we have encountered a fundamental problem with our hardware device. The detailed explanation of this problem is described in Chapter 3. Due to the vital necessity to complete the project with the only device available to us, which is presented in Chapter 2, we proposed a software solution to this problem. The way to this solution brought us to work with input signals originated not only from human eyes but with artificial input signals made by the signal generator.

Thus, the software we have implemented Horizontal Eye Tracker (HET) can work with real eye signals as well as simulated signals in form of a "Square Sine" with user defined frequency originated by signal generator.

The solution is widely explained and demonstrated in Chapter 4. The software we have implemented provides satisfactory results for detection and distinction between three observation points. This becomes possible with preliminary calibration which can be made in HET and that is described in Chapter 5.

The final results that can be obtained by using HET software are presented in Chapter 6. Each result is analyzed and conclusions for improvement for future work are made.

7

Chapter 1: EOG Measurement

In order to build a detecting system for eye movements which will use electronic signals originating from the eyes it's crucial to understand the eye structure and the source of the signals which are measured by the system. Understanding these signals and their nature will help to design a suitable system that will function properly and will simplify its use.

Eye structure:

Vision is one of our most valued senses and during the course of each day our eyes are constantly moving. Attached to the globe of the eye, there are three antagonistic muscle pairs, which relax and contract in order to induce eye movement. These pairs of muscles are responsible for horizontal, vertical and torsional (clockwise and counter clockwise) movement.

Figure 1.1: Horizontal section of the right human eye seen from above. The anteroposterior diameter averages 24 mm.

Eye signals:

Two specific categories exist which can be used to classify the four different types of conjugate eye movements: 1. Reflex eye movements - These provide stabilization of eye position in space during head

movement. 2. Voluntary eye movements - These are conscious eye movements involved in the redirection of

the line of sight in order to pursue a moving target (pursuit movement) or to focus on a new target of interest.

Signals Measurement:

The electro-oculography (EOG) is a measurement of biopotentials produced by changes in eye position. The fact that electrical activity could be recorded by placing electrodes on the surface of the skin in the eye region was discovered in the 1920’s. It was realized that the electrical potentials induced corresponded (almost linearly) to eye movement. Originally, it was thought that the induced electrical activity caused by eye movement corresponded to the action potentials in the above mentioned pairs of muscles.

8

It is now accepted that the generated electrical potentials arise due to the permanent potential

difference of between 10 to 30mV that exists between the cornea and the ocular fundus. This is commonly referred to as the cornea-retinal potential with the cornea being positive. An electrical field is set up in the tissues surrounding the eye and rotation of the eye causes a corresponding rotation of the field vector. For this reason, it is possible to detect eye movement with the appropriate placement of electrodes on the skin surrounding the eyes.

The EOG is one of the very few methods for recording eye movements that does not require a

direct attachment to the eye itself. For this reason, the EOG technique is preferred for recording eye movements in sleep and dream research and when recording eye movements in infants. Recently, this technique has become popular for evaluating reading ability and visual fatigue of subjects. One specific type of EOG measurements is the electronystagmograhy (ENG). ENG measurements are used to measure a condition called nystagmus. Measurement of nystagmus, a characteristic pattern of eye movement, is invaluable in the diagnosis of various vestibular and balance dysfunctions.

For instance Figure 1.2 illustrates the measurement of horizontal eye movements by the placement of a pair of electrodes at the outside of the left and right eye (outer canthi). With the eye at rest the electrodes are effectively at the same potential and no voltage is recorded. The rotation of the eye to the right results in a difference of potential, with the electrode in the direction of movement (i.e., the right canthus) becoming positive relative to the second electrode. (Ideally the difference in potential should be proportional to the sine of the angle.) The opposite effect results from a rotation to the left, as illustrated. The calibration of the signal may be achieved by having the patient look consecutively at two different fixation points located a known angle apart and recording the appropriate EOGs. Typical signal magnitudes range from 5-20 µV/°.

Figure 1.2: An illustration of the electro-oculogram (EOG). The signal generated by horizontal movement of the eyes.

The polarity of the signal is positive at the electrode to which the eye is moving.

An additional use of the EOG measurement system that is not directly associated with eye movement is a clinical test for retinal dysfunction.

The EOG is not a very stable signal and measurements can vary as a result of varying ambient light conditions. By having a patient carries out eye movements of constant amplitude in the dark and then in the light, any change in the biopotential would reflect a change in the corneal-retinal

9

potential. In normal eyes, this potential decreases during dark adaptation, and increases during light adaptation. This change arises mainly from the photoreceptor-retinal pigment epithelium complex. Thus, such a test is sensitive to conditions which affect these structures, such as retinitis pigmentosa, choroiditis, and retinal detachment.

Taking these factors into account, it is generally assumed for the purposes of analyzing and interpreting EOG recordings that the potential measured is linear to the movement of the eye within its orbit. When the above external factors are minimized and interference from biological signals such as EEG, EMG and ECG are appropriately filtered out, a resolution of about 1 degree can be achieved within the range ± 30 degrees for horizontal movement.

Electro-Oculography: Electrode placement:

EOG is based on electrical measurement of the potential difference between the cornea and the retina. This is about 10 mV under normal circumstances. The Cornea-retinal potential creates an electrical field in the front of the head. This field changes in orientation as the eyeballs rotate. The electrical changes can be detected by electrodes placed near the eyes. In clinical practice, the detected voltage changes are amplified and used to drive a plotting device, whereby a tracing of eye position is obtained.

It is possible to obtain independent measurements from the two eyes. However, the two eyes move in conjunction in the vertical direction. Hence it is sufficient to measure the vertical motion of only one eye together with the horizontal motion of both eyes. This gives rise to the four channel recording system shown next.

Figure 1.3: Placement of Transducer Pickups to Measure Eye Movements

10

Our eyes need to move in order to keep the image of whatever we are interested in at the central part (called the fovea) of the retina. Thus the act of "taking in" a visual scene consists of fixating (moving the fovea to image) all the objects in the scene that interest us. Human ocular movement has been widely studied in neurophysiology and psychology. These studies indicate that there are four types of eye movements, called vestibular, opto-kinetic, saccadic, and pursuit. The first two have to do with the largely involuntary head motion. The saccadic movement is used to "jump" from one object of interest to another. This is the fastest type of eye movement. The pursuit movement is used to maintain fixation on a moving object.

If the orientation of the eyes is measured, it is possible to locate the 3D position of a fixated

target object by triangulation. The accuracy of the location determination depends on the accuracy with which the eye orientation is determined. In this respect, EOG cannot compete with the direct reflectance methods. In EOG, a quantitative estimate of positional accuracy can be based on the observation that there is a change in potential of about 5-20 micro volt for every degree of change in eye orientation in either direction. Thus accuracy and resolution are determined by the sophistication of the electronic circuitry (and hence also its cost) built to amplify and condition this signal. The EOG technique cannot measure head movement directly. However, using the EOG signal we can estimate the distance of the fixated point from the head of the user. This might enable the system to determine that the user is fixating at a point outside the computer display.

By placing electrodes superior and inferior to the orbit of each eye, and a reference electrode

lateral to the eye of interest, vertical eye movements can also be measured. When a test subject is gazing straight ahead, the corneal-retinal dipole is symmetric between the two electrodes, and measured EOG output is zero. As the subject gazes to the left, the cornea becomes closer to the left lateral electrode, therefore causing the EOG output to become more positive. The inverse of this is true when the subject looks in the right direction.

As mentioned before, when measuring the EOG output, there is a fairly linear relationship between the horizontal angle of gaze and the EOG output. This relationship remains true up to approximately thirty degrees of arc.

11

Chapter 2: Hardware in use is PC-ECG 1200 The software that was built in this project was built to work with NORAV PC-ECG 1200, the only device which was available for us and is pictured here.

Figure 2.1: PC-ECG 1200S – used for cardiology imaging systems.

Technical Specifications

model Feature 1200B 1200S 1200M size [cm] 15x12x2 20x14x3.5 15x8x3 weight [gram] 250 (including belt/pouch) 500 200 ECG samples/sec 250, 500, 1000, 2000 A/D bits 12 ( 2.44 µV/LSB) defibrillation protection built in with cable C1-d simultaneously 12L yes CMMR > 100 dB Input Impedance > 100Mohm Signal dynamic range 10mV DC max. input + or - 330mV Frequency Range (-3db) 0.05 - 300 Hz Low Pass Filter (Software) 35 Hz Line noise Filter (software) 50/60Hz Safety Standard IEC 601-1 , IEC 601-2-25 , IEC 601-2-27

This device has 2 input channels with 4 electrodes marked by white electrode (Common), black electrode (Source 1), red electrode (Source 2) and green electrode (Ground). PC-ECG connection:

1. Black electrode (source 1) is connected to the red electrode of the signal generator. The rest of the electrodes are connected to the black electrode of the signal generated and thus are grounded.

2. Black electrode of the PC-ECG is places on the right side of the face (R). Red electrode is placed on the left side (L). White electrode is placed at the center (C). Green electrode is grounded. In practice, we have observed that connecting it to a hand finger, on the nail, close to the skin gives satisfactory results.

We use the sample frequency of 2000 [samples/sec].

12

Chapter 3: Hardware deficiency. The equipment we had for acquiring EOG data was NORAV PC-ECG 1200. At a very early stage

of the project we began making records of EOG signals generated by eye movement using this equipment for amplification and 12 bit/2kHz sampling of those signals. The results showed the following:

0 1 2 3 4 5 6 7 8 9

x 104

-200

-150

-100

-50

0

50

100

150

200

Figure 3.1: Initial output signal from the eyes

PC-ECG System analysis:

Figure 3.1 shows the signal generated by eye movement. The signal value of 150 indicating that the person is looking to the right and the value of -150 indicating the person is looking to the left, signal value of zero corresponds to a straight look. The sample frequency is 2 kHz which means that this record duration is approximately 45 seconds. It's easy to see the signal is exponentially fading towards zero. The first question we asked ourselves was whether this phenomenon was originated in the OEG signal or it's the side affects of our acquisition process, meaning that the PC-ECG equipment distorts the signal. To check that we connected a signal generator that was producing a "square sine" and recorded the results.

0 10 20 30 40 50 60 70 80 90-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

Time [sec]

Am

plitu

de [%

of F

ull S

cale

]

Figure 3.2: Initial output signal from the signal generator

13

0 5 10 15 20 25 30 35 40 45-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Time [sec]

Am

plitu

de [%

of F

ull S

cale

]

Figure 3.3

The last two figures display the results recorded on the computer when a "square sine" from a signal

generator was injected into the PC-ECG. Our conclusions are summarized in the next block diagram.

)(tx )(ty

)(th

Figure 3.4: PC-ECG Impulse response

Where )(ty is the step response of the PC-ECG system.

Consequently the impulse response that describes the system is )()( tydtdth = .

0 ,)( )()( )()(

1)( )()(>

+=⇒

⋅=⇒=

+=⇒⋅= ⋅−

αα

αα

sssH

sYssHtydtdth

ssYtuety t

14

This simple analysis shows that the PC-ECG system has a zero in the origin, which indicates that

such system can not be used to measure DC signal. For 1=α , the frequency response of PC-ECG system will looks like this:

-60

-50

-40

-30

-20

-10

0

Mag

nitu

de (d

B)

10-2

10-1

100

101

102

0

45

90

Phas

e (d

eg)

Bode Diagram

Frequency (rad/sec)

Figure 3.5: Frequency response of PC-ECG for 1=α

As you can see from this figure the DC amplification would be dBjwHw

)(0

−∞==

. Considering the fact that the EOG signal that we wish to measure is basically a DC signal (as shown in Figure 3.6) we have realized that our acquisition equipment (PC-ECG system) is completely inappropriate for this project! In a few words I would like to describe the behavior of an appropriate system that should have been used for EOG measurement:

)(tx )(ty

)(tg

Figure 3.6: Impulse response of appropriate system for EOG measurement

In such system as described in last figure we would have:

0 ,1)( >+

= ααs

sG

15

Consequently we will get the following frequency response for 1=α :

-40

-30

-20

-10

0

Mag

nitu

de (d

B)

10-2

10-1

100

101

102

-90

-45

0

Phas

e (d

eg)

Bode Diagram

Frequency (rad/sec)

Figure 3.7: Frequency response of appropriate system for EOG measurement

Such system can be used for DC signals acquisition because dBjwGw

0)(0

==

, but unfortunately such system was not available for us in this project.

16

Chapter 4: The Software solution

The only solution that was acceptable was to try to build a correction system in software, which will neutralize the distortion made by the PC-ECG system. The following diagram describes the correction process:

)( tx )(ty

sT

)(1 zH −

][~ kx

)(sH

Figure 4.1: The correction process done by the software

So basically we are looking for the inverse system of PC-ECG - )(1 sH − and then we would have to

find its discrete equivalent to be implemented in the software.

∫∞−

⋅⋅+=

+=+

=⇒>+

=

t

dytytx

ssssH

sssH

ττα

ααα

α

)()()(~

1)( 0 ,)( 1

After converting to discrete time by using the rectangle approximation, we get:

ST

∑−∞=

⋅⋅+=k

ns nyTkykx ][][][~ α

Figure 4.2: Discrete Inverse system

The expression on Figure 4.2 basically defines the action of the Discrete Inverse System show on

Figure_4.1, which is theoretically supposed to cancel the signal distortion made by the PC-ECG system.

17

After implementing the Inverse System in MATLAB we got the following results:

0 10 20 30 40 50 60 70 80 90-1

-0.5

0

0.5

1

Time [sec]

Am

plitu

de [%

of F

ull S

cale

]

0 10 20 30 40 50 60 70 80 90-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

Time [sec]

Am

plitu

de [%

of F

ull S

cale

]

Figure 4.3: Inverse system in Matlab

0 5 10 15 20 25 30 35 40 45-1

-0.5

0

0.5

1

Time [sec]

Am

plitu

de [%

of F

ull S

cale

]

0 5 10 15 20 25 30 35 40 45-1.2

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

Time [sec]

Am

plitu

de [%

of F

ull S

cale

]

Figure 4.4

18

Those two last figures show the signal generated by a signal generator and distorted by PC-ECG on

top, and on the bottom the signal after it was corrected according to formula on figure 4.2.

where 0005.02000

11===

ss F

T and α have empirical value of 3225.0≈α .

As you can see the Software Inverse System implemented in MATLAB was able to bring the signal back to its original "Square Sine" form, which was produced by the Signal Generator. The drift phenomenon:

Although by using the Software Inverse System we are able to correct the signal to its original form, the entire system is not stable! The PC-ECG system )(sH has a zero in origin, and )(1 zH − has a pole in origin which supposed to cancel the unwanted affect of the zero. The problem is that a system with a pole in the origin can be unstable, because in reality the zero and the pole are not necessarily cancel each other (in practice they are not exactly in the origin) and you get an unstable pole.

The drift phenomenon can be seen on figure 4.3, when at the end of the record (after ~80 seconds) the corrected signal doesn't return to zero value as it supposed to, instead its drifted down to negative value. In general after enough time has passed since the beginning of the acquisition the corrected signal won’t stay centralized around zero, instead it will be drifted towards infinity. Factors that affect the drift:

• The discrete integration is only an approximation and it's important to understand that this

action causes an accumulative error. The following figure displays the drift caused by accumulative error.

Figure 4.5: First drift phenomenon

This figure is a screenshot of the final software that we developed in this project for real-time EOG acquisition and analysis, the blue graph on the bottom shows the corrected signal as it is being drifted towards higher values. This drift is a side affect of the Software Inverse System.

Drift upwards

19

• Another problem that can cause the drift of the corrected signal is the fact the original

signal that is being recorded is not absolutely centralized around zero, but instead it has a small DC shift value. In this case the "RAW signal" (the one that comes out of the PC-ECG system, shown in red at the top of the figure) will stabilize on value different from zero, which will cause the integration result to diverge. The following figure displays the drift caused by a DC shift of the original signal.

Figure 4.6: Second drift phenomenon

On this screenshot from the final software you can see that the raw signal had stabilized on the value of -1 instead of zero, consequently the corrected signal is drifted towards minus infinity.

• The last factor that affects the accumulative error is the unknown value of α . Considering the fact that we characterized the PC-ECG system through experiments and measurement,

we only have the empirical value of α (PC-ECG system described by α+

=s

ssH )( ).

Consequently the Inverse System that we implemented in software is only an approximation to the real one, this of course causes an error.

Drift downwards

Non zero

20

Solutions for the drift problem:

• To cancel the affect of cumulative error the corrected signal maximum and minimum values should be constantly monitored in order to keep the corrected signal centralized around zero. The following figure displays this kind of periodic centralization applied to the corrected signal.

Figure 4.7: Correcting the first drift phenomenon

The little steps on the blue signal that marked with green circles are not real changes in the recorded signal, but software corrections which are done in order to cancel the drift and keep the corrected signal centralized.

21

• In order to cancel the problem caused by DC shift in the original signal, the software monitors the raw signal, once it is stabilized on a non zero value a DC shift is being made, that causes the raw signal to stay stable only on zero value. The following figure demonstrates this scenario.

Figure 4.8: Correcting the second drift phenomenon

The little correction which are marked by green circles are being made to the raw signal (red) forcing it to stabilize on zero value, and by doing so we prevent the constant drift of the corrected signal (blue).

Zero

22

Chapter 5: Implementation of the Software

The Horizontal Eye Tracker software we have developed was written in Visual Basic. Its main form is shown next.

Figure 5.1: Horizontal Eye Tracker Main Form • The "Raw Signal" drawn in red is the input signal to HET and its discrete value appears in the box

under its name called "Raw Signal Level". It can be one of the following:

1. Sigen – Electronic pulses generated by the Signal Generator which is connected to the 2 electrodes of the PC-ECG (according to specification in Chapter 2) and this is connected to the PC with HET software. Before drawing the initial signal it's first attenuated (divided by 10, because the amplitude of the artificial input signal generated by the signal generator is too high comparing with the signals originated from the eyes.) and filtered using LPF with 1000 samples length. The filter ensures the initial signal won't be defected by noises.

2. Eyes – The electronic pulses originated from the eyes of the tested subject are measured and

sampled by the PC-ECG and transferred to HET software. As in the previous case before drawing the signal in "Raw Signal" HET will filter it using LPF and won't attenuate it.

23

• The "Corrected Signal" drawn in Blue is the output signal of HET and its discrete value appears in the box "Corrected Signal Level" under its name. As it was explained earlier in Chapter 3 this is the signal we have expected to get from the PC-ECG in the first place. It is corrected to "square" structure by HET, as explained in the previous chapter. Using this output signal HET marks the current observation point of the eyes with red light on the horizontal scale located at the bottom of the main form. In order to achieve satisfactory results we preferred to work with three observation points: Center, Right and Left that appear on the horizontal scale.

• The "display length" option enables to define the decimation factor of the signal's display. • Correction Counter counts the number of the centralization corrections HET made to the

Corrected signal. In order to achieve the most accurate results for the observation point it is necessary to adjust HET

before every test using the following tools:

DC shift threshold: This value determines the maximum range in which the raw signal can change and still

considered stable by HET. As it was explained earlier in order HET to detect the changes in the signal properly and corresponding to the eye movements, it's important to detect when the Raw Signal is stabilized, meaning the subject looks at the same point.

Position Threshold: Defines the range of the Corrected signal values which considered by HET to correspond to the central observation point. This value varies from 1 to 50. For example when it chosen to be 9 it means whenever the Corrected signal is less then 9 or more than -9 HET determines the current observation point as Center. If the corrected signal is more than 9 then HET determines the observation point as Right. If the corrected signal is less than -9 then HET determines the observation point as Left. Window Size: The time frame (in seconds), which is used for calculating the maximum and the minimum values of the corrected signal for centralization purposes. For example choosing 15 seconds in Window size will set the time frame in which the corrected signal will be tested for centralization is from the current point and backward.

Correct Threshold: Enables the user to define the full movement range. This value varies from 0 to 100. In order HET to work properly the corrected signal must be centralized around zero. Window size determines the time frame in which HET finds the maximum and the minimum of the corrected signal. If the difference between them is more than the Correct Threshold defined by the user then in this time frame a full eye movement was made (from right to left or from left to right) and only then centralization is made. This difference is divided by 2 and compared to zero, if the result is different than zero a suitable correction is made and the centralization is obtained.

Correct Frequency: Define the time period in which the need for centralization is checked.

24

Chapter 6: Final results

In this chapter we have chosen to present several screen shots of the results obtained by the implemented software. We have especially chosen not only successful recordings of both eye signals and artificial signals. But also show unsuccessful recordings that were made when HET wasn't adjusted properly.

Example 1 - Good sigen recording:

Figure 6.1: Good generated signal ("sigen") recording

In this case the input signal was created by the signal generator. The "Start" button was clicked and the recording has begun. This scenario simulates slow movements of the eyes. At first the raw signal was at zero as if the observation point was at the center, then a movement to the left was made. As it can be seen from the red signal which is the output of the hardware device a distortion was made to the rectangular Sine created by the signal generator. The real input signal doesn't change as the red signal but stays in the same level as the corrected signal in blue. Also a centralization of the corrected signal is made in order a proper perception between the center and the left be made.

After several seconds we have increased the frequency of the signal created by the signal

generator in order to simulate a scenario of rapid full eye movements (left-right-left). As it can be seen from the corrected signal its behavior is relatively characteristic of eye movements and thus HET follows truly after the eyes of the tested subject.

25

Example 2: Bad sigen recording: In this case the DC Shift Threshold wasn't chosen adjectively.

Figure 6.2: Bad generated signal ("sigen") recording

As it can be seen from the picture, the DC Shift Threshold was chosen low then needed in order

to correct the raw signal correctly. The initial raw signal level was more than 3 so the correction to zero of the raw signal wasn't made. Although centralization is made and thus the corrected signal doesn't increases to infinity as it can be see from the fast and short changes in the corrected signal. But still this corrected signal can't simulate real eye movements and the detection HET will do can't be satisfactory. The conclusion derived from this simulation is that the DC Shift Threshold should be higher for this specific input signal created by the signal generator.

26

Example 3: Improved sigen recording:

This shot taken after the DC Shift Threshold was changed to 10. The tuning of the software in such manner provides satisfactory results for the specific input signal made by the signal generator. The new corrected signal can simulate real eye movements by which correct perception by HET is achieved.

Figure 6.3: Improved "sigen" recording

27

Example 4: Successful recording of Eye movements: The input signal to our hardware in this case is taken directly from human eyes. As it can be seen

from the screen shot HET succeeds to correct the raw signal and achieve satisfactory results. The corrected signal has the characteristic "steps" structure. This is achieved due to correct centralization and accurate recognition of the segments in which the raw signal is stable.

Figure 6.4: First successful recording of Eye movements

Figure 6.5: Second successful recording of Eye movements

28

Summary

The initial goal of this project was to build a simple detecting system which will allow the user to control a machine using only his eyes. The system had to detect and to distinguish between vertical and horizontal eye movements. In the first step of the design of such system we learned and analyzed the EOG measured by the only hardware device available to us. Due to the crucial deficiency discovered by us in the hardware it was decided to change the main task of our project.

We have built the recognition system using the available hardware to which we implemented appropriate software that enables horizontal detection and satisfactory recognition of human eye movements.

In order to do that we first studied the EOG method with which eye signals obtained and summarized that in the first chapter of this book. Then, we worked and made suitable experiences with our hardware device as described in the second chapter. These observations showed inconclusively that our hardware device is not suitable for this project. As it's widely explained and pictured in chapters four and five we overcame this basic discrepancy by implementing appropriate software to our hardware. This software (HET) receives the signal distorted by the hardware device and corrects it according to the parameters determined by the user. Both signals are exhibited in the main form of HET during the recording session. This enables the user to adjust HET in order to achieve the best correction available. Still, HET can't correct the distorted signal entirely and thus can't obtain an accurate EOG signal. The unsuitability between the hardware available and the initial goal of the project restricted significantly the options available to us. Thus, the recognition available by this device is strictly horizontal or strictly vertical depends on the connection of the device's electrodes.

HET functions and interface are shown in chapter five. Its results are presented in chapter six. These results show that using HET for tracking and recognition of eye movements is limited and

can be used as part of future projects. Future projects in which distinction of eye movements is made require a suitable hardware device as proposed next.

Picture 1: Proposed ML317 EOG Pod device

29

Picture 2: Data obtained with proposed hardware device

In general although the initial goals of our project could not have been achieved due to hardware limitation, we tried to make the best from this work and we feel it was very useful experience. And most importantly we enjoyed doing it.

30

Appendix

Picture 3: Improving advertisements with eye-movement research

Picture 4: Visual scan paths on instruments/dashboards – studies for the improvement of human-computer interfaces