blue eye report
TRANSCRIPT
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 1/42
ABSTRACT
Is it possible to create a computer, which can interact with us as we
interact each other? For example imagine in a fne morning you wal on to
your computer room an! switch on your computer, an! then it tells you
"#ey $rien!, goo! morning you seem to be a ba! moo! to!ay% An! then it
opens your mail box an! shows you some o$ the mails an! tries to cheer
you% It seems to be a fction, but it will be the li$e lea! by "B&'( ()(S*
in the +ery near $uture%
The basic i!ea behin! this technology is to gi+e the computer the
human power% e all ha+e some perceptual abilities% That is we can
un!erstan! each other-s $eelings% For example we can un!erstan! ones
emotional state by analy.ing his $acial expression% I$ we a!! these
perceptual abilities o$ human to computers woul! enable computers to
wor together with human beings as intimate partners% The "B&'( ()(S*
technology aims at creating computational machines that ha+e perceptual
an! sensory ability lie those o$ human beings%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
C0/T(/TS
6%
I/TR02'CTI0/%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%6
7%
S)ST(4 03(R3I(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%7
7%6%
T#( #ARAR(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%8
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 2/42
7%6%6%
2ATA AC9'ISITI0/ '/IT%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%8
7%6%7%
C(/TRA& S)ST(4 '/IT%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%:
7%7%
T#( S0FTAR(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%;
8%
(40TI0/ C045'TI/1%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%<
8%6%
T#(0R)%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%<
8%7%
R(S'&TS%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%=
:%
T)5(S 0F (40TI0/ S(/S0RS%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%6>
:%6%
#A/2%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%6>
:%6%6%
(40TI0/ 40'S(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%6>
:%6%7%
S(/TIC 40'S(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%68
:%7%
()(%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%68
:%7%6%
(5R(SSI0/ 1&ASS(S%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%68
;%
4A/'A& A/2 1A@( I/5'T CASCA2(2 4A1IC 50I/TI/1%%%%%%%%%%6:
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 3/42
;%6%
I45&(4(/TATI0/%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%78
;%7%
T#( IB4 A&4A2(/ ()( TRAC(R%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%78
;%8%
I45&(4(/TI/1 4A1IC 50I/TI/1%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%7;
;%:%
(5(RI4(/T%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%7D
;%;%
(5(RI4(/TA& 2(SI1/%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%7D
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
;%D%
(5(RI4(/TA& R(S'&TS%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%7D
D%
ARTIFICIA& I/T(&&I1(/T S5((C# R(C01/ITI0/%%%%%%%%%%%%%%%%%%%%%%%%%%%86
D%6%
T#( T(C#/0&01)%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%86
D%7%
S5((C# R(C01/ITI0/%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%87
D%8%
A55&ICATI0/S%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%88
E%
T#( SI45&( 'S(R I/T(R(ST TRAC(R S'IT0R%%%%%%%%%%%%%%%%%%%%%%%%%%%%%8:
<%
C0/C&'SI0/%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%8;
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 4/42
=%
R(F(R(/C(S%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%8D
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/16
6% I/TR02'CTI0/
Imagine yoursel$ in a worl! where humans interact with computers% )ou are
sitting in $ront o$ your personal computer that can listen, tal, or e+en scream alou!%
It
has the ability to gather in$ormation about you an! interact with you through
special
techniues lie $acial recognition, speech recognition, etc% It can e+en un!erstan!
your
emotions at the touch o$ the mouse% It +erifes your i!entity, $eels your presents,
an!
starts interacting with you %)ou ass the computer to !ial to your $rien! at his oGce%
It
reali.es the urgency o$ the situation through the mouse, !ials your $rien! at his
oGce,
an! establishes a connection%
#uman cognition !epen!s primarily on the ability to percei+e, interpret, an!
integrate
au!ioH+isuals an! sensoring in$ormation% A!!ing extraor!inary perceptual abilities to
computers woul! enable computers to wor together with human beings as intimate
partners% Researchers are attempting to a!! more capabilities to computers that
will
allow them to interact lie humans, recogni.e human presents, tal, listen, or e+en
guess
their $eelings%
The B&'( ()(S technology aims at creating computational machines that ha+e
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 5/42
perceptual an! sensory ability lie those o$ human beings% It uses nonHobtrusi+e
sensing
metho!, employing most mo!ern +i!eo cameras an! microphones to i!enti$y the
user-s
actions through the use o$ imparte! sensory abilities% The machine can un!erstan!
what
a user wants, where he is looing at, an! e+en reali.e his physical or emotional
states%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
7
7% S)ST(4 03(R3I(
Blue eyes system monitors the status o$ the operator-s +isual attention through
measurement o$ sacca!ic acti+ity% The system checs parameters lie heart beat
rate an!
bloo! oxygenation against abnormal an! triggers user !efne! alarms%
Blue(yes system consists o$ a mobile measuring !e+ice an! a central analytical
system%
The mobile !e+ice is integrate! with Bluetooth mo!ule pro+i!ing wireless inter$ace
between sensors worn by the operator an! the central unit% I2 car!s assigne! to
each o$
the operators an! a!euate user profles on the central unit si!e pro+i!e necessary
!ata
personali.ation so the system consists o$
4obile measuring !e+ice 2A'
Central System 'nit CS'
Fig% System 0+er+iew
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 6/42
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
8
The o+erall System !iagram is as $ollowsJH
Fig% System 2iagram
7%6% T#( #AR2AR(
7%6%6% 2ata Acuisition 'nit
2ata Acuisition 'nit is a mobile part o$ the Blue eyes system% Its main tas is
to $etch the physiological !ata $rom the sensor an! to sen! it to the central system
to be
processe!% To accomplish the tas the !e+ice must manage wireless Bluetooth
connections connection establishment, authentication an! termination% 5ersonal
I2
car!s an! 5I/ co!es pro+i!e operatorKs authori.ation% Communication with the
operator is carrie! on using a simple ;Hey eyboar!, a small &C2 !isplay
an! a
beeper% hen an exceptional situation is !etecte! the !e+ice uses them to noti$y
the
operator% 3oice !ata is trans$erre! using a small hea!set, inter$ace! to the 2A'
with
stan!ar! miniHLac plugs%
The 2ata Acuisition 'nit comprises se+eral har!ware mo!ules
Atmel <=C;7 microcontroller H system core
Bluetooth mo!ule base! on R06>6>><
#2::E<> H small &C2 !isplay
7:C6D H I7C ((5R04 on a remo+able I2 car!
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
:
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 7/42
4C6:;:<8 M 68bit 5C4 co!ec
Na.. 4ultisensor inter$ace
Beeper an! &(2 in!icators ,D AA batteries an! +oltage le+el monitor
Fig% 2A' Components
7%6%7% Central System 'nit
Central System 'nit har!ware is the secon! peer o$ the wireless
connection% The box contains a Bluetooth mo!ule base! on R06>6>>< an! a 5C4
co!ec $or +oice !ata transmission% The mo!ule is inter$ace! to a 5C using a parallel,
serial an! 'SB cable% The au!io !ata is accessible through stan!ar! miniHLac
socets
To program operatorKs personal I2 car!s we !e+elope! a simple programming
!e+ice% The programmer is inter$ace! to a 5C using serial an! 5SO7 power source
ports% Insi!e, there is Atmel <=C7>;6 microcontroller, which han!les 'ART
transmission an! I7C ((5R04 I2 car! programming%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
;
Fig% CS' Components
7%7 T#( S0FTAR(J
Blue (yes so$tware-s main tas is to loo a$ter woring operatorsK physiological
con!ition% To assure instant reaction on the operatorsK con!ition change the
so$tware per$orms real time buPering o$ the incoming !ata, realHtime
physiological
!ata analysis an! alarm triggering%
The Blue (yes so$tware comprises se+eral $unctional mo!ules System core
$acilitates the trans$ers Qow between other system mo!ules e%g% trans$ers raw
!ata
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 8/42
$rom the Connection 4anager to !ata analy.ers, processe! !ata $rom the !ata
analy.ers to 1'I controls, other !ata analy.ers, !ata logger etc%% The System
Core
$un!amental are singleHpro!ucerHmultiHconsumer threa! sa$e ueues% Any number
o$
consumers can register to recei+e the !ata supplie! by a pro!ucer% (+ery single
consumer can register at any number o$ pro!ucers, recei+ing there$ore
!iPerent
types o$ !ata% /aturally, e+ery consumer may be a pro!ucer $or other consumers%
This
approach enables high system scalability M new !ata processing mo!ules i%e% flters,
!ata analy.ers an! loggers can be easily a!!e! by simply registering as a
costumer%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
D
Connection 4anager is responsible $or managing the wireless communication
between the mobile 2ata Acuisition 'nits an! the central system% The
Connection 4anager han!lesJ
communication with the CS' har!ware
searching $or new !e+ices in the co+ere! range
establishing Bluetooth connections
connection authentication
incoming !ata buPering
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 9/42
sen!ing alerts
2ata Analysis mo!ule per$orms the analysis o$ the raw sensor !ata in or!er
to
obtain in$ormation about the operator-s physiological con!ition% The separately
running 2ata Analysis mo!ule super+ises each o$ the woring operators%
The mo!ule consists o$ a number o$ smaller analy.ers extracting !iPerent
types
o$ in$ormation% (ach o$ the analy.ers registers at the appropriate 0perator
4anager or
another analy.er as a !ata consumer an!, acting as a pro!ucer, pro+i!es the
results
o$ the analysis% The most important analy.ers areJ
Sacca!e !etector H monitors eye mo+ements in or!er to !etermine the le+el o$
operatorKs +isual attention
5ulse rate analy.er H uses bloo! oxygenation signal to compute operatorKs
pulse rate
Custom analy.ers H recogni.e other beha+iors than those which are builtHin the
system% The new mo!ules are create! using C:%; !ecision tree in!uction
algorithm
3isuali.ation mo!ule pro+i!es a user inter$ace $or the super+isors% It enables themto
watch each o$ the woring operator-s physiological con!ition along with a pre+iew
o$
selecte! +i!eo source an! relate! soun! stream% All the incoming alarm
messages are
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 10/42
instantly signale! to the super+isor% The 3isuali.ation mo!ule can be set in an
oPH
line mo!e, where all the !ata is $etche! $rom the !atabase% atching all the
recor!e! physiological parameters, alarms, +i!eo an! au!io !ata the super+isor is
able to
reconstruct the course o$ the selecte! operator-s !uty% The physiological !ata is
presente! using a set o$ customHbuilt 1'I controlsJ
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
E
a pieHchart use! to present a percentage o$ time the operator was acti+ely
acuiring the +isual in$ormation
A 3'Hmeter showing the present +alue o$ a parameter time series !isplaying a
history o$ selecte! parametersK +alue%
Fig%; So$tware Analysis 2iagram
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
<
8% (40TI0/ C045'TI/1
Rosalin! 5icar! 6==E !escribes why emotions are important to the
computing community% There are two aspects o$ aPecti+e computingJ gi+ing the
computer the ability to !etect emotions an! gi+ing the computer the ability to
express
emotions% /ot only are emotions crucial $or rational !ecision maing as 5icar!
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 11/42
!escribes, but emotion !etection is an important step to an a!apti+e computer
system%
An a!apti+e, smart computer system has been !ri+ing our ePorts to !etect a
person-s
emotional state% An important element o$ incorporating emotion into computing is
$or
pro!ucti+ity $or a computer user% A stu!y 2ryer #orowit., 6==E has shown that
people with personalities that are similar or complement each other collaborate
well%
2ryer 6=== has also shown that people +iew their computer as ha+ing a
personality%
For these reasons, it is important to !e+elop computers which can wor wellwith
its user%
8%6 T#(0R)
Base! on 5aul (man-s $acial expression wor, we see a correlation
between a person-s emotional state an! a person-s physiological measurements%
Selecte! wors $rom (man an! others on measuring $acial beha+iors !escribe
(man-s
Facial Action Co!ing System (man an! Rosenberg, 6==E% 0ne o$ his
experiments
in+ol+e! participants attache! to !e+ices to recor! certain measurements
inclu!ing pulse, gal+anic sin response 1SR, temperature, somatic
mo+ement
an! bloo! pressure% #e then recor!e! the measurements as the participants were
instructe! to mimic $acial expressions which correspon!e! to the six basicemotions% #e
!efne! the six basic emotions as anger, $ear, sa!ness, !isgust, Loy an! surprise%
From
this wor, 2ryer 6==8 !etermine! how physiological measures coul! be use! to
!istinguish +arious emotional states% The measures taen were 1SR, heart rate, sin
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 12/42
temperature an! general somatic acti+ity 1SA% These !ata were then subLect to
two
analyses% For the frst analysis, a multi!imensional scaling 42S proce!ure was
use! to
!etermine the !imensionality o$ the !ata%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
=
8%7 R(S'&T
The !ata $or each subLect consiste! o$ scores $or $our physiological
assessments 1SA, 1SR, pulse, an! sin temperature, $or each o$ the six
emotions
anger, !isgust, $ear, happiness, sa!ness, an! surprise across the f+e
minute
baseline an! test sessions% 1SA !ata was sample! <> times per secon!, 1SR an!
temperature were reporte! approximately 8H: times per secon! an! pulse was
recor!e!
as a beat was !etecte!, approximately 6 time per secon!% To account $or
in!i+i!ual +ariance in physiology, we calculate! the !iPerence between the baseline
an!
test scores% Scores that !iPere! by more than one an! a hal$ stan!ar! !e+iations
$rom the
mean were treate! as missing% By this criterion, twel+e score were remo+e! $rom
the
analysis% The results show the theory behin! the (motion mouse wor is
$un!amentally
soun!% The physiological measurements were correlate! to emotions using a
correlation mo!el% The correlation mo!el is !eri+e! $rom a calibration process in
which a baseline attributeHto emotion correlation is ren!ere! base! on statistical
analysis o$ calibration signals generate! by users ha+ing emotions that
are
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 13/42
measure! or otherwise nown at calibration time%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/16>
:% T)5(S 0F (40TI0/ S(/S0RS
For #an!J
(motion 4ouse
Sentic 4ouse
For (yesJ
(xpression 1lasses
4agic 5ointing
(ye Tracing
For 3oiceJ
Artifcial Intelligence Speech Recognition
:%6 #A/2
:%6%6 (motion 4ouse
Fig%D (motional 4ouse
0ne propose!, nonUin+asi+e metho! $or gaining user in$ormation through touch
is +ia a computer input !e+ice, the mouse% This then allows the user to relate the
car!iac
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 14/42
rhythm, the bo!y temperature, electrical con!ucti+ity o$ the sin an! other
physiological
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
66
attributes with the moo!% This has le! to the creation o$ the "(motion 4ouse*% The
!e+ice can measure heart rate, temperature, gal+anic sin response an! minute
bo!ily
mo+ements an! matches them with six emotional statesJ happiness, surprise,
anger, $ear,
sa!ness an! !isgust% The mouse inclu!es a set o$ sensors, inclu!ing in$rare!
!etectors
an! temperatureHsensiti+e chips% These components, 'ser researchers- stress, will
also
be cra$te! into other commonly use! items such as the oGce chair, the steering
wheel,
the eyboar! an! the phone han!le% Integrating the system into the steering wheel,
$or
instance, coul! allow an alert to be soun!e! when a !ri+er becomes !rowsy%
+
In$ormation 0btaine! From (motion 4ouse
6
Beha+ior
a%
4ouse mo+ements
b%
Button clic $reuency
c%
Finger pressure when a user presses hisOher button
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 15/42
7
5hysiological in$ormation
a%
#eart rate (lectrocar!iogram (C1O(1,
5hotoplethysmogram 551
b%
Sin temperature Thermester
c%
Sin electricity 1al+anic sin response, 1SR
!%
(lectromyographic acti+ity (lectromyogram, 41
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
67
+
5rototype
Fig%E System Confguration $or (motional 4ouse
+
Samples 0btaine! From (motional 4ouse
Fig%< 2iPerent Signals
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/168
:%6%7 Sentic 4ouse
It is a mo!ife! computer mouse that inclu!es a !irectional pressure sensor $or
ai!ing in recognition o$ emotional +alence liingOattraction +s% !isliingOa+oi!ance%
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 16/42
Fig%= sentic 4ouse
:%7 ()(
:%7%6 (xpression 1lasses
A wearable !e+ice which allows any +iewer to +isuali.e the con$usion an!
interest le+els o$ the wearer% 0ther recent !e+elopments in relate! technology are
the
attempt to learn the nee!s o$ the user Lust by $ollowing the interaction between the
user
an! the computer in or!er to now what heOshe is intereste! in at any gi+en
moment% For
example, by remembering the type o$ websites that the user lins to accor!ing to
the
moo! an! time o$ the !ay, the computer coul! search on relate! sites an! suggest
the
results the user%
Fig%6> (xpression 1lass
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
6:
;% 4A/'A& A/2 1A@( I/5'T CASCA2(2 4A1IC
50I/TI/1
This wor explores a new !irection in utili.ing eye ga.e $or computer input%
1a.e tracing has long been consi!ere! as an alternati+e or potentially superior
pointing
metho! $or computer input% e belie+e that many $un!amental limitations exist
with
tra!itional ga.e pointing% In particular, it is unnatural to o+erloa! a perceptual
channel
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 17/42
such as +ision with a motor control tas% e there$ore propose an alternati+e
approach,
!ubbe! 4A1IC 4anual An! 1a.e Input Casca!e! pointing% ith such an approach,
pointing appears to the user to be a manual tas, use! $or fne manipulation an!
selection% #owe+er, a large portion o$ the cursor mo+ement is eliminate! by
warping the
cursor to the eye ga.e area, which encompasses the target% Two specifc 4A1IC
pointing techniues, one conser+ati+e an! one liberal, were !esigne!, analy.e!,
an!
implemente! with an eye tracer we !e+elope!% They were then teste! in a pilot
stu!y%
This early stage exploration showe! that the 4A1IC pointing techniues might oPer
many a!+antages, inclu!ing re!uce! physical ePort an! $atigue as compare! to
tra!itional manual pointing, greater accuracy an! naturalness than tra!itional ga.e
pointing, an! possibly $aster spee! than manual pointing% The pros an! cons o$ the
two
techniues are !iscusse! in light o$ both per$ormance !ata an! subLecti+e reports%
In our +iew, there are two $un!amental shortcomings to the existing ga.e
pointing techniues, regar!less o$ the maturity o$ eye tracing technology% First,
gi+en
the oneH!egree si.e o$ the $o+ea an! the subconscious Littery motions that the eyes
constantly pro!uce, eye ga.e is not precise enough to operate 'I wi!gets such as
scrollbars, hyperlins, an! sli!er han!les In 5roc% C#I-==J AC4 Con$erence on #uman
Factors in Computing Systems% 7:DH7;8, 5ittsburgh, 6;H7> 4ay6=== Copyright AC4
6=== >H7>6H:<;;=H6O==O>;%%%V;%>> on to!ay-s 1'I inter$aces% At a 7;Hinch +iewing
!istance to the screen, one !egree o$ arc correspon!s to >%:: in, which is twice the
si.e
o$ a typical scroll bar an! much greater than the si.e o$ a typical character%
B&'( ()(S T(C#/0&01)
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 18/42
2I3IS0/ 0F C045'T(R (/1I/((RI/1
6;
Secon!, an! perhaps more importantly, the eye, as one o$ our primary perceptual
!e+ices, has not e+ol+e! to be a control organ% Sometimes its mo+ements are
+oluntarily
controlle! while at other times it is !ri+en by external e+ents% ith the target
selection
by !well time metho!, consi!ere! more natural than selection by blining E, one
has to
be conscious o$ where one loos an! how long one loos at an obLect% I$ one !oes
not
loo at a target continuously $or a set threshol! e%g%, 7>> ms, the target will not be
success$ully selecte!% 0n the other han!, i$ one stares at an obLect $or more than
the set
threshol!, the obLect will be selecte!, regar!less o$ the user-s intention% In some
cases
there is not an a!+erse ePect to a $alse target selection% 0ther times it can be
annoying
an! counterHpro!ucti+e such as uninten!e! Lumps to a web page% Furthermore,
!well
time can only substitute $or one mouse clic% There are o$ten two steps to target
acti+ation% A single clic selects the target e%g%, an application icon an! a !ouble
clic
or a !iPerent physical button clic opens the icon e%g%, launches an application%
To
per$orm both steps with !well time is e+en more !iGcult% In short, to loa! the +isual
perception channel with a motor control tas seems $un!amentally at o!!s withusers-
natural mental mo!el in which the eye searches $or an! taes in in$ormation an!
the
han! pro!uces output that manipulates external obLects% 0ther than $or !isable!
users,
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 19/42
who ha+e no alternati+e, using eye ga.e $or practical pointing !oes not appear to be
+ery
promising%
Are there interaction techniues that utili.e eye mo+ement to assist the control
tas but !o not $orce the user to be o+erly conscious o$ his eye mo+ement? e
wante!
to !esign a techniue in which pointing an! selection remaine! primarily a manual
control tas but were also ai!e! by ga.e tracing% 0ur ey i!ea is to use ga.e to
!ynamically re!efne warp the "home* position o$ the pointing cursor to be at the
+icinity o$ the target, which was presumably what the user was looing at, thereby
ePecti+ely re!ucing the cursor mo+ement amplitu!e nee!e! $or target selection%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
6D
0nce the cursor position ha! been re!efne!, the user woul! nee! to only mae a
small mo+ement to, an! clic on, the target with a regular manual input !e+ice% In
other
wor!s, we wante! to achie+e 4anual An! 1a.e Input Casca!e! 4A1IC pointing, or
4anual Acuisition with 1a.e Initiate! Cursor% There are many !iPerent ways o$
!esigning a 4A1IC pointing techniue% Critical to its ePecti+eness is the
i!entifcation
o$ the target the user inten!s to acuire% e ha+e !esigne! two 4A1IC pointing
techniues, one liberal an! the other conser+ati+e in terms o$ target i!entifcation
an!
cursor placement% The liberal approach is to warp the cursor to e+ery new obLectthe user
loos at See Figure 6%
The user can then tae control o$ the cursor by han! near or on the target, or
ignore it an! search $or the next target% 0perationally, a new obLect is !efne! by
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 20/42
suGcient !istance e%g%, 67> pixels $rom the current cursor position, unless the
cursor is
in a controlle! motion by han!% Since there is a 67>Hpixel threshol!, the cursor will
not
be warpe! when the user !oes continuous manipulation such as !rawing% /ote that
this
4A1IC pointing techniue is !iPerent $rom tra!itional eye ga.e control, where the
user
uses his eye to point at targets either without a cursor or with a cursor that
constantly
$ollows the Littery eye ga.e motion%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
6E
The liberal approach may appear "proHacti+e,* since the cursor waits rea!ily in
the +icinity o$ or on e+ery potential target% The user may mo+e the cursor once he
!eci!es to acuire the target he is looing at% 0n the other han!, the user may also
$eel
that the cursor is o+erHacti+e when he is merely looing at a target, although hemay
gra!ually a!apt to ignore this beha+ior% The more conser+ati+e 4A1IC pointing
techniue we ha+e explore! !oes not warp a cursor to a target until the manual
input
!e+ice has been actuate!% 0nce the manual input !e+ice has been actuate!, the
cursor is
warpe! to the ga.e area reporte! by the eye tracer% This area shoul! be on or in
the
+icinity o$ the target% The user woul! then steer the cursor annually towar!s the
target to
complete the target acuisition% As illustrate! in Figure 7, to minimi.e !irectional
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 21/42
uncertainty a$ter the cursor appears in the conser+ati+e techniue, we intro!uce!
an
"intelligent* bias% Instea! o$ being place! at the center o$ the ga.e area, the cursor
position is oPset to the intersection o$ the manual actuation +ector an! the
boun!ary $
the ga.e area% This means that once warpe!, the cursor is liely to appear in motion
towar!s the target, regar!less o$ how the user actually actuate! the manual input
!e+ice%
e hope! that with the intelligent bias the user woul! not ha+e to 1a.e position
reporte! by eye tracer (ye tracing boun!ary with =;W conf!ence True target will
be
within the circle with =;W probability% The cursor is warpe! to eye tracing position,
which is on or near the true target 5re+ious cursor position, $ar $rom target e%g%,
7>>
pixels Figure 6%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
6<
The liberal 4A1IC pointing techniueJ cursor is place! in the +icinity o$ a
target that the user fxates on% Actuate input !e+ice, obser+e the cursor position
an!
!eci!e in which !irection to steer the cursor% The cost to this metho! is the
increase!
manual mo+ement amplitu!e% Figure 7% The conser+ati+e 4A1IC pointing techniue
with "intelligent oPset* To initiate a pointing trial, there are two strategies a+ailable
to
the user% 0ne is to $ollow "+irtual inertiaJ* mo+e $rom the cursor-s current position
towar!s the new target the user is looing at% This is liely the strategy the user will
employ, !ue to the way the user interacts with to!ay-s inter$ace% The alternati+e
strategy,
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 22/42
which may be more a!+antageous but taes time to learn, is to ignore the pre+ious
cursor position an! mae a motion which is most con+enient an! least ePort$ul to
the
user $or a gi+en input !e+ice%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
6=
The goal o$ the conser+ati+e 4A1IC pointing metho! is the $ollowing% 0nce the
user loos at a target an! mo+es the input !e+ice, the cursor will appear "out o$ the
blue*
in motion towar!s the target, on the si!e o$ the target opposite to the initial
actuation
+ector% In comparison to the liberal approach, this conser+ati+e approach has both
pros
an! cons% hile with this techniue the cursor woul! ne+er be o+erHacti+e an! Lump
to a
place the user !oes not inten! to acuire, it may reuire more han!Heye
coor!ination
ePort% Both the liberal an! the conser+ati+e 4A1IC pointing techniues oPer the
$ollowing potential a!+antagesJ
6%
Re!uction o$ manual stress an! $atigue, since the cross screen longH!istance cursor
mo+ement is eliminate! $rom manual control%
7%
5ractical accuracy le+el% In comparison to tra!itional pure ga.e pointing whose
accuracy is $un!amentally limite! by the nature o$ eye mo+ement, the 4A1IC
pointing techniues let the han! complete the pointing tas, so they can be as
accurate as any other manual input techniues%
8%
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 23/42
A more natural mental mo!el $or the user% The user !oes not ha+e to be aware o$
the
role o$ the eye ga.e% To the user, pointing continues to be a manual tas, with a
cursor con+eniently appearing where it nee!s to be%
:%
Spee!% Since the nee! $or large magnitu!e pointing operations is less than with
pure
manual cursor control, it is possible that 4A1IC pointing will be $aster than pure
manual pointing%
;%
Impro+e! subLecti+e spee! an! easeHo$Huse% Since the manual pointing amplitu!e is
smaller, the user may percei+e the 4A1IC pointing system to operate $aster an!
more pleasantly than pure manual control, e+en i$ it operates at the same spee! or
more slowly%
The $ourth point wants $urther !iscussion% Accor!ing to the well accepte! Fitts-
&aw, manual pointing time is logarithmically proportional to the AO ratio, where A
is
the mo+ement !istance an! is the target si.e% In other wor!s, targets which are
smaller or $arther away tae longer to acuire%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
7>
For 4A1IC pointing, since the target si.e remains the same but the cursor
mo+ement !istance is shortene!, the pointing time can hence be re!uce!% It is lessclear
i$ eye ga.e control $ollows Fitts- &aw% In are an! 4iaelian-s stu!y, selection time
was shown to be logarithmically proportional to target !istance, thereby con$orming
to
Fitts- &aw% To the contrary, Silbert an! Nacob = $oun! that trial completion time with
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 24/42
eye tracing input increases little with !istance, there$ore !e$ying Fitts- &aw% In
a!!ition
to problems with to!ay-s eye tracing systems, such as !elay, error, an!
incon+enience,
there may also be many potential human $actor !isa!+antages to the 4A1IC
pointing
techniues we ha+e propose!, inclu!ing the $ollowingJ
6%
ith the more liberal 4A1IC pointing techniue, the cursor warping can be
o+eracti+e at times, since the cursor mo+es to the new ga.e location whene+er
the eye ga.e mo+es more than a set !istance e%g%, 67> pixels away $rom the
cursor% This coul! be particularly !istracting when the user is trying to rea!% It is
possible to intro!uce a!!itional constraint accor!ing to the context% For example,
when the user-s eye appears to $ollow a text rea!ing pattern, 4A1IC pointing
can be automatically suppresse!%
7%
ith the more conser+ati+e 4A1IC pointing techniue, the uncertainty o$ the
exact location at which the cursor might appear may $orce the user, especially a
no+ice, to a!opt a cumbersome strategyJ tae a touch use the manual input
!e+ice to acti+ate the cursor, wait $or the cursor to appear, an! mo+e the
cursor to the target manually% Such a strategy may prolong the target acuisition
time% The user may ha+e to learn a no+el han!Heye coor!ination pattern to be
eGcient with this techniue% 1a.e position reporte! by eye tracer (ye tracing
boun!ary with =;W conf!ence True target will be within the circle with =;W
probability The cursor is warpe! to the boun!ary o$ the ga.e area, along the
initial actuation +ector 5re+ious cursor position, $ar $rom target Initial manual
actuation +ector
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 25/42
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
76
8%
ith pure manual pointing techniues, the user, nowing the current cursor
location, coul! concei+ably per$orm his motor acts in parallel to +isual search%
4otor action may start as soon as the user-s ga.e settles on a target% ith
4A1IC pointing techniues, the motor action computation !ecision cannot
start until the cursor appears% This may negate the time sa+ing gaine! $rom the
4A1IC pointing techniue-s re!uction o$ mo+ement amplitu!e% Clearly,
experimental implementation an! empirical wor is nee!e! to +ali!ate, refne,
or in+ent alternati+e 4A1IC pointing techniues%
;%6 I45&(4(/TATI0/
e too two engineering ePorts to implement the 4A1IC pointing techniues%
0ne was to !esign an! implement an eye tracing system an! the other was to
implement 4A1IC pointing techniues at the operating systems le+el, so that the
techniues can wor with all so$tware applications beyon! "!emonstration*
so$tware%
;%7 T#( IB4 A&4A2(/ ()( TRAC(R
Since the goal o$ this wor is to explore 4A1IC pointing as a user inter$ace
techniue, we starte! out by purchasing a commercial eye tracer AS& 4o!el ;>>>
a$ter a maret sur+ey% In comparison to the system reporte! in early stu!ies e%g%
E,
this system is much more compact an! reliable% #owe+er, we $elt that it was still not
robust enough $or a +ariety o$ people with !iPerent eye characteristics, such as
pupil
brightness an! correction glasses% e hence chose to !e+elop an! use our own eye
tracing system 6>% A+ailable commercial systems, such as those ma!e by ISCA/
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 26/42
Incorporate!, &C Technologies, an! Applie! Science &aboratories AS&, rely on a
single light source that is positione! either oP the camera axis in the case o$ the
ISCA/(T&H:>> systems, or onHaxis in the case o$ the &CT an! the AS& (;>: systems%
Illumination $rom an oPHaxis source or ambient illumination generates a !ar pupil
image%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
77
hen the light source is place! onHaxis with the camera optical axis, the camera
is able to !etect the light reQecte! $rom the interior o$ the eye, an! the image o$ the
pupil
appears bright see Figure 8%
This ePect is o$ten seen as the re!Heye in Qash photographs when the Qash is
close to the camera lens%
Bright le$t an! !ar right pupil images resulting $rom onH an! oPHaxis
illumination% The glints, or corneal reQections, $rom the onH an! oPHaxis light sources
can be easily i!entife! as the bright points in the iris% The Alma!en system usestwo
near in$rare! IR time multiplexe! light sources, compose! o$ two sets o$ IR &(2Ks,
which were synchroni.e! with the camera $rame rate% 0ne light source is place!
+ery
close to the cameraKs optical axis an! is synchroni.e! with the e+en $rames% 0!!
$rames
are synchroni.e! with the secon! light source, positione! oP axis% The two light
sources
are calibrate! to pro+i!e approximately eui+alent wholeHscene illumination% 5upil
!etection is reali.e! by means o$ subtracting the !ar pupil image $rom the bright
pupil
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 27/42
image% A$ter threshol!ing the !iPerence, the largest connecte! component is
i!entife!
as the pupil% This techniue signifcantly increases the robustness an! reliability o$
the
eye tracing system% A$ter implementing our system with satis$actory results, we
!isco+ere! that similar pupil !etection schemes ha! been in!epen!ently !e+elope!
by
Tomonoetal an! (bisawa an! Satoh%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
78
It is un$ortunate that such a metho! has not been use! in the commercial
systems% e recommen! that $uture eye tracing pro!uct !esigners consi!er such
an
approach%
0nce the pupil has been !etecte!, the corneal reQection the glint reQecte! $rom
the sur$ace o$ the cornea !ue to one o$ the light sources is !etermine! $rom the
!ar
pupil image% The reQection is then use! to estimate the userKs point o$ ga.e in terms
o$
the screen coor!inates where the user is looing at% The estimation o$ the userKs
ga.e
reuires an initial calibration proce!ure, similar to that reuire! by commercial eye
tracers% 0ur system operates at 8> $rames per secon! on a 5entium II 888 4#.
machine running in!ows /T% It can wor with any 5CI $rame grabber compatible
with
3i!eo $or in!ows%
;%8 I45&(4(/TI/1 4A1IC 50I/TI/1
e programme! the two 4A1IC pointing techniues on a in!ows /T system%
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 28/42
The techniues wor in!epen!ently $rom the applications% The 4A1IC pointing
program taes !ata $rom both the manual input !e+ice o$ any type, such as a
mouse
an! the eye tracing system running either on the same machine or on another
machine
connecte! +ia serial port% Raw !ata $rom an eye tracer cannot be !irectly use! $or
ga.eH
base! interaction, !ue to noise $rom image processing, eye mo+ement Litters, an!
samples taen !uring sacca!e ballistic eye mo+ement perio!s% e experimente!
with
+arious fltering techniues an! $oun! the most ePecti+e flter in our case is similar
to
that !escribe! in E% The goal o$ flter !esign in general is to mae the best
compromise
between preser+ing signal ban!wi!th an! eliminating unwante! noise% In the case
o$ eye
tracing, as Nacob argue!, eye in$ormation rele+ant to interaction lies in the
fxations%
The ey is to select fxation points with minimal !elay% Samples collecte! !uring a
sacca!e are unwante! an! shoul! be a+oi!e!% In !esigning our algorithm $or picing
points o$ fxation, we consi!ere! our tracing system spee! 8> #., an! that the
4A1IC pointing techniues utili.e ga.e in$ormation only once $or each new target,
probably imme!iately a$ter a sacca!e% 0ur fltering algorithm was !esigne! to pic
a
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/17:
fxation with minimum !elay by means o$ selecting two a!Lacent points o+er two
samples%
;%: (5(RI4(/T
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 29/42
(mpirical stu!ies are relati+ely rare in eye tracingHbase! interaction research,
although they are particularly nee!e! in this fel!% #uman beha+ior an! processes
at the
perceptual motor le+el o$ten !o not con$orm to consciousHle+el reasoning% 0ne
usually
cannot correctly !escribe how to mae a turn on a bicycle% #ypotheses on no+el
interaction techniues can only be +ali!ate! by empirical !ata% #owe+er, it is also
particularly !iGcult to con!uct empirical research on ga.eHbase! interaction
techniues,
!ue to the complexity o$ eye mo+ement an! the lac o$ reliability in eye tracing
euipment% Satis$actory results only come when "e+erything is going right%* hen
results are not as expecte!, it is !iGcult to fn! the true reason among many
possible
reasonsJ Is it because a subLect-s particular eye property $oole! the eye tracer?
as
there a calibration error? 0r ran!om noise in the imaging system? 0r is the
hypothesis in
$act in+ali!? e are still at a +ery early stage o$ exploring the 4A1IC pointing
techniues% 4ore refne! or e+en +ery !iPerent techniues may be !esigne! in the
$uture% e are by no means rea!y to con!uct the !efniti+e empirical stu!ies on
4A1IC
pointing% #owe+er, we also $eel that it is important to subLect our wor to empirical
e+aluations early so that uantitati+e obser+ations can be ma!e an! $e! bac to the
iterati+e !esignHe+aluationH!esign cycle% e there$ore !eci!e! to con!uct a smallH
scale
pilot stu!y to tae an initial pee at the use o$ 4A1IC pointing, howe+er unrefne!%
;%; (5(RI4(/TA& 2(SI1/
The two 4A1IC pointing techniues !escribe! earlier were put to test using a
set o$ parameters such as the flter-s temporal an! spatial threshol!s, the minimum
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 30/42
cursor warping !istance, an! the amount o$ "intelligent bias* subLecti+ely selecte!
by
the authors without extensi+e user testing% 'ltimately the 4A1IC pointing
techniues
shoul! be e+aluate! with an array o$ manual input !e+ices, against both pure
manual an!
pure ga.eHoperate! pointing metho!s%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
7;
Since this is an early pilot stu!y, we !eci!e! to limit oursel+es to one manual
input !e+ice% A stan!ar! mouse was frst consi!ere! to be the manual input !e+ice
in the
experiment% #owe+er, it was soon reali.e! not to be the most suitable !e+ice $or
4A1IC pointing, especially when a user !eci!es to use the pushHupwar!s strategy
with
the intelligent oPset% Because in such a case the user always mo+es in one
!irection, the
mouse ten!s to be mo+e! oP the pa!, $orcing the user a!Lust the mouse position,o$ten
!uring a pointing trial% e hence !eci!e! to use a miniature isometric pointing stic
IB4 Trac 5oint I3, commercially use! in the IB4 Thin5a! D>> an! EE> series
noteboo computers% Another !e+ice suitable $or 4A1IC pointing is a touchpa!J the
user can choose one con+enient gesture an! to tae a!+antage o$ the intelligent
oPset%
The experimental tas was essentially a Fitts- pointing tas% SubLects were ase! topoint
an! clic at targets appearing in ran!om or!er% I$ the subLect clice! oPHtarget, a
miss
was logge! but the trial continue! until a target was clice!% An extra trial was
a!!e! to
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 31/42
mae up $or the misse! trial% 0nly trials with no misses were collecte! $or time
per$ormance analyses% SubLects were ase! to complete the tas as uicly as
possible
an! as accurately as possible% To ser+e as a moti+ator, a V7> cash pri.e was set $or
the
subLect with the shortest mean session completion time with any techniue%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
7D
The tas was presente! on a 7> inch CRT color monitor, with a 6; by 66 inch
+iewable area set at resolution o$ 67<> by 6>7: pixels% SubLects sat $rom the screen
at a
!istance o$ 7; inches% The $ollowing $actors were manipulate! in the experimentsJ
two target si.esJ 7> pixels >%78 in or >%;8 !egree o$ +iewing angle at 7; in
!istance an! D> pixels in !iameter >%E in, 6%D6 !egree
three target !istancesJ 7>> pixels 7%8: in, ;%8E !egree, ;>> pixels ;%<; in,
68%8E !egree, an! <>> pixels =%8< in, 76%7: !egree
three pointing !irectionsJ hori.ontal, +ertical an! !iagonal
A withinHsubLect !esign was use!% (ach subLect per$orme! the tas with all three
techniuesJ 6 Stan!ar!, pure manual pointing with no ga.e tracing /o 1a.eX 7
The conser+ati+e 4A1IC pointing metho! with intelligent oPset 1a.e6X 8 The
liberal 4A1IC pointing metho! 1a.e7% /ine subLects, se+en male an! two $emale,
complete! the experiment% The or!er o$ techniues was balance! by a &atin suare
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 32/42
pattern% Se+en subLects were experience! Trac 5oint users, while two ha! little or
no
experience% ith each techniue, a 8DHtrial practice session was frst gi+en, !uring
which subLects were encourage! to explore an! to fn! the most suitable strategies
aggressi+e, gentle, etc%% The practice session was $ollowe! by two !ata collection
sessions% Although our eye tracing system allows hea! motion, at least $or those
users
who !o not wear glasses, we !eci!e! to use a chin rest to minimi.e instrumental
error%
;%D (5(RI4(/TA& R(S'&TS
1i+en the pilot nature an! the small scale o$ the experiment, we expecte! the
statistical power o$ the results to be on the weaer si!e% In other wor!s, while the
signifcant ePects re+eale! are important, suggesti+e tren!s that are statistically
nonH
signifcant are still worth noting $or $uture research% First, we $oun! that subLects-
trial
completion time signifcantly +arie! with techniuesJ F7, 6D Y D%8D, p Z >%>6%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
7E
The total a+erage completion time was 6%: secon!s with the stan!ar! manual
control techniue 6%;7 secon!s with the conser+ati+e 4A1IC pointing techniue
1a.e6, an! 6%88 secon!s with the liberal 4A1IC pointing techniue 1a.e7% /ote
that the 1a.e6
Techniue ha! the greatest impro+ement $rom the frst to the secon! experiment
session, suggesting the possibility o$ matching the per$ormance o$ the other two
techniues with $urther practice%
As expecte!, target si.e signifcantly inQuence! pointing timeJ F6,< Y 6E<, p Z
>%>>6% This was true $or both the manual an! the two 4A1IC pointing techniues
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 33/42
Figure D%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
5ointing amplitu!e also signifcantly aPecte! completion timeJ F7, < Y =E%;, p
Z >%>>6% #owe+er, the amount o$ inQuence +arie! with the techniue use!, as
in!icate!
by the signifcant interaction between tech
Figure E%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
5ointing amplitu!e also signifcantly aPecte! completion timeJ F7, < Y =E%;, p
Z >%>>6% #owe+er, the amount o$ inQuence +arie! with the techniue use!, as
in!icate!
by the signifcant interaction between tech niue an! amplitu!eJ F:, 87 Y E%;, p Z
>%>>6
7<
5ointing amplitu!e also signifcantly aPecte! completion timeJ F7, < Y =E%;, p
Z >%>>6% #owe+er, the amount o$ inQuence +arie! with the techniue use!, as
in!icate!
niue an! amplitu!eJ F:, 87 Y E%;, p Z >%>>6
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/17=
As pointing amplitu!e increase! $rom 7>> pixels to ;>> pixels an! then to <>>
pixels, subLects- completion time with the /o[1a.e con!ition increase! in a nonH
linear,
logarithmicHlie pace as Fitts- &aw pre!icts% This is less true with the two 4A1IC
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 34/42
pointing techniues, particularly the 1a.e7 con!ition, which is !efnitely not
logarithmic% /onetheless, completion time with the 4A1IC pointing techniues !i!
increase as target !istance increase!% This is intriguing because in 4A1IC pointing
techniues, the manual control portion o$ the mo+ement shoul! be the !istance$rom the
warpe! cursor position to the true target% Such !istance !epen!s on eye tracing
system
accuracy, which is unrelate! to the pre+ious cursor position%
In short, while completion time an! target !istance with the 4A1IC pointing
techniues !i! not completely $ollow Fitts- &aw, they were not completely
in!epen!ent
either% In!ee!, when we lump target si.e an! target !istance accor!ing to the
Fitts- &aw
In!ex o$ 2iGculty I2 Y log7AO \ 6 6;,
we see a similar phenomenon% For the /o[1a.e con!itionJ
T Y >%7< \ >%86 I2 r]Y>%=67
The particular settings o$ our experiment were +ery !iPerent $rom those typically
reporte! in a Fitts- &aw experimentJ to simulate more realistic tass we use! circular
targets !istribute! in +arie! !irections in a ran!omly shu^e! or!er, instea! o$ two
+ertical bars !isplace! only in the hori.ontal !imension% e also use! an isometric
pointing stic, not a mouse% Consi!ering these $actors, the abo+e euation is
reasonable%
The in!ex o$ per$ormance I5 was 8%7 bits per secon!, in comparison to the :%; bits
per
secon! in a typical setting repeate! mouse clics on two +ertical bars 6D%
For the 1a.e6 con!itionJ
T Y >%< \ >%77 I2 r]Y>%E6D
I5 Y :%;; bits per secon!
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 35/42
For 1a.e7J
T Y >%D \ >%76 I2 r]Y>%<>:
I5 Y :%ED bits per secon!
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
8>
/ote that the !ata $rom the two 4A1IC pointing techniues ft the Fitts- &aw
mo!el relati+ely poorly as expecte!, although the in!ices o$ per$ormance :%;;
an!
:%ED bps were much higher than the manual con!ition 8%7 bps%
Finally, Figure < shows that the angle at which the targets were presente! ha!
little inQuence on trial completion timeJ F7, 6D Y 6%;E, /%S%
The number o$ misses clice! oP target was also analy.e!% The only
signifcant $actor to the number o$ misses is target si.eJ F6,< Y 6;%D, p Z >%>6%
'sers
ten!e! to ha+e more misses with small targets% 4ore importantly, subLects ma!e no
more misses with the 4A1IC pointing techniues than with the pure manual
techniue
/o[1a.e M <%7 W, 1a.e6 MEW, 1a.e7 M E%;W%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
86
D% ARTIFICIA& I/T(&&I1(/T S5((C# R(C01/ITI0/
It is important to consi!er the en+ironment in which the speech recognition
system has to wor% The grammar use! by the speaer an! accepte! by the
system, noise
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 36/42
le+el, noise type, position o$ the microphone, an! spee! an! manner o$ the user-s
speech
are some $actors that may aPect the uality o$ speech recognition %hen you !ial
the
telephone number o$ a big company, you are liely to hear the sonorous +oice o$ a
culture! la!y who respon!s to your call with great courtesy saying "elcome to
company % 5lease gi+e me the extension number you want*% )ou pronounce the
extension number, your name, an! the name o$ person you want to contact% I$ the
calle!
person accepts the call, the connection is gi+en uicly% This is artifcial intelligence
where an automatic callHhan!ling system is use! without employing any telephone
operator%
D%6 T#( T(C#/0&01)
Artifcial intelligence AI in+ol+es two basic i!eas% First, it in+ol+es stu!ying
the thought processes o$ human beings% Secon!, it !eals with representing those
processes +ia machines lie computers, robots, etc% AI is beha+ior o$ a machine,
which, i$ per$orme! by a human being, woul! be calle! intelligent% It maes
machines
smarter an! more use$ul, an! is less expensi+e than natural intelligence% /atural
language processing /&5 re$ers to artifcial intelligence metho!s o$ communicating
with a computer in a natural language lie (nglish% The main obLecti+e o$ a /&5
program is to un!erstan! input an! initiate action% The input wor!s are scanne! an!
matche! against internally store! nown wor!s% I!entifcation o$ a ey wor! causes
some action to be taen% In this way, one can communicate with the computer inone-s
language% /o special comman!s or computer language are reuire!% There is no
nee! to
enter programs in a special language $or creating so$tware%
B&'( ()(S T(C#/0&01)
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 37/42
2I3IS0/ 0F C045'T(R (/1I/((RI/1
87
D%7 S5((C# R(C01/ITI0/
The user speas to the computer through a microphone, which, in use!X a simple
system may contain a minimum o$ three flters% The more the number o$ flters
use!, the
higher the probability o$ accurate recognition% 5resently, switche! capacitor !igital
flters are use! because these can be customHbuilt in integrate! circuit $orm% These
are
smaller an! cheaper than acti+e flters using operational amplifers% The flter output
is
then $e! to the A2C to translate the analogue signal into !igital wor!% The A2C
samples
the flter outputs many times a secon!% (ach sample represents !iPerent amplitu!e
o$ the
signal %(+enly space! +ertical lines represent the amplitu!e o$ the au!io flter
output at
the instant o$ sampling% (ach +alue is then con+erte! to a binary number
proportional to
the amplitu!e o$ the sample% A central processor unit C5' controls the input
circuits
that are $e! by the A2CS% A large RA4 ran!om access memory stores all the
!igital
+alues in a buPer area% This !igital in$ormation, representing the spoen wor!, is
now
accesse! by the C5' to process it $urther% The normal speech has a $reuency range
o$
7>> #. to E #.% Recogni.ing a telephone call is more !iGcult as it has ban!wi!th
limitation o$ 8>> #. to8%8 #.%
As explaine! earlier, the spoen wor!s are processe! by the flters an! A2Cs%
The binary representation o$ each o$ these wor!s becomes a template or stan!ar!,
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 38/42
against which the $uture wor!s are compare!% These templates are store! in the
memory%
0nce the storing process is complete!, the system can go into its acti+e mo!e an!
is
capable o$ i!enti$ying spoen wor!s% As each wor! is spoen, it is con+erte! into
binary
eui+alent an! store! in RA4% The computer then starts searching an! compares
the
binary input pattern with the templates% t is to be note! that e+en i$ the same
speaer
tals the same text, there are always slight +ariations in amplitu!e or lou!ness o$
the
signal, pitch, $reuency !iPerence, time gap, etc% 2ue to this reason, there is ne+er
a
per$ect match between the template an! binary input wor!% The pattern matching
process there$ore uses statistical techniues an! is !esigne! to loo $or the best ft%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
88
The +alues o$ binary input wor!s are subtracte! $rom the correspon!ing +alues in
the templates% I$ both the +alues are same, the !iPerence is .ero an! there is
per$ect
match% I$ not, the subtraction pro!uces some !iPerence or error% The smaller the
error,
the better the match% hen the best match occurs, the wor! is i!entife! an!
!isplaye!
on the screen or use! in some other manner% The search process taes aconsi!erable
amount o$ time, as the C5' has to mae many comparisons be$ore recognition
occurs%
This necessitates use o$ +ery highHspee! processors% A large RA4 is also reuire! as
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 39/42
e+en though a spoen wor! may last only a $ew hun!re! millisecon!s, but the same
is
translate! into many thousan!s o$ !igital wor!s% It is important to note that
alignment o$
wor!s an! templates are to be matche! correctly in time, be$ore computing the
similarity
score% This process, terme! as !ynamic time warping, recogni.es that !iPerent
speaers
pronounce the same wor!s at !iPerent spee!s as well as elongate !iPerent parts o$
the
same wor!% This is important $or the speaerHin!epen!ent recogni.ers%
D%8 A55&ICATI0/S
0ne o$ the main benefts o$ speech recognition system is that it lets user !o other
wors simultaneously% The user can concentrate on obser+ation an! manual
operations,
an! still control the machinery by +oice input comman!s% Another maLor application
o$
speech processing is in military operations% 3oice control o$ weapons is an example%
ith reliable speech recognition euipment, pilots can gi+e comman!s an!in$ormation
to the computers by simply speaing into their microphonesUthey !on-t ha+e to
use
their han!s $or this purpose% Another goo! example is a ra!iologist scanning
hun!re!s o$
Hrays, ultrasonograms, CT scans an! simultaneously !ictating conclusions to a
speech
recognition system connecte! to wor! processors% The ra!iologist can $ocus hisattention
on the images rather than writing the text% 3oice recognition coul! also be use! on
computers $or maing airline an! hotel reser+ations% A user reuires simply to state
his
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 40/42
nee!s, to mae reser+ation, cancel a reser+ation, or mae enuiries about sche!ule%
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
8:
E% T#( SI45&( 'S(R I/T(R(ST TRAC(R S'IT0R
Computers woul! ha+e been much more power$ul, ha! they gaine! perceptual
an! sensory abilities o$ the li+ing beings on the earth% hat nee!s to be !e+elope!
is an
intimate relationship between the computer an! the humans% An! the Simple 'ser
Interest Tracer S'IT0R is a re+olutionary approach in this !irection%
By obser+ing the ebpage a neti.en is browsing, the S'IT0R can help by
$etching more in$ormation at his !estop% By simply noticing where the user-s eyes
$ocus on the computer screen, the S'IT0R can be more precise in !etermining his
topic
o$ interest% It can e+en !eli+er rele+ant in$ormation to a han!hel! !e+ice% The
success
lies in how much the suitor can be intimate to the user% IB4Ks Blue(yes research
proLect
began with a simple uestion, accor!ing to 4yron Flicner, a manager in Alma!enKs
'S(R groupJ Can we exploit non+erbal cues to create more ePecti+e user
inter$aces?
0ne such cue is ga.eUthe !irection in which a person is looing% Flicner an! his
colleagues ha+e create! some new techniues $or tracing a personKs eyes an!ha+e
incorporate! this ga.eHtracing technology into two prototypes% 0ne, calle! S'IT0R
Simple 'ser Interest Tracer, flls a scrolling ticer on a computer screen with
in$ormation relate! to the userKs current tas% S'IT0R nows where you are looing,
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 41/42
what applications you are running, an! what eb pages you may be browsing% _I$
IKm
rea!ing a eb page about IB4, $or instance,_ says 5aul 4aglio, the Alma!en
cogniti+e
scientist who in+ente! S'IT0R, _the system presents the latest stoc price or
business
news stories that coul! aPect IB4% I$ I rea! the hea!line oP the ticer, it pops up the
story in a browser win!ow% I$ I start to rea! the story, it a!!s relate! stories to the
ticer%
ThatKs the whole i!ea o$ an attenti+e systemUone that atten!s to what you are
!oing,
typing, rea!ing, so that it can atten! to your in$ormation nee!s%_
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
8;
<% C0/C&'SI0/
The nineties witnesse! uantum leaps inter$ace !esigning $or impro+e! man
machine interactions% The B&'( ()(S technology ensures a con+enient way o$
simpli$ying the li$e by pro+i!ing more !elicate an! user $rien!ly $acilities in
computing
!e+ices% /ow that we ha+e pro+en the metho!, the next step is to impro+e the
har!ware%
Instea! o$ using cumbersome mo!ules to gather in$ormation about the user, it will
be
better to use smaller an! less intrusi+e units% The !ay is not $ar when this
technology
will push its way into your house hol!, maing you more la.y% It may e+en reach
your
han! hel! mobile !e+ice% Any way this is only a technological $orecast%
7/23/2019 Blue Eye REPORT
http://slidepdf.com/reader/full/blue-eye-report 42/42
B&'( ()(S T(C#/0&01)
2I3IS0/ 0F C045'T(R (/1I/((RI/1
8D
=% R(F(R(/C(S
6 www%wiipe!ia%com
7 www%techre+iew%com
8 www%alma!en%ibm%com
: www%research%ibm%com
; www%metropolismag%com
D www%+isuallee%com
E www%howstuPwors%com