BLUE EYESThe Future of Sense Control

The BLUE EYES technology aims at creating computational machines that have perceptual and sensory ability like those of human beings. It uses non-obtrusive sensing method, employing most modern video cameras and microphones to identifies the users actions through the use of imparted sensory abilities . The machine can understand what a user wants, where he is looking at, and even realize his physical or emotional states.

Sudhanshu Singh07/ECE/08926th March 2011

6

ACKNOWLEDGEMENT

I take this opportunity to express my profound sense of gratitude and respect to all those who helped me in completing this seminar report.

I express my humble gratitude to the entire staff of ELECTRONICS DEPARTMENT who provided me with the relevant and required information to complete my report.

Sudhanshu Singh

Dated: 26th March, 2011

6

CONTENTS

1) What is blue eyes?2) Aims of blues eyes technology3) Tracks used 4) Affective computing 5) Blue eyes enabled devices6) Software and hardware7) IBM research8) Design objective9) Data security10)Conclusion11) References

6

What is Blue Eyes?

The U.S. computer giant, IBM has been conducting research on the Blue Eyes technology at its Almaden Research Center (ARC) in San Jose, Calif., since 1997. The ARC is IBM's main laboratory for basic research. The primary objective of the research is to give a computer the ability of the human being to assess a situation by using the senses of sight, hearing and touch.

Animal survival depends on highly developed sensory abilities. Likewise, human cognition depends on highly developed abilities to perceive, integrate, and interpret visual, auditory, and touch information. Without a doubt, computers would be much more powerful if they had even a small fraction of the perceptual ability of animals or humans. Adding such perceptual abilities to computers would enable computers and humans to work together more as partners. Toward this end, the Blue Eyes project aims at creating computational devices with the sort of perceptual abilities that people take for granted.

Thus Blue eyes is the technology to make computers sense and understand human behavior and feelings and react in the proper ways.

6

Aims1) To design smarter devices

2) To create devices with emotional intelligence

3) To create computational devices with perceptual abilities

The idea of giving computers personality or, more accurately, emotional intelligence" may seem creepy, but technologists say such machines would offer important advantages.

De-spite their lightning speed and awesome powers of computation, today's PCs are essentially deaf, dumb, and blind. They can't see you, they can't hear you, and they certainly don't care a whit how you feel. Every computer user knows the frustration of nonsensical error messages, buggy software, and abrupt system crashes. We might berate the computer as if it were an unruly child, but, of course, the machine can't respond. "It's ironic that people feel like dummies in front of their computers, when in fact the computer is the dummy," says Rosalind Picard, a computer science professor at the MIT Media Lab in Cambridge.

A computer endowed with emotional intelligence, on the other hand, could recognize when its operator is feeling angry or frustrated and try to respond in an appropriate fashion. Such a computer might slow down or replay a tutorial program for a confused student, or recognize when a designer is burned out and suggest he take a break. It could even play a recording of Beethoven's "Moonlight Sonata" if it sensed anxiety or serve up a rousing Springsteen anthem if it detected lethargy. The possible applications of "emotion technology" extend far beyond the desktop. A car equipped with an affective computing system could recognize when a driver is feeling drowsy and ad-vise her to pull over, or it might sense when a stressed-out motorist is about to explode and warn him to slow down and cool

6

Tracks used

Our emotional changes are mostly reflected in our heart pulse rate, breathing rate ,facial expressions ,eye movements ,voice etc. Hence these are the parameters on which blue technology is being developed

Affective Computing

The process of making emotional computers with sensing abilities is known as affective computing. The steps used in this are:

o Giving sensing abilitieso Detecting human emotionso Respond properly

The first step, researchers say, is to give machines the equivalent of the eyes, ears, and other sensory organs that humans use to recognize and express emotion. To that end, computer scientists are exploring a variety of mechanisms including voice-recognition software that can discern not only what is being said but the tone in which it is said; cameras that can track subtle facial expressions, eye movements, and hand gestures; and biometric sensors that can measure body temperature, blood pressure, muscle tension, and other physiological signals associated with emotion. In the second step, the computers have to detect even the minor variations of our moods. For eg, a person may hit the keyboard very fast either in the happy mood or in the angry mood. In the third step the computers have to react in accordance with the emotional states.

6

Blue Eyes Enabled Devices

Some of the blue Eyes enabled devices are discussed below:

1) POD: The first blue Eye enabled mass production device was POD , the car manufactured by TOYOTA. It could keep the driver alert and active. It could tell the driver to go slow if he is driving too fast and it could pull over the driver when he feels drowsy. Also it could hear the driver some sort of interesting music when he is getting bored.

2) PONG: IBM released a robot designed for demonstrating the new technology. The Blue Eyes robot is equipped with a computer capable of analyzing a person's glances and other forms of expressions of feelings, before automatically determining the next type of action. IBM has released a robot called PONG, which is equipped with the Blue Eyes technology. PONG is capable of perceiving the person standing in front of it, smiles when the person calls his name, and expresses loneliness when it loses sight of the person.

3) SECURE PAD: This is an electronic badge that can identify the wearer and track his movements and activities with an array of sensors. The device was designed for a major health-care provider to track the activities of doctors and other personnel at large medical facilities. It knows who you are and where you are, and it has a pretty good idea of what you are doing and when you are doing it. Although that may sound ominous, the device will ultimately benefit patients by enhancing the security and accountability of medical facilities. For example, the device will know who has accessed a drug locker and what drugs were removed. It will also allow doctors to access confidential medical information without carrying around paper charts, which can be misplaced or read by unauthorized personnel. When a doctor wearing the Secure Pad enters a patient's room, the patient's medical records will automatically appear on a wall monitor when the doctor looks at it. When he looks away, or another person enters the room, the records will disappear. Another advantage of Secure Pad is that it's interchangeable; when a wearer removes the badge from her body, the device automatically deactivates, its slate wiped clean until the next person puts it on.

6

SOFTWARE & HARDWARE

BLUE Eyes software is called “agent” which analyzes and modifies the mother program, according to the user’s needs and moods. The user modifies the mother program whenever the user requests or hardware finds that the user is changing his moods.

For eg, during browsing ,if a particular search is found to be failure ,the agent might improve the searching by changing the searching key words so as to get the desired results.

Hardware used is dedicated hardware. Usually embedded devices are used for eg. a pc camera may drain the processing abilities of the system and can not be used. The embedded devices have their own processing capabilities and can function on their own to achieve specific purposes.

6

IBM RESEARCH

Pioneers in this field are IBM, MIT(Massachusetts Institute of Technology), Sony egs of the technologies now under study have already been discussed. Limited success is achieved in translating neurological activities into identifiable emotional states by implanting electrodes in the rain. Researches are still going on and commercial availability is supposed to happen within the next few years.

BLUE EYE – EMOTIONAL MOUSE Sensors in the mouse, sense the physiological attributes which are correlated to emotions using correlation model-by simply touching the mouse, the computer will be able to determine a person’s emotional state.

BLUE EYE enabled TELEVISIONCould become active when the user makes an eye contact

.

6

The Eye Movement Sensor

The off-shelf eye movement sensor – Jazz multisensor that supplies raw digital data regarding:

o Eye positiono Level of blood oxygenationo Acceleration along horizontal and vertical axeso Ambient light intensity

Jazz Multisensor

6

Eye movements

Retina - central vision (macula) Central vision and the conscious brain

Saccade - a rapid eye jump to a new location assigned by the conscious attention process.

Saccades presence Û Exploratory mode

Retina - central vision (macula)

6

Design Objectives

A mobile data acquisition device A reliable real-time buffering, processing and recording A clear visualization interface Mechanisms for creating custom data analyzers To ensure a possibility of distributing the processing among a number

of computers

System overviewSystem overview

Central System UnitData Acquisition Unit

Atmel 89C52microcontroller

JazzMultisensor

ConnectionManager

Data Analysis

VisualizationModuleBluetooth Bluetooth

Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system.

DataLogger

6

Implementation & Engineering

Considerations

Three groups of users OPERATOR -a person whose physiological parameters are

supervised- authorization in the system- receiving alerts SUPERVISOR – a person responsible for analyzing operator’s

condition and performance. SYSTEM ADMINISTRATORS- maintains the system deliver tools for

adding- new operator’s to the database- defining alarm conditions- configuring logging tools- creating new analyzer modules

6

Data Security

Only registered mobile devicescan connect to the system

Bluetooth connection authentication Bluetooth connection encryption Access rights restrictions Personal and physiological data encryption

Blue Eyes – Benefits

Prevention from dangerous incidents Minimization of

o ecological consequenceso financial losso a threat to a human life

The reconstruction of the course of operator’s work

6

CONCLUSION

The Blue eyes technology ensures a convenient way of simplifying the life by providing more delicate and user friendly facilities in computing devices. The gap between the electronic and physical world will be reduce. The computers can be run by using the implicit commands instead of the explicit commands.

6

References

www.seminar4u.com www.howstuffswork.com www.cs.put.poznan.com www.whitepapers.com www.ibmresearchcenter.com www.almaden.ibm.com