the third eye-presentation
TRANSCRIPT
![Page 1: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/1.jpg)
The THIRD EYECORRELATING IMAGE-SOUND-VIDEO
Gunjan Gupta 10BEC112Romil Shah 10BEC093Sagar Patel 10BEC072
Mentor : Prof. Sachin Gajjar
1
![Page 2: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/2.jpg)
Outline
Motivation
Technological Objective
Present Scenario
Bionics: Concept and Cyborg
Our Innovation
Approach (Pros and Cons)
Applications
Algorithms
References
2
![Page 3: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/3.jpg)
Wah Taj !! 3
![Page 4: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/4.jpg)
Beautiful… 4
![Page 5: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/5.jpg)
Again Nice to See!! 5
![Page 6: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/6.jpg)
Oh, This bus is going to LAL DARWAJA 6
![Page 7: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/7.jpg)
BUT CAN EVERYBODY SEE THIS BEAUTY ?
CAN EVERYONE SAY THE BUS NUMBER BY SEEING IT ?
NO!!ALL PEOPLE DON’T HAVE SAME EYE-
SIGHT or EYE-VISION !
7
![Page 8: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/8.jpg)
WHAT FOR THESE GUYS ?? 8
![Page 9: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/9.jpg)
A project to build a system that converts visual input into audio signals which may lead to a practical product to help the blind or visually impaired
navigate
9
![Page 10: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/10.jpg)
Technological Objectives
The main objective is to obtain a device for the whole Visually Impaired(V.I.) community- totally blind, partially sighted, and people in a situationof progressive loss of vision.
To integrate the developments of image segmentation, imageenhancement, 3D Real World perception, audio representation of spaceinto a unique basic prototype with the purpose of assisting visuallyimpaired people in mobility in outdoor spaces.
To develop new ways of interfacing with V. I. people- to develop anacoustic interface (Ear-plugs for presentation) for the blinds.
To develop a system able to calculate a person’s head orientation in astatic and dynamic situation.
10
![Page 11: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/11.jpg)
Technological Objectives
To integrate the 3D modelling system and the intelligent system for thedetection of risks and obstacles
To integrate in a common platform all the data acquired from theenvironment from different sources- artificial vision system, 3D Real WorldRecognition System.
To finally develop a prototype which can be commercialised
During the project a simple device able to make a direct transformationfrom the acquisition system into an acoustic representation will be createdas a final product
11
![Page 12: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/12.jpg)
Gadgets/Technology for V.I. Person 12
Brainport Vision Device
Electric lollipop that allows blind people to 'see'using their tongue
Converts images captured by a tiny camera intoa series of electrical tingles, which can be felt onthe tongue
Nerves then send these messages to the brain,which turn the tingles back into pictures
Costs around 6000 Pounds ( 5 Lakh INR)
![Page 13: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/13.jpg)
Gadgets/Technology for V.I. Person
Devices that alert you when a pot of liquid begins to boil
Gadgets that make a sound when a cup or container you're pouring liquidinto is nearly full
Knives with an adjustable guide to help you cut slices of even thickness
Tactile watches and alarm clocks
13
![Page 14: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/14.jpg)
Gadgets/Technology for V.I. Person 14
![Page 15: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/15.jpg)
Cyborg / Bionics
An organism “to which exogenous components have been added for the purpose of adapting to new environments”.
Recent meaning of the term bionics refers to merging organism and machine.
This approach results in a hybrid system combining biological and engineering parts, which can also be referred as a cybernetic organism (cyborg).
15
![Page 16: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/16.jpg)
Natural-Artificial Feedback 16
![Page 17: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/17.jpg)
Neil Harbisson: A Real Cyborg17
![Page 18: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/18.jpg)
What is Image Processing?
In electrical engineering and computer science, image processing is any form of signalprocessing for which the input is an image, such as a photograph or video frame; theoutput of image processing may be either an image or, a set of characteristics orparameters related to the image.
Most image-processing techniques involve treating the image as a two-dimensionalsignal and applying standard signal-processing techniques to it.
In other words it is basically the transformation of data from a still or video camerainto either a decision or a new representation.
All such transformations are done for achieving some particular goal. The input datamay be a live video feed, the decision may be that a face has been detected, and a newrepresentation may be conversion of a color image into a gray scale image.
![Page 19: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/19.jpg)
Our Innovation
Locating everyday items using Image-Video Processing.
Detection of potential hazards (location and motion of objects, persons, obstacles).
Face Detection and Tagging.
Text to Speech Conversion.
Color Detection
Detecting and analyzing in-front objects (living and non-living) and perform operations on them.
19
![Page 20: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/20.jpg)
Approach
We are using OpenCV with Panda Board for this application.
Interfacing OpenCV and Panda Board along with webcam to generate real time images.
Coding to find distance of objects or any obstacle and provide output as an audio signal.
Provide with inbuilt images and search for them. Coding to match the images and provide the respective name of object as sound signal.
Advance further to develop for general usage amongst people.
20
![Page 21: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/21.jpg)
Advantage & Disadvantage
Advantage
Detection of known objects if they are lost.
Can read newspaper, books, or rather any reading material.
Can easily detect color and hence can be useful in various scenarios.
Face Detection and Face tagging mode.(He can easily know person and canalso save new face)
Most of the software are freeware, so cheap in cost.
Disadvantage
A fully-assembled device (Panda Board + Webcam) is to be carried for all theoperations.
Bulkier in comparison to Mobile.
21
![Page 22: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/22.jpg)
Hardware/Software
The Panda Board is a low-power, low-cost single-boardcomputer development platform based on the TexasInstruments OMAP4430 system on a chip (SoC). It is a communitysupported development platform.
The Panda Board ES is a newer version based on the OMAP4460 SoC, withthe CPU and GPU running at higher clock rates. Like its predecessor, it is acommunity supported development platform. We are working with PandaBoard ES.
22
![Page 23: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/23.jpg)
Pandaboard
![Page 24: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/24.jpg)
Pandaboard Overview
OMAP4 (Cortex-A9) CPU based open development platform.
OMAP4430 Application processor
1GB low-power DDR2
Display HDMI v1.3 Connector (Type A) to drive HD displays, DVI-D Connector,
3.5" audio in/out and HDMI Audio out
Full size SD/MMC card
Built in 802.11 & Bluetooth v2.1+EDR
On board 10/100 Ethernet
Expansion: 1xUSB OTG, 2xUSB HS host ports, General purpose expansion header
![Page 25: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/25.jpg)
What OS does Pandaboard support?
Debian
Ubuntu
Android
Fedora
Sabayon
Xubuntu
Angstrom
![Page 26: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/26.jpg)
OpenCV
OpenCV grew out of an Intel Research initiative to advance CPU-intensive applications.
The intent behind OpenCV was to provide a platform that a student can readily utilize for developing applications, instead of reinventing the basic functions from scratch.
![Page 27: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/27.jpg)
Why use OpenCV?
It is free.
It is fast.
Good documentation in the form of documentation, tutorials, user groups, forums etc.
Lot of pre-built functions and algorithms to get a head start.
There is an active development on interfaces for other languages like Ruby, Python, Matlab etc.
Pretty fast.
![Page 28: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/28.jpg)
License
OpenCV is released under a BSD license, it is free for both academic and commercial use.
There are only a few algorithms inside it that may have been patented and those are not free.
![Page 29: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/29.jpg)
KINECT
![Page 30: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/30.jpg)
Kinect SENSOR
Photo Albumby Gunjan Gupta
![Page 31: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/31.jpg)
BLOCK DIAGRAM
31
![Page 32: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/32.jpg)
Face Recognition 32
![Page 33: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/33.jpg)
Object Detection
Dynamic (Moving)Stationary
33
![Page 34: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/34.jpg)
Detection by Segmentation34
![Page 35: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/35.jpg)
Segmentation Algorithm
Segmentation
Discontinuity
Point Detection Line Detection Edge Detection
Similarity
Thresholding Region GrowingRegion splitting
and merging
35
![Page 36: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/36.jpg)
Using MATLAB inbuilt function
Using our ownalgorithm, we appliedlinear differentiator. Itworks fast and is moreefficient incomparison toMATLAB INBUILTALGO
36EDGE DETECTION
![Page 37: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/37.jpg)
Application
This project is applied mainly in the commercial field of helping visuallyimpaired people with lower eyesight or completely BLIND
Industrial applications can be devised and enhanced like robots andmachineries
Concept can be used for Autonomous Cars
For Security, Identifying and Tracking
37
![Page 38: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/38.jpg)
Projects Implemented
1. Capturing Image using hand gesture and uploading it directly to Google Drive (Internet needed)
2. Face Detection and Person Identity Detection for running lifetime of prog
3. Optical Character Recognition
4. Cartoonifying Real Time Images
5. Colour Detection and Diff Sound for Diff Colour
6. Ball Detection
7. Head Pose Estimation
8. Augmented Reality based Position Change Tracking
9. Combined Project of OpenGL and OpenCV for Image Operation
10. Number Recognition in real-time
11. Text to Speech Synthesis using C++
12. Kinect with WINDOWS
13. Kinect with Ubuntu
14. Sound Marking
![Page 39: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/39.jpg)
References
A collaborative European project CASBliP, or Cognitive Aid System for Blind People
Andhajan Mandal, Ahmedabad, Gujarat, India
Electric lollipop that allows blind people to 'see' using their tongue-Brainport Vision Device
OMAP4460 Pandaboard ES System Reference Manual
Neil Harbisson-http://www.ted.com/talks/neil_harbisson_i_listen_to_color.html
AMTS-www.amts.co.in
39
![Page 40: THE THIRD EYE-Presentation](https://reader034.vdocuments.net/reader034/viewer/2022050801/58854c841a28abb5368b7193/html5/thumbnails/40.jpg)
THANK YOU