sign language translator ieee power point
DESCRIPTION
TRANSCRIPT
Dept of Biomedical Engineering
AUTOMATIC LANGUAGE TRANSLATION SOFTWARE FOR AIDING COMMUNICATION BETWEEN INDIAN
SIGN LANGUAGE AND SPOKEN ENGLISH USING LABVIEW
By
YELLAPU MADHURI,
Reg.No.1651110002,
MTECH II YEAR,
SRM University.
Guided by Ms.G.ANITHA
Assistant professor (O.G) /BME
Dept of Biomedical Engineering
INTRODUCTION
SIGN LANGUAGE (SL)
Natural way of communication of speech and/or hearing-impaired people.
SIGN
Movement of one or both hands, accompanied with facial expression, which
corresponds to a specific meaning.
TRANSLATOR
Communication between speech and/or sound impaired person and person that do
not understand sign language, avoiding by this way the intervention of an
intermediate person. And allow communication using their natural way of speaking.
Dept of Biomedical Engineering
ANATOMY OF HUMAN EAR
Dept of Biomedical Engineering
EVENTS INVOLVED IN HEARING
Dept of Biomedical Engineering
SPEECH CHAIN
Dept of Biomedical Engineering
AIM
To develop a mobile interactive application software for automatic translation of
Indian sign language into speech in English and vice-versa to assist the
communication between speech and/or hearing impaired people with normal
people. This language translator should be able to translate one handed finger
spelling input of Indian Sign language alphabets A-Z and numbers 1-9 into spoken
English audio output and 165 spoken English words input to Indian Sign language
picture display output.
Dept of Biomedical Engineering
GRAPHICAL ABSTRACT
Dept of Biomedical Engineering
OBJECTIVE
For Sign to Speech conversion
1. Acquire images using the inbuilt camera of the device.
2. Perform vision analysis functions in the operating system and provide speech output
through the inbuilt audio device.
For Speech to Sign conversion
1. Acquire speech input using the inbuilt microphone of the device.
2. Perform speech analysis functions in the operating system and provide visual sign
output through the inbuilt display device.
Minimize hardware requirements and expense.
Dept of Biomedical Engineering
LITERATURE REVIEW
1. Jose l. Hernandez-rebollar et al [2004]
Discusses a novel approach for capturing and translating isolated gestures of ASL into spoken and written words using combined acceleglove and a two-link arm skeleton.
2. Paschaloudi N. Vassilia et al [may 2006]
Extensible system to recognize GSL modules for signs or finger-spelled words, using isolation or combined neural networks
3. Beifang yi [ may 2006] Explorations in the areas of computer graphics, interface design, and human-computer interactions with emphasis on software development and implementation in ASLT
4. Andreas domingo et al [2007]
ASLT using pattern-matching algorithm.
5. Rini akmeliawatil et al [may 2007]
ASLT for real-time english translation of the malaysia SL using neural networks.
6. Abang irfan halil et al [ 2007]
Extent of development details on recognition system by using state-of-the-art graphical programming software
Dept of Biomedical Engineering
ALGORITHM CRITERION
1. REAL-TIME
2. VISION-BASED
3. AUTOMATIC AND CONTINUOUS OPERATION
4. EFFICIENT TRANSLATION
Dept of Biomedical Engineering
MATERIALS
Software Tools used: National Instruments LabVIEW and toolkits
LABVIEW 2012 version Vision Development Module Vision acquisition Module
Hardware tools used:
Laptop inbuilt webcamera- Acer Crystal Eye Laptop inbuilt speaker-Acer eAudio
Dept of Biomedical Engineering
GUI OF SOFTWARE
Dept of Biomedical Engineering
PAGE 2- SPEECH TO SIGN LANGUAGE TRANSLATOR
Dept of Biomedical Engineering
BLOCK DIAGRAM OF SPEECH TO SIGN LANGUAGE TRANSLATOR
Dept of Biomedical Engineering
FLOW CHART OF SPEECH TO SIGN LANGUAGE TRANSLATION
Dept of Biomedical Engineering
WINDOWS SPEECH RECOGNITION TUTORIAL
Dept of Biomedical Engineering
WINDOWS SPEECH RECOGNITION SOFTWARE GUI
Dept of Biomedical Engineering
USER INTRFACE OF SPEECH TO SIGN LANGUAGE TRANSLATOR
Dept of Biomedical Engineering
PAGE 3- TEMPLATE PREPARATION
Dept of Biomedical Engineering
IMAGE ACQUISITION SEQUENCE OF FRAMES
Dept of Biomedical Engineering
USER INTERFACE OF TEMPLATE PREPARATION FOR SIGN LANGUAGE TO ENGLISH TRANSLATION
Dept of Biomedical Engineering
FLOW CHART OF TEMPLATE PREPARATION FOR SIGN LANGUAGE TO ENGLISH TRANSLATION
Dept of Biomedical Engineering
PAGE 4- PATTERN MATCHING
Dept of Biomedical Engineering
USER INTERFACE OF PATTERN MATCHING FOR SIGN LANGUAGE TO ENGLISH TRANSLATION
Dept of Biomedical Engineering
BLOCK DIAGRAM OF SIGN LANGUAGE TO SPEECH TRANSLATOR
Dept of Biomedical Engineering
DATABASE OF ONE HANDED ALPHABETS AND NUMBERS OF SIGN LANGUAGE
Dept of Biomedical Engineering
ADVANTAGES
Eliminates the need for an interpreter for communication between sign language and speech language.
Easy to incorporate and execute in any supporting operating system. Real time translation. Does not require any additional hardware.
Dept of Biomedical Engineering
FUTURE APPLICATIONS
Web conferenceCOMPUTER AND VIDEO GAMESPRECISION SURGERYDOMESTIC APPLICATIONSWEARABLE COMPUTERS
Dept of Biomedical Engineering
CHALLENGES
Background subtraction for robust usage.Making the system user independent.Pattern matching training.
Dept of Biomedical Engineering
LIMITATIONS
System is trained on a limited database..
Possibility of misinterpretation for closely related gestures.
Translates only static signs.
Not trained to translate dynamic signs.
Facial expressions are not considered.
Possibility of misinterpretation for words of similar pronunciation.
Dept of Biomedical Engineering
CONCLUSION
The feature vectors which include whole image frames containing all the aspects of
the sign are considered.
The geometric features which are extracted from the signers’ dominant hand, improve
the accuracy of the system to a great degree.
Training the speech recognition for shorter phrases is difficult than longer phrases.
Dept of Biomedical Engineering
FUTURE WORK
To increase the performance and accuracy of the ASLT, the quality of the training database used should be enhanced to ensure that the ASLT picks up correct and significant characteristics in each individual sign and further improve the performance more efficiently.
Current collaboration with Assistive Technology researchers and members of the Deaf community for continued design work should be considered for continued progress.
This project did not focus on facial expressions although it is well known that facial expressions convey important part of sign-languages.
This system can be implemented in many application areas examples include accessing government websites whereby no video clip for deaf and mute is available or filling out forms online whereby no interpreter may be present to help.
Dept of Biomedical Engineering
REFERENCES Andreas Domingo, Rini Akmeliawati, Kuang Ye Chow ‘Pattern Matching for Automatic
Sign Language Translation System using LabVIEW’, International Conference on
Intelligent and Advanced Systems 2007.
Beifang Yi Dr. Frederick C. Harris ‘A Framework for a Sign Language Interfacing
System’, A dissertation submitted in partial fulllment of the requirements for the degree
of Doctor of Philosophy in Computer Science and Engineering May 2006 University of
Nevada, Reno.
Helene Brashear & Thad Starner ‘Using Multiple Sensors for Mobile Sign Language
Recognition’, ETH - Swiss Federal Institute of Technology Wearable Computing
Laboratory 8092 Zurich, Switzerland flukowicz, junker [email protected]
Dept of Biomedical Engineering
Jose L. Hernandez-Rebollar1, Nicholas Kyriakopoulos1, Robert W. Lindeman2 ‘A
New Instrumented Approach For Translating American Sign Language Into Sound
And Text’, Proceedings of the Sixth IEEE International Conference on Automatic
Face and Gesture Recognition (FGR’04) 0-7695-2122-3/04 $ 20.00 © 2004 IEEE.
K. Abe, H. Saito, S. Ozawa: Virtual 3D Interface System via Hand Motion
Recognition From Two Cameras. IEEE Trans. Systems, Man, and Cybernetics, Vol.
32, No. 4, pp. 536–540, July 2002.
Dept of Biomedical Engineering
THANK YOU