asllenge
DESCRIPTION
ASLLENGE. He Zhou Hui Zheng William Mai Xiang Guo Advisor: Professor Patrick Kelly. Outline. Background System Requirements Design Challenges System Overview Hardware and Software Team Roles Timeline Q/A. 2. Background. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/1.jpg)
Department of Electrical and Computer Engineering
He ZhouHui ZhengWilliam MaiXiang Guo
Advisor: Professor Patrick Kelly
ASLLENGE
![Page 2: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/2.jpg)
Department of Electrical and Computer Engineering
Outline Background System Requirements Design Challenges System Overview Hardware and Software Team Roles Timeline Q/A
2
![Page 3: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/3.jpg)
Department of Electrical and Computer Engineering
Background Leverage Kinect technology to recognize and
offer feedback on American Sign Language ASL is the one of most commonly used language
in US(.5 to 2 million population use ASL as their primary form of communication)
Educational learning tool for ASL
3
![Page 4: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/4.jpg)
Department of Electrical and Computer Engineering
Motivation Helping hearing impaired people to learn ASL. Facilitate communication between ASL speaker
and non-speaker
4
![Page 5: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/5.jpg)
Department of Electrical and Computer Engineering
System Requirements Create a user-friendly interface that recognize
sign language and offer feedback• Software must be easy to use
Display translated word or phase being demonstrated by user• Provide quality feedback to user concerning
the accuracy of ASL Cost should be within the budget
5
![Page 6: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/6.jpg)
Department of Electrical and Computer Engineering
Design Challenges Human Factors Background Interferences Finger Gestures Hardware limitation• Kinect only has 320x240 resolution and 16 bit
depth
6
![Page 7: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/7.jpg)
Department of Electrical and Computer Engineering
Kinect Has a depth and regular Camera Associated with software development kit
• Able to track a human being.• Use computer generated algorithms
to form human skeleton data points Track upper body movements
7
![Page 8: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/8.jpg)
Department of Electrical and Computer Engineering
System Overview
8
![Page 9: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/9.jpg)
Department of Electrical and Computer Engineering
Video demonstration(Hand Rotation)
9
![Page 10: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/10.jpg)
Department of Electrical and Computer Engineering
Video demonstration(Hand Crossing)
10
![Page 11: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/11.jpg)
Department of Electrical and Computer Engineering
Video demonstration(Hand behind Head)
11
![Page 12: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/12.jpg)
Department of Electrical and Computer Engineering
Video demonstration(Hand behind Back)
12
![Page 13: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/13.jpg)
Department of Electrical and Computer Engineering
Hardware Implementation 1 Increasing the flex sensor
output voltage with bend Voltage output goes into
an AD converter board AD converter digitizes the
output signal PC captures the signal via
USB or Bluetooth
13
![Page 14: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/14.jpg)
Department of Electrical and Computer Engineering
Hardware Implementation 2 Kinect Camera captures the raw image and the
depth image data The data send to PC to be processed into skeleton
tracker Combining the signal from flex sensor and search
in database for matching motions
14
![Page 15: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/15.jpg)
Department of Electrical and Computer Engineering
Software Design Flow
15
![Page 16: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/16.jpg)
Department of Electrical and Computer Engineering
Development Model
16
![Page 17: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/17.jpg)
Department of Electrical and Computer Engineering
Cost of Major Components
Name of Component
Flex Sensor
A/D Converter
Board
A Pair of Gloves
Kinect Sensor
Bluetooth Module
Quantities 10 1 1 1 1
Unit Price $12 $50 $10 $150 $20
Total Price $350
17
![Page 18: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/18.jpg)
Department of Electrical and Computer Engineering
Team Roles He - test and integrate software prototype
Hui- image processing and programming
William - build microcontroller to detect finger and palm movements
Xiang- website, overall testing and building ASL database
18
![Page 19: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/19.jpg)
Department of Electrical and Computer Engineering
MDR deliverablesThe goal for our MDR is to be able to detect and track arm and hand positions, possibly be able to translate simple hand gestures
19
![Page 20: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/20.jpg)
Department of Electrical and Computer Engineering
ConclusionOur project is good for SDP:
• Contains both hardware and software• Expandable: database, audio recognition, braile
Our project is a good product:• Aid the disabled people• Personal/commercial use• Relatively inexpensive
20
![Page 21: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/21.jpg)
Department of Electrical and Computer Engineering
TimelinePDR
Purchase PartsPlanning
GUIInterface with the Sensors
Build Finger SensorsBuild Test Database
Prototype 1MileStone MDR Deliverables
System IntegrationWinter Break
Expand DatabasePrototype 2
MileStone CDR DeliverablesAlpha Testing
UpdateBeta TestingSpring Break
Finalize ProjectMileStone FRP Deliverables
10/16/2011 12/5/2011 1/24/2012 3/14/2012 5/3/2012
21
![Page 22: ASLLENGE](https://reader033.vdocuments.net/reader033/viewer/2022051821/56815d9e550346895dcbc4b9/html5/thumbnails/22.jpg)
Department of Electrical and Computer Engineering
Q&A
22