SHOW & TELLSIGN LANGUAGE TRANSLATOR GLOVE

Saron Paz Zvika Markfeld

Tomer Daniel Oleg Imanilov

WHAT IS IT?An open source software & hardwareCombining: Arduino Sensors Android ADK Android

WHAT IT DOES? A glove that translates a predefined

vocabulary of the deaf and mute sign language into text

HOW DOES IT WORK?

Lily-Pad Arduino samples sensors attached to glove

ADK board relays data from the Lily-Pad to Android

Android uses a back propagation Neural Network to analyze data

WHAT IS LILY-PAD?

Arduino compatible microcontroller board designed for wearables and e-textiles

GLOVE SENSORS

Flex Sensors measure finger bending

Wii Nunchuck = Accelerometer + Gyros

Conductive tape detects fingers touching

FLEX SENSOR Flex Sensor changes

resistance when bent

Using a voltage divider the voltage is read by Arduino ADC (10 bits)

WII NUNCHUCK

WII NUNCHUCK SENSORS

2 Buttons

Joystick

Accelerometer

(X,Y,Z)Gyros

(Roll, Pitch)

I2C & Power

WHAT IS ANDROID ADK?

WHAT IS A NEURAL NETWORK? Interconnected group

of artificial neurons Good at

pattern recognition

SOFTWARE VIEW

ArtificialNeural

Network

GUI

Relay

ADK library ADK library

SensorControl

Serial

Serial

HOW WE BUILT IT?

Click icon to add picture

PROTOTYPING FLEX SENSORS

PROTOTYPING FLEX SENSORS

FLEX SENSOR OUTPUT

ADDING NUNCHUCK SENSORS

THE PROBLEM WITH ASL

THE FINAL GLOVE

CHALLENGES How to position flex sensors?Fix tip to top and pass through sewn

loops How to distinguish between

R,U,V,S,E,T,N?Conductive tape detects fingers touching How to distinguish between each letter?Time based How to record movement?Freeze in the middle

DEMO

WHO ARE WE?

Saron Pazis an

experience designer & a hopeless

idealist

Zvika Markfeld

is a design

freshman and an aspiring inventor

Tomer Daniel

is a software architect

by day & a maker by

night

Oleg Imanilov

is a software

engineer & a brilliant

ideas generator

REFERENCE Show & Tell Google code project

http://code.google.com/p/adk-sign-language-translator-glove/

Sensors Flex Sensors

http://www.sparkfun.com/products/8606 http://www.sparkfun.com/products/10264

Nunchuck http://www.dealextreme.com/p/designer-s-nunchuck

-controller-for-wii-24529

Arduino Wiichuck class

http://www.arduino.cc/playground/Main/WiiChuckClass

Android ADK http://developer.android.com/guide/topics/usb/adk.h

tml