1

COMPUTER INTERFACE FOR THE PHYSICALLY

DISABLED WITH CUSTOM KEYBOARD AND

VOICE CONTROL

PRODUCT DESIGN LAB -2 REPORT

Submitted by

Team A

SHASHANK BHOSALE ED12B012

SAGAR JOSHI ED12B024

KISHAN JANI ED12B026

RANJIT VHANAMANE ED12B054

SUJATA KHANDARE ED12B057

2

DEPARTMENT OF ENGINEERING DESIGN

INDIAN INSTITUTE OF TECHNOLOGY MADRAS.

JANUARY - MAY 2015

CERTIFICATE

This is to certify that the project titled COMPUTER INTERFACE FOR THE

PHYSICALLY DISABLED WITH CUSTOM KEYBOARD AND VOICE

CONTROL, submitted by Shashank Bhosale, Sagar Joshi, Kishan Jani, Ranjit

Vhanamane and Sujata Khandare to the Indian Institute of Technology Madras,

Chennai is the bona fide record of the project work done by them under my supervision.

The contents of this report, in full or in parts, have not been submitted to any other

Institute or University.

Dr.Sandipan Bandyopadhyay

Project Guide

Assistant Professor

Dept. of Engineering Design Place: Chennai

IIT-Madras, 600 036

Date: 28th

April 2015

ACKNOWLEDGEMENTS

We would like to express our sincere thanks of gratitude to Dr. Nilesh J.Vasa, HoD, Dept. Of

Engg. Design, who gave us this golden opportunity to do this wonderful project. We thank

Nethrodaya and National Institute for the Visually Handicapped for the comments and support at

all times that helped us improve the design of the product. We also thank Electronics Lab of

Dept. Of Engg. Design, IIT Madras for all the assistance.

4

ABSTRACT

Visually impaired people and users with only one arm/hand often find it difficult to use

computers as they cannot type and navigate efficiently. So an input device which is single

handedly operated integrated with voice control to navigate easily over the computer is needed.

A one hand glove keyboard as an input device integrated with a software (named Aarya) having

Text to Speech(TTS) and Speech to Text(STT) functions is introduced to solve the problem

stated above. Human beings have an intrinsic sense of the body parts location. Hence, for a

visually impaired person, it will be easy to identify keys with little practice. Aarya eliminates the

need of graphical user interface which is redundant for visually impaired users.

KEYWORDS:

Visually impaired, Glove, Aarya, Switches, Conductive fabric, Aluminum foil, Conductive tape,

Ergonomics, Python, Optimization.

5

TABLE OF CONTENTS

ACKNOWLEDGEMENTS

ABSTRACT

TABLE OF CONTENTS

LIST OF FIGURES

1. PROBLEM STATEMENT 9

2. MOTIVATION 10

2.1 Visually impaired users

2.2 Users with one arm

2.3 Normal people as users

3. SPECIFICATIONS 13

3.1 Input Keyboard

3.2 Digital Assistant Aarya

4. ADVANTAGES 17

5. CUSTOMERS 18

6. APPROACH AND CHALLENGES FACED 21

6.1 Glove Keyboard (Hardware)

6.1.1 Concept Generation

6.1.1.1 Glove using switches

6.1.1.2 Conductive fabric and thread

6.1.1.3 Flexible PCBs

6.1.1.4 Aluminum foil on Leather Glove

6.1.1.5 Aluminum foil on Cotton Glove

6.1.1.6 Aluminum foil stuck using Fevicol on cotton Glove

6.1.1.7 Aluminum foil stuck using Anabond on Cotton glove

6.1.1.8 Conductive tape on Cotton Glove

6.1.2 Concept Screening

6.1.2.1 Cotton Glove with Conductive tape

6.1.2.2 Leather Glove with Aluminum foil

6.1.3 First Prototype

6

6.1.4 Final Prototype

6.2 Aarya (Software)

6.2.1 Software for the Keyboard

6.2.2 Features of the Code

7. ERGONOMIC ANALYSIS 39

7.1 Word-Letter Analysis

7.2 Formulating an Optimization Problem

7.3 Significance of Quadratic Assignment Problem

7.3.1 Solving the Linear Part

7.3.2 Solving the Non-Linear Part

8. USER TRIALS 45

9. CRITICAL ANALYSIS AND SCOPE FOR IMPROVEMENT 46

10. APPENDIX 48

7

LIST OF FIGURES

Figure 1. A keyer

Figure 2: A Microwriter MW4 (circa 1980)

Figure 3: One handed QWERTY keyboard

Figure 4: Front view of the Glove

Figure 5: Top view of the Glove

Figure 6: Flow chart for the working of glove

Figure 7: Algorithm for speech to text and action taken after that

Figure 8: Classification of customers

Figure 9: Glove having reset switches

Figure 10: Petite switches

Figure 11: Plush Game Controller with Flora & Conductive Fabric

Figure 12: Flex PCBs

Figure 13: Cotton Glove with patches made using aluminum foils

Figure 14: Glove with patches stuck using Anabond

Figure 15: Glove using conductive tape patches

Figure 16: Glove with wires connected to the conductive tape patches

Figure 17: First prototype – top view

Figure 18: First prototype – front view

Figure 19: Circuit board for the Glove

Figure 20: Final prototype - front view

Figure 21: Final prototype – top view

8

Figure 22: Final prototype with circuit board - Front view

Figure 23: Final prototype with circuit board - Top view

Figure 24: Voltage diagram

Figure 25: Aarya Arduino code 1

Figure 26: Aarya Arduino code 2

Figure 27: Aarya python program showing various cases.

Figure 28: Position-comfort and frequency values.

Figure 29: The Cij matrix (26x26)

Figure 30: The final assignment matrix (26x26)

Figure 31: Optimization in MS-Excel

Figure 32: User 1

Figure 33: User 2

9

1. PROBLEM STATEMENT

To develop an input method in the form of a glove keyboard having switches/keys on it with

all the necessary inputs and functions.

To develop a supporting digital assistant software ‘Aarya’ that will perform the following

functions

1. Process the keyboard interrupts from the designed keyboard.

2. Provide Text To Speech (TTS) output.

3. Incorporate Speech To Text (STT) for voice commands and implement computer

navigation using that.

Design the product from two significant user perspectives:

1. Visually impaired users

2. Users with one arm

10

2. MOTIVATION

2.1 Visually impaired users:

Corneal Blindness is one of the most common causes of blindness in India. India

shoulders the largest burden of global blindness, about 3.5 million across the country

with 30000 new cases being added each year.

Visually impaired people often find it difficult to use computers as they cannot type and

navigate efficiently

There is a need of an input device which is single handedly operated and integrated with

voice control to navigate easily over the computer

Hence an economical glove keyboard will help millions of people who have been

secluded from the world of computers.

Softwares like NVDA (Non Visual Desktop Access), JAWS (Job access with speech) are

available, but the user has to use the conventional keyboard which varies with size and

takes a long time getting used to.

All the softwares use English as the basic language of communication, but most of the

visually impaired people are poor and lack education (specifically English).

Most of the people are comfortable with regional languages but no interface available

which meets such requirements.

British/US English accent is difficult for the user to use efficiently.

During typing they need to map whole QWERTY keyboard in their mind and type

cautiously (hence slowly).

In case of change in keyboard, due to varied sizes, it is initially difficult for them to type

without errors.

11

Humans have an intrinsic sense of body location; hence visually impaired users can

easily locate different spots on the fingers. This feature can be exploited to design a glove

keyboard so that the user would commit fewer errors as the key location would be known

exactly.

Speech to text facility (currently not available) would enable easier navigation.

With a blue tooth module, this project can be expanded to smartphones

2.2 Users with one arm:

The QWERTY keyboard is designed for typing with two hands. Existing solutions are

not of much help as they do not reduce the efforts significantly.

A keyset or chorded keyboard is a computer input device that allows the user to enter

characters or commands formed by pressing several keys together, like playing a "chord"

on a piano. The large number of combinations available from a small number of keys

allows text or commands to be entered with one hand, leaving the other hand.

Figure 1: A keyer Figure 2: A Micro writer MW4 (circa 1980)

The above devices are not economical and require a lot of practice to develop an

expertise. Many stenotype users can reach 300 words per minute.

However, stenographers typically train for three years before reaching professional levels

of speed and accuracy.

12

Developing a glove keyboard for user lacking one hand would give them a better option

than the laid out QWERTY keyboard layout which is designed for two hand use.

This gives a portable option to the conventional keyboard.

2.3 Normal People as users:

One handed typing can be useful in situations where one would usually need to move

their dominant hand between one's keyboard and mouse or if the typist makes notes while

using one hand for another task.

However, one-handed typing is slower than two-handed typing. One-handed typing can

also become tiresome after long periods of time

Figure 3: One handed QWERTY keyboard

This product can be used for gaming purposes where user can keep his other hand free

and have as many as 24 different inputs at least in a hand.

This product can be expanded to smartphone keyboard, and other different areas like

glove piano

13

3. SPECIFICATIONS

Glove keyboard is an input device having keys/switches on the glove with a microcontroller

to communicate the input to the computer

Initial design had 26 switches which contains A-Z letters (capital and small), space,

backspace key and special characters

It has onboard switch to start speech to text software Aarya, which is accountable for easy

computer navigation

Aarya is basically a digital assistant to help visually impaired people accomplish tasks like

computer/web navigation as well as giving audio feedbacks for inputs entered by doing text

to speech (TTS) and speech to text (STT) processing

Keyboard module is integrated with Aarya such that as soon as a letter is typed from

keyboard, it is immediately pronounced by the computer

Aarya also has feature of text to speech which gives a voice output of the word typed

immediately after the space button is hit

The product developed is an innovative computer interface designed keeping in mind the needs

of the physically disabled people, but can be used by a normal user as well.

The product consists of the following:

3.1 Input Keyboard:

The keyboard developed is a glove keyboard. The glove has copper patches stitched, using

copper wires, at strategic locations on the fingers (6 on each of the finger) which act as keys.

The keys are pressed once the patch on the thumb makes a contact with the patch on the

14

fingers. The patches are placed taking into account the regions where the thumb is dexterous

enough and the user requires minimum effort while typing.

Very thin copper wires are connected to each of the patches and are woven through the glove

to reach the wrist portion of the hand where it is connected to a custom circuit board. An

Arduino micro –controller attaches itself on the circuit board and transmits the information to

the computer through serial communication.

This is a unique low cost hardware. Care is taken that the glove is tight enough on the hand

and the padding is optimum. This will play a crucial role for visually impaired users as they

need to identify one switch position from the other and the thumb needs to be dexterous

enough.

Figure 4: Front view of the Glove Figure 5: Top view of the Glove

15

Following Algorithm is used for implementing the keyboard inputs.

Figure 6: Flow chart for the working of glove

3.2 Digital Assistant Aarya:

Aarya is a custom software developed for processing the information sent by the micro-

controller.

It performs three main functions:

a) Processing the keyboard interrupts: Once the microcontroller sends information

regarding key press, the software accesses windows hooks to send the corresponding

interrupt. If the key is pressed for a longer time, the micro-controller transmits new signal

and corresponding capital case letter is pressed, if long pressed still a special character is

entered.

16

b) Text to speech: With every letter entered Arya speaks out the letter pressed for the

visually impaired users. After spacebar is pressed, the whole word is spoken.

c) Speech to text: The software accesses the computer’s on-board mike and records the

voice command spoken by the user. Then the file is sent to the voice processing engine.

The text string returned is processed and appropriate action is taken.

Figure 7: Algorithm for speech to text and action taken after that.

17

4. ADVANTAGES

Synchronization of hardware (Glove keyboard) and software (Digital assistant having Speech

to Text and Text to Speech facility) in the most efficient way possible by combining usage of

two senses – tactile and hearing along with speaking ability.

Arrangement of letters over glove is done using ergonomic analysis for attaining fast typing.

User friendly design: Even if the user changes the glove his hand remains the same and

henceforth he (or she) won’t feel any change in his navigation.

Cost effective design by using insulated copper wires and the contact patches instead of

costly conducting fabric.

The keyboard being single handed, visually impaired can use their other hand free for

reading braille texts.

One handed users can use this keyboard better due to inherent discomfort for them while

using laid out standard keyboard.

18

5. CUSTOMERS

Visually Impaired

User survey was undertaken at Nethrodaya, a Public charitable trust for persons with disabilities.

Following information was obtained:

Facilities available at Nethrodaya:

o Braille teaching center.

o School for visually impaired

o Library with braille printer.

o Computer lab facility with internet.

Their computer lab:

o Computers with internet facilities.

o Soft wares like NVDA (Non Visual Desktop Access), JAWS (Job access with speech)

installed.

o Typical QWERTY keyboard.

o ‘Text to speech’ available but no use of ‘Speech to text’.

Troubles reported by them:

o All the softwares use English as the basic language of communication but most of the

visually impaired people being poor, lack education (specifically English).

o Most of them are comfortable with regional languages but no interface available which

meets such requirements.

o British/US English accent difficult for user to use efficiently.

19

o During typing they need to map whole QWERTY keyboard in their mind and type

cautiously (hence slowly).

o While typing bigger statements (like if else conditional ladder in excel) they often

forget where they actually are. In these scenarios, Dynamic braille display would

have been helpful to catch last few characters.

o In case of change in keyboard, due to varied sizes, it is initially difficult for them to type

without errors.

Conclusions

o Accommodating Indian English accent in Aarya.

o Palm glove keyboard can be useful as it won’t be difficult to remember the key

positions.

o As palm of hand remains of constant size, it won’t be a problem glove is changed. Users

have an inherent sense of location of finger locations.Current softwares don’t have

speech to text, so it will be worth to integrate it in Aarya.

20

Figure 8: Classification of customers

21

6. APPROACH AND CHALLENGES FACED

6.1 GLOVE KEYBOARD (HARDWARE)

Aim:

The primary aim was to:

Integrate 26 switches on one palm of the hand.

Make a clean design so that the user is not hindered while operating.

6.1.1 Concept Generation

6.1.1.1 Glove using switches:

The glove had switches on the fingers to act as actuators for each letter. The switches were

first soldered onto a GCB with wires attached and then using the holes on the GCB, it was

stitched onto the glove.

Figure 9: Glove having reset switches

22

Problems faced:

After wearing on hand, orientation of switches got disrupted due to

shapes/width of fingers of different people.

Too many wires to pass from hand (approximated to 60+ if this switch is used)

Switch seemed too bulky. (Target was to accommodate 5 switches on one finger)

Bulkiness of switch was providing unwanted rigidity to fingers would have led to very

uncomfortable product once all switches were attached.

Solutions attempted:

For switches orientation: Using glove with hard fabric on finger part, typically used in

motorbike races.

For bulkiness: Using small sized switches.

Figure 10: Petite switches

6.1.1.2 Conductive fabric and thread:

Conductive fabric being stitched on a glove could solve the problem of large number of

wires. This is because here threads itself will serve as wires and number of wires required

will reduce to half if put contact patch switch on thumb.

23

It is a fabric which can conduct electricity. It consists of a non-conductive substrate

(Cotton/nylon), which is then either coated or embedded with electrically conductive

elements (nickel/copper) and then woven into the construction of the textile.

Here, patches of fabric stitched on the glove act as switches and threads (extracted from

the fabric) act as wires, therefore reducing the bulkiness of the glove by a huge amount.

Figure 11: Plush Game Controller with Flora & Conductive Fabric

6.1.1.3 Flexible PCBs:

Designing a PCB in the shape of fingers and putting it inside the glove could be attempted.

Figure 12: Flex PCBs

The above 2 solutions were not implemented due to their high cost.

24

Having a relatively hard fabric on the palm side of the glove may solve the orientation problem.

But this could have made the glove even more bulky. So, now the main focus was an alternative

for Conductive fabric.

6.1.1.4 Aluminum foil on Leather Glove:

Aluminum foil was used instead of the fabric and a few layers of it were stitched onto the

glove (6 patches per finger). A patch of copper was also stitched (using copper thread) as

an alternative for threads of the fabric.

A similar patch was made on the thumb tip, so that when it touches any patch on a finger,

the circuit is completed and the letter is typed. This time the glove was made of an elastic

cloth such that it fits any hand and is flexible.

The fact that it was a smooth cloth, stitching on it was easier. Various configurations of

patches on the finger and also the material that is best suitable for the patch was

experimented with

Configuration selected for placing the keys on the fingers consists of 6 keys each on the

finger and on the sidewalls of the glove rather than front and back of each finger

The advantage of this is that it is more convenient, ergonomic and faster.

6.1.1.5 Aluminum foils on cotton glove:

Layers of aluminum foil were stitched to make it act as patches.

Difficulty was faced in sewing using hemming stitch (or any other kind of stitch) as both

glove and foil were too thin

As the area available was insufficient on the planar side of the glove, it was difficult to

attach 6 keys on each finger. Touching the keys attached on the back side with thumb

was an onerous job.

25

Figure 13: Cotton Glove with patches made using aluminum foils

Attaching the keys on the back side was avoided by stitching the keys on those parts of

the finger where one finger comes in contact with the other. This formation, although

user friendly was difficult to stitch as it was difficult to maintain the orientation of the

aluminum foil with respect to the curved part of the finger and already stitched part of the

glove while sewing simultaneously.

6.1.1.6 Aluminum foil stuck using Fevicol on Cotton Glove:

Bond formed using fevicol was not strong enough to hold the fabric and the aluminum

foil together.

6.1.1.7 Aluminum foil stuck using Anabond on Cotton Glove:

Using anabond got us close to perfection but the cotton glove was too thin for the

anabond to not seep in side. This made the two layers of the glove to stick with each

other.

The area on which anabond was used no more remained flexible and curved and got

hardened. This made it difficult to insert and move fingers freely.

26

Figure 14: Glove with patches stuck using Anabond

6.1.1.8 Conductive tape on Cotton Glove

A conductive tape (made up of aluminum) was stuck on the cotton glove to act as

keys/patches

The patches were cut in circular fashion to prevent the tape from coming out from the

glove

Figure 15: Glove using conductive Figure 16: Glove with wires connected

tape patches to the conductive tape patches

27

6.1.2 Concept Screening

After experimenting with the above gloves, and comparing the advantages and

disadvantages of each of them, the following ideas were selected.

6.1.2.1 Cotton gloves with conductive tape

Pros:

The conductive tape is easy to stick.

Thin and tight cotton glove allows free movement of fingers (especially thumb) and

minimum loss in intrinsic sense of the location of keys on fingers.

Cons:

Copper wires can tear apart the extremely thin conductive tape in case of some load or a

sudden jerk.

Lack of available area results in a severe restriction on the number of keys on each finger.

6.1.2.2 Leather gloves with aluminum foils.

Pros:

Using leather glove solved the problem of the seepage of Anabond through it.

Sufficient area is available on each finger for sticking 8 aluminum foil keys with optimal

distance between each key.

Cons:

The leather glove is too bulky to allow free movement of fingers (especially the thumb) and

as a result, seriously affecting the intrinsic sense for locating keys on fingers.

28

6.1.3 FIRST PROTOTYPE

Hardware:

Conductive tape with wires:

A conductive tape (made up of aluminum) was stuck on the cotton glove to act as

keys/patches. Wires were stitched on the patches.

Figure 17: First prototype – top view Figure 18: First prototype – front view

Pros:

The hardware developed had all the basic functionalities.

There was no short circuit in the electronics.

It was comfortable handling, wearing and removing the glove.

Cons:

There were a lot wires sticking out from the GCB to the contact patch location on the finger

which was aesthetically not appealing and made the usage slightly uncomfortable.

The prototype was not robust enough and the wires soldered at the patches gave away when

the tension increased when the user flexed his/her fingers.

There was no tactile feedback to give the patch locations.

29

Electronics:

Circuit board:

30 wires were soldered to GCB to reduce the number of connections.

Insulated copper wires were used to prevent short circuit.

The thin layer of insulation was removed using a sharp blade at the ends, hence the ends of

the wire were non- insulated

Figure 19: Circuit board for the Glove

30

6.1.4 FINAL PROTOTYPE

Hardware:

Copper wires as buttons:

o The aluminum foil/conductive tape were replaced with copper wire buttons formed from

weaving the wires in a circular shape. This solved the earlier problem of the splintering of

the weak soldered joints and loosening of the conductive tape with weak adhesive

o Thin wires were woven in the fabric of the glove creating a nerve like structure and thus

preventing intertwining of the wires

o The new glove is sturdy and robust as compared to the previous one

Figure 20: Final prototype - front view Figure 21: Final prototype – top view

Electronics:

A GCB with bergs for the Arduino was made.

Two switches, one for power and other for voice control was integrated.

A power LED was soldered

31

Figure 22: Final prototype with circuit Figure 23: Final prototype with circuit

board - Front view board - Top view

Advantages of the final prototype:

It is robust and aesthetically more appealing.

It is more comfortable to use with patches kept at more ergonomically sound positions.

The final prototype also provided hardware necessary for integrating Aarya and the glove

keyboard.

32

6.2 AARYA (SOFTWARE)

Aim:

The primary aim was to:

Make the program robust so as to handle exceptions.

To implement parallel model whereby each of the two devices has a separate ‘Aarya’ script

and there is intercommunication.

6.2.1 Software for the keyboard:

The 30 berg pins are attached on the Arduino. All these pins are declared as

INPUT_PULLUP. This is useful for avoiding a floating state and uncertainty in the pin

digital state.

The thumb is in LOW (grounded) state. When the conductive patch on the finger and the

thumb make a contact, the digital pin reads a low and using serial communication this

information (pin and the corresponding alphabet) is passed on to the computer.

This information is received by software Arya written in Python. It receives this info from

Arduino and using windows hook an interrupt corresponding to the alphabet is passed and

the key is pressed.

6.2.2 Features of the code:

Algorithm:

Each patch (and the corresponding pin) is high (5 V) in the usual state. These are configured

as high input resistance input pins through a pull up resistor to avoid a floating state.

The thumb patch is given a low voltage (GND).

33

When the thumb touches a patch the corresponding pin goes to low and the character

assigned to that pin is transferred via serial communication to the computer.

Problems:

To avoid multiple key presses and choking of serial communication a delay has to be

introduced.

No further features can be added with this event driven algorithm.

Solution:

Instead of event based (pin going from high to low) a time based algorithm based on the

transitions in the pin voltage was implemented.

Algorithm:

A time based system was devised. When the pin goes low for the first time send the small

case char.

If the pin remains low (key is kept pressed) the capital case char is sent.

If the key is kept pressed still (beyond a certain time) send a corresponding special character

(like! @, # etc.)

This information will be sent to Arya, the digital assistant, which will first type small char, if

kept pressed, backspace will be pressed then the capital letter will be pressed, if kept pressed

still again backspace key will be pressed and after that the special char will be entered.

34

Voltage time diagram for the algorithm:

The transition periods (the vertical lines) are kept at 1.2 sec and 2.4 sec respectively

Figure 24: Voltage diagram

Code implanting the algorithm:

The ‘sendchar’ function

o It uses inbuilt millis function to calculate the required time intervals

o The transitions are monitored using iv (initial value) and cv (current value) flags and

updated consequently. Logic is developed using the same.

35

Figure 25: Aarya arduino code 1

36

Figure 26: Aarya arduino code 2

37

Depending on the length of the character sent by the arduino, Aarya enters the appropriate input.

If the length of data is one then keyboard hooks are invoked and the key is pressed.

If the length is two, and first letter is ‘s’, then backspace is first pressed and then the

second character transmitted along with the first ‘s’ is entered (here the ‘s’ is ignored and

is used only for identification). This is used for capital letters and special characters.

If ‘as’ is transmitted then space is pressed.

If ‘ab’ is transmitted then backspace is pressed.

If ‘aa’ is transmitted then voice control is activated.

If ‘ax’ is transmitted then program and the serial communication is terminated.

‘word’ array is declared so that Aarya speaks out the whole word when spacebar is

pressed.

38

Figure 27: Aarya python program showing various cases.

39

7. ERGONOMIC ANALYSIS

In order to increase the typing rate for the keyboard and to ensure that minimum effort is applied

to while typing, ergonomic analysis was done for the glove keyboard.

Two aspects were considered:

Finding points in the workspace (keys on hand) where the thumb is not as dexterous/ it is

difficult to reach.

Doing the word analysis, finding a word combination, letter frequency for the English

language.

For the first part, all the key positions were rated based on the comfort of the thumb to reach that

position and whether it is reaching the position in the required orientation.

7.1 Word/Letter analysis:

A python code was written to achieve the following things Import a text body and perform

character manipulations.

Return a result indicating how many times a letter occurs and 2 letter combinations occur (eg.

‘an’, ‘cb’) also find the maximum of each result.

Every combination from aa - zz is considered.

The text body used is a random collection of sentences.

Result format: the numbers in the square brackets indicate the character frequency from a-z.

Following aspects were to be considered:

Comfort index of a particular position

One letter frequency.

Distance between any two patches.

Two letter frequency between any two letters.

40

7.2 Formulating an optimization problem:

Considering the position index (cj) for each position (based on comfort) frequency index (fi) for

each letter (based on letter analysis), an assignment problem was formulated for obtaining an

optimized layout for this keyboard. Also, two letter combinations were considered in the

problem (for e.g. ‘e’ should be in the vicinity of ‘r’)

Cij= Fi*cj

Fi => frequency index of letter i

Cj => comfort index of position j

TFik => two letter frequency index of letters k and i

Djl => distance index between position j and l

We need the maximize the given objective function subject to conditions that

Every letter is allotted to only one position

Every position is allotted only one letter

This is a quadratic assignment problem (QAP)

41

7.3 Significance of the Quadratic Assignment Problem (QAP):

The QAP gives us a mathematical way of getting an optimum layout.

It is unique way which has not been implemented previously (as far as we know) and takes

into account the four major factors as listed above.

The QAP model can be applied to any keyboard.

Comfort index can be obtained either using surveys or a particular user can be asked for his

comfort indices for the set of positions and the QAP can be solved to obtain a personal custom

layout.

7.3.1 Solving the Linear part (Assignment problem):

Considering the comfort index and one letter frequency, the following data was obtained (as

shown in the next page)

42

Figure 28: Position-comfort and frequency values.

Figure 29: The Cij matrix (26x26)

43

Figure 30: The final assignment matrix (26x26)

7.3.2 Solving the Non-Linear QAP:

The quadratic assignment problem (QAP) is one of the fundamental combinatorial

optimization problems in the branch of optimization or operations research in mathematics,

from the category of the facilities location problems.

The problem is NP-hard, so there is no known algorithm for solving this problem in

polynomial time, and even small instances may require long computation time. It was also

proven that the problem does not have an approximation algorithm running in polynomial

time for any factor, unless P = NP.

The travelling salesman problem may be seen as a special case of QAP if one assumes that

the flows connect all facilities only along a single ring, all flows have the same non-zero

(constant) value. Many other problems of standard combinatorial optimization problems may

be written in this form.

Due to the problem size (576 variables)/ or for the lack of better formulation in excel, the excel

solver was unable to converge to a feasible solution.

44

Thus as a future work, it is needed to formulate the QAP in matlab, or search for better ways of

formulating the problem if possible.

Figure 31: Optimization in MS-Excel

45

8. USER TRIALS

User trial was conducted on 30 visually impaired students in The National Institute for the

Visually Handicapped, Poonamalle.

The outcomes (and user feedback) of the user trial were as follows

The middle three fingers were easily reachable while the switches on the little finger were

difficult to reach.

The switches on the right side of each finger could be easily operated while the operation of

the left side switches was little arduous.

The users suggested embossed button switches for easy identification of switch locations.

Variable size of each individual’s palm caused difficulties in wearing and the usage.

Users with relatively small thumbs compared to the other fingers found it difficult to reach

all switch locations.

Users demanded an identification mode to identify each alphabet.

The thumb switch position was shifting as a result of glove slippage.

USB cable was constraining the movement of palm.

Figure 32: User 1 Figure 33: User 2

46

9. CRITICAL ANALYSIS AND SCOPE FOR IMPROVEMENT

Inclusion of all the special keys like function, tab etc. is difficult on the glove as the number

of keys that can be accommodated is limited: Number of characters per finger can be

increased to 8 or the time press logic can be implemented to achieve this.

The user is unable to read input he gave in the past. There is a need to incorporate a third

module which is integrated with the glove keyboard and Aarya such that it displays last few

words/letters in braille through a tactile feedback. It can be used by the free hand of the user.

User needs to get accustomed to this keyboard to this keyboard for effortless use and

maximum efficiency: Intuitive and ergonomic layout necessary

There is a need to incorporate a custom micro controller built for glove keyboard :This will

be more economical as well as compact

Current hardware can be enhanced by introducing blue tooth feature for wireless interface. In

this case application of current product can be extended to smartphones and tablets as well.

This will increase the user spectrum. However cost of the product will also increase.

47

REFERENCES

http://www.netradaan.com/blind-statistics-india.php

http://timesofindia.indiatimes.com/india/India-has-largest-blind-

population/articleshow/2447603.cms

http://en.wikipedia.org/wiki/One_hand_typing

http://en.wikipedia.org/wiki/Chorded_keyboard

http://en.wikipedia.org/wiki/Dvorak_Simplified_Keyboard

http://www.tendtronic.com/Flexible-PCB

https://www.youtube.com/watch?v=qr3Ke57s3gU

https://learn.adafruit.com/plush-game-controller/overview

Special thanks to Mr. Krupananadam Yasarapudi , Regional Director, National Institute

for the Visually Handicapped, for his help during the user trials.

48

APPENDIX

Cost Projection

Overall project:

Arduino – INR 3270

Other electronics (GCB, pins, bergs, copper wires, switches etc) – 300

Glove (Different variants) – 600

Logistics (Stationary, Sewing kit, Transport etc) – 500

Total (till date) = INR 4670

Targeted cost of final product

INR 2500 (Arduino can be replaced with a custom Micro-controller, which will reduce the cost)