interpreter for visually impaired

7/30/2019 interpreter for visually impaired

1/40

Interpreter/Smart GlassForVisually Impaired

Guided By,Mrs J Jeyarani M.E., A.P.(Se.G)

Processed By,S AnbarasanS ArunrajG BharanidharanJ Jesuraj


2/40

INTRODUCTION

The only available text format for visually impaired isBraille

letter code.

This becomes a restriction for the blind, to read the common

texts (news papers, documents..etc) as we do.

Our proposal is to help the visually impaired to cross this

barrier.


3/40

Abstract

This project renders an aid to visually impaired, featuring them

by synthesising speech for the textual contents of a document.

This is possible with OCR engine And TTS technology.


4/40

Block Diagram

Save in text file

Generate speech

Wait for user

instruction

Image Segmentation

Conversion to

gray scale

Correlation

Conversion to

binary image

Image

enhancement

Yes

No

Discarded

OCR

TTS

Input image

Pre-processing


5/40

Conversion to gray scale

rgb2gray converts RGB values to gray scalevalues by forming a weighted sum of

theR, G, andB components:

0.2989 * R + 0.5870 * G + 0.1140 * B


6/40

Histogram Equalization

This is done to adjust the contrast of the gray scale

image.

The general formula of histogram equalization is givenby,

)1()(

)()()(

min

min

LcdfNM

cdfvcdfroundvh

where,

cdf- cumulative distribution function

M X Nimages no. of pixels

LNo. of gray levels used


7/40

9478766568697987

8365615559647185

7558687770606579

706888126104686167

6970106154122715863

7366104144113685962

72698510990555963

7364617066615552Gray scale image

Equivalent Matrix


8/40

9478766568697987

8365615559647185

7558687770606579

706888126104686167

6970106154122715863

7366104144113685962

72698510990555963

7364617066615552


9/40

Value Count Value Count

52 1 79 2

55 3 83 1

58 2 85 2

59 3 87 1

60 1 88 1

61 4 90 1

62 1 94 1

63 2 104 2

64 2 106 1

65 3 109 1

66 2 113 1

67 1 122 1

68 5 126 1

69 3 144 1

70 4 154 1

71 272 1

73 2

75 1

76 1

77 1

78 1


10/40

9478766568697987

8365615559647185

7558687770606579

706888126104686167

6970106154122715863

7366104144113685962

72698510990555963

7364617066615552


11/40

Value Count Value Count

52 1 79 2

55 3 83 1

58 2 85 2

59 3 87 1

60 1 88 1

61 4 90 1

62 1 94 1

63 2 104 2

64 2 106 1

65 3 109 166 2 113 1

67 1 122 1

68 5 126 1

69 3 144 1

70 4 154 1

71 2

72 1

73 2

75 1

76 1

77 1

78 1


12/40

1 st value,

cdf(v)=1

cdfmin=1

M X N= 8 X 8=64L=256

Then,

h(v) = 0

2 nd value,

cdf(v) = 3

cdfmin=1

M X N= 8 X 8=64L=256

Then,

h(v)=12

)1()(

)()()(

min

min

LcdfNM

cdfvcdfroundvh

By substituting cdf(v) corresponding h(v) values are found and values

are given in next slide.


13/40

21918217485117130190206

1948553123273154202

170201171781463685190

1461172102472271175397

1301462312552431542065

166932272512391173257

15813020223521512326516673531469353120

Equalized Image

Equivalent Matrix


14/40

CALCULATIONOFTHRESHOLDVALUE

Reshape the 2 dimensional gray scale image to 1 dimensional.

Find the histogram of the image using hist function.

Initialize a matrix with values from 0 to 255.

Find the weight , mean and the variance for the foreground and

background.

Calculate weight of foreground* variance of foreground + weight of

background* variance of background.

Find the minimum value.


15/40

Reshape the 2 dimensional gray scale image

to 1 dimensional.

21918217485117130190206

1948553123273154202

170201171781463685190

1461172102472271175397

1301462312552431542065

166932272512391173257

158130202235215123265

16673531469353120

219

194

.

.

.

.

.

206

202

190

97

65

57

65

0

8 X 8 Matrix of an image

64 X 1 Matrix of same image


16/40

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 . . . . 255

1 0 0 0 0 0 0 0 0 0 0 0 3 0 0 . . . . 0

1 1

2 0

3 0

4 0

5 0

6 07 0

8 0

9 0

10 0

11 0

12 3

13 0

14 0

. .

. .

. .

. .

256 0

Value count 1 x 256

Re-shaping

256 x 1


17/40

1 0

2 1

3 2

4 3

5 4

6 5

7 6

8 7

9 8

10 9

11 10

12 11

13 12

14 13

. .

. .

. .

. .

256 255

1 2 3 4 5 6 7 8 9 10 11 12 13 14 . . . . 256

0 1 2 3 4 5 6 7 8 9 10 11 12 13 . . . . 255

Creating a matrix for intensity level from 0 to 255

Re-shaping the matrix for

calculation simplicity


18/40

weight=sum(h(1:1))/sum(h);

weight=1/64=0.0156

1 1

2 03 0

4 0

5 0

6 0

7 0

8 0

9 0

10 0

11 0

12 3

13 0

14 0

. .

. .

. .

. .

256 0


19/40

Mean Calculation

value=H(m:n).*Index(m:n);

total=sum(value);

mean=total/sum(H(m:n));

Ex:

H(1:1) .* Index(1:1)

Variance calculation

value2=(Index(m:n)-mean).^2;

number=sum(value2.*H(m:n));

var=number/sum(H(m:n));

Ex:

value2= (Index(1:1)-mean).^2;

Threshold Calculation

[wbk,varbk]=calculate(1,i);

[wfg,varfg]=calculate(i+1,255);

After calculating the weights and the variance, the final computation is

stored in the array result.

result(i+1)=(wbk*varbk)+(wfg*varfg);

Value=min(result);

Thr=Value/256;


20/40

Conversion to binary image

BW = im2bw(I, level) converts the gray scale image I to a binary image.

The output image BW replaces all pixels in the input image with luminance

greater than level with the value 1 (white) and replaces all other pixels with

the value 0 (black).

Specify level in the range [0,1]. This range is relative to the signal levels

possible for the image's class.

Therefore, a level value of 0.5 is midway between black and white,

regardless of class.

To compute the level argument, you can use the function graythresh.

If you do not specify level, im2bw uses the value 0.5.


21/40

CLIPPING

[r c] = find(g);h= g(min(r):max(r),min(c):max(c));

where, r= rows of the matrix

c= columns of the matrixg= input image

User-defined function for clipping:


22/40

00000000

00111000

00111100

0111111000111100

00111100

00111100

00000000

Input image matrix


23/40

011100

011110

111111

011110

011110

011110

After applying the

clipping function


24/40

Non-zeros founded in r matrix

1 5

2 2

3 3

4 45 5

6 6

7 2

8 3

9 4

10 5

11 6

12 7

13 2

14 3

15 4

16 5

17 6

18 7

19 2

20 3

21 4

22 5

23 6

24 7

25 5

Non-zeros founded in c matrix

1 2

2 3

3 3

4 35 3

6 3

7 4

8 4

9 4

10 4

11 4

12 4

13 5

14 5

15 5

16 5

17 5

18 5

19 6

20 6

21 6

22 6

23 6

24 6

25 7


25/40

00000000

00111000

00111100

0111111000111100

00111100

00111100

00000000

5th row,

2nd column

5th row,

7th

column


26/40


27/40

LINE CROPPING

110110

010111

011011

000000

110110

110111 = 6

= 4

= 0

= 4

= 4

= 4

}

}

1st LINE

2nd LINE


28/40

011100100010

010101010010010101010101

010100100101

2 2 2 0 2 2 2 0 4 1 4 0

Y O U

LETTER CROPPING


29/40


30/40

Image Re-sizing

Re-size this cropped letter to 42 X 24(size of database image)

Cropped letter

42 X 24 image


31/40

Correlation

In this step the re-sized image is correlated with the images in

database.

The database image which has high correlative value is

assumed to be the letter and saved in a text file.


32/40

010

010101

101

[A ] [B] . [Y] . [a] . . . [9] [0]

. . . . . . . .

(0.146)

(0.9877)

1 2 3 25 26 27 35 37 52 61 62

sem=corr2(templates{1,n},imagn);comp=[comp sem];


33/40

0.146 . . 0.9877 . -0.44 . . . 0.567 0.111

1 2 3 25 26 27 35 37 52 61 62

Comp=

vd=find(comp==max(comp));


34/40

Calculating space between letters

Line without Y

Using clipping function

Difference between above two images gives the space value

Here value=4

These space values are stored in space vector array


35/40

[Y] [O] [U] [A] [R] [E] . . . . .

Word Matrix

4 4 3 14 3 2 . . . . .

Space Matrix

[Y] [O] [U] [A] [R] [E] . . . .

Word Matrix


36/40

Saved in Text File


37/40

TTS

There are number of TTS available.

Some parameters have to be checked before choosing an algorithm :

Accuracy

Processing Speed , etc.


38/40

Speech Application Programming Interface

The Microsoft text-to-speech voices are speech synthesizers provided for use

with applications that use the Microsoft Speech API(SAPI) or the Microsoft

Speech Server Platform.

In general the Speech API is a freely-redistributable component which can be

shipped with any Windows application that wishes to use speech technology.

There are both SAPI 4 and SAPI 5 versions of these text to speech voices.


39/40

OUTPUT


40/40

Conclusion

For precise recognition of the text the template database must be

trained well.

Phonetic rhythm must be maintained by the machine generated

sound to cope with natural voice.

For this linguistic analysis must be done.

And escape sequences are to be developed so that special

characters such as Mr. , St. , etc., are pronounced literally.

interpreter for visually impaired

Documents