SCENE TEXT RECOGNITION IN MOBILE APPLICATION BY

CHARACTER DESCRIPTOR AND STRUCTURE COMFIGURATION

CHERIYAN K M

INTRODUCING….. Valuable information form an image. To extract an information.

Automatic and Effective scene text detection. Recognition algorithm.

Factors affecting on extraction. Cluttered background. Difference in text pattern.

Difficult to model the structure of character. Lake of discriminative pixel level appearance. Structure features from non-text background

outliers. Different word , may diff. characters , in various

fonts , styles and size.

Two activities; Text detection.

Localize the image region containing the text characters. Based on

Color uniformity and Horizontal alignment of text char.

Text recognition. Transform pixel-based text into reliable codes. Distinguish diff. text characters , Properly compose the

text word. 62 identity category of text characters.

9 (0-9)26 (a-z)26 (A-Z)

Two schemes; Character recognizer to predict the category of text char. Binary character classifier to predict the existence of

ctgry.

RELATED WORKS

Optical Character Recognizer (OCR) system. Many algorithms are proposed;

Weinmen:- combined the Gabor-based appearance model.

Neumann:- based on extremal region. Smith:- based on SIFT. Mishra:- adopted conditional random field. Lu:- modeled the inner character structure. Coates:- extracted local features of character

patches.

LAYOUT BASED SCENE TEXT DETECTION A text;

Instruction Identifier Uniform color Aligned arrangement

Two processes are employed to complete layout analysis

1. Color Decomposition2. Horizontal Alignment

Improved to compatible with mobile app

LAYOUT ANALYSIS OF COLOR DECOMPOSITION

Boundary clustering algorithm base on bigram color uniformity.

Group pixels of same color into a layer. Character boundary boarder b/w txt and bg.

(color pair) Create a vector of color pair (txt and bg).

LAYOUT ANALYSIS OF HORIZONTAL ALIGNMENT

Text information(string)

Several character members

In similar size

Approximately horizontal alignment

The geometrical properties to detect the existence of text characters

Adjacent character grouping algorithm

Bounding box>siblings>similar size & vertical location>merge

For non-horizontal strings-> ±/6 degree set as range.

STRUCTURE BASED SCENE TEXT RECOGNITION

To extract text information. Binary classification problem. Character classes(Queried characters). Binary classifier:- to distinguish character

class from other classes or bg outliers. Eg: Character class A predict patches containing

A as positive. And other as negative. Two activities;

1. Character descriptor.2. Stroke configuration.

CHARACTER DESCRIPTOR

Extract structure features. 4 different key points features;

1. Harris Detector:- To extract Key points from corner and junction.

2. MSER Detector:-To extract Key point from stroke component.

3. Dense Detector:- To extract Key point uniformly.4. Random Detector:- To extract the preset

number of Key points in a random pattern.

Flowchart of our proposed character descriptor

HOG:-features are Calculated as observed feature vector x.(Histogram of Oriented Gradient)

•Selected as local feature descriptor( compatibility with all 4 key point detectors).

SIFT and SURF are not employed Normalization of character patches(128x128). Feature Quantization: to aggregate the

extracted features Bag-of-Words(BOW) Medel:- Applied to key points

from all 4 feature detector. Gaussian Mixture Model(GMM):Applied to key

points from DD & RD.(fixed number and location of key point)

Now mapping both into characteristic Histogram as feature representation.

Cascading BOW and GMM-based feature repr. ,we get Character Descriptor.

CHARACTER STROKE CONFIGURATION

Stroke:- Region bounded by two parallel boundary segments.

Stroke width Stroke orientation Characters are connected strokes with

configuration. Structure Map of Strokes is stroke

configuration.( is consistant)Eg: B have 1 vertical stroke

2 arc strokes.

B

Synthesized character generator: Estimate stroke configuration from computer s/w.(Provide accurate skeleton and boundary)

Synthetic font training dataset(20000 are selected out off 67400 character patches) Contain 62 class of characters(128x128 pixel)

Compose Stroke ConfigurationStep1

Discrete Contour Evaluation(DCE):obtain boundary and skeleton. Skeleton pruning on the basis of DCE.

DCE simplifies the character(using polygon and small no. of vertices)

DCE and Skeleton pruning are invariant to deformation and scaling.

Step2 Estimate stroke width and orientation

Width: length along normal Orientation: tangent

Sampling from character boundary 128 samples. So that no. of samples = length Estimating Taking two neighboring sample point to fit a

line. Approximately collinear. A quadratic curve.

Step3 Calculate Skeleton-based stroke map Consistency of stroke width and orientation.

Construct stroke section: If sample point satisfying the stroke related features.

Construct junction sections: If they are not. Skeleton points are extracted.

Width no larger than 3 Orientation no larger than /8

STROKE ALIGNMENT METHOD

To handle various fonts, styles …..etc Mean value of all stroke configuration.

Mean value,

D=Distance b/w stroke configurations of two samples

S=Mean value of stroke configurations. Ti=Transformations applied on strokes of i-th

stroke configuration. g(Ti)=Amplitude of the transformation.

DEMO SYSTEM

THANK YOU