Deep Automatic Licence Plate Recognition system

Vishal JainIndian Institute of Science

vjain20687@gmail.com

Zitha SasindranIndian Institute of Science

zithasasindran@gmail.com

Anoop RajagopalMyntra Designs

anoop.kr@myntra.comSoma Biswas

Indian Institute of Sciencesoma.biswas@ee.iisc.

ernet.in

Harish S BharadwajIndian Institute of Science

harishsb9490@gmail.com

K R RamakrishnanIndian Institute of Sciencekrr@ee.iisc.ernet.in

ABSTRACTAutomatic License Plate Recognition (ALPR) has impor-tant applications in traffic surveillance . It is a challengingproblem especially in countries like in India where the li-cense plates have varying sizes, number of lines, fonts etc.The difficulty is all the more accentuated in traffic videos asthe cameras are placed high and most plates appear skewed.This work aims to address ALPR using Deep CNN meth-ods for real-time traffic videos. We first extract license platecandidates from each frame using edge information and ge-ometrical properties, ensuring high recall. These proposalsare fed to a CNN classifier for License Plate detection obtain-ing high precision. We then use a CNN classifier trained forindividual characters along with a spatial transformer net-work (STN) for character recognition. Our system is evalu-ated on several traffic videos with vehicles having differentlicense plate formats in terms of tilt, distances, colors, illu-mination, character size, thickness etc. Results demonstraterobustness to such variations and impressive performance inboth the localization and recognition. We also make avail-able the dataset for further research on this topic.

CCS Concepts•Computing methodologies→ Computer vision; Ob-ject detection; Object recognition; Neural networks;

KeywordsConvolution Neural Network; Spatial Transformer Network;Image Transformation Pursuit; Automatic License Plate Recog-nition

1. INTRODUCTIONVehicle License plate (LP) detection and recognition is an

important applied area in traffic surveillance, management,parking management, vehicle recognition, highway toll col-lection, tracking vehicle for security purposes. Retrieving

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full cita-tion on the first page. Copyrights for components of this work owned by others thanACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-publish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from permissions@acm.org.

ICVGIP, December 18-22, 2016, Guwahati, Indiac© 2016 ACM. ISBN 978-1-4503-4753-2/16/12. . . $15.00

DOI: http://dx.doi.org/10.1145/3009977.3010052

and identifying the LP area in whole traffic image can bequite challenging due to many intrinsic factors such as di-versity in character pattern, font, shape, size, color, plate di-mensions as in Figure 1. Also, capturing viewpoint, blurringand uneven illumination causes character distortion. Exter-nal elements like complex background, text signs & boards,color similarity between background and vehicle body, re-flection from the surrounding objects further enhance com-plexity of the problem. Sample frames are shown in Figure 2

Figure 1: Left: Single line plates. Right : Doubleline plates

Figure 2: Sample frames from CCTV recordedvideo.

The system as in Figure 3 is typically developed in stageswith first being detection of LP in the traffic image. De-tection includes localization of LP and generating suitablebounding box over them. Next stage is recognition or iden-tification of characters inside the bounding box generated.This sequential process emphasizes each stage to be accurateand robust for the next stage to be successful.

For the detection part, feature based approaches is mostlyused [10]. Features of the LP derived from its edge, color,texture, or a combination of two or more of these are used

Figure 3: Typical stages of an ALPR system.

to identify the potential regions. The proposals generatedhave many false positives. These false positives are gener-ally eliminated using geometrical properties of the LP [6] viz.area, aspect ratio, dimensions of plate etc. LP detection isfollowed by recognition of the characters involving segmen-tation and extraction of individual characters from the LP.The segmentation methods may use pixel connectivity, pro-jections of characters, character contours or their combina-tion. Character recognition is performed on the segmentedcharacters with template matching and learning based meth-ods. Each process involves pre-processing to enhance per-formance of that module.

In this work, we adopted LP edge feature based schemeto generate candidate proposals for the LP regions. Thefalse positives are minimized using a convolution neural net-work which perform binary classification on the candidateplate regions. This integration of calculative and learningapproach ensures high recall and precision. As deep net-works requires large training data we build a new datasetwith Indian LP which have a lot of diversity in plate formfactor, fonts, number of lines etc. For recognition of the ex-tracted LP we use another CNN built for characters recog-nition. This CNN model has 11 parallel classifiers at theoutput to localize and encode individual character of thecharacter sequence of LP. As the extracted LP usually havevariations in form of tilt, non-uniform brightness a spatialtransformer layer (ST layer) [8] is used at the input layer ofthe CNN. This way it enhances the geometric invariance ofCNN in a flexible, data-driven manner without much train-ing overhead.

Most of the ALPR frameworks work well only in controlledenvironment with lots of assumptions. Some demand spe-cific viewpoint for image capturing, some require vehicle tobe parked or static and some may work for license platesof a specific region. Challenge lies in developing a systemthat works well in natural real-traffic images. Our majorcontributions are

• dataset creation for license plates with different for-mats, at different distances, in diverse background andcharacter style.

• a CNN framework for end-to-end LP detection andrecognition from video frames without segmentationin uncontrolled conditions. The system works on real-traffic videos and rapidly processes large frame area ofHigh-Definition images where amount of computationand delay is high, optimizing different implementationtools employed.

The rest of paper is organized as follows. Related workis discussed in Section 2. The detection and recognitionmodules of our framework are described in Section 3. Ex-periments performed on test videos and results obtained aresummarized in Section 4. Finally, conclusions are drawn andsome comments in general are made in Section 5.

2. RELATED WORKIn this section, we present a brief description of related

work on LP detection and recognition.

2.1 License plate detectionThis stage analyzes the input image and proposes LP re-

gions. The detection of license plates accurately in openenvironment is challenging as the view-point, illumination,direction of vehicle can be arbitrary. Existing approachesare broadly classified based on features like edge, texture,color character information they have used [10].

The edge-based approaches [3] finds regions with highervertical edge density. Morphological operations are per-formed on the edge map to remove smaller and bigger re-gions of image. Binarization followed by connected compo-nent analysis gives candidate license plates filtered for falsepositives with geometric filtering. Edge based approach issimple and fast but produces lots of false positives.

Texture-based approaches [4] detect the desired region ac-cording to its pixel intensity distribution as intensity patternof the LP region differs from the background. The methodsexamine local features of an image by calculating irregularityof local plate region in the texture of the image at differentscales. It works well in case of deformed plate boundariesand are robust to color, size and position of plate region.But high computational complexity for HD complex imageslimits its use for real-time vehicle-traffic images.

Color-based approaches [14, 10, 18] make use of the factthat color of the LP is different from the other regions of theimage. In [14], the image is segmented into different regionswith different colors using mean-shift algorithm. From thesecandidates, LP regions are distinguished using dimension,aspect ratio and vertical edge information. The methodcan detect deformed or tilted license plates as well withthe downsides being sensitivity to illumination changes andnoise sensitivity.

Character-based approaches [10] consider LP as sequenceof alphanumeric characters, and search for the character se-quences in the image. The method is robust to variationsof LP characters’ font, color, size, tilt and language-script.But computation time is high and detection errors occur incase of background texts.

In our work we use edge-based features, discussed above,as it performs fairly well in detecting almost all LP candi-dates with less computation time. Later a CNN based binaryclassification is performed for filtering out false-positives.

2.2 License plate recognitionThe detected LP regions are then recognized for their con-

tent. The general approach has mostly been segmenting thecharacter-sequence of LP into individual characters usingpixel connectivity, projections, character contours and tem-plate matching. This is followed by recognition of the seg-mented characters using suitable Optical Character Recog-nition (OCR) technique.

Pixel connectivity based methods label each pixel of the

binarized LP image based on 4 or 8 neighborhood connec-tivity. If characters are separated from each other with-out joins, they can be separated based on connectivity withneighboring pixels. The method is simple, fast and worksfor rotated plates but, the plain-vanilla usage results in falsepositives, joined or broken characters, plate edges, arbitraryspots. This issue is addressed in [16] based on some geo-metrical rules. In projection-based method [11], binarizedLP images is projected horizontally to get top and bottomboundaries of the characters and vertically to get left andright endpoints of each character. This method works for dif-ferent character positions and number of characters but per-forms poorly in case of rotated images. The contour-basedmethod [17] determines character boundary using statisti-cal boundary shape models and uses character template forsegmentation and recognition. Exact boundaries can be ex-tracted from this method but noisy images result in incom-plete and distorted contours. However, the segmentationis affected under tilt, broken characters and non uniformbrightness. The segmentation also fails to extract all thecharacters when there are joined or broken.

Template matching based methods [12] measure similar-ity between character and template and chooses class of thebest matching template as the predicted class. The methodis fast and simple but limited by variations of font, rota-tion, size, etc. of characters. Learning based method [5]are robust to variations of segmented characters. Featuresare extracted from character images, such as edge density,gradient, Local Binary Patterns, etc. which are classifieddiscriminatively with learning models like SVM, neural net-works, etc. Plate/non-plate CNN classifier previously usedin [19, 6] reject false positives and distinguish LP from back-ground objects including image text, giving high precision.However, these approaches adopt a sliding window approachfor locating LP making it computationally expensive.

In view of the issues with past approaches, as discussedabove, we adopt CNN based recognition on whole LP image,avoiding challenging task of image segmentation into eachcharacter.

3. DEEP LPR SYSTEMOur ALPR system shown in Figure 4 is a cascade frame-

work constituting three parts: (i) Generation of LP can-didates, (ii) Filtering of false positives with binary licenseplate/non-license plate classifier, (iii) Recognition of char-acters of extracted LP, with 37 class CNN with 11 parallelclassifier at output.

3.1 LP Detection

License plate candidates generationWe adopt LP edge feature based scheme for candidate pro-posals. The algorithm detects multiple LPs under differ-ent image capture conditions and extracts them using edgestatistics. Morphological operations are used to extract ver-tical edges of the LP regions while removing background.The process is divided into two sub-parts, candidate gen-eration for possible LP regions and candidate verificationto eliminate false positives based on geometric parameters.Below we outline the LP candidate generations steps.

• Original color image of traffic-scene is converted togray image. Black and white intensity values in LP

Figure 4: Illustration of our ALPR framework

regions is filtered out with median filtering as it elim-inates and minimizes unwanted noise and spots in theimage. Histogram equalization is performed to dealwith the illumination variations.

• Sobel vertical edge filter is applied to obtain verticaledge map as LP regions are seen to have higher ver-tical edge density. Mean filter is used to emphasizeLP region with the mask size of order of LP size elim-inating edges of car mirror, windows, LP border andother surrounding objects. It also smooths out theedge intensity maps removing high-frequency compo-nents pertaining to noise.

• Morphological filtering is performed to remove smallerand larger objects than the LP dimensions with open-ing and Mexican top-hat filtering. Structuring Ele-ments (SE) have size of MINHCHAR (minimum char-acter height), and MAXHCHAR (maximum characterheight) respectively.

• We convert the resulting image to binary image byOtsu thresholding so that candidate regions are sepa-rated from complex background.

Candidate verification is done based on criteria that can-didate regions should follow: a) width of the region to begreater than height for a LP b) height has to be greaterthan minimum LP height MINHCHAR c) region should nottouch image boundary d) two regions should not overlap e)aspect ratio should be between parameters LP MIN RATIOand LP MAX RATIO. Parameters along with size of struc-turing elements are selected after testing on multiple imagesfrom the desired location of application. The above stepsare visualized in Figure 5

CNN detectionCandidate generation technique described above detects allLP regions along with many non-license plate regions thatare to be filtered out. CNN classifiers show discriminativeability among different classes. We adopt binary licenseplate/non-license plate CNN for filtering. We use 5-layerCNN model for classifying candidate regions generated fromprevious stage into license plate or non-license plate regions.Configuration of the model is shown in Table 1.

All images are resized to 32 × 96, range normalized to[0, 1] for passing to input layer of CNN. We experiment withdifferent number of layers in CNN model and obtain high-est validation-set ( 20% of training set) accuracy for 5-layermodel. Accuracy numbers for different layer configurationsare shown in Table 2.

Figure 5: Candidate generation steps for LP detection

Table 1: Configuration of CNN model for LP detec-tion

Layer Type ParametersSoftmax 2 classesFully connected #neurons: 2Dropout Ratio : 0.5ReLUFully connected #neurons: 128Dropout Ratio : 0.25Maxpooling kernel : 2×2, stride : 2ReLUConvolution #filters: 48, kernel : 3×3, stride : 1Dropout Ratio : 0.25Maxpooling kernel : 2×2, stride : 2ReLUConvolution #filters: 64, kernel : 3×3, stride : 1Dropout Ratio : 0.25Maxpooling kernel : 2×2, stride : 2ReLUConvolution #filters: 64, kernel : 5×5, stride : 1Input 32×96 pixels gray-scale image

Table 2: LP detection accuracy with different num-ber of CNN layers.

CNN Model Validation-set Accuracy4-layers 94.8%5-layers 98.7%6-layers 97.2%

3.2 LP RecognitionTraditional approach for LP character recognition has been

to segment characters and recognize each separately. Toovercome these difficulty in character segmentation and recog-nition, we perform string recognition on whole LP image.This method avoids the challenging task of character seg-mentation.

We use character-sequence encoding CNN model [8] for al-phanumeric character recognition in LP images. We assumemaximum character length to be a fixed number N. Hence Ithas N parallel classifiers so that one classifier would classifyone of N characters of LP. A ’NULL’ (#) character class

is used to account for lesser number of characters than themaximum length. We also include Spatial Transformer (ST)module introduced by [8] which can make CNN spatially in-variant to large variations of input data. We include STmodule as first layer in recognition CNN.

We use character-sequence encoding CNN with 16 layers.We consider maximum length of character sequence to be 11.So there are 11 different classifiers at output of the CNN topredict each digit. As described earlier, we assume NULLcharacter in place of non-existing places of LP image. Total37 character-classes are 0, . . . , 9, a, . . . , z,NULL. Configu-ration of CNN layer is described in Table 3. Configurationof the localization network of ST layer is shown in Table 4.

Table 3: Configuration of CNN model for recogni-tion with ST LayerLayer Type ParametersSoftmax 11 parallel layers with 37 classesFully connected 11 parallel layers, each has #neurons: 37ReLUFully connected #neurons: 1024Dropout Ratio : 0.25Maxpooling kernel : 2×2, stride : 2ReLUConvolution #filters: 256, kernel : 3×3, stride : 1Dropout Ratio : 0.25Maxpooling kernel : 2×2, stride : 2ReLUConvolution #filters: 128, kernel : 3×3, stride : 1Dropout Ratio : 0.25Maxpooling kernel : 2×2, stride : 2ReLUConvolution #filters: 64, kernel : 5×5, stride : 1ST LayerInput 32×96 pixels gray-scale image

4. EXPERIMENTSWe contribute a new dataset for training CNN for ALPR.

We use video recordings from traffic monitoring 3 megapixelHD camera. It captures moving traffic footage with 20 fps,resolution of 1920× 1080 from a distance of 7-8 meters andheight of 4 meters. Few sample frames are shown in Figure 2.

Table 4: Configuration of ST layer’s localization net-work.

Layer Type ParametersFully connected #neurons: 6ReLUFully connected #neurons: 50Maxpooling kernel : 2×2, stride : 2ReLUConvolution #filters: 32, kernel : 5×5, stride : 1Maxpooling kernel : 2×2, stride : 2ReLUConvolution #filters: 48, kernel : 5×5, stride : 1Input 32×96 pixels gray-scale image

4.1 LP detectionThe localization of LP regions described above, involve

several parameters used in morphological operations andcandidate verification. These are arrived at, based on theplate dimensions in the frame images, and further tuned bytesting on more images. On an average, 2.45 non-LP regionsor false positives are generated per frame-image which arefiltered out with CNN. We use python and OpenCV libraryfor image processing tasks.

For training CNN, data is generated using the LP can-didate proposals from the video frames. Our dataset has8570 negative samples and 5981 positive examples, i.e. to-tal of 14, 551 images. Sample positive and negative trainingexamples are shown in fig 6

Figure 6: Top: Positive training samples. Below:Negative training examples.

We experiment with different combinations of CNN pa-rameters like initial learning rate, step size, batch-size. Wetrain with a batch size of 32 using Digits [2, 15] on NVIDIAGeForce GTX 660 GPU (2GB memory and 960 CUDA cores),Intel i7 8core CPU machine. We do batch-processing forreducing per-frame-processing time. Also, single-line anddouble-line type plates have different character arrangements,so a single model cannot learn the character arrangementfor both types of plates. Thus, we build separate models forsingle-line and double-line plates.

We perform a 80:20 split of our dataset for training andtesting. We obtain a test set accuracy of 99.02%. Positivesample accuracy is 98.46% and negative sample accuracyis 99.3% perhaps because number of negative samples aremore in training set. Thus CNN based false-positive rejec-tion gives very high test accuracy ensuring high precision.Average time for candidate generation is 150ms per frame

and for filtering false positives with CNN is 30ms per frame.Thus total time taken for LP extraction per frame is 180ms.So our LP detection algorithm is fast and accurate. Confu-sion matrix on the test set is shown in Table 5.

Table 5: Confusion matrix of CNN classifer for LPdetection

Confusion matrix License plate Non-license plateLicense plate 99.36% 0.64%Non-license plate 0.32% 99.68%

Some LP extraction results are shown in fig 7.

Figure 7: Detected license plates.

4.2 LP recognition

Using the detection algorithm, which is fast and accurate,we generate crops of License Plates from frames. We usearound 7, 400 of these crop images for the training set outof which 4, 400 are single-line type and 3000 are double-linetype. Test set has total 2, 080 images; 1, 800 of single-linetype and 280 of double-line type. We manually annotate theground truth labels for both train and test images. As wedo not have sufficient training samples for other states li-cense plates for training the CNN, we synthetically generateLP images as in [7] for balanced representation. Figure 8represents the semantic division of the Indian LPs. By per-muting over possible content-options for each semantic parthelps in generating many synthetic plates. We thus have28, 280 single-line type synthetic images and 17, 580 double-line type synthetic images. Some synthetic LP images areshown in Figure 9.

Data augmentation of the training set is performed us-ing [9] for obtaining better variability. We use 19 differenttransformations of scaling, rotation, homography and JPEGcompression. These are illustrated for a sample LP image inFigure 10. All images are in grayscale and resized to 32×96pixels for training. The data is normalized by subtractingthe mean over all training data and range-normalized to[0, 1]. For double line LP we bisect them horizontally. Wethen horizontally stack the lower part with the upper part,to make double-line images similar in appearance and di-mensions to single-line images. Figure 11 illustrates thispre-processing step.

We implement the model using theano[13] library in python.Average per image processing time is around 15ms-18ms for

Figure 8: License plate semantic parts.

Figure 9: Synthetic license plates.

Figure 10: Transformations used for the licenseplate images.

Figure 11: Bisection and joining of double-lineplates.

recognition. Table 6 shows accuracy on test set for bothsingle-line and double-line models. Character-level accuracyfor single-line type plates are shown in Table 7. Figure 12shows ST layer output at different epochs. It can be noticedthat ST layer applies affine transformation on different LPimages to get normalized orientation. As the number oftraining epoch increases, spatial variation of the trainingsamples reduces underlying the prominence of the ST layer.

Table 6: Accuracy for LP recognition.Modeltype

TotalImages

Fullycorrect

1 charactererror

2 charactererror

>2 charactererror

Single-line 1800 1018 (57%) 430 (24%) 146 (8 %) 206 (11%)Double-line 280 222 (79%) 18 (7%) 9 (3%) 31(11%)

Table 7: Character-level accuracy for LP.

Model typeNumber of

characters (A)Correctly

classified (B)Accuracy

(Bx100/A)Single-line 19778 18202 92%Double-line 2854 2663 93%

Figure 12: ST layer output at different epochs.

4.3 Framework EvaluationWe integrate LP detection and recognition to get a unified

cascade framework for ALPR as shown in Figure 4. There isno uniform way to evaluate performance of ALPR system [6].In this work, we use recall and precision rate to quantify LPdetection results. Recall rate is the number of true positivesdetected out of the total number of true positives. High re-call rate implies robustness of detection algorithm to variousoccurrences of the object to be detected. Precision rate is thenumber of true positives detected out of the total number ofdetected objects. High precision rate implies discriminativeability of algorithm between true positives and negatives.For evaluating LP recognition technique, we calculate plate-level accuracy and character-level accuracy. In plate-levelaccuracy, we calculate percentage plates with all charac-ters correctly classified, percentage plates with one charac-ter wrongly classified, percentage plates with two characterswrongly classified and percentage plates with more than twocharacters wrongly classified. In character-level accuracy, wecalculate percentage of all the characters, of all the licenseplates, correctly recognized.

We test the ALPR system on 4 traffic-video clips that have173 cars; 149 with single-line type LP and 24 with double-line type. Total time taken by the system for processingone frame is around 200ms on average, i.e. system’s speedis 5 fps. Table 8 shows LP detection module’s recall andprecision rate. The LP recognition module’s plate-level ac-curacy results are summarized in Table 9 and character-levelaccuracy results are summarized in Table 10.

Table 8: LP detection results.Platetype

Numberof plates

PlatesDetected

(B)

Falsepositives

(C)

Recall(Bx100/A)

Precision(Bx100)/(B+C)

Single-line 149 126 5 84.6% 96.2%Double-line 24 23 2 95.8% 92%

Figure 13 shows samples where our LP detections fails. Itcan be seen that the characters which are highly blurred arenot correctly recognized. Figure 15 shows some qualitative

Table 9: LP - Plate accuracy.

Plate-typeDetected

platesFully

correct1 char.error

2 char.error

>2 char.error

Single-line 126 101 (80.2%) 18 (14.3%) 5 (4.0%) 2 (1.5%)Double-line 23 15 (65.2%) 6 (26.1%) 2 (8.7%) 0

Table 10: LP recognition - Character accuracy.

Plate-typeNumber of

characters (A)Correctly

classified (B)Accuracy

(Bx100/A)Single-line 1260 1224 97%Double-line 184 174 94%

results of our ALPR on traffic-scene images. These imagesdemonstrate the accuracy and robustness of our ALPR sys-tem under diverse conditions of vehicle direction, LP color,font, format, size, character-arrangement, etc. We also qual-itatively compare our results with the publicly available Ope-nALPR [1] library in Figure 16. We see that our frame-work fares better as we do character recognition holisti-cally whereas OpenALPR involves character segmentationfollowed by OCR.

Figure 13: Sample LPs where our method failed

4.4 Performance for other locationsThough we have designed our ALPR system for one camera-

location, taking into consideration the properties of imagecaptured like viewpoint, dimensions, LP measurements, vehicle-direction, etc., we test it for two other locations as well.Some sample outputs are shown in Figure 14. For these lo-cations, we miss some license plates while detection and alsoget few characters wrongly classified. However the system isstill able to detect many plates and recognize the charactersacceptably in spite of motion blur, occlusion and tilt indi-cating the CNN framework has good generalization ability.

Figure 14: Sampe results for different viewpoint

5. CONCLUSIONIn this work, we have presented an automatic LP recogni-

tion system based on CNN. We integrated several conceptsand techniques for designing the system. Results have shownthat the system successfully extracts the multiple LP regionfrom the traffic image and correctly recognizes most of thecharacters under variety of conditions. We have also shownthat incorporating spatial transformer layer provides betterrepresentation of data for recognition. The processing timeper frame is around 200ms, suitable for real-time applica-tions. We also compare the superiority of our frameworkover publicly available OpenALPR system. Current frame-work cannot handle LP of two wheelers and generating adataset for two-wheelers is left for future. Further, we alsowant to incorporate video based information like optical flowand temporal information in our work for generating betterand fewer candidate regions for LP proposals.

References[1] Automatic license plate recognition library. https://

github.com/openalpr/openalpr, 2015.

[2] Nvidia digits. github.com/NVIDIA/DIGITS, 2016.

[3] R. Chen and Y. Luo. An improved license plate loca-tion method based on edge detection. Physics Procedia,24:1350–1356, 2012.

[4] X. He H. Zhang, W. Jia and Q. Wu. Learning-basedlicense plate detection using global and local features.In Internation Conference on Pattern Recognition, vol-ume 2, pages 1102–1105, 2006.

[5] Q. Ye J. Jiao and Q. Huang. A configurable methodfor multi-style license plate recognition. Pattern Recog-nition, 42(3):358–369, 2009.

[6] H. Li and C. Shen. Reading car license plates usingdeep convolutional neural networks and lstms. arXivpreprint arXiv:1601.05610, 2016.

[7] A. Vedaldi M. Jaderberg, K. Simonyan and A. Zis-serman. Synthetic data and artificial neural networksfor natural scene text recognition. arXiv preprintarXiv:1406.2227, 2014.

[8] K. Simonyan M. Jaderberg and A. Zisserman. Spatialtransformer networks. In Advances in Neural Informa-tion Processing Systems, pages 2008–2016, 2015.

[9] M. Paulin, Z. Harchaoui F. Perronnin C. Revaud,Jerome, and Schmid. Transformation pursuit for imageclassification. In IEEE Conference on Computer Visionand Pattern Recognition, pages 3646–3653, 2014.

[10] M. Shehata S. Du, M. Ibrahim and W. Badawy. Auto-matic license plate recognition (alpr): A state-of-the-artreview. IEEE Transactions on Circuits Systems VideoTechnology, 23(2):311–325, 2013.

[11] O. Katai H. Kawakami S. Nomura, K. Yamanaka andT. Shiose. A novel adaptive morphological approachfor degraded character image segmentation. PatternRecognition, 38(11):1961–1975, 2005.

Figure 15: Results of our ALPR framework.

Figure 16: Comparison of our framework with OpenALPR system

[12] O. Ishaq S. Rasheed, A. Naeem. Automated numberplate recognition using hough lines and template match-ing. In Proceedings of the World Congress on Engi-neering. and Computer Science, volume 1, pages 24–26,2012.

[13] Theano Development Team. Theano: A Python frame-work for fast computation of mathematical expressions.arXiv e-prints, abs/1605.02688, May 2016.

[14] H. Zhang W. Jia and H.Huaifeng. Region-based licenseplate detection. Journal of Network and Computer Ap-plications, 30(4):1324–1333, 2007.

[15] J. Donahue Y. Jia, E. Shelhamer et al. Caffe: Convo-lutional architecture for fast feature embedding. arXivpreprint arXiv:1408.5093, 2014.

[16] H. Yoon Hosub Y. Yoon, KD Ban and J. Kim. Blobextraction based character segmentation method for au-

tomatic license plate recognition system. In IEEE Con-ference on Systems, Man, and Cybernetics, pages 2192–2196, 2011.

[17] L. Bai Y. Zhang, Z. Zha. A license plate charactersegmentation method based on character contour andtemplate matching. In Applied Mechanics and Materi-als, volume 333, pages 974–979, 2013.

[18] H. Guo-Jiang Z. Wei-gang and J. Xing. A study oflocating vehicle license plate based on color feature andmathematical morphology. In International Conferenceon Signal Processing, volume 1, pages 748–751, 2002.

[19] S. Yang Z. Zhao and X. Ma. Chinese license platerecognition using a convolutional neural network. InPACIIA, volume 1, pages 27–30. IEEE, 2008.