Download - Automatic licence plate recognition
Team Members
Max Guise Electrical Engineering Technical Staff, Sandia National Labs
Stephan Goupille MS&E, Electrical Engineering Growth Analyst, Facebook
1
Project Aims - Adapted 3Character recognition particularly difficult and requires a massive database of license plates
Focus on license plate extraction:
Tools 4Motorola Droid Phone Great for snapping a quick picture Originally thought RGB values could be exported
directly (function hasn’t been implemented)
MATLAB One of the best, fastest mathematical analysis
tools Many built-in image processing functions
Other ALPR Systems 5Most systems in actual use benefit from a
controlled scenario: Lack of background objects Higher quality imaging Better flash (license plates are retroreflective) Minimal scaling and rotation
Ease of access to large database from which to build a model Typically use a machine learning algorithm (SVM,
LDA, etc.) to make final decisions
Our goal to accomplish task using basic properties of license plates – no machine learning necessary
Methods 6Pre-Processing
KMeans Adaptive Histogram Equalization
Image Transformation Histogram of Oriented Gradients Harris Corners Hough Transform Edge Detection Filtering
Decision-Making Thresholding Erosion
KMeans 7Unsupervised Nearest Centroids in 5-Space (X,Y,R,G,B) Variable number of centers can be chosen Can weight X,Y differently from R, G, B
Goal: License plate shows up as one block
KMeans 8 Issues: Optimal number of centers to use varies with size of
license plate As a result, can’t guarantee good separation of plate from
bumper, or might split license plate region into multiple parts Can be computationally expensive – either in itself or
later in the pipeline
Histogram Equalization 9 Image pixels may be far from uniformly spread across the display range Changing the distribution can add contrast
Can be done across whole image, or ‘adaptively’ Tiling across image might split up an individual license
plate region
0 100 200 3000
2
4
6
8
10x 10
5 Original
0 100 200 3000
1
2
3
4
5
6x 10
5 Equalized
0 100 200 3000
2
4
6
8
10x 10
5 Adaptive
Dalal Triggs ’05 paperObjects can be described by edge orientation
Compute basic gradient Since we have data in two directions, can calculate a direction
for each pixel Also snap to regular grid – 45 degree angles
Create histogram of these directions for a given window size Computation limitation here – would need to search at
various scales
Biggest problem here is that edges of plates themselves not always well defined Illumination changes, license plate holders, glare, etc.
Bottom line: need machine learning technique here to make this work
Histogram of Oriented Gradients 13
Detects ‘interest points’ in image Based on eigenvalues of ‘sensitivity matrix’
Our idea was to search the detected Harris corners for rectangles of approximately the right aspect ratio Turned out to be an incredible computational burden with the
detector sensitivity high enough to reliably detect license plate corners
Harris Corner Detector 14
200 400 600 800 1000 1200
100
200
300
400
500
600
700
800
900 20 40 60 80 100 120 140
10
20
30
40
50
60
70
80
Canny edge detection to get edge mapFor each edge pixel, record the set of all lines
that could pass through it Parameterized by distance from origin and angle
Resulting image has peaks at (ρ,Θ) pairs representing prominent lines in the image Idea was to search for peaks corresponding to the
approximate aspect ratio of license plates
Again, edges of plates not always well defined Meant that tuning the thresholds resulted in a
detector that delivered an incredible amount of false positives
Hough/Radon Transform 15
Underlying insight is that license plates have a lot of vertical edges (letters) and a specific aspect ratio Take horizontal gradient Filter with ‘matched’ filter
Designed with aspect ratio, approximate license plate size in mind
Threshold Bit of an art – ties in closely with the erosion
Erode Region label
Throw out regions that are too small Cascaded false positive rejection
Rely on context of license plate (car bumper, license plate holder etc.)
Simplest is best 17
Matched Filtering 19
0
20
40
60
0
10
20
30
40-1
-0.5
0
0.5
1
500 1000 1500 2000 2500
200
400
600
800
1000
1200
1400
1600
1800
200 400 600 800 1000 1200
100
200
300
400
500
600
700
800
900
500 1000 1500 2000 2500
200
400
600
800
1000
1200
1400
1600
1800
200 400 600 800 1000 1200
100
200
300
400
500
600
700
800
900
False Positive Rejection 21Expand candidate license plate window and look for prominent horizontal lines Reject if we don’t find any – bumper or license plate
holder typically shows up, and we interpret ‘prominent’ too stringently
Detection without learning = hard Lots of issues: background clutter, contrast,
scale/rotation invariance, computation, etc.
Color space changes didn’t seem to help Tried various transformations of RGB and LAB
coordinates as well as scale-by-max color balancing
Next steps would be window refinement, exploration of scale and rotation capabilities (and, of course, character segmentation and recognition)
Thanks!
Conclusions 22
Dalal, N. and Triggs, B. 2005. Histograms of Oriented Gradients for Human Detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cvpr'05) - Volume 1 - Volume 01 (June 20 - 26, 2005). CVPR. IEEE Computer Society, Washington, DC, 886-893.
Vahid Abolghasemi, Alireza Ahmadyfard. “An edge-based color-aided method for license plate detection.” Image and Vision Computing, Volume 27, Issue 8, 2 July 2009, pp. 1134-1142.
Stokman, Harro and Gevers, Theo. “Selection and Fusion of Color Models for Image Feature Detection.” IEEE Transactions on Pattern Analysis and Machine Intelligence 29.3 (2007): 371-381.
References 23