Max GuiseStephan Goupille

PSYCH 221Automatic License Plate Recognition

Team Members

Max Guise Electrical Engineering Technical Staff, Sandia National Labs

Stephan Goupille MS&E, Electrical Engineering Growth Analyst, Facebook

1

Project Aims 2

Project Aims - Adapted 3Character recognition particularly difficult and requires a massive database of license plates

Focus on license plate extraction:

Tools 4Motorola Droid Phone Great for snapping a quick picture Originally thought RGB values could be exported

directly (function hasn’t been implemented)

MATLAB One of the best, fastest mathematical analysis

tools Many built-in image processing functions

Other ALPR Systems 5Most systems in actual use benefit from a

controlled scenario: Lack of background objects Higher quality imaging Better flash (license plates are retroreflective) Minimal scaling and rotation

Ease of access to large database from which to build a model Typically use a machine learning algorithm (SVM,

LDA, etc.) to make final decisions

Our goal to accomplish task using basic properties of license plates – no machine learning necessary

Methods 6Pre-Processing

KMeans Adaptive Histogram Equalization

Image Transformation Histogram of Oriented Gradients Harris Corners Hough Transform Edge Detection Filtering

Decision-Making Thresholding Erosion

KMeans 7Unsupervised Nearest Centroids in 5-Space (X,Y,R,G,B) Variable number of centers can be chosen Can weight X,Y differently from R, G, B

Goal: License plate shows up as one block

KMeans 8 Issues: Optimal number of centers to use varies with size of

license plate As a result, can’t guarantee good separation of plate from

bumper, or might split license plate region into multiple parts

Can be computationally expensive – either in itself or later in the pipeline

Histogram Equalization 9 Image pixels may be far from uniformly spread across the display range Changing the distribution can add contrast

Can be done across whole image, or ‘adaptively’ Tiling across image might split up an individual license

plate region

0 100 200 3000

2

4

6

8

10x 10

5 Original

0 100 200 3000

1

2

3

4

5

6x 10

5 Equalized

0 100 200 3000

2

4

6

8

10x 10

5 Adaptive

Histogram Equalization

500 1000 1500 2000 2500

200

400

600

800

1000

1200

1400

1600

1800

10

Histogram Equalization

500 1000 1500 2000 2500

200

400

600

800

1000

1200

1400

1600

1800

11

500 1000 1500 2000 2500

200

400

600

800

1000

1200

1400

1600

1800

Histogram Equalization 12

Dalal Triggs ’05 paper

Objects can be described by edge orientation Compute basic gradient Since we have data in two directions, can calculate a direction

for each pixel Also snap to regular grid – 45 degree angles

Create histogram of these directions for a given window size Computation limitation here – would need to search at

various scales

Biggest problem here is that edges of plates themselves not always well defined Illumination changes, license plate holders, glare, etc.

Bottom line: need machine learning technique here to make this work

Histogram of Oriented Gradients 13

Detects ‘interest points’ in image Based on eigenvalues of ‘sensitivity matrix’

Our idea was to search the detected Harris corners for rectangles of approximately the right aspect ratio Turned out to be an incredible computational burden with the

detector sensitivity high enough to reliably detect license plate corners

Harris Corner Detector 14

200 400 600 800 1000 1200

100

200

300

400

500

600

700

800

900 20 40 60 80 100 120 140

10

20

30

40

50

60

70

80

Canny edge detection to get edge map

For each edge pixel, record the set of all lines that could pass through it Parameterized by distance from origin and angle

Resulting image has peaks at (ρ,Θ) pairs representing prominent lines in the image Idea was to search for peaks corresponding to the

approximate aspect ratio of license plates

Again, edges of plates not always well defined Meant that tuning the thresholds resulted in a

detector that delivered an incredible amount of false positives

Hough/Radon Transform 15

Hough/Radon Transform 16

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Underlying insight is that license plates have a lot of vertical edges (letters) and a specific aspect ratio Take horizontal gradient Filter with ‘matched’ filter

Designed with aspect ratio, approximate license plate size in mind

Threshold Bit of an art – ties in closely with the erosion

Erode Region label

Throw out regions that are too small Cascaded false positive rejection

Rely on context of license plate (car bumper, license plate holder etc.)

Simplest is best 17

Horizontal Gradient 18

200 400 600 800 1000 1200

100

200

300

400

500

600

700

800

900

Matched Filtering 19

0

20

40

60

0

10

20

30

40-1

-0.5

0

0.5

1

500 1000 1500 2000 2500

200

400

600

800

1000

1200

1400

1600

1800

200 400 600 800 1000 1200

100

200

300

400

500

600

700

800

900

500 1000 1500 2000 2500

200

400

600

800

1000

1200

1400

1600

1800

200 400 600 800 1000 1200

100

200

300

400

500

600

700

800

900

Threshold, Erode, Label 20

False Positive Rejection 21Expand candidate license plate window and look for prominent horizontal lines Reject if we don’t find any – bumper or license plate

holder typically shows up, and we interpret ‘prominent’ too stringently

Detection without learning = hard Lots of issues: background clutter, contrast,

scale/rotation invariance, computation, etc.

Color space changes didn’t seem to help Tried various transformations of RGB and LAB

coordinates as well as scale-by-max color balancing

Next steps would be window refinement, exploration of scale and rotation capabilities (and, of course, character segmentation and recognition)

Thanks!

Conclusions 22

Dalal, N. and Triggs, B. 2005. Histograms of Oriented Gradients for Human Detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cvpr'05) - Volume 1 - Volume 01 (June 20 - 26, 2005). CVPR. IEEE Computer Society, Washington, DC, 886-893.

Vahid Abolghasemi, Alireza Ahmadyfard. “An edge-based color-aided method for license plate detection.” Image and Vision Computing, Volume 27, Issue 8, 2 July 2009, pp. 1134-1142.

Stokman, Harro and Gevers, Theo. “Selection and Fusion of Color Models for Image Feature Detection.” IEEE Transactions on Pattern Analysis and Machine Intelligence 29.3 (2007): 371-381.

References 23

Questions? 24