automatic number plate recognition using mil string

Application Note / Automatic Number Plate Recognition using MIL String Reader / 1

Tools for the trade

Included in the Matrox Imaging Library (MIL) software development kit is String Reader, a character recognition tool that was designedwith Automatic Number Plate Recognition (ANPR) applications in mind. MIL is an application programming interface (API) used todevelop applications that incorporate image/video capture, processing and analysis, display and archiving.

MIL and thus String Reader can run on a standard PC platform as well as the Matrox Iris P-Series smart camera from Matrox Imaging.Image/video capture can be performed using Matrox Imaging's video capture boards, some of which can perform image/videocompression and even image authentication, which can be useful for ANPR applications. This white paper describes how MIL’s StringReader addresses the task that is ANPR, and also discusses ANPR application considerations beyond character recognition.

The ANPR challenge

ANPR, as used in access control, as an aid to criminal investigations, speed enforcement and toll collection, is one of the mostchallenging imaging applications around. There are many variables to take into account for ANPR. The following factors imposestringent demands on the imaging tool used for ANPR.

· Image quality and scene complexity (focus, background objects, multiple vehicles, vehicle occlusion, dirty plate, etc.), which introduces possible confusion.

· Weather (rain/sleet, snow, fog), which can introduce image “noise”.

· Time of day (dawn, day, dusk and night), which affects lighting (uniformity and contrast).

· Camera or plate positioning, which may introduce perspective distortions.

· Vehicle speed and spacing, which dictates recognition speed.

· Plate styles (number of characters, single versus multi-row, dark-on-light versus light-on-dark, background graphics).

· Different character fonts including varying sizes

· Plate size (scale), rotation and skew

· Required robustness (recognition and error rate), that is, the tolerance to negative results (i.e., plate is not read) and false positive results (i.e., plate is read incorrectly)..

MIL’s String Reader for ANPR

MIL’s String Reader is a sophisticated feature-based (as opposed to template-based) character recognition tool well suited for ANPRapplications. The String Reader tool automatically locates strings (i.e., a linear sequence of characters) within a whole and complexscene before reading the string(s) (Figure 1).

Automatic Number Plate Recognition using MIL String Reader

Figure 1: MIL’s String Reader handles complex scenes containing a platestring as well as several non-relevant strings.


MIL’s String Reader for ANPR (cont.)

The underlying algorithm is able to automatically handle much of the variability found in ANPR applications. MIL’s String Readerautomatically handles differences in string scale (50% to 200% from reference) and string rotation (±10°). Moreover, it handlescharacter skew due to a camera's perspective and can recognize characters as small as two pixels wide per stroke (Figure 2). TheString Reader tool maintains its effectiveness on a wide range of image conditions encountered in ANPR applications including poorcharacter clarity, underlying graphics on plates, image "noise", non-uniform lighting and poor image contrast (Figure 3).

The setup of MIL’s String Reader is context-based, which enables it to handle different plate formats, including different nationalitiesand regions, all at once. A String Reader context consists of string models and fonts. A string model is used to define a template for aparticular plate style (Figure 4). This string model includes the number of characters within a string (e.g., six for North Americanplates) and the number of strings (on successive lines). It also includes character tone (dark-on-light, light-on-dark or either),individual character scale (50% to 200% from reference), and individual character types (digit, lower case and/or upper case letter,etc.).


Figure 3: Conditions such as poor character quality (focus), non-uniform lighting (seen above) and underlying graphics/dirt on platesare easily handled by MIL’s String Reader algorithm.

Figure 4: MIL’s String Reader interactive utility can be used to definedstring models. For example, a string model (plate style) can be defined ashaving six characters consisting of three letters (L) followed by threedigits (D).

Figure 2: MIL’s String Reader algorithm handles character thicknessas small as two pixels in width.


MIL’s String Reader for ANPR (cont.)

To further increase the read accuracy, MIL ‘s String Reader provides control over parameters such as character, similarity andhomogeneity scores, all of which can be used to increase robustness and improve accuracy (Figure 5). The character's scorequantifies the similarity between the character in captured plate image and the character in the font, and also quantifies the similaritybetween the character in the target and the other characters of the string in the target. The character's similarity score quantifies thesimilarity between the character in the target and the character in the font. Homogeneity, which is used quite a bit in ANPR, providesa measure of how similar each character is to the others in the string. In ANPR where generally all characters are the same scale,this is useful for eliminating characters found to be much larger/smaller than its neighbors.

A font on the other hand, refers to the character set, style and size (Figure 6). While MIL’s String Reader supports plates with Latin-based fonts including accents and punctuations, it currently does not support other fonts such as Arabic, Cyrillic, Kanji, Sanskrit, etc.While a user can quickly and easily create font definitions using MIL’s String Reader interactive utility, a variety of font definitions forplates (or any system font) are readily available from Matrox Imaging.

The performance of MIL’s String Reader depends primarily on the image size and

system configuration. For example, reading an Italian plate in a 1k x 1k image takes an average of 52 ms on a 3 GHz Intel® Pentium 4equipped PC with only a single string model and font in the context. A recognition rate (i.e., plate read correctly) of 92.4% wasmeasured on a bank of real-world, challenging images consisting of European plates with only a 0.2% error rate (i.e., plate readincorrectly). The overhead for processing additional string models is negligible while performance is only loosely dependent on bythe number of fonts to process (about 30% per font).


Figure 6: MIL’s String Reader interactive utility enables font definition and fine tuning.

Figure 5: MIL’s String Reader tracks character, similarityand homogeneity scores, which can be used to increaserobustness and improve accuracy..


Application considerations

Working with a video of a vehicle traveling through a camera’s field-of-view can be used to further enhance reading accuracy (asopposed to just using a single image of a vehicle). In this case, triggering character recognition is based on the presence of the vehicle(i.e., plate) in almost each image that constitutes the video. The vehicle’s plate is thus read multiple times, strengthening theconfidence in the initial result. Consequently, false positives (i.e., incorrect reads) are greatly minimized. A further optimization forspeed would involve tracking the vehicle’s plate as the vehicle travels across a camera’s field-of-view and minimizing the ROI (Figure 7). An example demonstrating both of these principles (multiple reading and plate tracking) is available from Matrox Imaging.

While the String Reader algorithm can easily handle multiple vehicle plates in the camera's field-of-view, algorithm speed can becompromised if the exact number is unknown. Optimal performance is achieved when reading a single string per image. In addition toString Reader, MIL includes image processing functions that can be quite useful for ANPR such as perspective correction (tocompensate for camera positioning), de-interlacing (for eliminating motion artifacts that occur with interlaced video output cameras)and noise removal (Figure 8).


Figure 8: Using MIL’s image processing functions for perspective correction (left), noise removal and de-interlacing (right).

Camera’s field-of-view

Vehicle (i.e., plate) is tracked as it crosses the camera’s field-of-view

The plate is read multiple times as the vehicle crosses the camera’s field-of-view thus increasing the confidence in the initial results

Figure 7: Enhancing reading accuracy by tracking and reading a vehicle plate ina continuous video segment.



Application Note / Automatic Number Plate Recognition using MIL String Reader

Appendix

MIL meets the ANPR challenge

Below are actual field results using MIL’s String Reader to address some of the most common ANPR application demands.

All trademarks by their respective owners are hereby acknowledged. Matrox Electronic Systems Ltd. reserves the right to make changes in specifications at any time and without notice. The information furnished by MatroxElectronic Systems Ltd. is believed to be accurate and reliable. 10-6-2006.

Corporate headquarters:

Canada and U.S.A.

Matrox Electronic Systems Ltd.

1055 St. Regis Blvd.Dorval, Quebec H9P 2T4CanadaTel: +1 (514) 685-2630Fax: +1 (514) 822-6273

FFoorr mmoorree iinnffoorrmmaattiioonn,, pplleeaassee ccaallll:: 11--880000--880044--66224433 ((ttoollll ffrreeee iinn NNoorrtthh AAmmeerriiccaa)) oorr ((551144)) 882222--66002200 oorr ee--mmaaiill:: iimmaaggiinngg..iinnffoo@@mmaattrrooxx..ccoomm oorr hhttttpp::////wwwwww..mmaattrrooxx..ccoomm//iimmaaggiinngg

Images courtesy of Advanced Technologies S.r.l., IEG Inc., and Tannery Creek Systems Inc.

1. Image quality & scene complexity 3. Time of day 2. Weather

5. Vehicle speed and spacing4. Camera or plate positioning 6. Plate styles (European vs. North American)

8. Plate size (scale), rotation and skew 9. Required robustness (recognition or error rate)7. Character fonts including size

automatic number plate recognition using mil string

Documents