INTERNATIONAL JOURNAL OF IMAGING SCIENCE AND ENGINEERING (IJISE)

189

An Efficient Geometric feature based License Plate Localization and Recognition

C. Nelson Kennedy Babu, Krishnan Nallaperumal
edge density are some of the license plate features used by the above methods. High contrast between characters and background in a license plate is a strong feature which is considered in edge analysis [9]. Character segmentation is an important step in license plate recognition (LPR) system. There are many factors that cause the character segmentation task difficult, such as image noise, plate frame, rivet, space mark, plates rotation and illumination variance. Various methods such as Laplacian Transformation, Region marking method and the largeinterval-based character locating algorithm, Mean shift segmentation along with the Mahalanobis classifier, a method calculating same distance between characters as well as the inter-characters, compensation-based binarization technique are available for the segmentation of characters from License Plates to be recognized. The neural networks [10] (time-delay neural networks) are also used in the process of segmenting characters from the License Plate. In the recognition process of characters in the License Plates, the neural network may be considered to be a very successful approach for license plate recognition. Back Propagation Neural Network, feed-forward neural network, discrete-time cellular neural networks (DTCNNs) etc., are used in the process of recognition of characters of the License Plates. The Probabilistic Neural Network (PNN) is trained to identify alphanumeric characters from car license plates based on data obtained from algorithmic image processing. In this paper, a method is reported for detecting the license plate form a grayscale image and segment the characters contained by the License plate of a car. First a preprocessing stage is used, consisting of a certain number of transforms for this specific type of images. The plate rectangle is then detected, using the Geometric methods for locating lines forming a rectangle which is the detected license plate and a template for the expected rectangular form. An appropriate character segmentation method called Thin window Segmentation which uses the features of the license plate such as number of characters contained in it, is presented next for an effective recognition stage with the help of ANN. This work is based on the Indian License plates (ILP). II. GEOMETRY BASED PLATE LOCALIZATION ALGORITHM GPLA The algorithm uses geometry based features to capture characteristics of License plate in the image regions. This

Abstract License plate location is an important phase in vehicle

license plate recognition for intelligent transport systems. The objective of this work is to design and implement algorithms for License Plate Recognition (LPR) of Indian License Plates. This paper presents a robust time algorithms of license plate location, segmentation and reorganization of the characters present in the located plate. The proposed License Plate Localization algorithm consists of required preprocessing operations leading to successful plate identification. Geometry-based method is used to locate the position of the license plate. Thin window scanning method for Segmentation of Characters and an Artificial Neural Network (ANN) is designed to recognize characters of letter and characters of numbers. Experimental results show that the proposed method improves the effectiveness and robustness for License plate recognition. KeywordsLicense plate localization, recognition, image processing, segmentation, Artificial Neural Network.

I. INTRODUCTION

ICENSE License Plate Recognition (LPR) is an image-

processing technology used to identify vehicles by their license plates. LPR is one form of Intelligent Transport Systems (ITS) technology that not only recognizes and counts vehicles, but distinguishes each as unique. With the wide use of computing technology, Intelligent Transportation System becomes more and more important in Traffic management [1]. A LPR system can be conceptually considered as containing two separate processing stages [8]: License Plate Localization (LPL). License Plate Character Recognition (LPCR). License plate location is an important phase in vehicle license plate recognition of intelligent transport systems. Localization stage of the LPR yields a gray-scale plate clip which consists of printed characters. A class of algorithms describes a set of LPL methods which work effectively for a specific set of image conditions. License Plate Localization (LPL) methods are broadly classified into Morphology based LPL methods, Edge statistics, Neural networks, fuzzy, template based and so on. The aspect ratio, color, variance,

Nallaperumal Krishnan is with the Centre for Information Technology and Engineering, Manonmaniam Sundaranar University, Tirunelveli, India (phone: +91-462-2323650; fax:+91-462-2334363; e-mail:krishnan17563@ yahoo.com). C. Nelson Kennedy Babu is with the Department of Computer Science and Engineering, C.M.S. College of Engineering, CMS Nagar, Ernapuram Post, Namakkal District, Tamilnadu (e-mail: cnkbabu@yahoo.co.in).

IJISE,GA,USA,ISSN:1934-9955,VOL.2,NO.2, APRIL 2008

C. NELSON KENNEDY BABU, KRISHNAN NALLAPERUMAL : AN EFFICIENT GEOMETRIC FEATURE BASED LICENSE PLATE

190

algorithm generates exact features for segmentation and identification of text regions License plate image. A good number of plates are being localized so that to perform segmentation and recognition of License Plates.

Consider f(x,y), an image of NxN pixels and m gray levels. Assume Gm={0,1,...,(m-1)} the gray levels and fiGm the appearance frequency of the gray levels in image f. The probability of level i in image f, using the Equation (2-1).

Pi =
Input Image

fi , i Gm ( NxN )
r 1

(2-1)

Thus, the {pi | i Gm} distribution is obtained. For a given s gray level, if

Edge Detection

0 < pi < 1 , then the next two

i =0

distributions (Equation (2-2) and Equation (2.-3)) may be derived from this one, after normalizations:
Low-Pass Filtering for noise cleaning

Threshold with Max. Entropy Criterion

Po P1 Ps 1 A= , ,., P( s ) P( s ) P( s )
Ps Ps + 1 Pm 1 B= , ,. , 1 P( s) 1 P( s) 1 P( s)

(2-2) (2-3)

Selected Horizontal and Vertical Lines

Where P(s)=

p
i =0

s 1

is the total probability till (s-1) gray

Select candidate region

Candidate extraction

level. The basic idea of the MEC method is the appropriate choice of threshold, that maximize the amount of information obtained from the object and background. Recalling that the measure of information is the entropy, the total amount of information given by A and B is: TE(s) = EA(s)+ EB(s)
s 1 = Pi ln Pi P(s) P(s) i =0 m 1 Pi ln Pi

Yes Plate Detection in the original Image

1 P ( s )
i=s

1 P ( s )

Continue Character Segmentation

= ln [P(s)(1- P(s))] where P( s) =

s 1 s 1

H ( s ) H '( s ) P( s) 1 P( s)
ln(pi) H'(s) (2-4)

Figure 2.1 Design of GPLA Algorithm Preprocessing The work flow of GPLA is given in the Figure 2.1. As a first step in preprocessing, the sobel operator is used for detecting horizontal and vertical lines, basically required for the rectangle shape of the plate present in the license plate image. The edge in an image is the reasonably gradual or sudden change from dark to light or from light to dark of pixel intensity. The most common ways to extract the edges is by filtering the image with a suitable mask to calculate the gradient image of the edges. Sobel operator which is composed of two 3 x 3 masks is used for this purpose. The resulting image is then subjected to low-pass filter for noise removal and smoothing. In order to enhance the foreground of the resultant image a threshold is in need. The Maximum Entropy Criterion, (MEC) provides an unsupervised solution to the choice of threshold dilemma.
IJISE,GA,USA,ISSN:1934-9955,VOL.2,NO.2, APRIL 2008

i =0

pi

H ( s) = =

P
i =0

pi
i=s

m 1

ln( pi )

The Maximum Entropy Criterion assumes finding the threshold s that maximize the following measure: Equation (2-5). TE(s' ) = max{TE(s), sGm} (2-5) This method is computationally feasible and leads in short time to the solution. Also, thresholds are determined automatically. Plate Localization Next the vertical and horizontal lines are selected. The selection is made using a certain threshold, making near horizontal/vertical lines possible candidates. They are stored in a list. Using the horizontal and vertical lines, a rectangle

INTERNATIONAL JOURNAL OF IMAGING SCIENCE AND ENGINEERING (IJISE)

191

shape is detected using geometric properties. Geometry-based methods use measuring differential geometry features. Some of them are widely used in all kinds of applications. For example, the geometry-based corner detectors first segment the sharp, extract its boundary and then search for significant turnings at the boundary. The rectangle portion thus obtained is the location of the license plate in the image. So this rectangle portion is now embossed on the image of the license plate which exactly locates the plate. The located portion in the image of the license plate is cropped now to proceed to the next phase of segmenting the characters present in it. III. THIN WINDOW SCANNING AND NEURAL NETWORK BASED LICENSE PLATE RECOGNITION ALGORITHM TWSNN-LPRA This algorithm uses a thin window scanning method for Segmentation of Characters and an Artificial Neural Network (ANN) is designed to recognize 26 characters of letter and 10 characters of numbers. Character Segmentation Thin Window Scanning method is used here for character Segmentation. This method is proposed to extract characters from plate. This method is conducted by checking the mean of each partition in image (the size of partition will be 56 x 1 pixels). By checking the mean of each partition, it can be decided that the partition is a part of character or background. In this process, background is defined as white (1) and character is defined as black (0). After some experimenting, it is concluded that the thresholding value is 0.8-0.85. Partition value which larger than 0.8-0.85 is defined as background and another is defined as character. Plate is divided into three blocks. First block is contained with letter which corresponds to area domain of the car. Second block is contained with number, and the last block is contained with letter. Window size of characters, then equalized by adding the white object on right side of each windows. Finally, the size of each window is 51 x 26 pixels. In this system, the on-off method is used which represents the image in binary code. Black pixel will be represented by 0 and white pixel will be represented by 1. This representation will be conducted on each line from the window. This representation generate the binary code as much 1326 digits. The rank of binary code will be an input to two-dimension matrix with size 1326 x n pixels, n is a variable of input number. Character Recognition Artificial Neural Network (ANN) is designed to recognize 26 characters of alphabetic characters and 10 characters of number [4]. Here two separate ANN topologies are designed see Figure 3.1. 1. To recognize characters. 2. To recognize numbers.
IJISE,GA,USA,ISSN:1934-9955,VOL.2,NO.2, APRIL 2008

Character

classifier

Character image

number

Figure 3.1 Design of Neural Network The designed ANN is a back propagation network (supervised learning), which is able to recognize input matrix in size 1326 x n. A function is developed in designing this ANN has three layers consisting of input layer, hidden layer and output layer. For recognition of numbers, in the ANN there will be 10 neurons for hidden layer and output layer. For ANN which is used to recognize characters, there will be 20 and 26 neurons for hidden layer and output layer respectively. Target for the ANN are 26 elements for ANNs letter and 10 elements for ANNs number. The value of element of each targets are all zero except one element on specific position which represent the number or letter. For example, the first element is 1 in letters target, represent A. Learning process applied in this network, is using a training function with epoch = 3000, MSE = 1 x 10-5, and learning rate = 0.05. Output from the network will be two dimensional matrix with size 26x n and 10 x n for ANNs character and ANNs number, respectively. The value of output will be in range 0 and 1. To recognize, output should be processed first by converted the highest value to be 1 and other will be 0. Then, the location of element which has value 1 will be founded and the result will represent number or letter. For example, output from ANNs letter which has value 1 at first element will represent the letter A. Image recognition by using Artificial Neural Network is a usual method which customary used. This method is based on its capability to take decision of an input like the capability in human being. The robustness of the algorithm is proved by the results yielded out of it. The experimental results on the analysis of this algorithm are discussed. The style of the License plate used in the development of algorithms are given in the below Table 3.1. Table 3.1: Style of Indian License Plate Considered
Vehicle Category Private owned Cars Commercial (Taxi) Plate Color White Yellow Character Color Black Black Example TN 05 M 5399 PY 01 AP 6011

C. NELSON KENNEDY BABU, KRISHNAN NALLAPERUMAL : AN EFFICIENT GEOMETRIC FEATURE BASED LICENSE PLATE

192

The class of images is based the time and place where they are acquired. The Class 1 and Class 2 contain the images acquired from Tirunelveli and Tuticorin during day and night times. Class 3 and Class 4 images are acquired from those same places during day and night time (see table 3.2). Table 3.2: Image Database Composition
Sample Class 1 Class 2 Class 3 Class 4 Acquired From Tuticorin Tirunelveli Tuticorin Tirunelveli Description Parking Area Road Side Traffic Scene Parking Area Road Side Traffic Scene Average Plate Size in Pixels 626x153 522x131 626x160 520x130 Time Day Day Night Night Number of Images 100 98 80 75

Table 3.6: Plate Localization of Class 4 Comparison with other methods

Algorithms GPLA [3] [1] [2] Total Detections 75 75 75 75 Correct Detections 64 60 61 57 False Detections 11 15 14 18 Location rates (%) 85.3 80.0 81.3 76.0

The advantage of the proposed systems is that it works for all types of license plates having either white or black background with black or white characters respectively. The Experimental results obtained by the LPL algorithms are compared with other existing algorithms as given in the following Tables 3.3, 3.4, 3.5 and 3.6. Table 3.3: Plate Localization of Class 1 Comparison with other methods
Algorithms GPLA [3] [1] [2] Total Detections 100 100 100 100 Correct Detections 97 94 84 95 False Detections 3 6 16 5 Location rates (%) 97 94 84 95

A LPCR algorithms is developed to overcome the limitations with respect to segmentation and recognition of License plates. Each of the recognition algorithms is preceded by a segmentation process. In the case of segmentation algorithms, the performance is determined by accuracy in the separation of character content in the License Plate. The algorithm for segmentation, using Hough Transformation, can solve the problem of rivet, rotation, and illumination variance. The prior knowledge constrained segmentation algorithm can restrain the influence of plate frame and space mark. The results of the Character Segmentation algorithms in comparison with other existing methods are given in the below Tables 3.7, 3.8, 3.9 and 3.10. Table : 3.7 Comparison of Character Segmentation Algorithms Class 1 images
Algorithms TWSNN-LPRA [6] [7] Total Images 100 100 100 Segmented Images 95 97 94 Missing images 5 3 6 Success % 95 97.0 94

Table 3.4: Plate Localization of Class 2 Comparison with other methods

Algorithms GPLA [3] [1] [2] Total Detections 98 98 98 98 83 87 Correct Detections 86 False Detections 12 (Not Reported) 13 11 84.6 88.7 Location rates (%) 87.7

Table : 3.8 Comparison of Character Segmentation Algorithms Class 2 images

Algorithms TWSNN-LPRA [6] [7] Total Images 98 98 98 Segmented Images 93 92 94 Missing images 5 3 6 Success % 94.8 93.8 95.9

Table 3.5: Plate Localization of Class 3 Comparison with other methods

Algorithms GPLA [3] [1] [2] Total Detections 80 80 80 80 Correct Detections 70 69 67 64 False Detections 10 11 13 11 Location rates (%) 87.5 86.2 83.7 80.0

Table : 3.9 Comparison of Character Segmentation Algorithms Class 3 images

Algorithms TWSNNLPRA [6] [7] Total Images 80 80 80 Segmented Images 78 77 74 Missing images 2 3 6 Success % 97.5 96.2 92.5

IJISE,GA,USA,ISSN:1934-9955,VOL.2,NO.2, APRIL 2008

INTERNATIONAL JOURNAL OF IMAGING SCIENCE AND ENGINEERING (IJISE)

193

Table : 3.10 Comparison of Character Segmentation Algorithms Class 4 images

Total Images 75 75 75 Segmented Images 72 69 65 Missing images 3 6 10

IV. CONCLUSION The performance of the developed algorithms for License Plate Localization and License Plate Recognition is found reasonable competing with the existing class of algorithms.. The developed algorithms accurately localize and recognize the License Plates given as input. These algorithms are experimented and the results are obtained. The obtained results are compared and analyzed. The major findings from these tests are: (1) Primary advantages of automated license plate reading include the use of all license plate characters in matching for increased accuracy, ability to capture large samples of vehicles, and low labor intensity compared to other license matching techniques. (2) The major limitations of automated license plate reading include constraints on camera location and positioning, limited success in less-than-ideal conditions, and the evolving machine vision technology. Results of GPLA Algorithm

Algorithms TWSNNLPRA [6] [7]

Success % 96.0 92.0 86.6

In the case of Recognition algorithms, the way to measure the overall success is to calculate the percentage of license plates that have been correctly identified by the machine and then verified by a person supervising the test sample. This percentage is estimated using the Equation (3-1).

A = (T I )%

(3-1)

where A denotes the total system accuracy, T is per cent successful plate segmentation and I is the percentage of successful interpretation of the plate content (characters). Based on the observation from the experiments on discussed algorithms, it is found that for a car image with complex background, the system may find hard to locate the license plate. The system would possibly combine a background technology to subtract the image that only has car in a fixed situation. This may make the location simpler. Moreover, the system could be possible to combine other operators to improve the recognition rate. Some problems remain to be resolved. These problems include ambiguities between similarly shaped characters such as (B - 8), (O - o - 0), (I 1), (A 4) and confusion errors from the character pairs like (C G), (D O) and (K X). The comparison of Recognition rate with among all the three algorithms are given below in the Table 3.11. Table: 3.11 Comparison based on Recognition Rate

(a)

(b)

(c)
Images Category Recognition Rate TWSNN - LPRA Class1 Character Number Character Number Character Number Character Number 95.9 96.8 85.0 82.0 95.4 91.0 88.3 84.0 [4] 96.0 97.3 84.8 78.0 96.1 94.3 85.4 77.1 [5] 87.3 88.7 78.3 79.0 88.2 87.6 81.0 78.6 [3] 97.0 84.3 83.0 82.0 97.0 88.0 92.8 93.4

Result of Thin Window Scanning

Class2

(d) (a) Original Image (b) Plate Located Image (c) Extracted Image (d) Segmented Characters

Class3

Class4

IJISE,GA,USA,ISSN:1934-9955,VOL.2,NO.2, APRIL 2008

C. NELSON KENNEDY BABU, KRISHNAN NALLAPERUMAL : AN EFFICIENT GEOMETRIC FEATURE BASED LICENSE PLATE

194

REFERENCES
Faradji, F.; Rezaie, A.H.; Ziaratban, M.;, A Morphological-Based License Plate Location, IEEE International Conference on Image Processing, 2007, Volume 1, 2007 pp:I - 57 - I 60. [2] Suryanarayana, P.V.; Mitra, S.K.; Banerjee, A.; Roy, A.K.;, A Morphology Based Approach for Car License Plate Extraction, IEEE INDICON, 2005, pp:24 27. [3] S. Z. Wang and H. M. Lee, Detection and recognition of license plate characters with different appearances, in Proc. Conf. Intell. Transp. Syst., 2003, vol. 2, pp. 979984. [4] Qian Gao, Xinnian Wang, Gongfu Xie, License Plate Recognition Based On Prior Knowledge, IEEE International Conference on Automation and Logistics, pp: 2964 2968. [5] Paliy, I.; Turchenko, V.; Koval, V.; Sachenko, A.; Markowsky, G., Approach to recognition of license plate numbers using neural networks, IEEE International Joint Conference on Neural Networks, 2004, vol.4, pp 2965 2970. [6] Feng Yang; Zheng Ma; Mei Xie;, A Novel Approach for License Plate Character Segmentation, 1ST IEEE Conference on Industrial Electronics and Applications, 2006, pp:1 6. [7] Feng Yang; Zheng Ma; Mei Xie;, A Robust Character Segmentation Approach for License Plate, International Conference on Communications, Circuits and Systems, 2007, pp: 679 683. [8] Shyang-Lih Chang, Li-Shien Chen, Yun-Chung Chung, Sei-Wan Chen, Automatic license plate recognition, IEEE Transactions on IEEE Transactions on Intelligent Transportation Systems , March 2004, Vol 5, Issue 1, pp: 42 53. [9] Shridhar, M.; Miller, J.W.V.; Houle, G.; Bijnagte, L.; Recognition of license plate images: issues and perspectives, Proceedings of the Fifth International Conference on Document Analysis and Recognition, 1999, pp:17 20. [10] Haris Al-Qodri Maarif, and Sar Sardy, Plate Number Recognition by Using Artificial Neural Network, Prosiding Semiloka Teknologi Simulasi dan Komputasi serta Aplikasi 2006, pp: 176-182. Nallaperumal Krishnan received M.Sc. degree in Mathematics from Madurai Kamaraj University, Madurai, India in 1985, M.Tech degree in Computer and Information Sciences from Cochin University of Science and Technology, Kochi, India in 1988 and Ph.D. degree in Computer Science & Engineering from Manonmaniam Sundaranar University, Tirunelveli. Currently, he is the Professor and Head of Center for Information Technology and Engineering of Manonmaniam Sundaranar University. His research interests include Signal and Image Processing, Remote Sensing, Visual Perception, and mathematical morphology fuzzy logic and pattern recognition. He has authored three books, edited 18 volumes and published 25 scientific papers in Journals. He is a Senior Member of the IEEE and chair of IEEE Madras Section Signal Processing/Computational Intelligence / Computer Joint Societies Chapter. C. Nelson Kennedy Babu received M.Sc. degree in Computer Science from Madurai Kamaraj University, Madurai, India , M.Tech degree in Computer and Information Technology from Manonmaniam Sundaranar University, Tirunelveli, India in 2004 and submitted Ph.D. thesis in Computer Science and Engineering of Madurai Kamaraj University, Madurai. Currently, he is the Professor and Head of Department of Computer Science and Engineering, C.M.S. College of Engineering, CMS Nagar, Ernapuram Post, Namakkal District, Tamilnadu. His research interests include Signal and Image Processing, Remote Sensing, Visual Perception, and mathematical morphology fuzzy logic and pattern recognition. He is a Member of the IEEE.

[1]

IJISE,GA,USA,ISSN:1934-9955,VOL.2,NO.2, APRIL 2008