International Conference on Computational Intelligence and Multimedia Applications 2007

A Road Trafc Signal Recognition System based on Template matching employing Tree classier
Varun S, Surendra Singh, Sanjeev Kunte R, Sudhaker Samuel R D and Bindu Philip Department of Electronics & Communication S J College of Engineering Mysore - 570006, Karnataka, India Email: binduthomas25@yahoo.co.in, sudhakersamuel@yahoo.com
Abstract The trafc sign detection and recognition system is an essential module of the driver warning and assistance system. Smart vehicles are the order of the day. Such vehicles have the ability to warn drivers of pending situations, remind them of speed limits and even automatically take evasive action. Due to the visual nature of existing infrastructure, signs and line markings, image processing will play a large part in these systems. In this paper we present an automated Trafc sign recognition system allowing an invariance localization to changes in position, scale, rotation, weather conditions, partial occlusion, and the presence of other objects of the same color. The reliability demonstrated by the proposed method suggests that this system could be a part of an integrated driver warning and assistance system based on computer vision technology.

1. Introduction
Trafc signs provide regulatory, warning, or guidance information to drivers. They are designed to be easily recognized by human drivers mainly because their color and shapes differ from those present in natural environments. As such, effective and timely conveyance of information from trafc signs to drivers can denitely decrease the number of trafc accidents. The information conveyance depends in part on the legibility and recognition of the trafc sign content, which is a function of trafc sign and observer characteristics. In autonomous vehicles, they should thus provide the control system with information similar to that offered to human drivers. Trafc Sign Recognition (TSR) Systems are today being incorporated into the Autonomous Intelligent Vehicles of the future, which will take crucial decisions regarding speed, trajectory of motion, etc. Such vehicles will incorporate vision-based vehicle guidance systems that will perform road detection, obstacle detection and sign recognition. Some of the early works on trafc sign recognition system are: An automatic target recognition system developed by Olson and Huttenlocher in [1] uses a hierarchical search tree. Work by Lu et al. [2] is an example of neural network techniques for trafc sign recognition. Piccioli et al. [3] used black and white images. After the extraction of the edges, there is a shape analysis looking for circular and triangular contours. Besserer et al. [4] created a pyramidal structure from the original image. An edge detector is applied to every image of the pyramid. The edges of an image joined the edges of the upper image. By analyzing the generated contours, the signs are classied into triangular, circular, or rectangular signs. De La Escalera et al. in [5] have proposed system for European warning signs (equilateral triangles with one vertex upward) and circular prohibition
0-7695-3050-8/07 $25.00 2007 IEEE DOI 10.1109/ICCIMA.2007.190 360

signs. The system has different stages for color segmentation, corner detection, shape recognition, sign classication. The paper proposes a TSR system developed in two phases: Detection phase and Recognition phase. In the former, the color thresholding in the RGB color space is used to segment the image. The features of trafc signs are investigated and used to detect potential objects. The later phase will be trained to perform the classication and validation. The joint use of classication and validation modules can reduce the false recognition rates. The rest of the paper is organized as follows. Section 2 gives a brief overview of the proposed TSR system. Different modules used in TSR system are described in detail in section 3. Experimental results and conclusion are discussed in section 4 and 5.

2. Overview of the TSR system

After acquiring image of the object through a camera, the image is examined to determine whether a trafc sign is present in it or not in the rst phase. In the next phase, the detected trafc sign is matched with three templates corresponding to circle, circle-bar and triangular respectively. This is based on the fact that all the mandatory and cautionary signs used in India are characterized by equilateral triangles or circular shapes with or without a cross. These three templates form our basic templates. The remaining trafc signs are derived from these templates. Hence they are referred as derived templates. For example, all the speed signs are derived templates which belong to the circular basic template. Therefore the basic template category of the trafc sign is determined. In the last phase, the detected trafc sign is matched against the derived templates belonging to the same basic template class.

Figure 1. Three-phase decision tree used for trafc sign recognition.

3. Modules of TSR system

The TSR system gives an accurate output to the user through the interaction of ve modules namely, (a) Pictures module (b) Objects module (c) Database module (d) Classier module (e) GUI module. The interaction between these modules is illustrated in Figure 2. 3.1. Picture module This module deals with the following: (a) Image Capturing (b) Preprocessing the image (c) Segmentation of the image on the basis of color information (d) Retrieval of all the objects present in the current image.

361

Figure 2. Interaction between the various modules of the TSR system.

In this module, the image is rst preprocessed to remove the noise. The R, G and B values for each pixel are then determined. The sum of green and blue pixel components is then compared with 1.5 times the red pixel component for each pixel. Pixels corresponding to relatively higher red component values constitute the feature pixels and are an indication of the objects of interest. A binary segmented image is then created using the known coordinates of the feature pixels. Figures 3(a) and Figure 3(b) show the original image and its segmented version respectively.

Figure 3. Example of a binary segmented image (a)Original Image (b)Segmented Image.

3.2. Objects module This module deals with the following: (a) Detection of the various objects present in the image (b) Determines whether the objects are valid trafc signs or not (c) Determines the basic template class to which the detected trafc sign belongs to. In this module, all the 8-pixel connected objects are considered and labeled with a unique tag. If the number of detected objects is greater than 50, then it is an indication to the presence of unnecessary objects. Such objects are removed through a combination of morphological opening and closing processes [6]. For each of the detected objects, the following tasks are performed: The object is rst enclosed in a virtual quadrilateral. This quadrilateral is then examined to determine whether it is an approximate square with 40X40 or greater dimensions. These conditions are the preliminary requirements that need to be satised to enable efcient extraction of features.
362

 If these conditions are met, the portion of the segmented image that contains this object is extracted. This segment is called the feature image. The feature image is then resized to match the dimensions of the template. The templates are inverted such that features correspond to non-zero values. Euclidean distance transformation is then applied to both the resized feature image and the inverted template. The distance transformed mask elements corresponding to the feature pixels in the basic template are added. The sum constitutes a score. A perfect match is indicated by a score of 0. If any of these scores fall below the matching threshold, the template can be considered found which conrms the presence of template features in the image. This is called Forward Matching. The score of each distance transformed template is computed when correlated against the mask. If any of the scores fall below the matching threshold, it conrms the presence of image features in the template. This is called Reverse Matching. The nal score is thus the sum of scores obtained through both Forward and Reverse Matching. This operation is illustrated in Figure 5. The scores are thus computed for all the objects with respect to the three basic templates and the best scores for each of these basic templates will be considered when the nal decision regarding the basic template classication is made. This module will accordingly return the label of the relevant object and the basic template index. It also determines the location of the object and returns sections of the original and segmented binary image which contain the object. If there is no trafc sign present in the image than the nal score obtained will be huge. Therefore, a provision for an additional upper threshold has also been made.

Figure 4. Matching Methodology.

3.3. Classier module This module deals with (a) Validation of the detected trafc sign (b) Extraction of the object features (c) Recognition of the trafc sign In this module, the detected trafc sign is rst validated by computing the mean x and y coordinates using the Cartesian coordinates corresponding to the non-zero pixel components in the binary feature image. The city block distances of every pixel from the mean pixel are computed. After

363

normalization, the computed values are binned into ten containers. The divergence of the detected Trafc Sign is computed with respect to the previously computed normalized city block distances corresponding to the basic template it has been categorized under. If the divergence is below a certain threshold, the detected trafc sign is considered valid. Table 1 illustrates the previously computed binned city block distances corresponding to the three basic templates, namely circle-bar (CB), circle(C) and triangle (T). Table 1. Normalized City Block Distances Corresponding to the Basic Templates

The holes in the feature image are then lled thereby creating a temporary mask. The mask obtained is used along with the original feature image to modify the segment of the original image containing the trafc sign such that the trafc sign features are superimposed on a white background. This is called the Data Image. This is accomplished as follows: 1. The pixels in the original image segment, corresponding to non-zero mask elements, are retained. The remaining pixel components are set to 1. 2. The coordinates of all the pixel components, corresponding to the non-zero pixels in the binary feature image, are determined and the corresponding pixels in the image segment are set to 1. The Data image is then inverted such that the features correspond to the non-zero pixels. If the number of non-zero pixel components is greater than a suitable threshold indicating presence of sufcient feature data, the nal classication step is carried out. Forward matching and reverse matching is performed by correlating the inverted data image against the derived templates. The derived template corresponding to the minimum obtained score is determined and a suitable media output is provided to the driver. 3.4. Database module The template database is built which will be used to determine the information conveyed by the trafc signs. Upon construction, it takes the added functionality of inserting and retrieving records from the database as and when the TSR system dictates. The trafc sign templates, endorsed by the Delhi Trafc Police, are frequently used on national & state highways. These templates are used for compiling the template database. 3.5. GUI module This module deals with Interaction with the various modules of the TSR system Accepts the input images from a suitable image capturing device such as a camera Provision of an audio output to the user providing the driver with information that has been extracted from trafc signs.
364

4. Experimental results
A test set was created by capturing about 2000 images of trafc signs consisting of triangle shaped, circular shaped with or without a cross. The trafc sign recognition algorithm gave excellent results and met the constraints of real-time situations. The recognition rates for different basic shapes obtained are illustrated in Figure 6. The algorithm could detect the signs rotated up to 200. Also the system was tested with signs which are partially occluded and got recognition rate about 82%.

5. Conclusion
In this paper we propose a Trafc Sign Recognition system which is capable of recognizing the trafc signs at high accuracy. The system is invariant to rotation, partial occlusion of trafc signs, and the presence of other objects of the same color in the image. By further extending the proposed work it can be used in an autonomous vehicle guidance system which make crucial decisions regarding speed, trajectory of motion and has a myriad of applications.

Figure 5. System Performance.

References
[1] C. Olson and D. Huttenlocher, Automatic target recognition by matching oriented edge pixels, IEEE
Transactions on Image Processing, Vol. 6, No. 1, 1997, 103113. [2] S. Lu and A. Szeto, Hierarchical Articial Neural Networks for Edge Enhancement, Pattern Recognition, Vol. 26, No. 8, pp. 1993, 427435. [3] Piccioli, E. De Micheli, P. Parodi, M. Campani, A Robust Method for Road Sign Detection and Recognition, ECCV, 1994, 495500. [4] B. Bessere, S. Estable, B. Ulmer, and D. Reichardt, Shape classication for trafc sign recognition, In Proc. 1st IFAC Int. Workshop Intelligent Autonomous Vehicles, 1993, 487492. [5] Arturo de la Escalera, Luis E. Moreno and Miguel Angel Salichs, Road Trafc Sign Detection and Classication, IEEE Trans. on Industrial Electronics, Vol. 44, No. 6, 1997, 848859. [6] Anil K. Jain, Fundamentals of Digital Image Processing, Prentice Hall, 2001.

365