Chapter 2 Theoretical Background

Chapter 2 Theoretical Background

2.1 How the TSR System Works

TSR is an image processing system whereby it is used to assist the driver making decisions
navigating the vehicle by identifying the road-sign. Illustrate in figure 2-1.

Image Processing
Captured Image

Road Sign Detection

Road Sign Classification

Output

Figure 2-1 How the TSR System Works

Firstly, a camera mounted on the vehicle captures images of the road ahead and the captured
image will be sent to computer for further processing.
Secondly, the computer will apply certain algorithm to process and analyses the captured
image. For instance, the images will be deburred, color segmented, detecting the road-sign
position and extracting the road-sign from the captured image.
Next, the detected road-sign will be recognized by applying some algorithm.
Lastly, the recognized road-sign will be sent to driver support system, also Illustrate in
figure 2-2.

7
 Chapter 2 Theoretical Background

Figure 2-2 How the TSR System Works

2.2 Traffic Sign

Road and traffic signs, traffic lights and other traffic devices are used to regulate, warn,
guide or inform road users. They help achieve an acceptable level of road traffic quality
and increase safety with orderly and predictable movement of all traffic, both vehicular
and pedestrians [1, 2].
Road and traffic signs are designed to be easily recognized by drivers mainly because
their shapes and colors are readily distinguishable from their surroundings [3].

8
 Chapter 2 Theoretical Background

2.3 Properties of Road and Traffic Signs

Road and traffic signs are characterized by a number of features which make them
recognizable with respect to the environment:
1. Road signs are designed, manufactured and installed according to strict
regulations [4].
2. They are designed in fixed 2-D shapes such as triangles, circles, octagons, or
rectangles [5, 6].
3. The colors of the signs are chosen to contrast with the surroundings, which make
them easily recognizable by drivers [7].
4. The colors are regulated by the sign category [8].
5. The information on the sign has one color and the rest of the sign has another
color.
6. The tint of the paint which covers the sign should correspond to a specific
wavelength in the visible spectrum [4, 9].
7. The signs are located in well-defined locations with respect to the road, so that the
driver can, more or less, anticipate the location of these signs [9].
8. They may contain a pictogram, a string of characters or both [7, 8].
9. In every country the road signs are characterized by using fixed text fonts, and
character heights.
10. They can appear in different conditions, including partly occluded, distorted,
damaged and clustered in a group of more than one sign [7].

9
 Chapter 2 Theoretical Background

2.4 Various Type of Road Sign

Most countries post signage, known as road signs or traffic signs, at the side of roads to
impart information to road users. Since language differences can create barriers to
understanding, international signs using symbols in place of words have been developed in
Kurdistan and adopted in most countries of the world.

For this project, I will focus on Kurdistan road-sign. Road signs in Kurdistan closely follow
those laid down in the traffic sign regulations used in the Kurdistan.

Classification of Kurdistan Road-Sign as below:

Warning signs: warn of possible dangers or unusual conditions ahead and alert motorists
on the hazards to expect.

Figure 2-3 Warning Signs

Prohibitory signs: prohibits certain vehicles or actions on the road.

Figure 2-4 Prohibitory Signs

10
 Chapter 2 Theoretical Background

Restrictive signs: All restrictive signs are circular and have a red border with a white
background. These signs restrict certain vehicles which do not comply with the limits
displayed on the signs.

Figure 2-5 Restrictive Signs

Mandatory signs: give orders with which motorists must comply.

Figure 2-6 Mandatory Signs

Temporary work-zone signs: Temporary work-zone signs are displayed in the vicinity of
road works or construction projects affecting the normal flow of traffic.

Figure 2-7 Temporary Work-Zone Signs

11
 Chapter 2 Theoretical Background

Directional signs:

Figure 2-8 Directional Signs

2.5 Review on Recognition Methods

The first paper on the road sign recognition appeared in Japan in 1984. A growing number
of solutions have been proposed for road sign detection since then. In general, those
solutions include detection phase and classification phase. Detection subsystem is to search
for corresponding road signs. Classification is to evaluate regions found by the detection
subsystem.

Road signs are designed to offer their basic meaning by the combination of color and shape.
There are broadly two major methods applied for road sign detection:
Color-based and shape-based.

2.5.1 Color- Based Recognition of Traffic Signs

Color is the most important attribute that facilitates and improves driving conditions.
Moreover the color used in road sign are regulated by each country and are often nearly
simple primary colors (red, green or blue) with the exception of yellow, a secondary color.
Color is also regulated for the tint of the paint that covers the sign, which should
correspond, with a tolerance, to a specific wavelength in the visible spectrum [9]. Therefore
it is widely used in the systems of road sign recognition, especially in the area of
segmentation where road sign are extracted from the rest of a scene.

12
 Chapter 2 Theoretical Background

Color Segmentation is the common solution to the road signs recognition for initial
detection of signs. Typically, this is based on the assumption that the wavelength arriving
at the camera from a traffic sign is invariant to the intensity of incident light.
Color-based detection methods aim to segment the typical colors of traffic signs in order
to provide a region of interest for further processing. In some systems, the boundaries in
color space are hard-coded by the user, others use learning approaches.
Illustrate in figure 2-8.

Figure 2-9 Flow process of color based recognition

13
 Chapter 2 Theoretical Background

Several techniques on color-based recognition are being reviewed as followed:

Color Neural network: Neural networks can be trained to group and
Recognize patterns of colors. It is well known for its powerful classification capability.
Color segmentation neural network can be used to reduce the color resolution of image
from 65500 colors to a universal palette of eight colors [10]. When used in classification,
neural net can recognize traffic signs within a region of interest. There are different models
of neural network namely Multilayer Perceptron (MLP), Radial Basis Network (RBF),
Self-Organizing Map (SOM) and etc.

HSI / HSV transformation The image captured by camera is represented by its RGB
(Red Green Blue) value, and HSI value can be obtained after transformation. In HSI color
space, chromatic information is represented by the hue coordinate, and intensity coordinate
captures varying light conditions. Also, HSI color space is very similar to human
perception of colors [11]. The HSI system encodes color information by separating out an
overall intensity value from two values encoding hue and saturation to make it more
immune to lighting changes. HSI gives unique information in every component which
makes HSI color space appealing in color segmentation. This algorithm makes
segmentation in adversely illuminated road sign boards possible because hue value is
invariant for the illumination.

Transformation base on CIECAM97 model It first converts RGB space to CIE

standard XYA space. The lightness, chroma, and hue (LCH) are then obtained using
CIECAM97 model. The reason for using this model is that it takes viewing condition into
account and can predict colors as accurate as an average observer [12].

Color thresholding segmentation This is one of the earliest techniques used for image
segmentation. The purpose of thresholding is to classify pixels of an image into object
pixels or background pixels. By matching certain criteria level, a pixel is called an
object pixel if its color is close enough to a reference color, if not it will be a background
pixel.
14
 Chapter 2 Theoretical Background

2.5.2 Shape- Based Recognition of Traffic Signs

Shape, being one of the two important attributes of road signs, is often used for road sign
recognition. Shape detection does not require color information. However the selection of
an object recognition scheme for the detection of road signs based on their shape will have
to address more issues than color, for example:
Sign may appear in cluttered scenes.
Imperfect shape
Variance in scale.
Variation in size
Shape detection necessitates robust edge detection and/or matching algorithm to detect the
relevant shapes. This is particularly difficult when the road sign appears relatively small in
the image, a situation that is going to occur often in particular with low-resolution cameras.
Moreover even if a shape of interest is identified, it can be confused with several other
shapes of man-made objects such as commercial signs and building windows [13]. Shape
detection is more robust to changing illumination as it detects shapes based on edges, and
will efficiently reduce the search for a road sign from the whole image to a small number
of pixels [14].

The common recognition approaches based on shape are:

Shape Neural Network: Neural nets can be used to match a shape of an input object,
threshold value can be used tell whether or not the shape matches. It is usually used in
conjunction with Color Neural Network. For example in [15] and [16], shape neural
network is connected to color neural network, and the input for shape neural network is the
output of color neural network. Good results have been shown. It is more robust.

15
 Chapter 2 Theoretical Background

Template matching: All signs to be recognized are stored in a database.

Each potential sign is normalized in size and compared to every template of the same shape.
Normalized cross-correlations are used. With this encoding, a template is built for an
object, and a correlation computation can be defined, which serves as a measure for
computing matches between templates [17].
The method is fast and can easily be modified to include new classes of signs.

Similarity detection: Signs shape detection is done by computing a similarity factor

between a segmented region and set of binary image samples representing each road sign
shape. The method assumes that both sample and segmented image have the same
dimensions [18].

2.5.3 Other Recognition Approaches

Other than color-based and shape-based recognition, some of the common used approaches
are shown below.

Decision tree: By making use of the gray-level features computed on histogram,

projections and simple spatial moment invariants, decision can be made to recognize
particular road sign. The decision tree allows faster rejection of non-sign regions.

Space-Variant Sensor window: By placing a sensor window on the road sign center, it is
then able to recognize road signs, invariantly with respect to viewing and environmental
conditions. The closer the sensor window is to the road sign center, the quicker it gets
recognized. Road sign center is the chosen position because for most traffic signs the
geometrical center is also the center of information content.

Genetic Algorithm: This algorithm can be used to search for traffic sign in a scene image.
The image is matched by giving the gene information [19]. The gene of individuals can be
expression using a set of equation and determine its characteristic.
16
 Chapter 2 Theoretical Background

Histographic recognition: The histogram of road sign extracted from ROI can be
analyzed to determine if any sign of interest is presented in the image.
The histograph is obtained by overlaying a mask over the image.

2.6 Summarize and My Direction

An overview was provided for the different techniques commonly used, and some can be
incorporated with one another to realize a road sign recognition system.
Comparing to color-based detection, shape-based detection system faces more limitations
as addressed in section 2.5.

17