You are on page 1of 5

International Conference of Soft Computing and Pattern Recognition

Genetic Stereo Matching Algorithm with Fuzzy Fitness

Haythem Ghazouani
Department of Computer Science
École Supérieure de Technologie et d’Informatique
Charguia II, Tunis 2035, Tunisia
Université de Carthage
haythemghz@yahoo.fr

Abstract—This paper presents a genetic stereo matching when designating individuals for crossover or when electing
algorithm with fuzzy evaluation function. The proposed algo- individuals to build a new population [8].
rithm presents a new encoding scheme in which a chromosome Genetic algorithms (GAs) are adaptive search heuristics
is represented by a disparity matrix. Evolution is controlled by
a fuzzy fitness function able to deal with noise and uncertain that belong to the family of evolutionary algorithms. As
camera measurements, and uses classical evolutionary opera- such, they use natural selection and genetics to perform
tors. The result of the algorithm is accurate dense disparity a random search in order to solve optimization problems.
maps obtained in a reasonable computational time suitable for Although based on random processes, genetic algorithms are
real-time applications as shown in experimental results. not entirely arbitrary, instead they use fitness scores to guide
Keywords-Dense stereo matching; Fuzzy fitness; Genetic the search towards areas of better performance within the
algorithm; Disparity map; search space. The key techniques of GAs are performed to
mimic the natural evolutionary processes, particularly those
I. I NTRODUCTION adopt the Darwinian rule of ”survival of the fittest”. Since
in nature, the competition between individuals to achieve
Evolutionary algorithms try to solve complex problems results in terms of scarce resources results in the fittest
by imitating the Darwinian evolution process. In an EA, individuals surviving at the expense of the weaker ones [16].
artificial critters are created to search for a solution over the The application of artificial genetic algorithms in various
space problem. The artificial critters, known as individuals fields such as image processing, pattern recognition or
are constantly competing with each other to discover optimal machine learning has yielded encouraging results [1, 9].
areas of the search space. Each individual is defined by In particular, several genetic approaches have been used
a fixed encoding scheme representing a single possible to solve the problem of stereo matching [2, 3, 4]. To
solution to the problem. The EA is started by creating an obtain a disparity map, these methods use fitness functions
initial population of size µ including randomly generated defined from similarity and disparity smoothness constraints.
individuals [8]. An evaluation score, known as fitness, is They start by generating an initial population of individuals
then assigned to each individual. Fitness is generated so (chromosomes). Each individual encodes a possible match
that the individual approaches a potential solution as well as with respect to local constraints. After that, the evolutionary
possible. After this initial step, the evolutionary algorithm process is launched to reach a solution for which the pairing
goes into the main iterative cycle. It produces λ children is as compatible as possible with respect to matching con-
from the µ individuals in current population. The children straints. Genetic stereo matching algorithm provides good
are produced using perturbation of individual encoding (mu- results, but its major drawback is the computational effort
tation) and recombination between two or more individuals required to achieve a satisfactory solution. This disadvantage
(crossover). All individuals (children) newly created are can be explained by the fact that classical genetic approaches
assigned fitness scores. A second generation of population use binary encodings for individuals, which leads to some
of µ individuals is formed from µ individuals in the current matching ambiguities [5]. Furthermore, a binary encoding
population and the λ children. The same iterative cycle is requires more storage space and more computing time. To
applied to the new generation and all the generations that get around these limitations, we propose a new encoding
will follow. During the whole cycle, an adaptation pressure for individuals, more compact than binary encoding and
is applied to individuals. That is, evolutionary approach requiring much less space. Reducing the storage space
of the survival of the fittest is applied and individuals try has the immediate effect of reducing the computational
to outrace each other. The adaptation pressure is done by time and allows the search algorithm to explore the search
selection, with the fitter individuals more likely to be chosen space more efficiently [5]. Thus, the convergence time is
for the next generation. The selection pressure is applied considerably improved. The proposed algorithm uses also

978-1-4799-5934-1/14/$31.00 ©2014 IEEE 1


International Conference of Soft Computing and Pattern Recognition

a fuzzy fitness, to overcome noise and uncertain camera


measurement problems. In the next section, we present the
proposed algorithm.

II. T HE GENETIC FUZZY STEREO - MATCHING


ALGORITHM

In stereo-vision problems, we have a pair of images


depicting the same scene, taken from two different angles
of view. The goal is to determine the corresponding pixels
(the projections of the same punctual area on both images).
The first idea that comes to the mind is comparing the
Figure 1. A chromosome representing a 8x8 image, the value of the
areas around the two pixels to have a similarity score. Once element (5,4) is 3 which means that pixel (5,4) in the reference image
the similarity score calculated, the result can be improved corresponds to the pixel (5,7) in the target image.
by including restrictions and calculating the matching that
maximizes the global similarity [6, 14, 15]. The epipolar
restriction is used to reduce the search space [6]. In stereo matching problems, the solution is a disparity
Genetic algorithms are adaptive search methods based on map. A chromosome is an encoding scheme of a potential
the principles of evolution theory, i.e. crossover, mating, solution. Consequently, we define a chromosome as a matrix
mutation, natural selection, fitness. These principles are C with the same size (R × C) as the input image. The value
adapted to be applied in the context of stereo matching. of the cell (r, c) of the chromosome matrix represents the
Before defining genetic operators, we give assumptions on disparity d between the pixel (r, c) in the reference image
which our work is based: The input images used in this work (the right image) and the corresponding pixel in the target
are rectified to have a horizontal epipolar line. Consequently, image.
the disparity depends only on the column index of the pixel:
the pixel (r, c) in the reference image is paired to the pixel B. Fuzzy fitness
(r, c + d) in the target image. A 3D disparity space is In stereo matching, corresponding pixels are identified
defined. Dimension of the disparity space are r, c and d using several constraints such as similarity constraint and
to designate respectively row, column and disparity. Each disparity smoothness. Similarity assumption assumes that
element (r, c, d) of the disparity space is projected to the the projections of the same punctual area of the scene have
pixel (r, c) in the reference image and (r, c + d) in the target nearly the same intensities.
image. The element (r, c, d) refers to the pairing of the pixel Disparity smoothness constraint assumes that the dispar-
(r, c) of the reference image and the pixel (r, c + d) of the ities are smooth in a local neighbourhood. Accordingly, the
target image. Let R × C be the image size. evaluation function is formulated based on similarity and
smoothness constraints.
A. New encoding scheme Similarity assumption is commonly used by stereo vision
In genetic algorithms, a population of individuals repre- approaches to define a static fitness function by calculating
senting potential solutions is preserved within the space of the difference of intensities between two neighbourhood of
search. Each individuals is represented by a finite length the candidate pixels. A static fitness is not robust to noise
vector of components, or variables, by means of some caused by changes of illumination conditions, sampling,
symbols, mostly the binary symbols 0, 1. scanning... In this work, we propose to calculate a fuzzy
To continue the genetic analogy these individuals are quantification of the similarity assumption which can model
likened to chromosomes and the variables are analogous to more efficiently non ergodic phenomena. For this purpose,
genes. Thus a chromosome (potential solution) is composed a grey scale classification of pixels is defined [14]. This
of several genes (variables) [16]. classification is based on three classes; black pixels, white
pixels and average pixels. We define membership functions
State of-the-art genetic stereo algorithms provide good
(Eq.1) of theses classes as Gaussian centered in 0, 127.5 and
matching results, but their major limitation is the computa-
255.
tional time. This disadvantage can be explained by the fact
that classical genetic approaches use binary encodings for ( )
(I(m) − cclass )2
individuals, which leads to some matching ambiguities. Fur- µclass (m) = exp − 2 (1)
2σclass
thermore, a binary encoding requires more storage space and
more computing time [5]. To get around these limitations, I(m) is the intensity at the pixel m, cclass and σclass are
we propose a new encoding for individuals, more compact respectively the center and the standard deviation of the class
than binary encoding and requiring much less space. under consideration. Based on this classification and using

2
International Conference of Soft Computing and Pattern Recognition

similarity assumption, we can assume that the matching of


two pixels m1 and m2 projections of the same punctual
area M on the stereo images, is ”possible” if the two pixels
are in the same grey class, that means (m1 is black AND
m2 is black) OR (m1 is white AND m2 is white) OR (
m1 is average AND m2 is average). Using classical fuzzy
operators, we define a fuzzy matching possibility metric:
Π(m1 , m2 ) (given by Eq. 2). Π(m1 , m2 ) is a measure of
co-membership to a same grey class. It reflects how much
it is ”possible” to have m1 and m2 as corresponding pixels.
 
min(µblack (m1 ), µblack (m2 )),
Π(m1 , m2 ) = max  min(µaverage (m1 ), µaverage (m2 )), 
min(µwhite (m1 ), µwhite (m2 ))
(2)
µclass (m) is the degree of membership of the pixel m to
the class under consideration. The possibility of matching
ranges between 0 and 1. Thereafter, we will use the notation
Π(r, c, d)=Π(m1 , m2 ) with m1 = (r, c) and m2 = (r, c, d). Figure 2. Example of a crossover of two chromosomes in a random dot
During the evolutionary process, the fittest individual matrices
is determined by a fitness function, also known as the
evaluation function. This function aims to direct the selection
by assigning a fitness score to each chromosome of each
generation.
As classical approaches, our fitness function uses intensity
similarity and disparity smoothness. But instead of using
the difference of intensity measurements, which can be
easily affected by noise, we use the matching possibilities.
That makes the proposed fuzzy fitness more robust to
noise, change of view point, occlusions... The Fitness of
an individual matrix C is given by Eq. 3. Figure 3. Example of a chromosome mutation in a random dot matrices

∑ ∑ of the matrices (Figure 2). The child chromosome acquires


F (C) = Sr,c Π(r + i, c + j, C(r + i, c + j))
some characteristics from each parent. The crossover line is
r, c (i, j) ∈ N
chosen randomly in the interval [1, R] for a R × C image.
(3) Mutation is randomly applied to individuals to produce
a varied offspring. This operator randomly changes one or
several cells (a neighbouring N of uniform cells) in an
Sr,c = |∇(r, c)| |∇(r, c + C(r, c))| (4)
individual (Figure 3). Offspring may, thus, inherit several
C(r, c) is the disparity value of the cell (r, c) within the characteristics from their parents. Mutation prevents falling
chromosome matrix C. N is a neighbouring introduced to into local optima and increases the chance to find global
have a discriminating comparison between the projections. optima. The mutation rate is set to 40% and affects matrix
|∇(r, c)| and |∇(r, c + C(r, c))| are Sobel gradient norms cells with lower matching possibilities.
respectively on reference pixel (r, c) and target pixel (r, c +
C(r, c)). That is intended to penalize pixels which project D. Selection
onto uniform regions, i.e. less significant pixels. Once the choice of the chromosome encoding is fixed,
genetic algorithm starts by generating the initial population.
C. Genetic operators Subsequently, a varied offspring is produced using crossover
Some genetic operators are intended to preserve diversity and mutation operators. All the individuals in a given gen-
and to avoid local optima such as mutation. Other operators eration are evaluated by the proposed fuzzy fitness function.
are used to combine best individuals into others such as Based on this evaluation, some chromosomes are selected to
crossover [7]. Offspring is generated from two chosen in- play the role of parents in the next generation. This iterative
dividuals from the current population by exchanging a part process is repeated until a stop condition is reached.

3
International Conference of Soft Computing and Pattern Recognition

Tsukuba Venus Teddy Cones


error rate 2,74% 2,5% 14,7% 7,78%
Table I
E RROR RATE , ACCORDING TO THE M IDDLEBURY EVALUATION SITE FOR
OUR APPROACH

our approach after 80 generations. All of the performance


evaluations were conducted on-line on the Middlebury eval-
uation site [12], according to the benchmark protocol and
the test datasets (of 4 stereo pairs) agreed upon. The error
Figure 4. A) Reinforce image of Teddy (left) and Ground truth map rates obtained in this evaluation were 15.3%. Note that all
(right), B) Disparity map after 10 generations with initial population of
40 individuals (Left), Disparity map after 100 generations with initial
disparity maps were obtained with an initial population of
population of 70 individuals (Right). 70 individuals and a mutation rate of 40%, used in all of
the accuracy evaluation test data on the Middlebury site.

In our algorithm, we use an elitist deterministic selection.


Chromosomes are ordered according to their evaluation
score and only the fittest individuals are retained (around
40%). The fitness used to evaluate all individuals is given by
Eq. (3). The fitness of an individual represents his survival
probability to be selected for the next generation. Sigmoid
or rank functions are commonly used to convert from fitness
to a survival probability. Selection is controlled by a random
function which generates values between 0 and 1 for each
selection. If the fitness of the current individuals is greater
than the generated probability, then this individual will be
chosen for the next generation. The evolution process is
ended when the fittest individual of the current generation
satisfies the stop condition.
III. E XPERIMENTAL RESULTS
In this work, we acquire images using a pre-calibrated Figure 5. A) Reference images of the Sawtooth (left) and Venus (right)
stereo pair, B) Ground truth disparity maps, C) Disparity maps after 100
stereo rig. We also consider that the reference and target generations
images are rectified so that epipolar lines are horizontal
and have the same line number in the reference and target
images. We assume that the rectification is correct, which A main evaluation criterion used is the number of in-
means that the disparity depends only on the column index correct pixels, i.e., pixels for which the absolute disparity
of the pixel: the pixel (r, c) in the reference image is matched difference between the solutions provided and the ground
to the pixel (r, c + d) in the target image. truth is greater than 1. Table I shows the percentage of
The performance of our approach was assessed on the incorrect pixels obtained via the Middlebury evaluation site
Middlebury benchmark site [12], which is well recognized for our approach.
and widely used by the computer vision community. The To check the efficiency of our algorithm, we performed
site contains standard datasets for researchers to experiment a number of tests using the sawtooth image. Three genetic
with using an on-line interface. Also, it contains results for stereo matching algorithms (Table II) are considered for the
the most current algorithms and their relative ranking. comparison. The experiment is made on a dual core with
First pair of stereo images in Figure 4 shows the Teddy 3GHz and 1Go of RAM. Table II illustrates the computa-
stereo pair and the ground truth in (A), the disparity maps tional time for sawtooth with a half-size image (217x190).
obtained with different parameters in (B). We have used an initial population of 50 individuals and
Figure 5 below presents some results in test images. The the maximum number of generations is set to 10 in this test.
top two images (A) are the reference images of Sawtooth Table II shows the improvement in computational time of our
and Venus, the two images in (B) are ground truth (of the algorithm compared to state of-the-art algorithms. Results
disparity map) and the bottom two images correspond to show that our algorithm is suitable for real-time application.
the disparity maps obtained (on the Middlebury site) by

4
International Conference of Soft Computing and Pattern Recognition

Algorithms Execution time


[8] G. Jones, ”Genetic and Evolutionary Algorithms” Encyclope-
Our algorithm 3.4 s
dia of Computational Chemistry, (Schleyer, P.v.R. et al., eds),
Han’s algorithm [11] 9.7 s
Dong’s algorithm [10] 5s pp. 1127-1136, John Wiley & Sons, 1998.
Nguyen’s algorithm [13] 2.7 s
[9] V. Bevilacqua, G. Mastronardi, F. Menolascina and D. Nitti,
Table II ”Stereo-Matching Techniques Optimisation Using Evolution-
C OMPARISON OF EXECUTION TIME IN SECONDS WITH EXISTING
ary Algorithms”, Lecture Notes in Computer Science, Vol.
STEREO MATCHING ALGORITHMS FOR sawtooth IMAGE WITH A
HALF - SIZE IMAGE (217 X 190) AFTER 10 GENERATIONS .
4113, pp 612-621, 2006.

[10] N. Dong-Hu and E. Harbin, ”Stereo Matching Algorithm


Using Population-Based Incremental Learning on GPU”, In-
ternational Workshop on Intelligent Systems and Applications,
IV. C ONCLUSION pp. 1-4 Wuhan, 2009.
We presented in this paper a genetic fuzzy algorithm [11] K-P. Han, T. Bae and Y-H Ha, ”Hybrid Stereo Matching with
for the stereo matching problem. In this algorithm a new a New Relaxation Scheme of Preserving Disparity Disconti-
encoding scheme was proposed. A fuzzy formulation of nuity”, Pattern Recognition, Vol.33, No.5, pp.767-785, 2000.
the similarity assumption is used to propose a matching
[12] Middlebury stereo vision page Available Online:
possibility metric. A fuzzy fitness function using matching
http://vision.middlebury.edu/stereo).
possibilities and disparity smoothness is defined to evaluate
individuals. According to the proposed encoding scheme, [13] V. D. Nguyen, T. T. Nguyen, D. D. Nguyen, SangJun Lee
genetic operators were adapted. Experimental results show and J. W. Jeon, ”A Fast Evolutionary Algorithm for Real-
that the presented algorithm yields a significant improvement Time Vehicle Detection”, IEEE T. Vehicular Technology Vol.
in accuracy and computational time relative to state of-the- 62, No. 6, pp. 2453-2468, 2013.
art genetic stereo matching algorithm. [14] H. Ghazouani, M. Tagina and R. Zapata, ”Fast and Robust
Semi-Local Stereo Matching Using Possibility distributions”,
R EFERENCES International Journal of Computational Vision and Robotics
[1] D. Snyers and Y. Pétillot, ”Image processing optimization (IJCVR) - Vol. 2, No.3 pp. 237-253, 2011.
by genetic algorithm with a new coding scheme”, Pattern
Recognition Letters, 843-848, 1995. [15] H. Ghazouani, ”Navigation visuelle de robots mobiles dans un
environnement d’intérieur”, Doctoral dissertation, Université
[2] Z. Zhang, C. Hou and N. J. Yang, ”A Stereo Matching Montpellier II-Sciences et Techniques du Languedoc, 2012.
Algorithm based on Genetic Algorithm with Propagation
Stratagem”, International Workshop on Intelligent Systems [16] P. Bajpai, & M. Kumar, ”Genetic algorithm - an approach
and Applications, pp. 1-4, Wuhan, May 2009. to solve global optimization problems”, Indian Journal of
computer science and engineering, Vol. 1, No. 3, 199-206,
[3] D-J Lee, J. D. Anderson and J. K. Archibald, ”Hardware 2010.
Implementation of a Spline-Based Genetic Algorithm for
Embedded Stereo Vision Sensor Providing Real-Time Visual
Guidance to the Visually Impaired”, EURASIP J. Adv. Sig.
Proc. 2008.

[4] A. Arranz, M. Alvar , J. Boal , A. Sánchez and A. de


la Escalera, ”Genetic algorithm for stereo correspondence
with a novel fitness function and occlusion handling”, 8th
International Conference on Computer Vision Theory and Ap-
plications -VISAPP 2013. Barcelone, Spain. 21-24 February,
2013.

[5] Y. Ruichek, H. Issa and J. G. Postaire, ”An Evolutionary-


Based Stereo Matching Method with a Multilevel Searching
Strategy”, Soft Computing, Vol. 10, No. 12 pp. 1145-1159,
2006.

[6] M. D. Medeiros, L. M. G. Gonalves and A. C. Frery, ”Using


fuzzy logic to enhance stereo matching in multiresolution
images”, Sensors, Vol. 10, No 2, pp. 1093-1118, 2010.

[7] M. Kumar, M. Husian , N. Upreti and D. Gupta, ”Genetic


algorithm: review and application”, International Journal of
Information Technology and Knowledge Management, Vol. 2,
No. 2, pp. 451-454, July-December 2010.

You might also like