The Use of Genetic Algorithm for Feature Selection

in Video Concept Detection

Marjan Momtazpour*, Mohammad Hossein Saraee**, and Maziar Palhang***

Department of Electrical and Computer Engineering
Isfahan University of Technology,
Isfahan, Iran, 84156831112
*Momtazpour@ec.iut.ac.ir
** Saraee@cc.iut.ac.ir
*** Palhang@cc.iut.ac.ir

Abstract—Video semantic concept detection is considered as an
important research problem by the multimedia industry in recent
years. Classification is the most accepted method used for concept
detection, where, the output of the classification system is
interpreted as semantic concepts. These concepts can be
employed for automatic indexing, searching and retrieval of
video objects. However, employed features have high dimensions
and thus, concept detection with the existing classifiers
experiences high computation complexity. In this paper, a new
approach is proposed to reduce the classification complexity and
the required time for learning and classification by choosing the
most important features. For this purpose genetic algorithms are
employed as a feature selector. Simulation results illustrate
improvements in the behavior of the classifier.
Keywords-component; Video Concept detection, Genetic
Algorithm, Feature Selection, Classification, Support Vector
Machine.
I. INTRODUCTION
With the explosive growth of data (including text, sound
and video), the need for automatic indexing, searching,
organizing, and categorizing of data is increased [1], [2]. Most
of this data are multimedia ones including sound and videos.
Hence, one of the most important research problems in this
field is video concept detection.
The main goal of video concept detection is to determine
the presence (or absence) of certain semantic concepts such as
“outdoor”, “airplane”, “car” and “person” in a video frame.
Too much effort has been reported in the literature in order to
decrease the semantic gap between the low-level features and
the high-level ones. Examples include automatic concept
detection and high-level feature extraction [3], [4], [5], [6], and
[7].
The most popular approach in concept detection is to use a
classification method to determine if a concept exists in a video
frame or not. The most famous classifier for this type of
problem is Support Vector Machine (SVM) [3]. The output of
the classification system can be interpreted as high-level
semantic concepts used for retrieval and filtering in different
types of videos. A list of important concepts, a machine
learning algorithm, and a large-scale dataset are required for
these classification systems [4].
Proceedings of ICEE 2010, May 11-13, 2010
978-1-4244-6760-0/10/$26.00 ©2010 IEEE
Nowadays, great developments in video concept detection
have been reported in some domains [3], [4], [5], [6], and [7].
The semantic concepts can be from wide range of topics
including objects, scenes, and events. There are certain
standard datasets available for research purposes. For example
NIST TRECVID [8] provides a large-scale set of video data.
The TRECVID 2005 dataset includes hundreds of hours of
broadcast news video from different TV channels.
The existing classifiers have some disadvantages. Because
of high-dimensional features, they suffer from high
computation complexity with respect to the time and memory
consumption. Even with the long durations of time spending
for the learning process, these algorithms are unreliable, with
respect to different performance metrics [3].
In this paper, we propose a novel framework to decrease the
time which is required for learning and classification by
selecting the most important features from the traditional low-
level features, used in the existing classifiers. For this purpose,
Genetic Algorithm (GA) is employed to select a subset of
features. The goal of the paper is to improve the behavior of the
classification in terms of speed of the classification, and the
classification accuracy.
The rest of the paper is organized as follows. The related
works are reviewed in Section 2. In Section 3, an introduction
to feature selection and genetic algorithm is presented. In
Section 4, the proposed method is introduced. Experimental
and simulation results are presented in Section 5. Finally,
Section 6 concludes the paper.
II. RELATED WORKS
Semantic concept detection has a great importance in video
indexing and retrieval. There are several reported researches in
the literature. In this section, a literature review of concept
detection was discussed.
First of all, to process the raw film, the raw video should be
converted to the relational form. Video frames should be
converted into relational database to perform data mining
methods such as classification, clustering, and so on. Type of
extracted features can be different. Examples include sound
extraction from video to convert to text, or to get the subtitle
of videos to learn the actions and speeches during the movie.
The most important part of information in videos is visual
features. Visual features can be grouped into two parts: global
features and local ones. Global features such as color
histogram and edge direction histogram (EDH) provide
information about the whole video frame. However, local
features like SIFT and Bag-of-Features make information
available about specific objects in the images. Most of the
concept detection systems work based on visual features
especially global ones.
In [6] SVM-based classifier is employed for learning and
classification of concepts with global visual features. Also, the
fusions of local and global features, such as the ones which
have been considered in [6], have been employed in [7].
Statistical approaches based on global/local visual and audio
features, and their combinations have been employed in [4]
and several fusion frameworks have been proposed for
concept detection. It is shown that the fused multimodal
models considerably reduce the detection errors, compared to
single models.
The framework which is proposed in [9] is a combination of
Apriori algorithm and association rule mining to find frequent
item-sets in dataset. This model made classification rules to
classify the video shots into semantics concepts.
Bayesian networks have been also employed to build up
relations between different concepts [10]. Then, the structure
of the semantic network could be learned by an appropriate
learning method. An improved version of K2 algorithm has
been proposed in [10]. Parallel Self-Organizing Maps have
been also used for concept detection in [11].
In [12], authors want to have a concept detection system for
web-based videos like YouTube. They proposed an automatic
graph model generator from the online video-sharing websites
for concept relationship learning based on known ontology
(such as LSCOM). An SVM-based learning method is
employed to extract concepts from massive videos like
YouTube.
In [1], for each individual concept, the prior probability of
the concept is incorporated with detection score of an
individual SVM detector. Then, probabilistic estimates of the
target concept are computed using all of the individual SVM
detectors. Finally, these estimates are linearly combined using
weights learned from the training set.
Using all of the features for learning a classifier, is
computationally complex. Therefore, in next sections, a new
framework is presented to learn a classifier based on a partial
of visual features.
III. FEATURE SELECTION BY GENETIC ALGORITHM
Since each extra feature can increase the cost, space and
time of a classification system, image processing researchers
have a major motivation to design and implement classifiers
with small feature sets. However, there is a need to have a
sufficient set of features to achieve accurate recognition rates.
Hence, this led us to search for an optimum subset of features
from the available ones [13].
Feature selection strategies fall into two main categories.
The first category is to select features independent of their
effect on the classification performance. The second approach
directly selects a subset of the available features, in such a way
as to not significantly degrading the performance of the
classifier system [13].
Genetic Algorithm (GA) is one of the most effective tools
for data analysis and pattern recognition [14]. GAs are suitable
for feature selection due to their adaptive behavior and
randomized search technique [15]. Feature selection can
improve classification accuracy.
The major issues in applying GAs to a certain problem are
selecting a good representation of the chromosomes and a
suitable evaluation function as well as appropriate crossover
and mutation operators. In feature selection problems, the form
of the gene is a binary array with length equal to l with each
feature has a bit; indicating the presence (1) or absence (0) of
that feature in candidate feature set [13]. In the following
section, the application of GAs for feature selection in video
concept detection is analyzed.
IV. PROPOSED METHOD
In video concept detection through machine learning
approaches, a large number of features have been considered in
the literature. However, classification based on large number of
features, not only increase the classification time, but also
causes to get stuck in local minima. Therefore, feature selection
can be used as a good solution to decrease the number of
features. In this section, a new method is proposed to decrease
the number of features with Genetic Algorithm.
A. Classification Strategy
Classification is one of the most accepted methods
applicable for video concept detection. While different
classifiers could be employed for this purpose, SVM is more
popular. In this paper, SVM is used as the base classifier.
While SVM classifiers behave well in complex
classification problems, the learning phase can be a time
consuming process, especially when a large number of features
should be considered. Also, it has been shown that, in video
concept detection, SVM just memorizes the concept patterns,
behaving similar to memory-based classifiers such as KNN [3].
Therefore, reducing the number of features can be beneficial
from the time and performance perspectives.
The proposed classification strategy is illustrated in Fig. 1.
In the first step, GA is used to select a subset of features. At
each round of the GA, selected features are evaluated with
respect to an appropriate performance measure. For this
purpose, a randomly generated test data is temporarily
classified with the selected features. Then, classification
performance is measured in terms of correct rate. The temporal
classifier which is used in this paper, is classification based on
the Mahalanobis distance. Since classification based on
Mahalanobis is faster than SVM, this improves the feature
selection phase from the time perspective.
In the second step of the proposed strategy, selected
features are employed for classification with an SVM classifier.
Similar to [16], F-Score, accuracy, recall and precision are the
performance metrics that are measured at the end of the second
step.
B. Genetic Algorithm
When GAs are used for feature selection, a chromosome is
represented as a string of bits with length equal to the number
of features. A 0 or 1, in position i, indicates that whether the ith
feature is selected or not, respectively.
One of the most essential steps in applying GAs to a certain
application is to choose an appropriate evaluation function [13].
In order to select the best feature set, in the evaluation function,
we should evaluate the selected features on the test data.
Hence, in the evaluation function, with chromosomes
consisting of a subset of features, the classifier learns and tests
the dataset with specified features. The evaluation function has
to minimize the error rate of the classifier in each new
generation. It is almost clear that the best chromosome is the
one that has the minimum fitness function compared with other
chromosomes. Fig. 2 illustrates the pseudo code of the
evaluation function.
In any GA-based system, beside an appropriate evaluation
function, proper Crossover and Mutation operations are also
required. Crossover operation defines how the genetic
algorithm combines two individuals, or parents, to generate a
child for the next generation. Mutation specifies how the
genetic algorithm makes small random changes in the
individuals in the population to create mutated children. The Figure 2. Pseudo code of evaluation function.
Mutation and Crossover operations, used in this paper, are
illustrated in Figs. 3 and 4, respectively. It should be mentioned
that the selection function in this research is Tournament.
Selection function selects the population members that will be
used as the parents for the Crossover operation.
V. EXPERIMENTAL RESULTS Figure 3. Example of Mutation operator.
In order to evaluate the proposed method, extensive
experiments have been performed. In this section some
experimental results are presented.
A. Data Sets and Features
The experiments are performed on the development set of
2005 (TREC05) video collection in the TREC Video Retrieval
Evaluation [8]. The TREC05 is a collection of about one
hundred hours of broadcast news video from six channels Figure 4. Example of Crossover operator.
including CNN (English), NBC (English), MSNBC (English),
CCTV (English), NTDTV (Chinese), and LBC (Arabic). This
dataset contains around 60,000 of video shots, which are B. Semantic Concepts
equally distributed across the 6 channels.
A number of semantic concepts are chosen from the
After shot segmentation, each video shot is represented by
LSCOM-Lite lexicon [18], which covers different concepts
the middle frame named as the “key-frame”. Three visual
features have been extracted from each key-frame including presented in the broadcast news videos. These concepts have
Edged Direction Histogram (EDH) with 73 dimensions, Gabor been manually annotated by people to describe the visual
Texture (GBR) with 48 dimensions, and Grid Color Moment content of the key-frames in TRECVID 2005 data sets [19].
(GCM) with 225 dimensions. Detailed description of these The features and annotations of TRECVID 2005 are available
features could be found in [17]. at: http://www.ee.columbia.edu/dvmm/columbia374.
There is a large difference between the frequencies of
different concepts. Common concepts like “Outdoor” have
frequencies around 50%, while rare ones (like “Prisoner”)
have frequencies below 1% [3]. All of the concepts are
categorized into three sets [2]:
1) High-Frequency Concepts which are the concepts with
frequency of positive examples greater than 0.05 such as
“Person”, “Face”, “Outdoor”, and “Sky”,
Figure 1. The proposed classification strategy 2) Medium-Frequency Concepts which are the concepts
with frequency of positive examples between 0.05 and 0.01
such as “Meeting”, “Car”, “Animal”, and “Sports”.
 3) Low-Frequency Concepts which are the concepts with
frequency of positive examples below 0.01 such as “Airplane”,
“Snow”, “Maps”, and “Prisoner”.
The frequency of negative examples in the experimental
dataset is large. Therefore, in this paper, the idea of [6] is used
for balancing the data. To balance the frequency of the
positive and negative samples, randomly take 20% of negative
samples. Then, one of the following situations may occur:
1. If the number of examples in this 20% of negative
samples is more than the number of positive examples, then
this 20% of negative samples is used for training the classifier.
2. If the number of examples in this 20% of negative
samples is less than the number of positive examples, but the
total number of negative examples in dataset is greater than
Figure 5. Learning and classification time for different concepts.
the number of positive examples, a subset of negative
examples with the size equal to the size of positive examples
chosen randomly for training.
3. If the number of negative examples is less than the
number of positive examples, all negative examples are
selected.
C. Experiment Environment
The simulations have been performed on a Pentium IV
computer with a 2.5 GHz Dual Core CPU, and 2GBytes of
RAM. The simulation software is developed in
MATLAB2008. The SVM implementation, employed in the
experiments, is an SVM with Gaussian Radial Basis Function
(RBF) kernel with default sigma equal to 1. In order to find the
separating hyper-planes in MATLAB, we use Sequential
Minimal Optimization (SMO) method. In each experiment, the Figure 6. Precision of classification for different concepts.
number of generations, explored by the GA sub-system, is 50.
The population size should be greater than the number of Precision, recall, F-score, and accuracy of the regular SVM
features. For EDH and GBR this value is 100 and for GCM it are compared with the ones of the proposed solution in Figs. 6,
is equal to 250. In the evaluation function of the GA sub- 7, 8, and 9 respectively. It is almost clear that the related
system, 3-fold cross validation is used to ensure that system outcomes of the proposed method are comparable with the
works independent of the data. regular SVM. It can be seen that in different cases the
D. Numerical Results performance of the existing method is the same as the new one.
However, there are different cases that one of these solutions
For each concept, the GA has to select the best features. The performs better than the other one.
outcome of GA is an input for an SVM learner to learn the While the performance of the proposed solution is almost
dataset. Then the performance of the SVM is compared to the similar to the traditional method, it can reach the results in
performance of a regular SVM that learns with all the shorter period of time. Hence, the achievable accuracy of
available features. In this paper, only six concepts are different classifiers can be compared in conjunction with
examined. These concepts are from three different categories, learning and classification time. For this purpose, we define
introduced in Section 5.2. In other words, “airplane” and the following metric as the achievable accuracy per unit time
“snow”, “car” and “sports”, and “outdoor” and “person” or Accuracy Density:
belong to the Low-Frequency, Medium-Frequency and High-
Frequency categories, respectively. Classification Accuracy
Simulation results are illustrated in Figs. 5 to 10. The time Accuracy Density = (1)
Learning and Classification Time
required for learning and classification in regular SVM and the
proposed method (SVM with GA) are compared in Fig. 5. As
it is clear from this figure, applying GA for feature selection Fig. 10 compares the regular SVM with the proposed
reduces the learning and classification time. This is due to this solution with respect to the Accuracy Density. It is almost
fact that the number of features is smaller in the proposed clear that the proposed method, results in greater values of
method, which results in decreasing the dimension of the accuracy densities, compared with the traditional one.
problem. Table 1 shows the percentage of the selected features by the
GA sub-system for each individual concept. As an example,
for the EDH features of the Airplane concept, GA selects
about 68.49% of features.
VI. CONCLUSION
In recent years, the need for automatic indexing, searching
and categorizing of multimedia information has been increased
dramatically. One of the most important challenges in this field
is automatic image and video concept detection. Due to the
high dimensional features that are employed, concept detection
with existing classifiers suffers from high computation
complexity, in addition to their unacceptable performance in
some cases. In this paper, a new approach has been proposed to
reduce the classification complexity and the required time for
learning and classification by choosing the most important
features. For this purpose genetic algorithms are employed for Figure 9. Accuracy of classification for different concepts.
feature selection. Experimental results show that the proposed
method reduces the learning and classification time while the
performance of the classification remains almost unchanged.

Figure 10. Accuracy Density of classification for different concepts.

TABLE I. PERCENTAGE OF SELECTED FEATURES WITH THE GA SUB-

SYSTEM
Figure 7. Recall of classification for different concepts.
Concept Selected Concept Selected
- Feature features (%) - Feature features (%)
Airplane Snow
68.49 63.01
EDH EDH
Airplane Snow
93.75 87.5
GBR GBR
Airplane Snow
40.89 65.33
GCM GCM
Car Sports
38.36 52.05
EDH EDH
Car Sports
54.17 75
GBR GBR
Car Sports
54.67 38.66
GCM GCM
Outdoor Person
41.10 34.25
EDH EDH
Outdoor Person
58.33 64.58
GBR GBR
Outdoor Person
34.66 12.44
GCM GCM
Figure 8. F-score of classification for different concepts.

ACKNOWLEDGMENT
We would like to thank Dr. Pejman Khadivi and Mr.
Mahmoud Momtazpour for their comments and reviews.
 REFERENCES
[1] Y. Aytar, O.B. Orhan, M. Shah, “Improving Semantic Concept
Detection and Retrieval using Contextual Estimates”, ICME 2007, pp.
536-539, 2007.
[2] W. Jiang, E. Zavesky, S. Chang, A. Loui, “Cross-Domain Learning
Methods for High-Level Visual Concept Classification”, In IEEE
International Conference on Image Processing, October 2008.
[3] J. Yang and A. Hauptmann, “(Un)Reliability of video concept
detection”, Proc. Int. Conf. Image and Video Retrieval, p. 85–94, IV-D2,
2008.
[4] S. Chang, D. Ellis, W. Jiang, K. Lee, A. Yanagawa, A.C. Loui, J. Luo,
“Large-scale multimodal semantic concept detection for consumer
video”, MIR, pp. 255-264, 2007.
[5] A. Yanagawa, A.C. Loui, J. Luo, S. Chang, D. Ellis, W. Jiang, L.
Kennedy, and K. Lee, “Kodak consumer video benchmark data set:
concept definition and annotation”, Columbia University ADVENT
Technical Report 246-2008-4, Sep, 2008.
[6] A. Yanagawa, S. Chang, L. Kennedy, W. Hsu, “Columbia University's
Baseline Detectors for 374 LSCOM Semantic Visual Concepts”,
Columbia University ADVENT Technical Report #222-2006-8, March
20, 2007.
[7] Y. Jiang, A. Yanagawa, S. Chang, C. Ngo, “CU-VIREO374: Fusing
Columbia374 and VIREO374 for Large Scale Semantic Concept
Detection”, Columbia University ADVENT Technical Report #223-2008-
1, Aug. 2008.
[8] NIST. TREC video retrieval evaluation (TRECVID), http://www-
nlpir.nist.gov/projects/trecvid/.
[9] L. Lin, G. Ravitz, M. Shyu, and S. Chen, “Video Semantic Concept
Discovery using Multimodal-based Association Classification”,
Proceedings of the IEEE International Conference on Multimedia &
Expo (ICME), pp. 859-862, Beijing, China, July 2-5, 2007.
[10] F. Wang, D. Xu, H. Xu, W. Wu, “Construction of Semantic Network for
Videos”, ICICIC (2), pp. 217-220, 2006.
[11] M. Koskela, J. Laaksonen, “Semantic concept detection from news
videos with self-organizing maps”, In Proceedings of 3rd IFIP
Conference on Artificial Intelligence Applications and Innovations, pp.
591-599, June 2006.
[12] P. Yuan, B. Zhang, J. Li, “Semantic Concept Learning through Massive
Internet Video Mining”, ICDM Workshops, pp. 847-853, 2008.
[13] H. Vafaie, K.A. De Jong , “Genetic Algorithms as a Tool for Feature
Selection in Machine Learning”, ICTAI, 1992.
[14] M.L. Raymer, W.F. Punch, E.D. Goodman, L.A. Kuhn, A.K. Jain,
“Dimensionality Reduction Using Genetic Algorithms”, IEEE
Transactions on Evolutionary Computation, Vol. 4, pp. 164-171, 2000.
[15] J. Jarmulak, S. Craw, “Genetic algorithms for feature selection and
weighting”, In Proceedings of the IJCAI'99 workshop on Automating the
Construction of Case Based Reasoners, pp. 28-33, 1999.
[16] L. Lin, G. Ravitz, M. Shyu, S. Chen, “Correlation-based video semantic
concept detection using multiple correspondence analysis”, Proc. of 10th
IEEE International Symposium on Multimedia, pp. 316-321, 2008.
[17] A. Yanagawa, W. Hsu, and S. Chang, “Brief Descriptions of Visual
Features for Baseline TRECVID Concept Detectors”, Columbia
University ADVENT Technical Report #219-2006-5, July 2006.
[18] Large Scale Concept Ontology for Multimedia, http://www.lscom.org/
[19] L. Duan, I.W. Tsang, D. Xu, S.J. Maybank, “Domain Transfer SVM for
Video Concept Detection”, Proceedings of the 21st IEEE Computer
Society Conference on Computer Vision and Pattern Recognition
(CVPR 2009), June 2009.