You are on page 1of 9

Journal of King Saud University – Computer and Information Sciences xxx (xxxx) xxx

Contents lists available at ScienceDirect

Journal of King Saud University – Computer and


Information Sciences
journal homepage: www.sciencedirect.com

A Novel Classifier based on Firefly Algorithm


Emad Mohamed Mashhour a,⇑, Enas M.F. El Houby b, Khaled Tawfik Wassif c, Akram Ibrahim Salah c
a
Modern Academy for Computer Science-Computer Science Department, Egypt
b
Systems & Information Department -Engineering Research Division-National Research Centre, Cairo, Egypt
c
Computer Science Department-Faculty of Computers and Information-Cairo University, Egypt

a r t i c l e i n f o a b s t r a c t

Article history: During the past decade, assigning objects to one of several predefined classes with higher accuracy has
Received 17 May 2018 received considerable focus among researchers. Hence, many algorithms such as statistical algorithms
Revised 27 September 2018 have been developed to solve classification problem. Recently, considerable literature has grown up
Accepted 13 November 2018
around swarm intelligence algorithms. Firefly algorithm is a swarm intelligence algorithm that imitates
Available online xxxx
the behavior of the firefly in nature. In this research, a novel classifier approach based on firefly algorithm
is introduced as a supervised learning algorithm. Classification based on firefly algorithm is processed by
Keywords:
simulating the behavior of firefly in attracting other mates based on intensity and distances. The full pro-
Firefly algorithm
Model construction
cess of the proposed algorithm comprises three phases. Feature selection phase, which is responsible for
Model prediction reducing features and picking the most informative features, model construction phase responsible for
Swarm intelligence picking firefly class presenters, and model usage/prediction phase responsible for allocating testing or
Classification unseen samples in their relative classes using class presenters. Seven different datasets have been used
to test classifier performance. Some of the datasets results were compared with ant-miner algorithm.
Results proved that firefly classifier algorithm is a promising and competitive classifier.
Ó 2018 The Authors. Production and hosting by Elsevier B.V. on behalf of King Saud University. This is an
open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

1. Introduction on a pre-knowledge learning model. Classification is one of the


most frequently used processes in most problems that need to cat-
A machine learning technique refers to the ability of a machine egorize objects into a predefined class or category. Evaluation of
to identify or take a decision toward unseen data based on prede- classification performance is based on number of correctly and
fined data (Ponmani et al., 2017). Machine learning techniques are incorrectly predicted samples. Many researchers used statistical
classified into supervised learning techniques and unsupervised techniques in building classifiers but rare studies introduce swarm
learning techniques. Supervised learning techniques are responsi- intelligence algorithms as classification techniques.
ble for developing or creating a predictive model based on input Swarm intelligence is an approach that appears to illustrate
and output data. This model produces predictions about unseen how multiple agents in nature can interact and exchange informa-
data based on its pre-knowledge or training data. On the other side, tion with each other (Yang and He, 2015). A swarm contains a large
unsupervised learning techniques are responsible for studying the number of agents that corporate with each other to perform a
structure and distribution of input data without knowledge about specified task. These agents interact with each other in different
the output data categorization (Sathya and Abraham, 2013). A clas- ways, such as pheromone between ants, waggle dance between
sification is a supervised learning technique, it embodies the bees, flashing and rhyming light between fireflies, etc. Computer
meaning of recognizing, and categorizing unseen samples based scientists map a set of nature behaviors into algorithms and rou-
tines. They introduced the approach through the field of artificial
intelligence in late-80s (Das et al., 2008). Swarm intelligence
⇑ Corresponding author.
approach proved its efficiency in many optimization problems that
E-mail addresses: e_mashhour@hotmail.com (E.M. Mashhour), kwassif@fci-cu.
edu.eg (K.T. Wassif). seek for the best solution such as feature selection and other
Peer review under responsibility of King Saud University. problems.
In the last decade, statistical techniques such as support vector
machine (SVM), k-nearest neighbor (KNN), and other techniques
were used in classification, while t-test, relieff, and other
Production and hosting by Elsevier techniques were used in feature selection. Researchers were also

https://doi.org/10.1016/j.jksuci.2018.11.009
1319-1578/Ó 2018 The Authors. Production and hosting by Elsevier B.V. on behalf of King Saud University.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Please cite this article as: E. M. Mashhour, E. M. F. El Houby, K. T. Wassif et al., A Novel Classifier based on Firefly Algorithm, Journal of King Saud University
– Computer and Information Sciences, https://doi.org/10.1016/j.jksuci.2018.11.009
2 E.M. Mashhour et al. / Journal of King Saud University – Computer and Information Sciences xxx (xxxx) xxx

motivated to use swarm intelligence algorithms such as bee col- sidered as a best solution, where each path has an amount of pher-
ony, particle swarm, and other different algorithms in feature omone, which corresponds to the quality of the solution or the
selection\ranking problems, but rarely these techniques were used path. Ant-miner algorithm contains three phases, which are rule
as classifiers. construction, rule pruning, and pheromone updating. The algo-
A novel supervised learning technique for classification using rithm begins with empty rule as an initial value, and assigns all
firefly algorithm (Yang, 2010) is proposed in this research. The pro- trails (solutions) with same pheromone. Then search for term to
posed algorithm contains three different phases. The three phases be added to the empty rule according to a problem function and
have been integrated together to achieve the classification process. pheromone amount associated with this term. The algorithm
It works as follow, first phase is responsible for decreasing dimen- searches for the best combinations of attributes to form rules.
sionality while at the same time retain the most informative fea-
tures. The other two phases represented as model construction 3. Artificial firefly algorithm
phase and model prediction phase, both of the two phases will uti-
lize the informative features subset produced from feature selec- Firefly algorithm is inspired from the behavior of the real firefly
tion phase. Simulation of firefly algorithm for classification is insect. They interact and send information to each other through
represented by considering each sample as a firefly, and the inten- the rhyming flashing lights emitted from their bodies. The emitted
sity of each firefly is simulated with a set of fitness functions that light is used to attract others (Ali et al., 2014). This emitted light or
are used in optimization problems. The fitness functions used in intensity varies according to distance between fireflies. Mapping
this research are Rosenbrock, Sphere, and Ackley (Merkuryeva, firefly behavior into an algorithm must cover three rules: 1) Firefly
2005). is a unisex, so that all the fireflies will be attracted to each other
The remainder of the paper is organized as follows. Section 2 regardless of its sex. 2) Attractiveness is proportional to brightness,
shows a literature review. Section 3 introduces artificial firefly therefore for any flash lighting between two fireflies, the less bright
algorithm. Section 4 introduces the proposed classification algo- one will move to the brighter one. The attractiveness decreases as
rithm based on firefly. Section 5 illustrates experiment and results. the distance increases between two fireflies. The fireflies will move
Section 6 presents discussion and result analysis. Section 7 pre- randomly in the case that there is no brighter firefly. 3) Firefly
sents conclusion & future work. brightness is influenced or determined by the landscape of the fit-
ness function.
Two important variables in firefly algorithm govern the behav-
2. Literature review ior of the algorithm processing, the light intensity, and attractive-
ness between fireflies (BinWang et al., 2016). Light intensity varies
Due to the rapid prevalence and enhancing in swarm intelli- from each source according to the brightness of the firefly, which is
gence algorithms, researchers try to improve firefly algorithm as represented and calculated with a kind of fitness function. Attrac-
one of the swarm intelligence algorithms to be adaptive for differ- tiveness of each firefly is calculated using Eq. (1) (BinWang et al.,
ent applications. For example, in (Su et al., 2017; Karnan, 2014; 2016).
Vishwakarma and Yerpude, 2014; Rajinikanth and Couceiro,
2015; Chen et al., 2016) firefly algorithm was used to solve image bðr Þ ¼ b0 ecr2 ð1Þ
processing problems, where in (Khadwilard, 2012; Marichelvam Where b0 represents the attractiveness at distance (r) = 0 and some-
et al., 2014; Karthikeyan et al., 2014) firefly algorithm was used times for mathematical computation is considered as 1, and c rep-
in job scheduling, in (Devarajan and Paul Robert, 2015a,b; Tuba resents the value of the light absorption in the air. r is the distance
et al., 2017 firefly algorithm was used as a mobile network solu- between firefly i position and firefly j position. The distance
tion. In (Koodalsamy and Bairavan Veerayan, 2016) firefly algo- between fireflies is the measure of how two fireflies are attracted
rithm was improved to solve the problem of unit commitment of to each other. Fireflies are always in moving status from position
power system, where in (Shrivastav and Dhawan, 2018) firefly to position. According to the fact that attractiveness between fire-
algorithm was used with support vector machine classifier to flies is related to the distance between them, the distance between
enhance the process of intrusion detection in cloud computing any two fireflies i and j is computed by Euclidean distance using Eq.
field. Also firefly algorithm was applied in the field of robotics, in (2).
(Abdul Hassan and Fadhil, 2018) firefly was applied in mobile robot vffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
u d
path planning for 3D dynamic sphere. In (Raja et al., 2017) firefly uX
was used for Kinematic Analysis of Planar Robots. In the field of rij ¼ jjxi  xj jj ¼ t ðxi;k  xj;k Þ2 ð2Þ
object tracking, firefly algorithm proved its capability in tracking k¼1

systems (Gao et al., 2013). In (Amsaveni and Arunkumar, 2015) Where d represents the dimensionality of the problem, and xi,k is
firefly was used in data hiding in spatial domain. In the field of net- the kth component of the position of firefly i. After calculating the
work, researchers in (Reza Sedaghati et al., 2016) used the firefly distance between the two fireflies, suppose the firefly i is less
algorithm in network reconfigurations for distributed systems. brightness than firefly j, so the attractiveness between them occurs
Firefly algorithm proved its capability in solving various problems while moving the firefly i to the firefly j. Eq. (3) (BinWang et al.,
in different fields. 2016) controls this kind of movement and it is represented as
In this research, firefly algorithm was applied to solve classifica-
follow:
tion problems. The classification problems are considered as an  
¼ xti þ b0 eðcri j Þ  xtj  xti þ a  ðrand  1=2Þ
2
important issue in most researches due to its importance in cate- x1þ1
i ð3Þ
gorizing different objects. We found that there is a lack of applying
swarm intelligence algorithms as classifiers. where t represents the number of iterations, the coefficient a repre-
On the best of our knowledge, the only classifier based on sents a random number controlling the size of the random walk,
swarm intelligent is ant miner (Parpinelli et al., 2002). It appeared and rand represents a random number generator which falls
in 2002. It relies on ant colony and expressed with IF-THEN rules. between (Ponmani et al., 2017). The firefly with low brightness
In ant colony algorithm (Colorni et al., 1991) ants communicate moves to the higher one after considering three terms (BinWang
with each other using pheromone. They are designed to search et al., 2016). The first term is the current position of the low bright-
for the shortest path to the target (food). The shortest path is con- ness firefly. Second term is the movement toward the firefly with

Please cite this article as: E. M. Mashhour, E. M. F. El Houby, K. T. Wassif et al., A Novel Classifier based on Firefly Algorithm, Journal of King Saud University
– Computer and Information Sciences, https://doi.org/10.1016/j.jksuci.2018.11.009
E.M. Mashhour et al. / Journal of King Saud University – Computer and Information Sciences xxx (xxxx) xxx 3

higher brightness by the attraction coefficient b. Finally, the last ing sample to its class, if the classification accuracy is acceptable,
term is a kind of random walk calculated by a random generator then the model is considered a successful model in categorizing
multiplied by a. any future unknown data. Otherwise the process is repeated by
selecting other subset of informative features, then create class
presenter and so on. In the next sections, the three phases of the
4. Proposed classification algorithm based on firefly
proposed algorithm are discussed in details.
The idea behind the proposed firefly classification algorithm is
to utilize the behavior of fireflies to develop a classifier as well as 4.1. Feature selection phase based on firefly algorithm
a feature selection tool (Mashhour et al., 2018). The proposed clas-
sification algorithm includes three different phases as follow: As known most datasets suffer from irrelevant and noisy fea-
tures that may lead the classification process to undesirable
First phase: Feature selection. results, therefore in this phase feature selection process is used
Second phase: Model construction. to reduce dimensionality of datasets and eliminate undesirable
Third phase: Model usage\prediction. features. The feature selection process was implemented using
firefly algorithm (Mashhour et al., 2018), by considering each fea-
In feature selection phase, datasets are reduced and filtered ture as a firefly. Fireflies were filtered and selected for classifica-
from noisy, irrelevant, and redundant features. Model construction tion. The output of this phase is a subset that includes the most
phase is responsible for dividing dataset into a set of existing informative features\fireflies.
classes, where each class is processed individually using firefly
algorithm. In this phase, a set of firefly class presenters are pro- 4.2. Firefly model construction phase
duced to represent existing classes, where firefly class presenters
are the most informative firefly in their classes. Subsequently in In this phase each class C = {c1, c2, c3. . .cj} in the dataset will be
model usage/prediction phase, testing or unseen data has been considered as a kind of fireflies, where ci is the class or kind of fire-
compared with different class presenters produced from the model flies, i = {1, 2, 3. . . j} and j is the number of existed classes. All fire-
construction phase to select the nearest and the most fitting class. flies\samples of class ci have been set in a separate group and
Datasets are divided into two sets: training subset and testing sub- processed individually, where each class or swarm has a number
set. As stated above, the proposed algorithm is represented as a ni of fireflies or samples, in which ci = {s1, s2, s3,. . .,sni }, ni is divided
supervised learning technique. Therefore, the learning model will into 75% training ntri samples and 25% testing ntei samples. Suppose
be built based on training\testing samples and training\testing we have j classes/kinds of fireflies and each class ci contains ntri
class labels. training samples as illustrated in Fig. 2. This phase has to extract
Fig. 1 illustrates the block diagram of the proposed algorithm firefly fi class presenter for each class ci through applying firefly
containing different phases of the classification process. It begins algorithm for selecting the most informative fireflies\samples from
with feature selection phase that produce a subset of informative the training samples ntri, where firefly presenter for each class has
features, then passing these selected subset of features to the to represent this class. It is worth noting that the algorithm of
model construction phase. In model construction phase the selecting class presenter has been applied on selected features
reduced subset of features is used to extract firefly class presenters, subset.
while the model usage\prediction phase plays the role of allocating Firefly algorithm is applied on each class ci individually using a
each test firefly to its related class using class presenters. Firefly set of fitness functions for simulating intensities. Class presenter is
class presenters are passed in parallel with the testing fireflies to extracted from each class to represent this class, in which each
the model usage\prediction phase in order to categorize each test- class presenter is the most informative firefly in the class. So a

Most
Feature Model Construction phase based on selected fireflies
informative
Original selection using
fireflies
dataset Firefly
algorithm Firefly class presenters

Passing testing
Testing samples Model Usage/prediction phase
fireflies

Classified testing samples

No Is it acceptable
classification
accuracy?

Yes
Prediction of future unknown samples

Fig. 1. Different phases of classifier based on firefly algorithm.

Please cite this article as: E. M. Mashhour, E. M. F. El Houby, K. T. Wassif et al., A Novel Classifier based on Firefly Algorithm, Journal of King Saud University
– Computer and Information Sciences, https://doi.org/10.1016/j.jksuci.2018.11.009
4 E.M. Mashhour et al. / Journal of King Saud University – Computer and Information Sciences xxx (xxxx) xxx

Training dataset

c1 No of samples in each class (ntri)

Number of
classes/kinds of c2 Number of samples
.
fireflies ci .
.

cj

Fig. 2. Model construction phase responsible for separating classes and processing each class individually.

number of classes or swarms are separated, each swarm is flying effect of this feature for recommending this sample as a candidate
individually. These swarms contain ranked samples according to class presenter. The position’s value of each firefly has changed
their informative value (intensity) for their classes as shown in with each move of a lower intensity firefly to a higher one accord-
Fig. 3. ing to Eq. (3)
The main task for this phase is to search for or discover samples
(class presenters) that hold information about the class. This phase
contains a set of steps processed as follows: (1) input training sam- 4.2.3. Firefly\sample intensity calculation
ples and separate each class in a group, (2) assign values for firefly In this step, each firefly\sample (fi) is assigned a light intensity
positions, (3) calculate firefly intensity using different fitness func- value (Li) calculated by a fitness function. After applying a set of
tions. After applying steps a number of iterations, the most infor- different functions, it is found that Rosenbrock, Ackley, and Sphere
mative sample or firefly has been selected as class presenter for (Merkuryeva, 2005) were the best fitness functions to represent
different classes. The following sub-sections will discuss the firefly intensity. The goal of utilizing more than one fitness func-
sequence of steps in model construction phase. tion is to search for the best fitness function that can simulate
intensity to help in minimizing classifications errors. Intensity is
4.2.1. Input training samples and separate classes\kinds of fireflies used to compare among different fireflies in the same class. The
In this step, the model construction phase will receive the out- firefly with lower intensity updates its intensity after each move-
put of feature selection phase stated in Section 4.1, as a reduced ment (to a new position) through a set of iterations. The process
and refined (features) dataset from the original dataset. The classes from steps 2–3 is repeated m number of iterations until we reach
will be separated, where each class is an individual kind of firefly the best firefly (class presenter).
swarm. The objective of this step is to ensure picking the most pre-
sentable samples (class presenters) for different classes through
4.2.2. Assigning firefly\sample position comparing intensities for all samples in the same class individu-
Firefly position is one of the parameters that determine location ally. The output of this step will be the best firefly\sample in each
of firefly in space. In this research, the position of each firefly is rep- class.**
resented through considering each feature value in the sample as a The proposed pseudo code 1 describes the model construction
firefly position. The feature value in each sample represents the phase.

First class (swarm1) c1 Second class (swarm2) c2


Ranked
samples

Class (swarmj) cj

Firefly class presenter f2 for


second class c2

Firefly class presenter f1 for


Firefly class presenter fj for
First class c1
class cj
Fig. 3. Ranking fireflies or samples in each class according to the quality of these samples.

Please cite this article as: E. M. Mashhour, E. M. F. El Houby, K. T. Wassif et al., A Novel Classifier based on Firefly Algorithm, Journal of King Saud University
– Computer and Information Sciences, https://doi.org/10.1016/j.jksuci.2018.11.009
E.M. Mashhour et al. / Journal of King Saud University – Computer and Information Sciences xxx (xxxx) xxx 5

measure for allocating unseen fireflies to their classes. The distance


dk has been calculated between the position of the testing sample
Input: Matrix S (n, h), where S is the dataset after feature tk, where k = {1, 2,. . .,nte}flying randomly and the position of firefly
selection. n is number of samples, h is number of selected class presenters fbestci which are extracted from different classes
features.
through the previous phase. As stated in Section 3 firefly attractive-
Output: Set of created j firefly class presenters represent-
ness is estimated from distances and intensity, the near distance
ing j classes.
the more attractiveness. After calculating distance D between ran-
Model Construction phase
dom fireflies tk and firefly class presenters fbestci for different
Step 1: Separate each class individually; consider each
class ci as a kind of fireflies flying in swarm. classes, there will be D distances produced, where D = {d1, d2,
Step 2: Read training set containing selected features and d3,. . .,dj} and j is the number of distances calculated, the number
its class labels for each class. of distances must be equal to the number of classes. Distances will
Step 3: Convert training set into a swarm containing fire- be compared, the minimum dmin distance between tk and fbestci
flies f1,f2,f3,. . .,fntri , where (ntri) is number of training samples means the more probability to be attracted to the class ci that con-
in class ci. tains the current fbestci.
Step 4: Consider each value in sample vector as a position Fig. 4 illustrates the mechanism of assigning each testing sam-
for firefly. ples tk to its nearest class. The mechanism relies on the distance
Step 5: Apply firefly algorithm for each class separately calculated between testing samples in the random space and fbestci
with different fitness functions to represent intensity for dif- of each class. As shown in Fig. 4, consider testing sample tk flying
ferent samples. randomly in space, and 3 fireflies class presenters flying with their
Step 6: Choose the firefly class presenter (best sample) swarms C = {c1, c2, c3}, for testing sample tk there will be 3 distances
fbestci for each class that will represent the class ci using firefly
calculated Dcalculated = {dk1,dk2,dk3}. Table 1 illustrates the values of
algorithm, where I = {1, 2,. . . j}.
calculated distances for testing sample tk.
A comparison will be done among the three distances calcu-
lated, to detect the minimum distance between unseen sample
and the three class presenters. Suppose we have the minimum dis-
Pseudo code 1: Model construction phase responsible for pick- tance dk,2, this means that it detects a relationship between unseen
ing class presenters. firefly tk and class c2 represented by fbest2. This process will be con-
tinued for all unseen fireflies tk until each firefly reaches its correct
4.3. Firefly model usage/prediction phase swarm of fireflies.

The model prediction phase will focus on testing dataset. Num-


ber of testing samples nte are considered as unknown fireflies that 4.3.2. Firefly classification based on intensity (FF_Inte)
fly randomly in the space without any related swarm or class. Con- Firefly intensity is another important parameter for fireflies in
sider random space A contains many random fireflies/testing sam- space, it can be used to determine the relevance between the test-
ples A = {t1, t2, t3, t4. . .tnte} flying, where nte is number of random or ing fireflies that required to be classified, and swarms (class pre-
unseen fireflies. The aim of this phase is to discover the related senters) based on the difference between the intensity of the
class c for each testing sample tk. The classification process for testing fireflies and firefly class presenter intensity in each class.
unseen fireflies or testing samples is developed through three dif- Suppose a testing sample tk is flying randomly, and three differ-
ferent approaches, which are as follow: classification based on fire- ent swarms of fireflies are flying in space A. Intensity is calculated
fly distance, classification based on intensity and classification for a testing firefly tk, and then intensity differences are calculated
based on average intensity inside each class (swarm). between firefly class presenter intensity in each class and the test-
ing firefly’s intensity. The testing firefly tk is classified as the firefly
4.3.1. Firefly classification based on distance (FF_Dist) class presenter that its intensity has the minimum difference
Firefly distance is one of the controlling parameters in firefly (DI = fbestintensity – tkintensity) with testing firefly intensity.
algorithm. It measures how the attraction will be executed Fig. 5 illustrates the idea behind the firefly classifier based on
between fireflies. Distance measure plays the role for detecting intensity, as shown in the figure the number of differences (D) of
the nearest class for each testing firefly. In order to measure the intensities for each unseen firefly is equal to number of classes
relevance between testing fireflies and swarms (class presenters), have been compared with.
distance between fireflies that required to be classified (testing As shown in Table 2, three values were obtained for represent-
fireflies) and class presenters will be considered as a similarity ing the differences between testing firefly intensity and firefly class

Firefly class1 presenter f1

Class c1
dk1

Class c3 Unknown sample

dk2
Firefly class3 presenter f3 dk3 Class c2 Firefly class2 presenter f2

Fig. 4. Calculating distance in random space between unseen fireflies and firefly class presenters.

Please cite this article as: E. M. Mashhour, E. M. F. El Houby, K. T. Wassif et al., A Novel Classifier based on Firefly Algorithm, Journal of King Saud University
– Computer and Information Sciences, https://doi.org/10.1016/j.jksuci.2018.11.009
6 E.M. Mashhour et al. / Journal of King Saud University – Computer and Information Sciences xxx (xxxx) xxx

Table 1 Table 2
Example for calculating a number of similarity distance measures for unseen Example for calculating a number of D intensities for detecting the relevant class for
sample/firefly. unseen firefly/sample.

Class presenters Calculated distances Class presenters Calculated differences of intensities


Firefly class1 presenter fbest1 dk,1 Firefly class1 presenter f1 D1
Firefly class2 presenter fbest2 dk,2 Firefly class2 presenter f2 D2
Firefly class3 presenter fbest3 dk,3 Firefly class3 presenter f3 D3

presenter intensity in each class. Minimum difference D value is Unknown sample tk


representing the class of unseen sample. Suppose D2 is the mini- Δ1 Δ2
mum value for differences calculated, means that tk belongs to AvgI2
class 2. This process will be continued for all unseen fireflies tk until AvgI1
Δ3
each firefly reaches its correct swarm of fireflies.
Class c1 Class c2
4.3.3. Firefly classification based on average intensity (FF_AInte) AvgI3
As shown above, the proposed algorithm considers each swarm
Space A Class c3
as an individual class with its own intensity and distance values.
Fireflies in each class are sorted according to their intensity, thus
Fig. 6. Allocating unseen fireflies/samples based on the average intensity of each
the average intensity is calculated from the specified number of class.
best fireflies below class presenter in each class to act as a new
class presenter based on average intensity instead of intensity. This
process is repeated for each class to re-calculate class presenters Table 3
Example for calculating minimum difference between unseen firefly intensity and
for different classes. average intensity.
Suppose a testing sample tk is flying randomly, and three differ-
ent swarms of fireflies are flying in space A, the aim is to allocate Average Class presenters intensity Difference between Average
intensity and tIk intensity
each unseen sample tk to its class ci. Fig. 6 illustrates the mecha-
nism of allocating each unseen sample to its proper class. Average intensity for class 1 AvgI1 D1 = tIk – AvgI1
Average intensity for class 2 AvgI2 D2 = tIk – AvgI2
As shown in Fig. 6 average intensity (AvgI) is calculated for each
Average intensity for class 3 AvgI3 D3 = tIk – AvgI3
class individually, the intensity of unseen fireflies is calculated.
Then differences are calculated between the intensity of tk and
the AvgIi, where i = (1, 2,..j), the testing firefly tk is classified as
the firefly class that its average intensity has the minimum differ-
Input:
ence with the testing firefly intensity.
As shown in Table 3, after calculating differences among aver-
 j firefly class presenters representing j classes
age intensities for classes and unseen firefly’s intensities. Minimum
 Testing samples (represented by selected features)
difference is determined between testing firefly intensity and aver-
age intensities for different classes, which specifies the class of this
Output: Testing samples/fireflies tnte allocated for their rel-
testing sample. For example, as shown in Table 3 testing firefly ative classes ci., accuracy of classification is calculated
intensity tIk is compared with AvgI1, AvgI2 and AvgI3, suppose
the minimum difference is (D3), means that the unseen firefly tk Model Usage/Prediction phase
belongs to class 3. Step 1: Consider each testing sample as an unseen firefly
Pseudo code 2 represents model usage\prediction phase. In this t1,t2,t3,. . .,tnte flying in space A
phase, unseen fireflies\testing samples are passed to be catego- Step 2: Consider each value in unseen sample vector as a
rized. Pseudo code 2 represents model prediction phase for classi- position for firefly.
fication approach based on distance (FF_Dist) which is the most Step 3: Calculate distance using Euclidean distance
promising method among the different three methods mentioned between each testing firefly and j class presenters, in which
above, as will be seen in the ‘‘Experiment and Results” section. j different distances are calculated.
After classifying different testing fireflies, the classification accu- Step 4: Find the minimum distance in space between test-
racy has been calculated, if it is not acceptable, then the process ing samples and different class presenters.
will be repeated with other selected combination of features, Step 5: Set class of testing samples as the class presenter
otherwise the model is ready for predicting unseen data. with the minimum distance

Unknown sample tk
Space A Class c2
ΔI3 ΔI2

Class c3 ΔI1 Firefly class2 presenter


f2 (higher intensity)
Class c1
Firefly class3 presenter
f3 (higher intensity) Firefly class1 presenter f1 (higher intensity)

Fig. 5. Calculating D Intensity between Intensity of unseen fireflies and firefly class presenters Intensity.

Please cite this article as: E. M. Mashhour, E. M. F. El Houby, K. T. Wassif et al., A Novel Classifier based on Firefly Algorithm, Journal of King Saud University
– Computer and Information Sciences, https://doi.org/10.1016/j.jksuci.2018.11.009
E.M. Mashhour et al. / Journal of King Saud University – Computer and Information Sciences xxx (xxxx) xxx 7

Table 4
Different datasets used in the experiments.

Dataset Dataset type #classes #features Samples


Lung Medical dataset 3 56 32
Hepatitis Medical dataset Binary class 19 155
Dermatology Medical dataset 6 35 366
Prostate Microarray 2 7815 40
Leukemia1 Microarray 3 5327 72
DLBCL Microarray 2 5469 77
The small round blue cell tumors (SRBCT) Microarray 4 2308 83

Pseudo code 2: Model usage/prediction phase responsible for Focusing on classification method based on distance, results
classifying testing samples based on distance. prove the effect of this method in classifying different datasets
with different number of classes. Table 7 illustrates the classifica-
tion results along with fitness functions used for simulating inten-
5. Experiment and results
sity. By applying different fitness functions, it was found that
Rosenbrock, Ackley, and Sphere were the best fitness functions to
This section shows the empirical results of the proposed algo-
simulate firefly intensity. Among them, rosenbrock function was
rithm. MatlabÒ 2015a software on an Intel CoreTM i5-2400 CPU @
the best function for simulating intensity.
3.10 GHZ with 4 GB RAM computer is used for implementation.
As shown in Table 7 classification was applied using firefly dis-
The proposed algorithm has been tested using different datasets.
tance, the results of the experiments show that firefly algorithm is
The description of these datasets is shown in Table 4. Lung, Hepati-
a promising machine learning classifier that can be used for classi-
tis, and Dermatology datasets have been obtained from the Univer-
fying different datasets. Dermatology dataset achieved 90% classi-
sity of California at Irvine (UCI) machine learning repository (Bache
fication accuracy using 10 selected features, with intensity
and Lichman, 2002). Prostate microarray dataset has been obtained
simulated by rosenbrock fitness function, while for lung dataset,
from ncbi website (www.ncbi.nlm.nih.gov/geo). Leukemia1,
the achieved classification accuracy was 80% with 5 features using
DLBCL, and SRBCT microarray cancer dataset have been obtained
rosenbrock and sphere functions. Hepatitis dataset achieved accu-
from the GEMS website (www.gems-system.org). The original used
racy 82% with 3 features using rosenbrock and sphere. For microar-
datasets have been randomly divided into 75% for training and 25%
ray datasets, SRBCT achieved classification percentage 90% using 5
for testing.
features with intensity simulated by rosenbrock and Ackley func-
Extensive experimental studies had been tried on different
tions. While prostate achieved classification accuracy 90% using 2
datasets in order to tune the parameters with suitable values to
features by rosenbrock function, for DLBCL, classification accuracy
achieve best results. Table 5 illustrated best parameters used.
achieved was 90% with 2 features using Ackley function, and finally
The three different proposed prediction\usage methods, which
leukemia1 achieved classification accuracy 83% using 2 features,
are classification based on distance, classification based on inten-
with rosenbrock function.
sity and classification base on average intensity were applied.
In machine learning problems, extracting or selecting best fea-
Table 6 shows a comparison between the three different methods
tures has an important role in improving classification. To investi-
applied on the seven datasets used.
gate the importance of features selection on classification, different
As shown in Table 6, the results of three different methods were
acceptable. However, the most promising method applied was
classification based on distance, in which it achieves higher classi- Table 7
Classification accuracy for different datasets using distances between unseen firefly
fication accuracy for all datasets. Therefore, it will be studied in and firefly class presenter.
more details and extra experiments will be applied on this method.
Dataset No of selected Classification accuracy Fitness function
features using (FF_Dist) (Intensity)
Table 5 Dermatology 10 90% Rosenbrock
Parameters values used in classifier algorithm. Lung 5 80% Rosenbrock/
Sphere
Parameters Best values (recommended)
Hepatitis 3 82% Rosenbrock/
Firefly Intensity functions Rosenbrock, Sphere, Ackley Sphere
Firefly position Feature value in each sample SRBCT 5 90% Rosenbrock/
c 1, 2 Ackley
b 1, 2 Prostate 2 90% Rosenbrock
a 0.1, 0.5 DLBCL 2 90% Ackley
No of iterations 150, 250, 300 Leukemia1 2 83% Rosenbrock

Table 6
Classification accuracy for different datasets using different methods.

Datasets No of Classification accuracy using No of Classification accuracy using No of Classification accuracy using
features (FF_Dist) features (FF_Inte) features (FF_AInte)
Dermatology 10 90% 9 82% 9 77%
Lung 5 80% 5 80% 5 50%
Hepatitis 3 82% 4 79% 3 82%
SRBCT 5 90% 2 80% 5 70%
Prostate 2 90% 2 80% 2 80%
DLBCL 2 90% 2 75% 2 75%
Leukemia1 2 83% 2 72% 2 72%

Please cite this article as: E. M. Mashhour, E. M. F. El Houby, K. T. Wassif et al., A Novel Classifier based on Firefly Algorithm, Journal of King Saud University
– Computer and Information Sciences, https://doi.org/10.1016/j.jksuci.2018.11.009
8 E.M. Mashhour et al. / Journal of King Saud University – Computer and Information Sciences xxx (xxxx) xxx

datasets have been classified without features selection. Table 8 Table 10


illustrates the results of applying classification accuracy based on Comparison of ANT-MINER and FF_Dist.

distance (FF_Dist) before\after feature selection. It is shown from Dataset Ant-miner with Ant-miner without Firefly
the results that feature selection is considered as an important pruning pruning classifier
phase for classification, where datasets suffer from noisy and irrel- Dermatology 94% 83% 90%
evant data, which may affect classification accuracy negatively. Hepatitis 90% 92% 82%
Therefore, results without feature selection are poor due to the
dimensionality and noisy data, while on the other side with feature
selection, the accuracy has been improved. As stated in Section 2 ant-miner algorithm is considered as a
For analyzing the performance of the proposed classifier based first classifier based on swarm intelligence algorithm. In ant-
on distance. A confusion matrix is constructed for each dataset miner (Parpinelli et al., 2002.), researchers evaluated their work
individually. It analyzes the accuracy of the classifier on a number by using 6 datasets, with different number of samples, classes,
of datasets, where true values are known. The errors made by the and features. The two common data sets used in (Parpinelli et al.,
classification model are clarified through the confusion matrix. 2002.) and our research are dermatology and hepatitis. In their
Predictions made by the model are shown, in which rows represent work, results were shown on two approaches, without pruning
the known classes of data, and columns represent the prediction undesirable rules, and with pruning. The comparison shown in
made by the model. The diagonal elements made by the matrix Table 10 shows the performance of our classifier and ant-miner
are the number of correct classifications for each class, below classifier, with respect to classification accuracy for these datasets.
and above the diagonal shows the misclassifications (Joshi, 2002). As shown in Table 10 ant-miner with pruning approach outper-
form firefly. However, the proposed firefly classifier outperforms
Sensitiv ity ¼ TP=ðTP þ FNÞ ð4Þ
ant-miner without pruning approach in dermatology.
Regarding our experiment, we run the proposed classifier on a
SpecificityðSpecÞ ¼ TN=ðTN þ FPÞ ð5Þ larger dataset that contains high number of samples. It is called
Statlog ‘‘Australian credit approval” dataset. The dataset have been
Positiv e predictiv e v aluesðPPV Þ ¼ TP=ðTP þ FPÞ ð6Þ obtained from the University of California at Irvine (UCI) machine
learning repository (Bache and Lichman, 2002). This dataset con-
AccuracyðAccÞ ¼ ðTP þ TN Þ=ðTP þ FP þ TN þ FNÞ ð7Þ cerns credit card applications for number of customers. It contains
690 samples with 14 features, and it contains two classes. Our suc-
Table 9 presents the performance of the proposed classifier on
cessful classifier based on distance achieves 85% classification
different medical and microarray datasets. It is the result of calcu-
accuracy for unseen samples.
lating a confusion matrix for each dataset. Confusion matrices are
built based on the correct predictions detected by the proposed
classifier for each dataset. Eqs. (4)–(7) (Joshi, 2002) shows how 6. Discussion and results analysis
all metrics are calculated. PPV shows how the classifier succeeds
in predicting all true samples that are related to different classes In this research, a firefly classifier was proposed for classifying
relative to all positive samples. As shown in Table 9, the rate of different kinds of datasets. The goal of the proposed classifier
positive detection highly exceed in Lung, Hepatitis, SRBCT, Prostate was to discover the relevant class for each unseen firefly or testing
and DLBCL. Where sensitivity of the classifier shows the percent- sample. The novel classifier introduces a new direction for classifi-
age rate of detecting positive samples detected relative to all sam- cation through extracting the class presenter samples from train-
ples that should be related to the class. Finally, the specificity ing dataset. Class presenters are considered as fireflies/samples
measure shows how the classifier was capable of classifying nega- that have the advantage of providing the classifier with character-
tive samples or samples not related to the class as true non-relative istics/features of their classes. These class presenters can be used
samples. as a measure for the relationship between the class and testing
fireflies.
Fireflies interact with each other through intensity and dis-
Table 8
Classification before\after feature selection. tance. These two parameters are considered as important factors
for interaction. Intensity as stated before is simulated through a
Datasets Classification accuracy before Classification accuracy after
set of fitness functions tested all with the seven public datasets.
feature selection feature selection
The best of the tested fitness functions are Rosenbrock, Ackley,
Dermatology 75% 90%
and Sphere. These fitness functions proved their performance in
Lung 50% 80%
Hepatitis 25% 82%
simulating intensity, and help in converging to the optimum
SRBCT 70% 90% solution.
Prostate 70% 90% By comparing firefly classifier with ant-miner, firefly classifier
DLBCL 60% 90% performs better than the version of ant-miner without pruning in
Leukemia1 75% 83%
dermatology dataset. In case of ant-miner with pruning, firefly

Table 9
Evaluating the performance of the proposed classifier on different datasets.

Dataset PPV Sensitivity Specificity (Spec) Overall accuracy (Acc)


Dermatology 0.78 0.8 0.97 90%
Lung 0.8 0.8 0.91 80%
Hepatitis 0.9 0.56 0.56 82%
SRBCT 0.9 0.9 0.94 90%
Prostate 0.91 0.91 0.9 90%
DLBCL 0.875 0.925 0.928 90%
Leukemia1 0.76 0.7 0.876 83%

Please cite this article as: E. M. Mashhour, E. M. F. El Houby, K. T. Wassif et al., A Novel Classifier based on Firefly Algorithm, Journal of King Saud University
– Computer and Information Sciences, https://doi.org/10.1016/j.jksuci.2018.11.009
E.M. Mashhour et al. / Journal of King Saud University – Computer and Information Sciences xxx (xxxx) xxx 9

classifier has classification accuracy less than ant-miner by small Ali, Nadhirah, Othman, Mohd Azlishah, Husain, Mohd Nor, Misran, Mohamad
Harris, 2014. A review of firefly algorithm. ARPN J. Eng. Appl. Sci. 9 (10). ISSN
percentage. In ant-miner, the rule pruning differs with different
1819-6608.
datasets, as shown in Table 10 classification rate may decrease in Amsaveni, A., Arunkumar, C., 2015. An efficient data hiding scheme using firefly
dataset such as hepatitis and exceed in dermatology. It is supposed algorithm in spatial domain. In: 2nd International Conference on Electronics
that ant-miner with rule pruning is the best version for ant-miner and Communication Systems (ICECS). IEEE.
Bache, K., Lichman, M., 2002. UCI Machine Learning Repository. University of
due to its capability for pruning all rules that cover low number of California, School of Information and Computer Science, Irvine, CA.
cases. The problem will be addressed with ant-miner algorithm, as BinWang, Li, Dong-Xu, Jiang, Jian-Ping, Liao, Yi-Huan, 2016. A modified firefly
any dataset will need to be tested with and without pruning, this algorithm based on light intensity difference. J. Comb. Optim. 31 (3), 1045–
1060.
leads to more computational time processing. Ant-miner relies Chen, K. et al., 2016. Multilevel image segmentation based on an improved firefly
on measuring the quality of rules or data using entropy function. algorithm. Math. Probl. Eng.
Ant-miner must scan all features to form the rules that may cover Alberto Colorni, Marco Dorigo, Vittorio Maniezzo, 1991. Distributed Optimization
by Ant Colonies. In: ECAL91 – European conference on artificial life, Paris,
large amount of samples. Firefly classifier uses samples that con- France.
tain informative values (class presenters) which express the class, Das, Swagatam, Abraham, Ajith, Konar, Amit, 2008. Swarm intelligence algorithms
with complexity of algorithm is O(N2). in bioinformatics. Stud. Comput. Intell. (SCI) 94, 113–147.
Devarajan, Jinil Persis, Paul Robert, T., 2015. Modified AODV routing protocol for
MANET using firefly algorithm. Int. J. Appl. Eng. Res. 10 (34), 27176–27182.
7. Conclusion and future work Devarajan, Jinil Persis, Paul Robert, T., 2015. Reliable mobile ad-hoc network routing
using firefly algorithm. Int. J. Intell. Syst. Technol. Appl. 8 (5), 10–18.
Gao, Ming-Liang, He, Xiao-Hai, Luo, Dai-Sheng, Jiang, Jun, Teng, Qi-Zhi, 2013. Object
Classification methodology based on firefly algorithm proved its tracking using firefly algorithm. IET Comput. Vision 7 (4), 227–237.
success in allocating testing samples to their relevant classes. The Joshi, M.V., December 2002. On Evaluating Performance of Classifiers for Rare
classification process was implemented based on firefly distance, Classes. In: IEEE Intl. conf. on Data Mining, Maebashi City, Japan.
Karnan, A.S.D.M., . Medical image segmentation using firefly algorithm and
firefly intensity, and average intensity. Classification based on dis- enhanced bee colony optimization. Int. Conf. Inf. Image Process. (ICIIP-2014),
tance was the best approach for obtaining higher classification 316–321.
accuracy. However, we believe that structure and values of data- Karthikeyan, S., Asokan, P., Nickolas, S., 2014. A hybrid discrete firefly algorithm for
multi-objective flexible job shop scheduling problem with limited resource
sets affect the classification, therefore in other datasets other constraints. Int. J. Adv. Manuf. Technol. 72 (9–12), 1567–1579.
approaches may succeed. Obtained results are encouraging to con- Khadwilard, A. et al., 2012. Application of firefly algorithm and its parameter setting
sider a firefly algorithm as a promising classification tool for differ- for job shop scheduling. J. Ind. Technol, 8(1).
Koodalsamy, Banumalar, Bairavan Veerayan, Manikandan, 2016. Firey algorithm
ent datasets. It was a competitive classifier with the well-known
with multiple workers for the power system unit commitment problem. Turk J
classifier ant-miner, as it takes low time processing since it uses Elec Eng. Comp. Sci. 24, 4773–4789.
small refined subsets of solutions derived from large datasets. Marichelvam, M.K., Prabaharan, T., Yang, X.S., 2014. A discrete firefly algorithm for
the multi-objective hybrid flowshop scheduling problems. IEEE Trans. Evol.
Selecting and tuning parameters, is an important step in obtaining
Comput. 18 (2), 301–305.
and refining the classification results, utilizing different fitness Mashhour, Emad Mohamed, El Houby, Enas M.F., Wassif, Khaled Tawfik, Salah,
functions for simulating firefly intensity was the main key for Akram I., 2018. Feature Selection approach based on Firefly Algorithm and Chi-
our classification process. Testing different fitness functions for square. Int. J. Elect. Comput. Eng. (IJECE) 8 (4).
Merkuryeva, Galina, Bolshakovs, Vitalijs, 2005. Benchmark fitness landscape
simulating firefly intensity gives the opportunity to monitor the analysis. IJSST.
performance of each fitness function in searching for the optimum Parpinelli, Rafael S., Lopes, Heitor S., Freitas, Alex A., 2002.. Data mining with an ant
solution. Rosenbrock, Sphere, and Ackley were the best fitness colony optimization algorithm. IEEE Trans. Evol. Comput. 6 (4).
Ponmani, S., Samuel, Roxanna, Vidhu Priya, P., 2017. Classification, algorithms in
functions for simulating intensity; they all converge to the global data mining – A survey. Int. J. Adv. Res. Comput. Eng. Technol. (IJARCET) 6 (1).
optimum solution. Raja Prathab, T., Suja Mani Malar, R., Ahilan, T., 2017. A Method of Extended
Interestingly, the proposed classifier proved its success on dif- Jacobian and Firefly Algorithm for the Kinematic Analysis of Planar Robots. Int. J.
Rob. Autom. (IJRA) 6 (2).
ferent kinds of datasets such as SRBCT, DLBCL, leukemia1, and Rajinikanth, V., Couceiro, M., 2015. RGB histogram based color image segmentation
prostate as microarrays datasets, these kinds of datasets suffer using firefly algorithm. Procedia Comput. Sci. 46, 1449–1457.
from high noises that may affect the classification process. In addi- Sathya, R., Abraham, Annamma, 2013. Comparison of supervised and unsupervised
learning algorithms for pattern classification. (IJARAI) Int. J. Adv. Res. Artif.
tion, the classifier proved its success on other medical datasets
Intell. 2 (2).
such as hepatitis, lung, and dermatology. Sedaghati, Reza, Hakimzadeh, Mojtaba, Fotoohabadi, Hadi, Rajabi, AliReza, 2016.
A suggested future work, the proposed algorithm can be used to Study of network reconfiguration in distribution systems using an adaptive
modified firefly algorithm. Automatika 57 (1), 27–36.
classify medical images datasets, selecting the images with higher
Shrivastav, Shristi, Dhawan, Gagan, 2018. Detection of intrusion detection system in
intensity that may contain information about disease, and utilizing cloud using artificial intelligence. Int. J. Adv. Comput. Manage. Stud. (IJACMS) 3
this image as a presenter for this type of disease. Another research (2), 43–50. ISSN: 2456-1835.
area is to analyze the rhyme of the flashing intensity in seconds. Su, H., Cai, Y., Du, Q., 2017. Firefly-algorithm-inspired framework with band
selection and extreme learning machine for hyperspectral image classification.
Different rhymes express different fireflies, considering firefly- IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10 (1), 309–320.
rhyming flash as a new parameter to the firefly algorithm with Tuba, Eva, Tuba, Milan, Beko, Marko, 2017. Mobile wireless sensor networks
position, intensity, and attractiveness. coverage maximization by firefly algorithm. Int. Conf. Radioelektronika
(RADIOELEKTRONIKA).
Vishwakarma, B., Yerpude, A., 2014. A new method for noisy image segmentation
using firefly algorithm. Int. J. Sci. Res. (IJSR) 3 (5).
Yang, Xin-She, He, Xingshi, 2015. Swarm intelligence and evolutionary
References
computation: overview and analysis. Recent Adv. Swarm Intell. Evol. Comput.
Stud. Comput. Intell.
Abdul Hassan, A., Fadhil, D., 2018. Mobile robot path planning method using firefly Yang, Xin-She, 2010. Firefly algorithm, stochastic test functions and design
algorithm for 3D sphere dynamic & partially known environment. JUBPAS 26 optimization. Int. J. Bio-Inspired Comput. Arch. 2 (2).
(7), 309–320.

Please cite this article as: E. M. Mashhour, E. M. F. El Houby, K. T. Wassif et al., A Novel Classifier based on Firefly Algorithm, Journal of King Saud University
– Computer and Information Sciences, https://doi.org/10.1016/j.jksuci.2018.11.009

You might also like