25th International Conference on Computer

Theory and Applications

ICCTA 2015

24-26 October 2015

Alexandria, Egypt

Conference Proceedings
Contents:
Session # Session Title Page#
Session I Artificial Intelligence and Machine Learning 1
Session II Computer Control and Industrial Applications 39
Session III Information Systems and Software Engineering 71
Session IV Computer Networks, Security, and HPC 112

Session V Image Processing and Pattern Recognition 166

ICCTA 2015 Committees 208

 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Session I:
Artificial Intelligence and Machine
Learning

Title: Page#

Improving Sparse Subspace Clustering Using Subsampled 2

Randomized Hadamard Transform
Sabra Hechmi and Ezzeddine Zagrouba

Accuracy Improvement of WPBC Dataset-Based Breast Cancer 9

Diagnosis
Magdy Abd-elghany Zeid ,M. B. Abdelhalim, and Gouda I. Salama

Simple and Robust Design of A Neural-Based Ensemble Classifier 17

Hassan Wahbi Hassan, Amin Shoukry, and Mahmoud Farouk

Epistemic Hybrid Tree Logic 22

Ammar Mohamed, Rana Mohamed, and Hesham Hefny

Opinion Mining Framework For News Reviews To Build Good 28

Customer Loyalty
Fainan Nagy El sisi, Abd El-Fatah Hegazy, and Hatem Abdel Kader

Framework for Activity Recognition Using Smart Devices 34

Walid Mohamed Aly, Khaled Eskaf, and Alyaa Aly

1
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Improving Sparse Subspace Clustering Using Subsampled Randomized Hadamard

Transform

Sabra Hechmi and Ezzeddine Zagrouba

Equipe de Recherche en Syst`emes Intelligents en Imagerie et Vision Artificielle (SIIVA), Laboratoire RIADI,
Institut Superieur dInformatique (ISI), Universite Tunis El Manar, Abou Raihane Bayrouni, 2080, Ariana, Tunisie.
Email: hechmi sabra@yahoo.fr, ezzeddine.zagrouba@fsm.rnu.tn

AbstractIn the era of high dimensional data, trying to find
clusters is a challenging issue. In this context subspace cluster-
ing has showed surprising results. The tenet of this technique
is to uncover groups of data that exist in multiple underly-
ing subspaces. Recently, several methods have been proposed
that arise in many research fields, including signal/image
processing, system identification and computer vision. In the
presented work, we are interested to an algorithm, namely
Sparse Subspace clustering (SSC). This algorithm has showed
encouraging results to cluster high dimensional data. In spite,
the meaningful drawback of SSC is its computational complex-
ity due to the solving of the sparse optimization problem. In this
study, we propose a new method which can deal with large scale
issues. The key idea is what we call Sub-sampled Randomized
Hadamard Transform (SRHT) that approximates the amount
of data X T X (X is the data matrix), used to compute the
sparse solution. The performance of the proposed method is
tested on real datasets, especially Extended Yale B database
for face clustering and Hopkins 155 for motion segmentation
problem.
1. Introduction
Clustering is a fundamental task in machine learning
which organizes objects into sensible classes. Clustering
methods have been used in image analysis, information
retrieval and bio-informatics. Therefore, several methods [1]
have been proposed. However, the choice of an appropriate
method stays an open issue that depends essentially on the
nature of data and the domain application. Unfortunately,
applied to high dimensional data, conventional clustering
methods fail to generate meaningful results. Indeed, in
high-dimensional spaces, the data is often very close to
each other, which masks groups. In this case, clustering
algorithms based on a distance measurement become in-
creasingly meaningless. Subspace Clustering techniques [2]
succeed to overcome this problem by uncovering clusters
that exist in multiple underlying subspaces. Various tasks
in computer vision can be solved as a low-dimensional
subspace clustering issue, such as face clustering [3], motion
segmentation [4] and image segmentation and compression
[5]. Over the last few years, numerous subspace clustering
methods have been proposed. These methods can be broadly
divided into four principle groups: Algebraic methods in-
cluding Boult and Brown algorithm [6] and Costeira and
Kanade [7] method. Iterative methods: such as K-subspaces
[8] and K-planes [9] algorithms. Statistical methods: includ-
ing Mixtures of Probabilistic PCA [10] and Multi-Stage
Learning [11]. Finally, Spectral clustering-based methods:
including Local Subspaces Affinity (LSA) [12], Spectral Lo-
cal Best-Fit Flats (SLBF) [13] and Low Rank Representation
(LRR) [14].
In this paper, we study a special type of subspace
clustering methods which is Sparse Subspace Clustering
(SSC) [15]. To cluster a dataset according to their underlying
subspaces, SSC is based on the idea of sparse data represen-
tation [16], means that, every data point can be represented
as a linear combination of few other data points from the
same dataset. This representation is used after in a spectral
clustering framework to infer the segmentation of the data
into subspaces. The motivation behind this algorithm is that
a sparse representation of a data point corresponds to a
combination of points from its own subspace. The algorithm
showed very interested performance compared to other spec-
tral methods. In spite, due to the high dimensionality and the
solving of the sparse optimization solution, this algorithm is
very computationally demanding. Several algorithms have
been proposed to overcome this problem. Unfortunately,
they fail to preserve the clustering performance such as
Scalable Sparse Subspace Clustering (SSSC) [17] or to
accelerate dramatically the algorithm such as the Latent
Space Sparse Subspace clustering (LS3C) [18].
Here, we propose a straightforward method to speed up
the SSC algorithm. In fact, the algorithm is often applicable
only when a dimension reduction step is used a priori. Then,
the idea is to exploit a robust sampling strategy to eliminate
the dimensionality reduction step and to estimate the amount
of data X T X (X is the data matrix) which is the most
expensive step in terms of calculation.
The outline of this paper is as follows. Section 2 reviews
the subspace clustering problem and the sparse representa-
tion formulation. Section 3 discusses the related works. Sec-
tion 4 introduces the proposed method. Section 5 presents
the experimental results and the final section concludes the
paper.

978-1-5090-0217-7/15/$31.00 2015 IEEE 2

 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
2. BACKGROUND
2.1. Subspace Clustering Problem
Clustering is required to discover groups in datasets. A
lot of methods have been done in this area [1]. However,
the most of real-world problems are high-dimensional ones.
In such situation, traditional clustering algorithms tend to
fail. Subspace clustering [2] presents an evolving technique
to cluster high dimensional data.
Formally, Given N data points {yi }N
i=1 that lie in the union
of l subspaces {Sj }lj=1 and we denote the matrix containing
all the data points as:
Y = [y1 y2 ... yN ] <dN
The goal of subspace clustering is to find clusters in different
underlying subspaces within the high dimensional dataset,
instead of finding groups in the entire original space. More
precisely, subspace clustering aims to identify the number of
subspaces l embedded in the dataset, their dimensions, a ba-
sis for each subspace and the segmentation of the data points
according to their subspaces. Subspace clustering, has found
numerous applications including gene expression analysis
[19], systems identification [20] and computer vision [3]
[4] [5].
2.2. Sparse Representation Technique
Data-sparse representation [16] has received a great deal
of attentions over the past decade. The term Sparsity
implies the existence of many zeros in a vector or a ma-
trix. Therefore, sparse representation aims to discover the
most compact representation of a data vector y in terms of
linear combination of a few typical data vectors over an
overcomplete dictionary such that y = c where c is a
sparse coefficient vector.
More formally, sparse solution means to solve the following
non-convex program:
min k c k0 subject to y = c (1)
Where k . k0 denotes the l0 norm, which counts the number
of non-zero entries in c. The aforementioned problem is NP-
hard and numerically unstable [21]. Recent developments in
sparse representation theory reveal that this problem can be
replaced by its l1 -convex relaxation [22] [23]:
min k c k1 subject to y = c (2)
To deal with noisy data, the previous problem is generally
described as follows:
min k c k1 s.t k y c k2 (3)
Where 0 is the error tolerance.
The sparse representation technique was successfully ap-
plied in several machine learning fields specially, image
restoration [24], compressed sensing [25] and face clustering
[26].
3. RELATED WORKS
3.1. Sparse Subspace Clustering (SSC)
Sparse Subspace Clustering (SSC) [15] is one of the
state-of-the-art algorithms proposed for subspace clustering.
SSC can be considered as a spectral clustering method which
the neighbor graph connectivity is constructed based on the
sparse representation.
Precisely, let the data matrix Y = [y1 y2 ...yN ] that lies in
the union of the l subspaces, the key idea behind SSC is
that data points are self-expressive in a union of subspaces.
That means, each data point yi can be expressed as a
linear combination of other data points that lie in the same
subspace i.e.:
yi = Y ci , cii = 0 (4)
where ci = [ci1 ci2 ... ciN ]T and cii = 0 eliminates the trivial
solution of writing a data point as a linear combination of
itself. The solution is not unique, SSC seeks for the solution
with the smallest k ci k1 and the optimizing problem is
written as:
min k ci k1 s.t. yi = Y ci and cii = 0 (5)
Expressed in the matrix form as:
min k C k1 s.t. Y = Y C and diag(C) = 0 (6)
Where C = [c1 c2 ... cN ] RN N the sparse coefficient
matrix. This solution is called the sparse subspace repre-
sentation which constructs the neighbor graph connectivity
of SSC. In fact, a similarity matrix is built after as :
W =| C | + | C |T where wij =| cij | + | cji | is
the edge weight between a node i and an other node j in
a graph representation. Data segmentation is obtained then
by applying a spectral clustering technique to the graph. To
deal with real word issues, where data is corrupted by noise,
the optimizing problem is modified as a LASSO problem is
solved:
z
min k C k1 + k Y Y C k2F s.t diag(C) = 0 (7)
2
Where z > 0 is a regularization term in the objective
function, k . kF designs the Frobenius norm chosen as the
appropriate norm to detect noise.
Applied to face clustering and motion segmentation
[15], SSC appears effective in identifying the underlying
subspaces even with corrupted and noisy data. However
the main drawback of SSC is its computational complexity
due to the solving of sparse optimization problem.
Algorithm 1 summerizes the SSC method:
3
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Algorithm 1 Sparse Subspace Clusterin (SSC)
Input: A data matrix Y = [y1 y2 ... yN ] that lies in the
union of l subspaces.
1) Solve the problem (6) for free data or (7) in the
case of noise corrupted data.
2) Normalize the columns of C as ci = kcicki .
3) Form the graph W with N nodes representing the
N data points such that every node i is connected
to other node j by an edge weight wij =| cij | + |
cji |.
4) Apply spectral clustering to the similarity graph W .
Output: The cluster membership of Y .
3.2. Scalable Sparse Subspace Clustering (SSSC)
Scalable SSC [17] has proposed to overcome the scala-
bility problem of SSC. The first step states to select a subset
of points from the original dataset named in-sample data and
proceeds by applying SSC algorithm on this set. The next
step assumes to assign the out-sample data to their natural
clusters. But, since high-dimensional data are usually not
lie into a distance measurement such as Euclidian distance,
so the goal is to find an appropriate relationship measure
between data points. Therefore, a regression strategy is
adopted to code the out-sample data X by the in-sample
data X . the final step is to assign the out-sample data
to the cluster with minimal residual calculated from the
relationship previously found. Exactly, the Scalable SSC
proceeds as follows:
Algorithm 2 Scalable SSC
Input: A data matrix Y = [y1 y2 ...yN ] , the number of
clusters k and a rigid regression parameter  = 106 .
1) Select randomly p data points from Y which pre-
sented the in-sample denoted X . The remaining set
of points is denoted X as the out-sample data.
2) Apply SSC over X .
3) Compute the relationship between each out-sample
data point xi by using X as:
ci = (X T X + I)1 X T xi (8)
4) Calculate the residuals of xi overall subspaces by
solving:
k xi Xj (ci ) k2
rj (xi ) = (9)
k j (ci ) k2
5) Assign xi to the subspace that has the minimal
residual by :
identity(xi ) = argminj {rj (xi )} (10)
Output: The cluster assignment of Y .
The nonzero entries j (ci ) Rn are the elements
related to X which associated with the j -th cluster and
identity(xi ) designs the label of xi .
Scalable SSC helps to reduce the time complexity since
it requires only O(t1 dp3 + t2 pk 2 + N p2 ) comparing to
SSC which requires O(t1 dN 3 + t2 N k 2 ) with p << N
and d the data dimension. We note also that t1 and t2
presents the number of iterations of ADMM solver and k-
means clustering respectively. However, Scalable SSC fails
to preserve the clustering performance yielded by SSC.
3.3. Latent Space Sparse Subspace Clustering
(LS3C)
It turns out that to control the high computational time
demanding of SSC, a dimensionality reduction process is
generally applied before. The motivation of Latent Space
Sparse Subspace Clustering (LS3C) [18] is to project data
points into a low-dimensional space and to find in the same
time, the sparse coefficients in this space. This methodology
is in order to eliminate the reduction step and to simultane-
ously accelerate the computational time.
Specifically, let Y = [y1 y2 ...yN ] <dN be the matrix
containing a collection of N data, which are drawn from a
union of l linear subspaces S1 S2 ... Sl and P <td
be a linear transformation matrix that projects data from the
input space Rd to a t-low-dimensional space. The goal of
LS3C is to optimize the following function:
[P , C ] = min J(P, C, Y ) (11)
P,C
subject to P P T = I, diag(C) = 0,
where
J(P, C, Y ) =k C k1 +1 k P Y P Y C k2F (12)
T
+2 k Y P P Y k2F
The first two terms of J present the sparse research
problem. The other term is a PCA-like regularization term
that insures a non-significant information loss between the
two spaces. 1 and 2 are two non-negative regulariza-
tion parameters. Under the assumption that there exists
an optimal solution P to the optimizing problem where
P = T X T with RN t , the cost function can be
written as follows:
J(, C, Y ) =k C k1 +1 k T K (I C) k2F (13)
T
+2 k Y (I K) k2F
s.t. T K = I, diag(C) = 0
Where K = Y Y T and as a result: P P T = T K = I .
This formulation leads to up-date P via and the algorithm
proceeds by optimizing the previous problem iteratively over
and C . For more details of this algorithm see [18]. LS3C
shows good clustering performance comparing to Scalable
SSC. Unfortunately, LS3C fails to reduce the important
computational time of SSC and consequently, stills expen-
sive for use in applications requiring large datasets.
4
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

4. THE PROPOSED METHOD with this small change the time complexity of the entire
algorithm will be O(t1 sN 3 + t2 N k 2 + N d log(s)) where:
4.1. Preliminary
O(N d log(s)) to calculate Ye .
In this section, we describe the novel approach we O(t1 sN 3 ) for the ADMM solver.
proposed to reduce the SSC computational time. The entire O(t2 N k 2 ) is the cost of spectral clustering
idea behind our method is to approximate the amount of
data Y T Y . On the other hand, if Ye = Y T where Y is the SVD
It is known that the L1 minimization problem corresponding of Y and k the rank of Y , then with probability at least
to the formulation (7), is equivalent to a regression problem 1 ( + epk )
type Least Absolute Shrinkage and Selection Operator, more
known as Lasso regression. In this context, several solvers
have been proposed to solve the L1 regularization problem. (1 )G G
e (1 + )G (16)
An example would be the Alternating Direction Method
of Multipliers (ADMM) [27]. This solver is based on the
q
computing of the amount of data Y T Y well-known by the Where = klog(2k/)
s .
Gram matrix. For more details about the proofs of SRHT theory, see
Let us consider the data matrix Y RdN such as each [28],[29].
column is the j-th vector data of d-dimension. The Gram
matrix can be written as: So, we define the SSC-SRHT algorithm as follows:

G = Y T Y RN N (14)
Algorithm 3 SSC-SRHT
Thus, to get the sparse coefficients matrix, we need to calcu-
late the inverse of this matrix. The fact to calculate the Gram Input: a data matrix Y = [y1 y2 ...yN ] that lies in the union
matrix and its inverse cost O(dN 3 ) which slow down the of l subspaces.
execution time of the entire algorithm. Therefore, we have 1) Compute the SRHT of the data
the idea to approximate the matrix G i.e to get a matrix G e
as a way to speed up and robustify the SSC algorithm. This Y = Y T (17)
can be made by the Subsampled Randomized Hadamard 2) Compute
Transform or SRHT. e = Ye T Ye
G (18)

4.2. Subsampled Randomized Hadamard Trans- 3) Solve the ADMM problem using the amount of
data G.
form
4) Normalize the columns of C as ci = kcicki .
5) Form the graph W with N nodes representing the
Recently, Subsampled Randomized Hadamard Trans-
N data points such that every node i is connected
form (SRHT) has received particular attention [28][29].
to other node j by an edge weight :
For d = 2p where p is any positive integer, a SRHT is a
(s d) random projection matrix that can be defined as: wij =| cij | + | cji | (19)
p
= d/sSHD (15) 6) Apply spectral clustering to the similarity graph W .
S Rsd is a subsampling matrix. Output: The cluster membership of Y .
H Rdd is a normalized Walsh Hadamard matrix
which is defined recursively as:
H = d1/2 Hd with 5. EXPERIMENTAL RESULTS
   
Hd/2 Hd/2 +1 +1 In this section, we evaluate the results of the proposed
Hd = and H2 =
Hd/2 Hd/2 +1 1 method on publicly available datasets, which are the Ex-
tended Yale B dataset [30] for face clustering and the
D Rdd is a diagonal matrix which each diagonal Hopkins155 dataset [31] for motion segmentation. These
element is a i.i.d. Rademacher random variable. databases are commonly used in testing machine learning
algorithms especially subspace clustering methods.
There are two important properties of this transform. The performance of our algorithm is compared with previ-
First, due to the recursive structure of the matrix H , the ap- ously presented algorithms (SSC, Scalable SSC and LS3C).
proximate Gram matrix G e will take only O(N 2 s) comparing The comparison is done in terms of the computational times
2
with O(N d) in the normal case where s << d. Therefore, and clustering error.

5
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

5.1. Face Clustering Algorithms SSC SSSC LS3C SSC-SRHT

2 subjects
Face clustering refers to the problem of clustering face
Mean 2.035 3.45 2.36 1.04
images of multiple individuals acquired under fixed pose
and varying illumination conditions. According to the Lam- Median 0.78 2.86 1.10 0.703
bertian assumption [32], the set of face images taken under 3 subjects
all illumination conditions of a single object lie close to Mean 3.77 4.23 3.51 1.79
a 9-D linear subspace. Therefore, the set of face images
Median 2.60 4.03 2.40 1.73
of multiple subjects can be considered as a union of 9-
dimensional subspaces. 5 subjects
Mean 6.79 7.74 6.08 3.64
Median 5.31 7.19 4.93 2.25
8 subjects
Mean 10.28 15.56 9.85 5.86
Median 9.57 15.90 9.74 5.66
10 subjects
Mean 11.46 22.36 11.47 10.12
Figure 1: a Sample from the Extended Yale B Dataset. Median 11.09 23.45 11.00 9.53

TABLE 1: Clustering error on the Extended Yale B database.

We evaluate the clustering performance of our method
as well as the state-of-the-art methods on the Extended
Yale B dataset which contains 192 168 pixel cropped 5.2. Motion Segmentation
face images of 38 subjects, each of which consists of
64 frontal face images taken under varying illumination Motion segmentation is a fundamental pre-processing
conditions, see Figure 1. Face images are downsampled to step for different computer vision applications including
48 42 pixels in the purpose to reduce the computational tracking and surveillance. The main idea of motion
time of the algorithms.Then, we stacked them into 2016- segmentation is to classify a set of tracked feature points
dimensional column vectors. Since these images are into multiple groups corresponding to multiple objects on
corrupted by outliers, the sparse outlying entries variation the basis of their motions. see Figure 3.
of the optimizing problem is used with a value of e = 20e ,
for more details see [15].

Figure 3: Samples from the Hopkins 155 Database.

Under the affine camera model assumption, the trajec-

tories of tracked points belonging to the same object (or
motion) lie in a linear subspace of dimension at most four.
Figure 2: Average CPU times on the Extended Yale B Indeed, the relationship between the tracked feature points
database. and the corresponding 3-D coordinates of the points on the
object is :

TABLE I gives the clustering accuracy for different

x11 x21 ... xP 1

individuals, and Figure 2 depicts the runtimes for algorithms A1

X1 ... XP

y11 y21 ... yP 1
computing. Note that, the SSC algorithm is preceded by A2

.. .. .. .. Y1 ... YP
a dimension reduction step with PCA. We show that our . = .
. . .

. Z1 ... ZP
.

method yields to the best results compared to other state- x
1F x2F ... xP F
1 ... 1
of-the-art methods. AF
y1F y2F ... yP F
(20)

6
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Which can be simplified as: W = MS
where M R2F 4 is the motion matrix
and S R4N is called the stucture matrix.
Since rank(M ) 4 and rank(S) 4 so,
rank(W ) = rank(M S) min(rank(M ), rank(S)) 4.
Under the affine camera model and since the last row of S is
1, the feature points trajectories of a rigidly moving object,
lie in an affine subspace of R2F of dimension at most three.
Therefore, given N feature trajectories of n of multiple
rigid motions, these trajectories will lie in a union of n
low-dimensional subspaces in R2F .Thus, the problem
of motion segmentation becomes a problem of subspace
clustering. We evaluate our algorithm for motion on the
standard benchmark Hopkins 155 dataset. This database
contains 120 two motions video sequences and 35 three
video sequences which present checkerboard, traffic and
articulated sequences. The Hopkins 155 dataset is available
online at http://www.vision.jhu.edu/data/hopkins155.
TABLE 2 records the average and median
misclassification errors. Thus, we can see another time
that the method suggested gives the best results in term of
classification.
Algorithms SSC SSSC LS3C SSC-SRHT
[3] G.P. Teja and S. Ravi, Face recognition using subspaces techniques,
Recent Trends In Information Technology, pp. 103-107, 2012.
[4] Z. Li, J. Guo, C. L-F.Cheong and S.Z. Zhou. Perspective motion seg-
mentation via collaborative clustering, IEEE International Conference
on Computer Vision, pp.1369-13761, 2013.
[5] A. Boulemnadjel and F. Hachouf. An Improved algorithm for sub-
space clustering applied to image segmentation, International Con-
ference on Information Visualisation , pp. 297-301, 2012.
[6] T.E. Boult and L.G. Brown, Factorization-based segmentation of
motions, IEEE Workshop on Motion Understanding, pp. 179-186,
1991.
[7] J. Costeira and T. Kanade, A multibody factorization method for
independently moving objects, International Journal of Computer
Vision, vol. 29, no. 3, 1998.
[8] P. Agarwal and N. Mustafa, k-means projective clustering, Proceed-
ings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium
on Principles of database systems,vol. 4, pp. 155-165, 2004.
[9] P. S. Bradley and O. L. Mangasarian, k-plane clustering, Journal of
Global Optimization, vol. 16, no. 1, pp. 23-32, 2000.
[10] A. Gruber and Y. Weiss, Multibody factorization with uncertainty
and missing data using the EM algorithm, IEEE Conference on
Computer Vision and Pattern Recognition, vol. 1, pp. 707-714, 2004.
[11] Y. Sugaya and K. Kanatani, Geometric structure of degeneracy for
multi-body motion segmentation, Workshop on Statistical Methods in
Video Processing, vol. 3247, pp. 13-25, 2004.
[12] J. Yan and M. Pollefeys, A general framework for motion segmen-
tation: Independent, articulated, rigid, non-rigid, degenerate and non-
degenerate, European Conference on Computer Vision, vo. 3954, pp.
94-106, 2006.

2 Motions [13] T. Zhang, A. Szlam, Y. Wang, and G. Lerman, Hybrid linear model-
ing via local best-fit flats, International Journal of Computer Vision,
Mean 1.83 4.31 1.62 1.95 vol. 100, no. 3, pp. 217-240, 2012.
Median 0.00 0.54 0.00 0.00 [14] G. Liu. , Z. Lin , S. Yan,J. Sun, Y. Yu and Y. Ma, Robust Recovery of
Subspace Structures by Low-Rank Representation, IEEE Transactions
3 Motions on Pattern Analysis and Machine Intelligence, vol. 35, no. 01, pp.171-
Mean 4.40 5.08 4.38 3.73 184, 2013.
Median 0.56 1.12 0.56 0.23 [15] E. Elhamifar and R. Vidal, Sparse Subspace Clustering: Algorithm,
Theory, and Applications, IEEE Transactions on Pattern Analysis and
All Motions Machine Intelligence, vol.35, no. 11, pp.2765-2781, 2013.
Mean 2.41 7.40 2.31 2.61 [16] J. Wright, M.Yi, J. Mairal, G. Sapiro,Sparse Representation for
computer vision and pattern recognition, Proceedings of the IEEE,
Median 0.00 3.85 0.00 0.00 vol. 98 , no.6 , pp. 1031-1044, 2010.
TABLE 2: Clustering error on the Hopkins 155 database. [17] P. Xi , Z. Lei and Y. Zhang. scalable sparse subspace clustering,
IEEE Conference on Computer Vision and Pattern Recognition, pp.
430-437, 2013 .
[18] V. M. Patel, H. V. Nguyen and R. Vidal, Latent space sparse subspace
6. CONCLUSION clustering, IEEE Journal of Selected Topics in Signal Processing, vol.
9, no. 4, pp. 691-701, 2015.
In this paper, we proposed a novel sparse subspace [19] H.-P. Kriegel, P. Kroger, and A. Zimek. Clustering high-dimensional
clustering method that is based on the Subsampled Ran- data: A survey on subspace clustering, pattern-based clustering, and
domized Hadamard Transform. Our method outperforms correlation clustering, ACM Transactions on Knowledge Discovery
from Data , vol.3, no. 1, 2009.
when compared with other state-of -the art sparse subspace
clustering methods. The results are very encouraging and [20] R. Vidal, S. Soatto, Y. Ma, and S. Sastry. An algebraic geometric
approach to the identification of a class of linear hybrid systems,
the proposed method can deal with large scale clustering IEEE Conference on Decision and Control, vol. 1, pp. 167-172, 2003.
problems.
[21] B.K. Natarajan, Sparse approximate solutions to linear systems,
SIAM Journal on Computing, vol. 24, no. 2, pp. 227-234, 1995.
References [22] S.S. Chen, D.L. Donoho and M.A. Saunders, Atomic decomposition
by basis pursuit, SIAM Journal Review, vol. 43, no. 1, pp. 129-159,
[1] A. K. Jain and S. Maheswari, Survey of recent clustering techniques 2001.
in data mining, International Archive of Applied Sciences and Tech- [23] D.L. Donoho, For most large underdetermined systems of linear
nology,vol. 3 , no. 2, pp. 68-75, 2012. equations the minimal l1-norm solution is also the sparsest solution,
[2] R. Vidal, Subspace clustering, IEEE Signal Processing Magazine, Communications on Pure and Applied Mathematics, vol.59, no. 7,
vol. 28, no. 3, pp. 52-68, 2011. pp.907-934, 2006.

7
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

[24] J. Mairal, M. Elad and G. Sapiro. Sparse representation for color

image restoration, IEEE Transactions on Image Processing, vol. 17,
no. 1, pp. 53-69, 2008.
[25] E. Candes.Compressive sampling, Proceedings of the International
Congress of Mathematicians, vol. 3, pp. 1433-1452, 2006.
[26] J. Wright, A.Y. Yang, A. Ganesh and S.S.Sastry, Robust Face
Recognition via Sparse Representation, IEEE Transactions on Pattern
Analysis and Machine Intelligence vol. 31, no. 2, pp. 210-227, 2008.
[27] S. Boyd, N. Parikh, E. Chu, B. Peleato and J. Eckstein, Dis-
tributed optimization and statistical learning via the alternating di-
rection method of multipliers, Foundations and Trends in Machine
Learning, vol. 3, no. 1, pp. 1-122, Jan. 2011.
[28] J.A. Tropp, Improved analysis of the subsampled randomized
Hadamard transform, Advances in Adaptive Data Analysis, vol.3, no.
1-2 , pp. 115-126, 2011.
[29] C. Boutsidis and A. Gittens. Improved matrix algorithms via the
Subsampled Randomized Hadamard Transform, SIAM Journal on
Matrix Analysis and Applications, SIAM Journal on Matrix Analysis
and Applications, vol. 34, no. 3, pp. 1301-1340, 2013.
[30] K.C. Lee, J. Ho, and D. Kriegman, Acquiring linear subspaces for
face recognition under variable lighting, IEEE Transactions on Pattern
Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684-698, 2005.
[31] R. Tron and R. Vidal, A Benchmark for the Comparison of 3-
D Motion Segmentation Algorithms, IEEE Conference on Computer
Vision and Pattern Recognition, pp. 1-8, 2007.
[32] R. Basri and D. W. Jacobs, Lambertian reflectance and linear
subspaces, IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 25, no.2, pp. 218-233, 2003.

8
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Accuracy Improvement of WPBC Dataset-Based Breast Cancer Diagnosis

Magdy Abd-elghany Zeid, M. B. Abdelhalim and Gouda I. Salama

College of Computing and Information Technology
Arab Academy for Science Technology & Maritime Transport
Cairo, Egypt
e-mail: magdy_zeid83@yahoo.com, mbakr@ieee.org, dr_gouda80@yahoo.com

Abstract This paper discusses two different methods to
improve the resultant accuracy of Breast Cancer Diagnosis.
The methods were applied to Wisconsin Prognosis Breast
Cancer (WPBC) dataset.
The first method generates a reduced dataset feature vector by
deriving it from the available feature vector of WPBC dataset.
The second one generates a new dataset that combines the
Wisconsin Diagnosis Breast Cancer (WDBC) and WPBC
datasets and apply the different classifiers systems on the
generated dataset by using classification accuracy and Area
Under the Curve (AUC) metrics based on confusion matrix of
10-fold cross validation method. In addition, we introduce a
fusion at the classification level between these classifiers to get
the most suitable multi-classifier approach for each data set.
Reducing of WPBC dataset size gives better outcomes through
reducing the number of features.
The experiments were performed considering the following
classifiers individually (decision tree (J48), Multi-Layer
Perception (MLP), Naive Bayes (NB), Sequential Minimal
Optimization (SMO), and Instance Based for K-Nearest
neighbor (IBK)) along with their possible combinations.
The results were very promising, for example: The
Probabilistic Unification of Heterogeneous Recurrent
Malignancy (PUHRM) Dataset has up to 86.17 %
improvement in the accuracy. All experiments are conducted
in WEKA data mining tool.
Keywords: Breast cancer, Diagnosis, Prognosis, Feature
reduction, Classification, a priori algorithm, Expectation
Maximization (EM) algorithm.
I. INTRODUCTION
Breast cancer is a malignant tumor that develops when
cells in the breast tissue divide and grow without the normal
controls on cell death and cell division [1]. Breast cancer has
become a major cause of death among women in developed
countries [2], with an incidence rate more than twice that of
colorectal cancer and cervical cancer and about three times
that of lung cancer. However, breast cancer mortality
worldwide is just 25% greater than that of lung cancer in
women [3]. One of the most effective ways to reduce breast
cancer deaths is early detection. An accurate and reliable
diagnosis are required procedure on early diagnosis that in a
way offers physicians the ability to differentiate the benign
breast tumors from malignant ones with no need to the
surgical biopsy. Breast Cancer Diagnosis and Prognosis are
considered challenging medical applications to researchers.
Breast Cancer Diagnosis and Prognosis has been
revolutionized extensively by the use of machine learning
and data mining techniques. Breast Cancer Diagnosis
distinguishes benign from malignant breast lumps and Breast
Cancer Prognosis predicts when Breast Cancer is likely to
recur in patients that have had their cancers excised. one of
the highest fatality rate for women over the last decades is
breast cancer. By the end of the year 2008 in United States
alone 692,000 women are estimated to be diagnosed with
some type of cancer and 26% of these women will be
diagnosed with breast cancer. With 26% expected
occurrence rate, breast cancer has the highest percentage
among other cancer types [4].
The contribution of this paper is to reduce feature vector
from WPBC dataset by deriving a new feature vector
(Relative Standard Error (RSE)) from the available feature
vector. Then, manipulated Breast Cancer Wisconsin
Diagnostic and Prognostic datasets and created four datasets
with different classes and instances to improve the resultant
accuracy achieved through WPBC dataset: (the intersection
between WDBC and WPBC datasets, Probabilistic
Unification (PU) between WDBC and WPBC datasets,
Probabilistic Unification of Heterogeneous Malignancy
(PUHM) Datasets and Probabilistic Unification of
Heterogeneous Recurrent Malignancy (PUHRM) Datasets).
We perform a diverse set of experiments that include
combined successive binary classifications and multi-class
classifications using diverse classification algorithms. By
comparing the performance of multiple algorithms and
experimental settings, we try to find the best model system
that can predict both diagnosis and prognosis of breast
cancer. The above contribution is discussed in details in
section VII.
Of the many data mining algorithms that developed in
recent years for extraction of knowledge in database are
supervised learning algorithms. These algorithms are mostly
used for classification tasks. Basically, there are much
researchers on breast cancer prognosis. A principle factor
that determines the way of treatment is the prognosis, which
will immediately follow the disease diagnosis. Hence, many
of these researchers reflect good classification accuracy. In
[6], the performance criterions of supervised learning
classifiers, such as decision tree (J48), Multi-Layer
Perception (MLP), Naive Bayes (NB), Sequential Minimal
Optimization (SMO), and Instance Based for K-Nearest
neighbor (IBK) are compared, to find the best classifier in
WPBC dataset. The experimental results showed that the
fusion of MLP, J48, SMO and IBK classifier is superior to
the other classifiers when using it as a single; it obtains
accuracy of 77.31%. In [7], the performance of Support
Vector Machines (SVM) in WPBC dataset scores accuracy
of 76.3%. In [8], the neural network classifier is used on
WPBC dataset. It achieves accuracy of 70.725%. In [9], an

978-1-5090-0217-7/15/$31.00 2015 IEEE 9

 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
accuracy of 75.32% was obtained with the application
logistic model trees algorithm (LMT) with WPBC dataset.
The rest of this paper is organized as follows: Section 2
presents the preprocessing phase including details of all used
datasets. Feature Reduction is discussed in section 3. Data
Mining Classification Methods is discussed in details in
section 4. In section 5 Multi classifiers methodology is
proposed. Performance evaluations criteria is discussed in
section 6. Section 7 discusses Modification of the Breast
Cancer Wisconsin dataset. Section 8 reports the experimental
results and evaluation of the classification techniques.
Finally, Section 9 draws some conclusions of this paper and
highlights the main possible directions of future
enhancements.
II. DATASET DESCRIPTION
This paper deals with the breast cancer diagnosis and
prognosis problem using WDBC as well as WPBC datasets.
These datasets, that are publicly available [5], are used to
represent machine learning algorithm performance through
many studies. The above datasets include some
measurements selected according to (FNA) Fine Needle
Aspirate test. Here, the main function of diagnosis is to
provide the difference between malignant and benign breast
masses. If the patient is primarily diagnosed with breast
cancer, the malignant mass must be investigated. After
investigation, the expected disease course must be
predicated.
TABLE 1. WDBC/WPBC CELL NUCLEI CHARACTERISTICS/ATTRIBUTES.
Radius Mean of distances from center to points on the
perimeter
Texture Standard Deviation of gray-scale value
Perimeter Total distance between consecutive snake points
constitutes the nuclear perimeter
Area Measured by counting the number of pixels on the
interior of the snake and adding one-half of the
pixels on the perimeter
Smoothness Measuring the difference between the length of a
radial line and the mean length of the lines
surrounding it
Compactness Perimeter2 / area - 1:0
Concavity Severity of concave portions of the contour
Concave Points Number of concave portions of the contour
Symmetry Measuring the difference between lines
orthogonal to the major axis through the center of
the cell in both directions.
Fractal Dimension "coastline approximation" 1
However, prognostic prediction does not belong to the
classic learning paradigms of function approximation or to
simple classification problems. As a result of this, a patient
may be classified as a recur instance in case of observing
the disease, where there is no dominant point of considering
the patient a non-recur instance. The WDBC and WPBC
datasets are the result of efforts made at the University of
Wisconsin Hospital for the diagnosis and prognosis of breast
tumors solely based on the FNA test. Using a small gauge
needle is used in this test, which also involves fluid
extraction from a breast mass, and then a visual inspection of
the fluid came out under a microscope.
A. The Wisconsin diagnostic breast cancer dataset
The Wisconsin diagnostic breast cancer (WDBC)
dataset consists of 569 instances (357 benign and 212
malignant), where each represents FNA test measurements
for one diagnosis instance. For this dataset, each instance has
32 attributes, with the first 2 attributes corresponding to a
unique identification number and diagnosis status
(benign/malignant). The remaining 30 features are
computations for 10 real-valued features, along with their
mean, standard error and the mean of the three largest values
(worst value) for each cell nucleus, respectively. These 10
real values, which are depicted in Table 1, are determined
from a digitized image of a Fine Needle Aspirate (FNA)
from the breast tumor, describe characteristics of the cell
nuclei present in the image and are recorded with four
significant digits.
B. The Wisconsin prognostic breast cancer dataset
The Wisconsin prognostic breast cancer (WPBC) dataset
consists of 198 instances (151 non-recurrences and 47
recurrences), where each represents follow-up data for one
breast cancer case. These were consecutive in-patients at the
University of Wisconsin Hospital from the period of 1984 to
1995 and include only those cases exhibiting invasive breast
cancer and no evidence of distant metastases at the time of
diagnosis. Each instance has 34 attributes, with the first 2
attributes corresponding to a unique identification number
and prognosis status (recurrence/non-recurrence),
respectively. They follow the above-mentioned 30 features,
and the last 2 attributes are the diameter of the excised tumor
centimeters. and number of positive axillary lymph nodes
observed at the time of surgery.
III. FEATURE REDUCTION
In the following, we distinguish two classes of feature
reduction strategies: Feature Subset Selection (FS) and
Dimensionality Reduction (DR). The main idea of feature
subset selection is to remove redundant or irrelevant features
from the data set as they can lead to a reduction of the
classification accuracy or clustering quality and to an
unnecessary increase of computational cost [10]. The
advantage of FS is that no information about the importance
of single features is lost. With dimensionality reduction
techniques the size of the attribute space can often be
decreased strikingly without losing a lot of information of
the original attributes space. An important disadvantage of
DR is the fact that the linear combinations of the original
features are usually not interpretable and the information
about how much an original attribute contributes is often
lost.
A. Feature (Subset) Selection
Generally speaking, there are three types of feature
subset selection approaches: filters, wrappers, and embedded
10
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
approaches which perform the features selection process as
an integral part of a Machine Learning (ML) algorithm.
Filters are classifier agnostic pre-selection methods
which are independent of the later applied machine learning
algorithm. Besides some statistical filtering methods like
Fisher score [11] or Pearson correlation [12], information
gain, originally used to compute splitting criteria for decision
trees, is often used to find out how well each single feature
separates the given data set.
Wrappers are feedback methods which incorporate the ML
algorithm in the FS process, i.e., they rely on the
performance of a specific classifier to evaluate the quality of
a set of features. Wrapper methods search through the space
of feature subsets and calculate the estimated accuracy of a
single learning algorithm for each feature that can be added
to or removed from the feature subset. The feature space can
be searched with various strategies, e. g., forwards (i.e., by
adding attributes to an initially empty set of attributes) or
backwards (i. e., by starting with the full set and deleting
attributes one at a time). Usually an exhaustive search is too
expensive, and thus non-exhaustive, heuristic search
techniques like genetic algorithms, greedy stepwise, best first
or random search are often used [13]. On the other hand, the
wrapper method, used in this process, is computationally
more demanding, but takes dependencies of the feature
subset on the learning algorithm into account.
Dimensionality Reduction
B. Dimensionality Reduction
Dimensionality Reduction (DR) refers to algorithms and
techniques which creates new attributes as combinations of
the original attributes in order to reduce the dimensionality
of a data set [14]. The most important DR technique is the
Principal Component Analysis (PCA), which produces new
attributes as linear combinations of the original variables. In
contrast, the goal of a factor analysis [15] is to express the
original attributes as linear combinations of a small number
of hidden or latent attributes. The factor analysis searches for

 ,.., |
underlying (i e. hidden or latent) attributes that summarize a
 = (1)
 ,.., 
group of highly correlated attributes.
IV. DATA MINING CLASSIFICATION METHODS
The data mining consists of various methods. Different
methods serve different purposes, each method has its own
advantages and disadvantages. However, most data mining
methods commonly used for this review are of classification
category as the applied prediction techniques assign patients
to either a benign group that is non-cancerous or a
malignant group that is cancerous and generate rules for
the same. Hence, the breast cancer diagnostic problems are
basically in the scope of the widely discussed classification
problems. In data mining, classification is one of the most
important task. It maps the data in to predefined targets. It is
a supervised learning as targets are predefined. The aim of
the classification is to build a classifier based on some cases
with some attributes to describe the objects or one attribute
to describe the group of the objects. Then, the classifier is
used to predict the group attributes of new cases from the
domain based on the values of other attributes. The
commonly used methods for data mining classification tasks
can be classified into the following groups [16]:
Support vector machine (SVM) classifier, originally
developed by Vapnik [17]. It tries to find the tradeoff
between minimizing the training set error and maximizing
the margin, in order to achieve the best generalization ability
and remains resistant to over fitting. Additionally, one major
advantage of the SVM is the use of convex quadratic
programming, which provides only global minima hence
avoid being trapped in local minima. SVM is an algorithm
that attempts to find a linear separator between the data
points of two classes in multidimensional space. SVMs are
well suited to deal with interactions among features and
redundant features. Recently, Smola and Schlkopf [18]
[19], proposed an iterative algorithm, called Sequential
Minimal Optimization (SMO), for solving the regression
problem using SVM. This algorithm is an extension of the
SMO algorithm proposed by Platt [20] for SVM classifier
design. Computational speed and ease of implementation are
some of the noteworthy features of the SMO algorithm.
SMO is an approximate and fast method to SVM classifiers
[21]. SMO is a simple algorithm that can quickly solve the
SVM QP problem without any extra matrix storage and
without using numerical QP optimization steps at all.
Nave Bayes classifier [22] is a probabilistic classifier
based on the Bayes theorem, considering a strong (Naive)
independence assumption. Thus, a Naive Bayes classifier
considers that all attributes (features) independently
contribute to the probability of a certain decision. Taking
into account the nature of the underlying probability model,
the Naive Bayes classifier can be trained very efficiently in a
supervised learning setting, working much better in many
complex real-world situations, especially in the computer-
aided diagnosis than one might expect [23]. Because
independent variables are assumed, only the variances of the
variables for each class need to be determined and not the
entire covariance matrix.
PC|F , . . , F
Where P is the probability, C is the class variable and
F1.......Fn are Feature variables F1 through Fn The
denominator is independent of C.
The Multilayer Perceptron (MLP), the application of
neural network of the artificial intelligence world, in
healthcare is relatively new [24] [25] [26]. The goal of this
paper is to show how we can apply the data mining and
especially the neural network to the medical databases,
where this application can predict or classify the data with a
given accuracy. Multi-layer feed forward neural network,
such that multi-layer perceptron MLP is the structure of
network used in our study. As shown in figure 1. Each
network contains three layers, input, hidden and output layer.
The features of the dataset are the input of the networks. The
number of hidden neurons should be determined
experimentally. The output layer consists of two classes
11
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
corresponding to the two types of breast cancer (B for benign
and M for malignant).
FIG. 1: GENERAL ARCHITECTURE MLP [6]
K-Nearest Neighbor (KNN) classifiers are instance-
based or lazy learners [27]. It delays the process of modeling
the training data until it is needed to classify the test samples.
It can be used both for classification and prediction. The
training samples are described by n-dimensional
dimensional numeri
numeric
attributes. The training samples are stored in an nn-
dimensional space. When a test sample (unknown class
label) is given, the k-nearest
nearest neighbor classifier searches the
k training samples which are clsosest to the unknown
sample. Closeness is usually definedined in terms of Euclidean
distance. The Euclidean distance is between two points P
(p1, p2, ,pn) and Q (q1, q2, . ,qn) given by equation 1.
dP, Q = 
p  q  (2)
The advantages of KNN classifier is very too simple to
implement and easy to justify thee outcome of KNN.
Although KNN has this advantages, it has some
disadvantages such as: a) high Computation cost since it
needs to compute distance of each test instance to all training
samples b) requires large memory proportional to the size of
training set
et c) Low accuracy rate in multidimensional
datasets with irrelevant features d) there is no rule of thumb
to determine value of parameter K (number of nearest
neighbors).
Decision Trees (DTs) classifier is one of the widely
used classifiers in Data Mining. In this paper we use one
specific decision tree algorithm (C4.5)) that was designed by
Quinlan [29]. The algorithm made by WEKA project is
known as J48.. Decision trees provide the classification of the
instances by sorting them down the tree from the ro root to
some leaf node. An instance is classified by starting at the
root node of the tree, testing the attribute specified by this
node, then moving down the tree branch corresponding to
the value of the attribute in the given example. This process
is then repeated for the sub tree rooted at the new node. In
general, decision trees represent a disjunction of
conjunctions of constraints on the attribute values of
instances. Each path from the tree root to a leaf corresponds
to a conjunction of attribute tests and the tree itself to a
disjunction of these conjunctions [22] [28].
Fusion of classifiers is combining multiple classifiers to
get the best accuracy. It is a set of classifiers whose
individual predictions are combined in some way to classify
new examples. es. Integration should improve predictive
accuracy. In WEKA the class for combining classifiers is
called Vote. Different combinations of probability estimates
for classification are available.
V. MULTI CLASSIFIERS METHODOLOGY
MET
The study uses the feature vector file to achieve the best
accuracy of tumor instance. During the process, the
researcher uses a single classifier technique to check
accuracy. After achieving the highest accuracy, another
classifier iss added from a list of (N-1)
(N different classifiers in
a way to get the highest accuracy. In the next step, a third
classifier is added to the present two ones in order to get a
more accurate result. This methodology is applied until we
reach the (N) classifiers
fiers merged in an attempt to obtain the
best accuracy. Here the (N) classifiers are limited to the five
listed classifiers (NB, SMO, MLP, IBK, and J48). In the
final stage, single and fusion classifiers are checked
check to get
the highest possible accuracy as shown in Figure 2.
Test Sample feature vector file
NB SMO MLP IBK J48
Select the classifier with the highest accuracy and
call it the group of classifiers
Count = 1
Generate the complementary groups between the
available group and the other remaining classifiers
Select the complementary group of classifiers
with the highest accuracy
count
ount = count+1
No
count =N
Yes
Output complementary group of classifier with the
highest accuracy
FIG. 2: MULTI CLASSIFIERS METHODOLOGY
MET
12
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

VI. PERFORMANCE EVALUATION CRITERIA VII. MODIFICATION OF THE BREAST CANCER WISCONSIN
For each performed experiments, we obtain a confusion DATASET
matrix for the generated classification model. The confusion There are two different methods to improve the resultant
matrix [30] [6] is represented in table 3.3 in which rows accuracy achieved through WPBC dataset
represent the actual classes (benign and malignant) and The first one generates a reduced dataset feature vector
columns represent the predicated classes. by deriving a new feature vector from the available feature
TABLE 2: CONFUSION MATRIX vector of WPBC dataset and apply the single classifiers
system. Relative standard error (RSE) is used to reduce
Predicted features vector of WPBC dataset by dividing the standard
Benign Malignant error of the estimate by the mean and expressed as a
percentage, then multiplying that result by 100 as shown in
Benign TB FM equation 5.
Actual
Malignant FB TM +
RSE = * 100 (5)
,
Several standard terms have been defined for the 2 class where RSE= Relative Standard Error, S= samples standard
matrix: error and X = mean value of the sample dataset.
The accuracy (AC): is the proportion of the total The second one generates and evaluates a screening
number of predictions that were correct. It is determined algorithm that classifies the dataset into recurrent and non-
using the equation: recurrent (R, N) classes, benign, malignant, recurrent and
 ! non-recurrent (B,M,R,N) classes, benign and malignant
AC = (3)
 !  ! (B,M) classes and finally, benign, recurrent and non-
recurrent (B,R,N) classes. So, we used both the Breast
The Area Under the ROC (Receiver Operating Cancer Wisconsin Diagnostic dataset and the Breast Cancer
Characteristics) Curve, or simply AUC, has been Wisconsin Prognostic dataset from the UC Irvine Machine
traditionally used in medical diagnosis since the 1970s. It has Learning Repository [5]. The diagnostic dataset is to predict
recently been proposed as an alternative single-number whether it is benign or malignant and the prognostic dataset
measure for evaluating the predictive ability of learning is to predict whether it will recur or not (so all cases in the
algorithms. The (AUC) performance measure, which links Prognostic dataset are malignant instances). The attributes of
the hit rate to the false rate, has clearly become a traditional the two datasets are nearly the same (Table 1). So, we made
measuring tool of predictive modeling accuracy. a new datasets combining the two datasets as described in
Furthermore, it is an assessment of the probability that a table 3 and equations 6, 7, 7 and 9 where numerical
classifier may rank a randomly chosen positive sample examples are given to show the needed calculations.
higher than a randomly chosen negative sample. As a result
of this, the AUC is extensively thought to be a better
measure than the classification error rate, which is based on a No. of No. of No. of
single prior probability or KS statistic threshold. Formula for Dataset
Attributes Instances Classes
solving area under a curve, the calculus formula is intersection between WDBC and 2
S0  n0 n0 + 12 WPBC datasets
28 139
[R and N]
"=
A *
n0n1 Probabilistic Unification (PU)
4
between WDBC and WPBC 15 628
[R, N, B and M]
where n0 and n1 are the numbers of positive and negative datasets
examples respectively, and S0 = ri where ri is the rank of ith Probabilistic Unification of
2
heterogeneous Malignancy 13 198
positive example in the ranked list. (PUHM) Datasets
[B and M]
Probabilistic Unification of
K-fold cross validation is one way to improve over the 3
Heterogeneous Recurrent 10 555
holdout method. The data set is divided into k subsets, and [B, N and R]
Malignancy (PUHRM) Datasets
the holdout method is repeated k times. Each time, one of
the k subsets is used as the test set and the other k-1 subsets TABLE 3: DESCRIPTION OF THE GENERATED BREAST CANCER DATASETS
are put together to form a training set. Then the average error
across all k trials is computed. The advantage of this method The intersection between WDBC and WPBC datasets
is that it matters less how the data gets divided. Every data
WDBC WPBC = WDBC + WPBC  WDBC WPBC
point gets to be in a test set exactly once, and gets to be in a
= 569 + 198  569 198 
training set k-1 times. The variance of the resulting estimate
= 767  430 + 59 + 139
is reduced as k is increased. A variant of this method is to
= 767  628 
randomly divide the data into a test and training
= 139 (6)
set k different times. The advantage of doing this is that you
can independently choose how large each test set is and how
many trials you average over.

13
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

The Probabilistic Unification (PU) between WDBC and VIII. EXPERIMENTAL RESULTS
WPBC datasets To evaluate the proposed model, two experiments were
WDBC WPBC = WDBC + WPBC  WDBC WPBC performed.
= 569 + 198  569 198 
= 767  139  A. Experiment (1) using Reduction of WPBC dataset
= 628 (7) feature vector
This experiment is applied to study the effect of the
PUHM Dataset reduced dataset feature vector by deriving a new feature
Partial uni?icationBB, MD vector from the available feature vector of WPBC dataset.
= WDBC  M + WDBC  B WPBC Table 4 show the accuracies for the five classifiers (NB,
= 569  212 +  569  357 198  MLP, J48, SMO and IBK) based on 10-fold cross validation
= 357 +  212 198  as a test method. The results show the same best accuracy
= 357 +  271  using the WPBC dataset before reduction [6] shown in Table
= 628 (8) 4 but with lower number of features (24 features instead of
PUHRM Dataset 34 features). It means that the dataset compressed with a
Partial uni?icationBB, R, ND Compression Ratio = 34 / 24 = 1.417 which means less
= WDBC  M + WPBC  WDBC  M WPBC classification time with the same best accuracy.
= 569  212 + 198  569  212 198 
= 357 + 198   357 198 
TABLE 4: RESULTANT ACCURACY OF THE SELECTED FIVE CLASSIFIERS ON
= 357 + 198   0  THE REDUCED WPBC DATASET
= 555 (9)
Classifiers
NB MLP SMO IBK J48
In data mining, A priori algorithm is a classic algorithm
for learning association rules [31]. A priori is designed to Accuracy 55.6701% 61.8557% 77.31% 55.1546% 76.2887%
operate on databases containing transactions. Other
algorithms are designed for finding association rules in data In the case of combining two classifiers, table 5 show
having no transactions, or having no timestamps (DNA that a fusion between SMO (the highest single classifier
sequencing). The whole point of the algorithm (and data accuracy) and other classifiers. It could be noticed that the
mining, in general) is to extract useful information from resultant accuracy it the same in all cases (76.2887%).
large amounts of data. The algorithm aims to find the rules
which satisfy both a minimum support threshold and a TABLE 5 : RESULTANT ACCURACY OF COMBINING SMO WITH each
minimum confidence threshold (Strong Rules). OF THE OTHER CLASSIFIERS IN WPBC_RSE

Classifiers NB and MLP and J48 and IBK and

The expectation-maximization algorithm was introduced SMO SMO SMO SMO
by Dempster et al. (1977) [32], who also presented its main Accuracy 76.2887% 76.2887% 76.2887% 76.2887%
properties. Widely used for cases of missing data. In short,
the EM algorithm aims at finding maximum likelihood
estimation where the data are incomplete or the likelihood Accordingly, in the case of three classifiers fusion, the
function involves latent variables. The EM algorithm is used table 6 present that a fusion between SMO plus each one of
in tasks such as handwriting recognition, medical diagnosis classifiers for example J48 and different classifiers shows the
and machine translation. Informally, the EM algorithm starts same results of accuracy (76.2887%). The same techniques
with randomly assigning values to all the parameters to be in the fusion of four and five classifiers are applied and give
estimated. It then iterately alternates between two steps, the same accuracy results.
called the expectation step (i.e., the E-step) and the
maximization step (i.e., the M-step), respectively. In the E- TABLE 6: RESULTANT ACCURACY OF COMBINING SMO AND J48 WITH
step, it computes the expected likelihood for the complete EACH OF THE OTHER CLASSIFIERS IN WPBC_RSE
data (the so-called Q-function) where the expectation is
taken w.r.t. the computed conditional distribution of the Classifiers SMO, J48 and SMO, J48 and SMO, J48 and
latent variables (i.e., the hidden variables) given the NB MLP IBk
current settings of parameters and our observed (incomplete) Accuracy 76.2887% 76.2887% 76.2887%
data. In the M-step, it re-estimates all the parameters by
maximizing the Q-function. Once we have a new generation
of parameter values, we can repeat the E-step and another M-
step. This process continues until the likelihood converges,
i.e., reaching a local maxima.

14
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

B. Experiment (2) using combination of WDBC and WPBC TABLE 9: RESULTANT ACCURACY AND AUC OF THE SELECTED FIVE
CLASSIFIERS ON PARTIAL UNIFICATION (B AND M) BETWEEN WDBC AND
datasets WPBC DATASETS

1) Results of intersection between WDBC and WPBC Classifiers

NB MLP SMO IBK J48
datasets
Accuracy 94.45% 96.77% 97.29% 94.34% 95.04%
In this experiment, we manipulated the WDBC and
WPBC datasets and generated a new dataset from the AUC 98 99 97 93 95
intersection between the two datasets with two classes (R
and N) and it consists of 139 samples, each sample described 4) Outcomes of a PUHRM Dataset
by 28 features. We applied five single classifiers (NB, MLP, In this experiment, we manipulated the WDBC and
SMO, IBK and J48) individually. Cross validation of 10 WPBC datasets and generated a unified dataset with three
folds have been chosen as test method using WEKA data classes (B, R and N) it consist of 555 samples, each sample
mining tool kit. Table 7 shows the comparison of AUC and described by 10 features. We applied 5 single classifiers
accuracies for the five classifiers. SMO is more accurate than (NB, MLP, SMO, IBK and J48) individually. Cross
other classifiers (82.34%). validation of 10 folds have been chosen as test method using
WEKA data mining tool kit. Table 10 shows the comparison
of accuracies for the five classifiers. SMO is more accurate
TABLE 7: RESULTANT ACCURACY AND AUC OF THE SELECTED FIVE
CLASSIFIERS ON INTERSECTION BETWEEN WDBC AND WPBC DATASETS
than other classifiers (86.17 %).

Classifiers TABLE 10: RESULTANT ACCURACY AND AUC OF THE SELECTED FIVE
NB MLP SMO IBK J48 CLASSIFIERS ON PARTIAL UNIFICATION (B, N, AND R) BETWEEN WDBC
AND WPBC DATASETS
Accuracy 71.58% 78.01% 82.34% 71.09% 74.99%
AUC 73 77 67 64 66
Classifiers
NB MLP SMO IBK J48
2) Results of a PU between WDBC and WPBC Accuracy 78.02% 83.38% 86.17% 78.48% 82.88%
datasets AUC 95 95 91 89 90

In this experiment, we manipulated the WDBC and
WPBC datasets and generated a unified dataset with four
classes (R, N, B and M) it consist of 628 samples, each
sample described by 15 features. We applied 5 single
classifiers (NB, MLP, SMO, IBK and J48) individually.
Cross validation of 10 folds have been chosen as test method
using WEKA data mining tool kit. Table 8 shows the
comparison of accuracies for the five classifiers. SMO is
more accurate than other classifiers (76.4%).
TABLE 8: RESULTANT ACCURACY AND AUC OF THE SELECTED FIVE
CLASSIFIERS ON FULL UNIFICATION BETWEEN WDBC AND WPBC
DATASETS
Classifiers
NB MLP SMO IBK
Accuracy 66.36% 72.39% 76.4% 68.6%
AUC 80 81 79 60
3) Results of a PUHM Dataset
In this experiment, we manipulated the WDBC and
WPBC datasets and generated a unified dataset with two
classes (B and M) it consist of 628 samples, each sample
described by 13 features. We applied 5 single classifiers
(NB, MLP, SMO, IBK and J48) individually. Cross
validation of 10 folds have been chosen as test method using
WEKA data mining tool kit. Table 9 shows the comparison
of accuracies for the five classifiers. SMO is more accurate
than other classifiers (97.29%).
As mentioned in the above experiments, results shows a
clear distinction in results with a variety of percentages as a
result of reducing feature number of WPBC dataset
VIII. CONCLUSIONS
Most work done on the WPBC dataset produces
insufficient results, which motivates the researcher to tackle
such a problem to get satisfactory results. Reducing the
WPBC dataset features and generating new datasets with
new features are the outstanding contributions done here that
in a way improves results. Experimental results using
Reduction of WPBC dataset show the same accuracy using
the original WPBC dataset before reduction. But this reduces
the number of features used across the dataset from 34 to 24
J48 features. Experimental result, using combination of WDBC
and WPBC datasets, show that SMO classifier is superior to
71.6%
other classifiers for all generated datasets: The intersection
78 between WDBC and WPBC datasets presents (82.34%). The
Probabilistic Unification between WDBC and WPBC
datasets shows (76.4%). The Probabilistic Unification of
Heterogeneous Malignancy Datasets produces (97.29%).
Finally, the Probabilistic Unification of Heterogeneous
Recurrent Malignancy Datasets gives (86.17 %).
ACKNOWLEDGEMENT
We would like to thank to Dr. William H. Wolberg at the
University of Wisconsin for supporting us with the breast
cancer dataset which we have used in our experiments.

15
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

REFERENCES [22] http://www.d.umn.edu/~padhy005/Chapter5.html, last visited on

February, 25, 2011.
[1] Jerez-Aragones JM, Gomez-Ruiz JA, Ramos-Jimenez G, Munoz-
Perez J, Alba-Conejo E. A combined neural network and decision [23] S. Belciug, (2008) Bayesian classification vs. k-nearest neighbour
trees model for prognosis of breast cancer relapse. Artif Intell Med classification for the noninvasive hepatic cancer detection, Proc. 8th
2003;27:4563. International conference on Artificial Intelligence and Digital
Communications.
[2] Bothorel S., Meunier B.B., and Muller S.A., Fuzzy logic based
approach for semi logical analysis of micro calcification in [24] K.Hornik, M. Stinchcombe, and H. White, Universal approximators
mammographic images International Journal of Intelligent System, of an unknown mapping an its derivatives using multilayer
12 (1112): 819-848, 1997. feedforward networks, Neural Networks, vol.3, pp. 551-560, 1990.
[3] http://www.wnba.com/silverstars/community/breast_health_awarenes [25] M. Riedmiller, Advanced supervised learning in multi-layer
s.html, last accessed February 2011 perceptrons from back propagation to adaptive learning algorithms,
In Int. Journal of Computer Standards and Interfaces, Special Issue on
[4] American Cancer Society. Cancer Facts & Figures 2008 neural Networks (5), 1994.
http://www.cancer.org/downloads/STT/2008CAFFfinalsecured.pdf
(2008). [26] S. Haykin, Neural networks : A comprehensive foundation, New
York : Macmillan College Publishing company, 1994.
[5] UCI Machine Learning Repository [Internet]. UCI Machine Learning
Repository: 2010; [cited 2011 Dec 20]. Available from: [27] J. Han, and M. Kamber, Data Mining: Concepts and Techniques, San
http://archive.ics.uci.edu/ml. Francisco, Morgan Kauffmann Publishers, (2001).
[6] Salama. Gouda. I., Abdelhalim. M.B., and Zeid. Magdy. A., 2012: [28] Tom Mitchell M., "Machine Learning", Mc-Graw Hill
Science/Engineering/Math, 1997.
Experimental Comparison of Classifiers for Breast Cancer Diagnosis.
International Conference on Computer Engineering and Systems [29] Quinlan, 1993] :Quinlan, J.(1993) C4.5: programs for machine
(ICCES'2012), Cairo city, Egypt, 180-300,121. learning. San Francisco : Morgan Kaufmann Inc.
[7] Mu T. and Nandi A.K, 2008, BREAST CANCER DIAGNOSIS AND [30] Kohavi, R., & Provost, F. (1998). Glossary of terms. Editorial for the
PROGNOSIS USING DIFFERENT KERNEL-BASED Special Issue on Applications of Machine Learning and the
CLASSIFIERS. International Conference on Biomedical Electronics Knowledge Discovery Process, 30(23).
and Devices, BIOSIGNALS 2008, Funchal, Madeira, Portugal, [31] Rakesh Agrawal and Ramakrishnan Srikant Fast algorithms for
January 28-31, 2008, Volume 2. mining association rules in large databases. Proceedings of the 20th
[8] Vaibhav Narayan Chunekar, Hemant P. Ambulgekar (2009). International Conference on Very Large Data Bases, VLDB, pages
Approach of Neural Network to Diagnose Breast Cancer on three 487-499, Santiago, Chile, September 1994.
different Data Set. 2009 International Conference on Advances in [32] Dempster, A. P., Laird, N. M., & Robin, D. B. (1977).Maximum
Recent Technologies in Communication and Computing. likelihood from incomplete data via the EM algorithm (with
[9] Jin S., Won J., Lee H. and Choi H., 2011:Construction of an discussion). Journal of Royal Statistical Society B, 39, 138.
automated screening system to predict breast cancer diagnosis and
prognosis. International Conference on Internet (ICONI'2011), 5: 15
18 doi:10.1111/j.1755-9294.2012.01124.x.
[10] Avrim L. Blum and Pat Langley. Selection of relevant features and
examples in machine learning. Artificial Intelligence, 97:245271,
1997.
[11] Terrence Furey, Nello Cristianini, Nigel Duffyy, David W.
Bednarski, and David Hauessler. Support vector machine
classification and validation of cancer tissue samples using
microarray expression data. Bioinformatics, 16:906914, 2000.
[12] Koji Miyahara and Michael J. Pazzani. Collaborative filtering with
the simple Bayesian classifier. Pacific Rim International Conference
on Artificial Intelligence, pages 679689, 2000.
[13] Ron Kohavi and George H. John. Wrappers for feature subset
selection. Artificial Intelligence, 97(1-2):273324, 1997.
[14] Huan Liu and Hiroshi Motoda. Feature Selection for Knowledge
Discovery and Data Mining. Kluwer Academic Publishers, 1998.
[15] Richard L. Gorsuch. Factor Analysis. Lawrence Erlbaum, 2nd
edition, 1983.
[16] Han J. and Kamber M., Data Mining: Concepts and Techniques, 2nd
ed., San Francisco, Morgan Kauffmann Publishers,2001.
[17] Vapnik, V.N., The nature of statistical learning theory. 1995:Springer,
New York.
[18] A. J. Smola, Learning with Kernels, Ph.D. dissertation, GMD,
Birlinghoven, Germany, 1998.
[19] A. J. Smola and B. Schlkopf, A Tutorial on Support Vector
Regression, Royal Holloway College, London, U.K.,
NeuroCOLT Tech. Rep.TR 1998-030, 1998.
[20] J. C. Platt, Advances in Kernel Methods: Support Vector Machines,
B. Schlkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT
Press, Dec. 1998. Fast training of support vector machines using
sequential minimal optimization.
[21] J. Platt. Sequential Minimal Optimization. Technical Report MSR-
TR-98-14, Microsoft Research, 1998.

16
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Simple and Robust Design of A Neural-Based

Ensemble Classifier
Hassan Wahbi Hassan
College of Computing &
Information Technology,
AASTMT
Alexandria, Egypt
hassan_wahbi@yahoo.com
Amin Shoukry Mahmoud Farouk
Egypt-Japan University of Science College of Computing &
and Technology. New-Borg El- Information Technology,
Arab City, Alexandria, Egypt AASTMT
Amin.shoukry@ejust.edu.eg Alexandria, Egypt
m.farouk@aast.edu

Abstract-This paper proposes a simple, fast and robust
method for classifiers fusion (or ensemble design) using a single
hidden layer feedforward Neural Network. The parameters of the
hidden layer are chosen at random while those of the output
layer are optimally estimated using the pseudo-inverse technique.
Therefore, backpropagation (or gradient-descent like learning) is
not needed and instead one-shot learning is possible. Moreover,
the ensemble training error is controlled by the number of
neurons in the hidden layer. A case study on breast cancer
diagnosis, using the Wisconsin data set (from UCI data
repository) consisting of 569 samples with 30 attributes per
sample, is given in which 4 base classifiers (K-Nearest
Neighbors, Decision Tree, Support Vector Machine, feedforward
Neural Network) are fused using the proposed methodology.
Keywords-Classifiers, K-Nearest Neighbors, Decision Tree,
Support Vector Machine, feedforward Neural Network,
Ensemble, Fusion.
I. INTRODUCTION
Classification is a main task in any recognition system. It
is concerned with finding a decision rule that determines the
class of an unobserved (future) pattern from a limited set of
observed patterns. The observation phase is called
training/ learning. Learning has two different modes:
supervised learning (or learning using a teacher that teaches
the classifier the correct desired output) and unsupervised
learning (or learning without a teacher); also known as self-
organization. In this paper, we are mainly concerned with
supervised learning.
required decision rule by modifying their weights in a way
that minimizes an error (or objective) function.
The performance of any classifier is affected by several
factors including the size of the training data set, its
dimensionality, the number of classes to be discriminated
and their mutual separability. Ensemble methods have been
devised for reducing over-fitting and improving the
performance of individual classifiers by fusing their
decisions [1]. This paper proposes a simple, fast and robust
method for classifiers fusion (or ensemble design) using a
single hidden layer feedforward Neural Network (NN).
The paper is organized as follows. Section II presents a brief
review about ensemble classifiers. Section III introduces the
proposed ensemble design methodology. Section IV applies
the proposed methodology to design an ensemble classifier
for breast cancer diagnosis. Finally, the paper is concuded in
section V.
II. BACKGROUND
Ensembles are either homogeneous or heterogeneous.
Homogenous ensembles use a single, arbitrary learning
algorithm but manipulate training data to make it learn
multiple models (see Fig. 1 ).

Various statistical, syntactical and structural

classification algorithms have been proposed. In the
statistical approach, the decision rule is based on Bayes
rule (the a posteriori probability of each pattern class based Fig. 1- Ensembles Learning
on the detected features). In the syntactic approach, the
decision rule is based on parsing of the input string to
determine if it belongs to the grammar generating a given Data1 Data2 Datam and Learner1 = Learner2 =
class of patterns. In the structural approach, the decision rule Learnerm. Therefore, m models are fitted; using m
is based on graph matching. Neural Networks learn the bootstrap samples; and combined by averaging or voting.
This is known as bagging. Samples bootstrapping aims at

978-1-5090-0217-7/15/$31.00 2015 IEEE 17

 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
creating diversity in the training data while average/voting
aims at improving classification performance. Random
forests belong to homogeneous ensembles [2].
Heterogeneous ensembles (Committees) apply different
learning algorithms; to a single dataset; and combine their
individual predictions in some way.
Data1 = Data2 = = Datam and Learner1 Learner2
Learnerm. The predictions of the base-level learners/
classifiers together with the correct class values constitute a
meta-level dataset. Multi-level ensembles (also known as
stacking) can be considered as meta-learning (learning about
learning) [3].
Recent advances in ensemble design have studied
techniques that can make the errors committed by the base
classifiers uncorrelated [4, 5] as well as methods to deal
with high dimensional data; such as microarray gene
expressions data [6, 7].
The ensemble classifier, proposed in this paper, belongs to
the class of committees. It combines the decisions of four
base classifiers; whose characteristics are given in Table 1.
Table 1 - Main characteristics of the classifiers used in the
proposed ensemble.
Classifier Learning Algorithm
K-NN (K=3) Memory based learning
SVM Quadratic Optimization
Feed Forward Neural Back Propagation (Gradient
Network Descent)
Decision Tree Recursive - Greedy
The K-nearest neighbor (KNN) classifier algorithm is a non-
parametric technique that works by storing a subset of the
available observations, and classifying a new set of
unknown observations into classes based on similarity (or
distance) with the stored patterns. The KNN has been used
in statistical estimation problems and pattern recognition
field since late 60s [8].
Decision trees are a class of nonlinear classifiers. They
are multistage decision systems in which the feature space is
split into unique regions, corresponding to the classes, in a
sequential manner. Splitting is based on an attribute test that
optimizes certain criterion (e.g. Information Gain [9]). Both
Decision Trees and NNs aim at forming complex decision
boundaries in the feature space. In contrast to the sequential
nature of trees, neural networks utilize a set of soft decisions
in a parallel fashion [10].
Support Vector Machine (SVM) is a supervised binary
classification method that seeks an optimal hyperplane
(linear) that maximizes the margin between the classes. The
data points closest to the hyperplane are called support
vectors. The support vectors are the critical elements of the
training set.
SVM can be adapted to become a nonlinear classifier
through the use of nonlinear kernels. It can, also, function as
a multiclass classifier by combining several binary SVM
classifiers [11].
The ensemble combiner itself is a single hidden layer
feedforward Neural Network that is described in the next
section.
III. PROPOSED ENSEMBLE
The specific Neural Network- based ensemble works
according to the following theorem [12, 13]: A single-
hidden layer feed-forward network with at most N hidden
neurons (including bias) can learn N distinct input-output
pairs with zero error (It is possible to tolerate a certain
amount of error by letting the number of hidden neurons be
less than N). This remains true whether the activation
function of the hidden neurons is the signum (hard limit or
threshold) or sigmoid (logistic) functions. The activation
function of the output neuron(s) is (are) linear. The main
advantage of this kind of network is that the hidden layer
weights are chosen randomly between [-1.. 1] while the
output layer weights can be optimally estimated using the
pseudo-inverse solution of an over-determined set of linear
equations which is, also, the solution of the least-squares
error between the desired and actual outputs from the neural
network. Therefore, backpropagation (or gradient-descent
like learning) is not needed and one-shot (fast) learning can
be used instead. Therefore, the network enjoys both a simple
and fast design strategy. The design is robust because it is
based on a sound theoretical basis. Moreover, NNs are
model free, i.e, they do not impose any restrictions on the
statistical distribution of their input data.
IV. CASE STUDY
The ensemble proposed in section III is used for breast
cancer diagnosis, using the Wisconsin data set (from UCI
data repository). Four base classifiers (K-Nearest
Neighbors, Decision Tree, Support Vector Machine,
feedforward Neural Network) are fused using the proposed
methodology as shown in Fig. 2.
The data is permuted so that the training and testing sets
are selected at random each time the computation starts. The
neural network MATLAB toolbox has been used for
modeling and training of the network.
A. KNN Classficiation
The trained KNN Classifier produced 12 errors on the
training data set and 20 errors on the testing data set. The
confusion matrix on the testing data is as follows:
18
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

True Class 1 0
Malignant (= 1) 99. 7% 0.3%
Benign (= 0) 4.7% 95.3

C. Decision Tree Classficiation

The DT produced 4 errors on the training data set and 21
errors on the testing data set. The confusion matrix on the
testing data is as follows:

Assigned Class by the Classifier

True Class 1 0
Malignant (= 1) 94.8% 5.2%
Fig. 2- The Proposed Ensemble
Benign (= 0) 15.1% 84.9

Assigned Class by the Classifier

The confusion matrix for the whole data is as follows:
True Class 1 0
Malignant (= 1) 97.0 % 3.0%
Assigned Class by the Classifier
Benign (= 0) 17.2% 82.8
True Class 1 0
Malignant (= 1) 97.8% 2.2%
The confusion matrix for the whole data set is as
follows: Benign (= 0) 8.0% 92.0%
Assigned Class
True Class 1 0
Malignant (= 1) 98.0 % 2.0% The obtained regression tree is shown below:

Benign (= 0) 11.8% 88.2

B. SVM Classification
The trained SVM Classifier produced 3 errors on the
training data set and 8 errors on the testing data set. The
confusion matrix on the testing data is as follows:

Assigned Class by the Classifier

True Class 1 0
Malignant (= 1) 99.3% 0.7% Fig. 3- Decision Tree
Benign (= 0) 7.5% 92.5
D. Feed Forward NN Classficiation
A 30-10-1 feed forward network with sigmoid
The confusion matrix for the whole data is as follows: neurons in the hidden and output layers has been used.
Since the Mat lab toolbox allows the use of validation
data when training a NN, the data was divided into 60%
training, 20% validation and 20% testing. The trained
Assigned Class by the Classifier

19
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

NN Classifier produced 2 errors on the training data set

and 6 errors on the testing data set.

The confusion matrix on the testing data is as follows:

Assigned Class by the Classifier
True Class 1 0
Malignant (= 1) 99.3% 0.7%
Benign (= 0) 5.4% 94.6%

The confusion matrix for the whole data is as follows:

Assigned Class by the Classifier
True Class 1 0
Fig. 5- NN Training Performance
Malignant (= 1) 99.7 % 0.3%
Benign (= 0) 3.3% 96.7% The performance of the ensemble-based classifier
compared to the other base classifiers is summarized in the
tables 3 & 4 below:
The Receiver Operating Curve for the NN is shown below: TABLE 3: Performance of Ensemble Classifier against
the base classifiers
# of # of
Confusion Matrix
Classifier Training Testing
on Test Data
Errors Errors
97.0 3.0
KNN 12 20
17.2 82.8
99.3 0.7
SVM 3 8
7.50 92.5
99.3 0.7
NN 2 6
5.4 94.6
94.8 5.2
DT 4 21
15.1 84.9
99.3 0.7
Ensemble 2 5
4.3 95.7

TABLE 4: Performance against all the data

Total # of Confusion Matrix
Classifier
Errors on All Data
Fig. 4-ROC Curve 98.0 2.0
KNN 32
11.8 88.2
E. Neural Network-based Ensemble: 99.7 0.3
SVM 11
The adopted NN is a 4-11-1 feedforward network. The 4.70 95.3
hidden neurons have hyperbolic tangent activation functions 99.7 0.3
NN 8
while the output neuron is linear. This network has 67 3.3 96.7
degrees of freedom ((4+1bias)*11+(11+1bias)*1). The four 97.8 2.2
DT 25
inputs are fed from the KNN, SVM, NN, and DT classifiers. 8.00 92.0
The inputs as well as the output to/from this network are 99.7 0.3
Ensemble 7
adjusted to 1/ -1 corresponding to true/false cancer 2.8 97.2
diagnosis. Several random initializations of the weights of
the hidden layer are tried and the best neural network which
achieves the minimum sum of the squares of the errors is V. CONCLUSIONS
selected.
The mean Square Training Error versus the number of
epochs is shown below:

20
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
A methodology for designing a NN-based ensemble has
been presented. The designed ensemble offers the following
advantages:
i. The design is simple because it uses a single
hidden layer NN. The hidden layer neurons are non-
linear while the output neuron(s) are linear.
ii. The weights of the hidden layer neurons are chosen
at random while those of the output layer are obtained
by solving an overdetermined system of linear
equations.
iii. No gradient descent optimization technique is
needed.
iv. The training error is controlled by the number of
neurons in the hidden layer.
v. The proposed method produced better results than
that of the base classifiers when estimated on the whole
available data set.
For future research, we plan to test the robustness of the
proposed methodology using ensembles stacking and
using other data sets.
REFERENCES
[1] Michael P. Perrone and Leon N Cooper, When
Networks Disagree: Ensemble Methods for Hybrid
Neural Networks, October 1992.
[2] Breiman, L. Random forests. Machine Learning,
45(1), 532, 2001.
[3] Vilalta, R., & Drissi, Y. A perspective view and
survey of meta-learning. Artificial Intelligence Review,
18:2, 7795, 2002.
[4] H. Chen, X. Yao. Regularized negative correlation
learning for neural network ensembles, IEEE
Transactions on Neural Networks, 20 (12), pp. 1962
1979, 2009.
[5] H. Chen, X. Yao. Multiobjective neural network
ensembles based on regularized negative correlation
learning, IEEE Transactions on Knowledge and Data
Engineering, 22 (12) , pp. 17381751, 2010.
[6] Lausser, L., Buchholtz, M., & Kestler, H. A.
Boosting threshold classifiers for high-dimensional data
in functional genomics. In L. Prevost, S. Marinai,
Proceedings of the 3rd IAPR Workshop on Artificial
Neural Networks in Pattern Recognition, Paris, France,
pp. 147-156. 2008.
[7] L.I. Kuncheva, J.J. Rodriguez, C.O. Plumpton, D.E.
Linden, S.J. Johnston, Random subspace ensembles for
FMRI classification, IEEE Transaction on Medical
Imaging, 29 (2) (2010), pp. 531542.
[8] Cover, T.M., Hart, P.E. Nearest neighbor pattern
classification. IEEE Trans. Inform. Theory, IT-
13(1):2127, 1967.
[9] Quinlan, J. R. Induction of Decision Trees,
Machine Learning, vol. 1, issue 1, pp. 81-106, March
1986.
[10] Theodoridis S., Koutroumbas K. Pattern
recognition, 2nd edition, Elsevier, 2003.
[11] WANG, Lipo (Editor), Support Vector Machines:
Theory and Applications, Springer, 2005.
[12] Shih-Chi Huang and Yih-Fang Huang, Bounds
on the Number of Hidden Neurons in Multilayer
Perceptrons, IEEE Transactions on Neural Networks.
Vol. 2. No. I . January 1991, 47-55.
[13] Guang-Bin Huang and Haroon A. Babri, Upper
Bounds on the Number of Hidden Neurons in
Feedforward Networks with Arbitrary Bounded
Nonlinear Activation Functions, IEEE Transactions on
Neural Networks, Vol. 9, No. 1, January 1998, 224-22.
21
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Simple and Robust Design of A Neural-Based

Ensemble Classifier
Hassan Wahbi Hassan
College of Computing &
Information Technology,
AASTMT
Alexandria, Egypt
hassan_wahbi@yahoo.com
Amin Shoukry Mahmoud Farouk
Egypt-Japan University of Science College of Computing &
and Technology. New-Borg El- Information Technology,
Arab City, Alexandria, Egypt AASTMT
Amin.shoukry@ejust.edu.eg Alexandria, Egypt
m.farouk@aast.edu

Abstract-This paper proposes a simple, fast and robust
method for classifiers fusion (or ensemble design) using a single
hidden layer feedforward Neural Network. The parameters of the
hidden layer are chosen at random while those of the output
layer are optimally estimated using the pseudo-inverse technique.
Therefore, backpropagation (or gradient-descent like learning) is
not needed and instead one-shot learning is possible. Moreover,
the ensemble training error is controlled by the number of
neurons in the hidden layer. A case study on breast cancer
diagnosis, using the Wisconsin data set (from UCI data
repository) consisting of 569 samples with 30 attributes per
sample, is given in which 4 base classifiers (K-Nearest
Neighbors, Decision Tree, Support Vector Machine, feedforward
Neural Network) are fused using the proposed methodology.
Keywords-Classifiers, K-Nearest Neighbors, Decision Tree,
Support Vector Machine, feedforward Neural Network,
Ensemble, Fusion.
I. INTRODUCTION
Classification is a main task in any recognition system. It
is concerned with finding a decision rule that determines the
class of an unobserved (future) pattern from a limited set of
observed patterns. The observation phase is called
training/ learning. Learning has two different modes:
supervised learning (or learning using a teacher that teaches
the classifier the correct desired output) and unsupervised
learning (or learning without a teacher); also known as self-
organization. In this paper, we are mainly concerned with
supervised learning.
required decision rule by modifying their weights in a way
that minimizes an error (or objective) function.
The performance of any classifier is affected by several
factors including the size of the training data set, its
dimensionality, the number of classes to be discriminated
and their mutual separability. Ensemble methods have been
devised for reducing over-fitting and improving the
performance of individual classifiers by fusing their
decisions [1]. This paper proposes a simple, fast and robust
method for classifiers fusion (or ensemble design) using a
single hidden layer feedforward Neural Network (NN).
The paper is organized as follows. Section II presents a brief
review about ensemble classifiers. Section III introduces the
proposed ensemble design methodology. Section IV applies
the proposed methodology to design an ensemble classifier
for breast cancer diagnosis. Finally, the paper is concuded in
section V.
II. BACKGROUND
Ensembles are either homogeneous or heterogeneous.
Homogenous ensembles use a single, arbitrary learning
algorithm but manipulate training data to make it learn
multiple models (see Fig. 1 ).

Various statistical, syntactical and structural

classification algorithms have been proposed. In the
statistical approach, the decision rule is based on Bayes
rule (the a posteriori probability of each pattern class based Fig. 1- Ensembles Learning
on the detected features). In the syntactic approach, the
decision rule is based on parsing of the input string to
determine if it belongs to the grammar generating a given Data1 Data2 Datam and Learner1 = Learner2 =
class of patterns. In the structural approach, the decision rule Learnerm. Therefore, m models are fitted; using m
is based on graph matching. Neural Networks learn the bootstrap samples; and combined by averaging or voting.
This is known as bagging. Samples bootstrapping aims at

978-1-5090-0217-7/15/$31.00 2015 IEEE 17

 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
creating diversity in the training data while average/voting
aims at improving classification performance. Random
forests belong to homogeneous ensembles [2].
Heterogeneous ensembles (Committees) apply different
learning algorithms; to a single dataset; and combine their
individual predictions in some way.
Data1 = Data2 = = Datam and Learner1 Learner2
Learnerm. The predictions of the base-level learners/
classifiers together with the correct class values constitute a
meta-level dataset. Multi-level ensembles (also known as
stacking) can be considered as meta-learning (learning about
learning) [3].
Recent advances in ensemble design have studied
techniques that can make the errors committed by the base
classifiers uncorrelated [4, 5] as well as methods to deal
with high dimensional data; such as microarray gene
expressions data [6, 7].
The ensemble classifier, proposed in this paper, belongs to
the class of committees. It combines the decisions of four
base classifiers; whose characteristics are given in Table 1.
Table 1 - Main characteristics of the classifiers used in the
proposed ensemble.
Classifier Learning Algorithm
K-NN (K=3) Memory based learning
SVM Quadratic Optimization
Feed Forward Neural Back Propagation (Gradient
Network Descent)
Decision Tree Recursive - Greedy
The K-nearest neighbor (KNN) classifier algorithm is a non-
parametric technique that works by storing a subset of the
available observations, and classifying a new set of
unknown observations into classes based on similarity (or
distance) with the stored patterns. The KNN has been used
in statistical estimation problems and pattern recognition
field since late 60s [8].
Decision trees are a class of nonlinear classifiers. They
are multistage decision systems in which the feature space is
split into unique regions, corresponding to the classes, in a
sequential manner. Splitting is based on an attribute test that
optimizes certain criterion (e.g. Information Gain [9]). Both
Decision Trees and NNs aim at forming complex decision
boundaries in the feature space. In contrast to the sequential
nature of trees, neural networks utilize a set of soft decisions
in a parallel fashion [10].
Support Vector Machine (SVM) is a supervised binary
classification method that seeks an optimal hyperplane
(linear) that maximizes the margin between the classes. The
data points closest to the hyperplane are called support
vectors. The support vectors are the critical elements of the
training set.
SVM can be adapted to become a nonlinear classifier
through the use of nonlinear kernels. It can, also, function as
a multiclass classifier by combining several binary SVM
classifiers [11].
The ensemble combiner itself is a single hidden layer
feedforward Neural Network that is described in the next
section.
III. PROPOSED ENSEMBLE
The specific Neural Network- based ensemble works
according to the following theorem [12, 13]: A single-
hidden layer feed-forward network with at most N hidden
neurons (including bias) can learn N distinct input-output
pairs with zero error (It is possible to tolerate a certain
amount of error by letting the number of hidden neurons be
less than N). This remains true whether the activation
function of the hidden neurons is the signum (hard limit or
threshold) or sigmoid (logistic) functions. The activation
function of the output neuron(s) is (are) linear. The main
advantage of this kind of network is that the hidden layer
weights are chosen randomly between [-1.. 1] while the
output layer weights can be optimally estimated using the
pseudo-inverse solution of an over-determined set of linear
equations which is, also, the solution of the least-squares
error between the desired and actual outputs from the neural
network. Therefore, backpropagation (or gradient-descent
like learning) is not needed and one-shot (fast) learning can
be used instead. Therefore, the network enjoys both a simple
and fast design strategy. The design is robust because it is
based on a sound theoretical basis. Moreover, NNs are
model free, i.e, they do not impose any restrictions on the
statistical distribution of their input data.
IV. CASE STUDY
The ensemble proposed in section III is used for breast
cancer diagnosis, using the Wisconsin data set (from UCI
data repository). Four base classifiers (K-Nearest
Neighbors, Decision Tree, Support Vector Machine,
feedforward Neural Network) are fused using the proposed
methodology as shown in Fig. 2.
The data is permuted so that the training and testing sets
are selected at random each time the computation starts. The
neural network MATLAB toolbox has been used for
modeling and training of the network.
A. KNN Classficiation
The trained KNN Classifier produced 12 errors on the
training data set and 20 errors on the testing data set. The
confusion matrix on the testing data is as follows:
18
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

True Class 1 0
Malignant (= 1) 99. 7% 0.3%
Benign (= 0) 4.7% 95.3

C. Decision Tree Classficiation

The DT produced 4 errors on the training data set and 21
errors on the testing data set. The confusion matrix on the
testing data is as follows:

Assigned Class by the Classifier

True Class 1 0
Malignant (= 1) 94.8% 5.2%
Fig. 2- The Proposed Ensemble
Benign (= 0) 15.1% 84.9

Assigned Class by the Classifier

The confusion matrix for the whole data is as follows:
True Class 1 0
Malignant (= 1) 97.0 % 3.0%
Assigned Class by the Classifier
Benign (= 0) 17.2% 82.8
True Class 1 0
Malignant (= 1) 97.8% 2.2%
The confusion matrix for the whole data set is as
follows: Benign (= 0) 8.0% 92.0%
Assigned Class
True Class 1 0
Malignant (= 1) 98.0 % 2.0% The obtained regression tree is shown below:

Benign (= 0) 11.8% 88.2

B. SVM Classification
The trained SVM Classifier produced 3 errors on the
training data set and 8 errors on the testing data set. The
confusion matrix on the testing data is as follows:

Assigned Class by the Classifier

True Class 1 0
Malignant (= 1) 99.3% 0.7% Fig. 3- Decision Tree
Benign (= 0) 7.5% 92.5
D. Feed Forward NN Classficiation
A 30-10-1 feed forward network with sigmoid
The confusion matrix for the whole data is as follows: neurons in the hidden and output layers has been used.
Since the Mat lab toolbox allows the use of validation
data when training a NN, the data was divided into 60%
training, 20% validation and 20% testing. The trained
Assigned Class by the Classifier

19
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

NN Classifier produced 2 errors on the training data set

and 6 errors on the testing data set.

The confusion matrix on the testing data is as follows:

Assigned Class by the Classifier
True Class 1 0
Malignant (= 1) 99.3% 0.7%
Benign (= 0) 5.4% 94.6%

The confusion matrix for the whole data is as follows:

Assigned Class by the Classifier
True Class 1 0
Fig. 5- NN Training Performance
Malignant (= 1) 99.7 % 0.3%
Benign (= 0) 3.3% 96.7% The performance of the ensemble-based classifier
compared to the other base classifiers is summarized in the
tables 3 & 4 below:
The Receiver Operating Curve for the NN is shown below: TABLE 3: Performance of Ensemble Classifier against
the base classifiers
# of # of
Confusion Matrix
Classifier Training Testing
on Test Data
Errors Errors
97.0 3.0
KNN 12 20
17.2 82.8
99.3 0.7
SVM 3 8
7.50 92.5
99.3 0.7
NN 2 6
5.4 94.6
94.8 5.2
DT 4 21
15.1 84.9
99.3 0.7
Ensemble 2 5
4.3 95.7

TABLE 4: Performance against all the data

Total # of Confusion Matrix
Classifier
Errors on All Data
Fig. 4-ROC Curve 98.0 2.0
KNN 32
11.8 88.2
E. Neural Network-based Ensemble: 99.7 0.3
SVM 11
The adopted NN is a 4-11-1 feedforward network. The 4.70 95.3
hidden neurons have hyperbolic tangent activation functions 99.7 0.3
NN 8
while the output neuron is linear. This network has 67 3.3 96.7
degrees of freedom ((4+1bias)*11+(11+1bias)*1). The four 97.8 2.2
DT 25
inputs are fed from the KNN, SVM, NN, and DT classifiers. 8.00 92.0
The inputs as well as the output to/from this network are 99.7 0.3
Ensemble 7
adjusted to 1/ -1 corresponding to true/false cancer 2.8 97.2
diagnosis. Several random initializations of the weights of
the hidden layer are tried and the best neural network which
achieves the minimum sum of the squares of the errors is V. CONCLUSIONS
selected.
The mean Square Training Error versus the number of
epochs is shown below:

20
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
A methodology for designing a NN-based ensemble has
been presented. The designed ensemble offers the following
advantages:
i. The design is simple because it uses a single
hidden layer NN. The hidden layer neurons are non-
linear while the output neuron(s) are linear.
ii. The weights of the hidden layer neurons are chosen
at random while those of the output layer are obtained
by solving an overdetermined system of linear
equations.
iii. No gradient descent optimization technique is
needed.
iv. The training error is controlled by the number of
neurons in the hidden layer.
v. The proposed method produced better results than
that of the base classifiers when estimated on the whole
available data set.
For future research, we plan to test the robustness of the
proposed methodology using ensembles stacking and
using other data sets.
REFERENCES
[1] Michael P. Perrone and Leon N Cooper, When
Networks Disagree: Ensemble Methods for Hybrid
Neural Networks, October 1992.
[2] Breiman, L. Random forests. Machine Learning,
45(1), 532, 2001.
[3] Vilalta, R., & Drissi, Y. A perspective view and
survey of meta-learning. Artificial Intelligence Review,
18:2, 7795, 2002.
[4] H. Chen, X. Yao. Regularized negative correlation
learning for neural network ensembles, IEEE
Transactions on Neural Networks, 20 (12), pp. 1962
1979, 2009.
[5] H. Chen, X. Yao. Multiobjective neural network
ensembles based on regularized negative correlation
learning, IEEE Transactions on Knowledge and Data
Engineering, 22 (12) , pp. 17381751, 2010.
[6] Lausser, L., Buchholtz, M., & Kestler, H. A.
Boosting threshold classifiers for high-dimensional data
in functional genomics. In L. Prevost, S. Marinai,
Proceedings of the 3rd IAPR Workshop on Artificial
Neural Networks in Pattern Recognition, Paris, France,
pp. 147-156. 2008.
[7] L.I. Kuncheva, J.J. Rodriguez, C.O. Plumpton, D.E.
Linden, S.J. Johnston, Random subspace ensembles for
FMRI classification, IEEE Transaction on Medical
Imaging, 29 (2) (2010), pp. 531542.
[8] Cover, T.M., Hart, P.E. Nearest neighbor pattern
classification. IEEE Trans. Inform. Theory, IT-
13(1):2127, 1967.
[9] Quinlan, J. R. Induction of Decision Trees,
Machine Learning, vol. 1, issue 1, pp. 81-106, March
1986.
[10] Theodoridis S., Koutroumbas K. Pattern
recognition, 2nd edition, Elsevier, 2003.
[11] WANG, Lipo (Editor), Support Vector Machines:
Theory and Applications, Springer, 2005.
[12] Shih-Chi Huang and Yih-Fang Huang, Bounds
on the Number of Hidden Neurons in Multilayer
Perceptrons, IEEE Transactions on Neural Networks.
Vol. 2. No. I . January 1991, 47-55.
[13] Guang-Bin Huang and Haroon A. Babri, Upper
Bounds on the Number of Hidden Neurons in
Feedforward Networks with Arbitrary Bounded
Nonlinear Activation Functions, IEEE Transactions on
Neural Networks, Vol. 9, No. 1, January 1998, 224-22.
21
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Simple and Robust Design of A Neural-Based

Ensemble Classifier
Hassan Wahbi Hassan
College of Computing &
Information Technology,
AASTMT
Alexandria, Egypt
hassan_wahbi@yahoo.com
Amin Shoukry Mahmoud Farouk
Egypt-Japan University of Science College of Computing &
and Technology. New-Borg El- Information Technology,
Arab City, Alexandria, Egypt AASTMT
Amin.shoukry@ejust.edu.eg Alexandria, Egypt
m.farouk@aast.edu

Abstract-This paper proposes a simple, fast and robust
method for classifiers fusion (or ensemble design) using a single
hidden layer feedforward Neural Network. The parameters of the
hidden layer are chosen at random while those of the output
layer are optimally estimated using the pseudo-inverse technique.
Therefore, backpropagation (or gradient-descent like learning) is
not needed and instead one-shot learning is possible. Moreover,
the ensemble training error is controlled by the number of
neurons in the hidden layer. A case study on breast cancer
diagnosis, using the Wisconsin data set (from UCI data
repository) consisting of 569 samples with 30 attributes per
sample, is given in which 4 base classifiers (K-Nearest
Neighbors, Decision Tree, Support Vector Machine, feedforward
Neural Network) are fused using the proposed methodology.
Keywords-Classifiers, K-Nearest Neighbors, Decision Tree,
Support Vector Machine, feedforward Neural Network,
Ensemble, Fusion.
I. INTRODUCTION
Classification is a main task in any recognition system. It
is concerned with finding a decision rule that determines the
class of an unobserved (future) pattern from a limited set of
observed patterns. The observation phase is called
training/ learning. Learning has two different modes:
supervised learning (or learning using a teacher that teaches
the classifier the correct desired output) and unsupervised
learning (or learning without a teacher); also known as self-
organization. In this paper, we are mainly concerned with
supervised learning.
required decision rule by modifying their weights in a way
that minimizes an error (or objective) function.
The performance of any classifier is affected by several
factors including the size of the training data set, its
dimensionality, the number of classes to be discriminated
and their mutual separability. Ensemble methods have been
devised for reducing over-fitting and improving the
performance of individual classifiers by fusing their
decisions [1]. This paper proposes a simple, fast and robust
method for classifiers fusion (or ensemble design) using a
single hidden layer feedforward Neural Network (NN).
The paper is organized as follows. Section II presents a brief
review about ensemble classifiers. Section III introduces the
proposed ensemble design methodology. Section IV applies
the proposed methodology to design an ensemble classifier
for breast cancer diagnosis. Finally, the paper is concuded in
section V.
II. BACKGROUND
Ensembles are either homogeneous or heterogeneous.
Homogenous ensembles use a single, arbitrary learning
algorithm but manipulate training data to make it learn
multiple models (see Fig. 1 ).

Various statistical, syntactical and structural

classification algorithms have been proposed. In the
statistical approach, the decision rule is based on Bayes
rule (the a posteriori probability of each pattern class based Fig. 1- Ensembles Learning
on the detected features). In the syntactic approach, the
decision rule is based on parsing of the input string to
determine if it belongs to the grammar generating a given Data1 Data2 Datam and Learner1 = Learner2 =
class of patterns. In the structural approach, the decision rule Learnerm. Therefore, m models are fitted; using m
is based on graph matching. Neural Networks learn the bootstrap samples; and combined by averaging or voting.
This is known as bagging. Samples bootstrapping aims at

978-1-5090-0217-7/15/$31.00 2015 IEEE 17

 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
creating diversity in the training data while average/voting
aims at improving classification performance. Random
forests belong to homogeneous ensembles [2].
Heterogeneous ensembles (Committees) apply different
learning algorithms; to a single dataset; and combine their
individual predictions in some way.
Data1 = Data2 = = Datam and Learner1 Learner2
Learnerm. The predictions of the base-level learners/
classifiers together with the correct class values constitute a
meta-level dataset. Multi-level ensembles (also known as
stacking) can be considered as meta-learning (learning about
learning) [3].
Recent advances in ensemble design have studied
techniques that can make the errors committed by the base
classifiers uncorrelated [4, 5] as well as methods to deal
with high dimensional data; such as microarray gene
expressions data [6, 7].
The ensemble classifier, proposed in this paper, belongs to
the class of committees. It combines the decisions of four
base classifiers; whose characteristics are given in Table 1.
Table 1 - Main characteristics of the classifiers used in the
proposed ensemble.
Classifier Learning Algorithm
K-NN (K=3) Memory based learning
SVM Quadratic Optimization
Feed Forward Neural Back Propagation (Gradient
Network Descent)
Decision Tree Recursive - Greedy
The K-nearest neighbor (KNN) classifier algorithm is a non-
parametric technique that works by storing a subset of the
available observations, and classifying a new set of
unknown observations into classes based on similarity (or
distance) with the stored patterns. The KNN has been used
in statistical estimation problems and pattern recognition
field since late 60s [8].
Decision trees are a class of nonlinear classifiers. They
are multistage decision systems in which the feature space is
split into unique regions, corresponding to the classes, in a
sequential manner. Splitting is based on an attribute test that
optimizes certain criterion (e.g. Information Gain [9]). Both
Decision Trees and NNs aim at forming complex decision
boundaries in the feature space. In contrast to the sequential
nature of trees, neural networks utilize a set of soft decisions
in a parallel fashion [10].
Support Vector Machine (SVM) is a supervised binary
classification method that seeks an optimal hyperplane
(linear) that maximizes the margin between the classes. The
data points closest to the hyperplane are called support
vectors. The support vectors are the critical elements of the
training set.
SVM can be adapted to become a nonlinear classifier
through the use of nonlinear kernels. It can, also, function as
a multiclass classifier by combining several binary SVM
classifiers [11].
The ensemble combiner itself is a single hidden layer
feedforward Neural Network that is described in the next
section.
III. PROPOSED ENSEMBLE
The specific Neural Network- based ensemble works
according to the following theorem [12, 13]: A single-
hidden layer feed-forward network with at most N hidden
neurons (including bias) can learn N distinct input-output
pairs with zero error (It is possible to tolerate a certain
amount of error by letting the number of hidden neurons be
less than N). This remains true whether the activation
function of the hidden neurons is the signum (hard limit or
threshold) or sigmoid (logistic) functions. The activation
function of the output neuron(s) is (are) linear. The main
advantage of this kind of network is that the hidden layer
weights are chosen randomly between [-1.. 1] while the
output layer weights can be optimally estimated using the
pseudo-inverse solution of an over-determined set of linear
equations which is, also, the solution of the least-squares
error between the desired and actual outputs from the neural
network. Therefore, backpropagation (or gradient-descent
like learning) is not needed and one-shot (fast) learning can
be used instead. Therefore, the network enjoys both a simple
and fast design strategy. The design is robust because it is
based on a sound theoretical basis. Moreover, NNs are
model free, i.e, they do not impose any restrictions on the
statistical distribution of their input data.
IV. CASE STUDY
The ensemble proposed in section III is used for breast
cancer diagnosis, using the Wisconsin data set (from UCI
data repository). Four base classifiers (K-Nearest
Neighbors, Decision Tree, Support Vector Machine,
feedforward Neural Network) are fused using the proposed
methodology as shown in Fig. 2.
The data is permuted so that the training and testing sets
are selected at random each time the computation starts. The
neural network MATLAB toolbox has been used for
modeling and training of the network.
A. KNN Classficiation
The trained KNN Classifier produced 12 errors on the
training data set and 20 errors on the testing data set. The
confusion matrix on the testing data is as follows:
18
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

True Class 1 0
Malignant (= 1) 99. 7% 0.3%
Benign (= 0) 4.7% 95.3

C. Decision Tree Classficiation

The DT produced 4 errors on the training data set and 21
errors on the testing data set. The confusion matrix on the
testing data is as follows:

Assigned Class by the Classifier

True Class 1 0
Malignant (= 1) 94.8% 5.2%
Fig. 2- The Proposed Ensemble
Benign (= 0) 15.1% 84.9

Assigned Class by the Classifier

The confusion matrix for the whole data is as follows:
True Class 1 0
Malignant (= 1) 97.0 % 3.0%
Assigned Class by the Classifier
Benign (= 0) 17.2% 82.8
True Class 1 0
Malignant (= 1) 97.8% 2.2%
The confusion matrix for the whole data set is as
follows: Benign (= 0) 8.0% 92.0%
Assigned Class
True Class 1 0
Malignant (= 1) 98.0 % 2.0% The obtained regression tree is shown below:

Benign (= 0) 11.8% 88.2

B. SVM Classification
The trained SVM Classifier produced 3 errors on the
training data set and 8 errors on the testing data set. The
confusion matrix on the testing data is as follows:

Assigned Class by the Classifier

True Class 1 0
Malignant (= 1) 99.3% 0.7% Fig. 3- Decision Tree
Benign (= 0) 7.5% 92.5
D. Feed Forward NN Classficiation
A 30-10-1 feed forward network with sigmoid
The confusion matrix for the whole data is as follows: neurons in the hidden and output layers has been used.
Since the Mat lab toolbox allows the use of validation
data when training a NN, the data was divided into 60%
training, 20% validation and 20% testing. The trained
Assigned Class by the Classifier

19
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

NN Classifier produced 2 errors on the training data set

and 6 errors on the testing data set.

The confusion matrix on the testing data is as follows:

Assigned Class by the Classifier
True Class 1 0
Malignant (= 1) 99.3% 0.7%
Benign (= 0) 5.4% 94.6%

The confusion matrix for the whole data is as follows:

Assigned Class by the Classifier
True Class 1 0
Fig. 5- NN Training Performance
Malignant (= 1) 99.7 % 0.3%
Benign (= 0) 3.3% 96.7% The performance of the ensemble-based classifier
compared to the other base classifiers is summarized in the
tables 3 & 4 below:
The Receiver Operating Curve for the NN is shown below: TABLE 3: Performance of Ensemble Classifier against
the base classifiers
# of # of
Confusion Matrix
Classifier Training Testing
on Test Data
Errors Errors
97.0 3.0
KNN 12 20
17.2 82.8
99.3 0.7
SVM 3 8
7.50 92.5
99.3 0.7
NN 2 6
5.4 94.6
94.8 5.2
DT 4 21
15.1 84.9
99.3 0.7
Ensemble 2 5
4.3 95.7

TABLE 4: Performance against all the data

Total # of Confusion Matrix
Classifier
Errors on All Data
Fig. 4-ROC Curve 98.0 2.0
KNN 32
11.8 88.2
E. Neural Network-based Ensemble: 99.7 0.3
SVM 11
The adopted NN is a 4-11-1 feedforward network. The 4.70 95.3
hidden neurons have hyperbolic tangent activation functions 99.7 0.3
NN 8
while the output neuron is linear. This network has 67 3.3 96.7
degrees of freedom ((4+1bias)*11+(11+1bias)*1). The four 97.8 2.2
DT 25
inputs are fed from the KNN, SVM, NN, and DT classifiers. 8.00 92.0
The inputs as well as the output to/from this network are 99.7 0.3
Ensemble 7
adjusted to 1/ -1 corresponding to true/false cancer 2.8 97.2
diagnosis. Several random initializations of the weights of
the hidden layer are tried and the best neural network which
achieves the minimum sum of the squares of the errors is V. CONCLUSIONS
selected.
The mean Square Training Error versus the number of
epochs is shown below:

20
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
A methodology for designing a NN-based ensemble has
been presented. The designed ensemble offers the following
advantages:
i. The design is simple because it uses a single
hidden layer NN. The hidden layer neurons are non-
linear while the output neuron(s) are linear.
ii. The weights of the hidden layer neurons are chosen
at random while those of the output layer are obtained
by solving an overdetermined system of linear
equations.
iii. No gradient descent optimization technique is
needed.
iv. The training error is controlled by the number of
neurons in the hidden layer.
v. The proposed method produced better results than
that of the base classifiers when estimated on the whole
available data set.
For future research, we plan to test the robustness of the
proposed methodology using ensembles stacking and
using other data sets.
REFERENCES
[1] Michael P. Perrone and Leon N Cooper, When
Networks Disagree: Ensemble Methods for Hybrid
Neural Networks, October 1992.
[2] Breiman, L. Random forests. Machine Learning,
45(1), 532, 2001.
[3] Vilalta, R., & Drissi, Y. A perspective view and
survey of meta-learning. Artificial Intelligence Review,
18:2, 7795, 2002.
[4] H. Chen, X. Yao. Regularized negative correlation
learning for neural network ensembles, IEEE
Transactions on Neural Networks, 20 (12), pp. 1962
1979, 2009.
[5] H. Chen, X. Yao. Multiobjective neural network
ensembles based on regularized negative correlation
learning, IEEE Transactions on Knowledge and Data
Engineering, 22 (12) , pp. 17381751, 2010.
[6] Lausser, L., Buchholtz, M., & Kestler, H. A.
Boosting threshold classifiers for high-dimensional data
in functional genomics. In L. Prevost, S. Marinai,
Proceedings of the 3rd IAPR Workshop on Artificial
Neural Networks in Pattern Recognition, Paris, France,
pp. 147-156. 2008.
[7] L.I. Kuncheva, J.J. Rodriguez, C.O. Plumpton, D.E.
Linden, S.J. Johnston, Random subspace ensembles for
FMRI classification, IEEE Transaction on Medical
Imaging, 29 (2) (2010), pp. 531542.
[8] Cover, T.M., Hart, P.E. Nearest neighbor pattern
classification. IEEE Trans. Inform. Theory, IT-
13(1):2127, 1967.
[9] Quinlan, J. R. Induction of Decision Trees,
Machine Learning, vol. 1, issue 1, pp. 81-106, March
1986.
[10] Theodoridis S., Koutroumbas K. Pattern
recognition, 2nd edition, Elsevier, 2003.
[11] WANG, Lipo (Editor), Support Vector Machines:
Theory and Applications, Springer, 2005.
[12] Shih-Chi Huang and Yih-Fang Huang, Bounds
on the Number of Hidden Neurons in Multilayer
Perceptrons, IEEE Transactions on Neural Networks.
Vol. 2. No. I . January 1991, 47-55.
[13] Guang-Bin Huang and Haroon A. Babri, Upper
Bounds on the Number of Hidden Neurons in
Feedforward Networks with Arbitrary Bounded
Nonlinear Activation Functions, IEEE Transactions on
Neural Networks, Vol. 9, No. 1, January 1998, 224-22.
21
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Framework for Activity Recognition Using Smart
Devices
Walid Mohamed Aly*, Khaled Eskaf**, Alyaa Aly*#
College of Computing and Information Technology Arab Academy for Science, Technology & Maritime Transport
* walid.ali@aast.edu
** keskaf@aast.edu
*# alyaa.aly@aast.edu
Abstract Activity recognition has become of great importance
in many fields especially in fitness monitoring; health and elder
care by offering the opportunity for large amount of applications
which recognize humans daily life activities. The prevalence of
smart phones in our society with their ever growing sensing
power has opened the door for more sophisticated data mining
applications which takes the raw sensor data as input and
classify the motion activity performed. The main sensor used in
performing activity recognition is the accelerometer. This paper
presents a framework for activity recognition using smart phone
sensors. Features extracted from raw sensor data are used to
train and test supervised machine learning algorithms.
Keywords Activity recognition, mobile sensor, accelerometer,
human activity, smart devices
I. INTRODUCTION
Activity recognition which predicts the user daily life
motion such as walking, jogging, climbing up stairs, climbing
down stairs, sitting and standing has grabbed a great attention
with the large amount of applications built upon this area.
With the tremendous growth of smartphone and the
incorporation of many diverse and powerful sensors, this area
has been highlighted to be a new rich point of research.
Recent smartphones are powered with various types of sensors
including light sensors, temperature sensors, direction sensors,
acceleration sensors, and gyroscope and barometer sensors.
The addition of these sensors has opened the door for many
applications to be built to help the activity recognition area
with the help of data mining techniques to predict the users
motion which turned the mobile phone to be smarter that it
could provide the appropriate action based on the users
activity. This paper highlights the importance of activity
recognition which could help in various fields and most
importantly with elder people who can face a sudden fall.
Using activity recognition in these cases can be of great help.
Various applications were developed based on motion sensors.
These applications receive raw sensor data as input and output
the prediction of the users activity.
Most researches in human activities recognition use several
accelerometers located at particular body sites and with
specific orientations [1][2], in our framework, we use single
tri-axial accelerometer imbedded in a smart device which is
978-1-5090-0217-7/15/$31.00 2015 IEEE
located in a fixed position in the front leg pocket of each
subject [3]. The organization of this paper is as follows:
Section II describes previous work in this area. In section III,
details about the proposed framework used for the activity
recognition system and its stages. Section IV describes
detailed information about components of human activity
recognition framework and the sensors used in measurements.
Section V includes the transformation methods applied on raw
data extracted from smart phone systems, the activities
performed in this paper and the output graphs for each activity.
Section VI presents analysis for the results. Section VII
includes conclusion for this paper.
II. RELATED WORK
The history of human activity starts in the late of 90's [4][5]
but there are still some challenges facing this research such as
the collection of data in real conditions, the proper feature to
be extracted from raw data in order to provide an accurate
recognition system as well as the selection of classification
techniques. Previous work in human activity recognition using
accelerometer sensors has shown that it is possible to detect
real time human activities. Maurer, Smailagic et al. used
single accelerometer to classify six different activities using
decision trees classifier [6]. This sensor was placed on six
different body positions where most smart devices are
commonly used the resulted accuracy varies from 16.7% to
92.8% depending on the position of the accelerometer and the
features used. Choudhury, S. Consolvo, et al. Focused their
study on placing smart phone in different places and evaluate
the effect of the sensor position [7]. They tested the
accelerometers on the individuals hip, wrist, arm, ankle, thigh,
and combinations of them. Their conclusions suggest that only
two accelerometers are sufficient to recognize simple daily
activities
Multiple kinds of sensors were used in other researches for
activity recognition. Maurer, A. Smailagic, et al. uses
accelerometers, temperature sensors and microphones for
recognizing human activities [6]. Current smart phone version
are equipped with these sensors in additional to light sensor,
proximity sensors, direction sensors(compass) which make
these smart devices able to recognise the current state of the
user.
34
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Previous work used commercial smart devices for data
collection. J. R. Kwapisz, G. M. Weiss, et al. use Android-
based smart phone for recognizing very simple activities such
as walk, jog, climb up and down the stairs [8]. J. Yang used
the Nokia N95 cell phone for distinguishing between different
actions [9]
III. STRUCTURE OF ACTIVITY RECOGNITION FRAMEWORK
The activity recognition systems pass through stages in
order to predict the human activity. Figure 1 illustrates these
stages. Data is collected using smartphone sensors such as
accelerometer. The collected data is divided into training and
testing data. During the training phase, subjects are used to
perform specific actions for a specific time series. Time series
are divided into time windows from which, features are
extracted for each individual activity. These features are the
input of the learning techniques (Classifiers) to generate a set
of rules for each activity in order to build a model which can
predict the subjects motion. During the testing phase, the
collected data is transformed into features which will be
evaluated through a pre-trained model in order to predict the
activity label
Raw Data
Collection
Data
Transformation
Features Generation
Time Domain Frequency Domain
Features Features
Classification Techniques
Activity Recognized
Figure 1. ACTIVITY RECOGNITION PROCESS
IV. HUMAN ACTIVITY FRAMEWORK COMPONENTS
While designing human activity system, many concerns
must be taken into consideration.
A. Sensors used
The new generation of smart phones is highly equipped
with a wide variety of sensors, these sensors had a great help
in the activity recognition process. Sensors are categorized
into tree main type; motion sensor, environmental sensor and
position sensor. Motion Sensors: measure the acceleration
force and rotational force along three axes X, Y and Z. This
type of sensor includes accelerometer sensors, Gyroscope,
gravity sensors and rotational vector sensors. The most
important sensor used in this framework is the accelerometer.
Accelerometer sensors sense the acceleration event of
smartphones. Accelerometer reads the values of three axes.
The directions of those axes are shown in the following figure.
The raw data stream from the accelerometer is the
acceleration of each axis in the units of g-force. Along with
the three axes reading, a time stamp is also recorded. Most of
existing accelerometers provide a user interface to configure
the sampling frequency so that the user could choose a best
sampling rate through experiments. Accelerometer has been
used heavily in smartphone sensors based activity recognition.
Its popularity is due to the fact that it directly measures the
subjects physiology motion status. For example, if a user
changes his/her activity from walking to jogging, it will reflect
on the signal shape of the acceleration reading along the
vertical axis there will be an abrupt change in the
amplitude. Moreover, the acceleration data could indicate the
motion pattern within a given time period, which is helpful in
the complex activity recognition.
Figure 2. 3-axis accelerometer
B. Data Collection
Data collection phase is one of the critical processes which
affects the accuracy of the activity recognition. The number of
trained subjects is also a crucial factor in any HAR study. In
this paper we have collected 20 volunteers with different
characteristics for each in terms of gender, age, weight and
height. These subjects were asked to do some specific
activities such as walking, jogging, sitting, standing, going
upstairs and going downstairs [10]. These subjects were
provided with android devices and they put it in their front leg
pocket. We have developed mobile software which measures
the three axis of acceleration for the subject holding it. This
software is designed to allow the users submit their
characteristics through a simple graphical user interface (GUI)
before performing any of the mentioned activities. The rate of
collecting accelerometer data was every 66ms which is around
15 readings per second.
35
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

4- Binned distribution: This is calculated by First,

determine the range of values for each axis, then
C. Data Pre-Processing
divide this range into 10 equal sized bins and
In order to recognize the activities performed, some calculate the fraction of the values falling within each
classifier algorithms can be applied on the accelerometer data.
of the bins
But these algorithms cannot be applied on the raw data
collected instead data transformation must be used to generate
features based on the readings. Data collected is divided into
10 sec intervals. By experiment, we have found that the 10 sec B. Activities Measured
is a sufficient period to capture the activity repetition.

Activities recognized by the sensors data can be classified

D. Device Position in different ways. For example, activities can be categorized
based on the complexity of activities (simple activity,
The Device position during the data collection process was
complex activity, location based activity, vision based
an interesting point of research in this field. Many
activity,). A simple human activity could be walking,
experiments were performed to determine the recognition
jogging, walking downstairs, taking elevator, etc. The
accuracy in both position specific and position independent
complex activities are usually related to a combination of a
recognition. Doruk Coskun et al. have collected activity data
longer period of activities (e.g., taking bus and driving). The
from 15 volunteers performing the primary activities (walking,
activities may only correspond to the movements of certain
running, upstairs, downstairs, sitting and standing) while
parts of the body (e.g., typing and waving hand). In this paper
carrying smart devices in three different positions, in a
we have concentrated the research on the simple activities and
backpack, in the hand and in the pocket. On average, the
subjects were asked to perform each activity individually.
accuracy was the same for position specific and position
independent but only in the pocket the accuracy increased by
In the following section, a graph for each activity will be
2%[3].
shown describing the values measured using accelerometer
sensor
V. ACTIVITY RECOGNITION METHODS
A. Feature Extraction
Time Domain Features
Time domain features contain the basic statistics of each data
segment:
1- Mean: This is the mean value of each segment in
each dimension
2- Max, Min: the min and max values of each
dimension
3- Standard deviation: The variance and standard
deviation of each segment
4- Correlation: Calculated between each pair of axes
data
5- Signal-Magnitude Area: this is the sum of the Figure.3 walking activity
magnitude of the three axes acceleration
6- Average Resultant Acceleration: This is the average
of the square root of the sum of the acceleration

Frequency Domain Features

These features measure the periodicity of the signal:
1- Energy: It is the sum of the squared discrete FFT
component magnitudes.
2- Entropy: It is the normalized information entropy of
the discrete component.
3- Time between peaks: This is the time between peaks
in the sinusoidal waves. Figure.4 jogging activity

36
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

VI. GRAPH ANALYSIS

As shown in the above figures, the activities performed

Figure. 5 Climbing Upstairs
show periodic behaviour described by the magnitude of the
accelerometer values. For the walking pattern we can analyse
that the Y axis measures high peak every around half a second
while Z axis show the same behaviour, the difference between
the magnitude of both Y and Z axis demonstrate the time of
each stride. The x axis measures the lowest magnitude. For the
jogging graph, we can notice a similar behaviour as the
walking but the time between peaks decreases reaching
quarter a second. In the jogging activity, the values of y axis
are greater than in walking. For the climbing downstairs
activity, we can notice small peaks for z axis with negative
values comparing it with the running and jogging which
represent moving down in every single step. In climbing up
stairs, the space between peaks for y and z axis is increasing
representing the longer time taken to climb up stairs. By
evaluating the standing and sitting results, we found that the
acceleration is almost constant

A previous research [8] which used a similar framework

for activity recognition; activities were classified using weka
Figure. 6 Climbing Downstairs
learning machine tool. Multiple classifiers were used
including: J48, Logistic Regression and Multilayer Perceptron.
A high accuracy was achieved in most cases. Table 1 below,
demonstrates the accuracy for each classifier. Accuracy over
90% was easily reached in the most common activities
(walking and jogging). Jogging activity recorded the highest
accuracy as it involves more acceleration changes than in
walking. Although sitting and standing do not exhibit any
change in acceleration, but they can be easily recognized
through the relative magnitude of values for each axis.
Climbing upstairs and downstairs are the most confusing
activities so their accuracy was low.

Table I. Accuracies for Activity Recognition

Figure. 7 Standing

% of records Correctly Predicted

J48 Logistic Multilayer
Regression Perceptron
Walking 89.9 93.6 91.7
Jogging 96.5 98.0 98.3
Upstairs 59.3 27.5 61.5
Downstairs 55.5 12.3 44.3
Sitting 95.7 92.2 95.0
Standing 93.3 87.0 91.9
Overall 85.1 78.1 91.7

Figure.8 Sitting
VII. CONCLUSIONS

37
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Activity recognition has become a focus research point [9] J. Yang, Toward physical activity diary: Motion recognition
due to increasing availability of accelerometers in smart using simple acceleration features with mobile phones, In
devices and due to the tremendous number of applications First International Workshop on Interactive Multimedia for
based on this kind of sensor. In this paper we have proposed a Consumer Electronics at ACM Multimedia, 2009.
system to measure the acceleration of 20 subjects using an
android smart phone placed in the front leg pocket. By [10] Naciye C elenli, Kamile Nur Sevis.: An Unconstrained
Activity Recognition Method Using Smart Phones. International
analyzing the resulted measures and after data transformation
Conference of the Biometrics Special Interest Group (BIOSIG)
we came up with 43 features which will be the input of many 2014
classifiers in order to detect the users motion and then taking
action based on the result. Weka data mining tool will be used
to generate model to predict the user activities classify the
motion of the user

REFERENCES

[1] Karantonis, D.M., Narayanan, M.R., Mathie, M., Lovell, N.H.,

Celler, B.G.: Implementation of a real-time human movement
classifier using a triaxial accelerometer for ambulatory monitoring.
IEEE Trans. on Information Tecnhology in Biomedicine 10(1) (2006)

[2] Song, Y., Shin, S., Kim, S., Lee, D., Lee, K.H.: Speed estimation
from a tri-axial accelerometer using neural networks. In: 29th annual
international conference of the IEEE EMBS (2007)

[3] Doruk C., Ozlem D., Atay O.: Phone Position/Placement

Detection using Accelerometer: Impact on Activity Recognition.
IEEE Tenth International Conference on Intelligent Sensors,
Sensor Networks and Information Processing (ISSNIP) 2015

[4] O. X. Schlmilch, B. Witzschel, M. Cantor, E. Kahl, R.

Mehmke,
and C. Runge, Detection of posture and motion by
accelerometry:
a validation study in ambulatory monitoring, Computers in
Human
Behavior, vol. 15, no. 5, pp. 571---583, 1999.

[5] F. Foerster, M. Smeja, and J. Fahrenberg, Detection of

posture and motion by accelerometry: a validation study in
ambulatory monitoring, Computers in Human Behavior, vol.
15, no. 5, pp. 571---583, 1999.

[6] U. Maurer, A. Smailagic, D. P. Siewiorek, and M. Deisher.

Activity recognition and monitoring using multiple sensors
on different body positions. In International Workshop on
Wearable and Implantable Body Sensor Networks , Apr 2006.

[7] T. Choudhury, S. Consolvo, B. Harrison, J. Hightower,

A.LaMarca, L. LeGrand, A. Rahimi, A. Rea, G. Bordello,
B. Hemingway, et al., The mobile sensing platform:An
embedded activity recognition system, Pervasive Computing,
IEEE, vol. 7, no. 2, pp. 32-41, 2008.

[8] J. R. Kwapisz, G. M. Weiss, and S. A. Moore, Activity

recognition using cell phone accelerometers. In Proceedings
of the Fourth International Workshop on Knowledge Discovery
from Sensor Data, Washington DC, pp10-18, 2010.

38
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Session II:
Computer Control and Industrial
Applications

Title: Page#

Effect of SLA Process Parameters on Part Build-Time 40

Mootaz Ghazy and Mohab Hossam

Modeling and Simulation of MDF Bagasse Fiber Drying 46

Gafary Mahmoud, Esam H. Abdelhameed, G. Shabib, and Mahmoud A. Sayed

New Tuning Approach of Discrete PI/PID Controller Applied to 52

Bitumen System Based on Non-Minimal State Space Formulation
E. M. Shaban, AR. Hamed, R. R. Darwish, and A. M. Abdel ghany

Mechanism Design with Computational Algebra 58

Samuli Piipponen and Eero Hyry

Modified Model Predictive Control for a Positioning Mechanism 64

with Harmonic Drive Gear
Esam. H. Abdelhameed, Tarek Hassan Mohamed, and Makoto Iwasaki

39
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Effect of SLA Process Parameters
on Part Build-Time
Mootaz Ghazy
Industrial & Management Engineering (IME)
College of Engineering and Technology
Arab Academy for Science, Technology and Maritime
Transport (AASTMT)
Alexandria, Egypt
Mootaz.ghazy@aast.edu
Abstract Additive manufacturing (AM) processes are
relatively new manufacturing methods that directly produce
physical parts from 3D model data by deposing a layer-upon-
layer of material. Parts produced by AM processes are largely
affected by the process parameters; however, the knowledge
needed to fully understand the effect of these parameters over the
part build-time is insufficient. This study aims to investigate the
effect of some parameters of the SLA process (one the AM
processes) such as the hatch overcure, the part orientation, and
the layer thickness on the part build-time. Design of experiments
(DOE) method was used to investigate and to understand the
relationships between these process parameters and the build-
time. The 23 full factorial design was applied and the build-time
was measured as the output response. The regression analysis
and Analysis of Variance (ANOVA) techniques were conducted
to the investigated SLA parameters as inputs and the part build-
time as output. The results showed that the most significant
factors for the part build-time are the main effect of the hatch
overcure, the layer thickness, and the part orientation,
consecutively.
KeywordsAdditive Manufacturing (AM); Stereolithography
(SLA); Process Parameters; Build-Time; Design of Experiments
(DOE); Regression Analysis
I. INTRODUCTION
Additive Manufacturing (AM) is the process of producing
3D physical parts directly from CAD model by adding layer
upon layer of material. The AM technologies permit the
production of complex three dimensional structures, the ability
to generate near-net-shaped parts, and the reduction of the
time and the cost of production. AM processe can be classified
according to the starting material into powder and wire
material processes. Every AM process possesses its
advantages and disadvantages. The most famous AM
978-1-5090-0217-7/15/$31.00 2015 IEEE
Mohab Hossam
Department of Industrial Engineering and Systems
Management (IEM)
School of Innovative Design Engineering (IDE)
Egypt-Japan University of Science and Technology
(E-JUST)
Alexandria, Egypt
mohab.hossam@ejust.edu.eg
On Leave from:
Production Engineering Department
Faculty of Engineering
Alexandria University
Alexandria, Egypt
processes are the Fused Deposition Modeling (FDM), the 3D
printing (3DP), the Selective Laser Sintering (SLS), and the
Stereolithography (SLA) processes. The Stereolithography
process (SLA) was the first additive manufacturing technique
to be introduced to the market [1] [2].
The SLA machine consists of a computer, a container of
the photosensitive resin, a moveable platform in which the
model is built, a laser beam for curing the photosensitive resin,
and a dynamic mirror system to direct the laser beam. The
computer digitally control the mirror in order to direct the
laser beam over the surface of the resin to draw one cross
section of the CAD model of the part. Following the
termination of a layer, the platform goes down in the resin
container to permit the appearance of a thin film of the resin to
be cured for the following cross section of the CAD model. A
waiting period is allocated to recoat the surface of the previous
layer before the new layer is drawn. The purpose of recoating
is to cover the model with resin when it has been moved down
by one layer thickness [3][4].
The parameters of the SLA process can be classified into
part parameters, support parameters, and recoat parameters.
The part parameters are the parameters that affect the accuracy
of the produced part, and thus the selection and the setting of
these parameters are very important. Fig. 1 shows different
parameters of the SLA process including the layer thickness,
the hatch spacing, the fill spacing, the hatch overcure, the
border overcure, and the fill cure depth [5].
Another SLA parameter that affect the quality of the
produced part is the optimal orientation of build-up, also
called part orientation, which is an important parameter
affecting not only the part surface quality, but also the need of
supporting structure, the part build-time, and, consequently,
the part cost [6] [7] [8].
40
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Fig. 1. Parameters of the SLA process [5]
The determination of the SLA process parameters to
produce the required objectives set by users is not an easy task
due to the fact that these objectives are conflicting in nature.
The selection of the process parameters is also called the
setting of the building strategies or the building scenarios.
Generally, these process parameters affect the characteristics
of the produced parts produced by the SLA machine such as
the dimensional accuracy, the surface finish, the tensile
strength, and the build-time.
Section II of the paper presents the conducted
experimental work including the design of the produced part,
the specifications of the SLA machine used for producing the
parts, the part material specifications, the post-processing of
the part, and the applied Design of Experiments technique.
In Section III the part build-time is introduced. Section IV
discusses the regression analysis applied to derive a
relationship between the part build-time as response and the
three main three SLA process parameters (hatch overcure, part
orientation, and layer thickness) as factors. Finally, Section V
presents the drawn conclusions from this research.
II. EXPERIMENTAL WORK
The experimental work was intended to investigate the
effect of the SLA process parameters, i.e. layer thickness, part
orientation, and hatch overcure, over the build-time of the
produced part.
A. Part Design
The experiments were conducted using samples of a
pyramid-shape with a height of 50 mm, a square base of
37.5 mm per side, and a cylindrical hole of 12.4 mm diameter,
as illustrated in Fig. 2. The CAD file of the sample was
translated to the STL format (.stl) in order to be exported to
the SLA machine.
41
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Fig. 2. Pyramid-Shape of the Experimental Samples
B. SLA Machine
In order to produce the experimental samples, an
SLA 5000 machine manufactured by 3D Systems was used.
The SLA 5000 machine is a Stereolithography machine for
liquid photo reactive materials. The machine is typically used
for the production of casting patterns, patterns for Prosthetics,
marketing model, aerodynamic testing, and design
verification. The specifications of the machine are [9]:
Laser power at vat at 5000 hours: 216 mW
Laser beam diameter: 0.20 0.30 mm
Maximum recommended scan speed: 5.0 m/sec
Maximum part weight: 68.04 kg
Maximum build envelope (XYZ): 508 x 508 x 584 mm
Software used: 3D Lightyear
Recommended temperature: 20 26 C
Maximum change rate in temperature: 1C/hr
C. Part Material
The used material for producing the experimental samples
is the Accura Xtreme that is a commercial resin polymer. The
specifications of this liquid material (grey appearance) are [9]:
Liquid density at 25 C: 1.13 g/cm3
Solid density at 25 C: 1.19 g/cm3
Viscosity at 30 C: 250 300 cps
Penetration depth: 4.1 mm
Critical exposure: 11.7 mJ/ cm3
The Accura Xtreme possesses the look and the feel of a
durable molded plastic with an outstanding durability and
impact resistance, as well as, a thermal resistance over 60 C.
Thus, these characteristics of the Accura Xtreme permit using
this material for applications such as the production of
prototypes (form, fit, and functional testing prototypes),
durable assemblies (snap fit assemblies, tough enclosures, and
consumer electronic components), and master patterns for
Silicone molding. Moreover, this material replaces CNC
machining of Polypropylene and ABS. The use of the
Accura Xtreme for these applications is justified due to the
fact that the properties of the post-cured material are [9]:
Tensile strength: 38 44 Mpa
Tensile Modulus: 1790 1980 Mpa
Elongation at break: 14 22 %
Flexural strength: 52 71 Mpa
Flexural modulus: 1520 2070 Mpa
Impact strength (Notched Izod): 35 52 J/m
Heat deflection temperature at 66 PSI: 62 C
Heat deflection temperature at 264 PSI: 54 C
D. Part Post-Processing
Following the production of the part, it undergoes some
post processing operations. Post processing starts by removing
manually the support structure, then the part is dipped into an
Acetone bath to remove the residual resin on the part surface,
brushing the part, and the last step is to put the part into a UV
chamber for 30 minutes to complete the solidification of the
resin.
E. Design of Experiments (DOE)
This study is concerned with the factorial experiments that
focus on, purposefully, changing the inputs in order to observe
and to identify the reasons for changes in the output
response(s). The inputs can be classified as controllable and
uncontrollable factors that may or may not influence the
outputs. The variability of the uncontrollable factors has to be
minimized in order to better understand the relationship
between the controllable inputs and the outputs. The factorial
approach of the Design of Experiments (DOE) allows the
examination of the main (individual) effect, as well as the
interactions. The factorial design is denoted by the exponent-
base system 2k. The base, 2, represents the number of levels
that are tested for each factor. The levels are the values
considered to be the maximum and the minimum of the range
of the focus of the experiments. The number of factors, k,
represents the controllable inputs for the experiments [10].
In this study, the DOE technique was applied in order to
investigate the effect of the SLA process parameters over the
build-time of the produced part. An unreplicated 2 full-
factorial design was used, where the three considered factors
(each of them at two different levels) were the part orientation,
the layer thickness, and the hatch overcure. TABLE I. presents
the applied DOE scheme, where the levels of the layer
thickness and the hatch overcure were set to the lowest and the
highest values of the machine settings, while the part
orientation was set to 0 and 70, as shown in Fig. 3. The level
of 70 for the part orientation level was used for the purpose of
positioning one side face of the pyramid-shape parallel to the
XY plane of the machine, which permits the best surface
finish on the upper face of the part. The value of the hatch
overcure may be a positive, a zero, or a negative value.
42
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

TABLE I. 2 FULL FACTORIAL DOE estimated time. TABLE II. introduces the estimated build-
Hatch Part Layer time, the actual build-time (response variable), and the
Experimental
Overcure Orientation Thickness difference in the build-time of the produced parts for the
Setting
(mm) (Degrees) (mm) different experimental settings of the DOE. The actual build-
1 -0.025 0 0.10 times are greater than the estimated ones and the difference
2 0.125 0 0.10 percentages are noticeable. It is worth to mention that these
difference percentages would be considerably significant and
3 -0.025 0 0.15
affecting the part cost, especially for parts that are more
4 0.125 0 0.15 complex than the simple pyramid-shaped samples of the
5 -0.025 70 0.10 experimental work since the build-time, and consequently the
cost, for such complex parts would be higher.
6 0.125 70 0.10
7 -0.025 70 0.15
IV. REGRESSION ANALYSIS
8 0.125 70 0.15
Regression analysis using Minitab 17 was applied to the
23 full-factorial experiments in order to investigate the effect
of the three process parameters on the part build-time. In the
regression analysis, the hatch overcure was assigned as factor
A, the part orientation as factor B, and the layer thickness as
factor C. The main effects of the factors A, B, and C were
studied, as well as the interactions effects AC, AB, BC, and
ABC. Fig. 4 and Fig. 5 present the results of the regression
analysis, where the mains effects showed significance, while
the interactions effects showed no significance. The results
were attained with a 95% level of significance ( equals 0.05).
The main effect of factor A representing the hatch overcure
showed a 59.5% significance on the part build-time as
Fig. 3. Part Orientation: a) 0 and b) 70 response. Factor C representing the layer thickness and factor
B representing the part orientation showed a 32.5% and 20.5%
When the overcuring occurs exactly at the surface significance, respectively.
separating two consecutive layers of the material, the value of The derived regression equation for the part build-time as
the hatch overcure is zero. A positive value of the hatch the response function of the three SLA process parameters was
overcure is when overcuring occurs in the previous layer of as follows:
material under the surface separating two consecutive layers
of the material. And a negative value of the hatch overcure is Build-Time = 245.5 + 140.0 (A) 0.3381 (B) 753.3 (C) +
when overcuring occurs in the new layer of material above the 0.1905 (A*B) + 1867 (A*C) + 0.09524 (B*C) +
surface separating two consecutive layers of the material. 3.810 (A*B*C)
where: A, B, and C are the hatch overcure, the part orientation,
III. PART BUILD-TIME and the layer thickness, respectively.
The part build-time is a very important response variable TABLE III. shows the results of the Analysis of Variance
of the SLA process, which highly affect the part cost [11]. The (ANOVA) for the studied factors, where the degrees of
build-time is estimated by the operating software of the SLA freedom (DF), the adjusted sum of squares errors (Adj. SS),
machine according to the setting parameters of the process, and the adjusted mean squares errors (Adj. MS) are presented.
however, the actual build-time may vary differently than the

TABLE II. ESTIMATED AND ACTUAL BUILD-TIME

Hatch Part Layer Estimated Actual Actual Estimated Difference in

Experimental
Overcure Orientation Thickness Build-Time Build-Time Build-Time Build-Time
Setting
(mm) (Degrees) (mm) (minutes) (minutes) (minutes) (%)
1 -0.025 0 0.10 153 162 9 6
2 0.125 0 0.10 207 211 4 2
3 -0.025 0 0.15 115 122 7 6
4 0.125 0 0.15 181 185 4 2
5 -0.025 70 0.10 131 138 7 4
6 0.125 70 0.10 188 193 5 3
7 -0.025 70 0.15 95 98 3 3
8 0.125 70 0.15 156 169 13 8

43
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Fig. 4. Normal Plot of the Effects

Fig. 5. Pareto Chart of the Effects

44
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

deducted for these three parameters in their significance order

TABLE III. RESULTS OF ANALYSIS OF VARIANCE during the design and the setting of the part and of the process,
Source DF Adj. SS Adj. MS
respectively.
Model 7 10171.50 1453.07 The derived conclusions are encouraging the conduction of
Linear 3 10033.50 3344.50 further investigations for the purpose of studying the effect of
the SLA process parameters over other part characteristics, i.e.
A 1 7080.50 7080.50
dimensional and geometrical errors, as well as the mechanical
B 1 840.50 840.50 properties of the produced part.
C 1 2112.50 2112.50
2-Way Interactions 3 137.50 45.83 References
A*B 1 24.50 24.50
[1] F. Brenne, T. Niendorf, and H. J. Maier, Additively manufactured
A*C 1 112.50 112.50 cellular structures: Impact of microstructure and local strains on the
monotonic and cyclic behavior under uniaxial and bending load, J.
B*C 1 0.50 0.50 Mater. Process. Technol., vol. 213, no. 9, pp. 15581564, 2013.
3-Way Interactions 1 0.50 0.50 [2] L. Elsherif, M. Ghazy, K. S. Elkilany, Modeling stereolithography
A*B*C 1 0.50 0.50 process parameters using system dynamics, CIE 44 and IMS14
Proceedings. Istanbul, Turkey; 2014. p. 1188200.
Error 0 * * [3] T. Chartier, A. Badev, Y. Abouliatim, P. Lebuady, and L. Lecamp,
Total 7 10171.50 Stereolithography process: influence of the rheology of silica
suspensions and the medium on polymerization kinetics-cured depth and
width, J. Eur. Ceram. Soc., vol. Vol.32, pp. 16251634, 2012.
V. CONCLUSION
[4] P. J. Bartolo and J. Gaspar, Metal filled resin for stereolithography
The cost of producing a part using the SLA process, which metal part, CIRP Ann. - Manuf. Technol., vol. 57, no. 1, pp. 235238,
is relatively high, depends on the cost of the material used 2008.
(usually expensive materials) and the production time. The [5] H. S. Cho, W. S. Park, B. W. Choi, and M. C. Leu, Determining
production time is composed of the part build-time and the optimal parameters for stereolithography processes via genetic
algorithm, J. Manuf. Syst., vol. 19, no. 1, pp. 1827, 2000.
post-curing time that is neglected compared to the build-time.
Thus, the part cost depends largely on the part build-time. [6] H.-S. Byun and kwan H. Lee, Determination of the optimal build
direction for different rapid prototyping processes using multi-citerion
In this study, pyramid-shaped parts were produced using decision making, Robot. Comput. Manuf., vol. Vol.22, pp. 6980,
2006.
the SLA process. The aim of this study was to investigate the
[7] Y. Chen and J. Lu, RP part surface quality versus build orientation:
effect of some of the SLA process parameters, i.e. layer When the layers are getting thinner, Int. J. Adv. Manuf. Technol., vol.
thickness, part orientation, and hatch overcure, over the build- 67, no. 14, pp. 377385, 2013.
time of the produced part. [8] Y. S. F. Xu H.T. Loh; Wong, Considerations and selection of optimal
orientation for different rapid prototyping systems, Rapid Prototyp. J.,
The regression analysis for the produced experimental vol. 5, no. 2, 1999.
parts has shown that the 2-way and the 3-way interactions of
[9] 3D Systems. [Online]. Available:
the three SLA process parameters have no significance over
http://www.3dsystems.com/materials/accurar-xtreme. [Accessed: 25-
the part build-time. On the other hand, the main effects of the Jul-2015].
three SLA process parameters were significant with a 59.5% [10] D. C. Montgomery, Design and Analysis of Experiments. Wiley, 1997.
significance for the hatch overcure, a 32.5% significance for
[11] Y. Zhang, A. Bernard, J. M. Valenzuela, and K. P. Karunakaran, Fast
the layer thickness, and a 20.5% significance for the part adaptive modeling method for build time estimation in Additive
orientation. Therefore and in order to decrease the high cost of Manufacturing, CIRP J. Manuf. Sci. Technol., vol. 10, pp. 4960, 2015.
producing a part with the SLA process, more attention is to be

45
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Modeling and Simulation of MDF Bagasse Fiber Drying
Gafary Mahmoud Esam H. Abdelhameed, and G. Shabib
Nag-Hamady Fiberboard Company, Faculty of Energy Engineering,
Qena, Egypt Aswan University, Aswan, Egypt
gafry83@yahoo.com ehhameed@yahoo.com
AbstractDrying process is the most important stage in
Medium Density Fiberboard (MDF) production. On one hand,
this process is considered the most energy-consuming stage in
MDF manufacturing which affecting the final price of the
product, in addition, the loss of the energy represents50-60% of
the total energy. On the other hand, the product quality is
depending on how the drying performance is going on i.e. the
investigation of the produced MDF properties is directly related
to how the desired moisture content is obtained. The authors
propose drying process improvement through the control side of
view by optimizing the amount of energy that is required for
drying the wet resinated fibers before pressing it into aboard
during the pressing stage. Consequently, achieving a simulation
model for the drying process is the target in this study. The
obtained model has been verified using the experimental data
from, Nag-Hamady Fiberboard Company, Qena, Egypt. Using the
proposed mathematical models, the estimated values of the mixing
chamber outlet air temperature, and wet fiber temperature at the
inlet of the drying process were satisfactory comparing with the
experimental results, where the model of the processed fiber
temperature at the outlet of the drying process has to be modified.
KeywordsMedium density fiberboard; Drying process; Flash
dryer; Mathematical modeling
I. INTRODUCTION
Drying process is the key point for various industrial
sectors, it is considered one of the most common unit operations
in the process industries such as: drying of pulp and paper,
drying of wood, drying of foodstuffs, drying of grain, fruits and
vegetables, drying of herbal medicines and tea, drying of
pharmaceutical products, drying of ceramics, drying of fibrous
materials, drying of textile products, drying of coal, and many
other products. Sugarcane bagasse negatively affecting the
environment when it is treated as a waste, from the other side, it
can be considered as a resource when appropriate technologies
are implemented to process it and producing a consumable
product with market potential. Those products are such as
paper, pulp, Medium Density Fiberboard (MDF), and High
Density Fiberboard (HDF) manufacturing.
In Egypt, with its poor forests resource and high cost price
of imported wood, there is a high demand of wood products. So
that, wood based panel production, particleboard production,
was started in 1963 by Komombo Company in Aswan
governorate .Sugarcane bagasse was used as raw materials for
this manufacture. Where Medium Density Fiberboard (MDF)
was started in 1996 to introduce a new type of wood based
panel using sugarcane bagasse as raw materials in Qena
governorate by Nag-Hamady Fiberboard Company (NFBC)[1].
NFBC has two production lines, the MDF production capacity
978-1-5090-0217-7/15/$31.00 2015 IEEE
Mahmoud A. Sayed
Qena Faculty of Engineering,
South Valley University, Qena, Egypt
of the first line is 200 / , 4500 / , and with total
of 5400 / . Where, for the second line, it is about 50
/ , 1500 / , with total of 72000 / for
both lines.Both of the two lines use one-stage flash tube dryer
[2].
Production of MDF requires a large amount of thermal
energy and electricity. Fiber drying consumes 48% of the total
thermal energy in case of using direct flue gas in the drying
process, where it consumes around 54% in case of hot air
drying. All of the additional thermal energy (16 GWh/year) is
consumed by the fiber drying when air is used for drying[2]. In
addition to the energy consuming challenge in NFBC, control of
drying process is not a fully automatic control. As, with the
failure of control system, proportional-integral-derivative(PID)
controller, to dominate the high turbulent flow and uncertainties
in the input conditions at starting time, the operator switches the
control mode to manual mode to dominate these variations
manually anyway. This situation compounds the problem due to
the occurrence of large losses in thermal energy. In addition, the
operator only considers the failure of control system from the
outlet material characteristics. After getting steady state
operation by the operator, he switched to automatic mode. The
important impact of staring the drying process manually is that,
on one hand, process adjustment is depending on operators
experience. Consequently, not only a lack of security may there
be but also the starting will be a time and energy consuming
process, which can be easily translated as an additional cost for
the final product. on the other hand, the products during manual
starting time did not have an accepted quality so they may sold
with low price, recycled, or in some cases burned ,in both cases
that can be also translated as an additional cost for the final
product.
In the literature, many efforts have been done to solve MDF
production problems but most of those researches focus on the
physical properties of the drying process [3-5].The authors of
these researches proposed new methods for improvement the
drying process but they were from the viewpoint of drying
process design. Authors in [4-5] suggested the use of higher air
velocity during drying besides increasing a second-stage dryer.
They also suggested fiber conditioning in the fiber conveyers.
Where the authors in [3] suggested decreasing the steam flow
rate into the dryer from the refiner.
The significance of this study is that, the proposed method
for improving the drying process based on the control
viewpoint, by optimizing the amount of energy that is required
for drying the wet resinated fibers before pressing it into aboard
during the pressing stage .Wet fiber includes moisture of
30~35%where it is required to be decreased to 9~12% in the
46
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

dried fiber. This can be accomplished by designing
robust/optimized controllers for the drying process off line, and
then it can be applied to control the process at real-time .Been
known that, off-line controller design is basically based on
getting process model. Consequently, achieving a simulation
model for the drying process is the key point in this research.
The model has been developed based on the fundamental
knowledge of thermal properties of the adiabatic system and the
coefficients of heat and mass transfer rate between hot air and
fiber material. The model is able to predict fiber temperature at
the mixing chamber, dryer inlet and dryer outlet temperature
respectively after validation against the measured air
temperature and fiber moisture the authors can design a robust
controller to minimized the error between actual and target
values for the dryer exhaust temperature. The model has been
obtained and verified using the experimental data from NFBC,
Qena, Egypt.
II. ANALYSIS OF FIBER PREPARATION PROCESSES
Fiber drying Process is working according the principle of a
one-stage flash tube dryer where the conveying fiber flow is
heated directly with hot air, which is generated from a boiler
plant. Figure 1 shows the schematic diagram of fiber preparing
process including a drying process, which is the case in this
study. As indicated in the figure, washed bagasse material, in
wash beater, is fed to a plug screw. The plug screw directs
amount of the bagasse material [kg/min] to a pre-heater
according to the Fig. 2. During the pre-heater, which also called
digester, bagasse material is heated to a point where the lignin
present in greatest concentration in the material between the
individual fibers, becomes plastic, at a temperature generally
taken as 140C in material core [6]. Saturated steam with about
1 Mpa pressure and within 170 -180 C range of temperature is
used to heat the material. The required steam in the preheater, in
kg/h, can be determined from the flowing formula [3]:
( )/( (1 )) (1)
Exhaust
gases
5

4
Bagasse
Plug 1 Preheater 2 Refiner Blow
feeding

Cyclone
from the screw (digester) machine line
washer
Wet fiber
Servo-
3
Hot air from valve
source Mixing Hot
C
chamber iB
Fresh air
A
Processed
material Waste

Fig. 1. Schematic diagram of fiber preparing Process break chips into fibers through mechanical action in doing so,
the electricity is converted to thermal energy and the refiner
300
effectively acts as steam generator [7]. The generated steam, in
kg/h, can be calculated by the following equation [3]:
250
3600 / (2)
Material [kg/min]

200
Where is the coefficient of electricity-to-heat by the refiner,
150
in %, is the refiner machine load, in watt, and is the
average latent heat of water vaporization, in kj/kg. After
100
refining process fibers are discharged through a valve into a
blow line. In the blow line, at the normal operating pressure
50
of 0.4~0.5 MPa, a velocity of up to 100 m/s for the mixture of
0 10 20 30 40 50 60 70 80 90 100
fibers and steam can be achieved. The UF resin solution, in
Plug Screw Speed [rpm]
Kg/h, is added in the blow line where the turbulent flow
Fig. 2. Pre-heater feeding versus plug screw speed provides the desired mixing. There is no net energy supplied in
the blow line. However, it is necessary to calculate the
The refining process is the stage which follows the variations in fiber moisture and steam evaporation rate in the
preheating process, in the refiner, the electric energy is used to blow line because with the highly turbulent steam-fiber flow,

47
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
the pressure drop in the blow line is significant and can be
estimated from the following equation [3]:
0.014 ( )/( ) (3)
where, is the blow line pipe length, is the inner diameter of
the blow line pipe, is the steam-fiber velocity in the blow line,
in m/s. At the entrance of the blow line, the steam becomes
superheated steam once the pressure is reduced, and the
superheated steam provides sensible heat. The reached sensible
temperature can be calculated from the following equation
[3]:
5 ( ) 3.48 96.5
Blow line input and output energy can be expressed as in
Eqs. (5) & (6), through which, blow line energy balance can be
obtained [3].
( )
(5)
where is the resin temperature in at injection, and
are the specific heat of fibers at at the inlet and the outlet of
the blow line in kj/kg.k, respectively. The outlet fiber moisture
from the blow line related to the steam evaporation rate ( , in the mixing chamber. Secondly, modeling of dryer inlet air
kg/h) for a given amount of resin solution based on the mass
balance can be:
/( ) (7)
Equations (5) & (7) can be solved to determine the outlet
MC ( ) and the steam evaporation rate ( ), the other
parameters are known from either the operation conditions or
the refining.
III. DRYING PROCESS MODELING AND SIMULATION
Bagasse (fiber) as a hygroscopic material has equilibrium
moisture which is in direct relation to the ambient temperature
and the psychrometer (Humidity Tester) difference i.e. the
difference between a dry and a wet bulb thermometer. The
water evaporation occurs when the wet fiber surface comes in
direct contact with the hot drier air, creating a very high
evaporating rate (very high specific surface of the fibers). The
moisture is "flashed-off" and due to the fast heat exchange, the
temperature of the heating mixture is reduced. The prime
factors of dryer operation are the air throughput, the thermal
capacity of the heating medium and the temperature control,
thermo-dynamic capacity for evaporating, and exhausting the
moisture physical flow of the fibers in the dryer with a basic air
speed. These factors have a direct effect on the dryer efficiency,
which specifies the relationship between the amount of energy
required to heat the actual introduction of the dryer and the total
consumed amount of heat Energy. The difference between these
two values is the loss of energy or no benefit energy, this range
represents50-60% of the total Energy, 22.5GKcal/h [2]. The
proposed solution for decreasing energy loss as well as
improving the quality of the final product i.e. designing an
optimal/robust/intelligent control for the drying process, was
based on the following stated fact. At first, drying is a complex
thermal process because of multi-component moisture transport,
coupled with mass transfer, input parameters uncertain,
chemical/biochemical reactions, phase changes, shrinkage
transient, change in quality and change of physical structure.
Second, the temperature difference in the hot air before and
after contact with the fiber material is highly correlated to
moisture that lumped all of the variables associated with drying
(4) into one variable. Accordingly, fiber moisture MC can be
calculated by simple basics of thermodynamics rules to escape
from the above complexity since the authors interested with
improvement outlet fiber moisture by controlling input thermal
energy as manipulated variable. In addition, uniform moisture
content are required in the produced fiber, which can be
obtained through the precise control of the air temperature at the
out feed of the cyclone, due to, under constant working
conditions such as capacity, inlet material moisture content,
steam evaporation rate, and production speed, the final fiber
moisture is in direct relation (equilibrium) to the exhaust
temperature.
(6)
In this study, to model the drying process to be used for
optimal/robust/intelligent control system design, three physical
properties have been modeled. Firstly, modeling of the hot air
temperature that generated from a mixing chamber, dryer inlet
air temperature, and this case is considered as mixing humid air
temperature, exhaust. Thirdly, modeling of outlet fiber
temperature, which can be considered as an adiabatic process.
During dying process modeling, the thermal properties of the
hot air and fiber are needed to quantify the heat requirement.
The hot air average specific heat ( ) of 2200 J/kg.K and the
average latent heat of water vaporization (H ) of 2100 KJ/kg
have been used for temperature range of 100 to 180 [8].
A. Inlet Hot Air Temperature Modeling
As indicated in Fig.1, fresh air, a pure air from the
atmosphere, is mixed with the hot air, air generated from the
heater (source) with constant temperature of 350, to generate
a mixed air to be directed to the drying process tube. The
mixing process of the fresh air and the hot air is occurred in a
mixing chamber. The thermodynamic properties of hot air and
of fresh air are shown in Table I. The hot air flow rate and
consequently the final temperature of the mixed air is based on
the opening area of the servo valve gate, which is controlled by
a Proportional-Integral-Derivative (PID) controller as a sub-
control loop.
The properties of temperature, pressure, moisture content,
and enthalpy of the generated hot mixed air can be determined
as the properties of as mixing two humid air. When mixing air
of state A (fresh air) and C (hot air), the mixing point (mixed
air) will be on the straight line in point B. The position of point
B depends on the air volume (or mass) of A and C, the points
A, B, and C are as indicated by Fig.1. The mixing process is
indicated in a Psychrometric chart as shown in Fig. 3.
48
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

TABLE I. THERMODYNAMIC PROPERTIES OF HOT AIR AND OF FRESH AIR By substituting from Eq. (9), temperature of the mixed
properties Fresh air Hot air units air can be obtained as in Eq. (13)
Mass flow dry ca.32 Ca.30 Kg/sec
/ (13)
Temperature (dry blub) 34.5 350
Water content 27.7 30-130 g/kg dry At the inlet point of drying process, inlet of the drying tube,
Heat capacity 1.00 1.00 kJ/kg dry the mixed air from the mixing chamber is in direct contact with
the fiber particles. As shown in Fig. 1, the point (3) is the inlet
Dust content 0.5 300 Mg/Nm3
conditions, at which the hot air would be in direct contact with
Delivery pressure the moist fiber when this pressurized hot mixture is released to
0 +300 Pa
(overpressure)
atmospheric pressure in the dryer, the moisture on the fiber
surface quickly evaporates and the vapor diffuses into the air.
The moisture balance of the mixed fresh air and hot source The drying process just after the blow line is very fast, thus, this
air can be expressed as [14, 16]: process is sometimes called flash drying, the mixture
temperature at this point can be estimated based on the
(8) principal of mixing humid air as follows [14,16]:

(9) ( )/( ) (14)

where is the amount of steam at dryer inlet in kg/h, and is

where is the flow of the air generated from the mixing
the sensible temperature of flash steam in .
chamber, in kg/s, which is required for drying 1 Kg of moist
fiber, is constant. is the flow of hot air to the mixing B. Dried Fiber Temperature Modeling
chamber in kg/s, is the flow of fresh air to the mixing Along with the drying tube, just after the flash drying
chamber in kg/s. period, the required heat for removing moisture from fiber can
be supplied by the hot air i.e. the hot air supplies the heat to the
fiber and absorbs moisture vapor at the same time. Accordingly,
the air temperature decreases and humidity increases along the
Enthalpy dryer length. Drying process continues until the fiber
approaches its equilibrium moisture content at the dryer exit.
C The necessary heat consumption of a given flow rate of
Hot air drying material for heat and moisture mass balances, between
the hot air and the fiber surface, in an infinitesimally short
B
Mixed air length of the dryer can be estimated according to [4] as in (15)
A
Specific Humidity Fresh air . . . (15)

where is the particle concentration by volume in kg/m3, is

Fig. 3. Simplified Psychrometric Chart the air density kg/m3, is the fiber density kg/m3, is the air
temperature, is the fiber surface temperature, u is the air
/ (10) velocity m/s, is the particle velocity, is the specific heat
of the fiber in J/kg/K, is the fiber diameter in m, is the
As long as the mixed air is above the saturation line, the fiber length in m, is the hydraulic diameter in m, is the
temperature balance can be expressed by modifying the heat transfer coefficient in W/m2/K, and is the heat
enthalpy balance as [14, 16]: conductivity of air in W/m/K. This energy has to be delivered
by the hot air as its temperature declines from to at the
(11) end of the drying tube. can be calculated as follows [10]:

where is the temperature of the mixed air in , is the ( ) (16)

source air temperature in , is the is the ambient
temperature in , and is the specific heat capacity of air at where ( ) is the heat capacity of hot air in kJ/kg/C at
constant pressure in kJ/kg, the temperature of the mixed air constant pressure as a function of temperature. If the specific
can be expressed as follows: heat capacity of the combustion air is taken as constant and
equal to , the difference in enthalpy can be approximated as
/ (12) the difference in enthalpy, per kg, of the air at and ,
multiplied by the flow of hot air from the mixing chamber ,

49
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

which can be calculated as . , as expressed by illustrated at Fig. 5 (d). The output data, which has been used to
Eq.(17) [4-5]. model the temperature of the exhaust gases, which also is
considered same as the temperature of processed fiber at the
. ( ) (17) outlet of the drying process, has been compared with
experimental data of the exhaust gases temperature as shown in
The output temperature modeling can be obtained by Fig. 6.
approximating the solution of (15) & (17) 170

Mixed air temprature

165

. . /( . . . (1 )) (18)

[ oC]
160

155

150

where is a constant . . . 0 500 1000 1500

(c)
2000 2500 3000

Time [sec]

Steam amount at dryer inlet

5000

IV. SIMULATION RESULTS AND DISCUSSION 4500

[kg/h]
To verify the model of the drying process of bagasse fiber in 4000

MDF, the obtained results from the proposed model has been
3500

3000
compared with the experimental data from Nag-Hamady 0 500 1000 1500
(a)
2000 2500 3000

Fiberboard Company, Qena, Egypt, as shown in Figs. 4-6.

Coefficients of the proposed drying process model and process 144

Flash steam temprature

conditions were described in [9-17]. For each variable of the

[ oC]
143.5
drying process, a set of data which were taken during around
two hours have been illustrated as follows: The data set which 143
represents the inputs to the mixing chamber and its output 0 500 1000 1500
(b)
2000 2500 3000

temperature is illustrated in Fig. 4. The inputs have been used as 210

the experimental hot air temperature, as shown in Fig. 4 (a), the Exp.
Inlet air temprature
200 Model
experimental gate opening ratio, as shown in Fig. 4 (b), and the [oC] 190

fresh air temperature which has been taken around 27 . Figure 180

4 (c) represents the temperature of the resulting mixed air from 170
0 500 1000 1500 2000 2500 3000
the mixing chamber. It is clear from the figure that, the (c)
Time [sec]
temperature of the mixed air is well expressed.
380 Fig. 5. Wet fiber temperature at the drying process inlet
Hot air tem prature

370
[ o C]

360
100
350 Exp.
Model
340 95
0 500 1000 1500 2000 2500 3000
Hot air temprature at drying tube end

(a)
90
90
Gate opening ratio

85
[% ]

80 85

75

70 80
0 500 1000 1500 2000 2500 3000
(b)
170 75
M ix ed air tem prature

Exp.
165
Model
[ o C]

160 70
0 500 1000 1500 2000 2500 3000
Time [sec]
155

150
0 500 1000 1500 2000 2500 3000
(c)
Time [sec]
Fig. 6. Temperature of processed fiber at drying process outlet

Fig. 4. Inputs & output of the mixing chamber.

V. CONCLUSIONS
Controlling drying process is critical to produce a good
Figure 5 shows the wet fiber temperature at the inlet of the product and to save energy losses, therefore this research is
drying process of the data sets. The figure illustrate how good ongoing on this field. Consequently, a mathematical model of
the proposed model is, comparing to the experimental data. bagasse fiber drying process based on industry practice has been
Figure 5 (a), (b), (c) represent the experimental mixed air discussed in this study. The model has been validated by
temperature, the amount of the steam at input of the drying tube, comparing the simulated result with experimental data from
and the temperature of the flash steam, respectively. The Nag-Hamady Fiberboard Company, Qena, Egypt. The obtained
resultant temperature of the inlet dry hot air to the drying tube is mathematical models of mixing chamber outlet air temperature,

50
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
and wet fiber temperature at the inlet of the drying process were
sufficient where the model of the processed fiber temperature at
the outlet of the drying process has to be modified which is one
of the authors duties during future research.
ACKNOWLEDGEMENT
The authors would like to thank Prof. Ahmed Mahmoud
Abdel-Ghany for his constructive cooperation during this study.
Prof. Abdel-Ghany works as a professor Mechanical
Department in the Faculty of Energy Engineering, Aswan
University, Aswan, Egypt. Now He works in the Agriculture
Engineering Department, College of Food and Agriculture
Sciences, King Saud University, Saudi Arabia.
NOMENCLATURE
c Specific heat capacity of the fiber, in [kJ/Kg. K]
a Opening ratio for hot air source damper [0-100 %]
C Specific heat capacity of air [kJ/Kg. K]
H Average latent heat of water vaporization [kj/kg]
Heat loss ratio in preheater and refiner [%]
m Amount of steam generated from pre heater [kg/h]
m Amount of steam generated from the refiner [kg/h]
m Amount of steam at dryer inlet [kg/h]
Coefficient of electricity-to-heat by the refiner [%]
N Refiner machine load [watt]
M Inlet fiber moisture to pre heater [%]
M Outlet fiber moisture from pre heater [%]
M Inlet fiber moisture to dryer [%]
M Outlet fiber moisture from dryer [%]
Inlet fiber temp to pre heater []
Outlet fiber temp from pre heater []
Super heated steam temperature []
Resin temperature []
Mixing chamber outlet air temperature []
Dryer inlet temp after flash process []
Exhaust air temperature []
Source Air temperature for, in []
Ambient temperature []
Humid air flow at exhaust [kg/h]
Hot air from source temperature []
Water content in inlet air [kg/kg]
Water content in outlet air [kg/kg]
Maximum opening area [m2]
Quantity of moist fiber before drying [kg/s]
m Flow of hot air to the mixing chamber [kg/s]
m Flow of fresh air to the mixing chamber [kg/s]
m Flow of mixed air from the mixing chamber [kg/s]
Enthalpy of the moist air [J/kg]
Air density [kg/m3]
Fiber density [kg/m3]
Air temperature []
Fiber surface temperature []
Air velocity [m/s]
Particle velocity [m/s]
C Specific heat of wood fiber [J/kg/K]
d Fiber diameter [m]
L Fiber length [m]
d Hydraulic diameter [m]
h Heat transfer coefficient [W/m2/K]
Particle concentration by volume [kg/m3]
Heat conductivity of air [W/m/K]
REFERENCES
[1]. Zayed, A. A. Adam, A. H. Entesar, and M. A. Elkady,: Properties
of particleboard based on Date palm fronds as renewable Egyptian
lignocellulosic materials, Innovative Space of Scientific Research
Journals, Vol. 9, No. 2, pp. 326-334, Sep. 2014.
[2]. Technology manual operation Buttner document for Nag-Hamady
Fiberboard company
[3]. JinggeLi, ShushengPang, Eric W. Scharpf:Modeling of thermal
energy demand in MDF production,Forest Products Journal, Vol.
57, Source Issue: 9, Sep 2007.
[4]. S. Pang: Mathematical Modelling of MDF Fiber Drying: Drying
Optimization,Drying Technology International Journal, vol. 18,
pp. 1433-1448, Issue 7, 2000, Published online: 10 May 2007.
[5]. S. Pang.:Improving MDF Fiber Drying Operation by Application
of a Mathematical Model, Drying Technology International
Journal, vol. 19,pp. 1789-1805, Issue 8, 2001, published online: 06
Feb 2007.
[6]. Kelvin M. ghapman , A study of Two Aspects of Medium Density
fiberboard manufacture , University of Canterbury. Chemical and
Process Engineering, Issue Date March 2004, Copyright 2002-
2010.
[7]. Allen, D.G.saefstroem,c.wiecke,p. Design aspects of modern MDF
plants Appita journal, vol. 41(2),pp. 93-96,Mar 1988.
[8]. Don W. Green, Robert H. Perry: Perry's Chemical Engineers'
Handbook, Eighth Edition, Copyright 2008.
[9]. Pang. S.: External Heat and Mass Transfer Coefficients for Kiln
Drying of Timber, Drying Technology International Journal,vol.
14, pp. 859-871, Issue 3-4, 1996,Published online: Oct 2007
[10]. ErikW. Aslaksen:Mathematical Model of a Flash Drying
Process,Hindawi Publishing Corporation Journal of Industrial
Mathematics, vol. 2014, February 2014.
[11]. International Journal of Innovation and Scientific Research ISSN
2351-8014 Vol. 9 No. ,pp. 2 Sep. 2014, 326-334
[12]. J. Zhou: EPSRC thermal management of industrial processes; Case
Study: Thermal Design of a Biomass Drying Process Using Low
Grade Heat from Steel Industry, Report Prepared by: SUWIC,
Sheffield University, July 2011.
[13]. Anon, 1999. World MDF production capacity update Part II. Wood
Based Panel International, October/November, 19991216.
[14]. Engineering Toolbox, www.engineeringtoolbox.com
[15]. R. L. Earle: Unit Operations in Food Processing, 2nded,
Copyright 1983 Elsevier Ltd.
[16]. Engineering Toolbox, engineeringtoolbox.com.
[17]. G. Ayugi, E.J.K.B. Banda, F.M. DUjanga:Local Thermal
Insulating Materials For Thermal Energy Storage, Rwanda
Journal, vol. 23, 21-29, 2011.
51
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
New Tuning Approach of Discrete PI/PID Controller
Applied to Bitumen System Based on Non-Minimal
State Space Formulation
E. M. Shaban
Faculty of Engineering, Jazan University, on leave from
Faculty of Engineering (Mataria) Helwan University
Cairo, Egypt
modern3@hotmail.com
R. R. Darwish
Mechanical Engineering Department
Faculty of Engineering Helwan University
Cairo, Egypt
AbstractThis paper presents a new optimal tuning approach
for discrete PI/PID compensators via Linear Quadratic (LQ) cost
function by exploiting non-minimal state space (NMSS) form. In
this regard, first order discrete-time transfer function (TF) of
Bitumen system with samples time delays more than unity is
utilized to formulate the novel NMSS-PI/PID form. The
approach then develops state variable feedback (SVF) control
law for PI/PID implementation on Bitumen system, sited at
INSUMAT Company. The work also introduces proportional-
integral-plus (PIP) control as a well-established model-based, full
state feedback control, when applied to the same Bitumen system,
as benchmark. For consistent comparison, NMSS-PIP form is
formulated using the same TF, then optimizing the PIP gains via
LQ cost function. Here, the Bitumen temperature should not
exceed 5% of the specified temperature (90oC 180oC) in order
to achieve an accepted properties for the produced material.
Moreover, overshoot behaviour is not allowed in order to avoid
self-ignition for the vapor inside the Bitumen tank. Simulation
results verify the applicability of the new discrete PID approach.
Also both discrete PID and PIP control show satisfactory steady
state response with good control action, with little significant
improvement of PIP over new approach of discrete PID control.
The paper shows successful implementation onsite for the two
controllers, for which all the control design criteria are met.
Keywordsdiscrete PID control, proportional-integral-plus
(PIP) control, non-minimal state space (NMSS), Bitumen tank
I. INTRODUCTION
The state space formulation is, perhaps, the most natural
approach for use with digital controllers, since it allows for the
implementation of state variable feedback (SVF) control [1].
One major difficulty is that it is not measureable; therefore
researchers developed state observer to generate a surrogate
state vector which converges asymptotically to the true state
vector [2], e.g. Kalman filter [3] and Luenberger observer [4].
978-1-5090-0217-7/15/$31.00 2015 IEEE
AR. Hamed
Mechanical Engineering Department
Faculty of Engineering Helwan University
Cairo, Egypt
eng_ar_hamed@hotmail.com
A. M. Abdel ghany
Electrical Engineering Department
Faculty of Engineering Helwan University
Cairo, Egypt
ghanymohamed@ieee.org
Alternatively, the current work examines non-minimal
state space (NMSS) form, in which the non-minimal state
vector is composed only of those variables which can be
directly measured and stored in the digital computer. Various
authors have considered NMSS formulation, including Young
& Willems [5], Young et al. [6], Hesketh [7], Taylor et al. [8],
Gonzalez et al. [9] and Gawthrop et al. [10]. Some efforts
have been intended for formulating state space model in order
to tune PID compensators with LQ cost function as in He et al.
[11] and Yu and Hwang [12], considering the error and
integral of error as the state variables. Furthermore, Das et al.
[13] formulates a non-commensurate order state space model
while considering the error signal and its fractional order
differ-integral as the state variables. However, this paper
develops a novel formulation of NMSS model which suits
PI/PID controller design (NMSS-PI/PID), for which the
NMSS vector is composed of three elements (the integral of
error, the error and the difference of error). Optimization in
terms of LQ cost function is used to find the optimum SVF-
PI/PID gain vector that satisfies weighting criteria.
Regarding PIP controller, the dimension of the NMSS
vector depends upon the structure of the discrete-time TF, for
which its elements are the present and past sampled values of
the output variables and the past sampled values of the input
variables [1]. Successful applications include environmental
systems [14], heavy construction machinery [15], nuclear
decommissioning robots [16], navigation of ground vehicle
[17] and temperature system [18].
The industrial Bitumen system established in 1994, sited at
INSUMAT Company, is used as a practical demonstrator in
which the two proposed controllers are implemented for
verification of the novel NMSS-PID control and comparison
with the PIP controller. The demonstrator is a temperature
system in which the Bitumen is heated to a certain set point
priori to mixing process, see Fig. 1. The heating process of
52
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Bitumen is performed by means of hot oil (265oC) [19].
Overshoot is not allowed to avoid self-ignition of the vapor
inside the tank [20-22].
Fig. 1. Typical control process of Bitumen temperature at INSUMAT
Company, Tamouh Egypt.
The control process of the Bitumen temperature is carried
out by controlling the flow rate of the hot oil via three-way
valve. Here the control action manipulates the opening of the
valve by means of AC motor with potentiometer. As shown in
Fig. 1, a temperature sensor (PT100) is used to complete the
closed loop feedback control system.
The Bitumen tank had been repaired in 2014, consequently
the former KFM-PID controller failed to cope with the new
dynamic behaviour. There is a steady state error of 10%
with about 120 minutes of settling time, see Fig. 2. The
manual tuning of KFM-PID controller is chancy, since
Bitumen is a petrochemical substance.
Fig. 2. The response of Bitumen system using KFM-PID controller after tank
amendments, showing the unacceptable steady state error, February 2014 [19].
The contribution of this paper is developing an optimal
tuning approach for PI/PID gains via LQ cost function by
formulating a novel NMSS-PI/PID representation of discrete-
time TF. As a benchmark, the work introduces the NMSS-PIP
control methodology. For consistency, an optimal tuning of
SVF-PIP gain vector is developed via LQ cost function.
II. HARDWARE/SOFTWARE SETTING AND
BITUMEN SYSTEM MODELING
The original KFM-PID controller was replaced with
computer. The computer is connected with a motored three-
way valve, which serves as an actuator, and a temperature
sensor (PT100), which serves as a feedback element, via
LabJack (UE9). The actuator is normalized as a percentage
from 0 (fully closed) to 100 (fully opened) to manipulate the
flow rate of the hot oil. All the detailed explanation for the
hardware/software setting is documented by Hamed et al. [19].
The control system design for both approaches is carried
out entirely in discrete-time, starting from the identification
and estimation of a suitable linearized model of the Bitumen
system to the real-time implementation of the final control
law. Therefore, as a first step, it is required to find a single-
input single-output representation of the dynamic behaviour of
the Bitumen system based on discrete-time TF model. In terms
of backward shift operator, z 1 , this TF model takes the form
B ( z 1 )
yk = uk (1)
A( z 1 )
where yk is the output variable (Bitumen temperature), u k is
the control input (percentage of the valve opening) and
A(z 1 ) = 1 + a1 z 1 + a2 z 2 + " + an z n
(2)
B (z 1 ) = b1 z 1 + b2 z 2 + " + bm z m
in which a1 " an and b1 " bm are the TF parameters.
Data based modeling is utilized to identify the appropriate
structure for the TF (1) and estimate the TF parameters. The
algorithm of simplified refined instrumental variable [23],
combined with Young Identification Criterion (YIC) and the
coefficient of determination RT2 , is implemented using
CAPTAIN toolbox [24] and reveals that a TF with first order
and one numerator with three samples time delay provides an
acceptable description of Bitumen system [19], i.e.
b3 z 3
yk = uk (3)
1 + a1 z 1
The estimation process declares that a1 = 0.972 and
b3 = 0.0696 , at YIC = 11.6 and RT2 = 0.98 . The Bitumen
system shows relatively slow dynamics and the opening value
of the valve is 70% every 1 minute, therefore the sampling
rate had been chosen to be 0.25 sample/minute [19].
III. DISCRETE PI/PID CONTROLLER DESIGN
A. NMSS-PI/PID representation
There are many different structures to implement discrete
PID controllers. The most common structure is depicted in
Fig. 3 for which the proportional, integral and derivative
compensators {k P , k I , k D } are placed into the path of the
error, ek = rk yk . Here, the SVF-PID control action is
uk = k I
ek
1 z 1
( )
+ k P ek + k D 1 z 1 ek
zk (4)
= [k I kP k D ] ek
ek
53
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

for which x k = [z k ek ek ]T is the feedback state vector, 1 a1 0 b3

F = 0 a1 0, g = b3 , h = [0 1 0] (8)

given that ek is the error state, ek is the difference of error
state and z k is the integral of error state. Note that SVF-PI 0 (a1 + 1) 0 b3
control action can be achieved by discarding the derivative
compensator k D and its state ek in (4). The definition (8) describes the NMSS-PI form due to the
existence of the zero-column in the state transient matrix F .
In order to derive the NMSS-PID form, consider the first and
rk second difference of the error state of model (5) as
ek kI
(
+ k p + k D 1 z 1 ) uk ( )
B z 1 yk
1 z 1 A(z ) 1

ek =
(
b3 z 1 1 z 1 )u k
1
1 + a1 z
(9)
Fig. 3. The conventional structure of discrete PID controller when applied to 2
ek =
(
b3 z 1 1 z 1 )u
2

k
the Bitumen system. 1 + a1 z 1
Consider a unity output feedback system shown in Fig. 3. The numerators of both equations in (9) approach zero as
Here, as reported in He et al. [11] and Das et al. [13], the
external set point does not affect the result and therefore it is the system approaches steady state ( z 1 1 ), therefore it is
possible to assume that
possible to place rk = 0 . Thus the closed loop system is
reduced to a regulator problem [26, 27], where it is assumed
that the major disturbance is the change in the reference 2 ek = ek
signal. Consequently, the error signal is given by ek = yk . (10)
= (a1 + 1)ek 1 b3 u k 1
Given the feedback state vector of the PID controller (4), it
is expected that the matrices of the NMSS-PID form is of third Given that 2 ek = ek ek 1 , then equation (10) gives
order. This is not possible since the model (3) has three
samples time delay. The new approach suggest introducing an
intermediate input uk , for which u k = z 2 u k . Therefore, the ek = (a1 + 1)ek 1 b3 u k 1 + ek 1 (11)
TF (3) becomes y k = b3 z ( 1
1 + a1 z 1
)u k . Consequently,
The other two elements of the non-minimal state vector can be
derived from (11) as
b3 z 1
ek = uk (5)
1 + a1 z 1 ek = a1 ek 1 b3 u k 1 + ek 1 (i)
z k = z k 1 a1 ek 1 b3 u k 1 + ek 1 (ii) (12)
The three elements of the non-minimal state vector can
ek = (a1 + 1)ek 1 b3 u k 1 + ek 1 (iii)
now be derived from (5) as follows
A new definition for The NMSS form (7) can be now
ek = a1 ek 1 b3 u k 1 (i) constructed from (12) as
z k = z k 1 + ek
(ii) (6)
= z k 1 a1 ek 1 b3 u k 1 1 a1 1 b3
ek = ek ek 1
F = 0 a1 1, g = b3 , h = [0 1 0] (13)

(iii)
= (a1 + 1)ek 1 b3 u k 1 0 (a1 + 1) 1 b3

The set of equations (6) reveals the following NMSS form,

The definition (13) presents the NMSS-PID form in which
the three PID gains can be optimized using LQ cost function.
x k = F x k 1 + g u k 1
(7) B. PI/PID-LQ optimization
yk = h x k
One possible approach to calculate the control gain vector,
k = [k I k P k D ] , is the optimal state variable feedback
Recalling that x k = [z k ek ek ]T is the feedback state control which minimizes the Linear Quadratic type of
vector. The three dimensional state transient matrix F , input performance criterion, i.e.
vector g and command input vector h are defined as follows

54
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

The PIP control law is then takes the usual SVF-PIP form,
[ ]

J = xTk Q x k + R u k2 (14)
k =0
uk = k xk (19)
Here, Q is a weighing matrix of states and R is a scalar
for weighing the control action. The weighing state matrix is
Q = diag [q z q e q e ] , where qe is the error weighting,
where k = k p [ ]
k u1 ku2 k I is the SVF gain vector. The
SVF-PIP control (19) can be implemented as shown in Fig. 4,
whereas q z and qe are the weighting on the integral of error where it can be considered as one particular extension of the
and difference state variables z k and ek respectively. PI controller, where the control action is enhanced by the
higher order forward path compensators k u (z 1 ) , where,
Given the NMSS-PI/PID system description {F, g} , the
weightings Q and R ; the SVF-PI/PID gain vector can then
be obtained recursively using Algebraic Riccati Equation ku (z 1 ) = ku1 z 1 + ku2 z 2 (20)
(ARE) [28], derived from the cost function (14) as follows,

[
k = g T P (i +1) g + R ]
1 T
g P (i +1) F
(15)
rk ek kI uk ( )
B z 1 yk

P (i ) = F T P (i +1) [F g k ] + Q 1 z 1 A(z )
1

k u ( z 1 )
for which P is a symmetrical positive definite matrix, with
initial value P (i +1) = Q , and k is the control gain vector. kp

IV. PROPORTIONAL INTEGRAL PLUS (PIP) CONTROLLER DESIGN Fig. 4. The typical structure of PIP controller.
The Proportional-Integral-Plus (PIP) controller approach is
firstly introduced by Young and Wang [29] in which the Since PIP exploits the power of SVF within the NMSS
control system design is carried out entirely in discrete time, setting, NMSS-PIP control is inherently much more flexible
starting from the identification and estimation process of a and sophisticated, allowing for optimization in terms of ARE-
suitable linearized model (3) to the real time implementation LQ cost function (15). Here the state weighting matrix
of the final control law. One of the main features of PIP
methodology is the utilization of NMSS form, in which the
Q = diag q y [ qu1 qu2 qz .]
NMSS vector is composed of those states that can be
measured directly and stored for the use of control law, see V. COMPARISON BY SIMULATION
Fig. 4. Recalling that the states are the present and past
A comparison between PID and PIP controller had been
sampled values of the output variable, the past sampled value
carried out by Dixon et al. [30] and Shaban and Nada [31] in
of the input variable and the integral of error, the NMSS
which the comparison took place using different design
formulation of model (3) is
methods; they are frequency domain technique based on the
Nichols chart and a model-based, full state feedback approach.
x k = Fx k 1 + gu k 1 + dy d ,k However in this paper, a consistent comparison between the
(16) two controllers is considered. In this regard, a model-based
y k = hx k SVF is exploited within the NMSS setting which allows for
optimization in terms of ARE-LQ cost function for both
for which the matrix F , and the vectors g , d and h are discrete PID and PIP controller. A simulation for the tracking
defined as follows [1], test and input/output disturbance rejection tests are shown in
Fig. 6 and Fig. 7 respectively, when applied to the discrete
time TF model (3) of the Bitumen system.
a1 0 b3 0 0 0 The tracking test, Fig. 5, shows acceptable response for
0 0 0
0 0 both approaches with zero steady state error at all stages; also
1
F= , g = , d = , faster settling time for PID approach is noticed at the
0 1 0 0 0 0 (17)
beginning by about 4 samples (~16 minutes). In terms of
a
1 0 b3 1 0 1 control action, PIP controller shows superior performance
h = [1 0 0 0] over PID controller which suffers from some noisy behaviour.
Regarding input/output disturbance rejection test, Fig. 6, PIP
Here, the n + m non-minimal state vector x k is controller shows better performance regarding retrieving zero
steady state error quickly as well as its efficient ability to
withstand input disturbance due to the two input compensators
x k = [yk u k 1 u k 2 z k ]T (18) ku1 and ku2 .

55
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

160

Bitumen temperature, y k
TF model response, yk

150
140
120
100 100
80

50 NMSS-PID method 60 PID controller

NMSS-PIP method 40 PIP controller
0 50 100 150 200 250 300 350 50 100 150 200 250 300

100 100
Control input [%], uk

Control input, uk
80 80

60 60

40 40

20 20

0
0 50 100 150 200 250 300
0 50 100 150 200 250 300 350
Samples [0.25 sample/min]
Samples [0.25 sample/min]

Fig. 5. A simulation showing tracking test of PID controller versus PIP Fig. 7. Onsite practical comparison between NMSS-PID and NMSS-PIP
controller when applied to the TF model (3). controllers when applied to the Bitumen system.

Bitumen temperature, y k
120
TF model response, yk

100 100

80 Output
disturbance
20% output
50 60
disturbance PID controller
NMSS-PID method 40 PIP controller
NMSS-PIP method
0 10 20 30 40 50 60 70 80
0 10 20 30 40 50 60 70 80 90 100

100
100
Control input, uk
Control input [%], uk

20% input 80
80
disturbance
60
60
40
40
20
20
0
0 10 20 30 40 50 60 70 80
0 10 20 30 40 50 60 70 80 90 100 Samples [0.25 sample/min]
Samples [0.25 sample/min]

Fig. 8. Onsite practical comparison between NMSS-PID and NMSS-PIP

Fig. 6. A simulation showing 20% input/output disturbance rejection test of controllers when applied to the Bitumen system in case of output disturbance.
PID versus PIP controller when applied to the TF model (3).
Bitumen temperature, y k

160
VI. COMPARISON BY IMPLEMENTATION
140
This section illustrates typical onsite implementation of the 120
new NMSS-PID approach with comparison to NMSS-PIP
100
when both are applied to the Bitumen system. As shown in PID controller
Fig. 7, both controllers provide satisfactory closed loop 80
PIP controller
performance in which all the control criteria are met. However 10 20 30 40 50 60 70 80 90
PID control action suffers from noisy behaviour. Regarding
settling time, contrary to the simulation study, PIP controller 60

shows faster settling time by 2 samples (~ 8 minutes).

Control input, uk

40 Input
Two other practical tests were performed for both disturbace

controllers; they are output and input disturbance rejection test 20

which are shown in Fig. 8 and Fig. 9 respectively. For the
given disturbances, both controllers reveal the zero steady 0
10 20 30 40 50 60 70 80 90
state error in a reasonable time with satisfactory performance. Samples [0.25 sample/min]
However, PIP controller still has superior performance
regarding control action. This may be due to the enhancement Fig. 9. Onsite practical comparison between NMSS-PID and NMSS-PIP
provided by the higher order compensators k u (z 1 ) . controllers when applied to the Bitumen system in case of input disturbance.

56
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
VII. CONCLUSION
A novel formulation of NMSS model which suits PI/PID
controller design (NMSS-PI/PID) is developed in this paper,
for which the NMSS is composed of three elements; they are
the integral of error, the error and the difference of error.
Optimization in terms of LQ cost function is used to find the
optimum SVF-PI/PID gain vector that satisfies predetermined
weighting criteria. The approach utilizes first order discrete
time TF model for the Bitumen system which has three
samples time delay for the sake of tuning the three PID
compensators. For consistency, the NMSS-PI/PID approach is
compared to the well-established PIP methodology, as a model
based full state feedback control. Simulation and onsite
implementation results reveal that the new NMSS-PID
controller provides an acceptable closed loop performance for
which all the design criteria are met, albeit the noisy control
action compared to NMSS-PIP controller. Moreover the novel
NMSS-PID controller reveals satisfactory performance in
terms of output/input disturbance rejection. However, NMSS-
PIP controller still has outstanding performance regarding
control action. This may be due to the enhancement provided
by the higher order compensators k u (z 1 ) , see (20).
Acknowledgment
The authors are grateful for the support of Eng. Ahmed
Osman Ahmed Osman, INSUMAT Company, Tamouh
Giza, Egypt, http://www.insumat.com.
References
[1] Garnier, Hugues. System identification, environmental modelling, and
control system design. Edited by Liuping Wang. Springer, 2012.
[2] Dorf, R.C., Bishop, R.H. Modern Control Systems. Pearson Prentice
Hall. Upper Saddle River, 2008.
[3] Kalman, R.E. A new approach to linear filtering and prediction
problems. ASME Trans., Journal of Basic Engineering, 83: 95 108,
1960.
[4] Luenberger, D.G. Observing the state of a linear system. IEEE Trans.,
Mil. Electron., 8: 74 80, 1964.
[5] Young, P.C., Willems, J.C. An approach to the linear multivariable
servomechanism problem. International Journal of Control, 15: 961
979, 1972.
[6] Young, P.C., Behzadi, M.A., Qang, C.L., Chotai, A. Direct digital and
adaptive control by input-output, state variable feedback pole
assignment. International Journal of Control, 46: 1867 1881, 1987.
[7] Hesketh, T. Linear quadratic methods for adaptive control a tutorial.
Control Systems Centre Report 765, UMIST, Manchester, UK, 1992.
[8] Taylor, C.J., Young, P.C., Chotai, A., Wittaker, J. Non-minimal state
space approach to multivariable ramp metering control of motorway
bottlenecks. IEE Proc., Control Theory Applications, 145: 568 574,
1998.
[9] Gonzalez, A.H., Perez, J.M., Odloak, D. Infinite horizon MPC with non-
minimal state space feedback. Journal of Process Control, 19: 473 481,
2009.
[10] Gawthrop, P.J., Wang, L., Young, P.C. Continuous-time non-minimal
state-space design. International Journal of Control, 80: 1690 1697,
2007.
[11] He, Jian-Bo, Wang, Qing-Guo, Lee, Tong-Heng. PI/PID controller
tuning via LQR approach. Chemical Engineering Science, 55(13): 2429
2439, July, 2000.
[12] Yu, Gwo-Ruey, Hwang, Rey-Chue. Optimal PID speed control of brush
less dc motors using LQR approach. IEEE International Conference on
Systems, Man and Cybernetics, 1: 473 478, October, 2004.
[13] Das, S., Pan, I., Halder, K., and Gupta, A. Optimum weight selection
based LQR formulation for the design of fractional order PID
controllers to handle a class of fractional order systems. International
Conference on Computer Communication and Informatics (ICCCI), pp.
1 6, IEEE, January 2013.
[14] Taylor, C.J., Leigh, P.A., Chotai, A., Young, P.C., Vranken, E.,
Berckmans, D. Cost effective combined axial fan and throttling valve
control of ventilation rate. IEE Proceedings, Control Theory
Applications, 151(5): 577 584, 2004.
[15] Shaban, E.M., Ako, S., Taylor, C.J., Seward, D.W. Development of an
automated verticality alignment system for a vibro-lance, Automation in
Construction, 17: 645 655, 2008.
[16] Taylor, C.J., Chotai, A., Robertson, D. State dependent control of a
robotic manipulator used for nuclear decommissioning activities. In:
IEEE International Conference on Intelligent Robots and Systems
(IROS), Taipei, Taiwan, 2010.
[17] Abdelhamid, A., Shaban, E.M., Zied, K.M., Khalil, Younes.
Implementation of a Class of True Digital Control (TDC) in the
Navigation of a Ground Vehicle. American Journal of Research
Communication, 1(6): 99 111, www.usa-journals.com, 2013.
[18] Shaban, E. M., Elsayed, M. Design, simulation and implementation of a
class of True Digital Control (TDC) applied to natural gas burner. In
Computer Engineering & Systems, 2009. ICCES 2009. International
Conference on, pp. 109 114. IEEE, 2009.
[19] Hamed, AR., Darwish, R.R., Shaban, E.M., Abdel ghany, A.M.
Hardware Synthesis and Dynamic Modeling of Bitumen Tank. Journal
of American Science 10(12):183 189, 2014.
[20] Gorbatovskii, A.A., Rozental, D.A., Dronov, S.V. Structure and
Composition of Polymer-Bitumen Composites. Russian Journal of
Applied Chemistry 80(5):838 841, 2007.
[21] Behzadfar, Ehsan, Hatzikiriakos, Savvas G. Rheology of Bitumen:
Effects of Temperature, Pressure, CO2 Concentration and Shear rate.
The Science and technology of Fuel and Energy 116:578 587, 2014.
[22] Davie, F.M., Mores, S., Nolan, P.F., Hoban, T.W.S. Evidence of the
Oxidation of Deposits in Heated Bitumen Storage Tanks. Journal of
Loss Prevention in the Process Industries 6(3):145 150, 1993.
[23] Young, P.C. Data- Based Mechanistic Modelling of Engineering
Systems. Journal of Vibration and Control 4:5 28, 1998.
[24] Young, P.C., Taylor, C.J., Tych, W., Pedregal, D.J. (2007) The Captain
Toolbox. Center for Research on Environmental Systems and Statistics,
Lancaster University, UK. Internet: www.es.lancs.ac.uk/cres/captain.
[25] Vaishnav, Satish R., Khan, Zafar J. Performance of tuned PID controller
and a new hybrid fuzzy PD+I controller. World Journal of Modelling
and Simulation, 6(2): 141 149, 2010.
[26] Zafiriou, Evangbeloe, and Manfred Morari. "Setpoint Tracking vs.
Disturbance Rejection for Stable and Unstable Processes." American
Control Conference, 1987. IEEE, 1987.
[27] Brian D.O. Anderson, John B. Moore. Optimal Control: linear quadratic
methods. Prentice-Hall International, Inc., Englewood Cliffs, NJ, 1989.
[28] Astrom, K.J. and Wittenmark, B. Computer Controlled Systems: Theory
and Design. Prentice-Hall Information and System Sciences Series,
1984.
[29] Young, P.C., Behzadi, M.A., Wang, C.L., Chotai, A. Direct digital and
adaptive control by input-output, state variable feedback pole
assignment. International Journal of Control, 46: 1867 1881, 1987.
[30] Dixon, R., Taylor, C.J., Shaban, E.M. Comparison of classical and
modern control applied to an excavator arm. In International Federation
of Automatic Control 16th Triennial World Congress (IFAC), 4 8 July,
Prague, Czech Republic, 2005.
[31] Shaban, E. M., and Ayman A. Nada. Proportional Integral Derivative
versus Proportional Integral plus Control Applied to Mobile Robotic
System. Journal of American Science 9(12): 583 591, 2013.
57
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Mechanism Design with Computational
Algebra
Samuli Piipponen
School of information sciences
University of Tampere
Tampere, Finland, P.O Box 617, FI-33014
Email: samuli.piipponen@uta.fi
Telephone: +358 50 3465002
AbstractThe configuration spaces (c-
space) of mechanisms and robots can in many
cases be presented as an algebraic variety.
Dierent motion modes of mechanisms and
robots are usually irreducible components
of the c-space and their union the whole c-
space. Singularities of the variety correspond
usually (but not necessarily) to intersections
of irreducible components/motion modes of
the configuration space. If the mechanisms
purpose is to perform several tasks and it
contains closed kinematical loops then the
tasks are usually dierent motion modes of
the mechanism connected by singularities in
c-space. This means that in order to switch
the mechanism from one task to an other the
mechanism needs to go through a singularity
in c-space. Now there are dierent kinds of
singularities and our main purpose here is
introduce concepts and tools which allows
to investigate and design the properties of
these singularities. The concepts related to
singularities and their properties are well
understood in algebraic geometry and the
advances in computational algebraic geom-
etry and commutative algebra allows us ac-
tually investigate and design these proper-
ties if necessary. KeywordsMechanism the-
ory, Robotics, Kinematical analysis, Compu-
tational algebra, Algebraic geometry
I. Introduction
The crucial fact in our analysis is that the
ideal joints constraints appearing in engineering
can always be formulated as a set of polynomial
equations, say, f1 = 0, . . . , fk = 0 which in turn
span the constraint ideal I = f1 , . . . , fk A problem and analysis has proven to be partic-
in the polynomial ring A. This means that the
c-space is an algebraic variety V(I ) and thus
algebraic geometry and commutative algebra
provide exactly the right framework for the
analysis and design of the c-space.
The goal of this article is to find and analyze
c-space singularities of mechanisms and robots.
Intuitively a singularity is a point in the c-space
978-1-5090-0217-7/15/$31.002015 European Union
Eero Hyry
School of information sciences
University of Tampere
Tampere, Finland, P.O Box 617, FI-33014
Email: eero.hyry@uta.fi
Telephone: +358 50 3185855
of the mechanism where the rank of its con-
straint map drops. Here we present more precise
definition by means of algebraic geometry and
give also criteria to find out whether the c-space
of the mechanism has singularities and also to
compute them if needed. On the other hand,
singularities are not desirable since if you do
not have an idea where singularities are located
in the c-space then dynamical simulation of
the mechanism may prove dicult since stan-
dard DAE-solvers can not handle singularities.
In fact these types of mechanisms and their
constraints are often used when testing and
comparing dierent DAE-solvers [8].
But on the other hand if you want to de-
sign your mechanism to perform several tasks
which are represented by irreducible compo-
nents/motion modes Vi of the c-space V then in
order to move your mechanism from one mode
to another it needs to go through a singularity
in c-space. If one is able to design a mechanism
in such way that the singularities are a priori
known a dynamical simulation can be done
using the generators of each component sepa-
rately. Then in order to switch from one mode
to another it is possible to design a tailored
switching criteria based on the location and
nature of singularities on c-space V. This topic
will also be one goal of our future research.
The singularity problem and analysis is
therefore at the same time particularly impor-
tant, but also dicult area of mechanism design
[2], [3], [9], [17], [14], [13]. In past the singularity
ularly dicult because previously c-spaces has
been considered as smooth manifolds, which by
definition do not posses singularities or as ana-
lytic varieties for which constructive computa-
tional algorithms are rarely available. However
in case of algebraic varieties the singularities are
both well understood and and moreover they
can be eciently computed and analyzed by
computational algebraic geometry. 58
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Especially we mean to develop this method
further to a working design method of such
mechanisms and robots. This kind of care-
ful analysis of kinematics of mechanisms and
robots is extremely important for example in
space exploration since the autonomous ma-
chine can not be maintained and fixed after
they have begun their mission [16].
Suppose for example that the mechanism
has two 1-dimensional modes V1 and V2 and
its c-space V = V1 V2 is connected by singu-
larity p = V1 V2 . Now in a generic case the
modes V1 and V2 intersect non-tangentially at
p so that Tp V1 Tp V2 = {(p, 0)} in the tangent
bundle T V. So there is no regular curve (regular
motion) connecting the modes. Hence in order
to change the task from V1 to V2 we must
have a motion q : [t0 ,t2 ] 7 V (a smooth curve)
connecting V1 and V2 at q(t1 ) = p, but in generic
case [1] the mechanism has to stop at p, that is
1 ) = 0 at p and the motion therefore cannot
q(t
be regular. In most of our computations in this
article we have used a well established program
Singular [11] for computations in algebraic ge-
ometry and commutative algebra.
II. Mathematical Preliminaries
We give some basic facts about algebraic
geometry and computational commutative al-
gebra and refer to [4], [10], [5], [12] for more
detailed discussion.
A. Rings and ideals
We denote the polynomial ring with coe-
cient field K as
A = K[x1 , . . . , xn ].
whose element are polynomials depending on
variables xi . The following facts about ideals
I A in A are fundamental
(i) every ideal is finitely generated, i.e. it
has a basis with a finite number of
generators.
(ii) every radical ideal can be decomposed
to a finite number of prime ideals:
I = I1 Is
where each I is prime. This gives the
decomposition of the variety to irre-
ducible components:
V(I ) = V( I ) = V(I1 ) V(Is ).
B. Singularities
Definition 2.1 (Tangent space): Let
I = f1 , . . . , fk be an ideal and let us
denote by f = ( f1 , . . . , fk ) the map defined by
the generators. The dierential (or Jacobian)
of f is then denoted by d f , and its value at
p is d f p . Let us suppose that I is a prime
ideal. Then the tangent space of the variety
V = V(I ) at p is
TpV = {z Kn | d f p z = 0}.
Note that by definition TpV is a vector space so
its dimension is well defined.
Definition 2.2 (Singular points): A point
p V is singular if dim(TpV ) > dim p (V ).
Otherwise the point p is regular. The set of
singular points of V is denoted by (V ).
Recall that (V ) is itself a variety whose di-
mension is less than dim(V ). Hence almost all
points of a variety are regular.
Let V = V1 V be the decomposition
to irreducible components. Then there are ba-
sically two ways of a point p of a general variety
V to be singular: either p is a singularity of an
irreducible component Vi or it is an intersection
point of two components. That is, the variety
of singular points is
(V ) = (Vi ) Vi V j . (1)
i=1 i= j
Once we have the irreducible decomposition
it is easy to compute the intersections (the
second term in (1)). To compute the singular
points of irreducible components (the first term
in (1)) one needs the concept of Fitting ideals
[6], [10].
Let M be a matrix of dimension k n with
entries in A. The th Fitting ideal of M, F (M),
is the ideal generated by the minors of M.
If 0 we define F (M) = A and if > min{k, n},
then F (M) = 0. Note that
F+1 (M) F (M) V(F (M)) V(F+1 (M)).
Now the variety V(F (M)) consists of the points
where the rank of M is less than . In particular
if F (M) = A then the rank of M is at least
everywhere.
Let now f = ( f1 , . . . , fk ) : Kn 7 Kk be a
map corresponding to the prime ideal I =
f1 , . . . , fk and let V = V(I ) be the correspond-
ing irreducible variety. Let us suppose that
dim(V ) = n . The following result is usually
called the Jacobian criterion. 59
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Theorem 1 (Jacobian criterion): The
singular variety of V is
( ) ( ) ( )
(V ) = V I + Fl (d f ) = V I V Fl (d f ) .
In particular if I + Fl (d f ) = A then V is
smooth.
The number of minors in the Fitting ideal
grows quite quickly as a function of n and k.
Hence sometimes it is very useful to find and
analyze appropriate elimination ideals [4], [10],
[15].
Finally one can ask how does the variety
look like locally. Unlike manifolds varieties
dont have to be smooth or even locally Eu-
clidean. If p is a smooth point then naturally
the tangent space TpV can be thought of as the
best local approximation. In case of singular
points we need the concept of tangent cone C pV
which has the following properties
Definition 2.3 (Tangent cone): Suppose
that V(I ) Rn is an algebraic variety and
I = h1 , . . . , hl and let p V. The Tangent
cone of V at p, denoted by C p (V), is the variety
C p (V) = V( f(p,min) | f I(V)).
The tangent cone has the following basic prop-
erties:
if p is a smooth point then C pV = Tp V
dim(Cp V) = dim p (V).
Hence the tangent cone is the simplest possible
approximation of V in the neighborhood of p
such that dim(C p V) = dim p (V). The generators
of the tangent cone can be obtained eciently
using Grobner basis techniques and the gener-
ated ideal will in fact be homogeneous [4], [10].
C. Intersection multiplicity and singular points
Let p be a point of a subvariety
V Km . Consider an ideal I OV,p with
dimK (OV,p /I ) < . Then the Hilbert-Samuel
function of I is
HI (n) = dimK (OV,p /I n ) (n N).
It is known that there exists a polynomial PI (n)
such that HI (n) and PI (n) coincide for large
n. This polynomial is called the Hilbert-Samuel
polynomial of I. It is of degree d = dim OV,p . The
leading coecient is e(I )/d!, where e(I ) is an
integer. We say that e(I ) is the Hilbert-Samuel
multiplicity of I .
Let V1 ,V2 Kn be subvarieties. Then one
can define the intersection multiplicity of V1
and V2 at an irreducible component of the
intersection V1 V2 . We restrict to the special
case where the irreducible component is a point
p V1 V2 . The definition is based on the idea
of reduction to the diagonal [7]. For this, note
that the diagonal embedding
: Kn Kn Kn ,
(a1 , . . . , an ) 7 (a1 , . . . , an , a1 , . . . , an )
induces an isomorphism V1 V2
= (Kn ) (V1
V2 ).
Definition 2.4 (Intersection multiplicity):
The intersection multiplicity of V1 and V2 at p
is
i p (V1 ,V2 ) = e( OV1 V2 ,(p,p) ),
where
= (x1 y1 , . . . , xn yn ) K[x1 , . . . , xn , y1 , . . . , yn ]
is the ideal of the diagonal (Kn ) Kn Kn .
Fundamental theorem connecting multiplic-
ities of points of V to its tangent cone is
Theorem 2: Let V be a variety and denote
its multiplicity at p V by mult(V, p). Then
the tangent cone C p V and the variety V have
the same multiplicity at p
mult(V, p) = mult(C p V, p).
The tangent cone gives the geometric picture of
the variety near its singular points. For example
if we compute the irreducible decomposition of
the tangent cone at singularity by computing
the primary decomposition of its generators
we can get a clearer geometric picture of the
variety near the singularities. For example if
the variety V is one dimensional and at singular
point the cone is an union of lines intersecting
at p
C p V = L1 L2 ,
then near p the variety looks also like two in-
tersecting lines p. This can happen for example
for the plane curve V1 = V(y2 x2 ) or for the
nodal curve V2 = V(x2 + x3 y2 ) which both
have the same cone C0 V = V(y x) V(y + x)
at origin. Or for three leaved clover curve
V3 = V((x2 + y2 )2 x(3y2 x2 )) the cone of V3
at origin is an union
C0 V3 = L1 L2 L3 ,
and the variety V3 will also look as three inter-
secting lines in a small neighborhood of origin.
III. Test problems
A. Ordinary slider-crank mechanism
Let us analyze few basic examples of mech-
anism and count their multiplicities at singular
points of intersections and try to determine
the nature and tangency of singularities. The 60
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Figure 1. Ordinary slider-crank mechanism with equal
link lengths.
constraint equations for the slider-crank mech-
anism in polynomial for are
p1 = s1 + s2 = 0, p2 = c21 + s21 1 = 0
p3 = c22 + s22 1 = 0.
The constraint ideal is thus I = p1 , p2 , p3 so
let us analyze it in
Q[c1 , s1 , c2 , s2 ]
Now we can compute
I = I = I1 I2 .
The modes V(I1 ) and V(I2 ) are clearly smooth
so the singularities are intersections of the two
modes and we have
(V) = V(I1 + I2 ) = {(0, 1, 0, 1), (0, 1, 0, 1)}
= q1 q2
Let us then define ideals
i1 = a1 + a2 , b1 + b2 , 2b2 a22 b22
i2 = d1 d2 , e1 + e2 , 2e2 d22 e22
which correspond to I1 and I2 after q2 is
transferred to origin. Next we compute the sum
J = i1 + i2 and compute the localization of
quotient ring A/J in the localization O0 of
A = Q[a1 , b1 , a2 , b2 , d1 , e1 , d2 , e2 ]
In OA/J we define the ideal
= a1 d1 , b1 e1 , a2 d2 , b2 e2 OA/J .
and compute its Hilbert polynomial from which
we get intersection multiplicity i(V1 , V2 , q2 ) = 1.
Let us again transfer q2 to origin and investi-
gate the intersection of the modes at q2 at local
ring OV,0 O0 . The computation of the tangent
cone and it primary decomposition gives us
C0V = V(b1 , b2 , a1 + a2 ) V(b1 , b2 , a1 a2 )
= L1 L2 ,
which is now two intersecting lines a1 = a2 and
a1 = a2 in the plane b1 = b2 = 0 confirm-
ing our result about non-tangential intersec-
tion at q2 and about intersection multiplicity
i(V1 , V2 , q2 ) = 1.
B. Slider-crank mechanism of type 2
Let us then look at the slider-crank mecha-
nism whose end eector is restricted to move on
circles which osculate at origin. The constraint
Figure 2. The slider-crank mechanism where the end
eector is constrained to union of osculating circles.
equations for the mechanism are
p1 = c1 + c2 x = 0, p2 = s1 + s2 y = 0
p3 = ((x 1)2 + (y 1/2)2 1/4)
((x 1)2 + (y + 1/2)2 1/4) = 0
p4 = c21 + s21 1 = 0, p5 = c22 + s22 1 = 0
The equation p3 = 0 is clearly the constraint for
the end eector to move on the union of two
circles with radius 1/2 and centers at (1, 1/2)
and (1, 1/2). See Figure 2. The constraint
ideal is thus I = p1 , p2 , p3 , p4 , p5 so let us
analyze it in
Q[c1 , s1 , c2 , s2 , x, y]
Now we can compute
p1 , . . . , p5 = I = I = I1 I2 .
The singular points are again intersection of
two modes V(I1 ) and V(I2 ) which represent
the motion where the end eector is constrained
to move on one of the circles. A quick check us-
ing Jacobian criterion proves that both modes 61
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

V(I1 ) and V(I2 ) are smooth. The singular

points are thus

(V ) = V(I1 I2 ) = V(I1 + I2 )
= V(y, x 1, 4s22 3, 2c2 1, s1 + s2 , c1 + c2 1)
= k1 k2 .

which is can be seen from the figure and con-

struction. The two choices of s2 correspond
to two position of the mechanism from which
other one is reflection of the mechanism about
x-axis in Fig.2. Lets look at the mechanism
then locally at point
k1 = (1/2, 3/2, 1/2, 3/2, 1, 0).
As before we make coordinate transformation
in which k1 is origin and investigate the ideals
i1 = b1 + b2 y1 , a1 + a2 z1 , y21 + z21 y1
, 2b2 y1 + 2a2 z1 + 2a2 + ( 3 1)y1 z1
, a22 + b22 + a2 + ( 3)b2
i2 = e1 + e2 y2 , d1 + d2 z2 , y2 + z22 + y2
, 2e2 y2 + 2d2 z2 + 2d2 + ( 3 + 1)y2 z2
, d22 + e22 + d2 + ( 3)e2 ,
which correspond to I1 and I2 after coordi-
nate transformation. Next We compute the sum
J = i1 + i2 and the localization of the quotient
ring A/J in the localization O0 of
A = Q( 3)[a1 , b1 , a2 , b2 , d1 , e1 , d2 , e2 , z1 , y1 , z2 , y2 ]
In the localization of OA/J we define the ideal
= a1 d1 , b1 e1 , a2 d2 , b2 e2
, y1 y2 , z1 z2 OA/J
The Hilbert polynomial of has multiplicity 2
which indicates i(V1 , V2 , k1 ) = 2. Since V1 and
V2 are smooth and i(V1 , V2 , k1 ) = 2 we must
have tangential intersection of modes V1 and
V2 at k1 .
If we again compute the tangent cone at k1
we
C0V = V(a1 + a2 z, b1 + b2 y, 2a2 z 3y
, 6b2 3z 3y, y2 ) = (L1 )2
which is a doubled tangent line y2 = 0 in the
plane
T = V(6b2 3z, 2a2 z, 6b1 + 3z, 2a1 z),
and indicates the same result i(V1 , V2 , k1 ) = 2.
Figure 3. The slider-crank mechanism where the end
eector is constrained to move on the union of quadri-
folium and a line. The line intersects quadrifolium at
singularity.
C. A non generic situation
Let us finally analyze bit more complicated
slider-crank mechanism whose end eector is
restricted to the union of a line V(y (x 1))
and the quadrifolium curve shown in Figure
3. The constraint equations for the mechanism
are
p1 = c1 + c2 x = 0, p2 = s1 + s2 y = 0
p3 = (y x + 1)(((x 1)2 + y2 )3 4(x 1)2 y2 ) = 0
p4 = c21 + s21 1 = 0, p5 = c22 + s22 1 = 0
Again we can compute
p1 , . . . , p5 = I = I = I1 I2 .
Now however the mode V(I1 ) is not smooth
but has a singularity exactly at the point of
intersection of modes V(I1 ) and V(I2 ) which
is easy to check with Jacobian criterion and we
have
(V1 ) = {t1 ,t2 } (V) = {t1 ,t2 ,t3 ,t4 } = V1 V2 .
The (x,y)-coordinates of the other two singular-
ities t3 and t4 are of course caused by the two
other intersections of the line V(y (x 1)) and
quadrifolium V(((x 1)2 + y2 )3 4(x 1)2 y2 )
seen in Figure 3. Let us again investigate the
mechanism at the intersection t1
t1 = (1/2, 3/2, 1/2, 3/2, 1, 0).
As before we set k1 to origin, make the proper
change of variables and investigate ideals i1 and
i2 which correspond to I1 and I2 after change
of variables. The generators of i1 and i2 are
fairly long so we will not write them down.
As before we form the sum J = i1 + i2 and
compute the localization OA/J of the quotient
ring A/J in localization O0 of
A = Q( 3)[a1 , b1 , a2 , b2 , d1 , e1 , d2 , e2 , z1 , y1 , z2 , y2 ] 62
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Next we form the ideal
= a1 d1 , b1 e1 , a2 d2 , b2 e2
, y1 y2 , z1 z2 OA/J .
The Hilbert polynomial of has multiplicity
4 which indicates i(V1 , V2 ,t1 ) = 4. Since V1
and V2 are smooth and i(V1 , V2 ,t1 ) = 4. As
before we know that two modes of the mech-
anism intersect at t1 and now the formula gives
i(V1 , V2 ,t1 ) = 4 > 1.
However now the motion modes V1 and V2
do not intersect tangentially. If we compute the
primary decomposition of the tangent cone C0 V
we get
C0 V = a1 + a2 z, b1 + b2 y, 2 3)a2 3z 3y
, (2 3)b2 z + ( 3)y, 4z3 y2 4z2 y3
= L1 L22 L32 ,
where the lines are
L1 = T1 + z y, L22 = T2 + y2 , L32 = T3 + z2 .
The lines L22 and L32 correspond to doubled
x and y axis caused by double tangents z2 = 0
and y2 = 0 of the quadrifolium at (1, 0) and L1
corresponds to the intersecting line V(y (x
1)). The decomposition of the cone can in fact
be seen from its last generator
4z3 y2 4z2 y3 = 4y2 z2 (z y).
The planes V(Ti ) R6 fixes the angle variables
and locally the c-space looks geometrically like
three lines z = 0, y = 0 and z y = 0 crossing at
origin in planes V(Ti ).
This example demonstrates that further in-
formation is needed if the intersecting modes
are not smooth and we can not make definitive
conclusions about the tangency of the intersec-
tion by simply counting intersection multiplic-
ities.
IV. Conclusion
The ability to treat configuration spaces of
mechanisms and robots as algebraic varieties
has many advantages. Most important thing
is that crucial properties like singularities and
mobilities can be algorithmically computed and
designed. In this article we have introduced the
concept of intersection multiplicity to investi-
gate the order of tangency of intersections of
dierent motion modes of the mechanisms and
robots. In this article we have used three vari-
ations of simple slider-crank mechanism as our
test problems. However the methods presented
here have been tested for larger systems as well
and they scale up quite well. This is many
ways important topic since at tangential inter-
sections we do not need to stop the mechanism
which enables us control its motion easier. Also
the ability to transit regularly between motion
modes means we do not necessarily have to use
so much external forces to transit between its
dierent mode.
References
[1] T. Arponen, A. M uller, S. Piipponen, and
J. Tuomela. Kinematical of overconstrained
and underconstrained mechanisms by means of
computational algebraic geometry. Meccanica,
49(4):843862, 2014.
[2] T. Arponen, S. Piipponen, and J. Tuomela. Analy-
sis of singularities of a benchmark problem. Multi-
body System Dynamics, 19(3):227253, 2008.
[3] S. Bandyopadhyay and A. Ghosal. Analysis
of configuration space singularities of closed-loop
mechanisms and parallel manipulators. Mecha-
nism and Machine Theory, 39(5):519544, 2004.
[4] D. Cox, J. Little, and D. OShea. Ideals, Varieties
and Algorithms. Springer, Berlin, 3rd edition,
2007.
[5] W. Decker and C. Lossen. Computing in algebraic
geometry, volume 16 of Algorithms and Compu-
tation in Mathematics. Springer-Verlag, Berlin,
2006.
[6] D. Eisenbud. Commutative Algebra, volume 150 of
Graduate Texts in Mathematics. Springer, 1996.
corr. 2nd printing.
[7] H. Flenner, L. O. Carroll, and W. Vogel. Joins and
Intersections. Springer Berlin Heidelberg, Berlin,
1999.
[8] M. Gonz alez, D. Dopico, U. Lugrs, and
J. Cuadrado. A benchmarking system for
MBS simulation software. Multibody System
Dynamics, 16(2):179190, 2006.
[9] C. Gosselin and J. Angeles. Singularity analysis
of closed loop kinematic chains. IEEE Journal of
Robotics and Automation, 6(3):119132, 1990.
[10] G.-M. Greuel and G. Pfister. A Singular intro-
duction to commutative algebra. Springer-Verlag,
Berlin, 2002.
[11] G.-M. Greuel, G. Pfister, and H. Sch onemann.
Singular 3.1.6. A Computer Algebra System for
Polynomial Computations, Centre for Computer
Algebra, University of Kaiserslautern, 2012.
[12] K. Hulek. Elementary algebraic geometry, vol-
ume 20 of Student Mathematical Library. Ameri-
can Mathematical Society, Providence, RI, 2003.
[13] A. M uller. Geometric characterization of the
configuration space of rigid body mechanisms in
regular and singular points. In Proceedings of
IDETC/CIE 2005, ASME 2005, pages 114, Long
Beach, California, USA, September 22-28 2005.
ASME.
[14] F.-C. Park and J.-W. Kim. Singularity analysis of
closed kinematical chains. Journal of Mechanical
Design, 121(1):3238, 1999.
[15] S. Piipponen and J. Tuomela. Algebraic analysis
of kinematics of multibody systems. Mechanical
Sciences, 4(33):3347, 2013.
[16] B. Sch afer and A. C. Leite. Planetary robotics
exploration activities at dlr. Computational and
Applied Mathematics, 34(2):571586, 2015.
[17] J. Sefrioui and C. Gosselin. Singularity analysis
and representation of planar parallel manipula-
tors. Journal of Robotics and Autonomous Sys-
tems, 10:209224, 1993.
63
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Modified Model Predictive Control for a Positioning
Mechanism with Harmonic Drive Gear
Esam. H. Abdelhameed*, Tarek Hassan Mohamed
Faculty of Energy Engineering, Aswan University,
Aswan, Egypt.
ehhameed@yahoo.com
AbstractServo actuators, mechanisms used to produce or
control motion in mechanical systems, provided with harmonic
drives have been widely used in many applications such as
industrial precision positioning mechanisms. However, harmonic
drives exhibit performance features more turbulent than that of
conventional gear transmissions. In order to adapt to the
increasing demand for high-precision motion, nonlinear friction
forces and angular transmission error, which deteriorate the
control performance of the positioning mechanism, should be
compensated. In this study, a Modified Model Predictive Control
(MMPC), methodology has been proposed to improve positioning
precision of a harmonic drive mechanism, i.e. transient
characteristics, based on forcing the manipulated output to track
a predesigned reference trajectory. The MMPC has been designed
in order that, the effect of system disturbances could be reduced.
Numerical simulations have been provided to validate the
effectiveness of the proposed scheme. The performance
characteristics of the proposed scheme has been compared to
those obtained using the standard Model Predictive Control
(MPC) and conventional controllers. As a result, the proposed
system possesses good transient response and good robustness in
face of system disturbances.
KeywordsPrecision motion control; Positioning mechanisms;
model predictive control; harmonic drive.
I. INTRODUCTION
In motion control, the current trends in the applicable new
technologies demand ever-higher precision, productivity, and
accuracy of industrial positioning systems. Since the invention
of harmonic drive gearings (HDs) in the mid-50s of the last
century, HDs have been widely used in a variety of
applications, such as industrial and humanoid robots, and
industrial precision positioning mechanisms due to its excellent
positioning accuracy, compact design, high single-stage
reduction ratio, and high efficiency, zero backlash, back
derivability, and higher torque-to-weight ratio comparing to the
conventional gear drive. [1-2]. Over the last decades, this area is
considered one of the most challenging research areas as the
requirement for high precision positioning and high tracking
accuracy, consequently, many researches have been done to
model the dynamic characteristics of the HD, design robust
controllers and compensate for the nonlinearities of the system
characteristics e.g. nonlinear friction forces, and transmission
compliance [2-6].
On one hand, most of industrial systems are, inherently
nonlinear. With the rapid developments in the applicable new
technologies, industrial systems nonlinearities, higher product
* Corresponding author. IEEE member, Researcher in
Nagoya University, Nagoya, Japan, 2015/2016.
978-1-5090-0217-7/15/$31.00 2015 IEEE
Makoto Iwasaki
Department of Electrical and Electronic Engineering,
Nagoya Institute of Technology
Nagoya, Japan.
quality specifications, increasing productivity demands, and the
trend for every precise motion are control design challenges.
Consequently, classic control algorithms are often insufficient
to control the process adequately and modern control
approaches such as intelligent or adaptive ones should be
applied. This inadequacy of the classic control algorithms is one
of the motivations for the increasing interest in Model
Predictive Control (MPC) as an adaptive methodology. On the
other hand, the MPC appears to be an efficient strategy to
control many applications in industry [7-13]. MPC has many
advantages such as very fast response, and robust against
disturbance and parameters uncertainty. Moreover, MPC is an
appealing solution to the control of constrained discrete-time
nonlinear systems due to its ability to provide the optimal
solution while respecting the given constraints i.e. MPC has a
priori consideration of the constraints for computing the control
action. In addition, MPC methodology can be recruited to
design controllers for variety of dynamical systems as control
action prediction principles of the MPC does not depend on the
applied model type [14]. Furthermore, many theoretical and
applicable studies of MPC have been done [10, 15-16].
In this study, design of robust controller for constrained
discrete-time nonlinear systems using MMPC methodology
based on a reference trajectory, to make a gradual transition to
the desired set point, has been proposed. The MPC technique
produces its optimal output derived from a quadratic cost
function minimization based on simplified model of the
positioning mechanism. The technique calculates the optimal
control signal while satisfying the given constraints over the
output speed of the positioning mechanism. A comparison
between the response of the proposed controllers and the
traditional P-PI controller has been done. Simulation results
proved the capability of the proposed controllers to provide a
good performance comparing to P-PI controller.
This article has been organized as follows: in Section II,
modified MPC as a control scheme has been explained.
Configuration of the positioning mechanism, and control system
specifications have been described in Section III. Control
system configuration has been clarified in Section IV. In
Section V, MPC numerical simulated results have been
illustrated to evaluate the proposed control performance. The
conclusions have been presented in Section VI.
II. MODEL PREDICTIVE CONTROL
MPC has proved as an appealing solution to the control of a
wide range of applications in industry such as chemical process,
petrol industry, electromechanical systems and many other
64
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
applications. MPC is a control strategy that provides solutions
for even linear or nonlinear systems. MPC schemes use linear
models of the controlled system to predict the system dynamics
with consideration of linear constraints on the system states and
inputs and optimize the control actions accordingly. The
strategy of this control approach for solving the control problem
based on repeated solution of an open-loop optimal control
problem for a given state. The first part of the resulting open-
loop input signal is implemented and the whole process is
repeated [17]. To summarize, MPC solves an optimization
problem at each control interval , this solution determines the
manipulated variables to be used in the plant until the next
control interval 1. The MPC scheme is based on an explicit
use of a prediction model of the system response to obtain the
control actions by minimizing an objective function.
Optimization objectives include minimization of the difference
between the predicted and reference response, and the control
effort subjected to prescribed constraints. The modified
configuration of the MPC control system is shown in Fig.1. The
standard configuration of the MPC control system can be
explained as follows: the prediction has two main components,
the free response which being expected behavior of the output
assuming zero future control actions, and the forced response
which being the additional component of the output response
due to the candidate set of future controls. For a linear system,
the total prediction can be calculated by summing both of free
and forced responses, reference trajectory (a) signal is the target
values which the output should attain. The optimizer is used to
calculate the best set of future control action by minimizing a
cost function . The optimization is subject to constraints on
both manipulated and controlled variables [17-18].
Cost Function
Reference Trajectory
Constraints
(b)
Control
Error Input
Optimizer Plant
Reference +
Trajectory - where,
(a) Models
(Predictor)
Predicted output
Standard MPC
Fig. 1. MPC Modified Configuration
The general object is to tighten the future output error to
zero, with minimum input effort. At any control interval k, the
cost function to be minimized is a weighted sum of two terms,
each focusing on a particular aspect of controller performance,
square predicted errors and square future control values. For
Generalized Predictive Control (GPC), let the state-space model
of a single input single output (SISO) linear time invariant (LTI)
system be given as:
1 (1)
where, is the n-dimensional state vector, , , and are
system matrices with dimensions , 1 , and 1 ,
respectively. The internal model of MPC is used to predict the
future plant output based on the past and current values of the
inputs, outputs, and on the proposed optimal future control
actions, in addition to the constraints. The controller current
state, which consists of the plant model state variables and the
manipulated variable, expressed in Eq. (1) and Eq. (2), is used
during the operation of MPC controller as the basis for
predictions. System model yields a state observer, which can be
used by MPC controller in order to estimate the values of
unmeasured states needed as the basis for control input
predictions, and to predict how the controller's proposed
manipulated variable adjustments will affect future plant output
values. The controller updates its state automatically using the
latest plant measurements, and then supply these values to the
controller. The performance function to be minimized is given
by Eq. (3), a standard GPC control law for a SISO LTI system:
, , , ,
1 (3)
where is the predicted output i.e. the predicted value
of plant output velocity at jth prediction horizon step,
is the reference value for plant output at jth prediction horizon
step, N1 and N2 are the lower and upper prediction horizons
over the output, Nu is the control horizon, , are
weighting factors at jth prediction horizon step to help in
competing objectives balance, and is the controlled input
increment, defined as 1 as is the
controlled input. Moreover, the output predictions required to
be computed over the prediction horizon to calculate the
optimal control input by minimizing above cost function as
follows [19]:
(4)
Output
1 (5)
Solution of Eq. (3) gives the optimal sequence of control
signal over the horizon N while respecting the given constraint.
The effectiveness of the MPC is demonstrated to be equivalent
to the optimal control [7-8]. Constraints over the control signal,
the output, and the control signal changing can be added to the
cost function as follows:
where , , , , , and are the
lower and upper limits of the controlled input, controlled input
increment, and plant output, respectively.
In this study, to modify MPC controller principles, the
(2) manipulated variable is forced to track an additional reference
trajectory (b), as shown in Fig. 1, i.e. the internal model of MPC
65
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

is used to predict the future plant output based on the past and
current values of a velocity reference trajectory as well as those
of the inputs, outputs, and the proposed optimal future control
actions. The modified performance function to be minimized
is given by Eq. (6):
, , , , ,
1
(6)
where is the reference trajectory for plant output
velocity at jth prediction horizon step, and is a weighting
factor at jth prediction horizon step.
III. SYSTEM DYNAMICS
Actuators provided with harmonic drive has been widely
used in many applications such as industrial precision
positioning mechanisms. Components of the harmonic drive
gearing is illustrated in Fig.2 (a), adapted from [20], which
consists of three components: a an elliptical shape wave
generator (WG) made up of a rigid steel core, a flexspline (FS)
of an elastic thin walled made up of alloy steel with external
gear teeth at its open end, and a circular spline (CS) which is a
rigid internal gear. The circular spline has two teeth more than
those on the flexspline. The assembled harmonic transmission
system is shown in Fig. 2 (b) [2, 6, 21-22].
Actuators provided with harmonic drive normally controlled
by semi-closed feedback control, a current practice in industry,
for controlling the load position based on motor shaft position.
Harmonic drives are special flexible gear transmission systems,
consequently, they exhibits dynamic behavior more turbulent
than that of conventional gear transmissions. Conventional
control techniques such as P-PI cannot meet the required
characteristics of the control system performance i.e. they are
insufficient to compensate for neither the nonlinear disturbances
and therefore the optimal control accuracy corresponding to the
apparent resolution cannot be attained in the load side [6, 23].
Circular Spline
component model is illustrated in Fig. 3. Where is the motor
inertia, is the total load inertia, and are the viscous
damping coefficients at the motor side and the load side,
respectively. is the driving torque applied by the AC servo
motor. The driving toque can be obtained as ,
where is the motor torque constant and is the input
current to the motor. In this model, the system is analyzed as a
two-mass system with harmonic drive characteristics and the
HD is represented as a semi gray element with three ports that
transmit the torque and angular motion seen by the WG, FS, and
CS. The WG is attached to the motor shaft, the load attached to
FS, and CS is fixed i.e. the rotation of the circular spline
0.0, the rotation of the motor shaft attached to the
wave generator is represented by , and the load angular
position at the end side of the flexspline is represented by
. With ideal motion transmission, ignoring the effects of
friction, compliance, and kinematic error of HD, the behavior of
these three ports can be expressed by the following basic
equations [21, 24-25]:
(7)
/ (8)
1 (9)
where is the resulting torque of the circular spline, is the
transmitted torque across the flexspline ( represents the load
torque), is the resulting torque of the wave generator,
provided by the servo motor, and is the reduction gear ratio.
Equation (7) expresses torque balance around motor inertia,
where Eq. (8) expresses torque balance around load inertia.
Equations (10) & (11) represent the complete model of the
harmonic drive.
_ (10)
_ (11)

Flexspline
where _ represents a dry frictional torque component at the
bearings of the wave generator, and _ represents a dry
Wave Generator frictional torque component at the load.

Circular Spline, CS
,

Flexspline, FS
Motor Torque, ,
(a) (b) Motor Position,
Current
HD Load Torque,
Reference
Fig. 2. Harmonic Drive Gear (a) Components, and (b) Assembly. Semi gray Load Position,
element

The positioning mechanism provided with HD transmission [I] [Nm]

(system under study, is a FHA-8C-50-E200-C actuator with
metal disc load), can be described as a mechanism composed of Motor viscous friction Wave Generator, WG Load viscous friction
coefficient, coefficient,
an actuator (AC servo motor) with an encoder, a harmonic drive ,
gear for motion transmission, and an inertial load. In order to
simplify the description of the positioning device behavior and Fig. 3. Modeling Diagram of the Positioning Mechanism.
to derive the motion equations for the overall system, lumped

66
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

The HD can be represented as a damper and nonlinear
spring, which can be expressed by the following equation:
where is damping coefficient of the HD, is the stiffness
torque of the HD, and is the transmission error angle,
expressed as in Eq. (13) [5, 6, 24, 26].
where is the synchronous component and is the nonlinear
elastic component. The effects of and on the transmission
error angle was taken into consideration to improve the
accuracy of the static positioning. In this study, based on [24,
26], the general equation, which will be used to model the HD
as a non-linear spring characteristics, can be given as:
To consider the effect of the non-linear friction model at low
velocities, _ and _ were modeled as:
tanch
_
_ tanch
where is the motor angular velocity, is the load angular
velocity, and , , , and are constants. The synchronous
component can be expressed as a periodical pulsation for as
follows [24]:
cos
where is the velocity sign function, determines the
width of hysteresis, and are the transmission error and
the motor angle at the velocity reversal, and 2 is the level
(12) fluctuation after the non-stable region. is the bsolute value
of non-linear elastic deformation component during
reversal of rotation direction.
IV. CONTROL OBJECTIVES AND CONSTRAINTS
In this research, from an input/output viewpoint, the plant
has a single input , which is manipulated by the controller,
(13) and has two outputs, one measured and fed back to the
controller, , and the other is unmeasured, . For high-speed
and high-precision positioning, specifications of the target
position of the controlled object have been summarized in two
characteristics, the positioning accuracy and performance
settling time. At first, the positioning accuracy, variations of the
load position during multiple one-way positioning are within
30 load arc-sec ( 1 3600 arc-sec), from the average
response of multiple load positioning. Second, positioning
(14) settling time, motor response is required to settle within 1500
motor arc-sec from its target during 0.25 sec. In addition, the
applied reference current must stay within the range,
| | 0.64 A. These specifications require a fast servo
response despite existence of plant disturbances and/or
(15) constraints on plant input.
V. CONTROL SYSTEM CONFIGURATION
(16)
The structure of the feedback control system to build a
positioning mechanism is described in this section. The angular
position is controlled using what is called cascade control
scheme which has two nested feedback control loops. In which
there is a secondary (inner) control loop, velocity loop, located
inside the primary (outer) control loop, position-loop, as shown
in the Fig. 4 i.e. the controller of position-loop is used to
(17) calculate the set point for the velocity control loop.

where is the harmonic order for motor angular position, is

the amplitude of the harmonics, and is the phase of the Position Position Velocity Velocity Actual
harmonics. In [2], to study the dynamic behavior of the set point Error Position set point Error Velocity Position
Plant
complete harmonic drive system, the model of hysteresis will be +
-
Controller + -
Controller
combined with the wave generator and load dynamic models. +
Based on these nonlinear behaviors, can be expressed as
the following formula for the displacement after velocity Velocity-loop
Derivative
reversal [5]:
Position-loop
. 2 | |
| |
(18)
. Fig. 4. Block diagram of a cascaded control scheme
| |
In this study, the proposed control scheme has used MPC as
the controller for the velocity control loop (inner loop), in which
the velocity error , actual motor velocity , and the
. 1 2 commanded velocity generated by the position loop (outer
(19) loop) are used as MPC inputs. The predicted control signal
has been used as the plant input. The modified MPC-based
1 2
control architecture is derived, and a velocity trajectory is
forwarded to the velocity-loop controller to force the velocity
to track this trajectory as stated in the introduction, shown in
| |, / (20) Fig. 5.

67
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

TABLE I. SPECIFICATIONS OF THE FHA-8C-50-E200-C ACTUATOR

d/dt Maximum current 0.64 A
Maximum torque (related to load side) 3.3 load Nm
Maximum rotational speed (related to load 120 load rpm
Actuator side)
Position Modified
with HD Motor encoder resolution 8000 pulse/rev
+ Controller +- MPC
- Inertia of the motor (related to load side) 0.0074 kgm2
+
Inertia of the load 0.02126 kgm2
Torque constant 0.144 Nm/A
d/dt Reduction ratio 50
Velocity-loop
Position-loop
43.6

Fig. 5. Block diagram of the Proposed MPC 43.58

43.56
VI. RESULT AND DISCUSSION

Motor Displacement [deg]

43.54
Numerical simulations have been carried out in order to trajectory

(Load Angle)
validate the effectiveness of the proposed scheme. The 43.52
P-PI

MATLAB/Simulink software package has been used for this 43.5

MPC
MMPC
purpose. Parameters and specifications of the positioning 43.48
mechanism were described in [24, 26], as shown in TABLE I.
The MPC controller parameters are set as follows: 100 samples 43.46

prediction horizon, 20 samples control horizon, and 0.5 msec 43.44

sampling interval. Where the imposed constraints over the
43.42
control action are considered as: 0.64 A maximum control 0.3 0.4 0.5 0.6 0.7 0.8
Time [s]
action, -0.64 A minimum control action. The results obtained Fig. 6. Motor angular positions
using the proposed controller have been compared with those
obtained using standard MPC and conventional P-PI control 43.6

technique, which is considers a cascade control scheme. The 43.58

parameters of the P-PI controller were optimized by trial and 43.56
error with consideration of the system stability, as follows:
312 rev/sec, the speed
Load Displacement [deg]

position proportional gain 43.54

proportional gain 0.0133 A/(rad/sec), the speed integral trajectory

(Load Angle)

43.52 P-PI
gain 0.1575 A/rad. Figures 6-10 show the simulation 43.5
MPC

results of the proposed schemes, values of the position reference MMPC

trajectory and the obtained motor angles are indicated in the 43.48

figures related to the load side i.e. /50 , and /50 , 43.46
respectively. It has been noticed from Figs. 6 and 7 that, the
43.44
reference and the actual motor and load positions are aligned
and good tracking performance has been achieved with the 43.42
0.3 0.4 0.5 0.6 0.7 0.8
modified and standard MPC controllers comparing to those Time [s]

achieved with the conventional P-PI controller. The resultant Fig. 7. Load angular positions
transmission error is shown in Fig. 8, the effectiveness of the
modified MPC is indicated from the minimization of the 0.02

transmission error comparing to standard MPC and P-PI 0

controllers. The control effort of the proposed modified MPC
controllers is leading those of the other controllers, as reported -0.02
P-PI
in Fig. 9. Moreover, when evaluating the reproducibility of the
Transimission Error [deg]

MPC
-0.04
control system, through calculating the positioning accuracy, as MMPC
(Load Angle)

shown in Fig. 10, there is no sufficient reproducibility since -0.06

there is a little variety of load performance. However, the
mechanisms reproducibility with the proposed MMPC, Fig. 10 -0.08

(c), are better than those with standard MPC, Fig. 10 (b), and -0.1
with conventional P-PI controller, Fig. 10 (a). It has been
noticed also that, although variations of the load position during -0.12
multiple one-way positioning is within 1500 motor arc-sec
from the average response of multiple load positioning, a finite -0.14
0.25 0.3 0.35 0.4 0.45 0.5
steady state error appears in the load position. This error can be Time [s]

explained because of the nonlinear characteristics of the Fig. 8. Transmission Error

harmonic drive gear.

68
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

0.1
is robust against the nonlinear characteristics of the positioning
0.8 0
mechanism and achieved a better performance in comparison of
-0.1
P-PI classical P-PI controller. In addition, the motor position was
0.6 MPC
MMPC
settled at the target position, whereas, the effect of the nonlinear
-0.2

0.4
characteristics of the harmonic drive gear were a problem lead
to a finite steady state error in the load position. Therefore, the
-0.3
Reference Current [A]

0.2
-0.4
0.2 0.25 0.3 authors plan to modify the proposed control technique to
compensate for these effects in future research.
0

NOMENCLATURE
-0.2
State vector of the system
-0.4
Output vector of the system
-0.6 , , and State-space matrices
-0.8
Motor inertia
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5
Time [s] Total load inertia
Fig. 9. Control Input Viscous damping coefficients at the load side
Viscous damping coefficients at the motor side
43.65
Driving torque applied by the AC servo motor
Load Displacement [deg]

43.6
Motor torque constant
(Load Angle)

43.55
Input current to the motor.
43.5
Motor shaft angular position
43.45
Load angular position
0.2 0.25 0.3 0.35 0.4
(a)
0.45 0.5 0.55 0.6
Motor angular velocity
43.6
Load angular velocity
Dry frictional torque at the wave generator bearings
Load Displacement [deg]

43.55 _
(Load Angle)

_ Dry frictional torque at the load.

43.5
Damping coefficient of the harmonic drive
43.45
Stiffness torque of the harmonic drive
0.2 0.25 0.3 0.35 0.4
(b)
0.45 0.5 0.55 0.6 Transmission error angle
43.6
Synchronous component
Nonlinear elastic component
Load Displacement [deg]

43.55
Commanded velocity reference
(Load Angle)

43.5
43.45
0.2 0.25
Variation limit
Fig. 10. Load responses and variation (a) P-PI (b) MPC (c) MMPC
Designing a robust control for a positioning
servomechanism provided with a harmonic drive gear based on
the proposed modified model predictive control technique has
been investigated in this paper. Numerical simulations have
been carried out in order to validate the effectiveness of the
proposed scheme. The proposed controller has been tested for
nonlinear disturbances of the positioning system. A comparison
between performances of the proposed MPCs and conventional
P-PI controller has been carried out. Simulation results
demonstrated that, the closed loop system with MPC controller
Velocity reference Trajectory
REFERENCES
0.3 0.35 0.4
(c)
0.45 0.5 0.55 0.6
[1]. Wenjun Ye, Zhijun Li, and Chun-Yi Su: Development and
Human-like Control of an upper limb rehabilitation exoskeleton
Load position Average response Reference using sEMG Bio-feedback, Proceedings of 2012 IEEE
International Conference on Mechatronics and Automation, China,
pp. 2077- 2082, 2012.
[2]. Masafumi Yamamoto, Yoshifumi Okitsu, and Makoto Iwasaki.
VII. CONCLUSIONS GA-Based Auto-Tuning of Vibration Suppression Controller for
Positioning Devices with Strain Wave Gearings, Proceedings of
the 2015 IEEE International Conference on Mechatronics, pp. 610-
615, 2015.
[3]. Rached Dhaouadi, Torque Control in Harmonic Drives with
Nonlinear Dynamic Friction Compensation, Journal ref: Journal of
Robotics and Mechatronics, Vol.16, No.4, pp. 388-396, 2004.
[4]. Prasanna S. Gandhi, and Fathi H. Ghorbel, Control of Hysteresis
and Kinematic Error Nonlinearities in Harmonic Drives for High
Speed Precision Control Applications, Proceeding of the 2004
American Control Conference, Boston, pp. 1141-1146, 2004.
[5]. Makoto Iwasaki, Masafumi Yamamoto, Hiromu Hirai, Yoshifumi
Okitsu, Kozo Sasaki, and Toshio Yajima Modeling and
Compensation for Angular Transmission Error of Harmonic Drive

69
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Gearings in High Precision Positioning, 2009 IEEE/ASME IMACS World Congress, August 21-25, 2000, Lausanne,
International Conference on Advanced Intelligent Mechatronics, Switzerland.
Singapore, pp. 662-667 , 2009. [26]. Iwasaki, M.; Kainuma, M.; Yamamoto, M.; Okitsu, Y.
[6]. Masafumi Yamamoto, Makoto Iwasaki, Makoto Kainuma, Compensation by Exact Linearization Method for Nonlinear
Yoshifumi Okitsu, Kato Yuki, Kozo Sasaki, and Toshio Yajima,: Components in Positioning Device with Harmonic Drive Gearings,
Compensation for Synchronous Component of Angular Journal-Japan Society For Precision Engineering; 2012, 78(7):624-
Transmission Errors in Harmonic Drive Gearings, The 11th IEEE 630 ; Pub: Japan, JAP SOC PRESISION ENG, 2012.
International Workshop on Advanced Motion Control, Japan, pp.
361-365, 2010.
[7]. Thomas J., D. Dumur, J. Buisson and H. Gueguen.: "Model
Predictive Control for Hybrid Systems under a State Partition based
MLD Approach (SPMLD)", International conference on informatics
in control, automation and robotics ICINCO04, Vol. 3, pp. 78-85,
Setbal, 2004.
[8]. Tarek Hassan Mohamed, Jorge Morel, Hassan Bevrani, Takashi
Hiyama: " Model predictive based load frequency control_design
concerning wind turbines", International Journal of Electrical
Power and Energy Systems (IJEPES) 2012;43:859867.
[9]. Qin, S. J. and T. Badgwell, An Overview of Nonlinear Model
Predictive Control Applications, F. Allgwer and A. Zheng, Eds.,
Nonlinear Predictive Control, Birkhuser, p. 369 (2000).
[10]. Qin, S. J. and T. Badgwell, A Survey of Industrial Model
Predictive Control Technology, Accepted for Control Engineering
Practice (2003).
[11]. Garca, C., D. Prett, and M. Morari, Model Predictive Control:
Theory and Practice A Survey, Automatica, 25(3), 335 (1989).
[12]. Morari, M. and J. H. Lee, Model Predictive Control: Past, Present,
and Future, Comput. Chem. Eng., 23(4/5), 667 (1999).
[13]. Froisy, J. B., Model Predictive Control: Past, Present, and Future,
ISA Transactions, 33, 235 (1994).
[14]. Tarek Hassan Mohamed, Esam H. Abdelhameed, and Ammar
Mostafa Hassan: '' Real Time Robust Position Controller for a Cart
Moved by a DC Motor through MATLAB'', IEEE International
Conference, MEPCON14, Egypt, 2014.
[15]. Lee, J. and B. Cooley, Recent Advances in Model Predictive
Control and Other Related Areas, in J. Kantor, C. Garcia, and B.
Carnahan, Eds., 5th International Conference on Chemical Process
Control, American Institute of Chemical Engineers, p. 201 (1996).
[16]. Mayne, D. Q., J. B. Rawlings, C. V. Rao, and P. O. M.
Scokaert,Constrained Model Predictive Control: Stability and
Optimality, Automatica, 26(6), 789 (2000).
[17]. Frank Allgwer, Rolf Findeisen, and Zoltan K. Nagy, Nonlinear
Model Predictive Control: From Theory to Application, J. Chin.
Inst. Chem. Engrs., Vol. 35, No. 3, 299-315, 2004.
[18]. Richalet J., A. Rault, J. L. Testud and J. Japon: "Model predictive
heuristic control", application to industrial processes. Automatica,
14(5), pp. 413-428, 1978.
[19]. Jianjun Shi, Atul G. Kelkar, and Don Soloway Stable
Reconfigurable Generalized Predictive Control With Application to
Flight Control, Journal of Dynamic Systems, Measurement, and
Control, Vol. 128, pp-371:378, 2006.
[20]. The Virtual Industrial Exhibition, www.directindustry.com
[21]. Timothy D. Tuttle, and Warren Seering, Modeling a Harmonic
Drive Gear Transmission, Proceedings of IEEE International
Conference on Robotics and Automation, vol. 2, pp. 624-629, 1993.
[22]. Piotr Folegaa, Grzegorz Wojnara, and Rafa Burdzika Dynamic
model of a harmonic drive, Transaction on IoT and Cloud
Computing 1(1), pp.39-46, 2013.
[23]. H. D. Taghirad, and P. R. Belanger: Modeling and Parameter
Identification of Harmonic Drive Systems, Journal of Dynamic
Systems, Measurement and Control, 120, 12, (1998), 439444.
[24]. Masafumi Yamamoto, Makoto Iwasaki, Hiromu Hirai, Yoshifumi
Okitsu, Kozo Sasaki, and Toshio Yajima, Modeling and
Compensation for Angular Transmission Error in Harmonic Drive
Gearings, IEEJ transactions on electrical and electronic
engineering, vol. 4, pp.158165, 2009.
[25]. Dhaouadi, R. , Ghorbel, F., and Gandhi, P. "Modeling and Analysis
of Hysteresis in Harmonic Drive gears", Proceedings of 16th

70
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Session III:
Information Systems and Software
Engineering
Title: Page#

Adapting Balanced Score Card (BSC) technique to assess the 72

Egyptian Government Websites
Mai Mahmoud and Rasha Abd El Aziz

Software Release Planning Models: A Comparative Study 78

Asmaa H. Elsaid, Hamdy Ibrahim, and Hatem M. Abdul-kader

Evaluating Social Media Information Usage in Ranking the Web 84

A. S. Hashwa, N. El-Fishawy and S. M. Youssef

Efficient Dependable Rules Generation Approach for Data Quality 91

Enhancement
Asmaa S. Abdo, Rashed K. Salem, and Hatem M. Abdul-Kader

A GIS based-Modelling for Best Allocation of Voters Among 97

Polling Centers: Elections in Egypt Case Study
Ali Amasha, Dina El-Sayed Ibrahim, and Safaa Hassan

Decreasing ERP Implementation Failure in Egypt 106

George Sharmokh Wanas, Ali H. El-Bastawissy, and Mona Ahmed Kadry

71
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Adapting Balanced Score Card (BSC) technique to
assess the Egyptian Government Websites
Mai Mahmoud
Business Information Systems Department
College of Management and Technology, Arab Academy
for Science and Technology and Maritime Transport
Alexandria, Egypt
may.morsi@gmail.com
Abstract this research seeks to assess the 27 Egyptian
governorates website services and activities using an adapted
modified balanced scorecard. It utilizes a content analysis
strategy to get deep insight into the structure and performance of
the 27 different Egyptian governorates websites according to the
different criteria allocated into the four perspectives of the
modified balanced scorecard. The presence or absence of each
criterion is being analyzed and mapped into a frequency table
and then a statistical analysis is performed and the description of
each website is being provided. It has found that the status of the
Egyptian governorate websites differ in terms of website
structure and design efficiency. Moreover, the governorate
websites were found to be lacking privacy and security, calendar
of important governorate events and do not promote active
citizen participation.
Keywords E-government; balanced scorecard; evaluation
criteria
I. INTRODUCTION
Over the last decades, the Internet and the World Wide
Web has played a significant role in consolidating
governmental operations and the government interaction with
the citizens, businesses and with each other which results in
improved provision of citizens services, internal business
efficiencies, citizens participation in the democratic process
and cost reduction [1].
Accordingly, many governmental units have realized the
importance of their online presence and started launching their
own portals to create a one-point access to all the offered and
available governmental services. One of those governments
was the Egyptian government as in the year 2004; the Ministry
of State for Administrative Development (MSAD) has taken
the responsibility for convenient delivery of government
services through the Egyptian government portal, which is
updated on a continuous basis. MSAD introduced new
channels to access and deliver governmental services for
different segments of users such as Internet, landline phones,
mobile phones and service providers. MSAD has also
developed websites for the 27 Egyptian Governorates, which
is considered a one-stop entry point to each governorates
news, information and services provided to citizens.
Egypt is a developing country with a population of 94
million [2] which are mainly centered in the northeast corner
978-1-5090-0217-7/15/$31.00 2015 IEEE
Rasha Abd El Aziz
Business Information Systems Department
College of Management and Technology, Arab Academy
for Science and Technology and Maritime Transport
Alexandria, Egypt
rasha_a@aast.edu
of the country. Egypt has a potential to be a rich country but it
suffered from autocratic ruling hierarchy in the past three
decades, which did not permit a space of freedom or economic
opportunity or prosperity to reach the mass of population.
After the 25th revolution the government operates the country
with a totally different strategy which views the achievement
of citizens prosperity and welfare as one of its priorities, so
according to that the Egyptian governorates websites should
be assessed for their performance effectiveness and efficiency
and their ability to deliver better services with reduced cost
and time saving [3].
With the high population, poor provided services because
of employees lack of professionalism, long queues, dilemma
of paperwork and working hours; the Egyptian government
has launched different Egyptian Governorates websites which
exposes the efforts of providing news, information and
services to the citizens of each governorate, The Development
of the web sites is not self-sufficient, it is supposed to be
monitored and evaluated for their performance according to
different perspectives each contains a set of various criteria to
be able to map these websites to the appropriate e-
government maturity level.
II. LITERATURE REVIEW
A. Local Government
Local governments are the main point of contact for
delivery of services; they are the key players because what
they do directly affects citizens more than the whole
government [4, 5] Local governments are committed to work
with citizens and groups within the community to find
sustainable ways to meet their social, economic and material
needs to improve the citizens' life quality [6].
B. E-government
E-government is now a main theme in the information
society at all levels locally, nationally, regionally and globally.
E-government became a basic element for successful
implementation of different government policy targets. It
changed the way the government functions to be ready to
adapt new and creative developments in order to be citizen
centralized [7]. E-government is defined in literature as a tool
72
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
where governments use the recent information and
communication technologies, especially web-based Internet
applications, to offer citizens and businesses with easy
accessibility to government information and services. This in
return would result in the improvement of governmental
services quality and greater chances to participate in
democratic institutions and processes [8, 9].
C. Balanced score card
It is a performance management assessment tool that
measures organizational performance through aligning and
integrating four different perspectives which are financial,
customers, internal business processes, and learning and
growth. The Financial perspective is intended to assess the
financial performance of the company and determines how the
company wants to be viewed by its shareholders. The
customer perspective is intended to assess the customer
relationships Management by the company and how the
company wants to be viewed by its customers. The internal
business process perspective is intended to assess the business
operations inside the company through which shareholders
and customers satisfaction can be achieved. The learning and
growth perspective identifies the improvements and
modifications which the company needs to put into its
consideration [10].
D. Modified Balanced score card
Morrison [11] has developed a modified balanced
Scorecard (BSC) through which the website effectiveness can
be measured using four balanced perspectives, which are
technical, customer, internal and marketing. Each perspective
is being translated into a set of Critical success factors, which
represent criteria that a website must meet in order to be
effective [12]. The Technical perspective critical success
factors include link check, HTML check, browser
compatibility, load time, spell check and search engine
optimization. The Customer perspective critical success
factors include ease of navigation components and ease of
contact components.
The Internal perspective critical success factors include the
internal functionality of the websites. Marketing perspective
critical success factors include the presence of content or
features related to website marketing.
Morrison [11] also mentioned that the modified BSC
approach has been tested since the late 1990s and found to be
a reliable measurement instrument [13].The modified BSC is
being implemented in assessing websites of different types of
hospitality and tourism businesses. Morrison and Taylor
[11]are the pioneers in using the modified balanced scorecard
method to evaluate the design and maintenance of a group of
small Scottish hotels websites, Yuan [14] used the modified
BSC to evaluate the marketing effectiveness of small
wineries websites in a Mid-western U.S. state. Purnawan [13]
used it in assessing hotel websites of Singapore.
E. Local E-government Evaluation Criteria
Accessibility: According to[15]; [16]; [17]; [18]; [19]
accessibility refers to the website technical
compatibility and website loading speed
Navigation: According to [17]; [18]; [19] the website
pages should be consistent in terms of layout, coloring
and terminology. The pages should be well organized
with a clearly structured navigation menu, site map and
internal search engine.
Public outreach: According to [17]; [18]; [15] the
website should provide adequate contact details.
Privacy: According to [17];[18];[19];[15] a clear
privacy and security statement should be available on
the website and it should protect the citizen
information transition through a secure connection.
Multilingualism: According to [15]; [19]; [18] the
website should support multiple languages and there
should be consistency between the website and those
offered in other languages.
E-participation information Factor: According to [20]
the citizens should be able to access policy documents
online through the website.
E-participation active Factor: According [19]; [18] the
website should promote the citizens participation
through the availability of polls, surveys, bulletin
boards, blogs, chats and e-forums.
III. RESEARCH METHODOLOGY
The research adopted content analysis strategy which
refers to a detailed and systematic examination of the contents
of a particular area for the purpose of identifying patterns,
themes, or biases. Content analysis allows the researcher to
discover features in the content of large amounts of material
that is unnoticeable [21]. Using the content analysis will allow
the researcher get deep insight into the structure and
performance of the 27 different Egyptian governorates
websites to understand and interpret the status of each website
according to the defined criteria and allocate the main
differences between them.
This research is being guided using a modified balanced
scorecard that evaluates the Technical perspectives, the
Citizen perspective, the internal process perspective and the
Marketing perspective of the Egyptian governorates website
according to the different e-government website evaluation
criteria.
A. Technical Prespective
Technical Perspective, where the Egyptian E-government
website technical qualities is being assessed through two main
criteria, first, Accessibility which concentrates on the
download time of the website through a simple PSTN
connection, the availability links for downloading required
software which is necessary to view the websites content and
the website matches the first W3C (World Wide Web
73
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Consortium) accessibility level, number of bad links, HTML

errors, compatibility with different browsers and spelling
mistakes. Secondly, search engine optimization which checks
on which page the search results are shown and link popularity
which refers to the total number of websites link to this
website.

B. Citizen perspective
Citizen perspective focuses on Egyptian governorates
websites user friendliness including ease of navigation and
site attractiveness and it indicates the presence or absence of
specific site features such as navigation, it is directly
connected to websites functionality and ease of use and ease
of contact (public outreach).

C. Internal Business Process perspective Fig. 1. Governorate Accessibility

Internal Business Process shown assesses the website's
functionality of the Egyptian governorates websites to satisfy B. Governorate Search Engine Optimization
the customers intensions through the provision of the online
services offered and its depth and breadth. The search engine optimization of a website tests the
earlier and more frequently a site appears in the search results
and how popular the websites pages as shown in Fig 2.
D. Marketing Perspective
Marketing Perspective assesses the ability of the Egyptian
governorates website to handle transactions online through
secured connection and save citizens data, the availability of
the website in the national language and others, the news and
information updates frequency, governorates event calendar
and the content and width of the information offered.

IV. RESULTS
A content analysis was performed to measure the overall
performance of the Egyptian local e-government using a
modified balanced scorecard in terms of Accessibility, Search
engine optimization, Navigation, Public outreach, website
functionality, privacy, multilingualism, news and updating, Fig. 2. Governorate Search Engine Optimization
general content, specific content and E-participation.

A. Governorate Accessibility C. Governorate Navigation

Accessibility is an essential factor for any website to
operate, especially for E-government websites, being
accessible means the websites usage is available for All
citizens regardless their age, origin, disability and social
status, short downloadable time, no HTML errors and broken
links, compatibility with all browsers and links to Software
required to view the website content as shown in Fig1.
Being a navigational website measures the website user
friendless and eases of use through the presence of well-
organized and consistent navigational menu, consistency of
the websites pages font and coloring with a clear readable
text as shown in Fig 3.

74
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

F. Governorate Privacy
Governorate privacy checks the availability of privacy and
security statement on the website and the movement of the
data provided by the citizens using a secured connection as
shown in fig 6.

Fig. 3. Governorate Navigation

D. Governorate Public Outreach

Public outreach measures the availability of the
governorate contact details online including email and the Fig. 6. Governorate Privacy
ability to contact the elected representatives for requests as
shown in fig 4.
G. Governorate Multilingualism
Multilingualism checks the foreign languages the website
offers and the content completeness in each version as shown
in fig 7.

Fig. 4. Governorate Public Outreach

E. Governorate Internal Process

Governorate internal process measures the internal
capabilities of the website to satisfy the customer intensions
through offering services operating in the right way as shown
in fig 5.
Fig. 7. Governorate Multilingualism
H. Governorate News and updating
News and updating measures the website updating
frequency and checks for the presence of news and the event
calendar as shown in fig 8.

Fig. 5. Governorate Internal Process

75
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Fig. 8. Governorate News and Updating
I. Governorate General Content
General Content checks the availability of detailed
information about the governorate mission, authority
priorities, messages from the authoritys representative, the
structure of the authoritys internal organization and services it
offers, GIS maps and links to other relevant websites as shown
in fig 9.
Fig. 9. Governorate General Content
J. Governorate Specific Content
Specific content checks for the availability of e-
procurement services and job openings notices on the website
and information on governorate financial issues as shown in
fig 10.
Fig.10. Governorate Specific Content
K. Governorate Overall Performance
The overall performance shows the average rate of the
overall performance of the websites depending on the different
criteria which are measured before as shown in fig 11.
Fig.11. Governorate Overall Performance
V. CONCLUSION
This study is the result of the adaptation of a modified
balanced score card for evaluating the websites of 27 Egyptian
governorates by following a content analysis strategy. The
modified balanced scorecard included four main aspects, each
contains one or more criterion translated into items that
addressed the general website features as well as the
governmental websites functionality characteristics.
The study revealed that there is a relatively big difference
in the status between one governorate and another. For
example, some governorates have very clear and well-
structured websites, while others have poor presence or even
no online presence. It was noticeable that most of the
governorate websites lack privacy and security, calendar of
important governorate events, and citizen participation.
Firstly, privacy and security, is considered a key factor
when delivering e-government services; as the government
holds the responsibility of protecting citizens' personal data
and promotes citizens' rights by the provision of an online
privacy and security statement.
Secondly, the calendar of events which is considered an
access point to all events the governorate will hold during the
month and builds a general picture for the citizen on which
events he/she can participate at.
Finally, the citizen participation is divided into information
factor and active factor. For the information factor, the
governorate policy documents were not available in most of
the websites and for the active factor there were not any e-
forums, blogs or chats that allow the citizen to express his
opinion in a specific topic. The governorate websites do not
promote democracy, which is nowadays one of the key aspects
for any country that seeks to follow the development wheel.
76
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
References
[1] Mai Morsy and Rasha Abd El Aziz, A Road map Towards M-
Government Services in Egypt , International Conference IADIS e-
Society 2013, 13 16 March 2013, Lisbon, Portugal. , 2013.
[2] State Information Service (2014). Egyptian Population.[Available at
http://www.sis.gov.eg/En/Templates/Articles/tmpArticles.aspx?CatID=1
9#.Uzez3_mSxTw. Last Accessed: 05 of March 2015]
[3] El Batanony, S. and Abd El Aziz, R. (2015), Investing In Information
Technology To Acquire A Competitive Advantage In Egyptian
Companies: Methods Approach, International Journal of Applied
Engineering Research, Vol. 10, No. 9, pp. 21711-21728
[4] Heeks, R. (2006). Understanding and measuring e-government:
Internationalbenchmarking studies. Paper presented at the UNDESA
Workshop E-Participationand E-Government: Understanding the Present
and Creating the Future,Budapest, pp.27-28.
[5] Norris, D. (2005). E-government at the american grassroots: Future
trajectory.38th Annual Hawaii International Conference on System
Sciences,2005. HICSS'05, Hawaii. 125b.
[6] Asha, A., Belete, A., andMoyo, T. (2013).Analyzing Decentralization
and Local Government's Role Development in South Africa.Journal of
African and Asian Local Government Studies, Vol.2, No.2.
[7] Goundar, S. (2009). An Evaluation of Fijis E-Government Status:
Assessed According to UN Report on Benchmarking E-Government
Progress. Proceedings of the Assocition for Information Systems
Special Interest Group for ICT in Global Development [SIG GlobDev],
Pheonix, 22.
[8] Turban,E. (2012). Electronic commerce: A Managerial perspective.
London: Prentice Hall.
[9] Sprecher, M.H. (2000).Racing to e-government: using the Internet for
citizen service delivery, Government Finance Review, Vol.16, No.5 ,
pp.21-22.
[10] Huang, D.and Hu, Q. (2004).Integrating web services with competitive
strategies : The balanced Scorecard approach .Communications of the
Association for Information Systems , Vol.13, No.1, pp. 57-80.
[11] Morrison, A., Taylor, S., Morrison, A. J., & Morrison, A. D. (1999).
Marketing small hotelson the world wide web. Information
Technology & Tourism, Vol.2, No.2, pp. 97-113.
[12] Morrison, A, Taylor, J., & Douglas, A. (2004). Website evaluation in
tourism andhospitality: The art is not yet stated. Journal of Travel &
Tourism Marketing, Vol.17, No.3, pp.233-251.
[13] Purnawan, P(2011), Evaluating Websites by Features: Do Independent
Hotels in Singapore Get it Right?. UNLV
Theses/Dissertations/Professional Papers/Capstones.Paper 1102
[14] Yuan,J., Morrison,A., Linton,S., Feng,R, andJeon,S.(2004).Marketing
small wineries:An exploratory approach to website evaluation. Tourism
Recreation Research, Vol.29, No.3,pp. 15-25.
[15] West, D.M. (2007). Global E-Government, 2007, [ available at:
http://www.insidepolitics.org/egovt07int.pdf, [accessed 18 February
2015].
[16] Garcia A., Maciel C. and Pinto F. (2005), A quality inspection method
toevaluate e-government sites, Electronic Government: 4th
InternationalConference, EGOV 2005, Copenhagen, Denmark, August
22-26, 2005,Proceedings: Lecture Notes in Computer Science,3591,
Springer 2005, pp.198-209.
[17] Smith, A.G. (2001), Applying evaluation criteria to New Zealand
government websites, International Journal of Information
Management, Vol.21, No.2,pp. 137-149.
[18] Holzer, M. and Kim, S. (2005), Digital Governance in Municipalities
Worldwide, A Longitudinal Assessment of Municipal Websites
throughout theWorld, TheGovernance Institute,
availableat:http://andromeda.rutgers.edu/~egovinst/Website/researchpg.
htm [Last accessed 18 January 2014].
[19] Henriksson, A., Yi, Y., Frost, B. and Middleton M. (2006), Evaluation
instrument fore-government websites, Proceedings Internet Research
7.0: Internet Convergences, Brisbane, Queensland, Australia.
[20] European Communication (2003), The role of e-government for
Europes
futur,availableat:http://ec.europa.eu/information_society/eeurope/2005
/doc/all_about/egov_communication_en.pdf [accessed 18 December
2014].
[21] Leedy, P. D., & Ormrod, J. E. (2013). Practical research: planning and
design. Boston, Pearson.
77
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Software Release Planning Models:
A Comparative Study
Asmaa H. Elsaid, Hamdy Ibrahim, Hatem M. Abdul-kader
Information System Department
Faculty of computers and information, Menofia University
Menofia, Egypt
asma_haroun@yahoo.com
AbstractSoftware release planning is a challenge task as
it includes different stakeholder perspectives, competing
objectives, and different types of constraints. Additionally,
most of information is used during the process of software
release planning usually uncertain. Last few years,
significant progress has been made in software release
planning research. Many software release planning (SRP)
models have been proposed to address challenges
associated with software release planning problem.
Comparing SRP models with respect to these challenges
will help in identifying which challenges are not or
partially addressed. These challenges need to be studied
by researchers. In this paper, a comparative study is
proposed. In the proposed study, SRP problem and
solution spaces are explored, a number of comparison
criteria representing key challenges associated with SRP
problem are identified, 22 well-known SRP models are
identified and compared according to 11 comparison
criteria using Weighted Score Method (WSM). Results of
the comparative study identify a set of challenges that are
not or partially addressed and need more research and
investigation.
KeywordsSoftware release planning, Models, Weighted
Score Method (WSM).
I. INTRODUCTION
Last few years, researchers and practitioners paid
more attention to software release development because
of the increasing market demand of software systems,
extensive use of technology in life, and increasing
awareness for the use of software products. Inefficient
planning for developing a new software release may lead
to: (1) delaying or not including critical and key system
requirements, (2) Overrunning time (i.e., the software
release will reach the market late) and budget, (3)
Stakeholders dissatisfaction because their critical needs
are not implemented in the release, and (4) loss in the
market share.
Software release planning is the process of
developing a plan for delivering a software release to the
market. The software release planning includes selection
and assignment of requirements to create a sequence of
consecutive product releases that satisfies stakeholders
needs, technical resource, budget, and risk constraints. A
good release plan should provide a maximum business
value by offering the best possible set of requirements in
the right sequence of releases, satisfy the needs of key
978-1-5090-0217-7/15/$31.00 2015 IEEE
stakeholders being feasible with available resources, and
reflect existing dependencies between features [6].
It is difficult to clearly and completely define the
software release planning problem. Additionally, finding
an optimal solution for this problem is a challenge.
Information related to software release planning problem
is usually incomplete, inconsistent, and uncertain.
Making balance between competing stakeholders
demands and benefits for the developing organization
according to available resources is the purpose of
software release planning studies [1, 2].
There are many SRP models proposed in literature
[7, 11]. These SRP models represent research efforts
towards addressing challenges associated with SRP.
SRP models can be classified into: some specifically
targeted at the SRP problems while others are simply
requirements prioritization and selection models [9].
A study comparing current SRP models with
focusing on which challenges have been addressed and
to what extent will help in: (1) Identifying current
challenges that are partially or not addressed. (2)
Highlighting future research directions related to SRP
and associated challenges. This paper presents a
Weighted Score Method (WSM) [3, 5] oriented
comparative study. The proposed comparative study
aims at understanding SRP problem and context,
identifying, studying, and analysis of current SRP
models, their process, advantages and disadvantages.
In this study, 22 SRP models have been compared
according to pre-identified 11 comparison criteria. The
proposed comparative study consists of four steps. The
first step starts with exploring SRP problem and context.
The second step identifies a number of comparison
criteria according to challenges associated with SRP
problem and context. In the third step, current SRP
models are explored identifying features of each model,
its advantages, and disadvantages. Finally, the fourth
step applies WSM to compare 22 SRP models according
to 11 comparison criteria.
The rest of the paper is organized as follows: Section
2 discusses related work in particular pervious
comparative analysis. Section 3 presents the proposed
comparative method and discusses its process (i.e. SRP
Exploration, Criteria Identification, SRP Models
Exploration, and Conducting Comparison). In section 4,
results of the proposed comparative study are presented.
78
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Finally, the conclusion for the proposed study is
presented in Section 5.
II. RELATED WORK
Multiple comparative studies for software release
planning have been introduced in this section. Each
study compares different software release planning
models by selecting the proper factors and key aspects
but not by viewing software release planning as a
complex problem that needs a structured approach for
analyzing and understanding it deeply and efficiently.
By depending on a structured approach in this study,
software release planning researchers can find out
challenges points that each model covered and to
determine the topics of research that needs more
concentration in future.
In [8], existing release planning method based on
key technical and non-technical aspects impacting
software release planning is evaluated. The most
relevant aspects of release planning are scope, time
horizon, objectives, stakeholders involvement,
prioritization mechanism, technological constraints,
resource constraints, system constraints, character and
quality of solutions, and tool support. In [7], a
comparative analysis of existing models/approaches that
proved that most of the organizations are still using ad-
hoc approach for strategic Release Planning (RP) even
for their large products is presented. In [11], different
models and contribution of models of strategic RP
(requirements selection and packaging) presented in
academia and being used in Industry addition to the ones
presented in academia are studied. In [10], all strategic
release planning models, their degree of empirical
validation, their factors for requirements selection, and
whether they are intended for a bespoke or market-
driven requirements engineering context are
investigated. In [12], the various release planning
models and the factors considered by these models for
feature selection are analyzed. Thirty two release
planning models are considered and taxonomy of
requirement selection factors is constructed. This study
assists software engineers in finding out the real factors
that need to be considered in planning a release and to
assess the effect of these identified factors on a release
so as to plan releases efficiently and effectively.
In the past comparative studies, a limited number of
SRP models were compared with respect to a limited
number of criteria. Most of past studies did not explore
SRP problem and solution space as well as did not
assign weight to comparison criteria. This makes past
comparative studies subjective. The comparative study
proposed in the paper addresses these drawbacks. The
proposed comparative study uses WSM method to
compare 22 SRP models. This overcomes the
subjectivity of past comparative study.
III. PROPOSED COMPARITIVE METHOD
The steps of the proposed SRP comparative method
presented in this paper are shown in Fig. 1. The key
Fig. 1. Proposed Comparative Method.
objective of the proposed SRP comparative method is
defining a clear and well-structured comparison
framework.
The proposed method helps researchers and
practitioners to identify the SRP model(s) that best fit
with a specific SRP context. The proposed method can
be also used to select SRP models according to their
ability to address challenges associated with a SRP
context. This can be achieved by selecting SRP model
with the highest score in addressing challenges
associated with SRP context. The application of the
proposed method and meeting its objectives require
answering the following research questions:
RQ1: What are the current well-known SRP models?
RQ2: What are technical and non-technical
challenges (i.e. comparison factors or criteria) that can
be used to compare and evaluate these SRP models?
RQ3: What is the relative importance of these
comparison factors or criteria?
RQ4: To What extent the current SRP models
address these challenges?
As shown in Fig. 1, the proposed SRP comparative
method consists of four steps called SRP Exploration,
Criteria Identification, SRP Models Exploration, and
Conducting the Comparison. These steps are iteratively
applied to gather sufficient information about SRP
context and models required to determine a set of
challenges or comparison factors and accordingly SRP
models will be compared and evaluated to determine the
best SRP model addressing these challenges. The
following sections briefly discuss these steps.
IV. SRP EXPLORATION
SRP exploration step aims at understanding the
software release planning context and possible
associated challenges. In SRP exploration, research
papers and case studies related to SRP problem have
been studied and analyzed to explore the SRP problem
space. Studying and analyzing of collected research
papers and case studies helps in the following:
1. Identifying the people (i.e. stakeholders) who are
directly or indirectly impacted by the software
release plans.
2. Exploring stakeholders needs and their preferences
of system features.
3. Identifying resources and different constraints for
each case study.
79
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
4. Determining potential challenges associated with
each case study and corresponding importance level
(i.e. weight).
5. Identifying criteria (e.g., cost, effort, time) used for
selecting features implemented in a release [1, 6].
6. Identifying reasons behind the success and failure of
former SRP projects.
Collecting, studying, and analyzing SRP research
papers, case studies, and similar comparative studies
present keys for answering research questions RQ2 and
RQ3. Analysis of research papers and case studies
reveals many reasons for possible failure of a SRP
project. This includes for example:
1. Stakeholders are not involved in the SRP process.
2. Continuous change in requirements and
stakeholders needs.
3. Difficulty of gathering complete, certain, and
consistent SRP-related data.
4. Under-estimating resources.
5. Schedule pressure and limited resources.
In conclusion, there are common characteristics and
challenges between different SRP contexts. However,
each SRP context has its own unique characteristics and
challenges. Additionally, deciding which set of features
having a maximum business value and increasing
satisfaction level of all stakeholders is a real challenge.
V. CRITERIA IDENTIFICATION
In this step, challenges associated with SRP problem
are identified and defined. These challenges represent
comparison criteria used in and represent the basis for
the comparison study presented in this paper. These
challenges are identified and defined according to
analysis results of SRP exploration step. The identified
challenges are summarized in Table I. Table I presents a
name and a description of each challenge or criterion.
Each challenge or criterion is assigned a symbol (e.g.,
C1-C11). Finally the last column defines possible values
that can be assigned to a SRP model. These values
represent to what extent SRP model address the
corresponding challenge. For example criterion C6
represents number of foreseen future releases. It refers to
the ability of a SRP model to plan for one future release,
two releases, or more than two releases.
VI. SRP MODELS EXPLORATION
In SRP models exploration, SRP models have been
studied and analyzed to explore the solution space of
SRP problem space as well as determining which
challenges have been addressed by which SRP models
and to what extent the challenge has been addressed (see
Table II). This step answers the research question RQ1.
A list of 22 well known SRP models was identified and
used in the comparative study.
Information related to the identified SRP models was
collected and analyzed to determine:
1. Characteristics of SRP context at which the SRP
model was used.
2. What are challenges addressed by the SRP model?
3. To what extent the SRP model addressed a
challenge?
4. What are the advantages and disadvantages of a SRP
model?
VII. CONDUCTING THE COMPARISON
Once problem and solution spaces are explored and
comparison criteria are defined, SRP models will be
compared according to these criteria. Step 3 deals with a
detailed comparison of the identified 22 SRP models. In
this step, Weighted Score Method (WSM) oriented
comparative method is used to conduct the comparison
[3, 5].Conducting the comparison was completed by
applying the scoring values shown in Table III.
1-Each challenge is assigned a weight reflecting the
necessity to address the challenge. A 9-point scale is
used to represent the weight. A weight of 9 is assigned
to a challenge when it is very important to address it to
ensure effective software release planning. A weight of
0 is assigned to a challenge when it is not important at
all to address it. The weights may be normalized so that
their total is one.
2- For each challenge, a score is assigned to each
model reflecting how well the model addresses the
challenge. As shown in Table III, A scores can have
values between 0 and 1; 1 indicates full coverage of the
challenge by the model, 0 indicates no coverage, and
values between 0 and 1 represent different levels of
addressing the challenge.
3- The overall score of each model, reflects how well
the model performs with respect to all challenges, is
calculated using following equation.
= = 1,2, (1)
Where:
Wj: the weight of the jth challenge
Sij: the relative score of the ith model in terms of the jth
challenge.
Si: the overall score of the ith model in terms of all
challenges.
Table IV summarizes the comparison using WSM
method. Fig. 2 depicts the overall score of the 22 SRP
models. The SRP model with the highest overall score
(Si) represents the model who addresses most of SRP
key challenges.
80
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Table I THE COMPARISON CRITERIA IDENTIFICATION.

Target time of release (C1): [4] only proposed a genetic

VIII. RESULTS AND DISCUSSION
As shown in Figures 6 and 7, S-Evolve* has the
highest overall score. However, the SRP models
AEQWW and CVA have the lowest overall score.
According to the analysis and comparison conducted in
this study, a number of challenges that need further
investigation and research have been identified. These
challenges include the following:
algorithm oriented method to determine release plans
considering variation of release dates within a pre-
defined interval of feasibility. All other methods handled
software release planning problems with a fixed release
time. Accordingly, flexible time requires more research.
Further research is required to decide what the
internal/external factors are for flexible intervals and
how to determine the length of this interval.

81
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Table II THE COMPARISON OF 22 SOFTWARE RELEASE PLANNING MODELS.
Team project cooperation & uncertainty of data
(C2): This challenge has not been addressed by current
software release planning models. So far, handling data
uncertainty which obstacles the efficient cooperation
between team members is a real challenge. Addressing
the data uncertainty challenge should aim at decreasing
the uncertainty of data and its possible consequences on
software release planning decisions.
Interdependency & availability of data among
system (C8): system constraint): A further research is
required to address this challenge and to understand the
relationship between the future release of a system and
its previous releases. It is necessary to determine which
features implemented in the previous release are
required to be changed, deleted, or replaced in the future
release considering stakeholder and markets needs.
Interdependency between features should be analyzed to
minimize the impact of updating, deleting, or replacing a
feature on other dependent features.
Features are not well specified (C9): A formal
specification of features is required to ensure that all
features are well defined and completely described.
Clear and complete description of features makes it
easier for stakeholders to properly understand and
evaluate features and requirements.
Supporting old releases (C10): All SRP models do
not support old releases. It is required to study the
selection of features which will be included in the future
release considering supporting old releases. Supporting
old releases means future release should include and
improve key features implemented in old releases such
that stakeholders needs are satisfied.
Software support tool for release planning (C11): As
Release planning is a complex process, developing an
intelligent tool support would be of great value to help in
making release decisions. Release planning needs
intensive human expertise and knowledge and includes
many demanding tasks like resource estimation and
setting objectives in release plan generation and decision
making.
In conclusion, detailed investigation of these
challenges and addressing it aim at meeting the overall
objective of software release planning process and
enhancing the accuracy of associated decisions.
82
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Table III SCORING VALUES FOR EACH CRITERIA. IX. CONCLUSION AND FUTURE WORK
In this paper, a comparative study based on WSM
was proposed. The proposed study includes exploring of
SRP problem and solution space (e.g. SRP models). It
also defines 11 comparison criteria according to
technical and non-technical challenges. In the proposed
study, 22 SRP models were compared with respect to the
11 criteria. Comparison results helps us to determine
which challenges are addressed and to what extent as
well as which challenges need more investigation. Our
future research will focus on studying and addressing
challenges that are partially addressed or not addressed
at all.
REFERENCES
[1] Danesh A.S. and Ahmad R.(2012).Software release planning
challenges in software development: An empirical study
,African Journal of Business Management Vol. 6(3), pp. 956-
970.
[2] Khannur A. (2014).Structured Software Testing: The Discipline
of Discovering.
[3] Kontio J.(1995).OTSO: A Systematic Process for Reusable
Software Component Selection, University of Maryland,
Maryland, CS-TR-3478.
[4] McElroy J., Ruhe G. (2007) Software release planning with
time-dependent value functions and flexible release dates. In:
11th IASTED international conference on software engineering
and applications 2007, Cambridge, pp 429438.
[5] Ncube C. and Dean J. C.(2002).The Limitations of Current
Decision-Making Techniques in the Procurement of COTS
Software Components, In Proceedings of the 1st International
Conference on COTS-Based Software Systems (ICCBSS'02),
Orlando, Florida , pp. 176-187.
[6] Ruhe, G.(2005).SOFTWARE RELEASE PLANNING,
Handbook Software Engineering and Knowledge Engineering -
Vol. 3.
Table IV THE WSM TABLE
[7] SaleemS.B., and ShafiqueM. U.(2008). A Study on Strategic
Release Planning Models of Academia and Industry through
Systematic Review and Industrial Interviews, Master thesis,
School of Engineering, Blekinge Institute of Technology
Sweden.
[8] Saliu, O., Ruhe, G.(2005). Supporting Software Release
Planning Decisions for Evolving Systems, 29th Annual
IEEE/NASA Software Engineering Workshop, pp. 14-26.
[9] Saliu and Moshood O.(2005). SOFTWARE RELEASE
PLANNING FOR EVOLVING SYSTEMS, PhD Research
Proposal
[10] Svahnberg,M. ,Gorschek,T. , Feldt,R. , Torkar,R. , Saleem,S. ,
and Shafique,M U.(2010).Asystematic review on strategic
release planning models, Information and Software
Technology,v.52 n.3, p.237-248.
[11] SvahnbergM., GorschekT., FeldtR., TorkarR., SaleemS., and
ShafiqueM U.(2009). A systematic review on strategic release
planning models.
[12] ValsalaS.and Anil (2014). Review and Analysis of Software
Release Planning Models , International Journal of Engineering
and Advanced Technology (IJEAT) ISSN: 2249 8958,
Volume-3, Issue-5.

Fig. 2. Overall score of 22 SRP Models

83
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Evaluating Social Media Information Usage
in Ranking the Web
A. S. Hashwa, N. El-Fishawy
Computer Science and Engineering Department
Faculty of Electronic Engineering, Menoufia University,
Menouf, Menoufia, Egypt
{ahmed.samy, nawal.elfishawy}@el-
eng.menofia.edu.eg
Abstract Information retrieval is a critical issue for
everyone online. Searching for information happens all the time.
To better retrieve our needs from the web, its logical to leverage
data generated by users via Web 2.0 (web of interaction). Users
actions such as tagging a page, liking, sharing or other kinds of
interaction are used in this paper to better enhance information
retrieval. The work presented in this paper used a dataset from
Open Directory Project (ODP) to test the promising use of social
services for better ranking of web results. We propose a
framework to process social info and use it in the indexing and
ranking processes. The proposed framework retrieves data from
different kinds of social info (social bookmarking, social news,
social network, discovery engines). Raw data is parsed into fields
and significant values are either inserted into the index or used in
the ranking process. Mean Average Precision (MAP) is used to
evaluate results. Results are promising for using social tagging
services such as Delicious. An initial assessment of the usage of
other social services for instance Facebook and so on- reveals
that their data are not good enough to improve ranking of
webpages.
KeywordsInformation Retrieval; Social networks; Rank
Aggregation; Information Retrieval Model; Social Info.
I. INTRODUCTION
As the World Wide Web (WWW) is expanding all the time,
the content of web documents available online is exponentially
increasing [1] thus rendering users confused when finding
information they need. The advent of Search Engines made that
problem much easier to handle as users now have a destination
to resort to. As time went by, a lot of search engines emerged
such as Google, Bing, Yahoo, Ask, AOL Search and others [2].
Their mission is to find the users desired information need but
most of them use the content of these documents mainly to
match a query and the link structure between the documents to
rank the results - among other unpublished proprietary factors.
Recently, as the Web of interaction - also known as Web
2.0 - is evolving rapidly, many services emerged to give the
people the power to express their feedback about any web
resource in the form of comments, tags and rating.
978-1-5090-0217-7/15/$31.00 2015 IEEE
S. M. Youssef
Computer Engineering Department
Arab Academy for Science and Technology,
Alexandria, Egypt
sherin@aast.edu
The interaction of people with these resources via social
media gives valuable information about those resources, which
is known as "wisdom of the crowd"[3][4]. Social media include
social networks (Facebook1, Twitter2, Google+3, LinkedIn4,
etc.), social bookmarking and discovery systems (Delicious5,
StumpleUpon6, etc.) and social news sites (Reddit7).
Unfortunately, major search engines like Google doesn't make
use of this valuable information directly yet there are some
researches that use social annotations in search results [5].
The gap between social info generated by social networks
and Information Retrieval (IR) needs to be bridged to increase
the quality of the results requested by a user. The main tracks
to enhance the IR process and reduce the amount of irrelevant
documents are:
(i) rewriting query using extra knowledge, i.e. expansion
of the user query,
(ii) adding more information about the document from
other sources, i.e. document expansion,
(iii) improvement of the IR model, i.e. the way documents
and queries are represented and matched to quantify
their similarities, and
(iv) post filtering or re-ranking of the retrieved documents
(based on the user profile or context or some
document - related information).
Mainly many researches dealt with this gap using two
approaches:
(i) personalized approach by modeling the user
(extracting a profile or a group of keywords that
1
http://www.facebook.com
2
http://www.twitter.com
3
http://plus.google.com
4
http://www.linkedin.com
5
http://www.delicious.com
6
http://www.stumbleupon.com
7
http://www.reddit.com
84
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
represent topics interesting for a user) and
customizing the result for him/her,
(ii) non-personalized approach which uses social info
generally to better enhance information retrieval.
Our proposed approach is a non-personalized model of IR,
which depends on the social information available publicly by
social media as a source of extra information about web pages
included in the indexing phase of IR and used at the ranking
phase to enhance the quality of ranking.
The aim of our proposed work is to achieve the following:
1) The inclusion of social information in the indexing
process of IR for better results.
2) Re-ranking results using the available social information.
3) An evaluation study of the proposed approach and a
comparison with the closest works on a large public
dataset.
The remainder of this paper is structured as follows.
Section II reviews some related work. Section III describes our
IR Model. In section IV, we describe our dataset and show
evaluation methodology and metrics then discuss the results.
Finally, we conclude the paper and propose some future work.
II. RELATED WORK
Many researchers used social information to enhance
information retrieval in various methods.
Y. Yanbe et al. [6] tried to combine link-based ranking metric
(PageRank) with another metric derived using social
bookmarking data (SBRank). They also used metadata from
social bookmarking to enable more search capabilities such as
temporal search. That increases the precision of a standard
link-based search by incorporating popularity estimates from
aggregated data of bookmarking users. Their approach was
non-personalized. Their framework enhanced results but
depended on only two source for analysis. Their evaluation
used analytical studies of social bookmarks as well as
comparative analysis between PageRank and SBRank. Another
non-personalized approach was in S. Bao et al. [7] where the
authors used two factors for ranking pages; SocialSimRank
(SSR) which calculates the similarity between social
annotations and web queries and SocialPageRank (SPR) which
captures the popularity of web pages. They used Mean Average
Precision (MAP) and Normalized Discounted Cumulative Gain
(NDCG) to evaluate their results. However, both previous
approaches depend on social bookmarking only although many
other sources of social data are available. Personalized
approaches discussed in [8] and [9] used publicly available data
from Delicious to build profiles for users which can be used to
personalize results for that user. Unfortunately when trying to
gather social info of a particular person from different sources
it is hard and almost impossible for researchers. Tracking users
publicly through different social platforms is not available for
researchers to make better personalization for some user.
Nonetheless, it is increasingly becoming popular to for
example use your Facebook, Twitter or Google account to
login into other social platforms such as Delicious or
StumbleUpon and others. Also it is possible to add other social
platform accounts to your Google+ profile. Recently, a
research by I. Badache et al. [10] used social signals from
different sources to enhance search. Their dataset was collected
from IMDb which contains movie resources where as our
dataset is a general dataset from ODP. Their experimental
evaluation shows that the integration of social properties within
a textual search model allows to enhance the quality of the
search results.
S. Xu et al. [8], used a personalized rank derived from the
users tagging and another rank based on the query matching
with documents then aggregated two ranked lists into the final
ranked list and presented it as the final result. They also used
Delicious as a source for their data as well as Dogear (an
enterprise social bookmarking system). They used ODP as a
reference model. MAP was used mainly to evaluate their
results.
III. PROPOSED IR MODEL
A. Problem Definition
As a result of information revolution, users couldnt easily
get the information they need. Here we can see the great role of
Search Engines, which help users in retrieving the information
they need. Search engines such as Google, Yahoo and others
are helpful in finding the users desired information but most of
them use the content of documents mainly to match a query or
users need and the link structure between documents to rank
the results among other unpublished proprietary factors. As
stated in [3] info generated by the crowds is of value and needs
to be considered when retrieving webpages for users which is
what we consider in our paper.
B. Proposal Overview
The following figure (Fig. 1) presents the framework for
the proposal which uses social information from different
sources to enhance matching and ranking. It shows the main
components of the proposed system
The framework consists of two main parts, online part that
focuses on the interaction between the user and the system in
real time, and offline part that is concerned with the gathering
of social information for the indexing and ranking processes.
1. Online part:
The online part comprises the query or information needed
by the user as well as the re-ranking phase. Using the social
ranker, results of the search engine are re-ranked to bring about
the needed documents that fulfill the user need.
Fig. 1 Proposed IR Model framework
2. Offline part:
The offline part comprises many parts to store, index and
process the document collection as well as the social info
retrieved from the Web. After collecting documents from
whatever source from the Web, the system invokes a social
85
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

info crawler and parser to collect all social info available about
these documents from different social media services as score q, d ,
indicated in Fig. 1. All social info collected is processed and
stored in a database for later use. The document collection is (1)
then processed by the search engine for indexing and relative
social info is added to the index from Delicious tags entered by
,
users and sub-reddits (a form of categorization in a social news
website called Reddit- also chosen by users).
Where ti: query term,
1) IR model improvement
tfd (ti): term frequency in document =
In this solution document expansion has been done by
adding all tags associated with a web document from Delicious . ,
(a social tagging service), also a social news website called idf (ti): inverse document frequency for documents
Reddit contains a categorization of topics called sub-reddits
which have been added also as representative of the document. containing ti 1 ,
To further investigate the effects of expanding documents in
this manner seven query methods were implemented: docFreq (the number of documents in which the term ti
appears),
1. Querying using only information included in the getBoost (ti): boost value associated with a query term which
document (Title, URL, content) referred to as can be assigned when boosting a field at query time,
DocumentInfo. coord (q, d): is a score factor based on how many of the query
2. Querying using DocumentInfo in addition to information terms are found in the specified document.
from Delicious only referred to as DocumentInfo,
Delicious. 1
3. Querying using DocumentInfo in addition to information (2)
from Reddit only referred to as DocumentInfo, Reddit.

4. Querying using DocumentInfo in addition to information
from Delicious as well as Reddit referred to as For multi-field multi-term-query scoring formula is:
DocumentInfo, Delicious, Reddit. score q, d ,
5. Querying using DocumentInfo in addition to information
from Delicious only giving information in Delicious
more importance when finding a result referred to as , , (3)
DocumentInfo, Delicious, Boosted.
6. Querying using DocumentInfo in addition to information ,
from Reddit only giving information in Reddit more ,
importance when finding a result referred to as And queryNorm is calculated as
DocumentInfo, Reddit, Boosted.
7. Querying using DocumentInfo in addition to information 1
from Delicious and Reddit giving information in both (4)
Delicious and Reddit more importance when finding a
result referred to as DocumentInfo, Delicious, Reddit, ,

Boosted. ,
2) Re-ranking of the retrieved documents b) Social ranking
Re-ranking of retrieved documents occurs after the search All social information for a web page is consolidated into a
engine retrieves relevant documents from the index. This single metric Ws using weighted mean formula as follows in (5)
means the retrieved results are only reordered. There is the [12]:
ranking Lucene performs (textual ranking) and social ranking
and a combination of the two ranks (textual and social). (5)

a) Textual ranking (Lucene Score) where n is the number of social services used,
Textual rank is done using Lucene practical score formula
in [11]. This is the default formula in Lucene that uses a is the score calculated for the social service and
combination of the (term frequency / inverse document is the weight given to each social service which was
frequency - tf/idf) vector space and Boolean models for chosen as 1. In addition, for each social service the score is
scoring. Suppose you have a document d that consists of a calculated using the score of components of the social service
number of fields k {f1, f2, ,fk} and a query q which consists of (such as like or share in Facebook) using an equation just like
m terms { t1, t2, ,tm}, then the score that q get when related to (5) for which the weights are chosen as in Table I.
d ( if the document has one field ) is
Score calculation uses social information available from
different sources in two different ways:

86
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

(1) Non - weighted: all information is included from

different components of a social service for example
Facebook: share, like.
(2) Weighted: different components of a social service
have different weights for example Facebook share
count has more weight than like count for a certain web
page.
Components of the social services and their weights are
listed in Table I
TABLE I. SOCIAL SERVICES COMPONENTS AND THEIR WEIGHTS

Social Service Component Weight

Likes count 2
Facebook
Share count 4
Twitter Posts count 1
Total tags count 1
Delicious
Posts count 1 Fig. 2. Flow chart of the offline part of the algorithm
User lists 3
StumbleUpon
User likes 1
GooglePlus Likes count 1
LinkedIn Posts count 1
Total score sum 3
Reddit
Posts count 1
c) Combining textual and social rank
Finally re-ranking is done using only social information as
well as a combination between the textual rank (rt) and the
social rank(rs) resulting in a new aggregated rank (rt,s). This is
accomplished using rank aggregation using Weighted Borda-
Fuse (WBF) where in (6) as in [8].

, 1 (6)
3) Description of the proposed algorithm
Our proposed algorithm is constructed mainly to use social
information on the web to enhance web retrieval. The
algorithm is divided into offline and online parts. The
construction of each of them is described in the following
figures.
The flow chart in Fig. 2 shows the steps of processes done
in the background with no interaction with the user. It first
starts with getting social information by using the URLs of
cached web documents and requesting and retrieving all
possible data from different sources on the web. Then the
retrieved information is stored in a database for later use by the
system. To extract and use this data - which is usually found in
either JSON (Java Script Object Notation) format or RSS/XML
(Really Simple Syndicate/Extensible Markup Language)
format - parsers are used to split data into usable fields in the
database.
Fig. 3 Flow chart of the online part of the algorithm
Indexing documents then happen by including data from
Delicious and Reddit (from the database) as well as the
document itself in different fields in the index. All documents
indexed are stemmed (broken into terms) using Porter
Stemmer. In addition, a social rank is calculated using social
information retrieved for every document then stored in the
database. In Fig. 3, the user submits the query to the system
and the system searches the index for the information need and
retrieves the results which is then re-ranked using the social
information stored in the database.
IV. DATASET AND EVALUATION METHODOLOGY
A. Dataset description
The dataset used in this work - as a document collection
was collected from the DMOZ Open Directory Project
(ODP)8 during March 2014. ODP is the biggest, most
comprehensive directory of the web maintained by humans.
This high quality and free web taxonomy resource has been
8
http://www.dmoz.org

87
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
used in a number of previous researches for different purposes
such as topic extraction in [8], [13], finding relative pages ages
in [14], as a reference model in [7].
In this paper ODP is used as a reference model using the
same way as in [7]. Category paths were used as queries and
their corresponding URLs as the ground truth. A collection of
randomly selected 3960 categories were collected as queries of
which only 2439 were actually used and the rest has been
ignored.
Queries have been filtered as follows:
1. Queries for which relevant URLs had no social
information and/or no longer exist on the web were
ignored.
2. Because ODP categorizes some URLs alphabetically,
queries that had a capital alphabetic letter as the last
category were removed and if a letter appeared within
the query, the letter alone was removed.
3. Also, any duplicated words in the query were removed.
4. To ensure measurable quality, all queries must have at
least 10 relevant results.
5. Finally, all non-English queries were removed.
A total of 79275 URLs were used in the experiment. All of
them have social information from at least one of the used
social information sources and has a web document with a title
and html content. The web documents and all social info have
been downloaded using a custom C# application. Table II
shows the coverage percentage of social info for the dataset
URLs used in the experiment.
TABLE II. SOCIAL INFORMATION COVERAGE OF THE USED DATASET
Social Info Coverage
Source Percentage %
Facebook 80.944
Twitter 55.378
Delicious 52.621
StumbleUpon 50.523
GooglePlus 34.597
LinkedIn 28.868
Reddit 4.833
A. Proposal Evaluation Methodology
To evaluate the effectiveness of using social info to
enhance information retrieval process a framework was built
using C# and Lucene .Net (the .Net port of an Open Source
search engine called Lucene - originally written in Java). The
evaluation was executed in stages numbered and shown in Fig.
4.
The evaluation framework presented in Fig. 4 is numbered
to show the sequence used in the evaluation process. Phase 1
has input of data from ODP as categories and subcategories
with each category containing a set of URLs relative to that
category. Queries, URLs and reference dataset were extracted
from ODP categories. Then in phase 2 social services info is
queried from the web using the URL dataset and stored in a
database. In addition the document itself is also downloaded
and cached. Parsing all social info into usable form is done in
phase 3. To calculate social ranks phase 5 uses parsed social
info and stores results in social ranks database. Fifth phase is
responsible for indexing web documents cached from phase 2
and adding social info from Delicious and Reddit only as fields
in a Lucene document. Queries are executed using the seven
methods mentioned in IR model improvement in phase 6 and
only the first 100 results for each query are stored for the final
phase where they are re-ranked using five methods:
1. No re-ranking (Only results from querying and ranking
using Lucene tf/idf model).
2. Re-ranking by weighted mean rank method using the
social scores calculated by (5) and weights from Table I
and combining the rank of Lucene and social score using
(6).
3. Re-ranking by mean rank method using the social scores
calculated by (5) and weights of 1 for each component
and combining the rank of Lucene and social score using
(6).
Finally, in phase 7 evaluation metrics are calculated from
the results and final comparison result is extracted.
B. Evaluation Metric
To evaluate the system Mean Average Precision (MAP)
metric is used as defined in [15].
1) Precision
Precision (P) is the fraction of retrieved documents that are
relevant and is expressed as a percentage.
# (7)
#
For ranked results, Precision @k is used to represent
precision when only inspecting the first k items in the results,
calculating how many relevant items in the first k items
retrieved.
2) Mean Average Precision (MAP)
Most standard among the TREC (Text REtrieval
Conference) community is MAP, which provides a single-
figure measure of quality across recall levels. Among
evaluation measures, MAP has been shown to have especially
good discrimination and stability. For a single information
need, Average Precision is the average of the precision value
obtained for the set of top k documents existing after each
relevant document is retrieved, and this value is then averaged
over information needs. That is, if the set of relevant
documents for an information need qj Q is {d1, . . . dmj} and
Rjk is the set of ranked retrieval results from the top result until
you get to document dk, then
| |
1 1 (8)
| |
C. Evaluation Results and Discussion
Evaluation was performed using a Windows PC whose
specifications are as follows : ( CPU : Intel Core i7 3.9GHz
Quad Core with hyper-threading enabled, RAM: 32GB
DDR3), Microsoft SQL server was used to store all kinds of
information but index which was stored using Lucene.Net as
independent custom files. Code and libraries were used in C#
programming language and parallelism was used to improve
performance and minimize experiment time.
88
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

In this section the results from the evaluation phase (No.7)
in Fig. 4 is presented and discussed. Figures from Fig. 5 to Fig.
7 represent MAP results at different levels (10, 20, 100) for
different re-ranking methods in IR model improvement
Fig. 4. Proposal evaluation framework
paragraph (also included in Table III).
By inspecting Fig. 5, MAP is at max when using
DocumentInfo, Delicious, Boosted query method. This
suggests that Delicious is a very effective source of information
for expanding a document, because usually users add tags that
do represent the content of the document. Note that all tags for
a document are added, hence there could be duplicated tags that
means better matching for a document when the query contains
that tag as a term. Unfortunately because Delicious from Table
II covers only about 53% of the dataset not all documents have
info associated with them. Yet this percentage was enough to
make large enhancement (almost 30-50% over Document
Info), which means most relevant documents had Delicious
info associated and relevant. However, some relevant
documents had either no Delicious info or non-relevant info,
which would push them down the list.
Because Lucene scoring formula punishes not finding info
in a designated field by reducing the score of a document by
means of queryNorm factor in the scoring formula (3).
QueryNorm is calculated by (4) in which the idf factor is
inversely proportional to document frequency (field frequency)
that is zero if no documents (fields) have this term. Hence, as
idf goes higher queryNorm goes lower which causes penalty
effect. In addition, boosting may even make score lower also
because of queryNorm factor in Lucene in (4) which is also
inversely proportional to term boost in a field (getBoost (ti,fi)).
This explains why boosted version of DocumentInfo,
Delicious query method (DocumentInfo, Delicious, Boosted)
has a higher average. As either documents with no Delicious
info or non-relevant info is pushed down the list surpassing
the 100 limit set for the experiment. Using Reddit only as a
representative of a document is not effective enough mainly
because of coverage percentage of 5% from Table II and for
the same reasons as in Delicious boosting make it worse.
Combining it with Delicious lowers precision and MAP also
for the same reasons specially when a match is scored in a
Delicious field and not in a Reddit the score is less than in
Delicious, causing the combination to be lower than Delicious
alone but still higher than using DocumentInfo alone.
Unfortunately using re-ranking methods lowers the score
for all query methods used. The reason is that all social info
fail to capture the rank properly which is probably for many
reasons. Websites used are not all covered completely by every
social service but every website has at least info from one
social service. In addition, many of the websites are either
static or dynamic but has no social plugins. That make it harder
for users to interact and give ratings for pages. Lastly, the point
of view for social services users is probably different from
those made by ODP admins. In collected data, there is no way
for us to normalize data successfully because there is no way to
know how many have dealt with a specific webpage hence the
results are not that accurate.

TABLE I. EXPERIMENT RESULTS

Rank Method None WeightedMean Mean

Query Method MAP@10 MAP@20 MAP@100 MAP@10 MAP@20 MAP@100 MAP@10 MAP@20 MAP@100
DocumentInfo 6.711 8.412 10.832 4.850 6.680 9.250 4.858 6.691 9.256
DocumentInfo, Delicious 8.460 10.675 13.816 5.697 7.899 11.313 5.700 7.908 11.313
DocumentInfo,
9.006 11.205 14.195 5.892 8.077 11.378 5.886 8.074 11.364
Delicious, Boosted
DocumentInfo, Reddit 5.538 6.981 9.146 3.108 4.562 6.956 3.084 4.565 6.940
DocumentInfo, Reddit,
2.992 3.942 5.663 1.418 2.220 4.109 1.406 2.204 4.094
Boosted
DocumentInfo,
7.262 9.148 12.075 4.028 5.900 9.089 4.048 5.899 9.097
Delicious, Reddit
DocumentInfo,
Delicious, Reddit, 5.733 7.330 9.912 3.057 4.540 7.334 3.061 4.526 7.332
Boosted

89
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Fig. 5. MAP results using no rank method and different query methods
Fig. 6. MAP results using weighted mean rank method and
a different query
methods
Fig. 7. MAP results using mean rank method and differeent query methods
V. CONCLUSION AND FUTUREE WORK
This work contributes to the enhancemennt of search model
by incorporating social information from manny sources. As the
results indicated, some modifications actuallyy introduced better
results while others did not. The modificaations that led to
enhanced results were using delicious dataa in the indexing
process as if they were part of the web docuument. In addition,
giving delicious field of the document more significance gives
better results generally and specifically at the first 10 to 20
results. This gave a better ranking in com mparison to other
methods because usually delicious tags arre relevant to the
content of the webpage. Other methods faileed to perform well
as expected because social coverage of the websites
w of the test
is not high enough and the reference useed probably has a
different point of view of the websites than the average users
using social services. Also normalizationn issues are very
difficult to implement perfectly.
To better enhance the results some areass may be explored.
For example, many websites are associatedd with a Facebook
Page (mostly included as a link in the front f page of the
website) from which rank may be extrracted (using for
examples how many likes and the interactionn of people on the
Facebook Page). In addition, the normalization issue may be
addressed by normalizing for the largest valuev of a certain
aspect such as Facebook like or share for a ceertain webpage. As
a future work, the weights selected for the rannking methods can
be calculated using neural network or some other optimization
algorithm along with machine learning rather r than being
chosen by hand.
REFER
RENCES
[1] M. Hilbert, How much inforrmation is there in the information
society?, Significance, vol. 9, no.
n 4, pp. 812, Aug. 2012.
[2] Eb. Inc., Top 15 Most Populaar Search Engines | December 2014.
[Online]. Available: http://ww ww.ebizmba.com/articles/search-engines.
[Accessed: 03-Dec-2014].
[3] J. Surowiecki, The wisdom of croowds. New York: Anchor Books, 2005.
[4] S. K. M. Yi, M. Steyvers, M. D. D Lee, and M. J. Dry, The Wisdom of
the Crowd in Combinatorial Prooblems, Cognitive Science, vol. 36, no.
3, pp. 452470, Apr. 2012.
[5] A. Muralidharan, Z. Gyongyi, and a E. Chi, Social annotations in web
search, in Proceedings of the 20012 ACM annual conference on Human
Factors in Computing Systems - CHI C 12, 2012, p. 1085.
[6] Y. Yanbe, A. Jatowt, S. Nakkamura, and K. Tanaka, Can social
bookmarking enhance search in the web?, in Proceedings of the 2007
conference on Digital libraries - JCDL 07, 2007, p. 107.
[7] S. Bao, G. Xue, X. Wu, Y. Yuu, B. Fei, and Z. Su, Optimizing web
search using social annotatioons, in Proceedings of the 16th
international conference on Woorld Wide Web - WWW 07, 2007, pp.
501510.
[8] S. Xu, S. Bao, B. Fei, Z. Su, and a Y. Yu, Exploring folksonomy for
personalized search, in Proceeedings of the 31st annual international
ACM SIGIR conference on Research and development in information
retrieval - SIGIR 08, 2008, p. 1555.
[9] M. R. Bouadjenek, H. Hacid, andd M. Bouzeghoub, Sopra: a new social
personalized ranking functionn for improving web search, in
Proceedings of the 36th interrnational ACM SIGIR conference on
Research and development in info formation retrieval - SIGIR 13, 2013, p.
861.
[10] I. Badache and M. Boughanem, Harnessing
Social Signals to Enhance a
Search, in 2014 IEEE/WIC/AC CM International Joint Conferences on
Web Intelligence (WI) and Intellligent Agent Technologies (IAT), 2014,
pp. 303309.
[11] Similarity (Lucene 3.0.33 API). [Online]. Available:
http://lucene.apache.org/core/3_00_3/api/core/org/apache/lucene/search/S
imilarity.html. [Accessed: 15-Febb-2015].
[12] G. R. PRICE, Extension of covvariance selection mathematics, Annals
of Human Genetics, vol. 35, no. 4,4 pp. 485490, Apr. 1972.
[13] P. A. Chirita, W. Nejdl, R. Paaiu, and C. Kohlschtter, Using ODP
metadata to personalize search,, in Proceedings of the 28th annual
international ACM SIGIR conferrence on Research and development in
information retrieval - SIGIR 055, 2005, p. 178.
[14] P. Heymann, G. Koutrika, H. Garcia-molina, I. T. Report, L. U.
November, G. Koutrika, annd H. Garcia-molina, Can social
bookmarking improve web seearch?, in Proceeding WSDM 08
Proceedings of the 2008 Internaational Conference on Web Search and
Data Mining, 2008, pp. 195206..
[15] C. D. Manning, Prabhakar Raghaavan, and H. Schtze, An Introduction to
Information Retrieval. Cambridge University Press, 2008.
90
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Efficient Dependable Rules Generation Approach for
Data Quality Enhancement
Asmaa S. Abdo, Rashed K. Salem, and Hatem M. Abdul-Kader
Information System Department, Faculty of Computers and Information, Menoufia University, Egypt
{engasmaa.saad.mis, rashed.salim, hatem.abdelkader}@ci.menofia.edu.eg
Abstract In area of data quality research, enhancing data
quality is still big challenge, especially in large databases. Data
mining techniques can be efficiently utilized in data cleaning
process. Databases are often suffered from data inconsistency,
which has no vital solution up to now. In this paper, we tackle the
problem of detecting data inconsistency from large databases.
We propose an approach for discovering dependable rules from
databases themselves. Such generated rules are minimal and
non-redundant that covers all rules among patterns in database.
The proposed approach focuses mainly on generating precise
dependable data quality rules through extracting maximal fre-
quent patterns, as effective pruning mechanism to reduce the
search space domain. The proposed approach is validated against
several datasets from different application domains. Experimen-
tal Results demonstrate that our approach outperform other
approaches in terms of the efficiency, accuracy and scalability
using both real-life and synthetic datasets.
Keywords Data Quality, Data Mining, Data Cleaning, Func-
tional Dependency (FD), Conditional Functional Dependency
(CFD), Association Rule Mining (ARM), Maximal Frequent Pat-
terns (MFP).
I. INTRODUCTION
Business and Scientific data are most important value
among both public and private organization, which are ex-
ploited for management purposes [1]. However, real-life data
are often dirty holding inconsistencies, conflicts and errors.
The value of data highly depends on its degree of quality. Hen-
ceforward, the presence of errors and inconsistencies in data
decreases their value, making them insignificant, or even harm-
ful.
The consequences of poor data quality are serious problems on
result of data analysis, they often arise due to modifications in
usage of data and violation of integrity constraints (or deficien-
cy of these constraints). Such poor data quality leads to incor-
rect statistics, which results in inefficient daily tasks, wasting
of time and money, and poor business decisions [2], [3].
Data residing in databases is called to be dirty if it holds incon-
sistencies with respect to some set of constraints. Dirty data
costs US business billions of dollars annually that issue poor
decisions making resulting from the poor data quality [4].
Therefore, its very important task for detecting data inconsis-
tency in the data cleaning process. Doubtless, ensuring high
quality dependable data is a competitive advantage to all busi-
nesses, which requires precise data cleaning solutions [5], [6].
978-1-5090-0217-7/15/$31.00 2015 IEEE
Data cleaning (or data scrubbing) is considered as preliminary
and important phase of data preprocessing, i.e., essential part of
data mining [7]. It works for maintaining corrupted and/or
erroneous records. This phase is mandatory in data manage-
ment cycle before mining and analyzing data [23].
Indeed, data cleaning methods in the literature focus on record
matching [10], which match master cleaned records with prob-
ably erroneous records. It is still necessary to tackle the prob-
lem of data inconsistency with the help of data themselves and
without the need for external master copy of data. During re-
solving data inconsistencies, several integrity constraints are
ensured, e.g., Functional Dependencies (FDs), Conditional
Functional Dependencies (CFDs) [20, 28]. Identifying incon-
sistent values is a major step in the data cleaning process.
Currently, growing data in most application domains considers
a big challenge for ensuring consistent and accurate quality
data, to be employed for data management purposes. Such
most of these applications suffers from inconsistent and dirty
data problems. The incorrect data recorded electronically lead
to poor quality data. Herein, we are interested to generate de-
pendable accurate data quality rules, which then used for re-
solving data inconsistencies in several application domains.
The problem is formulated as follows: Given an instance r of a
relation schema R, support s, and confidence c thresholds. The
proposed approach firstly extract maximal frequent patterns in
r and their associated generators at defined support threshold.
Secondly, it generates dependable data quality rules with de-
fined confidence threshold, extracted patterns in r, retaining
interest measure greater than 1. The generated output rules are
minimal interest and non-redundant constant conditional func-
tional dependencies that cover all set of rules.
The main contribution of this paper is to propose an approach
for enhancing performance of generating data quality rules
based on extracting maximal frequent patterns and their asso-
ciated generators. Therefore, maximal frequent patterns reduce
number of patterns generated and search space domain. The
proposed approach utilizes data mining techniques for generat-
ing dependable rules, which mainly rely on extracting patterns
that speed up rules generation process. Also, the proposed
approach employs Interest (lift) measure as a filter to generate
accurate dependable interest data quality rules, and it prunes
other rules that are not statistically significant nor meaningful.
Such generated rules are exploited not only for detecting in-
consistent data, but also for correcting them.
91
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

The experimental results of the proposed approach compared III. PRELIMINARIES

with CCFD-ZartMNR algorithm [11] are conducted over dif-
There are several terminologies from the literature are related
ferent application domains. We verify that the proposed ap-
proach scales well with varying the dataset size, number of to the proposed approach. The definitions of such terminolo-
attributes and the minimal number generated data quality rules. gies are provided, including functional dependencies, condi-
tional functional dependencies, closed frequent patterns, max-
This paper is organized as follows: Section II discusses re- imal frequent patterns and their associated generators, constant
lated works. In section III, we present some notations which CFD problem, and pruning search space.
will be utilized by the proposed approach. Section IV introduc-
Functional Dependencies (FD): defined as constraint be-
es the proposed approach for generating dependable data clean-
ing rules. Section V discusses the experimental study and re- tween two sets of attributes in relation from a database. FD
sults conducted for different datasets from different application over Relation(r) represented as X Y where each X value is
domains. Finally, section VI concludes the proposed work and associated with precisely one Y value. Herein, X is considered
highlight future trends. as the determinant and Y is considered as the dependent. Func-
tional dependencies employed for working on schema design
II. RELATED WORK that not capture semantic of data for data cleaning [11].
Despite the urgent necessity for dependable techniques to Conditional Functional Dependencies (CFD): an extension
enhance data quality and data cleaning issue, this issue has not of FDs aims to detect inconsistencies of data between tuples in
vital solution up to now. Indeed, there have been little discus- single relation since CFD on relation r is a pair (X Y, tp),
sions and analysis for enhancing data inconsistency. To the where X Y is a standard Functional Dependency (FD) on r;
best of our knowledge, most of literature focus on record and tp is the pattern tuple of with attributes in X and Y. For
matching and duplicate detection [8]. each attribute A in X Y, tp[A] is either a constant in the
Database and data quality researchers have discussed variety domain of A, or an unnamed variable _ [6].
of integrity constraints based on Functional Dependencies
(FD) [9][12]. These constraints are developed essentially for Example 1: In this example, we address records from custom-
schema design, but are often not able to embed the semantic of er relationships have the following attributes Country Code
data for data cleaning. (CC), Area Code (AC), Phone Number (PN), Name (NM),
Other researchers focus on extension of FD, they have pro- Street (STR), City (CT) and Zip Code (ZIP). It explains the
posed Conditional Functional Dependencies (CFD) and Con- concept of functional dependency, and conditional functional
ditional Inclusion Dependencies (CID). Such integrity con- dependency.
straints are employed for capture errors in data [13][15]. TABLE 1. RECORDS FROM CUSTOMER RELATION
Moreover, several data quality techniques are proposed to CC AC PN NM STR CT ZIP
clean missy tuples in databases [16]. Statistical inference ap- t1 40 872 6666666 Mike Port PI UN 08422
proaches are studied in [17]. These approaches tackle missing t2 40 872 6666666 Rick Port PI UN 08422
values in order to enhance quality of data. From technological t3 40 101 9999999 Joe M.V.P EDI 01202
part, there are several open source tools which developed for t4 40 872 9999999 Jim Majestic UN 08422
handling messy data [18]. t5 32 222 4444444 Ben 6rd Str VIZAG EH4 1DT
t6 32 222 7777777 Ian 6rd Str VIZAG EH4 1DT
Besides, data transformation methods such as commercial t7 32 872 7777777 Ian XXX UN W1B 1JH
ETL (Extract, Transformation and Loading) tools are devel- t8 40 131 9999999 Sean 5rd Str MH 01332
oped for data cleaning [19].
The usage of editing rules with master data is discussed in
[20]. Such rules are able to find certain fixes by updating input Traditional FD that hold on Table 1 is
tuples with master data. However, this approach requires de- f1: [CC, AC] CT
fining editing rules manually for both relations, i.e., master f2: [CC, AC, PN] STR
relation and input relation, which is very expensive and time- f1 mean that if there are two customers that have the same
consuming. country and area code, they also have the same city, similarly
Furthermore, lots of work are proposed in the literature relying for f2.
on domain specific similarity and matching operators, such The following are CFDs that hold on table 1 is
works include record matching, record linkage, duplicate de- 0: ([CC, ZIP] STR, (32, -|| -))
tection, and merge purge [8], [21], [22]. These approaches 1: ([CC, AC] CT, (40, 872 || UN))
define two functions; namely match and merge [23]. While 2: ([CC, AC] CT, (32, 222 || VIZAG))
match function identifies duplication of records, the merge 3: ([CC, AC] CT, (40, 101 || EDI))
function combines the two duplicated records into one. Rules are classified to 0 as variable CFD, where 1, 2, 3 are
From the literature, we reutilized CFD as a special case of constant CFD. In this work, we are concerned with such type
association rules [14], [24]. The relationship between minimal of constant CFD. CFD 0 asserts that zip uniquely determines
constant CFD and association rule as similar relationship to STR. This FD hold on subset of tuples with the pattern
CFD that work on minimal non redundant rules [25]. CC=32, rather than on the entire relation. CFD 1 assures that
for any transaction tuple if customer country code=40 with

92
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
area code 872, then the customer city is UN. Similarly for 2
and 3 these cannot represent as FDs.
Closed Frequent patterns: Pattern is frequent closed if it is
not included in a proper superset having the same support. A
generator Y of a frequent closed pattern X, is a pattern con-
straint with it has the same support as X, and it does not have
any subset having the same support [14].
Maximal Frequent patterns: Pattern is maximal frequent if
none of its immediate supersets is frequent. The set of fre-
quent maximal patterns is thus a subset of the set of frequent
closed patterns, which is a subset of frequent patterns [26].
Constant CFD Problem: The problem is to discover minimal
set of frequent constant CFDs that include non-redundant CFD
[2], [27]. Example 1 has 1, 2, 3, which are constant CFD.
Pruning Search Space: Pruning is to remove infrequent
nodes from search space using predefined support value [2].
IV. PROPOSED WORK
The proposed approach focuses on generating precise depend-
able interest minimal and non-redundant constant CFD data
quality rules, which cover all set of rules with respect to speci-
fied support and confidence thresholds. Such generated rules
are exploited for detecting and repairing data inconsistencies
from large datasets. Since, the overall performance of rules
generation is based on fast extraction of patterns from data.
The proposed approach named minimal interest constant con-
ditional functional dependencies miner abbreviated as
(MICCFD-Miner). This mainly based on mining maximal
frequent patterns and their associated generators, applying the
effective pruning mechanism to reduce the search space do-
main. Maximal frequent patterns (MFP) are usually much
smaller than the set of frequent patterns and also smaller than
set of frequent closed patterns. In other words, maximal fre-
quent patterns closed frequent patterns frequent patterns.
MFP patterns are called maximal because they have no fre-
quent supersets. Moreover, the set of these patterns is a mi-
nimal set, i.e. the smallest set from which all frequent patterns.
The proposed approach is depicted in figure 1.
Figure 1: The proposed MICCFD-Miner approach
The main input of MICCFD-Miner are dataset and two prede-
fined thresholds, i.e., minimum support, and minimum confi-
dence. The first step is to extract list of maximal frequent pat-
terns and their associated generators by using minimum sup-
port threshold. MICCFD-Miner approach then utilizes such
extracted maximal frequent patterns instead of working on
discovering all frequent patterns or closed frequent patterns,
reducing the size of search space domain shown in figure 2.
Figure 2: Search Space Domain
Support CFD : (X A) is defined as the percentage of the
number of records in the dataset that contain X A to the total
number of records in the database. For example, the support of
1 in example 1 (Section 3) is 3/8 because three tuples satisfy
1. Support of a CFD : (X A) where X generator pattern
and A is (maximal /generator) [28], defined as:
N X A
support X A (1)
T
The second step depends on extracted patterns and their asso-
ciated generators from the first step. Then, the minimum con-
fidence threshold is utilized to generate the set of interest min-
imum non redundant constant CFD data quality rules. While
the literature utilizes only support and confidence for generat-
ing such rules, we consider interest (or lift) measure into ac-
count for generating more dependable and reliable rules.
The form of rules for each frequent generator pattern X finds
its proper supersets A from set of frequent maximal patterns.
Then, from X and A add rule antecedent (Generator) conse-
quence (maximal/generator) as : X A.
Confidence CFD is calculated as the number of records in the
dataset that contain X A divided by number of records that
contain X. For example, confidence of 1 in example 1 (Sec-
tion 3) is 1.0, defined as CFD : (X A) is
XA
confidence X A (2)
Confidence measures the reliability of rules, since the value of
confidence is real number between 0 and 1.0 [29]. However,
the pitfall of confidence is that ignores support of right hand
side of rules. As consequence, we add data quality measure
called Interest (Lift) which generates more dependent rules
when reserving it as greater than 1.
Lift CFD tries to measure the degree of compatibility of left
hand side and right hand side of rules as, i.e., occurrence of
both left hand side and right hand side [30]. We set here Lift
value greater than 1 to obtain dependent MICCFD-Miner
rules. For example, lift of this CFD rule : (X A) is
f
lift X A
A
(3)
This approach optimizes the process of the rules generation
compared with the most related methods.
93
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

V. EXPERIMENTAL STUDY
We present an experimental study for validating the proposed
approach. the proposed approach is evaluated w.r.t three di-
mensions: (1) Efficiency; extracting maximal frequent patterns
that results in reduction in the number of patterns generated
and space complexity without loss of information when gene-
rating dependable rules. (2) Accuracy; generating accurate
dependable rules, which are minimum and non-redundant. (3)
Scalability; measuring response time when increasing both
database size and number of attributes.
A. Experimental Setting
The experiments are conducted using both real-life and syn-
thetic datasets. Such datasets are taken from the UCI machine
learning repository (http://archive.ics.uci.edu/ml/) namely,
Mushroom, Thyroid (hypothyroid), Adult, Audiology. Table 2
shows the number of attributes and the number of instances
for each dataset.
equal to 0.4 and confidence =1, the number of maximal fre-
quent patterns extracted from the proposed approach is 41
patterns, but number of closed patterns generated from other
approach is 140. Such results show the effectiveness in ex-
tracting maximal frequent patterns as first step in generating
accurate and dependable rules instead of extracting closed
frequent patterns as current existing approaches. Focusing on
maximal frequent patterns also optimize the memory space,
which shown in figure 4. In this example, the usage memory
space of the proposed approach equal to 11.1 MB, but the
usage memory space of other approach equal to 12.24 MB.

TABLE 2: DATASETS DESCRIPTION

Size (Number of Arity (Number
Dataset name Instances) of Columns)
Mushroom 8124 23
Thyroid(hypothyroid) 3772 30
Adult 32561 15
Audiology 226 70
Figure 3: Total number of maximal frequent patterns
All our experiments were implemented using java (JDK1.7). extracted from mushrom dataset.
The implementation is tested on machine equipped with In-
tel(R) Pentium(R) Dual CPUT3400 with 2.00 GB of memory
running on windows 7 operating system. The proposed ap-
proach runs mainly in main memory. Each experiment is re-
peated at least five times and the average reported here.
B. Experimental Results
We next demonstrate and discuss the results from experimen-
tal study over the proposed approach against CCFD-ZartMNR
algorithm [24]. The experiments are conducted for different
dataset from different application domains described in pre-
vious section.
Experiment-1: Efficiency of extracting patterns. We here in
evaluate the efficiency of extracting maximal frequent patterns
Figure 4: Space Complexity measure about mushrom dataset.
from the proposed approach against closed frequent patterns
from CCFD-ZartMNR [24]. The results for mushroom dataset Experiment-2: Accuracy of rules generation. In this expe-
are shown in figures 3, 4. This dataset contains 23 attributes, riment, we validate accuracy of rules generated from the pro-
and 8124 records about mushroom physical characteristics. posed approach compared to CCFD-ZartMNR algorithm [24].
Figure 3 show the efficiency in reduction the number of pat- Results from thyroid (hypothyroid) diagnosis dataset is shown
terns generated form the proposed approach (MICCFD-Miner) in figures 5, 6. This dataset contains 30 attributes, and 3772
over other approach without loss of information. The x-axis is records of patient data describing patient information about
the different support values and y-axis is the total number of the hypothyroid diagnoses data. As shown, figure 5 confirm
extracted patterns, and set fixed constant confidence value effectiveness of the accurate minimal non-redundant data
equal 1. Figure 4 demonstrate the space complexity optimiza- quality rules generation of the proposed approach over other
tion of the number of patterns extracted from the proposed approach for detecting inconsistency problems. The proposed
approach against other approach. The x-axis is the support approach always generate less numbers of rules but more ac-
values and y-axis is the space complexity measure in mega- curate as compared to other approach, because the other ap-
bytes (MB), at fixed constant confidence value equal 1. For proach does not consider interest measure of rules. The x-axis
example in figure 3, with thresholds minimum support value is minimum support threshold values and y-axis is the number

94
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

of rules generated, and set fixed constant confidence value
equal 0.99. Figure 6 also show that the proposed approach
outperforms the other algorithm with respect to response time
for rules generations. The x-axis is minimum support thre-
shold values and y-axis is the response time in seconds (s), and
also set fixed constant confidence value equal 0.99.
milliseconds (ms). This experiment concludes that
the proposed approach outperforms other approach in
scalability of increasing number of tuples with less
response time.

Figure 7: Scalability per number of tuples on adult dataset.

Figure 5: Numbers of rules generated from thyroid diagnoses dataset.
b) Scalability of the proposed approach with increas-
ing arity of relation. We asses performance of sca-
lability on increasing number of attributes at fixed
number of tuples over the proposed approach against
CCFD-ZartMNR algorithm. Results from audiology
dataset are depicted in figure 7. This dataset contains
226 tuples and 70 attributes describing audiology
disease data about patients. Fixing database size=226
tuples, confidence=1, minimum support=0.99. We are
changing arity from 10 to 70. Figure 8 shows the re-
sponse time of the proposed approach against other
approach by varying number of attributes. The x-axis
is the number of attributes starting from 10 to 70 and
Figure 6: Response time measure of generated rules
from thyroid diagnoses dataset y-axis is the measure of response time in milliseconds
(ms). Such number of attributes increased signifies
Experiment-3: Scalability of the proposed approach with that the proposed approach scales well in less re-
the dataset size and arity of relation. In order to evaluate the sponse time compared to other algorithm.
scalability of the proposed approach against CCFD-ZartMNR
algorithm [24] by varying dataset size, and arity of relation.
a) Scalability of the proposed approach with varying
database size of relation. In this experiment, we
evaluate the scalability of the proposed approach
when increasing number of tuples at fixed number of
attributes against CCFD-ZartMNR algorithm. Results
from adult dataset is shown in figure 7. This dataset
contains 32561 tuples and 15 attributes describing
prediction about whether income exceeds $50K/yr or
not based on census data. Fixing arity=15, confidence
= 0.95 and minimum support= 0.5. We are changing
database size (number of tuples) from 5k to 30k
tuples. The response time of the proposed approach Figure 8: Scalability per number of attributes on audiology dataset.
against other approach by varying number of tuples
are reported in figure 7, which shows linear time VI. CONCLUSIONS AND FUTURE WORK
scaling of our approach over that always less than the We have proposed MICCFD-Miner, an approach that generat-
other approach. The x-axis is the number of tuples ing accurate minimum dependable interest data quality rules,
*1000 and y-axis is the measure of response time in as solution to data inconsistency problems. The proposed ap-

95
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
proach focus mainly on extracting maximal frequent patterns,
as effective patterns for generating dependable data quality
rules, which yields a promising method for detecting semantic
data inconsistency. The main goal of the proposed approach is
keeping the database in consistent state. Generated rules are
exploited as data cleaning solution to resolve inconsistency
problem in current different application domains specially that
have large databases. MICCFD-Miner relies on lift measure in
addition to support and confidence metrics for generating
depend-able minimal complete and non-redundant rules that
covers all sets of rules. The MICCFD-Miner is validated and
evaluated over both real life and synthetic datasets from dif-
ferent application domains. The experimental results confirm
the effective-ness and usefulness of the proposed approach
against CCFD_ZartMNR algorithm. The proposed approach
performs well across several dimensions such as scalability,
number of rules generated, response time, and space com-
plexity. Finally, we plan to investigate a technique for fixing
errors autonomously with generated rules from MICCFD-
Miner.
REFERENCES
[1] M. Mezzanzanica, R. Boselli, M. Cesarini, and F. Mercorio,
Automatic Synthesis of Data Cleansing Activities, 2011.
[2] J. Li, J. Liu, H. Toivonen, and J. Yong, Effective pruning for the
discovery of conditional functional dependencies, Comput. J.,
vol. 56, no. 3, pp. 378392, 2013.
[3] M. Yakout, A. K. Elmagarmid, and J. Neville, Ranking for data
repairs, Proc. - Int. Conf. Data Eng., pp. 2328, 2010.
[4] W. Fan, J. Li, S. Ma, N. Tang, and W. Yu, Interaction between
record matching and data repairing, Proc. 2011 Int. Conf. Manag.
data - SIGMOD 11, vol. 1, no. 1, p. 469, 2011.
[5] J. Wang and N. Tang, Towards dependable data repairing with
fixing rules, SIGMOD Conf., pp. 457468, 2014.
[6] W. Fan and F. Geerts, Foundations of Data Quality
Management, Synth. Lect. Data Manag., vol. 4, no. 5, pp. 1217,
2012.
[7] P. Srivastava, Mitali and Garg, Rakhi and Mishra, Analysis of
Data Extraction and Data Cleaning in Web Usage Mining
Categories and Subject Descriptors, in Proceedings of the 2015
International Conference on Advanced Research in Computer
Science Engineering \& Technology (ICARCSET 2015), 2015.
[8] D. Bharambe, S. Jain, and A. Jain, A Survey : Detection of
Duplicate Record, vol. 2, no. 11, 2012.
[9] G. Cong, W. Fan, F. Geerts, X. Jia, and S. Ma, Improving data
quality: consistency and accuracy, Proc. 33rd Int. Conf. Very
large data bases, vol. Vienna, Au, pp. 315326, 2007.
[10] S. Hartmann, M. Kirchberg, and S. Link, Design by example for
SQL table definitions with functional dependencies, VLDB J.,
vol. 21, no. 1, pp. 121144, 2012.
[11] H. Yao and H. J. Hamilton, Mining functional dependencies from
data, Data Min. Knowl. Discov., vol. 16, no. 2, pp. 197219,
2008.
[12] J. Liu, J. Li, C. Liu, and Y. Chen, Discover dependencies from
data - A review, IEEE Trans. Knowl. Data Eng., vol. 24, no. 2,
pp. 251264, 2012.
[13] P. Bohannon, W. Fan, F. Geerts, X. Jia, and A. Kementsietsidis,
Conditional functional dependencies for data cleaning, Proc. -
Int. Conf. Data Eng., pp. 746755, 2007.
[14] L. T. H. Vo, J. Cao, and W. Rahayu, Discovering conditional
functional dependencies, Conf. Res. Pract. Inf. Technol. Ser., vol.
115, no. 5, pp. 143152, 2011.
[15] J. Bauckmann, Z. Abedjan, U. Leser, H. Mller, and F. Naumann,
Discovering conditional inclusion dependencies, Cikm, no. 1, p.
2094, 2012.
[16] W. Fan and F. Geerts, Capturing missing tuples and missing
values, Proc. 29th ACM SIGACT-SIGMOD-SIGART Symp. Princ.
Database Syst., pp. 169178, 2010.
[17] C. Mayfield, J. Neville, and S. Prabhakar, ERACER: a database
approach for statistical inference and data cleaning, Proc. ACM
SIGMOD Int. Conf. Manag. Data, pp. 7586, 2010.
[18] P. Larsson, Evaluation of Open Source Data Cleaning Tools :
Open Refine and Data Wrangler, 2013.
[19] P. Vassiliadis and A. Simitsis, Extraction, Transformation, and
Loading, Encycl. Database Syst., pp. 10951101, 2009.
[20] W. Fan, J. Li, S. Ma, N. Tang, and W. Yu, Towards certain fixes
with editing rules and master data, VLDB J., vol. 21, no. 2, pp.
213238, 2012.
[21] W. Fan, H. Gao, X. Jia, J. Li, and S. Ma, Dynamic constraints for
record matching, VLDB J., vol. 20, no. 4, pp. 495520, 2011.
[22] J. Reiter, Data Quality and Record Linkage Techniques . Thomas
N. Herzog , Fritz J. Scheuren , and William E. Winkler . New
York : Springer , 2007 . ISBN 978-0-387-69502-0 . xiii + 227 pp.
$44.95 (P)., vol. 103, no. 482. 2008.
[23] O. Benjelloun, H. Garcia-Molina, D. Menestrina, Q. Su, S. E.
Whang, and J. Widom, Swoosh: A generic approach to entity
resolution, VLDB J., vol. 18, no. 1, pp. 255276, 2009.
[24] D. D. Kalyani, Mining Constant Conditional Functional
Dependencies for Improving Data Quality, vol. 74, no. 15, pp.
1220, 2013.
[25] M. J. Zaki, Mining Non-Redundant Association Rules, Data
Min. Knowl. Discov., vol. 9, pp. 223248, 2004.
[26] J. Quan, Z. Liu, D. Chen, and H. Zhao, High-efficiency algorithm
for mining maximal frequent item sets based on matrix, Proc. -
4th Int. Conf. Comput. Intell. Commun. Networks, CICN 2012, pp.
930933, 2012.
[27] B. Stefan, Addressing Internal Consistency with
Multidimensional Conditional Functional Dependencies, Manag.
Data, pp. 120, 2010.
[28] F. Chiang and R. J. Miller, Discovering data quality rules, Proc.
VLDB Endow., pp. 11661177, 2008.
[29] R. Medina and L. Nourine, A unified hierarchy for functional
dependencies, conditional functional dependencies and association
rules, Lect. Notes Comput. Sci. (including Subser. Lect. Notes
Artif. Intell. Lect. Notes Bioinformatics), vol. 5548 LNAI, pp. 98
113, 2009.
[30] N. Hussein, A. Alashqur, and B. Sowan, Using the interestingness
measure lift to generate association rules, J. Adv. Comput. Sci.
Technol., vol. 4, no. 1, p. 156, 2015.
=
96
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Efficient Dependable Rules Generation Approach for
Data Quality Enhancement
Asmaa S. Abdo, Rashed K. Salem, and Hatem M. Abdul-Kader
Information System Department, Faculty of Computers and Information, Menoufia University, Egypt
{engasmaa.saad.mis, rashed.salim, hatem.abdelkader}@ci.menofia.edu.eg
Abstract In area of data quality research, enhancing data
quality is still big challenge, especially in large databases. Data
mining techniques can be efficiently utilized in data cleaning
process. Databases are often suffered from data inconsistency,
which has no vital solution up to now. In this paper, we tackle the
problem of detecting data inconsistency from large databases.
We propose an approach for discovering dependable rules from
databases themselves. Such generated rules are minimal and
non-redundant that covers all rules among patterns in database.
The proposed approach focuses mainly on generating precise
dependable data quality rules through extracting maximal fre-
quent patterns, as effective pruning mechanism to reduce the
search space domain. The proposed approach is validated against
several datasets from different application domains. Experimen-
tal Results demonstrate that our approach outperform other
approaches in terms of the efficiency, accuracy and scalability
using both real-life and synthetic datasets.
Keywords Data Quality, Data Mining, Data Cleaning, Func-
tional Dependency (FD), Conditional Functional Dependency
(CFD), Association Rule Mining (ARM), Maximal Frequent Pat-
terns (MFP).
I. INTRODUCTION
Business and Scientific data are most important value
among both public and private organization, which are ex-
ploited for management purposes [1]. However, real-life data
are often dirty holding inconsistencies, conflicts and errors.
The value of data highly depends on its degree of quality. Hen-
ceforward, the presence of errors and inconsistencies in data
decreases their value, making them insignificant, or even harm-
ful.
The consequences of poor data quality are serious problems on
result of data analysis, they often arise due to modifications in
usage of data and violation of integrity constraints (or deficien-
cy of these constraints). Such poor data quality leads to incor-
rect statistics, which results in inefficient daily tasks, wasting
of time and money, and poor business decisions [2], [3].
Data residing in databases is called to be dirty if it holds incon-
sistencies with respect to some set of constraints. Dirty data
costs US business billions of dollars annually that issue poor
decisions making resulting from the poor data quality [4].
Therefore, its very important task for detecting data inconsis-
tency in the data cleaning process. Doubtless, ensuring high
quality dependable data is a competitive advantage to all busi-
nesses, which requires precise data cleaning solutions [5], [6].
978-1-5090-0217-7/15/$31.00 2015 IEEE
Data cleaning (or data scrubbing) is considered as preliminary
and important phase of data preprocessing, i.e., essential part of
data mining [7]. It works for maintaining corrupted and/or
erroneous records. This phase is mandatory in data manage-
ment cycle before mining and analyzing data [23].
Indeed, data cleaning methods in the literature focus on record
matching [10], which match master cleaned records with prob-
ably erroneous records. It is still necessary to tackle the prob-
lem of data inconsistency with the help of data themselves and
without the need for external master copy of data. During re-
solving data inconsistencies, several integrity constraints are
ensured, e.g., Functional Dependencies (FDs), Conditional
Functional Dependencies (CFDs) [20, 28]. Identifying incon-
sistent values is a major step in the data cleaning process.
Currently, growing data in most application domains considers
a big challenge for ensuring consistent and accurate quality
data, to be employed for data management purposes. Such
most of these applications suffers from inconsistent and dirty
data problems. The incorrect data recorded electronically lead
to poor quality data. Herein, we are interested to generate de-
pendable accurate data quality rules, which then used for re-
solving data inconsistencies in several application domains.
The problem is formulated as follows: Given an instance r of a
relation schema R, support s, and confidence c thresholds. The
proposed approach firstly extract maximal frequent patterns in
r and their associated generators at defined support threshold.
Secondly, it generates dependable data quality rules with de-
fined confidence threshold, extracted patterns in r, retaining
interest measure greater than 1. The generated output rules are
minimal interest and non-redundant constant conditional func-
tional dependencies that cover all set of rules.
The main contribution of this paper is to propose an approach
for enhancing performance of generating data quality rules
based on extracting maximal frequent patterns and their asso-
ciated generators. Therefore, maximal frequent patterns reduce
number of patterns generated and search space domain. The
proposed approach utilizes data mining techniques for generat-
ing dependable rules, which mainly rely on extracting patterns
that speed up rules generation process. Also, the proposed
approach employs Interest (lift) measure as a filter to generate
accurate dependable interest data quality rules, and it prunes
other rules that are not statistically significant nor meaningful.
Such generated rules are exploited not only for detecting in-
consistent data, but also for correcting them.
91
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

The experimental results of the proposed approach compared III. PRELIMINARIES

with CCFD-ZartMNR algorithm [11] are conducted over dif-
There are several terminologies from the literature are related
ferent application domains. We verify that the proposed ap-
proach scales well with varying the dataset size, number of to the proposed approach. The definitions of such terminolo-
attributes and the minimal number generated data quality rules. gies are provided, including functional dependencies, condi-
tional functional dependencies, closed frequent patterns, max-
This paper is organized as follows: Section II discusses re- imal frequent patterns and their associated generators, constant
lated works. In section III, we present some notations which CFD problem, and pruning search space.
will be utilized by the proposed approach. Section IV introduc-
Functional Dependencies (FD): defined as constraint be-
es the proposed approach for generating dependable data clean-
ing rules. Section V discusses the experimental study and re- tween two sets of attributes in relation from a database. FD
sults conducted for different datasets from different application over Relation(r) represented as X Y where each X value is
domains. Finally, section VI concludes the proposed work and associated with precisely one Y value. Herein, X is considered
highlight future trends. as the determinant and Y is considered as the dependent. Func-
tional dependencies employed for working on schema design
II. RELATED WORK that not capture semantic of data for data cleaning [11].
Despite the urgent necessity for dependable techniques to Conditional Functional Dependencies (CFD): an extension
enhance data quality and data cleaning issue, this issue has not of FDs aims to detect inconsistencies of data between tuples in
vital solution up to now. Indeed, there have been little discus- single relation since CFD on relation r is a pair (X Y, tp),
sions and analysis for enhancing data inconsistency. To the where X Y is a standard Functional Dependency (FD) on r;
best of our knowledge, most of literature focus on record and tp is the pattern tuple of with attributes in X and Y. For
matching and duplicate detection [8]. each attribute A in X Y, tp[A] is either a constant in the
Database and data quality researchers have discussed variety domain of A, or an unnamed variable _ [6].
of integrity constraints based on Functional Dependencies
(FD) [9][12]. These constraints are developed essentially for Example 1: In this example, we address records from custom-
schema design, but are often not able to embed the semantic of er relationships have the following attributes Country Code
data for data cleaning. (CC), Area Code (AC), Phone Number (PN), Name (NM),
Other researchers focus on extension of FD, they have pro- Street (STR), City (CT) and Zip Code (ZIP). It explains the
posed Conditional Functional Dependencies (CFD) and Con- concept of functional dependency, and conditional functional
ditional Inclusion Dependencies (CID). Such integrity con- dependency.
straints are employed for capture errors in data [13][15]. TABLE 1. RECORDS FROM CUSTOMER RELATION
Moreover, several data quality techniques are proposed to CC AC PN NM STR CT ZIP
clean missy tuples in databases [16]. Statistical inference ap- t1 40 872 6666666 Mike Port PI UN 08422
proaches are studied in [17]. These approaches tackle missing t2 40 872 6666666 Rick Port PI UN 08422
values in order to enhance quality of data. From technological t3 40 101 9999999 Joe M.V.P EDI 01202
part, there are several open source tools which developed for t4 40 872 9999999 Jim Majestic UN 08422
handling messy data [18]. t5 32 222 4444444 Ben 6rd Str VIZAG EH4 1DT
t6 32 222 7777777 Ian 6rd Str VIZAG EH4 1DT
Besides, data transformation methods such as commercial t7 32 872 7777777 Ian XXX UN W1B 1JH
ETL (Extract, Transformation and Loading) tools are devel- t8 40 131 9999999 Sean 5rd Str MH 01332
oped for data cleaning [19].
The usage of editing rules with master data is discussed in
[20]. Such rules are able to find certain fixes by updating input Traditional FD that hold on Table 1 is
tuples with master data. However, this approach requires de- f1: [CC, AC] CT
fining editing rules manually for both relations, i.e., master f2: [CC, AC, PN] STR
relation and input relation, which is very expensive and time- f1 mean that if there are two customers that have the same
consuming. country and area code, they also have the same city, similarly
Furthermore, lots of work are proposed in the literature relying for f2.
on domain specific similarity and matching operators, such The following are CFDs that hold on table 1 is
works include record matching, record linkage, duplicate de- 0: ([CC, ZIP] STR, (32, -|| -))
tection, and merge purge [8], [21], [22]. These approaches 1: ([CC, AC] CT, (40, 872 || UN))
define two functions; namely match and merge [23]. While 2: ([CC, AC] CT, (32, 222 || VIZAG))
match function identifies duplication of records, the merge 3: ([CC, AC] CT, (40, 101 || EDI))
function combines the two duplicated records into one. Rules are classified to 0 as variable CFD, where 1, 2, 3 are
From the literature, we reutilized CFD as a special case of constant CFD. In this work, we are concerned with such type
association rules [14], [24]. The relationship between minimal of constant CFD. CFD 0 asserts that zip uniquely determines
constant CFD and association rule as similar relationship to STR. This FD hold on subset of tuples with the pattern
CFD that work on minimal non redundant rules [25]. CC=32, rather than on the entire relation. CFD 1 assures that
for any transaction tuple if customer country code=40 with

92
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
area code 872, then the customer city is UN. Similarly for 2
and 3 these cannot represent as FDs.
Closed Frequent patterns: Pattern is frequent closed if it is
not included in a proper superset having the same support. A
generator Y of a frequent closed pattern X, is a pattern con-
straint with it has the same support as X, and it does not have
any subset having the same support [14].
Maximal Frequent patterns: Pattern is maximal frequent if
none of its immediate supersets is frequent. The set of fre-
quent maximal patterns is thus a subset of the set of frequent
closed patterns, which is a subset of frequent patterns [26].
Constant CFD Problem: The problem is to discover minimal
set of frequent constant CFDs that include non-redundant CFD
[2], [27]. Example 1 has 1, 2, 3, which are constant CFD.
Pruning Search Space: Pruning is to remove infrequent
nodes from search space using predefined support value [2].
IV. PROPOSED WORK
The proposed approach focuses on generating precise depend-
able interest minimal and non-redundant constant CFD data
quality rules, which cover all set of rules with respect to speci-
fied support and confidence thresholds. Such generated rules
are exploited for detecting and repairing data inconsistencies
from large datasets. Since, the overall performance of rules
generation is based on fast extraction of patterns from data.
The proposed approach named minimal interest constant con-
ditional functional dependencies miner abbreviated as
(MICCFD-Miner). This mainly based on mining maximal
frequent patterns and their associated generators, applying the
effective pruning mechanism to reduce the search space do-
main. Maximal frequent patterns (MFP) are usually much
smaller than the set of frequent patterns and also smaller than
set of frequent closed patterns. In other words, maximal fre-
quent patterns closed frequent patterns frequent patterns.
MFP patterns are called maximal because they have no fre-
quent supersets. Moreover, the set of these patterns is a mi-
nimal set, i.e. the smallest set from which all frequent patterns.
The proposed approach is depicted in figure 1.
Figure 1: The proposed MICCFD-Miner approach
The main input of MICCFD-Miner are dataset and two prede-
fined thresholds, i.e., minimum support, and minimum confi-
dence. The first step is to extract list of maximal frequent pat-
terns and their associated generators by using minimum sup-
port threshold. MICCFD-Miner approach then utilizes such
extracted maximal frequent patterns instead of working on
discovering all frequent patterns or closed frequent patterns,
reducing the size of search space domain shown in figure 2.
Figure 2: Search Space Domain
Support CFD : (X A) is defined as the percentage of the
number of records in the dataset that contain X A to the total
number of records in the database. For example, the support of
1 in example 1 (Section 3) is 3/8 because three tuples satisfy
1. Support of a CFD : (X A) where X generator pattern
and A is (maximal /generator) [28], defined as:
N X A
support X A (1)
T
The second step depends on extracted patterns and their asso-
ciated generators from the first step. Then, the minimum con-
fidence threshold is utilized to generate the set of interest min-
imum non redundant constant CFD data quality rules. While
the literature utilizes only support and confidence for generat-
ing such rules, we consider interest (or lift) measure into ac-
count for generating more dependable and reliable rules.
The form of rules for each frequent generator pattern X finds
its proper supersets A from set of frequent maximal patterns.
Then, from X and A add rule antecedent (Generator) conse-
quence (maximal/generator) as : X A.
Confidence CFD is calculated as the number of records in the
dataset that contain X A divided by number of records that
contain X. For example, confidence of 1 in example 1 (Sec-
tion 3) is 1.0, defined as CFD : (X A) is
XA
confidence X A (2)
Confidence measures the reliability of rules, since the value of
confidence is real number between 0 and 1.0 [29]. However,
the pitfall of confidence is that ignores support of right hand
side of rules. As consequence, we add data quality measure
called Interest (Lift) which generates more dependent rules
when reserving it as greater than 1.
Lift CFD tries to measure the degree of compatibility of left
hand side and right hand side of rules as, i.e., occurrence of
both left hand side and right hand side [30]. We set here Lift
value greater than 1 to obtain dependent MICCFD-Miner
rules. For example, lift of this CFD rule : (X A) is
f
lift X A
A
(3)
This approach optimizes the process of the rules generation
compared with the most related methods.
93
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

V. EXPERIMENTAL STUDY
We present an experimental study for validating the proposed
approach. the proposed approach is evaluated w.r.t three di-
mensions: (1) Efficiency; extracting maximal frequent patterns
that results in reduction in the number of patterns generated
and space complexity without loss of information when gene-
rating dependable rules. (2) Accuracy; generating accurate
dependable rules, which are minimum and non-redundant. (3)
Scalability; measuring response time when increasing both
database size and number of attributes.
A. Experimental Setting
The experiments are conducted using both real-life and syn-
thetic datasets. Such datasets are taken from the UCI machine
learning repository (http://archive.ics.uci.edu/ml/) namely,
Mushroom, Thyroid (hypothyroid), Adult, Audiology. Table 2
shows the number of attributes and the number of instances
for each dataset.
equal to 0.4 and confidence =1, the number of maximal fre-
quent patterns extracted from the proposed approach is 41
patterns, but number of closed patterns generated from other
approach is 140. Such results show the effectiveness in ex-
tracting maximal frequent patterns as first step in generating
accurate and dependable rules instead of extracting closed
frequent patterns as current existing approaches. Focusing on
maximal frequent patterns also optimize the memory space,
which shown in figure 4. In this example, the usage memory
space of the proposed approach equal to 11.1 MB, but the
usage memory space of other approach equal to 12.24 MB.

TABLE 2: DATASETS DESCRIPTION

Size (Number of Arity (Number
Dataset name Instances) of Columns)
Mushroom 8124 23
Thyroid(hypothyroid) 3772 30
Adult 32561 15
Audiology 226 70
Figure 3: Total number of maximal frequent patterns
All our experiments were implemented using java (JDK1.7). extracted from mushrom dataset.
The implementation is tested on machine equipped with In-
tel(R) Pentium(R) Dual CPUT3400 with 2.00 GB of memory
running on windows 7 operating system. The proposed ap-
proach runs mainly in main memory. Each experiment is re-
peated at least five times and the average reported here.
B. Experimental Results
We next demonstrate and discuss the results from experimen-
tal study over the proposed approach against CCFD-ZartMNR
algorithm [24]. The experiments are conducted for different
dataset from different application domains described in pre-
vious section.
Experiment-1: Efficiency of extracting patterns. We here in
evaluate the efficiency of extracting maximal frequent patterns
Figure 4: Space Complexity measure about mushrom dataset.
from the proposed approach against closed frequent patterns
from CCFD-ZartMNR [24]. The results for mushroom dataset Experiment-2: Accuracy of rules generation. In this expe-
are shown in figures 3, 4. This dataset contains 23 attributes, riment, we validate accuracy of rules generated from the pro-
and 8124 records about mushroom physical characteristics. posed approach compared to CCFD-ZartMNR algorithm [24].
Figure 3 show the efficiency in reduction the number of pat- Results from thyroid (hypothyroid) diagnosis dataset is shown
terns generated form the proposed approach (MICCFD-Miner) in figures 5, 6. This dataset contains 30 attributes, and 3772
over other approach without loss of information. The x-axis is records of patient data describing patient information about
the different support values and y-axis is the total number of the hypothyroid diagnoses data. As shown, figure 5 confirm
extracted patterns, and set fixed constant confidence value effectiveness of the accurate minimal non-redundant data
equal 1. Figure 4 demonstrate the space complexity optimiza- quality rules generation of the proposed approach over other
tion of the number of patterns extracted from the proposed approach for detecting inconsistency problems. The proposed
approach against other approach. The x-axis is the support approach always generate less numbers of rules but more ac-
values and y-axis is the space complexity measure in mega- curate as compared to other approach, because the other ap-
bytes (MB), at fixed constant confidence value equal 1. For proach does not consider interest measure of rules. The x-axis
example in figure 3, with thresholds minimum support value is minimum support threshold values and y-axis is the number

94
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

of rules generated, and set fixed constant confidence value
equal 0.99. Figure 6 also show that the proposed approach
outperforms the other algorithm with respect to response time
for rules generations. The x-axis is minimum support thre-
shold values and y-axis is the response time in seconds (s), and
also set fixed constant confidence value equal 0.99.
milliseconds (ms). This experiment concludes that
the proposed approach outperforms other approach in
scalability of increasing number of tuples with less
response time.

Figure 7: Scalability per number of tuples on adult dataset.

Figure 5: Numbers of rules generated from thyroid diagnoses dataset.
b) Scalability of the proposed approach with increas-
ing arity of relation. We asses performance of sca-
lability on increasing number of attributes at fixed
number of tuples over the proposed approach against
CCFD-ZartMNR algorithm. Results from audiology
dataset are depicted in figure 7. This dataset contains
226 tuples and 70 attributes describing audiology
disease data about patients. Fixing database size=226
tuples, confidence=1, minimum support=0.99. We are
changing arity from 10 to 70. Figure 8 shows the re-
sponse time of the proposed approach against other
approach by varying number of attributes. The x-axis
is the number of attributes starting from 10 to 70 and
Figure 6: Response time measure of generated rules
from thyroid diagnoses dataset y-axis is the measure of response time in milliseconds
(ms). Such number of attributes increased signifies
Experiment-3: Scalability of the proposed approach with that the proposed approach scales well in less re-
the dataset size and arity of relation. In order to evaluate the sponse time compared to other algorithm.
scalability of the proposed approach against CCFD-ZartMNR
algorithm [24] by varying dataset size, and arity of relation.
a) Scalability of the proposed approach with varying
database size of relation. In this experiment, we
evaluate the scalability of the proposed approach
when increasing number of tuples at fixed number of
attributes against CCFD-ZartMNR algorithm. Results
from adult dataset is shown in figure 7. This dataset
contains 32561 tuples and 15 attributes describing
prediction about whether income exceeds $50K/yr or
not based on census data. Fixing arity=15, confidence
= 0.95 and minimum support= 0.5. We are changing
database size (number of tuples) from 5k to 30k
tuples. The response time of the proposed approach Figure 8: Scalability per number of attributes on audiology dataset.
against other approach by varying number of tuples
are reported in figure 7, which shows linear time VI. CONCLUSIONS AND FUTURE WORK
scaling of our approach over that always less than the We have proposed MICCFD-Miner, an approach that generat-
other approach. The x-axis is the number of tuples ing accurate minimum dependable interest data quality rules,
*1000 and y-axis is the measure of response time in as solution to data inconsistency problems. The proposed ap-

95
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
proach focus mainly on extracting maximal frequent patterns,
as effective patterns for generating dependable data quality
rules, which yields a promising method for detecting semantic
data inconsistency. The main goal of the proposed approach is
keeping the database in consistent state. Generated rules are
exploited as data cleaning solution to resolve inconsistency
problem in current different application domains specially that
have large databases. MICCFD-Miner relies on lift measure in
addition to support and confidence metrics for generating
depend-able minimal complete and non-redundant rules that
covers all sets of rules. The MICCFD-Miner is validated and
evaluated over both real life and synthetic datasets from dif-
ferent application domains. The experimental results confirm
the effective-ness and usefulness of the proposed approach
against CCFD_ZartMNR algorithm. The proposed approach
performs well across several dimensions such as scalability,
number of rules generated, response time, and space com-
plexity. Finally, we plan to investigate a technique for fixing
errors autonomously with generated rules from MICCFD-
Miner.
REFERENCES
[1] M. Mezzanzanica, R. Boselli, M. Cesarini, and F. Mercorio,
Automatic Synthesis of Data Cleansing Activities, 2011.
[2] J. Li, J. Liu, H. Toivonen, and J. Yong, Effective pruning for the
discovery of conditional functional dependencies, Comput. J.,
vol. 56, no. 3, pp. 378392, 2013.
[3] M. Yakout, A. K. Elmagarmid, and J. Neville, Ranking for data
repairs, Proc. - Int. Conf. Data Eng., pp. 2328, 2010.
[4] W. Fan, J. Li, S. Ma, N. Tang, and W. Yu, Interaction between
record matching and data repairing, Proc. 2011 Int. Conf. Manag.
data - SIGMOD 11, vol. 1, no. 1, p. 469, 2011.
[5] J. Wang and N. Tang, Towards dependable data repairing with
fixing rules, SIGMOD Conf., pp. 457468, 2014.
[6] W. Fan and F. Geerts, Foundations of Data Quality
Management, Synth. Lect. Data Manag., vol. 4, no. 5, pp. 1217,
2012.
[7] P. Srivastava, Mitali and Garg, Rakhi and Mishra, Analysis of
Data Extraction and Data Cleaning in Web Usage Mining
Categories and Subject Descriptors, in Proceedings of the 2015
International Conference on Advanced Research in Computer
Science Engineering \& Technology (ICARCSET 2015), 2015.
[8] D. Bharambe, S. Jain, and A. Jain, A Survey : Detection of
Duplicate Record, vol. 2, no. 11, 2012.
[9] G. Cong, W. Fan, F. Geerts, X. Jia, and S. Ma, Improving data
quality: consistency and accuracy, Proc. 33rd Int. Conf. Very
large data bases, vol. Vienna, Au, pp. 315326, 2007.
[10] S. Hartmann, M. Kirchberg, and S. Link, Design by example for
SQL table definitions with functional dependencies, VLDB J.,
vol. 21, no. 1, pp. 121144, 2012.
[11] H. Yao and H. J. Hamilton, Mining functional dependencies from
data, Data Min. Knowl. Discov., vol. 16, no. 2, pp. 197219,
2008.
[12] J. Liu, J. Li, C. Liu, and Y. Chen, Discover dependencies from
data - A review, IEEE Trans. Knowl. Data Eng., vol. 24, no. 2,
pp. 251264, 2012.
[13] P. Bohannon, W. Fan, F. Geerts, X. Jia, and A. Kementsietsidis,
Conditional functional dependencies for data cleaning, Proc. -
Int. Conf. Data Eng., pp. 746755, 2007.
[14] L. T. H. Vo, J. Cao, and W. Rahayu, Discovering conditional
functional dependencies, Conf. Res. Pract. Inf. Technol. Ser., vol.
115, no. 5, pp. 143152, 2011.
[15] J. Bauckmann, Z. Abedjan, U. Leser, H. Mller, and F. Naumann,
Discovering conditional inclusion dependencies, Cikm, no. 1, p.
2094, 2012.
[16] W. Fan and F. Geerts, Capturing missing tuples and missing
values, Proc. 29th ACM SIGACT-SIGMOD-SIGART Symp. Princ.
Database Syst., pp. 169178, 2010.
[17] C. Mayfield, J. Neville, and S. Prabhakar, ERACER: a database
approach for statistical inference and data cleaning, Proc. ACM
SIGMOD Int. Conf. Manag. Data, pp. 7586, 2010.
[18] P. Larsson, Evaluation of Open Source Data Cleaning Tools :
Open Refine and Data Wrangler, 2013.
[19] P. Vassiliadis and A. Simitsis, Extraction, Transformation, and
Loading, Encycl. Database Syst., pp. 10951101, 2009.
[20] W. Fan, J. Li, S. Ma, N. Tang, and W. Yu, Towards certain fixes
with editing rules and master data, VLDB J., vol. 21, no. 2, pp.
213238, 2012.
[21] W. Fan, H. Gao, X. Jia, J. Li, and S. Ma, Dynamic constraints for
record matching, VLDB J., vol. 20, no. 4, pp. 495520, 2011.
[22] J. Reiter, Data Quality and Record Linkage Techniques . Thomas
N. Herzog , Fritz J. Scheuren , and William E. Winkler . New
York : Springer , 2007 . ISBN 978-0-387-69502-0 . xiii + 227 pp.
$44.95 (P)., vol. 103, no. 482. 2008.
[23] O. Benjelloun, H. Garcia-Molina, D. Menestrina, Q. Su, S. E.
Whang, and J. Widom, Swoosh: A generic approach to entity
resolution, VLDB J., vol. 18, no. 1, pp. 255276, 2009.
[24] D. D. Kalyani, Mining Constant Conditional Functional
Dependencies for Improving Data Quality, vol. 74, no. 15, pp.
1220, 2013.
[25] M. J. Zaki, Mining Non-Redundant Association Rules, Data
Min. Knowl. Discov., vol. 9, pp. 223248, 2004.
[26] J. Quan, Z. Liu, D. Chen, and H. Zhao, High-efficiency algorithm
for mining maximal frequent item sets based on matrix, Proc. -
4th Int. Conf. Comput. Intell. Commun. Networks, CICN 2012, pp.
930933, 2012.
[27] B. Stefan, Addressing Internal Consistency with
Multidimensional Conditional Functional Dependencies, Manag.
Data, pp. 120, 2010.
[28] F. Chiang and R. J. Miller, Discovering data quality rules, Proc.
VLDB Endow., pp. 11661177, 2008.
[29] R. Medina and L. Nourine, A unified hierarchy for functional
dependencies, conditional functional dependencies and association
rules, Lect. Notes Comput. Sci. (including Subser. Lect. Notes
Artif. Intell. Lect. Notes Bioinformatics), vol. 5548 LNAI, pp. 98
113, 2009.
[30] N. Hussein, A. Alashqur, and B. Sowan, Using the interestingness
measure lift to generate association rules, J. Adv. Comput. Sci.
Technol., vol. 4, no. 1, p. 156, 2015.
=
96
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Decreasing ERP Implementation Failure in Egypt
George Sharmokh Wanas
College of Computing and Information Technology
Arab Academy for Science, Technology & Maritime
Transport
Cairo, Egypt
Gwanas20@yahoo.com
AbstractERP Implementation projects have high failure
rates. Any mistakes made during the ERP implementation
process can derail business decision-making, productivity and
profitability. Various risks can affect and cause the failure of
ERP Implementation. The researchers are handling ERP failures
by using risk management before, during and post ERP
Implementation phase in Egyptian organizations. We study how
to resolve ERP Failures by decreasing the risks, for that we
create MSAM model based on FMEA calculations to measure the
readiness of the organization for ERP Implementation and
assure successful implementation and maintenance.
KeywordsFailure Modes and Effects Analysis (FMEA), Multi
Stages Assessment Model (MSAM), Information and
Communications Technology (ICT), Information Technology
Transfer (ITT), Critical Success Factors (CSF), Critical Failure
Factors (CFF).
1. INTRODUCTION
The paper consists of 7 parts, part 1 is the Introduction, part
2 contains the Literature review, part 3 contain Main Failure
Factors Assessment, part 4 includes the overall model
MSAM, part 5 includes the Experimentation model
MSAM, part 6 contains paper conclusion and finally last part
including the references.
1.1. ERP System :
Many researchers describe the ERP Expression with many
aliases as Enterprise application, Enterprise Resource Planning
and ERP systems, they describe ERP as a standardized
software package that integrate all business processes,
information, and information flow across an organization, also
managing and optimizing the available resources, improve its
competitiveness and business processes [1-12] [14].
ERP systems Control and manage complex business
processes and information effectively [3] [17], the important
benefits from ERP Implementation for organizations are to
allow better access, and improve the visibility of information
[15]. In Addition, there are technical and business reasons
978-1-5090-0217-7/15/$31.00 2015 IEEE
Ali H. El-Bastawissy
Faculty of Computer Science
MSA University
Cairo, Egypt
aelbastawissy@msa.eun.eg
Mona Ahmed Kadry
Arab Academy for Science, Technology & Maritime Transport
Cairo, Egypt
monakadry@hotmail.com
which motivate the Egyptian organizations for implementing
ERP such as replacing legacy systems and improving
operational performance and system efficiency [18].
The elimination of work and data redundancy and business
processes simplicity are important strengths of ERP system. So
it is a high technical cross-functional information system, that
improves organizational performance and competitiveness [16]
[20].
1.2. ERP Implementation stages definition:
ERP pre-implementation phase comprises issues of
selecting ERP software, vendors, consultant, and define
requirements. ERP Implementation phase includes activities
related to software rollout, configuration, data conversion,
integration, testing, training of super-users and end users, and
stabilization. ERP Post-implementation phase, encompasses,
training of super-users and end users, maintenance, upgrading,
new-release management, and evolution.
1.3. The Importance of study :
ERP Systems are considered as major investment requiring
effective and efficient planning and facts reveal that 66% to
75% of ERP implementation projects fail to achieve their
implementation goals in many organizations world wide
including Egyptian organizations [12-15], for that, the
management of ERP risk is highly important, to insure the ERP
system success in achieving the business requirements.
The researchers will explore and analyze the ERP
Implementation failure in Pre, during and Post Implementation
phases, to create Recipe for ERP implementation success.
Many researchers found that Risk management absence is
the main factor that leads to the ERP implementation failure
[26-28]
For that, the organization must validate the ERP to
authenticate that the ERP system meets the organization
business requirement, and create the risk assessment for the
ERP to evaluate and minimize the risk.
106
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

2. LITERATURE REVIEW

2.1. ERP implementation difficulties in developing countries TABLE I. RISK ASSESSMENT TOOLS
such as egypt :
There are several constraints In ERP implementation facing
Tools
developing countries compared with developed countries.
Firstly, The National Factors are the greatest challenge for ERP
Implementation in developing countries, It include Information Criteria

HAZOP

HACCP
and Communications Technology (ICT) infrastructure,

FMEA
PHA

FTA
economical status of the country, manufacturing industry
strength, regional location, and governmental regulations [21-
22]. Use in Many stages for X X X X 9
Secondary, Organizational Factors such as IT Maturity, ERP implementation [30] [30] [30] [48] [33]
Computer culture, Business Size, Management Commitment, [41] [36]
ERP cost , and BPR experience. [14] [18] [19] [21] [23], all [42]
[43]
business units at different countries had their own way of doing Quantitative X X X 9 9
business because of different business processes and local Tools [30] [30] [30] [49] [50]
requirements [21]. Many of ERP failures in Egypt are caused [41] [45] [44]
by customization of an ERP system to match the existing [43] [47]
[54]
processes instead of restructuring them [14] [22].
Consider Severity of 9 X 9 9 9
Thirdly, Information Technology Transfer (ITT) problems failures [44] [34] [54] [48] [50]
or the cultural issues facing the eastern developing countries [49]
when implementing and using western technologies, Prioritizing critical 9 X 9 9 9
management processes, and information systems and failures [41] [45] [54] [53] [37]
[41]
techniques. [14] [18-19] [21] [23] 9 9 9 9 9
Give clear description
Fourthly, Most of the implementation failures for ERP were for failure mode [45] [41] [45] [49] [33]
early ERP adoptions which did not have strong business (Reason of Hazard) [45] [53] [50]
justifications. [19] [51]
Powerful in complex 9 X 9 X 9
Most of ERP implementation failure factors are similar
system such as ERP [44] [30] [44] [48] [50]
between developing countries and Egypt. [14] [18-19]. [32]
[45]
2.2. Risk assessment tools: Used in Varity of 9 9 X X 9
There are many tools that are able to assess risk in general industry over the world [42] [55] [46] [38] [24]
such as Preliminary Hazard Analysis (PHA), Hazard and including Software [41] [29]
[48] [36]
Operability study (HAZOP), Fault Tree Analysis (FTA), [49] [51]
Hazards Analysis and Critical Control Point (HACCP), and [52]
Failure Mode and Effect Analysis (FMEA), the PHA, FTA, Product error free 9 9 9 9 9
and HAZOP tools are used in early stages in the requirement [41] [41] [41] [48] [36]
[49]
analysis or at the very beginning of the design process. They
Service operation error 9 9 9 X 9
support qualitative not quantitative analysis [30] that is why free [41] [45] [54] [48] [25]
these tools are not preferred for ERP Implementation, the FTA
does not consider the severity of the failures [34] and in According to the above table its concluded that FMEA is
complex system such as ERP that includes large number of powerful Quantitative risk assessment tool for creating ERP
equipment and process variables the fault tree becomes lager Risk Assessment.
and takes long time to be completed, and they become much The researchers developed MSAM model to assess the risk
more difficult to solve [30-32], and also these makes FTA not a factors related to different implementation stages. The
preferred choice, and takes long time to be completed, and they researchers depend on using FMEA as the principle assessment
become much more difficult to solve [30-32], and also these tool for the MSAM model. The model starts with Failure
makes FTA not a preferred choice, HACCP is originated in the Factors Assessment as in Point No. 3 and proceeds as defined
food industry [38][40], and hence it is more logic to used it for in Point No. 4.
ERP systems Implemented in food industry.
FMEA methodology gives a clear description of the failure 3. FAILURE FACTORS ASSESSMENT
modes [33], FMEA methodology is now extensively used in a 3.1. Critical Failure Factors:
variety of industries including Software [24] [29][35-36][39- Critical Failure Factors aid to develop the ERP without
40], FMEA purpose is to examine possible failure modes and failure in any organization.
determine the impact of these failures on many stages by
The Critical Success Factors (CSF) and the Critical Failure
Design FMEA (DFMEA), Process FMEA (PFMEA),
Factors (CFF) is mirroring each from other, the absence of CSF
Machinery or Equipment FMEA (MFMEA) or Service FMEA
lead to the ERP project failure, and then we can use CFF
(SFMEA) [33][36], FMEA is a strategic technique for creation
expression. The absence for CFF lead to the ERP project
of error free service operation [25]. And it is a methodology
Success, then we can use CSF expression.
that focuses on prioritizing critical failures to improve the
Many researchers study and research for CSFs, CFFs that
safety [37]. FMEA considers each mode of failure of every
widely used in the information systems arena, that lead to the
component of a system, [39].
ERP success or failure [4] [12] [14].

107
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
3.2. Critical Failure Factors Questionnaire in Egypt:
The researchers use the questionnaire technique to study
various failure factors which can affect the ERP
implementation
leading to its failure for the organization in Egypt by the scale
of 1-5, and If any Extra factors can affect ERP pre, during and
Post the implementation faces.
3.3. Data Analysis:
Data collection of this study is done based on combination
of the questionnaire with ranking 5 Likert scale from 1 to 5,
and interviews with ERP Users and experts. The survey
obtained 29 response samples out of 41 (70.7% response rate).
All experts and Users in this study agreed on the
appropriateness, clarity of the items and contents in this
questionnaire.
Some candidate reply with extra factors as the following:
Unclear user requirement definition
Supplier Selection
Poor Team Selection
3.4. Questionnaire Results:
To evaluate the factors, in pre Implementation phase found
that Inadequate project planning and controls has highest
rank degree as found by this study.
In During Implementation phase found that Top
management commitment has highest rank degree as found by
this study.
In Post Implementation phase found that Users resistance
to change has highest rank degree as found by this study.
4. OVERALL MSAM MODEL
Fig. 1
Multi Stages Assessment Model
Symbol ( i ) Refer to the implementation stage, i=1 Refer to
Pre ERP implementation phase, i=2 Refer to During ERP
implementation phase, and i=3 Refer to Post ERP
implementation phase.
4.1. Purpose:
The researchers create MSAM model based on FMEA
calculations for aid to measure the readiness of the organization
for ERP Implementation, and also if any Failures are expected
in any stage in ERP implementation.
This model applied on all ERP Implementation phases, Pre
ERP Implementation phase, During ERP Implementation
phase, and Post ERP Implementation phase.
4.2. Model Inputs:
4.2.1. Predefined Factors Risk Check List is a output
from Critical Failure Factors Questionnaire in Egypt
that was created by the reserchers
4.2.2. Project specific Potential Failure Involve company
stakeholders and consultant aid to identify potintial
failure for ERP Project.
4.2.3. Predefined Recommendationsis an output from
Questionnaire for the consultant in Egypt that was
created by the reserchers.
4.3. Model Calculation sectionFactors Assessment &
Evaluation:
4.3.1. Risk Management Rating criteria:
Severity SEV: How severe is effect ERP Implementation?
TABLE II. SEVERITY
Severity Rating Criteria
(Effect)
Insignificant No effect on data security, integrity and/or
accuracy Data can be retrieved and or stored
1
during normal operating environment, No
hardware damage.
Minor System Downtime of up to 15 minutes but
without affecting data security, integrity and/or
2
accuracy.
No hardware damage.
Moderate Downtime of greater than 15 minutes and less
than 1 hours and/or loss of data which has been
3 previously backup.
No hardware damage.
No effect on product safety and/or quality.
Major Downtime of greater than 1 hours and/or loss of
data security, integrity and/or accuracy.
4 Hardware damage that can be fixed or corrected
with moderate maintenance.
No effect on product safety and/or quality.
Catastrophic Downtime of greater than 1 day and/or loss of
(very High) data security, integrity and/or accuracy.
5 Hardware damage that cannot be fixed and
require replacement.
Possible effect on product safety and/or quality.
108
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Occurrence OCC: How frequent is the cause likely to occur 4.4. Model Output:
the risk?
4.4.1. Risk Evaluation:
TABLE III. OCCURENCE Evaluate ERP Implementation risks Depending on Severity,
Occurence and Detection
Occurrence Rank Criteria
Remote 1 Failure occurs every year or more RPN = Risk priority number in order to rank concerns,
( 1 failure per 8760 hours of operation) Calculated as SEV x OCC x DET
Rare 2 Failure occurs every 3 months(quarter)
( 1 failure per 2160 hours of operation)
TABLE VI. RISK RANGE
Occasionally 3 Failure occurs every week
( 1 failure per 168 hours of operation)
Risk Range Criteria
Frequently Failure occurs every day
4 ( 1 failure per 24 hours of operation) 001 : 010 Insignificant
Continually 5 Failure occurs every shift
( 1 failure per 8 hours of operation) 011 : 050 Minor

051 : 080 Moderate

Detection DET: How probable is detection of failure
cause? 081 :100 Major

TABLE IV. DETECTION 100 : 125 Catastrophic (very High)

Detection Rank Criteria
Certain 1 Controls certainly detect any potential
cause and subsequent failure.
Controls will prevent an imminent failure
and isolate the cause. 4.4.2. Modified Recommendations List:
High 2 High chance that controls will detect a
potential cause and subsequent failure. Its a repository for all recommendations to decrease the ERP
Controls will prevent an imminent failure Implementation risks, Initialy we fill the Modified
and isolate the cause. Recommendation List with the Predefined
Medium 3 Medium chance that controls will detect
Recommendations and the Recommendation from risk
likelihood a potential cause and subsequent failure.
Controls will provide on indication of analysis.
imminent failure and may, or may not,
prevents failure. 4.4.3. Action Plan:
Low 4 Controls do not prevent failure from
likelihood occurring. Use Modified Recommendations List to Create Action plan for
Controls will isolate the cause and failure every ERP Failure; clarify for every action, who is the
mode after the failure has occurred. Responsible person or persons that will implement this action,
Remote 5 Very remote chance that controls will the resourses that will be used in the Action implementation,
likelihood detect a potential cause and subsequent
failure mode, or there are no controls.
and time schedule for action implementation.
4.5. Action Plan Implementation:

4.5.1. Implementation for Stages:

4.3.2. Risk management Ranking criteria:
Action plan implementation for Pre, During, and Post ERP
Depending on Severity and Occurence of Risk
Implementation stages.
TABLE V. RANKING CRITERIA 4.5.2. Acceptance:
Severity After Acceptance test, if implementation of stage is accepted
proceed to next stage, and if stage implementation did nt pass
Insignificant Minor Moderate Major Catastrophic
acceptance criteria return to Factors Assessment &
1 2 3 4 5 Evaluation, make new risk evaluation, and update the
Continually 5 5 10 15 20 25 Modified Recommendation List with the new
Occurrence

recommendations, new action plan, and new implementation

Frequently 4
4 8 12 16 20 for the new action plan.
Occasionally 3
3 6 9 12 15 5. EXPERIMENTATION MSAM MODEL
Rare 2
2 4 6 8 10 The researchers Apply MSAM model on two case studies
in different manufacturing field, automotive and
Remote 1
1 2 3 4 5 pharmaceutical industries for MSAM Model experimentation,
in these case studies we did not implement the complete model
life cycle, put we just apply the model to Modified
Recommendations list step.

109
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
5.1. EXPERIMENTATION:
The researchers use Likert Scale and FMEA calculations
for model experimentation. They experiment the model with
the two case studies in automotive and pharmaceutical
industries, for 51 potential failures and they create Modified
Recommendations list for risk reduction.
5.2. THE MODEL EXPERIMENT STEPS:
Create questionnaires to study various failure factors
which can affect the ERP implementation in Egypt,
and common recommendation for the failures in
Egypt.
Prepare the inputs for MSAM model from the
questionnaires and Company stakeholders and
consultants involvement for the two companies that
involved in our study.
In Factor Assessment and evaluation phase, We
calculate Severity, Occurrence and Detection, to
calculate RPN for every Potential failure, which is a
very important output values in this phase.
Create Modified recommendations List as the Output
of Factor Assessment and evaluation phase detailed
study.
The detailed study show results of two case studies in
automotive and pharmaceutical industries. These
results show the readiness for the organization to
Implement ERP. Also these results are very
important to propose the Recommendation that can
lead to success of the ERP Implementation.
6. CONCLUSION
It is concluded that, ERP systems are complex and cross
functional systems. Successful ERP implementation is
considered as one of the core competencies for any
organization worldwide and in Egyptian market.
ERP systems include multiple risks that need to be
addressed and controlled on scientific bases.
Our developed model used for ERP systems risk analysis
and measurement.
Appling our model to ERP systems assure that
implementation teams will formulate an actionable, reliable
and robust action plan that assures correct implementation,
operation, and maintenance of ERP systems with scientific
rational.
REFERENCES
[1] Mr. Rajendra Marulkar, Prof. Vijay Kumar
Dharmadhikari and Dr. B. V. Cadambi ERP Software
Development For Pressure Die-casting Industry ,Mar
2012
[2] Chetan S. Sankar factors that improve ERP
implementation strategies in an organization ,June 2010
[3] Hany Elbardan Enterprise Resource Planning (ERP)
Systems Implementation and Internal Audit Function
(IAF) Change: Case Study Approach, 28 Mar 2012
[4] Azadeh Pishdad, Abrar Haider,and Andy Koronios ERP
ASSIMILATION: A TECHNOLOGY
INSTITUTIONALISATION PERSPECTIVE ,June
2012
[5] Mutaz M. Al-Debei and Enas M. Al-Lozi
Implementations of ICT Innovations: A Comparative
Analysis in terms of Challenges between Developed and
Developing Countries ,2012
[6] Mustafa Abdullabhai and Freddie Acosta Impact of
Adopting Enterprise Resource Planning Systems by
Commercial Organizations in Kenya ,Feb 2012
[7] Belal Saleh Mareai and Dr. S. Y. Patil TAXONOMY
OF ENTERPRISE RESOURCE PLANNING SYSTEM
,June 2012
[8] L Kambarami, S Mhlanga and T Chikowore
EVALUATION OF ENTERPRISE RESOURCE
PLANNING IMPLEMENTATION SUCCESS: CASE
STUDY IN ZIMBABWE ,July 2012
[9] Syed Iftikhar H. Shah, Abdul Zahid Khan, Dr. Rahat. H.
Bokhari and Muhammad Abbas Raza Exploring the
Impediments of Successful ERP Implementation ,Dec
2011
[10] ALA'A HAWARI and RICHARD HEEKS Explaining
ERP Failure in Developing Countries: A Jordanian Case
Study , 2010
[11] Abeer I. ALdayel, Mashael S. Aldayel and Abdullah S.
Al-Mudimigh The Critical Success Factors of ERP
implementation in Higher Education in Saudi Arabia
,Oct 2011.
[12] Syed Iftikhar H. Shah, Rahat H. Bokhari, Shabbir
Hassan, Mahmood H. Shah and Madad Ali Shah Socio-
Technical Factors Affecting ERP Implementation
Success in Pakistan ,2011
[13] Mahmood Ali and Ying Xie The quest for successful
implementation: A new dynamic model for ERP Systems
Implementation Innovation ,Aug 2012
[14] M. H. Rasmy, Assem Tharwat and Sondoss Ashraf
Enterprise Resource Planning (ERP) Implementation in
the Egyptian Organizational Context , 2005
[15] Hany Abdelghaffar SUCCESS FACTORS FOR ERP
IMPLEMENTATION IN LARGE ORGANIZATIONS:
THE CASE OF EGYPT , 2012
[16] Ayyub Ansarinejad, Mohsen-Sadegh Amalnick,
Mohammad Ghadamyari, Samad Ansarinejad and
Loghman Hatami-Shirkouhi EVALUATING THE
CRITICAL SUCCESS FACTORS IN ERP
IMPLEMENTATION USING FUZZY AHP
APPROACH ,2011
[17] T.SUGANTHALAKSHMI and C
MOTHUVELAYUTHAN GROUPING OF
CRITICAL SUCCESS FACTORS FOR ERP
IMPLEMENTATIONS ,April 2012
[18] Hany Abdelghaffar and Reem Hamdy Abdel Azim
SIGNIFICANT FACTORS INFLUENCING ERP
IMPLEMENTATION IN LARGE ORGANIZATIONS:
EVIDENCE FROM EGYPT , April 2010
[19] Ahmed A. Elragal and Ayman M. Al-Serafi The Effect
of ERP System Implementation on Business
Performance: An Exploratory Case-Study , 2011
[20] Arun Madapusi An Overview of ERP in Indian
Production Firms , Mar 2011
[21] Light. Zaglago, Idisemi. Apulu, Craig. Chapman and
Hanifa Shah, Member The Impact of Culture in
Enterprise Resource Planning System Implementation ,
Jul 2013
[22] Ahmed O. R. Kholeif, Magdy Abdel-Kader and Michael
Sherer ERP Customization Failure: Institutionalized
Accounting Practices,Power Relations and Market
Forces , 2007
[23] Dr. Salwa Alsamarai and Dr. Ziad. M.S Almashaqba
The Organizational Factors Affecting Enterprise
Resource Planning Systems (ERPs) Implementation
Success , 2009
[24] Hadi Shirouyehzad, Reza Dabestani and Mostafa
Badakhshian The FMEA Approach to Identification of
Critical Failure Factors in ERP Implementation , July
2011
[25] Agung Sutrisno and TzongRu, Service reliability
assessment using failure mode and effect analysis
(FMEA): survey and opportunity roadmap 2011
[26] S. Vijayakumar Bharathi, R. Raman and Dhanya Pramod
A FPN Based Risk Assessment Model for ERP
Implementation in Small and Medium Enterprises ,
2014
[27] Nazli Sadat Safavi, Mahyar Amini, Azam
Abdollahzadegan and Nor Hidayati Zakaria An
110
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Effective Model for Evaluating Organizational Risk and [41] ICH-Q9 standard, INTERNATIONAL CONFERENCE
Cost in ERP Implementation by SME , Jun 2013 ON HARMONISATION OF TECHNICAL
[28] Vijayakumar Bharathi, Dhanya Pramod and REQUIREMENTS FOR REGISTRATION OF
Ramakrishnan Raman A FUZZY PETRI-NET BASED PHARMACEUTICALS FOR HUMAN USE
CONCEPTUAL MODEL FOR RISK PREDICTION IN QUALITY RISK MANAGEMENT Q9 09 Nov 2005.
ENTERPRISE RESOURCE PLANNING [42] Mohamed-Habib MAZOUNI et al. A PHA based on a
ACQUISITION DECISIONS FOR SMALL AND systemic and generic ontology 2007.
MEDIUM ENTERPRISES , 18 Mar 2013 [43] Roland Mader et al. Computer-Aided PHA, FTA and
[29] T. Frank et al. Quality Risk Management Principles and FMEA for Automotive Embedded Systems 2011.
Industry Case Studies, 28 Dec 2008 [44] JongGyu Hwang,and HyunJeong Jo
[30] S. Bernardi et al. Model-Driven Dependability Hazard Identificaiton of Railway Signaling System Usi
Assessment of Software Systems, 2013 ng PHA and HAZOP Methods 2 FEB 2013.
[31] Clifton A. Ericson Fault Tree Analysis A History, [45] HE Xin, and TAO Xin A Software Safety Test
1999 Approach Based on FTA and Bayesian Networks 2011.
[32] Ahmed Ali Baig, Risza Ruzli, and Azizul B. Buang [46] XU Xin, et al. Improvement & Application of HAZOP
Reliability Analysis Using Fault Tree Analysis: A Analysis for Batch Process with SDG 2010.
Review, June 2013 [47] Wei WANG, and Tingdi ZHAO The Application of the
[33] Yonas Mitiku Degu, R. Srinivasa Moorthy Root Causes of Human Error Analysis Method Based on
Implementation of Machinery Failure Mode and Effect HAZOP Analysis in Using Process of Weapon 2009.
Analysis in Amhara Pipe Factory P.L.C., Bahir Dar, [48] S.I. Lao et al. A real-time food safety management
Ethiopia, 2014 system for receiving operations in distribution centers
[34] Rodrigo de Queiroz Souza, and Alberto Jos lvares 2012.
FMEA AND FTA ANALYSIS FOR APPLICATION [49] Ioan Alfred Letia, and Adrian Groza Developing
OF THE RELIABILITY-CENTERED Hazard Ontology for Supporting HACCP Systems in
MAINTENANCE METHODOLOGY: CASE STUDY Food Supply Chains 2010.
ON HYDRAULIC TURBINES, 2008
[50] Xi-Ping Zhu et al. A Quantitative Comprehensive
[35] Dr. D. R. Prajapati Implementation of Failure Mode Safety Evaluation Method for Centrifugal Compressors
and Effect Analysis: A Literature Review , July 2012 Using FMEA-fuzzy Operations 2013.
[36] Murat OZKOK RISK ASSESSMENT IN SHIP HULL [51] Guoqi Li Ontology-based Reuse of Failure Modes for
STRUCTURE PRODUCTION USING FMEA, Feb FMEA: Methodology and Tool 2012.
2013
[52] Youngjung Geum et al. A systematic approach for
[37] Rupy Sawhney et al. A Modified FMEA Approach to diagnosing service failure: Service-specific FMEA and
Enhance Reliability of Lean Systems, 2010 grey relational analysis approach 29 JUL 2011.
[38] Abdelaziz Berrado et al. A Framework for Risk [53] Guoqi Li A 14-Step Strategy of HACCP System
Management in Railway Sector: Application to Road- Implementation in Snack Food Manufacturing 30 AUG
Rail Level Crossings2011 2008.
[39] DAVID W. VINCENT, and BILL HONECK Risk
Management Analysis Techniques For Validation
Programs May 2004 [54] Silvianita et al. Operational Risk Assessment
Framework of Mobile Mooring System 2011.
[40] Tim Sandle The use of risk assessment tools for
microbiological assessment of cleanroom environments, [55] Yoshio Sakai et al. An extended notation of FTA for
2010 risk assessment of software-intensive medical devices.
2013.

111
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Session IV:
Computer Networks, Security, and
HPC

Title: Page#

Accelerated Parallel Training of Logistic Regression using 113

OpenCL
Hamada M.Zahera and Ashraf B.El-Sisi

ROEF: A Smart Search Engine of the 3rd Generation World Wide 118
Web (WWW)
Karim Mahmoud, Hanafy Ismail, and Mohamed Kholief

Mitigating ARP Spoofing Attacks in Software-Defined Networks 126

Ahmed M.AbdelSalam, Ashraf B. El-Sisi and Vamshi Reddy K

DNA-Based Data Hiding Using Variable Length Substitution 132

Technique
Dina A. Ibrahim, Elsayed A. Sallam, and H.M. Abdalkader

PRNG Based Modified MSLDIP Watermarking 138

Abdelmgeid A. Ali and Ahmed H. Ismail

Energy-Aware Enhancement of LEACH Protocol in Wireless 144

Sensor Network
Ahmed AL-BAZ and Ayman EL-SAYED

Intelligent Collision Detection and Avoidance in Railways using 150

GPS/RF Module
S. A. Qunber, M. S. Zaghloul, O. Badawy, and M. Farouk

112
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Accelerated Parallel Training of Logistic Regression
using OpenCL
Hamada M.Zahera, Ashraf B.El-Sisi
Faculty of Computers and Information, Menofia University
Shebin El Kom, EGYPT
{hamada.zahra, ashraf.elsisi}@ci.menofia.edu.eg
AbstractThis paper presents an accelerated approach for
training logistic regression in parallel and running on Graphics
Processing Units (GPU). Many researchers have worked out in
boosting the performance of logistic regression using different
techniques. Our study focuses on showing the ultimate capabilities
of GPU processing and OpenCL framework. GPU and OpenCL
are the low cost and high-performance solutions for scaling
up logistic regression to handle large datasets. The proposed
approach is implemented in OpenCL C/C++ and tested with
different datasets. All results showed a significant improvement
in execution time in large datasets, which is reduced almost with
the available GPU devices.
KeywordsLogistic Regression, Parallel Computing, GPU,
OpenCL
I. I NTRODUCTION
Recently, machine learning have been well applied in many
complicated problems like prediction, recommendation and
classification. Logistic regression is one of machine learning
algorithms used in Predicting tasks, it can anticipate the output
value based on a set of attributes or input variables. First,
We need to build a logistic model through training with
previous instances. The datasets should have a variety of
training examples and consider many cases. The complexity of
training process in logistic regression depends on the problem
characteristics and datasets volume.
Such training like this can take long hours and even days
to reach the desired accuracy in logistic model[1]. On the
other way, finding the best fit setting for building a logistic
model needs a certain amount of cross-validation experiments
which can also be very time-consuming. In the past decade,
many techniques were proposed to speed up training in com-
plex applications. Multithreaded, Multi-core CPUs, Message
Passing Interface (MPI) and OpenCL have been investigated
in different applications. Each technology has its own de-
ployment characteristics and execution cost. An exponential
growth of performance is expected in the near future since
many researchers focus their attention in parallelizing a variety
of computational intelligence algorithms[2] using the new
hardware architectures.
OpenCL is a development framework developed by
Khronos group to support general purpose computations on
GPUs. OpenCL provides an industry standard for parallel
programming of heterogeneous computing platforms[2], its
not dedicated to specific GPU vendors opposite to CUDA
which is restricted to Nvidia GPUs. The recent developments
of GPU have shown a superb computational performance
978-1-5090-0217-7/15/$31.00 2015 IEEE
in multiple applications with respect to current multi-core
CPU. Nevertheless, GPUs are specially designed to facil-
itate accelerated graphics processing, they have been used
as general purpose computing devices (often called General
Purpose GPU: GPGPU[3]). Certain high-performance GPUs
are now designed to solve general-purpose processing instead
of graphics rendering, which it was the only usage of GPUs
before.
The specifics of OpenCL architecture are strongly taken
into account so as to carefully devise the parallelization
method. Consequently, this method would be effective in
different conditions and useful for potential researchers. The
results showed that our parallel implementation have a clear
advantage over the original algorithm, which can be obtained
in a wide range of available GPU hardware.
This paper is organized as follows. The proposed approach
is presented in detail in section 2 along with sequential
training process of logistic regression. Section 3 analyzes the
results obtained from running several experiments on different
platforms. In section 4, we discussed the relevant works that
have done before in accelerating the training process with
respect to our approach. Finally, the paper is concluded in
section 5.
II. R ELATED W ORKS
Recently, the size of datasets has increased rapidly by such
a way that became difficult to be handled with current im-
plementations. Several techniques were purposed to accelerate
the training process of logistic regressions using technolo-
gies such as Multithreading,MPI and Open Multiprocessing
(OpenMP). In [8], Thanh-Nghi et al have presented a new
parallel multiclass logistic regression algorithm that achieves
high performances for classifying large amounts of images
into many classes. The parallel multiclass algorithm is also
developed for efficiently classifying large image datasets into
a very large number of classes on multi-core computers. They
applied two major parallel models: MPI and OpenMP are
mostly famous for shared memory. While OpenCL use the
powerful abilities provided in GPU devices. Anther approach
for training logistic regression in a distributed fashion has
been suggested by Yong Zhuang et al[9]. Many interesting
techniques are discussed for reducing the communication cost
and speeding up the computation. They have released an MPI-
based implementation for public use.
Peng et al[10] analyzed three optimization approaches
along with two computing platforms to train logistic model
113
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

on a large scale and high-dimensional datasets for classifica-

tion. Based on extensive experiments. Sequential algorithms
with memory intensive operations can perform very well if
datasets can fit in memory. For massive datasets, if limited by
machine resources, Mahout with its online algorithm is a good
choice with a slightly lower precision. The authors relied their
implementation on distributed frameworks to scale up logistic
regression.
Nowadays, GPU devices are well utilized for executing
general application in addition to processing graphics and
games. Complex applications could be run on CPU and GPU
devices in parallel. This implementation has been investigated
before by Kulkarni et al[11] in designing database processing Fig. 1. Platform model of OpenCL architecture: one host and one or more
compute devices each with one or more compute units each with one or more
model by Linear Regression on GPU using CUDA. It is ob- processing elements. While the host executes the main program, processing
served that the multithreading architecture and SIMD approach elements execute small kernel program in parallel.[4]
of CUDA helps for performance improvement in a great sense.
There is a tremendous difference in the results obtained on
CPU and GPU. So, CUDA is one of the best programming
approaches to optimize the time for various algorithms which
require a huge amount of data. Also, Mark van Heeswijk
et al[12] presented an approach that allows for performing
regression on large data sets in reasonable time. The main
component of this approach consists in speeding up the slowest
operation of the used algorithm by running it on GPU, instead
of the processor CPU. Experiments show a speedup of an order
of magnitude by using the GPU, and competitive performance
on the regression task.
In this paper, we used OpenCL framework for accelerating
training of logistic regression. The proposed approach worked
out on available GPU devices in parallel and achieve the
highest performance speed. The following section discusses in
detail the reasons behind our choice of considering OpenCL Fig. 2. Memory model of OpenCL architecture with respect to the compute
in developing the proposed algorithm. units and device[4].

III. PARALLELIZATION USING O PEN CL
A. OpenCL Programming Model
OpenCL[4] is an open standard which defines a parallel
programming framework for programs that execute across het-
erogeneous platforms. Programs that conform to the standard
may be executed on CPU, GPU and other devices that have
interfaces to the standard specification. Processing elements in
OpenCL are the compute device (such as GPU), which may
have one or more compute units (e.g. a GPU multiprocessor),
which in turn may consist of one or more processing elements
(e.g. a GPU scalar processor). The processing elements are
accessed by a host, which may be a computer system where
OpenCL programs are initiated. The programming elements
in OpenCL are based on work items or kernels, which are
equivalent to tasks in parallel programming models, and which
execute on a single processing element. A set of multiple work
items that are processed (in parallel) by a computing unit
is called a workgroup. OpenCL memory model distinguishes
different levels of memory:
Global memory is a multipurpose memory on the
compute device that is the largest and the slowest one.
Constant memory is a part of the global memory that
remains unchanged during the kernel execution.
Local memory is allocated to a computing unit and is
smaller and faster than the global memory.
private memory is the smallest and the fastest (e.g.
registers), and is accessible only from the processing
element it belongs to..
The global memory is the only way of communication between
host and computing device, whereas the other types of memory
may be used for internal computation. In case more private
memory is requested than physically available, global memory
will be used instead.
B. Training with sequential Logistic Regression
Logistic regression is a statistical method for analyzing a
dataset in which there are one or more independent variables
that determine an outcome. The outcome is measured with a
dichotomous variable (in which there are only two possible
outcomes). In order to build a predictive model using logistic
regression, first we need to train this model using previous
training instances. The output value is computed based on a
set of instance attributes. Each attribute has a coefficient value
named attributes weight. The goal of the training process is to
find which are the best weights values that fit this problem and
could achieve a high prediction accuracy later on. The steps
of logistic regression are stated in Algortihm1[5]: Sequential

114
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Fig. 3. Sequential Training of Logistic Regression
training of logistic regression. the algorithm runs sequentially
from step2 to step4. The predicted output is computed in step2
using a logistic function which based on instance attributes
values.
Algorithm 1 Sequential Training of Logistic Regression
1: Initialize weights to small random values
2: Compute the actual output using sigmoid function
sigmoid(Z) = P1/1 + ez
k
where : Z = i=k Wi Xi
X : Instance attribute
W : Attribute weight
3: Estimate error and update weights using delta rule
4: If (Actualoutput(O) 6= T argetoutput(T ))
Wi = Wi + (T O) Xi
5: ELSE : try new Instance
We did a comparative study between sequential and parallel
training of logistic regression. Despite in complex problems,
the training process requires very large datasets which con-
sume to the extent long time (e.g.hours, days). As shown in
Fig.1, the training process goes sequentially to train a logistic
model instance by instance. This approach fits well simple
prediction problems that require small datasets for training.
Recently, the logistic models are designed to work out complex
problems with many attributes. This type of problems required
to be trained with very large training datasets. Unfortunately,
sequential logistic regression takes a long time in training and
building efficient model. Many researchers addressed this prob-
lem in the past few years using different parallel techniques.
The motivation of our research work is to parallelize training
in logistic regression using available resources in a standard
PC such as GPU devices as well as running on CPU.
C. Training with parallel Logistic Regression
This section discusses main aspects of logistic regression
implementation that uses OpenCL to exploit multi-core CPU
and GPU processors. The proposed method includes three
main steps in order to train logistic regression with a large-
scale dataset in a response time. Main issues required to deal
with big data; it should fit memory by processing it in parallel
Fig. 4. Parallel Training of Logistic Regression
or distributed manner. The first step is to divide training dataset
T into N equal parts where N corresponds to a number of
parallel kernels. Each part of data can fit in memory and
trained by logistic regression. We considered batch training
approach because it can be parallelized directly rather than
online training. In batch learning[6], we accumulated the error
values over all parts and then updated the weights. Secondly,
for each part of the dataset; logistic regression is trained by
initialized weights with random values at first[7]. Then, train
instance by instance to adapt weights and find the best fit
values using delta rule as shown in equation 1.
Wij = Wij + (T O) (1)
where Wij represents attribute weight j in instance i and T ,
O corresponds to target value and actual value calculated by
the algorithm respectively. At each kernel, the actual output
O is computed using a sigmoid function and compared to the
target value T . Then, weights need to be adapted in case of
actual output is not the same as the target value. The algorithm
runs in parallel starting from step 2, each kernel output delta
weights which used to get final weights in the last step.
Algorithm 2 Parallel Training of Logistic Regression
1: Initialize weights to small random values
2: for each node i N do
3: Compute the actual output using sigmoid function
sigmoid(Z) = P1/1 + ez
k
where : Z = i=k Wi Xi
X : Instance attribute
W : Attribute Weight
4: Estimate error and update weights using delta rule
5: If (Actualoutput(O) 6= T argetoutput(T ))
Wi = Wi + (T O) Xi
6: ELSE : Try new Instance
7: end for
We considered applying data parallelism in GPU due to
many reasons: The GPU-based solutions are cost-effective and
dont require any additional resources for extreme processing
application. We used OpenCL C/C++ in GPU programming.
Finally, once each kernel finished adapting weights, the final
weights are obtained by accumulating all weights come from
115
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

TABLE I. E XECUTION TIMES IN MICROSECONDS FOR SEQUENTIAL

AND GPU L OGISTIC R EGRESSION ON PLATFORM A

Data Size Sequential Logistic Regression GPU-Implemented Logistic Regression

1000 334 142

5000 1209 159
10000 2188 192
50000 10038 292
100000 15000 349
111000 80000 395

TABLE II. E XECUTION TIMES IN MICROSECONDS FOR SEQUENTIAL

AND GPU- BASED L OGISTIC R EGRESSION ON PLATFORM B

Data Size Sequential Logistic Regression GPU-Implemented Logistic Regression

1000 299 150

5000 1076 165
10000 1947 171
50000 8745 230
100000 15000 272 Fig. 5. Execution time between sequential and GPU-Logistic Regression on
111000 80000 289 platform A

different kernels as shown in equation 2.

N
X
4Wij = 4WijTk (2)
n=1

IV. E XPERIMENTAL R ESULTS

A. Environment Setup
We analyzed the performance between sequential and
parallel training of logistic regression in different datasets
and platforms. We carried out many experiments to estimate
the speedup in both approaches. Our experiments were im-
plemented in OpenCL C/C++ and run onto two different
platforms: Platform A has Mac OSX Yosemite with Intel(R)
Core(TM) i5-2415M CPU 2.30GHz and Platform B has
Mac OSX Yosemite with Intel(R) Core(TM) i5-3210M CPU
2.50GHz. The following subsections give more information
about datasets and results to demonstrate the performance
boosting when GPU is involved in the computation.
B. Dataset
At first, we preprocessed the datasets from raw structure
format into a vector space model. Each record represents a user
review as a set of products features and users rating value.
We tried out with various sizes 1000, 5000, 10000, 50000
and 100000 instances as shown in table 1 and table 2. The
speedup in all datasets was measured in microseconds. It can
be clearly seen that the speed raised exponentially with the
increasing size in sequential training. While, in parallel training
the trend roughly goes constantly despite the dataset size.
These results demonstrated the high computation capabilities
of GPU devices when they were used in other applications.
C. Result Analysis
The results are visualized in figure five and six to highlight
the comparison between execution time in parallel and sequen-
tial logistic algorithm. It can be clearly seen that the perfor-
mance of proposed approach outperforms sequential training
of logistic regression. As shown in figure 5 and figure 6, the
speedup of GPU-accelerated logistic regression is 11 times
Fig. 6. Execution time between sequential and GPU-Logistic Regression on
platform B
faster than the speed of sequential approach. GPU has power-
ful computing capabilities that points out rendering graphics
applications in the past. Lately, GPU-based applications are
developed using programming frameworks such as CUDA and
OpenCL. CUDA could be used only with Nvidia GPUs. We
chose to implement the proposed approach in OpenCL to build
platform independent approach.
OpenCL has many other capabilities in building parallel
applications on multiple GPU devices. OpenCL supports vec-
tor processing which means it can add two vectors of n length
at the same execution cycle instead of scalar processing in
CPU that takes n execution cycle. Second, OpenCL provides
parallel programming at two levels: data parallelism and task
parallelism. In our implementation, we consider data paral-
lelization in training process in order to handle larger datasets
and process them in parallel.
OpenCL provides an effective solution for many complex
problems that usually demand high resources of memory and
CPU. OpenCL shows extremely high performance in execu-
tion time when the problem is well designed with OpenCL
parallelization parameters: work-item and work-groups.

116
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

V. C ONCLUSION
In this paper, we proposed a method for accelerating
training process of logistic regression using GPU. The speedup
is analyzed and compared in sequential and GPU-based logistic
algorithm. The results showed that OpenCL can afford a
new trend in application development to have performance
boosting and make a full resource utilization. The OpenCL
model allows implementation to be deployed onto different
platforms; however, the maximum efficiency is achieved on
GPU device, the program can also be executed on CPU.
The proposed methods speed approximately reaches 11 times
faster than the speed of sequential logistic regression. The
results demonstrated high impact in performance when taking
into account developing GPU-based applications. In future
work, we will implement the proposed approach in MapReduce
to run it on Hadoop framework as well. The final version of
the proposed approach can process big datasets in distributed
framework like Hadoop and in parallel on GPU devices

ACKNOWLEDMENT
This research was partially supported by XEROX Research
Center in Europe. Thanks to Frederic Roulland and Jean-Marc
Andreoli for their advice in developing the proposed approach.

R EFERENCES
[1] D. W. Hosmer Jr, S. Lemeshow, and R. X. Sturdivant, Applied logistic
regression, vol. 398. John Wiley & Sons, 2013.
[2] U. Lotric and A. Dobnikar, Parallel implementations of recurrent neural
network learning. Springer, 2009.
[3] M. Harris, Gpgpu: General-purpose computation on gpus, SIGGRAPH
2005 GPGPU COURSE, 2005.
[4] K. O. W. Group et al., The opencl specification, version: 2.0 document
revision: 22, URL http://www. khronos. org/registry/cl/specs/opencl-
1.0, vol. 29, 2014.
[5] P. D. Allison, Logistic regression using SAS: Theory and application.
SAS Institute, 2012.
[6] O. Schuessler and D. Loyola, Parallel training of artificial neural
networks using multithreaded and multicore cpus, in Adaptive and
Natural Computing Algorithms, pp. 7079, Springer, 2011.
[7] Z.-P. Liu, Logistic regression, in Encyclopedia of Systems Biology,
pp. 11421143, Springer, 2013.
[8] T.-N. Do and F. Poulet, Parallel multiclass logistic regression for
classifying large scale image datasets, in Advanced Computational
Methods for Knowledge Engineering, pp. 255266, Springer, 2015.
[9] Y. Zhuang, W.-S. Chin, Y.-C. Juan, and C.-J. Lin, Distributed newton
methods for regularized logistic regression, in Advances in Knowledge
Discovery and Data Mining, pp. 690703, Springer, 2015.
[10] H. Peng, D. Liang, and C. Choi, Evaluating parallel logistic regression
models, in Big Data, 2013 IEEE International Conference on, pp. 119
126, IEEE, 2013.
[11] J. B. Kulkarni, A. Sawant, and V. S. Inamdar, Database processing
by linear regression on gpu using cuda, in Signal Processing, Com-
munication, Computing and Networking Technologies (ICSCCN), 2011
International Conference on, pp. 2023, IEEE, 2011.
[12] M. van Heeswijk, Y. Miche, E. Oja, and A. Lendasse, Gpu-accelerated
and parallelized elm ensembles for large-scale regression, Neurocom-
puting, vol. 74, no. 16, pp. 24302437, 2011.

117
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

ROEF: A Smart Search Engine of the 3rd Generation World Wide Web (WWW)
Karim Mahmoud, Hanafy Ismail, Mohamed Kholief
Computer Science
Arab Academy for Science, Technology & Maritime Transport
Cairo - Egypt
kareem.mahmoud.shawqy@gmail.com, drhanafy@yahoo.com, kholief@aast.edu

Abstract The purpose of this paper is to discuss the
new personalized sematic web semantic search engine
ROEF (Register Once Enjoy Forever). ROEF was
created to be a step towards bridging the gap between the
current search engines implementations and the future of
smart and personalized Web3.0. ROEF is a search engine
where a user can register his personal information and
areas of interest, and link his social networking profiles
to his ROEF profile. After registration, every time the
user searches for any query, the results returned are
filtered and ranked by cross-referencing the search query
with the users ROEF profile information. These results
are returned as semantic search documents, which can
then be processed and displayed in both user-friendly
GUI and machine-friendly XML formats respectively,
thus providing a unique and personalized searching
experience for the end-user, and simple service-based
integration with other applications. The product of this
paper is a fully functioning proof-of-concept that is
limited by the lack of a complete semantic knowledge
base, and future enhancements include more advanced
query optimization, performance enhancements, and
developing semantic web crawlers and scutters which
will maintain and expand the semantic knowledge base.
Keywords Web3.0; Semantic web; Personalized web;
Semantic search
I. Introduction
Rapid growths in the fields of communication and
information technology have resulted in the creation of a
near infinite pool of raw data. Data that keeps expanding
greatly with every passing second. This has grown to
become a serious challenge for content providers who
need to catalog and classify this data into related bits and
pieces of information. And it has made it equally
challenging for users who want to find relevant and
personalized information; That is, users would need to
exert time and effort to browse through nearly endless
pages of search results in order to find an answer to their
query [1].
And on the other hand, it has become costly and difficult
for web search engines to categorize the ever expanding
pools of data manually. Hence they created spiders,
robots and crawlers to search the internet automatically
for related pieces of data.
Such problems have motivated researchers to devise
automated techniques to deal with web documents to
facilitate cataloging, organizing and browsing even
further and with minimum intervention. Hence, the idea
to create a relevant and personalized semantic web
semantic search engine "ROEF" was born.
In this section of the paper we will discuss the evolution
of web and the significance of semantic web and
semantic search integration.
Then in the following sections of this paper, we will
highlight related work in section two and system
design in section three, in system design section we will
discuss problem definition, ROEF design and technical
architecture. Next section will explain how ROEF
works with live examples, then shine a light on the
limitations in section 5 that we've faced, and lastly we
will compare ROEF against other partially similar
solutions and evaluate ROEF0 in section six.
Finally we will conclude this paper in section 7 and also
discuss areas of future work.
A. Evolution of Web
The World Wide Web Consortium (W3C) has
overseen multiple WWW milestones, and there are three
primary turning points that determine its life cycle.
Figure 1, shows the evolution of the WWW over the
years; the first turning point is Web1.0 which is based on
static pages, the second turning point is Web2.0 which is
based on dynamic pages, and the last one is Web3.0
which is based on personalization techniques and
semantic technologies [2][7].
Till now there is no standard definition for Web3.0.
However, we believe that the most accurate description
for Web3.0 is that it is more intelligent, open and
connected to semantic web technologies, natural
language processing, distributed databases, machine
learning, autonomous agents and machine reasoning [6].
The main reason for the absence of standards for Web3.0
concepts is its instability [2]. There are numerous
concepts associated with Web3.0 like artificial
intelligence, personalization and semantic web [1][2]. In
short Web3.0 aims at enabling search engines to
understand and respond to complicated human requests
depending on the users personal information and the
context of the request.
Search engines have become the basic backbone of the
web and are an indispensable part of any platform which
makes it quite important for information sources to
become semantically structured by applying the Web3.0
theories to provide the new generations with automated,
smart and powerful search engines [2][10].

978-1-5090-0217-7/15/$31.00 2015 IEEE 118

 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Figure 1: Past, Present and Future of WWW
B. Significance of Semantic Web and Semantic
Search Integration
There is a common confusion between semantic
search and semantic web. This confusion can be clarified
by considering semantic search as a way to improve
search accuracy by comprehending the searchers intent
and the contextual meaning associated with the terms
appearing within the data space. While, semantic web
targets converting the existing semi-structured and
unstructured data web into structured documents
represented as interconnected complicated data with easy
access, and makes it possible for machines to accomplish
tedious tasks like finding, combining and acting upon
information from the web without human directions; as
web pages before it were human readable, not machine
readable.
Utilizing the stable and reliable infrastructure provided
by semantic web, semantic search will be able to perform
its tasks of understanding users requests more
accurately, and providing more context relevant results,
in less time.
II. Related Work and State of the Art
There are multiple research papers discussing the
different types of search engines and their features and
latest changes. Most of which usually classify search
engines into 3 types; semantic web search engines,
semantic search engines and keyword-based search
engines.
Semantic web search engines use the provided
keywords as input to search through semantically
structured data sources. Swoogle [3] can be defined as a
crawler-based retrieval and indexing system for semantic
web documents like OWL or RDF documents. It plays a
significant role in analyzing the documents for
computing the metadata relationships and properties
existing between them [3]. An information retrieval
system is used for indexing the documents using either
URIs or N-Gram to seek documents that match the query
of the users for computing the similarities between the
existing set of documents. The rank associated with every
semantic web document is the most important property to
determine the significance of each document on the
semantic web.
Watson [4] is a semantic web search engine that is not
limited to a service which supports the development of
semantic web applications. It generates a number of APIs
comprising of high-level functions to find, explore and
query semantic ontologies and data that are published
online. Based on such APIs, novel applications have
come up which link the different activities like sense
disambiguation, matching and ontology construction. It
acts as an extraordinary resource for researchers studying
semantic web focusing on ways to produce, share and
consume formalized data and knowledge online [4].
Semantic search engines attempt to understand the
users query and use it as input to search through
non-semantically structured or semi-structured data
sources. Kngine1 which is a famous semantic search
engine, features search outcomes classified either into
image results or web results. They are generally headed
by information regarding the search term referred to as
Concepts.
For instance, looking for iPhone 3GS would be headed
by the specs of the device1. Looking for a film would be
headed by information regarding the film, reviews,
quotes and links to its trailers. Looking for a city would
be headed by information regarding the hotels, weather,
events, local attractions and city. Kngine presently
comprises of 8 million Concepts approximately, which is
the core strength of the site. The results from Kngine can
easily be shared on Facebook and twitter. One can also
set Kngine as the default search engine while using
Internet Explorer, Google Chrome or Firefox.
Another famous semantic search engine is Hakia. The
search results of Hakia2 can be classified as videos,
images, tweets, blogs, news and web pages sorted per
date or relevance. Based on the term being searched, the
results might also include an excerpt from the Wikipedia
entry.
The Falcons Semantic Web Search engine emphasizes
mainly on the different aspects of user interface
providing the primary API comprising of a sub-set of
functions which are offered by Watson. The rest of the
system emphasizes on a limited number of functionalities
or scenarios not being used and developed further [5].
1
http://www.kngine.com/Technology.html
2
http://en.wikipedia.org/wiki/Hakia
119
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

It should be noticed that the collection of semantic a clear and machine-readable XML based structure.
data from web has acquired a wider scope with few of the Figure 2, explains how ROEF process queries.
features appearing in the typical web search engine
which is a way of exploiting the structured data for
improving the presentation and search experience.
Semantic information which is embedded within web
pages as micro formats or RDFs provide, enriched
snippets which describe the different web pages within
the search results. Google Rich Snippets in the same way
utilize the accumulated semantic data with the help of
particular schemes for adding information to the results.

III. System Design

A. Problem Definition
As described above, todays available search engines
can be classified as semantic web search engines,
semantic search engines or keyword-based search
engines. Still, these search engines are far from fully
applying the principles of Web3.0; they lack either one or
more of the concepts of personalization, intelligence and
the ability to visualize semantic data in an understandable
and user-friendly format.
This study discusses the design and development of a
new search engine named ROEF. It targets the full
implementation of Web3.0 concepts. It aspires to surpass
all existing search engines by providing a full package of
Web3.0 functionalities that most of the other engines are
lacking. ROEF offers an intelligent and personalized
search experience that evolves and learns from the nature
of the users search patterns and their areas of interest. It
also provides semantic web data visualization in a format
that is both understandable to technical users and
user-friendly to non-technical users.

B. ROEF Design
Figure 2: ROEF Workflow

The proposed solution implements Web3.0 standards;

semantic web, intelligent web, and personalization, aiming
to create a new search engine that is able to understand user
search queries, personalize and present the results in both
a user friendly machine-to-human GUI, and a semantic
machine-to-machine XML based ontology.
User interface: Interface component represents all the
user interfaces which are involved in the ROEF engine
used by clients starting from the welcome page and
registering moving to displaying the search results. Its
characterized by simplicity and easy navigation through
results.

From a machine-to-human perspective; ROEF works
as a personal digital assistant that is able to understand
the users search queries and provide context-relevant
results that are tailored to that specific user, based on the
users personal ontology.
From a machine-to-machine perspective; ROEF is
able to understand and respond to parameterized machine
initiated requests in the same way it understands and
responds to human user queries, but provide the results in
Semantic Search (Query Analyzer): This component
analyzes the query and understand the query context
using semantic search methodologies. For such it is
required that the user has logged in ROEF. ROEF will
have two types of search: Faceted Search and Semantic
Search.
Personalization: This module will search interest/hobby
belonging to Category & Sub Category in user ontology.

120
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Semantic Web Data Discovery: It is responsible for the

If user location is relevant for search, it will also get the
round trip of the search parameters between receiving it
user location form ontology.
after personalization, then passing it as input to an
For example: If the user is searching for I want to external invocation for a set of a restful web service for
eat then from above module we come to know that the following semantic search engines (Watson -
he/she is looking for Category-Food & Sub Category Swoogle - Sindice) and retrieving semantic web
like-Meal, Dessert, Restaurant. So, this module will documents (SWDs) to send it to Filter & Ranker.
return users favorite Meal, Dessert, restaurant from
his/her ontology[11]. It will also give the users location Filter & Ranker: Each search engine has its own indexer
as it will be required to find Restaurant. Now search and this module is using the default indexer of these
string will be modified based on the above information. search engines. The system doesnt have its own indexer
Apart from above personalization based on user ontology, as it has no own semantic data to search but this module
this module is also used following approach to improve rank the results returned from these search engines based
personalized search string to further improve search on a simple algorithm to refine & filter order of search
result before passing to semantic search engine to get results.
results.
This module will take a list of URLs populated from
1. Incorporates current location, sometime user semantic web data discovery module as a result of search
location may be different from users native location. and rank (or assign weight) to each URL based on
This functionality will add current location to further Table1.
enhance results.
Example: Suppose user searches for car repair then Table 1: Ranking Criteria
adding its current location may help in deterring Criteria Descriptions Weight
car-repairing workshop. Valid:1000
Ping each URL to check valid or
2. Determine the intent and contextual meaning or Ping Invalid:
invalid
-1000
keyword-to-concept mapping: Example: Supposes user
searches for car then intention can be bought or RDF The URL points in RDF documents 500
repaired. OWL The URL points in OWL documents 500
3. Pattern matching, fuzzy logic, word variation to The URL points in HTML
HTML 100
documents
allow errors in key Key
4. Synonyms of a term. For each key found in URL String 50
Mapping
Example: I want to buy a house.The house can be
synonymous with property.So; this sentence can also
be considered as I want to buy a property.
Figure 3, explains the process of semantic search &
personalization.

121
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
After assigning a weight to each URL, the system
calculates total weight for each URL and short URL on
the value of total weight in descending order.
It allows selecting minimum allowable weight/rank as
cutoff value. A URL with a rank value less than the
minimum allowable weight/rank will not be displayed to
the user.
Visualization Component: This component is
responsible to convert SWDs returned from a search
result in an understandable format for normal user. It can
render SWD as HTML or Graph. Figure 4: explains the
architecture of HTML and Graph visualization service.
It uses Apache Jena to parse SWDs and HTML & SVG
for rendering. It draws one HTML page for all ontology
instances in an understandable format for normal user. In
Graph, an ontology instance is represented as node
which is connected to other ontology instance node or
object with arrow having suffix as the name of the
predicate. And Technical user also can convert SWDs in
different format like: N3, N-Triples, RDFa and JSON.
Figure 4: HTML & Graph Visualization
C. ROEF Technical Architecture
In the previous section we have discussed ROEFs
main components and ROEFs workflow. In this
section we will dig deeper into the technical details of
ROEFs architecture (Figure 5), we will discuss
ROEFs main UI screen, ROEFs search module, and
how ROEF handles users ontologies.
Main pages of user interface:
Facebook login: Login validation will be done through
Facebook accounts.
General login: Login validation will be done through
user id & password stored in ROEF database.
Facebook Registration: User will require to login
through Facebook account in ROEF. On successful login,
ROEF will get personal details from Facebook and
missing information will be required to enter by the user.
General Registration: User will input personal
information, user id & password. The user should enter
maximum information such as education, religion,
addresses, nationality, work, date of birth, sports, hobbies,
preferences in movies, actors/ actresses, news, articles,
etc. to get benefit of ROEF.
Search Page: Through this page user can perform search
& it also displays the test results.
Ontology Store & Ontology Query: This module stores
personal details (User Ontology) of user as RDF model
& provide a way for the SPARQL query. It stores RDF in
dataset TDB. The Apache Jena framework is used to
serve this purpose. The benefit of using TDB instead
of DB :
User information is getting stored as an ontology.
User ontology can be displayed in different formats
like-RDF/XML, Turtle, N-Triples.
We can retrieve information using SPARQL.
Data can be easily shared with other semantic website.
Faceted Search: It provides the way to query based on
predefined category and sub category based on user
ontology. Apache Lucene is used for this purpose.
Benefits of faceted search are faster searching through
the structured filtering of results, therefore enabling
users to find the content he/she is looking for quicker,
faster indexing of content and drilling down in search
results by data grouping.
Plane Text Semantic Search: Here user can input text for
search. Like-I want to eat or I want watch movies or I
want to read books etc.
This module preprocesses search string to understand
the concept of the search string. It has two parts:-
POS Tagger: It tags every word in the search string
with its appropriate parts of speech (POS). Based on
tagging, search string will be broken into different
pairs/words (like-noun, verb, adverb, adjective etc.).
Apache open NLP POS Tagger is used for tagging.
WordNet: WordNet is used to find hyponym,
meronym & synonyms of verb & noun return from
POS Tagger. It helps to understand the concept of
search as well as to map search context to Category
(like-Food) and Sub Category (like-Meal, Dessert,
Restaurant) of the user interest / hobby. For
example: if a user is searching for pizza, then the
122
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
system should understand it belongs to Food
category and can be found in Sub Category-
Restaurant (like Pizza Hut). If user wants to eat, then
system should co-relate that user is looking for Food
Category& Sub Category like-Meal, Dessert,
Restaurant. If, this module is able to identify
Category & Subcategory, it will be passed to
personalization module to identify users
interest/hobby related to that Category &
Subcategory from user ontology.
IV. How ROEF Works
Figure 6 shows ROEF home page, which provides
the search textbox.
Figure 6: ROEF Home Page
Let us, suppose a user from Nasr City, Cairo, Egypt
login to ROEF & type a text I want to eat Dessert in the
text search box. Flow of control through different
modules will be as following:
1. User input will go to the Semantic Search
module.
2. POS Tagger of Semantic Search module will
split sentence to noun, verb. The output of this
module will be-want, eat, dessert.
3. Output (want, eat, dessert) from step-2 will pass
to WordNet module and its output will be given
context that dessert is a type of food.
4. Now dessert will be searched in the food
category of users ontology.
5. Let us assume in user ontology, ice cream was
found as favorite.
6. Favorite restaurants will also be searched in
user ontology. Say in user ontology Smug
International Restaurant is mentioned as
favorite restaurants.
7. Now, Personalization module will build a new
search string like-dessert, Ice Cream, Smug
International Restaurant, Nasr City, Cairo,
Egypt. This module gets location information
from user address which was entered during
registration.
8. The new search string will be passed to
Semantic Web Data Discovery module.
123
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

9. Filter & Ranker module will rank and filter the 2. It displays ontology instances as graph/html
result returned from Semantic Web Data and hides information about classes or
Discovery modules. namespace which are not relevant to normal
10. Now the result will be displayed to the user.
users. Due to this, by seeing graphics and
11. Normal users can view SWDs as HTML or a
graph. HTML, it is easy to understand content of the
12. Technical user can see SWDs in different file.
format like: RDF, RDFa and OWL.
The following comparison shown in (Table 2)
V. Limitations highlights ROEFs functions against the other
state-of-the-art search engines, excluding all features that
For ROEF to achieve its full potential as a smart are not targeted by ROEFs implementation.
semantic web search engine, it needs to have its own
complete semantic knowledge base, which doesnt exist. Table2: ROEFs functions against the other state-of-the-art
After creating this knowledge base ROEF also needs search engines
semantic web crawlers and semantic web scutters which Swoogle Watson Falcon ROEF
will be responsible for grabbing and indexing new
Semantic
information in order to append to the already established Search
N Y Y Y
ever-expanding semantic knowledge base. SWD/Graph/
Result Type SWD SWD HTML
HTML
Semantic Web
Currently and despite the fact that neither the semantic Service
Y Y N Y
knowledge base or the semantic web crawlers or scutters Personalized
N N N Y
exist, by using other already established, but incomplete Search
semantic web data sources; ROEF was able to showcase
its features as a smart search engine and a baseline for Ontology-Based Y Y Y Y

future works in the field. Visualize SWD as

N N Y Y
HTML
Visualize SWD as
VI. Comparison and Evaluation Graph
N N N Y

A. Comparison Let us consider following example to discuss the

above comparison and Table 3 will explain; suppose, we
The following features make ROEF differ from other have a 16 year old student, who lives in London and is
semantic search engines like Swoogle, Watson and studying at the faculty of computer science. His interests
Sindice: include watching movies(The Dark Knight and Harry
1. It uses semantic search component to Potter), eating (Pizza, Rice, Milk), reading (Times
understand the concept behind the search. Magazine), watching TV (ABC News), watching
2. By understanding the concept behind the football (FC Barcelona), reading books (Java
search, it personalizes search by retrieving Programming).
information from user ontology.
3. Result (RDF or OWL file) can be displayed as Q1: I want to eat.
HTML page or Graph. Q2: Where can I find a club?
Q3: I want to watch a movie.
4. It is expandable. Currently it is using Swoogle,
Q4: I want to read and play.
Watson and Sindice but can be expanded very
Q5: What is the latest news?
easily by adding more web services to retrieve
better results.
Table 3: ROEF results against other search engine results
The following features make ROEF differ from Query No.
Swoogle

Watson

Falcon

ROEF

other SWD visualization tools, like OWLGrd1, No. of results from

LODE 2 [9] and Rhizomik3: search engine
1. It is designed, so that a normal user can easily Q1 0 0 100 78
understand the content of RDF & OWL file. Q2 0 368 34 62
1
Q3 0 0 52 111
http://owlgred.lumii.lv/ Q4 1 0 39 55
2
http: //www.essepuntato.it/lode Q5 2 1415 98 89
3
http://rhizomik.net/html/

124
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
The quality of ROEFs results depends mainly on the
quality and quantity of information present in the users
ontology. Hence, the user should enter correct personal and
interests information during registration. ROEF also
evolves users ontology based on search history by
identifying new areas of interest. If ROEF finds that any
word is getting searched frequently, it prompts the user to
map it to the right interests category -make user ontology
learnable ontology- and this information will be appended
to users ontology.
B. Evaluation of ROEF
Most search engines are evaluated by calculating
absolute precision and recall metrics. However, ROEF -as
stated above- depends on other semantic web search
engines' web services and crawlers to fetch its results. Thus
we should calculate precision and relative recall [8][12]
instead of absolute values.
Yet, due to the fact that ROEF has another unique feature
"Personalization" so precision and relative recall
metrics alone will not be able to fairly evaluate its
power. Hence, we have innovated two new terms to
evaluate ROEF in a more holistic manner;
Personalized Precision, and Personalized Relative Recall
which we will discuss next.
Sum of the scores of personalized
documents retrieved by ROEF
(1)Personalized Precision =
Total number of personalized documents
selected for evaluation
Total number of personalized documents
retrieved by ROEF
(2) Personalized Relative
Recall =
Sum of personalized documents retrieved by three
search engines (Swoogle, Watson, Sindice)
Unfortunately, even after innovating the new terms we
will still not be able to accurately calculate and represent
solid numbers for this metric, due to the lack of a complete
dataset at the time of conception of this research. Yet, as
soon as a complete dataset becomes available, and by
applying the above formulas we will be able to accurately
calculate and represent both the Personalized Precision,
and Personalized Relative Recall metrics.
VII. Conclusion and Future Work
This research has attempted to study and implement
Web3.0 principle concepts by creating a new smart
search engine ROEF. ROEF is a strong step towards
achieving the intelligent, personalized, and semantic web
revolution vision.
It has the ability to interact with both humans and
machines, and it provides tailored results that are both
accurate and relevant to the search query and its context.
Future plans for ROEF include 1Query Optimization in
order to be able to understand user queries in a more
accurate way. 2Performance Enhancements in order to be
able to analyze user queries and fetch the most relevant
results in less time. 3Semantic knowledge base which
needs to be created in order to serve as a stand-alone
comprehensive semantic data source for ROEF.
4
Semantic web crawlers and scutters which need to be
implemented in order to fetch new information and
expand the ever growing semantic knowledge base.
References
[1]. Farah, Josiane. "Predicting the Intelligence of Web 3.0
Search Engines." International Journal of Computer Theory and
Engineering, 2012: 11-32.
[2]. Sabbagh, Karim, and Olaf Acker. "Designing the
Transcendent Web the Power of Web 3.0 ." Booz and
Company, 2011: 44-87.
[3]. Finin, Tim, and Yun Peng. "Swoogle: A Search and
Metadata Engine for the Semantic Web ." CIKM, 2004: 44-87.
[4]. dAquin, Mathieu, and Enrico Motta. "Watson, more than
a Semantic Web search engine." IOS Press, 2011: 44-67.
[5]. Gong Cheng, WeiyiGe, and Yuzhong Qu. Falcons:
searching and browsing entities on the semantic web. In WWW
conference, pages 11011102. ACM, 2008.
[6]. Nova Spivack, Lifeboat Foundation Advisory Boards.
Available online[http://lifeboat.com/ex/web.3.0], access date:
30 Jan 2014.
[7] . Nasr, E. A. . Agent Based Services for the Semantic
Web. Master of Science in Systems and Computers
Engineering, Al- Azhar University Faculty of Engineering
33-89, (2012).
[8]. T. Usmani, D. Pant, A.Bhatt. A Comparative Study of
Google and Bing Search Engines in Context of Precision and
Relative Recall Parameter, International Journal on Computer
Science and Engineering (IJCSE), 2012.
[9]. Silvio Peroni , David Shotton , and Fabio Vitali. The Live
OWL Documentation Environment: a tool for the automatic
generation of ontology documentation. Protg, 12-89. (2012).
[10]. Rani, Monika, Satendra Kumar, and Vinod Kumar Yadav.
"Optimize space search using fcc_stf algorithm in fuzzy
co-clustering through search engine."International Journal of
Advanced Research in Computer Engineering & Technology 1
(2012): 123-127.
[11]. Rani, Monika, Maybin K. Muyeba, and O. P. Vyas. "A
hybrid approach using ontology similarity and fuzzy logic for
semantic question answering." In Advanced Computing,
Networking and Informatics-Volume 1, pp. 601-609. Springer
International Publishing, 2014.
[12]. B.T. Sampath Kumar. "Precision and Relative Recall of
Search Engines: A Comparative Study of Google and Yahoo "In
Singapore Journal of Library & Information Management .
2009:Volume 38.
125
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Mitigating ARP Spoofing Attacks in
Software-Defined Networks
Ahmed M.AbdelSalam, Ashraf B. El-Sisi
Faculty of Computers and Information,
Menoufia University.
Menoufia, Egypt
ahmed.abdelsalam@ci.menofia.edu.eg
ashraf.elsisi@ci.menofia.edu.eg
Abstract Software-Defined Networks (SDN) are emerging
as an attractive solution to overcome the limitations of traditional
networks. They provide network programmability and promote
rapid innovation in protocol design, network management and
network security. Today, network security is the most important
concern for any computer network administrator. Traditional
networks had several security problems, some of them no longer
exist with the advent of SDN and others still do exist. ARP
spoofing or ARP cache-poisoning attack is mainly seen in LAN
networks, which has no efficient solution to mitigate in
traditional networks but SDN provides a unique way to solve this
problem without any changes in the network. ARP spoofing is
exploited in different forms, mainly request and reply based
attacks. In this paper, a solution is proposed to mitigate both of
these types of ARP attacks in Software defined networks. The
proposed solution extends the SDN controller to include an ARP
module, which detects and stops the attack, and avoids
overloading or Denial of Service on the controller. The solution is
fast, reliable and tested for different attack scenarios. Openflow
is used for the communication between the controller and
switches, emulated by Mininet.
Keywords: Software-Defined Networking; ARP; ARP spoofing
mitigation; Security.
I. INTRODUCTION
Traditional computer networks are built from large number
of end devices like PCs, Servers, Printers, IP Phones and
intermediate devices such as hubs, switches, routers, access
points and firewalls. These intermediate devices mainly
forward data from one end device to another. They incorporate
special algorithms to build their forwarding tables and forward
the traffic accordingly. They are vertically integrated: the
control plane and date plane are coupled together. Control
plane has the logic to handle the traffic and the data-plane
forwards the traffic based on the decisions made by control-
plane. Devices operating in traditional networks have many
proprietary features that are not easy to configure and may lead
to some incompatibility between devices from different
vendors. In addition, traditional networks have become very
hard to manage in a large scale like Datacenters. They require
network engineers for configuration and maintenance.
Consequently, this increases the operational cost of a network.
Network protocols are also hard to evolve, they require years
of testing and standardization before being used into
978-1-5090-0217-7/15/$31.00 2015 IEEE
Vamshi Reddy K
Department of Computer Science & Engineering,
Manipal Institute of Technology.
Manipal, India
k.vamshi2008@gmail.com
production network. Moreover, they have no built in security
features.
Software-Defined Networking (SDN) is a new paradigm in
networking that changed the way the traditional networks are
built and managed. It aims to solve most of the limitations in
traditional networks by decoupling the control plane from the
data plane. In SDN, network devices have become simple
forwarding devices and only implement the data plane logic.
The control or network intelligence is implemented in a
centralized controller, as the controller is the network brain.
SDN will shape networking by giving the network owner or
operator the ability to customize and optimize their networks.
It speeds up the innovation in computer networking at software
development speeds rather than a slow hardware driven
approach present in the traditional networks.
Traditional networks have several security problems, as
they are not designed with security as the main criteria. With
the increase in the number of nodes connected to a network,
security becomes the most important factor for the successful
communication. With emergence of SDN, most of these
problems no longer exist but there are some, which threaten
even todays networks. Spoofing attacks were first discovered
in early traditional networks, they cannot be mitigated with a
plain SDN controller. When a user impersonates another users
identity on a network it will be considered a spoofing attack.
By using this attack, an attacker can steal other users data or
bypass the access control. IP spoofing, DNS spoofing, and
ARP spoofing are different forms of spoofing attacks.
ARP spoofing is the most common form of spoofing in
Local Area Networks (LAN). Section II of this paper will
present the details of ARP Protocol and ARP spoofing attacks.
The efforts previously done to mitigate ARP spoofing Attacks
in traditional networks will be examined in section III. In
Section IV, the proposed solution to ARP spoofing problem in
SDN will be shown. Furthermore, in Section V, we have
tested the proposed solution in different scenarios to show that
the attack can be detected and completely mitigated. In Section
VI, the results for the conducted experiments are discussed.
Finally, in Section VII, the paper is concluded with further
modifications, extensions and enhancements to the proposed
architecture.
126
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
II. ARP SPOOFING
ARP [1] protocol is specified in RFC 826 [2]. It is crucial
for successful communication in Computer Networks. Sending
devices use ARP protocol to know the MAC address of the
next hop device in path from the sender to receiver. Every host
in the network has its own ARP cache that contains the recent
IP to MAC bindings. These mappings can be learnt
dynamically by ARP protocol or can be added manually as
static rules. If two hosts host_A and host_B want to
communicate, host_A searches in its ARP cache for the next
hops MAC address. If search is successful, then it uses the
returned MAC address, or else, ARP protocol will start to
operate for it. In ARP protocol, host_A sends an ARP request
for the next hop IP by broadcasting its request into the
network. host_C, which is one hop away from host_A in the
path to host_B, replies with an ARP reply message. host_A
updates its ARP table with the next hop IP, MAC pair sent by
host_C and subsequent IP packets use this cached entry present
in the table. One more consequence of ARP protocol is that
host_C also adds an entry for host_A IP-MAC in its ARP cache
table.
ARP spoofing is a technique used by attackers to perform
cache poisoning by inserting false IP to MAC address
mappings in victims ARP cache. ARP spoofing attack comes
in different forms namely request and response attacks. In
request attack, an attacker broadcasts ARP request message
with forged source IP-MAC in the ARP header. When victim
receives this spoofed ARP message, it updates its ARP cache
table with the attackers forged IP-MAC pair. When the victim
sends its subsequent packets destined to the forged IP, the
packets will be destined to the MAC of the host specified by
the attacker in the forged IP-MAC pair. This way attacker can
intercept or deny the traffic sent to a user in the network. The
other form of attack is the response attack, in which the
attacker will either respond to normal ARP request with forged
ARP replies that maps the next hop IP to the attacker MAC
address or send spoofed ARP replies without having requests
being issued. ARP spoofing may be used to launch either one
of the following attacks [3]:
1. DoS attacks: the attacker will prevent the two
communicating hosts from getting connected to each
other.
2. Host impersonation attack: the attacker will receive
packets intended to the victim and can reply to these
packets on behalf of the victim.
3. Man-In-The-Middle (MITM) attack: the attacker will
be able to monitor all the traffic between two
communicating hosts.
Due to the popularity, importance, and danger of the ARP
spoofing attacks, various tools have been implemented to
launch this attack. These tools are available on most of the
operating systems. Ettercap [4], Dsniff [5], Yersinia [6] and
Cain and able [7] are the most popular tools used to launch
ARP spoofing attacks. They can also be used to execute other
data link layer attacks such as MAC flooding attacks, STP
attacks, DHCP attacks, and VLAN attacks.
III. MITIGATING ARP SPOOFING ATTACKS IN
TRADITIONAL NETWORKS
The several solutions proposed to solve ARP spoofing
problem in traditional can belong to either one of the following
categories [8]:
1) ARP authentication: ARP protocol will be modified in
order to not process ARP packets unless it contains a
predefined cryptographic code. S-ARP [9] and TARP [10]
are examples of these techniques. But this method leads to
some incompatibility with those devices that still operate
using the standard ARP protocol.
2) Operating system patching: The operating system will
be patched to prevent ARP spoofing. However, this technique
require a different patches for different operating systems
kernels. Moreover, the patched operating system may not
communicate correctly with other operating systems that
might not have been patched. Anticap [11] and antidote [12]
are the solutions for ARP spoofing problems, that patch the
operating system kernel.
3) Dynamic ARP Inspection (DAI): Some switches (e.g.,
Cisco catalyst) are equipped with a security feature called
Dynamic ARP Inspection (DAI) [13]. Switches use DAI to
stop ARP spoofing attacks by intercepting all ARP requests
and responses and dropping packets with invalid IP-to-MAC
address mappings. The main problem of this solution is that
not all switches are equipped with the DAI feature. It also not
viable for most organizations to buy new switches equipped
with the DAI feature instead of the old ones.
4) ARP mitigation tools: There are several software tools
that have been developed to mitigate the ARP spoofing
attacks. Some of these tools prevent the attack and others only
detect the attack. XArp [14], ARPWatch [15], Anti Netcut
[16], NoCut [17], and AntiARP [18] are example of these
software tools. However, these have many problems such as
high rate of false positives and false negatives and
ineffectiveness.
5) Static ARP mappings: The most effective way to
prevent ARP spoofing attacks is to configure the ARP entries
manually. This way, the attacker will not have any chance to
spoof the other hosts ARP cache. But, this solution is not
viable for large scale networks. It cant be used in networks
that use DHCP addressing schemes and in Datacenters, where
there is a frequent change in IP addresses. In addition, it will
be error prone and requires a great overhead on the network
administrator. Several solutions have been proposed in attempt
to ovecome these drawbacks. DAPS [8], NIDPS [3], and the
techniques proposed in [19], [20], [21] are examples of
solutions that make use of static ARP entries. Each of these
techniques have their own limitations.
127
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
IV. MITIGATING ARP SPOOFING ATTACKS IN
SOFTWARE DEFINED NETWORKS
Network security in SDN can be improved either by using
SDNs features and capabilities or by solving the security
problems of SDN itself [22]. SDN gives us the ability to gather
the required data from the network and provides a way to
analyze and detect the attack signature. After the analysis,
network can also be reprogramed for enforcing any security
policy. The proposed method is an application over SDN POX
controller written in python; it will prevent LAN attackers from
poisoning the ARP cache tables of other hosts present in the
network. This application does the following:
It detects and mitigates ARP request and ARP reply
spoofing attacks.
It does port level ARP packet count monitoring, to
detect large influx of malicious packets and stops them
by installing Openflow rules.
It avoids controller overloading by installing flow
entries on switches to drop packets coming towards the
controller, in case the attacker tries to bring down the
controller.
It differentiates edge switches (where security policy
enforcement should be applied) from the other
intermediate switches. (Where usual forwarding logic is
applied).
The flowchart of the proposed application is shown in
Figure 1. The proposed application initializes the L2 learning
Fig. 1: ARP spoofing mitigation flowchart
switch modules present on the controller for each of the
switches present in the network. It then starts the DHCP server
for serving the lease requests by the hosts on the network.
DHCP server is required to be on the controller, as the
controller will keep track of the IP leases and uses them later in
detecting the attack. Edge switches have the hosts connected to
them
Then, the controller starts monitoring on each of the ports
on the switches connected to it. It also installs flow entries on
the edge switches to forward ARP and DHCP lease packets to
it for analysis. When the controller receives a packet from the
edge switch, it will process according to the protocol present in
the packet. ARP and DHCP packets are handled according to
the proposed logic to detect any attack. If a packet comes from
an intermediate switch, controller does plain L2-L3 processing
on the packet. The following classification is done to describe
how the DHCP and ARP packets are handled.
A. DHCP packets
The DHCP protocol is used to dynamically assign IP and
other network configuration parameters to the hosts. A DHCP
client will exchange messages with the DHCP server present
on the controller to get IP address and other parameters. The
controller extracts the IP and MAC addresses from the DHCP
header and uses them to add an entry into the known hosts list.
This known hosts list will keep track of all the IP addresses
leased by the DHCP server.
128
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
B. ARP packets
The controller handles ARP packets based on the packet
type namely request or reply.
1) ARP request packets: ARP request packet will be
considered spoofed, if it satisfies one of the following
conditions
Source MAC address of ethernet header and source
MAC address of ARP header are not the same.
Destination IP address in the ARP header is not present
in the known hosts list of the DHCP server.
The MAC address binding present in the known hosts
list for the Source IP of the ARP header doesnt match
with the source MAC address of the ARP header.
2) ARP reply packets: the ARP reply packet will be
considered spoofed, if it satisfies one of the following
conditions
Source MAC address of ethernet header and source
MAC address of ARP header are not the same.
Destination MAC address of ethernet header and
destination MAC address of ARP header are not the
same.
The MAC address binding present in the known hosts
list for the Source IP of the ARP header doesnt match
with the Source MAC address of the ARP header.
The MAC address binding present in the known hosts
list, for the Destination IP of the ARP header doesnt
match with the Destination MAC address of the ARP
header.
Destination MAC address of the ethernet header has a
value of (FF:FF:FF:FF:FF:FF).
If any of these stated conditions is true, the controller
knows that this is an attack signature, installs a flow entry on
the forwarding device, to stop the incoming malicious traffic.
If an attacker knows the current algorithm that the ARP
packets are sent to the controller for inspection, then he can
simply send large number of ARP packets to perform DOS
attack. To avoid such problem, the controller in this proposed
solution keeps track of the number of received ARP packets on
a certain switch port by using the port-monitoring algorithm.
This algorithm collects the ARP packet count at successive
intervals. Port monitoring algorithm is run on every port
present on all of the edge switches that keeps track of the ARP
packet counts for every second. At any time, if controller
witnesses a spike in the ARP packet count for a port, it installs
a flow entry on the switch to stop packets coming from that
port for a specified amount of time. This prevents the attack
packets entering into the network and protects the controller.
V. IMPLEMENTATION
Experiments are conducted for various attack scenarios to
prove the feasibility of the proposed solution. The details of
these experiments are shown in this section.
A. Test Enviroment
The experiment is conducted on an Ubuntu Virtual
Machine (VM) with 2 cores and 2 GB of RAM. The host
machine is running on Windows 8 and has Intel Core i5
processor with 4 GB of RAM, 4x1 Intel Core i5 cores with
Intel VT enabled.
Mininet [23], a network emulator is used for creating the
testbed. It creates a virtual network of hosts, switches,
controllers and links according to a given topology. Hosts and
switches created by Mininet have Linux OS and Open Virtual
Switch software (OVS) respectively. These switches support
Openflow and are orchestrated by a SDN controller. The
Mininet emulator is installed on Ubuntu 14.10 Linux virtual
machine (VM) that runs on VirtualBox. The VM has two
network adapters, Network Address Translation (NAT) adapter
and Host-only adapter. The NAT adapter allows the VM to
connect to the internet for downloading the required software
packages. The Host-only adapter is used for the
communication between the VM (guest machine) and the
physical machine (host machine). POX [24] is used as the SDN
controller. DHCP server is also run on the POX controller as a
service, with a predefined lease IP range to serve the clients on
the network. The proposed solution is run as a module on the
POX controller for the tests.
B. Experiment Setup
The topology used to test the proposed solution is shown in
Figure 2. The topology consists of three Openflow OVS
switches (S1, S1, and S3). Switches S1 and S2 are edge
switches and S3 is intermediate (core) switch. These switches
are connected in a tree topology. It has 4 end users (H1, H2,
H3, and H4), three of them (H1, H2, and H3) are normal users
and the fourth one (H4) acts as an attacker. This topology also
has a single controller (default POX controller with the
proposed ARP spoof detection module), created by the Mininet
emulator. All the emulated links have 100 Mbps bandwidth
with 5ms delay, 0% loss and with a maximum packet queue
size of 1000. Every host in the emulated topology has the same
CPU-performance.
C. Test Scenario
Three different attack scenarios are created to verify the
attack mitigation by using the proposed algorithm. In order to
execute the tests, an ARP spoofing software (C code) called
arp-spoof is developed by using UNIX raw sockets, to generate
spoofed ARP request and reply traffic. ARP request attack,
ARP reply attack, and DoS attack against the SDN controller
are the attack scenarios tested in this section.
Tests are performed on each of the attack scenarios to
evaluate attack detection time, attack mitigation time, load on
the controller and throughput of the network. Iperf, a tool,
which collects statistics about the network connectivity and
throughput of the path between two hosts, is used. Packet loss
and throughput statistics between two non-malicious hosts in
the network are collected while an attacker is executing his
ARP cache-poisoning exploit. At the same time the other
statistics like Attack detection and mitigation time are
tabulated.
129
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

the controller by the switches. Controller analyzes these

incoming packets and does nothing but forwarding; this may
bring the controller down as the controller is oblivious to the
fact that attacker can overhaul it by sending a large number of
good ARP packets. This problem is solved in the proposed
solution by introducing Port Monitor module that counts the
number of ARP packets received on every switch port. If this
number reaches a pre-defined threshold (threshold is 100 in
this scenario), the controller installs a flow entry at the switch
to drop traffic from this port for a pre-defined amount of time.
To test this, we have used the same tool arp-spoof to generate
good ARP packets with a high rate and injected them into the
network. POX controller running the proposed algorithm
detects that this packet flow is abnormal and stops it from
entering into the network.
Fig. 2: Test Experiment Setup

For example, the attacker Trudy is the one, sending VI. PERFORMANCE EVALUATION
spoofed ARP packets, and the victim Alice is the one, which Attack detection time and mitigation time for the
gets affected by the Trudys packets. Trudy spoofs the identity topology shown in Figure 2 are measured and shown in the
of another host Bob with its own MAC address and poisons Figure 3.
Alices ARP cache, to prevent Alices communication with
Bob.
Attack detection time is the time elapsed between the
initiation of an attack by the attacker and the detection of the
attack by the controller. Attack mitigation time is the time
between the detection of the attack by the controller and the
installation of the flow rules on the switch. Load on the
controller is the process load on the CPU.

D. ARP request attack

The first form of ARP spoofing attack is the ARP request
attack. This is exploited by sending a storm of spoofed ARP
requests to into the network to poison the victims ARP cache.
To simulate this, the arp-spoof is run with the request attack
option selected. This will inject the packets into the network at
a rate of 1000 packets/second. Now, the victim host in the Fig. 3: Attack detection and mitigation time
network will accept the attackers spoofed packet and fills its
ARP cache table with a wrong IP-to-MAC address mapping of Throughput test is also performed for the topology shown
the spoofed identity. The victim host will not be able to in Figure 2. Link bandwidth of 100 Mbps is assumed for all
communicate with others in the network until its cache gets the links in this test. Figure 4 shows the graph with the various
refreshed. When we run the proposed mitigation solution on values of TCP connection throughput before, during and after
the controller, it will detect the request attack at the switch on the attack.
which the attacker is connected and installs a flow entry to
drop packets coming from the attackers port. This way,
attackers packers are filtered at the switch nearest to him,
thereby protecting the entire network.
E. ARP reply attack
ARP reply attack is another type of spoofing attack similar
to the request attack. Here, the cache of the victim is poisoned
by the unsolicited ARP replies sent by an attacker with a
spoofed identity.
F. Denial of service against SDN Controller
Denial of service attack (DOS) is not a form of ARP
spoofing but always comes in conjunction with ARP attacks.
This happens when the attacker tricks the proposed algorithm
by sending a continuous burst of non-spoofed ARP packets.
According to the algorithm, the ARP packets are diverted to Fig. 4: Throughput test between the victim and a host.

130
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Load on the controller for the two attack scenarios are
shown in the Figure 5 below. Firstly, in the request ARP
spoofing attack, the controller as soon as it detects an attack,
installs a flow entry on the switch to drop all packets at the
switch level, thereby avoiding any load. Lastly, DOS attack
with the non-malicious ARP packets; controller uses the port
monitor to prevent any load. The results are quite impressive
showing minimal or no overhead on the controller.
Fig. 5: CPU Utilization on the POX controller.
VII. CONCLUSION AND FUTURE WORK
In this paper, a solution to ARP spoofing problem in
Software-Defined Networks is presented. We started with a
discussion of the main differences between traditional and
Software-Defined Networks and then focused on security
concerns for these networks. The proposed solution makes use
of the features of SDN to reliably mitigate both ARP request
and reply attacks with minimum latency. This solution uses
port level ARP packet monitoring to prevent Denial of Service
(DoS) attacks against the controller and doesnt have any extra
overhead in the network. It also requires no infrastructure
changes, when compared to other traditional ARP spoofing
solutions.
This solution is currently described for a single controller
in a LAN network, it can be extended to handle ARP attacks in
a multiple controller setup with high availability. The proposed
solution can be extended not only to mitigate ARP spoofing
attacks but all kinds of data link layer attacks. A survey can be
made to research on how to use SDN features to provide a
complete solution to prevent most of the Spoofing attacks.
REFERENCES
[1] W. R. Stevens, TCP/IP Illustrated, Vol. 1: The Protocols, Addison-
Wesley Professional Computing Series, 1994.
[2] D. C. Plummer, "An Ethernet Address Resolution Protocol," November
1982. [Online]. Available: https://tools.ietf.org/html/rfc826. [Accessed
01 August 2015].
[3] Bhirud, S.G. and Katkar, V., "Light weight approach for IP-ARP
spoofing detection and prevention," in 2011 Second Asian Himalayas
International Conference on Internet (AH-ICI), Kathmandu, 2011.
[4] A. Ornaghi and M. Valleri, "Ettercap," [Online]. Available:
http://ettercap.github.io/ettercap. [Accessed 01 August 2015].
[5] D. Song, "DSniff," [Online]. Available:
http://naughty.monkey.org/dugsong/dsniff. [Accessed 01 August 2015].
[6] "Yersinia hacking tool," [Online]. Available:
http://yersinia.sourceforge.net. [Accessed 01 August 2015].
[7] "Cain and able," [Online]. Available: http://www.oxid.it/cain.html.
[Accessed 01 August 2015].
[8] Puangpronpitag, S. and Masusai, N., "An efficient and feasible solution
to ARP Spoof problem," in 6th International Conference on Electrical
Engineering/Electronics, Computer, Telecommunications and
Information Technology, Chonburi, 2009.
[9] Bruschi, D. , Ornaghi, A. and Rosti, E., "S-ARP: a secure address
resolution protocol," in 19th Annual Computer Security Applications
Conference, 2003.
[10] Wesam Lootah , William Enck and Patrick McDaniel , "TARP: Ticket-
based address resolution protocol," Computer Networks: The
International Journal of Computer and Telecommunications Networking,
vol. 51, no. 15, pp. 4322-4337 , 2007.
[11] M. Barnaba, "anticap," Antifork, [Online]. Available:
https://antifork.org/git/anticap. [Accessed 01 August 2015].
[12] I.Teterin, "Antidote," [Online]. Available:
http://www.securityfocus.com/archive/l/299929. [Accessed 01 August
2015].
[13] "Dynamic ARP Inspection," Cisco Systems, [Online]. Available:
http://www.cisco.com/c/en/us/td/docs/switches/lan/catalyst6500/ios/12-
2SX/configuration/guide/book/dynarp.html.. [Accessed 01 August
2015].
[14] "XArp - Advanced ARP Spoofing Detection," [Online]. Available:
http://www.xarp.net/. [Accessed 01 August 2015].
[15] "arpwatch," Lawrence Berkeley National Laboratory, August 2009.
[Online]. Available: ftp://ftp.ee.lbl.gov/arpwatch.tar.gz. [Accessed 01
August 2015].
[16] "Anti netcut version 2.0," [Online]. Available: http://www.tools4free.net.
[Accessed 01 August 2015].
[17] A. Ali, "NoCut 1.001a," [Online]. Available:
http://www.download.com/NoCUT/3000-2085_410520090.. [Accessed
01 August 2015].
[18] ColorSoft, "AntiARP," [Online]. Available:
http://www.antiarp.com/English/e_index.htm. [Accessed 01 August
2015].
[19] X. HOU, Z. JIANG and X. TIAN, "The detection and prevention for
ARP Spoofing based on Snort," in International Conference on
Computer Application and System Modeling (ICCASM 2010), 2010 .
[20] A. P. Ortega, X. E. Marcos, L. D. Chiang and C. L. Abad, "Preventing
ARP Cache Poisoning Attacks: A Proof of Concept using OpenWrt," in
Latin American Network Operations and Management Symposium,
2009.
[21] A.-z. Qian, "The Automatic Prevention and Control Research of ARP
Deception and Implementation," in World Congress on Computer
Science and Information Engineering, 2009.
[22] Diego Kreutz, Fernando M. V. Ramos, Paulo Verissimo, Christian
Esteve Rothenberg, Siamak Azodolmolky and Steve Uhlig, "Software-
Defined Networking: A Comprehensive Survey," Proceedings of the
IEEE , vol. 103 , no. 1, pp. 14 - 76, 2015.
[23] B. Lantz, B. Heller and N. McKeown, "A network in a laptop: rapid
prototyping for software-defined networks," in Proceedings of the 9th
ACM SIGCOMM Workshop on Hot Topics in Networks, 2010.
[24] NoxRepo.org, "About POX," [Online]. Available:
http://www.noxrepo.org/pox/about-pox/. [Accessed 01 August 2015].
131
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
DNA-Based Data Hiding Using Variable Length
Substitution Technique
Dina A. Ibrahim
Master Student ,Computers and Systems Department Faculty
of Engineering, Kafrelsheikh University, Kafrelsheikh, Egypt.
H.M. Abdalkader
Professor , Information Systems Department Faculty of
Computers and Informatics, Monyfia University, Shebien,
Egypt.
Abstract The field of using DNA in cryptography is a
new and very promising direction in cryptographic
research. Deoxyribonucleic Acid (DNA) cryptography has
shown to be very effective. In this paper a data hiding
algorithm based on DNA sequence is proposed. We apply
playfair encryption to the secret message first to secure the
secret message then a substitution technique is used for
hiding. The secret message can be in any format not only
binary. An evaluation for the proposed algorithm is
performed according to randomness testing by using the
National Institute of Standards and Technology (NIST)
test. The study showed that the proposed algorithm is
secure against cracking.
Keywords DNA, data hiding, substitution and NIST.
I. INTRODUCTION
Important information transmitted via the Internet
must be secured so that only the authorized receiver can
retrieve it. Thus, data hiding has become a well-known field of
research. Traditionally, data hiding approaches usually used
images as a host for hiding information [6,7,8]. However,
images are important for some people and hiding messages
could distort the host image to some degree. This distortion
has two backgrounds: first, the hidden message may be
detected by the intruder. The second disadvantage is that this
distortion has limited the capacity of hiding so researches
moved towards another host for hiding data which was DNA
[1,2,4,14]. DNA sequences have some inherent properties that
can be utilized to hide data because it is difficult to distinguish
between a real DNA sequence and a fake one.
Firstly, some background knowledge should be
introduced. A DNA sequence is a sequence consisting of four
letters: A; C; G and T. Each letter is related to a nucleotide,
adenine(A), thymine(T), guanine(G), and cytosine(C). For
instance, the DNA sequence appears as follows: This is the
DNA sequence from Litmus with 154 nucleotides retrieved
from the European Bioinformatics Institute(EBI) [9]:
"ATCGAATTCGCGCTGAGTCACAATTCGCGCT
GAGTCACAATTCGCGCTGAGTCACAATTGTGACTCA
GCCGCGAATTCCTGCAGCCCCGAATTCCGCATTGCAG
978-1-5090-0217-7/15/$31.00 2015 IEEE
Elsayed A. Sallam
Associate Professor , Computer and Control Department
Faculty of Engineering, Tanta University, Tanta, Egypt.
AGATAATTGTATTTAAGTGCCTGCTCGATACAATAAA
CGCCATTTGACC".
DNA molecules are arranged as two oppositely oriented
strands with sugar-phosphate backbones joined together in an
alpha-helical structure by hydrogen bonding between the
complementary nitrogenous bases. The bases, adenine(A),
thymine(T),guanine(G), and cytosine(C) represent the genetic
code. A bonds with T and G bonds with C. The DNA
sequence determines the arrangement of amino acids which
form a protein. Proteins are responsible for almost everything
in the cells. RNA is an intermediary copy of the instructions
contained in DNA. Both DNA and RNA are composed of
many codons. Each codon has three nucleotides. A codon
indicates which amino acid will be attached next. As shown in
Table 1, all distinct amino acids are: Phe, Leu, Ile,Val, Ser,
Pro, Thr, Ala, Tyr, His, Gln, Asn, Lys, Asp, Glu, Cys, Trp,
Arg, Met and Gly.
For a real DNA sequence, it is easy to discover that it is
very difficult to distinguish between a real DNA sequence and
a fake one. Another useful element is the fact that there is a
large number of DNA sequences, around 163 million ,
publicly available on various web-sites such as the EBI
database, NCBI database.
The main target of this paper is to propose a new
algorithm for hiding using key features of DNA computing
and playfair cryptography. The proposed algorithm combined
between the substitution technique as a method for hiding data
and the playfair encryption as a prior phase to secure the secret
message. The evaluation of the proposed scheme was carried
out using the randomness testing, NIST. The security analysis
and experimental results indicate better performance and low
computational requirements for the proposed algorithm.
The rest of this paper is organized as follows the next
section highlights some of the related work Section3 presents
Encrypting data using Amino Acid-Based Playfair Cipher.
Section4 describes the variable-length substitution technique.
Section5 discuss the security analysis using NIST. Finally,
Section 6 presents results and comparison against previous
work. Finally, section 7 contains the conclusion.
132
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Table 1: Codon to amino acid mapping[2]
U C A G recovers the secret message M, changing S' back to the
U UUU Phe UCU Ser UAU Tyr UGU Cys U reference sequence S. However, we modified the technique to
UUC Phe UCC Ser UAC Tyr UGC Cys C deal with DNA alphabets instead of the binary data form. We
UUA Leu UCA Ser UUA STOP UGA STOP A also made some modifications to shiu's original substitution
UUG Leu UCG Ser UAG STOP UGG Trp G method to be compatible with our scheme.
C CUU Leu CCU Pro CAU His CGU Arg U
CUC Leu CCC Pro CAC His CGC Arg C
CUA Leu CCA Pro CAA Gln CGA Arg A
CUG Leu CCG Pro CAG Gln CGG Arg G Hongjun Liu, Da Lin , and Abdurahman Kadir in 2013
A AUU Ile ACU Thr AAU Asn CGU Ser U published a novel data hiding method based on DNA coding
AUC Ile ACC Thr AAC Asn CGC Ser C using the Word document as carrier [12]. They concluded that
AUA Ile ACA Thr AUA Lys CGA Arg A
AUG Met, ACG Thrt AAG Lys CGG Arg G the proposed method is well suited to some data hiding
start applications, such as fragile watermarking, secret
G GUU Val GCU Ala GAU Asp GGU Gly U communication.
GUC Val GCC Ala GAC Asp GGC Gly C
GUA Val GCA Ala GUA Glu GGA Gly A
GUG Val GCG Ala GAG Glu GGG Gly G In [16] a modification of the original substitution method
was introduced. The capacity of the updated version has been
II. RELATED WORK doubled.
In this section, we briefly review some previous DNA Mona Sabry et al. [17] proposed one way of distribution of
sequence-based data-hiding schemes. In recent years, a Amino Acid along the English alphabet that is summarized in
number of methods have been proposed for hiding data within table 2. They proposed their method of Amino Acid-Based
DNA sequences. Shimanovsky et al. exploited the codon Playfair cipher.
redundancy to hide data in RNA [2]. Hence, there are many
possible combinations to form an RNA codon, while there are Another improvement of the original substitution method
only twenty distinct amino acids shown in Table 1 encoded is proposed to minimize modification rate in DNA reference
from the RNA codon. It is obvious that some codons might be sequence [19].
mapped to the same amino acids. For example, the codons
CCU, CCC, CCA and CCG are mapped to the same III. ENCRYPTING DATA USING AMINO-ACID
amino acid Pro. The redundancy could be exploited to embed BASED PLAYFAIR CIPHER
information in the RNA codon. In their scheme, if the codon
should be encoded with CCU, but the secret message is four, The first phase in our algorithm is that we apply Amino
they use the codon CCG to replace the original because Acids-Based Playfair encryption to the secret message. The
CCG is the fourth codon of the set of codons whose mapping traditional Playfair algorithm is based on the use of a 5- 5
amino acid is Pro. As a result, a reversible hiding mechanism matrix of letters constructed using a secret key. The Playfair
that can both conceal information into the DNA sequence and cipher is a great advance over simple monoalphabetic ciphers
completely restore the original one was proposed. Note that to [19]. Cryptanalysis of the Playfair cipher is much more
convert table1 from RNA to DNA, just replace each U in the difficult than normal simple substitution ciphers, because
distribution with T. digraphs (pairs of letters) are being substituted instead of
monographs (single letters) [19]. One of the modern
In [3] a randomly chosen DNA sequence is used to hide modifications of Playfair is playfair based on DNA. It
secret message. This DNA sequence is indexed and used to introduced some modifications to the Playfair cipher
hide secret message. The message is converted to binary using processing by using some Biological concepts such as DNA
8-bit ASCII representation then encoded to DNA using binary and amino acids structures to the improve Playfair ciphering
coding. Finally message index is produced and sent to the process. We used the table proposed by Mona Sabry et.al [17]
receiver. for the distribution of Amino Acid along the English alphabet
to convert message from DNA form to English letters needed
to apply playfair. Therefore, we applied Playfair cipher as the
Recently, Chang et al. proposed two schemes to hide data first phase of the proposed technique. In this phase, the secret
in DNA sequences based upon a software view [5]. text plaintext is encrypted using DNA and Amino Acids
concepts [16] as follows: Convert plaintext to binary form
such as 8-bits coding. After that, a binary coding scheme used
Shiu et. al. [11] proposed three DNA-based data hiding to transform binary form into DNA alphabets A, C, G and T.
methods. All these methods secretly select a reference For instance, it may be a binary coding used: ((A 00) (C 01)
sequence S from some publicly available DNA sequence (G 10) (T 11)). It should be noted that more digits may be
databases. Only the sender and the receiver are aware of this used. Next, The DNA form is transferred to the Amino acids
reference sequence. The sender transforms this selected DNA form according to table 2 and save ambiguity number. Now,
sequence S into a new sequence S' by incorporating the secret English form of Amino Acids can go through traditional
message M into the DNA sequence S. This transformed Playfair cipher process using the secret key [19]. Amino Acids
sequence S' is sent by a sender to the receiver. The receiver
then examines the received sequence, identifies S' and

133
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
form of encrypted data transferred to DNA Sequence form.
The encryption and decryption process are illustrated in Fig.1.
IV. The variable-length substitution technique
Shiu et al.[11] proposed the original substitution method.
We adopted the general idea of substitution in our scheme, but
we developed our own scheme. We use a random number seed
to divide the message into segments with different sizes. We
divide the reference sequence into blocks. The number of
blocks is equal to the number of segments to hide each
segment into one of the blocks. We used another random
number generator to generate a set A that will determine
.
where in each block the segment will be placed. If there is part
of the message remaining, it will be concatenated at the end of
the last block. The hiding algorithm is illustrated in Algorithm
1. The recovery algorithm is illustrated in Algorithm2. The
receiver receives the fake sequence. It is very difficult to
distinguish between a real sequence and a fake one. He uses
Algorithm2 to recover secret message. In our proposed
scheme, the substitution method is applied after playfair. We
increased the security and decreased the probability of attack
by using a random number generator that divided message into
segments with variable length instead of the original method
that always substituted one character of reference sequence by
one character of secret message.

Algorithm 1: Data hiding algorithm of the variable-length Substitution

Convert secret message to binary
form [BP] using ASCII coding
for example.
code binary data into a DNA
sequence [DP] by using the
binary coding rule
Transfer DNA sequence to
Amino acids [AP] and save
Ambiguity number AMBIG
Construct the Playfair matrix
using the secret key [SK] and
then apply traditional Playfair
encryption process to get Amino
acid of cipher text [AC]
Apply the substitution recovery
to get DNA CIPHER(DNAC).
Remove Ambiguity number
AMBIG and Transfer DNA
sequence to Amino acids [AC]
Construct the Playfair matrix
using the secret key [SK] and
then apply traditional Playfair
encryption process to get Amino
acid of plain text [AP]
Method
Input: Reference Sequence S, secret message in DNA form,
random number seed r, random number seed k,
Output: a fake DNA sequence s'.
Step1. Determine length of reference sequence to be len_s and length of
secret message to be len_m.
Step2. Use random number seed k to generate the number sequence k1; k2;
. . . ; ki; Find the smallest integer t such that | |. Sequentially
divide the secret message M into segments with lengths k1; k2; . . . ; kt-1 in
order, denote these segments by m1; m2; . . . ;m t-1and let the residual part
be m t. Determine the number of segments to be n.
Step3. Divide len_s into n blocks to hide each segment of message into
one of the blocks.
Step4. Use a random number generator to generate n distinct integers,
called set A, where every integer is no larger than block size.
Step5. For each message segment mi , hide each segment into one block
of the reference sequence starting at the position Ai of the block
Step6. Attach mt at the end of the last block.
Step7. Concatenate all blocks to get a fake sequence s'
Step8. Send s' to the receiver

transfer Amino acids [AP] to

DNA sequence [DNAP] Algorithm 2: Data recovery algorithm of the variable-length
Transfer Amino acid back to Substitution Method
DNA form using Table2 to get Input: a fake DNA sequence s', random number seed r, random
DNAC(DNA CIPHER). number seed k, len_m. len_m can be hidden in any part of sent
sequence and told about it to the receiver to extract it.
Output: secret message in DNA form .
code DNA sequence [DNAP] Step1. Determine length of used sequence to be len_s, determine number
the Substitution method is of blocks n= abs(len_s/len_m).
into binary form using binary
applied to get fake sequence s'
coding rule then back to its
Step2. Divide len_s into n blocks to hide each segment of message from
original form if necessary one of the blocks
Step3. Use random number seed k to generate the number sequence k1; k2;
. . . ; ki;
Step4. Use a random number generator to generate n distinct integers,
called set A, where every integer is no larger than block size.
Fig.1 Encryption process Decryption process Step5. For each block B , recover one segment from each block of the
reference sequence starting at the position Ai of the block and for a length
equal to ki.
Step6. Concatenate all recovered segments to get message M. if
the recovered length is not equal to len_m. find:
len_residue=len_m-len_recovered.
Step7. Recover mt from the end of the last block.

134
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
V. SECURITY ANALYSIS
The main characteristic that identifies any encryption
algorithm is its ability to secure the protected data against
attacks. The NIST statistical testing used to evaluate the
secrecy of the proposed scheme. The NIST Test Suite is a
statistical package consisting of 15 tests that were developed
to test the randomness of binary sequences produced by either
hardware or software. The NIST statistical testing as they
appear in the tables for simulation results namely are; The
Approximate Entropy Test, The Block Frequency Test, The
Cumulative Sums Test, The Discrete Fourier Transform (DFT)
Test, The Frequency Test, The Linear Complexity Test, Tests
for the Longest-Run-of-Ones in Block, The Non Overlapping
Template Matching Test, The Overlapping Template
Matching Test, The Random Excursions Test, Test, The Runs
Test, The Serial Test and The Universal Test. For each test we
compute what so called P-value; this value used to determine
whether the tested bit stream is random or not. For any bit
stream to be random its P-value must be greater than 0.01.
Very small P-values would support non-randomness for given
measure that less than 0.01.
According to NIST statistical testing the more randomness
binary sequence is the more secrecy it is, but as the final
cipher here is presented in a DNA form so this rule will be
quietly different. Running NIST statistical testing on real
DNA sequences; Six DNA sequences downloaded from NCBI
database and mentioned in Table 3 proved that DNA isnt
random .So the evaluations here focused on proving that the
final cipher behaves like any real DNA sequence not to prove
that it is random. Table3 introduced the results of P-value for
each real DNA sequence with the fifteen tests.
Table4 introduced the results of applying NIST statistical
tests on the final cipher generated from the proposed scheme
for each message size. According to the similarities between
the results from Table3 and Table4, we may conclude that the
final cipher is so close to any real DNA sequence. It proved
that the final cipher of the proposed scheme looks like any real
DNA sequence, this ensure the secrecy of the proposed
algorithm. Also, it is very difficult for anyone to differentiate
between any real DNA sequence and the faked one resulting
from the proposed scheme. Running NIST statistical testing
with DNA sequence also defines new ranges for P-value of
each test with DNA sequence. So dealing with DNA isnt
similar to dealing with any other binary bit stream.
VI. RESULTS and COMPARISON
In the field of data hiding, techniques are compared
according to a number of parameters. We used a combination
of performance parameters including the ones introduced by
Shiu et al. [11], namely, capacity, payload, bpn, and the
cracking probability or the probability of a successful brute-
force attack Pbf.
The capacity ( ) is defined as the total length of a
reference sequence that encodes or includes the encrypted
message. The payload ( ) is the remaining length of the new
sequence after excluding the reference DNA sequence. The
payload by a steganographic technique represents the
maximum hiding capacity offered by this algorithm. In other
words, it measures the maximum size of bits that can be
embedded in the cover media. The bpn (bit per nucleotide)
is another measurement that is used to estimate the hiding
capability of a certain algorithm. The previous parameters
utilize the following notations: N is the length of a DNA
sequence, M is the message that will be encrypted, and |M| is
its length.
The capacity of our algorithm is: , while
payload and bpn are the same as the original substitution
method[11] as listed in table 5. The improvement that we
made to the original one is that the input to our method may be
in alphabetical form also not only in the binary form as the
original method was. We also decreased the probability of a
successful attack. The original method was one-to-one
substitution but our method is a variable-length which added
extra security. The substitution method has the best capacity
over the other methods proposed by shiu et. al. [11] so we
improved it.
As for the probability of brute force attack for the variable
length substitution technique, an intruder can discover the
secret message if he knows the following information: (1) the
reference DNA sequence or the initial carrier. There are
roughly 163 million DNA sequences available publicly. Thus,
the probability of an attacker making a successful guess is 1/
1.63x108. and (2) two random seeds, if we assume everyone is
in the range of 1 to 10000 for example then the probability of
guessing them is 1 10000 1 10000 The total probability
that an attacker makes a successful guess is: Pbf =
.
Now we are going to compare our results with other
steganographic approaches. We will compare our algorithm
with Shiu et.al.[11] The comparison is summarized in table 5
Given that N is the length of a DNA sequence, |M| is the
length of the original message, and is a method-specific
parameter that represents the length of the longest
complementary pairs in the reference DNA sequence.
135
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Table 2:distribution of Amino Acids upon English letters[16]

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

4 2 2 2 2 2 4 2 3 1 2 4 1 2 2 4 2 4 4 4 2 4 1 2 1 1

GCU UAA UGU GAU GAA UUU GGU CAU AUU UGA AAA CUU AUG AAU UUA CCU CAA CGU UCU ACU AGA GUU UGG AGU UAU UAC

GCC UAG UGC GAC GAG UUC GGC CAC AUC AAG CUC AAC UUG CCC CAG CGC UCC ACC AGG GUC AGC

GCA GGA AUA CUA CCA CGA UCA ACA GUA

GCG GGG CUG CCG CGG UCG ACG GUG

GC UA UG GA GA UU GG CA AU UG AA CU AU AA UU CC CA CG UC AC AG GU UG AG UA UA

Table 3. Results of NIST testing with real DNA sequences

NIST Tests
DNA/Test
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0.00-
AC167221 0.00 0.00 0.00 0.00 0.00 0.74 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.85
0.00-
AC168874 0.00 0.00 0.00 0.00 0.00 0.67 0.00 0.00 0.00 0.00 0.93 0.00 0.00 0.00
0.92
0.00-
AC168897 0.00 0.00 0.00 0.00 0.00 0.39 0.00 0.00 0.00 0.00 0.31 0.00 0.00 0.00
0.77
0.00-
AC168901 0.00 0.00 0.00 0.00 0.00 0.85 0.00 0.00 0.00 0.00 0.07 0.00 0.00 Error
0.99
0.00-
AC168907 0.00 0.00 0.00 0.00 0.00 0.28 0.00 0.00 0.00 0.00 0.03 0.00 0.00 Error
0.93
0.00-
AC168908 0.00 0.00 0.00 0.00 0.00 0.285 0.00 0.00 0.00 0.00 0.2 0.00 0.00 0.00
0.68

Table 4. Results of NIST testing with the final DNA sequence result from the proposed algorithm

NIST Tests
Msg/Test
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0.00-
AC168908 0.00 0.00 0.00 0.00 0.00 0.285 0.00 0.00 0.00 0.00 0.2 0.00 0.00 0.00
0.68
0.00-
10 KB 0.00 0.00 0.00 0.07 0.00 0.5 0.00 0 0.00 0.00 0.38 0.00 0.00 0.00
0.77
0.00-
40 KB 0.00 0.00 0.00 0.00 0.00 0.314 0.00 0.00 0.00 0.00 0.154 0.00 0.00 0.00
0.727
0.00-
100 KB 0.00 0.00 0.00 0.00 0.00 0.285 0.00 0.00 0.00 0.00 0.21 0.00 0.00 0.00
0.66
0.00-
300 KB 0.00 0.00 0.00 0.00 0.00 0.202 0.00 0.00 0.00 0.00 0.81 0.00 0.00 0.00
0.727
0.00-
600 KB 0.00 0.00 0.00 0.00 0.00 0.243 0.00 0.00 0.00 0.00 0.2 0.00 0.00 0.00
0.68
0.00-
1 MB 0.00 0.00 0.00 0.00 0.00 0.24 0.00 0.00 0.00 0.00 0.23 0.00 0.00 0.00
0.66

136
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Bits per nucleotide Probability of brute Format of

Method Capacity [C] Payload [P] secret message
[bpn] force [Pbf]
Only Binary
N +|M|/N N/2 |M|/ (N+|M|/2)
| |
.
Insertion [11]

Only Binary
Complementary |M|/ (N+|M|
N+|M|(K+3.5) |M| (K+3.5)
pair [11] (K+3.5)) .

Only Binary
Substitution [11] N 0 |M|/N .

Alphabetical or
. binary
Variable-Length
Substitution
N 0 |M|/N

VII. CONCLUSION

In this paper, it has been demonstrated that DNA
sequences have special properties which can be utilized for
data hiding purposes. The proposed scheme first
implemented Amino Acid-Based playfair to encrypt the
message needed to be hidden then a variable-length
substitution for hiding data in a DNA sequence. It is difficult
for an attacker to detect whether or not there are secret
messages hidden in a DNA sequence. Our proposed data
hiding scheme increased security than previous schemes by
hiding variable length segments of the message into different
blocks of the reference sequence. NIST statistical tests have
been performed to improve that it very hard for an attacker
to retrieve secret message from a sequence because the fake
sequence after hiding message within it is very similar yo the
real sequence.
REFERENCES
[1] A. Leier, C. Richter, W. Banzhaf, H. Rauhe , Cryptography with DNA
binary strands, BioSystems,volume:57,2000,PP.1322.
[2] B. Shimanovsky, J. Feng, M. Potkonjak , Hiding data in DNA, in: Revised
Papers from the 5th International Workshop on Information Hiding,
Lecture Notes in Computer Science 2578,2002,pp. 373386.
[3] B. Debnath, K. B. Samir, "Hiding Secret Data in DNA Sequence, "
International Journal of Scientific & Engineering Research, vol. 4, no. 2,2013,
pp. 1-4.
[4] Cheng Guo1, Chin-Chen Chang and Zhi-HuiWang A New Data Hiding
Scheme Based On DNA Sequence , International Journal of Innovative
Computing, Information and Control Volume 8, Number 1(A),
2012,pp.1349-4198.
[5] C.-C. Chang, T.-C.Lu, Y.-F.Chang and C.-T. Lee, Reversible data hiding
schemes for deoxyribonucleic acid (DNA) medium, International
Journal of Innovative Computing, Information and Control, vol.3, no.5,,
2007, pp.1145-1160.
[6] C.C. Chang, C.C. Lin, C.S. Tseng, W.L. Tai, Reversible hiding in DCT-
based compressed images, Information Sciences 177 , 2007, pp. 2768
2786.
[7] C.C. Chang, W.C. Wu, Y.H. Chen, Joint coding and embedding
techniques for multimedia images, Information Sciences 178,2008,pp.
35433556.
[9] European Bioinformatics Institute, <http://www.ebi.ac.uk/>.
[10] E. I. Fatma, I. M. Mahmoud and S. A. Hatem, "A Symmetric Encryption
Algorithm based on DNA Computing," International Journal of Computer
Applications, vol. 97, no. 16, 2014,pp. 41-45.
[11] H.J. Shiua,d, K.L. Ng b, J.F. Fang c, R.C.T. Lee d, C.H. Huang e, Data
hiding methods based upon DNA sequences, Information Sciences
180,2010,pp. 21962208.
[12] Hongjun Liu, Da Lin, and Abdurahman Kadir.,2013,A novel data hiding
method based on deoxyribonucleic acid coding, Computers & Electrical
Engineering, 2013,pp.1164-1173.
[13] H.W. Tseng, C.P. Hsieh, Prediction-based reversible data hiding,
Information Sciences 179,2009,pp. 24602469.
[15] I. Peterson, Hiding in DNA, Muse 22,2001.
[16] J. Taur, H. Lin, H. Lee and C. Tao, " Data hiding in DNA sequences
based on table lookup substitution," International Journal of Innovative
Computing, Information and Control , vol. 8, no. 10,2012, pp. 6585-6598.
[17] Mona Sabry et al., A DNA and Amino Acids-Based Im-plementation of
PlayfairCipher,International Journal of Computer Science and
Information Security (IJCSIS), Vol.8 No. 3,2010.
[18] R. A. Mohammad., N. Pourya., Ordi, A. and R. N. Mohammad. "DNA
Base Data Hiding Algorithm, International Journal on New Computer
Architectures and Their Applications, vol. 2, no. 1,2012, pp. 183-192.
[19] S. Kevin, K. KiRyong, L. SukHwan and K. Seong-Geun, "High Capacity
Data Hiding Method in DNA with Mutation Handling," ACM, vol. 3, 2014,
pp. 56-63.
[20]. William Stallings, Cryptography and Network Security Principles and
Practices, Fourth Edition,2005.
[21] Watson JD, et al. , Molecular Biology of the Gene,2004.
[2] DNA codon table - Wikipedia, the free ency- clopedia.
Wikipedia. [Online] http://en.wikipedia.org/wiki/DNA codon
table.Wikipedia. [Online] http://en.wikipedia.org/wiki/DNA codon table.

[8] C.H. Huang, J.L. Wu,Fidelity-guaranteed robustness enhancement of

blind-detection watermarking schemes, Information Sciences
179,2009,pp. 791808.

137
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
PRNG Based Modified MSLDIP Watermarking
Abdelmgeid A. Ali
Computer Science Dept.
Faculty of Science,
Minia University
Minia, Egypt
abdelmgeid@yahoo.com
Abstract Watermarking of digital media is an imperative and
interactive method for identification and protection of digital data.
It allows veritable watermarks to be hidden in digital media for
example image, audio, and video. Procedure of embedding and
extracting of watermark from original image and watermarked
image is complicated process. These include randomization of the
watermark and selection of locations to embed and extract it from
the selected locations. Proposed work gives more secured digital
watermarking algorithm that follows Modified Substitute Last
Digit in Pixel Technique. Embedding of watermark is done based
on pseudorandom number generator (PRNG) and RC4 encryption
of each pixel value of the cover image. Watermarked image is used
to extract the watermark without any help of the original image.
Parameters used to evaluate the performance watermarking
technique are Peak Signal to Noise Ratio (PSNR) and Mean
Square Error (MSE) and the results of the proposed method were
better.
Keywords PRNG (Pseudorandom Number Generator),
Cryptography, Encryption, Decryption, Watermarking, Spatial
Domain, MSLDIP (Modified Substitute Last Digit in Pixel),
Security.
I. INTRODUCTION
A. Watermarking
Digital content, with the appearance of Internet and swift
evolution of information technologies, has become an essential
part of ones day to day life. As it is easy to make an exact copy
of the digital media, illegitimate sharing and replication of
digital contents has become major concern for authors,
publishers and legitimate owners of the contents [1]. It has
become imperative to keep the intellectual characteristics of
digital content of a given media. In the beginning, ciphering and
control access methods were used to protect from harm the
authenticity of digital content of a given media [2]. But this
avoids free sharing and transmission of the content through the
network, which is mostly not preferable to the author of the
content. To deal with this problem, digital watermark has been
proposed [1]. Watermarking is the process of adding concealed
information by modifying the pixel values of an image with least
amount of perceptual interruption. A recent survey of major
methods appears in [3]. Some preferable features of valuable
watermarking methods include imperceptibility, security and
robustness.
Visible and invisible are the popular classification of
Watermarks. Visible watermark is seen with the content of the
image and unauthorized removal of the same causes damage to
the image. Example of scheme like that is the IBM digital
978-1-5090-0217-7/15/$31.00 2015 IEEE
Ahmed H. Ismail
Computer Science Dept.
Faculty of Science,
Minia University
Minia, Egypt
ahamdycs2012@gmail.com
watermarking scheme for the Vatican library. Invisible
watermarking is needed for copyright protection especially in
multimedia products. Though the Watermark modifies an
image, it should not change the image as seen by human eye.
Digital watermarking techniques can be classified into two main
categories based on watermark embedding domain, i.e. spatial
domain (which depends on modifying the pixels of one or two
randomly selected subsets of images) and frequency domain
(which values of specific frequencies are modified from their
original) [4]. Basic watermarking system mainly consists of two
basic parts [5] (Figure 1), watermark embedder which
responsible for embedding the watermark image and watermark
key into the cover image (original), and watermark detector
which responsible for finding and extracting the watermark
image only using watermark key and watermarked image which
executes on the other side after transmission of data.
Fig 1. Basic Watermarking System
B. Cryptography
Cryptography is an area within the field of cryptology. The
expresion cryptology is a combination of the Greek (cruptos =
hidden and logos = study, science). Therefore, the word
cryptology literally means the science of concealing. The
cryptography can be divided into two areas: cryptography and
cryptanalysis [6]. Cryptanalysis is the area within cryptology
which is concerned with techniques for deciphering encrypted
data without prior knowledge of which key has been used. This
more commonly known as 'Hacking'. The cryptanalyst is the
person who tries to find weaknesses in encryption schemes. He
will often figure out how to break the cryptography scheme, and
138
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
then the developer of the scheme will use that information to
make it stronger [7].
C. Pseudo-random number generator (PRNG)
Pseudo-random number generators act as a black box, which
takes one number (called the seed and produces a sequence of
numbers; this sequence is said to be pseudo-random if it passes
a number of statistical tests, and it appears random. A pseudo
random number generator is formed of an initialization function,
a state (a sequence of bits of bounded length), a transition
function, and an output function [8] (Figure 2):
The initialization function takes a number (the seed), and
puts the generator in its initial state.
The transition function transforms the state of the
generator.
The output function transforms the current state to produce
a list of integer numbers.
Fig 2. Schematic diagram of a pseudo-random number generator [8]
The rest of this paper is organized as follows. In section II,
we present LSB, SLDIP, MSLDIP, and Modified MSLDIP as a
related work, then PRNG Based MSLDIP (proposed method)
has been explained in section III, next the experimental results
of our proposed method has been recorded and discussed in
section IV, and finally Conclusion and Future Work in section
V, and References in Section VI.
II. RELATED WORK
In general, the embedding techniques can be classified into
two categories: spatial domain techniques and frequency domain
techniques. Spatial domain watermarking techniques do a bit
modifications on the pixels of one or two randomly selected
subset of cover image. The main technique is LSB technique. Its
modifications contain flipping the least significant bit of each
pixel. The LSB technique is the simplest technique of watermark
insertion [9].
Other method was done before is Substitute Last Digit in
Pixel (SLDIP) that takes the cover image and the watermark as
input, convert the blue layer of the cover image into one row,
and divide the row into blocks each of which contains 9 values,
then consider the watermark is color image then each pixel will
be represented in 3 bytes, according to the color image
representation (which each pixel is specified by three values
one each for red, green, and blue components of the pixel's color
and each value represented by one byte, so each pixel will be
represented in three bytes) [10]. Each byte in the watermark
image will be ranges from 0 to 255, and make each byte value's
length equal to 3 digits, for example we have byte of value 19,
this value equal to 019 which has length of 3 digits, finally
substitute each 9 digits of each pixel with the last digit of each
pixel in the current block, so each pixel of the watermark image
will be embedded in only one block, and output the
watermarked image and 2 keys which be required in the
extraction process [11]. Assuming that watermark image of
height 1 pixels and width 1 pixels, and cover image of height 3
pixels and width 3 pixels. The SLDIP will represent blue layer
of the cover image in one row which contains one block of 9
values (123, 255, 086, 192, 145, 210, 035, 099, and 004), and
the watermark image will be represented as (130, 105, and 080),
the SLDIP will substitute 3 (last digit in cover image) in 123
with 1 (first digit in watermark image) in 130, the result is 121
and also substitutions done until reaching the last digit in the
last value of the watermark. The watermarked image will be
(121, 253, 080, 195, 140, 215, 030, 098, and 000) [10].
By using SLDIP method capacity of embedding has been
increased, the maximum area of watermark image that can be
embedded in any cover image can be calculated by using this
formula [11]:
If the watermark image is grayscale image this formula can be
used:
Modified Substitute Last Digit in Pixel (MSLDIP) is a
modified method of SLDIP that depends on a modification on
SLDIP by update the substitution step to decrease the difference
between the original pixel and the substituted pixel, for example
embedding value digit 8 in pixel 100, by using SLDIP the pixel
will be 108, but by MSLDIP two possible values can be taken
for each substitution and choose the value that has the smallest
difference, so the two values will be 108 and 98, then the value
with the smallest difference must be chosen, so the pixel value
will be 98, the difference will be 2 instead of 8 and this increases
the PSNR value of the image [11].
Modified MSLDIP is a modified method of MSLDIP which
the watermark is encrypted using RC4 encryption before the
embedding process to increase the security of the watermarked
data and also in the extraction the extracted watermarked should
be decrypted by RC4 decryption using the same key used in the
encryption process [12].
III. PROPOSED METHOD
In this section the proposed method will be presented, at the
first the PRNG Algorithm will presented and why PRNG
Algorithm is used.
In the field of symmetric ciphers RC4 is the most popular
algorithm that is immune to attack [13]. It used variable key size
for ciphering with byte-oriented operation. A number of
research activities have been done to analyze the RC4 algorithm
139
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

[14-20]. Some of these papers mentioned the weaknesses 1. Take W and encrypt it by performing RC4 Encryption
present in the algorithm [14-16]. algorithm with K, the output of this step is called W'.
The ideal PRNG for this problem is one which can generate 2. Divide the W' into blocks B(i). Each block contains 8
a unique, random integer the first 232 times we call it, then repeat numbers.
the same sequence the next 232 times it can be called ad 3. Generate RS(i) using PRNG generator and K. Each
infinitum. In other words, a repeating cycle of 232 values. That number of these random numbers will represent the
way, we could begin the PRNG at any point in the cycle, always location of the pixels in C.
having the guarantee that the next 232 values are repeat-free. One 4. Apply MSLDIP watermarking embedding procedure to
way to implement such a PRNG is by defining a one-to-one embed the 8 numbers of each block B(i) of W' in the
function on the integers a function which maps each 32-bit random 8 locations of RS(i) at C.
integer to another, uniquely. Lets call such a function a 5. Repeat steps 3, 4 until the whole blocks of W' are
permutation [21]. If we have a good permutation as a result, all embedded in C.
we need is to call it with increasing inputs {0, 1, 2, 3, ..}. We
could even begin the input sequence at any value. For some
reason, it is known in Finite Mathematics that when p is a prime W RS(i)
number, x2 mod p has some interesting properties. Numbers W' B(i)
produced this way are called quadratic residues, and we can K K C
compute them in C using expression (3). In particular, the
quadratic residue of x is unique as long as 2x<p. For example,
Embedding
when p=11, the quadratic residues of 0, 1, 2, 3, 4, 5 are all unique RC4 Encryption
Dividing Blocks &
B(i) in RS(i) to
Generating
(0, 1, 4, 9, 5, 3) (Figure 3) [21]. Random Sets output
PRNGWI

Fig 5. PRNG Based MSLDIP Embedding Process

B. Watermark Extraction Algorithm

Fig 3. Quadratic residues of 0, 1, 2, 3, 4, 5 [21]
As luck would have it, it also happens that for the remaining
integers, expression (4) fits perfectly into the remaining
numbers. This only works for primes p which satisfy expression
(5), so new output numbers of 6, 7, 8, 9, 10 are all unique (8, 6,
2, 7, 10) (Figure 4) [21].
Fig 4. Quadratic residues of 6, 7, 8, 9, 10 [21]
Algorithm: PRNG Based MSLDIP Extracting Algorithm
Input: PRNG Watermarked Image PRNGWI, Secret Key K.
Output: PRNG Random Sets RS(i),
Encrypted Watermark into Blocks B(i),
Encrypted Watermark W',
PRNG Watermarked Image PRNGWI.
Steps: (Figure 6)
1. Generate RS(i) using PRNG generator and K. Each
number of these random numbers will represent the
location of the pixels in PRNGWI.
2. Apply MSLDIP watermarking extraction procedure to
extract B(i) from the random 8 locations RS(i) at
PRNGWI.
3. Repeat steps 1, 2 until the whole blocks collected to output
the encrypted watermark W'.
4. Take W' and decrypt it by performing RC4 Decryption
algorithm with K, the output of this step is called W.

The proposed method will be divided into two algorithms B(i)

which are watermark embedding algorithm and watermark
extraction algorithm. K RS(i) W'
PRNGWI
K
A. Watermark Embedding Algorithm
Algorithm: PRNG Based MSLDIP Embedding Algorithm Generating Enctypted Collect B(i),
Watermar Blocks RC4
Input: Watermark W; Cover Image C; Secret Key K. Random Sets
Extraction Decryption to
Output: Encrypted Watermark W', output W
Encrypted Watermark into Blocks B(i),
PRNG Random Sets RS(i),
PRNG Watermarked Image PRNGWI. Fig 6. PRNG Based MSLDIP Extraction Process
Steps: (Figure 5)

140
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

IV. EXPERIMENTAL RESULTS Table 1. PRNG Based MSLDIP Embedding Process

In order to evaluate the performance of the watermarked Cover PSNR MSE Watermark
images, there are some quality measures such as PSNR and Image Image
MSE.
(150 x 150) 43.56 2.86 (45 x 45 )
A. MSE (Mean Square Root)
(192 x 192 ) 45.84 1.69 (45 x 45 )
The MSE (Mean Square Root) is defined as an average
squared difference between a reference image and a distorted (256 x 256 ) 48.31 0.96 (45 x 45 )
image. It can be calculated by the formula as
(512 x 512 ) 53.49 0.29 (45 x 45 )
(768 x 768 ) 57.74 0.11 (45 x 45 )
Where X and Y are height and width respectively of the (1024 x 1024) 60.78 0.05 (45 x 45)
cover image, the c(i, j) is the pixel value of the cover
image and w(i, j) is the pixel value of the watermarked In (Figure 8), Chart showing the results of MSE and PSNR
between the cover images and watermarked images in the PRNG
image.
based MSLDIP Algorithm.
B. PSNR (Peak Signal to Noise Ratio)
The PSNR (Peak Signal to Noise Ratio) is an expression PRNG an d M SE b etween cover s
for the ratio between the maximum possible value (power) an d water m ar ked im ages (PRNG
of a signal and the power of distorting noise that affects the b ased M SLDIP
quality of its representation. It can be calculated by the
formula given below

IMAGE SIZE
Where L is the peak signal value of the cover image which
is equal to 255 for 8 bit images [22].
RC4 Encryption was performed before the MSLDIP Embedding
based, and in order to give the algorithm more security the
PSNR & MSE
MSLDIP algorithm was done based on Pseudorandom Number
Generator (PRNG) (Figure 7),
Fig 8. Chart showing the results of PRNG based MSLDIP

PRNG based MSLDIP has been compared with [23] (Table 2),
supposed four cover image with squared sizes 128, 256, 512, and
1024, and a watermark with full capacity with cover images
according to [23], thus the full watermark capacity can be
calculated using cover image sizes according to [23] by using
formula
Cover Image (a) Watermark (b)
(512 x 512) (45 x 45)

Then the PRNG based MSLDIP has been compared with [24]
(Table 4), supposed the grayscale baboon.bmp as a cover image
and, the grayscales lena.bmp, and barbara.bmp as watermark
and the full capacities of the embedded watermark according to
Watermarked Image (c)
each algorithm have been calculated using formulas
(512 x 512)

Fig 7. PRNG Based MSLDIP Embedding Process

PRNG based MSLDIP Watermarking implemented in

MATLAB 2014 platform and the experiment has been
conducted on various images.
PRNG based MSLDIP Watermarking has been applied on set of
images different in sizes and the Peak Signal to Noise Ratio
(PSNR), and Mean Square Root (MSE) have been calculated, all
results recorded in (Table 1).

141
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Then PRNG based MSLDIP has been compared with Modified
MSLDIP [12] (the previous version method) (Table 3), the
results of PRNG based MSLDIP were slightly better and in the
Modified MSLDIP if the user type incorrect key the watermark
will be different but the user can deduce the original watermark
with human eyes, in PRNG based MSLDIP if the user type
incorrect key the watermark will be big different as the user
cannot deduce the original with human eyes (Figure 9).
Original Watermark (Correct
Watermark)
Fake Watermark in Modified Fake Watermark in PRNG
MSLDIP Method based MSLDIP Method
Fig 9. Watermark Extraction with incorrect Key in Modified
MSLDIP Method and in PRNG based MSLDIP
From the comparison in table (2), the reason of why PRNG
based MSLDIP has been compared with [23] 3rd Bit? Has been
clarified as, in PRNG based MSLDIP substitutions can change
the value of pixel which the difference ranges from 0 to 5 and
change in the 3rd Bit in pixel can change the value of pixel
which the difference ranges from 0 to 7 which include the
PRNG based MSLDIP difference range. However results in
PRNG based MSLDIP are better.
From the comparison in table (3), the results of the PRNG based
MSLDIP are compared with the results of the modified
MSLDIP [12] (the previous version), and It can be concluded
that the results of PRNG based MSLDIP were better, and in the
PRNG based MSLDIP the data which has been watermarked is
more secured as if the user type incorrect key, the user cannot
deduce the original watermark with the human eyes.
From the comparison in table (4), the results of PRNG based
MSLDIP are compared with results of [24], and from the
comparison it can be concluded that the two algorithm have
very good PSNR results that mean no one can discover the
watermark when looking at the image, also it can be conducted
that the watermark full capacity of PRNG based MSLDIP is
greater than [24], suppose cover image (512 x 512) and
watermark (256 x 256) algorithm of [24] cannot embed the
watermark in cover image but the PRNG based MSLDIP can
embed this watermark successfully.
After Implementing and analyzing the results, conclude that,
the visual quality of the image doesn't change significantly, on
the other hand this algorithm is more robust than LSB
techniques [23][24], because in LSB technique some attackers
can possibly zero out several least significant bit of pixels of the
image and hence clear the watermark. This technique has
increased the capacity of watermark in embedding process.

Table 2. Results Comparison between [23] 3rd Bit and PRNG based MSLDIP

Watermark [23] 3rd Bit PRNG based MSLDIP

Cover Image
FC [23] PSNR MSE PSNR MSE
Baboon 128 ( 27 x 27 ) 31.68 dB 44.50 46.71 dB 1.39
Bird 256 ( 53 x 53 ) 31.68 dB 44.50 46.76 dB 1.37
Boat 512 (125 x 125) 31.68 dB 44.50 46.04 dB 1.62
Pepper 1024 (210 x 210) 31.68 dB 44.50 47.12 dB 1.26

Table 3. Results Comparison between [12] Modified MSLDIP and PRNG based MSLDIP

Watermark [12] Modified MSLDIP PRNG based MSLDIP

Cover Image
Image PSNR MSE PSNR MSE

( 150 x 150 ) ( 50 x 50 ) 43.21 dB 3.11 43.13 dB 3.16

( 200 x 200 ) ( 50 x 50 ) 45.87 dB 1.68 45.98 dB 1.64
( 500 x 500 ) ( 50 x 50 ) 53.29 dB 0.31 53.49 dB 0.29
( 700 x 700 ) (50 x 50) 56.22 dB 0.16 56.38 dB 0.15
( 1000 x 1000 ) ( 50 x 50 ) 59.82 dB 0.07 59.85 dB 0.07

142
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

[4] Prabhishek Singh, R S Chadha, "A Survey of Digital Watermarking

Table 4. Results Comparison between [24] and PRNG based MSLDIP Techniques, Application and Attacks", IJEIT, March 2013.
PRNG based [5] Abdullatif, Mohammad, Akram M. Zeki, Jalel Chebil, and Teddy Surya
Watermark [24] Gunawan. "Properties of digital image watermarking." In Signal
Cover Image MSLDIP Processing and its Applications (CSPA), 2013 IEEE 9th International
Image Colloquium on, pp. 235-240. IEEE, 2013.
PSNR PSNR
[6] Jan C A, Van Der Lubbe, "Basic Methods of Cryptography", English
baboon.bmp lena.bmp translation Cambridge University Press 1998.
58.64 dB 52.00 dB [7] E. Cole, "Hiding In Plain Sight: Steganography and The Art of Covert
(512 x 512) (64 x 64)
Communication", ISBN 0-471-44449-9, Wiley publishing, inc, 2003.
baboon.bmp Barbara.bmp [8] Clment PitClaudel (July, 2012), "How random is pseudo-random?
58.99 dB 51.81 dB Testing pseudo-random number generators and measuring randomness",
(512 x 512) (64 x 64)
fom http://www.apastyle.org/learn/quick-guide-on-references.aspx.
Watermark (FC) applying [9] R. Chandramouli and Nasir Memon, Analysis of LSB Based Image
Steganography Techniques, IEEE 2001.
equations (6,7) With Cover (181 x 181) (295 x 295)
[10] Ahmed A. Radwan, Ahmed Swilem, Al-Hussien Seddik, "A High
Image size (512 x 512) Capacity SLDIP method", ICICIS, July 2011.
[11] Abdelmgeid A. Ali, Ahmed A. Radwan, and Ahmed H. Ismail, "Digital
Image Watermarking using MSLDIP (Modified Substitute Last Digit in
V. CONCLUSION Pixel)", IJCA, Volume 108 No 7, Pages 30-34, December 2014.
[12] Abdelmgeid A. Ali and Ahmed H. Ismail, "Cryptography based MSLDIP
Digital watermarking with Security is the current area of Watermarking Algorithm", CSC open acess library IJCSS, Volume 9
research where lot of scope exists. Currently digital No 4, Pages 208-217, August 2015.
watermarking with cryptographic technique is being used by [13] W. Stallings, "Cryptography and Network Security", Delhi, Pearson
several countries for secretly transfer of hand written Education, 4th Edition, 2008.
documents, text images, financial documents, internet voting [14] S. Maitra, G. Paul, Analysis of RC4 and proposal of additional layers for
etc. This paper starts from some basic knowledge of digital better security margin, Proc. Indocrypt, 2008, IIT Kharagpur, LNCS
5365, 27-39.
image watermarking, cryptography and pseudorandom number
[15] S. Paul, B. Preneel, A New Weakness in the RC4 Keystream. Generator
generator (PRNG). From results conclude that the visual quality and an Approach to Improve the. Security of the Cipher, Proc. Fast
of the image doesn't change significantly, this algorithm is more Software Encryption, 2004, Berlin, LNCS 3017, 245-259.
robust than LSB technique because in LSB technique some [16] S. Fluhrer, I. Mantin, A. Shamir, Weakness in the Key Scheduling
attackers can possibly zero out several least significant bit of Algorithm of RC4, Proc. Int. Workshop on Selected Areas in
Cryptography, 2001, Toronto, LNCS 2259, 1-24.
pixels of the image and hence clear the watermark, this
[17] I. Mantin, A. Shamir, A Practical Attack on Broadcast RC4, Proc. Fast
algorithm is more secure because of using cryptographic Software Encryption, 2001, Japan, LNCS 2355, 152-164.
technique which has been merged with the watermarking [18] S. Fluhrer, D. McGrew, Statistical Analysis of the Alleged RC4 Key
technique that based in pseudorandom number generator Stream Generator, Proc. Fast Software Encryption, 2000, New York,
(PRNG), and this technique has increased the capacity of LNCS 1978, 19-30.
watermark which will be embedded. In the future, the aim of [19] L. Knudsen, et al., Analysis Method for Alleged RC4, Proc.
this paper is to extend the security to higher dimensions and ASIACRYPT, 1998, Beijing, LNCS 1514, 327- 341.
apply it in frequency domain in order to consider more security [20] S. Mister, S. Tavares, Cryptanalysis of RC4-Like Ciphers, Proc. Int.
Workshop on Selected Areas in Cryptography, 1998, Canada,LNCS
and robustness. 1556, 131-143.
[21] Jeff Preshing, (December 24, 2012), " How to Generate a Sequence of
REFERENCES Unique Random Integers", from http://preshing.com/20121224/how-to-
[1] R.G. Schyndel, A. Tirkel, and C.F Osborne, "A Digital Watermark", generate-a-sequence-of-unique-random-integers/.
Proceedings of IEEE International conference on Image Processing, ICIP, [22] Amit Kumar Singh, Nomit Sharma, Mayank Dave, Anand Mohan, "A
1994. Novel Technique for Digital Image Watermarking in Spatial Domain",
[2] Preeti Gupta, "Cryptography based digital image watermarking algorithm 2012 2nd IEEE International Conference on Parallel, Distributed and Grid
to increase security of watermarked data", International Journal of Computing.
Scientific & Engineering Research, September 2012. [23] Deepshikha Chopra, Preeti Gupta, Gaur Sanjay, Anil Gupta, "LSB based
[3] Christine I. Podilchuk, Edward J. Delp, "Digital watermarking: digital image watermarking for gray scale image", IOSRJCE, October
Algorithms and applications, IEEE Signal processing Magazine, July 2012.
2001. [24] Krishna Kumar, and Shashank Dwivedi, "Digital Watermarking using
Asymmetric Key Cryptography and Spatial Domain Technique",
IJARCSMS, Volume 2, Issue 8, August 2014.

143
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Energy-Aware Enhancement of LEACH Protocol
in Wireless Sensor Network
Ahmed AL-BAZ
Computer Science & Eng. Dept., Faculty of Electronic Eng.,
Menoufia University, 32952 Menouf, Egypt.
engahmedbaz@yahoo.com
Abstract Clustering is an effective technique that can
participate to entire system lifetime, scalability and energy
efficiency in wireless sensor networks. Low energy adaptive
clustering hierarchy protocol is a very effective protocol that
enhances the network lifetime. A clustering based protocol
balances the energy usage by giving equal chance to all nodes to
become a cluster head. This paper presents a new algorithm to
select the cluster head with increasing sensor lifetime,
throughput, and the delivery packet ratio. The results show that
our proposed algorithm is more efficient than previous
algorithms.
KeywordsClustering, Data Aggregation, Wireless Sensor
Networks, cluster head selection.
I. INTRODUCTION
A Wireless Sensor Networks (WSN) considers a base station
(BS) that communicates with a large number of wireless
sensor nodes through a radio channel. Data gathering at the
wireless sensor node, is compressed, and is transmitted either
directly to the BS or via other wireless sensor nodes to the
BS. The BS is a superior computer where data from the
sensor network will be compiled and processed and may
communicate with the remote controller node via Internet or
satellite. Data aggregation technique has been used to achieve
energy efficiency and data transfer optimization in a number
of routing protocols [1].The sending the unprocessed data to
the BS needs more energy than local data aggregation.
Data aggregation is the combination of data from different
sources by using functions such as suppression (eliminating
duplicates), min, max and average [2]. Some of these
functions can be performed either partially or fully in each
sensor node.
In WSN, sensor nodes sense the target area and transmit their
collected information to the sink node for further operations
in order to decrease the number of transmitted messages to
the sink node [3, 4].
Architectural layers of a WSN are summarized into
application layer, transport layer, network layer, data Link
layer, physical layer. Application layer defines a standard set
of services and interface primitives available to a
programmer independently on their implementation on every
kind of platform. Transport layer provides services such as
loss recovery, congestion control, and packet fragmentation
and ordering. Transport layer helps to maintain the flow of
data if the sensor the system is planned to be accessed
through Internet or other external networks [5, 6].Network
layer takes care of routing the data, directing the process of
path selection along which to send data in the network. Data
978-1-5090-0217-7/15/$31.00 2015 IEEE
Ayman EL-SAYED (IEEE Senior Member)
Computer Science & Eng. Dept., Faculty of Electronic Eng.,
Menoufia University, 32952 Menouf, Egypt.
ayman.elsayed@el-eng.menofia.edu.eg
Link layer is primarily responsible for data stream
multiplexing, data frame transmission and reception, medium
access, and error control. Physical layer is responsible for
signal transmission and reception over a physical
communication medium, including frequency generation,
signal modulation, transmission and reception and data
encryption [7].
Energy consumption is the critical aspect in wireless sensor
networks because nodes are battery-powered devices, so
clustered-based protocols have been accelerated to balance
and prolong lifetime of the sensor nodes in WSN [5].
Low Energy Adaptive Clustering Hierarchy protocol
(LEACH) is a clustering based protocol that is proposed by
Heinzelmon. The author introduced a hierarchical clustering
algorithm for sensor networks. LEACH splits the sensor
nodes into small clusters and chooses one of them as the
cluster-head (CH). LEACH utilizes randomized rotation of
CH role among sensor nodes to evenly distribute the energy
load among them to avoid draining the battery of any one
sensor in the network [8].The BS periodically changes both
the cluster membership and the cluster-head (CH) to
conserve energy. In LEACH protocol, cluster head
probability becomes within reach of all the sensor nodes,
which makes the nodes achieve economize energy as the
main objective of prolonging the network lifetime [9].
Our proposed modification on LEACH protocol considers
both the distance and residual energy information of sensor
nodes to select more efficient CH.
This paper is organized as follows: section II describes the
related works, which discusses hierarchical routing protocols
research point concerns with cluster-head selection
mechanism. Section III depicts the LEACH overview and
problem definition. Our proposed for CH selection is
described in section IV. The performance evaluation of our
proposed and the other is discussed in section V. Finally, the
paper is concluded in section VI.
II. RELATED WORK
In this section, briefly existing works relating to proposed
scheme were presented. Many studies have resulted in
hierarchical routing protocols are adequate for static sensor
networks, and suitable clustering algorithm is the premise
key of the research of hierarchical routing protocols. LEACH
is a classic clustering routing protocol.
144 1
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Compared with ordinary routing protoccols, LEACH can
reduce communication energy as much as a eight times and
thus significantly improve the network performance [10].
The selection process of cluster head nodes n in LEACH
protocol is random; causing the uneven disstribution of cluster
head nodes [9].Single-hop and multi-hopp are two types of
data communications. In single-hop, clustter heads send data
directly to the sink so; the farthest clusteer away from sink
node consumes more energy than other cluusters. In multi-hop
data transmission model, some intermediiate nodes (cluster
heads) can be used to forward the data towards the sink.
These cluster heads group form a relayy network among
themselves to send data to the sink [10, 111].
A Hybrid, Energy Efficient, Distributed (HEED)
( clustering
protocol was proposed that it could minnimize the control
overhead and prolong the network lifetim me. HEED uses a
distributed algorithm that can converge quuickly and has been
shown to have low overhead.
HEED protocol considered the residual energy of sensor
nodes and the cost of communication within the cluster
during the selection of cluster heads [122]. The impact of
heterogeneity of sensor nodes was studied in terms of their
energy, and this clustered scheme was bassed on the residual
energy significance.
An energy efficient clustering protocol (E EECPL) organizes
sensor nodes into clusters and ring topology used to send
data. Thus, each sensor node can receive daata from a previous
neighbor sensor node and can transmit dataa to a next neighbor
sensor node.
Energy Aware Intra Cluster Routing (EAICR) is appeared to
increase the energy efficiency up to 17% and to increase the
network lifetime up to 12%. EAICA conssidered the number
of packets sent in the network, energgy consumed, and
residual energy level of sensor nodes at specific time and
network lifetime [13].
Some of LEACH protocols depict in
i the following
subsections.
A. E-LEACH protocol
Energy-LEACH (E-LEACH) protocoll improves the CH
selection procedure by making residual eneergy of node as the
main metric to decide whether the node is a CH or not [14].
In first round, every node has the samee probability to be a
CH; it means that the CHs are randomly seelected as the same
in LEACH. In the next rounds, the residuual energy of each
node is different after the first round coommunication and
taken into account for the selection of the CHs. It means that
the nodes having more energy will becomee a CHs rather than
nodes with less energy.
B. TL-LEACH protocol
A new version of LEACH called Tw wo-Level LEACH
(TL-LEACH) was proposed. In this protoocol, there are CHs
collect data from other members in its cluster as original
LEACH, but not transfer the data to the BS
B directly; it uses
one of the CHs that lie between the CH annd the BS as a relay
station [15].
C. M-LEACH protocol
Multi-hop-LEACH (M-LEACH) protoocol selects optimal
path between the CH and the BS via other CHs and use these
CHs as a relay station to transmit data over through them [16].
Not like in LEACH, each CH diirectly communicates with BS
no matter the distance between them.
t If CH and BS are a little
far, CH will consume a lot of its energy. The communication
among CHs is multi-hop; thee optimal path is needed to
transmit the data from a CH thrrough others CHs to a nearest
CH of BS.M-LEACH protocol is i almost the same as LEACH
protocol, but the communicationn mode between CHs and BS
are multi-hop instead of single hop
h as in LEACH.
D. LEACH-C protocol
LEACH has no guaarantee about both the
placement/number of cluster heeads. In order to improve the
LEACH protocol, the centralizeed clustering algorithm is used
with the same steady-state phasse as LEACH, this protocol is
called LEACH-C. Because off that, LEACH-C [17] has a
better performance by dispersingg the cluster heads throughout
the network. In LEACH-C, each e node sends information
about its current location by ussing GPS and residual energy
level to the BS to both determiine good clusters. In addition,
the BS ensure that the energy loaad is evenly distributed among
all the nodes.BS determines opttimal clusters and broadcasts a
message containing cluster headd ID for each node [11,13,18].
Centralized approach cannot sccale to very large numbers of
sensors.
E. V-LEACH protocol
A new version of LEACH H protocol called vice-Cluster
selection approach in LEACH (VLEACH) [19]. The cluster
contains both CH that is responssible only for sending data that
is received from the cluster mem mbers to the BS, vice-CH that
is a node that will become a CH H of the cluster in case of CH
dies, and cluster nodes that gathher data from environment and
send it to the CH. There is improovement of VLEACH in [20],
the vice-CH selection proceess basis on three factors:
minimum distance, maximum residual energy, and minimum
energy. Because of vice-CH, thee network lifetime increases as
never the cluster head will not die
d and cluster nodes data will
always reach the BS; no need to select a new CH each time the
CH dies.
III. CH PROTOCOL
LEAC
A. LEACH overview
The BS periodically changess both the cluster membership
and the cluster-head (CH) to connserve energy. Figure 1 shows
LEACH clustering hierarchical model.
Fig. 1: LEACH clusteriing hierarchical model.
The CH aggregates informaation from sensor nodes in its
private cluster and forwards information
i to the BS. Data
aggregation and data compressiion performed by CH and BS
145 2
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
receives a compressed data. Randomness in cluster head
selection without considering nodes' residual energy may lead
to energy draining in such cluster head [21].
The operation of LEACH is divided into rounds. A desired
percentage of total network nodes are selected as cluster
heads. Each round begins with a set-up phase when the
clusters are organized, followed by a steady-state phase when
data are transferred from the nodes to the BS.
Set-Up Phase split into advertisement, cluster set-up and
schedule creation phases. In the set-up phase, a random
number between 0 and 1 is selected by each node n. If this
random number is less than a threshold T (n), the node
becomes lucky cluster head for the current round. The
threshold is set as shown in equation (1).
(1)
0,
Where P is the percentage of cluster heads to all nodes,
and r is the selected rounds number, r mod (1/P) stands for the
number of selected cluster head nodes before this round, and
G is the group of nodes that have not been elected as cluster
head nodes previously. When r = 0, the possibility of each
node becoming the cluster head is P. If it becomes the cluster
head node in the first r rounds, it can be no longer re-elected in
the future (1/P-r) round which enhances the possibility of
other nodes to become a cluster head. After (1/P) rounds, all
nodes have a possibility of P to be a cluster head once again,
over and over again [22].
The threshold value depends on the desired percentage to
become a cluster-head- p, the current round r, and the set of
nodes that have not become the cluster-head in the last 1/p
rounds, which is symbolic by G. In every round, if one node
has been cluster head before, then T (n) is set to 0, so that this
node will not be selected again. For the nodes that have not
been selected once, the possibility of being selected is T (n)
[23, 24].
As the number of nodes which have been cluster head
increases, T (n) will increase, so the possibility for the rest
nodes to be selected will increase. When there is only one
node left, T (n) =1, which means this node will be selected for
sure. Higher probabilities nodes will be chosen as the Cluster
Heads.
Informing the cluster-nodes by advertisement message is
done by elected CH to the rest of the nodes in the network,
which include welcome message to join their clusters. The
non-cluster head nodes decide to join the clusters according to
received advertisement signal strength.
The non-cluster head nodes then reply with
acknowledgement message to their respective cluster-heads
that they will be under control in their cluster. The
cluster-heads creates a Time Division Multiple Access
(TDMA) schedule and assigns each node a time slot in which
it can transmit the sensed data.
Since the environment is noisy and sensor nodes can be
mobile, the MAC protocol must be power aware and able to
minimize collision with neighbors broadcast.
Each non-Cluster Head transmits a join-request message
(Join-REQ) back to its chosen Cluster Head using a CSMA
MAC protocol.
TDMA Schedule Prevents collision among data messages
and Energy conservation in non-cluster-head nodes. It allows
several users to share the same frequency channel by dividing
the signal into different time-slots. It has a natural advantage
of collision free medium access.
During Steady-State Phase, TDMA schedule permits the
nodes to turn off their transmitters if there is no activity in the
cluster. The non-cluster head nodes start sensing data and
transmit it in their-own time-slot [25].The cluster-head node,
after receiving data from all the member nodes, aggregates it
and then sends it to the base-station. After a certain time,
which is determined a priori, the network again goes back into
the setup phase and new cluster-heads are chosen. Each
cluster communicates using different Code Division Multiple
Access (CDMA) codes in order to reduce interference from
nodes belonging to other clusters [24].
In the original LEACH, the CH is always on receiving
data from cluster members, it aggregates these data. Then, it
sends this data to the BS that might be located far away from
it.
Death of CH means data aggregation misses use and
segment covered is out of service.
B. problem definition
Residual energy of nodes is not considered in the CH
random selection. Therefore, if the node loaded with usual
functions not compatible with residual energy. The CH will
then be out of service without warning. In large-scale
network, the CH suffers from communication cost especially
in energy consumed for long distance transmission.
IV. PROPOSED ALGORITHM
Take residual energy and distance into CH election
question of life and death for WSN lifespan. Inharmonic
distribution of nodes leads to dense and slight clusters in the
same network at the same time, which affirms that some CHs
may be loaded more than others sensors.
In set-up phase, residual energy and distance are
considerable factors in calculation the value of T (n).The
cluster head will be rotated by computing both distance and
residual energy of nodes and select the node with maximal
parameter. Enhanced LEACH (ELEACH) in [26] decision on
basis of distance and residual energy .ELEACH calculates
decision variable (dij) as shown in equation (2).
, Where (2)
Where Ej is the initial energy for each node, Em is the
remaining energy of cluster head j, Sij is the distance between
the nodes i and the Cluster head j, and dij is the node i decision
value for cluster head j.
This is fair load distribution among the nodes in one
cluster [9]. However, it is not satisfied for the good
performance, so the decision variable has to be verified again.
Our proposed takes into account both the distance and
residual energy as well to calculate the decision value by the
equation (3), where is the average of the first order
differences of the energy consumed and is the average of the
first order differences of the distance. By the equation (3), CH
can be selected to improve the performance of ELEACH. Our
proposed is Improved LEACH (ILEACH).
,
146 3
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Where and (3) (4) Average energy ratio (AER): Represented as the average
energy consumed (Econsumed) at each node divided by the total
ILEACH is an optimization equation (3) which leads to energy available (Eavailable) for simulation time. AER is
more improvement on packet delivery ratio, average delay, evaluated as shown in equation (7).
throughput and number of nodes alive.
At the beginning of each round, each node receives the (7)
residual energy from all the neighbors in its cluster range.
(5) Number of alive nodes (Nalive): Number of remaining
Cluster head selected based on its weight calculated based
nodes after simulation time.
on its residual energy and the received signal strength for the
signal broadcasted by the sink. C. Simulation Results
The proposed system will improve the network life and (1) Throughput (Thr)
total communication over the network. The selection of Figure2 depicts the throughput versus number of
cluster head node in LEACH has serious defect that leads to sensor nodes. It is observed that the throughput increases with
unreasonable CH selection that the nodes have different number of nodes increase because of increasing the amount of
energy while big clusters and small clusters may exist in the data sent to the base station. Noted also that our proposed has
network at the same time. a good throughput with comparing with both the original and
ELEACH protocol. This is because there is a load balance of
the CH role distribution among the nodes in one cluster by
V. PERFORMANCE EVALUATION taking into account both the energy consumed of each node
and the distance among nodes in cluster. Also, our proposed
A. Simulation setup makes dij increases by multiply energy consumed by the rate
The simulations are often used for understanding and of distance in order to decrease the effect of distance changes.
prediction of the behavior of protocols and data streams in
networks. All simulation results in this paper are obtained
using NS2 [27] simulator.NS2 is used to evaluate the
performance of original LEACH, ELEACH and our proposed
(ILEACH).WSN area size in meter is 600x600, the initial
energy of each node in the WSN is assumed as 10 j, the
number of sensor nodes varies from10 to 250 and 20% of
sensor nodes per round become cluster heads.
B. Evaluation metrics
In order to check the protocols performance in terms of its
effectiveness, many different metrics can be used. In this
study, one can use throughput, average end-to-end delay,
observe the packet delivery ratio, average energy ratio and
number of alive nodes for protocols evaluation.
(1) Throughput (Thr): The ratio of total packets received
(Nrecv) by a receiver from a sender for a total time (T) needed
for receiving the last packet by receiver measures in bit/sec
and byte/sec. It can be expressed mathematically as shown in Fig. 2: Throughput versus number of nodes for original
equation (4). LEACH, ELEACH and our proposed.
(4)

(2) Average end-to-end delay of data packets (Davg): is the

average delay between the sending of the data packet source
and its receipt at the corresponding received. This delay
includes the time due to route acquisition, buffering, link, the
processing at intermediate nodes, retransmission at the MAC
layer, etc. If the value of end-to-end delay is high then it
means the protocol performance is not good due to the
network congestion. Equation (5) depicts the average
end-to-end delay of data packets.

(5)

(3) Packet delivery ratio (PDR): is the ratio of number of data

packets successfully received (Nrecv) by the sink to the total
number of data packets sent (Nsend). PDR is shown in Fig. 3: Average delay versus number of nodes for original
equation (6). LEACH, ELEACH and our proposed.

(6)

147 4
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
(2) Average end-to-end delay of data packets (Davg)
Figure 3 shows the Average delay versus number of
sensor nodes. Noted that our proposed has a minimum delay
with comparing with both the original and ELEACH
protocol. Because of considering the distance when selecting
the CH.
(3) Packet delivery ratio (PDR)
Figure 4 depicts the packet delivery ratio versus number of
sensor nodes. It is noted that PDR decreases with increasing
the number of nodes because of the fixed available bandwidth
in the network and the worked area. Noted that our proposed
has a good PDR with comparing with both the original and
ELEACH protocol. Because there is a good distribution of the
nodes and a good selection of CH for each cluster, which
makes a good data flow from CH to BS.
Fig. 4: PDR versus number of nodes for original LEACH,
ELEACH and our proposed.
(4) Average energy ratio (AER):
Figure 5 depicts the average energy ratio versus number of
sensor nodes. Noted that the AER slightly increases with
increasing the number of nodes. Also noted that our proposed
has a lowest energy consumed with comparing with both the
original and ELEACH protocol. Because the role of CH is
distributed among cluster nodes in good manner.
Fig. 5: A.E.R versus number of nodes for original LEACH,
ELEACH and our proposed.
(5) Number of alive nodes (Nalive)
Figure 6 depicts the number of alive nodes versus number
of sensor nodes. It is noted that our proposed increases the
nodes lifetime with comparing with both the original and
ELEACH protocol. Because of using the cluster based
protocols. In addition, the role of CH is distributed among
cluster nodes in good manner.
Fig. 6: Number of nodes alive versus number of nodes for
original LEACH, ELEACH and our proposed.
CONCLUSION
Unjust distribution of cluster heads often causes a
significant loss of energy. To solve this problem, Cost
function algorithm based on the remaining energy and their
distance from sink exists. The simulation results show that our
algorithm is much more efficient. Our proposed algorithm
increases sensor lifetime, throughput, and the delivery packet
ratio and decreases both the packet delay and the sensors
power consumption.
References
[1] Potdar, V.; Sharif, A.; Chang, E., "Wireless Sensor Networks: A
Survey," International Conference on Advanced Information
Networking and Applications Workshops (WAINA '09), PP.636-641,
26-29 May 2009.
[1] Akkaya, Kemal, and Mohamed Younis. "A survey on routing
protocols for wireless sensor networks." Elsevier Journal of Ad hoc
networks, Vol. 3, No. 3,PP. 325-349, May 2005
[2] Di Marco, Piergiuseppe. "Protocol design and implementation for
wireless sensor networks." M.Sc.,Stockholm, Sweden April 2008.
[3] Mohamed Guerroumi, Al-Sakib Khan Pathan, NadjibBadache, and
Samira Moussaoui, "Strengths and Weaknesses of Prominent Data
Dissemination Techniques in Wireless Sensor Networks",
International Journal of Communication Networks and Information
Security (IJCNIS) Vol. 5, No. 3, December 2013.
[4] Rajesh Patel, Sunil Pariyani, and Vijay Ukani, "Energy and
Throughput Analysis of Hierarchical Routing Protocol (LEACH) for
Wireless Sensor Network", International Journal of Computer
Applications Vol. 20, No.4, April 2011.
[5] Ramesh, Siddharth. "A Protocol Architecture for Wireless Sensor
Networks." Proc. of the first ACM International Workshop on
Wireless Sensor Networks and Applications (WSNA). 2008.
[6] Akyildiz, Ian F., Weilian Su, YogeshSankarasubramaniam, and
ErdalCayirci. "Wireless sensor networks: a survey", Elsevier Journal
of Computer networks, Vol. 38, No. 4, PP. 393-422, 2002.
148 5
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
[7] Mitra, Rudranath, and Anurupa Biswas,"Enhanced Clusterhead
Selection Algorithm Using LEACH Protocol for Wireless Sensor
Networks", International Journal Of Computational Engineering
Research (IJCER), Vol. 2, No. 3, PP.766-770, May-June 2012.
[8] Ma, Xiao Wen, and Xiang Yu, "Improvement on LEACH Protocol of
Wireless Sensor Network", Applied Mechanics and Materials, Vol.
347, PP. 1738-1742, August 2013.
[9] Ruan, Feng, Chuanyang Yin, Jinhui Chen, Jin Wang, and
ShengjunXue. "A Distance Clustering Routing Algorithm
Considering Energy for Wireless Sensor Networks." International
Journal of Future Generation Communication and Networking, Vol.
6, no. 5 PP. 73-80, 2013.
[10] Gnanambigai, J., Dr N. Rengarajan, and K. Anbukkarasi. "Leach and
Its Descendant Protocols: A Survey." International Journal of
Communication and Computer Technologies, Vol. 1, No. 3,
PP.15-21, September 2012.
[11] Kour, Harneet, and Ajay K. Sharma. "Hybrid energy efficient
distributed protocol for heterogeneous wireless sensor network."
International Journal of Computer Applications Vol. 4, No. 6, PP.1-5,
July 2010.
[12] Hong, Jiman, Joongjin Kook, Sangjun Lee, Dongseop Kwon, and
Sangho Yi. "T-LEACH: The method of threshold-based cluster head
replacement for wireless sensor networks." Information Systems
Frontiers, Vol. 11, No. 5, PP.513-521, 2009.
[13] Fan Xiangning; Song Yulin, "Improvement on LEACH Protocol of
Wireless Sensor Network," International Conference on Sensor
Technologies and Applications, 2007, pp.260-264, 14-20 Oct. 2007.
[14] Loscri, V.; Morabito, G.; Marano, S., "A two-level hierarchy for
low-energy adaptive clustering hierarchy (TL-LEACH)," IEEE 62nd
Vehicular Technology Conference (VTC-2005-Fall), pp.1809-1813,
25-28 Sept. 2005.
[15] Mo Xiaoyan, Study and Design on Cluster Routing Protocols of
Wireless Sensor Networks, Dissertation, Hang Zhou, Zhe Jiang
university, 2006.
[16] Heinzelman, W.B.; Chandrakasan, A.P.; Balakrishnan, H., "An
application-specific protocol architecture for wireless micro sensor
networks," IEEE Transactions on Wireless Communications, vol.1,
no.4, pp.660-670, October 2002.
[17] Heinzelman, Wendi B., Anantha P. Chandrakasan, and
HariBalakrishnan. "An application-specific protocol architecture for
wireless micro sensor networks." IEEE Transactions on Wireless
Communications,Vol. 1, No. 4, PP. 660-670, October 2002.
[18] NutanSindhwani and RohitVaid,"V Leach: An Energy Efficient
Communication Protocol for WSN", Journal of MECHANICA
CONFAB, Vol. 2, PP.79-84, Feb-March 2013.
[19] Ahlawat, A.; Malik, V., "An Extended Vice-Cluster Selection
Approach to Improve V Leach Protocol in WSN," Third International
Conference on Advanced Computing and Communication
Technologies (ACCT), pp.236, 240, 6-7 April 2013.
[20] K. Ramesh and K. Somasundaram, "A comparative study of cluster
head selection algorithms in wireless sensor networks." International
Journal of Computer Science & Engineering Survey (IJCSES) Vol.2,
No.4, PP.153-164, November 2011.
[21] W Heinzeman, A Chandrakasan, H Balakrishnan. "Energy-efficient
routing protocols for wireless microsensor networks", Proc 33rd
Hawaii International Conference of System Sciences, Maui, 2000, PP.
534-546.
[22] Arbab, Erfan, VaheAghazarian, AlirezaHedayati, and
NimaGhazanfariMotlagh. "A LEACH-Based Clustering Algorithm
for Optimizing Energy Consumption in Wireless Sensor Networks."
In second International Conference on Computer Science and
Information Technology (ICCSIT'2012), pp. 147-150, April 28-29,
2012.
[23] Sindhwani, Nutan, and RohitVaid. "VLEACH: An Energy Efficient
Communication Protocol for WSN." Mechanica Confab, Vol. 2, no. 2,
PP. 79-84, February-March 2013.
[24] Yong, Zhu, and Qing Pei. "An energy-efficient clustering routing
algorithm based on distance and residual energy for wireless sensor
networks", International Workshop on Information and Electronics
Engineering (IWIEE) and Procedia Engineering, Vol. 29,
PP.1882-1888, 2012.
[25] N. M. Nawar, S. E. Soliman, H. M. Kelash, and H. S. El-sayed,
"Enhancement of Mobility Model for Cluster Hierarchical Routing
Protocol for wireless sensor networks", International journal of
Computer Applications (IJCA), Vol. 94, No. 9, PP. 12-16, May 2014.
[26] Ns-2 network simulator (ver. 2). LBL, URL:
http://wwwmash.cs.berkeley.edu/ns.
149 6
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Intelligent Collision Detection and Avoidance in
Railways using GPS/RF Module
S. A. Qunber, M. S. Zaghloul, O. Badawy, and M. Farouk
Sarah.aast@gmail.com, dr_mszaghloul@yahoo.com, obadawy@aast.edu, mahmoud.farouk@aast.edu
College of Computing and Information Technology
Arab Academy for Science, Technology, and Maritime Transport, Alexandria, Egypt
Abstract - Railways are an important mode of transport
because it can carry large number of people and goods
at the same time, so it is important to protect the
peoples life and property. There are many collision
accidents between trains, because of irresponsibility of
drivers and signaling problems. Several techniques have
developed in order to avoid such danger loss of life and
property. In this paper, we have proposed an approach
using GPS and RF module for detecting and avoiding
train collisions. The GPS module identifies the location,
speed, and direction of the trains with highest accuracy.
This information transfers to the train microcontroller
in the same vicinity via RF module. The availability of
the information allows the train microcontroller type
PIC16F877 to take accurate decision as for train
location. This system warns the driver both visually and
by giving an alert sound, to allow avoiding the collision
between trains when they are separated by one kilo. We
used a vibration sensor for sensing the train on the
track. The prototype was designed using microcontroller
and tested successfully through RF communication.
Index Terms - Wireless LAN, Collision Detection, Micro
Controller, Train Anti-Collision System.
I. INTRODUCTION
Railways are a mode of transport essential to the daily life
of people. It can carry large numbers of people and goods
speedily at the same time. It is cheap and affordable mean of
transportation for millions of passengers. According to
COMESA Infrastructure Statistics, it carried more than 292
million people and 5.8 million tons of goods in year 2010 as
[1]. According to COMESA Infrastructure Statistics, the
recorded drop in the number of passengers transported is
due to lack of safety, as many accidents occur. Over the past
ten years, more than one hundred people were killed and
another about four hundred were injured due to railway
collisions as [2]- [3].
Communication is a key issue in traffic management
and tracking and railroad traffic control is not an exception.
The human factor has proven to be the direct cause of these
several train collisions as [4]. The driver does not know
anything happening around. He does not have any
information about other trains. He just sees the signal aspect
and follows it. The signal aspect depends upon the ASM
978-1-5090-0217-7/15/$31.00 2015 IEEE
(Assistant Station Manager), so if the ASM gives wrong
signals and gives wrong route, accidents will occur. Whole
working is ASM dominant; therefore, the drivers cannot
take any decisions. Rail accidents are inevitable where
communication is inefficient, hence the need to device
systems which will enable trains to communicate
automatically. With the aim of reducing injury and accident
severity, pre-crash sensing is becoming an active research
area.
According to an aspect of the present situation, this
paper describes an intelligent detect and avoidance collision
system that can be used to avoid collision between trains.
The proposed system is based on direct train-to-train
communication. The system allows trains autonomous
detection to imminent collisions. Designed as a safety
overlay system, it shall warn train drivers in such situations.
This approach is based on GPS and RF module and
receivers. The GPS receiver is capable of identifying the
longitudinal and latitudinal position and the speed of the
specific train by receiving information from satellite. The
time delay of GPS is very small, and the accuracy of GPS is
from 10 to15 m which i 0.3 to 0.5 % from the 3 km we
calculate so we can neglect this value.
The Radio Frequency (RF) module is used in this
system to allow exchanging the trains information between
trains in the same vicinity.
The rest of this paper is organized as follows: section II
discusses the related works, section III describes the system
overview, section IV describes the proposed system
structure, section V explains the scenarios of collision,
section VI explains working flow, section VII indicates the
results, and finally conclusion is discussed in section VIII.
II. RELATED WORKS
Many methods have been developed and used for avoiding
collision and for getting proper communication. Intelligent
Collision Avoidance System ICAS as [5] is based on Radio
Frequency RF transmitters and receivers to sense the
direction and the position of the trains. RF transmitters
determine the track IDs and the position of trains in main
tracks or sub tracks. They put vibration sensor at starting
and ending of sub tracks. When the trains are on the same
track and are separated by three-kilo meters, the system
150
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
warns the driver both visually and gives alert sound for
avoiding frontal collisions. The disadvantages of this system
that eliminates the potential for collisions, includes; the
position of trains are not accurate and the speed is unknown.
Another method using powerful combination of mobile
computing, Global System for Mobile Communication
(GSM), Global Positioning System (GPS), Geographical
Information System (GIS) technologies and software for
improving railway security and safety and predict the
dynamics of trains collisions as [6]- [7]-[8]. The in-built
GPS module identifies the train location and speed with a
highest accuracy and transfers the information via GSM to
the central system to identify the possible safety issues.
Location data processed to provide visual positioning using
maps granting a wholesome view on train location.
Positioning data along with train speed helps the
administration to identify the possible safety issues and
react to them effectively using the communication methods
provided by the system. This system reduces waiting time,
provide automatic information of train, knowing about the
train arrival time & location. Using a SMS people can find
the location & time of arrival but it needs infrastructure.
Other system uses WIMAX for detecting and avoidance
collisions as [9]. In this system using latest 4G technology
WiMAX (Worldwide Interoperability for Microwave
Access) i.e. IEEE802.16e standards for preventing train
accidents. The approach helps the train to know the location
of its own with the help of fixed WiMAX base stations
whose positions are known. The approach does not require
extra GPS service for calculating location, like the one
existing railway system uses. The WiMAX base station and
the train (mobile stations) contribute in sending the
information to the trains whenever a train comes in its
vicinity and range. The trains will be equipped with
WiMAX enabled laptops/tablets for communicating with
others. The speed, location and distance of other trains are
known in advance to increase awareness and safety. The
distance calculated between the train and the base station is
based on angle of arrival (AOA) and received signal
strength (RSS). Using this information, the trains will be
able to take possible precautions to avoid collision. The
technique will be more efficient and beneficial for avoiding
situations like collision and catastrophic hazards. The
advantages of this system includes, not requiring GPS
receiver and no need for infrastructure. The disadvantage is
being expensive.
III. DESIGN PATTERN
The Intelligent Collision Detection and Avoidance system is
based on integration of GPS and RF model as shown in FIG.
1. This system consists of three sections: 1) base station:
which identify the track ID, 2) train unit: where a GPS
identifies the train location, speed and direction and where
controller takes the accurate decision to avoid collisions, 3)
sub track unit: which detect the train location if it enters or
leaves the sub tracks. The train has a controller unit that
contains several parts; like transmitter which transmits the
information about train that can be identified by GPS and
the data about the track that identified by the transmitter at
the base station. This information transmitted through Radio
Frequency RF transmitter and received by RF receiver.
Then the receiver sends this information to microcontroller
where the controlling procedure is processed. The
microcontroller processes the data and takes accurate
decision to avoid collisions. The system also consists of sub
track block for detecting the train position whether it is
entering the sub track or leaving the sub track by using
vibration sensor under the track, which senses the train
moving.
1 (RF Transmitter)
Base
Station
B
2(Train
controller
unit)
2(Train controller unit)
3 (RF Transmitter &
1 (RF Transmitter)
Vibration sensor)
Base
Station
A
FIG. 1. TOP VIEW OF THE INTELLIGENT AVOIDANCE COLLISION
SYSTEM
IV. PROPOSED SYSTEM STRUCTURE
A. Base Station Unit
Two RF transmitters (1 in FIG. 1) mounted at base stations.
One is at the beginning of the track and the other is at the
end of the track. Each transmitter has a particular code
signal that identifies the track IDs. When the train starts to
move, the transmitter transmits the code of the track to the
train to avoid the collision.
B. Train Controller Unit
Every train has a train controller unit that processes the
information to take the correct decision to avoid the
collision. The system consists of GPS, Transmitter,
Receiver, Microcontroller, LCD and Alarm. The
microcontroller unit in vehicle section module sends alarm
sound and message for different events to the driver as:
151
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

1 BASE STATION UNIT 2 TRAIN COMTROLLER UNIT

BASE STATION M
Main Track ID
RF Transmitter LCD
TRANSMITTER

RF Receiver
3 SUBTRACK UNIT

Sub Track ID
MICRO
CONTROLL
LER
Amplifier RF Traansmitter GPS Receiver
Vibration
Sensor

MIC
CRO Alarm Driver
ADC CONTR
ROLLER

FIG. 2. STRUCTURE OF INTELL

LIGENT COLLISION DETECTION AND AVOIDANCE SYSTEM

Green LED and GO message: when thhe trains move at

different tracks and no collision occurs,,
Vibration Amplifier RF Transmitter
Red LED and STOP message: when thee trains move in
Sensor
the same track and different direction annd
Red LED and STOP OR REDUCE E THE SPPED
message: when the trains move in thee same track and MICRO
the same direction. ADC CONTROLLER
C. Sub Track Unit
FIG. 3 THE BLOCK DIAGRAM OF SUB TRACK SENSING
Sub track is the path that branches from m main track. Both
starting and ending of all the sub tracks arre equipped with
vibration sensors and a low power ttransmitter. The V. COLLISION SCE
ENARIOS
transmitters Tx1, Tx2 used to inform that thee train entering or
In railroad transport, well-defined collision
c scenarios will be
leaving from the sub track, respectively. Thhe block diagram
distinguished in different modess of operation. Table1
of the sensing unit shows as in FIG. 3. Thhe sensors consist
illustrated scenarios of collisions an
nd the expected output to
of crystal that is mounted between two iron pieces and
the drivers.
placed under the track. The force exerted onn the track by the
train makes the crystal to vibrate and oscillate. These
Scenario 1
oscillations amplified and converted to diigital format and
gives to the microcontroller. The sensor willl produce signals When two trains are at different traccks as in FIG. 4 and each
only when trains are on the specified trackk where it placed. track has a particular code, so each train has a different track
The sensor will not produce signal when thhere is a train on ID. The microcontroller compares between
b tracks IDs of the
nearby track. trains that train receivers receivees, when track IDs are
When a train is moving on the trrack, the sensor different, so the output light will still
s be green that means,
produces a signal due to vibration of the track. We have safe case and no collision danger.
selected vibration sensors because it will produce signals
only when train is on the specified track where it placed.
The sensor will not produce signal when thhere is a train on
nearby track. It is a generalized system easily
accommodated in any train. The output leevel of vibration
sensor is very low, in terms of millivoltss (150 mv). The
output is amplified and converted into diggital signal using FIG. 4. NO COLLISION SCENARIO
ADC 0808 and given to micro controller.

152
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

TABLE 1 between them, if the distance is less than 2 km and the

ILLUSTRATED COLLISION SCENARIIOS AND OUTPUT
vers by giving alert sound
direction is different, alerts the driv
TO THE DRIVERS
and the red output light on, and by message to stop the
EXPECTED O
O\P trains to avoid the collision between n them.
COMBINATION
OF CASE STUDY vid
deo
audio
writing light
1. Two trains at 3 beep (TrackID, positionn, speed, green FIG. 6. HEAD ON COLLISIO
ON SCENARIO
different tracks direction) of two traains
COMMENT: Each train has different trackID, so thhere is no collision Scenario 4
occurs. This case was implemented.
2. Two trains at the 5 beep (TrackID, positionn, speed, red When train moves above other train n as in FIG. 7 each track
same track & same direction) of two traains has different track ID, so the micrrocontroller found it safe
direction
case and does nothing.
COMMENT: Each train has the same trackID, tthe microcontroller
compare between the direction and depending on thhe speed, alerts the
driver to reduce or stop the second train. this case was implemented
3. Two trains at the 5 beep (TrackID, positionn, speed, red
same track & direction) of two traains
different directions
COMMENT: Each train has the same trackID, tthe microcontroller
compares between the directions then alerts the driverrs to stop the trains.
This case was implemented.
4. Train 3 beep (TrackID, positionn, speed, green
underground and direction) of two traains
FIG. 7. TRAIN MOVES ON BRIDGE AN
ND OTHER TRAIN MOVES
the other above it UNDER IT
COMMENT: Each train has different trackID, so thhere is no collision
occurs. This case was implemented.
Scenario 5
5. When the train 3 beep (TrackID, positionn, speed, Green
enters to sub track direction) of two traains or red When the train switches from thee main track to the sub
COMMENT: Just the train moves on vibration sensorr a TrackID will track: Starting and ending of each sub track equipped with
change.
Green light ON if the two trains in different tracks andd red light ON if the
sub track module to identify the lo ocation of the train when
two trains in the same sub track. entering or leaving the sub track k. Each sub track has a
6. two trains will 5 beep (TrackID, positionn, speed, red particular code. When the train leaaves the main track and
Converge at direction) of two traains enters to sub track, the train receiives the code of the sub
intersection point track and changes its track ID and d when it leaves the sub
COMMENT: Put vibration unit just 1km before inntersection point to track, it receives the same track ID again then automatically
change the trackID of the train. The trains have thee same trackID and changes its track ID again to main track
t ID. When a train is
depending on the speed, alert the driver to reduce or stoop the second train.
moving on the sub track, the sensorr produces a signal due to
vibration of the sub track. The transsmitter Tx1 that is present
in the sub track transmits with the signal SUB111. The RT
Scenario 2 receive the signal by and pass itt to the controller. The
controller will store it and transm mit through TT indicating
When two trains are on the same track and iin same direction that the train is in sub track. At the end
e of sub track, receiver
as in FIG. 5, the memory of the trains has thhe same track ID. RT receives SUB111 again from trransmitter Tx2 indicating
The microcontroller calculates the distance between them in the end of sub track and hence thee controller automatically
addition to the length of the train, if the disttance is less than generates the signal LIN111.
2 km and the direction is the same then aleert the drivers by
giving alert sound and the red output ligght on, and by a Scenario 6
message to stop or reduce the speed of the ssecond train.
If the train enters from main traack to other main track
directly as in FIG. 8 the vibration seensor placed at a distance
FIG. 5. REAR-END COLLISION SCEN
NARIO of 1km from the intersection beforee the train enters the other
main track, and transmitter Tx transmits signal code
Scenario 3 LIN111which is the code of the maiin track.

When two trains are on the same track and in different

directions as in FIG. 6, the memory of thhe trains has the
same track ID. The microcontroller calculates the distance

153
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

VII. RESUL
LTS

Our system implemented and teested as shown in the

following figures. Here the variaables Lat, Long and H
indicate latitude, longitude and d height of the train
respectively. The direction 0 ind dicates that a train moves
FIG. 8. FLANK COLLISION SC
CENARIO from base station A to base stationn B for example and the
direction 1 means that the train moves from base station
VI. WORKING FLOW CHA
ART B to base station A as in FIG. 9.

In order to detect and avoid collisions betw ween trains, train

locations must be identified. The Follow paattern algorithm 1
Appendix A shows the steps for locating tthe trains in this
system. The input to the system is a longittude and latitude
from GPS receiver and track ID from RF trransmitter that is
at the base station. The first task processinng the data from
GPS by microcontroller then stores this innformation in the
memory. When the microcontroller receiives information
about other trains, then it compare betweenn track IDs of the
two trains. When the track IDs are thee same, then it
calculates the distance between them and ccompare between
the directions of the trains to take accuratee and appropriate FIG. 9 SHOWS THE DIRECTIO
ON OF THE TRAINS FROM
decision for avoiding collision. STATION A TO
O STATION B

Algorithm 1 Flow chart of the decision-m making system FIG. 10 shows the result for thee scenario 1 of FIG. 4.
Inputs: GA GPS Information , IDA Track ID D, According to FIG. 10, the track ID is different so appear the
GB GPS Information , IDB Track ID GO message to driver. The result forf the scenario 2, shown
Outputs: L Light , Str Messages , B Beep in FIG. 11, so alert message appear to the driver of the
1. IDA = GetMyTrack ID() peed and GO message to
second train to stop or reduce the sp
2. GA = GetMyGPSINformation() the first train. FIG. 12, show the scenario
s 3 and the trains
3. WHILE LOOP move in different direction. So alert the drivers to stop the
4. IDB = GetOtherTrackID() train by giving STOP messages. Th he scenario 4 tested as in
5. GB = GetOtherGPSINformation()) FIG. 13 that shows that the track IDs of the trains are
6. Distance = CalcDistance(GA, GB) different so it is safe case and no collision will occur and
7. IF IDB == IDA THEN GO message appear to the drivers. When the train enter to
8. IF Distance < 1km THEN sub track as illustrated in scenario 5, here we tested when
9. IF GB.Direction == GA.Diirection nd the other one enter to
one train still in the main track an
THEN sub track as in FIG 14.
10. Str = Stop The flank collision scenario o of FIG. 8 tested and
11. Return(5B , Red L , S
Str) indicated that the two trains have the same track ID and
12. ELSE same direction.
13. Str= Reduce speed oor Stop
14. Return(5B , Red L , S
Str)
15. END IF
16. ELSE
17. Str = Go
18. Return(0B , Green L , Str))
19. END IF
20. ELSE
21. Str = Go
22. Return(0B , Green L , Str)
23. END IF
24. END LOOP
FIG. 10 DETECT TWO TRAINS IN DIFFERENT TRACKS

154
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
FIG. 11 DETECT TWO TRAINS ON THE SAME TRACK WITH SAME
DIRECTION AND AVOID REAR-END COLLISION BETWEEN THEM
FIG. 12 DETECTS TWO TRAINS ON THE SAME TRACK WITH
DIFFERENT DIRECTIONS AND AVOIDS HEAD-ON COLLISION
BETWEEN THEM
FIG. 13 SHOW TWO TRAINS ONE OF THEM MOVES ON BRIDGE
FIG. 14 DETECT ONE TRAIN ENTER SUB TRACK AND
ANOTHER IN MAIN TRACK
VIII. CONCLUSION
In this research, a scalable rule-based system for detecting
and avoiding collisions between trains has been designed
and developed based on microcontroller. A GPS/RF module
for locating trains is proposed for detecting and preventing
collisions. The system demonstrates different expected
scenarios for single and multiple tracks. The system has a
graphical user interface and information displayed in a way
that is familiar for administrators, and drivers. This system
implementation and tests are successful. The software
designed using PICBASIC PRO language, and completed
using a Java program. The allotted code of a particular
transmitter can be re-used. A prototype designed to test the
feasibility of the collisions prevention. Results show that
this approach is appropriate to validate the collisions
prevention and increases the reliability of the system.
Moreover, simulations show the feasibility of the proposed
solutions.
The goal of this work is to design and implement a cost
effective and intelligent system to avoid collisions. We
believe that the system is more suitable for the developing
countries. Enriching the system by adding more knowledge
rules is a continuous process.
REFERENCES
[1] Secretariat, COMESA. Annual Bulletin of Infrastructure Statistics .
s.l. : COMESA, 2012. ISBN 978-9982-26-055-8.
[2] Series of disasters [online]. Available:
http://www.egyptianrailways.com/a-series-of-disasters/
[3] Egyptian National Railways [Online]. available:
http://en.wikipedia.org/wiki/Egyptian_National_Railways
[4] Statistic of train accidents. [Online]. Available:
http://www.egylovers.net/news/802783-
[5] Chellaswamy, C., Arul, S. ; Balaji, L. (2011).Design and analysis of an
intelligent collision avoidance system for locomotives. In IEEE, second
international conference on Sustainable Energy and Intelligent System
(SEISCON 2011).
[6] S.Punekar, Nisha and Raut, Archana. (2013). A Survey on Railway
Security in Wireless Network.. International Journal of Computer
Science and Network (IJCSN), Vol. 2:1.
[7] S.Puneka, Nisha and Raut, Archana A. (2013). Improving Railway
Safety with Obstacle Detection and Tracking System using GPS-GSM
Model. International Journal of Scientific & Engineering Research, vol.
4:8.
[8] Mohamed, Hossam Abdel Rahman. s.l. (2014). A Proposed Model for
Radio Frequency Systems to Tracking Trains via GPS(The Study for
Egyptian National Railways). International Journal of Intelligent
Systems and Applications, Vol. 6:3.
[9] Jain, Anjali and Tyagi, Dr. Neeraj (2013). Collision Detection and
Avoidance in Railways Using WiMAX. Indian Journal of Computer
Science and Engineerin (IJCSE), Vol. 3:1.
155
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

AppendixA

Flow chart of the decision-making system

Start

Initialize GPS

YES
Any message Compare between
Received? tracks ID
Set track ID

Read GPS YES NO

Calculate the Track ID is
distance equal?

Display

Distance < 1 YES

YES
km?
Beep 5 times & Red
Read any RF Are same
LED ON & stop
message direction?
message

NO Beep 5 times & Red

LED ON & stop or
reduce the speed
message

NO
Green LED ON

156
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Session V:
Image Processing and Pattern
Recognition

Title: Page#

Lossless Image Compression Using Agglomerative Hierarchical 167

Clustering and the Jensen-Shannon Divergence
Sameh Samir, Sherif F. Fahmy, and Gamal Selim

An Enhanced Gene Silencing Algorithm Using Hash Table 172

Hoda Elhamy Mohamed, Essam M. Hamed, and Mohammad H. Abdel-Rahman

Novel Single Image deblurring Technique using Sparse 179

representation and Discrete Radon Transform
I.M. EL-Henawy, A. A. Ali, Kareem Ahmed, and Hadeer Adel

FPGA Implementation of Iris Recognition Based on Fast DCT 185

Coding
Ramadan Gad, M. Zorkany, Nawal El-Fishawy, and Ayman El-SAYED

A Novel 3D Image Fusion Approach Using FTransform 191

Saad M. Darwish and Maha M. Ghoneim

An Elliptical Basis Function Network for Non-blind Image 197

Deconvolution
Mohammad Hashem, Essam A. Rashed, and Fatma Harby Mohamed

Multiclass Object Recognition Using Object-based and Local 202

Image Features Extractors
Menna Maged Kamel, Mohamed Waleed Fakhr, H. I. Saleh, and M. B.
Abdelhalim

166
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

978-1-5090-0217-7/15/$31.00 2015 IEEE 172

ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

173
ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

174
ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

175
ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

176
ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

177
ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

178
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Novel Single Image deblurring Technique using
Sparse representation and Discrete Radon Transform
I.M. EL-Henawy A. A. Ali
Computer Science Department, Computer Science Department,
Zagazig University. Menia University.
henawy2000@yahoo.com abdelmgeid@yahoo.com
AbstractIn this paper, a novel single image deblurring
technique based on sparse representation and radon transform is
presented. The Sparse representation is used to make an initial
estimation of the latent sharp image. Then, a set of directional
filters are applied to the blurry and noisy image to reduce the
noise while maintaining the blurry information on the orthogonal
direction. After that an initial kernel estimation at different
angles is performed using the initial latent image that was
produced using sparse representation. Each estimated kernel is
transformed to radon transform and the set of projections at
different angles are stored. An inverse radon transform is applied
to make a final kernel estimation. Finally, the Wiener
deconvolution is performed to estimate the final latent sharp
image. Experimental results showed the effectiveness of the
proposed technique. The best obtained PSNR is 28.233 at 0.04
noise ratio, whereas the best obtained SSIM is 0.896769 at also
0.04 noise ratio.
KeywordsSparse representation; radon, deblurring, blur,
PSF, image restoration.
I. INTRODUCTION
Photographs are very important in our life for because they
are part of our legacy and they record important memories for
us. There is an old adage says "A picture is worth a thousand
words". Many applications such as satellite navigation,
monitoring, medical imaging, astronomy, microscopy, and
remote sensing are totally dependent on the quality of recorded
images. Each image contains a set of pixels organized in a
grid. The intensity value at each pixel controls the tone at this
pixel. Unfortunately, the moving objects or camera shake
during shooting may cause images to be less or more blurry.
The blurring may be caused for several reasons, such as
relative motion between the camera and the scene, movement
of camera or object during light exposure, using long exposure
times, using wide angle lens, defocused optical system.
Practically, it is impossible to prevent the blur altogether
because it is a result of several overlapped factors. As a result,
the blur may completely destroy the visual characteristics of
the image. The blur can be occurred at two levels: (i) the entire
image level, where the blur is occurred because of camera
shake which causes all pixels in the image to be affected, and
(ii) the region level, where the image is blurred by object
motion which causes a specific region to be blurred. A simple
example to illustrate blurring is: when the image to be acquired
is a black image that contains only one bright (white) pixel and
all the surrounding pixels are dark. When capturing, the single
white pixel, also called a point source, is spread over its
neighboring pixels. A function called point spread function
978-1-5090-0217-7/15/$31.00 2015 IEEE
Kareem Ahmed Hadeer Adel
Computer Science Department, Computer Science Department,
BSU University. NUB University.
Kareem_ahmed@hotmail.co.uk hadeer_adel25@yahoo.com
(PSF) is used to describe the response of an imaging system to
a point source. In the other hand, Image deblurring and
restoration is one of the most important fields of image
processing. Blind Image deblurring is the process of restoring
the latent sharp image from the blurred one by solving
mathematical blurring model. The process is blind, i.e. there is
no information about the sharp image or the characteristics of
the blur kernel such as length and orientation. Image deblurring
is useful for a variety of fields include: Personnel Photograph
recovery, using telescopes to watch distant objects, watching
space vehicles and satellites, Radar imaging, Tomography and
medical imaging. In this paper, a novel single image deblurring
technique based on sparse representation and radon transform
is presented. The rest of the paper is organized as follows:
Section II presents the mathematical blur model and a brief
description of the available single image deblurring techniques,
Section III presents the first component of the proposed
algorithm which is estimating an initial latent image using
sparse representation, Section IV presents the subsequent
stages of the proposed algorithm that estimates the final blur
kernel using radon transform and estimates the final latent
image using Wiener deconvolution.
II. THE BLUR MODEL
The sharp image is a clear image that the we can identify
and recognize all its objects correctly. For example, a face
image is sharp if we can recognize eyes, nose, and lips
correctly. The edges in the image are responsible for
identifying and recognizing objects. The blur reduces the edge
content and makes the transition from one color to another very
smooth. Zooming is one type of blurring. When zooming an
image using pixel replication and large zooming factor, the
result will be blurry. However, the number of pixels in the
zoomed image is greater than the number of pixels in the
original image, which differentiate it from pure blurring. The
blurring is a linear process that leads to image degradation. The
blurring model is a combination of two operations: convolving
the original image with the blur kernel; then adding some
noise. Mathematically, the blurring model [31] can be
expressed as , Where is blurred image, is
the original image, is blur kernel, and is the additive noise.
The blur kernel is a linear shift-invariant point spread function
(PSF) or impulse response. Point Spread Function (PSF) is the
degree to which the imaging system spreads a point of
light[33]. The Fourier transform of the point Spread Function
(PSF) is called Optical Transfer Function (OTF) which can be
used to describe the response of a linear, position-invariant
179
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
system to an impulse[32]. Image deblurring techniques are
divided into non-blind or blind. Non-blind single image
deblurring techniques require a priori information about the
characteristics of the blur kernel and the original image to
estimate the blur kernel and restore the sharp image. Blind
single image deblurring techniques are able to estimate the PSF
kernel and restore the sharp image without any kind of data
about the blurring process or the sharp image. There are four
kind of filters [10] that can cause image blurring: (i) average
filter, (ii) weighted average filter, (iii) Gaussian filter, and (iv)
motion filter. There are several technique for single image
deblurring. A comprehensive review of the state-of-the-art in
image deblurring can be found in [10,30]. The Lucy-
Richardson algorithm, or RichardsonLucy deconvolution[34,
35], is an iterative procedure for recovering the sharp image
that has been blurred by a known point spread function (PSF).
Some techniques employed neural network [36, 37] to
converge the recorded blurred image to the sharp ones after
training the system on a private dataset. Several iterative
approaches [38-40] have been discussed for removing the blur
and restoring the sharp image. Several techniques use the
Wiener filter [41-43] which makes an optimal trade-off
between inverse filtering and noise smoothing and it is able to
remove the additive noise and invert the blurring
simultaneously. Blind deconvolution approaches [10] are
divided into categories: projection based blind deconvolution
and the maximum likelihood restoration. Sparse representation
techniques [44-50] have been widely used after developing l1-
norm optimization techniques, and also because natural images
are intrinsically sparse in some domain. Sparse decomposition
is used to estimate sparse multi-dimensional vector that
satisfies a set of equations in a linear system given high-
dimensional observed data and a design matrix. In [50] a new
deblurring approach based on sparse representation is
proposed. The authors used the KSVD algorithm to train an
over-complete dictionary on a set of images and curvelet to
represent the blur kernel. In [46], the authors propose to
express the problem of the deconvolution of Poisson noisy
images as the minimization of a new constrained problem. In
[49], the authors proposed an image restoration method based
on sparse regularized with learning dictionary. They first
construct a degradation model under the sparse representation
theory, with the Haar dictionary as initial over-complete
dictionary. Then they solve the degradation model by parallel
coordinate descent iteration algorithm. In[3], the authors
proposed a novel blind image deblurring method based on the
sparse prior of dictionary pair, estimating the sparse
coefficient, sharp image and blur kernel alternately. The
proposed method could avoid the deconvolution problem
which is an ill-posed problem, and obtain the result with fewer
artifacts.
III. ESTIMATING THE INITIAL LATENT IMAGE USING SPARSE
REPRESENTATION
In most cases, the blurry and noisy image is constituted
by convolving the latent sharp image with point spread
function (PSF) then adding some Gaussian noise .
Mathematically, the blur model can be shown as:
(1)
The objective of the blind deblurring is to estimate the
latent sharp image and the kernel from the blurry and noisy
image . Several techniques for single image deblurring have
been widely used to restore the latent sharp image blur from the
blurry and noisy image. However, most of these techniques are
based on deconvolution process. Since is the only known
variable and all of the other variables and are unknown,
so the number of unknowns is larger than the number of
knowns. So the deconvolution is an ill-posed inverse
problem[1] which requires many priors and regularities to
weaken the influence of artifacts and to stabilize the solution.
Recently, sparse prior is used in image deblurring to improve
performance via constraining the coefficients in some
transform domains[3]. M. Elad and M. Aharon [2] proposed an
image denoising technique to remove a zero-mean white and
homogeneous Gaussian additive noise from an image based on
sparse and redundant representations over trained dictionaries.
After they proposed the over-complete dictionary for image
denoising, the sparse prior under over-completed dictionary is
employed for image deblurring. Lou et al. [6] proposed a non-
blind image deblurring method named direct sparse deblurring
using two dictionaries to deblur the image. Cai et al. [4]
estimated the blur kernel and sharp image using the sparsity of
the blur kernel and the clear image under certain over-complete
frame systems. Yang et al. [5] proposed an approach to single-
image super-resolution (SR), based on sparse signal recovery.
They found that image patches can be well represented as a
sparse linear combination of elements from an appropriately
chosen over-complete dictionary. So they seek a sparse
representation for each patch of the low-resolution input, and
then use the coefficients of this representation to generate the
high-resolution output[5]. Haisen Li et al. [3] claimed that
super-resolution ideas in [5] contribute to solving the
deblurring problem using the sharp and blur dictionary pair,
instead of the traditional deconvolution techniques. The
advantage of this approach is that it avoids the ill-posed
deconvolution problem and bring more high-frequency
information from the dictionary pair.
In the rest of this section, we will illustrate the necessary
steps [1, 3] to use sparse representation for single image
deblurring. The input to this technique is the blurry image ,
dictionary , iteration number , and kernel size. The output is
an estimated blurring kernel and an estimated deblurred
image . The following three steps are performed times on
all patches of the blurry image: (A) updating sparse
representation coefficient, (B) updating the estimated latent
image and (C) updating the estimated blur kernel.
A. Updating the Sparse representation
The latent sharp image is sparsely represented [1,
3] as follows:
(2)
180
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Where is an over-complete dictionary with initial kernel estimation and an initial latent image
atoms and is the sparse representation coefficient. estimation , where is the number of performed iterations to
Substituting from equation 2 in equation 1 and eliminating the update Sparse coefficients, kernel estimation, and latent image
noise term, we obtain: estimation. At this stage, for simplicity, we make and
. Then the following stages are performed until the
( ) ( ) (3)
converges.
Where is the blur dictionary. The dictionary pair A. Applying Directional low-pass filter
consists of sharp dictionary and corresponding blur dictionary.
It is clear that the blurry image and the sharp image have the In this step, we apply a set of directional low-pass filter to
same coefficient under corresponding dictionary. So, the sharp the input blurry and noisy image . Lin Zhong et al [8]
image can be obtained through sparse reconstruction and by discovered that applying a directional low-pass filter to the
searching the blur image sparse coefficients on sharp input image greatly reduces the noise level, while preserving
dictionary. The sparse coefficient for each patch is updated by the blur information in the orthogonal direction to the filter.
minimizing the following equation[1,3]: First, we determine the number of directional low-pass filters
, in our experiments we make . Then we apply the
{ } (4) eighteen directional low-pass filters to the blurry and noisy

image , where the direction of each filter is determined as
Where is -th patch in the blurry image, and is the . The eighteen low-pass filters , where
sparse coefficient of the -th image patch. , are convolved with as follows:
B. Updating the estimated latent sharp image
At each iteration , the latent image is updated until the ( ) ( ) ( ) (7)
maximum number of iterations is reached. Equation 5 shows
the update process of the estimated latent image[1,3,7]:
Where is a pixel location, is the spatial distance from
(5) one pixel to , ( ) ,
{ }

is a unit vector of direction , and
Where is the first order gradient extraction filters controls the strength of the filter [8]. In our experiments, we
, . always set the extent to . The result of current step is a
set of filtered images , , , , each of which is
C. Updating the estimated blur kernel
The final stage in each iteration is to update the estimation the result of convolving with the appropriate directional
of the blur kernel[1,3]: low-pass filter , with .

(6) B. Blur kernel estimation at each filtered image

{ }
IV. ESTIMATING THE FINAL LATENT SHARP IMAGE USING
DISCRETE RADON TRANSFORM
There are several causes for the noise such as low light
conditions, slow shutter speeds and high sensitivity modes. As
a result, the acquired image always contains a considerable
amount of noise. Almost, all image deblurring techniques [9-
16] are highly sensitive to Gaussian noise because small
amount of noise may degrade the quality of latent image
estimation dramatically. Most of the state-of-the-art deblurring
techniques have a serious problem when the amount of the
noise is large enough in the blurry image. Several authors tries
to simultaneously reduce noise and deblur the image[17-21].
Lin Zhong et al [8] claimed that directly applying image
denoising methods often partially damages the blur information
and leads to biased kernel estimation. In this section, we
modified the technique in [8] to make benefit of sparse
representation for estimating the initial latent image. The input
to the algorithm is the blurry and noisy image . The output is
the blur kernel and the latent sharp image . The first step is
to apply the technique described in section II to estimate an
Each filtered image is treated as an independent blurry
and not noisy image. So, it is assumed that it was obtained by
just convolving the sharp image with the PSF. In other words,
the noise has been eliminated. The next step is to use the initial
latent sharp image (referred to as ) that was produced from the
sparse representation to estimate the blur kernel for each
filtered image.
C. Applying Discrete Radon Transform on the orthogonal
direction
The Radon transform computes the projection of an image
intensities along a radial line oriented at specific angle. The
Radon Transform [22] is an important tool in many areas such
as the reconstruction of an image from its projections (CT
scanning)[23]. The radon transform of a two dimensional
signal ( ) is a set of projections along an angle ,
:
( ) ( ) ( ) (8)

181
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Where ( ) is the Fourier transform of , ( ) is the

( ) ( ) (9) mean power spectral density of ( ), ( ) is the mean
power spectral density of the noise ( ) . Then the
Fourier transform of the estimated sharp image ( ) is
Where the -function is the Dirac delta function that
calculated as follows:
converts the two-dimensional integral into a line integral
along the line [24]. The radon ( ) ( ) ( ) (14)
transform maps Cartesian rectangular coordinates ( ) to
polar coordinates, i.e. a distance and an angle . It was Where ( ) is the Fourier transform of the observed
shown in [8] that each directional filter filters the image blurred image. Finally, an inverse Fourier transform is applied
to produce the final latent sharp image ( ). In our
only along the direction and it has no impact on the blur
notations, we shall apply the Wiener deconvolution on the
information in the orthogonal direction. So the authors of [8]
blurry and noisy image using the estimated blur kernel that
proposed to calculate radon transform of each estimated blur
was produced from inverse radon transform in the previous
kernel along the orthogonal direction to . So, we
stage. After that the latent sharp image is produced.
calculate , then compute the radon transform
( ) of each estimated blur kernel at an angle . V. EXPERIMENTAL RESULTS
D. Estimating the final kernel using Inverse Discrete Radon In this section, we performed a set of experiments to
Transform evaluate the proposed technique. The quality of the produced
sharp image is an important measure to judge the effectiveness
Inverse radon transform reconstructs the original image of the deblurring technique. There are two measures that are
from a set of projections at different angles. The inverse radon used to evaluate the quality: subjective measures and objective
transform is computed using equation 11[24]. measures[30]. Subjective measures measure the users
satisfaction or perception. There is no explicit formula to
( ) ( ) ( ) (10) calculate a numeric value for the percentage of the quality.
These metrics are very important and required because they are
directly records users requirements and comments on the
deblurred images. In the other hand, the objective metrics use
( ) ( ) (11) an explicit formulas to calculate numeric values. There are two
types of objective metrics that are widely used to evaluate
It is clear that it is an integral over all angles of all various image deblurring techniques: Peak Signal-To-Noise
projections that pass through the point ( ). As in [8], we use Ratio (PSNR) and Structural Similarity (SSIM).
the inverse radon transform to make the final estimation of the
blur kernel from the set of projections ( ), with
.
E. Estimating the final latent sharp image using Wiener
deconvolution
The Wiener filter [25-29] performs an optimal trade-off
between inverse filtering and noise smoothing because it
removes the additive noise and inverts the blurring
simultaneously[10]. The Wiener deconvolution has been
widely used for de-convolving images because the frequency
spectrum of most visual images is fairly well behaved and may
be estimated easily[29]. For the following blurring model:
b( ) ( ) ( ) ( ) (12)
Where ( ) is the unknown sharp image, ( ) is the
known blur kernel, ( ) is some unknown additive noise
independent of ( ), and b( ) is the observed blurred and
noisy image. All Wiener filter calculations are in the frequency
domain. Equation 13 calculates a set of Fourier transform
coefficients called ( ).
( ) ( )
( ) (13)
| ( )| ( ) ( )

182
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Fig. 2. Comparison between achieved PSNR among different techniques.

Fig. 1. Blurred and noisy image (Left) Deblurred image (Right)

PSNR [30] is used to measure the quality of image in dB
(decibels). PSNR between two images (deblurred image and
original image) measures how far the two images are equal.

[ ] (15)

( )
( ) (16)
Where is the original image, is the restored image, is
the number of rows, is the number of columns and is
the maximum possible intensity value of the image[10]. The Fig. 3. Comparison between achieved SSIM among different techniques.
Structural Similarity (SSIM) [30] is calculated as follows: VI. CONCLUSION
( )( )
(Eq. 23) This research proposes a novel single image deblurring
( )( ) technique based on sparse representation and radon transform.
Where and are the means of and , respectively; The Sparse representation is used to make an initial estimation
and are the variance of and , respectively; and are of the latent sharp image. Then, a set of directional filters are
used to stabilize the division with weak denominator[10]. We applied to the blurry and noisy image to reduce the noise while
have performed 30 experiment at each noise level. At each maintaining the blurry information on the orthogonal direction.
experiment, the PSNR and SSIM are recorded. The best and After that an initial kernel estimation at different angles is
worst PSNR and SSIM are summarized in table 1. The best performed using the initial latent image that was produced
obtained PSNR is 28.233 at 0.04 noise ratio, whereas the best using sparse representation. Each estimated kernel is
obtained SSIM is 0.896769 at also 0.04 noise ratio. Figure 2 transformed to radon transform and the set of projections at
and 3 compare average values of PSNR and SSIM among different angles are stored. An inverse radon transform is
different image deblurring techniques. We tested each applied to make a final kernel estimation. Finally, the Wiener
technique on ten fixed blurred and noisy images and record deconvolution is performed to estimate the final latent sharp
the average values for PSNR and SSIM. Our technique image.
achieved the highest average value for both PSNR and SSIM.
TABLE I. BEST AND WORST PSNR AND SSIM AT DIFFERENT NOISE REFERENCES
BEST WORST [1] H. Zhang, J. Yang, Y. Zhang, and T. S. Huang. Sparse representation
NOISE BEST SSIM WORST SSIM
PSNR PSNR based blind image deblurring. In IEEE In-ternational Conference on
0.01 23.12 17.066 0.815853 0.630543895 Multimedia and Expo (ICME), PP. 16, 2011.
0.02 26.141 21.045 0.841068 0.777836476 [2] M. Elad and M. Aharon. "Image denoising via sparse and redundant
representations over learned dictionaries." Image Processing, IEEE
0.03 21.055 15.215 0.812397 0.67265998 Transactions on 15, no. 12, PP. 3736-3745, 2006.
0.04 28.233 24.122 0.896769 0.731004029 [3] H. Li, Y. Zhang, H. Zhang, Y. Zhu, and J. Sun. "Blind image deblurring
based on sparse prior of dictionary pair." In Pattern Recognition (ICPR),
0.05 25.009 20.062 0.88157 0.715426885 21st International Conference on, PP. 3054-3057. IEEE, 2012.
0.06 24.162 21.002 0.826934 0.727617254 [4] J. Cai, H. Ji, C. Liu, and Z. Shen. Blind motion deblurring from a single
image using sparse approximation. In IEEE Conference on Computer
0.07 25.02 20.243 0.76632 0.691838616 Vision and Pattern Recognition, PP. 104111, 2009.
0.08 24.132 20.06 0.83431 0.79625216 [5] J. Yang, J. Wright, H. Thomas, and Y. Ma. Image super-resolution via
sparse representation. IEEE Trans. on Image Processing, 19(11), 2010.
0.09 23.063 18.25 0.79192 0.694005369
[6] Y. Lou, A. Bertozzi, and S. Soatto. Direct sparse de-blurring. Journal of
0.1 25.062 21.125 0.761593 0.684599173 Mathematical Imaging and Vision, Vol. 39, No. 1, PP. 112, 2011.

183
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
[7] K. Dilip and F. Rob. Fast image deconvolution using hyper-laplacian
priors. In Annual Conference on Neu-ral Information Processing
Systems, PP. 10331041, 2009.
[8] L. Zhong, S. Cho, D. Metaxas, S. Paris, and J. Wang. "Handling noise in
single image deblurring using directional filters." In Computer Vision
and Pattern Recognition (CVPR), PP. 612-619, 2013.
[9] L. Zhang, A. Deshpande, and X. Chen., "Denoising vs. deblurring: HDR
imaging techniques using moving cameras." In Computer Vision and
Pattern Recognition (CVPR), IEEE Conference on, PP. 522-529., 2010.
[10] I. M. El-Henawy, A. E. Amin, K. Ahmed, H. adel, "A Comparative
Study On Image Deblurring Techniques", IJACST, Vol.3, No.12, 2014
[11] R. Fergus, B. Singh, A. Hertzmann, S. T. Roweis, and W. T. Freeman.,
"Removing camera shake from a single photograph." ACM Transactions
on Graphics (TOG) 25, no. 3, PP. 787-794, 2006.
[12] Y.-W. Tai, X. Chen, S. Kim, S. J. Kim, F. Li, J. Yang, J. Yu, Y.
Matsushita, and M. S. Brown. "Nonlinear camera response functions and
image deblurring: Theoretical analysis and practice." Pattern Analysis
and Machine Intelligence, IEEE Transactions on 35, no. 10, 2013.
[13] Y. Tai and S. Lin. "Motion-aware noise filtering for deblurring of noisy
and blurry images." In Computer Vision and Pattern Recognition
(CVPR), IEEE Conference on, PP. 17-24., 2012.
[14] L. Yuan, J. Sun, L. Quan, and H. Y. Shum., "Progressive inter-scale and
intra-scale non-blind image deconvolution." In ACM Transactions on
Graphics (TOG), vol. 27, no. 3, p. 74. ACM, 2008.
[15] N. Joshi, R. Szeliski, and D. J. Kriegman., "PSF estimation using sharp
edge prediction." In Computer Vision and Pattern Recognition, CVPR.
IEEE Conference on, pp. 1-8. IEEE, 2008.
[16] S. Y. Kim, Y. W. Tai, S. J. Kim, M. S. Brown, and Y. Matsushita.,
"Nonlinear camera response functions and image deblurring." In
Computer Vision and Pattern Recognition (CVPR), PP. 25-32., 2012.
[17] A. Marquina, and S. Osher. "Explicit algorithms for a new time
dependent model based on level set motion for nonlinear deblurring and
noise removal." SIAM Journal on Scientific Computing 22, no. 2, 2000.
[18] J.-F. Cai, R. H. Chan, and M. Nikolova. "Fast two-phase image
deblurring under impulse noise." Journal of Mathematical Imaging and
Vision 36, no. 1, PP. 46-53, 2010.
[19] L. Bar, N. Sochen, and N. Kiryati. "Image deblurring in the presence of
salt-and-pepper noise." In Scale Space and PDE Methods in Computer
Vision, PP. 107-118. Springer Berlin Heidelberg, 2005.
[20] L. Bar, N. Kiryati, and N. Sochen. "Image deblurring in the presence of
impulsive noise." International Journal of Computer Vision 70, no. 3,
PP. 279-298, 2006.
[21] U. Schmidt, K. Schelten, and S. Roth. "Bayesian deblurring with
integrated noise estimation." In Computer Vision and Pattern
Recognition (CVPR), IEEE Conference on, pp. 2625-2632. 2011.
[22] G. Beylkin, "Discrete radon transform." Acoustics, Speech and Signal
Processing, IEEE Transactions on 35, no. 2, PP. 162-172, 1987.
[23] A. Averbuch, and Y. Shkolnisky. "3D Fourier based discrete Radon
transform." Applied and Computational Harmonic Analysis 15, 2003.
[24] W. H. Press, "Discrete Radon transform has an exact, fast inverse and
generalizes to operations other than sums along lines." Proceedings of
the National Academy of Sciences 103, no. 51, PP. 19249-19254, 2006.
[25] Y. Zheng, "A Fast Image Deblurring Algorithm Using the Wiener Filter
and the Hartley Transform." In Review of Progress in Quantitative
Nondestructive Evaluation, pp. 735-742. Springer US, 1989.
[26] H. Yang, Z. B. Zhang, D. Y. Wu, and H. Y. Huang. "Image deblurring
using empirical Wiener filter in the curvelet domain and joint non-local
means filter in the spatial domain." The Imaging Science Journal 62,
no. 3, PP. 178-185, 2014.
[27] P. D.Sankhe, M. Patil, and M. Margaret. "Deblurring of grayscale
images using inverse and Wiener filter." In Proceedings of the
International Conference & Workshop on Emerging Trends in
Technology, pp. 145-148. ACM, 2011.
[28] Y. C. Eldar, "Robust Deconvolution of noisy signals." In Signal
Processing Conference, 2005 13th European, pp. 1-4. IEEE, 2005.
[29] B. H. Shakibaei, and P. Jahanshahi. "Image deconvolution by means of
frequency blur invariant concept." The Scientific World Journal 2014.
[30] R. Wang, and D. Tao. "Recent Progress in Image Deblurring." arXiv
preprint arXiv:1409.6838, 2014.
[31] J. Jiaya. "Single image motion deblurring using transparency." In
Computer Vision and Pattern Recognition, 2007. CVPR'07. IEEE
Conference on, PP. 1-8. 2007.
[32] D. S. Rao, K. S. Deepthi, and K.M.S. Deep. "Application of Blind
Deconvolution Algorithm for Image Restoration." International Journal
of Engineering Science and Technology (IJEST), 2011.
[33] S. S. Al-Amri, and A. S. Ali. "Restoration and Deblured Motion Blurred
Images." International Journal of Computer Science Issues (IJCSI) 11,
no. 1, 2014.
[34] Y. Tai, P. Tan, and M. S. Brown, Richardson-Lucy deblurring for
scenes under projective motion path, IEEE Trans. Pattern Anal. Mach.
Intell., vol. 33, no. 8, pp. 16031618, Aug. 2011.
[35] S. Prasad, "Statistical-information-based performance criteria for
Richardson-Lucy image deblurring." Journals of Optical Society of
America, 19, no. 7, PP. 1286-1296, 2002.
[36] A.-P. Yang, Z.-X. Hou, and C.-Y. Wang. "Image deblurring based on
wavelet and neural network." In Wavelet Analysis and Pattern
Recognition, ICWAPR'07. , vol. 2, PP. 647-651. IEEE, 2007.
[37] N. Kumar, M. Shrivastava, and N. Rahim. "Image Deblurring Using a
Neural Network Approach", International Journal of Engineering and
Innovative Technology (IJEIT), Vol. 2, No. 3, September 2012.
[38] A. Beck, and Marc Teboulle. "A fast iterative shrinkage-thresholding
algorithm with application to wavelet-based image deblurring." In
Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE
International Conference on, PP. 693-696, 2009.
[39] J. L. Herring, "The Effects of Projection on Iterative Methods in Image
Deblurring." PhD diss., Emory University, 2010.
[40] J. G. Nagy, K. Palmer, and L. Perrone. "Iterative methods for image
deblurring: a Matlab object-oriented approach." Numerical Algorithms
36, no. 1, PP. 73-93, 2004.
[41] S. kumar, P. kumar,M. Gupta, A. K. Nagawat, Performance
Comparison of Median and Wiener Filter in Image De-noising,
International Journal of Computer Applications, Vol. 12, Issue No.-
10.5120, PP. 1664-2241, Dec. 2010.
[42] J. Wang, G. Wang, and M. Jiang. "Blind deblurring of spiral CT images
Based on ENR and Wiener filter." Journal of X-Ray Science and
Technology13, no. 1, PP. 49-60, 2005.
[43] P. Bojarczak, and Z. Lukasik. "Image DeblurringWiener Filter Versus
TSVD Approach." Advances in Electrical and Electronic Engineering 6,
no. 2, PP. 86-89, 2011.
[44] L. Xu, S. Zheng, and J. Jia. "Unnatural l0 sparse representation for
natural image deblurring." In Computer Vision and Pattern Recognition
(CVPR), IEEE Conference on, pp. 1107-1114, 2013.
[45] Y. Zhang, and Z. Yanning "Sparse representation based iterative
incremental image deblurring." In Image Processing (ICIP), 2009 16th
IEEE International Conference on, pp. 1293-1296. IEEE, 2009.
[46] M. Carlavan, and L. B.-Fraud. "Sparse Poisson noisy image
deblurring." Image Processing, IEEE Transactions on 21, no. 4, 2012.
[47] H. Zhang, D. Wipf, and Y. Zhang. "Multi-image blind deblurring using
a coupled adaptive sparse prior." In Computer Vision and Pattern
Recognition (CVPR), IEEE Conference on, pp. 1051-1058, 2013.
[48] L. Ma, J. Yu, and T. Zeng. "Sparse Representation Prior and Total
Variation--Based Image Deblurring under Impulse Noise." SIAM
Journal on Imaging Sciences 6, no. 4, PP. 2258-2284, 2013.
[49] H. Huang, and N. Xiao. "Image deblurring based on sparse model with
dictionary learning." Journal of Information and Computational
Science10, no. 1, PP. 129-137, 2013.
[50] L. Feng, Q. Huang, T. Xu, and S. Li. "Blind image deblurring based on
trained dictionary and curvelet using sparse representation." In Selected
Proceedings of the Photoelectronic Technology Committee Conferences,
PP. 95222G-95222G, 2015.
184
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
FPGA Implementation of Iris Recognition Based on
Fast DCT Coding
Ramadan Gad
Computer Science and Engineering Dept.
Faculty of Electronic Engineering, Menoufia University,
Egypt.
ramadangad@el-eng.menofia.edu.eg
Nawal El-Fishawy
Computer Science and Engineering Dept.
Faculty of Electronic Engineering, Menoufia University,
Egypt.
nelfishawy@hotmail.com
Abstract Iris is a touch-less automated real-time biometric
system for user authentication. Most iris recognition approaches
suffer from high cost, long development times, power
consumption, and computationally intensive. Extracting valuable
features from iris is computational burden, computational time,
and limited memory storage. Classification of irises is most time
consuming phase, repeated in recognition. In addition, general
purpose systems are low speed and not portable solution. In this
paper, off-line iris approach implemented on PC and hardware.
Both Fast Discrete Cosine Transform (FDCT)-based feature
extraction and Hamming Distance for matching stages are
synthesized and implemented using low-cost Xilinx Spartan-3E
FPGA chip. Simulation and implementation results indicate that
the proposed solution improved the execution time with
equivalent accuracy and error rates achieved by computer
solution. Iris recognition based on FDCT system is more reliable,
more saving in computations cost, small size, and good interclass
separation in minimum time.
Keywords Iris Recognition; Authentication; FDCT;
Hamming Distance; FPGA; Parallel Processing.
I. INTRODUCTION
Nowadays, security of computer and financial systems in
governments and organizations invested in biometric
authentication for its high security level at critical access
points. Remembering many passwords and ID cards is not
suitable; as it may be forgotten or stolen. From among of other
biometrics, iris considered more accurate, reliable, and stable.
Based on recent research, it is the lowest recognition error rate;
so it is the best choice in different potential applications need
high security [1]. Iris overview and iris system components,
benefits, drawbacks, and challenges are in details discussed in
[2].
Building such robust, more accurate, non-intrusive to
humans, and fast real-time iris recognition system, by using
only PC platform, has some insufficiency. Most iris recognition
systems are operating on a slow general purpose system.
Furthermore, sequential processing devices based systems are
not especially portable. It is not suitable for use in the
978-1-5090-0217-7/15/$31.00 2015 IEEE
M. Zorkany
Electronic and Communication Engineering Dept.
National Telecommunication Institute,
Egypt.
m_zorkany@nti.sci.eg
Ayman El-SAYED
Computer Science and Engineering Dept.
Faculty of Electronic Engineering, Menoufia University,
Egypt.
ayman.elsayed@el-eng.menofia.edu.eg
environments where high portability is required. Consequently,
some applications are sensitive to battery life, heat dissipation,
etc. In general, state of the art biometrics techniques suffer
from high cost and long development times. In addition, these
techniques have high consumption of computational power.
Therefore, the current trend is hardware/software co-design. [3,
4].
Thus, a hardware implementation of an iris recognition
system could be exceptionally faster than its general purpose
counterpart. Also, being small size enough to be part of a
digital device. Moreover, hardware/software co-design is a
suitable solution, widely used for developing specific and high
computational cost devices. This provides a means of
embedded systems [4]. One of the embedded system examples
is the Field Programmable Gate Arrays (FPGAs). FPGA
architecture has a dramatic effect on the quality of the final
device speed performance, area efficiency, and power
consumption. The re-configurability property and a good
flexibility in design, makes them ideal prototyping tools for
hardware designers. Recent advances in fabrication technology
allow FPGAs capable of performing many complex
computations in parallel while hosted by conventional
computer hardware [5].
One of the main objectives is to minimize the complexity of
operations as much as possible while maintaining low delays
and high throughputs. Also, minimize the time-to-market cost,
and simplify debugging and verification. Architectures based
on hardware/software co-design combine the advantages of
both hardware and software solutions. So, the purpose of this
paper is to implement both iris feature extraction technique
phase due to its computational power, and matching phase as it
is a repeated function and large computation time requirement,
targeting low-cost Xilinx Spartan-3E FPGA device by using
VHDL language. Because of Feature extraction for iris code
based on Discrete Cosine Transform (DCT) achieves less size
extracted normalized iris data codes, due to DCT energy
compaction characteristic, giving such less time real time
implementation, more reliable, low computational cost and
185
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
good interclass separation in minimum time; this algorithm
implemented, here, followed by the Hamming Distance (HD)
as our classifier in hardware for authentication tasks.
The rest of this research paper is organized as: section II,
illustrates the previous work done in this topic. Section III,
discusses the adopted architecture for system design explaining
iris recognition algorithms with phases, and the hardware
implementation. Finally, the experimental results and
discussion for performance evaluation are in section IV.
II. RELATED WORK
Judith Liu-Jimenez [6] implemented Gabor filter for iris
feature extraction and Hamming Distance (HD) for matching,
on Xilinx Virtex2 FPGA. Using pipelined architectures for
hardware reduced time 80% compared to software. In Raida
Hentati [6], Hough Transform and HD implemented on
CycloneII2c35 FPGA manufactured by ALTERA. High
performance, reliability, and speed achieved through iris
localization and matching phases. Judith Liu-Jimenez et al. [7]
developed different algorithms for off-line iris system. Fast
Wavelet Transform (FWT) algorithm for feature extraction and
HD for matching are both implemented on both microprocessor
ARM7TDMI and Virtex4SX35 FPGA. Results obtained from
the two platforms beside PC compared according to
performance, cost, area, processing time, and security level. In
[8, 9], Verilog HDL used to implement HD on Spartan-3AN
FPGA. Ten iris templates from MMU1 database are generated
by Matlab first, stored in text file then, transferred to ROM in
FPGA using Universal Asynchronous Receiver/Transmitter
(UART). The threshold is set to 30% for identification.
Babasaheb G. Patil et.al. [10], implemented Singular Value
Decomposition (SVD) using Jacobi method with tournament
ordering and HD for both iris feature extraction and
classification phases respectively. Virtex5xc50vxl FPGA
device used beside Matlab to implement this approach.
Execution time was 250s at clock 100 MHZ for iris image of
size 40x40, and utilization of 80% of device resources. In [11],
the whole iris system implemented using 50 MHZ clock speed
CycloneII2C35F672C6 FPGA. SoPC Builder Mega Wizard
used to create NIOS embedded processor from ALTERA. C2H
compiler and accelerators used to gain 33% enhancement in
time execution compared with software solution running in
microprocessor. A selected review for iris implementation for
recognition on hardware devices discussed in [6]. Hamming
Distance classifier only implemented in [12] and [13] using
FPGA platform. Youngkyoon Jang et.al.[14] implemented a
portable iris system using Sony VGN-UX17LP ultra-mobile
personal computer (UMPC) and a small USB iris camera.
Recognition rate and usability evaluation like user acceptance,
convenience, satisfaction and resistance was experimentally
tested. In [13], ten samples from IITD database identified at
threshold 0.3, and execution time was 4 ns. They used Verilog
HDL source file written in Integrated Software Environment
(ISE 14.2) environment. Huiyan Xu [15] developed iris
acquisition sensor based on embedded microprocessor
(Advanced RISC Machines) ARM S3C2410 with Windows
CE operating system. Iris image processing was applied on
ADSP-BF533 using Embedded Visual C++ (EVC). Recently,
Mahesh Patil et al. [16], implemented iris recognition
operations on Beagleboard-xM by using OpenCV library
functions. Circular Masking technique used in normalization,
Biorthogonal Wavelet Transform and SVD algorithms are used
to extract iris features, and iris matched by Euclidean Distance
measure. Only 4 images from CASIA ver. 1 used in
experimental tests indicating the performance of SVD then,
results of hardware and CPU-based compared.
As shown, facing the tasks of iris algorithms
implementation on hardware, designers could use several
different architecture alternatives beside FPGA to implement
the system as microprocessor, General Purpose Processor
(GPP), full-custom Application-Specific Integrated Circuits
(ASICs), and Digital Signal Processors (DSPs). In addition,
using a combination solution obtains the inherent advantages of
both systems such as: reduced time, reduced area and low
power consumption. These are commonly referred to as
System on Chip (SoC) solutions [3]. The use of FPGA in the
integrating design of iris recognition system combines the
benefits of hardware speed and re-configurability, flexibility,
and reprogramming from software advantages. In addition, its
in-system programming yields a more cost effective procedure
[5]. FPGA routinely outperforms General Purpose Processor
(GPP) when computing algorithms that have iteration level
parallelism. FPGAs have the potential for higher performance
and lower power consumption than microprocessors [17].
FPGAs compared with (ASICs), offer lower non-recurrent
engineering (NRE) costs, reduced development time, shorter
time to market, easier debugging and reduced risk. For DSPs,
the speed is limited by the clock speed of the DSPs. Re-
configurability can be achieved by changing the memory
content of its program. In contrast, re-configurability in FPGAs
can be performed by downloading reconfiguration data to the
RAM. Power consumption in a DSP depends on the number of
memory elements used regardless of the size of the executable
program. For FPGA, the power consumption depends on the
circuit design [17].
Daugman's overall system [18, 19] has an excellent
performance and accuracy. It uses a binary representation for
iris code. Moreover, this speeds the matching through HD. In
addition, it handles the rotation of iris easily. Also,
interpretation of matching as a result of statistical test of
independence. On the other hand, the system is iterative and
computationally expensive. In addition, evaluation of iris
image quality reduces to the estimation of a single or a pair of
factors such as defocus blur, motion blur, and occlusion.
Wildes [20, 21] implemented effectively gradient-based iris
segmentation by using Laplacian of Gaussian filters (Laplacian
pyramid). The overall system was more stable to noise
perturbations and encompassed eyelid detection and
localization. Also, it uses more of the available data, in
matching. So, it might be capable of finer distinctions.
However, it is less use of available data, due to binary edge
abstraction, and therefore it might be less sensitive to some
details. In addition, computations by using mathematical
calculations to compare irises is more exhaustive. In our
previous research [22], efficient and effective mixture of
Daugman and Wildes techniques described after some
modifications to implement an iris recognition system. Our
proposed system will be discussed next section.
186
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
III. OUR ADOPTED ARCHITECTU
URE
FPGA. The architecture of the proposed syystem is shown in
fig.1. Enrolling iris image, localization, normalization,
n and
enhancement is on PC platform. Then, the inpput normalized iris
template transferred from a PC to FPGA deviice via a serial port
(RS-232), stored in the device memory. Heree; the images were
converted to binary text files (.COE) using MATLAB.
M The file
was applied as vector to the hardware interrface and stored in
memory using ROM generated by using Inttellectual Property
(IP) core generator tool. Verification of thhe two iris images
then, done through HD module, by calculating the distance
between the ROM contents as vectors. After implementation
and downloading the bit file generated, the result
r of matching
decision (authorized or imposter) based on a threshold will be
given by a Light-Emitting Diode (LED). Optionally,
O the HD
value could be displayed on the device Liquuid Crystal Display
(LCD).
Fig. 1. Architecture of our proposed system
A. Iris Recognition Algorithm
Iris recognition system is divided into five phases as
follows: iris image acquisition, localization, normalization and
unwrapping, encoding, and iris matching. EachE phase will be
discussed below.
1) Iris Image Acquisition
All images tested in this paper are takenn from the Chinese
Academy of Sciences Institute of Automationn (CASIA) version
(1) iris database; this database is clearly thee most known and
widely used by the majority of researchers. TheT characteristics
of this database illustrated in [2, 22]. A random subset
database of different person's eyes is selectted for test, under
unbiased conditions.
2) Iris Localization
The segmentation stage is critical to the success of an iris
recognition system, Wildes approach [21] chosen
c to localize
the iris, but some modifications occurred. Fiirst, pupil detected
by thresholding techniques after resamplingg eye image. The
threshold value used is (200 pixel) giving best segmentation
results. Second, iris boundary detected usinng Circular Hough
Transform (CHT). Therefore, after canny eddge detection with
parameters (threshold=0.1 and sigma=1), iterations of circles
drawing in Hough matrix space done in raange values (100 -
130 pixel). In addition, we considered the sm mall area occluded
by eyelids and random scattering of eyelashees, which is darker
as pupil. It could be considered as a part of iris
i code as the iris
region is small size rich data. That means removing random
The proposed architecturre divided into two sections.
Software solution based on PC,, and hardware solution based on
scattering eyelashes will reducee the iris data signature. Finally,
iris isolation done giving pupil circle (center coordinate and
radius) and iris circle too. These parameters used in iris
normalization and unwrapping in next stage.
3) Iris Normalization and Enhancement
E
Before doing comparisonns, the different inconsistency
sources need to be fixed fiirst. The iris region must be
normalized to compensate for all states like listed in [22]. Iris
region points, in this work, are converted from Cartesian
coordinates to polar coordinattes using Rubber Sheet Model
proposed by Daugman [19].. Compensation for rotational
inconsistencies done by shiftinng the iris templates in the ()
direction until all two iris tempplates are aligned together. The
normalized iris image still hass low contrast and may has non
uniform illumination caused byy the position of light sources. In
order to obtain more welll distributed texture image;
enhancement is required. Thiss can be reached by histogram
equalization [23]. Normalizedd and enhanced normalized iris
templates are under angular resolution of 512 and radial
resolution of 46.
4) Iris Encoding
In this paper, 1D-DCT are typically used for analyzing the
human iris patterns and extraacting significant features from
them. The N-point 1D-DCT is defined
d by [24]:
, 0,1, . . , 1(1)
1 , 0
Where: 2
1, 0
Feature extraction for iris coode based on DCT achieves less
size extracted normalized iris data codes, due to DCT energy
compaction characteristic givving such less time real-time
implementation. Another advanntage, is that the former involves
only real multiplications, whicch reduces the total number of
required multiplications [24].
Real-time implementation of o the DCT operation is highly
computationally intensive. Acccordingly, the focus has been on
reducing the number of multiiplications required. Additional
design criteria has included minimizing the complexity of
control logic, memory requirem ments, power consumption and
complexity of interconnect. Thhe computational complexity can
be further reduced by replaacing the cosine form of the
transforms with a fast algorithm
m, which reduces the operations
to a short series of multiplicattions and additions [4, 24]. So,
equation (1) of DCT could be written in the form
(Y=A*X)[25], Where (A) is coefficient matrix and (X) is the
input signal vector. Fast Discrrete Cosine Transform (FDCT)
consisting of alternating cossine/sine butterfly matrices to
reorder the matrix elements to a form, which preserves a
recognizable bit reversed patterrn at every node, is set by Wang
[26]. Due to the symmetry of the
t (8 x 8) multiplication matrix,
it can be replaced by two (4x4)) x (4x4) matrices, which can be
computed in parallel when exeecuted in FPGA chip, as can the
sums and differences forming thhe vectors below [24, 25]:
187
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

modules, adder tree, and comparator as shown in fig.2. Each 8

bit received from FDCT outtput, computed by XOR then,
(2) accumulated by adders. Finallly, it compared with threshold
value. Figure 3 shows a snappshot of the proposed hardware
system under Matlab Simulinkk simulation. The output display
sink tool added to display the HD
H operator. Also, image from
file and video viewer tools added to input the test image and
(3) plot the result iris signature affter DCT block. This system is
before using HDL coder convverter tool execution. Then, this
design converted to VHDL in i Xilinx ISE environment to
Where: A=cos (1/4), B=cos (/8), C=coos (3/8), D=cos complete the implementation by b using parallel process blocks
(/16), E=cos (3/16), F=cos (5/16), and G=cos
G (7/16). The statements. In the Spartan--3E (XC3S1200E-FGG3204C)
complexity of direct 1D-DCT, expressed as a the number of FPGA, there are 28 RAM blockks (516,096 bit) available for on-
operations O, is equal to N2. The formula caan be rearranged in chip storage. The iris templaates must be stored either in
even and odd terms to obtain a FDCT, in which
w O = N log N memory on the FPGA or off ff-chip. In one instance of our
[25]. implementation, we have impllemented a ROM in VHDL by
using memories available in IP P core generator. The iris image
5) Iris Matching has size 512*46 pixels. Each pixel
p represented in 8 bits; each
In the comparison stage, a dissimillarity measure is memory has 23552 locations with w 8 bits wide. The HD value
computed between two codes to decide whhether the two iris accumulated every clock to outt the final result value after 2944
images are the same eye or not. Hamming Distance (HD), in clock pulse. Each clock acceptts 8 pixels values and computes
this work, is the metric for recognition. The hamming distance the HD of them, and then the seecond clock pulse enters another
is defined as followed [27]: 8 pixels with adding the new HD value to the previous,
increasing the total number off different binary bits. Until the
(4) accumulator equals the threshoold value, or full size achieved,
the decision signal asserted to show the final verification
decision.
Where S and T are the enrolled sam mple and template
patterns that we compared and N is the total number of bits.
The value of the HD closer to one means thhe two patterns are
different. And the closer this distance value to zero states that
the two patterns are to be identical. Thhis technique for
matching is fast because the templates vecctors are in binary
format. In addition, it is simple and suitable for
f comparisons of
millions of template in large database [199]. A threshold is
needed to differentiate between intra-classs and inter-class
comparisons. The optimum threshold in ouur system is 0.48.
However, the selection of that value is depeending on the final
application and trade-off between False Acccept Rate (FAR)
and False Reject Rate (FRR).
B. Hardware Implementation
In feature extraction module, the FDCT algorithm requires
Fig. 2. Components of matching module.
an addition butterfly and a number of 4-input Multiply
Accumulators (MACs) that can be realized with
w only one Look
Up Table (LUT) per MAC instead of 3 FP PGA LUTs. The
structure of this algorithm implemented as each vector has 8
values. Direct factorization methods usse sparse matrix
factorization. The speed gain when using thhis method comes
from the unitary matrix used to represent thee data. These direct
factorization algorithms have been custtomized to DCT
matrices and necessitate a smaller number off multiplications or
additions. The FDCT algorithm presented by Wang requires
the use of a different type of DCT in additiion to the ordinary
DCT. Since the direct implementation of butterfly network
needs 29 additions, and 13 multiplications, annd a multiplication
consumes much more time than an additionn; this will directly
decreases the delays and the area required while
w increasing the
output. The decrease of complexity and the reduction of
multipliers make hardware implementationn easier [24, 28].
Regards matching module, it consists inteernally from XOR Fig. 3. The proposed hardware systtem under Simulink simulation tool.

188
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

IV. RESULTS AND DISCUSSION achieve faster implementation thet floating point can be replace
The performance of the previously discuussed method was by fixed point, with cost of inntroducing results with rounding
tested and simulated, using MATLAB (2009a) version error. In Matlab, this converssion is called quantization, (ii)
7.8.0.347(video and image processing toolbbox, .m files, and Hardware Supported Functionns: In hardware, only a limited
Simulink) , using PC with operating system m WINDOWS XP, set of operations can be synthhesized, that is hardware is not
processor Dual-Core 1.6 GHZ/2MB Cache,, RAM 2GB. The able to realize all functions in Matlab.
M Such division and square
recognition rate equals 93.07 %. The associaated FAR and FRR root are not synthesizable [33], (iii) Pipeline and Parallel
is 0.88 % and 6.04 % respectively. Equal Errror Rate (EER) is Computing: Real-time Image processing
p applications requires
4.48% at hamming distance value 0.48. The average estimated handling a large amount of data and it poses a challenge to the
time executing only feature extraction and matching
m phases is machine vision designers. In computing, a several pipeline
1.93s. The proposed system simulated annd synthesized for stages, on the other hand, acceppt an input pixel value from the
FPGA technology, namely Xilinx Spartan-3E S using stream and output a processedd pixel value each clock cycle
ModelSimTM SE from Model Technology, version 6.4.a, for with several clock cycles of latency, equal to the number of
simulating the VHDL source code and thee ISE design suite pipeline stages, between the input and output [3, 29], (iv)
12.1. In this paper, iris feature extraction annd iris matching, a Design Methodology: Top-doown and bottom-up are two
repeatedly executed portions of an iris recognition algorithm is methodologies available. Top-down design is the preferred
parallelized on an FPGA system. methodology for chip design ass this methodology simplifies the
design task and allows moore than one engineer, when
The proposed architecture simulated on ModelSimTM SE necessary, to design the chip. Second, it allows flexibility in the
with 100 ps clock for each 1-D DCT block and HD matching design, and (v) Debugging: Duue to large volume of image data
module. It was found to be working satisfactorily.
s The and complex algorithms; it is extremely
e difficult to design test
simulated results are shown in fig. 4. The matching decision vectors that exercise all of thhe functionality of the system,
(authorized signal) asserted to binary '1' as the distance value especially when there may be complex interactions [29].
reached at the threshold, indicating that the entered irises was
different (in this test case shown in fig.4). Thhe implementation
of the hardware architecture of the iris recoggnition system was
successfully synthesized Xilinx Spartan-33E (XC3S1200E-
4FG320) FPGA device, with a working frequuency of 50 MHZ.
The device utilization summary is presented in i table 1. It shows
the resources used by the implementationn of the hardware
architecture of the system. The system synthesized and
implemented successfully with equivalent accuracy
a and error
rates achieved by computer solution implem mented with Matlab
(.m) files and Simulink. It occupies 1% of o chip CLBs and Fig. 4. Simulation of the iris hardw
ware architecture with fixed point using
achieved 58.88 s to process and take decisiion compared with Modelsim.
current software implemented which takee 1.93s. Timing
simulation report indicates 16.229 ns on-chiip total delay time TABLE I. XC3S1200E FPG
GA DEVICE UTILIZATION SUMMARY.
(11.453 ns for Logic, and 4.776 ns forr route). Table 2
summarizes the execution time for the propposed architecture Utilization
Logic utilization Used Available
(%)
compared with some previous architectures implemented with 100 17344 1
No. of slice flip flops
different algorithms. Our FPGA based iris prroposed is the least Total No. of 4input LUTs 187 17344 1
execution time. No. of bonded IOBs 35 250 14
No. of RAMB16s 24 28 85
From the reported results, we can conclude c that all No. of BUFGMUXs 1 24 4
investigated FPGA implementations can speed s up the iris
recognition system dramatically; due to the parallelism on the TABLE II. EXECUTION TIME FOR THE PROPOSED ARCHITECTURE
FPGA system when compared to a state-of-tthe-art CPU-based REVIOUS ARCHITECTURES.
COMPARED WITH SOME PR
version. However, for computationally inttensive algorithms
Architectu
ure Maximum Execution
like DCT, better results can be achieved by b coarser-grained solution
n frequency time
reconfigurable logic, like the one realized byy the Spartan-3E of Youngkyoon Jang
Xilinx. This utilization of coarse-grain reconffigurable resources Microproceessor 1.2 GHZ 986.2 ms
et.al.[14]
available in the kit, speeds the system. In paarticular, the usage Babasaheb G. Patil
FPGA 100 MHZ 250 s
of hardwired multipliers and fast carry chainns lead to a severe et.al. [10]
acceleration of the implemented computationns. Computeer 2.8 GHZ 5660 s
Judith Liu-Jimenez
Microproceessor 60 MHZ 56600 s
[7]
Issues in Hardware Implementation: Manny key differences FPGA 122.264 MHZ 295.459 s
between software and hardware mustt be thoroughly Our Proposed
Computeer 1.6 GHZ 1.93 sec
considered. Several important issues are like follows: (i) FPGA 50 MHZ 58.88 s
Floating-point and Fixed-point Number: Reports
R show that
speedup gained when implementing directd fixed-point
execution compared to emulating floatingg point [28]. To

189
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
V. CONCLUSION
To overcome the problems of obtaining real-time decision
of human iris in an accurate, robust, less complexity, reliable
and fast technique; Fast DCT algorithm is developed. As
General Purpose Systems are low speed and not portable;
FPGA-based system prototype implemented by using VHDL
language and Xilinx Integrated Software Environment (ISE
12.1) platform. Hardware systems are small enough and fast.
A PC and a hardware platforms designed and developed.
Selecting one of them depends on application requirements.
Applications with large number of users and low error rates is
important; PC solution is recommended. But, for small number
of users and execution time is critical; the dedicated hardware
solution is preferred. In this work, Fast DCT-based feature
extraction (butterfly network needs 29 additions and 13
multiplications) and Hamming Distance implemented and
simulated by ModelSimTM SE from Model Technology,
version 6.4.a tool. The proposed approach with accuracy
93.07%, implemented and synthesized on few benchmark
images from CASIA v.1, using Xilinx FPGA chip Spartan-3E
(XC3S1200E-4fg320), 50 MHZ clock frequency and occupied
1% of chip CLBs and 85% of RAMB16s. The implementation
needs 58.88 s and delay time on-chip needs 16.229 ns for
processing the input values and presenting a result. This is a
very fast implementation when compared with current Matlab
system implemented by software (feature extraction and
matching average time 1.93 sec). This prototype is suitable for
small low-cost applications.
Newest database from CASIA or other could be used in the
test in future. Real-time camera with high resolution also may
be used. This prototype, if the acquisition, segmentation, and
normalization implemented, this will achieve a full iris
recognition system, can then apply to fast ASIC devices.
Microblaze soft core processor in C language instead of using
Matlab, can be used to implement iris system on single chip.
REFERENCES
[1] E. Harinda and E. Ntagwirumugara, "Security & Privacy Implications in
the Placement of Biometric-Based ID Card for Rwanda Universities,"
Journal of Information Security, vol. 6, no. 2, pp. 93-100, 2015.
[2] R. M. Gad, N. A. El-Fishawy, and M. A. Mohamed, "An Algorithm for
Human Iris Template Matching," Menoufia Journal of Electronic
Engineering Research (MJEER), vol. 20, no. 2, pp. 215- 227, 2010.
[3] C. Johnston, K. Gribbon, and D. Bailey, "Implementing image
processing algorithms on FPGAs," in Proceedings of the Eleventh
Electronics New Zealand Conference, ENZCon04, Palmerston North,
New Zealand, 2004, pp. 118-123.
[4] M. Lpez, J. Daugman, and E. Cant, "HardwareSoftware Co-Design
of an Iris Recognition Algorithm," IET Information Security, vol. 5, no.
1, pp. 60-68, 2011.
[5] I. Kuon, R. Tessier, and J. Rose, "FPGA Architecture: Survey and
Challenges," Foundations and Trends in Electronic Design Automation,
vol. 2, no. 2, pp. 135-253, 2008.
[6] T. Singh and S. Gupta, "Hardware Implementation of Iris recognition on
FPGA: A Selected Review," International Journal of Advanced
Research in Computer Engineering & Technology (IJARCET), vol. 2,
no. 11, pp. 2813-2816, 2013.
[7] J. Liu-Jimenez, R. Sanchez-Reillo, and B. Fernandez-Saavedra, "Iris
Biometrics for Embedded Systems," IEEE Transactions on Very Large
Scale Integration (VLSI) Systems, vol. 19, no. 2, pp. 274-282, 2011.
[8] D. Bethuna, S. Tadisetty, and P. Yadav, "FPGA based Implementation of
Iris Recognition," International Journal of Emerging Technology and
Advanced Engineering, vol. 2, no. 12, pp. 614-618, 2012.
[9] D. Bethuna, "Hardware Implementation Of Iris Matching," International
Journal of Scientific Research (IJSR), vol. 1, no. 6, pp. 46-48, 2012.
[10] B. G. Patil, N. N. Mane, and S. Subbaraman, "Iris Feature Extraction and
Classification using FPGA," International Journal of Electrical and
Computer Engineering (IJECE), vol. 2, no. 2, pp. 214-222, 2011.
[11] H. Raida and M. A. YassineAoudni, "HW\SW Implementation of Iris
Recognition Algorithm in The FPGA," International Journal of
Engineering Science (IJEST), vol. 4, no. 2, pp. 805-810, 2012.
[12] P. Rajasekar, G. Sundar, P. S. Ashok, and S. Shyamaladevi,
"Reconfigurable Accelerator for Biometric Search Engine Using FPGA
and MATLAB," International Journal of Engineering Research and
Science and Technology, vol. 3, no. 2, pp. 271-275, 2014.
[13] T. p. S. Tajinder pal Singh and S. g. Sheifali gupta, "Enhancing
Performance of Iris Recognition Algorithm through Time Reduction,"
International Journal of Signal Processing, Image Processing and
Pattern Recognition, vol. 7, no. 4, pp. 57-64, 2014.
[14] Y. Jang, B. J. Kang, and K. R. Park, "A Novel Portable Iris Recognition
System and Usability Evaluation," International Journal of Control,
Automation and Systems, vol. 8, no. 1, pp. 91-98, 2010.
[15] H. Xu, "Embedded Design Research of Iris Information Acquisition
System," Sensors & Transducers (1726-5479), vol. 178, no. 9, pp. 154-
160, 2014.
[16] M. Patil and K. Raghuveer, "SVD and DWT Based Iris Recognition
Using Beagleboard-xM," International Journal of Computer Science and
Mobile Computing (IJCSMC), vol. 4, no. 4, pp. 532-540, 2015.
[17] S. M. Qasim, A. A. Telba, and A. Y. AlMazroo, "FPGA Design and
Implementation of Matrix multiplier Architectures for Image and Signal
Processing Applications," International Journal of Computer Science
and Network Security, vol. 10, no. 2, pp. 168-176, 2010.
[18] J. G. Daugman, "High Confidence Visual Recognition of Persons by A
test of Statistical Independence," IEEE Transactions on Pattern Analysis
and Machine Intelligence, vol. 15, no. 11, pp. 1148-1161, 1993.
[19] J. Daugman, "How Iris Recognition Works," IEEE Transactions on
Circuits and Systems for Video Technology, , vol. 14, no. 1, pp. 21-30,
2004.
[20] R. P. Wildes, "Iris Recognition: An Emerging Biometric Technology,"
Proceedings of the IEEE, vol. 85, no. 9, pp. 1348-1363, 1997.
[21] R. P. Wildes, J. C. Asmuth, G. L. Green, S. C. Hsu, R. J. Kolczynski, J.
R. Matey, et al., "A System for Automated iris Recognition," in
Applications of Computer Vision, 1994., Proceedings of the Second
IEEE Workshop on Applications of Computer Vision, Sarasota, FL,
1994, pp. 121-128.
[22] R. Gad, M. Mohamed, and N. El-Fishawy, "Iris Recognition Based on
Log-Gabor and Discrete Cosine Transform Coding," Journal of
Computer Science and Engineering vol. 5, no. 2, pp. 19-26, 2011.
[23] L. Ma, Y. Wang, and T. Tan, "Iris Recognition Using Circular
Symmetric Filters," in Pattern Recognition, 2002. Proceedings. 16th
International Conference on, 2002, pp. 414-417.
[24] E.-B. Hassan, A. A. El-Fattah, and W. Fakhr, "An Efficient
Implementation of The 1D DCT Using FPGA Technology," in
Engineering of Computer-Based Systems, IEEE International
Conference on the, 2004, pp. 356-356.
[25] D. Babic, "Polynomial Transform Based DCT Implementation," in
Student Contest Proc. of the 4th Electronic Circuits and Systems
Conference (ESC03), Bratislava, Slovakia, 2003, pp. 5-9.
[26] Z. Wang, "Fast Algorithms for The Discrete W Transform and for The
Discrete Fourier Transform," IEEE Transactions on Acoustics, Speech
and Signal Processing, vol. 32, no. 4, pp. 803-816, 1984.
[27] L. Masek, "Recognition of Human Iris Patterns for Biometric
Identification," MSC., Computer Science and Software Engineering,
Masters thesis, University of Western Australia, 2003.
[28] R. Uma, "FPGA Implementation of 2-D DCT for JPEG Image
Compression," International journal of advanced engineering sciences
and technologies (IJAEST), vol. 7, no. 1, pp. 001-009, 2011.
[29] B. V. Jaspinder Sidhu, Dr. H. K.Sardana, "Real Time Image Processing-
Design Issues " presented at the NCCI 2010 -National Conference on
Computational Instrumentation CSIO, Chandigarh, INDIA, 2010. pp. 93-
96.
190
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
A Novel 3D Image Fusion Approach Using F-
Transform
Saad M. Darwish , Maha M. Ghoneim
Alexandria University, Institute of Graduate Studies and Research, Department of Information Technology
163Horreya Avenue, El-Shatby, 21526 P.O. Box 832, Alexandria, Egypt.
Saad.Darwish@alex-igsr.edu.eg , maha_ghoneim@yahoo.com
Abstract- The fusion of information from sensors with
different physical characteristics enhances the understanding of
our surroundings and provides 3D view. Moreover, 3D color
images can provide geo-scientists, environment planners, mapping
experts and military officers with easy-to-understand color images
and useful height information, which make the interpretation of the
environment of the areas of interest more efficient and more
accurate. The aim of this study is to show that the F-transform
technique is a promising 3-D multisensor image fusion method that
surpasses the previous approaches based on Hermite and Wavelet
transform due to its computational simplicity. An experimental
study based on numerical and visual observations was conducted
and eventually showed that the performance of the proposed
method outperforms current non-fuzzy methods and is comparable
with wavelets methods.
Keywords- Image fusion, 3D Fusion system, F-Transform.
I. INTRODUCTION
Image fusion refers the process of combining multiple
images of a scene to obtain a single composite image. This
fused image should be more useful for human visual or
machine perception and should contain a more useful
description of the scene than provided by any of the individual
source images[*]. In the past decades data fusion has been
applied to different fields such as pattern recognition, visual
enhancement, object detection and area surveillance[1].
The purely image-based 3D reconstruction of scene
geometry, for instance via a stereo method is still a highly
challenging problem. The primary reason for this is the
notorious difficulty of finding multi-view correspondence
when visible texture is sparse or complex occlusions are
present. Although these difficulties could be partially
remedied by increasing the set of views or resolution of the
images, intrinsic problems still remain such as the random
noise incorporated by the use of Time of Flight (ToF) active
sensors, that use laser light to probe the subject [4].
In the literature, Image Fusion can be categorized
generally into two main categories: Single Sensor Image
Fusion System and Multi-Sensor Image Fusion System [1].
Multi-sensor image fusion system overcomes the limitations
of a single sensor vision system by combining the images
from these sensors to form a composite image. The most
important issue concerning image fusion is to determine how
to combine the sensor's images.
978-1-5090-0217-7/15/$31.00 2015 IEEE
In recent years, several image fusion techniques have
been proposed, they differ according to different mathematical
fields such as: statistical methods (e.g., using aggregation
operators, such as the Min Max method [1]), estimation theory
[2], fuzzy methods (see [3,4]), optimization methods (e.g.,
neural networks, genetic algorithms), and multi-scale
decomposition methods, which incorporate various
transforms, for example, discrete wavelet transforms.
Wavelets are a type of multi-resolution function
approximation that allow for the hierarchical decomposition of
a signal or an image. The wavelet transform has several
advantages over other pyramid-based transforms: It provides a
more compact presentation, separates spatial orientation in
different bands, and decorrelates interesting attributes in the
original image. The majority of applications of a fusion
scheme are interested in features within the image, not in the
actual pixels [5]. Therefore, it seems reasonable to incorporate
feature information into the fusion process. Region based
fusion schemes initially transform pre-registered images using
a wavelet transform. Regions representing image features are
then extracted from the transform coefficients. The regions are
then fused based on a simple region property such as average
activity.
Similarly to traditional transforms (Fourier and wavelet),
the F-transform (an abbreviated name for the fuzzy transform)
performs a transformation of an original universe of functions
into a universe of their skeleton models (vectors of F-
transform components) in which further computation is easier.
In this study, we show that the F-transform technique is an
efficient method for 3-D multisensor image fusion that could
overcome the various shortcomings encountered in the
previous approaches based on Hermite and Wavelet transform.
such as: Their computational complexity; Spectral content of
small objects often lost in the fused images; They are not shift
invariants, and consequently the fusion methods using DWT
lead to unstable and flickering results. Moreover, the F-
transform can be used as an effective means to enhance spatial
resolution by enhancing the edges [6]. This method is
computationally simple and can be applied in real time
applications. Experimental results show that it outperforms the
Wavelet based method and Hermite method. The rest of the
paper is organized as follows: Section II summarizes the
previous work. The proposed fusion scheme is described in
section III. Experimental results are given in section IV and
finally discussion along with conclusion remarks are outlined
in section V.
191
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

existence of a pattern. The fusion rule selects the coefficient

II. RELATED WORK with the highest dependency value. A higher value will
represent a stronger pattern.
Recently, many approaches for 3D image fusion have The goal of this paper is to show that higher quality 3D
been introduced. In [7], Qingxiong Yang et al. envision a fused image can be obtained using the F-transform technique
conference room of the future where depth sensing systems which has been proved to be a promising and efficient method
are able to capture the 3D position and pose of users, and for 3D image fusion. Unlike the Wavelet transform, which
enable users to interact with digital media and contents being uses a single mother wavelet that determines all basic
shown on immersive displays. For this purpose, fusing two functions with different shapes, the F-transform performs a
kinds of sensors is proposed in a synergistic fashion, relying transformation of an original universe of functions into a
on passive stereo in highly textured regions while using data universe of their skeleton models (vectors of F-transform
from active depth sensors in featureless regions. One key components) in which further computation is easier.
challenge which was faced is that data from the active sensors
are of a much lower resolution than those from typical RGB III. THE PROPOSED METHOD
cameras used in the passive subsystem. To address this
problem an algorithm for up-sampling a depth map at real time The main characteristic of the F-transform method is to
based on hierarchical bilateral filtering was developed for maintain an acceptable quality in the reconstructed image even
synergistic fusion, a Time of Flight (ToF) signal strength under strong compression rates. In fact, it was shown that the
image provided by the active depth sensor was used. This PSNR of the image compressed with the FTR method gives
sensor indicates the signal strength received at each sensor PSNR values close to the PSNR obtained using the standard
pixel. This allowed the computation of a ToF sensor JPEG method under high compression rates [9].
confidence map. A stereo confidence map was also computed The following block diagram summarizes the proposed
based on local image features. The two confidence maps are algorithm:
then incorporated into the cost function used to populate the
3D volume created by a plane-sweeping stereo matching
algorithm. Multiple input
Input images
images
Although ToF depth sensors can provide 3D information registration (1)...(n(
regardless of texture, they only have limited resolution and
accuracy. To find an optimal reconstruction, an integrated
multi-view sensor fusion approach was proposed in [4] that F-transform for
combines information from multiple color cameras and Registered F-Transform
multiple ToF depth sensors. First, multi-view ToF sensor images
measurements are combined to obtain a coarse but complete
model. Then, the initial model is refined by means of a
probabilistic multi view fusion framework, optimizing over an Inverse F-
Inverse F-
transform for the
energy function that aggregates ToF depth sensor information fused image transform
with multi view stereo and silhouette constraints.
The proposed scheme in [8] fuses volumes at the voxel-
level using a multi-resolution directional-oriented Hermite Multi-image
transform of the source images by means of a decision map. fusion Fusion
First a multi-resolution HT of the input volumes is applied.
Then, for each level of decomposition, the orientation of
maximum energy is detected to rotate the coefficients, so the Final output
first order rotated coefficient that has most edge information. image output
Afterwards, taking this rotated coefficient of each volume we image
apply a linear dependence test. The result of this test is then
used as a decision map to select the coefficients of the fused Fig. 1. Block diagram of the proposed method.
volume in the multi-resolution HT domain of the input
volumes. The approximation coefficients in the case of HT are The algorithm can be summarized as follows:
the zero order coefficients, which, in this case, just are 1: Read input registered images u.
averaged to generate the zero order coefficient of the fused 2: Apply F-transform F[u]H.
volume. Finally the fused volume is obtained by applying the 3: Apply Inverse F-transform to get unm(i,j).
inverse multi-resolution HT where fused coefficients in each 4: Use the simple F-transform-based fusion algorithm (SA)
level can retrieve zero order coefficients of the higher level. which is based the one-level decomposition.
The linear dependence test evaluates the voxels inside a cube 5: For i=1 to n
of dimension wswsws, if those voxels are linearly 6: For j= 1 to m
independent, then there is no relevant feature in the cube. 7: u(x,y)= unm(x, y) + eNM(x,y), (x, y) P
However, if the voxels are linearly dependent, it indicates the 8: End
9: End

192
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
10: Apply fusion operator xp
11: For i=1 to k
12: If xp=max(x)
13: x=xp
14: End
15: End
A. Input Images Registration
Image registration is the process of overlaying images (two or
more) of the same scene taken at different times, from
different viewpoints, and/or by different sensors [10]. The
registration geometrically aligns two images (the reference
and sensed images). Fig.2 illustrates the four steps of image
registration:
Feature Detection
Feature Matching
Transform Model Estimation
Image Re-sampling & Transformation
Fig. 2. Images registrations steps.
* Feature detection: Salient and distinctive objects
(closed-boundary regions, edges, contours, line intersections,
corners, etc.) are manually or, preferably, automatically
detected. For further processing, these features can be
represented by their point representatives (centers of gravity,
line endings, distinctive points), which are called control
points (CPs) in the literature.
* Feature matching: In this step, the correspondence
between the features detected in the sensed image and those
detected in the reference image is established. Various feature
descriptors and similarity measures along with spatial
relationships among the features are used for that purpose.
* Transform model estimation: The type and parameters
of the so-called mapping functions, aligning the sensed image
with the reference image, are estimated. The parameters of the
mapping functions are computed by means of the established
feature correspondence.
* Image re-sampling and transformation: The sensed
image is transformed by means of the mapping functions.
Image values in non-integer coordinates are computed by the
appropriate interpolation technique [11].
B. F-Transform
Generally speaking, the F-transform produces an image
by a linear mapping from a set of ordinary continuous/discrete
functions over a domain P onto a set of functions within a
fuzzy partition of P. We assume that the reader is familiar with
the notion of the fuzzy set and how is it represented. Below,
we explain the F-transform in more detail. The explanation
will be given for the example of a discrete function that
corresponds to the image u.
Let u be represented by the discrete function u: P R of
two Variables, where P = {(i, j) |i = 1. . . N, j = 1, . . . , M} is
an N M array of pixels, and R is the set of reals. If (i, j) P
is a pixel, then u (i, j) represents its intensity range. The F-
transform of u corresponds u to the matrix Fnm[u] of F-
transform components:
(1)
Each component F[u]kl is a local mean value of u over a
support set of the respective fuzzy set Ak Bl. The latter is an
element of a fuzzy partition of the Cartesian product of
intervals [1,N] [1,M]. Using the fact that a fuzzy partition of
a Cartesian product is the Cartesian product of fuzzy
partitions, we first introduce this notion for a single interval
and then for a Cartesian product of intervals.
Let [1,N] = {x | 1 x N} be an interval on the real line R,
n 2, a number of fuzzy sets in a fuzzy partition of [1,N], and
h = N1/ n1 the distance between nodes x1, . . . , xn [1,N],
where x1 =1, xk = x1 +(k1)h, k=1, . . . ,n.
Fuzzy sets A1, . . . ,An : [1,N][0,1] establish a h-uniform
fuzzy partition of [1,N] if the following requirements are
fulfilled:
(i) For every k = 1, . . . ,n, Ak(x) = 0 if x [1,N] \ [xk1, xk+1],
where x0 = x1, xN+1= xN ;
(ii) For every k = 1, . . . ,n, Ak is continuous on [xk1, xk+1],
where x0 = x1, xN+1 = xN ;
(iii) For every i = 1, . . . ,N, nk =1 Ak(i) = 1;
(iv) For every k = 1, . . . ,n, Ni=1 Ak(i) > 0;
(v) For every k = 2, . . . ,n 1, Ak is symmetrical with respect
to the line x = xk.
The membership functions of the respective fuzzy sets in a
fuzzy partition are called basic functions.
The example of triangular basic functions A1, . . , An, n 2 on
the interval [1, N] is given below:
(2)
Note that the shape (e.g., triangular or sinusoidal) of a
basic function in a fuzzy partition is not predetermined and
can be chosen according to additional requirements. We now
introduce two extreme fuzzy partitions of [1, N] that will be
used in the following.
Largest partition. The largest partition contains only one fuzzy
set, A1: [1,N][0,1], such that for all x [1,N], A1(x) = 1.
Finest partition. The finest partition is established by N fuzzy
sets, A1, . . . ,AN : [1,N] [0,1], such that for all k, l = 1, . . . ,N,
k l, Ak(xk) = 1 and Ak(xl) = 0.
If fuzzy sets A1, . . . ,An establish a fuzzy partition of
[1,N] and B1, . . . ,Bm do the same for [1,M], then the Cartesian
product {A1, . . . ,An} {B1, . . . ,Bm} of these fuzzy partitions
193
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
is the set of all fuzzy sets Ak Bl , k = 1, . . . ,n, l = 1, . . . ,m.
The membership function Ak Bl: [1, N] [1, M] [0,1] is
equal to the product Ak Bl of the respective membership
functions. Fuzzy sets Ak Bl , k=1, . . . ,n, l =1, . . . ,m establish
a fuzzy partition of the Cartesian product [1,N][1,M]. Let u:
P R and fuzzy sets Ak Bl , k = 1, . . . ,n, l = 1, . . . ,m,
establish a fuzzy partition of [1,N] [1,M]. The (direct) F-
transform of u (with respect to the chosen partition) is an
image of the mapping F[u]: {A1, . . , An} {B1, . . . ,Bm}R
defined by:
(3)
Where k = 1, . . . ,n, l = 1, . . . ,m.
The value F[u](Ak Bl) is called an F-transform component of
u and is denoted by F[u]kl. The components F[u]kl can be
arranged into the matrix representation as in (1) or into the
vector representation as follows:
(F[u]11, . . . , F[u]1m, . . . , F[u]n1, . . . , F[u]nm).
C. Inverse F-transform
The inverse F-transform of u is a function on P, which is
represented by the following inversion formula,
(4)
Where i = 1, . . . , N, j = 1, . . . ,M. It can be shown that the
inverse F-transform un*m approximates the original function u
on the domain P.
D. Fusion
We now proceed with a detailed description of the
simple F-transform-based image-fusion algorithm (SA). The
fusion is performed between multiple input images from
different sensors having multiple viewpoints, which results in
a 3D image output. We assume that the image u is a discrete
real function, u = u(x, y) defined on the N M array of pixels
P = {(i, j) |i = 1, . . ,N, j = 1, . . . ,M} so that u : P R.
Moreover, let fuzzy sets Ak Bl , k = 1, . . . , n, l = 1, . . . ,m,
where 0 < n N,0 < m M , establish a fuzzy partition of
[1,N] [1,M]. We begin with the following representation of u
on P:
u(x,y)=unm(x, y) + e(x, y), (5)
Where 0 < n N,0 < m M,
e(x, y)=u(x, y)unm(x, y), (x, y) P, (6)
Where unm is the inverse F-transform of u and e is the
respective residuum. If we replace e in (5) by its inverse F-
transform eNM with respect to the finest partition of [1,N]
[1,M], the above representation can then be rewritten as
follows:
u(x,y)= unm(x, y) + eNM(x,y), (x, y) P (7)
We call (7) a one-level decomposition of u. If function u
is smooth, then the error function eNM is small, and the one-
level decomposition (7) is sufficient for our fusion algorithm.
However, images generally contain various types of
degradation that disrupt their smoothness. As a result, the error
function eNM in (7) is not negligible, and the one-level
decomposition is insufficient for our purpose. In this case, we
continue with the decomposition of the error function e in (5).
We decompose e into its inverse F-transform en'm' (with respect
to a finer fuzzy partition of [1,N] [1,M] with n' : n < n_ N
and m' : m < m' M basic functions, respectively) and a new
error function e'. Thus, we obtain the second-level
decomposition of u:
u(x, y) = unm(x, y) + en'm' (x, y) + e'(x,y), (8)
e'(x,y) = e(x,y) en'm' (x, y), (x, y) P.
In our work, we use the simple F-transform-based fusion
algorithm (SA) which is based the one-level decomposition
(7). The main role in fusion algorithms is played by the so-
called fusion operator : RK R, defined as follows:
(9)
Assume that we are given K 2 input images c1, , cK
with various types of degradation. Our aim is to recognize
undistorted parts in the given images and to fuse them into one
image. In this section, we describe the algorithm for image
fusion based on the one-level decomposition. Each input
image ci, i = 1, . . . ,K, is assumed to be a discrete real function
ci = ci(x,y) defined on the N M array of pixels
P = {(x, y) | x = 1, . . . ,N, y = 1, . . . ,M}
so that ci : P R. Moreover, the set [1,N] [1,M] is assumed
to be partitioned by fuzzy sets Ak Bl,
where k = 1, . . . ,n, l = 1, . . . ,m and 0 < n N, 0 < m M.
Denote I = {1, 2, . . . ,K}.
IV. EXPERIMENTAL RESULTS
Experiments have been performed on two set of images.
The first set of natural images was given in the Figures- 3, (a)
and (b) and the second set of tree source images was given in
Figures-4 (a) and (b). The resultant fused images obtained
using the wavelet and the proposed method were given in the
figures- 3(c), (d) and 4(c), (d), respectively.
Looking at the complexity of fuzzy transform, it is
O(md2d) {d: dimension, m: data size}, that is the complexity
of the same order as in the case of wavelet transform. Finally
the complexity of the image fusion using the fuzzy transform
is O(pm2log2m) {p: number of input images}[21].
Experimental results emphasize the fact that, The F-
transform fusion is remarkably less in computational
complexity as compared to the Wavelet transform. This will
let him a good candidate in the fusion of real-time images.
In many applications, the end user of the fusion result is
a human. Thus, the human perception of the composite image
is of paramount importance and therefore, fusion results are
mostly evaluated by subjective criteria [20]. Objective
performance assessment is a difficult issue due to the variety
of different application requirements and the lack of a clearly
defined ground-truth. Mean squared error (MSE) based metrics
are widely used for these comparisons, and will be used in this
paper as an objective evaluation of the fusion algorithm in
addition to Peak Signal to Noise Ratio (PSNR) in decibels
(dB) and Maximum Error (MAXERR).
The mean square error between the input image, X, and the
fused image, Y, is the squared norm of the difference divided
by the number of elements (N) in the signal or image:
(10)
Whereas the following equation defines the PSNR:
194
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

TABLE I. Comparison between different quality measures for the composite TABLE II. Comparison between different quality measures for the composite
images in Fig. 2. images in Fig. 4.
Algorithm PSNR MSE MAXERR
Algorithm PSNR MSE MAXERR
WAVELET 21.2571 486.8267 114
WAVELET 16.21033454 1.56E+03 132
F-TRANSFORM 17.92249472 1.05E+03 210
F-TRANSFORM 10.7776322 5.44E+03 255

PSNR= (11) V. CONCLUSION

Where, B represents the bits per sample. Finally, we calculate
MAXERR which is the maximum absolute squared deviation
of the data, X, from the approximation, fused image Y.
This paper focused on the application of the F-transform
to the problem of image fusion. After a brief introduction to
the theory of F-transform, detailed description of the fusion
algorithm was given. This algorithm is based on one-level
decomposition of input images. We then proposed an
appropriate fusion operator and applied it on several types of
pictures. With the investigation of various types of images, ie.,
natural and medical, we showed that the proposed approach
can be successfully applied in cases when input images are
available as multi-sensor input images. Both subjective and
objective results show that the proposed scheme outperforms
other methods based on the wavelet transform. The F-
transform has been proved to be an efficient model for the
representation of signals.

REFERENCES

[1] J. Dong, D. Zhuang, Y. Huang and J. Fu, "Advances in Multi-Sensor

Fig. 3. (a) left input robin image, (b) right input robin image, (c) fused image
using wavelet (d) fused image using F-transform
Fig. 4: (a) left input tree image, (b) right input tree image, (c) fused image
using wavelet (d) fused image using F-transform
Data Fusion: Algorithms and Applications," Sensors 2009, 9(10), pp.
7771-7784, Switzerland, 2009.
[2] T. Zaveri, and M. Zaveri, "A Novel Region Based Multimodality Image
Fusion Method," Journal of Pattern Recognition Research, Vol.6,No.2,
pp. 140-153, 2011.
[3] T. Zaveri, and M. Zaveri," A Novel Two Step Region Based Multifocus
Image Fusion Method," International Journal of Computer and Electrical
Engineering, Vol. 2, No. 1, pp. 1793-8163, February 2010.
[4] Y. Kim, Ch. Theobalt, J. Diebel, J. Kosecka, B. Miscusik, and S. Thrun,
"Multi-View Image and ToF Sensor Fusion for Dense 3D
Reconstruction," George Mason University, Austrian Institute of
Technology GmbH, 2011.
[5] A. Malviya and S. G. Bhirud, "Image Fusion of Digital Images,"
International Journal of Recent Trends in Engineering, Vol. 2, No. 3,
November 2009.
[6] T. Zaveri, I. Makwana, and M. Zaveri, "Novel Hybrid Multispectral
Image Fusion Method Using Fuzzy Logic," International Journal of
Computer Information Systems and Industrial Management Applications
(IJCISIM), Vol. 2, pp. 96-103, 2010.
[7] Q. Yang, K. Han, T. B. Culbertsony and J. Apostolopoulos, "Fusion of
Active and Passive Sensors for Fast 3D Capture," IEEE International
Workshop on Multimedia Signal Processing, Saint-Malo, France, pp.
69-74, October 2010.
[8] S. C.Techica, B. E.Ramrez, and E. M. Albor, "Medical Image Fusion in
3-D by the Multiresolution Directional-Oriented Hermite Transform,"
Technical Report, Department of Signal Processing, Faculty of
Engineering, Mexico University, D.F., Mexico.
[9] F. Di Martino, V. Loia, S. Sessa, "Direct and Inverse Fuzzy Transforms
for Coding/Decoding Color images in YUV Spaces," Journal of
Uncertain Systems, Vol.3, No.1, pp.11-30, 2009.
[10] M. V. Wyawahare, P. M. Patil, and H. K. Abhyankar, "Image
Registration Techniques: An overview," International Journal of Signal
Processing, Image Processing and Pattern Recognition Vol. 2, No.3, pp.
11-28, Australia, September 2009.
[11] B. Zitova, and J. Flusser, "Image Registration Methods: A Survey,"
Image and Vision Computing,Vol. 21, pp. 9771000, 2003.

195
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

[12] I.Perfilieva, I. Kreinovich, "Probabilistic Interpretation of Fuzzy

Transforms and Fuzzy Control," Departmental Technical Reports (CS),
Paper 44, 2009.
[13] I. Perfilieva, V. Pavliska, M. Vajgl, and B. De Baets, "Advanced Image
Compression on the Basis of Fuzzy Transforms," Proceedings of
Information Processing and Management of Uncertainty in Knowledge-
Based systems, pp.1167-1174, Malaga, Spain, June 2008.
[14] M. Stepnicka, "Fuzzy Transform for Practical Problems," Institute for
Research and Applications of Fuzzy Modeling, Research report No. 114,
Czech Republic, 2007.
[15] T. Wan, N. Canagarajah, and A. Achim, "Compressive Image Fusion,"
IEEE International Conference of Image Processing, pp 1308-1311, San
Diego, California,USA,October 2008.
[16] A.Das, and K. Revathy, "A Comparative Analysis of Image Fusion
Techniques for Remote Sensed Images," Proceeding of the World
Congress on Engineering, Vol. IWCE 2007, London, July 2007.
[17] L. Grammatikopoulos, I. Kalisperakis, G. Karras, T. Kokkinos, and E.
Petsa, "Automatic Multi-Image Photo-Texturing of 3D Surface Models
Obtained With Laser Scanning," CIPA International Workshop on
Vision Techniques Applied to the Rehabilitation of City Centers,
Lisbon,Portugal, pp. 25-27, October 2004.
[18] I.Perfilieva,M.Dankova,P.Hodakova and M.Vajgl, "F-Transform Based
Image Fusion," Image Fusion, Osama Okimura(Ed.), ISBN: 978-953-
307-679-9, DOI: 10.5772/14504, InTech., January 2011.
[19] P. Vlanek, "Generating Suitable Basic Functions Used in Image
Reconstruction by F-Transform," Advances in Fuzzy Systems, Vol.
2013, Article ID 593694, 6 pages.
[20] R. Maruthi," Spatial Domain Method for Fusing Multi-Focus Images
using Measure of Fuzziness ",International Journal of Computer
Applications, volume 20 No.7, April 2011, pp.0975 8887.
[21] M. Dankova, R. Valasek, "Full fuzzy transform and the problem of
image fusion ", Journal of Electrical Engineering (2006), Vol. 57, Iss.
12.

196
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
An Elliptical Basis Function Network for Non-blind Image
Deconvolution
Mohammad Hashem*1
*Ain Shams University
Faculty of Computers and Information Sciences
Cairo, Egypt
1 2
mhashem100@yahoo.com
AbstractBiomedical imaging deblurring attempts to recover
original human organ boundaries degraded by an imaging
system due to low intensity signals and blur like defocus or
motion blur that are frequently occurred. In this paper, we aim
to deliver an improved procedure in deblurring images that are
degraded, particularly CT medical images. Our work relies on
probabilistic image patches prior Expected patch log likelihood
(EPLL). The challenge is to find suitable prior in the presence of
blur and noise. We propose a framework based on Gaussian
Mixture prior trained by elliptical basis function (EBF) network
to restore CT images. An expectation-maximization (EM)
algorithm is used to estimate the basis functions. Extensive
experimental results indicate in terms of visual quality, PSNR
values that the EM-based EBF network outperforms the original
EPLL-GMM model. The performance of the proposed method is
assessed by experimental results pertaining to restoration of
blurred images.
Keywords Image deblurring, Gaussian mixture model,
Elliptical basis function, expectation maximization
I. INTRODUCTION
Medical images such as computed tomography (CT) are
often deteriorated by noise or blurring. Images gathered from
imaging system usually possess defective resolution due to
losing information during the acquisition process and other
technical reasons [1]. Patient motion, cardiac and respiratory
can cause misregistration artifacts, such as blurring, double
images and long range streaks which also appear as shading in
the reconstructed image. The most widely kind of breathing
artifact brings about curvilinear cold regions when CT scans
are acquired while patient full inspiration. Quicker scanners
decrease motion artifact, as throughout the acquisition the
patient has less time to move. If the patient is able to hold their
breath during scanning it will help respiratory motion
forestalling [2]. There are three specific effects of blurring in
medical imaging: Reduced visibility of details, image
unshrpness and reduced spatial resolution. Blurring reduces
the visibility of details such as small objects and structures in
the medical images as it spreads small components in the
image into the surrounding background area which reduces the
contrast of small objects and image features [2]. Image
deblurring is a vital phase in recovering back the undegraded
form of the images from different types of degradations such
as noise, blur and contrast imperfections. Image deblurring is
the process of granting the filtered image with sharpened view
by decreasing the blur amount in the meanwhile enhancing the
978-1-5090-0217-7/15/$31.00 2015 IEEE
Essam A. Rashed #2 , Fatma Harby Mohamed #3
#
Suez Canal University
Faculty of Science, Department of Mathematics
Ismailia, Egypt
erashed@science.suez.edu.eg ,3 fatma_harby@yahoo.com
overall image quality. In this paper, we consider the problem
of non-blind deconvolution with the help of image prior
learned over small image patches. In non-blind deconvolution,
the true image is estimated by utilizing the degraded image
and well-known Point Spread function (PSF). Restoring image
details is still hard although the point spread function (PSF) is
known [4]. From a clustering perspective any image cannot be
adequately modeled by a single cluster Gaussian model but,
they can often be accurately expressed as mixture density
distribution that can be modeled via a Gaussian Mixture
Model (GMM). So we extend the EPLL/GMM framework
from Zoran and Weiss [5] ((Expected Patch Log Likelihood
(EPLL) framework using Gaussian Mixture Model (GMM))
which model image patches via a simple Gaussian Mixture
Model (GMM). In particular, (GMM) prior learned from DC-
removed patches turns out to be extremely powerful for
removing noise and blur. Deblurred results often contain
artifacts, such as broken lines and painterly structure details.
Challenge is to find a good learning algorithm to estimate the
GMM parameters which play a central role in the deblurring
process [6].
In this study, an elliptical basis function network (EBFN)
is introduced as density estimator and used as a new
nonparametric method of estimating probability density
functions for training data. Generally the EBFNs with full
covariance matrices are an extension of the RBF networks.
The EBFNs do not require a large number of function centers
as RBFNs to model data involve clusters with complicated
shape, so it helps distribution of complex data without the
commitment for a large number of basis functions. The basis
functions are estimated using the expectation-maximization
(EM) algorithm. EBF basis functions assessment is equivalent
to mixture density models estimation; therefore mixture
distributions can be represented by the hidden centers with no
need to a large number of basis functions. Finally, the
adaptation process is based on obtaining the minimum mean
square error using least mean square (LMS) algorithm that
adjusts the output weights [7].
In section II, the degradation model is introduced, and
then EPLL framework is represented. In section III, EBF
network, which is extended on the RBF networks, is described
and the framework of the EM-based EBF network is
subsequently outlined. The EM algorithm is then employed to
estimate the Gaussian parameters of the EBF network is
197
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
introduced in section IV. Finally in section V, the EM-based
EBF network is applied to deblurring framework and its
performance is evaluated and compared with common
conventional EPLL-GMM classifier. The paper is then
concluded with a summary and outlook for further research.
II. IMAGE DEGRADATION/DECOVOLUTION MODEL
Image restoration aims to enhance the corrupted image
quality by recovering an original and sharp image from its
distortions observation. In degradation model, the original
image is blurred using degradation function and additive
noise. Image degradation is commonly modeled as:
where y, x are n are vectors representing an observed blurry
image, its latent image to be recovered, and noise respectively.
For image deconvolution, A is a degradation matrix with a
known kernel and denotes the mathematical operation of
convolution. We follow the EPLL framework of [5], [9] to
estimate the restored image using Maximum A Posteriori
(MAP) estimation, with a Gaussian likelihood function and
Gaussian noise:
where is a parameter set to be related to the standard
deviation of the noise variance .
Using GMM based patch prior proposed by Zoran and
Weiss [5], [9], the log likelihood of a given patch is defined as:
(3)
here are mixing weights for each mixture component
satisfy , are the mean and covariance of
the Gaussians respectively, and k is the number of Gaussian
mixture components.
The parameter set of GMM is learned
from a large collection of natural image patches.
It is hard to solve (2) directly, a common strategy is
introduced for efficient optimization using auxiliary variables
as done in [5]. By using "Half Quadratic Splitting " and
reformulating (2), the cost function to be minimized using
patch prior p is:
(4)
The choice of is setting to be such that is the
estimated amount of noise in the image .
The optimization of (4) is carried out in an iterative
manner as follows:
Solve x keeping { } constant.
(5)
Solve for { } given x. (using MAP).
For each iteration, value of is kept constant.
We follow the approximation procedure from [5] to
obtain a MAP estimate which is a Wiener filter solution:
= (6)
where is the Gaussian component with the highest
conditional weight.
III. ESTIMATION MIXTURE DENSITY DISTRIBUTION
PARAMETERS
Clearly image priors play a central role in the deblurring
process. Gaussian mixture prior learned from small image
patches with their DC removed will be presented for image
deblurring. The most common form for the mixture density is
the radial basis functions (RBFs) or generally elliptical basis
functions (EBFs). One of the most ways to describe radial
basis functions is to interpret them as mixtures of Gaussians
and deal with the basis functions as components of a mixture
density model [10]. By describing RBFs as mixtures of
Gaussians many statistical tools applications as the (EM)
algorithm can be utilized for parameter estimation. The feature
space density with a GMM using the EM algorithm will be
modeled to obtain the RBF centers position. Once the center
positions have been selected, the spread parameters can be
estimated from the covariance matrices of the corresponding
Gaussian components [11]. Briefly, the EBF network
overcomes the RBF network disadvantage as it requires a
large number of function centers to model data with
complicated shape clusters by incorporating full covariance
matrices into the basis functions of a RBF network. By
extending RBF structure into EBF network, there is no need to
use a large number of basis functions to identify complex
distributions of mixture densities. At final, the spread of each
component should correspond to a definite category and the
overall distributions of the feature space will be expressed as a
finite mixture with ellipsoidal spreads [12].
An elliptical basis function (EBF) network, with the basis
function [11] is defined as:
(6)
where is the pth input vector and are the mean vector
and covariance matrix of the jth basis function respectively.
Smoothing parameter which controls the spread of the jth
basis function can be determined as:
[11] (7)
Such that is the kth nearest neighbour of in the
Euclidean sense. The weight between unit i in the hidden
layer and unit j in the output layer in EBF network can be
iteratively adjusted as follow until convergence:
(8)
where is the weight from unit i in the hidden layer to
unit j in the output layer at time t (or the tth iteration) and
is the weight adjustment at the current step.
The weight adjustment may be computed by the delta
rule:
where learning is rate and is an error term for node j. If
node j is an output node then:
(9)
where is the desired output of node j and is the actual
output.
198
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
IV.EXPECTATION-MAXIMIZATION ALGORITHM
The (EM) algorithm is a computationally efficient
algorithm for solving parameter estimation problems for the
GMM and supervised or unsupervised neural networks
training [11],[13]. Assume Gaussian mixture parameters:
} denote respectively the prior distribution, mean
vector, a full rank covariance matrix of the i-th component
density.
The GMM's output is given by:
(10)
The maximum likelihood estimates of the mean vectors
and covariance matrices of a Gaussian mixture distribution in
the training phase are estimated by utilizing the EM algorithm.
The iterative EM algorithm is utilized for estimating the
parameters of the component densities to make the training
process more stable and efficient as follows:
(a) E-Step
(11)
(12)
Where is the posterior probability of the jth data point
belonging to the ith component at tth step.
(b) M-Step
(13)
(14)
(15)
By the EM-based EBF network, hidden centers represented
with elliptical spreads of probability distribution functions can
be expressed as mixture density distribution of training
patches [10],[11],[13].
V.EXPERIMENTS
A. Image quality measurements
It is an important task to faithfully evaluate the quality of
images degraded with the proposed model. The simplest and
most extensively used full-reference quality metric is peak
signal-to-noise ratio (PSNR) and mean square error (MSE).
The PSNR has been utilized to calculate similarity between
the original image and the restored image. The higher the
PSNR and lower the MSE indicate better quality in the
deblurred image.
To measure the performances of proposed method we use
two tests. First, the restored images are compared to original
image using MSE. The MSE averaging the squared intensity
differences of distorted and native image pixels and it is
given as:
(16)
where x is the original reference image, y is the restored
image, and N is the number of pixels. The PSNR is also
measured via the mean squared error (MSE) as:
(17)
Since, MAXI is the maximum possible pixel value within the
image.
B. Dataset and GMM prior learning
The GMM parameters are learned using patch data
collected from CT image dataset [14] with their DC removed.
Subsequently we adopt a-100 mixture component GMM as the
prior for both our proposed approach and predefined EBLL
framework. A set of 2 x patches is trained using EM-based
EBF network with zero mean and full covariance matrices. In
all experiments, a tuning parameter related to the noise
variance where N is the number of pixels in each
patch. For the GMM prior, the values for were set to 50*[1,
2, 4, 8, 16, 32, 64]. Each model was trained on 8x8 pixels of
non-overlapped patches with DC component of all patches is
removed.
C. Evaluation on Image Deblurring
Image deconvolution yields an indirect noise reduction, a
resolution improvement and a contrast enhancement. To
qualify and compare these benefits, we convolve 50 test
images from [14] with blur kernels, and then a white Gaussian
noise with standard variance 0.01 is added. In Table I, we
show quantitative comparison based on PSNR and PCC
(Person Correlation Coefficient) [15]. The PCC is a well-
established measure of correlation and it is broadly used in
statistical analysis, pattern recognition and image processing,
it is described as:
(18)
where is the intensity of the ith pixel in the original image,
is the intensity of the ith pixel in the restored image, is
the mean intensity of the original image and is the mean
intensity of the restored image. In the linear training phase
from the hidden layer to the output layer, the training rate is
kept to 0.01. The system is implemented, and experiments
were conducted with 15 15 kernel. The time for the training
phase of the EBF network is about few seconds. Figure 1
shows a comparison between the results from proposed
approach and original EPLL. Results are superior both in
PSNR and Structural Similarity Index Measure (SSIM). The
SSIM estimates the visual impact of shifts in image
luminance, changes in photograph contrast, as well as any
other remaining errors by comparing local patterns of pixel
intensities that have been normalized for luminance and
contrast. The SSIM is computed locally within a sliding
window that moves pixel by pixel across the image. The
measure between two windows x and y of common size NN
is:
199
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

With is the average of window x, is the average of VI. CONCLUSION

window y, is the variance of the window x, is the In this work, we have developed a non-blind
variance of the window y, is the covariance of x and y deconvolution framework relying on patch-based priors. The
, , , L is the dynamic range of the proposed technique combines a probabilistic image patch
images. By default: L = 255, and . priors, Expected patch log likelihood (EPLL) which achieves
a high performance on image deblurring and elliptical basis
TABLE I function (EBF) network based expectation maximization (EM)
DEBLURRING EXPERIMENTS: SHOWING PSNR VALUES IN DB AND
PCC OF THE RESTORED IMAGES BLURRED WITH 15x15 MOTION algorithm to restore the given image which is degraded by
KERNELS AND NOISE LEVEL = 0.01. motion blur and additive noise. Experimental results indicate
that the EM-based EBF network performs training in a faster
Original EPLL EPLL based EBF manner, more precise, and simpler in structure. The analysis is
PSNR PCC PSNR PCC
Patient (1) done on the basis of various performance metrics like PSNR
27.12 0.9961 27.91 0.9992 and SSIM, we obtain results that are overhead those in [5].
Patient (2)
32.92 0.9976 38.91 .9996
REFERENCES
Patient (3)
31.70 0.9979 38.34 0.9996 [1] B. De Man, J. Nuyts, P. Dupont, G. Marchal, and P. Suetens, Metal
streak artifacts in x-ray computed tomography: a simulation study, In
Patient (4)
27.51 0.9972 32.3 0.9994 IEEE Trans. On Nuclear Science, 1999, vol. 46, pp. 691696.
[2] J. Hsieh, Computed tomography: principles, design, artifacts, and
Patient (5)
36.09 0.9980 39.51 0.9995 recent advances , SPIE and John Wiley and Sons, 2003.
[3] D. Kundur and D. Hatzinakos, Blind image deconvolution, IEEE
Mag.Signal Process.13,4364, 1996.
(b) Blurred [4] S. Cho, Y. Matsushita, and S. Lee. Removing non-uniform motion
(a) Original
PSNR= 22.96 , SIM=0.85 blur from images. In Proc. ICCV, pp. 18, 2007.
[5] D. Zoran and Y. Weiss, From learning models of natural image
patches to whole image restoration, In Proceedings of IEEE
International Conference on Computer Vision, pp. 479-486, 2011.
[6] M. Elad and M. Aharon. Image denoising via sparse and redundant
representations over learned dictionaries . IEEE Trans. on Image
Process., 15(12):37363745, 2006.
[7] S. Gutjahr and J. Feist: Elliptical basis function networks for
classification tasks . In International Conference on Artificial
Neural networks .Lecture Notes in Computer Science , vol .1327, pp.
373-378,Springer Verlag,Berlin,1997
[8] S.C. Mariana. Almeida and Luis B. Almeida., Blind and Semi-
Blind Deblurring of Natural Images, IEEE Transactions on Image
Processing, Vol 19, pp.36-52, No. 1, January 2010.
[9] J. Z. Feng, L. Song, X. K. Yang and W. J. Zhang, Image restoration
(c) Deblurring with original EPLL (d) Deblurring with proposed EPLL via efficient Gaussian mixture model learning,In Proceedings of
PSNR=27.85,SSIM=0.87 based EBF, PSNR=35.71, SSIM=0.96 IEEE International Conference on Image Processing, pp. 1056-
1060, 2013.
[10] C. E. Thomaz, R.Q. Feitosa, A. Veiga, Design of Radial Basis
Function Network as Classifier in Face Recognition Using
Eigenfaces,Neural Networks, Proceedings. Vth Brazilian
Symposium on, pp. 118 123, 9-11 Dec. 1998.
[11] J.-C. Luo , Q.-X. Chen , J. Zheng , Y. L. Lreis , Y. Leung and J.-H.
Ma "An elliptical basis function network for classification of
remote-sensing images", Proc. IEEE Int. Geosci. Remote Sens.
Symp., vol. 6, pp.3489-3494,2003.
[12] S. S. Salankar and B. M. Patre "RBF neural network based model as
an optimal classifier for the classification of radar returns from the
Fig.1 : Visual comparison between deblurring with original EPLL- ionosphere", Proc. IEEE Int. Conf. Ind. Techno., pp.2043-2048
GMM and proposed method using PSNR and SSIM as image 2006 .
quality measurement. [13] M.-W. Mak and S.-Y. Kung "Estimation of elliptical basis function
parameters by the EM algorithm with application to speaker
Fig.1 Visual comparison between deblurring with original EPLL-GMM and
verification", IEEE Trans. Neural Netw, vol. 11, no. 4, pp.961-
proposed method using PSNR and SSIM as image quality
969, 2000 .
measurement.
[14] Chest Screeing CT dataset, JAMIT medical image database the
Visual results show that proposed method is especially Japanese Society of Medical Imaging Technology
(http://www.jamit.jp/cadcommitte/caddbinfo).
good at preserving details concurrently allowing for very [15] A. Miranda Neto, A. Correa Victorino, et al, Image processing
smooth regions, which visually shows artifacts in case of the using Pearson's correlation coefficient: Applications on autonomous
original approach. The PSNR and SSIM values obtained for robotics, 2013.
three CT images blurred with uniform motion blur for various
parameter values are shown in figs 2-4.

200
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

(a)Original (b) Blurred PSNR=16.37db , (C) Deblurring with original EPLL (d) Deblurring with proposed
SSIM=0.5117 PSNR= 34.17db, SSIM=0.9021 EPLL based EBF
PSNR=38.25db , SSIM=0.9598

Fig.2 Deblurring example: PSNR values in dB of the blurred and restored images blurred with motion kernel of size 9 9 and =0.01.

Fig.2 : Deblurring example PSNR values in dB of the blurred and restored images blurred with motion kernel of size
9x9 and =0.01

(b) Blurred (d) Deblurring with proposed

(a) Original (C) Deblurring with original EPLL
PSNR=17.15db, SSIM=0.6005 EPLL based EBF
PSNR= 27.51db, SSIM=0.8853
PSNR=32.33db, SSIM=0.9569

Fig.3 Deblurring example: PSNR values in dB of the blurred and restored images blurred with motion kernel of size 9 9 and =0.001.

(b) Blurred (d) Deblurring with proposed

(a) Original (C) Deblurring with original EPLL
PSNR=20.11db, SSIM=0.7225 EPLL based EBF
PSNR= 22.95db, SSIM=0.7132
PSNR=33.98db , SSIM=0.9991

Fig.4 Deblurring example: PSNR values in dB of the blurred and restored images blurred with motion kernel of size 5 5 and =0.001.

201
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Multiclass Object Recognition Using Object-based
and Local Image Features Extractors
Menna Maged Kamel Mohamed Waleed Fakhr
College of Computing and College of Computing and
Information Technology Information Technology
Arab Academy for Arab Academy of Science
Science and Technology and Technology
Cairo, Egypt Cairo, Egypt
mennakamel@aast.edu waleedf@aast.edu
Abstract- This paper examines a combined feature extraction
method for visual object recognition. The method is based on
applying Bag of words (BoW) using the object-based Zernike
Moment (ZM) shape descriptor and the Speeded up Robust
Features (SURF) local descriptor on the detected objects in an
image. Support Vector Machine (SVM) classifier is trained on the
extracted features using a one versus all method. The
experiments are tested on two benchmark data sets for object
recognition like COREL 1000, and Caltech 101. Using this
combined feature extraction method, the derived results
outperform the other published methods applied on the same
databases.
Keywords Object Recognition; Speeded up Robust Features;
Bag of Words; Zernike moment
I. INTRODUCTION
The field of visual object recognition has been growing in
the recent years and became a challenging problem; it didnt
reach the human level performance yet. Image recognition and
classification is an important research area for manipulating
large databases. As the features of an image have strong
relationship with its semantic meaning; object recognition
systems seeks to recognize the content of images automatically
using descriptors to classify the objects in an image. Some
approaches rely on the local features of an image and dont
capture the global features in it, although, shape is important
and gives a powerful clue to identify an object.
Recent approaches, like speeded up robust features
(SURF), use local descriptors to extract local information from
an image [1]. They tend to detect the local interest areas and
represent them in numeric vectors to quantify the image
invariant descriptors.
Other studies use complex moments as a global feature
extractor. An effective shape descriptor is the Zernike moment
(ZM) which is based on a set of orthogonal complex moments.
It was first introduced to image analysis by Teague who
constructed rotation invariants by ZM [2].
ZMs represent the image by a set of descriptors with a
minimal amount of information redundancy. As well, ZMs are
978-1-5090-0217-7/15/$31.00 2015 IEEE
H. I. Saleh M. B. Abdelhalim
Radiation Engineering College of Computing and
Department Information Technology
Atomic Energy Authority Arab Academy of Science
Cairo, Egypt and Technology
Cairo, Egypt
h_i_saleh@hotmail.com
mbakr@ieee.org
rotation and scale invariant so they can deal with the shape
problems in images.
Practically, neither global nor local features are enough
alone to recognize scene images. As global ZM works on the
whole scenery image which is poor as the image contains
multiple objects in it and extracting features for the whole
image wont represent them. As well, local SURF descriptor
ignores the object shape which might give a good clue to
identify it.
This paper presents a combined approach of local and
object-based feature extractors. The approach is composed of
the following steps;
First, all objects in an image are annotated and detected. So
an image is treated as a set of objects. Second, to extract
object-based feature for each image, ZM features are calculated
for the detected objects. Third, interest points detection and
features extraction; local interest points are detected from each
object in an image and features are calculated accordingly
using the SURF technique.
For each of the feature extraction techniques in second and
third steps; a dictionary of visual words is built from a set of
extracted random patches. The key points of these patches are
clustered using k-means to form the visual word dictionary. By
mapping the key points to the visual words each image is then
represented by a bag of words (BoW). A visual word vector is
then constructed with the frequency of the presence or absence
of each visual word in an image.
Finally, the resulted visual word vectors of the ZM
extractor and the SURF extractor are concatenated. Images are
then classified using non-linear support vector machine (SVM)
classifier.
The method was tested on popular benchmark data sets.
Experimental results obtained on the Caltech-101 and COREL
1000 datasets, presented later in this paper, achieve accuracies
that are superior to the best published results to date on the
same databases.
202
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
After discussing the background and related work in the points of an image from the prominent features of its linear
next section, section III explains the proposed approach to space, and then builds local features based on the image
extract features from an image. Experiments and results are intensity distribution.
presented in section IV. Finally, conclusion and future research
In SURF, the Hessian matrix approximation is used in the
are discussed in section V.
interest point detection.
II. BACKGROUND AND RELATED WORK Given a point X = (x, y) in an image U, the Hessian matrix
A. Background H(X, ) in X at scale is defined as follows:
1) Zernike Moments
L xx (X, ) L xy (X, )
Zernike moments consist of a set of complex polynomials H(X, ) =
that form a complete orthogonal set over the interior of the unit L xy (X, ) L yy (X, )
circle [4]. The computation of ZMs from an input image
consists of three steps; computation of radial polynomials, Where Lxx (X, ), is the convolution of the Gaussian
computation of Zernike basis functions, and computation of 2
second order derivative g () 2 with the image U in point x,
ZMs by projecting the image on to the basis functions. x
and similarly for Lxy (X, ) and Lyy (X, ).
First the Zernike radial polynomials are computed. The
real-valued 1-D radial polynomial Rnm() is defined In interest point description, a distribution of intensity
as: content within the interest point neighborhood is extracted. It is
built using the distribution of first order Haar wavelet response
Rnm () = s = 0
(n - |m|) / 2 n-2s
c (n, m, s) (1) in x and y direction.
B. Related Work
n-s !
Where, c (n,m,s) =(1) s 1) Local Descriptors
s!((n + |m|)/2 - s)!((n - |m|)/2 - s)!

n is the non-negative integer that represents the order, and Many different local extraction approaches for object
m is an integer represents the repetition satisfying n |m| = recognition have been introduced in the literature. Hu [3] and
(even) and |m| n. Vogel [4] use regular grid of the local regions to extract
patches from an image.
Using the radial polynomial, complex-valued 2-D
Zernike basis functions, which are defined within a Other like Maree [5] uses random sampling and
unit circle, are formed by: segmentation methods; then images are classified using
randomly extracted sub-windows. Recent popular approaches
Vnm (, ) = Rnm () exp (j m ) (2)
use local features to detect key points in an image. In 1999,
Where, | | 1, j = 1 Lowe and David G. [6] introduced the well-known scale
invariant feature transform (SIFT) descriptor for object
Zernike basis functions are orthogonal and imply no recognition. In SIFT; a 128 dimensional feature vector is
redundancy or overlap of information between the moments obtained from a grid of histograms of oriented gradients with
with different orders and repetitions. This property enables the automatic scale selection.
contribution of each moment to be unique and independent of
Later in 2006, Bay, Herbert, Tuytelaars, and Van Gool [7]
the information in an image.
proposed the SURF, which is a much faster scale and rotation
Complex Zernike moments of order n with repetition m invariant interest point descriptor. Like SIFT, both approaches
are finally defined as; produce hundreds of interest points per image. Each of these
points is characterized by a dictionary of visual words. An
n+1 2 1
image is then represented by a bag of words (BoW). This
Znm = (,)V*nm (,) d d (3) representation is then quantized and represented in a feature
0 0
vector that contains the presence or absence of information of
Where, f (x, y) is the image function and * denotes the each visual word for the image.
complex conjugate.
2) Features Quantization
To compute Zernike moments from a digital image, the The most popular approach today is the (BoW) [8]. Its goal
integrals in the equation are replaced by summations and the
is to transform the image key points features into a fixed
coordinates of the image must be normalized into [0, 1] by a
vector of weights; each weight resembles the importance of the
mapping transform. visual word in the image.
2) Speeded up Robust Features In BoW, the key points descriptors are clustered using k-
Speeded up Robust Features (SURF) is a fast and robust means clustering algorithm which encodes each key point with
algorithm for local image representation. It selects interest the index of its cluster by mapping it to the nearest centroid.

203
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
Each cluster is considered to represent a visual word; thus
clustering process generates a codebook of visual words. Each
image is then represented by a bag of words; which is
quantized as a histogram of the frequency of occurrence of
visual words.
Several approaches as presented by Mu [9], Jgou [10], and
Kesorn [11] worked on enhancing the BoW model. An
extension to BoW called Spatial Pyramid Matching (SPM) was
proposed by Lazebnik [12]; it creates geometrical relationships
between features. It partitions the image into increasingly finer
spatial sub-regions and computes histograms of local features
from each sub-region. Yang [13] introduced an extension to
SPM; sparse coding is used followed by multi-scale spatial
max pooling, and propose a linear SPM kernel based on SIFT
sparse codes.
3) Global Descriptors
Other studies rely on global feature extractors. Due to
moments ability to represent global shape features they have
been used extensively in image processing as by Belkasim in
[14]. Hu and Ming-Kuei [15] introduces one of the early
studies illustrating the potential of image moments invariants
that enable successful recognition against scaling, translation,
and rotation.
ZMs are one of the most popular global descriptors. It is a
complex orthogonal rotation invariant moment composed of a
set of circle polynomials in two polar coordinates. Since, the
moments are sensitive to rotation; the magnitudes of the
moments were used as image features.
In many studies like Khotanzad [16], Kadir [17], Fleyeh
[18], and Hwang [19], ZM was used in different image
processing applications. In addition, Ono [20], Kim [21],
Vretos [22] and Singh [23] used ZM in face recognition. As
well, Wang [24] reported promising results using ZM in
Chinese character recognition.
III. PROPOSED ALGORITHM
Global Zernike moments are mostly applied to images that
have explicit shapes so they can be described well. On the
other hand, local SURF is concerned more with the object
details. In this work, a combined local/ object-based feature
extractor is presented. In this method, the SURF features are
calculated for each explicit object in the image and its moments
to get the object shape characteristics of the texture around it.
The proposed algorithm works as follows:
1) For an input image, detect the objects in it.
2) Extract object-based features for each object in an
image.
3) Detect the local key points in an object and extract the
local features for patches around each of the key
points detected.
4) Construct visual codebooks for each extractor;
randomly selecting training patches and cluster them
using k-means.
5) Encode the image; create a feature vector of a
histogram of visual word occurrences in it.
6) Concatenate the resulted visual vectors for each
image.
7) Train a support vector machine classifier and predict
the image category.
Figure 1 illustrates the main steps in the proposed method.
First, images are divided equally into training and testing sets.
For each input image, a preprocessing is done; all objects are
annotated with a bounding box around it and then cropped, so
that single objects are detected from an image instead of
dealing with the entire image. ZM features are extracted for
each object designated from the image. As well, the object
local key points are detected using the SURF technique which
results in hundreds of key points for a single image. After
balancing all the strongest features among all images, a feature
vector of SURF is calculated for a variable size block width
around each key point; which is determined from the scale of
each key point.
An image ends up with multiple feature vectors with the
same dimensions describing each object in it. For each of the
ZM and SURF extraction methods, the descriptor is then
represented using BoW. The descriptors extracted from the
images are grouped into clusters using k-means; resulting in a
visual word dictionary. The number of clusters determines the
size of dictionary. Each key point descriptor is encoded by
mapping it to the index of the nearest cluster centroid; in which
it belongs to. Only training images are used in building the
dictionary.
Each image is then represented by a histogram of the visual
words same length as the dictionary size, which is then
converted to a single visual word vector with the frequency of
the occurrence of each visual word in it accordingly. This
histogram is normalized using L2 norm to make it invariant to
the number of descriptors used.
Finally, the visual word vectors resulting from each
extractor on a single image is concatenated to form the final
feature vector for an image. Figure 2 depicts the method steps
applied on a sample image from COREL data set.
Since there are more than two groups in the data sets, a
multiclass Support Vector Machine (SVM) classifier is used to
train the data set and predict a new image category.
A one-vs-all classification schema is used which constructs
M binary SVM classifiers, each of which separates one class
from all the remaining classes. The ith SVM is trained with all
the training examples in class i with positive labels and all
other classes with negative ones. When the M classifiers are
combined to make the final decision, the classifier which
generates the highest value from its decision function is
selected as the winner. Accordingly, the corresponding class
label is assigned without considering the competence of the
classifiers.
204
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
dictionaries are trained for the tests with 2000 basis and
random patches from the training image sets.
All results report the average of 10 runs with random
selection of images for training and testing.
A. Caltech-101
The Caltech-101 data set contains 9144 images in 101
classes including vehicles, animals, objects, flowers, etc. The
number of images per category varies from 31 to 800 images.
Figure 3 shows sample of images from the data set.
The annotations given with the data set were used that
outlines the objects in the image and crop the object
accordingly. Examples of annotated objects are shown in Fig.
4.
Table I presents the results compared with other recently
proposed ones. To make fare comparison, as recommended by
the original data set [25] and suggested by other authors Griffin
[26] and Zhang [27], the whole data set was divided into 5, 10,
15, 20, 25, and 30 training images per class and a maximum of
Fig.1. Block diagram for the proposed model 50 images for testing.
In the evaluation, 81.6% accuracy was obtained compared
with several existing approaches as in [11], [12] and [26-31] as
shown in Table I.

TABLE I. IMAGE CLASSIFICATION RESULTS ON CALTECH-101

Training Images
Authors
5 10 15 20 25 30

Wang [31] 51.1 59.77 65.43 67.74 70.16 73.44

Zhang [27] 46.6 55.8 59.1 62.0 - 66.20

Lazebnik [11] - - 56.40 - - 64.60

Griffin [26] 44.2 54.5 59.00 63.3 65.8 67.60
Boiman [30] - - 65.00 - - 70.40
Jain [29] - - 61.00 - - 69.10
Gemert [28] - - - - - 64.16
Yang [12] - - 67.00 - - 73.20
Proposed Method 54.5 61.3 65.0 68.3 75.6 81. 6

Fig.2. The proposed method applied on a sample image from COREL

1000 data set.

IV. EXPERIMENTAL RESULTS AND DISCUSSION

In this section, the results on benchmark data sets are
reported; Caltech-101 and Corel 1000. As well, a discussion of
the results is offered. First, pre-processing is done on images
by annotating objects and cropping them. For each object, ZM
features with order 20 are calculated. Then using the SURF-64
descriptor, identify the region of interest points and pick the
80% strongest descriptors to calculate the features around
them. The images were all pre-processed into gray scale. All
Fig.3. Sample images from Caltech-101 data set

205
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

Fig.4. Object annotation in Caltech-101 data set

Fig.6. Objects annotation in COREL 1000 data set
B. COREL 1000
The results confirm that object-based shape features and
The COREL 1000 dataset consists of 1000 images divided
local features are both capable together to recognize objects in
among 10 classes. Each category has 100 images. The
scene images with high accuracy. That is, neither applying ZM
categories are Africans, Architecture, Beach, Buses, Dinosaurs,
global feature descriptor nor SURF local descriptor alone was
Elephants, Flowers, Food, Horses, and Mountains. Figure 5
as effective as combining them for enhanced object recognition
shows a sample from each category in the data set.
rates.
Objects annotations and cropping are done manually on the
The main finding is that working on the detected objects in
whole data set. Samples of the manually cropped objects from
the image is way better than dealing with the whole image at
images are shown in Fig. 6. The same parameter setup was
once. Particularly, in ZM shape-based extractor, working on
prepared as mentioned before for Caltech-101. Experiments
explicit objects helps to define clear boundary for the object
were performed using 50 images for training and 50 for testing
rather than multiple objects in the same image.
as recommended by Oliveira [32].
Certainly, differences in dealing with the image objects
In the evaluation, 90.8% recognition performance was
contribute to the correctness variances between this study and
reached that is higher than SMK [34], ScSPM [12], LScSPM
the other previously stated ones. However, gray scale images
[33], and SSC [32] methods. Table II summarizes the results on
were used in features extraction. Future research should include
Corel 1000 data set.
color features to enhance the recognition accuracy.
TABLE II. IMAGE CLASSIFICATION RESULTS ON COREL 1000 V. CONCUSION AND FUTURE WORK
Authors 50 Training Images The paper presented an approach for object recognition,
Lu [34] 77.9
which uses a combined feature extractor method. Objects in an
Yang [12] 86.20 image are detected first; some object annotations were already
Gao [33] 88.40 offered with the Caltech-101 data set and others for COREL
1000 were done manually.
Oliveira [32] 88.40
Proposed Method 90.80 The method then extracts the object-based features using
ZMs and the local feature SURF descriptors for the captured
objects in each image. As well, the technique uses bag of
words for compact representation of images so that the image
can be represented in a single feature vector.
Based on COREL 1000 and Caltech-101 data sets, the
experimental results show that, the proposed method achieves
better accuracy according to the published ones on the same
databases. Future work will focus on automating the object
annotations in images and enhancing the recognition
performance by including the color parameter in the SURF
extractor instead of dealing with gray images.
REFERENCES
Fig.5. Sample image from each category in COREL 1000 data set
[1] Schmitt, Drew, and Nicholas McCoy. "Object classification
and localization using SURF descriptors." CS 229 Final
Projects, 2011.

206
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt
[2] Teague, Michael Reed. "Image analysis via the general
theory of moments*."JOSA 70.8 (1980): 920-930.
[3] Hu, Ming-Kuei. "Visual pattern recognition by moment
invariants." Information Theory, IRE Transactions on 8.2
(1962): 179-187.
[4] Vogel, Julia, and Bernt Schiele. "On performance
characterization and optimization for image
retrieval." Computer VisionECCV 2002. Springer Berlin
Heidelberg, 2002. 49-63.
[5] Maree, Raphael, et al. "Random subwindows for robust
image classification."Computer Vision and Pattern
Recognition, 2005. CVPR 2005. IEEE Computer Society
Conference on. Vol. 1. IEEE, 2005.
[6] Lowe, David G. "Object recognition from local scale-
invariant features."Computer vision, 1999. The proceedings
of the seventh IEEE international conference on. Vol. 2.
Ieee, 1999.
[7] Bay, Herbert, Tinne Tuytelaars, and Luc Van Gool. "Surf:
Speeded up robust features." Computer visionECCV 2006.
Springer Berlin Heidelberg, 2006. 404-417.
[8] Lavou, Guillaume. "Combination of bag-of-words
descriptors for robust partial shape retrieval." The Visual
Computer 28.9 (2012): 931-942.
[9] Mu, Yadong, et al. "Randomized locality sensitive
vocabularies for bag-of-features model." Computer Vision
ECCV 2010. Springer Berlin Heidelberg, 2010. 748-761.
[10] Jgou, Herv, et al. "Aggregating local descriptors into a [26]
compact image representation." Computer Vision and
Pattern Recognition (CVPR), 2010 IEEE Conference on.
IEEE, 2010.
[11] Kesorn, Kraisak, and Stefan Poslad. "An enhanced bag-of-
visual word vector space model to represent visual content
in athletics images." Multimedia, IEEE Transactions
on 14.1 (2012): 211-222.
[12] Lazebnik, Svetlana, Cordelia Schmid, and Jean Ponce.
"Beyond bags of features: Spatial pyramid matching for
recognizing natural scene categories."Computer Vision and
Pattern Recognition, 2006 IEEE Computer Society
Conference on. Vol. 2. IEEE, 2006.
[13] Yang, Jianchao, et al. "Linear spatial pyramid matching
using sparse coding for image classification." Computer
Vision and Pattern Recognition, 2009. CVPR 2009. IEEE
Conference on. IEEE, 2009.
[14] Belkasim, Saeid O., Malayappan Shridhar, and Majid
Ahmadi. "Pattern recognition with moment invariants: a
comparative study and new results."Pattern
recognition 24.12 (1991): 1117-1138.
[15] Hu, Ming-Kuei. "Visual pattern recognition by moment
invariants." Information Theory, IRE Transactions on 8.2
(1962): 179-187.
[16] Khotanzad, Alireza, and Yaw Hua Hong. "Invariant image
recognition by Zernike moments." Pattern Analysis and
Machine Intelligence, IEEE Transactions on 12.5 (1990):
489-497.
[17] Kadir, Abdul, et al. "A comparative experiment of several
shape methods in recognizing plants." arXiv preprint
arXiv:1110.1509 (2011).
[18] Fleyeh, Hasan, et al. "Invariant road sign recognition with
fuzzy artmap and zernike moments." Intelligent Vehicles
Symposium, 2007 IEEE. IEEE, 2007.
[19] Hwang, Sun-Kyoo, and Whoi-Yul Kim. "A novel approach
to the fast computation of Zernike moments." Pattern
Recognition 39.11 (2006): 2065-2076.
[20] Ono, Atsushi. "Face recognition with Zernike
moments." Systems and Computers in Japan 34.10 (2003):
26-35.
[21] Kim, Hyoung-Joon, and Whoi-Yul Kim. "Eye detection in
facial images using Zernike moments with SVM." ETRI
journal 30.2 (2008): 335-337.
[22] Vretos, Nicholas, Nikos Nikolaidis, and Ioannis Pitas. "3D
facial expression recognition using Zernike moments on
depth images." Image Processing (ICIP), 2011 18th IEEE
International Conference on. IEEE, 2011.
[23] Singh, Chandan, Neerja Mittal, and Ekta Walia. "Face
recognition using Zernike and complex Zernike moment
features." Pattern Recognition and Image Analysis 21.1
(2011): 71-81.
[24] Wang, Tiansheng, and Shengcai Liao. "Chinese character
recognition by Zernike moments." Audio, Language and
Image Processing (ICALIP), 2014 International
Conference on. IEEE, 2014.
[25] Fei-Fei, Li, Rob Fergus, and Pietro Perona. "Learning
generative visual models from few training examples: An
incremental bayesian approach tested on 101 object
categories." Computer Vision and Image
Understanding 106.1 (2007): 59-70.
Griffin, Gregory, Alex Holub, and Pietro Perona. "Caltech-
256 object category dataset." (2007).
[27] Zhang, Hao, et al. "SVM-KNN: Discriminative nearest
neighbor classification for visual category
recognition." Computer Vision and Pattern Recognition,
2006 IEEE Computer Society Conference on. Vol. 2. IEEE,
2006.
[28] Van Gemert, Jan C., et al. "Kernel codebooks for scene
categorization."Computer VisionECCV 2008. Springer
Berlin Heidelberg, 2008. 696-709.
[29] Jain, Prateek, Brian Kulis, and Kristen Grauman. "Fast
image search for learned metrics." Computer Vision and
Pattern Recognition, 2008. CVPR 2008. IEEE Conference
on. IEEE, 2008.
[30] Boiman, Oren, Eli Shechtman, and Michal Irani. "In
defense of nearest-neighbor based image
classification." Computer Vision and Pattern Recognition,
2008. CVPR 2008. IEEE Conference on. IEEE, 2008.
[31] Wang, Jinjun, Fengjun Lv, and Kai Yu. "Locality-
constrained linear coding systems and methods for image
classification." U.S. Patent No. 8,233,711. 31 Jul. 2012.
[32] Oliveira, Gabriel L., et al. "Sparse spatial coding: A novel
approach for efficient and accurate object
recognition." Robotics and Automation (ICRA), 2012 IEEE
International Conference on. IEEE, 2012.
[33] Gao, Shenghua, et al. "Local features are not lonely
Laplacian sparse coding for image
classification." Computer Vision and Pattern Recognition
(CVPR), 2010 IEEE Conference on. IEEE, 2010.
[34] Lu, Zhiwu, and Horace HS Ip. "Image categorization by
learning with context and consistency." Computer Vision
and Pattern Recognition, 2009. CVPR 2009. IEEE
Conference on. IEEE, 2009.

207
 ICCTA 2015, 24-26 October 2015, Alexandria, Egypt

ICCTA 2015 Committees

Conference Chair
H. E. Prof. Dr. Yousry S. El-Gamal
Former Minister of Education
President, Computer Scientific Society

Honorary Chair
Prof. Dr. Ismail Abdel Ghafar Ismail Farag
President, Arab Academy for Science, Technology, and Maritime Transport (AASTMT), Egypt

Conference Coordinators and Program co-Chairs

Prof. Dr. Ayman Abdel-Hamid, AASTMT, Alexandria, Egypt.
Dr. Essam Kosba, AASTMT, Alexandria, Egypt.

Organizing Committee
Prof. Dr. Ossama Ismail, AASTMT, Alexandria, Egypt
Prof. Dr. Khaled Mahar, AASTMT, Alexandria, Egypt
Dr. Mahmoud Farouk, AASTMT, Alexandria, Egypt
Eng. Hassan Bakr, AASTMT, Alexandria, Egypt
Dr. Mohamed Kholief, AASTMT, Alexandria, Egypt
Dr. Mohamed El-Habrouk, Alexandria University, Alexandria, Egypt
Dr. Mohamed El-Shaib, AASTMT, Alexandria, Egypt

Finance Chair
Eng. Mahmoud Morsy
Honorary Secretary, Computer Scientific Society

208