You are on page 1of 8

International Journal of Research in Computer and ISSN(Online) 2278-5841

Communication Technology, Vol 2, Issue 9, September -2013 ISSN (Print) 2320-5156

Classification Of Heart Disease Using Svm And ANN

Deepti Vadicherla1, Sheetal Sonawane2


1
Department of Computer Engineering, Pune Institute of Computer and Technology,
University of Pune, Pune, India
deepti.vadicherla@gmail.com
2
Department of Computer Engineering, Pune Institute of Computer and Technology,
University of Pune, Pune, India
sssonawane@pict.edu

Abstract: example, many clinical symptoms are associated with


many human organs other than the heart and very
Classification of heart disease can be useful for the
often heart diseases may exhibit various syndromes.
physicians if it is computerized for the purpose of fast
Due to this complexity, there is a need to automate
diagnosis and accurate result. Predicting the existence
the process of medical diagnosis which can help
of heart disease accurately can save patients life. The
medical practitioners in the diagnostic process [1],
objective of this paper is to analyze the application of
[2].
AI tools for classification and prediction of heart
disease. The work includes the classification of heart To reduce the diagnosis time and improve the
disease using Support Vector Machine and Artificial diagnosis accuracy, it has become more of a
Neural Network. Comparison is carried out among demanding issue to develop reliable and powerful
two methods on the basis of accuracy and training medical decision support systems to support the
time. This paper presents a Medical decision support diagnosis decision process. Basically medical
system for heart disease classification in a rational, diagnosis is complicated process, hence the approach
objective, accurate and fast way. The dataset used is for solving this issue, is to develop such an intelligent
the Cleveland Heart Database taken from UCI system, such as Support Vector Machine and
learning data set repository. In the proposed model Artificial Neural Network[4],[5]. It has shown great
we classify the data into two classes using SMO potential to be applied in the design and
algorithm in Support Vector machine and Artificial implementation of decision support system of heart
Neural Network (ANN). disease. The system uses features extracted from the
ECG data of the patients. The experiments however,
Keywords: Support Vector Machine, Sequential have been performed taking the Cleveland Heart
Minimal Optimization ,Optimization problem, Heart Database taken from UCI learning data set repository
disease, Artificial Neural Network. which was donated by Detrano[23]. Results obtained
from support vector machine model are satisfactory.
This paper presents a medical decision support
I. INTRODUCTION system for heart disease classification. In the
At present, the number of people suffering from heart proposed model we classify the data into two classes
disease is on a rise. Accurate diagnosis at an early using Support Vector machine and Artificial Neural
stage followed by proper subsequent treatment can Network [21], [22].
result in significant life saving. New data released by The rest of the paper organized as, support vector
the National Heart, Lung, and Blood Institute machine described in section 2. Section 3 includes
(NHLBI) of the show that especially women in older artificial neural network, in which functioning of
age groups are more at risk of getting heart disease. A ANN is explained. Proposed model of MDSS and
recent study fielded Heart disease can be controlled related work is mentioned and explained in section 4.
effectively if it is diagnosed at an early stage [24]. Experiments and results are shown in section 5.
But its not easy to do accurate diagnosis because of Section 6 has conclusion followed by the future
many complicated factors of heart diseases. For work in section 7.

www.ijrcct.org Page 694


International Journal of Research in Computer and ISSN(Online) 2278-5841
Communication Technology, Vol 2, Issue 9, September -2013 ISSN (Print) 2320-5156

Therefore we can say that, any point from one class


II. SUPPORT VECTOR MACHINE lies above the separating hyperplane satisfies, f(X) >
0. In the same way any point from another class lies
Support Vector Machine, is a promising method of below the separating hyperplane satisfies, f(X) < 0.
learning machine, based on statistical learning theory
developed by Vladimir Vapnik. Support vector Above equations were processed to make the linearly
machine (SVM) used for the classification of both separable set D to meet the following inequality,
linear and nonlinear data [6], [7]. It performs yi ( f(x) ) ≥ 1, i
classification by constructing a linear optimal
separating hyperplane within higher dimension, with 1
Here the margin m is , m
the help of support vectors and margins, which || w ||2
separates the data into two categories (or classes).
With an appropriate nonlinear mapping the original Using above equation, maximizing margin can be
training data is mapped into a higher dimension. written in the form of optimization problem as below:
Within this the data from two classes can always be 1
min 2 || w|| y w.x  b   1, i
2
Subject to
separated by a hyperplane[8]. w, b
i

This optimization problem can be solved by using


Relevant mathematics associated with the project is
dual Lagrange multiplier,
as given below. N N N
  
Let S represents Medical decision support system.
  ( )  min
min

   yi y j ( xi  x j ) i j    i ,

1
2
i 1 j 1 i 1
This system provides classification by two methods,
one is SVM and another is ANN. It can be shown in
following form, The output of a non-linear SVM is explicitly
S= { SVM, ANN} computed from the Lagrange multipliers [17, 9],
Suppose f is a function for Support vector machine, N
 
then, u   y j j K ( x j , x )  b,
f: IO j 1

where, I is domain ( set of inputs) where K is a kernel function. We used Radial Basis
Kernel Function (RBF) [10] here, which is denoted as
I= {D, E} follow:
D= { X, Y} K(xi, xj) = exp(-γ|| xi – xj ||2), γ>0
X= {xi | 1<=i<=n} The non-linearity alter the quadratic form, but the
dual objective function is still quadratic in α,
Y= {yi |1<=i<=n}  N N N
 
min  ( )  min 1
  y y
i j K ( x i , x j )  i  j   i,
}
n   2
 
i.e. D= { (xi, yi) ∈ (X × Y) i 1 j 1 i 1
i 1 0   i  C , i,
N
E (set of constants) = {C, e}  y
i 1
i i  0.
O is co domain (set of output) Sequential minimal optimization algorithm solves
O = { opi | 1<=i<=n} above quadratic programming problem by repeatedly
finding two Lagrange multipliers that can be
The support vector machine computes a linear optimized with respect to each other.
classifier of the form,
Sequential minimal optimization (SMO) [5] is an
f(x) = WX + b algorithm for efficiently solving the optimization
Where, W is weight vector problem which arises during the training of support
vector machine. At every step, SMO chooses two
X is input vector Lagrange multipliers to jointly optimize, finds the
B is bias optimal values for these multipliers and updates the
SVM to reflect the new optimal values [15].
The separating hyperplane is the plane, f(x) = 0.
Functioning of SMO as in the below algorithm.

www.ijrcct.org Page 695


International Journal of Research in Computer and ISSN(Online) 2278-5841
Communication Technology, Vol 2, Issue 9, September -2013 ISSN (Print) 2320-5156

SMO training algorithm: }while(training error > 0.01 && epoch


<25000);
Step 1: Input C, kernel, kernel parameters, and
Step 6: Compute the training accuracy.
epsilon.
Step 7: Use this trained ANNnetwork for testing.
Step 2: Initialize αi = 0 and b= 0
Step 3: Let f(x) = b+
and τ the Suppose g is a function for Artificial Neural

tolerance. Network, then,


Step 4: Find Lagrange multiplier αi, which violates g: MN
KKT optimization.
Step 5: Choose second multiplier and optimize pair. where, M is domain ( set of inputs)
Repeat steps 4 and 5 till convergence.
M= {A, B}
Step 6: Update α1 and α2 in one step.
A= { X, Y}
α1 can be changed to increase f(x1).
α2 can be changed to decrease f(x2).
}
P
Step 7: Compute new bias weight b. A= { (xi, yi) ∈ (X × Y)
i1

III. ARTIFICIAL NEURAL NETWORK B={ wij, Δij(t), E, η, epochs}


Let wij =weights
An artificial neural network is a computational model Δij(t) = individual update value
based on the structure and functions of biological Δij(t) exclusively determines the magnitude of the
neural networks. Information that flows through the weight-update.
network affects the structure of the ANN because a This update value can be expressed
neural network changes, in a sense based on that mathematically according to the learning rule for
input and output. ANNs are considered nonlinear each case based on the observed behavior of the
statistical data modeling tools where the complex partial derivative during two successive weight-steps
relationships between inputs and outputs are modeled by the following formula:
or patterns are found.Training a neural network
model essentially means selecting one model from
the set of allowed models that minimizes the cost
criterion. There are numerous algorithms available
for training neural network models; most of them can
be viewed as a straightforward application of
optimization theory and statistical estimation. In the
proposed model Resillient Backpropogation
algorithm is used for training ANN.
where, 0 < η- < 1 < η+ .
Classification algorithm of ANN: Whenever the partial derivative of the equivalent
weight wij varies its sign, it indicates that the last
Step 1: Initialize the training class buffers, input data
update was large in magnitude and the algorithm has
buffers,
skipped over a local minima then,
Declare BasicNetwork ANNnetwork in ENCOG.
Δij(t) = Δij(t) - η−
Step 2: Extract input data and update as,
d= Input Layers, out= Output Read, Hidden layers = Otherwise, the update-value will do some extent
2d -1. increase.
Step 3: Initialize trainigSet with InputVal[] and If the derivative is positive, the weight is decreased
OutputVal[]. by its update value, if the derivative is negative, the
Step 4: Use Resillient Propagation for training of
neural network.
Step 5: epoch = 1;
do{
train the ANNnetork.
Epoch++;
www.ijrcct.org Page 696
International Journal of Research in Computer and ISSN(Online) 2278-5841
Communication Technology, Vol 2, Issue 9, September -2013 ISSN (Print) 2320-5156

update-value is added as shown below: result in speeding up the computation process. They
have high tolerance of noisy data. The major
disadvantage of neural networks is that, they have
poor interpretability. Fully connected networks are
difficult to articulate. Whereas various empirical
studies of Bayesian classifier in comparison with
decision tree and neural network classifiers have
found out that, in theoretical way Bayesian classifiers
have minimum error rate in comparison to all other
wij(t+1) = wij(t) + Δwij(t) classifiers. However, in practice this is not always the
case, owing to inaccuracies in the assumptions made
However, there is one exception. If the for its use, such as class conditional independence
partial derivative changes sign that is the previous and the lack of available probability data.
step was too large and the minimum was missed, the
previous weight-update is reverted: 2. Pre-processed data
Δwij(t) = - Δwij(t-1),
The experiments are carried out on heart dataset
if using Sequential Minimal Optimization in Support
Vector Machine.
To avoid a double penalty of the update-value, set the
above update rule by putting below value in Δij. The experiments are carried out on heart dataset
using Sequential Minimal Optimization in Support
Vector Machine.
Heart disease is diagnosed with the help of some
The partial derivative of the total error is given by the complex pathological data. The heart disease dataset
following formula: used in this experiment is the Cleveland Heart
Disease database taken from UCI machine learning
dataset repository [23]. This database contains 14
attributes as below:
1. Age of patient, 2. Sex of patient, 3. Chest pain
This indicates that the weights are updated only after type, 4. Resting blood pressure, 5. Serum cholesterol,
the presentation of all of the training patterns. 6. Fasting blood sugar, 7. Resting ECG results, 8.
Resilient back-propagation (RPROP) Maximum heart rate achieved, 9. Exercise induced
training algorithm [21] was adopted to train the angina, 10. ST depression induced by exercise
proposed ANN model as mentioned previously. After relative to rest, 11. Slope of the peak exercise ST
the selection of network, the network has been segment, 12. number of major vessels colored by
trained using resilient backpropagation training flourosopy, 13. thal, 14. Diagnosis of heart disease.
scheme. The training parameters have been modified
several times as explained above until the optimum 3. Flow diagram of MDSS
performance has been achieved. Maximum number
The purpose of this proposed model is to diagnose
of iterations has been set to 25000 epochs.
the heart disease by classifying the dataset of heart
disease. This classification process is shown in
IV. PROPOSED MODEL OF MEDICAL Figure 1.
DECISION SUPPORT SYSTEM

1. Related work

Medical decision support system work has been


carried on the basis of performance of different
methods like SVM[20], Artificial neural network,
Bayesian classification method, etc. [1], [2]. Neural
network algorithms are inherently parallel, which

www.ijrcct.org Page 697


International Journal of Research in Computer and ISSN(Online) 2278-5841
Communication Technology, Vol 2, Issue 9, September -2013 ISSN (Print) 2320-5156

minimal optimization in Support vector machine is


more effective than Resilient backpropagation in
Artificial neural network.
The proposed system is tested with many datasets.
The experimental results are shown in the following
figures.

Figure 2: Pie chart of multiclass SVM classification


1. Multiclass classification by using Support
Vector Machine
Following Table1 gives the result of multiclass
classification of SVM.

Figure 1. Flow diagram of MDSS for heart disease Figure 2 of pie chart shows the performance of the
system with fifth sample from table1, which gives
100% accuracy.
V. RESULT AND DISCUSSION
In the proposed model, we used dataset having 297
2. Multiclass classification by using
total number of patient records. Large part of records
Artificial neural network
in the dataset is used for training and rest of them are
used for testing. The main difference between the Following analysis done with, Input Layer = 13,
dataset given as input to training and testing is that, Hidden Layer = 25 and Output Layer = 1. In this case
the input we are giving to training is the data with training is done till error becomes less than 0.010 or
correct diagnosis (14th field in the dataset) and epochs are less than 25000. Table2 and Figure 3
whereas the input data of testing doesn’t have the gives the result of ANN multiclass classification.
correct diagnosis purposely. The Diagnosis (14 th)
field refers to the presence or absence of heart Following Figure 3 of pie chart shows the
disease of that respective patient. It is integer valued performance of fifth sample from table 2 which gives
field, having value 1(absence of disease) or 65% accuracy.
-1(presence of disease). So that at the end of testing
process we can check the result in the output file
created after testing and verify the efficiency of the
proposed model in terms of accuracy.
In the proposed system two methods of classification
are provided, Support vector machine and Artificial
neural network. Performance of both the
classification techniques is compared in terms of time
needed for classification and accuracy of the system.
From the below analysis we can say that, Sequential

www.ijrcct.org Page 698


International Journal of Research in Computer and ISSN(Online) 2278-5841
Communication Technology, Vol 2, Issue 9, September -2013 ISSN (Print) 2320-5156

Figure 3: Pie chart of multiclass ANN classification

Testing
No. of Class Class Class Class
Class I Accuracy Time Accuracy-
samples II III IV V
(10 samples)

50 30 7 4 5 4 100 0.078 100

100 57 19 10 10 4 100 0.063 100

150 83 27 15 18 7 100 0.078 100

200 110 37 20 23 10 100 0.094 100

297 160 54 35 35 13 100 0.218 100

Table 1: Performance of the multiclass SVM decision support system

No. of Class Class Class Class Training Testing


Class I Time
samples II III IV V Accuracy Accuracy
50 30 7 4 5 4 100 1.43 100

100 57 19 10 10 3 95 31.02 90

150 83 25 14 18 5 91.2 45.6 90

200 110 36 10 15 4 70.4 36.65 70

297 159 40 21 23 4 68.2 56.5 65

Table 2: Performance of the multiclass ANN decision support system

www.ijrcct.org Page 699


International Journal of Research in Computer and ISSN(Online) 2278-5841
Communication Technology, Vol 2, Issue 9, September -2013 ISSN (Print) 2320-5156

VI. CONCLUSION svm classifier design,” Neural Computation,


vol.13,pp:637-649, 2002.
The results of the SVM classification algorithm
compared to the ANN classification, are very [5] John C. Platt, “Sequential Minimal
encouraging. The difference in the accuracy is Optimization: A Fast Algorithm for Training
noticeable. Moreover the difference in the execution Support Vector Machines”, Microsoft
times is even more noteworthy. The enhanced Research, Technical Report MSR-TR-98-14.
performance of the SVM classification is due to the
[6] Gong Wei Wang Shoubin, “Support Vector
fact that they can avoid repetitive searches in order to
Machine for Assistant Clinical Diagnosis of
find the best two points to use for each optimization
Cardiac Disease”, IEEE Global Congress on
step. It is found that SMO performs better with high
Intelligent Systems, pp: 588-591, 2009.
accuracy when the data is preprocessed and given as
input. Applied to the task of solving classification [7] Ya Gao; Shiliang Sun, “An empirical
problem of heart disease and the features extracted evaluation of linear and nonlinear kernels
based on statistical properties, the accuracy is higher for text classification using Support Vector
in proposed SVM classification model which uses Machines”, IEEE Seventh International
SMO. Conference on Fuzzy Systems and
Knowledge Discovery (FSKD), Pages:
VII. FUTURE WORK 1502-1505, 2010.
SMO is a carefully organized algorithm which has [8] Qinghua Jiang, Guohua Wang, Tianjiao
excellent computational efficiency. However, Zhang, Yadong Wang, “Predicting Human
because of its way of computing and use of a single microRNA-disease Associations Based on
Support Vector Machine”, IEEE
International Conference on Bioinformatics
threshold value it can become inefficient. In future and Biomedicine, pp:467-472 , 2010.
multiple threshold parameters can be used to improve
the performance in terms of speed. In case of [9] Browne, K.E.; Burkholder, R.J., “Nonlinear
Optimization of Radar Images From a
artificial neural network, window momentum is a
Through-Wall Sensing System via the
standard technique that can be used to speed up
Lagrange Multiplier Method”, IEEE
convergence and maintain generalization
Geoscience and Remote Sensing Letters,
performance. Window momentum can give
significant speed-up over a set of applications with Pages: 803-807, 2012.
same or improved accuracy. [10] Gao Daqi; Zhang Tao, “Support vector
machine classifiers using RBF kernels with
clustering-based centers and widths” IEEE
REFERENCES Intrnational Joint Conference on Neural
[1] Long Wan, Wenxing Bao, “Research and Networks, 2007 , Pages: 2971-2976.
Application of Animal Disease Intelligent [11] Olvi L.Mangasarian and Michael
Diagnosis Based on Support Vector E.Thompson, “Massive Data Classification
Machine” IEEE International Conference on via Unconstrined Support Vector Machines”
Computational Intelligence and Security, Journal of Optimization Theory and
Pages 66-70, 2009. Applications, 131, Pages315–325, 2006.
[2] S.N. Deepa, B. Aruna Devi, “Neural [12] P. S. Bradley and O. L. Mangasarian.
Networks and SMO based Classification for “Massive data discrimination via linear
Brain Tumor”, IEEE World Congress on support vector machines. Optimization
Information and Communication Methods and Software, 13:1-10,
Technologies, Pages 1032-1037,2010. 2000.ftp://ftp.cs.wisc.edu/math-prog/tech-
[3] C. Cortes and V. Vapnik, “Support-vector reports/98-05.ps
network,” Machine Learning, vol.20, 1995. [13] Osuna, E., Freund, R., Girosi, F., “Improved
[4] S. S. Keerthi, S. K. Shevade, C. Training Algorithm for Support Vector
Bhattacharyya, and K. R. K. Murthy Machines”, Proc. IEEE NNSP ’97, 1997.
“Improvements to platt’s smo algorithm for

www.ijrcct.org Page 700


International Journal of Research in Computer and ISSN(Online) 2278-5841
Communication Technology, Vol 2, Issue 9, September -2013 ISSN (Print) 2320-5156

[14] Yiqiang Zhan, Dinggang Shen, “Design [24] Mediline Plus: Information related to Heart
efficient support vector machine for fast Diseases.
classification”, The journal of pattern http://www.nlm.gov/medlineplus/heartdiseas
recognition society 38, Pages 157-161,2004. es.html
[15] Peng Peng, Qian-Li Ma, Lei-Ming Hong,
“The Research of the Parallel SMO
algorithm for solving SVM”, Proceedings of
the Eighth International Conference on
Machine Learning and Cybernetics,
Baoding, pp:1271-1274, 12-15 July 2009.
[16] Chin-Jen Lin, “Asymptotic convergence of
an SMO algorithm without any
assumptions”, IEEE Transactions on Neural
Networks, vol.13,issue 1, pp:248-250, 2002.
[17] Baxter Tyson Smith, “Lagrange Multipliers
Tutorial in the Context of Support Vector
Machines”, Memorial University of
Newfoundland St. John’s, Newfoundland,
Canada.
[18] Ya-Zhou Liu; Hong-Xun Yao; Wen Gao;
De-bin Zhao, “Single sequential minimal
optimization: an improved SVMs training
algorithm”, Proceedings of 2005
International Conference on Machine
Learning and Cybernetics, 2005. Pages
4360- 4364.
[19] Kjeldsen, Tinne Hoff. "A contextualized
historical analysis of the Kuhn-Tucker
theorem in nonlinear programming: the
impact of World War II". Historia Math. 27
(2000), no. 4, 331–361.
[20] Deepti Vadicherla, Sheetal Sonawane,
“Decision support system for heart disease
based on sequential minimal optimization in
support vector machine” International
Journal of Engineering Sciences and
Emerging Technologies, Volume 4, Issue 2,
Pages: 19-26, 2013.
[21] Ian H. Witten, Eibe Frank, Data Mining,
Elsevier Inc. 2005.
[22] Jiawei Han and Micheline Kamber, Data
Mining: Concepts and Techniques, 2/e,
Morgan Kaufmann Publishers, Elsevier Inc.
2006.
[23] UCI Machine Learning Repository: Heart
Disease Data Set.
http://archive.ics.uci.edu/ml/datasets/Heart+
Disease

www.ijrcct.org Page 701

You might also like