A Support vector machines (SVMs) are a family of supervised machine learning methods

developed in recent years. SVMs were first suggested by Vapnik in the 1960s for classification.
The main idea behind the SVM algorithm is to map input vectors into a feature space of higher
dimension, construct a linear decision surface (hyperplane), and then optimize that hyperplane
for generalization. SVMs are used for classification, regression, and ranking, and are used for
related tasks such information retrieval and optical character recognition.
The support vector machine (SVM)

is a training algorithm for learning classification and
regression rules from data. SVMs arose from statistical learning theory; the aim being to solve
only the problem of interest without solving a more difficult problem as an intermediate step.
SVMs are based on the structural risk minimization principle, closely related to regularization
theory.
Two key elements in the implementation of SVM are the techniques of mathematical
programming and kernel functions. The parameters are found by solving a quadratic
programming problem with linear equality and inequality constraints; rather than by solving a
non-convex, unconstrained optimization problem. The kernel forms the basic parts of SVM and
the flexibility of kernel functions allows the SVM to search a wide variety of hypothesis spaces.
The basic principles of an SVM are as follows
1. The SVM performs a nonlinear mapping of the input vectors (objects) from the input space R
d
into a high dimensional feature space H; the mapping is determined by a kernel function.
2. It finds a linear decision rule in the feature space H in the form of an optimal separating plane,
which is the one that leaves the widest margin between the decision boundary and the input
vectors mapped into H.
3. This plane is found by solving the following constrained quadratic programming problem:
Maximize
( ) W o

( )
1 1 1
1
,
2
n n n
i i j i j i j
i i j
y y K x x o oo
= = =
=

under the constraints
1
0
n
i i
i
y o
=
= =

and
0 C o s s for i=1,2,.., n where
i d
x R e are the training sample set (TSS) vectors, and
{ } 1, 1
i
y e + the corresponding class labels. C is a constant needed for non-separable classes;
( ) , K u v is a kernel function.
There are three types of support vector machines:
Polynomial SVM:
( ) ( ) , , 1
d
K u v u v = + (


This type of machine describes the mapping of an input space into the feature space that
contains all products .....
i i j j k k
u v u v u v up to degree d.
Radial Basis Function SVM:
( )
{ }
2
, exp K u v u v

=
This type of machine forms a separation rule according to radial hyperplanes.
Two-layer neural network SVM:
( ) ( ) , tanh , K u v w u v c = + (


This type of machine generates a kind of two-layer neural network without back propagation.