Octobre 2010

RÉSEAU DE NEURONS
NETWORK ARCHITECTURE
 Normally the neural network is divided into:

 Input layer (data is presented to the NN)

 Output layer (presents the result to the user)

 Hidden layers (refer to one or more layer)

NETWORK ARCHITECTURE
NETWORK ARCHITECTURE
 What is a neuron ?
w1 Neuron is a parameterized algebraic
function with borders.
w2


w3

w4
∑
w5
Sum Activatio
n
function
b

weight
NETWORK ARCHITECTURE
 Type of connection between layers:

 Fully connected

 Partially connected

 Bi-directional
NETWORK ARCHITECTURE
 Number of nodes:
 Input and output layers  automatically determined by
the number of inputs and outputs.

 For hidden layer

 no steadfast rule for determining the neurons number.

 There is 2 approach to do it:

 Start with few and then increase until the overall results
are improved.
 Start with twice the number of input nodes plus one
([x*2]+1)
NETWORK ALGORITHMS
 First we must see the activation function inside
the neurons:
 Threshold
 Linear
 Sigmoid
 Note: the sigmoid function is the most common
function used.
NETWORK ALGORITHMS
 While training of artificial neural network:
1. First the connection between neurons are set to
random weight values.
2. During the training process the input-output data are
fed into the network.
3. Difference between training output and actual output
is then calculated.
4. Considering this deference as error, using the training
algorithm the weight is updated to reduce this error.
5. Once trained, the network is hopefully ready to predict
accurate output.
NETWORK ALGORITHMS
 Training ANN:
 “Training is defined as a search process for the optimized set
of weight values, witch can minimize the squared error
between the estimation and the experimental data of units in
output layer.”

 Learning a neural network can be supervised or unsupervised.

 The most common algorithm is supervised training called
standard-back Propagation.
NETWORK DATA SETS
 Data set is often divided into “training set” and
“test set”.
 “Training set” is for learning (weights)
 “Test set” is to tune parameters (architecture not
weight number of hidden units)
 TRAINING SETS
 A large training set is required with a variety of data.
 Numeric and nominal variables can be handled, all
other sets need to be converted or discarded.
 Redundant variables are minimized or eliminated.
 Cases with missing values can be used but outliers
may cause problems. (if necessary)
 Choose independent variables … ???
Training algorithms

PRUNING ALGORITHM
PRUNING ALGORITHM BASIC IDEA
sortie

+1 +2 +3 +4

Entree
PRUNING ALGORITHM

Adjusting the weight to follow the equations :

w y  (w
[ p] [ p]
ij j  ij   ij ) yi
jRi jRi {h}

 ij y j
[ p] [ p]
To simplify: w y ih h
jRi {h}