7 Anand ANN PDF

Artificial Neural Networks
Outline
Introduction to Neural Network

Introduction to Artificial Neural Network
Properties of Artificial Neural Network
Applications of Artificial Neural Network
Demo Neural Network Tool Box
Case-1 Designing XOR network
Case-2 Power system security assessment
CIARE-2012, IIT Mandi 2

What are Neural Networks?
Models of the brain and nervous system
Highly parallel
Process information much more like the
brain than a serial computer
Learning
Very simple principles

Very complex behaviours
BIOLOGICAL NEURAL NETWORK
Figure 1 Structure of biological neuron

The Structure of Neurons
A neuron has a cell body, a branching input
structure (the dendrIte) and a branching
output structure (the axOn)
Axons connect to dendrites via synapses.

Electro-chemical signals are propagated
from the dendritic input, through the cell
body, and down the axon to other neurons

The Structure of Neurons
A neuron only fires if its input signal

exceeds a certain amount (the threshold)
in a short time period.
Synapses vary in strength
Good connections allowing a large signal
Slight connections allow only a weak signal.
Synapses can be either excitatory or
inhibitory.

The Artificial Neural Network
Figure 2 Structure of artificial neuron
Mathematically, the output expression of the

network is given as Y = F ( S ) = F X K W K + b
N

CIARE-2012, IIT Mandi
K =1 7
ANNs The basics
ANNs incorporate the two fundamental
components of biological neural nets:
1. Neurones (nodes)
2. Synapses (weights)

Properties of Artificial Neural
Nets (ANNs)

Properties of Artificial Neural
Nets (ANNs)
Many simple neuron-like threshold switching

units
Many weighted interconnections among units
Highly parallel, distributed processing
Learning by tuning the connection weights

Appropriate Problem Domains
for Neural Network Learning
Input is high-dimensional discrete or real-

valued (e.g. raw sensor input)
Output is discrete or real valued
Output is a vector of values
Form of target function is unknown
Humans do not need to interpret the results
(black box model)

Applications
Ability to model linear and non-linear systems
without the need to make assumptions implicitly.
Applied in almost every field of science and
engineering. Few of them are
Function approaximation, or regression analysis, including time series
and modelling.
Classification, including pattern and sequence recognition, novelty
detection and sequential decision making.
Data processing, including filtering, clustering, blind signal separation
and compression.
Computational neuroscience and neurohydrodynamics
Forecating and prediction
Estimation and control
Applications in Electrical
Load forecasting
Short-term load forecasting
Mid-term load forecasting
Long-term load frecasting
Fault diagnosis/ Fault location
Economic dispatch
Security Assessment
Estimation of solar radiation, solar heating, etc.
Wind speed prediction
Designing ANN models
Designing ANN models follows
a number of systemic
procedures. In general, there
are five basics steps:
(1) collecting data,
(2) preprocessing data
(3) building the network
(4) train, and
(5) test performance of model
as shown in Fig.
Fig. 3. Basic flow for designing

artificial neural network model
Neural Network
Problems
Many Parameters to be set

Overfitting
long training times
...

INTRODUCTION TO NN TOOLBOX
The Neural Network Toolbox is one of the

commonly used, powerful, commercially
available software tools for the development
and design of neural networks. The software is
user-friendly, permits flexibility and convenience
in interfacing with other toolboxes in the same
environment to develop a full application.

Features
It supports a wide variety of feed-forward and
recurrent networks, including perceptrons, radial
basis networks, BP networks, learning vector
quantization (LVQ) networks, self-organizing
networks, Hopfield and Elman NWs, etc.
It also supports the activation function types of
bi-directional linear with hard limit (satlins) and
without hard limit, threshold (hard limit), signum
(symmetlic hard limit), sigmoidal (log-sigmoid),
and hyperbolic tan (tan-sigmoid).
Features
In addition, it supports unidirectional linear with
hard limit (satlins) and without hard limit, radial
basis and triangular basis, and competitive and
soft max functions. A wide variety of training and
learning algorithms are supported.

Case-1 Problem Definition
The XOR problem requires one hidden layer &
one output layer, since its NOT linearly
separable.

Design Phase

NN Toolbox
NN toolbox can be open by entering

command
>>nntool
It can also be open as shown below
It will open NN Network/ Data Manager
screen.

Getting Started

NN Network/ Data Manager

Design
Let P denote the input and T denote the
target/output.
In Matlab as per the guidelines of
implementation these are to be expressed in
the form of matrices:
P = [0 0 1 1; 0 1 0 1]
T = [0 1 1 0]
To use a network first design it, then train it
before start simulation.
We follow the steps in order to do the above:
Provide input and target data
Step-1: First we have to enter P and T to the NN
Network Manager. This is done by clicking New
Data once.
Step-2: Type P as the Name, and corresponding
matrix as the Value, select Inputs under
DataType, then confirm by clicking on Create.
Step-3: Similarly, type in T as the Name, and
corresponding matrix as the Value, select
Targets, under DataType , then confirm.
See a screen like following figures
Providing input

Providing target data

Create Network
Step-4: Now we try to create a XORNet. For

this click on New Network.
See a screen like in the following figure.
Now change all the parameters on the screen
to the values as indicated on the following
screen:

Defining XORNet network

Setting network parameters
Make Sure the parameters are as follows:

Network Type = Feedforword Backprop
Train Function = TRAINLM
Adaption Learning Function = LEARNGDM
Performance Function = MSE
Numbers of Layers = 2

Define network size
Step-5: Select Layer 1, type in 2 for the

number of neurons, & select TANSIG as
Transfer Function.
Select Layer 2, type in 1 for the number of
neurons, & select TANSIG as Transfer
Function.
Step-6: Then, confirm by hitting the Create
button, which concludes the XOR network
implementation phase.

Step-7: Now, highlight XORNet with DOUBLE
click, then click on Train button.
You will get the following screen indicated in
figure.

Training network

Defining training parameters

Step-8: On Training Info, select P as Inputs,
T as Targets.
On Training Parameters, specify:
epochs = 1000
Goal = 0.000000000000001
Max fail = 50
After, confirming all the parameters have
been specified as indented, hit Train Network.

Training process

Various Plots
Now we can get following plots

Performance plot
It should get a decaying plot (since you are
trying to minimize the error).
Training State Plot
Regression Plot

Performance plot
plots the
training,
validation, and
test
performances
given the
training
record TR
returned by
the function
train.

Training state plot
plots the training

state from a
training record
TR returned by
train.

Regression plot
Plots the linear

regression of
targets relative to
outputs.

View weights and bias
Step-8: Now to confirm the XORNet structure
and values of various Weights and Bias of the
trained network click on View on the
Network/Data Manager window.
NOTE: If for any reason, you dont get the figure
as expected, click on Delete and recreate the
XORNet as described above.
Now, the XORNet has been trained
successfully and is ready for simulation.

XORNet Structure

Network simulation
With trained network, simulation is a way of
testing on the network to see if it meets our
expectation.
Step-9: Now, create a new test data S (with a

matrix [1; 0] representing a set of two inputs)
on the NN Network Manager, follow the same
procedure indicated before (like for input P).

Step-10: HighLight XORNet again with one
click, then click on the Simulate button on the
Network Manager. Select S as the Inputs,
type in ORNet_outputsSim as Outputs, then
hit the Simulate Network button and check
the result of XORNet_outputSim on the NN
Network Manager, by clicking View.
This concludes the whole process of XOR
network design, training & simulation.

Simulated result

Case-2 Problem Definition
Power system security assessment determines
safety status of a power system in three fold
steps: system monitoring, contingency analysis
and security control.
load flow equations are required to identify the
power flows and voltage levels throughout the
transmission system
The contingencies can be single element outage
(N-1), multiple-element outage (N-2 or N-X) and
sequential outage
Here single only outage
CIARE-2012, at a time is considered47
IIT Mandi
Data Collection
The input data is obtained from offline Newton-
Raphson load flow by using the MATLAB
software. The data have matrix size [12X65].
In data collection, these input data are divided

into three groups which are train data, validate
data, and test data. The matrix size of train data
is [12X32] while the matrix size of test data
is [12X23].

Data Collection
The bus voltages V1, V2 and V3 are not included
in the train data and test data because they
are generator buses. They will be controlled by
the automatic voltage regulator (AVR) system.
In train data, there are 10 train data in secure
condition while 12 train data in insecure
condition. For test data, there are 1 test data
which is secure status while 10 test data are
insecure status.

DATA COLLECTION

DATA COLLECTION

DATA PRE-PROCESSING
After data collection, 3 data preprocessing

procedures train the ANNs more efficiently.
solve the problem of missing data,
normalize data, and
randomize data.
The missing data are replaced by the average

of neighboring values.

Normalization
Normalization procedure before presenting

the input data to the network is required
since mixing variables with large magnitudes
and small magnitudes will confuse the
learning algorithm on the importance of each
variable and may force it to finally reject the
variable with the smaller magnitude.

Building the Network
At this stage, the designer specifies the

number of hidden layers, neurons in each
layer, transfer function in each layer, training
function, weight/bias learning function, and
performance function.

TRAINING THE NETWORK
During the training process, the weights are
adjusted to make the actual outputs
(predicated) close to the target (measured)
outputs of the network.
Fourteen types of training algorithms for
developing the MLP network.
MATLAB provides built-in transfer functions
linear (purelin), Hyperbolic Tangent Sigmoid (tansig)
and Logistic Sigmoid (logsig). The graphical
illustration and mathematical form of such functions
are shown in Table 1.
TRAINING THE NETWORK
Table 1. MATLAB built-in

transfer functions
Parameter setting
Number of layers
Number of neurons
too many neurons, require more training time
Learning rate
from experience, value should be small ~0.1
Momentum term
..

TESTING THE NETWORK
The next step is to test the performance of the
developed model. At this stage unseen data are
exposed to the model.
In order to evaluate the performance of the
developed ANN models quantitatively and verify
whether there is any underlying trend in
performance of ANN models, statistical analysis
involving the coefficient of determination (R), the
root mean square error (RMSE), and the mean
bias error (MBE) are conducted.
RMSE
RMSE provides information on the short

term performance which is a measure of
the variation of predicated values around
the measured data. The lower the RMSE,
the more accurate is the estimation.

MBE
MBE is an indication of the average deviation

of the predicted values from the corresponding
measured data and can provide information on
long term performance of the models; the
lower MBE the better is the long term model
prediction.

PROGRAMMING THE NEURAL
NETWORK MODEL
ANN implementation is a process that results in
design of best ANN configuration.
Percentages of classification accuracy and mean
square error are used to represent the
performance of ANN in terms of accuracy to
predict the security level of IEEE 9 bus system.
Steps of ANN implementation is shown in the
following flow chart.

FLOW CHART

USING NN TOOLBOX
First run the MATLAB file testandtrain.m.
This file contains test data (input data) and
target data.
Name of input data is train
Name of target data is target
Network can be initialized from command prompt
as
>>nftool
or by using following step

OPENING nftool

NETWORK FITTING TOOL
Network fitting tool appears as show below

PROVIDING INPUT AND TARGET
DATA
Clicking on next button provide option to give
input and target data.

VALIDATING AND TEST DATA
Here we define training, validating, and test data.

DEFINING NETWORK SIZE
Here we set the number of neurons in the fitting
networks hidden layer.

TRAIN NETWORK

TRAINING PROCESS
By clicking on train button training process
starts.

PERFORMANCE PLOT

TRAINING STATE PLOT

REGRESSION PLOT

EVALUATE NETWORK

SAVE RESULTS

SIMULINK DIAGRAM
Following are the simulink diagram of the network.

Query

Epoch- During iterative training of a neural network ,
an Epoch is a single pass through the entire training
set, followed by testing of the verification set.
Generalization- how well will the network make
predictions for cases that are not in the training set?
Backpropagation- refers to the method for computing
the gradient of the case-wise error function with
respect to the weights for a feedforward network.
Backprop- refers to a training method that uses
backpropagation to compute the gradient.
Backprop network- is a feedforward network trained
by backpropagation.

7 Anand ANN PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

7 Anand ANN PDF

Uploaded by

Copyright:

Available Formats

Artificial Neural Networks

Introduction to Neural Network

CIARE-2012, IIT Mandi 2

Very simple principles

Figure 1 Structure of biological neuron

CIARE-2012, IIT Mandi 4

Axons connect to dendrites via synapses.

CIARE-2012, IIT Mandi 5

A neuron only fires if its input signal

CIARE-2012, IIT Mandi 6

Figure 2 Structure of artificial neuron

Mathematically, the output expression of the

CIARE-2012, IIT Mandi 8

CIARE-2012, IIT Mandi 9

Many simple neuron-like threshold switching

CIARE-2012, IIT Mandi 10

Input is high-dimensional discrete or real-

CIARE-2012, IIT Mandi 11

Fig. 3. Basic flow for designing

Many Parameters to be set

CIARE-2012, IIT Mandi 15

The Neural Network Toolbox is one of the

CIARE-2012, IIT Mandi 16

CIARE-2012, IIT Mandi 18

CIARE-2012, IIT Mandi 19

CIARE-2012, IIT Mandi 20

NN toolbox can be open by entering

CIARE-2012, IIT Mandi 21

CIARE-2012, IIT Mandi 22

CIARE-2012, IIT Mandi 23

CIARE-2012, IIT Mandi 26

CIARE-2012, IIT Mandi 27

Step-4: Now we try to create a XORNet. For

CIARE-2012, IIT Mandi 28

CIARE-2012, IIT Mandi 29

Make Sure the parameters are as follows:

CIARE-2012, IIT Mandi 30

Step-5: Select Layer 1, type in 2 for the

CIARE-2012, IIT Mandi 31

CIARE-2012, IIT Mandi 33

CIARE-2012, IIT Mandi 34

CIARE-2012, IIT Mandi 35

CIARE-2012, IIT Mandi 36

CIARE-2012, IIT Mandi 37

Now we can get following plots

CIARE-2012, IIT Mandi 38

CIARE-2012, IIT Mandi 39

plots the training

CIARE-2012, IIT Mandi 40

Plots the linear

CIARE-2012, IIT Mandi 41

CIARE-2012, IIT Mandi 42

CIARE-2012, IIT Mandi 43

Step-9: Now, create a new test data S (with a

CIARE-2012, IIT Mandi 44

CIARE-2012, IIT Mandi 45

CIARE-2012, IIT Mandi 46

In data collection, these input data are divided

CIARE-2012, IIT Mandi 48

CIARE-2012, IIT Mandi 49

CIARE-2012, IIT Mandi 50

CIARE-2012, IIT Mandi 51

After data collection, 3 data preprocessing