Short-term Load Forecasting using Artificial Neural Network based on Particle Swarm Optimization Algorithm

Bashir, Z.A. El-Hawary, M.E.

Department of Electrical & Computer Engineering Dalhousie University Halifax, Nova Scotia, Canada B3J 2X4

Abstract The quality of short term load forecasting can improve the efficiency of planning and operation of electric utilities. In this work, for determining the competitive learning model, the particle swarm optimization (PSO) technique is used as a training algorithm to adjust the weights of the artificial neural networks (ANNs) model to predict hourly loads. The feature of PSO is to fly potential solutions through hyperspace, accelerating toward better solutions. Thus the training phase should result in obtaining the weights configuration associated with the minimum output error. The historical load and weather information were trained and tested over a period of one season through two years. Generalized error estimation is done by using the reverse part of the data as a test set. The results were compared with conventional back-propagation algorithm and yielded encouraging results. Keywords - Predicted hourly load, Neural networks, Particle swarm optimization algorithm.

I. INTRODUCTION The quality of the short-term hourly load forecasting with lead times ranging from one hour to several days ahead has a significant impact on the efficiency of operation of any electrical utility. Many operational decisions such as economic scheduling of the generating capacity, scheduling of fuel purchase and system security assessment are based on such forecasts. Many statistical methods such as the autoregressive moving average, linear regression, stochastic time series, general exponential smoothing etc. have been used to predict the hourly load [1]. However, these techniques can not properly represent the complex nonlinear relationships between the consumed load and a series of stochastic factors such as time of day, special events and correlation properties of weather condition which are highly affecting power load curve. For the past several years, ANNs methods received a great deal of attention and are now being proposed as powerful computational tools to solve the load forecasting problem and are able to give better performance in dealing with the nonlinearity and other difficulties in modeling time series forecasting [2],[3]. ANNs is able to extract an implicit nonlinear relationship among input variables (higher-order combination input) by learning from training data. Using

trained supervised networks requires a measure of the discrepancy between the networks output value and desired value. The difference between these values results in an error function. The back propagation algorithm is one of the most popular techniques used to train neural network parameters (weights and biases) based on gradient descent or conjugate gradient decent method by calculating the partial derivative of the performance with respect to the weights and biases. However, using this technique adds some constraints to a neural network where the transfer relation of each neuron must be differentiable. Moreover, it has been proven that this gradient information technique is slow to train and sensitive to the initial guess which could possibly be trapped in a local minimum [4]. The advantage of using PSO algorithm over other techniques it can be computationally inexpensive, easily implemented, and does not require gradient information of the objective function but only its values. Therefore, the particle swarm optimization algorithm is applied, in this work, to the neural network as a training phase to obtain a set of weights and biases that will minimize the error function. Weights are progressively updated until the convergence criterion is satisfied. Thus the error function is the objective function to be minimized by PSO in our problem. Weather components such as temperature, wind speed, cloud cover play a major rule in short-term load forecasting by changing daily load curves. The characteristics of these components are reflected in the load requirements although some are affected more than others. Therefore temperature, wind speed, and cloud cover are used as variables with historical load date to improve predict hourly load. In this work, better results are obtained neural networks that significantly influence the overall performance such as determination of the network topology, the selection of data pattern for network training and the selection of appropriate and efficient training algorithm parameters. Predefined parameters of PSO, i.e. numerical coefficients, swarm size, and topology are also investigated.

272 0840-7789/07/$25.00 2007 Crown Copyright

II. NEURAL NETWORK TOPOLOGY A neural network consists mainly of three layers: input layer, hidden layer and output layer. These layers are arranged in some way to have a multilayer feed-forward structure. A general network model consists of simple processing elements called neurons with adjustable parameters called weights. These neurons are arranged in a distinct layered topology and perform a biased weighted sum of their inputs and pass this activation level through a transfer relation (tan sigmoid function) to produce their output. Thus, the parameters of data flow from the input neurons, forwards through any hidden neurons, eventually reaching the output neurons. All of the layers are fully interconnected with each other by weights as shown in Fig. 1.
Input Layer Hidden Layers Output Layer Neuron

solution. Each potential solution is represented as a particle in population called swarm. The social sharing of information among the particles of a population may provide an evolutionary advantage which is the main idea behind the development of the PSO algorithm [6]. Each particle is flown through the multi-dimension search space with random and adaptable velocity in order to find the lower function values (global minimum) based on function minimization task. Particles are manipulated according to (1) and (2) where each particle tries to adjust its velocity according to best positions ever visited that is stored in its memory (personal best, pbest) and according to best positions currently found for all swarm (global best, gbest), trying to search for better position. The later process takes place when particles communicate with each other and distribute their information among each other.
vi (t + 1) = wv i (t ) + 1 * r1[ pbest xi (t )] + 2 * r2 [ gbest xi (t )] i = 1,2,...... N

X1

(2) (3)

xi (t + 1) = xi (t ) + vi (t + 1)

oi
Xm
Connection weights (

Where w is an inertia weight, provides balance between the local and global exploration. vi & v i (t + 1) are current velocity and updated velocity for each iteration respectively. 1 and 2 are positive numbers, they represent cognitive and social components respectively and control the movement of particle at each iteration. r1 and r2 are uniform distribution numbers in to the range [0 , 1]. xi (t) and

w ij

Figure. 1. Multi-layer feed-forward neural network architecture

A neural network uses a learning function to modify the variable connection weights at the inputs of each processing element according to some neural based algorithm. Multiple layers of neurons with nonlinear activation functions allow the network to learn nonlinear and linear relationships between the input and output of the network. The training process in our network requires a set of examples of proper network behavior. Hence, the network can be trained for function approximation. Through the training process, the weights and biases are taken to be a dimension in space and updated iteratively to minimize the error function (error surface) to find the lowest point in this multi-dimensional surface. The performance function of the neural network is normally chosen to be the mean squared error for each pattern on the training set, (1).

xi (t + 1) are current position and updated position for each iteration respectively. N is the number of particles.
IV. INPUT DATA SELECTION It was observed from [7] that load patterns for working days were similar while those for weekends were different from weekdays. Therefore, loads and weather information data for weekdays and weekends were forecasted separately. The data was collected from Nova Scotia Power & Weather Information Center, Halifax. The network input data for both models are normalized such that they fall within the range [0, 1], to avoid convergence problems during the training process. As shown in Table 1, the networks are trained with 12 neurons as input data, 6 & 4 neurons for hidden layers, and 1 neuron for an output layer. There is no such general rule that can be followed to determine the number of neurons in the hidden layer. Hence, the optimum number of neurons in the hidden layers has been determined based on trial and error experimentation. The data is trained for 87 days (workdays for winter 2003) for 24 hours of each day while the testing date is done by using the same time period for winter 2004. V. TRAINING ALGORITHM The perception training algorithm is a form of supervised learning algorithm where the weights and biases are updated to

Ep =

1 N
th

(t
i =1

pi

o pi ) 2

(1)

Where t pi is the target at i pattern, o pi is a networks output at i pattern and N is number of neural network pattern. III. PARTICLE SWARM OPTIMIZATIOM Particle swarm optimization is an evolutionary computation technique proposed by Kennedy and Eberhart in 1995 [5]. This technique is a stochastic optimization method which was developed through simulation of bird flocking in two dimensional space. PSO has a population with random search
th

273

reduce errors whenever the network output does not match the desired values. The process in this application was trained by using TRAINPSO for NNs PSO model [8]. The modified PSO parameters which have been chosen are: The optimal size of swarm was 30 (starts from min. 20 to max.50), the search space range was [-100 100], the acceleration value (1 = 2 = 2) , and the inertia weight was gradually decreased from 0.9 to 0.4. TRAINLM training function was used for NNs _ PB model. TRAINLM is a training function that updates weights and biases values according to Levenberg-Marquardt optimization algorithm [9]. Initialization weights and biases for both models were done by using a layer initialization function according to Nguyen-Widrow [9]. Training functions repeatedly apply a set of input vectors to a network, updating the network each time, until stopping criteria is met. Stopping criteria consists of a maximum number of epochs, A number of epochs is used for training ANNs was 1000 epochs to meet the performance goal when the maximum error goal was 1e-03. VI. SIMULATION RESULTS The network models are implemented in MATLAB 7. The numerical simulations were run on a portable/laptop PC. (Centrino Duo, 1.8 GHz, 1 GB RAM). The training time for NNs _ PSO model was relatively shorter than

TABLE I. NEURAL NETWORK ARCHITECTURES FOR NN_PB and NN_PSO Layer Input Input Input Input Hidden Output Data Type Load for current hour and previous two hours Temp. for current hour and previous two hours Wind for current hour and previous two hours Cloud for current hour and previous two hours N/A Normalized Load No.& type of T.F./Neuron 3 3 3 3 6,4 Tan-sigmoid 1 Purelin

NNPSO,performance(MSE) = 0.0021882 & Iter.time = 0.063 0.8 0.75 Normalized actual and predicted load 0.7 0.65 0.6 0.55 0.5 0.45 0.4 0.35 forecast load A ctual load

NNs _ BP model. Tan-sigmoid function as in (4) has been

chosen in the hidden layer and purelin (linear) transfer function in the output layer. This is a useful structure for function approximation problem [9]. Forecasting was tested to the neural networks based on designed network architecture described in Table 1. A plot of actual hourly load curve for winter season is compared to forecast values for the same period is shown in Fig. 2. The average percentage error APE (5) of one arbitrary day for two models is shown in Table 2. The actual and forecasted daily load curves for two models are represented in Fig 3.

10

20

30

40 50 60 Days for winter 2004

70

80

90

Figure. 2. Forecasting load for one hour in winter season

tan sig (k ) =

2 (1 + exp(2 * k )) 1

(4)

950 Actual load 900 NNs-PSO NNs- PB

Where k is the neurons activation function

i =1

Load MW

w x .
ij i

850

800

750

APE =

[
i =1 j =1

ti o j ti

] 100 0 0
(5)

700

650

Where n is the number of testing sample data, ti and oi are the desired data and a networks output respectively.

600

10 15 Daily load curve

20

25

Figure. 3. The arbitrary daily load curve for both models

274

TABLE II.. AVERAGE PERCENTAGE ERROR Average percentage error %

Hour 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

NN-BP 2.76 2.40 2.58 1.86 2.34 2.13 2.07 2.06 2.26 2.42 1.79 1.89 1.65 1.23 -1.79 1.02 0.83 0.83 2.18 1.28 -1.52 -1.73 2.62 1.33

NN-PSO 1.91 1.63 1.58 1.78 1.68 1.53 1.22 1.48 1.37 1.21 1.23 1.25 0.56 -0.66 -0.94 -083 -1.69 -1.01 1.06 -2.32 0.99 -1.12 1.44 1.08

load forecasting. PSO shows its ability to minimize the error function with high efficiency. In this study, it was found that selecting proper values of network structure, input data, and PSO parameters are major factors that affect the performance accuracy of the network. It shows also that using weather condition data can positively influence the training and testing results. We have checked our prediction by doing linear regression test to our results. Based on the PSO characteristics that are represented by parallel search technique and global optimization gives the training neural network faster convergence. The correlation coefficient (R-value = 0.86), measures of how well the variation in the output is explained by the target as shown in Fig. 4. The number is close to one, which indicates a good fit. REREFENCES
[1] I. Moghram, S. Rahman, Analysis and Evaluation of Five Short-Term Load Forecasting Techniques, Proceedings of the IEEE Transaction on Power Systems, pp. 1484-1491, Vol. 4, No. 4, October 1989. B. S. Kermanshahi, C. H. Poskar, G. Swift, W. Buhr and A. Silik Artificial Neural Network for Forecasting Daily Loads of A Canadian Electric Utility, Proceedings of the IEEE, pp. 302-307, 1993. T. W. S. Chow, C. T. Leung Neural Networks Based Short Term Load Forecasting using weather Compensation, Proceedings of the IEEE, pp. 1736-1742, 1996. D. E. Rumelhart, G. E. Hinton and R. J. Williams, Learning representation by back propagating errors, Nature, pp. 533-536. 1986. Kennedy and R. C. Eberhart, Particle Swarm Optimization,Proceeding of the IEEE international conference on neural networks, pp. 1942-1948, volume 4, 1995. R. C. Ebehart, P.K. Simpson, R. W. Dobbins, Compuational Intelligence,PC Tools, Academic Press Professional, 1996. Bashir, Z. and M.E. El-Hawary, Short Term Load Forecasting by using Wavelet Neural Networks, proceeding of IEEE Canadian Conference on Electrical and Computer Engineering, pp. 163-166, 2000. B. Birage, Particle Swarm Optimization ToolBox, ,2005. Howard Demuth and Mark Beal, Neural Network ToolBox Users Guide, The Math Works Inc. 1998.

[2]

[3]

[4] [5]

[6] [7]

[8] [9]
Best Linear Fit: A = (1.01) T + (-0.0255) 0.8 0.75 0.7 0.65 0.6 A 0.55 0.5 0.45 0.4 0.35 0.4 R = 0.86 Data Points Best Linear Fit A=T

0.45

0.5

0.55

0.6 T

0.65

0.7

0.75

0.8

Figure. 4. The correlation coefficient between output and target

VII.

CONCLUSIONS

This work investigated the ability of neural network to predict the hourly load using particle swarm optimization algorithm to adjust weights and biases in order to minimize the error objective function of a neural network to obtain a short term

275