ARTICLE IN PRESS

Neurocomputing ] (]]]]) ]]]]]]

Contents lists available at ScienceDirect

Neurocomputing
journal homepage: www.elsevier.com/locate/neucom

Letters

Energy demand prediction using GMDH networks

Dipti Srinivasan
Department of Electrical and Computer Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117576, Singapore

a r t i c l e in fo
Article history: Received 12 March 2008 Received in revised form 1 August 2008 Accepted 5 August 2008 Communicated by T. Heskes Keywords: Energy demand Forecasting Articial neural networks Self-organizing networks Group method of data handling (GMDH) networks

abstract
The electric power industry is in transition as it moves towards a competitive and deregulated environment. In this emerging market, traditional electric utilities as well as energy traders, power pools and independent system operators (ISOs) need the capability to predict as precisely as possible how much energy their customers will use in the near future. This paper presents a medium-term energy demand forecasting system that helps utilities identify and forecast energy demand for each of the end-use consumption sector of the energy system, representing residential, industrial, commercial, non-industrial, entertainment and public lighting load. The demand forecasting system is organized and implemented in a modular fashion using high accuracy forecast models. These models are developed for each sector to account for the growth trends and seasonal effects. A comparative evaluation of various traditional and neural network-based methods for obtaining the forecast of monthly energy demand was carried out. Among the models tested, articial neural network (ANN)-based models were determined to produce better results. In particular, group method of data handling (GMDH) neural network, composed of self-organizing active neurons, was proven very effective in producing forecasts that were signicantly more accurate and less labor-intensive than traditional time-series and regression-based models. & 2008 Elsevier B.V. All rights reserved.

1. Introduction Forecasting of electrical load has always been an essential part of efcient power system planning and operation, especially short-term forecasts as it has become increasingly important since the rise of the deregulated competitive energy markets. In this emerging market, accurate load and energy forecasts, with lead times from a few hours to several weeks ahead, are critical for efcient operation of traditional utilities as well as a wide range of other market players. Inaccuracy in demand forecasts may give rise to appreciable increase in operating costs and loss of opportunities to sell excess capacity into power markets [3,13]. The medium-term loads have strong statistical seasonality that can be observed in terms of weekly, monthly and annual characteristics. The medium-term demand for a given area consists of various deterministic components reecting residential, commercial and industrial energy requirements, and a weather-dependent stochastic component. This stochastic component of load is dependent on the current weather conditions and the recent history. Any practical forecasting scheme should make full use of the strongly periodic and/or predictable components of the load before attempting to investigate the more stochastic elements [11].

E-mail address: dipti@nus.edu.sg 0925-2312/$ - see front matter & 2008 Elsevier B.V. All rights reserved. doi:10.1016/j.neucom.2008.08.006

A comprehensive review of the literature [12,1,14,6,9,10, 7,16,15,17,18,2,5] reveals the dominant electricity demand forecasting methodologies as time-series modeling, load prole analysis, pattern recognition, group handling models, expert systems, articial neural networks (ANN) and hybrid fuzzy neural approaches. The most commonly used methodology; time-series analysis relies on explicit descriptive, stochastic or other models of processes that describe the observed energy demand data [11]. Usually, the parameters of a standard model like the autoregressive moving average (ARMA) technique are derived from the autocorrelation of the time series [8]. Although ANN methods have recently had considerable success, they are often criticized for their lack of transparency and shift of emphasis to the training of the network [5]. This paper provides a comparative evaluation of various methods for obtaining the forecast of monthly energy demand. Four commonly used conventional methods that can determine in a consistent fashion the short-term and long-term characteristics of a time series [5] were tested. Multi-layer perceptron (MLP) neural network, which was trained using backpropagation learning algorithm, and a relatively unexplored model, group method of data handling (GMDH) neural network [4,8] were also implemented. Recognizing that in modern electric systems, the complex and varied needs of utility customers can no longer be adequately represented by forecasts of peak demand and total energy use

Please cite this article as: D. Srinivasan, Energy demand prediction using GMDH networks, Neurocomputing (2008), doi:10.1016/ j.neucom.2008.08.006

ARTICLE IN PRESS
2 D. Srinivasan / Neurocomputing ] (]]]]) ]]]]]]

alone, the proposed system has been developed to forecast the expected demand of each category of consumer. Total demand was considered to be composed of six categories of consumers representing residential, commercial, industrial, non-industrial, entertainment and public lighting loads. A key advantage of this method is that as well as providing overall energy demand forecast; it allows the user to obtain forecasts for individual group of customers. Of the various models tested, those based on selforganizing networks with active neurons (GMDH) have proven very effective in providing forecasts that are signicantly more accurate and less labor-intensive than traditional time-series analysis or regression-based systems. This paper has, for the rst time, applied the relatively unexplored GMDH neural networks for electricity demand forecasting. An evaluation of the prediction accuracy of these neural network-based models and comparison with methods used by other electricity utilities is presented in the paper.

such as manufacturing, agriculture, mining, sheries and construction. Non-industrial sector includes hospitals, educational institutions, religious organizations, as well as a wide range of facilities that would not be considered commercial in a traditional economic sense. Fig. 1 shows the percentage share of each of these sectors in the total load/energy demand. 2.2. Data processing As the number of days in a month varies from month to month, to avoid false patterns created by the resulting inconsistency, the data was normalized by dividing the total monthly energy demand by the number of days within that month. The normalized series is given by X0 t Xt Dt (1)

2. Characteristics of load series The data used in this project for monthly energy demand forecast was obtained from an electricity utility in Asia. The data consisted of monthly energy sales units to the six categories of consumers and total energy sales units for a period of 78 months. Discussions were carried out with current and past electricity personnel concerning the types of load forecasting requirements envisaged by the power utility. Appropriate load and meteorological information was acquired and processed for use in the load forecasting models developed. 2.1. Description of data series In this paper, the monthly energy demand is forecasted for each of the six individual load series representing commercial, industrial, residential, non-industrial, entertainment and public lighting sectors that make up the total demand. The forecast obtained for individual series is aggregated to obtain the total energy demand forecast. The commercial sector consists of business establishments and other organizations that provide services. This sector includes ofce and nancial establishments, and service businesses, such as retail and wholesale stores and hotels. Industrial sector load comprises all industries, including the goods-producing industries
45

where Xt is the total monthly energy demand, X0 t is the normalized demand and Dt is the number of days in the tth month. Fig. 2(af) shows the resulting normalized series for each category of user for a period of 66 months. The remaining 12 months of data was used for testing and validation of the models. From Fig. 2, it can be seen that there is an year-on-year increase in the average monthly energy demand observed by all categories of consumers. However, the characteristics of each of the above six series is unique. The residential, commercial and industrial load series show a period component. The stochastic component of load is dependent on the current weather conditions and the recent history. This recurrent statistical pattern produces most of the load variability. A smaller component (variously estimated at 1020%) in load demand is related to the meteorological conditions, with maximum demand typically occurring when extreme temperatures during summer daytime result in high airconditioning loads. This effect is most pronounced for the residential sector. The relationship with meteorological variables can be quite non-linear and may produce unexpected changes.

3. Model development This research involved investigation using this historical database of the efcacy of a number of models considered useful for the forecasting of electricity demand [11,12,1,14,6,9,10,7,16,1 5,17,18,2,5] including double moving average, single exponential smoothing, double exponential smoothing, ARMA and multi-layer backpropagation neural network. The sixth model, GMDH model, made up of self-organizing active neurons, is new technology implemented for this work. Six forecasting networks, one for each category of consumer, was developed for each of these techniques. A seventh network was created for obtaining the total forecast demand for the entire region. 3.1. Time-series models

cial mer Com 40

35 % of total demand 30 25 20 15 10 5 0

l stria Indu

tial iden Res l stria indu on_ N ing ight ent lic L inm Pub erta Ent

The rst four of the models listed above were built in the following fashion [11]: (1) Model identication: In this step, tentative models are identied. Determination of the need for power transformation and differencing (to render the series stationary in the mean) is rst evaluated. For ARMA model, plots of the autocorrelation functions (ACF) and partial autocorrelation functions (PACF) of the possibly transformed series are examined to assist in determining the order of the AR and MA components.

Fig. 1. Division of demand among various sectors, expressed as a percentage of total demand.

Please cite this article as: D. Srinivasan, Energy demand prediction using GMDH networks, Neurocomputing (2008), doi:10.1016/ j.neucom.2008.08.006

ARTICLE IN PRESS
D. Srinivasan / Neurocomputing ] (]]]]) ]]]]]] 3

3.5 3 2.5 2 1.5 1 0.5 0

11

21

31 41 month

51

61

7 6 5 4 3 2 1 0

GWhr/day

GWhr/day

11

21

31 41 month

51

61

7 6 5 4 3 2 1 0

2 GWhr/day 1 11 21 31 41 month 51 61 1.5 1 0.5 0 1 11 21 31 41 month 51 61

GWhr/day

0.1 GWhr/day GWhr/day 1 11 21 31 41 51 month 61 0.08 0.06 0.04 0.02 0

0.2 0.15 0.1 0.05 0 1 11 21 31 41 51 month 61

Fig. 2. Monthly average energy demand for various categories of consumers. (a) Residential series load prole, (b) commercial series load prole, (c) industrial series load prole, (d) non-industrial series load prole, (e) entertainment series load prole, (f) public lighting series load prole.

(2) Parameter estimation: Following selection of a potential model(s), estimates of the parameters are calculated. Standard errors are also computed, and parameters judged to not be signicantly different from zero are dropped. The remaining parameters are then re-estimated. (3) Model diagnostic checking: With a tentative model selected and parameters estimated, the adequacy of the model must be assessed to determine if model assumptions are met. Several model selection criteria have been developed to assist in model selection [11]. In this analysis, we compared competing models using the mean square error (MSE), which measures the average one-step-ahead prediction error; and the unbiased residual variance, equal to the residual sum of squares divided by degrees of freedom. Once the best model had been selected, it was used for forecasting the energy demand. 3.2. Articial neural network-based models ANNs are suitable for many tasks in pattern recognition and machine learning. Many successful applications of ANNs for electricity demand forecasting have been reported in the literature [18]. Unlike conventional techniques for time-series analysis and prediction, an ANN needs little information about the time-series data and can be applied to a broad range of problems. Two different types of neural network models have been developed for demand forecasting. The rst one is a multi-layer feedforward network, which has been proven to be very effective for demand forecasting by many researchers. Several experiments were conducted to determine the optimum number of hidden

layers and the number of hidden neurons. The network with one hidden layer was found to best capture the characteristics of the load series. The network was trained using backpropagation learning algorithm. The learning rate used was 0.1 and the momentum rate was 0.5. The second is a relatively unexplored neural network known as GMDH network. In these networks, the most important input variables, number of layers, neurons in hidden layers and optimal model structure are determined automatically. The network is thus composed of active neurons that organize themselves. The GMDH network learns in an inductive way and tries to build a function (called a polynomial model) which would result in the minimum error between the predicted value and expected output. The majority of GMDH networks use regression analysis for solving the problem [4]. The rst step is to decide the type of polynomial that regression should nd. General connection between input and output variables can be expressed by Volterra functional series, discrete analog of which is KolmogorovGabor polynomial: y a0
M X i1

ai x i

M M XX i1 j1

aij xi xj

M M M XXX i1 j1 k1

aijk xi xj xk

(2)

where X(x1,x2, y, xM) is the input variables vector and A(a1,a2, y, aM) is the vector of coefcients or weights. Components of the input vector X can be independent variables, functional forms or nite difference terms. Other nonlinear reference functions, such as difference, probabilistic, harmonic, logistic can also be used. The method simultaneously nds the structure of model and the dependence of modeled system output on the values of most signicant inputs of the

Please cite this article as: D. Srinivasan, Energy demand prediction using GMDH networks, Neurocomputing (2008), doi:10.1016/ j.neucom.2008.08.006

ARTICLE IN PRESS
4 D. Srinivasan / Neurocomputing ] (]]]]) ]]]]]]

system. In the original GMDH algorithm [8], these functions are linear or second-order polynomial functions of two or three variables, such as f 1 xi ; xj a0 a1 xi a2 xj f 2 xi ; xj f 1 xi ; xj a3 xi xj a4 x2 a5 x2 i j (3) (4)

the forecasting accuracy of which is higher and structure simpler than those of a usual fully connected MLP-type neural network model described earlier.

4. Implementation and results This section contains the results obtained from the forecasting models described above. Actual data from an electric utility was used in this project. The data consisted of monthly energy sales units to the six categories of consumers, and total energy sales units for a period of 78 months. Out of this, the data for the rst 66 months was used for training of forecasting models, and the remaining data was used for evaluating their forecasting performance. Mean absolute percentage error (MAPE), which is a commonly used indicator of forecasting performance [5], was used for this purpose. Six separate forecasting models were developed for each forecasting technique used in this paper. The forecasting results from different models are summarized and tabulated in Table 1. The results show that ARMA models produce better results than those using Single and double exponential smoothing methods, or double moving average. The self-organizing network using active neurons (GMDH) produces the best results. The total monthly energy demand was forecasted using two approaches. The rst was to independently forecast the total demand using a seventh model. The second approach used a

The next step is to construct a linear combination of all of the polynomial terms with variable coefcients. The algorithm determines values of these coefcients by minimizing the squared sum (overall samples) of differences between sample outputs and model predictions. GMDH works by building successive layers with complex links (or connections) that are the individual terms of a polynomial. These polynomial terms are created by using linear and non-linear regression. The initial layer is simply the input layer. The rst layer created is made by computing regressions of the input variables and then choosing the best ones. The second layer is created by computing regressions of the values in the rst layer along with the input variables. This means that the algorithm essentially builds polynomials of polynomials. Again, only the best are chosen by the algorithm. These are called survivors. This process continues until a pre-specied selection criterion is met. In this paper, experiments were carried out to nd the right parameters for the GMDH forecasting networks. For each consumer sector of the energy demand series, which is a short, noisy data series, an optimal simplied GMDH model was found,

Table 1 Comparison of results for various end-use sectors End-use sector Forecasting method Double moving average (%) Residential Commercial Entertainment Public lighting Industrial Non-industrial 6.00 2.58 5.16 1.82 6.14 6.06 Single exponential smoothing (%) 5.41 2.46 4.80 1.59 5.66 6.20 Double exponential smoothing (%) 4.96 3.07 4.70 1.61 4.68 5.30 ARMA (%) Backpropagation neural network (%) 2.54 2.56 3.86 1.34 3.02 3.2 GMDH network (%)

3.74 3.11 3.90 1.86 3.51 3.7

1.74 2.04 2.69 0.83 2.65 2.64

Actual demand Single Exponential smoothing ARMA GMDH network

Double moving average Double exponential smoothing ANN

20 Normalized demand (GWhr/day) 19 18 17 16 15 14 1 2 3 4 5 6 7 8 9 10 11 12 month

Fig. 3. Forecast of total energy demand by various forecasting models.

Please cite this article as: D. Srinivasan, Energy demand prediction using GMDH networks, Neurocomputing (2008), doi:10.1016/ j.neucom.2008.08.006

ARTICLE IN PRESS
D. Srinivasan / Neurocomputing ] (]]]]) ]]]]]] 5

Table 2 Aggregated individual series forecast v/s forecast of total demand Forecasting method Combined forecast from individual series (% MAPE) 2.75 2.86 2.79 2.44 2.29 1.52 Forecast of total load series (% MAPE)

Double moving average Single exponential smoothing Double exponential smoothing ARMA Articial neural network GMDH neural network

2.98 3.23 3.10 2.58 2.63 1.73

traditional time-series analysis or regression-based systems, and supervised neural network. A key advantage of the forecast method presented in this paper is that as well as providing overall energy demand forecast; it allows the user to obtain forecasts for individual group of customers. The paper also shows that the forecasting results from the aggregated scheme, using forecast combination from six end-use categories, are superior compared to those from the model that treats the total demand as one time series. References
[1] R.E. Abdel-Aal, Modeling and forecasting electric daily peak loads using abductive networks, Int. J. Electr. Power Energy Systems 28 (2) (2006) 133141. [2] D. Benaouda, F. Murtagh, J.L. Starck, et al., Wavelet-based nonlinear multiscale decomposition model for electricity load forecasting, Neurocomputing 70 (13) (2006) 139154. [3] D.W. Bunn, Forecasting loads and prices in competitive power markets, Proc. IEEE 88 (2) (2000) 163169. [4] S.J. Farlow, Self-organizing method in modeling: GMDH type algorithms, Statistics: Textbooks and Monographs (1984). [5] H.S. Hippert, C.E. Pedreira, R.C. Souza, Neural networks for short-term load forecasting: a review and evaluation, IEEE Trans. Power Systems 16 (1) (2001) 4455. [6] Shyh-Jier Huang, Kuang-Rong Shih, Application of a fuzzy model for shortterm load forecast with group method of data handling enhancement, Int. J. Electr. Power Energy Systems 24 (8) (2002) 631638. [7] Syed M. Islam, Saleh M. Al-Alawi, Forecasting monthly electric load and energy for a fast-growing utility using an articial neural network, Electr. Power Systems Res. 34 (1) (1995) 19. [8] A.G. Ivakhnenko, G.A. Ivakhnenko, J.A. Muller, Self-organisation of neuronets with active neurons, Pattern Recogn. Image Anal. 4 (2) (1994) 177188. [9] Nicolaos B. Karayiannis, Mahesh Balasubramanian, Heidar A. Malki, Shortterm electric power load forecasting based on cosine radial basis function neural networks: An experimental evaluation: Research Articles, Int. J. Intell. Syst. 20 (6) (2005). [10] H.A. Malki, N.B. Karayiannis, M. Balasubramanian, Short-term electric power load forecasting using feedforward neural networks, Expert Systems 21 (3) (2004) 157167, (11). [11] Douglas C. Montegomery, L.A. Johnson, J.S. Gardiner, Forecasting and Time Series Analysis, McGraw-Hill, New York, 1990. [12] A.D. Papalexopoulas, T.C. Hesterberg, A regression-based approach to shortterm system load forecasting, IEEE Trans. Power Systems 5 (4) (1990) 15351547. [13] D.K. Ranaweera, G.G. Karady, R.G. Farmer, Economic impact analysis of load forecasting, IEEE Trans. Power Systems 12 (3) (1997) 13881392. [14] Setsuo Sagara, Junichi Murata, Monthly load forecasting using GMDH, Electr. Eng. Japan 106 (4) (1986) 125132. [15] D. Srinivasan, M.A. Lee, Survey of hybrid fuzzy neural approaches to electric load forecasting, in: Proceedings of the IEEE Conference on System, Man and Cybernetics, Vancouver, Canada, 1995, pp. 40044008. [16] D. Srinivasan, S.S. TAN, C.S. Chang, E.K. Chan, Parallel neural network-fuzzy expert system strategy for short-term Load forecasting: system implementation and performance evaluation, IEEE Trans. Power Systems 14 (3) (1999) 11001106. [17] H.K. Temras, M.M.A. Salama, A.Y. Chikhani, Review of electric load forecasting methods, in: IEEE 1997 Canadian Conference on Electrical and Computer Engineering, vol. 1, 1997, pp. 289292. [18] M. Ulagammai, P. Venkatesh, P.S. Kannan, et al., Application of bacterial foraging technique trained articial and wavelet neural networks in load forecasting, Neurocomputing 70 (1618) (2007) 26592667. Dipti Srinivasan (M) obtained her M.Eng. and Ph.D. degrees in EE from the National University of Singapore (NUS) in 1991 and 1994, respectively. She worked at the University of California at Berkeleys CS Division as a postdoctoral researcher from 1994 to 1995. In June 1995, she joined the faculty of the EE department at the NUS, where she is an Assistant Professor. Her research interests are in the application of soft computing techniques in power system operation, economics and control.

linear combination of forecasts from the six end-use categories to forecast the total demand as an aggregated forecast combination. Fig. 3 shows the plot of forecasted total demand from various models. Table 2 compares the results from these two methods. Although, the MAPE from individual series are high (Table 1), when combined to produce aggregated forecast (Table 2), the errors for most of the data points cancel out, thus reducing the overall error. Over the past few years, several neural network-based methods have been effectively applied for electric load forecasting [5]. However, the most commonly used ANN for load forecasting, the multi-layer backpropagation neural network, requires the user to estimate the model structure by choosing the number of layers and the number and transfer functions of nodes. The difculty in estimating the optimal model structure is eliminated by GMDHtype neural networks that adaptively synthesize the network architecture directly from data sampling. The model selection is done automatically based on its ability to accurately forecast the future demand. In addition, the results presented in this paper show that the GMDH models outperform traditional forecasting techniques and supervised neural network in terms of the accuracy of the forecast obtained. The forecast combination pursued in this paper improves the overall forecasting accuracy and minimizes the variance of the forecast errors. An additional advantage of these networks is that in contrast to supervised neural networks, the GMDH algorithm is able to construct the network in a relatively short time on the basis of extremely short samples. The sector-wise forecasting approach used in this paper can be used to show the total demand and provide information on the structure of energy consumption in simulations for demandmanagement programs, long-term forecasting and customer identication. This approach can be further extended for spatial forecasting where individual growth trends of these unique series can be projected and utilized to forecast the future demand.

5. Conclusion This paper has presented a medium-term energy demand forecasting system for predicting the demand for each of the enduse consumption sector of the energy system using GMDH. The GMDH method, that uses self-organizing networks of active neurons, has resulted in substantial forecast accuracy improvement for all categories of consumers. The paper compares the results from the GMDH network with those obtained from ve commonly used load forecasting techniques. The GMDH results are signicantly more accurate and less labor-intensive than

Please cite this article as: D. Srinivasan, Energy demand prediction using GMDH networks, Neurocomputing (2008), doi:10.1016/ j.neucom.2008.08.006