2009 (1)

 (31)      _ 

    
 
Tishreen University Journal for Research and Scientific Studies - Engineering Sciences Series Vol. (31) No. (1) 2009

  ! "# $  
   

*

% ! &  & 
**
 ()  
*** * +

(2009/2/1 34  (2 %1 .2008 / 10 / 23 , - .

)

567

!" !#$% &$ '(   
 

    
 

  
%
 #$%
'+ )



 ,  * 
 %
)
*+   ' ' 
 %
!
/   ,

 6 7
 
 8
 ( .$+
/+


0 1  /"2 1 / %.
3 4
% 

 %

") ' 
 
&$ 9"
: ;<
   %

.



:  , %
)
  &$

.
 ? @
+  $% ?.$+
)
 @
+ ? 
 : 
8


*
. 9%+)9 
 9
 
    93: 

  ; % 9 #
 +
!
. 9%+)9 
 9
 
    93: 

  ; % 9<  **
9? ?
9?
 
   93: 

  ; % 9 (
)
#
 =
> ***
. 9%+)

213
 A    ( 2 <
2 &$  %






 ,  * 


2009 (1)  (31)      _ 

    
 
Tishreen University Journal for Research and Scientific Studies - Engineering Sciences Series Vol. (31) No. (1) 2009

Studying the Effect of the Neural Network's Parameters

Change on the Network Performance

Dr. Kinda Abo Khasem*

Dr. Belal Sheha**
Monzer Skef***

(Received 23 / 10 / 2008. Accepted 1 / 2 / 2009)

ABSTRACT

This research aims at studying and designing a three-layer neural network, capable of
dealing with massive data, relying on the back-propagation error method. It also explores
the effect of parameters-change (learning step, number of nodes, type of activation
function for a few different input signals) and the major impact, caused by this change on
the neural network performance.
The results of these experiments have demonstrated the extreme sensitivity of the
designed network which depends on the back-propagation mechanism to change these
parameters.

Key words: neural networks, back propagation algorithm, learning algorithm, layers.

*
Associate Professor, Department of Computer and Automatic Control, Faculty of Mechanical and
Electrical Engineering, Tishreen University, Lattakia, Syria.
** Assistant Professor, Department of Computer and Automatic Control, Faculty of Mechanical and
Electrical Engineering, Tishreen University, Lattakia, Syria.
*** Postgraduate Student, Department of Computer and Automatic Control, Faculty of Mechanical
and Electrical Engineering, Tishreen University, Lattakia, Syria.

214
 Tishreen University Journal. Eng. Sciences Series 2009 (1) %
(31) $"
 
$%

1  %" $"

:  @
!$
 !B 
9!8
 "  /  '  /
 &$ C  %


/ 
. D
..... C @ .
(%
C
)  

Warren McCulloch /!2  B

1

  %2   %

$  
 E

2
F$2   2    $% 2
  1  %


 (  0 &$  1 $
Walter Pitts 
!  ! ! 12 &0 1  ' /( 1
  %
 /%

 .Perceptron ' "
 
 $
Back !%
!)
 @
+
+

:  . [1] '
 . /%
&$ '(  
!+* !  Feed-Forward 4
1

'%    9  Propagation algorithm
! 8
 @
+
 %
)
 @
+ 10 .   G+ /+ 3  %
$
 
.  %


 $%  $  1I
H 
9!J
 
 $%.
G+
 ( 1  *+
/   1
@8
/ % &$  @
+
: ' %
*+
 ,  .  % 9$ 72 K1
@8
&0  *+
F 9  
: 9$ . [2]G+$  
/!% 
1
@8
&0 AB 1* G+
  .+

1  1
@8
/% 
6 ( &0 
!)
!
&!2 
 /+
 &    $%
: % [3] 1
@8
:  *+
 ,
.  @
+
: %

: B4 ;!  ;!

! ,  *! 
  
 %


 % 
/
@
0  
A L+$
! .  ( !.$+
/+!


0 1  /"2 1 / %.
3 4 %
  $%
'+ )




!" 0 ! !

! 2 1! ..$+
)
 @
+ &$ )  "
C+8
&$ 9$,

.1 *+ /(2 &$ /$  %

$   





: C   @>

: 


 

  
. %


  %" 
 ?
.%
)
 @
+ 
?
. %

/ 



2  ,  * ?
.
")
J  

;< ( ?

:
@  D ! $
:  


+
  
C
2   9$
.[3] E
/  .  '   .+ 
  1 .M    C ?
. BP %
)
 @
+   %

9   @
+ ?
.(error)*+
3   %

C
2   3 ?

215
 A    ( 2 <
2 &$  %






 ,  * 


: 34  7 E7

3 
F E7
: N B /"2 1 %
)
 @
+ 1 .E

'%  %


 
(1) /
 NB   
 G+ /+ 3  %
$
  +*  $%

f1 (e1 )

W11 W14

X W12 f 2 (e 2 ) W24 f 4 (e 4 ) Y

W13 W34
f3 (e3 )

@> &  $#  (1)(

   
'
. /+


0 1
@8

% ; &8
'
1  1 9 1 /
/+!
x /   1 / L+
 
3 f ( ) /    1%
  3 C  

!

 1   
1
@8
w /  6 %
1
@8 /+
C
" 4" /  e G+
y
.G+
 /+

O% 3 B%
 9   $ / % . ( X 1 , X 2 ,.....)
 % " 9  G 1  %



.[4] %

1
@2  ( / %
: $  
 @
+

+ 9 / %

0  (   3  $
: % 9 
" 1 /+
'0  , 2  $% '+ /
.(2) /
 1    
1  /  1 / G+

Back Propagation)3 

F
:6  /  
  $%$  @
+ 
G!+
12   H $+
 /  12 /+
 7, 

$%
3 

'% 
 •
.[5] $+

/ 

 1 

.
%  $+


 $%
/+ 1  G+
10 •

216
 Tishreen University Journal. Eng. Sciences Series 2009 (1) %
(31) $"
 
$%

1  %" $"

"# H
# (7 
> 
H%)> G7 =
 8  (2)(
NN $  

(The Learning Scenerio)  


. (9 
") G+
– /+
G
@2 1 $ 8
1 " 
  
•
.[4] 
G+
– /+
G
@2 1 3
9   /% 12 1 1
@8
•
.9$

C
2 &$  /+ 1 
L. % •
: *+
N  •
. G+


 &$  
1 "
H  F  
 /+
3B 9 
C 2  ?
:9$
A
3 1  1 6  $%.
G+
?
. [2]
  , 72   A
G+
 $%.
G+
1  F "
0 ?1
. ,
 F
"   ?2
3 
F # 37
 E
7!
?Amari ! 1967   1

'%

 
)
 9 /2 •
.+
A 
/  '
  .+  +

."  %
 ,
    " Werbos A
1974   •
.1985    
 E "F
$% " &$ Parker $ 1982   •

217
 A    ( 2 <
2 &$  %






 ,  * 


"   @
+

&$ E8
 *
–   1 
 /  2 1985   •
. %


(Learning Algorithm)E7 

:$  *+
1 % 0 [6]*+
  &$ %  @
+
 $%
E = ∑ EP ( 1)
P

.$
*+
 E 
 /+ /"2 1 *+
 E P  
1
EP = ∑
2 i
(t i − ai )2 ( 2)

 
.$%.
G+
 a i
.

  i
*+
   ,
&$ C  , 1
@8
/%" A
∆w = −η∇E ( 3)
. '+
"  ( $%

 )G

 /  η  
: j 1%
3 i 1%
 7
1@
  ,
1 
∂E
∇w ji = −η∇ ji E = −
∂w ji
∂E
.[7] NN  %

  &$
H

1  %
1 
∂wij
∂E ∂E ∂a j ∂net j
= ⋅ ⋅ ( 4)
∂w ji ∂a j ∂net j ∂w ji
. net j   /+ 1 (4) %
  
<@"
F
9 
n
net j = ∑ ai * w ji  
i =0

∂net j ∂ n n ∂w jk a k
∂w ji
=
∂w ji
∑ w jk a k = ∑
K =0 k =0 ∂w ji
( 5)

∂w jk
. k = i 1   .
7 1* E <@"
F
L.
∂w ji
:(5) %
N 
∂net j
= ai
∂w ji

218
Tishreen University Journal. Eng. Sciences Series 2009 (1) %
(31) $"
 
$%

1  %" $"

∂a j
a j = f (net j ) 12  : <@"
F
&0 '% 2
∂net j
1
f (net j ) = − net j

1+ e
∂a j
= f ′(net j ) =
d 1 + e −x ( )−1

" 
∂net j dnet j
(
d 1 + e −x )−1

= (−1)(1 + e − x ) − 2 e − x (−1)
dnet j
e −x 1
= ⋅
1 + e 1 + e−x
−x

1 + e−x − 1 1  1 + e−x 1  1
= −x
⋅ −x
=  −x
−  ⋅
1+ e 1+ e 1+ e 1 + e−x  1+ e
−x

= (1 − a j )a j

:1

∂a j
= (1 − a j )a j ( 6)
∂net j
∂E
:(2) %
1  . :(4) %
1 /8
F
9
∂a j
&0 '%
1
EP = ∑
2 i
(t i − ai )2 ( 7)

<@"
F
/"2 1 1  R 12   


 G+ &$ 1 4"
 
' G+  J •
(hidden layer) .+
$  J •
:$   F
9 12 3   ' G+ j

0
∂E ∂ 1
⋅ ∑ (t i − a i )
2
=
∂a j ∂a j 2 i
∂ (t i − ai )
= ∑ (t i − a i )
i ∂a j
= (t i − ai )(− 1) = −(t i − a i )
∂ j = −(t i − a i )(1 − a j )a j

219
 A    ( 2 <
2 &$  %






 ,  * 


&!$ F!
G!
1!( &$ /)
&0 G 1 % (h) .+
$ a j

0 2
.[8] j '
&0 /
k



∂E
= δ ⋅ wij ⋅ f ′(net h ) ⋅ X i (8)
∂w ji
:  
∂out h
f ′(net h ) =
∂net h
∂net o
wij =
∂out h

∂net h
xi =
∂wi

:( $) 
  ($ 34 &  E7

S ' 6 7
*+
3 *+ /   C+2
$      
  %
*+
L( 2 & ' @
:  %
 ' @  L( 
0
H " 9 
1 *+
L(
. (3) /
 1   

220
Tishreen University Journal. Eng. Sciences Series 2009 (1) %
(31) $"
 
$%

1  %" $"

/+

 ‫د ا‬
N

‫ج ا‬ ‫ج ام‬
Yn Ys

(‫)ا‬
E=Ys-Yn

N=N+1

+ *
E>0.03

stop


  ($ &  E7 :(3) (

:%
 I

:   2   '.
:  T(
.( STEP_W )  $%
'+  , ?1
..$+ /+

0 –3 . ( NO_OF_NODE) %
  , ?2
( STEP_W)  &>7  9FH !
.
&$ )
H + 6. +
3
 sin(2πt / 250) 3 12  

221
‫‪A    ( 2‬‬ ‫
 *  ‪<
2 &$  %






 ,‬‬

‫ ()‪.$  D ! "#  &>7  87  :(1‬‬
‫"‪*+
3 4‬‬ ‫
‪%‬‬ ‫

‪*+‬‬ ‫‪'+‬‬ ‫
‪%‬‬ ‫ 

‬
‫)‪(9 
%‬‬ ‫‪ .
%‬‬ ‫
‪ $%‬‬ ‫
‪E8‬‬ ‫
‪ .‬‬
‫‪16.06‬‬ ‫‪200‬‬ ‫‪-0.002718‬‬ ‫‪0.01‬‬ ‫‪200‬‬ ‫‪1000‬‬
‫‪188.83‬‬ ‫‪200‬‬ ‫‪-0.317749‬‬ ‫‪0.00‬‬ ‫‪200‬‬ ‫‪1000‬‬
‫‪1‬‬
‫‪7.20‬‬ ‫‪120‬‬ ‫‪-0.000080‬‬ ‫‪0.06‬‬ ‫‪200‬‬ ‫‪1000‬‬

‫‪1.5‬‬
‫‪1‬‬
‫‪0.5‬‬
‫ج ا‬
‫‪0‬‬
‫ج ام‬
‫‪-0.5 1‬‬ ‫‪139 277 415 553 691 829 967‬‬

‫‪-1‬‬
‫‪-1.5‬‬

‫()‪0.01 =  &>7

 # =
@  (4‬‬

‫

=‪.1000‬‬
‫
‪.200=E8
%‬‬
‫‪.0.01=  $%
'+‬‬

‫‪1.5‬‬
‫‪1‬‬
‫‪0.5‬‬
‫ج ا‬
‫‪0‬‬
‫ج ام‬
‫‪967 829 691 553 415 277 139‬‬ ‫‪1 -0.5‬‬

‫‪-1‬‬
‫‪-1.5‬‬

‫()‪0.001= $!
 #  &>7  (5‬‬
‫

=‪.1000‬‬
‫
‪.200=E8
%‬‬
‫‪0.001=  $%
'+‬‬

‫‪222‬‬
Tishreen University Journal. Eng. Sciences Series 2009 (1) %
(31) $"
 
$%

1  %" $"

1.5
1
0.5
‫ج ا‬
0
‫ج ام‬
-0.5
-1
-1.5

0.06 =  &>7

 # =
@  (6)(
.1000=


.200=E8
%

.0.06=  $%
'+
( NO_OF_NODE) @ #  –



&$ )
H + 6. +
3
 sin(2πt / 250) 3 10  

.$  D ! "# @ #  87  :2 ( 

3 4" % %
 *+

 '+ %


 
9 
% *+
9 
 $%
E8
 .

10.39 50 -0.001157 0.06 50 1000

5.01 100 -0.000284 0.06 100 1000
3.29 150 -0.000125 0.06 150 1000

1.5
1
0.5
‫ج ا‬
0
‫ج ام‬
-0.5 1 139 277 415 553 691 829 967

-1
-1.5

50= @ #
 #  G7  (7)(
.1000=


.50=E8
%

.0.06=  $%
'+

223
A    ( 2 <
2 &$  %






 ,  * 


1.5
1
0.5
‫ج ا‬
0
‫ج ام‬
-0.5 1 139 277 415 553 691 829 967

-1
-1.5

100= @ #
 #  G7  (8) (
.1000=


.100=E8
%

.0.06=  $%
'+

1.5
1
0.5
‫ج ا‬
0
‫ج ام‬
-0.5 1 139 277 415 553 691 829 967

-1
-1.5

150= @ # 

 #  G7  (9) (
.1000=


.150=E8
%

0.06=  $%
'+
: 87 (7 
L –


.$  D ! "# (7  &
L , 87  :3 ( 
  *+

 4"  %
 '
4
% %


 *+
3 

E8
/+

 .
 .

200 1063 -0.000102 3.16 1000 200 SIN

200 1063 -0.003784 375.78 1000 200 COS
150 2063 0.000001 6.56 2000 150
SIN 2
115 1063 -0.306595 48.42 1000 200 SIN*COS

224
Tishreen University Journal. Eng. Sciences Series 2009 (1) %
(31) $"
 
$%

1  %" $"

1.5
1
0.5
‫ج ا‬
0
‫ج ام‬
-0.5 1 139 277 415 553 691 829 967

-1
-1.5

sin (7  &
L 

 #  D !  (10)(
.1000=


.200=E8
%

0.06=  $%
'+

1.5
1
0.5
‫ج ا‬
0
‫ج ام‬
-0.5 1 139 277 415 553 691 829 967

-1
-1.5

cos (7  &
L 

 #  D !  (11)(
.1000=


.150=E8
%

0.05 =  $%
'+
1.5

0.5

‫ج ا‬
0
‫ج ام‬
997 914 831 748 665 582 499 416 333 250 167 84 1
-0.5

-1

-1.5

sin*cos (7  &
L 

 #  D !  (12)(

225
 A    ( 2 <
2 &$  %






 ,  * 


.1000=


.200=E8
%

.115 =  .
% %


1.2

0.8

0.6
‫ج ا‬
0.4
‫ج ام‬
0.2

0
1959 1781 1603 1425 1247 1069 891 713 535 357 179 1
-0.2

-0.4

sin 2 (7  &
L 

 #  D !  (13)(
.1000=


.150=E8
%

0.05 =  $%
'+

! 
G+
1  9
N2 $ (STEP_W )  $%
'+
 $ 62 " (1) /"

3! 4" 1 0.001 7  $%
'+
  0 K (2)/
6B 
 22  
G+

. 1  2 1 "+
1  
 188.83 7 *+

! 12 (9) /

 " 0 K /B2 9
1 $ %
 
@ $ 62 1   (2) /"
 2
*!+
3! ! (   /B8
 1 "+
1  9
12 /"
  ( &$2  (150 ) %

.  
: 1 /B8
 9
 (2) /"
 *+
3  
( 1 /(2  (3.29)
62 )
# 0  1
H %  2  "
3
3 /+
'0 1 J
&$ 62 E (3) /"

/
 1   
H " 
G+ E
G+ 1  9
1 < /



 +
 12 %
.(13)

:
$

F
. %


9  ' " $  V % .$+
)
 @
+ 12 

;<
  ?1
.'
  .+  1 A*      N ?2

226
 Tishreen University Journal. Eng. Sciences Series 2009 (1) %
(31) $"
 
$%

1  %" $"

3! ! 1! 

1# 68 KH 1 (sigmoid)  , 
3  +
12 


  ?3
.A

  "
 @
+

+ N W K
 @
+

+ 9  " ;<

  ( ?4
. %


    $ 8
;<
&0 /$

: M

1- ERIC,D.and PATRICK,N.", Neural Networks",university of Manchester,Edition

Eyrolles.Paris1991,145P.
2- SARTORI,A. and ANTSAKLIS,P.,"Implementations of learning control systems using
neural networks,"IEEE control systems magazine,April 1992,57.
3- http://www.techguide.com.
4- LEE,J.,"Astuy on speeding up learning of neural networks",The Institute of Electrical
Engineering ,chung-Hua polytechnic institute,Taiwan,1994,105.
5- RUMELHART,D.,HINTON,G.and WILLIAMS,R.,"learning internal Representations
by error propagation,"vol .1.MIT press,1986,536.
6- NG,S.,LEUNG,S. and LUK,A."Ageneralized Back-propagation algorithm for faster
convergence",IEEE international conference on neural networks,Washington D.C.,3-
6,vol.l,1996,409-413.
7- ADNAN,S.,TAYFUN,M.&SINAN,U.,"Determination of efficiency of flat-plate solar
collectors using neural network approach",Ankara,Turkey,2008,1533-1539.
8- MELLIT,A.,BENGHANEM,M.,&KALOGIROU,A.,"Modeling and simulation of a
stand-alone photovoltaic system using an adaptive artificial neural network :
proposition for anew sizing procedure",Renewable Energy,2007,285-313.

227
A    ( 2 <
2 &$  %






 ,  * 


228