Wojciech Gryc - Neural Network Predictions of Stock Price Fluctuations

Neural Network Predictions of
Stock Price Fluctuations

By Wojciech Gryc (wojciech@gmail.com)

7
7
0
0
+
+
D
D
V
V
D
D
s
s
F
F
O
O
R
R
S
S
A
A
L
L
E
E
&
&
E
E
X
X
C
C
H
H
A
A
N
N
G
G
E
E

w ww ww w. .t tr ra ad de er rs s- -s so of ft tw wa ar re e. .c co om m

w ww ww w. .f fo or re ex x- -w wa ar re ez z. .c co om m

w ww ww w. .t tr ra ad di in ng g- -s so of ft tw wa ar re e- -c co ol ll le ec ct ti io on n. .c co om m

w ww ww w. .t tr ra ad de es st ta at ti io on n- -d do ow wn nl lo oa ad d- -f fr re ee e. .c co om m

C Co on nt ta ac ct ts s

a an nd dr re ey yb bb br rv v@ @g gm ma ai il l. .c co om m
a an nd dr re ey yb bb br rv v@ @y ya an nd de ex x. .r ru u
S Sk ky yp pe e: : a an nd dr re ey yb bb br rv v
Table of Contents
Abstract...................................................................................................................................................... 3
Introduction................................................................................................................................................ 3
Part 1: The Stock Market
Stock Market Psychology.......................................................................................................................... 5
Technical Analysis..................................................................................................................................... 6
Variable Moving Average (VMA........................................................................................................ !
Trading "ange #reakout (T"#............................................................................................................ !
#ollinger #ands................................................................................................................................... $%
Money &lo' Inde( (M&I....................................................................................................................$%
Te)*oral +ata..................................................................................................................................... $$
Prices................................................................................................................................................... $$
&unda)ental Analysis o, Stocks..............................................................................................................$-
.onsu)er Price Inde( (.PI............................................................................................................... $3
.onsu)er Senti)ent Inde( ................................................................................................................ $3
.onsu)er .on,idence Inde(............................................................................................................... $/
0e' 1rders +i,,usion Inde(............................................................................................................... $/
2eading Indicators Inde(.....................................................................................................................$/
Interest "ate o, the 3%34ear .onventional Mortgage......................................................................... $5
&ederal &unds "ate.............................................................................................................................. $5
Price35arnings (P65 "atio.................................................................................................................. $6
Target Stocks............................................................................................................................................$6
Part 2: Neural Networks
0eurons and 0eural 0et'orks................................................................................................................. $7
In*ut .onnections................................................................................................................................$!
Su))ing and Activation &unctions.................................................................................................... $!
1ut*ut .onnections............................................................................................................................. -%
&eed,or'ard 0et'orks.............................................................................................................................-$
#ack*ro*agation..................................................................................................................................--
+ra'backs........................................................................................................................................... -3
#ack*ro*agation A**lied....................................................................................................................-/
Threshold Analysis..............................................................................................................................-5
"ecurrent 0et'orks................................................................................................................................. -6
5l)an 0et'orks.................................................................................................................................. -8
Part 3: Results
Statistical Analysis................................................................................................................................... -7
"esults...................................................................................................................................................... -!
&eed,or'ard 0et'orks and Ty*es o, Stocks...................................................................................... 3%
1ther 0et'ork Architectures ,or $3+ay Predictions...........................................................................3$
0eural 0et'ork Predictions o, Stock Price &luctuations $ 6 /-
Prices................................................................................................................................................... 3$
2onger Ter) Predictions and &unda)ental Analysis......................................................................... 3-
.onclusion and &uture 9ork................................................................................................................... 33
A**endi( A: 1vervie' o, Trials............................................................................................................. 3/
A**endi( #: 1vervie' o, .usto) So,t'are........................................................................................... 38
A**endi( .: 5,,ects o, "ando) 9eights on 0et'ork "esults............................................................... 3!
9orks .ited............................................................................................................................................. /$
0eural 0et'ork Predictions o, Stock Price &luctuations - 6 /-
Abstract
This research *a*er is the result o, a three3)onth3long inde*endent study ,ocusing on the ability
o, ,eed,or'ard and recurrent neural net'orks in *redicting stock *rice ,luctuations o, co)*anies 'ithin
the ;.onsu)er +iscretionary< category o, the =lobal Industry .lassi,ication Standard (=I.S. The
*a*er ,ocuses on *redictions ,or .ircuit .ity Stores Inc. (.. ,or $3> 53> and -%3business day *eriods
through the use o, in*ut *atterns based on ,unda)ental and technical indicators ,ro) the stock )arket
and econo)y. The *ro?ect is uni@ue in its ,ocus on co)*anies selling to consu)ers. The results o, the
*ro?ect are *ro)ising> 'ith statistically signi,icant results a**earing ,or $3day single3layer ,eed,or'ard
and 5l)an net'orks> as 'ell as net'orks based on ,unda)ental in*uts and *redicting ,or -%3day
*eriods.
Introduction
9ithin the last t'o decades> a great deal o, attention has been ,ocused on the idea o, *redicting
stock *rices and *rice ,luctuations through the use o, neural net'orks. Such research> i, success,ul> has
the *otential to bring nu)erous re'ards A both acade)ic and ,inancial A to researchers or analysts that
,ind any *atterns. There are nu)erous reasons 'hy neural net'orks o,,er an advantage in the @uest to
*redict stock *rice ,luctuations. Many theories ,ocusing on econo)ics and ,inance o,ten begin 'ith a
set o, assu)*tions> and ho*e to build rational and believable )odels o, the 'orld. At this ti)e>
ho'ever> there is only one 'idely acce*ted theory on stock *rices and )arkets> called the 5,,icient
Markets By*othesis (5MB> 'hich ulti)ately concludes that such *rice changes )ay never be
*redicted.
9hat is uni@ue about neural net'orks is that the a)ount o, assu)*tions )ade through their use
is )ini)iCed> and ,ocuses solely on the data being ,ed into the). "esearchers can avoid assu)*tions
about *er,ectly rational hu)an beings> used by the 5MB> and si)*ly 'atch i, and ho' their net'orks
converge. 9hether rando) or ,ully *redictable> a correctly designed neural net'ork 'ill theoretically
converge on the o*ti)al result. Most other )odels do not have this lu(ury.
9ith this in )ind> the research *ro?ect *resented belo' ai)s to test ho' 'ell neural net'orks
*redict stock *rice ,luctuations by *resenting a nu)ber o, diverse architectures using t'o very di,,erent
sets o, in*uts and co)binations thereo,. S*eci,ically> the 'ork ai)s to shed light on the di,,erences
0eural 0et'ork Predictions o, Stock Price &luctuations 3 6 /-
bet'een using ,unda)ental and technical analysis co)bined 'ith neural net'orks to *redict stock
*rice ,luctuations 'ithin $> 5> and -%3business day ti)e ,ra)es> starting in the year -%%% and ending in
-%%5. A second ai) o, this *ro?ect is to co)*are ho' 'ell *redictions ,are in both ,eed,or'ard and
recurrent net'orks.
This *a*er is organiCed into three *arts> beginning 'ith a theoretical overvie' o, the
*sychology o, the stock )arket> and 'hy stock *rices )ay be *redictable. The *a*er then goes into
so)e discussion o, o, the theory behind neural net'orks> and ho' they 'ork. &inally> it ends 'ith the
results o, the *ro?ect> based on over $-5 di,,erent co)binations o, net'ork architectures and training
settings> and their i)*lications on the *redictive ability o, neural net'orks 'ithin the stock )arket.
0eural 0et'ork Predictions o, Stock Price &luctuations / 6 /-
Part 1: The Stock Market
Stock Market Psychology
A great deal o, 'ork has gone into analyCing the *sychology o, the stock )arket. Si)*ly *ut>
the stock )arket is a *lace> be it abstract or real> 'here large grou*s o, *eo*le gather to buy and sell
stocks. The dyna)ics o, such a *lace is o,ten linked 'ith that o, grou* *sychology and )any have tried
to build theories o, ho' the stock )arket ,unctions based on the *sychology o, large grou*s.
The only 'idely acce*ted theory o, ho' the stock )arket 'orks is called the 5,,icient Market
By*othesis (5MB (9Drneryd> -%%$> and in its si)*lest ,or)> states that stock *rices al'ays re,lect all
the available in,or)ation about the )arket and co)*anies in @uestion (Shlei,er> -%%%. The theory rests
on three assu)*tions> all ,ocusing on investors the)selves. These are:
Investors are ,ully rational and res*ond rationally to all changes 'ithin the )arket. This )ay go
so ,ar as saying that investors subconsciously use tools like #ayesian *robability to analyCe
*robabilities and )ake decisions.
Irrational investors trade rando)ly> and as such> all rando) invest)ents cancel the)selves out>
leaving no e,,ect on the )arket.
5ven i, investorsE irrational decisions do not cancel out through *robabilistic )ethods> other
rational investors 'ill take advantage o, these irrational decisions to )ake a *ro,it. Such
sche)es bring the )arket back to an e,,icient state.
The 5MB has been 'idely acce*ted> though it does have shortco)ings. 1n a si)*le
behavioural level> it is clear that *eo*le do not use #ayesian *robability to learn ,ro) their )istakes
(Shlei,er> -%%% and are rarely ,ully rational. The theory also assu)es that every investor has access to
all the re@uired in,or)ation to value a stock. This *roble) o, F*er,ect in,or)ationF is its inter*retation
A 'hile ,inancial state)ents and econo)ic statistics are 'idely available> one cannot assu)e that such
in,or)ation is inter*reted correctly by all investors (+yck)an G Morse> $!76. Indeed> this
inter*retation o,ten de*ends on ho' in,or)ation is ,ra)ed and 'here it co)es ,ro) (Shlei,er> -%%%.
&urther)ore> 'ell3docu)ented stock )arket ano)alies (+i)son> $!77 such as an increase in share
*rices in Hanuary and a continuation o, &riday *rice trends on Mondays run counter to the idea o,
un*redictable and *er,ectly e,,icient )arkets.
In ter)s o, large grou*s> one o, the biggest *roble)s 'ith the 5MB is that the assu)*tions
i)*ly investors act inde*endently o, each other: a naive investor )aking )istakes 'ill be taken
advantage o, by a rational investor. Bo'ever> it is o,ten the case that investors act in unison (Shlei,er>
-%%%> as can be seen during *eriods o, irrationally increasing *rices> like the late $!!%sE dot3co) stock
craCe. Indeed> a co))on stock trading strategy> es*ecially a)ong technical analysts> is the Fcontrarian
strategyF (Siegel> $!!7: by observing the )arkets and seeing ho' large grou*s o, investors act> one
can )ake a great deal o, )oney by doing the o**osite o, the grou*. This> ho'ever> de*ends on the
assu)*tions that investors act in a herd3like 'ay and that their grou*3based decisions> 'hether
conscious or not> are incorrect.
9ithin econo)ics> there are t'o i)*ortant ti)e ,ra)es: the long and the short run. Su**orters
o, the 5MB theory argue that in the long run> *atterns and ano)alies are o,ten noticed by investors and
eventually cancel the)selves out. &or e(a)*le> i, enough investors learn about increasing share *rices
in Hanuary> )any investors 'ill try to sell their stocks in that )onth to )ake a *ro,it> thus lo'ering the
*rice and canceling out the e,,ect. Bo'ever> there is a di,,erence bet'een e,,iciency in the long3 and
short3ter)s. It is *ossible ,or the )arket to be e,,icient in the long run 'hile e(hibiting irrational and
*otentially *redictable *atterns in the short ter) (Malkiel> -%%3. Thus> it is )ay be *ossible> even
under the 5MB> to have a neural net'ork *redict short3ter) *atterns 'ithin the stock )arket or
individual co)*aniesE shares. 9ithin the conte(t o, this *ro?ect> it is these short3ter) irregularities and
collective )istakes by investors that )ay be e(tre)ely use,ul> es*ecially ,or a neural net'ork. #y
,ocusing on a ti)e ,ra)e o, at )ost one )onth> any *atterns that )ay arise are not long3ter)> and a
neural net'ork )ay learn to *redict the) be,ore investors do.
To )ake such *redictions> a neural net'ork )ust have access to in,or)ation. 5ven i, one
ackno'ledges that stock *rices )ay be *redicted in a given s*an o, ti)e> there are )ulti*le strategies
and ,or)s o, analysis that )ay or )ay not 'ork. 9hen *redicting share *rice ,luctuations> there are
t'o )a?or ca)*s: technical and ,unda)ental. #oth are used 'ithin this *ro?ect> and an analysis o, their
e,,ects and use,ulness based on the accuracies o, their res*ective neural net'orks is given near the end.
Technical Analysis
9hereas ,unda)ental analysis looks at )any di,,erent as*ects o, a co)*any and econo)y to
*redict ho' 'ell a stock 'ill *er,or)> the assu)*tion inherent in technical analysis is that all such
re@uired in,or)ation is available in a shareEs *ast *rices. S*eci,ically> technical analysis is o,ten
de,ined as ;The *rocess o, analyCing a securityIs historical *rices in an e,,ort to deter)ine *robable
,uture *rices.< (Achelis> -%%$> *t. $ So)e ,or)s o, technical analysis also incor*orate volu)e A ho'
o,ten a security is traded during a day or 'ithin another ti)e *eriod A to hel* *redict *rices.
The 'ay technical analysis hel*s *redict 'hether a stock 'ill increase or decrease in value is by
signaling trend reversals. According to technical analysts> )any securities e(hibit di,,erent ty*es o,
gro'th over ti)e> and the key to )aking a *ro,it in the stock )arket is to *redict 'hen a stockEs
tendency to increase in value 'ill rise> or a ,alling stock *rice 'ill sto* and begin to increase. #y
vie'ing a stockEs *ast *er,or)ance> one )ay begin to understand ho' *eo*le investing in that stock act
and 'hich *rices they are co),ortable 'ith. Indeed> so)e (9Drneryd> -%%$J Iihara> Kato G Tokunaga>
-%%$ have co)*ared the theory behind technical analysis to herd behaviour in the stock )arket.
&or e(a)*le> one technical indicator> called the Trading "ange #reakout> assu)es that there is
an u**er and lo'er *rice that investors 'ill tolerate ,or their stocks. I, one considers a lo'er bound ,or
the *rice> investors )ay o,ten allo' the *rice to ,all to the lo'er bound and then bounce back> but the
grou* has a *sychological barrier to letting the *rice ,all belo' this bound. 1ne can *redict a trend
reversal A in this case> a stockEs *rice tending to ,all 'ithin the co)ing days A 'hen this lo'er bound is
,inally broken and the stockEs *rice ,alls lo'er. 9hen this ha**ens> technical analysts argue that one
can be sure the *rice change 'ill be signi,icant> and in the do'n'ard direction. 1bserving such
*atterns a,ter they occur is easy> and an e(a)*le 'ith share *rices o, &ord is sho'n belo'. In this case>
one can see a lo'er bound in starting )id3August -%%5 and lasting until the end o, Se*te)ber -%%5.
A,ter the lo'er bound is breached> the *rice dro*s signi,icantly.
Ford Share Prices
7.50
8.00
8.50
9.00
9.50
10.00
10.50
11.00
17-Jun-05 07-Jul-05 27-Jul-05 16-Aug-05 05-Sep-05 25-Sep-05 15-Oc-05 0!-"o#-05 2!-"o#-05 1!-$ec-05
Ad%used &losing Price
$
a
e
The use o, technical analysis 'ithin invest)ent decisions has been controversial. According to
5,,icient Market Theory> investors act co)*letely rationally and have access to all re@uired in,or)ation
to value a stockEs *rice during the *resent A as such> a stockEs current *rice re,lects its value at that ti)e>
and has no connection 'ith *ast *rices. Such assu)*tions have been criticiCed> and statistical analysis
o, technical indicators has sho'n their use,ulness in *redicting trend reversals and share *rice
,luctuations (#rock> 2akonishok G 2e#aron> $!!-. Interestingly> historical technical analysis o, *re3
$!-! stock )arket data sho's that technical analysis 'ould have hel*ed investors *redict the stock
)arket crash and the =reat +e*ression (Siegel> $!!7.
9ithin the conte(t o, this research *ro?ect> technical analysis is e(tre)ely use,ul 'hen
co)bined 'ith neural net'orks. This ty*e o, analysis de*ends largely on *ast *rices o, share sales>
'hich are o,ten available instantaneously and in real3ti)e. The data is o,ten @uite e(tensive as 'ell>
including daily high and lo' *rices> ad?usted closing *rices to account ,or dividends and s*lit shares>
and nu)erous other variables. Indeed> the *roble) ,or technical analysis is that there is so )uch data
and ,or)s o, analysis that one does not kno' 'hat to do 'ith it all. &or e(a)*le> Technical Analysis
from A to Z (Achelis> -%%$> a re,erence guide ,or technical analysts> lists $35 di,,erent tools ,or doing
such analysis. #y entering all this in,or)ation into a neural net'ork> the net'ork )ay be able to
discern relevant *atterns 'ithout having an analyst do it. &urther)ore> because everything de*ends on
the analysis o, *ast *rices> it is theoretically *ossible to use *rices as in*uts and have the neural
net'ork develo* its o'n technical indicators.
1ne o, the *roble)s 'ith technical analysis is that 'hile so )any tools e(ist> )any o, the
details are sub?ective: it is still u* to the investor to choose values ,or the variables 'ithin each tool>
and it is the investorEs decision to ,ollo' a buy or sell signal ,ro) a technical indicator. 9ith so )any
o*tions> neural net'orks can *lay a crucial role in deciding 'hich technical variables to ,ollo' 'hen
analyCing *ast *rices. 9ithin this *ro?ect> neural net'orks are *resented 'ith a nu)ber o, technical
variables and buy6sell signals> and it is u* to the net'ork to learn 'hich indicators are use,ul and 'hich
are not.
#ased on discussions 'ith ,inancial e(*erts and reading into *rior research (#rock> 2akonishok
G 2e#aron> $!!-> si( ty*es o, indicators 'ere chosen as in*uts ,or neural net'orks using technical
analysis.
Variable Moving Average (VMA)
9hile technical analysis *osits that *ast stock *rices are not rando)> any technical analyst 'ill
ad)it that there is a great deal o, noise 'ithin the data. To s)oothen the noise> a variable )oving
average is used> using the ,ollo'ing e@uation:
i=$
k
p
i
k
'here p
i
is the *rice ,or the i
th
day> and the )oving average e(tends ,or k days. As de,ined by #rock>
2akonishok G 2e#aron ($!!-> a VMA indicator uses t'o se*arate )oving averages> one ,or -%% days
and one ,or $ day. 9hen the *rice o, the $3day )oving average is $L higher than the -%%3day )oving
average> the stock is considered overbought> and its *rice is likely to ,all in the ,uture. Thus> a sell
signal (3$ is given to the net'ork ,or that day. 2ike'ise> i, the $3day )oving average is $L belo' the
-%%3day )oving average> a buy signal ($ is given> because the stock is oversold and has a lo' *rice.
A second variation on the above is also used as an in*ut to the net'ork> 'ith a -%%3day )oving
average and a -3day )oving average.
Trading Range Breakout (TRB)
The *re)ise behind the T"# is that investors holding securities have a *sychological barrier to
e(tre)ely high and lo' *rices. As sho'n earlier 'ith &ordEs *rice dro*> share *rices o,ten tend to
bounce o,, o, this *sychological barrier. 1nce the barrier is broken> ho'ever> and a stockEs *rice ,alls
belo' the historical )ini)u)> a sell signal is generated> 'hile a buy signal is generated 'hen a stockEs
*rice rises above a historical )a(i)u).
The i)*le)entation o, the T"# is si)*le: ,ind the lo'est and highest *rices ,or the last k days>
and i, the stockEs closing *rice rises above the )a(i)u) or ,alls belo' the )ini)u)> the stock should
be bought or sold> res*ectively. In this *ro?ect> 5%3day> $%%3day> and -%%3day T"#s are used> 'ith buy
signals generated 'hen a *rice rises $L above the )a(i)u)> and instructions to sell are sent 'hen the
stockEs *rice ,alls $L belo' the )ini)u). This $L di,,erence is signi,icant> as it avoids regular
buy6sell signals i, a *rice ,luctuates around a )a(i)u)> and 'as sho'n to 'ork 'ell in *revious
investigations (#rock> 2akonishok G 2e#aron> $!!-.
0eural 0et'ork Predictions o, Stock Price &luctuations ! 6 /-
Bollinger Bands
#ollinger #ands are used to )easure a stockEs *rice volatility and then *rovide insights as to
'hether or not the stock should be bought> sold> or held. The buy6sell signal de*ends on three di,,erent
lines> the ,irst and si)*lest o, 'hich is a variable )oving average> as *resented earlier. 1,tenti)es> the
)oving average is set ,or the *ast -% days. T'o other bands are dra'n around the )oving average> the
,irst being a )ulti*le o, the standard deviation added to the )iddle band> and the other being a )ulti*le
o, the standard deviation subtracted> as sho'n belo':
M!d
.
i
k
( p
i
M)
-
n
'here M is the )oving average> p
i
is the closing *rice (or in the *ro?ectEs case> the ad?usted close> and
n is the nu)ber o, days o, the )oving average. d is the nu)ber o, standard deviations that one 'ould
like to use. &or the data being analyCed> d M -> as reco))ended by Technical Analysis ,ro) A to N
(Siegel> -%%$.
9hile there are )any 'ays to inter*ret the in,or)ation given by #ollinger #ands> the
in,or)ation su**lied ,or the neural net'orks ,ollo's a *attern si)ilar to the T"#: i, a stockEs *rice di*s
belo' standard deviations o, the )oving average> a buy signal is sent> 'hile a sell signal is sent i, the
stock *rice rises t'o standard deviations above the )oving average.
Money Flow Index (MFI)
M&I is the only technical variable 'ithin this *ro?ect that incor*orates volu)e. The M&I
analysis ho' )uch )oney is being s*ent on *urchasing securities and ho' )uch is )ade by selling
the)> and de*ending on the ratio bet'een the t'o> an inde( is created. .alculations begin by creating a
Money &lo' (M& variable:
Money Flow=
p
high
+p
low
+p
close
3
Volume
The M& is di,,erent ,ro) the M&I. In this case> one adds all the days and *rices thereo, 'here a
securityEs *rice increased to )ake the *ositive M&> 'hile all the negative daysE *rices are added to get a
negative )oney ,lo'. This allo's one to calculate the Money "atio (M" 'hich is then used to
calculate the M&I.
0eural 0et'ork Predictions o, Stock Price &luctuations $% 6 /-
MR=
MF
positive
MF
negative
MFI =$%%
$%%
$+MR
I, the M&I di*s belo' -%> a buy signal is created> 'hile an M&I rising over 7% generates a sell
signal (Siegel> -%%$.
Temporal ata
+ata on the )onth o, the year and day o, the 'eek is also included in the data set> as so)e stock
analysts have noted ,luctuations in security *rices based on the day o, the 'eek and )onth o, the year
(+i)son> $!77. T'o notable e,,ects are called the ;Monday 5,,ect<> 'here stocks tend to ,ollo' the
*atterns e(hibited on &ridays> and the ;Hanuary 5,,ect<> 'hen stock *rices are )ore likely to rise in this
)onth than other )onths.
+ue to the nature o, the stocks being analyCed> this data can be considered both technical and
,unda)ental A 'hile the te)*oral *atterns above are technical> because the stocks being analyCed tend
to re*resent goods and services that are o,ten *erceived as lu(uries and e(hibit cyclical *atterns>
consu)*tion decisions )ade by households (,or e(a)*le> to go to lu(ury hotels )ore o,ten in the
su))er than 'inter )ay a,,ect certain stock *rices. Since no assu)*tions about such events are )ade
in the data and because the data changes on a daily basis 'hile all other ,unda)ental variables do not>
this 'as treated as a technical in*ut.
!ri"es
&inally> *rices 'ere still included in the in*uts. In this case> *rices 'ere nor)aliCed as ,ollo's:
p
norm
=
p
original
p
average
3d
'here p
average
re*resents the average *rice ,or the entire data set and d is the standard deviation o, the
entire data set. Such a syste) *uts virtually all *rices into a range bet'een 3$ and $> and allo's the
gra*h to retain its ,or)er sha*e> as sho'n belo' ,or &ord shares.
0eural 0et'ork Predictions o, Stock Price &luctuations $$ 6 /-
Fundamental Analysis of Stocks
Onlike its technical counter*art> ,unda)ental analysis *osits that co)*anies that do 'ell in their
line o, 'ork> be it by having high *ro,its> a good )anagerial structure> a success,ul ,ocus on research
and innovation> or any other si)ilar ,actors> 'ill do 'ell in the stock )arket. Many argue that this ty*e
o, analysis is )uch )ore reliable than technical analysis ,or longer ter) invest)ents> as stocks are
su**osed to re,lect the value o, the co)*any they re*resent.
The stocks 'ithin this *ro?ect 'ere chosen based on their *otential res*onses to ,unda)ental
variables 'ithin the )acroecono)y. The ;.onsu)er +iscretionary< (Morgan Stanley .a*ital
International Inc.> -%%5 categoriCation o, stocks> ,ro) 'hich the list o, chosen stocks co)es ,ro)> is
kno'n ,or its res*onsiveness to business cycles> including ones that a,,ect the entire econo)y. This
)akes it *ossible to ,ind *atterns 'ithin the stocks based on ho' certain econo)ic indicators and
variables are doing.
There are t'o caveats to be )entioned 'ith regards to using the variables above> or any others
available to one using neural net'orks ,or a**lications 'ithin the stock )arket. &irst and ,ore)ost>
)any ,unda)ental variables are released long *ast the *eriod they re*resent. 5ven so)ething as si)*le
as a variable re*resenting 'hether an econo)y is in a recession could lag by )onths> i, not years: the
recession o, $!!% to $!!$> ,or e(a)*le> 'as only announced by the 0ational #ureau o, 5cono)ic
"esearch (0#5" a,ter it 'as over (Kaca*yr> $!!6. Si)ilarly> aside ,ro) the leading indicators> the
variables chosen ,or this *ro?ect )ay co)e long a,ter the *eriod 'hich they discuss.
0eural 0et'ork Predictions o, Stock Price &luctuations $- 6 /-
Ford Share Prices
7.00
8.00
9.00
10.00
11.00
12.00
1'.00
1!.00
15.00
16.00
17.00
Oc-0' Jan-0! Apr-0! Aug-0! "o#-0! Fe(-05 )a*-05 Sep-05 $ec-05 )ar-06
$ae
&
l
o
s
i
n
g

P
r
i
c
e
-0.8
-0.6
-0.!
-0.2
0.0
0.2
0.!
0.6
0.8
"
o
r
+
a
l
i
,
e
d

P
r
i
c
e
Ad%used &lose "or+ali,ed &lose
Secondly> 'hile stock *rices change on a regular basis> the sa)e is not true ,or ,unda)ental
variables. Many o, the variables belo' are released )onthly> though cor*oration earnings in,or)ation
is released @uarterly A incor*orating data that a**ears ,ro) nu)erous ti)e *eriods 'ithin a neural
net'ork is di,,icult> es*ecially 'hen one has access to a nearly3continuous strea) o, in,or)ation> like
share *rices.
As 'ith technical variables and indicators> there are nu)erous o*tions ,or analysts> and one
)ust be e(tre)ely care,ul in choosing the). The indicators 'ere chosen 'ith this in )ind> and are
listed and e(*lained belo'.
#onsumer !ri"e Index (#!I)
There are )ulti*le 'ays o, )easuring in,lation 'ithin an econo)y> and one o, the )ost 'idely
used is the .PI. There are t'o reasons 'hy the .PI 'as chosen as an in*ut ,or the neural net'ork.
.hanging *rices can o,ten a,,ect ho' consu)ers s*end their )oney> and ho' 'ell co)*anies do. This
is es*ecially true ,or co)*anies that deal directly 'ith consu)ers> such as those selling *roducts. The
co)*anies in the .onsu)er +iscretionary category all deal 'ith goods that are seen as lu(uries rather
than necessities> and thus )ay be sensitive to *rice ,luctuations A i, *eo*le have less )oney to s*end>
lu(uries are o,ten the ty*es o, goods that are avoided.
The .PI is )easured by taking a ,i(ed basket o, goods that consu)ers usually buy (household
goods> ,ood> and so on and co)*aring the *rice o, these goods during the current year or )onth to a
base value> nor)aliCed to a value o, $%%. Thus> i, the .PI is $5% today> it )eans *rices rose by 5%L
since the base year (Manki' G Scarth> -%%/. All o, the data being used is based on the OS econo)y>
and 'as ac@uired ,ro) the #ureau o, 2abor Statistics (n.d..
#onsumer $entiment Index
The Oniversity o, Michigan (-%%6 conducts a )onthly survey o, consu)ers that gauges a
nu)ber o, di,,erent variables> and based on these variables> the Oniversity co)*iles an inde( allo'ing
one to see the relative changes in consu)er senti)ent each year. Again> due to the nature o, the
.onsu)er +iscretionary category o, co)*anies> ho' consu)ers ,eel about their ,uture ,inances>
e)*loy)ent> and other econo)ic and ,inancial ,actors )ay a,,ect 'hat they s*end )oney on> and thus>
ho' 'ell the co)*anies do.
The inde( is based on a nu)ber o, variables ,ro) the universityEs surveys> and includes
0eural 0et'ork Predictions o, Stock Price &luctuations $3 6 /-
(Oniversity o, Michigan> -%%6:
.urrent Personal &inances
5(*ected Personal &inances
5(*ected #usiness .onditions in 5 Months
5(*ected #usiness .onditions in $- Months
#uying .onditions
#onsumer #on%iden"e Index
Si)ilar to the .SI> the .onsu)er .on,idence Inde( (.on,erence #oard> -%%6a is based on a
)onthly survey o, 5%%% re*resentative households in the Onited States> and is conducted by the
.on,erence #oard o, the Onited States. The survey is )eant to learn ho' consu)ers see the current
and ,uture econo)ic and ,inancial situation> and ho' they ,eel it 'ill a,,ect the) *ersonally. This is
very si)ilar to the .SI> though the historical values do di,,er> and there are ti)es 'hen one value rises
and another ,alls.
&ew 'rders i%%usion Index
The Institute ,or Su**ly Manage)ent (ISM is a an international association co)*osed o,
businesses 'orking ,or the ;identi,ication> ac@uisition> access> *ositioning and )anage)ent o,
resources that an organiCation needs or *otentially needs in the attain)ent o, its strategic ob?ectives.<
(Institute ,or Su**ly Manage)ent> n.d.. S*eci,ically> this includes )anu,acturing> trans*ortation> and
inventory )anage)ent. The ISMEs 0e' 1rders +i,,usion Inde( is an indicator that )easures the level
o, ne' inventory orders ,ro) businesses> and ho' dis*ersed these ne' orders are bet'een industries.
This inde( can be used as an indicator o, su**lier senti)ent and e(*ectations A businesses order ne'
goods only i, they e(*ect to sell the)> and the higher the di,,usion o, ne' orders> the )ore likely it is
that su**liers 'ithin the entire econo)y have a *ositive outlook.
(eading Indi"ators Index
Indicators are econo)ic variables that are correlated 'ith the general state o, the econo)y> and
co)e in three ty*es: lagging> coincident> and leading (=ranger> $!7!. 2agging indicators are variables
that shi,t in a si)ilar direction as the econo)y> but do so a,ter the econo)y has e(*erienced the
0eural 0et'ork Predictions o, Stock Price &luctuations $/ 6 /-
change> 'hile coincident indicators do so at the sa)e ti)e. 2eading indicators are e(tre)ely use,ul ,or
,orecasting> because they o,ten e(*erience changes be,ore the econo)y as a 'hole e(*eriences si)ilar
shi,ts. This )akes the) very use,ul ,or *redicting ho' the econo)y 'ill look in the near ,uture.
An e(a)*le o, a leading indicator is the average 'ork'eek in the Onited States (=losser G
=olden> $!!8. In this case> 'hen e)*loyers begin to e(*erience higher de)and ,or *roducts> they 'ill
have their e)*loyees 'ork longer hours to )eet *roduction de)and. This increases the average
'ork'eek> and the econo)y only e(*eriences an increase in activity once the e)*loyees are ,inished
'orking and the *roducts have been shi**ed. The ISM 1rders +i,,usion Inde( )ay be a leading
indicator ,or a si)ilar reason.
This inde( is *roduced by the the .on,erence #oard and is co)*rised o, ten leading indicators
(.on,erence #oard> -%%6b:
average 'eekly initial clai)s ,or une)*loy)ent insurance (inverted
average 'eekly )anu,acturing hours
building *er)its
inde( o, consu)er e(*ectations
interest rate s*read
)anu,acturersI ne' orders ,or consu)er goods and )aterials
)anu,acturersI ne' orders ,or non3de,ense ca*ital goods
real )oney su**ly
stock *rices
vendor *er,or)ance
Interest Rate o% t)e *+,-ear #onventional Mortgage
The &ederal "eserve *ublishes 'eekly statistics on the interest rates given ,or ,i(ed3rate ,irst
)ortgages (&ederal "eserve #oard> -%%6b. Such interest rates o,ten a,,ect )a?or buying decisions
'ithin ,a)ilies> es*ecially ,or things such as ne' ho)es. It is assu)ed that this interest rate also a,,ects
s)aller *urchases indirectly> by a,,ecting buying habits 'ith regards to auto)obiles and other goods.
Federal Funds Rate
This is another interest rate> but is charged by the &ederal "eserve to banks and other lending
institutions 'hen they need )oney to balance their o'n reserves and accounts. The &ederal "eserve
uses this rate to *ro)ote *rice stability and sustainable econo)ic gro'th (&ederal "eserve #oard>
-%%6a> and o,ten> this rate in,luences consu)ers using banks and credit.
!ri"e,.arnings (!/.) Ratio
The only stock s*eci,ic ,unda)ental variable in this data set> the P65 "atio ,or a stock is the
value o, all shares being traded divided by a co)*anyEs annual earnings. Mathe)atically>
P Ratio=
!hare Price"um#er of !hares
Annual arnings
The generally acce*ted rule in investing is that as the P5 ratio gro's> the likelihood that a
stockEs *rice 'ill ,all increases. 9hile there are nu)erous ,actors a,,ecting a stockEs *rice and a lo' P5
ratio does not necessarily )ean one should buy shares o, a s*eci,ic stock> it does *lay a role in )any
investorsE decisions.
Target Stocks
There are nu)erous o*tions available to those interested in researching neural net'orks and
their a**lications to the stock )arket. &or this research> an initial set o, $7 stocks 'ere chosen ,or
analysis> and the choice 'as based on a nu)ber o, ,actors. The )ethod to choosing the stocks began by
observing the SGP 5%%. This inde( is used as a standard gauge ,or )odels and *redictions> and the
inde( contains so)e o, the largest and 'ell3re*uted co)*anies on the )arket.
The SGP 5%%> as 'ell as )any other stocks> uses the =lobal Industry .lassi,ication Standard
(=I.S (Morgan Stanley .a*ital International Inc.> -%%6 to classi,y its stocks into s*eci,ic categories>
industries> and other sectors. 9hile one can ,ocus on a large a)ount o, diverse stocks> this *ro?ect has
chosen to ,ocus on the subset labeled ;.onsu)er +iscretionary<. According to Morgan Stanley .a*ital
International Inc.>
$The %I&! &onsumer 'iscretionary !ector encompasses those industries that tend to #e the
most sensitive to economic cycles( Its manufacturing segment includes automotive) household
dura#le goods) te*tiles + apparel and leisure e,uipment( The services segment includes hotels)
restaurants and other leisure facilities) media production and services) and consumer retailing
and services(-
This category 'as ?udged to be the )ost *ro)ising ,or *redictions. I, the econo)y or stock
)arket as a 'hole is e(hibiting an cycles or *atterns> this category o, stocks is )ost likely to sho' it>
assu)ing that such cycles 'ould be re,lected in share *rices. &urther)ore> ,unda)ental variables o,ten
de*ict business cycles and )ay even be used to *redict the)> )aking the) )ore use,ul than )ore
stable and unchanging categories.
There are> ho'ever> )ore than $7 stocks in the SGP 5%% ,alling into this category. The list o,
stocks 'as ,urther narro'ed to ensure @uality o, data and to ensure that stocks are traded o,ten and by
large a)ounts o, *eo*le> to even out any discre*ancies that )ay be caused in the short ter). As such>
stocks 'ith a high *ercentage ($%L or )ore o, insider o'ners> or 'ith )ore than !5L o'nershi* by
institutional investors> 'ere not included. As 'ere stocks that s*lit a,ter -%%%> 'hen the data is
collected. 9hile an ad?usted closing *rices are used in )ost data to ensure dividend *ay)ents do not
distort *redictions> s*lits have a )uch larger e,,ect (o,ten halving the stock *rice and 'ere convenient
,or narro'ing the list o, stocks.
&inally> stocks 'ith an average volu)e o, less than $ )illion shares> and 'ith a di,,erence
bet'een their 5-3'eek high6lo' *rices greater than 7%L and less than -%L 'ere also not included to
ensure high volu)es o, trading and a larger *otential ,or technical indicators to a**ear based on grou*s
o, investors acting in unison. Stocks that 'ere relatively constant in *rice and highly volatile 'ere
ignored to avoid any distortions they )ay introduce into the neural net'orks *icking u* *atterns.
#elo' is a list o, the ,inal $7 stocks:
Auto0ation> Inc.
.ircuit .ity =rou*
.lear .hannel .o))unications
&ord Motor
=annett .o.
=oodyear Tire G "ubber
Bilton Botels
Bo)e +e*ot
Mattel> Inc.
Mc+onaldEs .or*.
0e' 4ork Ti)es .l. A
0e'ell "ubber)aid .o.
"adioShack .or*
Sher'in39illia)s
Star'ood Botels G "esorts
Tribune .o.
9alt +isney .o.
9endyEs International
9hile it 'ould have been ideal to ,ocus on all $7 stocks> due to a lack o, ti)e (the *ro?ect lasted
about three )onths> it 'as not *ossible to test net'orks on all o, these stocks. #y chance> the stocks
that 'ere ,ocused on 'ere .ircuit .ity and &ord> 'ith a )uch greater e)*hasis on the ,or)er. It is
reco))ended that ,urther research be taken to e(*lore ho' other stocks ,unction in these research
settings> though one should stick to the $7 chosen> or at )ost> deviate only 'ithin the .onsu)er
+iscretionary category.
Part 2: eural et!orks
eurons and eural et!orks
2ike the brain> the basic building block o, any neural net'ork is the neuron. &ro) a biological
*ers*ective> a neuron is ;a cell in the brain 'hose *rinci*al ,unction is the collection> *rocessing> and
disse)ination o, electrical signals.< ("ussell G 0orvig> -%%3> *g. 836 Its ,unction in neural net'orks
is very si)ilar. 2ike biological neurons> arti,icial neurons are connected to other neurons A they can
acce*t in,or)ation ,ro) others> as 'ell as send it to the). "ather than receiving electrical signals>
arti,icial neurons receive a nu)ber ,ro) other neurons> and *rocess these nu)bers accordingly.
5ach arti,icial neuron is ,unda)entally the sa)e. In and o, the)selves> neurons can *er,or)
si)*le calculations based on the in*uts they receive. 9hat is uni@ue and )ost *o'er,ul about the)>
ho'ever> is their ability to *er,or) calculations collectively: by co)bining grou*s o, neurons> one can
*er,or) e(tre)ely co)*le( o*erations> even 'hile each neuron acts inde*endently o, others. &or this
reason> neural net'orks have o,ten been described as ;*arallel distributed *rocess< (P+P net'orks.
Such net'orks are uni@ue because rather than being set u* to solve a s*eci,ic *roble)> the net'orks are
trained to solve *roble)s by )odi,ying ho' they receive in,or)ation ,ro) other neurons.
The )ain dra'back o, such net'orks is their ;black bo(< @uality: a,ter a net'ork is trained> it
)ay be e(tre)ely di,,icult to inter*ret its state to learn 'hich in*uts and @ualities are i)*ortant to the
solution received. Bo'ever> analysis o, 'eights and net'orks in the *ast has been success,ul in
learning the relationshi*s bet'een in*uts and out*uts o, a neural net'ork (4ao> Teng> Poh G Tan>
$!!7J +a'son> -%%/. This is e(citing in the conte(t o, stock )arkets> because i, a neural net'ork is
trained to *redict stock *rice ,luctuations> there is a chance A albeit a s)all one A that the net'orkEs
state could be analyCed to learn 'hich ,actors are use,ul in ,orecasting changes in stock *rices.
0eurons are all ,unda)entally the sa)e> though )ay di,,er in s*eci,ic details> such as
connections and ,unctions used to analyCe data. 9hen neurons are co)bined in such a 'ay> they o,ten
take one o, three roles: in*ut> hidden> and out*ut neurons. In*ut neurons are ones that are ,ed data ,ro)
an e(ternal source> such as a ,ile> rather than other neurons. Bidden neurons acce*t their in,or)ation
,or) other neuronsE out*uts. 1ut*ut neurons are like hidden neurons> but rather than *assing the
*rocessed in,or)ation to a ne' set o, neurons> they save the in,or)ation and allo' it to be read and
inter*reted by an e(ternal actor.
&or e(*lanatory *ur*oses> a neuron )ay be broken do'n into three *arts:
in*ut connections
su))ing and activation ,unctions
out*ut connections
Input #onne"tions
Onless the arti,icial neuron is an in*ut neuron> a neuron is connected to other neurons and
de*ends on the) to receive the in,or)ation that it *rocesses. There is no li)it to the a)ount o,
connections a neuron )ay receive in,or)ation ,ro).
The in,or)ation that a neuron receives ,ro) others is regulated through the use o, 'eights.
9hen a neuron receives in,or)ation ,ro) other neurons> each *iece o, in,or)ation is )ulti*lied by a
'eight 'ith a value bet'een 3$ and $> 'hich allo's the neuron to ?udge ho' i)*ortant the in,or)ation
it receives ,ro) its in*ut neurons is. These 'eights are integral to the 'ay a net'ork 'orks and is
trained: s*eci,ically> training a net'ork )eans )odi,ying all the 'eights regulating in,or)ation ,lo' to
ensure out*uts are correct.
$umming and A"tivation Fun"tions
The second *ortion o, a neuron is the su))ing and activation ,unctions. The in,or)ation sent
to the neuron and )ulti*lied by corres*onding 'eights is added together and used as a *ara)eter
'ithin an activation ,unction. In a biological conte(t> a neuron beco)es activated 'hen it detects
electrical signals ,ro) the neurons it is connected to (1E"eilly G Munakata> -%%%. I, these signals are
su,,icient> the neuron 'ill beco)e ;activated< 3 it 'ill send electrical signals to the neurons connected
to it. An activation ,unction is si)ilar: the arti,icial neuron 'ill out*ut a value based on these in*uts. It
is al)ost al'ays the case that a neuron 'ill out*ut a value bet'een P%> $Q or P3$> $Q> and this
nor)aliCation o, data occurs by using the su))ed in*uts as a *ara)eter to a nor)aliCing ,unction>
called an ;activation ,unction<.
0u)erous activation ,unctions e(ist> but 'ithin this *ro?ect> three ty*es o, activation ,unctions
'ere e(*lored:
Threshold Function: a si)*le ,unction that co)*ares the su))ed in*uts to a constant> and
de*ending on the result> )ay return a 3$> %> or $. S*eci,ically> ,or su))ed in*ut
0eural 0et'ork Predictions o, Stock Price &luctuations $! 6 /-
f ( *)=
$ if *>%
% if *=%
$ if *%
Piecewise./inear Function: i, the su))ed in*uts are in the range P3%.5> %.5Q> the value is ke*t
as be,ore> 'hile anything else 'ill return a 3$ or $:
f ( *)=
$ if *
$
-
% if
$
-
*
$
-
=%
$ if *
$
-

0yper#olic Tangent Function: a continuous ,unction 'ith a do)ain o, (3> and a range o,
(3$> $. S*eci,ically (9eisstein> $!!!>
tanh ( *)=
e
-(
$
e
-(
+$
9ithin this *ro?ect> the tanh(( ,unction is )ost used. #y *roviding a ,unction 'ith a li)itless
do)ain and a range o, (3$> $> it is *er,ect ,or *redicting 'hether or not a stock 'ill rise (tanh(( M $ or
,all (tanh(( M 3$. It should also be noted that the above ,unctions> 'hile having ranges o, P3$> $Q also
have corres*onding versions 'ith ranges P%>$Q. &or e(a)*le> the sig)oid ,unction> belo'> is a
re*lace)ent ,or the hy*erbolic tangent:
f ( *)=
$
$+e
a*
'here a allo's one to set the slo*e o, the ,unction.
'utput #onne"tions
&inally> once the activation ,unction returns a corres*onding value ,or the su))ed in*uts> these
values are sent to the neurons that treat the current neuron as an in*ut. The *rocess re*eats again> 'ith
the current neuronEs out*ut being su))ed 'ith others> and )ore activation ,unctions acce*ting the su)
o, these in*uts. The only ti)e this )ay be ignored is i, the current neuron is an out*ut neuron. In this
case> the su))ed in*uts and nor)aliCed su) is sent as an out*ut and not *rocessed again.
0eural 0et'ork Predictions o, Stock Price &luctuations -% 6 /-
Feedfor!ard et!orks
9hile each neuron is> in and o, itsel,> a co)*utational unit> neurons )ay be co)bined into
layers to create co)*le( and e,,icient grou*s that can learn to distinguish bet'een *atterns 'ithin a set
o, given in*uts. Indeed> by co)bining )ulti*le layers o, such grou*s> it is theoretically *ossible to
learn any *attern.
There are )any co)binations o, neurons that allo' one to create di,,erent ty*es o, neural
net'orks> but the si)*lest ty*e is a single3layer ,eed,or'ard net'ork. In this case> a net'ork is
co)*osed o, three *arts: a layer o, in*ut nodes> a layer o, hidden neurons> and a layer o, out*ut nodes.
9ithin this net'ork> there is a neuron ,or each in*ut variable in an in*ut *attern> 'hich is then
*ro*agated to the layer o, hidden neurons. As )entioned earlier> each neuron in the hidden layer has
the in*uts )ulti*lied by a 'eight> and all in*uts are su))ed. A,ter this is done> the value is *assed to
an activation ,unction> and each neuron in the hidden layer then *asses the out*ut on to neurons in an
out*ut layer> 'hich also )ulti*ly the values by 'eights and su) the).
A )ultilayer ,eed,or'ard net'ork is si)ilar to a single3layer one. The )ain di,,erence is that
instead o, having a hidden layer *ass its calculated values to an out*ut layer> it *asses the) on to
another hidden layer.
#oth ty*es o, net'orks are ty*ically i)*le)ented by ,ully connecting each layerEs neurons 'ith
the *receding layerEs neurons. Thus> i, /ayer A has k neurons and sends its in,or)ation to /ayer 1>
'ith n neurons> each neuron in /ayer A has n connections ,or its calculated out*ut> 'hile each neuron
in /ayer 1 has k in*ut connections.
Interestingly> such a net'ork can be re*resented )athe)atically in a si)*le )anner. Su**osing
there are k neurons in /ayer A> let a re*resent a vector> 'here a
i
is the i
th
neuronEs activation ,unction
out*ut. 2et # re*resent the in*ut values to neurons in /ayer 1> 'ith #
?
be the 2
th
neuron. 2et W be a n by
k )atri( 'here w
?i
re*resents the 'eight a,,ecting the connection ,ro) a
i
to #
?
. Kee*ing this in )ind> 'e
can see that ,or a single3layer ,eed,or'ard net'ork> 'e can )athe)atically re*resent the ,lo' o,
in,or)ation by>
Wa M b
and the learning thus beco)es a )odi,ication o, each w
?i
in W. A si)ilar )athe)atical analogy a**lies
to )ultilayer ,eed,or'ard net'orks> but in this case> there is a W ,or every layer and b is used as the
value ,or a 'hen )oving to subse@uent layers.
0eural 0et'ork Predictions o, Stock Price &luctuations -$ 6 /-
The )ost *o*ular ty*e o, learning 'ithin a single3layer ,eed,or'ard net'ork is the +elta "ule>
'hile )ultilayer ,eed,or'ard net'orks i)*le)ent the #ack*ro*agation algorith)> 'hich is a
generaliCation o, the +elta "ule.
Ba"kpropagation
#oth the ,eed,or'ard and recurrent net'orks in this *ro?ect i)*le)ent the back*ro*agation
algorith)> one o, the )ost co))on learning )ethods 'ithin neural net'orks. 9hile 'idely used> the
algorith) has its *ros and cons> and it hel*s to understand ho' it 'orks.
The back*ro*agation algorith) is use,ul in that it generaliCes the +elta "ule ,or single3layer
,eed,or'ard neural net'orks (1E"eilly G Munakata> -%%% to )ultilayer ,eed,or'ard ones. 1ne o, the
biggest *roble)s in training neural net'orks is the .redit Assign)ent Proble) (Baykin> $!!!: ho'
does one assign credit to individual neurons ,or errors in the ,inal out*uts o, a neural net'orkR
The +elta "ule is one solution to this *roble)> but only a**lies to single3layer ,eed,or'ard
net'orks. 9hile its derivation is beyond the sco*e o, this *ro?ect> the +elta "ule )ay be su))ariCed
'ith the ,ollo'ing e@uation:
Aw
i2
=c6
2
*
i
In this case> Sw
i?
re*resents the change o, the 'eight connecting the i
th
neuron 'ith the 2
th
out*ut
neuron> *
i
is the out*ut value o, the i
th
neuron> 3 is the learning rate> and 4
?
is the error ter) in the out*ut
layer> de,ined as:
6
k
=(t
k
o
k
)
'here t
k
is the e(*ected out*ut> 'hile o
k
is the actual out*ut.
9hile this rule 'orks 'ell 'hen there is only one hidden layer> credit assign)ent beco)es
)uch )ore di,,icult 'ith )ulti*le hidden layers. &or this> one )ust use a ne' e@uation:
6
2
=(
k
6
k
w
2k
)(h
2
($h
2
))
In this case> one uses the sa)e e@uation ,or Sw
i?
but uses the ter) above instead> 'ith k re*resenting
the neurons receiving in,or)ation ,ro) the current neuron being )odi,ied. 4
k
w
?k
is the error ter) o, the
k
th
neuron in the receiving layer> 'ith w
?k
being the connecting 'eight. The activation ,unctions o, all
the neurons in a net'ork i)*le)enting back*ro*agation )ust be di,,erentiable> because:
h
2
=c5 ( 6
2
)
'ith 6
?
being the net in*ut ,or the neuron.
0eural 0et'ork Predictions o, Stock Price &luctuations -- 6 /-
&inally> i, biases are *resent> they are treated like regular neurons> but 'ith their ( values e@ual
to $:
AB
2
=c 6
2
9hen i)*le)ented> this algorith) has t'o *hases. The ,irst deals 'ith having the net'ork evaluate the
in*uts 'ith its current 'eights. 1nce this is done> and all the neuron and out*ut values are recorded>
*hase t'o begins. The algorith) begins this *hase by a**lying the original +elta "ule to the out*ut
neurons> )odi,ying 'eights as necessary. Then the generaliCed +elta "ule is i)*le)ented> 'ith the
*revious T values sent to hidden neurons> and their 'eights changing as re@uired.
rawba"ks
5ssentially> back*ro*agation is a learning algorith) that allo's a net'ork to ,ind a state that
)ini)iCes the a)ount o, error the net'ork e(hibits (.hurchland G Se?no'ski> $!!/. It does this by
)odi,ying 'eights connecting neurons using a gradient descent (Arbib> -%%3> 'hich allo's the
'eights to be )odi,ied in a )anner ,ollo'ing a negative slo*e along the ;error landsca*e< A a n3
di)ensional sur,ace re*resenting all *ossible errors 'ithin the net'ork.
The *rocess> ho'ever> has a ,e' dra'backs. &irst and ,ore)ost> back*ro*agation does not
guarantee to ,ind the global net'ork )ini)u). 9hile it does )ini)iCe error> there is a chance the
'eights 'ill be changed to ,it a local )ini)u) in the error landsca*e> but the net'ork 'ill not be
o*ti)iCed.
9ith the above in )ind> the learning coe,,icient (3 )ust be chosen 'ith great care A one too
s)all 'ill ,orce the net'ork to take an e(cruciatingly long ti)e in o*ti)iCing itsel,> 'hile a large one
'ill ,orce a net'ork to oscillate around a )ini)u) value. Another *otential solution is using a
momentum term U (Baykin> $!!!. Such a ter) can allo' the delta and generaliCed delta rules to
)odi,y 'eights at di,,erent )agnitudes> rather than have the) de*end on a constant learning
*ara)eter.
This dra'back is es*ecially i)*ortant ,or this *ro?ect because o, the co)*le(ity o, the data
being dealt 'ith. Baykin 'rites that ,or co)*le( net'orks )odeling real 'orld *heno)ena> ;It is
generally agreed that it is inadvisable ,or a )ultilayer *erce*tron to be ,ully connected.< ($!!!> *g.
-33 This *roved to be a challenging issue in this *ro?ect> )ainly because the nu)ber o, 'eights in
co)*le(> ,ully3connected net'orks 'as o,ten )uch greater than that o, the total nu)ber o, training
*atterns.
0eural 0et'ork Predictions o, Stock Price &luctuations -3 6 /-
Ba"kpropagation Applied
9ithin the research *ro?ect> back*ro*agation 'as used 'ithin all the ,eed,or'ard net'orks. The
algorith) is e(tre)ely e,,ective at learning *atterns> and is *ro,icient in ada*ting to various ty*es o,
data. The Hava 0eural 0et'ork Si)ulator (H00S allo's ,or the creation o, neural net'orks
i)*le)enting back*ro*agation> and allo's one to )odi,y a nu)ber o, settings> including (Nell>
Ma)ier> Vogt> Mache> BVbner G +Wring> n.d.:
X: the learning rate (Y above.
T
)a(
:the )a(i)u) error rate back*ro*agated in the net'ork.
cycles: the nu)ber o, ti)es H00S should run through the training data.
ste*s: the nu)ber o, ti)es each in*ut should be *ro*agated through the net'ork during a cycle.
&or net'orks i)*le)enting back*ro*agation> $ ste* is su,,icient.
shu,,le: asks 'hether or not each in*ut *attern should be shu,,led. I, unchecked> the in*uts are
sent to the net'ork in the sa)e order each ti)e.
The various o*tions above give one an unli)ited a)ount o, choice in setting u* a net'ork> and
choosing the settings )ust be done 'ith care: incorrect settings can sto* a net'ork ,ro) converging on
the training data and *atterns held therein.
9ithin this *ro?ect> the learning *ara)eter takes values o, %.%$> %.%%3> and %.%%%5. Initially>
values 'ere set higher> near %.$> but it a**ears that the net'orks using such a learning rate 'ere
ine,,ective at learning in*ut *atterns> and the value 'as abandoned to conserve co)*uting ti)e and
,ocus resources on )ore accurate settings.
+ue to li)itations 'ith co)*uting> cycles 'ere o,ten ke*t at -5%%% and lo'er. Again> al)ost all
net'orks being analyCed 'ithin this *ro?ect 'ere tested at -5%%%> $%%%%> 5%%%> $%%%> and 5%% cycles.
In the e(a)*les *resented belo'> only the best e(a)*les are sho'n> but statistics and in,or)ation on
all such trained net'orks is available.
The learning rate and nu)ber o, cycles 'ere the only t'o settings that 'ere e(*eri)ented 'ith.
&eed,or'ard net'orks are not de*endent on *revious or ,orthco)ing in*ut *atterns and as such> the set
'as shu,,led to avoid overtraining o, the net'ork to the s*eci,ic order being *resented. &urther)ore>
the nu)ber o, ste*s 'as ke*t constant to avoid any discre*ancies in the data. The nu)ber 'as ke*t at
$%%.
T
)a(
'as ke*t at %.% at all ti)es as 'ell. +ue to the nature o, back*ro*agation> it is i)*ortant to
0eural 0et'ork Predictions o, Stock Price &luctuations -/ 6 /-
kee* the out*uts in a net'ork continuous to allo' ,or the error rate bet'een out*uts and correct
*atterns to get s)aller. As such> *redictions ,ro) the neural net'orks range ,ro) 3$ to $> and are o,ten
in the )iddle. 9hile such *redictions allo' the net'ork to e(hibit con,idence in *redictions (i.e. a
*rediction o, $.% is )ore likely to be correct than one o, %.!> this con,idence is relative> and it is
i)*ossible ,or one to kno' 'hether an out*ut o, %.65 )eans the stock is likely to rise> or i, it )eans
the net'ork is unsure. #y setting T
)a(
to a value greater than %> one )akes an assu)*tion that a
*rediction other than $ or 3$ can still be correct.
Such an assu)*tion> ho'ever> is not necessarily incorrect. "ather than setting T
)a(
,ro) H00S>
ho'ever> an analysis 'as undertaken o, *redictions to see ho' accurate *redictions are at various
threshold values.
T)res)old Analysis
#y leaving T
)a(
e@ual to %> H00S kee*s training a net'ork until all neurons )ake correct
*redictions o, $ or 3$ ,or all in*uts> or until the nu)ber o, training cycles is co)*lete. +ue to the
a)ount o, training data (over $>%%% in*uts in all cases> the net'ork never converged on the training
data. As such> )any *redictions ,or ,uture *rices 'ere not e@ual to Z$ but 'ere so)e'here in bet'een.
It 'as assu)ed that the closer a *rediction 'as to Z$> the )ore sure the net'ork 'as that the *rediction
'as correct. Again> this 'as *urely relative: a value o, %.65 'as )ore accurate than %.!%> but one
cannot be sure ho' )uch )ore accurate.
To test 'hether or not this 'as true> a threshold analysis 'as *er,or)ed on *redictions to see
ho' accurate they get. A threshold> [> 'as selected> and the ,ollo'ing ,unction a**lied:
f ( *)=
$ if *
$ if *>
% otherwise
'ith 3$ )eaning the stock 'ill ,all> and $ )eaning the stock 'ill rise. A nu)ber o, di,,erent thresholds
'ere chosen> and the gra*h belo' sho's the threshold analysis ,or .ircuit .ity (...
In the case above> it is clear that as the [ a**roaches $> the accuracy o, the net'ork increases>
'hile the nu)ber o, *redictions decreases. It is clear that by having setting [ M %.!> one can have a
)eaning,ul nu)ber o, accurate *redictions. &ro) the above chart> it 'as decided that [ M %.85 'as an
acce*table threshold value.
"ecurrent et!orks
9hile ,eed,or'ard net'orks are e(tre)ely e,,icient at learning static *atterns> it is di,,icult to
have such a net'ork learn *atterns that change over ti)e. The )ost i)*ortant reason ,or this is that
,eed,or'ard net'orks do not take into account earlier in*ut *atterns 'ithin their analysis o, a *attern
*resented to the). To deal 'ith this issue> ,eed,or'ard net'orks 'ere *resented 'ith a ,e' daysE 'orth
o, historical data. Bo'ever> this a**roach is li)ited to the nu)ber o, in*ut *atterns one *resents> and
also drastically increases the a)ount o, in*uts.
A *otential solution to this *roble) is the use o, recurrent net'orks. "ecurrent neural net'orks
are net'orks 'ith neurons that have ,eedback connections A connections to the)selves> or send their
out*uts to a *revious layer. These ,eedback connections can be a global or local ty*e. &or e(a)*le> a
neuron )ay send its out*ut to itsel,> allo'ing it to incor*orate this in,or)ation in the ne(t *atternEs
analysis> or the out*ut layer )ay send its ,inal ans'er as an in*ut ,or the ne(t *attern.
-hreshold Anal*sis .&&/
0
20
40
60
80
100
120
140
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
-hreshold
"
u
+
(
e
r

o
0

P
r
e
d
i
c
i
o
n
s
40%
50%
60%
70%
80%
90%
100%
&
o
r
r
e
c

P
r
e
d
i
c
i
o
n
s

.
1
/
# of Predictions (500 Cycles) # of Predictions (10000 Cycles) % Correct (500 Cycles) % Correct (10000 Cycles)
.lman &etworks
There are nu)erous recurrent net'ork architectures> but one o, the si)*lest is the 5l)an
net'ork. Such net'orks are based on ,eed,or'ard net'orks in that they have a ty*ical in*ut layer and
hidden layers sending each other in,or)ation in a se@uential order. The )ain di,,erence> ho'ever> is
that every hidden layer sends its out*uts to a coincident layer> 'here the in,or)ation is stored and sent
back to the hidden layer during the ne(t *ass3through in the net'ork. 5very neuron in this coincident
layer ('ithin S00S has a ,eedback loo* to itsel, as 'ell.
Such a setu* allo's 5l)an net'orks to retain in,or)ation ,or longer than one *ass3through in
the net'ork. A,ter the ,irst *ass> subse@uent *asses 'ill have in,or)ation based on all earlier atte)*ts>
and it is assu)ed that this 'ill allo' the net'ork to incor*orate *revious in,or)ation into its o'n
*redictions.
I)*le)enting such a net'ork 'ithin H00S is di,,icult> but its *redecessor> the Stuttgart 0eural
0et'ork Si)ulator (S00S> contains a tool that builds an 5l)an net'ork based on a list o, variables>
including nu)ber o, in*uts> out*uts> and nu)ber o, hidden net'orks. Training takes *lace through the
use o, a )odi,ied version o, back*ro*agation that a**lies the generaliCed delta rule to coincident
layers. &urther)ore> all training ,unction *ara)eters are as in the case ,or back*ro*agation> but 'ith
one addition> a teacher ,orcing *ara)eter (Nell> Ma)ier> Vogt> Mache> BVbner G +Wring> n.d.. "ather
than training based on the out*ut o, the net'ork> one can use a 'eighted average bet'een the e(*ected
out*ut and actual out*ut o, the net'ork. &or e(a)*le> i, the *ara)eter is set at %.%> the training is based
solely on the original out*ut> 'hile $.% )eans the e(*ected out*ut is used to train the net'ork. The
de,ault value in S00S is %.5.
Input Layer
Hidden Layer Coincident Layer
Output Layer
Part #: "esults
Statistical Analysis
9ith the t'o net'ork ty*es listed above and data ,or .ircuit .ity (..> &ord (&> and
Mc+onalds (M.+> a total o, $! di,,erent trials 'ere run that e(*lored ho' various ty*es o, net'ork
architectures and data in*uts a,,ected *redictions ,or stock *rices 'ithin $ day> $ 'eek> and / 'eek
*eriods. 9hile nu)erous co)binations 'ere e)*loyed> all o, 'hich are e(*lained in the a**endi(> it is
i)*ortant to note that this is by no )eans a co)*rehensive list. I, a researcher constrains hi) or hersel,
to ?ust 5l)an and ,eed,or'ard net'orks> the nu)ber o, o*tions he or she has are still e(tre)ely large>
and i)*ossible to analyCe in a ,e' )onths. Bo'ever> it is ho*ed the results belo' shed light on 'hat
ty*e o, ,eed,or'ard or recurrent net'ork )ay *rovide the best results ,or *redicting *rice ,luctuations
in share *rices o, s*eci,ic stocks.
+ue to the breadth o, this *ro?ect> a ,e' i)*ortant notes )ust be ke*t in )ind 'hen co)*aring
data and results bet'een net'orks. This *ro?ect 'as )eant as an acade)ic study into neural net'orks
and stock *rices and as such> no atte)*ts 'ere )ade to develo* a *otentially *ro,itable trading strategy
based on these results. &urther)ore> because o, the )any )odels e(*lored> the data in*uts as 'ell as
nu)ber o, *redictions di,,ers bet'een so)e net'orks.
&urther)ore> longer ti)e *eriods tend to ,ocus on *atterns that reach ,urther into the *ast>
)aking it di,,icult to co)*are short3ter) results and the actual nu)bers o, *redictions. This is
es*ecially true ,or net'orks solely using ,unda)ental indicators: due to the constraints o, the data
(,ocusing bet'een -%%% and -%%5> training sets 'ere built 'ith only 55 training units and $%
validation *atterns> 'hile other sets had over $-% validation *atterns and $$%% training ones.
A net'orkEs ;correctness< is )easured by the nu)ber o, correct *redictions above the threshold
> as discussed earlier. It is assu)ed that 'hen a value is belo' [> the net'ork is unsure o, the data and
does not )ake a reliable *rediction> so those 'ere ignored. As such> the *ercentage accuracies
discussed later are:
Accuracy=
"um#er of &orrect Predictions with !core A#ove
"um#er of Total Predictions with !core A#ove
&or an individual net'ork> the accuracy above is co)*ared to t'o other values. The ,irst o, these is
5%L> 'hich signi,ies the accuracy o, a *er,ectly rando) *rediction syste).
The second value 7> is based on the #ernoulli +istribution. #ecause every stock *rice either
goes u* or do'n in a day> a stock *rice can be treated as a variable ! \ #ernoulli(7 that takes a value
o, 3$ or $. #ased on the training data> the best esti)ate ,or 7> re*resenting the *robability a stock *rice
'ill rise> is:
0=
"um#er of 'ays A!tock Price Increased
Total "um#er of 'ays
In this case 7 also re*resents the *ercentage that a stock *rice rises. This is i)*ortant ,or co)*arisons>
because i, one uses a co)*letely rando) *redictor based on S \ #ernoulli(7> one can get a better result
than an unin,or)ed guess 'hile still de*ending on a rando) variable.
The accuracy o, individual net'orks 'as thus co)*ared to both ] and %.5> 'hile grou*s o,
net'orks 'ere co)*ared to to the t'o values using t3tests. It is i)*ortant to note that de*ending on the
trial> 7 took on di,,erent values.
1ne caveat re)ains in )aking co)*arisons. 9hile a net'ork )ay be accurate $%%L o, the
ti)e> due to the de,initions above> this )ay not be signi,icant. A nu)ber o, net'orks (notably Trial
$/> had e(tre)ely ,e' *redictions A so)eti)es as little as - or 3. +ue to this> a net'ork could have a
$%%L accuracy rate 'ithout being use,ul. It is i)*ortant to check the nu)ber o, total *redictions i, one
sees such high (or lo' rates o, accuracy.
"esults
The results o, the various trials and net'orks tested are very *ro)ising> 'ith a nu)ber o,
net'orks sho'ing very good results. The analysis took *lace through t'o di,,erent grou*s A by
,ocusing on the various trials and )aking generaliCations about the ty*es o, net'orks therein> and by
co)*aring s*eci,ic net'orks. In all cases> a threshold value o, %.85 'as used.
9hile nu)erous grou*s o, net'orks 'ere tested> the grou*s belo' sho' *ro)ising results. In
this case> *
%.5
re*resents the *3value ,or the grou* o, net'orks 'hen testing ,or average accuracy o, %.5>
'hile the *
]
is the *3value ,or 'hen testing i, accuracy is e@ual to ]. +etailed in,or)ation about each
trial and the )eaning o, the na)e )ay be ,ound in the a**endi(.
0eural 0et'ork Predictions o, Stock Price &luctuations -! 6 /-
Trial Name Avg. Accuracy
!."
#
#
$orecast%
Trial 3& ' ( !.!!!" 55.-%L %.%3-/ %.5%3! %.%/$!
Trial "& ' ( !.!!!" 56.%$L %.%%!6 %.5%87 %.%$53
Trial 1"& ' ( !.!1 6$.%6L %./$$% %.5/65 %.6$$%
Trial 1"& ' ( !.!!3 6%.3!L %.$!-7 %.5/65 %.3-87
1 )ay
Trial 11& ' ( !.!1 5/.!3L %.$356 %.65$- %.%$// " )ays
Trial 12& ' ( !.!1 7-.8%L %.%%%5 %.8%%% %.%$65
Trial 12& ' ( !.!!3 7%.33L %.%%$% %.8%%% %.%/-3
Trial 13& ' ( !.!1 6$.78L %.%%%$ %.8//- %.%%%%
Trial 13& ' ( !.!!3 58.75L %.%-!! %.8//- %.%%38
Trial 1*& ' ( !.!!3 58.$3L %.$375 %.8//- %.%$66
2! )ays
8 #usiness days
Feed%orward &etworks and Types o% $to"ks
The results above are interesting ,or a nu)ber o, reasons. &irst and ,ore)ost> Trial 3 re*resents
a si)*le one3layer ,eed,or'ard net'ork trained 'ith a learning *ara)eter o, %.%%%5. It 'as noticed
'ithin )any o, the ,eed,or'ard net'orks that as the learning *ara)eter decreases> accurate results
increase. 9hether this is because the net'ork avoids overtraining or because the data set (and stocks
'ere e(tre)ely sensitive to in,or)ation ,lo' 'as undeter)ined. Trial 3 ,ocused on .ircuit .ity (..>
'hile Trial /> not sho'n above> ,ocused on &. Interestingly> & 'as statistically signi,icant ,or ho' *oor
its results 'ere. &ro) this> one can gather that so)e stocks are )ore inclined to ,ollo' *redicted *aths
than others. &or e(a)*le> &Es volu)e 'as $% ti)es as high as that o, ..> and .. has t'ice as )any
institutional investors as &> in *ercentage ter)s. .. 'as also )ore volatile than &. This> along 'ith the
ty*es o, *roducts (consu)er retail versus auto)obiles and investor *sychology could all *lay a role in
ho' *redictable the stocks are.
9hat is interesting> ho'ever> is that 'hile & did not 'ork 'ell 'ith neural net'orks and ..
did> Trial 5 'as once again statistically signi,icant. Trial 5 re*resents nothing )ore than a co)bination
o, the training and validation data o, Trials 3 and /. +oing so doubled the a)ount o, in,or)ation
*resented to each net'ork 'ithout changing the net'orkEs structure> )aking overtraining less likely
'hile also *resenting )ore *atterns to train the net'ork.
9ith )ore ti)e> it 'ould be interesting to see ho' incor*orating a larger grou* o, stocks could
hel* *ro)ote net'ork convergence. 9hile the ;.onsu)er +iscretionary< category o, stocks 'as used>
these also have subcategories A ..> ,or e(a)*le> is in the "etailing category> and *resenting a net'ork
0eural 0et'ork Predictions o, Stock Price &luctuations 3% 6 /-
'ith a nu)ber o, such stocks could allo' it to learn general and industry3s*eci,ic *atterns.
An interesting observation 'ith the data above also relates to the nu)ber o, cycles re@uired to
obtain a good rate o, training. 9hile Trial 3 re@uired $%%%%% cycles> Trial 5 re@uired $%%%. This )ay
be a result o, the rando)ness o, 'eight initialiCations. So)e 'ork> *resented in A**endi( .> 'as done
in this regard> and sho's that the use o, rando) 'eights can indeed ske' results. The best solution to
this *roble) is to run )ulti*le training sessions on the sa)e net'orks and see 'here )ost o, the)
converge. Bo'ever> this could not be done in this *ro?ect due to a lack o, ti)e.
't)er &etwork Ar")ite"tures %or 0,ay !redi"tions
9hat is signi,icant about the $3day *redictions is that t'o3layer or three3layer ,eed,or'ard
net'orks are not *resent. 9hile additional layers can hel* a net'ork converge> they )ay also re@uire
)ore training and 'ith the cycles and li)ited )ultilayer architectures used> use,ul results 'ere not
obtained. 9hen increasing the nu)ber o, layers> ho'ever> the nu)ber o, nodes *er layer 'as
decreased to avoid overtraining. 9ith )ore data> such a concern and li)itation )ay not be necessary.
"ecurrent net'orks 'ere used ,or $3day *redictions as 'ell> and *roved to be interesting in ho'
they 'ork. A one hidden layer 5l)an net'ork 'as constructed> 'here only the *revious dayEs
in,or)ation 'as sent to the net'ork. +ue to the net'orkEs recurrent nature> it should theoretically take
into account all *revious data sets 'ithin its *redictions. This net'ork> ho'ever> did *oorly.
A second ,eed,or'ard 6 5l)an hybrid 'as constructed> 'here in*uts 'ere *rovided ,or ,ive
days> as in the ,eed,or'ard net'ork> but the net'ork also had a coincident hidden layer 'here *revious
hidden layer out*uts 'ere sent. This net'ork *roved to be )ost accurate> 'ith an average accuracy rate
higher than that o, the ,eed,or'ard net'orks. It should be noted ho'ever> that the validation sets 'ere
di,,erent because the 5l)an net'orks 'ere recurrent and could not be easily tested on data at the
)iddle o, the training set> 'hile ,eed,or'ard net'orks could.
!ri"es
9hile )ost net'orks received in*uts in the ,or) o, technical or ,unda)ental indicators> Trial
$! 'as a uni@ue trial in that it tested 'hether or not a net'ork could )ake *redictions a,ter receiving
nothing )ore than in,or)ation on a stockEs *ast *rices. A single3layer ,eed,or'ard net'ork 'as
*resented 'ith 5% days o, historical *rices and 'as trained to )ake *redictions ,or $> 5> and -% business
0eural 0et'ork Predictions o, Stock Price &luctuations 3$ 6 /-
days. &or each ti)e ,ra)e> a net'ork 'as trained at both -5%%% and $%%%%% cycles. &or $ and 5 day
*eriods> the results 'ere very *oor> 'ith net'orks )aking ,e'> i, any *redictions over a threshold o,
%.85. 9hat is interesting> ho'ever> is the net'orkEs ability to )ake *redictions ,or -% days. At $%%%%%
cycles> the net'ork al'ays *redicts the stock 'ill go u*> 'hich does ha**en the )a?ority o, the ti)e>
)aking the net'ork correct 8$.3-L o, the ti)e. 9hen trained at -5%%% cycles> it )akes -5 *redictions>
all o, 'hich *redict u*'ard )ove)ent> and the net'ork is correct !6.%%L o, the ti)e. 9hile )ore
study is 'arranted to ?udge 'hether such results are signi,icant> it is *ossible that the net'ork is
generaliCing its *redictions to 'hat the stockEs long3ter) gro'th *attern a**ears to be A an u*'ard
direction.
(onger Term !redi"tions and Fundamental Analysis
9hile *redicting ,or $3day *eriods 'as )et 'ith so)e success 'hen it ca)e to the use o,
technical analysis> longer3ter) *redictions did not see) to 'ork so 'ell. Indeed> there 'as no ade@uate
net'ork ,or 53day *redictions> and the one *resented in the gra*h is statistically signi,icant ,or its lack
o, accuracy. 9hile it correctly *redicted al)ost 55L o, all atte)*ts> a net'ork ?ust *redicting an
u*'ard increase in stock *rices every day 'ould get better results. This is because a large *ortion o,
the validation data ,ocused on a region o, .. that gre' regularly. It should be o, so)e signi,icance>
ho'ever> that the net'ork does e(hibit so)e generaliCation.
9hile technical neural net'orks did *oorly ,or 53day and -%3day *eriods> ,unda)ental3based
net'orks in this *eriod did e(tre)ely 'ell> and 'ere the only ones that had statistically signi,icant
average correctness rates. This 'as very sur*rising> as )any discussions be,ore this research ,ocused
on the idea that not only are ,unda)ental neural net'orks di,,icult to i)*le)ent> but are likely to ,ail.
9ithin the table *resented earlier> one should note that Trials $$> $3> and $8 are signi,icantly above
average ,or 5%L accuracy> but belo' average co)*ared to a net'ork *redicting a rise in stock *rices
all the ti)e.
9ith this in )ind> a neural net'ork co)bining both technical and ,unda)ental in*uts 'as
created. The results (Trial $8> 'hile better than )ost atte)*ts> 'ere not as good as the ,unda)ental
net'orks. It should be noted that these net'orks 'ere e(tre)ely large> 'ith 36% in*uts> and as such> a
lack o, co)*uting ability severely detracted ,ro) e,,orts to train the net'orks. #y increasing the
nu)ber o, cycles to $%%%%% or even higher and co)bining )ulti*le stocks> such a net'ork )ay *rove
to be the )ost use,ul in )aking *redictions. Indeed> i, nothing else> such a net'ork could learn to
0eural 0et'ork Predictions o, Stock Price &luctuations 3- 6 /-
ignore everything but the ,unda)ental variables.
Conclusion and Future $ork
9ith the results above> it is clear that even si)*le net'orks like single3layer ,eed,or'ard nets
)ay be o, so)e use in )aking *redictions about ,uture )ove)ents o, stock *rices. The trick> it see)s>
)ay lie in *icking the correct stocks: 'hile .. 'orked 'ell 'ith *redictions> initial atte)*ts at )aking
*redictions 'ith & did not 'ork 'ell. 9hile little ti)e 'as s*ent on analyCing the a**licability o,
various stocks ,or )aking *redictions> it 'ould be interesting to see a larger scale study A o, all $7
chosen stocks> ,or e(a)*le A on ho' 'ell *redictions 'ork ,or various stocks.
1ne o, the initial reasons ,or this *ro?ect 'as to e(a)ine ho' 'ell technical and ,unda)ental
analysis 'orks in *redicting ,uture *rice )ove)ents. In the short3ter) o, $ or 5 business days>
technical analysis is really the only tool one )ay i)*le)ent. 2onger ti)e *eriods> ho'ever> sho' good
results 'hen i)*le)enting ,unda)ental analysis> though the study could be e(*osed to a longer data
set A all the research here 'as restricted to the year -%%% and on'ard.
It 'as ,ound that co)binations o, data A be it through co)bining )ulti*le stocks or )ulti*le s
o, data A is a good 'ay to strengthen a net'ork. +ue to co)*uting li)itations> this 'as not e(*lored to
the e(tent it should> and is an obvious direction ,or ,uture 'ork. A si)ilar li)itation 'as encountered
in e(*loring the e,,ects o, rando) initial 'eights on the net'orks: so)e o, the results sho'ed net'orks
*roviding use,ul results a,ter as little as $%%% training cycles> 'hile si)ilar net'orks re@uired $%%%%%.
A second goal o, this *ro?ect 'as to co)*are recurrent and ,eed,or'ard net'orks. The results
here are interesting. The ty*ical 5l)an net'ork> *roviding only one dayEs historical data and using the
coincident hidden layer to kee* track o, historical in,or)ation> did not ,are 'ell. 9hat is interesting>
ho'ever> is a )odi,ied version o, the 5l)an net'ork> that still received )ulti*le daysE 'orth o,
historical data 'hile kee*ing a coincident layer to kee* track o, longer3ter) historical in,or)ation> did
very 'ell. It 'ould be use,ul to e(a)ine this net'ork architecture ,urther> and see ho' 'ell it 'orks in
other settings.
1verall> it is clear that neural net'orks a**lied to individual stocks can yield statistically
signi,icant *redictions. The net'orks e(*lored here are so)e o, the )ost co))only used 'ithin the
'orld o, neural net'orks> but a**eared to have good results 'ithin the stocks e(*lored. &urther
research is 'arranted> and this 'ork )ay have nu)erous i)*lications 'ithin ,inance> as 'ell as the
theory o, e,,icient )arkets and analysis thereo,.
A%%endices
A%%endi& A: '(er(ie! of Trials
This is a record o, the various trials that took *lace during the *ro?ect. 1nly the )ost general
outline is given> so one can have an idea o, 'hat ty*es o, net'orks 'ere trained> and at 'hat settings.
To obtain all the results> *lease consult the .+ available 'ith the *a*er. Onless other'ise noted>
training took *lace 'ith T
)a(
set at %.% and $%% ste*s *er cycle. &or all ,eed,or'ard nets> the in*uts 'ere
shu,,led> 'hile shu,,ling 'as not used in recurrent net'orks. Si)ilarly> al)ost all trials ,ocused on
.ircuit .ity and used solely technical in*uts. Any re,erence to ;days< should be noted as ;business
days<.
Trial +1
A single3layer ,eed,or'ard net'ork 'ith 5% in*uts> one hidden layer 'ith 5% neurons> and
*redicting ahead ,or one day based on ,ive day historical data. The logistic activation ,unction 'as used
every'here e(ce*t the out*ut> 'hich 'as i)*le)enting the sign(( ,unction. This has a range o, ^3$> %>
$_ (9eisstein> $!!!> )aking out*uts easy to inter*ret. This ,or)at 'as abandoned because the result
'as either 3$ or $> 'hich )eant it 'as either right (error o, % or 'rong (error o, - or 3-> )aking it
di,,icult ,or back*ro*agation to converge.
Trial +2
This trial used ne' activation ,unctions and had the sa)e net'ork ty*e as the ,irst trial. 0o
records 'ere ke*t A validation *atterns 'ere used to ensure that there is so)e convergence a)ong data>
as early results regularly had the sa)e *redictions ,or each in*ut.
Trial +3
Single3layer ,eed,or'ard net'ork 'ith 5% in*uts> one hidden layer 'ith 5% neurons> and
*redicting ahead ,or one day based on ,ive days o, historical data. All activation ,unctions 'ere using
the hyberbolic tangent. Various nu)bers o, learning cycles 'ere used.
0eural 0et'ork Predictions o, Stock Price &luctuations 3/ 6 /-
Trial +,
2ike Trial `3> but a**lied to stock ticker &.
Trial +"
2ike Trial `3> but the data ,or stock tickers & and .. 'ere co)bined.
Trial +-
A net'ork e(actly like that in Trial `3> but instead o, technical indicators> 5% historical days
'orth o, *rices 'ere *ut in and the net'ork 'as trained at $%%>%%% and -5>%%% cycles. Stock ticker
M.+ 'as used. The results 'ere not e(tre)ely relevant> es*ecially since M.+ 'as not used in other
analyses. See Trial `$! ,or ,urther analysis.
Trial +*
Ontrained one3> t'o3> and three3layer net'orks 'ere created ,or stock ticker .. and validation
*attern results saved. 2ike'ise> a one3layer net'ork ,or stock & 'as also created. This 'as used in
so)e initial statistical analysis to see ho' di,,erent these net'orks are ,ro) their trained counter*arts>
as 'ell as rando) *redictions.
Trial +.
A t'o3layer ,eed,or'ard net'ork 'ith -5 neurons in each hidden layer and hy*erbolic tangent
,or its activation ,unctions.
Trial +/
A three3layer ,eed,or'ard net'ork 'ith $8 neurons in each layer and hy*erbolic tangent ,or its
activation ,unctions.
Trial +1!
&ive3day *redictions ,or a single3layer ,eed,or'ard neural net'ork 'ith one hidden layer
containing 5% neurons and $%% in*ut neurons ($% days o, historical data.
Trial +11
2ike Trial `$%> but the hidden layer had $%% neurons.
Trial +12
$ )onth ($5
th
o, every )onth *redictions based on ,unda)ental data. A single3layer
,eed,or'ard net'ork 'as used> receiving ,unda)ental indicators ,or the three )onths *rior to the
)onth in @uestion. Since the *ro?ect only ,ocused on the years -%%% to -%%5> very little data 'as
available> and only $% di,,erent in*uts ,or validation 'ere available.
Trial +13
A single3layer ,eed,or'ard net'ork receiving -%% in*uts (-% days 'orth o, historical data and
*redicting ,or -% days ahead.
Trial +1,
An 5l)an net'ork 'ith $% in*uts (,or the current day> *redicting the outco)e o, the *rice the
,ollo'ing day. The hidden layer and coincident layers both had $% neurons as 'ell. Teacher ,orcing
'as le,t at %.5> and Hordan 5l)an #ack*ro*agation 'as used.
Trial +1"
An 5l)an net'ork 'ith 5% in*uts> or 5 days 'orth o, historical data> *redicting the ne(t dayEs
results o, the stock )arket *rice.
Trial +1-
Various 5l)an net'orks 'ith Teacher &orcing set to %.-5> %.5%> and %.85. The goal o, this 'ork
'as to see i, Teacher &orcing had any real e,,ect on learning.
Trial +1*
.o)bination o, ,unda)ental and technical indicators. +ata 'as *ut in ,or the *ast -% days>
'hich resulted in 36% in*uts. The net'ork 'as a one3layer ,eed,or'ard net'ork> 'ith 5% neurons in the
hidden net'ork. Since the nu)ber o, 'eights co)*ared to nu)ber o, in*ut *atterns 'as so high> T
)a(
'as set to %.$.
Trial +1.
The sa)e neural net'ork as Trial `3> but -% such net'orks 'ere trained at 5%% cycles 'ith
rando) 'eight initialiCations to test ,or the range o, *redictions and variance caused by the rando)
'eights. All net'orks above also had rando) 'eight initialiCations> and it 'as i)*ortant to see ho'
likely it 'as that the results achieved above 'ere due to the initial 'eights.
Trial +1/
Theoretically> the only thing one needs to have neural net'orks )ake *redictions about share
*rices using technical analysis is *ast *rices. This trial ,ocused on ho' 'ell a one3layer hidden
net'ork> like the one in Trial `3> 'ould 'ork on *redicting ,or $> 5> and -% business days based on the
*ast 5% days o, *rices. $5%%%% and -5%%% cycles 'ere used ,or training> 'ith 5 ste*s and a learning
*ara)eter o, %.%%3.
A%%endi& ): '(er(ie! of Custom Soft!are
9hile the Hava 0eural 0et'ork Si)ulator (H00S and Stuttgart 0eural 0et'ork Si)ulator
(S00S 'ere used ,or creating and training the neural net'orks> and 5(cel and " 'ere used to analyCe
data> a nu)ber o, *ieces o, so,t'are 'ere created to ,acilitate analysis and research. All o, the custo)
so,t'are is described belo' and included in the acco)*anying .+. The so,t'are belo' 'as 'ritten in
Hava> using the 5cli*se Integrated +evelo*)ent 5nviron)ent (I+5. All the so,t'are )ust be run ,ro)
the co))and line. Please note the so,t'are 'as created solely to serve itEs short3ter) *ur*ose> and is
by no )eans e,,iciently coded.
get)ata.0ar
All o, the re@uired data 'as taken ,ro) ,inance.yahoo.co) and a *rogra) 'as 'ritten that
do'nloads the re@uired data and *uts it into a tab deli)ited ,ile. To run the *rogra)> use the ,ollo'ing
co))and:
java -jar getdata.jar <ticker file> <target file>
'here ticker file is a ,ile 'ith one stock ticker sy)bol *er line> and the second *ara)eter is the ,ile
na)e 'here the data 'ill be stored> 'ith data ,or each stock on a se*arate line.
get1u')ame'tals.0ar
Since ,unda)ental data is released once a 'eek> once a )onth> or another *eriod> it is di,,icult
to have a good idea o, 'hich data a**lies to 'hat day 'hen creating in*ut *atterns ,or a neural
net'ork. #y s*eci,ying a date and data ,ile to this *rogra)> the user 'ill get a ne' ,ile outlining 'hat
the values o, various ,unda)ental variables 'as on that day. To run this *rogra)> ty*e:
java -jar getfundamentals.jar <data file> <date file> <target file>
'here the data file is a ,ile o, data 'ith the ,irst colu)n having dates organiCed in a dd6))6yyyy
,or)at. The date file is a list o, re@uired dates in a si)ilar ,or)at> and the target file is the co))a
se*arated values (.SV ,ile that the in,or)ation 'ill be saved at.
co'vert.0ar
This *rogra) takes a .SV ,ile and converts it into an S00S 6 H00S *attern ,ile. The .SV ,ile
)ust be organiCed 'ith the in*ut *atterns ,irst and the out*uts ,ollo'ing> 'ith one *attern *er ro'.
5nter *ara)eters in the ,ollo'ing ,or)at:
java -jar convert.jar
<CSV file> <number of inputs per row> <include outputs> <target file>
'here include outputs is either TR9 or FA/! and allo's one to choose 'hether the e(*ected out*ut
should be encoded 'ith the in*ut *attern.
com2i'e.0ar
Since so)e o, the net'orks in the *ro?ect had )ore than -56 in*uts> 5(cel could not handle the
a)ount o, data and convert it to .SV ,or)at ,or the .SV to S00S *attern converter. As such> a second
*rogra) had to be 'ritten that co)bines t'o .SV ,iles into one. The code belo' is used to run the
*rogra):
java -jar combine.jar <first CSV file> <second CSV file> <target file>
'here the ,irst t'o *ara)eters are the re@uired .SV ,iles and the target file is the ,ile to save the t'o
others. This *rogra) 'ill co)bine every line o, each se*arate .SV ,ile into one line.
results.0ar
9hen you use the !ave 'ata o*tion in H00S> the ,ile is saved to a R! ,ile 'ith its o'n ,or)at.
This *rogra) converts the R! ,ile to a ,ile 'ith one result *er line> so it can then be o*ened in 5(cel
or another *rogra). This *rogra) is used on entire directories> so that all R! ,iles in a directory are
converted to &!V ,iles. The original ,iles> ho'ever> are not )odi,ied. Ose the code belo' to run this
*rogra):
java -jar results.jar <target directory>
A%%endi& C: *ffects of "andom $eights on et!ork "esults
The *ur*ose o, Trial `$7 'as to see ho' 'ell the results o, the various net'orks tested adhere
to the assu)*tions )ade in the statistical analysis> and 'hether the use o, rando) 'eights had a )a?or
i)*act on results.
&irst and ,ore)ost 'ere tests ,or nor)ality. The assu)*tion behind the t3tests used in the *a*er
to co)*are net'orks to rando) *redictions and #ernoulli distributions rests on the ,act that a net'orkEs
accuracy ,ollo's a nor)al distribution. Trial `$7 consisted o, -% 5%%3cycle training runs on a one3layer
,eed,or'ard net'ork> e(actly like the one in Trail `3. This *rovided a)*le data to use the Sha*iro3
9ilk (0ational Institute o, Standards and Technology> n.d. test ,or nor)ality. The ,ollo'ing results
'ere achieved:
Trai'i'g Set: 9 M %.!/%$
3ali)atio' Set: 9 M %.!//%
9ithin such a test> the higher the value ,or 9> the )ore likely the data ,ollo's a nor)al
distribution. &or statistical interest> it 'as also ,ound that the )ean and standard deviation o, the
validation set 'as 5%.7$L and 5.87L> res*ectively. #elo' is a gra*h re*resenting the results o, the -%
net'orks.
0eural 0et'ork Predictions o, Stock Price &luctuations 3! 6 /-
Interestingly> one o, the net'orks trained had an accuracy rate o, 66.68L> )aking it a
signi,icant result i, results ,ollo' a nor)al distribution as described above. Indeed> it see)s rando)
'eights )ay actually *lay a )a?or role in ho' 'ell a net'ork is trained> at least 'hen the nu)ber o,
training cycles is lo'. 9hile these are ?ust *reli)inary results> it is use,ul to kee* the) in )ind 'hen
analyCing results *resented in the *a*er: *oor net'ork results )ay si)*ly be a result o, unlucky
rando) 'eight initialiCations.
0eural 0et'ork Predictions o, Stock Price &luctuations /% 6 /-
$isri(uion o0 "e2or3 Accuracies
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.40 0.43 0.46 0.49 0.52 0.55 0.58 0.61 0.64 0.67 0.70
Accurac* o0 "e2or3
F
r
e
4
u
e
n
c
*
$orks Cited
Achelis> S.#. (-%%$. Technical analysis from A to Z. Onited States: Mc=ra'3Bill.
Arbib> M.A. (5d.. (-%%3. The 0and#ook of 1rain Theory and "eural "etworks. .a)bridge: MIT
Press.
#rock> 9.> 2akonishok> 2. G 2e#aron> #. ($!!-. Si)*le Technical Trading "ules and the Stochastic
Pro*erties o, Stock "eturns. The :ournal of Finance> /8> $83$3$86/.
#ureau o, 2abor Statistics. (n.d.. &onsumer Price Inde* 0ome Page. "etrieved on A*ril 6> -%%6 ,ro)
htt*:66'''.bls.gov6c*i6ho)e.ht)
.on,erence #oard. (-%%6a. &onsumer &onfidence Inde*. "etrieved on A*ril 6> -%%6 ,ro)
htt*:66'''.con,erence3board.org6econo)ics6consu)er.on,idence.c,)
.on,erence #oard. (-%%6b. conomic "ews. "etrieved A*ril 6> -%%6 ,ro) htt*:66'''.con,erence3
board.org6econo)ics6*ress.c,)R*ressaI+M-7/%
.hurchland> P. G Se?no'ski> T.H. ($!!/. The &omputational 1rain. .a)bridge: MIT Press.
+a'son> M. (-%%/. Minds and Machines; &onnectionism and Psychological Modeling. Onited States:
#lack'ell Publishing.
+i)son> 5. (5d.. ($!77. !tock Market Anomalies. .a)bridge: .a)bridge Oniversity Press.
+yck)an> T.". G Morse> +. ($!76. fficient &apital Markets and Accounting; A &ritical Analysis.
Onited States: Prentice Ball.
5vans> M. G "osenthal> H. (-%%/. Pro#a#ility and !tatistics; The !cience of 9ncertainty. Onited
States: 9. B. &ree)an.
&ederal "eserve #oard. (-%%6a. <pen Market <perations. "etrieved A*ril 6> -%%6 ,ro)
htt*:66,ederalreserve.gov6,o)c6,undsrate.ht)
&ederal "eserve #oard. (-%%6b. !elected Interest Rates. "etrieved A*ril 6> -%%6 ,ro)
htt*:66'''.,ederalreserve.gov6releases6h$56.urrent6
=losser> S.M. G =olden> 2. ($!!8. Average 'ork hours as a leading econo)ic variable in OS
)anu,acturing industries. International :ournal of Forecasting> $3> $853$!5.
=ranger> ..9.H. ($!7!. Forecasting in 1usiness and cononomics (-nd ed.( Onited States: Acade)ic
Press> Inc.
Baykin> S. ($!!!. "eural "etworks; A &omprehensive Foundation (-nd ed.. Onited States: Prentice
Ball.
0eural 0et'ork Predictions o, Stock Price &luctuations /$ 6 /-
Iihara> 4.> Kato> B.K.> G Tokunaga> T. (-%%$. InvestorsE Berding on the Tokyo Stock 5(change.
International Review of Finance> -> 8$3!7.
Institute ,or Su**ly Manage)ent. (n.d.. A#out I!M. "etrieved A*ril 6> -%%6 ,ro) htt*:66'''.is).'s6
Kaca*yr> 5. ($!!6. conomic Forecasting; The !tate of the Art. 0e' 4ork: M.5. Shar*e.
Malkiel #.=. (-%%3. The 5,,icient Market By*othesis and Its .ritics. :ournal of conomic
Perspectives> $8> 5!37-.
Manki'> =. G Scarth> 9. (-%%/. Macroeconomics; &anadian dition (-nd ed.. Onited States: 9orth
Publishers.
Morgan Stanley .a*ital International Inc. (-%%5. %lo#al Industry &lassification !tandard =%I&!>(
"etrived A*ril 6> -%%6> ,ro) htt*:66'''.)sci.co)6e@uity6gics.ht)l
0ational Institute o, Standards and Technology. (n.d.. ngineering !tatistics 0and#ook. "etrieved
A*ril 8> -%%6> ,ro) htt*:66'''.itl.nist.gov6div7!76handbook6
1E"eilly> "...> G Munakata> 4. (-%%%. &omputational *plorations in &ognitive "euroscience.
.a)bridge: Massachusetts Institute ,or Technology.
"ussell> S. G 0orvig> P. (-%%3. Artificial Intelligence; A Modern Approach (-nd ed.. Onited States:
Pearson 5ducation.
Siegel> H. ($!!7. !tocks for the /ong Run. 0e' 4ork: Mc=ra'3Bill.
Shlei,er> A. (-%%%. Inefficient Markets; An Introduction to 1ehavioral Finance. 1(,ord: 1(,ord
Oniversity Press.
Oniversity o, Michigan. (-%%6. !urvey of &onsumers. "etrieved on A*ril 6> -%%6 ,ro)
htt*:66'''.sca.isr.u)ich.edu6
9Drneryd> K.5. (-%%$. !tock.Market Psychology. Onited Kingdo): 5d'ard 5lgar.
9eisstein> 5.9. ($!!!. Mathworld. "etrieved A*ril 6> -%%6 ,ro) htt*:66)ath'orld.'ol,ra).co)6
4ao> H.> Teng> 0.> Poh> B.> G Tan> ..2. ($!!7. &orecasting and Analysis o, Marketing +ata Osing
0eural 0et'orks. :ournal of Information !cience and ngineering> $/> 7/3376-.
Nell> A.> Ma)ier> =.> Vogt> M.> Mache> #.> BVbner> ".> G +Wring> S. (n.d.. !tuttgart "eural "etwork
!imulator; 9ser Manual) Version ?(@. "etrieved A*ril 6> -%%6> ,ro) htt*:66'''3
ra.in,or)atik.uni3tuebingen.de6do'nloads6S00S6S00Sv/.-.Manual.*d,
0eural 0et'ork Predictions o, Stock Price &luctuations /- 6 /-

Wojciech Gryc - Neural Network Predictions of Stock Price Fluctuations

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Wojciech Gryc - Neural Network Predictions of Stock Price Fluctuations

Uploaded by

Copyright:

Available Formats

Neural Network Predictions of

Stock Price Fluctuations

You might also like