TESTERS TESTERS

GUIDE GUIDE
TABLE OF CONTENTS
1. POLICY........................................................................................................
2. Terms to understand......................................................................................
2.1. What is software 'quality'?......................................................................
2.2. What is erifi!ation? alidation?............................................................
2.". What's an 'ins#e!tion'?............................................................................
2.$. %& ' Testin(? )ifferen!es.....................................................................
". Life Cy!le of Testin( Pro!ess........................................................................
$. Leels of Testin(...........................................................................................
$.1. *nit Testin(.............................................................................................
$.2. Inte(ration testin(...................................................................................
$.". +ystem testin(.........................................................................................
$.$. &!!e#tan!e testin(..................................................................................
,. Ty#es of Testin(............................................................................................
,.1. In!remental inte(ration testin(...............................................................
,.2. +anity testin(..........................................................................................
,.". Com#ati-ility testin(..............................................................................
,.$. ./#loratory testin(..................................................................................
,.,. &d0ho! testin(.........................................................................................
,.1. Com#arison testin(.................................................................................
,.2. Load testin(............................................................................................
,.3. +ystem testin(.........................................................................................
,.4. 5un!tional testin(....................................................................................
,.16. 7olume testin(......................................................................................
,.11. +tress testin(.........................................................................................
,.12. +o!ia-ility Testin(................................................................................
,.1". *sa-ility testin(....................................................................................
,.1$. 8e!oery testin(...................................................................................
,.1,. +e!urity testin(.....................................................................................
,.11. Performan!e Testin(.............................................................................
,.12. .nd0to0end testin(.................................................................................
,.13. 8e(ression testin(.................................................................................
,.14. Parallel testin(......................................................................................
,.26. Install9uninstall testin(..........................................................................
,.21. :utation testin(....................................................................................
,.22. &l#ha testin(.........................................................................................
,.2". ;eta testin(...........................................................................................
1. Testin( Te!hniques........................................................................................
1.1. ;la!< ;o/ testin(....................................................................................
1.1.1. .quialen!e Testin(..........................................................................
1.1.2. ;oundary testin(..............................................................................
1.1.". Cause0.ffe!t =ra#hin( Te!hniques..................................................
1.1.$. .rror =uessin(..................................................................................
1.2. White ;o/ testin(...................................................................................
1.2.1. Path Testin(......................................................................................
1.2.2. Condition testin(..............................................................................
1.2.". Loo# Testin(.....................................................................................
1.2.$. )ata 5low Testin(.............................................................................
2. We- Testin( +#e!ifi!s...................................................................................
2.1. Internet +oftware 0 %uality Chara!teristi!s............................................
2.2. WWW Pro>e!t Pe!uliarities....................................................................
2.". ;asi! ?T:L Testin(...............................................................................
2.$. +u((estions for fast loadin(....................................................................
2.,. Lin< Testin(............................................................................................
2.1. Com#ati-ility Testin(.............................................................................
2.2. *sa-ility Testin(.....................................................................................
2.2.1. *sa-ility Ti#s...................................................................................
2.3. Porta-ility Testin(...................................................................................
2.4. Coo<ies Testin(.......................................................................................
3. Testin( 0 When is a #ro(ram !orre!t?...........................................................
4. Test Plan........................................................................................................
16. Test !ases.....................................................................................................
16.1. What's a 'test !ase'?...............................................................................
11. Testin( Coera(e.........................................................................................
12. What if there isn't enou(h time for thorou(h testin(?.................................
1". )efe!t re#ortin(..........................................................................................
1$. Ty#es of &utomated Tools...........................................................................
1,. To# Ti#s for Testers.....................................................................................
1. POLICY
We are committed to Continuous Improvement of Quality of Products and Customer Services by adhering to
International Standards.
2. TERMS TO UNDERSTAND
3. What is softwa! "#$a%it&"'
Quality software is reasonably bug/defect-free delivered on time and within budget meets re!uirements and/or
e"pectations and is maintainable.
(. What is )!ifi*atio+' )a%i,atio+'
#erification typically involves reviews and meetings to evaluate documents plans code re!uirements and
specifications. $his can be done with chec%lists issues lists wal%throughs and inspection meetings. #alidation
typically involves actual testing and ta%es place after verifications are completed. &oth validation and verification
process continue in a cycle till the software becomes defect free '#alidation 00@ 8e#ortin( 00@ 5i/in( Aand
enhan!ementsB 00@ 7erifi!ation(.
-. What"s a+ "i+s.!*tio+"'
&n ins#e!tion is more formaliCed than a 'wal<throu(h'D ty#i!ally with "03 #eo#le in!ludin( a moderatorD reader Athe author
of whateer is -ein( reiewedBD and a re!order to ta<e notes. The su->e!t of the ins#e!tion is ty#i!ally a do!ument su!h as a
requirements s#e! or a test #lanD and the #ur#ose is to find #ro-lems and see what's missin(D not to fi/ anythin(.
/. 0A 1 T!sti+2' Diff!!+*!s
+oftware %& inoles the entire software deelo#ment P8OC.++ 0 monitorin( and im#roin( the #ro!essD ma<in( sure
that any a(reed0u#on standards and #ro!edures are followedD and ensurin( that #ro-lems are found and dealt with. It is
oriented to '#reention'
Testin( inoles o#eration of a system or a##li!ation under !ontrolled !onditions and ealuatin( the results Ae.(.D 'if the user
is in interfa!e & of the a##li!ation while usin( hardware ;D and does CD then ) should ha##en'B. The !ontrolled !onditions
should in!lude -oth normal and a-normal !onditions. Testin( should intentionally attem#t to ma<e thin(s (o wron( to
determine if thin(s ha##en when they shouldn't or thin(s don't ha##en when they should. It is oriented to 'dete!tion'.
3. LIFE CYCLE OF TESTIN4 PROCESS
1. Plannin(
2. &nalysis
". )esi(n
$. ./e!ution
,. Cy!les
1. 5inal Testin( and im#lementation
2. Post Im#lementation
The followin( are some of the ste#s to !onsiderE
O-tain requirementsD fun!tional desi(nD and internal desi(n s#e!ifi!ations and other ne!essary do!uments
O-tain s!hedule requirements
)etermine #ro>e!t0related #ersonnel and their res#onsi-ilitiesD re#ortin( requirementsD required standards and
#ro!esses Asu!h as release #ro!essesD !han(e #ro!essesD et!.B
Identify a##li!ation's hi(her0ris< as#e!tsD set #rioritiesD and determine s!o#e and limitations of tests
)etermine test a##roa!hes and methods 0 unitD inte(rationD fun!tionalD systemD loadD usa-ility testsD et!.
)etermine test enironment requirements AhardwareD softwareD !ommuni!ationsD et!.B
)etermine testware requirements Are!ord9#lay-a!< toolsD !oera(e analyCersD test tra!<in(D #ro-lem9-u( tra!<in(D
et!.B
)etermine test in#ut data requirements
Identify tas<sD those res#onsi-le for tas<s
+et s!hedule estimatesD timelinesD milestones
)etermine in#ut equialen!e !lassesD -oundary alue analysesD error !lasses
Pre#are test #lan do!ument and hae needed reiews9a##roals
Write test !ases
?ae needed reiews9ins#e!tions9a##roals of test !ases
Pre#are test enironment and testwareD o-tain needed user manuals9referen!e do!uments9!onfi(uration
(uides9installation (uidesD set u# test tra!<in( #ro!essesD set u# lo((in( and ar!hiin( #ro!essesD set u# or o-tain test
in#ut data
O-tain and install software releases
Perform tests
.aluate and re#ort results
Tra!< #ro-lems9-u(s and fi/es
8etest as needed
:aintain and u#date test #lansD test !asesD test enironmentD and testware throu(h life !y!le
5. LE6ELS OF TESTIN4
7. U+it T!sti+2
The most 'mi!ro' s!ale of testin(F to test #arti!ular fun!tions or !ode modules. Ty#i!ally done -y the #ro(rammer and some
times -y testersD as it requires detailed <nowled(e of the internal #ro(ram desi(n and !ode. Got always easily done unless
the a##li!ation has a well0desi(ned ar!hite!ture with ti(ht !odeF may require deelo#in( test drier modules or test
harnesses.
18. I+t!2atio+ t!sti+2
Testin( of !om-ined #arts of an a##li!ation to determine if they fun!tion to(ether !orre!tly. The '#arts' !an -e !ode modulesD
indiidual a##li!ationsD !lient and serer a##li!ations on a networ<D et!. It is not truly system testin( -e!ause the
!om#onents are not im#lemented in the o#eratin( enironment. This ty#e of testin( is es#e!ially releant to !lient9serer
and distri-uted systems.
)urin( the inte(ration testin(D the units or !om#onents !an -e assem-led usin( 2 a##roa!hesE
Gonin!remental inte(ration 0 H-i( -an(H a##roa!h
)isadanta(e 0 When failure o!!ursD it is ery diffi!ult to lo!ate the faults. &fter the modifi!ationD we hae to (o throu(h
the testin(D lo!atin( faultsD modifyin( faults a(ain.
In!remental inte(ration
In!remental Inte(ration !an -e to#0down or -ottom0u#E
To#0down testin( starts with main and su!!essiely re#la!es stu-s with the real modules.
o :a>or ste#sE
1. The main !ontrol module is used as a test drier and stu-s are su-stituted for all !om#onent dire!tly su-ordinate
to the main module.
2. de#endin( on inte(ration a##roa!hD su-ordinate stu-s are re#la!ed on!e a time with a!tual !om#onents.
". Tests are !ondu!ted as ea!h !om#onent is inte(rated.
$. When failures are en!ounteredD lo!ate faultsD and #erform re(ression testin( to (uarantee the modifi!ation won't
adersely affe!ted other #ie!es. Then (o throu(h 2 D"D $ a(ain
&danta(esE !an erify ma>or !ontrol or de!ision #oint early in the testin( #ro!ess.
)isadanta(esE +tu-s are required when #erform the inte(ration testin(D and (enerallyD deelo# stu-s in ery
diffi!ulties.
;ottom0u# testin( -uilds lar(er module assem-lies from #rimitie modules.
o :a>or ste#s
1. Low0leel !om#onents are !om-ined into !lusters whi!h #erforms a s#e!ifi! su-fun!tion.
2. & drierAa !ontrol #ro(ram for testin(A is written to !oordinate test !ase in#ut and out#ut.
". The !luster is tested
$. )riers are remoed and !lusters are !om-ined moin( u#ward in the #ro(ram stru!ture.
&danta(esE +tu-s are su-stituted -y driersD whi!h are mu!h easier to -e deelo#ed.
)isadanta(esE :a>or !ontrol and de!ision #ro-lem will -e identified later in the testin( #ro!ess.
+andwi!h testin( is mainly to#0down with -ottom0u# inte(ration and testin( a##lied to !ertain widely used
!om#onents
11. S&st!9 t!sti+2
The system test #hase -e(ins on!e modules are inte(rated enou(h to #erform tests in a whole system enironment. +ystem
testin( !an o!!ur in #arallel with inte(ration testD es#e!ially with the to#0down method.
12. A**!.ta+*! t!sti+2
5inal testin( -ased on s#e!ifi!ations of the end0user or !ustomerD or -ased on use -y end0users9!ustomers oer some limited
#eriod of time.
13. TYPES OF TESTIN4
1(. I+*!9!+ta% i+t!2atio+ t!sti+2
Continuous testin( of an a##li!ation as new fun!tionality is addedF requires that arious as#e!ts of an a##li!ation's
fun!tionality -e inde#endent enou(h to wor< se#arately -efore all #arts of the #ro(ram are !om#letedD or that test driers -e
deelo#ed as neededF done -y #ro(rammers or -y testers.
1-. Sa+it& t!sti+2
Ty#i!ally an initial testin( effort to determine if a new software ersion is #erformin( well enou(h to a!!e#t it for a ma>or
testin( effort. 5or e/am#leD if the new software is !rashin( systems eery , minutesD -o((in( down systems to a !rawlD or
destroyin( data-asesD the software may not -e in a 'sane' enou(h !ondition to warrant further testin( in its !urrent state.
1/. Co9.ati:i%it& t!sti+2
Testin( how well software #erforms in a #arti!ular hardware9software9o#eratin( system9networ<9et!. enironment.
13. E;.%oato& t!sti+2
Often ta<en to mean a !reatieD informal software test that is not -ased on formal test #lans or test !asesF testers may -e
learnin( the software as they test it.
15. A,<ho* t!sti+2
+imilar to e/#loratory testin(D -ut often ta<en to mean that the testers hae si(nifi!ant understandin( of the software -efore
testin( it.
17. Co9.aiso+ t!sti+2
Com#arin( software wea<nesses and stren(ths to !om#etin( #rodu!ts.
28. Loa, t!sti+2
Testin( an a##li!ation under heay loadsD su!h as testin( of a we- site under a ran(e of loads to determine at what #oint the
system's res#onse time de(rades or fails.
21. S&st!9 t!sti+2
;la!<0-o/ ty#e testin( that is -ased on oerall requirements s#e!ifi!ationsF !oers all !om-ined #arts of a system.
22. F$+*tio+a% t!sti+2
;la!<0-o/ ty#e testin( (eared to fun!tional requirements of an a##li!ationF this ty#e of testin( should -e done -y testers.
This doesn't mean that the #ro(rammers shouldn't !he!< that their !ode wor<s -efore releasin( it Awhi!h of !ourse a##lies to
any sta(e of testin(.B
23. 6o%$9! t!sti+2
7olume testin( inoles testin( the software or We- a##li!ation usin( !orner !ases of Htas< siCeH or in#ut data siCe. The
e/a!t olume tests #erformed de#end on the a##li!ation's fun!tionalityD its in#ut and out#ut me!hanisms and the
te!hnolo(ies used to -uild the a##li!ation. +am#le olume testin( !onsiderations in!ludeD -ut are not limited toE
If the a##li!ation reads te/t files as in#utsD try feedin( it -oth an em#ty te/t file and a hu(e Ahundreds of me(a-ytesB
te/t file
If the a##li!ation stores data in a data-aseD e/er!ise the a##li!ationIs fun!tions when the data-ase is em#ty and when
the data-ase !ontains an e/treme amount of data
If the a##li!ation is desi(ned to handle 166 !on!urrent requestsD send 166 requests simultaneously and then send the
161
st
request
If a We- a##li!ation has a form with doCens of te/t fields that allow a user to enter te/t strin(s of unlimited len(thD try
#o#ulatin( all of the fields with a lar(e amount of te/t and su-mit the form
2(. St!ss t!sti+2
Term often used inter!han(ea-ly with 'load' and '#erforman!e' testin(. &lso used to des!ri-e su!h tests as system fun!tional
testin( while under unusually heay loadsD heay re#etition of !ertain a!tions or in#utsD in#ut of lar(e numeri!al aluesD
lar(e !om#le/ queries to a data-ase systemD et!.
2-. So*ia:i%it& T!sti+2
This means that you test an a##li!ation in its normal enironmentD alon( with other standard a##li!ationsD to ma<e sure they
all (et alon( to(etherF that isD that they don't !orru#t ea!h other's filesD they don't !rashD they don't !onsume system
resour!esD they don't lo!< u# the systemD they !an share the #rinter #ea!efullyD et!.
2/. Usa:i%it& t!sti+2
Testin( for 'user0friendliness'. Clearly this is su->e!tieD and will de#end on the tar(eted end0user or !ustomer. *ser
interiewsD sureysD ideo re!ordin( of user sessionsD and other te!hniques !an -e used. Pro(rammers and testers are usually
not a##ro#riate as usa-ility testers.
23. R!*o)!& t!sti+2
Testin( how well a system re!oers from !rashesD hardware failuresD or other !atastro#hi! #ro-lems.
25. S!*$it& t!sti+2
Testin( how well the system #rote!ts a(ainst unauthoriCed internal or e/ternal a!!essD willful dama(eD et!F may require
so#histi!ated testin( te!hniques.
27. P!fo9a+*! T!sti+2
Term often used inter!han(ea-ly with 'stress' and 'load' testin(. Ideally '#erforman!e' testin( Aand any other 'ty#e' of testin(B
is defined in requirements do!umentation or %& or Test Plans.
38. E+,<to<!+, t!sti+2
+imilar to system testin(F the 'ma!ro' end of the test s!aleF inoles testin( of a !om#lete a##li!ation enironment in a
situation that mimi!s real0world useD su!h as intera!tin( with a data-aseD usin( networ< !ommuni!ationsD or intera!tin( with
other hardwareD a##li!ationsD or systems if a##ro#riate.
31. R!2!ssio+ t!sti+2
8e0testin( after fi/es or modifi!ations of the software or its enironment. It !an -e diffi!ult to determine how mu!h re0
testin( is neededD es#e!ially near the end of the deelo#ment !y!le. &utomated testin( tools !an -e es#e!ially useful for this
ty#e of testin(.
32. Paa%%!% t!sti+2
With #arallel testin(D users !an easily !hoose to run -at!h tests or asyn!hronous tests de#endin( on the needs of their test
systems. Testin( multi#le units in #arallel in!reases test throu(h#ut and lower the manufa!turer's !ost.
33. I+sta%%=$+i+sta%% t!sti+2
Testin( of fullD #artialD or u#(rade install9uninstall #ro!esses.
3(. M$tatio+ t!sti+2
& method for determinin( if a set of test data or test !ases is usefulD -y deli-erately introdu!in( arious !ode !han(es
A'-u(s'B and retestin( with the ori(inal test data9!ases to determine if the '-u(s' are dete!ted. Pro#er im#lementation requires
lar(e !om#utational resour!es.
3-. A%.ha t!sti+2
Testin( of an a##li!ation when deelo#ment is nearin( !om#letionE minor desi(n !han(es may still -e made as a result of
su!h testin(. Ty#i!ally done -y end0users or othersD not -y #ro(rammers or testers.
3/. B!ta t!sti+2
Testin( when deelo#ment and testin( are essentially !om#leted and final -u(s and #ro-lems need to -e found -efore final
release. Ty#i!ally done -y end0users or othersD not -y #ro(rammers or testers.
33. TESTIN4 TEC>NI0UES
35. B%a*? Bo; t!sti+2
;la!< -o/ testin( Adata drien or in#ut9out#ut drienB is not -ased on any <nowled(e of internal desi(n or !ode. Tests are
-ased on requirements and fun!tionality. ;la!< -o/ testin( attem#ts to derie sets of in#uts that will fully e/er!ise all the
fun!tional requirements of a system. ;la!< -o/ testin( is !on!erned only with testin( the s#e!ifi!ationF it !annot (uarantee
that all #arts of the im#lementation hae -een tested. Thus -la!< -o/ testin( is testin( a(ainst the s#e!ifi!ation and will
dis!oer fa$%ts of o9issio+D indi!atin( that #art of the s#e!ifi!ation has not -een fulfilled. It is not an alternatie to white
-o/ testin(. This ty#e of testin( attem#ts to find errors in the followin( !ate(oriesE
,. In!orre!t or missin( fun!tionsD
1. Interfa!e errorsD
2. .rrors in data stru!tures or e/ternal data-ase a!!essD
3. Performan!e errorsD and
4. InitialiCation and termination errors.
White -o/ testin( should -e #erformed early in the testin( #ro!essD while -la!< -o/ testin( tends to -e a##lied durin( later
sta(es.
35.1.1. E#$i)a%!+*! T!sti+2
This method diides the in#ut domain of a #ro(ram into !lasses of data from whi!h test !ases !an -e deried. .quialen!e
#artitionin( stries to define a test !ase that un!oers !lasses of errors and there-y redu!es the num-er of test !ases needed.
It is -ased on an ealuation of equialen!e !lasses for an in#ut !ondition. &n equialen!e !lass re#resents a set of alid or
inalid states for in#ut !onditions.
.quialen!e !lasses may -e defined a!!ordin( to the followin( (uidelinesE
1. If an in#ut !ondition s#e!ifies a ran(eD one alid and two inalid equialen!e !lasses are defined.
2. If an in#ut !ondition requires a s#e!ifi! alueD then one alid and two inalid equialen!e !lasses are defined.
". If an in#ut !ondition s#e!ifies a mem-er of a setD then one alid and one inalid equialen!e !lass are defined.
$. If an in#ut !ondition is ;ooleanD then one alid and one inalid equialen!e !lass are defined.
Test!ase )esi(n for .quialen!e #artitionin(
1. =ood test !ase redu!es -y more than one the num-er of other test !ases whi!h must -e deelo#ed
2. =ood test !ase !oers a lar(e set of other #ossi-le !ases
". Classes of alid in#uts
$. Classes of inalid in#uts
Consider the followin( data aria-le
Game of data aria-leE &(e
)es!ri#tion of data aria-leE the len(th of time that one has e/isted
)ata aria-le ty#eE Inte(er
&!!e#ta-le ran(eE 2, to 16
.quialen!e #artitionin( essentially ta<es the in#ut ran(e and attem#ts to !learly !om#artmentaliCe this ran(e into su-
ran(esD and then #resents test !ases that will s#e!ifi!ally test for ea!h of these su- ran(es.
GowD for the &(e data aria-leD the a!!e#ta-le ran(e is from 2, to 16D in!lusie. &!!ordin( to the rules of the te!hniqueD
sin!e the data aria-le is a ran(eD that means one alid !lassD and two inalid !lasses will -e s#e!ified. 7alid !lass refers to
the alid su- ran(e of in#ut aluesD inalid !lass refers to the inalid su- ran(es.
?en!eD for equialen!e #artitionin(E
7alid equialen!e !lassE 2, to 16
5irst inalid equialen!e !lassE less than 2,
+e!ond inalid equialen!e !lassE more than 16
?en!eD e/am#les of test !ases !ould -e as followsE
,. 7alid equialen!e !lassE &(e J "6. ./#e!ted result K a!!e#ted
1. 5irst Inalid equialen!e !lassE &(e J 26. ./#e!ted result K re>e!ted
2. +e!ond Inalid equialen!e !lassE &(e J 26. ./#e!ted result K re>e!ted
35.1.2. Bo$+,a& t!sti+2
This method leads to a sele!tion of test !ases that e/er!ise -oundary alues. It !om#lements equialen!e #artitionin( sin!e it
sele!ts test !ases at the ed(es of a !lass. 8ather than fo!usin( on in#ut !onditions solelyD ;7& deries test !ases from the
out#ut domain also. ;7& (uidelines in!ludeE
1. 5or in#ut ran(es -ounded -y a and -D test !ases should in!lude alues a and - and >ust a-oe and >ust -elow a and
- res#e!tiely.
2. If an in#ut !ondition s#e!ifies a num-er of aluesD test !ases should -e deelo#ed to e/er!ise the minimum and
ma/imum num-ers and alues >ust a-oe and -elow these limits.
". &##ly (uidelines 1 and 2 to the out#ut.
$. If internal data stru!tures hae #res!ri-ed -oundariesD a test !ase should -e desi(ned to e/er!ise the data stru!ture
at its -oundary.
Test !ase )esi(n for ;oundary alue analysisE
+ituations onD a-oeD or -elow ed(es of in#utD out#utD and !ondition !lasses hae hi(h #ro-a-ility of su!!ess
5or -oundary alue analysisD the idea is to identify the e/treme alues of the data ran(eD and then !hoosin( test !ases
around these e/treme alues.
?en!e for the a-oe e/am#leD for -oundary alue analysisD itIll (o li<e thisE
:inimum alueE 2, Aminima alueB
:a/imum alueE 16 Ama/ima alueB
The e/am#les of test !ases would -eE
1. Test !ases for minima alueE &(e J 2$ Are>e!tedBD &(e J 2, Aa!!e#tedBD &(e J 21 Aa!!e#tedB
2. Test !ases for ma/ima alueE &(e J ,4 Aa!!e#tedBD &(e J 16 Aa!!e#tedBD &(e J 11 Are>e!tedB
35.1.3. Ca$s!<Eff!*t 4a.hi+2 T!*h+i#$!s
Cause0effe!t (ra#hin( is a te!hnique that #roides a !on!ise re#resentation of lo(i!al !onditions and !orres#ondin( a!tions.
There are four ste#sE
1. Causes Ain#ut !onditionsB and effe!ts Aa!tionsB are listed for a module and an identifier is assi(ned to ea!h.
2. & !ause0effe!t (ra#h is deelo#ed.
". The (ra#h is !onerted to a de!ision ta-le.
$. )e!ision ta-le rules are !onerted to test !ases.
35.1.(. Eo 4$!ssi+2
.rror =uessin( is the #ro!ess of usin( intuition and #ast e/#erien!e to fill in (a#s in the test data set. There are no rules to
follow. The tester must reiew the test re!ords with an eye towards re!o(niCin( missin( !onditions. Two familiar e/am#les
of error #rone situations are diision -y Cero and !al!ulatin( the square root of a ne(atie num-er. .ither of these will result
in system errors and (ar-led out#ut.
Other !ases where e/#erien!e has demonstrated error #roneness are the #ro!essin( of aria-le len(th ta-lesD !al!ulation of
median alues for odd and een num-ered #o#ulationsD !y!li! master file9data -ase u#dates Aim#ro#er handlin( of du#li!ate
<eysD unmat!hed <eysD et!.BD oerla##in( stora(e areasD oerwritin( of -uffersD for(ettin( to initialiCe -uffer areasD and so
forth. I am sure you !an thin< of #lenty of !ir!umstan!es unique to your hardware9software enironments and use of
s#e!ifi! #ro(rammin( lan(ua(es.
.rror =uessin( is as im#ortant as .quialen!e #artitionin( and ;oundary &nalysis -e!ause it is intended to !om#ensate for
their inherent in!om#leteness. &s .quialen!e Partitionin( and ;oundary &nalysis !om#lement one anotherD .rror =uessin(
!om#lements -oth of these te!hniques.
37. Whit! Bo; t!sti+2
White -o/ testin( Alo(i! drienB is -ased on <nowled(e of the internal lo(i! of an a##li!ation's !ode. Tests are -ased on
!oera(e of !ode statementsD -ran!hesD #athsD !onditions. White -o/ testin( is !on!erned only with testin( the software
#rodu!tF it !annot (uarantee that the !om#lete s#e!ifi!ation has -een im#lemented. White -o/ testin( is testin( a(ainst the
im#lementation and will dis!oer fa$%ts of *o99issio+D indi!atin( that #art of the im#lementation is faulty. & failure of a
white -o/ test may result in a !han(eD whi!h requires all -la!< -o/ testin( to -e re#eatedD and the re0determination of the
white -o/ #aths. White -o/ testin( is a test !ase desi(n method that uses the !ontrol stru!ture of the #ro!edural desi(n to
derie test !ases. Test !ases !an -e deried that
1. (uarantee that all inde#endent #aths within a module hae -een e/er!ised at least on!eD
2. e/er!ise all lo(i!al de!isions on their true and false sidesD
". e/e!ute all loo#s at their -oundaries and within their o#erational -oundsD and
$. e/er!ise internal data stru!tures to ensure their alidity.
37.1.1. Path T!sti+2
& #ath0!oera(e test allows us to e/er!ise eery transition -etween the #ro(ram statements Aand so eery statement and
-ran!h as wellB.
1. 5irst we !onstru!t a #ro(ram (ra#h.
2. Then we enumerate all #aths.
". 5inally we deise the test !ases.
Possi-le !riteriaE
1. e/er!ise eery #ath from entry to e/itF
2. e/er!ise ea!h statement at least on!eF
". e/er!ise ea!h !ase in eery -ran!h9!ase.
37.1.2. Co+,itio+ t!sti+2
& !ondition test !an use a !om-ination of Com#arison o#erators and Lo(i!al o#erators.
The Com#arison o#erators !om#are the alues of aria-les and this !om#arison #rodu!es a -oolean result. The Lo(i!al
o#erators !om-ine -ooleans to #rodu!e a sin(le -oolean result that is the result of the !ondition test.
e.(. Aa JJ -B 8esult is true if the alue of a is the same as the alue of -.
:yersE ta<e ea!h -ran!h out of a !ondition at least on!e.
White and CohenE for ea!h relational o#erator e1 L e2 test all !om-inations of e1D e2 orderin(s. 5or a ;oolean !onditionD
test all #ossi-le in#uts AMB.
;ran!h and relational o#erator testin(000enumerate !ate(ories of o#erator alues.
;1 NN ;2E test O;1JtD;2JtPD OtDfPD OfDtP
;1 NN Ae2 J e"BE test OtDJPD OfDJPD OtDLPD OtD@P.
37.1.3. Loo. T!sti+2
This white -o/ te!hnique fo!uses e/!lusiely on the alidity of loo# !onstru!ts. 5our different !lasses of loo#s !an -e
definedE
1. sim#le loo#sD
2. nested loo#sD
". !on!atenated loo#sD and
$. unstru!tured loo#s.
37.1.3.1 Si9.%! Loo.s
The followin( tests should -e a##lied to sim#le loo#s where n is the ma/imum num-er of allowa-le #asses throu(h the
loo#E
1. s<i# the loo# entirelyD
2. only #ass on!e throu(h the loo#D
". m #asses throu(h the loo# where m L nD
$. n K 1DnDn Q 1 #asses throu(h the loo#
37.1.3.2 N!st!, Loo.s
The testin( of nested loo#s !annot sim#ly e/tend the te!hnique of sim#le loo#s sin!e this would result in a (eometri!ally
in!reasin( num-er of test !ases. One a##roa!h for nested loo#sE
1. +tart at the innermost loo#. +et all other loo#s to minimum alues.
2. Condu!t sim#le loo# tests for the innermost loo# while holdin( the outer loo#s at their minimums. &dd tests for out0
of0ran(e or e/!luded alues.
". Wor< outwardD !ondu!tin( tests for the ne/t loo# while <ee#in( all other outer loo#s at minimums and other nested
loo#s to ty#i!al alues.
$. Continue until all loo#s hae -een tested.
37.1.3.3 Co+*at!+at!, Loo.s
Con!atenated loo#s !an -e tested as sim#le loo#s if ea!h loo# is inde#endent of the others. If they are not inde#endent Ae.(.
the loo# !ounter for one is the loo# !ounter for the otherBD then the nested a##roa!h !an -e used.
37.1.3.( U+st$*t$!, Loo.s
This ty#e of loo# should -e !,!si2+!, not testedMMM
37.1.(. Data F%ow T!sti+2
)ata flow testin( sele!ts test #aths a!!ordin( to the lo!ations of definitions and uses of aria-les in the #ro(ram
)ef0use !hainsE
$. def J definition of aria-le
,. use J use of that aria-leF
1. def0use !hains (o a!ross !ontrol -oundaries.
2. Testin(000test eery def0use !hain at least on!e.
St$:s fo T!sti+2
& +tu- is a dummy #ro!edureD module or unit that stands in for an unfinished #ortion of a system. +tu-s are mostly used for
To#0)own Testin(
+tu-s !an -e used for the followin( when standin( in for an unfinished !odeE
o )is#lay a tra!e messa(e
o )is#lay #arameter alueAsB
o 8eturn a alue from a ta-le
o 8eturn ta-le alue sele!ted -y #arameter
Di)!s fo T!sti+2
Test ?arness or a test drier is su##ortin( !ode and data used to #roide an enironment for testin( #art of a system in
isolation.
(8. WEB TESTIN4 SPECIFICS
(1. I+t!+!t Softwa! < 0$a%it& Chaa*t!isti*s
1. 5un!tionality 0 7erified !ontent
2. 8elia-ility 0 +e!urity and aaila-ility
". .ffi!ien!y 0 8es#onse Times
$. *sa-ility 0 ?i(h user satisfa!tion
,. Porta-ility 0 Platform Inde#enden!e
(2. WWW Po@!*t P!*$%iaiti!s
+oftware Consists of lar(e de(ree of !om#onents
*ser Interfa!e is more !om#le/ than many =*I -ased Client0+erer a##li!ations.
*ser may -e un<nown Ano trainin(9 user manualsB
+e!urity threats !ome from anywhere
*ser load un#redi!ta-le
(3. Basi* >TML T!sti+2
Che!< for ille(al elements #resent
Che!< for ille(al attri-utes #resent
Che!< for ta(s !lose
Che!< for the ta(s L?.&)@D LTITL.@D L;O)Y@D L)OCTYP.@
Che!< that all I:= ta(s should hae &LT ta(R&LT ta(s must -e su((estieS
Che!< for !onsisten!y of fonts 0 !olors and font siCe.
Che!< for s#ellin( errors in te/t and ima(es.
Che!< for HGon +enseH mar< u#
./am#le for Gon +ense mar< u#
L;@?elloL9;@ may -e written as L;@?L9;@L;@ellL9;@L;@oL9;@
((. S$22!stio+s fo fast %oa,i+2
We- #a(es wei(ht should -e redu!ed to lesser siCe as mu!h as #ossi-le
)onIt <no!< door of the )ata-ase eery time. =o for the alternate.
./am#leE If your we- a##li!ation has 8e#ortsD (enerate the !ontent of the re#ort in a +tati! ?T:L file in a
#eriodi! time. When the user iew the re#ort show him the stati! ?T:L !ontent. Go need to (o to the data-ase
and retriee the data when the user hits the re#ort lin<.
Ca!hed %uery 0 If the data whi!h is fet!hed usin( a query only !han(es #eriodi!ally
Then we !an !a!he the query for that #eriod. This will aoid unne!essary data-ase a!!ess.
.ery I:= ta(s must hae WI)T? and ?.I=?T attri-utes.
I:= 0 ;ad ./am#le
L;@ ?ello L9;@
LI:= +8C J H5&T.=I5H@
L;@ World L9;@
I:= 0 =ood ./am#le
L;@ ?ello L9;@
LI:= +8C J H5&T.=I5H WI)T? JH126H ?.I=?TJH1,6H @
L;@ World L9;@
&ll the #hoto(ra#hi! ima(es must -e in H>#(H format
Com#uter !reated ima(es must -e in H(ifH format
;a!<(round ima(e should -e less than ".,< Rthe -a!<(round ima(e should -e same for all the #a(es Ae/!e#t for
fun!tional reasonsBS
&oid nested ta-les.
Tee# ta-le te/t siCe to a minimum Ae.(. less than ,666 !hara!tersB
(-. Li+? T!sti+2
You must ensure that all the hy#erlin<s are alid
This a##lies to -oth internal and e/ternal lin<s
Internal lin<s shall -e relatieD to minimiCe the oerhead and faults when the we- site is moed to #rodu!tion
enironment
./ternal lin<s shall -e referen!ed to a-solute *8Ls
./ternal lin<s !an !han(e without !ontrol 0 +oD automate re(ression testin(
8emem-er that e/ternal non0 home #a(e lin<s are more li<ely to -rea<
;e !areful at lin<s in HWhat's GewH se!tions. They are li<ely to -e!ome o-solete
Che!< that !ontent !an -e a!!essed -y means ofE +ear!h en(ineD +ite :a#
Che!< the a!!ura!y of +ear!h .n(ine results
Che!< that we- +ite .rror $6$ AHGot 5oundHB is handled -y means of a user0friendly #a(e
(/. Co9.ati:i%it& T!sti+2
Che!< for the site -ehaior a!ross the industry standard -rowsers. The main issues inole how different the
-rowsers handle ta-lesD ima(esD !a!hin( and s!ri#tin( lan(ua(es
In !ross -rowsers testin(D !he!< forE
;ehaior of -uttons
+u##ort of Uaa s!ri#ts
+u##ort of ta-les
&!ro-atD 8ealD 5lash -ehaior
&!tieV !ontrol su##ort
Uaa !om#ati-ility
Te/t siCe
Bows!
Bows!
)!sio+
A*ti)!
A
*o+to%
s
6B
S*i.t
Ba)aS*
i.t
Ba)a
a..%!ts
D&+a9i*
>TML
Fa9!s
CSS
1.8
CSS
2.8
Internet
./#lorer
$.6 and
later
.na-le
d
.na-le
d
.na-le
d
.na-le
d
.na-led .na-led
.na-le
d
.na-le
d
Internet
./#lorer
".6 and
later
.na-le
d
.na-le
d
.na-le
d
.na-le
d
)isa-led .na-led
.na-le
d
)isa-le
d
Gets!a#e
Gai(ato
r
$.6 and
later
)isa-le
d
)isa-le
d
.na-le
d
.na-le
d
.na-led .na-led
.na-le
d
.na-le
d
Gets!a#e
Gai(ato
r
".6 and
later
)isa-le
d
)isa-le
d
.na-le
d
.na-le
d
)isa-led .na-led
)isa-le
d
)isa-le
d
;oth
Internet
./#lorer
$.6 and
later
)isa-le
d
)isa-le
d
.na-le
d
.na-le
d
.na-led .na-led.na-le
d
.na-le
d
and
Gai(ato
r
;oth
Internet
./#lorer
and
Gai(ato
r
".6 and
later
)isa-le
d
)isa-le
d
.na-le
d
.na-le
d
)isa-led .na-led
)isa-le
d
)isa-le
d
:i!rosoft
We- T7
*naaila
-le
)isa-le
d
)isa-le
d
)isa-le
d
)isa-le
d
)isa-led
)isa-le
d
)isa-le
d
)isa-le
d
(3. Usa:i%it& T!sti+2
&s#e!ts to -e tested with !areE
Coheren!e of loo< and feel
Gai(ational aids
*ser Intera!tions
Printin(
With res#e!t to
Gormal -ehaior
)estru!tie -ehaior
Ine/#erien!ed users
(3.1.1. Usa:i%it& Ti.s
1. )efine !ate(ories in terms of user (oals
2. Game se!tions !arefully
". Thin< internationally
$. Identify the home#a(e lin< on eery #a(e
,. :a<e sure sear!h is always aaila-le
1. Test all the -rowsers your audien!e will use
2. )ifferentiate isited lin<s from unisited lin<s
3. Geer use (ra#hi!s where ?T:L te/t will do
4. :a<e =*I desi(n #redi!ta-le and !onsistent
16. Che!< that #rinted #a(es fit a##ro#riately to #a#er #a(esR!onsider that many #eo#le >ust surf and #rint. Che!<
es#e!ially the #a(es for whi!h format is im#ortant. ..(.E an a##li!ation form !an either -e filled on0line or
#rinted9filled9fa/edS
(5. Pota:i%it& T!sti+2
1. Che!< that lin<s to *8Ls outside the we- site must -e in !anoni!al form A./E htt#E99www.ar<insys.!om9B
2. Che!< that lin<s to *8Ls into the we- site must -e in relatie formAe.(. W.9ar<insys9ima(es9ima(es.(ifB
(7. Coo?i!s T!sti+2
What are !oo<ies?
& HCoo<ieH is a small #ie!e of information sent -y the we- serer to store on a we- -rowser. +o it !an later -e read -a!<
from the -rowser. This is useful for hain( the -rowser remem-er some s#e!ifi! information.
Why must you test !oo<ies?
1. Coo<ies !an e/#ire
2. *sers !an disa-le them in ;rowser
?ow to #erform Coo<ies testin(?
1. Che!< the -ehaior after !oo<ies e/#iration
2. Wor< with !oo<ies disa-led
". )isa-le !oo<ies mid0way
$. )elete we- !oo<ies mid0way
,. Clear memory and dis< !a!he mid0way
-8. TESTIN4 < W>EN IS A PRO4RAM CORRECT'
There are leels of !orre!tness. We must determine the a##ro#riate leel of !orre!tness for ea!h system -e!ause it !osts
more and more to rea!h hi(her leels.
1. Go synta!ti! errors
2. Com#iles with no error messa(es
". 8uns with no error messa(es
$. There e/ists data whi!h (ies !orre!t out#ut
,. =ies !orre!t out#ut for required in#ut
1. Corre!t for ty#i!al test data
2. Corre!t for diffi!ult test data
3. Proen !orre!t usin( mathemati!al lo(i!
4. O-eys s#e!ifi!ations for all alid data
16. O-eys s#e!ifi!ations for li<ely erroneous in#ut
11. O-eys s#e!ifi!ations for all #ossi-le in#ut
-1. TEST PLAN
& software #ro>e!t test #lan is a do!ument that des!ri-es the o->e!tiesD s!o#eD a##roa!hD and fo!us of a software testin(
effort. The #ro!ess of #re#arin( a test #lan is a useful way to thin< throu(h the efforts needed to alidate the a!!e#ta-ility of
a software #rodu!t. The !om#leted do!ument will hel# #eo#le outside the test (rou# understand the 'why' and 'how' of
#rodu!t alidation. It should -e thorou(h enou(h to -e useful -ut not so thorou(h that no one outside the test (rou# will
read it. The followin( are some of the items that mi(ht -e in!luded in a test #lanD de#endin( on the #arti!ular #ro>e!tE
1. Title of the Pro>e!t
2. Identifi!ation of do!ument in!ludin( ersion num-ers
". 8eision history of do!ument in!ludin( authorsD datesD a##roals
$. Ta-le of Contents
,. Pur#ose of do!umentD intended audien!e
1. O->e!tie of testin( effort
2. +oftware #rodu!t oeriew
3. 8eleant related do!ument listD su!h as requirementsD desi(n do!umentsD other test #lansD et!.
4. 8eleant standards or le(al requirements
16. Tra!ea-ility requirements
11. 8eleant namin( !onentions and identifier !onentions
12. Test or(aniCation and #ersonnel9!onta!t0info9res#onsi-ilities
1". &ssum#tions and de#enden!ies
1$. Pro>e!t ris< analysis
1,. Testin( #riorities and fo!us
11. +!o#e and limitations of testin(
12. Test outline 0 a de!om#osition of the test a##roa!h -y test ty#eD featureD fun!tionalityD #ro!essD systemD moduleD
et!. as a##li!a-le
13. Outline of data in#ut equialen!e !lassesD -oundary alue analysisD error !lasses
14. Test enironment 0 hardwareD o#eratin( systemsD other required softwareD data !onfi(urationsD interfa!es to other
systems
26. Test enironment setu# and !onfi(uration issues
21. Test data setu# requirements
22. )ata-ase setu# requirements
2". Outline of system0lo((in(9error0lo((in(9other !a#a-ilitiesD and tools su!h as s!reen !a#ture softwareD that will -e
used to hel# des!ri-e and re#ort -u(s
2$. )is!ussion of any s#e!ialiCed software or hardware tools that will -e used -y testers to hel# tra!< the !ause or
sour!e of -u(s
2,. Test automation 0 >ustifi!ation and oeriew
21. Test tools to -e usedD in!ludin( ersionsD #at!hesD et!.
22. Test s!ri#t9test !ode maintenan!e #ro!esses and ersion !ontrol
23. Pro-lem tra!<in( and resolution 0 tools and #ro!esses
24. Pro>e!t test metri!s to -e used
"6. 8e#ortin( requirements and testin( deliera-les
"1. +oftware entran!e and e/it !riteria
"2. Initial sanity testin( #eriod and !riteria
"". Test sus#ension and restart !riteria
"$. Personnel #re0trainin( needs
",. Test site9lo!ation
"1. 8eleant #ro#rietaryD !lassifiedD se!urityD and li!ensin( issues.
"2. &##endi/ 0 (lossaryD a!ronymsD et!.
-2. TEST CASES
-3. What"s a "t!st *as!"'
& test !ase is a do!ument that des!ri-es an in#utD a!tionD or eent and an e/#e!ted res#onseD to determine if a
feature of an a##li!ation is wor<in( !orre!tly. & test !ase should !ontain #arti!ulars su!h as test !ase identifierD
test !ase nameD o->e!tieD test !onditions9setu#D in#ut data requirementsD ste#sD and e/#e!ted results.
Gote that the #ro!ess of deelo#in( test !ases !an hel# find #ro-lems in the requirements or desi(n of an
a##li!ationD sin!e it requires !om#letely thin<in( throu(h the o#eration of the a##li!ation. 5or this reasonD it's
useful to #re#are test !ases early in the deelo#ment !y!le if #ossi-le.
-(. TESTIN4 CO6ERA4E
1. Li+! *o)!a2!. Test eery line of !ode AOr Stat!9!+t *o)!a2!C test eery statementB.
2. Ba+*h *o)!a2!. Test eery lineD and eery -ran!h on multi0-ran!h lines.
". N<%!+2th s$:<.ath *o)!a2!. Test eery su-0#ath throu(h the #ro(ram of len(th G. 5or e/am#leD in a 16D6660line
#ro(ramD test eery #ossi-le 160line sequen!e of e/e!ution.
$. Path *o)!a2!. Test eery #ath throu(h the #ro(ramD from entry to e/it. The num-er of #aths is im#ossi-ly lar(e
to test.
,. M$%ti*o+,itio+ o .!,i*at! *o)!a2!. 5or!e eery lo(i!al o#erand to ta<e eery #ossi-le alue. Two different
!onditions within the same test may result in the same -ran!hD and so -ran!h !oera(e would only require the
testin( of one of them.
1. Tri((er eery assertion !he!< in the #ro(ram. *se im#ossi-le data if ne!essary.
2. Loo. *o)!a2!. H)ete!t -u(s that e/hi-it themseles only when a loo# is e/e!uted more than on!e.H
3. Low M!9o& Co+,itio+s 0 In the eent the a##li!ation fails due to low memory !onditionsD it should fail in
(ra!eful manner. This means that error lo(s should -e written detailin( what the !onditions were !ausin( the
failure to o!!urD orders the user may hae o#en are !losedD an error messa(e is (ien informin( the user of the
!ondition and the a!tions that will -e ta<enD et!
4. E)!& 9o,$%!D o:@!*tD *o9.o+!+tD too%D s$:s&st!9D !t*. This seems o-ious until you realiCe that many
#ro(rams rely on off0the0shelf !om#onents. The #ro(rammin( staff doesn't hae the sour!e !ode to these
!om#onentsD so measurin( line !oera(e is im#ossi-le. &t a minimum Awhi!h is what is measured hereBD you need
a list of all these !om#onents and test !ases that e/er!ise ea!h one at least on!e.
16. F$EE& ,!*isio+ *o)!a2!. If the #ro(ram ma<es heuristi!ally0-ased or similarity0-ased de!isionsD and uses
!om#arison rules or data sets that eole oer timeD !he!< eery rule seeral times oer the !ourse of trainin(.
11. R!%atio+a% *o)!a2!. HChe!<s whether the su-system has -een e/er!ised in a way that tends to dete!t off0-y0one
errorsH su!h as errors !aused -y usin( L instead of LJ. This !oera(e in!ludesE
12. .ery -oundary on eery in#ut aria-le.
1". .ery -oundary on eery out#ut aria-le.
1$. .ery -oundary on eery aria-le used in intermediate !al!ulations.
1,. Data *o)!a2!. &t least one test !ase for ea!h data item 9 aria-le 9 field in the #ro(ram.
11. Co+stai+ts a9o+2 )aia:%!sC Let X and Y -e two aria-les in the #ro(ram. A and Y !onstrain ea!h other if the
alue of one restri!ts the alues the other !an ta<e. 5or e/am#leD if X is a transa!tion date and Y is the
transa!tion's !onfirmation dateD Y !an't o!!ur -efore X.
12. Ea*h a..!aa+*! of a )aia:%!. +u##ose that you !an enter a alue for X on three different data entry s!reensD
the alue of X is dis#layed on another two s!reensD and it is #rinted in fie re#orts. Chan(e X at ea!h data entry
s!reen and !he!< the effe!t eerywhere else X a##ears.
13. E)!& t&.! of ,ata s!+t to !)!& o:@!*t. & <ey !hara!teristi! of o->e!t0oriented #ro(rammin( is that ea!h o->e!t
!an handle any ty#e of data Ainte(erD realD strin(D et!.B that you #ass to it. +oD #ass eery !on!eia-le ty#e of data
to eery o->e!t.
14. >a+,%i+2 of !)!& .ot!+tia% ,ata *o+f%i*t. 5or e/am#leD in an a##ointment0!alendarin( #ro(ramD what ha##ens
if the user tries to s!hedule two a##ointments at the same date and time?
26. >a+,%i+2 of !)!& !o stat!. Put the #ro(ram into the error stateD !he!< for effe!ts on the sta!<D aaila-le
memoryD handlin( of <ey-oard in#ut. 5ailure to handle user errors well is an im#ortant #ro-lemD #artially -e!ause
a-out 46X of industrial a!!idents are -lamed on human error or ris<0ta<in(. *nder the le(al do!trine of
foreseea-le misuseD the manufa!turer is lia-le in ne(li(en!e if it fails to #rote!t the !ustomer from the
!onsequen!es of a reasona-ly foreseea-le misuse of the #rodu!t.
21. E)!& *o9.%!;it& = 9ai+tai+a:i%it& 9!ti* a2ai+st !)!& 9o,$%!D o:@!*tD s$:s&st!9D !t*. There are many
su!h measures. Uones lists 26 of them. Peo#le sometimes as< whether any of these statisti!s are (rounded in a
theory of measurement or hae #ra!ti!al alue. ;ut it is !lear thatD in #ra!ti!eD some or(aniCations find them an
effe!tie tool for hi(hli(htin( !ode that needs further inesti(ation and mi(ht need redesi(n.
22. Co+fo9it& of !)!& 9o,$%!D s$:s&st!9D !t*. a2ai+st !)!& *o.oat! *o,i+2 sta+,a,. +eeral !om#anies
-eliee that it is useful to measure !hara!teristi!s of the !odeD su!h as total lines #er moduleD ratio of lines of
!omments to lines of !odeD frequen!y of o!!urren!e of !ertain ty#es of statementsD et!. & module that doesn't fall
within the HnormalH ran(e mi(ht -e summarily re>e!ted A-ad ideaB or re0e/amined to see if there's a -etter way to
desi(n this #art of the #ro(ram.
2". Ta:%!<,i)!+ *o,!. The ta-le is a list of addresses or #ointers or names of modules. In a traditional C&+.
statementD the #ro(ram -ran!hes to one of seeral #la!es de#endin( on the alue of an e/#ression. In the ta-le0
drien equialentD the #ro(ram would -ran!h to the #la!e s#e!ified inD sayD lo!ation 2" of the ta-le. The ta-le is
#ro-a-ly in a se#arate data file that !an ary from day to day or from installation to installation. ;y modifyin( the
ta-leD you !an radi!ally !han(e the !ontrol flow of the #ro(ram without re!om#ilin( or relin<in( the !ode. +ome
#ro(rams drie a (reat deal of their !ontrol flow this wayD usin( seeral ta-les. Coera(e measures? +ome
e/am#lesE
2$. !he!< that eery e/#ression sele!ts the !orre!t ta-le element
2,. !he!< that the #ro(ram !orre!tly >um#s or !alls throu(h eery ta-le element
21. !he!< that eery address or #ointer that is aaila-le to -e loaded into these ta-les is alid Ano >um#s to im#ossi-le
#la!es in memoryD or to a routine whose startin( address has !han(edB
22. !he!< the alidity of eery ta-le that is loaded at any !ustomer site.
23. E)!& i+t!$.t. &n interru#t is a s#e!ial si(nal that !auses the !om#uter to sto# the #ro(ram in #ro(ress and
-ran!h to an interru#t handlin( routine. LaterD the #ro(ram restarts from where it was interru#ted. Interru#ts mi(ht
-e tri((ered -y hardware eents AI9O or si(nals from the !lo!< that a s#e!ified interal has ela#sedB or software
Asu!h as error tra#sB. =enerate eery ty#e of interru#t in eery way #ossi-le to tri((er that interru#t.
24. E)!& i+t!$.t at !)!& tas?D 9o,$%!D o:@!*tD o !)!+ !)!& %i+!. The interru#t handlin( routine mi(ht !han(e
state aria-lesD load dataD use or shut down a #eri#heral dei!eD or affe!t memory in ways that !ould -e isi-le to
the rest of the #ro(ram. The interru#t !an ha##en at any time0-etween any two linesD or when any module is -ein(
e/e!uted. The #ro(ram may fail if the interru#t is handled at a s#e!ifi! time. A./am#leE what if the #ro(ram
-ran!hes to handle an interru#t while it's in the middle of writin( to the dis< drie?B
The num-er of test !ases here is hu(eD -ut that doesn't mean you don't hae to thin< a-out this ty#e of testin(.
This is #ath testin( throu(h the eyes of the #ro!essor Awhi!h as<sD HWhat instru!tion do I e/e!ute ne/t?H and
doesn't !are whether the instru!tion !omes from the mainline !ode or from an interru#t handlerB rather than #ath
testin( throu(h the eyes of the reader of the mainline !ode. .s#e!ially in #ro(rams that hae (lo-al state
aria-lesD interru#ts at une/#e!ted times !an lead to ery odd results.
"6. E)!& a+ti*i.at!, o .ot!+tia% a*!. Ima(ine two eentsD A and B. ;oth will o!!urD -ut the #ro(ram is desi(ned
under the assum#tion that A will always #re!ede B. This sets u# a ra!e -etween A and B 0if B eer #re!edes AD the
#ro(ram will #ro-a-ly fail. To a!hiee ra!e !oera(eD you must identify eery #otential ra!e !ondition and then
find waysD usin( random data or systemati! test !ase sele!tionD to attem#t to drie B to #re!ede A in ea!h !ase.
8a!es !an -e su-tle. +u##ose that you !an enter a alue for a data item on two different data entry s!reens. *ser 1
-e(ins to edit a re!ordD throu(h the first s!reen. In the #ro!essD the #ro(ram lo!<s the re!ord in Ta-le 1. *ser 2
o#ens the se!ond s!reenD whi!h !alls u# a re!ord in a different ta-leD Ta-le 2. The #ro(ram is written to
automati!ally u#date the !orres#ondin( re!ord in the Ta-le 1 when *ser 2 finishes data entry. GowD su##ose that
*ser 2 finishes -efore *ser 1. Ta-le 2 has -een u#datedD -ut the attem#t to syn!hroniCe Ta-le 1 and Ta-le 2 fails.
What ha##ens at the time of failureD or later if the !orres#ondin( re!ords in Ta-le 1 and 2 stay out of syn!h?
"1. E)!& ti9!<s%i*! s!tti+2. In some systemsD you !an !ontrol the (rain of swit!hin( -etween tas<s or #ro!esses.
The siCe of the time quantum that you !hoose !an ma<e ra!e -u(sD time0outsD interru#t0related #ro-lemsD and
other time0related #ro-lems more or less li<ely. Of !ourseD !oera(e is a diffi!ult #ro-lem here -e!ause you aren't
>ust aryin( time0sli!e settin(s throu(h eery #ossi-le alue. You also hae to de!ide whi!h tests to run under
ea!h settin(. =ien a #lanned set of test !ases #er settin(D the !oera(e measure loo<s at the num-er of settin(s
you'e !oered.
"2. 6ai!, %!)!%s of :a*?2o$+, a*ti)it&. In a multi#ro!essin( systemD tie u# the #ro!essor with !om#etin(D
irreleant -a!<(round tas<s. Loo< for effe!ts on ra!es and interru#t handlin(. +imilar to time0sli!esD your
!oera(e analysis must s#e!ify
"". !ate(ories of leels of -a!<(round a!tiity Afi(ure out somethin( that ma<es senseB and
"$. all timin(0sensitie testin( o##ortunities Ara!esD interru#tsD et!.B.
",. Ea*h .o*!sso t&.! a+, s.!!,. Whi!h #ro!essor !hi#s do you test under? What tests do you run under ea!h
#ro!essor? You are loo<in( forE
"1. s#eed effe!tsD li<e the ones you loo< for with -a!<(round a!tiity testin(D and
"2. !onsequen!es of #ro!essors' different memory mana(ement rulesD and
"3. floatin( #oint o#erationsD and
"4. any #ro!essor0ersion0de#endent #ro-lems that you !an learn a-out.
$6. .ery o##ortunity for file 9 re!ord 9 field lo!<in(.
$1. .ery de#enden!y on the lo!<ed Aor unlo!<edB state of a fileD re!ord or field.
$2. .ery o##ortunity for !ontention for dei!es or resour!es.
$". P!fo9a+*! of !)!& 9o,$%! = tas? = o:@!*t. Test the #erforman!e of a module then retest it durin( the ne/t
!y!le of testin(. If the #erforman!e has !han(ed si(nifi!antlyD you are either loo<in( at the effe!t of a
#erforman!e0si(nifi!ant redesi(n or at a sym#tom of a new -u(.
$$. 5ree memory 9 aaila-le resour!es 9 aaila-le sta!< s#a!e at eery line or on entry into and e/it out of eery
module or o->e!t.
$,. ./e!ute eery line A-ran!hD et!.B under the de-u( ersion of the o#eratin( system. This shows ille(al or
#ro-lemati! !alls to the o#eratin( system.
$1. 6a& th! %o*atio+ of !)!& fi%!. What ha##ens if you install or moe one of the #ro(ram's !om#onentD !ontrolD
initialiCation or data files to a different dire!tory or drie or to another !om#uter on the networ<?
$2. Ch!*? th! !%!as! ,is?s fo th! .!s!+*! of !)!& fi%!. It's amaCin( how often a file anishes. If you shi# the
#rodu!t on different mediaD !he!< for all files on all media.
$3. E)!& !9:!,,!, sti+2 i+ th! .o2a9. *se a utility to lo!ate em-edded strin(s. Then find a way to ma<e the
#ro(ram dis#lay ea!h strin(.
$4. O#eration of eery fun!tion 9 feature 9 data handlin( o#eration underE
,6. .ery #ro(ram #referen!e settin(.
,1. .ery !hara!ter setD !ode #a(e settin(D or !ountry !ode settin(.
,2. The #resen!e of eery memory resident utility AinitsD T+8sB.
,". .a!h o#eratin( system ersion.
,$. .a!h distin!t leel of multi0user o#eration.
,,. .a!h networ< ty#e and ersion.
,1. .a!h leel of aaila-le 8&:.
,2. .a!h ty#e 9 settin( of irtual memory mana(ement.
,3. Com#ati-ility with eery #reious ersion of the #ro(ram.
,4. A:i%it& to !a, !)!& t&.! of ,ata a)ai%a:%! i+ !)!& !a,a:%! i+.$t fi%! fo9at. If a file format is su->e!t to
su-tle ariations Ae.(. C=:B or has seeral su-0ty#es Ae.(. TI55B or ersions Ae.(. d;&+.BD t!st !a*h o+!.
16. Wit! !)!& t&.! of ,ata to !)!& a)ai%a:%! o$t.$t fi%! fo9at. &(ainD -eware of su-tle ariations in file
formats0if you're writin( a C=: fileD full !oera(e would require you to test your #ro(ram's out#ut's reada-ility
-y !)!& o+! of the main #ro(rams that read C=: files.
11. E)!& t&.!fa*! s$..%i!, with th! .o,$*t. Che!< all !hara!ters in all siCes and styles. If your #ro(ram adds
ty#efa!es to a !olle!tion of fonts that are aaila-le to seeral other #ro(ramsD !he!< !om#ati-ility with the other
#ro(rams Anonstandard ty#efa!es will !rash some #ro(ramsB.
12. E)!& t&.! of t&.!fa*! *o9.ati:%! with th! .o2a9. 5or e/am#leD you mi(ht test the #ro(ram with Amany
differentB TrueTy#e and Posts!ri#t ty#efa!esD and fi/ed0siCed -itma# fonts.
1". E)!& .i!*! of *%i. at i+ th! .o,$*t. Test ea!h with this #ro(ram. Test ea!h with other #ro(rams that should -e
a-le to read this ty#e of art.
1$. E)!& so$+, = a+i9atio+ .o)i,!, with th! .o,$*t. Play them all under different dei!e Ae.(. soundB driers 9
dei!es. Che!< !om#ati-ility with other #ro(rams that should -e a-le to #lay this !li#0!ontent.
1,. E)!& s$..%i!, Fo *o+st$*ti:%!G s*i.t to drie other ma!hines 9 software Ae.(. ma!rosB 9 ;;+'s and
information seri!es A!ommuni!ations s!ri#tsB.
11. &ll !ommands aaila-le in a su##lied !ommuni!ations #roto!ol.
12. R!*o2+iE!, *haa*t!isti*s. 5or e/am#leD eery s#ea<er's oi!e !hara!teristi!s Afor oi!e re!o(nition softwareB
or writer's handwritin( !hara!teristi!s Ahandwritin( re!o(nition softwareB or eery ty#efa!e AOC8 softwareB.
13. .ery ty#e of <ey-oard and <ey-oard drier.
14. .ery ty#e of #ointin( dei!e and drier at eery resolution leel and -allisti! settin(.
26. .ery out#ut feature with eery sound !ard and asso!iated driers.
21. .ery out#ut feature with eery ty#e of #rinter and asso!iated driers at eery resolution leel.
22. .ery out#ut feature with eery ty#e of ideo !ard and asso!iated driers at eery resolution leel.
2". .ery out#ut feature with eery ty#e of terminal and asso!iated #roto!ols.
2$. .ery out#ut feature with eery ty#e of ideo monitor and monitor0s#e!ifi! driers at eery resolution leel.
2,. .ery !olor shade dis#layed or #rinted to eery !olor out#ut dei!e Aideo !ard 9 monitor 9 #rinter 9 et!.B and
asso!iated driers at eery resolution leel. &nd !he!< the !onersion to (rey s!ale or -la!< and white.
21. .ery !olor shade reada-le or s!anna-le from ea!h ty#e of !olor in#ut dei!e at eery resolution leel.
22. .ery #ossi-le feature intera!tion -etween ideo !ard ty#e and resolutionD #ointin( dei!e ty#e and resolutionD
#rinter ty#e and resolutionD and memory leel. This may seem e/!essiely !om#le/D -ut some times !rash -u(s
o!!ur only under the #airin( of s#e!ifi! #rinter and ideo driers at a hi(h resolution settin(. Other !rashes
required #airin( of a s#e!ifi! mouse and #rinter drierD #airin( of mouse and ideo drierD and a !om-ination of
mouse drier #lus ideo drier #lus -allisti! settin(.
23. .ery ty#e of C)08O: drieD !onne!ted to eery ty#e of #ort Aserial 9 #arallel 9 +C+IB and asso!iated driers.
24. .ery ty#e of writa-le dis< drie 9 #ort 9 asso!iated drier. )on't for(et the fun you !an hae with remoa-le
dries or dis<s.
36. Com#ati-ility with eery ty#e of dis< !om#ression software. Che!< error handlin( for eery ty#e of dis< errorD
su!h as full dis<.
31. .ery olta(e leel from analo( in#ut dei!es.
32. .ery olta(e leel to analo( out#ut dei!es.
3". .ery ty#e of modem and asso!iated driers.
3$. .ery 5&V !ommand Asend and re!eie o#erationsB for eery ty#e of 5&V !ard under eery #roto!ol and drier.
3,. .ery ty#e of !onne!tion of the !om#uter to the tele#hone line Adire!tD ia P;VD et!.F di(ital s. analo(
!onne!tion and si(nalin(BF test eery #hone !ontrol !ommand under eery tele#hone !ontrol drier.
31. Toleran!e of eery ty#e of tele#hone line noise and re(ional ariation Ain!ludin( ariations that are out of s#e!B in
tele#hone si(nalin( AintensityD frequen!yD timin(D other !hara!teristi!s of rin( 9 -usy 9 et!. tonesB.
32. .ery ariation in tele#hone dialin( #lans.
33. E)!& .ossi:%! ?!&:oa, *o9:i+atio+. +ometimes you'll find tra# doors that the #ro(rammer used as hot<eys to
!all u# de-u((in( toolsF these hot<eys may !rash a de-u((erless #ro(ram. Other timesD you'll dis!oer an .aster
.(( Aan undo!umentedD #ro-a-ly unauthoriCedD and #ossi-ly em-arrassin( featureB. Th! :oa,! *o)!a2!
9!as$! is !)!& .ossi:%! ?!&:oa, *o9:i+atio+ at !)!& !o 9!ssa2! a+, !)!& ,ata !+t& .oi+t. You'll
often find different -u(s when !he!<in( different <eys in res#onse to different error messa(es.
34. R!*o)!& fo9 !)!& .ot!+tia% t&.! of !#$i.9!+t fai%$!. 5ull !oera(e in!ludes ea!h ty#e of equi#mentD ea!h
drierD and ea!h error state. 5or e/am#leD test the #ro(ram's a-ility to re!oer from full dis< errors on writa-le
dis<s. In!lude flo##iesD hard driesD !artrid(e driesD o#ti!al driesD et!. In!lude the arious !onne!tions to the
drieD su!h as I).D +C+ID :5:D #arallel #ortD and serial !onne!tionsD -e!ause these will #ro-a-ly inole
different driers.
46. F$+*tio+ !#$i)a%!+*!. 5or ea!h mathemati!al fun!tionD !he!< the out#ut a(ainst a <nown (ood im#lementation
of the fun!tion in a different #ro(ram. Com#lete !oera(e inoles equialen!e testin( of all testa-le fun!tions
a!ross all #ossi-le in#ut alues.
41. H!o ha+,%i+2. 5or ea!h mathemati!al fun!tionD test when eery in#ut alueD intermediate aria-leD or out#ut
aria-le is Cero or near0Cero. Loo< for seere roundin( errors or diide0-y0Cero errors.
42. A**$a*& of !)!& 2a.hD a!ross the full ran(e of (ra#ha-le alues. In!lude alues that for!e shifts in the s!ale.
4". A**$a*& of !)!& !.ot. Loo< at the !orre!tness of eery alueD the formattin( of eery #a(eD and the
!orre!tness of the sele!tion of re!ords used in ea!h re#ort.
4$. &!!ura!y of eery messa(e.
4,. &!!ura!y of eery s!reen.
41. &!!ura!y of eery word and illustration in the manual.
42. &!!ura!y of eery fa!t or statement in eery data file #roided with the #rodu!t.
43. &!!ura!y of eery word and illustration in the on0line hel#.
44. .ery >um#D sear!h termD or other means of nai(ation throu(h the on0line hel#.
166.Che!< for eery ty#e of irus 9 worm that !ould shi# with the #ro(ram.
161..ery #ossi-le <ind of se!urity iolation of the #ro(ramD or of the system while usin( the #ro(ram.
162.Che!< for !o#yri(ht #ermissions for eery statementD #i!tureD sound !li#D or other !reation #roided with the
#ro(ram.
16".7erifi!ation of the #ro(ram a(ainst eery #ro(ram requirement and #u-lished s#e!ifi!ation.
16$.6!ifi*atio+ of th! .o2a9 a2ai+st $s! s*!+aios. *se the #ro(ram to do real tas<s that are !hallen(in( and
well0s#e!ified. 5or e/am#leD !reate <ey re#ortsD #i!turesD #a(e layoutsD or other do!uments eents to mat!h ones
that hae -een featured -y !om#etitie #ro(rams as interestin( out#ut or a##li!ations.
16,.7erifi!ation a(ainst eery re(ulation AI8+D +.CD 5)&D et!.B that a##lies to the data or #ro!edures of the #ro(ram.
161.*sa-ility tests ofE
162..ery feature 9 fun!tion of the #ro(ram.
163..ery #art of the manual.
164..ery error messa(e.
116. .ery on0line hel# to#i!.
111. .ery (ra#h or re#ort #roided -y the #ro(ram.
112. Lo!aliCa-ility 9 lo!aliCation testsE
11". E)!& sti+2. Che!< #ro(ram's a-ility to dis#lay and use this strin( if it is modified -y !han(in( the len(thD usin(
hi(h or low &+CII !hara!tersD different !a#italiCation rulesD et!.
11$. Com#ati-ility with te/t handlin( al(orithms under other lan(ua(es Asortin(D s#ell !he!<in(D hy#henatin(D et!.B
11,. .ery dateD num-er and measure in the #ro(ram.
111. ?ardware and driersD o#eratin( system ersionsD and memory0resident #ro(rams that are #o#ular in other
!ountries.
112. .ery in#ut formatD im#ort formatD out#ut formatD or e/#ort format that would -e !ommonly used in #ro(rams
that are #o#ular in other !ountries.
113. Cross0!ultural a##raisal of the meanin( and #ro#riety of eery strin( and (ra#hi! shi##ed with the #ro(ram.
--. W>AT IF T>ERE ISN"T ENOU4> TIME FOR T>OROU4> TESTIN4'
*se ris< analysis to determine where testin( should -e fo!used. +in!e it's rarely #ossi-le to test eery #ossi-le as#e!t of an
a##li!ationD eery #ossi-le !om-ination of eentsD eery de#enden!yD or eerythin( that !ould (o wron(D ris< analysis is
a##ro#riate to most software deelo#ment #ro>e!ts. This requires >ud(ment s<illsD !ommon senseD and e/#erien!e. AIf
warrantedD formal methods are also aaila-le.B Considerations !an in!ludeE
114. Whi!h fun!tionality is most im#ortant to the #ro>e!t's intended #ur#ose?
126.Whi!h fun!tionality is most isi-le to the user?
121.Whi!h fun!tionality has the lar(est safety im#a!t?
122.Whi!h fun!tionality has the lar(est finan!ial im#a!t on users?
12".Whi!h as#e!ts of the a##li!ation are most im#ortant to the !ustomer?
12$.Whi!h as#e!ts of the a##li!ation !an -e tested early in the deelo#ment !y!le?
12,.Whi!h #arts of the !ode are most !om#le/D and thus most su->e!t to errors?
121.Whi!h #arts of the a##li!ation were deelo#ed in rush or #ani! mode?
122.Whi!h as#e!ts of similar9related #reious #ro>e!ts !aused #ro-lems?
123.Whi!h as#e!ts of similar9related #reious #ro>e!ts had lar(e maintenan!e e/#enses?
124.Whi!h #arts of the requirements and desi(n are un!lear or #oorly thou(ht out?
1"6.What do the deelo#ers thin< are the hi(hest0ris< as#e!ts of the a##li!ation?
1"1.What <inds of #ro-lems would !ause the worst #u-li!ity?
1"2.What <inds of #ro-lems would !ause the most !ustomer seri!e !om#laints?
1"".What <inds of tests !ould easily !oer multi#le fun!tionalities?
1"$.Whi!h tests will hae the -est hi(h0ris<0!oera(e to time0required ratio?
-/. DEFECT REPORTIN4
The -u( needs to -e !ommuni!ated and assi(ned to deelo#ers that !an fi/ it. &fter the #ro-lem is resoledD fi/es should -e
re0testedD and determinations made re(ardin( requirements for re(ression testin( to !he!< that fi/es didn't !reate #ro-lems
elsewhere. The followin( are items to !onsider in the tra!<in( #ro!essE
1",.Com#lete information su!h that deelo#ers !an understand the -u(D (et an idea of it's seerityD and re#rodu!e it if
ne!essary.
1"1.;u( identifier Anum-erD I)D et!.B
1"2.Current -u( status Ae.(.D 'O#en'D 'Closed'D et!.B
1"3.The a##li!ation name and ersion
1"4.The fun!tionD moduleD featureD o->e!tD s!reenD et!. where the -u( o!!urred
1$6..nironment s#e!ifi!sD systemD #latformD releant hardware s#e!ifi!s
1$1.Test !ase name9num-er9identifier
1$2.5ile e/!er#ts9error messa(es9lo( file e/!er#ts9s!reen shots9test tool lo(s that would -e hel#ful in findin( the !ause
of the #ro-lem
1$".+eerity Leel
1$$.Tester name
1$,.;u( re#ortin( date
1$1.Game of deelo#er9(rou#9or(aniCation the #ro-lem is assi(ned to
1$2.)es!ri#tion of fi/
1$3.)ate of fi/
1$4.&##li!ation ersion that !ontains the fi/
1,6.7erifi!ation )etails
& re#ortin( or tra!<in( #ro!ess should ena-le notifi!ation of a##ro#riate #ersonnel at arious sta(es.
-3. TYPES OF AUTOMATED TOOLS
1,1.!ode analyCers 0 monitor !ode !om#le/ityD adheren!e to standardsD et!.
1,2.!oera(e analyCers 0 these tools !he!< whi!h #arts of the !ode hae -een e/er!ised -y a testD and may -e oriented
to !ode statement !oera(eD !ondition !oera(eD #ath !oera(eD et!.
1,".memory analyCers 0 su!h as -ounds0!he!<ers and lea< dete!tors.
1,$.load9#erforman!e test tools 0 for testin( !lient9serer and we- a##li!ations under arious load leels.
1,,.we- test tools 0 to !he!< that lin<s are alidD ?T:L !ode usa(e is !orre!tD !lient0side and
1,1.serer0side #ro(rams wor<D a we- site's intera!tions are se!ure.
1,2.other tools 0 for test !ase mana(ementD do!umentation mana(ementD -u( re#ortin(D and !onfi(uration
mana(ement.
-5. TOP TIPS FOR TESTERS
Ti# 1E Play ni!ely. ;e #art of the We- team. This <ee#s you in the loo#D and (ies you a oi!e in de-ates of fun!tionality
ersus desi(n. 8emem-erD a tester is really a user ado!ate. Prote!t that #osition -y ma<in( friends.
Ti# 2E & (ood s#e! is a tester's -est friend. =et desi(n and fun!tional s#e!ifi!ations 00 een if you hae to whine or threaten.
+#e!s hel# you determine what is a HrealH -u( and what is H-y desi(n.H
Ti# "E TI++ ATee# It +im#leD +tu#idB. If you hae a form that's su##osed to a!!e#t only inte(ers 1 to 16D you !an start your
tests -y enterin( G*LLD 6D 1D 16D 11D 1666666666D 01D aD and the other a!!e#ta-le alues. You needn't enter eery #ossi-le
<ey stro<e to find a !ode hole.
Ti# $E ./#e!t the une/#e!ted. ;efore you thin< the #reious ti# allows you to !ut -a!< your testin(D remem-er the user. If
your audien!e is non0te!hni!alD the #ossi-le <eystro<esD -rowser settin(sD and other HmysteryH influen!es are unlimited.
Thin< li<e a userD and stress the We- site the way a user would.
Ti# ,E *se all the automated testin( tools you !an (et your hands on. :any useful tools are on the We- already. With a little
sear!hin(D you may -e a-le to find the ri(ht ones for you.