A SUMMARY ON TEST DRIVEN DEVELOPEMENT
Bharat Rajan
Graduate student
Department of computer science
University of Alabama
+001-(860) 465-6991
brajan@crimson.ua.edu
ABSTRACT
This summary deals with a contemporary topic of software
engineering, Test driven development. In traditional
development,
tests are for verification and validation purposes and are built
after the target product feature exists. In test driven
development, tests are used for specification purposes in
addition to verification and validation. In extreme programming
XP, test first concept is used this is the starting point which
leads to the development of Test driven development .Kent Beck
formally introduced this development technique in 2003. There
are many development techniques available in software
engineering, but this Test driven development, has a unique
feature of developing the code after developing the test cases
and it has many advantages.
Categories and Subject Descriptors
D.2.8 [Software Engineering]: General analysis,
application, advantages, limitation, measurements and future
work
General Terms
Measurement, Performance, Economics, Reliability.
Keywords
Test-Driven Development, software quality, test
cases, refactoring,
1. INTRODUCTION
The concept of test driven development came from extreme
programming. In the system testing, the entire behavior of the
clients application is at least simulated then tested for
performance. This performance testing is done with respect to
the time and the old ones. This type of development has some
problems they are, different confirmation of the system , a good
test is hard to write ,in the developer point of view, he has to
track a bug and fix it and in the management perspective it’s a
time consuming process. In extreme programming the test first
concept is introduced .Then it is developed into a new
development technique. In this summary we are going to discuss
about the performance, reliability advantages and limitations of
test driven development by analyzing some case studies. Test
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
driven development has got many advantages such as reduction
in the testing time, improves the ability and productivity of the
developer etc., here the test suite is written first before writing
the code. As it uses a unique technique of developing test cases
before writing the system code there exists a conflict of, how to
determine the test cases before knowing the code. Because
normally we used to write the unit test cases to test the system
code at the end.
In this summary, Section 2 describes the working of test driven
development .Section 3 describes the performances .Section 4
describes the analysis.
2. TEST FIRST CONCEPT
In the extreme programming the unit test cases are written first
before writing the code this is the base for the test driven
development. The unit test cases are either written by keeping
the customer requirement in mind or it is generated using some
tools. In the former process the developer has to know the exact
and entire requirement of the customer so as to write the unit
test cases. This feature improves the productivity of the
developer. As much of the code is written in the test cases the
development of the system code becomes easy.
2.1 TEST DRIVEN DEVELOPMENT
In this technique the unit test cases are written first then the
system code is generated according to this unit test cases. The
code thus generated is then tested and refactoring is done. [1]
There are several steps should be followed in this test driven
development technique.
• Writing a (very) small number of automated unit
test case(s);
• Running the new unit test case(s) to ensure they
fail (since there is no code to run yet);
• Implementing code which should allow the new
unit test cases to pass;
• Re-running the new unit test cases to ensure they
now pass with the new code;
• Refactoring the implementation or test code, as
necessary, and
• Periodically (preferably once a day or more) re-
running all the test cases in the code base to ensure
the new code does not break any previously-
running test cases.
These test cases are automated and are small in
size. They are prepared from the requirements of
the customers. So the customer requirement is
completely analyzed by the developer or the use
cases and/or the user stories are the main thing for
the automation of the unit test cases. These unit
test cases contain assertions which are the normal
check statement used in the test cases. After
generating these automated test cases they are
made to run. By default they should produce the fail
condition. Because there is no code developed till
now to implement the unit test cases. So the unit
test cases are just executed as it is with null inputs
being given to them. This ensures that the
automated unit test cases works well. Now we are
ready with the unit initial test cases. At this stage
the code for the system is generated by keeping
these automated test cases in mind. Thus written
codes should correspond to the unit test cases i.e.
the code should be written so as to satisfy the
produce a pass in all the unit test cases. Then again
new codes are generated and is passed to the tests.
This stage is very important because the system
code is generated and also tested at the same time.
So to produce the system codes behave with all the
expected functionalities, the test cases should be
written properly.
After writing the system code, the process of refactoring is
carried out. It is the place where the duplication is eliminated.
As it is refactoring process no change in the functionality is
done but the architecture and build of the code is altered to
eliminate the duplication. The duplication occurs because both
the codes i.e. the code for unit test cases and the system code
upon which this test cases are implemented are present in the
same part of the system code. So there is high possibility of
occurrence of the duplication. There several methods to track
and refractor. Refactoring is done either by changing or
modifying the test methods or by changing the system codes. In
either method after a modification the system code is tested
using the test cases and a verification is made for the tests. This
is refectored until it gives a more efficient piece of code and a
high quality is achieved. Now for the full set of the system code,
all the unit test cases are again implemented validated and
verified. Kent Beck says “Fake it, till you make it” [8] i.e. the
test cases are modified (refactoring) until the required
functionality is achieved.
The diagram figure 1 represents the above mentioned six steps
in a flowchart. As the process of refactoring is made repetitively
the entire process looks to be an iterative process. On the other
hand it gives a feedback to the system and hence improves the
quality of the system. In this diagram the first four steps are
minute by minute process whereas the next two steps are carried
out periodically once in a day.
2.2 WORKING
In the test driven development unit test cases codes are written
inside the system itself without compromising the
functionalities and features of the system code. The features that
are mentioned includes the information hiding, encapsulation
etc., But there occurs a contradiction when the system code
make use of the databases. Because the units test cases cannot
be produced effectively without knowing about the databases
their usage and influence inside the system code. Similarly in
the private classes, we are facing the same problem. The MSDN
library has created the new set of steps for the object oriented
approach of the system code.
• Creating and running automated tests inside
• Abstracting dependencies in an object-oriented world
• Refactoring new and old features to remove duplication in
code
• Author a Unit Test
• Organize Tests into Test Lists
• Run Selected Tests
This follows the steps same as that of the normal development
process but the difference is that the dependencies in the object
oriented code is also kept in mind while refactoring the code.
The MSDN library also says about Red Green Refractor cycle
concept [2] i.e. the Red: Create a test and make it fail. Imagine
how the new code should be called and write the test as if the
code already existed. You will not get IntelliSense because the
new method does not yet exist. Create the new production code
stub. Write just enough code so that it compiles. Run the test. It
should fail. This is a calibration measure to ensure that your test
is calling the correct code and that the code is not working by
accident. This is a meaningful failure, and you expect it to fail.
Green: Make the test pass by any means necessary. Write the
production code to make the test pass. Keep it simple. Some
advocate the hard-coding of the expected return value first to
verify that the test correctly detects success. This varies from
practitioner to practitioner. If you've written the code so that the
test passes as intended, you are finished. You do not have to
write more code speculatively. The test is the objective
definition of "done." The phrase "You Ain't Gonna Need It"
(YAGNI) is often used to veto unnecessary work. If new
functionality is still needed, then another test is needed. Make
this one test pass and continue. When the test passes, you might
want to run all tests up to this point to build confidence that
everything else is still working.Refactor: Change the code to
remove duplication in your project and to improve the design
while ensuring that all tests still pass. Remove duplication
caused by the addition of the new functionality. Make design
changes to improve the overall solution. After each refactoring,
rerun all the tests to ensure that they all still pass. Repeat the
cycle. Each cycle should be very short, and a typical hour should
contain many Red/Green/Refractor cycles.
3 PERFORMANCES
This test driven development is said to be have a reduced testing
time less error rate and a less implementation time. But they are
bounded within the nature of the customer requirements. In the
industrial scale this test driven development plays a significant
roll. The ability of the project team to predict the effort required
to complete the target program may be greater if the developers
use a test-first process [3] .the results of A. Geras M. Smith J.
Miller says that it is more difficult to implement the test driven
development in the industrial case because it is a kind of
empirical experiment where only a little training can be
provided to the developers. This little training makes them to
perform low in the test cases derivation. In the customer point of
view the customer must be familiar with the developers test
tools because many times the unit test cases are automated in
from the tools. So to satisfy both the above constraints the
developers are made to communicate with the customer to know
about the requirements and the customer to know about the tools
used. So the communication improves between the stakeholders.
3.1 CASE STUDIES
George and Williams performed a set of structured experiments
[7] in which 24 pairs of professional programmers were
involved. One group developed a small JAVA program by
applying test driven development, whereas the other (control)
group used the waterfall lifecycle model. The pairs using test
driven development produced a better quality code (18% higher)
than the pairs who did not use test driven development, although
the former required 16% longer time. This study provided
evidence that test driven development increases the level of tests
passed and improves the quality of the code. Williams et al.
carried out a case study in IBM [5]; the process consisted of
developing an automatic package of test cases once the system
was designed with UML. As a result, the code developed by
applying test driven development had 40% fewer defects when
compared with the code of an experienced team using an ad-hoc
testing approach. Besides, test driven development had a
minimal impact on the developer’s productivity. All these above
cases studies may show that it produces a good result while
using the test driven development technique but the
experiments are bound to some limitations.
. development technique. Mean Time per Assertion is assumed as
3.2 TEST DRIVEN DEVELOPEMENT IN
INDUSTRY
A few evaluative research studies have been conducted on test
driven development with professional practitioners. North
Carolina State University seems to be the only source of such a
study to date. Researchers at NCSU have performed at least
three empirical studies on test driven development in industry
settings involving fairly small groups in at least four different
companies. These studies examined defect density as a measure
of software quality, although some survey data indicated that
programmers thought test driven development promoted simpler
designs. In one study, programmers’ experience with test driven
development varied from novice to expert, while programmers
new to test driven development anticipated in the other studies.
These studies showed that programmers using test driven
development produced code that passed 18 percent to 50 percent
more external test cases than code produced by corresponding
control groups. The studies also reported less time spent
debugging code developed with test driven development.
Further, they reported that applying test driven development had
an impact that ranged from minimal to a 16 percent decrease in
programmer productivity—which shows that applying test
driven development sometimes took longer. In the case that took
16 percent more time, researchers noted that the control group
wrote far fewer tests than the test driven development group.
4 ANALYSIS
Gerardo Canfora, Aniello Cimitile and Felix Garcia
[4]conducted an experiment with Evaluating Advantages of Test
driven development : a Controlled Experiment with
Professionals and took two things to analyze the performance of
the test driven development they are 1) productivity from the
viewpoint of testing. In the case, the product is intended as the
set of test cases and correct code; the code is considered correct
if all the related tests succeed. Thus, ‘productivity’ is seen as the
efficiency in producing test cases and correct code. 2) Quality of
unit testing? We evaluate the differences between test driven
development and formal method in terms of accuracy and
precision of unit tests.
In their experiment they prove that the Mean Time per
assertion.i.e. the time required to write and execute a assertion
in the test suite. And the mean time for writing and executing a
test suite is more for the test driven development than the formal
an indicator of the productivity. The mean time indicates the
effort spent by subjects when performing the practices. The time
consumed by the test driven development is more. This excess
time is used to improve the quality of the software and more
feedbacks are given to the product. yet there is no statistical
evidence for the improvement in the quality of the product. But
one of the major advantages is that the predictability of the part
of the system is given by the test driven development using
which the cost of the product can be found. Even here all the
experiments are done in the controlled environment.
4.1 TEST DRIVEN DEVELOPMENT FOR
DEFECT REDUCTION
Laurie Williams [5] states that when new codes are added to the
already existing code, faults and/or defects may occur. These
faults are detected quickly using the test driven development
and hence the time is saved. This leads to minimization of
schedule. In the formal development technique, the process of
finding and fixing these faults and/or defects may cause more
errors or the needs more modifications in the system code. But
in test driven development the debugging process may be made
easy by changing the test cases codes. In test driven
development the probability of the error rate is reduced and also
the fixing of error also follows the same procedure. It is because
most of the automated unit test cases are used as the validation
and verification test cases. On applying the test driven
development repeatedly, Laurie Williams proves that the
probability of error reduces for new systems than the legacy
system. In the case of a system using both the legacy part and a
new system, the test cases are likely to be same because of the
fact that the new system has to come across all the test cases the
legacy system does apart from that the new system goes through
some more test cases. Also for each class of device there is a
percentage of the test cases that are only ran once because they
were common for all devices. Hence, the number of test cases
needed for a class of device could be reduced by this factor.
(commonTCFactor is used to account for this effect) The total
number of test cases (TC) run on each product may be
approximated by the following formula:
TC= Σ (number Models*TCforDevice*commonTCFactor)*
deviceClass
numberOfOS*numberOfSystemFamily
4.2 ADVANTAGES AND LIMITATIONS OF
TEST DRIVEN DEVOLOPEMENT
As testing is done side by side, complete testing is done with a
positive result so the quality of the software improves. This
development process is based on the complete requirement
analysis. This method finds the error in the code and generates
the target which is to be kept in mind during the process of
debugging so the process of debugging can be done quick.
While refactoring, if the modification in the program leads to a
different unexpected answer, then the last successful version can
be taken as the most effective code. It has less implementation
time and debug time.
The limitations are it cannot be used for all type of applications
like the application using databases because full functional test
cases cannot be prepared. This technique should be followed by
all the levels of the team and throughout the entire application.
Bad testing increases the maintenance cost. If the customer
changes the requirement then the entire process has to be done
from the start.
5 CONCLUSIONS AND FUTURE WORK
In this analysis we have investigated the utility of test driven
development from the view point of software quality and
productivity in terms of development time. Drawing general
conclusions from empirical studies in software engineering is
difficult because any process depends to a large degree on a
potentially large number of relevant context variables. There is
not statistical evidence that test driven development brings
about more accurate and precise unit tests than formal
development method, even if subjects who used test driven
development outperformed those who use formal development
method, during all the experimental runs. We are convinced that
test driven development increases such quality aspects and that
evidence might be obtained in a longer experiment, where
differences between the two practices could be more evident.
Future studies should consider the effectiveness of test driven
development at varying levels in the curriculum and the
programmer’s maturity. The studies can also examine how test
driven development compares to test-last methods that fix the
design ahead of time, as well as iterative test last methods that
build an emergent design.
6 REFERENCESES
[1] http://en.wikipedia.org/wiki/Test-
driven_development#cite_ref-Beck_0-1
[2] http://msdn.microsoft.com/en-
us/library/aa730844(VS.80).aspx
[3] Geras M. Smith J. Miller “A Prototype Empirical Evaluation
of Test Driven Development, Proceedings of the Software
Metrics”, 10th International Symposium Pages: 405 - 416
[4] Gerardo Canfora Aniello Cimitile Felix Garcia, “Evaluating
advantages of test driven development: a controlled experiment
with professionals”, international Symposium on Empirical
Software Engineering Proceedings of the 2006 ACM/IEEE
international symposium on Empirical software
engineering SESSION: Test-driven development Pages: 364 -
371
[5] Laurie Williams, E. Michael Maximilien, Mladen Vouk,
“Test-driven development as a defect-reduction practice”,
Software eliability Engineering,2003. ISSRE 2003. 14th
International Symposium Publication Date: 17-20 Nov. 2003 On
page(s): 34- 45
[6] “Evaluating the efficacy of test-driven
development: industrial case studies”, Thirumalesh Bhat,
Nachiappan Nagappan, nternational Symposium on Empirical
Software Engineering
Proceedings of the 2006 ACM/IEEE international symposium
on Empirical software engineering SESSION: Test-driven
development Pages: 356 - 363
[7] George, B. and Williams, L. A structured experiment of
testdriven development. Information and Software Technology
46 (May 2004), pp.337–342.

[8] Test Driven Development: By Example –by Kent Beck