Application Test Process in Product Life Cycle

Oner Tekin

Gulsah Bayram Cetin

Technology Leader
Netas
Istanbul, Turkey
oner@netas.com.tr

R&D ICT Test and Automation

Netas
Istanbul, Turkey
gulsah@netas.com.tr

AbstractThis article explains the phases of application

testing process in product life cycle and methodologies of software
testing. Software testing is a series of processes begin with
requirements step in the early phases of product life cycle and
there are many different types of testing methods or techniques
used as part of the software testing methodology. The
methodology of software deployment chosen depends on the
nature of project. Most software development projects involve
periodic testing, but some methodologies focus on getting the
input from testing earlier in the product life cycle.
Index TermsApplication Test Process, Test Automation

I. INTRODUCTON
Software testing is a series of processes begin with
requirements step in the early phases of product life cycle and a
part of the Software Quality Assurance (SQA) process. Various
stages of testing that occur throughout a project are illustrated
in Fig. 1. These stages are:
x
x
x
x
x
x
x
x
x
x

Requirements
Test Strategy documentation and reviews
Test Plan documentation and reviews
Test Case documentation and reviews
Test development for Software Test Automation
applicable projects
Test execution
Result and Defect Reporting
Final Assessment Testing
Trial Preparation Testing
Customer Acceptance Preparation Testing

The software testing techniques could be implemented in

two ways - manually or by automation. There are many
software testing techniques used as part of the software testing
methodology:
x
x
x
x
x
x
x

Unit Testing
Integration Testing
Feature Verification
Product Integration Testing
Product Verification
Solution Verification
Regression Testing

x
x
x
x
x

Performance Testing
Recovery Testing
Security Testing
Conformance Testing
Compatibility Testing

There are numerous methodologies available for developing

and testing software. One of the common methodologies is
Waterfall Model. This model is a sequential development
process which allows dividing life cycle into various phases.
Iterative models break the cycle into parts and "Waterfall
Model" is applied to each part. In "V Model", development and
testing takes place at the same time with the same kind of
information.
II. APPLICATION TESTING PROCESS
This section explains, stages of testing that occur throughout
a project regardless of the testing methodology. Product Life
Cycle is a procedural process and software test teams involve at
the customer requirements step of software development
process.
A. Software Requirements Specification
Software testing is a series of processes begin with
requirements step in the early phases of product life cycle. Test
architects and/or specialists should take part in requirements
stage. Timeline, milestones of sub processes of test cycle,
hardware/software requirements for testing, and resources
should be identified at this phase. All requirements should be
documented properly for further use in Software Requirements
Specification.
B. Test Strategy Documentation and Reviews
Test Strategy describes the high level planning which
identifies the resources, equipment, tools, test activities and
internal milestones required to meet the program objectives.
The strategy defines the holistic product or solution view of the
system verification work in a given release. Strategy document
is reviewed by Product Line Management, Development,
Product Support teams, and Test Managers/Architects.

978-1-4673-1740-5 /12/$31.00 2012 IEEE

F. Final Assessment Testing

Defines the work required to perform the final assessment
of a product or solution by identifying and executing select test
suites upon receipt of the Final Compile loads.
G. Trial Preparation Testing
Focuses on outlining the work requirements associated with
product or solution preparation for customer specific trials. It
also addresses trial specific planning, testing and tracking.
H. Customer Acceptance Preparation Testing
Defines the workflow associated with the introduction of
customer acceptance criteria testing.
III. SOFTWARE TEST MODELS
A. Unit Testing
The primary goal of unit testing is to take, individual
component within a subsystem, and isolate it from the
remaining components of the code, and determine whether it
behaves exactly as you expect. Each component/unit is tested
separately before integrating them into modules to test the
interfaces between modules. Development teams are
responsible for unit testing and it is an example of White-Box
testing. White-Box testing assumes that the test specialist could
follow the related part of the code, and create the test cases for
the potential failures.
The number of defects found in the development is high and
the cost to fix those defects is relatively low when comparing to
further stage of product life cycle.

Fig. 1. Test Process

C. Test Plan and Test Case Documentation and Reviews

Test Plan describes the low level planning which identifies
the detailed resources, equipment, tools, and test activities, etc.
Detailed test scenarios are defined in Test Case document.
D. Test Development for Software Test Automation
In todays software projects, test automation becomes an
increasingly critical and strategic necessity. Automation Scripts
are developed in this phase and updated during execution as
necessary.
E. Test Execution, Result and Defect Reporting
Test Cases which have been developed in the Test Activity
Planning phase are executed on the target system and/or
product. The strategy addresses the mechanics of issue tracking
as it interacts with other processes as well as the test execution
metrics collection and reporting. Defects are not caused only
due to coding errors, but most commonly due to the
requirement gaps in the non-functional requirement. A failure is
caused due to the deviation between an actual and an expected
result.

B. Integration Testing
Next step is to test the interoperability of the different
components of a subsystem which is known as Integration
Testing. Software Integration Testing covers off network
element specific stress testing, regression testing, and search
and destroy. Also ensuring that network element interfaces are
working correctly, and design teams are responsible for
software integration.
Feature Integration Testing verifies the basic feature
operation following integration into the release build or
integration stream. Tests verify that the feature interactions
with other architecture components or activities. Design and/or
test teams are responsible for feature integration tests.
There are three common approaches of implementing
integration tests:
1) Top-Down Approach
High level units are tested and combined first. This enables
early testing of high-level logic.
2) Buttom-Up Approach
In this approach, lower level units are tested and combined
first.
3) Umbrella Approach
This involves testing the control flow paths and the
functional data. Initially, the inputs for the different functions
are combined using the bottom-up approach. The outputs are
then
combined
utilizing
the
top-down
approach.

C. Feature Verification
Feature Verification provides a complete verification of
feature operation in a working system for both success path and
failure path against the requirements in the Feature
Requirements Spec and Feature Technical Spec. Design and/or
test teams are responsible for feature verification testing. This
type of testing is an example of Black-Box testing. Black-Box
testing assumes that the test specialist creates and implements
the test cases without any internal knowledge of the unit under
test. This approach tests all possible combinations of end-user.
D. Product Integration Testing
Product Integration is a regression testing of earlier release
features to ensure components and interfaces are still intact
amongst network elements. Design and/or test teams are
responsible for product integration testing.
E. Product Verification
At the phase of product verification, the main goal is to
ensure the system as a whole including all new features can
respond favorably to robustness scenarios, capacity stress tests,
reliability testing, extended soaking, end to end upgrade and
rollbacks and deployment scenarios. Test teams are responsible
for the product verification.

K. Conformance Testing
Conformance determines whether a product meets the
international and industrial standards and interoperates
properly.
L. Compatibility Testing
Compatibility testing determines whether a product
compatible with other elements of the system or environment.
Software test automation is applicable to compatibility testing.
IV. SOFTWARE TESTING METHODOLOGIES
The methodology of software deployment chosen depends
on the nature of project. Most software development projects
involve periodic testing, but some methodologies focus on
getting the input from testing earlier in the product life cycle.
A. Waterfall Model
One of the common methodologies is Waterfall Model
which is a non-iterative approach. This model is a sequential
development process which allows dividing life cycle into
various phases. Waterfall Model is shown in Fig. 2.

F. Solution Verification
Solution verification verifies specific end-to-end
functionality and performance of the solution in a customer
simulated environment, where realistic conditions are critical to
the tests, and test teams are responsible for the solution
verification.
G. Regression Testing
Regression testing verifies the existing behavior of the
software is unbroken after new implementations. Regression
testing is a good application area for software test automation.
H. Performance Testing
Performance testing verifies that a system performs in terms
of responsiveness and stability under a particular workload and
meets the specifications claimed by its manufacturer or vendor.
Software test automation is applicable to performance testing.
I. Recovery Testing
Recovery testing verifies that how a system or application is
able to recover from hardware and software crashes and
failures. This type of testing determines the ability to restart the
applications after integrity lost. The main focus is to ensure that
the functionality continuous after disaster cases.
J. Securtiy Testing
Security testing determines whether a product protects data
and maintains functionality of the system properly. Main focus
is to protect confidentiality. Vulnerability scanning finds
weakness of the system. Penetration testing simulates the
malicious attacks to the system.

Fig. 2. Waterfall Model

B. V Model
In this approach, both development and test teams drive the
process in parallel (It could be visualized forming the letter 'V').
Development and testing takes place at the same time with the
same kind of information. Fig. 3, represents the V model.
C. Spiral Model
In the spiral development environment, software testing is
again described as a continuous improvement process that must
be integrated into a rapid application development
methodology. Testing as an integrated function prevents
development from proceeding without testing. There are a
number of cycles of all the sequential steps of the waterfall
model. Once the initial cycle gets completed, a thorough
analysis and review of the achieved product or output is
performed. Fig. 4, represents the Spiral Model.
D. Agile Model
Agile is a hybrid method which mixes sequential and
iterative approaches. The main focus is to create quick,
practical and visible outputs. This is a practice-based
methodology. Agile development is successful when software
test automation is implemented properly. This model relies on
the repetition of a very short development cycle.

User
Acceptance
Testing

Requirements
Analysis

System
Design

Functional
Testing

Integration
Testing

Architectural
Design

Unit Testing

Modal Design

1.Determine
Objectives

2.Identify
and Resolve
Risk

4.Plan the
Next
Iteration

3.Develop
and Test

Fig. 4. Spiral Model

Figure 5 determines Agile methodology. Agile principles

are:
x
x
x
x
x
x
x

Continuous Delivery
Quick handling of changing requests
Quick software delivery
Involvement of business units and developers
throughout the project
Self-organized teams
Good communication between the business units
Technical excellence and good design

There are different methods of Agile software development

methodology.
1) Scrum
The basic idea of scrum is, there are several technical and
environmental variables changing during the process. These
variables are requirements, resource, time, and technology.
Scrum can help organizations to achieve better engineering
activities, as it has some frequent activities for management to
check the system regularly for any kind of deficiencies.

Design
Analyse

Build

Plan

Coding

Test

Fig. 5. Agile Model

Fig. 3. V Model

Deploy

2) Extreme Programming
Extreme Programming is a collection of different models.
The main characteristics are, short iterations with rapid
feedback, and small releases, regular participation of the
customer, continuous testing and integration, collective code
ownership, not a detailed documentation.
3) Feature Driven Development
Feature Driven Development (FDD) is a client-centric,
architecture-centric software process. In FDD, testing is done
through inspection. In FDD process, unit tests are taken and
implemented according to the project and situation.
4) Crystal Clear Method
Crystal Clear method is used for small teams. It attempts to
describe a full methodology of the lightest, most habitable kind
that will produce good results.
V. SOFTWARE TEST AUTOMATION
Software test automation is a process which uses software
for testing a software product. Automation is a kind of
development process in this manner. If there are repeatable
actions in test scenarios, software test automation is an
effective way to implement these test cases.
It is critical to describe the automation test objectives for
the first step. Test architects and/or test automation experts are
responsible for designing the test framework and architecture.
Test specialists must then comply with and contribute to the
architecture and design automation scripts. Parameterization of
scripts is one of the important steps for software test
automation.
While the development is in progress, as soon as the
functional description is completed, test plan and detailed test
cases are created. Test libraries for common functions and test
scripts for base-unrelieved applications could be created, at this
point but this is an early phase to create all test scripts. Test
architects should start to build test framework. It is an
important phase to identify automated, semi-automated, and
manual test cases. When the development completed,
automation team could start to create test scripts for new
activities. These scripts will be updated when the functionality
change. Test automation maintenance is an important issue,
and automation teams should handle any changes in related
product. Automation provides many advantages in software
testing:
A. Speed
Testing process is expedited, since a program naturally
works quicker than the pace of a human tester. Many
automated testing tools can replicate the activity of a large
number of users using a single computer. Therefore, load/stress
testing using automated methods require only a fraction of the
computer hardware that would be necessary to complete a
manual test. Parallel execution reduces the execution effort and
time.
B. Cost Effectiveness
The cost of performing manual testing is prohibitive when
compared to automated methods. Automation is not time

dependent, so the scheduling methods allow us to execute the

automated test suites any time.
C. Reliability
In automation, human factor is virtually eliminated, giving
less consideration for subjectivity and possible margins of
error. Full-featured automated testing systems also produce
convenient test reporting and analysis. These reports provide a
standardized measure of test status and results, thus allowing
more accurate interpretation of testing outcomes. Manual
methods require the user to self-document test procedures and
test results.
D. Reusability
Automated tests are reusable on different versions of an
application.
E. Comprehensive
A suite of tests that covers every feature in the application
could be build. The productivity gains delivered by automated
testing allow and encourage organization to test more often and
more completely. Greater application test coverage also
reduces the risk if exposing users to malfunctioning or noncompliant software.

Fig. 6. Test Automation Framework

F. Repeatable Testing
Automation allows the testing organization to perform
consistent and repeatable test. When applications need to be
deployed across different hardware or software platforms, tests

can be created and repeated on target platforms to ensure that

new platforms operate consistently.
The methods that are employed to carry out testing remain
repetitious throughout the development life cycle.
G. Programmable Testing
In automation projects, more sophisticated test suites that
bring out hidden information from the application could be
created.
H. Good Result Analysis and Resulting
Automated test results could be tracked easily, and all
iterations of execution are reported.
VI. CONCLUSION
Successful software testing projects depends on a standard
testing process, good testing objectives and the right team
roles, skills and tools. The automation test team needs to have
knowledge of testing, programming, and automation tool as
well. In todays system, rapid deployments and quick
responses to the customers are essential. Software development
and test processes during a product life cycle are adapted to
these requirements.

REFERENCES
[1] Koomen, T. and M. Pol, Test Process Improvement: A practical
step-by-step guide to structured testing, Addison-Wesley, 1999
[2] Copeland, L., A Practitioners Guide to Software Test Design,
STQE Publishing, 2004
[3] Graham, D. and M. Fewster, Experiences of Test Automation:
Case Studies of Software Test Automation, Pearson Education,
2012
[4] Hetzel, B., The Complete Guide to Software Testing, Wiley,
1993
[5] Galin, D., Software Quality Assurance: From Theory to
Implementation, Addison-Wesley, 2003
[6] Dustin, E., Rashkal J. and J. Paul, Automated Software Testing:
Introduction, Management, and Performance: Introduction,
Management, and Performance, Addison-Wesley Professional,
1999
[7] Venkatasubramanian, A. and V. Vinoline, Software Test Factory
(A proposal of a process model to create a Test Factory),
International Journal of Computational Intelligence Techniques,
vol. 1, no.1 2010, pp.14-19.
[8] Boehm, B., "A Spiral Model of Software Development and
Enhancement," Computer, vol.21, no.5 1988, pp.61-72.
[9] Nawaz, A and M. Masood, Software Testing Process In Agile
Development, Master Thesis, Dept. of Computer Science,
Blekinge
Tekniska
Hogskola,
2008.