OVERVIEW:

FUNDAMENTALS OF SOFTWARE
TESTING
By Dung Nguyen
Date: Sep. 23, 2009
Duration: 4 hours
Course Code: GCS-C119

Contents
Introduction
Concepts: Black Box And White Box Testing
PART 1
TESTING LIFECYCLE IN WATERFALL AND RUP

The Lifecycle Of Testing in Software Development Process

(SDP)
PART 2
OVERVIEW PHASES OF TEST AND TEST TYPES

Concepts: Test Phases and Test Types

PART 3
THE RELATED ACTIVITIES IN PER TEST PHASES

Unit Test Phase

Integration Test Phase
System Test Phase
Acceptance Test Phase
Glossary

Sep. 23, 2002

Confidential

Introduction
The test process defines 4 phases (including Test Types) of
software testing which occur between product coding and product
delivery.
It also covers, in brief, pre-test activities on which the test effort
depends. This document is intended to provide an overview about
Fundamentals of Software Testing.

Sep. 23, 2002

Confidential

CONCEPT: BLACK BOX TESTING

Black box testing:
Not based on any knowledge of internal design or code.
Tests are based on requirements and functionalities.

Sep. 23, 2002

Confidential

CONCEPT: WHITEBOX TESTING

White box testing:
Based on knowledge of the internal logic of an application's
code. Tests are based on coverage of code statements,
branches, paths, and conditions.

Sep. 23, 2002

Confidential

PART 1:
TESTING LIFECYCLE IN WATERFALL AND
RUP

Sep. 23, 2002

Confidential

Testing Lifecycle in Waterfall

Sep. 23, 2002

Confidential

Testing Lifecycle in Iteration

Sep. 23, 2002

Confidential

PART 2
OVERVIEW PHASES OF TEST AND TEST
TYPES

Sep. 23, 2002

Confidential

CONCEPTS: PHASES OF TEST

There are essentially 4 test phases that have to be completed
between coding and product delivery:

1
Unit Test
Phase

Integration
Test Phase

System Test
Phase

4
Acceptance
Test Phase

Each test phase is structured to detect specific categories of

defects
The types of tests that are performed in each test phase

Sep. 23, 2002

Confidential

10

CONCEPTS: PHASES OF TEST (cont.)

Test Phases
Unit
Test Phase
Unit
Test

Responsibility
Project Team

Oversees the entire testing process

Primary Test
Platform
SDE Host

QA/QC/SW Team
Integration Test
Phase:
Software - Module
Software - Subsystem
Software, HardWare Deliverable Product

SDE Host
SDE Host
Target Platform

System Test Phase

QA/QC/SW Team

Target Platform

Acceptance Test
Phase

Acceptance Test Group

Target Platform

Sep. 23, 2002

Confidential

11

Unit Test Phase

1
Unit Test
Phase

Integration
Test Phase

System Test
Phase

4
Acceptance
Test Phase

NOTE
1. Feature testing is
NOT performed in this
phase of testing.
TEST TYPES
2.This phase catches
65% of all bugs- Ross
Collard.
Statement Coverage
3 . Effective unit
Basis Path Coverage
testing
is
the
Help and Error Messages
foundation of all later
phases of testing. No
File Handling
amount of integration
or system testing can
compensate
for
Sep. 23, 2002
Confidentialinadequate
12unit

Integration Test Phase

1
Unit Test
Phase

Integration
Test Phase

System Test
Phase

4
Acceptance
Test Phase

TEST TYPES
Inter-unit/module/subsystem
communication
Functional tests
Timing and synchronization tests
Reliability
Load and Performance

Sep. 23, 2002

Confidential

13

Integration Test Phase (cont.)

NOTE
1. Feature testing is NOT performed in this phase of
testing.
2. Stubs are used to replace components which are not
ready for the initial phase of this testing. Stubs are
replaced by actual components in the latter phase of
this testing.
3. Performance, load, and reliability test types are all
subsystem based that could be performed in this test
phase.
4. Integration testing is hierarchical. Units are
integrated into modules, and modules may be integrated
into other modules or subsystems. Full system
integration is the final stage of integration.

Sep. 23, 2002

Confidential

14

System Test Phase

1
Unit Test
Phase

Integration
Test Phase

System Test
Phase

4
Acceptance
Test Phase

TEST TYPES

Feature

Performance
Load
Reliability

Storage

Capacity
Systems Operations and
Maintenance
Sep. 23, 2002

Confidential

15

Acceptance Test Phase

1
Unit Test
Phase

Integration
Test

System Test
Phase

4
Acceptance
Test Phase

TEST TYPES

End-User tests
Administrator tests
Distributor tests
Support tests

Sep. 23, 2002

Confidential

16

PART 3
THE RELATED ACTIVITIES IN PER TEST
PHASES

Sep. 23, 2002

Confidential

17

UNIT TEST PHASE

Objective of the Unit Test Phase :
Effective unit testing is the foundation of all later phases of
testing. No amount of integration or system testing can
compensate for an inadequate unit test - Boris Beizer
A unit may be regarded as a single callable routine or function
(eg. A function in C). Unit test cases require the unit to be
defined in this manner.
Note:
The functionality provided by the unit is most likely NOT
visible to the end-user of the system

Sep. 23, 2002

Confidential

18

UNIT TEST RELATED ACTIVITIES

Write Detailed Design (includes Unit & Module
Level Integration Test Plans)
Detailed Design Review (includes review of
Unit & Module Level Integration Test Plans)
Code & Write Unit Test Cases
How many Unit
Test-related
activities are
ready?

Code Inspection
Unit Test Execution
Unit Test Report / Unit Handover

Design Group (Author or Programmer)

Sep. 23, 2002

Confidential

19

UNIT TEST PHASE (cont.)

Deliverables to Unit Test
A unit which meets the Unit Test Entrance Criteria
Unit test cases
Unit test case data (input and expected output)

Note:
Anynewormodifiedunitmustundergounittesting.

Sep. 23, 2002

Confidential

20

UNIT TEST PHASE (cont.)

Unit Test Entrance Criteria
The following must occur prior to the start of unit testing:
Review of all specifications (Product, and Feature)
Review of all appropriate design documents (High Level, and
Detailed Level)
Unit must have been code inspected
Reviewed unit test plan available

Sep. 23, 2002

Confidential

21

UNIT TEST PHASE (cont.)

Deliverables from Unit Test
The following items are delivered to the integration test primes:
Unit
Unit test data
Unit test results
Test report

Sep. 23, 2002

Confidential

22

UNIT TEST PHASE (cont.)

Unit Test Exit Criteria
A unit must successfully pass unit testing (100% unit test
case execution on complete inspected code with 100% of all
identified defects resolved)

Note:
Defect: A defect may be identified in the code, specifications, test
cases,ortestplans.

Sep. 23, 2002

Confidential

23

UNIT TEST PHASE (cont.)

Responsibility for the Unit Test Phase
The author or designer from the author s design group are
responsible for ensuring that this test phase is carried out

Sep. 23, 2002

Confidential

24

INTEGRATION TEST PHASE

Objective of the Integration Test Phase:
Integration testing tests modules, subsystems, and finally
the full- system to ensure distinct components of the
application still work in accordance to customer
requirements.

Note:
Thetestiteminintegrationtestingiseitheramodule,subassembly,
subsystem,orfull-system.

Sep. 23, 2002

Confidential

25

INTEGRATION TEST RELATED ACTIVITIES

Write High Level Design
High Level Design Review
Write Integration Test Plan (subsystem
and full-system) & Test Cases
How many
Integration Testrelated activities
are ready?

Integration Test Plan (subsystem and

full-system) & Test Case Review
Integration Test Execution (SW & HW)
Integration Test Report / Integrated System
Handover (System Test Acceptance
Meeting)

Design / System Test Groups

Sep. 23, 2002

Confidential

26

INTEGRATION TEST PHASE (cont.)

Integration Test Coverage
Focus on uncovering interfacing problems (e.g. Interface
problems between units within a module)
Inter-unit/module/subsystem communication
Functional tests
Timing and synchronization tests
Reliability
Load and Performance

Note:
Performance,load,andreliabilitytesttypesmaybeperformedinthe
integrationtestphaseifitisappropriatetotheprimaryfunctioningofthe
testitemorindicativeoffullsystemperformance.

Sep. 23, 2002

Confidential

27

INTEGRATION TEST PHASE (cont.)

Deliverables to Integration Test
The following items must be delivered prior to integration
testing:
Units which have successfully passed unit testing
Unit test report
Integration test cases
Integration test data
Target hardware platform (before full system integration
testing)

Sep. 23, 2002

Confidential

28

INTEGRATION TEST PHASE (cont.)

Integration Entrance Criteria
The following must occur prior to the start of integration
testing:
Before a unit may enter into integration testing, it must have
successfully completed unit testing
All defects identified in unit test must have been resolved
Unit test results must be provided for inspection
The reviewed integration test plan and test cases must also
be available

Sep. 23, 2002

Confidential

29

INTEGRATION TEST PHASE (cont.)

Deliverables from Integration Test
The following items are delivered to System Test:
Release notes
Integration test report
Integration Test Exit Criteria
Integration testing must be fully integrated on the target
hardware platform, and successfully complete before it
meets the integration test exit criteria (100% integration test
case execution, with 100% of all identified defects resolved )
Test results must be made available to System Test for
inspection
Note:

Defect:Adefectmaybeidentifiedinthecode,specifications,test
cases,ortestplans.

Sep. 23, 2002

Confidential

30

INTEGRATION TEST PHASE (cont.)

Responsibility for the Integration Test Phase
Designers and/or design primes are responsible for low level
integration testing. Design primes and the System Test
Group are responsible for higher level integration testing and
full system integration testing. In some instances, the
System Test Group may be augmented by a designated set
of designers

Sep. 23, 2002

Confidential

31

SYSTEM TEST PHASE

Objective of the System Test Phase
In this test phase, product features are fully tested as well
as system-wide capabilities as a production-like. The tests
are focused on ensuring that the system operates in conformity
with the product and feature specifications.

Sep. 23, 2002

Confidential

32

SYSTEM TEST RELATED ACTIVITIES

Participate in reviews of Product and
Feature Specifications, and High Level
Design Documents
Write System Test Plan & Test Cases

System Test Plan & Test Case Review

How many
System Test
related activities
are ready?

System Test Report / System Handover

(Verification Acceptance Meeting) Optional

System Test Execution

System Test Group

Sep. 23, 2002

Confidential

33

SYSTEM TEST PHASE (cont.)

System Test Coverage
All system features (feature testing)
Feature interactions
System performance
Capacity limits
Load and overload
Reliability
Recovery
Security
Installation
Conversion, and expansion.

Sep. 23, 2002

Confidential

34

SYSTEM TEST PHASE (cont.)

Deliverables to System Test
The following items must be delivered prior to system testing:
Release notes
Integration test report
Minimum hardware required
Test tools as identified in the system test plan.

Sep. 23, 2002

Confidential

35

SYSTEM TEST PHASE (cont.)

System Test Entrance Criteria
Test results/reports must be provided to System Test for
inspection

Sep. 23, 2002

Confidential

36

SYSTEM TEST PHASE (cont.)

Deliverables from System Test
Problem reports (PR s) if and when any problem is discovered
during testing which have not been reported before
At the conclusion of the system test phase, the distribution
media containing the successfully tested software will be
delivered to Acceptance
System test report and test results
System Test will also issue a report on the post-mortem of the
test effort for this test phase

Sep. 23, 2002

Confidential

37

SYSTEM TEST PHASE (cont.)

System Test Exit Criteria
System testing must successfully complete before it meets
the system test exit criteria (100% system test case
execution with the product quality requirements)
The system test report and test results must be made
available to the Acceptance Test group for inspection

Responsibility for the System Test Phase

The System Test Group is responsible for ensuring that this
test phase is carried out. In some instances, the System Test
Group may be augmented by a designated set of designers

Sep. 23, 2002

Confidential

38

ACCEPTANCE TEST PHASE

Objective of the Acceptance Test Phase
Acceptance testing is focused upon determining the products
readiness to enter the marketplace. Acceptance test will simulate
typical customer scenarios in order to ensure that the product
operates acceptably in that environment. Testing in this phase is
not expected to uncover any significant technical problems.

Sep. 23, 2002

Confidential

39

ACCEPTANCE TEST RELATED ACTIVITIES

Participate in reviews of Commercial,
Product and Feature Specifications

Write Acceptance Test Plan & Test Cases

Acceptance Test Plan & Test Case Review

How many
Acceptance Test
related activities
are ready?

Acceptance Test Execution

Acceptance Test Report / Verified

System Handover (to Field Trial)
Acceptance Test Groups

Sep. 23, 2002

Confidential

40

ACCEPTANCE TEST PHASE (cont.)

Acceptance Test Coverage
There are essentially four customer perspectives which provide
the direction for Product Verification:
End-user
Administrator
Distributor
Support (Product Support)

Test types:
Installation
Conversion
Upgrade
Modification
Network
Administration

Sep. 23, 2002

Confidential

41

ACCEPTANCE TEST PHASE (cont.)

Acceptance Test Coverage(cont.)
Compatibility
Support
Scenario
Documentation
On-line Help

Sep. 23, 2002

Confidential

42

ACCEPTANCE TEST PHASE (cont.)

Deliverables to Acceptance test
The following items must be delivered prior to acceptance
testing :
Distribution media containing software to be tested
Release notes providing all appropriate information on the
software release
Minimum hardware as required for the product under test
All test tools identified in the acceptance test plan
System test results/report

Sep. 23, 2002

Confidential

43

ACCEPTANCE TEST PHASE (cont.)

Acceptance Entrance Criteria
These items must be made available at the acceptance
meeting:
Product (hardware, firmware, software, and documentation)
quality must meet or exceed criteria
Workarounds and/or description must be available for all
open PR s
All system test cases must have been executed by the
System Test Group

Sep. 23, 2002

Confidential

44

ACCEPTANCE TEST PHASE (cont.)

Acceptance Entrance Criteria (cont.)
All system and designer test cases must be available for
audit by Acceptance
Test results for each test must be provided by development
for inspection. In the event of a fail, a PR number must also
be provided
Reviewed acceptance test plan must be available. All
acceptance test activity is provided in the test plan. The final
test plan and test cases are made available
Reviewed acceptance test cases must be available

Sep. 23, 2002

Confidential

45

ACCEPTANCE TEST PHASE (cont.)

Deliverables from Acceptance
Problem reports (PR s) if and when any problem is
discovered during testing which have not been reported
before
Acceptance test report and test results
Acceptance will also issue a report on the post-mortem of
the test effort for this test phase

Sep. 23, 2002

Confidential

46

ACCEPTANCE TEST PHASE (cont.)

Acceptance Exit Criteria
Acceptance will recommend a product that it tests if it has
quality and meets or exceeds customers requirements and
expectation based on the Product specifications

Responsibility for the Acceptance Test Phase

The Acceptance Group is responsible for ensuring that this
test phase is carried out

Sep. 23, 2002

Confidential

47

GLOSSARY
See the Glossary - attached file.

Sep. 23, 2002

Confidential

48

REFERENCES
Referent Documents:
Software Testing and Quality Assurance White Papers
Pointe Technology Group, Inc.
Toronto Lab Software Test Process
Northern Telecom
Rational Unified Process
Rational Software Corp.

Sep. 23, 2002

Confidential

49

Question/ answer

Sep. 23, 2002

Confidential

50

THANK YOU

Sep. 23, 2002

Confidential

51