Software Testing Processes and Automation HIT3166 & HIT 8166 Lecture 1

Semester 1, 2011 Basics of Software Testing T. Y. Chen

Faculty of Information and Communication Technologies Swinburne University of Technology

Definitions
Fault
An incorrect definition or process in the software

Errors, Faults and Failures

M. Roper: When developing software, people make errors, these become faults in the software which then manifest themselves as failures when the software is run. One error may lead to several different faults, each of which in turn leads to several different failures.
3 4

Failure
An observable violation against the specifications

Phases of a software life cycle

Requirements analysis Specifications Design Coding Testing Operation Maintenance
5

Validation and Verification

Boehm:
Validation Are we building the right product? Verification Are we building the product right?

Verification
Debugging
Cost of fixing a fault

Locating and fixing the error

Testing
An attempt to reveal errors

Proving
Proving the correctness of the program

Requirement Specification Analysis

Design

Coding

Testing

Operation
7 8

Testing
Proving Complexity

demonstrates the presence of fault does not demonstrate the absence of fault

Testing

Debugging

Quality

10

Main Problems
Reliable Test Set Problem Oracle Problem

Reliable Test Set Problem

How to select good test cases? Different intuitions to develop different test case selection strategies

11

12

Oracle Problem
An oracle is a mechanism to verify against the correctness of the computed output Oracle Problem no oracle or practically infeasible to apply the oracle

Testing Activities
Define the objectives Design the test cases Generate the test cases Execute the test cases Analyze the test results

13

14

Error Analysis
What Where When What was done incorrectly Why was not detected earlier How was the error found
15

Some Guidelines
Analyze and think carefully Errors tend to cluster Not just fix one instance of the error Correction may give rise to new errors Avoid experimentation Explain the problems to others
16

Program Testing Methodologies

Static analysis Dynamic analysis Program Instrumentation Data flow analysis Selection of test cases Mutation analysis Metamorphic Testing Symbolic execution
17

Various Levels of Testing

Unit test (Module test)
Each module is individually tested

Integration test (incremental test)

A set of modules are tested collectively

Systems test (evaluation test)

The entire system is tested

Acceptance test
18

Approaches towards Integration Test

Top-down
Used with the top-down software development Need to define stubs (same interface as modules)

References:
Pfleeger, S. L., Software Engineering, Chapters 7-8. Myers, G., Software Reliability: Principles & Practices, Chapter 2.

Bottom-up
Need to define drivers

Hybrid
19 20