Prepared by : 1.Mohammed Siddig Ahmed. 2.Mohammed Zeinelabdeen. 3.Omer Salih Dawood.

Introduction to Testing. Verification and validation. Black & White-box testing. Types of testing that involve both black & white-box techniques. Strategies for writing test cases. Using a template for writing repeatable, defined test cases.

Software testing is the process of analyzing a software item to detect the differences between existing and required conditions (that is, bugs) and to evaluate the features of the software item. Software testing is an activity that should be done throughout the whole development process . Software testing is one of the V&V, software practices.

Verification is the process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. Validation is the process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements. Verification: Are we building the product right? Validation : Are we building the right product?
4

 Standard or specification to measure or

identify correct results from incorrect results must define some terms:
Mistake . Fault [or Defect] . Failure . Error . Specification.
5

 In software development, there are costs

associated with testing our programs. Quality is much more important for safety- or mission critical software e.g. aviation software. Goal of testing covering many defects as possible with a little testing.
6

There are two basic classes of software testing definitions below: Black box testing (also called functional testing) is testing that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution conditions. White box testing (also called structural testing and glass box testing) is testing that takes into account the internal mechanism of a system or component.
7

1. Unit Testing
Opacity: White box testing. Specification: Low-level design and/or code structure. (Unit testing is the testing of individual hardware or software units or groups of related units).

2. Integration testing
Opacity: Black- and white-box testing. Specification: Low- and high-level design. (Integration test is testing in which software components, hardware components, or both are combined and tested to evaluate the interaction between them).
8

3. Functional and system testing

Opacity: Black-box testing
Specification: high-level design, requirements specification. ( Functional testing involves ensuring that the functionality specified in the requirement specification works,& System testing is testing conducted on a complete, integrated system to evaluate the system compliance with its specified requirements). Stress testing Performance testing Usability testing
9

4. Acceptance testing
Opacity: Black-box testing
Specification: requirements specification. (Acceptance testing is formal testing conducted to determine whether or not a system satisfies its acceptance criteria (the criteria the system must satisfy to be accepted by a customer) and to enable the customer to determine whether or not to accept the system).

10

5. Regression testing
Opacity: Black- and white-box testing
Specification: Any changed documentation, high-level design (Regression testing is selective retesting of a

system or component to verify that modifications have not caused unintended effects and that the system or component still complies with its specified Requirements).
A subset of the regression test cases can be set aside as smoke tests.

11

6. Beta testing
Opacity: Black-box testing. Specification: None.

The advantages of beta test:

Identification of unexpected errors A wider population search for errors Low costs Lack of systematic testing Low quality error reports Much effort is necessary to examine error reports
12

The disadvantages

13

( Document describing the scope, approach, resources, and schedule of intended test activities. It identifies test items, the features to be tested, the testing tasks, who will do each task, and any risks requiring contingency plans).

Test case ( set of test inputs, execution conditions, and expected results developed for a particular objective, such as to exercise a particular program path or to verify compliance with a specific requirement).
14

The earlier testing is planned at all levels, the better. Very important to consider test planning and test execution as iterative processes. It is best to begin to write functional and system test cases. When requirements change, revise the test cases. When code changes, run the test cases again. All in all, testing should be considered an iterative and essential part of the entire development process.
15

16

17

Black-Box Testing

Called :
functional testing behavioral testing

focuses on:
whether or not a program does what it is supposed to do based on its functional requirements.

19

1.

2.
3.

4.
5.

incorrect or missing functionality. interface errors . errors in data structures used by interfaces. behavior or performance errors. initialization and termination errors.

But: no amount of testing can demonstrate the absence of errors.

20

blackbox tester is not the programmer of the code (it is best). Programmer:

If the program does what they programmed it to do?.

Needed:
If The program does what the customer wants it to do?.

21

Focus on input and output of the software without regard to the internal code of the program

22

1.

2.

3.

4.

test case a unique identifier. describe the set of steps and/or input for the particular condition. the expected results for an input/output oracle. the actual results.

23

24

Why test Strategies?: test every possible thing (Cost). find many defects in few test cases . avoid writing redundant test cases. design the simplest test cases .

25

1.

2.
3. 4. 5.

Tests of Customer Requirements. Equivalence Partitioning. Boundary Value Analysis. Decision Table Testing. Failure (Dirty) Test Cases.

26

Black box test cases are based on customer requirements Have two paths:

success path. failure path.

risky requirements should tested first.

27

dont want to write several test cases that test the same aspect of our program(cost). A good test case uncovers a different class of errors. Equivalence partitioning is used to reduce the number of test cases . divides the input of a program into classes

28

For each of these classes : the set of data should be treated the same by the module under test . And should produce the same answer.

29

 Bugs lurk in corners and congregate at boundaries Boris Beizer need to focus testing at the boundaries. Boundary value:

a data value that corresponds to a minimum or maximum input, internal, or output value specified for a system or component.

30

When creating BVA test cases, consider the following :

If input conditions have a range from a to b (such as a=100 to b=300), create test cases: immediately below a (99) at a (100) immediately above a (101) immediately below b (299) at b (300) immediately above b (301)
31

If input conditions specify a number of values that are allowed, test these limits.

32

record complex business rules that must be implemented in the program, and therefore tested. conditions represent possible input. actions are the events that should trigger. Each column in the table is a unique combination of input conditions that result in triggering the action(s) associated with the rule.
33

34

every possible thing a user could possibly do with your system to demolish the software.

35

White-Box Testing

White-box testing is a verification technique software engineers can use to examine if their code works as expected. White-box testing is testing that takes into account the internal mechanism of a system or component . It also known as structural testing, clear box testing, and glass box testing (Beizer, 1995).

types of testing: o There are six basic types of testing: 1. Unit Testing. 2. Integration Testing. 3. Regression Testing. 4. function/system Testing. 5. Acceptance Testing. 6. Beta Testing. White-box testing is used for 1,2,and 3 respectively.

white-box testing techniques :

1. 2. 3. 4. 5. 6. White-Box Testing by Stubs and Drivers. Deriving Test Cases. Control-flow/Coverage Testing. Data Flow Testing. Failure (Dirty) Test Cases. Flow Graphs Revisited.

With white-box testing, you must run the code with predetermined input and check to make sure that the code produces predetermined outputs. Often programmers write stubs and drivers for white-box testing. A driver: is a software module used to invoke a module under test and, often, provide test inputs, control and monitor execution, and report test results A stub: is a computer program statement substituting for the body of a software module that is or will be defined elsewhere.

A driver: movePlayer(Player1, 2); A stub: public void movePlayer(Player player, int diceValue) { player.setPosition(1); }

statistics show that the act of careful, complete, systematic test design will catch as many bugs as the act of testing . The test design process, at all levels, is at least as effective at catching bugs as is running the test case designed by that process.

Basis Path Testing : A testing mechanism proposed by McCabe. is a means for ensuring that all independent paths through a code module have been tested. An independent path is any path through the code that introduces at least one new set of processing statements or a new condition.

cyclomatic number : Number of conditions (diamonds) and add 1.

1-2-3-4-5-10 (property owned by others, no money for rent) 1-2-3-4-6-10 (property owned by others, pay rent) 1-2-3-10 (property owned by the player) 1-2-7-10 (property available, dont have enough money) 1-2-7-8-10 (property available, have money, dont want to buy it) 1-2-7-8-9-10 (property available, have money, and buy it)

Equivalence Partitioning/Boundary Value Analysis :

Equivalence partitioning (EP) and boundary value analysis (BVA) provide a strategy for writing white-box test cases. Undoubtedly, whenever you encounter any kind of number or limit in a requirement, you should be alert for EP/BVA issues. then we will check the object by giving boundary values as :min,min1,min+1,max,max-1,max+1

Equivalence Partitioning/Boundary Value Analysis : 1. property costs $100, have $200 (equivalence class have enough money). 2. property costs $100, have $50 (equivalence class, dont have enough money). 3. property costs $100, have $100 (boundary value). 4. property costs $100, have $99 (boundary value). 5. property costs $100, have $101 (boundary value).

Coverage: is a measure of the completeness of the set of test cases

Statement Coverage Decision/Branch Coverage Condition Coverage Decision/Condition Coverage Path Coverage

 If ((A>1) & (B=0)) then Do; X=X/A; END; If ((A==2) | (X>1)) then Do; X=X+1; END; END;
Source: The Art of Software Testing Glenford Myers

Statement Coverage
Exercise all statements at least once How many test cases?
A=2 and B=0 (ace)

Source: The Art of Software Testing Glenford Myers

Decision/Branch Coverage
Each decision has a true and a false outcome at least once How many test cases?
A=2 and B=0 (ace) A=1 and X=1 (abd)

Source: The Art of Software Testing Glenford Myers

Condition Coverage
Each condition in a decision takes on all possible outcomes at least once Conditions: A>1, B=0, A=2, X>1 How many test cases?
A=2, B=0, and X=4 (ace) A=1, B=1, and X=1 (abd)

Source: The Art of Software Testing Glenford Myers

Decision/Condition Coverage
Each condition in a decision takes on all possible outcomes at least once, and each decision takes on all possible outcomes at least once How many test cases?
A=2, B=0, and X=4 (ace) A=1, B=1, and X=1 (abd)

What about these?

A=1, B=0, and X=3 A=2, B=1, and X=1
(abe) (abe)

Multiple Condition Coverage

Exercise all possible combinations of condition outcomes in each decision Conditions:
A>1, B=0 A>1, B<>0 A<=1, B=0 A<=1, B<>0 A=2, X>1 A=2, X<=1 A<>2, X>1 A<>2, X<=1

Source: The Art of Software Testing Glenford Myers

Multiple Condition Coverage

How many test cases?
A=2, B=0, X=4 A=2, B=1, X=1 A=1, B=0, X=2 A=1, B=1, X=1
(ace) (abe) (abe) (abd)

Source: The Art of Software Testing Glenford Myers

Path Coverage
Every unique path through the program is executed at least once How many test cases?
A=2, B=0, X=4 (ace) A=2, B=1, X=1 (abe) A=3, B=0, X=1 (acd) A=1, B=1, X=1 (abd)

Source: The Art of Software Testing Glenford Myers

the control flowgraph is annotated with information about how the program variables are defined and used.

Look at the structure of your code and think about every possible way a user might break it. Some suggestions follow: - Look at every input into the code you are testing. - Look at every calculation

When you encounter a compound predicate, such as in, you must break the expression up so that each Boolean subexpression is evaluated on its own.

Why Do Both?
Black-box Impossible to write a test case for every possible set of inputs and outputs Some of the code may not be reachable without extraordinary measures Specifications are not always complete White-box Does not address the question of whether or not the program matches the specification Does not tell you if all of the functionality has been implemented Does not discover missing program logic

References : Cem Kaner, Testing Computer Software. http://www.win.tue.nl/~mousavi/testing/7.pdf . www.sisqa.com/Training/White_Box_Testing200709.ppt

Questions ?

64