Black box and white box testing compared

The example above indicates that white box testing can indicate test considerations which are not produced by black box testing. The converse is also true, black box testing can produce test considerations which are not produced by white box testing. White box testing is concerned only with testing the software product, it cannot guarantee that the complete specification has been implemented. Black box testing is concerned only with testing the specification, it cannot guarantee that all parts of the implementation have been tested. Thus black box testing is testing against the specification and will discover faults of omission, indicating that part of the specification has not been fulfilled. White box testing is testing against the implementation and will discover faults of commission, indicating that part of the implementation is faulty. In order to fully test a software product both black and white box testing are required. White box testing is much more expensive than black box testing. It requires the source code to be produced before the tests can be planned and is much more laborious in the determination of suitable input data and the determination if the software is or is not correct. The advice given is to start test planning with a black box test approach as soon as the specification is available. White box planning should commence as soon as all black box tests have been successfully passed, with the production of flowgraphs and determination of paths. The paths should then be checked against the black box test plan and any additional required test runs determined and applied. The consequences of test failure at this stage may be very expensive. A failure of a white box test may result in a change which requires all black box testing to be repeated and the re-determination of the white box paths. The cheaper option is to regard the process of testing as one of quality assurance rather than quality control. The intention is that sufficient quality will be put into all previous design and production stages so that it can be expected that testing will confirm that there are very few faults present, quality assurance, rather than testing being relied upon to discover any faults in the software, quality control. A combination of black box and white box test considerations is still not a completely adequate test rationale, additional considerations will be introduced in Chapter 6 of this section.

A white box test rationale for the SquareRoot function

The intention in white box testing is to ensure that all possible feasible flow of control paths through a subprogram are traversed whilst the software is under test. This is not the same as saying that all statements in the subprogram will be executed as it is possible for all statements to be executed but for not all of the possible paths to be traversed. However, the converse is true; if all possible paths through a subprogram are traversed then all statements in the subprogram will necessarily be executed. When considering the number of possible paths through a subprogram two other factors need to be remembered. The first is that some of the possible paths through a subprogram turn out upon investigation to be non-feasible paths. The second consideration is that the number of possible paths through a subprogram indicated by a flowgraph analysis will indicate the minimum number of paths to ensure complete coverage. This may be less than the total number of paths which are possible when combinations of paths are allowed for. Examples of these two considerations will be given after the white box testing of the SquareRoot function has been presented. The number of possible paths through a subprogram is equal to the number of regions in the subprogram's flowgraph. The final flowgraph for the SquareRoot function, with the regions numbered, is as follows.

This indicates that there are five possible paths through the flowgraph which can be described by listing the sequence in which the nodes must are visited. The five paths in the SquareRoot flowgraph are as follows.

The first two paths differ only in the arc which is followed between nodes c and d, to indicate which node is intended the nodes have been labelled 1 and 2 and the paths annotated to indicate which arc is intended. In the subsequent paths the arc traversed between c and d is shown as x, indicating that it does not matter which particular arc is followed. The white box process continues by taking the black box test plan and manually determining which of the paths will be traversed by each of the test runs. It is immediately obvious that with only three test runs in the black box test plan there are not enough test runs to be certain that all five paths will be traversed. A trace of the first test run from the black box test plan will indicate that the second part of the if statement will be followed before the loop is entered and that the value of CurrentError when computed on line 0028 would be 0.0, causing the if condition on line 0036 to be true. Consequently the value of CloseEnough will be set to True and the loop will terminate after the first iteration. Relating this to the flowgraph this produces the path a c d f d e b, which is Path3 on the white box test plan. The second test run will also follow the second path of the selection before entering the loop. The if on line 0036 will evaluate false causing the if on line 0039 to be evaluated. This condition will evaluate false as the current guess will be 0.95 whose square (0.9025) is be greater than Number (0.9). Following the end of both if statements the loop condition will evaluate true again and the loop will iterate for a second time. The path traversed so far would be a c d f g 1 h d and as the algorithm is known to terminate at some stage the path d e b would have to be traversed at some stage. Thus the complete path a c d f g 1 h d e , Path 4 from the white box test plan, will have been known to be traversed during test run 2. The third black box test will differ from the second only in the arc which it traverses between nodes g and h. In this case the square will be larger than Number and the alternative path will be followed. The full path is a c d f g 2 h d e , which is Path 5 from the white box test plan Paths 1 and 2 will now have to be examined to determine if they are feasible. Both of these paths will be followed only if the body of the while loop is never entered. An examination of the source code will indicate that this can never happen. The loop is controlled by the value of the Boolean variable CloseEnough, for the loop body never to execute this variable will have to have the value true when the condition is evaluated for the first time. This is not possible as the value of the variable is set to false when the variable is declared. Paths 1 and 2 are thus examples of possible paths through a flowgraph which, upon investigation, turn out to be a non-feasible path. Unfortunately the total number of feasible possible paths is not always the same as the number of possible paths which are produced from a flowgraph. The following two

flowgraphs and example implementations both have three regions, but differ in the number of paths.

if Condition1 then -- First actions. else -- Second actions. end if; if Condition2 then -- Third actions. elsif -- Fourth actions. end if;

if Condition1 then -- First actions. else if Condition2 then -- Third actions. else -- Fourth actions. end if; end if;

The reasons why the flowgraph on the left has a total of four feasible paths and the one on the right only three, despite the fact that both flowgraphs have the same degree of cylometric complexity are complex and are rooted in a branch of mathematics known as graph theory. Despite this problem the technique of flowgraph analysis and the subsequent counting of regions is still a valuable measure of cognitive complexity, and the number of bias paths which it indicates is the smallest number of distinct feasible paths which should be looked for. As this example indicates, there may also be more possible paths which be checked for when the flowgraph is used as the basis of white box testing.

Source The Internet