IEEE 829 Documentation

Over the years a number of types of document have been invented to allow for the
control of testing. They apply to software testing of all kinds from component testing
through to release testing. Every organisation develops these documents themselves
and gives them different names, and in some cases confuses their purpose. To provide
a common set of standardised documents the IEEE developed the 829 Standard for
Software Test Documentation for any type of software testing, including User
Acceptance Testing.
This article outlines each of the types of document in this standard and describes how
they work together.

The Types of Document

There are eight document types in the IEEE 829 standard, which can be used in three
distinct phases of software testing:

1. Preparation Of Tests

Test Plan: Plan how the testing will proceed.

Test Design Specification: Decide what needs to be tested.
Test Case Specification: Create the tests to be run.
Test Procedure: Describe how the tests are run.
Test Item Transmittal Report: Specify the items released for testing.

2. Running The Tests

Test Log: Record the details of tests in time order.

Test Incident Report: Record details of events that need to be investigated.

3. Completion of Testing

Test Summary Report: Summarise and evaluate tests.

Documentation For Preparation Of Tests

The preparation for testing is the most important part of any software testing project and
easily accounts for most of the paper work. The purpose this stage is to prepare an
effective and efficient set of tests, and create the environment for them to run in.

IEEE 829 - Test Plan

The Test Plan is the pivotal document around which all the software testing projects
revolve. It describes:

what has to be done,

to what quality standard,
with what resource,
to what time scale,
and outlines the risks and how they would be overcome.

More details about this document are in the Test Plan overview article. If you wish to
see a more detailed version of a test plan then a UAT Plan Template which uses the
standard can be bought from this site.

IEEE 829 - Test Design Specification

Creating the test design is the first stage in developing the tests for a software testing
project. It records what needs to be tested, and is derived from the documents that
come into the testing stage, such as requirements and designs. It records which
features of a test item are to be tested, and how a successful test of these features
would be recognized.
As an example lets use a Billing project from which the following testing requirements
may be defined:

A normal bill can be produced.

A final bill can be produced.
The volume discount is properly calculated.

The test design does not record the values to be entered for a test, but describes the
requirements for defining those values.
This document is very valuable, but is often missing on many projects. The reason is
that people start writing test cases before they have decided what they are going to test.

IEEE 829 - Test Case Specification

The test cases are produced when the test design is completed. Test cases specify for
each testing requirement:

The exact input values that will be input and the values of any standing data that is
required,

The exact output values and changes of value of the internal system state that are
expected,
And any special steps for setting up the tests.

Defining the expected values is very important, for only by doing this can discrepancies
be spotted. However in some projects they are not defined which results in a very poor
quality set of test cases.
A feature from the Test Design may be tested in more than one Test Case, and a Test
Case may test more than one feature. The aim is for a set of test cases to test each
feature from the Test Design at least once. Taking the Billing project example all three
requirements could be tested using two test cases:

The first test case could test both that a normal bill is produced and that a volume
discount is properly calculated.
A second test case could check that a final bill is produced and a volume discount
is calculated.

IEEE 829 - Test Procedure Specification

The Test Procedures are developed from both the Test Design and the Test Case
Specification. The document describes how the tester will physically run the test, the
physical set-up required, and the procedure steps that need to be followed. The
standard defines ten procedure steps that may be applied when running a test.

IEEE 829 - Test Item Transmittal Report

This curiously named document is not derived from the Test Plan but is the handover
document from the previous stage of development. In User Acceptance Testing this
may be the completion of System Testing.
It describes the items being delivered for testing, where to find them, what is new about
them, and gives approval for their release. The importance of the document is to
provide to the testers a warranty that the items are fit to be tested and gives a clear
mandate to start testing. Do not start testing without receiving one!

Documentation For Running The Tests

When the tests have been developed then they can be run. The schedule of what Test
Cases are run and when, is defined in the Test Plan. The test results are recorded in
the Test Log, and in Test Incident Reports.

IEEE 829 - Test Log

The Test Log records the details of what Test Cases have been run, the order of their
running, and the results of the test. The results are either the test passed, meaning that
the actual and expected results were identical, or it failed and that there was a
discrepancy. If there is a discrepancy than one or more Test Incident Reports are raised
or updated, and their identities recorded on the Test Log.
The Test Log is important as it allows progress of the testing to be checked, as well as
providing valuable information for finding out what caused an incident. If an incident is a
coding fault, the fault may have occurred not in the Test Case that failed but in one that
was run previously. Thus the sequence of the tests enables the fault to be found.

IEEE 829 -Test Incident Report

This document is deliberately named as an incident report, and not a fault report. The
reason is that a discrepancy between expected and actual results can occur for a
number of reasons other than a fault in the system. These include the expected results
being wrong, the test being run wrongly, or inconsistency in the requirements meaning
that more than one interpretation could be made.
The report consists of all details of the incident such as actual and expected results,
when it failed, and any supporting evidence that will help in its resolution. The report will
also include, if possible, an assessment of the impact upon testing of an incident.
The relationship between the Test Log and the Test Incident Report is not one to one. A
failed test may raise more than one incident, and at the same time an incident may
occur in more than one test failure. Taking the Billing project example, if both test cases
completely failed than three Test Incident Reports would be raised:

The first would be for failure to produce a normal bill,

The second would be for failure to produce a final bill,
The third for failure to calculate the volume discount for both the normal and the
final bill.

It is important to separate incidents by the features being tested so as to get a good

idea of the quality of the system, and allow progress in fixing faults to be checked.
A useful derivative document from the Test Incident Report is a Test Incident Log to
summarise the incidents and the status. This is not an IEEE 829 document as all it
values can be derived from the Test Incident Reports.

Documentation For Completion of Testing

Eventually testing will be completed according the criteria specified in the Test Plan.
This is when the success or failure of the system is decided based on the results. The
Test Summary records this information.

IEEE 829 - Test Summary

The Test Summary brings together all pertinent information about the testing, including
an assessment about how well the testing has been done, the number of incidents
raised and outstanding, and crucially an assessment about the quality of the system.
Also recorded for use in future project planning is details of what was done, and how
long it took.
This document is important in deciding whether the quality of the system is good
enough to allow it to proceed to another stage.

Use of the Standard

The standard is generic to cover all types of testing. As a result it allows the documents
to be tailored to each situation. This means using the basic structure as given, but other
documents can be added to it, sections can be added to each document, and further
descriptions can be written. In addition some content can be referenced in another
document. By using the standard means that anybody joining a project will know what
documents are being used, and for what purpose, allowing them to become productive
faster.