You are on page 1of 24

System test,

Evaluation, and
Validation
By:
Ocampo,
Abbygail
Zabat, Jenilyn

System test, Evaluation and


Validation
System test, evaluation, and validation activities should be established
during the conceptual design phase of the life cycle, concurrently with
the definition of the overall system design requirements. From that
point on, The test and evaluation continues by testing of individual
components, the testing of various system elements and major
subsystems and then by the testing of overall system as an integrated
entity.
The objective is to adopt a progressive approach that will lend it self to
continuous implementation and improvement as the system design
and development process evolves.
Test and evaluation activities in this chapter can be aligned initially with
the design activities describe in chapter3-5 and then extended through
production/construction and system utilization.

The purpose of this chapter


is to:
to present an integrated approach for system

test and evaluation


to facilitate the necessary validation of the
proposed system configuration and
to provide assurance that will indeed meet
the customer requirements

System
Requirements
and Evaluation
Relationship

Stages of System Evaluation During Life Cycle

Categories of System test


and Evaluation

6.2.1 Analytical and Simulation Evaluation

The utilization of analytical models, CAD methods, and


various combinations of these can be effective in the
verification of certain design relationships before system is
finalized.
for example, the development of system through
computer graphics can illustrate the relationships among
various system elements and accesses for maintenance
and support. From this, potential interferences, space
utilization conflicts can be detected.

Categories of System test


and Evaluation

Type 1 Testing
During the early phase of detail design, breadboards, bench-test models,
engineering models, engineering software, and service test models are
build with the intent of verifying certain performance and physical design
characteristics. These models, representing either an entire system or a
designated system component, usually operate functionally (electrically
and mechanically) but do not by any means represent production
equipment. In the development of software, the application of rapid
prototyping is sometimes used to verify design adequacy.

Categories of System test


and Evaluation

Type 2 Testing
Formal tests and demonstrations are accomplished
during the latter part of detail design when
preproduction prototype equipment, software, and
similar to production equipment (that which will be
deployed for operational use), but it is not necessarily
fully qualified at this point in time. A test program
may constitute a series of individual tests tailored to
the need. Such a program might include the following:

Type 2 test..

1. Performance tests are accomplished to verify individual


system performance characteristics. For instance, tests are
designed to determine whether the electric motor will provide
the necessary output, whether the airplane will perform its
intended mission successfully, and so on. Also, it is necessary
to verify form, fit, interchangeability, product safety, and other
comparable features.
2. Environmental qualification temperature cycling, shock
and vibration, humidity, wind, salt, spray, dust and sand,
fungus, acoustic noise, pollution emission, explosion proofing,
and electromagnetic interference tests are conducted. These
factors are oriented to what the various system elements will
be subjected to during operation, maintenance, and during
transportation and handling function.

Type 2 test..
3. Structural tests are conducted to determine material
characteristics relative to stress, strain, fatigue, bending,
torsion and general decomposition.
4. Reliability qualification are accomplished on one or more
system elements to determine MTBF. Also, special tests
are often designed to measure component life, to
evaluate degradation, and to determine models of failure.
5. Maintainability demonstration tests are conducted on
one or more system elements to assess the values for
mean active maintenance time (M), mean corrective
maintenance (Mct), mean preventive maintenance time
(Mpt), maintenance labor-hours per operating hour
(MLH/OH), and so on

Type 2 test..
6. Support equipment compatibility tests are often
accomplished to verify compatibility among the prime
equipment, test and support equipment, and
transportation and handling equipment
7. Personnel test and evaluation are often accomplished
to verify the relationships between people and
equipment, the personnel quantities and skill levels
required and training needs, both operator and
maintenance tasks are evaluated.
8. Technical data verification the verification of
operational and maintenance procedures is
accomplished.
9. Software verification the verification of operational
and maintenance software is accomplished.

Categories of System test


and Evaluation
Type 3 Testing
Formal tests and demonstrations, started
after initial system qualification and prior to
the completion of production, are
accomplished at a designated field test site
by user personnel, operation.

Categories of System test


and Evaluation
Type 4 Testing
During the operational-use phase, formal tests are
sometimes conducted to gain further insight in a
specific area. It may be desirable to vary the
mission profile or the system utilization rate to
determine the impact on total system effectiveness,
or it might be feasible to evaluate several alterative
support policies to see whether system operational
availability can be improved

Planning for Testing and Evaluation

1.
2.
3.

The definition and schedule of all tests


requirements, including anticipated test
output for each individual test and
integrated where possible.
The definition of organization,
administration and control responsibilities.
The definition of test conditions and
maintenance and logistic resource
requirements.

Planning for Testing and


Evaluation
4. A description of the test preparation phase
for each type of testing method, training of
testing.
5. A description of the formal test phase.
6. A description of condition and provisions for
a retest phase.
7. The identification of test documentation

Evaluation of Test Requirements

Preparation for system test


and evaluation

The selection of the items to be tested,


Establishment of test procedures,
Test site selection,
Selection and training of test personnel,
Preparation of test facilities and resources,
Acquisition of support equipment,
Test supply support.

Test performance and


reporting
This requires operating and supporting the

system in a prescribed manner as defined in


the system test and evaluation plan.

Test performance and


reporting
The following questions will be answered:
1. what is the true performance and
effectiveness of the system?
2. what is the true effectiveness of the
maintenance and logistic support capability?
3. are the initial specified requirements being
met?

Test data requirements


1.
2.

It provides ongoing data that are analyzed


to evaluate and assess the performance,
effectiveness, operation, maintenance,
logistic support capability, and so on.
It provides historical data that are
applicable in the design and development
of new systems having a similar function
and nature.

Development of a data
subsystem
1.
2.

Success data constitute information


covering system operation and use on a
day-to-day basis.
Maintenance data cover each event
involving scheduled and unscheduled
maintenance.

Test reporting
1.
2.
3.

The final effort in the evaluation process,


It should reference the initial system
evaluation planning document,
It should describe all test conditions,
incorporated system modifications during
the test, test data, and the results of data
analysis.

System
Evaluation
and
Corrective
Action Loop

Summary

You might also like