Validation and Verification

Adapted from Jerry

Banks

Verification
Concerned with building the model
right
Comparison of conceptual model and
computer representation
Is the model implemented correctly
in the computer?
Are the inputs and logical
parameters represented properly?

Validation
Concerned with building the right
model
Accurate representation of the real
system
This is achieved through the
calibration of the model
Iterative process until accuracy is
acceptable

REAL SYSTEM

Calibration and
Validation

Conceptual
Validation

Conceptual Model
1Assumptions on system components
2Structural Assumptions (defines the interactions
between the system components)
3Input parameters and data assumptions

Model
Verification

Operational Model
(Computer
Representation)

Model
Building,
Verification,
and
Validation

Common sense suggestions

for verification
Have someone check the
computerized model
Make a flow diagram (with logical
actions for each possible event)
Examine model output for
reasonableness
Print the input parameters at the
end of the simulation

Common sense suggestions

for verification
Make the computerized
representation as self documenting
as possible
If animated, verify what is seen
Use IRC or debuggers
Use graphical interface

Three Classes of Techniques

for Verification
Common sense techniques
Thorough documentation
Traces

Calibration and Validation

Validation is the overall process of
comparing the model and its
behavior to the real system and its
behavior
Calibration is the iterative process of
comparing the model to the real
system and making adjustments to
the model, and so on.

Iterative Process of
Calibration
Compare Model to
Reality

REAL SYSTEM

Compare Revised
Model to Reality

Compare second
Revised Model to
Reality

Initial Model

First Revision of
Model

Second
Revision of
Model

3 Step Approach by Naylor

and Finger (1967)
Build a model with high face validity
Validate model assumptions
Compare the model input-output
transformations to corresponding
input-output transformations of the
real system

Possible validation techniques in

order of increasing cost-value
ratio by Van Horn (1971)

High face validity. Use previous

research/studies/observation/experience
Conduct statistical test for data
homogeneity, randomness, and goodness
of fit test
Conduct Turing test. Have a group of
experts compare model output versus
system output and detect the difference
Compare model output to system output
using statistical tests

Possible validation techniques in

order of increasing cost-value
ratio by Van Horn (1971)
After model development, collect
new data and apply previous 3 tests
Build a new system or redesign the
old one based on simulation results
and use this data to validate the
model
Do little or no validation. Implement
results without validating