UNIT 4

Experimental Evaluation
Modelling and simulation

stochastic Petrinets

Fault injection

NFTAPE fault injector

networks

Modelling for performance

dependability and perform

SAN/NAS.

stochastic

Application:

activity

Ultra

ability:

dependability-specific

methods

(fault

trees,

reliability

block diagrams), queues

Roll No: 15

Modelling
Definition
Modeling is the process of producing a model.
A model is a representation of the construction and
working of
some system of interest.
A model is similar to but simpler than the system it
represents.
One purpose of a model is to enable the analyst to
Real Time and Fault Tolerance
predict the effect of changes to the system.

Modelling
A model should be
close approximation to the real system
incorporate most of its salient features.
Less complex
Realism and simplicity.

Real Time and Fault Tolerance

Modelling Application
A model should be
Banking
Space
Military
Commerce
Education

Real Time and Fault Tolerance

Mathematical model classifications

include

Deterministic (input and output variables are fixed

values)
Stochastic (at least one of the input or output
variables is probabilistic);
Static (time is not taken into account)

Dynamic

(time-varying

interactions

among

variables are taken into account).

Typically, simulation models are stochastic and

dynamic.

Real Time and Fault Tolerance

Simulation
Definition
A simulation of a system is the operation of a model
of the system.

The operation of the model can be studied, and

hence, properties concerning the behavior of the
actual system or its subsystem can be inferred.

Real Time and Fault Tolerance

Definition

Simulation is a tool to evaluate the performance of a

system,

existing

or

proposed,

under

different

configurations of interest and over long periods of

real time.

Real Time and Fault Tolerance

Simulation is used before an existing system is

altered or a new system built,:

To reduce the chances of failure to meet

specifications,
To eliminate unforeseen bottlenecks,
To prevent under or over-utilization of resources,
To optimize system performance.

sombody@gmail.com

simulation can be used to answer questions

What is the best design for a new
telecommunications network?
What are the associated resource requirements?

How will a telecommunication network perform when

the traffic load increases by 50%?

How will a new routing algorithm affect its

performance?
Which network protocol optimizes network
performance?

What will be the impact of a link failure?

Real Time and Fault Tolerance

Real Time and Fault Tolerance

Steps in simulation
The steps involved in developing a simulation
model, designing a simulation experiment, and
performing simulation analysis are:
Step 1. Identify the problem.
Step 2. Formulate the problem.
Step 3. Collect and process real system data.
Step 4. Formulate and develop a model.
Step 5. Validate the model.
Real Time and Fault Tolerance

 Step 6. Document model for future use.

Step 7. Select appropriate experimental design.
Step 8. Establish experimental conditions for runs.
Step 9. Perform simulation runs.
Step 10. Interpret and present results.
Step 11. Recommend further course of action.

Real Time and Fault Tolerance

Simulation Used In
government, defense, computer and
communication
systems, manufacturing, transportation (air traffic
control), health care, ecology and environment,
sociological and behavioral studies, biosciences,
epidemiology, services (bank

Real Time and Fault Tolerance

Fault Injection
Definition

Fault injectionis a technique for improving

thecoverage of a test by introducing faults to test
code paths, in particularerror handling code
paths, that might otherwise rarely be followed.

It is often used withstress testingand is widely

considered to be an important part of
developingrobustsoftware

Real Time and Fault Tolerance

Fault Injection Types

Definition
Compile time injection
Mutation testing
Runtime injection (memory corruption and n/w
fault)
Physical methods
Triggers
Time
Interrupt based

Real Time and Fault Tolerance

Real Time and Fault Tolerance

Fig explained
The fault injector injects faults into the target
system as it executes commands from the
workload generator
The monitor tracks the execution of the commands
and
initiates data collection whenever necessary.
The data collector performs online data collection.
The data analyzer, which can be offline, performs
data processing and analysis.

The controller controls the experiment.

Real Time and Fault Tolerance

Tools

Ferrari
FTAPE
Orchestra
Doctor
Xception

Real Time and Fault Tolerance

FTAPE
Introduction
FTAPE (Fault Tolerance and Performance Evaluator) can
inject faults, not only into memory and registers, but
into disk accesses as well.

This is achieved by inserting a special disk driver into

the system that can inject faults into data sent and
received from the disk unit.

FTAPE also has a synthetic load unit that can simulate

specific
purposes

amounts

of

load

for

robustness

testing

Real Time and Fault Tolerance

NFTAPE
Introduction

NFTAPE is a configurable tool for

injecting faults,

triggering injections,

producing workloads,

detecting errors

logging results.

Real Time and Fault Tolerance

Modelling for performance, dependability and

perform ability

Introduction
Dependability, performance, and performability
evaluation techniques provide a useful method for
understanding the dynamic behavior of a computer or
communication system.

To be useful, the evaluation should reflect important

system characteristics such as fault- tolerance,
automatic reconfiguration, and repair; contention for
resources; concurrency and synchronization; deadlines
imposed on the tasks; and graceful degradation.
Real Time and Fault Tolerance

Dependability
Measure of following attributes

Availability
% of time delivering correct service
Reliability
Expected time until incorrect service
Safety
Absence of catastrophic consequences
Confidentiality
Absence of unauthorized disclosure
Real Time and Fault Tolerance

Means to Dependable system

Fault prevention
Fault tolerance
Fault removal
Fault forecasting

Real Time and Fault Tolerance

Performability

It is a measure of the system ability to achieve a

given performance level, despite the occurrence of
failures.

Performability

differs

from

reliability

in

that

reliability is a measure of the likehooh that all of

the

functions

are

performed

correctly,

while

performability is a measure of likehood that some

subset of the functins is performed correctly.
Real Time and Fault Tolerance

Fault tree diagrams

Fault tree diagrams consist of gates and events
connected with lines.
The AND and OR gates are the two most
commonly used gates in a fault tree.

To illustrate the use of these gates, consider two

events (called "input events") that can lead to
another event (called the "output event").

If the occurrence of either input event causes the

output event to occur, then these input events
are connected using an OR gate.
Real Time and Fault Tolerance

Fault tree diagrams symbol

Basic
External
Undeveloped
Conditioning

Used in
aerospace, nuclear power, chemical and process,
pharmaceutical, petrochemical and other highhazard industries

Real Time and Fault Tolerance

Fault tree diagrams

Alternatively, if both input events must occur in
order for the output event to occur, then they are
connected by an AND gate.

Figure 1 shows a simple fault tree diagram in

which either A or B must occur in order for the
output event to occur. In this diagram, the two
events are connected to an OR gate

Real Time and Fault Tolerance

stages

Identify undesired event to study

Understand system
Construct tree
Evaluate
Control hazards identified

Real Time and Fault Tolerance

Reliability Block Diagram

Directed graph:
Exactly one starting node E, exactly one terminal
node A.
Other nodes represent the binary random
variable of a
component (stating whether "faultless" or "faulty").

Notice that multiple nodes are allowed for a single

component.
Additional virtual nodes H help to simplify the
representation.
Time if
and Fault Tolerance
Semantics: The system is faultless if and Real
only

Reliability Block Diagram

Real Time and Fault Tolerance

Stochastic activity
Includes

Stochastic

activity

networks,

or

convenient, graphical, high-level

SANs,

are

language for

describing system behavior.

SANs are useful in capturing the stochastic (or

random) behavior of a system.

Real Time and Fault Tolerance

Examples

The amount of time a program takes to execute

can be computed precisely if all factors are known,
but

this

is

nearly

impossible

and

sometimes

useless.

At a more abstract level, we can approximate

the running time by a random variable.

Fault arrivals almost always must be modeled by

a random process.
Real Time and Fault Tolerance

Stochastic Petri Net Review

Concepts
One of the simplest high-level modeling
formalisms is called stochastic Petri nets.

A stochastic Petri net is composed of the following

components:

Real Time and Fault Tolerance

continue

Real Time and Fault Tolerance

A stochastic Petri net is made from a Petri net by

Assigning an exponentially distributed time to all

transitions.
Time represents the delay between enabling and
firing of a timed transition.
Transitions execute in parallel with independent
delay distributions.
Since the minimum of multiple independent
exponentials is itself exponential, time between
transition firings is exponential.

If a transition t becomes enabled, and before t fires,

some other transition fires and changes the
Realstate
Time and of
Faultthe
Tolerance

Stochastic Activity Networks

The need for more expressive modeling languages
has led to several extensions to stochastic Petri
nets.

One extension that we will examine is called

stochastic activity networks.

Because there are a number of subtle distinctions

relative to SPNs, stochastic activity networks use
different words to describe ideas similar to those of
SPNs.

Real Time and Fault Tolerance

Stochastic activity networks have the following

properties:
A general way to specify that an activity (transition)
is enabled
A general way to specify a completion (firing) rule
A way to represent zero-timed events
A way to represent probabilistic choices upon
activity completion
State-dependent parameter values
General delay distributions on activities

Real Time and Fault Tolerance

SAN Symbols

Real Time and Fault Tolerance

SAN Terms
1.activation - time at which an activity begins

2. completion - time at which activity completes

3. abort time: after activation but before

completion, when activity is no longer enabled

4. active - the time after an activity has been

activated but before it completes or aborts.
Real Time and Fault Tolerance

Completion Rules
When an activity completes, the following events take
place (in the order listed),

possibly changing the

marking of the network:

1. If the activity has cases, a case is (probabilistically)
chosen.
2. The functions of all the connected input gates are
executed (in an
unspecified order).
3. Tokens are removed from places connected by input
arcs.
4. The functions of all the output gates connected to the
Real Time and Fault Tolerance

chosen case are executed (in an unspecified order).

THANK YOU