A Calculation Method for Software Safety Integrity Level

Takaji Fujiwara

Juan Manuel Estevez

Business Cube & Partners, Inc.

1-20-18 Ebisu, Shibuya-ku, Tokyo, Japan

Business Cube & Partners, Inc.

1-20-18 Ebisu, Shibuya-ku, Tokyo, Japan

fujiwara@biz3.co.jp
Yoshinobu Satoh

jme@biz3.co.jp
Shigeru Yamada

Graduate School of Marine Sci. and Tech.

Tokyo University of Marine Sci. and Tech.
2-1-6 Etchujima, Koto-ku, Tokyo, Japan

Graduate School of Engineering

Tottori University
4-101 Minami, Koyama-cho, Tottori, Japan

yoshi@kaiyodai.ac.jp

yamada@sse.tottori-u.ac.jp
with the hardware, though software reliability and safety
has been studied so far, these have still not necessarily been
satisfactory. Therefore, once system failures due to defects
or faults latent in the software system come to the surface,
the computer system is entirely useless and many people
sustain great damage. Occasionally, there are also the worst
defects or faults which would bring about serious and critical accidents to human life. In such present circumstances,
the development of highly reliable and safe software is an
important issue.
As one of the solutions to this issue, developers have been
conforming their development to the functional safety standards (IEC 61508 and ISO/DIS 26262) [1],[2]. In the functional safety standards, the development and quantitative
analytical methods are dened for the hardware of safetyrelated systems (abbreviated as SRSs). However, only development methods are dened for the software SRSs. That
is, the safety integrity level (in IEC 61508, it is called Automotive Safety Integrity Level in case of ISO/DIS 26262)
(abbreviated as SIL) for software is determined only by the
number of the development methods applied to practical
SRS development. This is not reasonable to evaluate the
SIL, because various risk factors should be included. As a
clear risk factor, the implemented software varies in quality
according to the skill level of the developers. Thus, a developer has to grasp reliability and safety quantitatively as a
quality level when verifying the implemented software.
In this paper, we propose a method for calculating the SIL
for software. Especially, we propose a calculation method
based on the software reliability growth model (abbreviated
as SRGM) [5],[6] that has long been used in the large-scale
system development.

ABSTRACT
In the functional safety standards (IEC 61508 and ISO/DIS
26262), development methods and quantitative analytical
methods are dened for establishment of safety-related systems. However, only development methods are recommended
to establish the software of safety-related systems. That is,
the safety integrity level for software is determined only by
the number of the development methods applied to practical safety-related system development. This is not reasonable to evaluate the safety integrity level, because various
risk factors should be taken up. In this paper, we propose
how to calculate the safety integrity level for software. Especially, we propose the calculation method based on the
software reliability growth model that has long been used in
the large-scale system development.

Categories and Subject Descriptors

G.3 [Probability and Statistics]: Statistical computing; G.4
[Mathematical Software]: Reliability and robustness

General Terms
Software Reliability and Safety

Keywords
safety integrity level, software reliability growth model, calculation method

1.

INTRODUCTION

In today's highly informative society, computer systems

are used in various elds, and play important roles in our
social life. However, the hardware of the computer system
has been suciently studied and attained high reliability as
well as safety, and has been applied to business. In contrast

2.

DESCRIPTION OF SRGM

In this Section, we discuss software reliability growth modeling based on a non-homogeneous Poisson process (abbreviated as NHPP) [5],[6], because the SRGM based on NHPPs
have been widely and successfully applied to practical software development activities.
Generally, during the testing phase in the software development process, the software developer has to execute many
various test-cases in order to verify the implemented functions based on the requirement specication. At that time,
they detect the faults which are latent in the software, and
those corrections and removals are performed in accordance

Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
CARS 10, April 27, Valencia, Spain
Copyright 2010 ACM 978-1-60558-915-2/10/04 ...$10.00.

31

Table 1: The probability distribution for the existing SRGMs.

NHPP-based SRGMs
Goel & Okumoto model (Goel and Okumoto, 1978)
delayed S-shaped model (Yamada et al., 1983)
hyperexponential model (Ohba, 1984)
inection S-shaped model (Ohba, 1984)
modied Duane model (Littlewood, 1984)
generalized exponential model (Goel, 1985)
generalized S-shaped model (Zhao and Xie, 1996)
log-logistic model (Gokhale and Trivedi, 1998)
log-normal model (Achcar et al., 1998)
phase-type model (Okamura and Dohi, 2006)

Accordingly, we can conclude that the SRGM having the

smallest value of the MSE ts best the analytical data.
Next, the AIC can be calculated with the following equation;

with the specied procedures. That is, a software reliability growth during the testing-phase means the relationships
between the testing-phase and the cumulative number of
faults detected by testing or the time-interval between software failures. Then, the reliability growth curve represents
the time-dependent behavior of the cumulative number of
detected faults with progress of the testing. By describing
fault-detection phenomenon by a stochastic model based on
an NHPP, software reliability assessment during the testingphase and the reliability prediction for the operating-phase
can be performed.
In order to describe the fault-detection phenomenon at
arbitrary testing time t, let {N(t) , t 0} denote a counting process representing the cumulative number of faults
detected up to arbitrary testing time t. Then, the faultdetection phenomenon can be formulated by an NHPP as
follows;
{H(t)}n
exp[H(t)]
n!
(n
= 0, 1, 2, ) ,
Z t
H(t) =
h(x) dx,

Distribution
exponential
2-Erlang
hyperexponential
truncated logistic
Pareto
Weibull
n-Erlang
log-logistic
log-normal
phase-type

AIC = 2 {M (the logarithmic maximumlikelihood of an SRGM)} ,

(3)

where M is the number of parameters in an SRGM. It is

because these parameters are estimated by the maximum
likelihood. When the values of AIC for compared SRGMs
are calculated, if the dierence among the calculated values
is 12 or more, we can judge that the dierence is signicant
and the SRGM with this small value has good suitability.
And if the dierence of the AIC is 1 or less, the superiority or
inferiority of compared SRGMs cannot be judged, and the
same degree of goodness-of-t is meant as both compared
SRGMs t equally well.

Pr {N(t) = n} =

3.

(1)

3.1

CALCULATION OF SOFTWARE SIL

About SIL [1]

Safety integrity and the SIL are dened by IEC 61508-4

as follows:

where Pr {A} is dened as the probability of event A, and

H(t) is called the mean value function of an NHPP, which
represents the expected value of N(t). Further, h(t) is called
the intensity function of an NHPP, and represents the instantaneous fault-detection rate at testing time t.
The SRGM based on NHPPs have been proposed by Goel
& Okumoto in 1978 [4], and many researchers have proposed SRGMs based on Goel & Okumoto Model in consideration of various testing- and operational-environment.
Consequently, the SRGMs which can represent various probability distributions are proposed (see Table 1). A developer
can obtain various information for the software to be tested
by selecting the most suitable SRGM for analytical data
from these SRGMs. As criteria for selecting the most suitable SRGM for analytical data, we can use the mean square
errors (abbreviated as MSE) [5],[6] and the Akaike Information Criterion (abbreviated as AIC) [3].
b
Letting n be the observed number of data, and H(t)
the
estimated mean value function, we can calculate the value
of the MSE by using the following equation:
n
i2
1 Xh
b (tk ) .
yk H
(2)
M SE =
n
k=1

Safety Integrity: probability of an SRS satisfactorily performing the required safety functions under all the stated
conditions within a stated period of time.
SIL: discrete level for specifying the safety integrity requirements of the safety functions to be allocated to the
E/E/PE SRSs, where SIL 4 has the highest level of safety
integrity and SIL 1 has the lowest.

As a note, target failure measures for the four safety integrity levels are specied in Table 2 (see IEC 61508-1).
Further, target failure measure is dened as intended probability of dangerous mode failures to be achieved in respect
of the safety integrity requirements, and specied viewpoint
of following either by IEC 61508-4:
low demand mode: where the frequency of demands for
operation made on an SRS is no greater than one per
year and no greater than twice the proof-test frequency.
high demand or continuous mode: where the frequency
of demands for operation made on an SRS is greater
than one per year or greater than twice the proof-test
frequency.

32

Table 2: SILs: target failure measures for a safety function, allocated to an E/E/PE SRS.

SIL

low demand mode of operation

(average probability of
failure to perform its
design function on demand)

4
3
2
1

105
104
103
102

to
to
to
to

< 104
< 103
< 102
< 101

109
108
107
106

That is, target failure measure, i.e., SIL for high demand
and continuous mode of operation can be considered the
dangerous failure rate or dangerous failure intensity of SRS.
These values are almost equivalent, given that the time to
repair is negligible comparing to the mean time to failure.
Consequently, we can show SIL by the following equation:
SIL failure rate =

1
.
Mean Time Between Failure

(4)

< 108
< 107
< 106
< 105

1. Case where the fault is classied into the dangerous

and safe failures clearly:
We propose that DFR is dened by dividing the number of dangerous failures (abbreviated as NDF) by
the total number of failures (abbreviated as TNF), as
shown in the following equation:

MTBF for Software

The MTBF for software can be represented as the mean

time between software failure occurrence or the mean time
between fault detection. Accordingly, as the MTBF for software, we can use the Instantaneous MTBF [5],[6] and the
Cumulative MTBF [5],[6]. These measures can be assessed
by using the SRGM described in Section 2, and we can obtain various information in regard of the software quality
from these analysis results.
Next, we explain the Instantaneous MTBF and the Cumulative MTBF briey as follows:

DFR =

NDF
.
TNF

Ex: DFR is 0.01, where TNF is 100 and NDF is 1 in

it.
2. Case where the fault is classied into the safe and unclear failures:
The unclear failure means that cannot be determined
as the dangerous and safe failure. The number of this
failure is denoted as UCF. Thus, we propose that DFR
is dened as follows;

Instantaneous MTBF: is represented with the inverse

number of the expected value of the fault detected by
testing in the testing time t. We can compute by the
following equation:
1
.
Instantaneous MTBF =
h(t)

to
to
to
to

Generally, we use the fault count data detected during the

testing, in order to assess the Instantaneous MTBF and the
Cumulative MTBF. However, we have to focus on dangerous
failure in order to calculate software SIL for high demand
or continuous mode of operation, because both of dangerous and safe failure are included in the fault count data.
That is, we need to classify into dangerous and safe failure
by analyzing the detected the fault's contents. Then, since
there is a dicult case according to the fault's contents, we
calculate a dangerous failure ratio (abbreviated as DFR) by
the method shown below:

The mean time between failure (abbreviated as MTBF) is

failure occurrence frequency, i.e., the measure expressing a
steady operating state. In the following sections, we discuss the MTBF for software and a calculation method for
software SIL.

3.2

high demand or
continuous mode of operation
(probability of a dangerous
failure per hour)

DFR =

(5)

UCF
,
TNF + UCF

That is, we have to pay attention to show an optimistic

estimate because the analysis result by the Instantaneous
MTBF analyzes using the latest data.

where a unclear failure is counted to each of the dangerous and safe failure.
Ex.: DFR is 0.0099, where TNF is 100 and UCF is 1
in it.

Cumulative MTBF: is represented with the value divided

testing time by the number of cumulative faults detected
by the present from the testing-beginning time. We can
calculate by the following equation:

3. Case where the fault is classied into three types, i.e.,

the dangerous, safe, and unclear failures:
We propose that DFR is dened as follows:

Cumulative MTBF =

t
.
H(t)

DFR =

(6)

That is, we have to pay attention to show a pessimism

value because the analysis result by the Cumulative MTBF
analyzes using all data.

3.3

NDF + UCF
.
TNF + UCF

Consequently, we can get dangerous failure rate by multiplying the Instantaneous MTBF or the Cumulative MTBF
obtained as the analysis result by calculated DFR. Then,
we can judge the calculation result to be the software SIL
of the tested at the analysis time.

Calculation Method

33

500

500
Analysis result based on

Analysis result based on

all cumulative data

400

300
Reflect
analysis results

200

to testing-policy

100

Instantaneous MTBF (days)

Instantaneous MTBF (days)

cumulative data up to 16 days

400

300

200

100

0
5

10

15

20

25

30

Testing-Time (days)

10

15

20

25

30

Testing-Time (days)

Figure 1: The estimated Instantaneous MTBF for DS.

3.4

As a future issue, we are going to study a software SIL

calculation method for the low demand mode of operation.

Numerical Examples

We can apply the technique to an example numerical data

set, DS, and calculate the SIL. This data set consists of 23
data pairs in form of

5.

DS: (tk , yk ) (k = 1, 2, , 23 ; 0 < t1 < t2 < < t23 ),

where yk is the cumulative number of faults detected in the
time-interval ( 0, tk ] and the measurement unit of testing
time tk represents calendar days. DS has been observed in
the embedded software system development. We analyze DS
by the most suitable SRGM discussed in Section 2.
The analysis results in DS are shown in Fig. 1. Lefthand side of Fig. 1 is the Instantaneous MTBF analyzed
using the fault count data from the testing-beginning time
to the 16th, and right-hand side shows the result analyzed
by 23 sets of all data. Thus, the developer can get not
only software quality information but also software SIL by
performing analysis at any time during the testing.
For example, in Fig. 1, we can obtain the software SIL as
follows:
1
DFR = 8.25 107 ,
SIL =
500 24

6.

REFERENCES

[1] IEC 61508 Functional Safety of Electrical / Electronic /

Programmable Electronic Safety-Related System. 1998.
[2] ISO/DIS 26262 Road Vehicles - Functional Safety -.
2009.
[3] H. Akaike. What is the information criterion aic ? Suri
Kagaku, 14(3):511, 1976.
[4] A. L. Goel and K. Okumoto. A time dependent error
detection rate model for a large scale software system.
In Proc. 3rd USA-Japan Computer Conference, pages
3540, 1978.
[5] H. Pham. Software Reliability. Springer-Verlag,
Singapore, 2000.
[6] S. Yamada. Software Reliability Models - Fundamentals
and Applications. JUSE Press, Tokyo, Japan, 1994.

where the tested software can operate without a fault occurring for 500 days after several days testing and DFR is
0.0099 from the detected faults' contents. Consequently,
we can see that this calculation result shows software SIL-2
from Table 2.

4.

ACKNOWLEDGMENTS

The authors wish to thank Prof. Mitsuhiro Kimura, Faculty of Science and Engineering, Hosei University for helpful
suggestions and generous supports.

CONCLUSIONS

In this paper, we have proposed a method for adapting the

reliability assessment method of using the fault count data
collected during the testing-phase as the basis for calculating the software SIL. As a result, a developer can obtain
not only information about the quality of the tested software, but also the software SIL. Furthermore, by making
this assessment quantitatively, a developer can determine
the degree to which the targeted safety level is achieved, providing greater feedback to the testing policy. Accordingly,
the developer can secure system-wide safety and realise the
development of highly reliable and safe products.

34