Hazard Analysis

Alan Wassyng
McMaster Centre for Software Certification

COMP
January 2014

Why do we need it?

! When we consider building devices or
processes that have to be safe how do we
ensure they are safe?
Always remember that 100% safety is not
achievable
What we want is to avoid unacceptably unsafe

! An obvious idea identify unsafe behaviour

What does that mean?
How do we do it?

! And then try to make sure those unsafe

behaviours never happen!
! Simple eh?

ALARA/ALARP

! As low as (is) reasonably achievable

! As low as reasonably practical
! Standards are a minimum that we have to
achieve!
! It is expected that we do better than what the
standards mandate
! The cost involved in reducing the risk further
would be grossly disproportionate to the benefit
gained
! Hazard analysis is an important aspect of
preventing errors we know that so you have
to do it (and it will help you do a better job)

Regulatory Regimes

NRC:

What is a hazard?
! Its a property or condition that has the
potential to cause {harm or damage} = loss

What is hazard analysis?

! Document hazards
! Document hazard controls how to mitigate
each hazard
! Hazard analysis introduces a different way of
thinking about our systems and processes
There is lots of anecdotal evidence to suggest
that we find and mitigate more hazards when we
follow some systematic process designed to
perform the hazard analysis

! Hazard analysis is mandatory in safety

critical domains
5

Nuclear, medical, chemical process industry,

Medical Devices
! ISO 14971 Application of risk management
to medical devices lists hazards for medical
devices
Energy hazards: Electromagnetic energy,
radiation energy, thermal energy, etc.
Operational hazards: Erroneous data transfer,
loss or deterioration of function, incorrect
measurement, use error, etc.

! IEC 62304 Medical device software

Software life cycle processes references
ISO 14971 and mandates hazard analysis,
does not mandate a particular type of HA
6

HA Flavours
! Lots to choose from

Hazard & Operability Study (HAZOPS)

Fault Tree Analysis (FTA)
Failure Modes and Effects Analysis (FMEA)
Failure Modes and Effects Criticality Analysis (FMECA)
Functional Resonance Accident Model (FRAM)
System-Theoretic Process Analysis (STPA)

HA Flavours
! And ones I see in use regularly

Hazard & Operability Study (HAZOPS)

Fault Tree Analysis (FTA)
Failure Modes and Effects Analysis (FMEA)
Failure Modes and Effects Criticality Analysis (FMECA)
Functional Resonance Accident Model (FRAM)
System-Theoretic Process Analysis (STPA)

HA Flavours
! And ones I see in use regularly

Hazard & Operability Study (HAZOPS)

Fault Tree Analysis (FTA)
Failure Modes and Effects Analysis (FMEA)
Failure Modes and Effects Criticality Analysis (FMECA)
Functional Resonance Accident Model (FRAM)
System-Theoretic Process Analysis (STPA)

! And some people talk about

Preliminary Hazard Analysis

as though it is a different kind of hazard analysis. It is
simply a hazard analysis done at the earliest stage of
system development

Concept ISO 14971

10

FTA
! Top down
! Process
Define the TOP event to be analyzed
Identify the lower level events which may lead to
the TOP event and complete the gates
(optional) Find minimal cut sets (qualitative)
(optional) Calculate the failure rate of TOP event
(quantitative)
! Aside: Good for identifying single points of failure
! Cut set = events that together cause the top event
(sometimes called a fault path)
11

Example FTA: Insulin Pump

! Insulin Pump Extract top level FTA

12

Example FTA: Insulin Pump

! Insulin Pump Extract FTA - expand underdosed

13

Example FTA: Insulin Pump

! Insulin Pump Extract FTA - expand too low

14

Graphics?

15

Aside: One solution to SE

display problems

16

FMEA
! Bottom up approach need to know all
details
! Was not designed to consider combination
failure initiating events
! Performed on both processes and products
! Many people use RPN to prioritize so
mitigate only those hazards with RPN > x
RPN = Risk Priority Number
= Severity * Probability of Occurrence * Detection Rating

17

! Aside: I do not endorse the use of probabilistic approaches

(risk) to prioritize mitigation of hazards! Too many accidents
have resulted from that is not likely to happen. We have no
real basis for our risk assessments in general.

Example Process FMEA

18

Example FMEA: Insulin Pump

Functional decomposition of the insulin pump

19

Example FMEA: Insulin Pump

20

FMEA Zoomed-in Left

21

FMEA Zoomed-in Right

22

FMEA ! Safety Requirements

23

STPA
From Nancy Leveson

24

NRC View

25

STPA
From Nancy Leveson

! Four categories of control actions to consider

A control action required for safety is not
provided or is not followed
An unsafe control action is provided that leads to
a hazard
A potentially safe control action is provided too
early, too late, or out of sequence
A safe control action is stopped too soon (for a
continuous or non-discrete control action)

! For me, this is an important breakthrough there is some idea of completeness that
helps us consider all possibilities
26

Example STPA: Insulin Pump

Viewed as a control system

27

Example STPA: Insulin Pump

Hazards included

28

Example STPA: Insulin Pump

29

Example STPA: Insulin Pump

Zoomed-in
extract

30

Motivation for STPA

! Many failures are traced back to interaction
failures components work well, but put
them together in a specific environment and
we get unanticipated failures
! STPA may help us find those hazardous
interactions in the environment, for example
! Question: Why do so many cars in Canada
not have their rear lights on at night?
31

Useful References
! There is a pretty good book on the subject that
discusses (too) many different kinds of hazard
analyses:
Hazard Analysis Techniques for System Safety by Clifton
A. Ericson II, (2005)

! Interesting maybe, rather than useful:

U.S. Nuclear Regulatory Commission, Fault tree handbook
(NUREG 492). URL: http://www.nrc.gov/reading-rm/doccollections/nuregs/staff/sr0492/sr0492.pdf
MIL-STD-1629 "Procedures for Performing a Failure Mode,
Effects and Criticality Analysis (cancelled but still used)

32

! Engineering a Safer World: Systems Thinking

Applied to Safety (Engineering Systems) by
Nancy G. Leveson (2012)

New & Interesting

! U.S. N.R.C. DRAFT RESEARCH
INFORMATION LETTER 1101: Technical basis
to review hazard analysis of digital safety
systems, Dec 2013, Rev. 3
Public document out for comment
Interesting. NRC seems to have added to Nancys 4
control categories

33

Example from RiL 1101

34

How do we construct correct,

safe and reliable software?
! Rigorous software engineering
Validation Test and
Reliability Qualification
Reports

HAR

Requirements
Review
Report

Hazard analysis
is iterative over the
life of the project!

Formal
Requirements
Documents

Design Review and

Verification Reports

HAR

Software Integration
Test Report

Software
Design
Document

Unit Test
Report

HAR

Legend:
Documents produced in the
forward going development process
Documents produced for
verifications, reviews and
testing
Activities and data flow

35

HAR

Hazards Analysis Report

HAR

Code
Code Review and
Verification Reports

How do we construct correct,

safe and reliable software?
! Rigorous software engineering
Validation Test and
Reliability Qualification
Reports

HAR

Requirements
Review
Report

We have a defencein-depth approach to

the software development
process itself driven by
identification of SPOF.

Formal
Requirements
Documents

Design Review and

Verification Reports

HAR

Software Integration
Test Report

Software
Design
Document

Unit Test
Report

HAR

Legend:
Documents produced in the
forward going development process
Documents produced for
verifications, reviews and
testing
Activities and data flow

36

HAR

Hazards Analysis Report

HAR

Code
Code Review and
Verification Reports

Acknowledgements

37

! Yao Song, MASc Graduate - FTA, FMEA, STPA

applied to Darlington SDS1
! Ben Breimer, MASc Graduate STPA applied to
adaptive cruise control
! Mischa Geven, Nicholas Proscia, MASc candidates
working on the Insulin Pump
! John Stribbell, MEng Graduate, Grant Whinton,
MASc Candidate & Linna Pang, PhD Candidate FTA Insulin Pump
! Vera Pantelic (slides from COMP 2013)
! Colleagues: Paul Joannou, Mark Lawford, Tom
Maibaum (all at McMaster), Sushil Birla, Luis
Betancourt (both at US NRC), Paul Jones (US FDA),
Nancy Leveson (MIT)

Acceptably Safe?

38