Safety Performance

Indicators
Stefano PROLA
Manager Safety and Flight Ops EUR IATA

Overview
SMS > why SPIs
Measuring Safety
Examples

SMS Basics
Safety is dependent on this System

working
SMS requires organizations to make
data-based decisions and actions
Dependent on the collection and analysis of correct
information
Must understand the system as a whole and the risks
associated with it
Shift of responsibility for safety risk mitigation

Proactive safety: from theory to practice

29 October 2015

Fundamental Shift
Reactive to proactive management

Prescriptive to performance based management and

oversight

Proactive safety: from theory to practice

29 October 2015

Shift to Performance Based

Management
New requirements
New roles
New needs

Proactive safety: from theory to practice

29 October 2015

Performance Based Management

In context of operation
Domestic / International / Regional
Wide-body / Narrow-body
Bulk load / Container load
Cargo / Pax / Combi
Risk tolerance of the organization
Complexity of the operation

An alternative means to prescriptive compliance

Proactive safety: from theory to practice

29 October 2015

Why?
Proactive management
Technology improvements
Vast majority of accidents / incidents related to
technology / human interface
Improvements in the understanding of

human factors

Drill into causal factors

Proactive safety: from theory to practice

29 October 2015

SMS purpose
Safety Management processes identify hazards with the

potential to adversely affect safety ~ ICAO SMM

Purpose is to improve safety performance
Also requirement for continuous improvement, system
must continue to evolve

Proactive safety: from theory to practice

29 October 2015

ICAO SMS Framework - Annex 19

Safety Policy and Objectives
1.1 Management commitment and responsibility
1.2 Safety accountabilities
1.3 Appointment of key safety personnel
1.4 Coordination of emergency response planning
1.5 SMS documentation
Safety Risk Management
2.1 Hazard identification
2.2 Risk assessment and mitigation

Proactive safety: from theory to practice

29 October 2015

ICAO SMS Framework Annex 19

Safety Assurance

3.1 Safety performance monitoring and measurement

3.2 The management of change

3.3 Continuous improvement of the SMS
Safety Promotion
4.1 Training and education
4.2 Safety communication

Proactive safety: from theory to practice

10

29 October 2015

Measuring Safety
Performance
You cannot manage what you dont
measure
How do you measure safety performance?

Definitions
Safety ~ the state in which risks associated with aviation

activities, related to, or in direct support of the operation

of aircraft, are reduced and controlled to an acceptable
level - ICAO

Safety Performance ~ a States or Service Providers

safety achievement as defined by its safety performance

targets and safety performance indicators - ICAO SMM
Ed3 Effective 2013

Proactive safety: from theory to practice

13

29 October 2015

Definitions
Safety Performance Indicator ~ a data based safety

parameter used for monitoring and assessing safety

performance - ICAO SMM

Performance Measures ~ Metrics (or values) that are

set as a target (usually a number or rate) in order to

measure the level of operational performance being
achieved - IATA Reference Manual for Audit Programs
(IRM)

Proactive safety: from theory to practice

14

29 October 2015

IOSA Reference
ORG 3.2.1 The Operator should have

processes for setting performance

measures as a means to monitor the
operational safety performance of the
organization and to validate the
effectiveness of safety risk controls.
[SMS] (GM)
1 September 2016 should > shall

Proactive safety: from theory to practice

15

29 October 2015

Performance Measurement and

Monitoring
Dependent on all SMS framework

requirements met
Managing operational risks
Maintaining regulatory compliance

Proactive safety: from theory to practice

16

29 October 2015

Measuring Safety Performance

Set measureable (SMART) safety objectives
Track performance
Compare against targets
Achievement of a target > improvement in performance

Proactive safety: from theory to practice

17

29 October 2015

IOSA examples
Flight operations (e.g., landing tail strikes, unsatisfactory

line or training evaluations)

Operational control (e.g., fuel diversions due to fuel)
Engineering and maintenance (in-flight engine
shutdowns, aircraft component/equipment failures)
Cabin operations (inadvertent slide deployments)
Ground handling (aircraft damages due to vehicles or
equipment)

Proactive safety: from theory to practice

18

29 October 2015

SMS 3rd phase Doc 9859 SMM

Develop high-consequence SPIs and associated targets
and alert settings.
Typically focused on number of serious accidents /
incidents
High profile
Easy to measure
Reactive

Does not expose systemic issues or hazards

Proactive safety: from theory to practice

19

29 October 2015

Proactive Shift
SPIs measure performance of safety controls
Shift focus to precursors of serious events

Proactive safety: from theory to practice

20

29 October 2015

SMS 4th phase Doc 9859 SMM

Develop lower-consequence SPIs and associated
targets/alert settings
Enhance the safety data collection and processing
system to include lower-consequence events.
Establish lower-consequence safety/quality indicators
with target/alert level monitoring as appropriate.
Reach an agreement with the State oversight authority
on lower-consequence safety performance.

Proactive safety: from theory to practice

21

29 October 2015

Data
Must measure leading and lagging indicators
Leading indicator ~ Metrics that provide information on

the current situation that may affect future performance

Lagging Indicator ~ Metrics that measure safety events
that have already occurred including those unwanted
safety events you are trying to prevent
Established for any operational or maintenance area

Proactive safety: from theory to practice

22

29 October 2015

Example ~ Long Landing (leading)

Identify touchdown points of ALL flights

Proactive safety: from theory to practice

23

29 October 2015

Example Unstable approach paradox

Extract from Data Report for Evidence-Based Training
While airline Standard Operating Procedures (SOPs)
mandate a go-around if an approach is unstable, data
indicates that landing from an unstable approach may be
less risky.
Landings that follow an unstable approach are usually
uneventful (98% land, 90% uneventful)
Missed approaches are rarely handled well
Proactive safety: from theory to practice

24

29 October 2015

Summary
SPIs not driven to simply get a good score
Must improve overall safety performance
Process approach focusing on inputs and their direct

impact on output(s)
Results to be regularly reviewed and communicated
Can be established for virtually any operational or
maintenance area
Must be challenging and result in enhancing the SMS as
a whole

Proactive safety: from theory to practice

25

29 October 2015

IATA Activity
Creation of SPI Guidance Material
Sample SPIs
Supporting methodology
Assist organizations in developing the means to verify

their safety performance against meaningful safety

objectives
Facilitate understanding of the need to look beyond
events

Proactive safety: from theory to practice

26

29 October 2015

Thank you!

Proactive safety: from theory to practice

27

29 October 2015