Safety Science 47 (2009) 10561067

Contents lists available at ScienceDirect

Safety Science
journal homepage: www.elsevier.com/locate/ssci

A method for assessing health and safety management systems

from the resilience engineering perspective
Marcelo Fabiano Costella a, Tarcisio Abreu Saurin b,*, Lia Buarque de Macedo Guimares b
a

UNOCHAPEC (Regional University of Chapec), Rua Quintino Bocaiuva, 390-D, Chapec, SC, CEP 89801-080, Brazil
DEPROT/UFRGS (Industrial Engineering and Transportation Department, Federal University of Rio Grande do Sul), Av. Osvaldo Aranha, 99,
5. andar. Porto Alegre, RS, CEP 90035-190, Brazil
b

a r t i c l e

i n f o

Article history:
Received 12 June 2008
Received in revised form 8 November 2008
Accepted 21 November 2008

Keywords:
Health and safety management systems
Resilience engineering
Health and safety audits
Manufacturing industry

a b s t r a c t
This article introduces a method for assessing health and safety management systems (MAHS) that has
two innovative characteristics: (a) it brings together the three main auditing approaches to health and
safety (HS) the structural approach (which assesses the system prescribed), the operational approach
(which assesses what is really happening on the shop-oor) and the performance approach (which
assesses the results of performance indicators); (b) it emphasizes the resilience engineering perspective
on HS, which takes into consideration four major principles (exibility, learning, awareness, and top management commitment). Such principles underlie seven major assessment criteria, which in turn are
divided into items (e.g. hazard identication from a resilience perspective is an item that belongs to
the criteria of production processes). The items are sub-divided into statements, which are the requirements that should be assessed based on interviews, analysis of documents and direct observations.
Within the 112 requirements proposed, 38 of them have clear links with at least one out of the four resilience engineering principles adopted. The remaining requirements are based on traditional assumptions
underlying the so-called best practices of HS management. The results of the assessment for each item
are expressed by a score on a scale of compliance with the established requirements, ranging from 0%
to 100%. The specic score within that scale is obtained from tables used to assess applications for the
Brazilian national quality award. The MAHS was tested in a case study that was carried out in a factory
that manufactures automobile exhaust systems, located in Brazil.
2008 Elsevier Ltd. All rights reserved.

1. Introduction
Health and safety (HS) management best practices are well
known among the leading companies in the area. Nevertheless,
best practices are generally not applied systematically but rather
normally amount to fragmented actions, in addition to which the
companies which use them have reached a performance plateau
(Amalberti, 2006; Mitropoulos et al., 2005).
Thus, advances are needed in terms of concepts and methods to
ameliorate this state of affairs. The traditional HS management
best practices analyze people, technology and the work context
separately by means of focusing on a sociological, technological
or organizational approach. The socio-technical approach brings
these ways of focusing together and emphasizes their interfaces,
and thus furnishes an analysis which is closer to the complex reality of mutual interactions and adaptations between people, technology and work (Clegg, 2000).
* Corresponding author. Tel.: +55 51 3223 8009; fax: +55 51 3308 4007.
E-mail addresses: costella@nostracasa.com.br (M.F. Costella), saurin@ufrgs.br
(T.A. Saurin), lia@producao.ufrgs.br (L.B. de Macedo Guimares).
0925-7535/$ - see front matter 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ssci.2008.11.006

The socio-technical vision served as a basis for the development

of principles and concepts of cognitive systems engineering (CSE),
which is a cross-disciplinary approach for the design of complex
socio-technical systems, being concerned with the study of how
joint cognitive systems (a human machine ensemble that cannot
be separated) perform, rather than cognition as a mental process
(Hollnagel and Woods, 2005). Differently from behaviourist approaches, CSE takes into account all the complexity that exists in
the organizational environment in which human behaviour takes
place. In CSE, humans are no longer regarded as simply deterministic inputoutput devices but as goal-oriented creatures who actively select their goals and seek relevant information
(Rasmussen, 1983) i.e. both adaptations and the role of the context are core issues. A set of key CSE principles for the design of safe
and adaptive work systems was proposed by Jens Rasmussen (Rasmussen et al., 1994; Rasmussen, 1997), whose ideas have been further developed under the label of resilience engineering (RE),
which is a term adopted by a number of studies to refer to CSE
applications on safety management (Hollnagel, 2006; Hollnagel
and Woods, 2006; Leveson et al., 2006). This approach has been
adopted in different domains (e.g. surgery and reghting), even

M.F. Costella et al. / Safety Science 47 (2009) 10561067

though some studies use neither the label of CSE nor RE, such as
that by Montagna and Ferrari (2006).
The challenge for HS management in the context of RE is to
draw up prevention strategies which adequately address complex,
dynamic and unstable systems. In particular, strategies are needed
which adequately take account of system variations which cannot
be totally foreseen at the design stage. Therefore the challenge is to
construct dynamically stable systems, with a view to ensuring that
adaptations, despite their being necessary at any given moment,
allow for the system to remain under control (Hollnagel, 2006).
Although RE has been mostly studied in the context of complex
systems of high risk, such as in the aviation, petro-chemical and
nuclear power industries (Hollnagel, 2006), its concepts also tend
to be benecial for manufacturing industry, especially for medium-sized and large industries positioned in highly competitive
supply chains, such as the automobile sector. This occurs since
manufacturing also possesses characteristics of complex systems,
such as high interdependence between processes (most strongly
felt in environments of just-in-time production), the huge number
of variables and various trade-offs to be managed (Christoffersen
and Woods, 1999).
Given that all control systems tend to deteriorate over time or
become obsolete as a consequence of changes, the continuous performance measurement is essential for HS management, whether
or not under the RE paradigm. Such measurement can occur at different levels, such as individual workstations, individual management processes or at the level of the HS management system
(HSMS) as a whole. A particular type of measurement, which is
dealt with in this study, is auditing. In fact, auditing goes beyond
measurement, since it explains performance and builds on measurement to identify gaps between current and desired performance, to identify where there are problems and needs, and to
provide information that can be used in developing actions plans
to improve performance (Chiesa et al., 1996). The European Foundation for Quality Management (EFQM, 1995) adds that auditing
means a comprehensive, systematic and regular review of an organizations activities and results referenced against a model of business excellence.
Currently, the structural approach is the one most used to audit
HSMS, as it is based on the analysis of documents which prove that
the organization is meeting certain HS requirements which it itself
has dened or which are dened by standards, like OHSAS 18001
(occupational health and safety assessment systems). The structural emphasis is typical of most current safety audit tools, and is
concerned with assessing if there is a documented HSMS (Ahmad
and Gibb, 2004). As to the operational approach, it veries if the
documented HSMS has in fact been implemented in practice, by
means of observations and interviews with the companys operational and management staff. There is also the approach of auditing
by performance, based on analyzing the results of normally reactive performance indicators (Cambon et al., 2006).
The literature indicates a series of limitations to the existing
models for auditing HSMS, such as (Bluff, 2003; Le Coze, 2005):
(a) non-specication of the context in which the HSMS implicit
in the models is valid or more effective; (b) focus on relation to
imminent risks of accidents, so giving little importance to the latent hazards to HS in the long term; (c) the items assessed are static, with there being no guidelines for their continuous adaptation
to the dynamic conditions of modern work environments this is
partly a result of not making explicit what HS principles underlie
the audit model; and (d) strategic and cultural dimensions which
interfere in HS management are not considered.
Although there are no models for auditing which explicitly embrace the principles of RE, some studies have re-interpreted models which originally did not take account of RE, with a view to
verifying the extent to which its principles were indirectly borne

1057

in mind (Hale et al., 2006). Other auditing models, such as the Dupont system for process safety management (Dupont, 2006), focuses on behaviour-based safety. This has been criticized for not
emphasizing the capacity of workers to adapt and for not having
an impact on incidents which occur without there being any safe
or unsafe behaviour by workers on the shop-oor (Hopkins,
2006). Other auditing models do not adopt explicit assumptions
about what HS management philosophy should underpin the system, as for example ISRS international safety rating system (Eisner and Leger, 1988), CHASE complete health and safety
evaluation (Chase, 2006) and MISHA method for industrial safety
and health activity assessment (Kuusisto, 2001).
Considering this context, this study aims to present a method
for assessing HSMS (MAHS) with the focus on RE. Such a method
takes account of the three main approaches to auditing HSMS
and was tested in a case study in a factory which manufactures
motor vehicle exhaustion systems. Even though the MAHS might
be considered as an audit tool, according to the denition of an
audit that was previously given, the A of the acronym MAHS
stands for assessment rather than auditing. This choice was made
since the term audit is often narrowly interpreted as looking for
non-compliance with and deviations from regulations and standards, such as OHSAS 18001.

2. Resilience engineering
Resilience engineering is a paradigm for safety management
that focuses on how to help people to cope with complexity under
pressure to achieve success (Resilience Engineering Network,
2008). Due to this fact, a distinctive feature of RE is its emphasis
on understanding how success is obtained, how people learn and
adapt themselves by creating safety in an environment which
has faults, hazards, trade-offs and multiple objectives (Hollnagel
and Woods, 2006). Wreathall (2006) also associates resilience with
the ability of an organization to keep, or recover quickly to, a stable
state, allowing it to continue operations during and after a major
mishap or in the presence of continuous signicant stresses. Thus,
resilience includes both the property to avoid failures and losses, as
well as the property to respond effectively after these have
occurred.
The applications of RE are particularly suitable for high risk systems with complex characteristics, such as (Christoffersen and
Woods, 1999): (a) the high degree of inter-connection between
the components of the system, the consequence of which is the
very great difculty which the operator has to foresee the effects
of his actions and the rapid propagation of the errors; (b) uncertainty and variability. Under such conditions of complexity, the
attributes associated with RE grow in importance.
Considering that there is no one set of RE principles which is
widely accepted in academic circles and also that there are differences in the terminology adopted by different authors, for the purposes of this article, there was a need to endeavour to compile a set
of principles which would serve as a reference for the assessment
method now put forward. It is worth stressing that RE principles
may be used at any level of aggregating the cognitive system, ranging from the focus of a single worker at his workstation to the focus
of the organization as a whole. Thus, based on various studies (Rasmussen, 1997; Hollnagel and Woods, 2005; Hale and Heijer, 2006;
Wreathall, 2006; Saurin et al., 2008) four principles were identied
which have interfaces with each other and do not possess strictly
dened limits:
(a) Top management commitment: this implies demonstrating
a devotion to HS above or to the same extent as the companys other objectives.

1058

M.F. Costella et al. / Safety Science 47 (2009) 10561067

(b) Increase exibility (exibility): a basic assumption of RE is

that human errors are inevitable because of individual and
organizational pressures (e.g. workload and cost) (Rasmussen et al., 1994). Therefore, work system design must be
exible, recognizing that variability management is as
important as variability reduction. In fact, design should
support the natural human strategies for coping with hazards, rather than enforce a particular strategy. This means
that no more than what is absolutely essential should be
specied in the design of jobs (Clegg, 2000). This implies
studying what people actually do and then considering
whether it is possible to support that through design (Hollnagel and Woods, 2005). For instance, a mechanism to comply with this principle is to design error-tolerant boundaries
(Rasmussen et al., 1994). Wreathall (2006) also emphasizes
that exibility requires that people at the working level (particularly rst-level supervisors) should be able to make
important decisions without having to wait unnecessarily
for management instructions.
(c) Learn from both incidents and normal work (learning): RE
emphasizes understanding normal work rather than just
learning from incidents, in order to learn and to disseminate
successful working strategies. Nevertheless, learning
requires an organizational environment that encourages
the reporting of incidents and recognizes adaptive strategies,
although it will not tolerate culpable behaviours (Wreathall,
2006). Also, learning must take into account the way procedures are implemented. In fact, monitoring the implementation of procedures should be considered as important as
devising procedures, since it may contribute to reducing
the gap between work as imagined by managers and work
as performed by front-line operatives. The smaller this gap,
the greater the evidence that learning is taking place (Wreathall, 2006; Hale and Heijer, 2006).
(d) Be aware of system status (awareness): this principle
implies that actors should be aware both of their own current status and the status of the defences in the system. This
is critical for anticipating future changes in the environment
that may affect the systems ability to function (Resilience
Engineering Network, 2008). Awareness is also important
for the assessment of the trade-offs between production
and safety (Hale and Heijer, 2006; Hollnagel and Woods,
2005). Rasmussen et al. (1994) suggest two broad
approaches for implementing this principle: performance
measurement based on proactive indicators and the design
of visible boundaries of performance.
Besides these four principles, another which permeates them
can be highlighted. This is proactiveness which refers to anticipating problems, needs or changes, and which leads to actions being
drawn up which directly alter the surroundings of the environment. In terms of HS, proactiveness refers to anticipating hazards
and control measures so as to interrupt the evolutionary course
of incidents.
Of course, the RE principles presented overlap somewhat with
principles of other safety management paradigms (and even with
generic management principles, such as the well-known idea of
top management commitment) and they are fully in line with
the general principles for the design of socio-technical systems
established by previous studies, such as that by Clegg (2000). For
example, there is an overlap with the perspective of safety culture
taken by Reason (1997), since he agrees with the stance that safety
culture might be engineered and managed and it should encompass four subcomponents: a reporting culture, a just culture, a exible culture and a learning culture. In fact, Reason (2001) included
safety culture principles in a checklist aimed at assessing what he

referred to as the institutional resilience of airlines. Nevertheless,

this article assumes that the characteristic of RE which distinguishes it most from other paradigms is that it emphasizes the positive side of safety (i.e. understanding how adaptive strategies
ensure safe and productive work), although it does not neglect
learning based on incidents. Moreover, it is based on the assumption that resilience is a property of a system that may be consciously designed and managed, even though the development of
a strong RE framework in terms of concepts, principles and methods is still an on-going process (Woods and Hollnagel, 2006).

3. Research method
The selection of the elements assessed by the MAHS was made
based on the standards OHSAS 18001 and ILO-OSH 2001 (guidelines on occupational safety and health management systems), as
well as based on a review of the literature which covered three
areas: HSMS, RE and HSMS audits. An exploratory case study to assess a HSMS conducted in an agricultural equipment factory also
contributed to constructing the audit model. The elements assessed by MAHS can be classied into criteria and items. The former correspond to the large categories of assessment and the
items to the sub-categories, which, for their part, consist of the
requirements to be assessed. Thus, 28 items and seven criteria
were established.
Since the framework of the MAHS was dened, it was applied
and assessed by means of a case study, carried out over a 3 months
period in 2007. The case study took place in the Brazilian company
which is the market leader for replacing automobile exhaust systems. It was chosen because of the ease of access which the
researchers had to it, as well as on account of its size (450 employees) and integration with a highly competitive supply chain, which
were indicative of characteristics of complexity. The company produces exhaust pipes and accessories, such as protective caps,
crankcase guards, tow bars and trailers for motor vehicles, ranging
from automobiles to tractors. The main clients are the sales outlets
for spare parts for all models, there being 12,000 clients in Brazil
and Latin America.
Throughout 2005, 2006 and 2007, the company went through
nancial difculties as a result of an investment in a new line of
products that produced a disappointing return. This negatively
inuenced HS in terms of there being a shortage of resources for
the area. Despite this, the company holds an ISO 9001 certicate.
The workforce consists predominantly of men from the rural zone,
which requires substantial investment in training events.
The companys portfolio boasts more than 850 products, with
most of its components being manufactured in the same plant in
which the nal products are assembled. This contributes to the frequent change of dies and tools, to the existence of functional layouts and difculties in standardizing processes, which makes the
production process more complex.
Three examiners applied the MAHS in the case study: one of
them was the rst author of this article, who was designated as
the lead examiner (LE) and is a certied auditor of quality management systems by the Brazilian foundation for the national quality
award. The other two examiners were selected in accordance with
their knowledge of the area of HS. One of these examiners, deemed
examiner 1 (E1), is a civil engineer and was completing a specialization course in safety engineering. The other, deemed examiner
2 (E2), is a business administrator and was completing a masters
degree in industrial engineering with emphasis on ergonomics.
Due to the disparity of knowledge about RE among the examiners,
the LE fostered a levelling up of these concepts using the MAHS
assessment instrument which was fully discussed. The three examiners took part in all the stages of the case study jointly. Taking into

1059

M.F. Costella et al. / Safety Science 47 (2009) 10561067

account only the collection of eld data, the examiners made a

weekly 4-h visit over 2 months, a total of eight visits and 32 h
per examiner in the company. The activities undertaken by the
examiners throughout the case study correspond to the application
of all the stages of the MAHS, which are described in the following
section of this article. At the end of the case study, interviews were
conducted with some of the companys representatives who had
accompanied the study, so that they could express their opinions
about the MAHS and the application procedures.
After concluding the case study reported in this article, another
application of MAHS was undertaken in a small-sized school for
training pilots for civil aviation (Carim Jr. et al., 2008). On the
one hand, this study indicated that all items assessed by MAHS
were relevant and potentially necessary for the school of aviation.
On the other hand, various terms and requirements established by
MAHS were unfamiliar to the aviation schools instructors and
managers and therefore needed to be claried (for example, the
meaning of personal protective equipment was not clear, nor were
they familiar with concepts of HSMS). In fact, this is a reection of
the fact that the MAHS was designed for the context of manufacturing industry, so that adaptations of terms, among others, may
be necessary to make it viable to apply it in other contexts.
4. Method for assessing health and safety management systems
(MAHS)
4.1. Criteria and items of the MAHS
In Fig. 1, the criteria and items of MAHS are presented as well as
the association between each item of the MAHS and both the RE
principles and the requirements of OHSAS 18001, with additions
from standard ILO-OSH 2001.
An overview of the scope of each item is presented below:
(1) HSMS planning
(1.1) HSMS policy and objectives: policy and objectives were
put together in a single item because they are closely inter-con-

nected, since the objectives must be consistent with the policy.

With regard to RE, a requirement of this item is to emphasize continuous improvement, in order not to be complacent about the current situation, even when safety performance is good (Hale and
Heijer, 2006).
(1.2) HSMS planning: this item has requirements such as establishing priorities for HS planning and establishing tasks, resources,
deadlines and responsibilities for achieving the HS objectives.
(1.3) Structure and responsibility: this encompasses bureaucratic issues, such as dening HS responsibilities throughout all
hierarchical levels of the organization and establishing mechanisms to let all those involved in it be aware of their HS
responsibilities.
(1.4) Documentation and records: this has requirements such as
a way of easily nding out and tracking documents which are distributed to company personnel.
(1.5) Legal requirements: this is a core issue to any HSMS and, at
MAHS, there are also requirements related to preparedness for and
response to emergency, since this is mandatory in accordance with
Brazilian regulations.
(1.6) Top management commitment: this item has requirements such as top management monitoring of HS metrics and
managing production pressures on HS.
(2) Production processes
(2.1) Hazard identication from a traditional perspective: this
checks whether there are mechanisms to identify those hazards
that are easily observable and that are usually emphasized by regulations, such as physical, chemical and biological hazards.
(2.2) Hazard identication from an RE perspective: this item
checks whether there are mechanisms for identifying organizational hazards (e.g. production pressures, monotony, excessive labour division), which could be broadly understood as all hazards
that make the traditional hazards riskier than they should be.
(2.3) Risk assessment: this item checks whether there are
mechanisms for prioritising risks based on their severity and
probability.

1.1 HSMS policy and objectives

1.2 HSMS Planning
1 HSMS Planning

1.3 Structure and responsability

1.4 Documentation and records
1.5 Legal requirements
1.6 Top management commitment
2.1 Hazard identification from traditional perspective
2.2 Hazard identification from RE perspective

2 Production
processes

2.3 Risk assessment

2.4 Hazard responses from traditional perspective
2.5 Hazard responses from RE perspective
3.1 Workers' participation

3 People
management

3.2 Training and competence

4.1 Management systems integration

4.2 Management of change

4 General safety
4.3 Maintenence
factors
4.4 Procurement and contracting
4.5 External environment
5 Planning of
performance
monitoring

5.1 Reactive indicators

5.2 Proactive indicators
5.3 Internal audit
6.1 Incident investigations

6.2 Real work investigations

6 Feedback and
6.3 Preventive actions
learning
6.4 Corrective actions
6.5 Management review and continuous improvement
7 Performance

7.1 Reactive performance

7.2 Proactive performance

Fig. 1. Association between the items of the MAHS and the principles of RE and OHSAS 18001 elements, with additions from ILO-OSH 2001.

Worker participation

Continual improvement

Management systems
integration

Procurement e contracting

Management review

Audit

Performance measurement
and monitoring
Accidents, incidents,
nonconformances, and
corrective and preventive
actions
Records and records
management

Emergency preparedness and

response

Operational control

Document and data control

Documentation

Consultation and
communication

Training, awareness and

competence

Structure and responsability

OHS Management Program(s)

Objectives

Legal and other requirements

Planning for hazard

identification, risk assessment
and risk control

OHS Policy

OHSAS 18001 requirements plus ILO-OSH 2001

Awareness

Flexibility

Criteria and itens

Learning

Top management
commitment

RE principles

1060

M.F. Costella et al. / Safety Science 47 (2009) 10561067

(2.4) Hazard responses from a traditional perspective: this

checks whether there is an action plan that is consistent with hazard identication (traditional focus) and risk assessment. For instance, it may be appropriate to check whether the handling of
manual materials is properly dealt with by organizational and
technological preventive measures.
(2.5) Hazard responses from RE perspective: this is a core item
of MAHS since it checks whether there are action plans for dealing
with the hazards identied at item 2.2. For instance, a check is
made on how the difference between real and prescribed work is
managed, whether there are initiatives to design error-tolerant
performance boundaries and whether there are formal guidelines
for carrying out sacricial judgments of production in favour of HS.
(3) People management
(3.1) Workers participation: this is based on the assumption
that workers participation concerning HS issues should imply they
learn and become aware of the boundaries of safe performance.
(3.2) Training and competence: this item emphasizes that
workers should receive training on non-technical skills (e.g. communication, error detection and recovery, development of proactive attitudes). It also requires the integration of HS and
production training.
(4) Generic safety factors
(4.1) Management systems integration: this requires the integration of HS, quality and environmental management systems.
(4.2) Management of change: this item has strong connections
with RE, since it checks whether the organization has mechanisms
for anticipating and managing any changes in the work environment, taking into account their HS implications as early as possible.
(4.3) Maintenance: since maintenance errors are well-known
factors that contribute to mishaps in complex systems (Reason
and Hobbs, 2003), this item has requirements such as that all risk
management tasks mentioned in criteria 2 also be extended to
maintenance.
(4.4) Procurement and contracting: from a RE perspective, this
item is important because it implies anticipating HS issues during
the procurement and contracting of any resources, such as people,
machinery and materials.
(4.5) External environment: in line with the socio-technical approach underlying RE, this item has requirements such as that the
organization have mechanisms for being made aware of threats to
and opportunities for HS imposed by the external environment,
which in turn include socioeconomic, educational, political, cultural and legal aspects (Hendrick and Kleiner, 2001). For instance,
this item checks whether the organization maintains either an
adversary or collaborative relationship with government agencies
which are responsible for enforcing HS regulations.
(5) Planning of performance monitoring
(5.1) Reactive indicators: this item checks what the HS reactive
metrics adopted (e.g. frequency accident rates) are, why they are
used, how they are collected, how they are analyzed and how the
organization learns from the metrics. Also, this requires the existence of both personal and process reactive safety metrics, which
are essential since excellent personal safety performance does
not necessarily imply excellence in process safety performance
(Hopkins, 2009). Indeed, while poor personal safety means that
individual accidents take place, poor process safety means that
hazardous technologies have been damaged.
(5.2) Proactive indicators: this has similar requirements to the
previous item, and emphasizes proactive metrics (e.g. amount of
hours dedicated to HS training). This item also requires mechanisms
to monitor the trade-off between safety and production. As to item
5.1, both personal and process proactive metrics are required.
(5.3) Internal audits: this item is demanded by all major HSMS
standards and it checks how HS internal audits are undertaken,
such as the regularity and use of multiple sources of evidence.

(6) Feedback and learning

(6.1) Incident investigations: this is a well-known item of any
HSMS. However, at MAHS the requirements explicitly state that
any situation of lack of safety should be investigated from a systems perspective.
(6.2) Real work investigations: in sharp contrast with other
HSMS audits, MAHS requires real work to be regularly audited with
the aim of understanding workers adaptive strategies and, consequently, to reduce the gap between real and prescribed work. For
instance, such audits of real work could be undertaken in a similar
way to audits that are conducted in behaviour observation programs. However, rather than the enforcement of rules, which is
the usual focus in behaviour-based safety programs, audits of real
work from an RE perspective should emphasize both an understanding of adaptations to and the identication of workers degrees of freedom.
(6.3) Preventive actions: this item stresses that preventive actions should be documented and monitored as well as aimed at
closing the gap between real and prescribed work.
(6.4) Corrective actions: this item has similar requirements to
item 6.3, though emphasizing corrective actions.
(6.5) Management review and continuous improvement: at
MAHS, distinctive requirements of this item are that learning and
continuous improvement should be based on an understanding
of adaptations and successful performances, based on data provided by item 6.2.
(7) Performance
(7.1) Reactive performance: this item assesses the results and
tendency of reactive HS metrics, as well as how they are benchmarked against external competitors and how they are disseminated throughout the organization.
(7.2) Proactive performance: this item is similar to item 7.1,
though emphasizing proactive metrics.
4.2. MAHS assessment tool
Applying MAHS takes place by means of assessing each item
based on a series of questions about the organizations management
practices. Throughout the questionnaire, in each item, what are
made explicit are the type of assessment approach (performance,
structural or operational) and the sources of evidence recommended
for assessing each requirement, such as: interviews with managers,
interviews with representatives of the HS department, interviews
with workers, analysis of documents and records, and direct observation. With regard to the interviews with workers, it is recommended that it be carried out in groups with a view to reducing the
time taken to collect data, when compared to individual interviews.
The group interview approach has been adopted with good results in
ergonomics assessment studies by the laboratory responsible for
this research, with typically 30% of the total number of workers being
interviewed, in groups of, at most, eight workers (Saurin and Guimares, 2008). It is recommended that the sample of workers interviewed is diversied, including both the most experienced and those
who have been recently hired. The sample should also cover people
from various departments (should this not be possible, focus on the
departments at greatest safety risks) and the interview should be
conducted without the presence of the HS managers and staff.
It is important to emphasize that the scripts to be followed in the
interviews correspond to the very requirements laid down in the
detail of the guidelines for the MAHS. For example, in order to hold
the interviews with the workers, it is necessary for the examiner to
select all the paragraphs which contain the indication that the
interviews with workers are a recommended source of evidence.
The requirements in each paragraph were drawn up in accordance with the point of view of the examiner, with a view to facilitating his/her work. For example, in item 1.1, paragraph a, the exact

1061

M.F. Costella et al. / Safety Science 47 (2009) 10561067

(a) to highlight what the participative approach of the workers

is like. To check what the workers degree of involvement is in
improving everyday safety at work, by highlighting the modalities
of participation, whether they are more active or passive;
(b) to highlight if the workers opinions are observed in the context of the work process design (awareness and learning).

4.3. Summary of the sources of evidence of the MAHS

Fig. 2 presents a summary of the sources of evidence required
by MAHS, the sources being divided in accordance with the evaluation approach.
From Fig. 2, it can be seen that the MAHS items are evaluated
based on, at least, two sources of evidence (for example, the item

1.1 HSMS policy and objectives

1.2 HSMS Planning
1.3 Structure and responsability
1.4 Documentation and records
1.5 Legal requirements
1.6 Top management commitment
2.1 Hazard identification from traditional perspective
2.2 Hazard identification from RE perspective
2.3 Risk assessment
2.4 Hazard responses from traditional perspective
2.5 Hazard responses from RE perspective
3.1 Workers' participation
3.2 Training and competence
4.1 Management systems integration
4.2 Management of change
4.3 Maintenance
4.4 Procurement and contracting
4.5 External environment
5.1 Reactive indicators
5.2 Proactive indicators
5.3 Internal audit
6.1 Incident investigations
6.2 Real work investigations
6.3 Preventive actions
6.4 Corrective actions
6.5 Management review and continuous improvement
7.1 Reactive performance
7.2 Proactive performance
Fig. 2. Association between the MAHS items and the sources of evidence for evaluation.

Interviews with sample of

workers

Interviews with OSH

representatives

Interviews with managers

Evidence sources
Operational

Interviews with top

management

Items

Performance indicators
analysis

Documentation and
records analysis

Structural Performance

Direct observation

words are: to highlight what HS objectives have been established by

the company. If this text had the format set by the HSMS standards,
the text might be: the company should establish objectives for HS.
Beside each requirement related to RE, the principle which is
being assessed is presented. Among the 112 paragraphs, 38 (34%)
are directly related to RE. As an example, the following presents
the requirements linked to item 3.1 (workers participation). The
complete questionnaire can be found in the study by Costella
(2008) and other examples of requirements are presented in the
Appendix A.
3.1 Workers participation
Approach: operational.
Sources of evidence: interview with representatives from the
HS department (requirements: a, b) interview with workers
(requirements: a, b):

1062

M.F. Costella et al. / Safety Science 47 (2009) 10561067

of external environment factors) and, at most, ve sources of evidence (for example, hazards response from RE perspective).
Fig. 2 also indicates that the MAHS reconciles in a balanced way
the structural and operational approaches, given that 25 of 28
items adopt both the sources of evidence linked to documentation
(typical of the structural approach) as well as the sources linked to
interviews or direct observation (typical of the operational
approach).

in the margins of the scoring table, from which one obtains the corresponding percentage.
Should there be characteristics in distinct score bands, one
should always opt for the lower score band. For example, when
we consider the following performance for a given item in relation
to the focus: the management practices are adequate for most of
the requirements of the item (line C); almost all the practices are
rened (line E); meeting almost all the requirements is proactive
(line E); innovation is present in some practices (line E). In this
case, the score will be determined by line C, given that in the lines
above, at least, one of the factors would not be met. It is worth
stressing that that the coverage of each assessment factor should
be determined in accordance with the note in Fig. 3: some (less
or equal to 50%), most (greater than 50%), almost all (greater than
75%) and all of them (100%).

4.4. Attribution of scores to the items assessed

MAHS includes the possibility that a score be attributed to each
item, based on the scoring system adopted by the Brazilian foundation for the national quality award (FPNQ, 2006). The scoring tables
adopted for such an award have been widely tested in the practice
of assessing quality management systems in Brazil, and a substantial number of professionals are familiar with them (FPNQ, 2006).
Thus, for each item among the criteria from 1 to 6, the percentage
should be dened which best approximates the reality observed, in
accordance with the assessment factors being focused on (sub-divided into suitability, proactiveness, renement and innovation)
and being applied (sub-divided into advertising, continuity and
integration). The items of criterion 7 of the MAHS (results) should
be assessed based on the factors of relevance, current level and
tendency, also dened by the FPNQ in order to assess the results
of the quality management system.
To attribute the scores, with a view to assessing the focus and
application of the management practices (Fig. 3), the following sequence should be strictly followed: (a) choose the line from Fig. 3
which is best adjusted to the situation observed; (b) choose the
column from Fig. 3 the description of which best approximates
the situation observed; (c) identify the inter-section of the column
with the line selected by means of the letters and numbers located

Dissemination,
continuity and
integration
Suitability,
proactiveness,
refinement and
innovation

Management practices are suitable for all requisites of

the item.
Meeting all requisites is pro-active.
All practices are refined.
A inovao est presente em algumas prticas.
Management practices are suitable for all requisites of
the item.
Meeting almost all the requisites is pro-active.
Almost all the practices are refined.
A inovao est presente em algumas prticas.

4.5. Steps for applying mahs

Six major steps are needed to apply MAHS:
(a) denition of the team of examiners, which should comprise
at least two people, external or internal to the company.
(b) Initial meeting with the following objectives: mutual introductions by the company team and the examiners; explanation of the main objectives, of the items and criteria, as well
as of the MAHS assessment tool; negotiation of the schedule
of the assessment process; checking the level of knowledge
about the concepts of RE.
(c) Application of the assessment tool: the examiners should
certify that all the items have been answered, and should
also be familiar with the prole of the company in order to
better understand the management practices adopted and
to estimate how much the companys characteristics have

Management practices
presented are
disseminated in some
areas, processes,
products and/or by the
pertinent interested
parts.
Continued use in some
management practices.
No evidence of
integration.

Management practices
presented are
disseminated by most of
the main areas,
processes, products
and/or by the pertinent
interested parts.
Continued use in most
of the management
practices.
Some evidence of
integration.

Management practices
presented are
disseminated by the
main areas, processes,
products and/or by the
pertinent interested
parts.
Continued use in
almost all management
practices.
Presented most of the
evidence expected from
integration.

Management
practices presented are
disseminated in almost
all areas, processes,
products and/or by the
pertinent interested
parts.
Continued use in all
management practices.
Presented most of the
evidence expected
from integration.

Management
practices presented
are disseminated in all
areas, processes,
products and/or by the
pertinent interested
parts.
Continued use in all
management
practices.
Presented all
evidence expected
from integration.

10%

30%

50%

70%

90%

100%

10%

30%

50%

70%

80%

90%

Management
practices presented
are not disseminated.
Use not reported.
No evidence of
integration.

Management practices are suitable for almost all

requisites of the item.
Meeting most of the requisites is pro-active.
Most practices are refined.

10%

30%

50%

60%

70%

70%

Management practices are suitable for most of the

requisites of the item.
Meeting some of the requisites is pro-active.
Some practices are refined.

10%

30%

40%

50%

50%

50%

Management practices are suitable for some requisites of

the item.

10%

20%

30%

30%

30%

30%

Management practices are unsuitable for the requisites of

the item or are not reported.

0%

0%

0%

0%

0%

0%

Notes: Coverage: some (less or equal to 50%), most (more than 50%), almost all (more than 75%) and all (100%).

Fig. 3. Framework of the Brazilian foundation for the national quality award for assessment of the factors in focus and being applied.

1063

M.F. Costella et al. / Safety Science 47 (2009) 10561067

in common with the characteristics of complex systems, the

context which gives potential to the utility of the MAHS. A
sequence is proposed for applying the assessment instrument based on the approach (structural, operational and
by performance) and on the respective sources of evidence.
Initially, the structural approach may be analyzed, which is
restricted to analyzing documents and records described in
the sources of evidence for each item of the MAHS. By starting in this way, it is possible to become familiar with the
HSMS idealized by the company in order to check, later, if
it is being put into practice, besides obtaining support to
enrich the interviews. Next, what is suggested is the analysis
following the approach by performance, which consists of
analyzing records which contain the results of the indicators
collected by the company. After the structural and by performance assessment, the operational approach of the HSMS
can be evaluated. This phase consists of direct observation
and of interviews with those involved in the HSMS. It is
important to emphasize that the sequence suggested may
not prove to be the most appropriate because of the particularities of each company or because of the availability of
the interviewees. It is worth pointing out that the sequence
suggested was not followed rigidly during the case study,
but rather than it can be understood as a proposal of an ideal
situation, which was established based on what was learned
during the case study.
(d) Denition of the scoring of each examiner: after collecting
the data, each examiner, individually, should undertake
their evaluation, the product of which is a given score based
on the tables already quoted.
(e) Meeting to reach consensus on dening the nal score:
under the coordination of the lead examiner, with the aim
of such a meeting being to analyze the divergences of evaluation among the examiners. In extreme cases, in which
there is no consensus, the lead examiner should decide the
scoring, and register this fact. In the minutes of the meeting,
the lead examiner should record the items which suffered
modications, report on the discussions and the new scores
obtained by means of consensus.
(f) Preparation and presentation of the assessment report: the
lead examiner should present the nal report on applying
the MAHS to the companys representatives who took part
in the rst meeting, to top management and to representatives of the workers.

5. Main results of the case study

5.1. Scores obtained in each item assessed
The scores obtained in the case study are presented in Table 1,
which indicates that the company investigated aimed, rst and
foremost, to try to full the legal requirements, which demand full
documentation and records.
As an example of the argumentation process needed to attribute
scores, the case of item 3.1 (workers participation) is presented. In
this case, it was observed that those in charge generally pass on the
workers demands to the production manager and to the safety
specialists, there not being any formal system for collecting suggestions. However, the safety specialists report a low take-up of
the workers suggestions because they are related to reducing steps
or excluding requirements of the procedures, which, according the
safety specialists report, are aimed at excluding preventive measures. Besides this, there are the quality control circles (QCC),
which suggest projects, including in the area of HS. However, the
QCC meetings are held outside normal working hours and there

Table 1
Scores obtained in the case study, in accordance with the items of the MAHS.
Items of the MAHS

Score (%)

1.4
1.5
2.4
3.2
4.3
2.1
3.1
5.1
5.3
6.1
6.5
1.2
6.4
1.1
2.5
7.1
1.3
1.6
2.2
2.3
4.1
4.2
4.4
4.5
5.2
6.2
6.3
7.2

70
70
50
50
50
40
30
30
30
30
30
20
20
10
10
10
0
0
0
0
0
0
0
0
0
0
0
0

Documentation and records

Legal requirements
Hazard responses from traditional perspective
Training and competence
Maintenance
Hazard identication from traditional perspective
Workers participation
Reactive indicators
Internal audit
Incident investigations
Management review and continuous improvement
HSMS Planning
Corrective actions
HSMS policy and objectives
Hazard responses from RE perspective
Reactive performance
Structure and responsibility
Top management commitment
Hazard identication from RE perspective
Risk assessment
Management systems integration
Management of change
Procurement and contracting
External environment
Proactive indicators
Real work investigations
Preventive actions
Proactive performance

Average score

20

are no incentives for participation, which is voluntary. The workers

reported that they valued participating in the QCC a lot and believe
that they add value to HS. Generally speaking, the degree of workers participation is limited, given that the only channels of direct
participation are the QCC and a monthly meeting of a safety committee required by regulations. Both of which require management
approval in order to implement suggestions, which, normally, also
demand resources. In this item, management practices showed
themselves to be adequate to some requirements (50%, 1 requisite
of a total of 2), without renement, innovation and proactiveness
(line B of Fig. 3). At the same time, the practices were seen to have
spread to all the main areas, and demonstrated continuity and evidence of integration with other areas, mainly by means of the QCC
(column 3 of Fig. 3). Thus the result was 30%, corresponding to inter-section B3 of Fig. 3.
With regard to the individual scores which the examiners
attributed to each item and the scores from the consensus meeting,
among the 28 items, in 13 of them, the scoring was identical
among the three examiners and, in 10 items, the scoring was equal
for, at least, two examiners. Thus, the scoring was different for the
three examiners in only 5 items, which needed greater effort to obtain consensus, although this was always obtained. With the
exception of only one item, the scoring of the consensus was equal
to the scoring attributed by the lead examiner.
The fairly good uniformity in the attribution of scores is indicative of the consistency of the MAHS assessment tool. According to
examiners E1 and E2, the fact of the requirements being organized
in paragraphs facilitated the attribution of scores. Examiners E1
and E2 reported that the complexity of the assessment of the items
related to RE was greater than for the other items, in addition to
which they also reported the difculty of assessing two aspects
of the scoring system (dissemination and integration of the management practices), because of the greater subjectivity perceived.
It is also worth stressing that examiner E2, who had less experience in HS, showed greater disparity in scoring for the items related to legislation and the structural aspects of the HSMS.

1064

M.F. Costella et al. / Safety Science 47 (2009) 10561067

5.2. Analysis of meeting Re principles

5.2.1. Top management commitment
With regard to the principle of top management commitment,
what was detected was the lack of giving value to HS in comparison with other management functions. One piece of evidence for
this is the subordination of the HS area to a management unit little
related to this issue (information technology) and the lack of structure for the HS department, in which the safety specialists are
overloaded with bureaucratic tasks and dedicate little time to the
activities of planning and control.
As another piece of evidence that HS is not among the priorities
of top management, one of the directors reported that, after the
investments made in implementing and maintaining ISO 9001, resources will be invested in environmental management and, after
implementing ISO 14001, the focus will be on HS. Neither is there
any formal assessment of the HS department, its performance only
being questioned if there is a serious or fatal accident. In fact, the
workers reported that top management only takes an interest in
HS when an accident happens. Moreover, the comment was also
made by representatives of the HS department that, due to the
companys nancial difculties in recent years, the large-scale
changes needed have not been turned into concrete reality, with
only demands needing minor nancing being carried out.
5.2.2. Learning
With regard to the principle of learning, observations are not
undertaken to assess how safety procedures are met and by means
of these to monitor the differences between real work and prescribed work. The assessment made it clear that the main function
of the procedures is bureaucratic, as they do not form a document
linked to the dynamics of real work. On being hired by the company, the workers receive training from the HS department based
on the content of the procedures. This training is later reviewed,
with the addition of more details of the function by the ofcerin-charge in the sector where the worker will work. Thereafter,
the worker gets used to carrying out duties with the help of a more
experienced colleague who works alongside the newly contracted
person for approximately 1 month, until the latter acquires sufcient knowledge and experience to operate the machine alone.
According to the report of the workers interviewed, after initial
training, they do not consult the procedures again, but only follow
the verbal instructions given by the ofcer-in-charge. Should they
have any doubts, the procedure is not consulted, but rather the
ofcer-in-charge is. Therefore, the worker does not acquire the habit of checking the written procedures, given that he is instructed
not to waste time on this but rather to ask the ofcer-in-charge,
who, for his part, does not conduct audits to check if the procedures are in fact being complied with. Although this may indicate
that the differences between prescribed and real work are great, it
was not possible to specify the magnitude of this difference, for
two reasons: (a) it is not part of the scope of the MAHS to measure
the gap between real and prescribed work, but rather its scope includes verifying if the company has this concern and if it seeks to
close the gap between real and prescribed work and (b) deep
knowledge of each process would be needed to undertake this
measurement.
Apart from this, moments that would be propitious for learning
such as carrying out preventive actions and a critical analysis of the
HSMS have not been duly exploited. With regard to corrective and
preventive actions, they do not emphasize investigation of the
causes of the deviations and are neither fully broadcast in the factory nor formally recorded. Such actions normally arise from intense inspection by the safety specialists instead of coming from
the initiative of others who may intervene. With regard to the critical analysis of HSMS, this is conducted without being based on

reliable and full information, which limits the participants understanding of the safety performance, and consequently, of the
causes of the problems. It is worth stressing that critical analysis
of HS is analyzed supercially by top management as part of the
critical analysis of the quality management system.
Still with regard to the principle of learning, two other opportunities have been wasted: (a) in the investigations of accidents,
since the company operates under the paradigm of the culture of
blaming the worker, which is felt to be insisted on during the process of investigation and (b) in the management of changes, a very
faulty process in the company, since it does not include alterations
in training or procedures should there be changes (for example,
technologies). Thus, changes occur and their impact on HS is perceived late on, which characterizes reactive learning.
5.2.3. Flexibility
Performance in relation to the principle of exibility is ambiguous. As an example of this, in 2006, at the request of the operator of
an overhead crane, members of the HS department inspected the
main supporting cable of the bridge and issued a report declaring
that there was the grave and imminent risk of the cable snapping,
and requesting the immediate stoppage of the equipment, without
consulting the production supervisors. This attitude generated
huge upsets in production on that day and friction between the
production and safety staff because the former were not consulted.
The positive fact of this example was that the safety staffs action
prevailed over the pressures from production staff. At the same
time, this was seen as a circumstantial event, since there are no
guidelines for managing the trade-off between safety and production. Although top management supported the stoppage decision,
they also held that there should be better communication between
HS and production.
Another ambiguity arises from the fact that, both in the procedures and in the discourse of the production manager, it is recognized that a very quick rhythm can generate accidents, although
there are no mechanisms to identify what characterises an excessive rhythm. For example, in the procedure of the pressing operation there is a recommendation not to accelerate production with a
view to making up for lost time after a stoppage, the aim of which
is to prevent accidents. In this example, at least under the structural approach, the principle of exibility is fairly well satised, because of not ceding to the pressures of production, in favour of
safety. Besides this, according to the representative from the production planning and control department, there are no individual
production targets in each workstation. There is only the concern
that the factory as a whole meets the global production targets.
The workers who declared that they did not feel pressurised into
attaining production targets conrmed this.
The deciencies already cited in the management of changes
also indicate that the company is not prepared to respond exibly
to the dynamic environment in which it nds itself. The lack of definition of responsibilities regarding HS is also detrimental from the
point of view of exibility, since, especially during emergencies, it
may not be clear what the channels of assistance are nor do those
who intervene feel obliged to act in favour of HS.
In operational terms, the lack of exibility is evident in the nonexistence of fail-safe devices in machines with the highest risks,
which would make the limits error-tolerant. According to a traditional HS approach, a factor which was emphasized by the managers interviewed was the consideration of requirements concerning
attention, concentration and discipline when selecting skilled
workers who operate machines of greatest risk, such as the presses
in the stamping department, the folding machines and milling.
Nevertheless, this criterion could be less important should the machines of greatest risk have fail-safe devices, as is the case of a single machine which has a system of sensors which switches the

M.F. Costella et al. / Safety Science 47 (2009) 10561067

machine off when something or someone gets close to the hazardous parts.
During the interviews with the managers, situations were also
reported of refusal to work on machines shortly after an accident,
as the member of staff was afraid. According to the production
manager, in this case, the worker was transferred to another function. Similarly, the company has demonstrated concern for staff
who presents health complaints, by immediately arranging for
them to have medical attention and transferring the worker to
activities with less risk of accident. These facts display the autonomy of production management to deal with aspects of HS, which
is one dimension of the principle of exibility.
5.2.4. Awareness
Although the examples of re-allocation of functions are positive
from the point of view of exibility, they are indicative of the lack
of meeting the principle of awareness, since they are evidence of a
belated awareness in which the workers and managers become
aware of the risk after tragic events. Another piece of evidence that
the principle of awareness is characterised by great reactiveness
arises from a change of attitudes with regard to safety on the occasion of hiring new welders coming from a large multi-national
company. Such welders were accustomed to wearing protective
spectacles under the welders mask and demanded that they
should only work under the same conditions. The company supplied the requested equipment, and the other welders, slowly but
surely, also began to use the same equipment. Currently, most
welders use this double protection, which occasioned the reduction of eyes injuries.
Although, as has already been commented on, there is a systematic effort to train the workers, as well as there being a few channels
for their participation, the potential of these mechanisms to support
the principle of awareness could be better exploited. With regard to
the training, it does not tackle management skills, which would favour hazard identication and control, such as critical observation
of their own and their colleagues work, as well as communication
of hazards. As to participation, the existing channels do not include
opportunities for work enrichment, which would lead the worker to
get to know the nature of their activities in more depth.
Nor is HS planned from the early conception of products and
processes, whether because of the lack of support and release of resources by top management or whether because of the lack of
awareness of the hazards. There is a procedure relating to the
development of new dies and tools in the company itself, it having
been foreseen in this procedure that there was a need for approval
by the HS during a pilot test of the new pieces of equipment. Nevertheless, this posture is reactive for it does not require the participation of operators and safety specialists from the initial steps of
the design for new pieces of equipment.

6. Conclusions
The method for assessing health and safety management systems (MAHS) proposed in this article contributes to lling gaps
regarding the assessment of the HSMS. One of these gaps concerns
the deciency of the current audits of HSMS when it comes to reconciling the structural, operational and by performance approaches in a single audit model. Besides this, the MAHS also
contributes to adopting explicitly the focus of RE on HS. With regard to the practical application of the MAHS, what can be stressed
is its use to re-structure the HSMS, and its presenting the main positive and negative points as well as the action priorities. These are
assessed from a focus which, although still little known in the
industry, makes it possible for there to be wide-ranging assessment of the HSMS performance.

1065

Indeed, key characteristics of the MAHS are the broad scope of

hazard identication and its proactiveness, since the established
requirements may be interpreted as guidelines for implementing
improvement measures consistent with the RE approach. The differences between the MAHS and standard-based audits were evident in the case study. Before this study, the only audit in the
company was based on the ISO 9001 standard, which despite
encompassing HS issues mostly relied on the structural approach.
Due to this narrow focus, the ISO 9001 audit indicated good HS
performance, which was in sharp contrast with the results obtained by the MAHS, which indicated low HS performance.
Regarding the MAHS scope of hazard identication, the results
of the case study also indicated that a positive feature was the
detection of failures from strategic issues (e.g. lack of top management commitment) to operational ones (e.g. lack of safeguards in
some equipment). The consideration of generic safety factors was
also benecial, since this provided a broad perspective of responsibilities for HS, which was strongly limited to the HS staff. Even
though the company performed poorly in the items encompassed
by generic safety factors, the simple questioning of these issues
provided insights for the company staff. For instance, it was realized to what extent the nancial crisis the company was going
through was impacting on HS and how the lack of external pressures (e.g. lack of enforcement of regulations by government agencies) in favour of safety contributed to a state of complacency.
Nevertheless, a limitation of the MAHS in terms of scope is that
it looks for the application of RE principles only in the items
encompassed by the assessment tool. Of course, this implies limited exibility, since RE principles might be applied in other
dimensions of the organization that are not encompassed by the
HSMS (e.g. product design). Also, in its current version, the MAHS
does not provide tools for systematically analyzing the correlations
among the items assessed, which could be important because of
the systems approach underlying RE. The case study indicated that
such correlations exist, such as with regard to the following items:
(a) planning the monitoring of performance and reactive and proactive performance and (b) hazard identication, risk assessment
and hazard response. For example, since proactive indicators have
not been planned, the result of the proactive indicators will also be
non-existent. A characteristic such as this indicates that it is possible that the improvement in the result of an assessment based on
MAHS may occur in large increments. These correlations also indicate which items should be prioritized since they are prerequisites
for other items.
Another limitation of the MAHS is the need for both company
representatives and examiners to know about RE, a difculty which
could be tackled by means of drawing up training mechanisms for
examiners (for example, games and manuals with examples of good
practices), with a view to facilitating and making the assessment
process uniform. The difculty of assessment also springs, as it also
does for other audits of management systems, from the MAHS not
indicating how the requirements should be met, and thus what is
required is an auditor with great experience. In fact, since RE is
not yet a fully established discipline, as well as assuming that it is
not being widely disseminated in the industry, it can be considered
that there is not even a sufcient base of knowledge in the literature
for establishing prescriptive requirements.
Concerning the length of time necessary for applying the MAHS,
it took approximately 96 man-hours, without computing the time
needed for data analysis. The fact of MAHS having been applied
over 4 weeks (one-day visit per week), was considered by company
staff as an advantage in relation to the audits of ISO 9001, which
are generally concentrated into a single week, which facilitates
the companys circumstantial preparation to receive the auditors.
It is appropriate to recognize that the scenario of the case study
was a limiting factor for the validation of the MAHS. If the company

1066

M.F. Costella et al. / Safety Science 47 (2009) 10561067

studied had possessed more advanced HS management practices, it

would probably have been possible to have identied a larger
number of good examples of meeting RE principles, which would
have contributed to the renement of the model.
Among the opportunities for future studies arising from what
have been presented in this article, adapting and applying the
MAHS in other contexts stands out, principally in organizations
acknowledged as examples of excellence in HS management,
which have a real interest in improving their performance in this
area and which possess characteristics of complex systems.

such as to stop production should there be an imminent risk

of accidents (exibility);
(d) to highlight when judgments of sacricing production can
be observed (stopping production in favour of HS). To highlight if the procedures include guidelines regarding how to
make these judgments and if there is any indication of the
most likely situations or workstations in which such judgments may become necessary (exibility);
(e) to check if the planned preventive actions take into account the
systemic focus, the macro ergonomic point of view and the
organizational hazards, not only the physical ones (learning).

Appendix A
References
Extracts from the MAHS assessment tool
1.6 Top management commitment
Approach: operational.
Sources of evidence: interview with top management representatives (requirements a, b, c, d, e, f), interview with production
management representatives (requirements a, b, c, d, e, f), interview with representatives from the HS department (requirements
a, b, c, d, e, f) and interview with workers (requirements a, b, c, e):
(a) to highlight if top management participates in HS efforts and
interacts with the interested parties, thus demonstrating
commitment and seeking opportunities to develop the
HSMS (top management commitment and learning);
(b) to highlight if top management is aware of the satisfaction,
motivation and well-being of the workers (top management
commitment and awareness);
(c) to highlight if top management ensures there are resources
for HS (top management commitment);
(d) to highlight if top management shows concern for improving HSMS performance and if there is oversight regarding
the level of the safety performance indicators, especially
the proactive ones (top management commitment and
awareness);
(e) to highlight if top management demonstrates commitment
both to HS and other business functions, such as production,
marketing and sales (top management commitment);
(f) to check the position of top management with regard to the
management of the pressures of production in relation to
safety (top management commitment and exibility).
2.5 Hazard response from an RE perspective
Approach: structural and operational.
Sources of evidence: direct observation (requirements a, d),
analysis of the procedures relative to HS (requirements a, b, d, e),
interview with representatives from top management (c, d), interview with the production manager (requirements a, b, c, d, e),
interview with representatives from the HS department (requirements a, b, c, d, e) and interview with workers (requirements a,
c, d):
(a) to observe how the difference between real and prescribed
work is managed. To describe how the monitoring and modication of the HS procedures are carried out and how the
adaptations carried out by the workers in relation to the
HS procedures are managed (exibility);
(b) to highlight how the procedures indicate what the limits of
safe work are and how to detect faults and regain control. In
addition, to highlight what the mechanisms are so that the
limits are made visible, respected and/or error-tolerant
(exibility);
(c) to check how workers and supervisors on the shop-oor
have autonomy to take decisions which inuence safety,

Ahmad, K., Gibb, A., 2004. Towards effective safety performance measurement
evaluation of existing techniques and proposals for the future. In: Rowlinson, S.
(Ed.), Construction Safety Management Systems. Routledge Published, pp. 425
442.
Amalberti, R., 2006. Optimum system safety and optimum system resilience:
agonistic or antagonistic concepts? In: Hollnagel, E., Woods, D., Leveson, N.
(Eds.), Resilience Engineering: Concepts and Precepts. Ashgate, London, pp.
238256.
Bluff, L., 2003. Systematic Management of Occupational Health and Safety. National
Research Centre for Occupational Health and Safety Regulation, Australian
National University. Working Paper 20.
Cambon, J., Guarnieri, F., Groeneweg, J., 2006. Towards a new tool for measuring
safety management systems performance. In: Proceedings of the 2nd
Symposium on Resilience Engineering, Juan-les-Pins, France, November 810,
2006, France.
Carim Jnior, G., Silva, M., Saurin, T.A., 2008. Auditoria de sade e segurana no
trabalho sob o enfoque da engenharia de resilincia: estudo de caso em uma
empresa de aviao civil. In: XV Simpsio de Engenharia de Produo, Bauru,
Brasil. Anais. Universidade Estadual Paulista Jlio de Mesquita Filho. <http://
www.simpep.feb.unesp.br/anais.php>.
Chase Complete Health and Safety Evaluation, 2006. Chase Evaluation and Audit
System. <http://www.hastam.co.uk/chase.htm> (accessed February 2006).
Chiesa, V., Coughlan, P., Voss, C., 1996. Development of a technical innovation audit.
Journal of Product Innovation Management 13, 105136.
Christoffersen, K., Woods, D., 1999. How complex humanmachine system fail:
putting human error in context. In: Karwowski, W., Marras, W.S. (Eds.), The
Occupational Ergonomics Handbook. CRC Press, Boca Raton, FL, pp. 585600.
Clegg, C., 2000. Sociotechnical principles for system design. Applied Ergonomics 31,
463477.
Costella, M.F., 2008. Mtodo de avaliao de sistemas de gesto de segurana e
sade no trabalho (MASST) com enfoque na engenharia de resilincia. Porto
Alegre, Tese (Doutorado em Engenharia de Produo) Programa de PsGraduao em Engenharia de Produo, PPGEP/ UFRGS.
Dupont (DuPont Safety Resources), 2006. <http://www.dupont.com/safety>
(accessed March 2006).
EFQM European Foundation for Quality Management, 1995. Self-assessment,
1995 Guidelines. EFQM, Brussels.
Eisner, H.S., Leger, J.P., 1988. The international safety rating system in South African
mining. Journal of Occupational Accidents 10, 141160.
FPNQ Fundao para o Prmio Nacional da Qualidade, 2006. Critrios de
excelncia: o estado da arte da gesto para a excelncia do desempenho.
<http://www.fpnq.org.br/criterios_2006.htm> (accessed October 2006).
Hale, A.R., Heijer, T., 2006. Is resilience really necessary? The case of railways. In:
Hollnagel, E., Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts
and Precepts. Ashgate, London, pp. 115137.
Hale, A.R., Guldenmund, F., Goossens, L., 2006. Auditing resilience in risk control
and safety management systems. In: Hollnagel, E., Woods, D., Leveson, N. (Eds.),
Resilience Engineering: Concepts and Precepts. Ashgate, London, pp. 270295.
Hendrick, H., Kleiner, J., 2001. Macroergonomics: An Introduction to Work System
Design. Human Factors and Ergonomics Society, Santa Monica.
Hollnagel, E., 2006. Resilience: the challenge of the unstable. In: Hollnagel, E.,
Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts and Precepts.
Ashgate, London, pp. 817.
Hollnagel, E., Woods, D., 2005. Joint Cognitive Systems: An Introduction to Cognitive
Systems Engineering. Taylor and Francis, London.
Hollnagel, E., Woods, D., 2006. Resilience engineering precepts. In: Hollnagel, E.,
Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts and Precepts.
Ashgate, Epilogue, London, pp. 326337.
Hopkins, A., 2006. What are we to make of safe behaviour programs? Safety Science
44, 583597.
Hopkins, A., 2009. Thinking about process safety indicator. Safety Science 47, 460
465.
Kuusisto, A., 2001. Safety management systems: audit tools and reliability of
auditing. Doctor of Technology Thesis, Tampere University of Technology, VTT
Technical Research Centre of Finland.
Le Coze, J., 2005. Are organisations too complex to be integrated in technical risk
assessment and current safety auditing? Safety Science 43, 613638.

M.F. Costella et al. / Safety Science 47 (2009) 10561067

Leveson, N.G., Marais, K., Saleh, J.H., 2006. Archetypes for organizational safety.
Safety Science 44, 565582.
Mitropoulos, P., Abdelhamid, T., Howell, G., 2005. Systems model of construction
accident causation. Journal of Construction Engineering and Management 131
(7), 816825.
Montagna, A., Ferrari, E., 2006. Improving risk management in complex systems:
from reliability to collective competence. In: Proceedings of Triennial Congress
of the International Ergonomics Association, 16, Maastricht, Netherlands, Delft
University of Technology (CD-ROM).
Rasmussen, J., 1983. Skill, rules and knowledge; signals, signs, and symbols, and
other distinctions in human performance models. IEEE Transactions on Swems,
Man and Cybernetics SMC-13 (3).
Rasmussen, J., 1997. Risk management in a dynamic society: a modeling problem.
Safety Science 27 (2/3), 183213.
Rasmussen, J., Petersen, A., Goodstein, L., 1994. Cognitive Systems Engineering. John
Wiley & Sons, New York.
Reason, J., 1997. Managing the Risks of Organizational Accidents. Ashgate,
Burlington, p. 252.

1067

Reason, J., 2001. Score your safety culture. Flight Safety Australia, JanuaryFebruary.
Reason, J., Hobbs, A., 2003. Managing Maintenance Error. Ashgate, Burlington, p.
183.
Resilience Engineering Network, 2008. Resilience engineering. <http://
www.resilience-engineering.org> (accessed April 2008).
Saurin, T.A., Guimares, L.B.M., 2008. Ergonomic assessment of suspended scaffolds.
International Journal of Industrial Ergonomics 38 (2), 238246.
Saurin, T.A., Formoso, C.T., Cambraia, F., 2008. An analysis of construction safety
best practices from the cognitive systems engineering perspective. Safety
Science 46, 11691183.
Woods, D., Hollnagel, E., 2006. Resilience engineering concepts. In: Hollnagel, E.,
Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts and Precepts.
Ashgate, Prologue, London, pp. 16.
Wreathall, J., 2006. Properties of resilient organizations: an initial view. In:
Hollnagel, E., Woods, D., Leveson, N. (Eds.), Resilience Engineering: Concepts
and Precepts. Ashgate, London, pp. 258268.