Rootcauseanalysismasterplan 150120235926 Conversion Gate02

Failure
Is Trying to Tell Us Something
+
18.0 Root Cause Analysis
Root Cause Analysis is the method of problem solving that
identifies the root causes of failures or problems. A root cause is
the source of a problem and its resulting symptom, that once
removed, corrects or prevents an undesirable outcome from
recurring.
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016 842

18. Root Cause
+ The Notion of Root Cause Analysis 843
Symptom: The result or outcome of the problem.

An observation
§ We have late additions to the release that break the software
§ We have “core” defects that should been caught long before
production release
§ We make changes to software, stored procedures, or the
database only to discover it was a mistake.
§ We make promises to the customer before assessing the
impact on our resources or the technical difficulty
Problem:
§ Test coverage insufficient the detect latent bugs in software
§ We commit before understanding the consequences
Root Cause:
§ No software structure to determine test coverage or change
impacts on baseline.
§ No detailed understanding of our capacity for work and
productivity of our technical staff
Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

18. Root Cause
+ Four Root Causes for Program Cost 844
and Schedule Overruns
Unrealistic Performance Expectations,

missing Measures of Effectiveness (MOE)
and Measures of Performance (MOP).
The Lens of Performance

Unrealistic Cost and Schedule estimates
based on inadequate risk adjusted growth
Assessment
Unanticipated
models.
Cost and
Schedule
Inadequate assessment of risk and
Growth
unmitigated exposure to these risks
without proper handling plans.
Unanticipated technical issues without

alternative plans and solutions to maintain
effectiveness. “Borrowed” with permission from Mr. Gary
Bliss, Director, Performance Assessment and
Root Cause Analysis (PARCA), Office of
Assistant Secretary of Defense for Acquisition,
Technology and Logistics.

18. Root Cause
+ Our Path to Better Root Cause 845
Analysis
n Principles of Root Cause Analysis
n Understanding the weaknesses in our current method
n Introduction the Apollo Method
n Steps to applying Apollo
n Transition from the current method to Apollo
ISO/IEC 17025:2005 (4.11.2) ‒ The procedure for corrective action

shall start with an investigation to determine the root cause(s) of the
problem.

18. Root Cause
+ Evaluation of Significance 846
n Can the nonconformity

recur or does it raise
doubt about compliance
of the laboratory’s
operations with its own
policies and procedures?
(ISO/IEC 17025, Clause
4.9.2)
n What criteria is utilized to

determine significance?
(ISO/IEC 17025, Clause
4.9.1.b)

18. Root Cause
+ A Simple Perspective of Problem 847
Solving
n Every Problem in our lives
has three basic elements
Action
connected through Cause
causality.
n Each Effect, has at least two Effect
causes:
Condition
n An Action Cause
n A Condition
This Cause and Effect relationship is the core principle of

effective problem solving

18. Root Cause
+
Ignorance is a most wonderful thing.
It facilitates magic.
It allows the masses to be led.
18.1 Beyond the
Conventional It provides answers when there are none.
Wisdom of It allows happiness in the presence of
danger.
Problem
Solving All this, while the pursuit of knowledge
can only destroy the illusion. It is any
The common approach to problem
solving is to categorize causes or wonder mankind chooses ignorance?
identify causal factors and look for
root causes within the categories.
Categorization schemes do not reveal Original Quote from George Bernard
the cause and effect relationships Shaw
needed to find effective solutions.
It is the effective solution we are after. In Apollo Root Cause Analysis: Effective
Solutions to Everyday Problems, Every
Time, Dean L. Gano

848
18. Root Cause
+ The Persistent Problem … 849
We can not solve problems by using the same kind of

thinking we used when we created them.
— Albert Einstein
Increasing use of larger and more complex systems
potentially results in greater number of problems,
evidenced by the symptoms of unstable software, latent
defects, unexpected performance issues.
Many of these problems and their symptoms, have a serious
impact on business operations than ever before. In many
case these problems are also more difficult to solve.
— A Management System for the Information Business,
Edward Van Schaik
18. Root Cause
+ … and the Persistent Unsuccessful 850
Solution to recurring problems.
In every human endeavor, a critical component to success is

the ability to solve problems.
Unfortunately, we often set ourselves up to fail with our
problem–solving strategies and our inherent prejudices.
We typically rely on what we believe to be common sense,
storytelling, and categorizing to resolve our problems.
Conventional wisdom has us believe that problem solving is
inherent to the subject at hand.
— Dean L. Gano, The Apollo Method

18. Root Cause
+ Some sources of Guidance for Root 851
Cause Analysis†
§2.4.5.1 Event Management – is the process that monitors all events
that occur through the IT infrastructure to allow for normal operation
and also to detect and escalate exception conditions.
§2.4.5.2 Incident Management – concentrates on restoring the

service to users as quickly as possible, in order to minimize business
impact.
§2.4.5.3 Problem Management – involves root–cause analysis to

determine and resolve the cause of events and incidents, proactive
activities to detect and prevent future problems/incidents and a
Known Error sub process to allow quicker diagnosis and resolution if
further incidents do occur.
† §2.4.5. Processes within Service Operation, ITIL V3 Service Operation

18. Root Cause
+ Root Cause Analysis 852
n A structured approach to investigating and analyzing of significant

adverse events or system deficiencies and their required
improvement.
n Root Cause Analysis provides information and tools to be

incorporated into risk management, quality management,
independent verification and validation and improvement
procedures in order to:
n PREVENT future occurrence of adverse events that cause or can cause
harm to individuals; and,
n CORRECT practices that have led to identified deficiencies

18. Root Cause
+ Do’s and Don’ts of Root Cause 853
Analysis
n Use RCA When … n Do Not Use RCA …
n The cause of a system defect n To review every incident or

or failure needs to be system failure
identified so the cause can be n Only symptoms, failures, and
eliminated in the future. incidents that impact business
n The symptom and problem are functions, Service Level
recurring Agreements, maintenance,
operations, security, and
n There are repeated failures integrity of the system in the
ascribed to human error, future.
process errors, technical
n When the reason for the failure
failures.
is obvious
n If it is obvious – fix it.

18. Root Cause
+ Root Cause 854
n Direct causes often result from another set of causes, which could be
called intermediate causes, and these may be the result of still other
causes.
n When a chain of cause and effects is followed from a known end-

state, back to an origin or starting point, root causes are found.
n The process used to find root causes is called root cause analysis ---
systematic problem solving.
n A root cause is an initiating cause of a causal chain which leads to an

outcome or effect of interest.

18. Root Cause
+ 855
Discovering Root Cause starts with …

18. Root Cause
+ The Five Whys 856
Five Whys involves holding meetings immediately – in our case the Friday
immediately following the release.
The problems can be anything – development mistakes, infrastructure
performance issues, process failures, or even internal missed schedules.
Any time something unexpected happens, we start the root cause analysis
with the 5 Ways in the domain from the chart above.
Process – how do the process not catch the problem?

Tools – where in the path did the tool not catch the problem?
Training – what training was missing that created the problem?
Environment(s) – how did the environment fail to protect the baseline or
the production system?
Communications – what communications created the problem?
Management – how did management fail in its actions?
The 5 Whys NEVER seek blame. But are exclusively focused on

discovering the root cause that creates the problem, evidenced by the
symptom
18. Root Cause
+ Five Steps in the Five Whys 857
n Invite all affected parties
n Select the meeting leader

(someone training in RCA)
n Ask Why 5 times for each

topic area
n Assign responsibilities for

collected actual factual data
n Publish the results
Start this process with the planned release of 29 January.

18. Root Cause
+ Why Root Cause Analysis is Hard 858
n Many problems are poorly defined.

n A systematic approach is not used to classify problem and cause.
n Investigations are stopped prematurely – moving on the next problem.
n Decisions are based on guesses, hunches or assumptions.
n An inadequate level of detail is used to get to the real root cause.
n Interim containment fixes are sometimes allowed to become
"permanent."
n The skills, knowledge and experience needed to uncover the root
cause are not available.
n Lack of organizational will to address the bigger issues
n Fear of being blamed
n I really don’t have time for this, we have bigger problems to solve
18. Root Cause
+ Testing Answers from the 859
Five Whys
n What evidence is there that this cause exists?
n Is it concrete?
n Is it measurable?
n What evidence is there that this cause could lead to the observed effect?
n Are we merely asserting causation?
n What evidence is there that this cause actually contributed to the problem?
n Even given that it exists and could lead to this problem, how do we know it wasn't
actually something else?
n Is anything else needed, along with this cause, for the stated effect to occur?
n Is it self-sufficient?
n Is something needed to help it along?
n Can anything else, besides this cause, lead to the stated effect?
n Are there alternative explanations that fit better?
n What other risks are there?

18. Root Cause
+ n A common Root Cause Analysis

approach is to start with a narrative of
what happened and how we think that
18.2 Understanding undesirable state was achieved.
the Weakness
of our Current n This is a storytelling approach.
Storytelling uses linear language and
Approach to linear thinking
Root Cause n Stories start in the past, while causal
Analysis† relationships start in the present
The overriding theme of traditional n Stories are linear while causal
Root Cause Analysis is the focus on
the Root Cause. We can eliminate the
relationships follow the branches of
problem if we eliminate the Root an infinite set
Cause. n Stories use inference to communicate
This assumes the causal relationships meaning, while problems are known
are linear and that problems come
from a single source. by sensed causal relationships
† Apollo Root Cause Analysis: Effective Solution's to Everyday

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016 Problems, Dean L. Gano 860
Root
+ Cause Analysis is Not about Story Telling
Stories seldom identify causes because they are busy setting the stage for
who was where and when some action occurred.
A story is a sequence of events starting in the past, leading to the
consequences disguised as a root cause 861
18. Root Cause
+ Core Failure of Story Telling and 862
the Filling Out of Forms

n Stories rely on experience and judgment of the authors to connect
the causes of the problem. The mapping between Event, Cause, and
Effect not provided in the story narrative.
n Story telling can be used to document the investigation and

describe the corrective actions. But stories are poor in providing the
analytical connections between cause and effect.
n Measures of the effectiveness of corrective actions can not be

provided by narratives. Traceability between Effect, Action, and
Condition can not be provided by the narrative.
n It is a false premise that analysis of a problem, its causes, effects,

and solutions can be reduced to filling out a form and checking
boxes.

18. Root Cause
+ Story Telling Is Not Good Root Cause 863
Analysis Approach
n Story telling describes an event by relating people (who), places
(where), and things (what) in a linear time frame (when).
n When using storytelling to analyze an event (system, outage for

example), the causes identified in the report are difficult to follow
and hinder our ability to understand the relationships between all
the causes and provide a critique of the analysis.
n The investigators may well understand all the causal relationships,

but because they are not presented causally it is difficult to know
these relationships.
n Peer reviews will result in more questions because of the missing

connections between Primary, Intermediate Effects, and their
Causes.

18. Root Cause
+ Problem with the Story Telling Approach 864
to Root Cause Analysis
n Stories start with the past – we saw this happen and something else
happened after that, and then something else happened…
n Causal relationships leading to the Root Cause start with the present
and work backwards to the causes – both Activities and Conditions of
this cause.
n Stories are linear – they come from the minds of the story tellers,
usually as a linear time line.
n The linear understanding of an event in a time sequence from past to
present, ignores the cause–and–effect principle.
n Since we do not understand the branched causes, we use our own
understanding of cause rather than the actual causal connections.
n Stories use inference to communicate causes.

n Causal relationships require clear evidence of the existence of each
cause and its connection to the effects.
18. Root Cause
+ Root Cause Analysis is the 865
Event, the Cause, and the Resulting Effect

n We need a structured approach to investigating and analyzing
significant adverse events or system deficiencies and their required
improvement – not based on Story Telling.
n We need an approach that provides information and tools to be

incorporated into risk management, quality management,
independent verification and validation and improvement
procedures in order to:
n PREVENT future occurrence of adverse events that cause or can cause
undesired performance of our systems.
n CORRECT practices that have led to identified deficiencies.
n This approach separates story telling from the Primary Effect, and
the cause–effect chain leading to the Primary Effect.

18. Root Cause
+ A Better Approach to Root Cause Analysis for 866
Primary Effect, Cause, and Effect

n Direct causes often result from another set of causes – the
intermediate causes – and these may be the result of still other
causes.
n This chain of cause and effect needs to be revealed in a way that

clearly points to the corrective actions.
n When a chain of cause and their effects is followed from a known

end–state (time now), back to an origin or starting point, root causes
are revealed and corrective actions can be applied.
n A root cause is an initiating cause of the causal chain which leads to

an outcome or effect of interest.

18. Root Cause
+ Why Root Cause Analysis is Hard 867
n The problem is poorly defined.

n Systematic approach is not used to classify problem and cause.
n Investigations stopped prematurely – moving on to next problem.
n Decisions based on guesses, hunches or assumptions.
n Inadequate level of detail used to get to the Primary Effect.
n Interim containment fixes sometimes allowed to become
"permanent.”
n Skills, knowledge, and experience needed to uncover the root
cause not available.
n Lack of organizational will to address bigger issues.
n Fear of being blamed.
n I really don’t have time for this, we have bigger problems to solve.

18. Root Cause
+ The Problems with Categorical 868
Thinking
n We need to put order to the things we perceive.
n This is a natural process, but creates laziness in our thinking

processes.
n The notion of good and bad is categorical thinking at its base level.
n Categorical thinking creates the believe that once categorized, we

can establish relationships, and act on the according to other
perceived solutions.
n Filling out root cause forms or assigning elements to Fishbone

charts reinforces the perception we can put the rot causes in
categories (boxes) and assign solutions.

18. Root Cause
+ The Real Problem with Categorical 869
Thinking
n When interacting with others, we assume there is a single reality
and therefore their categories are like ours.
n They are not.
n We assign value that establishes our basis of understanding and

prejudices.
n If this is not recognized there is danger these prejudices set us up

for failure when trying to produce an effective solution to the root
cause.

18. Root Cause
+ Testing Answers from the Five 870
Whys Question Stream

n What evidence is there that this cause exists?
n Is it concrete?
n Is it measurable?
n What evidence is there that this cause could lead to the observed effects?
n Are we merely asserting causation without evidence?
n What evidence is there that this cause actually contributed to the Primary
Effect?
n Even given that it exists and could lead to this problem, how do we know it wasn't actually
something else?
n Is anything else needed, along with this cause, for the stated effect to occur?
n Is it self–sufficient?
n Is something needed to help it along?
n Can anything else, besides this cause, lead to the stated effect?
n Are there alternative explanations that fit better?
n What other risks are there?
18. Root Cause
871
To navigate the path to the actual

Root Cause, we need to connect
the Action and Condition
causes to the primary Effect and
all the Intermediate Effects in
a single picture revealing the
corrective actions that prevent
the Primary Effect in the future.
18. Root Cause
18.3 The Apollo

Method
Nothing happens without a cause.
Every time we ask WHY we must find
at least two causes – the Action and
the Condition in which that action
causes the effect.

872
18. Root Cause
+ Principles of the Cause and Effect 873
Map
For each Primary Effect we need ask why that Effect
occurred.
n Causes are never part of a Linear Chain found in standard Fishbone
diagram or narrative approach.
n Look for causes to create the effect. Two causes are needed for each
Effect.
n Conditions – may exist prior to the Effect. Or conditions may be in motion or
active during the Effect. Conditions are the causes often ignored or beyond our
knowledge.
n Actions – momentary causes that bring conditions together to cause an effect.
Actions are causes most easily recognized.
n Connect all causes (Actions and Conditions) with a Caused By phrase to

either an action or a condition.
n Support each Cause with evidence or an answered question.

18. Root Cause
+ Five Steps to using the Five Whys 874
n Invite all affected parties to

contribute to the map.
n Select the leader (someone

trained in RCA)
n Ask Why 5 times for each

topic area in the Cause and
Effect map.
n Assign responsibilities for

collected actual factual data.
n Publish the map.
Start this process using Apollo now.

18. Root Cause
+ The Apollo Principles 875
n Cause and Effect are the same thing

n If we look closely at cause and effect, we see that a “cause” and an
“effect” are the same thing.
n A single thing may be both a cause and an effect.
n They differ only by how we perceive them in time.
n Each effect has at least two causes in the form of actions and
conditions.
n This is the most important and overlooked principle of causation.
n Unlike storytelling used to capture the Fishbone style charts, which
focuses on linear action causes, reality demands that each effect have at
least one action cause and one or more conditional causes.

18. Root Cause
+ Cause and Effect are the Same 876
Thing
Effects Cause
Injury Caused by Fall
Fall Caused by Slipped
Slipped Caused by Wet Surface
Wet Surface Caused by Leaky Faucet
Leaky Faucet Caused By Seal Failure
Seal Failure Caused by Not Maintained
n The cause of one thing becomes the effect when you ask why.
n The cause of the “Injury” was a “Fall”, and when you ask why “Fall”,
it changes to an effect and the cause is “Slipped.”
n This relationship continues as long as we continue to ask why.

18. Root Cause
+ Each Effect Has At Least Two Causes in 877
the form of Actions and Conditions

n Primary Effect – is any effect
we want to prevent
n Action – momentary causes that
bring condition together to
cause an effect
n Conditions – the fundamental
causal element of all that
happens. It is made up of an
effect and its immediate causes
that represent a single causal
relationship.
n As a minimum, the causes in this
set consist of an action and one or
more conditions.
n Causal sets, like causes, cannot
exist alone.
n They are part of a continuum of
causes with no beginning or end,
which leads us to the next
principle:
n Causes and Effects are Part of
an Infinite Continuum of
Causes.
18. Root Cause
+ Causes and Effects are Part of an 878
Infinite Continuum of Causes

n Causes are not linear.
n They branch out into at

least two causes each
time we ask why of an
effect and if we ask why
of each of those causes
we find an ever
expanding set of causes
as shown in the example
to the right.

18. Root Cause
+ An Effect Exists Only if Its Causes Exist 879
in the Same Space and Time Frame

n Cause-and-effect
relationships exist with
or without the human
understanding
n We perceive them
relative to time and
space.
n Every causal
relationship is made up
of conditional causes
with a history of
existence over time
combining with an
action cause in some
defined time frame and
existing in the same
space to create an
effect.
18. Root Cause
1. Define the Problem
18.4 Four Phases of 2. Create the Cause and Effect Chart

the Apollo
3. Identify effective solutions
Method
These four phases are the basis of 4. Implement the best solutions
discovering the corrective actions for
the undesirable Effects we see in our
development, testing, and
deployment efforts.

880
18. Root Cause
+ Four Phases of the Apollo Method 881
1. Define the problem 3. Identify effective solutions

• What is the problem? must
• When did it happen? • Prevent recurrence
• Where did it happen? • Be within our control
• What is the significance of • Meet our goals and
the problem? objectives
2. Create the Cause and Effect 4. Implement the best solutions
chart • Measure the effectiveness of
• For the primary Effect, ask these solutions in units
Why did this happen defined in the Action and
• Look for causes in Actions Condition causes
and Conditions
• Connect all the causes with
Caused By for the next
cause and its effect
• Support causes with
evidence or an open
Question

18. Root Cause
+ No Fishbone Charts or Narratives 882
Allowed in the Apollo Method

18. Root Cause
+ The Flaw of our Linear Thinking 883
Process
n Like a string of dominos, asking why in the conventional Five Whys
method assumes, A caused B, B caused C, and C caused D.
n At the end of this chain we believe the Root Cause of the

undesirable outcome can be found.
n In the traditional Fishbone approach we are looking for the event

that caused the Effect.
n Instead we need to find the Actions and Conditions that ALLOWED

the event to happen.
n These Actions and Conditions are the actual Root Cause.

18. Root Cause
+ The Principles of the Apollo 884
Method
n Cause and Effect are the same thing.
n If we look closely at cause and effect, we see that a “cause” and an
“effect” are the same thing.
n A single thing may be both a cause and an effect.
n They differ only by how we perceive them in time.
n Each Effect has at least two causes in the form of Actions and
Conditions.
n This is the most important and overlooked principle of causation.
n Unlike storytelling used to capture the Fishbone style charts, which
focuses on linear action causes, reality demands that each effect have at
least one action cause and one or more conditional causes.
n Causes and Effects are part of a continuum of causes.
n An Effect exists only if its causes exist in the same space and time
frame.
18. Root Cause
+ Cause and Effect are the Same 885
Thing
Effect Caused by Action or Condition
Injury Caused by Fall
Fall Caused by Slip
Slip Caused by Wet Surface
Wet Surface Caused by Leaky Faucet
Leaky Faucet Caused By Seal Failure
Seal Failure Caused by Seal Not Maintained
n The cause of one thing becomes the effect when you connect
caused by.
n The cause of the “Injury” was a “Fall”, and when you ask why “Fall”,
it changes to an effect and the cause is “Slip.”
n This relationship continues as long as we continue to ask why.
18. Root Cause
+ An Effect Exists Only if Its Causes Exist 886
in the same Space and Time Frame

n Cause–and–effect
relationships exist with or
without the human
understanding.
n We perceive them relative
to time and space.
n Every causal relationship
is made up of Conditional
causes with a history of
existence over time,
combining with an Action
cause in some defined
time frame and existing in
the same space to create
an effect.

18. Root Cause
+ The Apollo Method structures our 887
current information collection method

to identify solutions to the Root Cause
Every time-series entry in our current
narrative method is an ACTION cause.
By focusing on ACTION’s and not
associated CONDITION causes, we
leave out important causes that can be
acted on to provide an effective
SOLUTION.

18. Root Cause
+ Example of a Naïve RCA 888
Why?
Action Cause?
Condition Cause?
Without following the Why’s to the terminal node with Action and Condition,
the actual Root Cause is still buried in the narrative, waiting to reoccur
18. Root Cause
+ Actual System Outage Root Cause 889
Analysis using Apollo Method
Each stopping condition points to the Root cause.

Until these are found only symptoms are
discovered, not the Root cause

18. Root Cause
+ Missing Elements of Success 890
n Linear thought process

n A caused B, B caused C, C caused D.
n No causal chain from Primary Effect to related Cause and Effect
n No Actions and Conditions to connect to the Effect

n Under what conditions was the Effect observed?
n What actions triggered the Effect?
n No evidence for each Action and Condition
n Stopping too soon, before actual Root Cause found
Every Effect is caused by momentary Action coming

together with existing Condition in the same time
and space.
18. Root Cause
+ 1. Define the Problem.
2. Determine the Causal Relationships.
3. Provide a Graphical Representation of

Cause and Effect that is not linear
thinking.
4. Provide Evidence for each Cause and

Effect.
18.5 Seven Steps to 5. Determine if each Cause is Sufficient and
Discovery Necessary.
Effective problem solving and
Strategies for business success that
6. Identify Effective Solutions THIS IS
move away from blame finding and WHAT WE’RE AFTER.
linear thinking of Fish Bone diagrams. § Finding the cause is needed.
And move toward finding the § Preventing the effect is needed.
interconnected factors where Cause
and Effect are intertwined. § But installing an effective solution is
the desired business outcome.
These Seven Steps expand on the
Four phases, to further detail the
process of arriving at the Root Cause 7. Implement And Track the Effective
Solutions.

891
18. Root Cause
+ Seven Steps of the Apollo Method 892
1. Define the Problem.
2. Determine the Causal Relationships.
3. Provide a Graphical Representation of Cause and Effect that is not

linear thinking.
4. Provide Evidence for each Cause and Effect.
5. Determine if each Cause is Sufficient and Necessary.
6. Identify Effective Solutions

This is what we’re after in Root Cause Analysis
§ Finding the cause is needed.
§ Preventing the effect is needed.
§ But installing an effective solution is the desired business outcome.
7. Implement And Track the Effective Solutions.

18. Root Cause
+ An Effective Solution … 893
n Prevents recurrence of the Primary Effect.
n Assures corrections and prevents actions within our control.
n Meets our goals and objectives, including a solution that …

n Does not cause unacceptable problems.
n Prevents similar occurrences.
n Provides reasonable value for the cost.

18. Root Cause
+ 894
A Recent Example of Cause and Effect Analysis

18. Root Cause
18.6 An Ugly Truth

About Root
Causes
The truth?
You can’t handle the
Truth!

895
18. Root Cause
+ In the end it usually comes down to the 896
People. The People are the Root Cause
This does NOT and CAN NOT mean a Blame Game.

If we ask properly people will see their role in the failure of the process.
Human beings cause problems, not tools, processes, systems, or culture.
18. Root Cause
+ This is our core problem† 897
n Getting the staff engaged in Preventing problems not just Correcting

problems means
n If I do this what will happen?
n Is this the right thing to do at this time?
n Is this really what the customer wanted me to do?
n Did I consider the impact of my actions?
n If I don’t have time to be careful, then what damage will result from my
actions?
n Did I consult with someone who knows more than I do about what the
solution should look like?
n Am I being as careful as I should be, when I make a change?
n Am I being pressured to do something I know isn’t the right thing to do?
If the people change, the culture will change

† IEEE 13th Annual Workshop on Human Performance / Root Cause / Trending / Operating Experience / Self Assessment, August 26-31,
2007, Monterey Marriott, Monterey, CA
18. Root Cause
+
n Critical Elements
n Infrastructure
Management Buy-In and Support

18.7 Effective n
Problem n Crating a Program Champion

Solving
Dedicated Incident Investigators
Culture n
Every organization is different, so n Integration

before implementing this plan you
should determine who the players
are and what level of training they n Implementation Strategies
need.
n Deployment
http://www.realitycharting.com/training/problemsolvingculture/plan

898
18. Root Cause
+ Critical Elements 899
The Work Force should
n Exposed to the principles of causation to understand that “stuff” does

not just happen.
n Should know that we can find effective solutions to event-based

problems by using RealityCharting® and RC Simplified™.
n Must understand that different perspectives are a key to effective

solutions and easily accommodated when you use the Reality
Charting process.
n Must know their role in defining problems and finding effective

solutions to prevent recurrence.
n Must know that management is behind this initiative.

18. Root Cause
+ Infrastructure 900
The Plan musty implement
n Top-level management support.
n A Program Champion.
n Dedicated Incident Investigation Facilitators.
n Incorporation of the Reality Charting process into existing

procedures and protocol.
n Involve every employee in this effective problem-solving initiative.
n Utilize RealityCharting®, the RC Learning Center™, and RC

Simplified™ throughout the organization.

18. Root Cause
+ Management Buy-In and Support 901
Top level support of the RCA process is done
n Show the Effective Problem-Solving processes to top-level managers

so they know the principles of effective problem-solving.
n Show RealityCharting® Overview processes to all managers so they

know what the software does and why it is so effective.
n Read the RealityCharting – Seven Steps to Effective Problem Solving

and Personal Success and work the interactive exercises in this book.
n Provide a manager’s workshop on Managing Effective Problem-

Solving. Discuss application to Defect Elimination, Continuous
Improvement programs
n Six Sigma, Lean, Proactive Maintenance, Chronic (Systemic) issues.

18. Root Cause
+ Create a Program Champion 902
The Program Champion is the Cheer Leader for the RCA

Improvement initiative
n Qualifications
n An experienced incident investigator and facilitator.
n Training
n Familiar with RCA concepts
n Familiar with Apollo method
n User of reality Charting®
n Role
n The Go To persons for all things RCA
n Effectively facilitate incident investigations

18. Root Cause
+ Dedicated Incident Investigator 903
n Identify stakeholders ‒ who have to solve problems as part of their

daily work scope.
n System or Process Engineers
n Supervisors or Team Leaders
n Some Managers
n Training ‒ online training takes ½ the time classroom training does
n Mentoring ‒ program champion and investigators mentor

practitioners and support them to develop skills for conducting Root
Cause Analyses

18. Root Cause
+ Integration 904
n Entry Points ‒ Identify problem-solving entry points by reviewing

the work processes.
n Problems can occur anywhere and evaluation includes what the actions of
any employee should be if they identify a problem.
n This may include Non-Conformance Reports, Corrective Action Reports,
and Customer Feedback Forms.
n Minor Problems ‒ If problem is below the threshold criteria for

performing a full-blown RCA, use Simplified process to document
the problem.

18. Root Cause
+ Implementation Strategy 905
n Initial Findings ‒ If the initial problem analysis provides effective

solutions, implement them according to existing procedures and
approval protocol.
n Send the final analysis to the Program Champion for review and approval and
put into the organizations tracking and trending system and/or print, or
otherwise transmit the final copy to stakeholders who may need it.
n The Program Champion will review to determine if the problem exceeds the
threshold criteria for a formal investigation and respond accordingly.
n This review provides the Program Champion oversight of how well the
program is working and when and where to adjust it.
n Incomplete Analysis ‒ If the event-analysis does not find effective

solutions, then send the initial analysis to Program Champion
n The Program Champion will determine if the problem exceeds the threshold
criteria for a formal investigation and respond accordingly.
n The Program Champion determines if analysis can be finished or if a team is
needed to work the issue.
n If a team is required, Program Champion will gather team and work with them
to finalize the analysis.

18. Root Cause
+ Deployment 906

18. Root Cause
n Do just enough Root Cause Analysis

18.7 Root Cause sessions
Analysis in
Agile Software n Have a knowledgeable facilitator
Development n Communicate the corrective actions
Root Cause Analysis is used in
software development to build a n Do Agile Root Cause Analysis
shared understanding of a problem
to determine the first or “root”
causes.
Knowing these causes identifies
effective improvement actions to
prevent similar problems in the
future.
Root Cause Analysis in Agile stops
problems that have been inhibiting
the team for too long.

907
18. Root Cause
+ Do Just Enough RCA 908
n For any problem to be investigated, the loss or potential for loss

must be significant to the business or the project
n The definition of significant is held by those paying never by those
providing the solution
n The loss needs to monetized or defined in some unit of measure
meaningful to the decision makers
n There must exist a significant probability that similar problems will

occur in the future if no corrective action is taken

18. Root Cause
+ Knowledgeable Facilitator 909
n The team itself cannot produce a credible Root Cause Analysis
n Facilitation of the process is needed by a separate party
n This can be:

n Quality Assurance
n Systems Architecture
n Program Management
n Release Management
n The facilitator must follow a process ‒ the Apollo Method is one

shown here
n Tools are needed to capture the information and avoid the narrative
biases also shown here

18. Root Cause
+ Communicate the Corrective 910
Actions
n Finding the Root Cause is just the start.
n Taking corrective actions is next, but that starts with communicating

to all stakeholders what those actions are.
n The communication must show

n The problem
n The Root Cause
n The Corrective Actions
n The expected outcomes in quantifiable measures of effectiveness and
performance

18. Root Cause
Root Causes Have Root Causes

It’s not a matter of asking 5 Whys
You must keep asking until you reach the end of the chain or action and condition

18. Root Cause
“Kto Nie Pamięta

Historie Skazany
Jest Na Jej Ponowne
Preżycie” ‒ George
Santayana
“Those Who Do Not

Remember the Past
are Condemned to
Repeat it” ‒ George
Santayana

18. Root Cause
+ Root Cause 913
Analysis
n Events have two contributors ‒ the condition and the action. Both must be found
before root cause can be determined for the primary effect.
n There is an endless chain of cause and effect, stopping too early is a common
failure mode of Root Cause Analysis
n Formal Root Cause Analysis processes and tools provide information not
available with the narrative approach
n Without determining the Root Cause and suggested solution just address the
symptoms. This approach does not remove the cause and allows the symptoms
to recur in a loop of fix, break, fix.

Rootcauseanalysismasterplan 150120235926 Conversion Gate02

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Rootcauseanalysismasterplan 150120235926 Conversion Gate02

Uploaded by

Copyright:

Available Formats

Failure

Is Trying to Tell Us Something

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016 842

+ The Notion of Root Cause Analysis 843

Symptom: The result or outcome of the problem.

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ Four Root Causes for Program Cost 844

and Schedule Overruns

Unrealistic Performance Expectations,

The Lens of Performance

Unanticipated technical issues without

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ Our Path to Better Root Cause 845

n Understanding the weaknesses in our current method

n Introduction the Apollo Method

n Steps to applying Apollo

n Transition from the current method to Apollo

ISO/IEC 17025:2005 (4.11.2) ‒ The procedure for corrective action

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ Evaluation of Significance 846

n Can the nonconformity

n What criteria is utilized to

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ A Simple Perspective of Problem 847

n Each Effect, has at least two Effect

This Cause and Effect relationship is the core principle of

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ The Persistent Problem … 849

We can not solve problems by using the same kind of

+ … and the Persistent Unsuccessful 850

Solution to recurring problems.

In every human endeavor, a critical component to success is

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ Some sources of Guidance for Root 851

§2.4.5.2 Incident Management – concentrates on restoring the

§2.4.5.3 Problem Management – involves root–cause analysis to

† §2.4.5. Processes within Service Operation, ITIL V3 Service Operation

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ Root Cause Analysis 852

n A structured approach to investigating and analyzing of significant

n Root Cause Analysis provides information and tools to be

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ Do’s and Don’ts of Root Cause 853

n The cause of a system defect n To review every incident or

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ Root Cause 854

n When a chain of cause and effects is followed from a known end-

n A root cause is an initiating cause of a causal chain which leads to an

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

Discovering Root Cause starts with …

Performance–Based Project Management®, Copyright © Glen B. Alleman, 2002 ― 2016

+ The Five Whys 856

Process – how do the process not catch the problem?

The 5 Whys NEVER seek blame. But are exclusively focused on

+ Five Steps in the Five Whys 857

n Invite all affected parties

n Select the meeting leader

n Ask Why 5 times for each

n Assign responsibilities for

n Publish the results

Start this process with the planned release of 29 January.

+ Why Root Cause Analysis is Hard 858

n Many problems are poorly defined.