You are on page 1of 5

VIEWPOINT

The pursuit of better diagnostic


performance: a human factors
perspective
Kerm Henriksen, Jeff Brady

US Department of Health and ABSTRACT systems-oriented approach to patient


Human Services, Agency for
Despite the relatively slow start in treating safety. With the spotlight on recognisable
Healthcare Research and Quality,
Rockville, Maryland, USA diagnostic error as an amenable research topic at system failuresmedication mix-ups,
the beginning of the patient safety movement, communication lapses and wrong-site
Correspondence to interest has steadily increased over the past few surgeriesdiagnostic error seemed left in
Dr Kerm Henriksen,
US Department of Health and
years in the form of solicitations for research, the shadows. Yet diagnostic mishaps
Human Services, Agency for regularly scheduled conferences, an expanding involve both system-related and individ-
Healthcare Research and Quality, literature and even a new professional society. ual components as well as many other
540 Gaither Road, Rockville, Yet improving diagnostic performance factors.
MD 20850, USA;
Kerm.Henriksen@ahrq.hhs.gov increasingly is recognised as a multifaceted It did not take long, however, for con-
challenge. With the aid of a human factors cerned investigators to draw attention to
Received 14 January 2013 perspective, this paper addresses a few of these diagnostic error, raise awareness of it,
Revised 12 April 2013
challenges, including questions that focus on and undertake studies.36 Funding agen-
Accepted 25 April 2013
Published Online First who owns the problem, treating cognitive and cies and foundations have taken notice.
23 May 2013 system shortcomings as separate issues, why Over the past 7 years, the Agency for
knowledge in the head is not enough, and what Healthcare Research and Quality
we are learning from health information (AHRQ) in the USA has supported a
technology (IT) and the use of checklists. To number of diagnostic error conferences
encourage empirical testing of interventions that and has posted a special emphasis notice
aim to improve diagnostic performance, a on its web site, soliciting research on
systems engineering approach making use of diagnostic performance in ambulatory
rapid-cycle prototyping and simulation is care settings. To further increase aware-
proposed. To gain a fuller understanding of the ness, research and education, a new pro-
complexity of the sociotechnical space where fessional society, The Society to Improve
diagnostic work is performed, a final note calls Diagnosis in Medicine, was launched by
for the formation of substantive partnerships the emerging disciplines thought leaders.
with those in disciplines beyond the clinical Diagnostic error is undeniably gaining
domain. respect and recognition as a worthy
research domain. However, despite the
enthusiasm, challenges remain. Two
A few years ago the issue was raised why recent reviews of the literatureone on
diagnostic error had not received much system-related interventions and the
attention compared to other adverse other on cognitive interventionsfound
events that were afforded greater patient a large gap between suggestions and ideas
safety focus.1 Much of the neglect was for interventions and those that had been
traced to the Institute of Medicines operationalised and tested empirically.7 8
(IOM) seminal To Err is Human report.2 To have a lasting patient safety impact,
The reports overarching take-home there is a need to candidly confront and
message was that preventable adverse critically examine these challenges.
http://dx.doi.org/10.1136/ events arose from a complex web of
bmjqs-2013-002387 system factors, not from the failings of
individual clinicians. The report had an WHO OWNS THE PROBLEM?
immediate media impact. Long aware of Just as many physicians have viewed
To cite: Henriksen K, Brady J. the fragmented nature of their profession, system-based failures as an institutional
BMJ Qual Saf 2013;22:ii1ii5. many healthcare leaders embraced a problem, so have healthcare CEOs and

Henriksen K, et al. BMJ Qual Saf 2013;22:ii1ii5. doi:10.1136/bmjqs-2013-001827 ii1


Viewpoint

administrators viewed diagnostic error as an individual limiting our view that improved diagnostic perform-
physician matter. Both views are short-sighted. They ance means more knowledge in the head or solely to
fail to take into account the reciprocal influences and what takes place during a physicianpatient encounter,
interdependencies between imperfect humans and a more encompassing view holds that diagnostic work
their imperfect work environments. With each ceding is distributed across time and placedistributed cog-
part of the diagnostic error equation to the other, nition,13 shared mental models14 and joint cognitive
meaningful communication and collaborative effort systems15 are a few of the related terms usedand is
are stymied. Both own the problem. Physicians need continuously subject to the direct and indirect effects
to be just as concerned about health IT systems that of multiple interactions among providers, specialists,
lack interoperability, add complexity to the workflow, technicians, patients, test results, artefacts, tools, tech-
and introduce usability issues that threaten patient nology, organisational structures and cultures, and
safety as purchasing officers and system administra- local contextual factors as well as shifting health
tors. Likewise, administrators and unit directors policy and sentiment.16 In brief, diagnostic work fre-
should be just as concerned about the host of cogni- quently involves more than a between the ears reve-
tive limitations and the working conditions that facili- lation. For patients, family members and clinicians, it
tate such limitations as the clinicians themselves. In can be a disjointed journey across confusing terrain,
the absence of a meaningful dialogue and a sense of aided or impeded by different agents, with no destin-
joint ownership, it should not be surprising if were ation in sight and few landmarks along the way.
doing fine here is the mindset.
IS SEGREGATION INTO CAMPS A GOOD THING?
WHEN KNOWLEDGE IN THE HEAD IS NOT In much of the emerging literature and conferences to
ENOUGH date, cognitive issues and biases (including perceptual
A distinction between knowledge in the head and and affective biases) and system failures typically are
knowledge in the world was made years ago by Donald treated as separate entities. This division results from a
Norman.9 As a cognitive psychologist, Norman cer- key question that investigators seem to face. Are efforts
tainly appreciated the information processing and to improve diagnostic performance better spent on
storage capacities for which humans are known, but trying to correct the dispositions to cognitive and affect-
knowledge in the head isnt always retrievable when ive biases (eg, premature closure, overconfidence and
needed. When it comes to considering a full range of visceral bias17) and other cognitive short-comings or can
possibilities that are available for making an optimal diagnostic performance more easily be improved by
diagnosis, the full range does not get considered. To system solutions such as decision support systems that
use a term introduced by Simon10 that predates our sidestep concerns about cognitive bias?18 Both camps
current use of the term premature closure, we satis- have their advocates and both approaches have less than
fice instead by expending minimal cognitive effort and sterling track records. Convincing demonstrations of
accepting the first possibility that seems satisfactory. effective cognitive debiasing techniques are few and
Norman argued that our daily and professional lives cumbersome decision-support systems that are poorly
could be made much easier and less error laden by integrated with the clinical workflow have not gained
putting more knowledge out in the world rather than adoption by busy physicians.
relying solely on knowledge in the head. Framing the question this way encourages the need-
Many of the process errors associated with diagnos- less choosing of camps. While camp life may provide
tic investigations can be reduced by visible and access- a sense of easy agreement and unity for its members,
ible information display and tracking systems for there are drawbacks for those who become too com-
up-dating the status of patients referrals, test results fortable in camps.19 The downside can be a distrust of
and follow-up actions. The lead author recently outsiders with alternative views, a disregard for infor-
received a solicitation from a nearby community hos- mation not compatible with prevailing beliefs, and a
pital requesting donations for an electronic white- lot of self-referential endorsement. Taken collectively,
board (census board) and tracking system for their these are not the best qualities for understanding
emergency department. While it is an encouraging system complexity. Yet humans with their cognitive
sign when local hospitals recognise that knowledge in limitations also are capable of remarkable and adapt-
the head is insufficient in todays information- able real-life decision making20 21 and systems with
intensive clinical environments, the transition from their glitches progressively get better in terms of func-
dry-erase boards to electronic boards has not always tionality, interoperability and usefulness. Neither are
taken into account the distributed and social nature of going to disappear: both with their strengths and lim-
clinical work. If disconnected from regular workflow itations inextricably interact and will continue to
patterns and the needs of providers, the usefulness of impact the diagnostic process.
electronic boards likely will be limited.11 12 The putative distinction between cognitive and
Diagnostic work, like other clinical work, is embed- system becomes somewhat spurious when one con-
ded in a greater sociotechnical system. Rather than siders the diagnostic work of a busy emergency

ii2 Henriksen K, et al. BMJ Qual Saf 2013;22:ii1ii5. doi:10.1136/bmjqs-2013-001827


Viewpoint

department with its chaotic mix of system-based, cog- built unintentionally into our most sophisticated and
nitive, affective, perceptual, temporal and variable promising technologies as users interact with them in
patient factors. Of course, both cognitive and system unkind and unforgiving work environments deserve
variables can be manipulated and tested separately, but continued attention. Healthcare organisations,
with both types of variables interacting in the clinical vendors and researchers need to work together, in the
setting, the interaction term in our analyses should be spirit of a learning community, on design, usability
of just as much interest as the main effects. In fact, in and implementation issues. Providers need to be
a multi-site survey of primary care physicians on diag- involved at the earliest stages of design. As a start in
nostic challenges, when respondents were asked spe- this direction, the US Department of Veteran Affairs
cifically about the role of cognitive factors, they has established a usability laboratory to support the
referenced system and patient factors at the same rapid prototyping of new health IT designs, formal
time.22 usability testing and the development of analysis tools
The perils of embracing dichotomies too eagerly are to assess existing technologies.29 The results of risk
ever presentno less for the learned than the uniniti- assessments used to identify the unanticipated and
ated. Of course, they may serve as useful fictions or unintended consequences of health IT need to be fed
labels initially in helping to simplify complex phe- back to vendors. Vendors, likewise, may need encour-
nomena. But dichotomies tend to assert too much, agement and assistance in conducting their own
feeding delusions of understanding when their usability testing and risk assessments, and in under-
overuse impedes it. Instead of serving as convenient standing the broader sociotechnical safety conse-
short-hand labels, they uncritically take on explana- quences of their products. Beyond accessibility and
tory power, serving as causes rather than conse- usability issues that have long been cornerstone con-
quences. Parsing the world into imperfect humans and cerns of the human factors community, is the greater
imperfect systems, into cognitive versus system-based challenge of using health IT in ways that better
research approaches, and into system 1 (intuitive) educate and empower patients to view themselves as
versus system 2 (analytical) modes of thinking,23 24 active partners in their own medical histories, diagnos-
misses much of the human factors work on shared tic work-ups and improved care.30
mental models and distributed cognition cited earlier.
IS THERE A ROLE FOR CHECKLISTS?
WHAT ARE WE LEARNING FROM HEALTH IT? While used in other hazardous industries for decades,
The leveraging and potential benefits of electronic checklists have found their way into healthcare given
health records in helping to improve diagnostic per- successful efforts in reducing bloodstream infections in
formance have been duly noted.25 A few of the possi- the intensive care unit, in reducing surgical morbidity
bilities include providing access to the patients and mortality in diverse global settings, and in
evolving medical history; providing a forward-moving re-engineering the hospital discharge process to decrease
space for documenting patient and clinician assess- avoidable rehospitalisations.3135 More recently, papers
ments, concerns and uncertainties; enabling continu- have appeared calling for the further exploration of
ous updating and rearranging of problem lists; their use in diagnostic work.36 37 To decrease inappro-
providing prompts to aid in the asking of key ques- priate reliance on memory and heuristics and to help
tions that should not depend on memory; tracking curb overconfidence, diagnostic checklist suggestions
test ordering, results and follow-up with patients; and range from general steps well known to residents but
providing feedback on outcomes given that physicians frequently neglected by busy practitioners, to more
and organisations lack systematic mechanisms to learn comprehensive differential lists and those with more
from diagnostic efforts and calibrate their perform- critical possibilities that ought to be considered and dis-
ance.26 But potential benefits are not the same as counted before a final diagnosis is made.
actual benefits. A recent IOM report on health IT Checklist development, use and acceptance come
noted that its adoption and widespread use in the with challenges. Development requires a team of indi-
USA has been slow.27 At the same time, there is viduals or a consensus body that is adept in best prac-
concern that if poorly designed and implemented, tice guidelines and the underlying evidence base, in
health IT can create new hazards and threaten patient the realities of clinical work, in measurement and in
safety in a healthcare delivery environment that is human factors design principles, and has the persever-
already known for its complexity and fragmentation. ance to engage in successive pilot-testing trials and
Of all the hazard categories identified in a govern- improvements. If put together too rapidly, checklists
ment report that examined health IT hazards, soft- can be excessively lengthy, ambiguous, devoid of clin-
ware design and usability issues (eg, difficult ical reality and insensitive to the needs of front-line
information access, difficult data entry, confusing users. Even when well developed and accepted by
information displays, excessive demands on memory, end-users, there is potential for cognitive drift that
confusing feedback to user) were mentioned the most repetition, by itself, seems to induce. Tasks that are
(52% and 49%, respectively).28 The hazards that are repetitive and become routine are performed with

Henriksen K, et al. BMJ Qual Saf 2013;22:ii1ii5. doi:10.1136/bmjqs-2013-001827 ii3


Viewpoint

nominal cognitive resources. If clinicians tune out resources have been encumbered) is the engineering
and use checklists in a perfunctory manner, subtle and mantra. There are many diagnostic challenges that do
unexpected cues to the patients condition may be not require the actual clinical environment or even a
missed. Checklists are largely based on past failures high-fidelity simulation environment, but simply require
and reluctant adherence is no substitute for heigh- a requisite level of functional fidelity; that is, require the
tened sensitivity to other ways a process can fail (of diagnostician or team to process the same cues, the
course, close the barn door appears on the checklist same variable patient conditions, the same incomplete
once the horse has bolted, but has the owner checked information and uncertainty while subject to the same
the loose side-planking in the horses stall?). Finally, constraints and time pressures, make the same decisions,
investigators who have successfully implemented carry out the same actions, and be informed of the same
checklists are quick to tell us that it is not all about consequences as would occur in the clinical setting.42 43
the checklist. A prevailing patient safety culture, team- By engaging in an iterative testassessimprove
work, leadership commitment, well-conceived meas- process in a simulated setting, researchers are more
urement, and attention to implementation, workflow likely to resist the temptation to seek the immediate
and organisational change issues all need to be care- scientific gratification of comparing a premature inter-
fully aligned before checklists can be properly tested. vention with a control group in a resource-intensive
So far, checklists have been most successful with dis- clinical setting and coming up with equivocal
crete, observable tasksthose associated with surgical, results.44 However, once there is satisfaction with the
central venous catheter and discharge procedures. At interventions efficacy, there continues to be a need to
the same time, a certain amount of the diagnostic test the interventions effectiveness in the environment
process involves individual mental activityperceiv- of usethe flesh and blood clinical setting with its
ing, thinking and interpretingthat is less observ- noise, time pressures and interruptions. Similar refine-
able.36 Do these mental activities have discernible ments are likely to be needed as work-system and con-
start- and stop-points for which a checklist could be textual factors that need to be aligned are identified.
used? Other diagnostic pursuits have been charac-
terised as wicked problems16 38 where there is no
clear end goal or path, where a trusted progression of A FINAL NOTE
tests does not exist, where decisions taken lead to new One of the many constructive comments made by the
uncertainties, and where tentative solutions with their papers reviewers makes for a fitting parting message.
known and unknown effects are difficult to evaluate It was observed that most of the funded research and
and compare. A better understanding of the effective published work on improving diagnoses has come
uses and limitations of checklists in diagnostic work is from clinicians. Why has there been a failure to
clearly needed. engage with scientists who are expert in human per-
formance, perception, cognition and decision making
AN ENGINEERING TACTIC TO IMPROVE THE is the question that was raised. While there are excep-
EVIDENCE BASE tions, they are still exceptions. In other patient safety
Hospitals and primary care offices typically are fluid domains, clinicians and human factors professionals
and dynamic places where interruptions, slips in the have joined together, forming integrated teams to
schedule and encountering the unexpected are com- advance the field. The emerging discipline of diagnos-
monplace. While dynamic environments might be ideal tic improvement needs a human factors voicenot
for research aims that are facilitated by observational or just one voice, but a number of voices, and not just
ethnographic approaches, they are less ideal for testing from those that reside in camps. Orthodoxy in safety
the effects of an intervention and safely attributing the research does not serve anyone well.45 To gain a fuller
results to the independent variable of interest without understanding of the interactivity and complexity of
the results also being influenced by contextual and the sociotechnical space where diagnostic work is per-
organisational variables over which investigators have formed, an opportunity exists for clinicians and their
little control. Yet there is a need for prospective empir- human factors counterparts as well as other disciplines
ical testing of approaches that aim to improve diagnostic to form substantive partnerships for the long-term
performance. Simulation and systems engineering work ahead. Until we learn to do this, progress is
approaches are gaining use as a test-bed for health IT likely to be less than desired.
and medical devices.3941 One tactic is to engage in
rapid-cycle prototyping in a simulated setting to test the Competing interests None.
various promising features of an interventions design. Provenance and peer review Not commissioned; externally
Upon assessing the results, improving the prototype and peer reviewed.
testing again, the testassessimprove cycle continues Open Access This is an Open Access article distributed in
until there is satisfaction with the prototypes efficacy accordance with the Creative Commons Attribution Non
Commercial (CC BY-NC 3.0) license, which permits others to
(or it is discarded, if there is dissatisfaction). Fail early, distribute, remix, adapt, build upon this work non-commercially,
not later (at a later stage of development when extensive and license their derivative works on different terms, provided the

ii4 Henriksen K, et al. BMJ Qual Saf 2013;22:ii1ii5. doi:10.1136/bmjqs-2013-001827


Viewpoint

original work is properly cited and the use is non-commercial. Graber ML, eds. Adv in Health Sci Educ 2009;14(Supp 1):
See: http://creativecommons.org/licenses/by-nc/3.0/ 2735.
24 Norman G. Dual processing and diagnostic errors. In: Berner
REFERENCES ES, Graber ML, eds. Adv in Health Sci Educ 2009;14(Supp 1):
1 Wachter RM. Why diagnostic errors dont get any respect 3749.
and what can be done about them. Health Aff 25 Schiff GD, Bates DW. Can electronic clinical documentation
2010;29:160510. help prevent diagnostic errors? N Engl J Med
2 Institute of Medicine. To err is human: building a safer health 2010;362:10669.
system. Washington, DC: National Academy Press, 2000. 26 Schiff GD. Minimizing diagnostic error: the importance of
3 Graber M, Gordon R, Franklin N. Reducing diagnostic errors follow-up and feedback. Am J Med 2008;121(Suppl 5):S3842.
in medicine: whats the goal? Acad Med 2002;77:98192. 27 Institute of Medicine. Health IT and patient safety: building
4 Graber M. Diagnostic errors in medicine: a case of neglect. safer systems for better care. Washington, DC: National
Jt Comm J Qual Patient Saf 2005;31:10613. Academy Press, 2011.
5 Croskerry P. The importance of cognitive errors in diagnosis 28 Walker JM, Hassol A, Bradshaw B, et al. Health IT hazard
and strategies to minimize them. Acad Med 2003;78:77580. manager beta-test: final report. AHRQ Publication No. 12
6 Schiff GD, Hasan O, Kim S, et al. Diagnostic error in medicine 0058-EF. Rockville, MD: Agency for Health Care Research
analysis of 583 physician-reported errors. Arch Intern Med and Quality, 2012.
2009;169:18817. 29 Russ AL, Weiner M, Russell SA, et al. Design and implementation
7 Singh H, Graber ML, Kissam SM, et al. System-related of a hospital-based usability laboratory: insights from a
interventions to reduce diagnostic errors: a narrative review. Department of Veterans Affairs Laboratory for Health
BMJ Qual Saf 2012;21:16070. Information Technology. Jt Comm J Qual Pt Saf 2012;38:53140.
8 Graber ML, Kissam SM, Payne VL, et al. Cognitive 30 Shneiderman B. Leonardos laptophuman needs and the new
interventions to reduce diagnostic errors: a narrative review. computing technologies. Cambridge, MA: MIT Press, 2003.
BMJ Qual Saf 2012;21:153557. 31 Pronovost PJ, Gpeschel CA, Colantuoni E, et al. Sustaining
9 Norman DA. The design of everyday things. New York: reductions in catheter-related bloodstream infections in
Doubleday, 1988. Michigan intensive care units. BMJ 2010;340:c309.
10 Simon HA. The sciences of the artificial. 2nd edn. Cambridge, 32 Pronovost PJ, Needham D, Berenholtz S, et al. An intervention
MA: MIT Press, 1981. to decrease catheter-related bloodstream infections in the ICU.
11 Aronsky D, Jones I, Lanaghan BS, et al. Supporting patient N Engl J Med 2006;355:272532.
care in the emergency department with a computerized 33 Haynes AB, Weiser TG, Berry WR, et al. A surgical safety
whiteboard system. J Am Med Inform Assoc 2008;15:18494. checklist to reduce morbidity and mortality in a global
12 Patterson ES, Rogers ML, Tomolo AM, et al. Comparison of population. N Engl J Med 2009;360:49197.
extent of use, information accuracy, and functions for manual 34 Gawande A. The checklist manifesto: how to get things right.
and electronic patient status boards. Int J Med Inform New York: Metropolitan Books, 2009.
2010;79:81723. 35 Jack BW, Chetty VK, Anthony D, et al. A reengineered hospital
13 Hutchins E. Cognition in the wild. Cambridge, MA: MIT discharge program to decrease rehospitalizations: a randomized
Press, 2006. trial. Ann Intern Med 2009;150:17887.
14 Cannon-Bowers JA, Salas E, Converse S. Shared mental models 36 Ely JW, Graber ML, Croskerry P. Checklists to reduce
in expert team decision making. In: Castellan NJ, ed. diagnostic errors. Acad Med 2011;86:17.
Individual and group decision making. Hillsdale, NJ: Lawrence 37 Schiff GD, Leape LL. How can we make diagnosis safer? Acad
Erlbaum, 1993:22146. Med 2012;87:13538.
15 Woods DD, Hollnagel E. Joint cognitive system: patterns in 38 Rittal HWJ, Webber MM. Dilemmas in a general theory of
cognitive systems engineering. Boca Raton, FL: CRC planning. Policy Sci 1973;4:15569.
Press-Taylor & Francis Group, 2006. 39 March CA, Steiger D, Scholl G, et al. Use of simulation to
16 Wears RL. What makes diagnosis hard? In: Berner ES, Graber assess electronic health record safety in the intensive care unit:
ML, eds. Adv in Health Sci Educ 2009;14(Supp1):1925. a pilot study. BMJ Open 2013;3:e002549.
17 Croskerry P. Cognitive and affective dispositions to respond. 40 Pronovost PJ, Bo-Linn GW. Preventing patient harms though
In: Croskerry P, Cosby KS, Schenkel SM, et al, eds. Patient systems of care. JAMA 2012;308:76970.
safety in emergency medicine. Philadelphia, PA: Wolters 41 Tropello SP, Ravitz AD, Romig M, et al. Enhancing the quality
Kluwer Lippincott Williams & Wilkins, 2009:21927. of care in the intensive care unit: a systems engineering
18 Newman-Toker DE, Pronovost PJ. Diagnostic errorsthe next approach. Crit Care Clin 2013;29:11324.
frontier for patient safety. JAMA 2009;301:10602. 42 Henriksen K, Patterson MD. Simulation in health care: setting
19 Janis I. Victims of groupthink. Boston, MA: Houghton-Mifflin, realistic expectations. J Patient Saf 2007;3:12734.
1972. 43 Rudolph JW, Simon R, Raemer DB. Which reality matters?
20 Klein G. Naturalistic decision making. Hum Factors Questions on the path to high engagement in healthcare
2008;50:45660. simulation. Simul Healthcare 2007;2:16163.
21 Klein G. Streetlights and shadowssearching for the keys to 44 Henriksen K. Opportunities and challenges in the pursuit of
adaptive decision making. Cambridge, MA: MIT Press, 2009. patient safety. In: Carayon P, eds. Handbook of human factors
22 Sarkar U, Bonacum D, Strull W, et al. Challenges of making a and ergonomics in health care and patient safety. Boca Raton,
diagnosis in the outpatient setting: a multi-site survey of FL: CRC Press-Taylor & Francis Group, 2012:1726.
primary care physicians. BMJ Qual Saf 2012;21:6418. 45 Wears RL, Kneebone RL. The problem of orthodoxy in safety
23 Croskerry P. Clinical cognition and diagnostic error: research: time for a reformation. Ann Emerg Med
applications of dual process model of reasoning. In: Berner ES, 2012;60:58081.

Henriksen K, et al. BMJ Qual Saf 2013;22:ii1ii5. doi:10.1136/bmjqs-2013-001827 ii5

You might also like