Supplement

Annals of Internal Medicine

Simulation Exercises as a Patient Safety Strategy

A Systematic Review
Eric Schmidt, BA; Sara N. Goldhaber-Fiebert, MD; Lawrence A. Ho, MD; and Kathryn M. McDonald, MM

Simulation is a versatile technique used in a variety of health care

settings for a variety of purposes, but the extent to which simulation may improve patient safety remains unknown. This systematic
review examined evidence on the effects of simulation techniques
on patient safety outcomes. PubMed and the Cochrane Library
were searched from their beginning to 31 October 2012 to identify
relevant studies. A single reviewer screened 913 abstracts and selected and abstracted data from 38 studies that reported outcomes
during care of real patients after patient-, team-, or system-level
simulation interventions. Studies varied widely in the quality of

THE PROBLEM
It is well-known, both in medical practice and in other
professions, that error rates decrease with experience (1).
Yet, an important challenge is how to train physicians and
ensure that they maintain competency while minimizing
the potential for patient harm. Many medical educators
now regard the traditional medical training model see
one, do one, teach one as unstructured and inadequate
(2). In contrast, simulation exercises allow patient-safe clinician training. Although all physicians must eventually
perform procedures on and manage critical events for an
actual patient for the first time, simulation can make initial
interactions with patients safer.
Clinical expertise and mastery within a specialty do
not increase simply as a function of experience (3), and
likewise, patient safety issues are not likely to decrease simply as a function of more practice hours. Deliberate practice, or practice that includes reflection on performance,
increases mastery (4), and simulation exercises offer opportunities for deliberate practice with flexibility to adjust procedure complexity and provide regular practice for rare
treatments. Experienced clinicians also must maintain proficiency in a wide array of skills, most of which are known
to deteriorate over time without practice (5). Simulation
can serve to maintain clinical skills and may be part of
maintenance of board certification, as is the case for the
American Board of Anesthesiology (6).
The versatility of simulation techniques affords many
potential benefits to those working to improve patient
safety. First, simulation is designed to match user needs
and has been associated with increased technical perforSee also:
Web-Only
CME quiz (Professional Responsibility Credit)
Supplement
426 5 March 2013 Annals of Internal Medicine Volume 158 Number 5 (Part 2)

methodological design and description of simulation activities, but

in general, simulation interventions improved the technical performance of individual clinicians and teams during critical events and
complex procedures. Limited evidence suggested improvements in
patient outcomes attributable to simulation exercises at the health
system level. Future studies would benefit from standardized reporting of simulation components and identification of robust patient safety targets.
Ann Intern Med. 2013;158:426-432.
For author affiliations, see end of text.

www.annals.org

mance (7) and knowledge acquisition and clinical reasoning (8). Second, simulation can replicate rare, complex, or
high-stakes scenarios known to affect individual and team
performance (9, 10). Third, mistakes are not only allowed
in simulation, they enhance learning through reflection
and debriefing (11, 12). Fourth, new technologies or procedures may be tested in simulation before implementation
in real time with real patients (13). Finally, teams can simulate patient care flow in situ for critical events (14) or
adequacy of new facilities and equipment (15).
However, studies evaluating the relationship between
these benefits and patient safety outcomes, including potential harms, have not been thoroughly evaluated. The
purpose of this systematic review is to examine evidence on
the benefits and harms of using simulation to improve
patient safety in medicine.

PATIENT SAFETY STRATEGIES

Research demonstrating the benefits of simulation
comes from studies about simulation as well as studies using simulation (16). Research about simulation directly examines the effect of a simulation technique as an intervention on behaviors and actions at the health professional or
team level that could directly improve patient safety if that
training were widely implemented. In contrast, studies using simulation harness these techniques as a laboratory to
investigate new technologies and human performance for
insights into potential causal pathways to improve safety.
Simulation is used along the translational pathway
from health care provider actions in simulated laboratory
contexts to similar actions in clinical settings to patient
outcomes (17). As such, simulation is considered a technique rather than any 1 specific technology (18).
Simulation to enhance patient safety has 4 general
purposes: education (for example, in transitioning trainees
from content knowledge to experiential practice, and in
continuing education); assessment (for example, in quality
control or quality improvement, or usability testing); rewww.annals.org

This article has been corrected. The specific correction appears on the last page of this document. The original version (PDF) is available at www.annals.org.
Downloaded From: http://annals.org/ on 02/11/2014

Simulation Exercises as a Patient Safety Strategy

search (for example, regarding clinician behaviors) and

health system integration (for example, team processes)
(19). These purposes are not mutually exclusive, and each
may span a range of complexity. A classic low-fidelity example of partial task training is simulation of intramuscular medication administration by inserting a needle into an
orange. Individual dynamic medical management exercises
may include high-fidelity simulations that utilize anatomically accurate mannequins and vital sign monitors. Patient
safety may also be enhanced through full scenario team
management, in which a human patient simulator and a
fully simulated care environment, such as entire operating
rooms or emergency department bays, are utilized.
On a basic level, simulations improve patient safety by
allowing physicians to become better trained without putting patients at risk and, importantly, by providing protected time for reflection and debriefingwhere most of
the learning takes place (11, 12). The challenge is matching the best simulation method to the desired learning
objectives while recognizing the costs of each method (18).
Because simulation is a broad technique, faculty training
and time are often a more important investment than are
specific expensive simulation equipment. Practitioners
must be appropriately trained to effectively use simulation
techniques, as well as any specific technologies, to accomplish the relevant training, assessment, or systems probing
goals.
The simulation needs to feel real enough for participants to be able to suspend disbelief, enabling them to feel,
think, and act much as they would in a real scenario (12,
19). If the learning objective is mainly to practice cognitive
skills for diagnosis or treatment, a verbal simulation, such
as, What would you do if . . . , may be sufficient. In
contrast, if development of management skills, such as situational awareness or team communication, is the focus, a
more accurate replication of the actions and team presence
become important for the simulation experience.

REVIEW PROCESSES
Methodology to capture literature in this review involved 3 mechanisms. First, we used structured search
strategies to search PubMed and the Cochrane Library
from their beginning to 31 October 2012. These searches
were limited to meta-analyses; systematic reviews; and randomized, controlled trials (RCTs) or observational studies
published in English. Second, practitioners with expertise
in simulation provided recommendations on key articles,
including issues in implementation and empirical research
on simulation and patient safety. Finally, the reference lists
of articles captured by using the first 2 methods were
scanned for relevant literature. Abstracts of references captured by these searches (n 913) were screened by a single
reviewer.
Studies, including systematic reviews and metaanalyses, were included in the review if they reported evalwww.annals.org

Downloaded From: http://annals.org/ on 02/11/2014

Supplement

Key Summary Points

Simulation is a versatile technique that may be applied in
patient safety strategies across a variety of factors that
contribute to patient harm.
Heterogeneous evidence across multiple topic areas shows
that training with simulation-based exercises increases
technical and procedural performance.
Heterogeneous evidence shows that simulation-based
exercises can improve team performance and interpersonal dynamics.
Limited evidence suggests that improvements in patient
outcomes attributable to simulation exercises can occur
at the health system level.

uative results of patient outcomes or changes in clinician

actions in patient care. Studies that only provided
laboratory-based results were excluded.
Data from the 38 studies that met the inclusion criteria were abstracted by a single investigator. Given the varied nature of the included studies and the broad area of
simulation, quality assessment of individual studies was
limited to reporting study design. Selected studies are described with narrative synthesis. The Supplement (available at www.annals.org) provides a complete description of
the search strategies, article flow diagram, and evidence
tables.
This review was supported by the Agency for Healthcare Research and Quality, which had no role in the selection or review of the evidence or the decision to submit the
manuscript for publication.

BENEFITS

AND

HARMS

Of the 38 included studies, 22 were RCTs, 11 were

prospective observational studies, and 5 were retrospective
analyses of previous simulation interventions. Table 1
shows the distribution of study methodology and aspects of
simulation interventions by targeted areas for improvement
in patient safety. Thirty-four studies reported patient outcomes from care provided by trainees at varied levels of
education or specialties; postgraduate residents and fellows
were highly represented. Of the 27 studies that specified a
setting for the simulation, academic medical settings predominated (n 23).
Diagnostic Procedures

Five RCTs and 1 prospective beforeafter study on

training for colonoscopy and upper gastrointestinal endoscopy found better initial performance in actual patients
when physicians received simulation-based training (20
25). Studies generally reported a similar training period
requirement to ultimately reach desirable levels of procedure mastery (20 25). Safety outcomes focused primarily
5 March 2013 Annals of Internal Medicine Volume 158 Number 5 (Part 2) 427

Supplement

Simulation Exercises as a Patient Safety Strategy

Table 2. Eleven Dimensions to Consider When Designing

and Setting Up Simulation Exercises*
Purpose and Participants

Setup

Purpose and aims of simulation

activity
Type of knowledge, skills, attitudes,
or behaviors addressed in
simulation
Unit of participation in the simulation:
individuals or teams
Experience level of simulation
participants
Health care domain in which the
simulation is applied
Health care disciplines of personnel
participating in the simulation

Age of the patient being

simulated
Technology applicable or
required for simulations
Extent of direct participation
Site of simulation: in situ
clinical setting vs.
dedicated simulation center
Feedback method
accompanying simulation

* Adapted from reference 18.

participation to training in the particular simulation technique. In addition, sufficient time for creating meaningful
exercises with debriefing, equipment matched to the simulation need that recreates sufficient realism, and adequate
space or storage for in situ simulations will increase the
likelihood of success (18). Resources used in simulations
must be available when needed and kept safe from being
used inappropriately for patient care (for example, expired
medications). Technical support and maintenance may be
required for complex or high-tech simulators.
Rosen and colleagues (19) highlighted the importance
of cognitive fidelity (vs. physical fidelity) in a simulated
exercise: Simulations that engage the participant in ways
that cognitively best reflect the actual task are likely to be
more effective. Debriefing is considered crucial when implementing simulation requires instructor training (11, 12)
and is considered a best practice in simulation-based medical education (62).
Costs

The cost of implementing simulation exercises ranges

from low to high, depending on the type of exercise and
personnel and equipment resources involved (18). Instructor and learner time are likely to be the most expensive and
crucial aspects of simulation in the long run. Start-up costs
for a comprehensive simulation center may be accounted
for differently from ongoing costs for exercises, which
complicates the ability to categorize the expected cost for
simulation as a patient safety strategy. Unfortunately, research addressing cost savings attributable directly to simulation remains sparse, although some studies have reported up to a 7-to-1 return on simulation costs through
reduction in hospital days for bloodstream infections (21,
21a, 49).

DISCUSSION
Simulation has continued to gain momentum in patient safety efforts in the past decade because it allows for
exercising and improving aspects of health care delivery
430 5 March 2013 Annals of Internal Medicine Volume 158 Number 5 (Part 2)

Downloaded From: http://annals.org/ on 02/11/2014

without any known risks to patients. Simulation has been

used in patient safety for the purposes of education, assessment, research, and integration of system-level strategies.
These efforts have been reported in the literature as research about simulation: that is, research evaluating the
translation of simulation-based education to enhanced patient safety. In contrast, other research has focused on using simulation as a laboratory to discover potential leverage
points for patient safety (16).
Our review found that studies reporting patient outcomes or systems of care have been done primarily in academic settings, although researchers have used simulation
in diverse clinical specialties, experience levels, and care
settings. These studies varied in terms of individual quality,
but the majority were randomized or had methodologically
sound controlled prospective designs. Researchers have
replicated standardized simulation training for central venous catherization, and although this approach is promising for patient safety in that area, we did not find other
examples of replication studies in our review. We also did
not find analysis of contextual effects on the validity of
simulation to improve patient safety. The generalizability
of any one technique is likely to vary according to many
factors, such as those in Gabas 11-dimensional framework
(18), and the adequacy of resources dedicated to simulation (for example, debriefing).
At this juncture, simulation seems to have a favorable
effect on quicker acquisition and improved performance of
technical skills. Although not yet thoroughly studied, simulation of complex or high-stakes procedures seems to be a
promising technique to increase patient-safe behavior at
the clinician and team levels. Simulation has the potential
to enhance patient safety through structured assessment
and debriefing in quality improvement initiatives. It has
been used to assess practices that would be difficult or
unsafe to study empirically in real time with actual patients. Likewise, simulation has been endorsed for ongoing
competency and continuing education, as well as advancement to mastery-level practices.
A previous systematic review (7) reported that simulation contributes to enhanced knowledge acquisition and
improved clinical performance. Simulation techniques
have been used in translating results from the withinsimulation laboratory to patient- and health care system
level outcomes (17). Another systematic review (4) suggested that protected time for debriefing in a learning
experience is a crucial component of simulation techniques. To our knowledge, our review is the first to examine the effects that simulation exercises have on patient
safety outcomes, and in particular outcomes in patients
outside of simulation laboratory settings (that is, during
clinical care).
Our review has limitations. First, it is possible that the
broad search strategies missed studies that may be captured
with targeted and comprehensive strategies dedicated to
each simulation technique, clinical specialty, or applicawww.annals.org

Simulation Exercises as a Patient Safety Strategy

Several RCTs and observational studies that we reviewed confirmed that simulation-based education improved performance (40 51). Two prospective studies and
1 RCT reported that simulation training decreased rates of
catheter-related bloodstream infection (41, 46, 49), but 1
prospective controlled cohort study reported no difference
in rates (48) attributable to simulation-based training.
Studies showed mixed results for other major complications and critical patient safety events.
Other Procedures and Processes

Three RCTs reported data on other procedures and

processes. In 1 RCT, a simulation-based training curriculum for pediatric residents using high-fidelity models was
associated with nonstatistically significant increases in
performance of basic clinical procedural skills, such as bag
mask ventilation, venipuncture, peripheral venous catheter
placement, and lumbar puncture (52). Bachelors-level
nursing students made fewer medication administration errors in external training rotations when simulation training
was added to coursework (53). Among paramedic students,
simulation-based training did not lead to improved performance during their first 15 intubations in terms of overall
success rate, success rate on first attempt, or complications
(54).
Team and Systems Performance

Researchers retrospectively investigated the effect of an

annual mandatory 1-day workshop and training program
for all midwifery (including community-based practitioners) and obstetric emergency staff in a tertiary care center
(55). The workshop used simulation exercises for 7 common obstetric emergencies: shoulder dystocia, postpartum
hemorrhage, eclampsia, delivery of twins, breech presentation, adult resuscitation, and neonatal resuscitation. Compared with the 2-year period before the training program
was implemented, there was a statistically significant decrease in the rate of births with 5-minute Apgar scores of 6
or less and hypoxicischemic encephalopathy in the 2-year
period after implementation. The decrease in rate of moderate to severe hypoxicischemic encephalopathy only approached statistical significance.
In an RCT (56), primary care physicians in a large
multidisciplinary medical group were randomly assigned to
1 of 3 groups: no simulation control, simulation alone, or
simulation combined with a physician leader program. The
simulation training provided a series of interactive virtual
encounters with patients who had newly diagnosed diabetes or who had indicated or contraindicated adjustments to
their insulin regimen. When combined for comparison
with the control group, physicians in the simulation groups
prescribed renal-contraindicated metformin significantly
less often to patients with diabetes. In another RCT (57),
residents who participated in full-scenario simulation training for elective coronary artery bypass graft surgery had
increased Anesthesiologists Nontechnical Skills Assesswww.annals.org

Downloaded From: http://annals.org/ on 02/11/2014

Supplement

ment scores, and this difference was observed at 5-week

follow-up.
Three studies reported patient outcomes after
simulation-based training for resuscitation teams (14, 58,
59). An RCT (58) reported no differences attributable to
simulation training for actual team performance on rates of
ventilation, return of spontaneous circulation, or survival
to discharge. However, a prospective beforeafter study examined resuscitation outcomes after implementation of the
TeamSTEPPS team-building program coupled with simulation (59). This study reported several improvements in
communication, as well as reductions in time to computed
tomography, intubation, and the operating room. Finally,
in a retrospective case control study (14), simulation
training was associated with a higher correct response rate
based on the American Heart Association standards for
resuscitation.
Harms

Studies generally provided additive or supplemental

interventions to training as usual, and no study reported
data indicating increased potential for or actual harm to
patients that resulted from implementing simulation techniques. However, it is conceivable that simulation exercises
would place demand on valuable resources that could be
applied elsewhere in patient safety efforts. We found no
evaluations of such considerations.

IMPLEMENTATION CONSIDERATIONS

AND

COSTS

The Context for Simulation

A meta-analysis (8) of simulation in education programs for health professionals found that 564 of 609 studies (92.6%) examined techniques provided through dedicated simulation centers. Thirty-four studies (5.6%)
examined simulation in situ, and 11 studies (1.8%) reported from both contexts. Among studies cited in our
review, academic medical systems and academically affiliated hospitals predominated (2123, 2729, 35, 36, 38,
41 46, 49 54, 58, 60, 61). However, studies also reported
outcomes of use of simulation in tertiary care facilities (25,
45, 50, 58), trauma centers (44, 59), and multispecialty
medical groups (55, 56). We found no reported data on
the effect of context on the effectiveness of simulation exercises for improving patient safety.
Implementing Simulation

Gaba (18) conceptualized a framework for simulation

techniques that may aid implementation and ultimately
enhance patient safety (Table 2). The framework includes
11 dimensions that form a comprehensive set of considerations proposed to enhance the development and the effectiveness of simulation exercises. Application of each dimension guides specification and decision making on critical
choices about the simulation exercise. In practice, objectives of implementing simulation are aligned with the
needs of learners and the goals of trainers from level of
5 March 2013 Annals of Internal Medicine Volume 158 Number 5 (Part 2) 429

Supplement

Simulation Exercises as a Patient Safety Strategy

Table 2. Eleven Dimensions to Consider When Designing

and Setting Up Simulation Exercises*
Purpose and Participants

Setup

Purpose and aims of simulation

activity
Type of knowledge, skills, attitudes,
or behaviors addressed in
simulation
Unit of participation in the simulation:
individuals or teams
Experience level of simulation
participants
Health care domain in which the
simulation is applied
Health care disciplines of personnel
participating in the simulation

Age of the patient being

simulated
Technology applicable or
required for simulations
Extent of direct participation
Site of simulation: in situ
clinical setting vs.
dedicated simulation center
Feedback method
accompanying simulation

* Adapted from reference 18.

participation to training in the particular simulation technique. In addition, sufficient time for creating meaningful
exercises with debriefing, equipment matched to the simulation need that recreates sufficient realism, and adequate
space or storage for in situ simulations will increase the
likelihood of success (18). Resources used in simulations
must be available when needed and kept safe from being
used inappropriately for patient care (for example, expired
medications). Technical support and maintenance may be
required for complex or high-tech simulators.
Rosen and colleagues (19) highlighted the importance
of cognitive fidelity (vs. physical fidelity) in a simulated
exercise: Simulations that engage the participant in ways
that cognitively best reflect the actual task are likely to be
more effective. Debriefing is considered crucial when implementing simulation requires instructor training (11, 12)
and is considered a best practice in simulation-based medical education (62).
Costs

The cost of implementing simulation exercises ranges

from low to high, depending on the type of exercise and
personnel and equipment resources involved (18). Instructor and learner time are likely to be the most expensive and
crucial aspects of simulation in the long run. Start-up costs
for a comprehensive simulation center may be accounted
for differently from ongoing costs for exercises, which
complicates the ability to categorize the expected cost for
simulation as a patient safety strategy. Unfortunately, research addressing cost savings attributable directly to simulation remains sparse, although some studies have reported up to a 7-to-1 return on simulation costs through
reduction in hospital days for bloodstream infections (21,
21a, 49).

DISCUSSION
Simulation has continued to gain momentum in patient safety efforts in the past decade because it allows for
exercising and improving aspects of health care delivery
430 5 March 2013 Annals of Internal Medicine Volume 158 Number 5 (Part 2)

Downloaded From: http://annals.org/ on 02/11/2014

without any known risks to patients. Simulation has been

used in patient safety for the purposes of education, assessment, research, and integration of system-level strategies.
These efforts have been reported in the literature as research about simulation: that is, research evaluating the
translation of simulation-based education to enhanced patient safety. In contrast, other research has focused on using simulation as a laboratory to discover potential leverage
points for patient safety (16).
Our review found that studies reporting patient outcomes or systems of care have been done primarily in academic settings, although researchers have used simulation
in diverse clinical specialties, experience levels, and care
settings. These studies varied in terms of individual quality,
but the majority were randomized or had methodologically
sound controlled prospective designs. Researchers have
replicated standardized simulation training for central venous catherization, and although this approach is promising for patient safety in that area, we did not find other
examples of replication studies in our review. We also did
not find analysis of contextual effects on the validity of
simulation to improve patient safety. The generalizability
of any one technique is likely to vary according to many
factors, such as those in Gabas 11-dimensional framework
(18), and the adequacy of resources dedicated to simulation (for example, debriefing).
At this juncture, simulation seems to have a favorable
effect on quicker acquisition and improved performance of
technical skills. Although not yet thoroughly studied, simulation of complex or high-stakes procedures seems to be a
promising technique to increase patient-safe behavior at
the clinician and team levels. Simulation has the potential
to enhance patient safety through structured assessment
and debriefing in quality improvement initiatives. It has
been used to assess practices that would be difficult or
unsafe to study empirically in real time with actual patients. Likewise, simulation has been endorsed for ongoing
competency and continuing education, as well as advancement to mastery-level practices.
A previous systematic review (7) reported that simulation contributes to enhanced knowledge acquisition and
improved clinical performance. Simulation techniques
have been used in translating results from the withinsimulation laboratory to patient- and health care system
level outcomes (17). Another systematic review (4) suggested that protected time for debriefing in a learning
experience is a crucial component of simulation techniques. To our knowledge, our review is the first to examine the effects that simulation exercises have on patient
safety outcomes, and in particular outcomes in patients
outside of simulation laboratory settings (that is, during
clinical care).
Our review has limitations. First, it is possible that the
broad search strategies missed studies that may be captured
with targeted and comprehensive strategies dedicated to
each simulation technique, clinical specialty, or applicawww.annals.org

Simulation Exercises as a Patient Safety Strategy

tion. Second, given the relative infancy of the research on

simulation exercises, the field may be prone to selective
reporting of studies with positive findings, leading to potential publication bias. Finally, we limited our assessment
of quality of evidence to study design and did not perform
a structured assessment of the strength of evidence. Therefore, the overall strength of the evidence for simulation
exercises to improve patient safety should be interpreted
with caution.
In conclusion, simulation is a versatile technique that
continues to gain momentum in a variety of clinical settings and applications, including patient safety strategies.
Although evidence is largely heterogeneous at this time,
our review suggests the potential for simulation exercises to
contribute to patient safety through increased technical
and procedural performance and improved team performance. Limited research using health systemlevel observations suggests that simulation may enhance patient
safety, although more research is needed on the potential
for simulation to contribute to system-level differences in
patient safety outcomes. Systematic reviews of simulation
for specific procedures have begun reporting patient safety
outcomes (26, 30); more reviews of this nature would enhance our understanding of the overall contribution of
simulation techniques to patient safety. Future systematic
reviews would benefit from investigators using a consistent
framework, such as that developed by Gaba (18), to describe the intervention and its context and implementation.
From Stanford Center for Health Policy/Center for Primary Care and
Outcomes Research and Stanford University School of Medicine, Stanford University Hospital and Clinics, Stanford, California.
Note: The Agency for Healthcare Research and Quality reviewed

contract deliverables to ensure adherence to contract requirements and

quality, and a copyright release was obtained from the Agency for
Healthcare Research and Quality before the manuscript was submitted
for publication.
Disclaimer: All statements expressed in this work are those of the authors

and should not in any way be construed as official opinions or positions

of Stanford University, the Agency for Healthcare and Quality, or the
U.S. Department of Health and Human Services.
Financial Support: From the Agency for Healthcare and Quality, U.S.
Department of Health and Human Services (contract HHSA-290-200710062I).
Potential Conflicts of Interest: Mr. Schmidt: Grant (money to institu-

tion): Agency for Healthcare Research and Quality. Ms. McDonald:

Grant (money to institution): Agency for Healthcare Research and Quality. All other authors have no disclosures. Disclosures can also be viewed at
www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum
M12-2572.
Requests for Single Reprints: Kathryn M. McDonald, MM, Stanford
University, 117 Encina Commons, Stanford, CA 94305-6019; e-mail,
Kathryn.McDonald@stanford.edu.
www.annals.org

Downloaded From: http://annals.org/ on 02/11/2014

Supplement

Current author addresses and author contributions are available at www

.annals.org.

References
1. Meltzer D, Manning WG, Morrison J, Shah MN, Jin L, Guth T, et al.
Effects of physician experience on costs and outcomes on an academic general
medicine service: results of a trial of hospitalists. Ann Intern Med. 2002;137:86674. [PMID: 12458986]
2. Ziv A, Wolpe PR, Small SD, Glick S. Simulation-based medical education: an
ethical imperative. Acad Med. 2003;78:783-8. [PMID: 19088599]
3. Ericsson KA. Deliberate practice and acquisition of expert performance: a
general overview. Acad Emerg Med. 2008;15:988-94. [PMID: 18778378]
4. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. Effect of practice on
standardised learning outcomes in simulation-based medical education. Med
Educ. 2006;40:792-7. [PMID: 16869926]
5. Semeraro F, Signore L, Cerchiari EL. Retention of CPR performance in
anaesthetists. Resuscitation. 2006;68:101-8. [PMID: 16325986]
6. Park CS. Simulation and quality improvement in anesthesiology. Anesthesiol
Clin. 2011;29:13-28. [PMID: 21295750]
7. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does
simulation-based medical education with deliberate practice yield better results
than traditional clinical education? A meta-analytic comparative review of the
evidence. Acad Med. 2011;86:706-11. [PMID: 21512370]
8. Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health
professions education: a systematic review and meta-analysis. Acad Med. 2010;
85:1589-602. [PMID: 20703150]
9. Blum RH, Raemer DB, Carroll JS, Dufresne RL, Cooper JB. A method for
measuring the effectiveness of simulation-based team training for improving communication skills. Anesth Analg. 2005;100:1375-80. [PMID: 15845689]
10. Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb Y. Simulation-based
training in anesthesia crisis resource management (ACRM): a decade of experience. Simul Gaming. 2001;32:175-93.
11. Fanning RM, Gaba DM. The role of debriefing in simulation-based learning. Simul Healthc. 2007;2:115-25. [PMID: 19088616]
12. Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefing
with good judgment: combining rigorous feedback with genuine inquiry. Anesthesiol Clin. 2007;25:361-76. [PMID: 17574196]
13. Lipman S, Daniels K, Cohen SE, Carvalho B. Labor room setting compared
with the operating room for simulated perimortem cesarean delivery: a randomized controlled trial. Obstet Gynecol. 2011;118:1090-4. [PMID: 22015877]
14. Wayne DB, Didwania A, Feinglass J, Fudala MJ, Barsuk JH, McGaghie
WC. Simulation-based education improves quality of care during cardiac arrest
team responses at an academic teaching hospital: a case-control study. Chest.
2008;133:56-61. [PMID: 17573509]
15. Lighthall GK, Poon T, Harrison TK. Using in situ simulation to improve
in-hospital cardiopulmonary resuscitation. Jt Comm J Qual Patient Saf. 2010;36:
209-16. [PMID: 20480753]
16. Gaba DM. The futures here. We are it. Simul Healthc. 2006;1:1-2. [PMID:
19088617]
17. McGaghie WC, Draycott TJ, Dunn WF, Lopez CM, Stefanidis D. Evaluating the impact of simulation on translational patient outcomes. Simul Healthc.
2011;6 Suppl:S42-7.
18. Gaba DM. The future vision of simulation in healthcare. Simul Healthc.
2007;2:126-35.
19. Rosen MA, Salas E, Tannenbaum SI, Provonost P, King HB. Simulationbased training for teams in health care: designing scenarios, measuring performance, and providing feedback. In: Carayon P, ed. Handbook of HFE in Health
Care and Patient Safety. 2nd ed. Boca Raton, FL: CRC Press; 2011:571-92.
20. Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D. Virtual
reality colonoscopy simulation: a compulsory practice for the future colonoscopist? Endoscopy. 2005;37:1198-204. [PMID: 16329017]
21. Cohen J, Cohen SA, Vora KC, Xue X, Burdick JS, Bank S, et al. Multicenter, randomized, controlled trial of virtual-reality simulator training in acquisition of competency in colonoscopy. Gastrointest Endosc. 2006;64:361-8.
[PMID: 16923483]
21a. Cohen ER, Feinglass J, Barsuk JH, Barnard C, ODonnell A, McGaghie
WC, et al. Cost savings from reduced catheter-related bloodstream infection after
5 March 2013 Annals of Internal Medicine Volume 158 Number 5 (Part 2) 431

Supplement

Simulation Exercises as a Patient Safety Strategy

simulation-based education for residents in a medical intensive care unit. Simul

Healthc. 2010;5:98-102. [PMID: 20389233]
22. Ferlitsch A, Schoefl R, Puespoek A, Miehsler W, Schoeniger-Hekele M,
Hofer H, et al. Effect of virtual endoscopy simulator training on performance of
upper gastrointestinal endoscopy in patients: a randomized controlled trial.
Endoscopy. 2010;42:1049-56. [PMID: 20972956]
23. Park J, MacRae H, Musselman LJ, Rossos P, Hamstra SJ, Wolman S, et al.
Randomized controlled trial of virtual reality simulator training: transfer to live
patients. Am J Surg. 2007;194:205-11. [PMID: 17618805]
24. Sedlack RE, Kolars JC. Computer simulator training enhances the competency of gastroenterology fellows at colonoscopy: results of a pilot study.
Am J Gastroenterol. 2004;99:33-7. [PMID: 14687137]
25. Thomson M, Heuschkel R, Donaldson N, Murch S, Hinds R. Acquisition
of competence in paediatric ileocolonoscopy with virtual endoscopy training.
J Pediatr Gastroenterol Nutr. 2006;43:699-701. [PMID: 17130753]
26. Walsh CM, Sherlock ME, Ling SC, Carnahan H. Virtual reality simulation
training for health professions trainees in gastrointestinal endoscopy. Cochrane
Database Syst Rev. 2012;6:CD008237. [PMID: 22696375]
27. Blum MG, Powers TW, Sundaresan S. Bronchoscopy simulator effectively
prepares junior residents to competently perform basic clinical bronchoscopy.
Ann Thorac Surg. 2004;78:287-91. [PMID: 15223446]
28. Duncan DR, Morgenthaler TI, Ryu JH, Daniels CE. Reducing iatrogenic
risk in thoracentesis: establishing best practice via experiential training in a zerorisk environment. Chest. 2009;135:1315-20. [PMID: 19017865]
29. Tongprasert F, Wanapirak C, Sirichotiyakul S, Piyamongkol W, Tongsong
T. Training in cordocentesis: the first 50 case experience with and without a
cordocentesis training model. Prenat Diagn. 2010;30:467-70. [PMID:
20440735]
30. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual reality
training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev.
2009:CD006575. [PMID: 19160288]
31. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C,
McClusky DA 3rd, et al. Proficiency-based virtual reality training significantly
reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg. 2007;193:797-804. [PMID: 17512301]
32. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J,
Funch-Jensen P. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg. 2004;91:146-50. [PMID: 14760660]
33. Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, et al.
Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg. 2000;191:272-83. [PMID: 10989902]
34. Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Fayez R, Fried GM.
Fundamentals of laparoscopic surgery simulator training to proficiency improves
laparoscopic performance in the operating rooma randomized controlled trial.
Am J Surg. 2010;199:115-20. [PMID: 20103076]
35. Calatayud D, Arora S, Aggarwal R, Kruglikova I, Schulze S, Funch-Jensen
P, et al. Warm-up in a virtual reality environment improves performance in the
operating room. Ann Surg. 2010;251:1181-5. [PMID: 20485133]
36. Hamilton EC, Scott DJ, Kapoor A, Nwariaku F, Bergen PC, Rege RV,
et al. Improving operative performance using a laparoscopic hernia simulator.
Am J Surg. 2001;182:725-8. [PMID: 11839347]
37. Rogers GM, Oetting TA, Lee AG, Grignon C, Greenlee E, Johnson AT,
et al. Impact of a structured surgical curriculum on ophthalmic resident cataract
surgery complication rates. J Cataract Refract Surg. 2009;35:1956-60. [PMID:
19878829]
38. Kallstrom R, Hjertberg H, Svanvik J. Impact of virtual reality-simulated
training on urology residents performance of transurethral resection of the prostate. J Endourol. 2010;24:1521-8. [PMID: 20677993]
39. Ma IW, Brindle ME, Ronksley PE, Lorenzetti DL, Sauve RS, Ghali WA.
Use of simulation-based education to improve outcomes of central venous catheterization: a systematic review and meta-analysis. Acad Med. 2011;86:1137-47.
[PMID: 21785310]
40. Andreatta P, Chen Y, Marsh M, Cho K. Simulation-based training improves
applied clinical placement of ultrasound-guided PICCs. Support Care Cancer.
2011;19:539-43. [PMID: 20306091]
41. Barsuk JH, Cohen ER, Feinglass J, McGaghie WC, Wayne DB. Use of
simulation-based education to reduce catheter-related bloodstream infections.
Arch Intern Med. 2009;169:1420-3. [PMID: 19667306]
42. Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use
of simulation-based mastery learning to improve the quality of central venous
432 5 March 2013 Annals of Internal Medicine Volume 158 Number 5 (Part 2)

Downloaded From: http://annals.org/ on 02/11/2014

catheter placement in a medical intensive care unit. J Hosp Med. 2009;4:397403. [PMID: 19753568]
43. Barsuk JH, McGaghie WC, Cohen ER, OLeary KJ, Wayne DB.
Simulation-based mastery learning reduces complications during central venous
catheter insertion in a medical intensive care unit. Crit Care Med. 2009;37:2697701. [PMID: 19885989]
44. Britt RC, Novosel TJ, Britt LD, Sullivan M. The impact of central line
simulation before the ICU experience. Am J Surg. 2009;197:533-6. [PMID:
19249739]
45. Evans LV, Dodge KL, Shah TD, Kaplan LJ, Siegel MD, Moore CL, et al.
Simulation training in central venous catheter insertion: improved performance
in clinical practice. Acad Med. 2010;85:1462-9. [PMID: 20736674]
46. Khouli H, Jahnes K, Shapiro J, Rose K, Mathew J, Gohil A, et al. Performance of medical residents in sterile techniques during central vein catheterization: randomized trial of efficacy of simulation-based training. Chest. 2011;139:
80-7. [PMID: 20705795]
47. Martin M, Scalabrini B, Rioux A, Xhignesse MA. Training fourth-year
medical students in critical invasive skills improves subsequent patient safety. Am
Surg. 2003;69:437-40. [PMID: 12769219]
48. Miranda JA, Trick WE, Evans AT, Charles-Damte M, Reilly BM, Clarke
P. Firm-based trial to improve central venous catheter insertion practices. J Hosp
Med. 2007;2:135-42. [PMID: 17549773]
49. Sherertz RJ, Ely EW, Westbrook DM, Gledhill KS, Streed SA, Kiger B,
et al. Education of physicians-in-training can decrease the risk for vascular catheter infection. Ann Intern Med. 2000;132:641-8. [PMID: 10766683]
50. Smith CC, Huang GC, Newman LR, Clardy PF, Feller-Kopman D, Cho
M, et al. Simulation training and its effect on long-term resident performance in
central venous catheterization. Simul Healthc. 2010;5:146-51. [PMID:
20651476]
51. Velmahos GC, Toutouzas KG, Sillin LF, Chan L, Clark RE, Theodorou D,
et al. Cognitive task analysis for teaching technical skills in an inanimate surgical
skills laboratory. Am J Surg. 2004;187:114-9. [PMID: 14706600]
52. Gaies MG, Morris SA, Hafler JP, Graham DA, Capraro AJ, Zhou J, et al.
Reforming procedural skills training for pediatric residents: a randomized, interventional trial. Pediatrics. 2009;124:610-9. [PMID: 19651582]
53. Sears K, Goldsworthy S, Goodman WM. The relationship between simulation in nursing education and medication safety. J Nurs Educ. 2010;49:52-5.
[PMID: 19810664]
54. Hall RE, Plant JR, Bands CJ, Wall AR, Kang J, Hall CA. Human patient
simulation is effective for teaching paramedic students endotracheal intubation.
Acad Emerg Med. 2005;12:850-5. [PMID: 16141019]
55. Draycott T, Sibanda T, Owen L, Akande V, Winter C, Reading S, et al.
Does training in obstetric emergencies improve neonatal outcome? BJOG. 2006;
113:177-82. [PMID: 16907952]
56. OConnor PJ, Sperl-Hillen JM, Johnson PE, Rush WA, Asche SE, Dutta
P, et al. Simulated physician learning intervention to improve safety and quality
of diabetes care: a randomized trial. Diabetes Care. 2009;32:585-90. [PMID:
19171723]
57. Bruppacher HR, Alam SK, LeBlanc VR, Latter D, Naik VN, Savoldelli GL,
et al. Simulation-based training improves physicians performance in patient care
in high-stakes clinical setting of cardiac surgery. Anesthesiology. 2010;112:98592. [PMID: 20234305]
58. Weidman EK, Bell G, Walsh D, Small S, Edelson DP. Assessing the impact
of immersive simulation on clinical performance during actual in-hospital cardiac
arrest with CPR-sensing technology: a randomized feasibility study. Resuscitation. 2010;81:1556-61. [PMID: 20724057]
59. Capella J, Smith S, Philp A, Putnam T, Gilbert C, Fry W, et al. Teamwork
training improves the clinical care of trauma patients. J Surg Educ. 2010;67:43943. [PMID: 21156305]
60. Barsuk JH, Ahya SN, Cohen ER, McGaghie WC, Wayne DB. Mastery
learning of temporary hemodialysis catheter insertion by nephrology fellows using
simulation technology and deliberate practice. Am J Kidney Dis. 2009;54:70-6.
[PMID: 19376620]
61. Griswold-Theodorson S, Hannan H, Handly N, Pugh B, Fojtik J, Saks M,
et al. Improving patient safety with ultrasonography guidance during internal
jugular central venous catheter placement by novice practitioners. Simul Healthc.
2009;4:212-6. [PMID: 21330794]
62. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. A critical review of
simulation-based medical education research: 2003-2009. Med Educ. 2010;44:
50-63. [PMID: 20078756]
www.annals.org

Current Author Addresses: Mr. Schmidt and Ms. McDonald: Stanford

Center for Health Policy/Center for Primary Care and Outcomes Research, Stanford University, 117 Encina Commons, Stanford, CA
94305-6019.
Drs. Goldhaber-Fiebert and Ho: Stanford University School of Medicine, Stanford University Hospital and Clinics, 291 Campus Drive,
Stanford, CA 94305.
Author Contributions: Conception and design: E. Schmidt, S.N.
Goldhaber-Fiebert, K.M. McDonald.

W-188 5 March 2013 Annals of Internal Medicine Volume 158 Number 5 (Part 2)

Downloaded From: http://annals.org/ on 02/11/2014

Analysis and interpretation of the data: E. Schmidt, S.N. GoldhaberFiebert, L.A. Ho, K.M. McDonald.
Drafting of the article: E. Schmidt, S.N. Goldhaber-Fiebert, L.A. Ho,
K.M. McDonald.
Critical revision of the article for important intellectual content: E.
Schmidt, S.N. Goldhaber-Fiebert, L.A. Ho, K.M. McDonald.
Final approval of the article: S.N. Goldhaber-Fiebert, K.M. McDonald.
Obtaining of funding: K.M. McDonald.
Administrative, technical, or logistic support: E. Schmidt, K.M. McDonald.
Collection and assembly of data: E. Schmidt, S.N. Goldhaber-Fiebert.

www.annals.org

CORRECTION: SIMULATION EXERCISES

SAFETY STRATEGY

AS A

PATIENT

The following reference should be added to a recent review (1):

21a. Cohen ER, Feinglass J, Barsuk JH, Barnard C, ODonnell A,
McGaghie WC, et al. Cost savings from reduced catheter-related
bloodstream infection after simulation-based education for residents
in a medical intensive care unit. Simul Healthc. 2010;5:98-102.
[PMID: 20389233]
This has been corrected in the online version.
Reference
1. Schmidt E, Goldhaber-Fiebert SN, Ho LA, McDonald KM. Simulation exercises as
a patient safety strategy: a systematic review. Ann Intern Med. 2013;158:426-32.
[PMID: 23460100]

Downloaded From: http://annals.org/ on 02/11/2014