CEB Learning & Development Leadership Council

Guidebook: Achieving Impact with Simulations

A FRAMEWORK FOR MEMBER CONVERSATIONS The mission of The Corporate Executive Board Company and its affiliates (CEB) is to unlock the potential of organizations and leaders by advancing the science and practice of management. When we bring leaders together, it is crucial that our discussions neither restrict competition nor improperly share inside information. All other conversations are welcomed and encouraged. CONFIDENTIALITY AND INTELLECTUAL PROPERTY These materials have been prepared by CEB for the exclusive and individual use of our member companies. These materials contain valuable condential and proprietary information belonging to CEB, and they may not be shared with any third party (including independent contractors and consultants) without the prior approval of CEB. CEB retains any and all intellectual property rights in these materials and requires retention of the copyright mark on all pages reproduced. LEGAL CAVEAT CEB is not able to guarantee the accuracy of the information or analysis contained in these materials. Furthermore, CEB is not engaged in rendering legal, accounting, or any other professional services. CEB specically disclaims liability for any damages, claims, or losses that may arise from a) any errors or omissions in these materials, whether caused by CEB or its sources, or b) reliance upon any recommendation made by CEB.

ACHIEVING IMPACT FROM SIMULATIONS USER GUIDE

OVERVIEW Although most L&D functions have been using simple simulations (such as role plays) for years, an increasing number of L&D functions are beginning to integrate more complex simulations (such as running a simulated business) into their programs. This guidebook reviews the three action steps that L&D must take to ensure that these complex simulations drive performance outcomes rather than just offer an entertaining experience.

AUDIENCE The guidance in this book is aimed at L&D professionals and executives interested in implementing more complex, detailed simulations.

HOW TO USE Use this three-step process to create high-impact simulations:

Step 1Assess Opportunities: Assess whether simulation is an appropriate method to address a specic development need. Step 2Determine Type: Determine the appropriate type of simulation to use to meet learning objectives. Step 3Design for Impact: Design and deliver simulations to maximize learner impact.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

There are three action steps that L&D must take to create effective simulations.
To realize the benets of simulations, L&D must ensure it is focusing simulations on the right learning needs, using the right type of simulation, anddesigning simulations tomaximize learner impact.

ROADMAP FOR ACHIEVING IMPACT WITH SIMULATIONS

Three Steps to Achieving Impact with Simulations

Step 1 Assess Opportunities

Assess whether simulation is an appropriate method to address a specic development need.

Step 2 Determine Type

Determine the appropriate type of simulation to use to meet learning objectives.

Step 3 Design for Impact

Design and deliver simulations to maximize learner impact.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

Before creating a simulation, L&D must carefully consider whether a simulation is an appropriate method tomeet a specic learning need.
Simulations are not appropriate for every development needL&D must consider the learning content, the audience, and logistics when deciding whether to use a simulation. Use the tool to the right as a guide to determine if simulation is an appropriate delivery method in comparison with traditional learning methods.

ASSESS THE SUITABILITY OF SIMULATION AS A DEVELOPMENT METHOD

Guide to Assessing the Suitability of Simulation
Directions: For the learning content you are evaluating, consider each of the selection criteria below. Circle the rating that is most appropriate for each of the criteria. Tally your score and consult the key on the following page to interpret the score and determine the viability of simulations as an effective learning delivery method for this particular content. Selection Criteria
Target Population: To what extent is the learning content relevant to a large population of learners within the target audience? Need for Practice: To what extent is there a need for learners to practice applying the content? Emphasis on Skills Versus Knowledge: To what extent does the learning content address building skills as opposed to imparting knowledge? Risk of Content Application: How risky is it for the target audience to apply the learning back on the job without supervision? Barriers to Content Application: How many barriers (e.g., workow, manager relationships) are there to learners applying content on the job? Importance of Feedback: How important is it that learners receive immediate feedback on their ability to apply learning? L&D Resources: What level of investment (money, time, people, etc.) can the organization devote to this initiative?

Rating Scales
1 Small Population 1 Little Need to Practice 1 Imparting Knowledge 1 No Risk 1 Few Barriers 1 Not Important 1 No Resources 3 Medium Population 3 Some Need to Practice 3 Equal Focus on Skills and Knowledge 3 Moderate Risk 3 Some Barriers 3 Somewhat Important 3 Limited Resources 5 Large Population 5 Substantial Need to Practice 5 Building Skills 5 High Risk 5 Many Barriers 5 Very Important 5 Substantial Resources

Point Total
25 and Above 1524 14 and Below
2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

Deploying a simulation is a highly viable approach. See step 2 to identify which simulation is right for you. Simulation may be effective when also blended with more traditional learning approaches. Simulation is not likely to be effective. Consider a more traditional approach.

Source: CEB analysis.

There are three action steps that L&D must take to create effective simulations.
To realize the benets of simulations, L&D must ensure it is focusing simulations on the right learning needs, using the right type of simulation, anddesigning simulations tomaximize learner impact.

ROADMAP FOR ACHIEVING IMPACT WITH SIMULATIONS

Three Steps to Achieving Impact with Simulations

Step 1 Assess Opportunities

Assess whether simulation is an appropriate method to address a specic development need.

Step 2 Determine Type

Determine the appropriate type of simulation to use to meet learning objectives.

Step 3 Design for Impact

Design and deliver simulations to maximize learner impact.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

L&D should consider three main categories of simulations commonly used for training purposes.
Once L&D has determined that a simulation is the right development method, staff must take a disciplined approach to considering which type of simulation to use given learning objectives and logistical considerations.

THREE CATEGORIES OF SIMULATIONS

Simulation Categories
Live Simulations Live simulations involve real employees operating in a real environment.a Live simulations include (but are not limited to) activities such as role plays, board games, live gaming situations, and live case studies.

Technology-Based Simulations Technology-based simulations involve real employees operating in a computersimulated system.a When teams of learners work together in technology-based simulations, they communicate through the technology platform in the simulated system. Technology-based simulations involve activities undertaken within video games, virtual worlds, and virtual reality.

Blended Simulations Blended simulations involve real people operating in both real and computersimulated systems. When teams of learners work together in blended simulations, they communicate through both the platform in the simulated system and the real system. Blended simulations include (but are not limited to) activities such as a live simulation enhanced by virtual reality simulators and a live simulation enhanced by multi-player game interface.
Source: CEB analysis.
a

Modeling & Simulations Coordination Office, DoD Modeling and Simulation Glossary, http://www.msco.mil/MSGlossary.html.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

Live simulations involve real employees operating in a real environment.

Live simulations include activities such as role plays, board games, live gaming situations, and live case studies. Live simulations are wellsuited for developing higher-order soft skills, such as communication and managing relationships.

LIVE SIMULATIONS
Benets
Can be owned and completed by L&D using existing staff skills Are easily tailored to t individuals and their challenges/context Can be easily monitored and changed by L&D in the moment (to increase engagement, challenge, etc.) Emotionally immerses learners in the learning situation

Drawbacks
Need convincing participants to deliver value Coordination with multiple stakeholders can be complex Must occur in the right environment and situation to feel realistic

Case in Point
During its leadership development programs, Swiss Reinsurance runs live simulations to test senior leaders ability to perform in uncertain and ambiguous situations. To do this, it models the simulation around an emerging business opportunity with which the organization has little existing expertise and that presents the possibility for dilemmas. Swiss Re then hires actors or works with partners to bring the business opportunity to life for senior leaders. In a past simulation, Swiss Re brought its senior leaders together with a Chinese delegation (played by hired actors) to discuss expanding their current business with China. Swiss Re ensures leaders perform in the simulation as they would in real life by not letting them know that they are in a simulation.
Source: Swiss Reinsurance Ltd.; CEB analysis.

Choosing the Right Live Simulation When choosing the type of live simulation to deploy, L&D staff should consider the resources available to them and the level of realism (delity) necessary to meet the objectives of the simulation.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

Technology-based simulations involve real employees operating in a computer-simulated system.

Technology-based simulations involve activities undertaken within video games, virtual worlds, and virtual reality. Technology-based simulations are well-suited for developing higher-order technical skills.

TECHNOLOGY-BASED SIMULATIONS
Benets
Are often a highly-scalable simulation method

Drawbacks
Bought off the shelf or L&D must use dedicated consultants or internal staff with the right technical skills to create them (which are hard to nd) Can seem gimmicky to learners and leaders when not done well Learners need a high level of technology uency The organization must have the appropriate technology infrastructure

Ensures that learning is consistent across the learner population Enables L&D to re-create important scenarios but without the risk (e.g., to customer relationships) Enables employees to practice using expensive equipment (e.g., a driving simulator) Allows learners to operate in discrete time (e.g., compressing a year of events into a day) to see the future implications of decisions

Case in Point
Choosing the Right Technology-Based Simulation When choosing the type of technology-based simulation to deploy, L&D staff should consider the level of interactivity between learners necessary and whether the simulation needs to be available on-demand to learners. Workplace Technologies Research, Inc. (WTRI) worked with SAIC to create an interactive simulation game to train open pit miners on safety measures. In their quest to extract the most gold under budget and on time, teams of miners confront a variety of challenges, such as bad weather conditions, maintenance problems, and dangerous unforeseen events. The teams must address these challenges while continuing to meet production goals, which teaches them effective mine safety practices and maintenance procedures when operating in real-life pressure situations. The training program can identify and track game players who are working and behaving in an unsafe way and deduct points from the teams score for this behavior.
Source: http://www.saic.com/products/simulation/olive/customers.html; CEB analysis.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

Blended simulations involve employees operating in both real and computer-simulated systems.
Blended simulations include (but are not limited to) activities such as live simulation enhanced by virtual reality and live simulation enhanced by a multi-player game interface. Blended simulations capture the benets of both technology-based and live simulations.

BLENDED SIMULATIONS
Benets
Allows learners to work in both discrete and real time (i.e., the actual time during which a process occurs) High level of immersion for learners Enables learners to see the outcomes of their decisions while still allowing for a high level of collaborative decision making

Drawbacks
Highly resource-intensive simulation Requires a high level of coordination Often requires both L&D resources and thirdparty vendors Difficult to run effectively if participants are not colocated for the simulation

Case in Point
To help leaders learn how to execute against Sodexos new strategy, L&D created a custom-built strategy simulation. Teams composed of ve to six leaders are placed in charge of a company that mirrors Sodexo. The simulation has three stages. In the rst stage, teams hold weekly conference calls to create a strategy for their company. The second stage takes place over a three-day offsite where teams work together to overcome a number of common challenges (identied by senior executives) provided by the simulation software. The simulation software uses decision trees that require executives to live with the outcomes of their decisions and adjust their strategies accordingly. At the end of each offsite day (equal to one year in the simulation company), the groups pull up to debrief their results and decide how they will adapt their strategy. Finally, in the third stage, participants pull up to debrief the experience and provide each other with feedback on what it was like to work on a team with them and on their performance more broadly.
Source: Sodexo; CEB analysis.

Choosing the Right Blended Simulation When choosing the type of blended simulation to deploy, L&D staff should consider the resources available to them and their ability to colocate learners for the duration of thesimulation.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

10

Use the table on the right to determine which type of simulation is best suited for a particular initiative.

WHICH SIMULATION IS RIGHT FOR YOUR NEEDS?

Guide to Identifying the Right Type of Simulation

Live Simulations Best Suited for Higher-order soft skills (communication, relationship management, etc.) Real time Low Medium to High Low to High

Technology-Based Simulations Higher-order technical skills (nancial acumen, risk management, etc.) Discrete time Medium to High Low High

Blended Simulations Combination of soft and technical skills

Simulation Operating Time Learner Technology Prociency Required L&D Time Investment Monetary Investment

Combination of real-time and discrete time Low to Medium High High

Source: CEB analysis.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

11

There are three action steps that L&D must take to create effective simulations.
To realize the benets of simulations, L&D must ensure it is focusing simulations on the right learning needs, using the right type of simulation, and designing simulations to maximize learner impact.

ROADMAP FOR ACHIEVING IMPACT WITH SIMULATIONS

Three Steps to Achieving Impact with Simulations

Step 1 Assess Opportunities

Assess whether simulation is an appropriate method to address a specic development need.

Step 2 Determine Type

Determine the appropriate type of simulation to use to meet learning objectives.

Step 3 Design for Impact

Design and deliver simulations to maximize learner impact.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

12

L&D must go beyond designing an entertaining experience to designing a simulation that enables employees to take action and apply learning back to their job.
Use the tips on the right to design high-value simulations that enable learners to apply learning on the job.

DESIGNING HIGH-IMPACT SIMULATIONS

Design and Implementation Tips
DESIGNING THE SIMULATION

Consider the skill level of participants. When designing the simulation, be sure to consider the skill level of participants to ensure you are making the simulation challenging enough to stretch learners but not so challenging that participants will be overstretched and break down. Find complex problems. Try to avoid situations that have easy answers when designing the simulation to engage learners critical thinking skills. Create a compelling story. For more complex simulations, ensure you have a realistic and interesting story to ensure participant engagement in the simulation. Consider the timeline in which the simulation story will need to unfold and the setting. Ensure relevance. Speak with learners and their managers to ensure that the problems and stories presented in the simulation are relevant to learners day-to-day work. Align simulation with existing content. If the simulation is part of a larger development solution, L&D will need to think about how to integrate it with the rest of the development solution during the design phase. See the bottom of the following page for tips on aligning the simulation with existing development programs. Consider level of realism. Consider the level of realism (delity) necessary to meet the learning objectives of the simulation. Identify success measures. Create concrete and measurable objectives for the simulation. Run a pilot. Before rolling out the simulation, pilot it with HR employees and identify any areas you might need to change before launching.

PRE-SIMULATION ADMINISTRATION

Contact managersContact the managers of participants about the objectives of the simulation and larger development solution. Ask them to discuss the objectives with their direct reports before the simulation so that they understand what they are expected to learn. Assign pre-workReach out to participants with any prework for the session.

Source: Chris Musselwhite, Sue Kennedy, and Nancy Probst, Best Practices for Facilitating Simulations. T+D, 64, no. 8 (August 2010): 2628; CEB analysis.

2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

13

L&D must go beyond designing an entertaining experience to designing a simulation that enables employees to take action and apply learning back on the job.

DESIGNING HIGH-IMPACT SIMULATIONS (CONTINUED)

Design and Implementation Tips (Continued)
DELIVERING THE SIMULATION

Establish ground rules. Before beginning the simulation, explain the purpose and the roles participants will play and explain any ground rules for participant behavior during the simulation. Set the expectation for performance. Let participants know that they are expected to perform to their fullest potential during the simulation. Observe. Monitor the simulation to identify anything you need to change in the moment to increase or decrease the challenge or intensity. Make sure to take notes on things to discuss during the debrief session. Know when to intervene. Be alert to conict and sudden changes in behavior (such as withdrawal); if necessary, pull the participant aside to discuss what is happening.

AFTER THE SIMULATION

Debrief the simulation. Let participants discuss the experience and provide feedback to each other about their experiences working with them. Make sure to focus the discussion on the learning, not the outcome of the simulation. Identify areas to change. Ask participants if they have any suggestions for how the simulation could be more effective or engaging.

Source: Chris Musselwhite, Sue Kennedy, and Nancy Probst, Best Practices for Facilitating Simulations. T+D, 64, no. 8 (August 2010): 2628; CEB analysis.

Aligning Simulations with Existing Development Programs If the simulation is part of a larger development program, L&D must determine how to align the simulation with the program (and the other solutions within the program) during the design phase. L&D staff should follow the steps below to ensure alignment: 1. Identify how the simulation supports the objectives of the development program. 2. Determine the unique value-add from the simulation compared to the other solutions within the development program. 3. Check for redundancies in the program that may arise from adding the simulation and cut them. 4. Keeping in mind the goals for each of the solutions in the development program, identify the ideal timing for each solution (e.g., if the goal of the simulation is to help learners apply, it should come after they have learned the content). 5. Identify any content that needs to be added to the development program to support the simulation.
2013 The Corporate Executive Board Company. All Rights Reserved.LDR7422113PRO

Source: CEB analysis.

14