Secure.
Resilient. Competitive.
Design & Evaluation
“Without a sound understanding of the organization’s core capabilities and critical tasks, sound evaluation may not be completed.”
ESI provides training, exercise design, and evaluation services using real-life situations, enhanced through the use of dynamic simulation injects.
On a regular basis, Emergency Solutions International (ESI) is asked to assist firms to design exercises and evaluate the preparedness and response capabilities of their executive teams and organization to manage challenging crises.
Setting the Stage
One of the greatest challenges when evaluating an exercise is to have participants perform as they would in an actual event.
Anxiety at the individual or group level can cause poor performance and thus, inhibit accurate measurement of capabilities. The facilitator must conduct the exercise in a manner that ensures each participant understands that the exercises are practice evaluations, not personal tests.
Scope
When defining the scope of an exercise, the following must be considered:
Type of Exercise
Discussion, Table-top, Functional,
Full Scale, etc.
Level of Participant Capabilities
Duration
Interdependent Organizations and Parties of Interest
Risk Scenario Chosen from the Organization Risk Assessment.
Each year the risk scenario will change.
Exercise Objectives
exercise objectives
The SMART model outlined below may serve as a guide in the creation of exercise objectives.
Objectives will be based upon existing organizational core capabilities. Objectives must be succinct and understood by all participants.
It is critical that the exercise objectives align with the planning documents.
Realistic and appropriate objectives are set with the client, including both strategic and tactical objectives. Strategic objectives are long-term organizational goals to help guide decision-making, while tactical objectives are operational short-term goals, set with strategic objectives in mind.
After setting exercise objectives, common stakeholder and agency-specific critical tasks are identified and linked to the objectives, as are the interdependent core capabilities. Core Capabilities are defined by the critical tasks that must be completed by individuals or groups within the crisis organization.
It is the critical tasks which become the subject of the evaluator’s attention. Often, firms wish to improve upon their existing core capabilities. As a result, Target Capabilities are written, and the associated critical tasks are identified and validated as part of the exercise process. Together, these linked objectives, measures and metrics prompt the creation of objective-testing injects that simulate a worst-case crisis scenario that the organization may face.
Specific
State exactly what you’d like to accomplish: who, what, where, when.
Measurable
How will you demonstrate and evaluate the extent to which the goal has been met?
Achievable
Challenging goals with continued ability to achieve outcome.
Relevant
How does the goal tie into your key responsibilities? How is it aligned to objectives?
Time-Bound
Set one or more target dates to guide your goal to a successful and timely completion using deadlines, dates and frequency.
Simulated Real-Life Challenges For Training and Exercising
ESI challenges your organization with real-life situations.
Our challenges are enhanced through the use of dynamic simulation injects to enrich the learning experience. Custom and interactive visuals help increase the realism of the exercise while keeping participants interested, excited to learn, and eager to practice the organization’s plans, policies, and procedures in response to an incident.
Audio:
- A call from a concerned family member of a participant or a bothered member of the community can have an immense impact on exercise participants in comparison to reading the words off of a PowerPoint slide.
- A special radio broadcast or breaking news report can help add a layer of realism to your exercise scenario, or perhaps the power is out and radio communication is the only option.
Video:
- A Breaking News Video from your ‘local’ news company, mentioning your organizations name can really liven up an exercise. Whether the news report be at the first sign of an unfolding situation, during an incident, and/or in follow-up to the event during the recovery phase, this is sure to break up an exercise of text-based injects.
- Digital Simulation software can also be used to create a custom video (or image) simulation of your facility(s) during a fire, smoke/vapor incident, explosion, Haz Mat incident, and other emergency situations.
Custom Injects
Now more than ever, we get our first tip-off of an unfolding incident through social media platforms.
Social Media:
Oftentimes this is the case even before the local and national news agencies run the story. Because of its instantaneous reach, organizations can find themselves facing critical messages and/or misinformation going viral almost immediately. Through manipulated imagery, organizations and their Crisis Communications teams can proactively practice their key messaging strategies or plan ahead to address the developing situation through their social media platform followers at a pace that is representative of a real-world situation.
Digital Simulation:
Bring your training and exercise scenario to life with an image or video illustrating the scenario the participants are facing, i.e. perhaps a facility is on fire, or a gas leak is observed.
Newspaper Article:
Having your participants see the company name in the local news paper is a real eye-opener. An article on the front page of the newspaper can be a great tool to help participants experience and practice their roles within the Response or Recovery phases of the incident.
Process
On a regular basis, Emergency Solutions International (ESI) is asked to assist firms to design exercises and evaluate the preparedness and response capabilities of their executive teams and organization, to manage challenging crises.
Gap Indentification
Planning documents guide critical tasks when supporting core capabilities.
During or after a well-run exercise, participants should be in a position to honestly identify the gaps in planning documents or the execution of critical tasks. Participants get this chance to be honest during a hotwash, which follows quickly after the exercise. The hotwash is a facilitated session where the evaluator keeps notes on the strengths and areas for improvement or refinement. A more formal debrief is held with the evaluators and, many times, senior staff.
During ESI facilitated exercises, an Issues Board is created whereby participants chart their concerns about gaps as the exercise proceeds. Oftentimes, participants will say “I’ve always worried about this gap.”
After Action Review (AAR)
An After Action Review is created following a training drill or exercise.
Organizational learning requires that teams continuously assess their performance to identify and learn from successes and failures.
The AAR is a simple but powerful tool to help achieve this. An AAR is a form of retrospective analysis on a given sequence of goal-oriented actions previously undertaken; the analytical AAR is exercised as part of a process of performance evaluation and improvement as part of the Public Safety Canada (PSC) Capability Improvement Process. AARs identify specific corrective actions, assign them to responsible parties, and establish targets for their completion.
Drill
A drill is narrower in scope than an exercise, and is typically operations-based. Drills are primarily designed to assist personnel to practice specific tasks or procedures related to their role or functional area.
Table-Top Exercise
Exercises are broader in scope than Drills. A Table-top Exercise is a boardroom style session, administered by a facilitator. Team members discuss their roles and share strengths and opportunities for refinement regarding a simulated, reasonable worst-case emergency scenario. Table-top exercises usually run a few hours in duration.
Functional Exercise
A Functional Exercise is larger in scale than a Table-top exercise. Participants perform their emergency management roles in a simulated reasonable worst-case emergency environment created using a series of injects that simulate realistic problems that need to be addressed. Functional exercises typically focus on specific team members and procedures, the linkage between the Emergency Operations/Coordination Centre (EOC/ECC), and are often used to identify strengths, process gaps and opportunities for refinement. A functional exercise does not involve any “boots on the ground” components. Functional exercises usually run a half-day in duration.
Full Scale Exercise
A Full Scale Exercise is the closet to the real thing as possible and typically includes personnel from multiple functions, community first responders, stakeholders, and Regulators. Full Scale Exercises are multi-agency, multi-jurisdictional, multi-discipline exercises involving a “boots on the ground” response component (i.e., Security Guard Force response to a security threat, firefighters decontaminating equipment, paramedics triaging mock victims). Full Scale Exercises utilize, to the extent possible, the actual policies/procedures, systems, equipment, resources, and response stakeholders that would be used in response to a real event. From a duration standpoint, full scale exercises often take place over the course of an entire business day.