Exercise Design and Evaluation of Corporate Emergency Management and Response Capabilities

Without a sound understanding of the organization’s core capabilities and critical tasks, sound evaluation may not be completed”(DHS, 2013).

On a regular basis, Emergency Solutions International (ESI) is asked to assist firms to design exercises and evaluate the preparedness and response capabilities of their executive teams and organization, to manage challenging crisis.

Setting the Stage

One of the greatest challenges when evaluating an exercise is to have participants perform as they would in an actual event. Anxiety at the individual or group level can cause poor performance and thus, inhibit accurate measurement of capabilities. The facilitator must conduct the exercise in a manner that ensures each participant understands that the exercises are practice evaluations, not personal tests.

Scope

When defining the scope of an exercise, the following must considered:

  1. Type of exercise (Discussion, Table-top, Functional, Full Scale, etc.)
  2. Level of Participant Capabilities
  3. Duration
  4. Parameters for engagement (usually determined from previous engagements)

Exercise Objectives

Objectives will be based upon existing organizational core capabilities. Objectives must be succinct and understood by all participants. The SMART model outlined below may serve as a guide in the creation of exercise objectives.

It is critical that the exercise objectives align with the planning documents.

Exercise Design Components
Exercise Design Components, or Injects, are fictitious events used to create the structure of the exercise and provide the basis for evaluating the core capabilities as identified in the plan. Injects are grouped to form a storyline that is a reasonable worst-case scenario. Injects are written by the exercise designer and documented in a Master Inject Log (MIL). The MIL lists each event in chronological order, and also outlines the event details, time of occurrence, and the expected action or outcome.
Core Capabilities
Core Capabilities are defined by the critical tasks that must be completed by individuals or groups within the crisis organization. It is the critical tasks which become the subject of the evaluator’s attention. Often firms wish to improve upon their existing core capabilities. As a result, Target Capabilities are written, planning and training is conducted, and the associated critical tasks are identified and validated as part of the exercise process.
Exercise Design and Evaluation Process
Gap Identification

During or after a well-run exercise, participants should be in a position (if they are honest) to identify the gaps in planning documents or the execution of critical tasks. Planning documents guide critical tasks supporting core capabilities. Participants get this chance to be honest during a “hotwash,” which follows quickly after the exercise. The hotwash is a facilitated session where the evaluator keeps notes on the strengths and areas for improvement or refinement. A more formal debrief is held with the evaluators and, many times, senior staff.

During ESI facilitated exercises, an Issues Board is created whereby participants chart their concerns about gaps as the exercise proceeds. Often times, participants will say “I’ve always worried about this gap…”

Improvement Planning

Capability gaps become the output of the exercise evaluation process. An Options Analysis will aid in the selection of the proper solutions to address the gaps. Action must be taken to remedy the identified gaps in procedure. There must be a communication process to let the participants understand the remediation process. Plans are modified, budgets are set, and responsibility delegated. When appropriate, training is provided and, at the next exercise, the newly filled gaps become the focused priority.

References:Federal Emergency Management Agency U.S. Department of Homeland Security. 2013. Homeland Security Exercise and Evaluation Program (HSEEP). Retrieved from http://www.fema.gov/media-library-data/20130726-1914-25045-8890/hseep_apr13_.pdf.

Go back
Tell us your thoughts

Filed under:

December 9, 2020

Mark Gillan

About the author:

Mark leads the ESI team in conducting risk assessments, training, program, and plan review, exercise scenario development, exercise facilitation, and evaluation as well as compiling final After Action Review documentation and recommendations. Under Mark’s leadership, there have been six (6) Federal research and development projects conceived and executed by the ESI team. Service has been provided for various critical infrastructures, such as Port Saint John, Point Lepreau Generating Station, Canaport Liquefied Natural Gas, Saint John Energy, as well as corporations like Atlantic Potash, Mead Johnson Nutrition, and the Mosaic Company.