Design & evaluation
“Without a sound understanding of the organization’s core capabilities and critical tasks, sound evaluation may not be completed.”
On a regular basis, Emergency Solutions International (ESI) is asked to assist firms to design exercises and evaluate the preparedness and response capabilities of their executive teams and organization, to manage challenging crises.
Setting the Stage
One of the greatest challenges when evaluating an exercise is to have participants perform as they would in an actual event.
Anxiety at the individual or group level can cause poor performance and thus, inhibit accurate measurement of capabilities. The facilitator must conduct the exercise in a manner that ensures each participant understands that the exercises are practice evaluations, not personal tests.
Scope
When defining the scope of an exercise, the following must considered:
Type of Exercise
Discussion, Table-top, Functional,
Full Scale, etc.
Level of Participant Capabilities
Duration
Parameters for Engagement
usually determined from previous engagements
exercise objectives
The SMART model outlined below may serve as a guide in the creation of exercise objectives.
Objectives will be based upon existing organizational core capabilities. Objectives must be succinct and understood by all participants.
It is critical that the exercise objectives align with the planning documents.
Realistic and appropriate objectives are set with the client, including both strategic and tactical objectives. Strategic objectives are long-term organizational goals to help guide decision-making, while tactical objectives are operational short-term goals, set with strategic objectives in mind.
After setting exercise objectives, common stakeholder and agency-specific critical tasks are identified and linked to the objectives, as are the interdependent core capabilities. Core Capabilities are defined by the critical tasks that must be completed by individuals or groups within the crisis organization.
It is the critical tasks which become the subject of the evaluator’s attention. Often, firms wish to improve upon their existing core capabilities. As a result, Target Capabilities are written, and the associated critical tasks are identified and validated as part of the exercise process. Together, these linked objectives, measures and metrics prompt the creation of objective-testing injects that simulate a worst-case crisis scenario that the organization may face.
Specific
State exactly what you’d like to accomplish: who, what, where, when.
Measurable
How will you demonstrate and evaluate the extent to which the goal has been met?
Achievable
Challenging goals with continued ability to achieve outcome.
Relevant
How does the goal tie into your key responsibilities? How is it aligned to objectives?
Time-Bound
Set one or more target dates to guide your goal to a successful and timely completion using deadlines, dates and frequency.
Exercise design components
Exercise Design Components, or Injects, are fictitious events used to create the structure of the exercise and provide the basis for evaluating the core capabilities as identified in the plan.
Injects are grouped to form a storyline that is a reasonable worst-case scenario. They are written by the exercise designer and documented in a Master Inject Log (MIL). The MIL lists each event in chronological order, and also outlines the event details, time of occurrence, and the expected action or outcome.
Simulated Real-Life Challenges For Training and Exercising
ESI challenges your organization with real-life situations, enhanced through the use of dynamic simulation injects to enrich the learning experience. Custom and interactive visuals help increase the realism of the exercise while keeping participants interested,
Find out about our exercise design and evaluation process: