Troubleshooting FAQs
Why can't I find my custom scenario in the catalog when scheduling an exercise?
The new scenario won't be listed when scheduling an exercise if it was not published when you created it. In this case, the scenario is in a draft state, and won't appear in the scenario catalog.
To publish the scenario, select the 'Create Scenario' button on the top right.
Below the two 'Create' options, you'll find a list of your 'Draft Scenarios'.
- Search for the scenario and select 'Edit'.
- Select the 'Review' button on the top right, and then 'Publish'.
You'll now be able to schedule an exercise with this scenario.
Why are performance indicators not shown?
Performance indicators will only appear if feedback is enabled for your scenario. To ensure you see these indicators, check your scenario settings and make sure the feedback option is turned on. This allows participants to receive valuable insights into their performance during and after the exercise.
How can I find specific text within a scenario?
To locate specific text, you need to open each inject (scenario step) individually and use the search function within your browser or editor. Press Ctrl+F (or Cmd+F on Mac) to bring up the search bar, then type the text you are looking for. This method helps you quickly navigate to relevant information within each inject.
What are interludes, and how should I use them in a scenario?
Interludes are special sections within a scenario designed to stimulate discussion, encourage participants to reflect, or provide additional context and information. Unlike regular injects, interludes do not require participants to make decisions or submit votes—they are purely informational or thought-provoking. Use interludes to break up the flow, introduce new concepts, or prompt group discussion without advancing the scenario timeline.
Why can’t I see or run a scenario I just created?
If you have recently created a scenario but cannot find it in the scenario catalog, it is likely still in draft mode. To make it available for use, you must ‘Publish’ your draft scenario. Once published, it will appear in the scenario catalog, where you can select it and choose the appropriate exercise type to run your new scenario.
Where can I find the answers to justification questions after an exercise?
After completing a scenario or exercise, you can review all participant responses—including answers to justification questions—on the ‘Results’ page. This page provides a comprehensive overview of how participants responded and the reasoning behind their decisions, which can be valuable for debriefs and lessons learned.
How can I test a scenario before running it with participants?
You can preview your scenario in ‘Test’ mode, which allows you to experience the scenario flow without consuming a credit. This is useful for checking content, timing, and logic. Please note that reporting features are disabled in test mode, so results and analytics will not be generated during your preview.
Why does my After Action Report (AAR) appear empty or incomplete?
An After Action Report (AAR) may appear empty or lack detail if certain scenario settings were not enabled. For a comprehensive AAR, ensure that both Ranking and Confidence options are enabled when running your scenario. These settings allow the system to collect and display detailed participant data in the final report.
Why can’t I see any scenarios in the catalog?
If you are unable to view scenarios in the catalog, it may be due to your access rights. Only users with Crisis Sim Admin or Manager permissions can see the full catalog of scenarios. If you have standard user access, you will only see content that has been specifically assigned to you. Contact your administrator if you need broader access.
How can I see or read responses from specific participants?
To view individual participant responses, you must enable the “Show Participant Name” option when setting up your scenario. This setting prompts users to enter their names when joining the exercise, allowing you to track and review their answers. Please note that this feature is available only in “Presentation” mode.
How can I collect lessons learned or feedback from participants?
You can gather feedback and lessons learned by enabling justification questions within your scenario. Additionally, you can prompt users for feedback or next steps at the final inject. This approach encourages participants to reflect on their experience and share insights that can be used to improve future exercises.
What are some suggestions for customizing a scenario effectively?
When creating multiple injects that cover similar topics or share the same title, it’s helpful to add differentiators such as “ransomware 1/2/3.” This practice makes it easier to track the logic flow and understand how each inject fits into the overall scenario. Clear labeling also aids in scenario editing and review.
What should I do if I receive the error: “Participant can’t join a crisis sim exercise, and the WebSocket connection status is red” in Presentation Mode?
This error typically indicates a problem with WebSocket connectivity, which is essential for real-time updates in Crisis Simulation Presentation Mode. As a quick workaround, try connecting using mobile data or a different network. For a permanent solution, work with your IT team to ensure your browsers and network meet the system requirements, especially regarding WebSockets. For more details, refer to the WebSockets Diagnostics article in this guide.
Why aren’t the participant counter or votes updating in Crisis Simulation Presentation Mode?
If the participant counter or voting results are not updating in real time, it is likely due to WebSocket connection issues. Please refer to the previous FAQ on WebSocket diagnostics for troubleshooting steps and solutions.
After Action Report FAQs
What is an AAR?
At the end of a crisis sim exercise, an After Action Report (AAR) will be available to Crisis Sim Managers. This will inform them on:
Overall Performance
Inject Breakdown (Metrics relating to decision points)
Participant Breakdown
Next Steps (through related scenarios and lab collections that can be assigned)
How is a new AAR different from previous reports (View Results)
AAR is in addition to and complements the ‘View Results’ Report. AAR equips you with actionable insights and metrics on performance, whereas the ‘View Results’ report outlines the specific decisions made by participants and allows you to export these to CSV. Use ‘View Results’ to examine specific responses; use the AAR to determine next steps.
AAR also includes support for reporting by teams in single-player mode. Managers can assign exercises that include team reporting options by region, department, or enterprise when an exercise is created or edited.
Who can view AARs?
Anyone designated as Crisis Sim Manager can access and view AARs. This role is typically allocated to those responsible for crisis management in your organization (e.g., Incident Response Manager).
Are AARs available for all exercise types (e.g., Presentation, Single-Player, and Drills)?
Yes, as long as either response confidence or ranking is enabled in the scenario settings (this is the default case for all Immersive Labs scenarios). However, support for reporting by teams is only available in single-player mode.
The report will become available 30 minutes after the first completion of the exercise.
Why are AARs valuable?
The report provides the insights crisis managers need to effectively plan next steps for teams and individuals to improve their crisis response readiness.
What are some typical use cases for reporting by teams?
Assign team exercises that scale across the organization.
Compare teams’ performance against one another to prioritize exercising strategies.
Assess organizational cyber readiness and risks by specific teams.
How do I view an AAR?
At the end of an exercise, Crisis Sim Managers can follow these steps:
Navigate to ‘Exercise’ in the main navigation menu and select ‘Immersive Crisis Simulations.’
Search for the specific exercise in the ‘Managed by Me’ or ‘All Exercises’ area: use the search bar and/or filters on the left-hand side to bring up the exercise you want to view results for.
Select the exercise and then the ‘View After Action Report’ button.
Note: the AAR button will be accessible 30 minutes after the first completion.
Are AARs available retroactively (i.e., for exercises ended before its release)?
Yes. All exercises (including exercises that ran in the past) will have AARs available. If you don’t have an AAR appearing, jump to the next FAQ.
Why didn’t I get an AAR for an exercise?
The report will become available 30 minutes after the first completion. Contact our Support Team if a report isn’t generated, and we’ll investigate.
What data points can I see in the report?
What is visible depends on what you have chosen to measure in your exercise.
All data points will be available if your exercise is set up to measure response confidence and ranking of inject options. We measure both by default in all Immersive Labs scenarios.
If you create your own custom scenarios and want to obtain After Action Reports, ensure you’ve enabled these options when creating your content. Response confidence can be found in the ‘Analysis’ area of the content creator, which is just below ‘Ranking.’
If your exercise isn’t measuring response confidence, the data points related to this, such as decision confidence by inject, won’t be available.
Similarly, if you haven’t enabled ranking inject options, the data points related to this, such as Decision Score by Inject, won’t be in your report.
Note: data points relating to reporting by teams are only available in a single-player mode.
Why doesn’t my AAR for an exercise include a High Performers and Low Performers section? This feature is available in AARs for other exercises I’ve run.
If you did your exercise in Presentation mode, the AAR wouldn’t include these two sections in the Participant Breakdown area of the report. There will only be data for one participant in this area because, in Presentation exercises, only one participant selects answers: the facilitator.
Can I export/download an AAR?
Future releases will allow you to export the report to PDF format.
Terminology FAQs
What is a good overall score?
This is located In the Overall Performance section.
Based on our experience with IL clients today:
>= 75% – Excellent
>= 50% – Good
>= 25% – Fair
>= 0% – Needs improvement
The scoring guidelines apply to Immersive Labs scenarios. Consider this when interpreting the score if you have created your own scenarios or heavily customized option rankings within Immersive Labs scenarios.
What are the related scenarios and collections in ‘Next Steps’ based on?
These are chosen from the ‘attack vector’ of the exercise’s scenario.
What is an inject?
We call every decision point in scenarios an inject; scenarios are made up of injects. Injects (decision points) comprise inject options that rank weak, okay, good or great.
What is a playthrough?
A playthrough is a completion of the exercise.
Calculation FAQs
How is the Overall Score calculated?
The score on the exercise is an average of participants’ decisions (selection of inject options), with an adjusted element to account for confidence levels.
For example, selecting a weak option with a high level of confidence is a weaker decision than choosing a weak option with a low confidence level. This is because a high level of confidence could lead to the participant swaying others in a crisis or not checking with team members before acting, which could translate to higher risk for the organization.
We use the Dunning-Kruger curve to adjust for overconfidence bias in the score. If the exercise does not measure participants’ response confidence, the overall score will be the Decision Score (see next FAQ).
How is the Decision Score calculated and ranked?
It is an average of an individual’s scores based on their selected inject option. Inject options are ranked:
Great – 100%
Good – 75%
Okay – 50%
Weak – 25%
For example, in an exercise with five injects, where a participant selected the Great option for three injects and the Okay option for two, the Decision Score would be calculated as: (100+100+100+50+50)/5 = 80.
How is Decision Confidence calculated?
This is found in the Participant Breakdown section. It is the average of the response confidence scores.
Very confident – 100%
Confident – 75%
Somewhat confident – 50%
Not very confident – 25%
Not at all confident – 0%
How are the Strongest and Weakest Injects determined?
We consider the strongest injects to be the highest-scoring answers as voted by the participants. The weakest injects are the lowest-scoring answers as voted on by the participants.
How are High and Low Performers determined?
High performers are individual participants (Drill exercises) or teams (Single Player exercises) with an overall score equal to or higher than 50%.
Low performers are those individuals or teams that have scored lower than 50% in the exercise.
AI Scenario Generator FAQs
What is the AI Scenario Generator?
The AI Scenario Generator enables you to use AI to create Crisis Sim scenarios from scratch. You can access the AI Scenario Generator in the Create Scenario area of Crisis Simulations by clicking Create with AI. You can then select the type of scenario you wish to create using the pre-specified drop-down lists and give the generator additional context by specifying a briefing prompt and uploading a reference file, if you wish. After you select Generate briefing, the generator will present you with the briefing it has created as the basis for your scenario. You can edit this briefing if you wish, and then go on to create the remainder of the scenario. Once created, these scenarios can be edited and published as normal in your organization's catalog.
What type of content does the AI Scenario Generator create?
The AI Scenario Generator creates a Crisis Simulation scenario. These generations are based on publicly available data related to crisis management and create generic Crisis Simulation scenarios.
If you choose to add additional context via the briefing prompt or reference file field, the AI will take this into account when generating your scenario.
If your organization has enabled Crisis Sim - Share custom scenarios for enhanced AI responses, your previous and future custom scenarios will be used to generate scenarios tailored to your particular context. The sections below explain more about Scenario Sharing functionality.
Before being published, review the AI tool's outputs to ensure they meet your requirements (see section below).
What should I write in the Briefing prompt field?
The Briefing prompt field is a space for you to give additional information about the scenario you want to create to increase the relevance to your organization and the outcomes you are trying to achieve. Effective prompting of AI tools is important to ensure you get the best results. Here are some suggestions to get the most out of the tool:
Instruct the AI to adopt a specific persona or role: This can significantly influence the tone and style of its response. For example, “Act as a Crisis Incident Expert designing an ongoing crisis simulation exercise.”
Offer background information: Give the AI relevant details about the subject matter or the situation. This helps it understand the nuances to generate more accurate responses.
Include examples: Providing examples of the desired output can be highly effective. This allows the AI to learn from your preferences and produce similar results.
Consider the audience: Specify the target audience for the AI output. This will help the AI tailor its language and level of detail appropriately.
For more information and examples, please see our guidance documentation on effective prompting.
What documents should I upload in the Reference file field?
The purpose of this field is to allow you to add additional context in the form of news articles or reports on cyber attacks that might be relevant to your industry, organizational context, or any content that you want to focus on for exercising purposes.
The Reference file field will only accept PDF files. If you add a file into this field, it will be sent as additional context to the AI to generate your scenario. It will not be stored by our third party AI vendor or used to train its models. Users are responsible for any documents that are uploaded to the AI.
You are responsible for the content of any documents you upload. This includes ensuring that you have all necessary rights and permissions to share the content, and that it does not contain any sensitive, confidential, or personal information that you are not authorized to disclose to third-party AI.
Will I need to add anything else to my scenario?
The AI Scenario Generator currently only generates text content, so you may want to add additional rich media (for example, images and videos) to your scenario.
We also recommend enabling, adding, or checking certain elements after the scenario has been generated while in “edit mode” to get the most out of your Crisis Simulation.
This includes:
Checking that you are happy with the text formatting and narrative content
Checking that the injects and options generated link logically
Checking that you are satisfied with the role
Enabling and adding response feedback or performance indicators
If you want to capture ranked response data, you will need to select the ranked options setting and add a rank (great, good, weak, or okay) to each response option to suit your organization's preferred situational response
Turning on response confidence or justifications
How does management of the AI Scenario Generator in Immersive work?
Organization admins can choose to use the AI Scenario Generator in a layered approach:
Without Scenario Sharing enabled but with document upload and free-text interaction enabled. (default)
With Scenario Sharing enabled and with document upload and free-text interaction enabled.
No AI access at all
These settings can be updated in the AI section on the organization's Platform Settings page.
What does "without Scenario Sharing" mean?
This means that the AI will not use information from previous or future custom scenarios you have created when generating your scenario.
The AI will only use the title you give your scenario, the pre-filled drop-down filters (currently organizational sector, attack vector, and threat actor), any information entered in the Briefing prompt field, and any documents added in the Reference file section to create your scenario.
What does "Scenario Sharing" mean?
If your organization chooses to enable Scenario Sharing, Immersive will include relevant information from any previous custom scenarios that you have published in the context window for requests to our third-party AI vendor. This will make the scenario more relevant to your particular context. Further details on the context window and exactly what data will be shared can be found below.
Will my data be used for model training?
No. None of the data you choose to share will be used by the third party AI vendor for the purpose of training its models. Shared data will only ever be included in the temporary "context window" of generation and will not be stored by the third-party AI vendor.
What is a "context window"?
A "context window" is an extension of the query sent to an AI model. It exists only during the processing of the query and is not saved by any third parties.
Which third-party AI vendor does Immersive work with?
Immersive is using technology provided by OpenAI to support this feature. OpenAI’s business terms of use are available at https://openai.com/policies/business-terms/ .
We will notify you in advance if we intend to engage alternative third-party AI vendors to support feature releases containing AI.
If I choose to enable Scenario Sharing for AI, exactly what will be shared?
When enabled, the AI Scenario Generator will include data from your previously published (and any future) custom scenarios in the "context window" of its request.
The data captured from your custom scenarios will include the following:
Scenarios:
Title
Description
Industry Sector
Attack Vector
Threat Actor
Injects & Options:
Title
Description
The data shared will not include feedback, exercise information, reporting, account or organization information fields, or metadata.
The tool is not intended to process personal information. We recommend that you review your custom scenarios before choosing to enable Scenario Sharing to check whether they contain any sensitive or personal information and ensure its removal.
Immersive will endeavor to scan and de-identify fields for any personal information; however, we do not guarantee that any automated de-identification will be 100% effective.
Is Immersive compliant with the UK National Cyber Security Centre Guidelines for secure AI system development?
Yes. Immersive has implemented secure design and development procedures in line with the NCSC Guidelines document.
In which Geographic Region will my scenario information be processed?
OpenAI’s servers are located in the US. All requests made to this third party vendor will therefore be transferred to the US.
In which Geographic Region will my scenario information be stored?
All AI Service data will be stored in Immersive US region. This means that if you enable Scenario Sharing, the scenario data specified above will be stored in the Immersive US region until the scenario sharing option is switched off. At this point, the data will be deleted.
Disclaimer
The outputs are generated by artificial intelligence and are not verified by Immersive for accuracy and do not represent Immersive’s views. We do not make any warranty or guarantee as to the accuracy, completeness, or reliability of any outputs.