So, you want to customize the Workforce Exercising content?
That’s great! A key benefit of Workforce Exercising is that you can customize the content of scenarios to suit your needs, tailoring them to your specific organizational context.
How you customize the content will depend on the outcomes you want to achieve.
- Education and engagement: If you’re purely focused on engagement and education, some aspects, such as learning quality, will be more important than others.
- Data: If you also want to use the response data captured during Workforce Exercising, whether to inform risk management or demonstrate the impact of interventions, then you need to make sure that you customize the scenarios with data in mind.
Maximizing learning
Creating an educational experience that engages, is relevant to people, and provides in-the-moment feedback is important to maximize learning quality. The language that you use, the feedback you provide, and the way you use narrative and story will all influence how people experience your content and what they get from it.
Here are some things to consider when customizing scenarios.
1. The language that you use
At Immersive Labs, we want to make it as easy as possible for everyone to be able to engage with our content and understand the key points. We write in a conversational style, avoiding the use of unnecessarily technical language and jargon. We also try to avoid using idioms where possible since these can be difficult to understand for non-native speakers and are not always easily understood.
Using tools such as Grammarly can help you simplify the language you use and spot any potential typos or errors.
|
ey Takeaway
|
2. In-the-moment learning
Most scenarios provide immediate feedback on people’s responses. Although you can turn this off, we don’t advise it if education is your primary goal as it facilitates in-the-moment learning. You may wish to customize feedback to better reflect your organizational context and policies.
When customizing feedback, we recommend that you clearly state the best action or choice in the circumstances and why, alongside why any particular choice that has been made, should be avoided if possible.
Remember, it is the chosen action that you are providing feedback on, not the person!
|
Key Takeaway
|
3. Telling an engaging story
The role of narrative and story-telling in learning is well known. Scenarios provide a great opportunity to engage people in a story, and the ability to customize this means that you can make it as relevant to their role and context as possible. Incorporating rich media assets in your story, such as images, audio files, and video files, all contribute to a more engaging experience. If you decide to create your own images, remember to add any required alt text!
The ability to branch within the content means that you can change the outcomes and parts that the person encounters based on their choices should you wish to. However, if you make everyone’s journey through the story different, be aware that this can impact your data as it will be less comparable across individuals.
Even if you don’t use branching, if you decide to customize any response options, feedback, or part content, ensure that there is still a narrative flow between these and that the story does not become illogical or disjointed. You can often maintain narrative flow by using careful wording at the beginning of each part to maximize linkage with varied response options.
|
Key Takeaway
|
Ensuring data quality
Measuring behavior is a tricky thing. Within our Workforce Exercising scenarios, we do not directly measure how an individual behaves. To do that, you’d have to take an objective measurement using your internal systems, such as checking whether an individual has plugged a USB device into their laptop when they shouldn’t. This can be difficult to do in practice and also doesn’t tell you much about why they made that choice.
So we’re doing the next best thing, using scenario-based judgments that ask people what they would be most likely to do in a particular situation. This allows them to practice their responses in a safe environment and free from judgment and is essential if people are to be honest and open in their responses.
When customizing content, there are several things you need to consider if you want to make the data from your scenarios as robust as possible.
4.Number and type of decision points
We strive to keep our scenarios as short as possible to maximize engagement. However, we also need to ensure that everyone experiences a sufficient number of decisions to provide multiple data points for each person. This increases the reliability of the data you collect by reducing the impact of lucky guesses.
Our scenarios contain a minimum of five decision points. This allows them to still be completed within 10 minutes and provides a minimum number of data points to refer to across related areas. We recommend that you do not use fewer than five decision points to maximize the reliability of the produced data.
Type of decision point
We also recommend that, where possible, the decisions that individuals encounter in a scenario cover related risk areas (e.g., all parts may relate to behaviors within the social engineering topic and subsequent reporting). This will help ensure you have reliable data related to each risk area.
Some scenarios cover multiple risk areas. For example, each decision part may relate to a completely different risk area. This is due to the nature of the scenario and is a good way to have a high-level overview across different areas as efficiently as possible, but be aware that the data from these exercises for each individual risk area is likely to be less reliable as a result.
|
Key Takeaway
|
5. Language used
When designing questions and response options, it’s important to consider whether you unintentionally lead people to respond in a certain way. You may be using adjectives within the scenario that suggest that a particular action, process, or activity is negative or positive, such as ‘a suspicious email’ rather than ‘an email’.
By referring to things in a particular way, you can ‘prime’ an individual to respond in a certain way. You can read more about priming here. When a potential phishing email is received, the question ‘What do you do next?’ is very different from ‘What do you do to ensure that the email is legitimate?’ or ‘Do you report the email?’. The latter two are more likely to suggest to people that the email is in some way suspicious and influences their decision.
This can increase the likelihood that people will choose a response based on what they think you want them to choose or lead them to notice or respond differently than they normally would. Try not to give any clues to individuals in the wording of either the question or response options!
Response level feedback
All of our narrative scenarios include response level feedback. This provides an opportunity to maximize in-the-moment learning whilst also collecting data. However, our Security Hygiene Compass is a completely data-focused tool, so does not provide feedback on each response to reduce the chance that this will influence future choices.
It’s up to you whether you would like to use response-level feedback and if so, if you’d like to customize the provided feedback. If you do, make sure that you think about the language used and the information provided to try and minimize any potential impact on remaining responses across the scenario.
|
Key Takeaway
|
6. Focusing on behavior
We focus on the actions people would choose to take rather than what they think or know about a situation. Knowing the right response does not necessarily mean that they would act in a secure manner when put in that situation. This may be for various reasons, including competing priorities, external pressures, or processes that are poorly implemented or difficult to follow.
You want people to respond honestly to identify potential problem areas. For our Workforce Exercising scenarios to work best, encourage people to be as honest as possible and use the scenario reporting to facilitate an open culture of honesty, reflection, and learning rather than focusing on penalizing those who have not responded how you would like.
To make sure that we focus on behavior in our scenarios, our instructions reflect ‘what would you do?’, i.e., the behavior you would take, rather than ‘what should you do?’, which is a more knowledge-focused question.
Actions and options
When writing response options, it’s easy to overcomplicate things. For effective measurement, keep things simple. Ensure each response option represents a single action, rather than multiple parts. For example, ‘report the email to X’ or ‘ask what a colleague thinks’ rather than ‘ask what a colleague thinks, then report the email to X’. Otherwise, it’s hard to know the first and primary choice people make and what action they prioritize.
Risk areas
Each Workforce Exercising scenario is tagged to a risk area. These reflect overarching topic areas that the decisions, and associated behaviors, in the scenario relate to. For example, identifying that an email may not be genuine is a behavior that would form part of the wider social engineering area.
Customizing a scenario extensively may mean that the risk area provided is no longer appropriate, and a different one should be chosen. Always check that the risk area listed still reflects the key behaviors and actions covered in the scenario. There may be overlap between risk areas, but choose the one that you feel reflects it most strongly.
|
Key Takeaway
|
7. Maximising variance in the data
To effectively use data to understand potential risk areas, we need to ensure sufficient variance in the data we collect. We do this through the design of response options. For example, if everyone scored 100% across every scenario part and risk area, that wouldn’t tell you much. It might make you think that you’re perfect in every area, but that’s unlikely to be the case. What’s more likely is that the way you’re measuring something is flawed. Maybe the measure is too easy? Maybe you’re measuring the wrong thing?
You need to encourage variation in your data to get useful data and actionable insights. This enables you to identify where individuals, teams, or the organization as a whole may be stronger or weaker. You can then use this information to target resources or upskill where needed most.
To maximize variation, we recommend using at least four response options and that these should cover the full range of potential rankings. For example, you ideally don’t want one response option that is clearly correct alongside three weak response options. This increases people's choices and allows you to use ‘distractor’ options to increase complexity. For example, adding options that would not be considered negative, but also do not do anything to help the situation.
|
Key Takeaway
|