Adaptive Assessments Manager Overview

What are adaptive assessments?

Adaptive assessments accurately measure an individual's current skill level, allowing you to target upskilling content and monitor their improvement over time.

How they work

Imagine a bank of questions, ranging from easy to challenging, all designed and tested by subject matter experts. When a user takes an adaptive assessment, they start with questions that match their initial responses. 

• If they answer correctly, they will get progressively harder questions.
• If they answer incorrectly, they will get easier questions. 

This continues until the assessment accurately pinpoints their current skill level.

Most adaptive assessments take around 10 minutes to complete.

Benefits

• Personalized Learning: These assessments are a valuable tool to identify the most suitable learning content for their team members, ensuring everyone gets the most out of their learning journey.
• Track Growth: Users can retake adaptive assessments later (for example, after completing a program or career path) so that you can clearly see how much their skills have improved.

Where can adaptive assessments be found on the platform?

Assignable adaptive assessments

The new assignable adaptive assessments can be found in the Upskill catalog alongside our other assignable content. If a specific area has adaptive assessments available, they will appear above the existing lab collections for that area.  

For example, when you click Upskill > Defensive Cyber, a heading labeled Assessment appears at the top of the page.  This will show if there are any baseline and re-assess adaptive assessments available for this persona / content type.

We also have Career Paths that feature a baseline and reassess adaptive assessment, which can be identified by the Career Path with Assessment label on their cards.

User onboarding (Assess and Recommend) 

We also offer a subset of adaptive assessments as part of the onboarding process for certain technical roles (Cyber Professional, Infrastructure Penetration Tester, Web Application Penetration Tester, and SOC Analyst). These are used in our Assess and Recommend feature to help new team members identify relevant career paths (for example, guiding them to beginner, intermediate, or expert learning paths). 

Note: These specific assessments do not provide a score and are not assignable by managers; they are a voluntary tool for learners to get recommendations on where to start.

How are adaptive assessments created?

Our adaptive assessments are designed by our cybersecurity subject matter experts (SMEs). For each assessment, they develop questions tailored to the essential knowledge, skills, and abilities required for a particular role.

Each assessment draws from a comprehensive question bank, featuring a wide range of questions categorized by difficulty: beginner, intermediate, and advanced. 

The assigned difficulty level of each question directly influences the Computerized Adaptive Testing (CAT) algorithm that underpins our adaptive assessments.

To ensure the accuracy and validity of both the assessments and their question banks, an independent verification process is undertaken. This involves further collaboration with our SMEs to establish an expert consensus on the difficulty levels and to confirm appropriate coverage across beginner, intermediate, and advanced topics.

What is Computerized Adaptive Testing (CAT)?

Our adaptive assessments feature is built on Computerized Adaptive Testing (CAT) standards. CAT is a modern and efficient way for users to take an assessment. Unlike traditional tests where every test-taker receives the exact same set of questions, a CAT system dynamically adjusts the difficulty of the questions presented based on a user's performance.

CAT offers the following benefits:

• More precise measurement: Because the test adapts to a user's skill level, it can more accurately pinpoint their strengths and weaknesses. Users won't waste time on questions that are too easy or become overly frustrated by questions that are far too difficult.
• Shorter test times: CATs are  shorter than traditional fixed-form tests while still providing a highly reliable score. This is because the system doesn't need to ask every possible question; it efficiently hones in on a user's ability.
• Fairer assessment: By adapting to each individual, CAT helps to ensure that everyone is assessed at an appropriate level, leading to a fairer and more accurate evaluation of their knowledge or skills.

What kinds of adaptive assessments are available?

We currently offer adaptive assessments in areas such as:

• Offensive Security
• Application Security
• Operational Technology*
• IT Operations
• Infrastructure Security
• Web Application Security
• Security Operations
• Cybersecurity

*Subject to license package

What reporting is available?

At the end of the assessment, users will see their score and skill level ranking (see above).  They will also be shown their personal best score, and a rocket icon will highlight their personal best score.  If they have taken the assessment more than once, they can view their previous attempts in the drop-down list. 

Assignment report for managers

Managers can view the scores of their users as part of their assignment report. 

To access the assignment report, from the main navigation menu:

• Click Launchpad > Manage assignments, and then click the assignment.
  Or
• Click Manage > Assigned Activities, and then click the assignment.

If the user has been assigned an assessment, the user can also view their score by navigating to Launchpad > Assigned to me, clicking the assignment card, and then selecting the My Progress tab. 

How can adaptive assessments be used by managers?

Baseline and re-assess adaptive assessments can be assigned to individuals and teams by managers to:

• provide an initial baseline assessment of their strengths and capability level.
• inform the planning of targeted upskilling programs that are tailored to an individual's current skill level.
• reassess strengths and capability level at a later point to demonstrate improvement over time.

Adaptive assessments can be assigned as a single assignment for users to complete and use the same flow as the current assignment of lab collections.

As with the assignment of lab collections, managers can view completion data for open and closed adaptive assessments. They can also view overall scores.

How many times can a user take an adaptive assessment?

If the assessment is standalone and not part of a career path, a user can take the assessment as many times as you want.

If the assessment is part of a career path, users take the assessment at baseline and are re-assessed at the end of the career path.

How do adaptive assessments fit within our program’s vision?

Adaptive assessments are a crucial element of our program’s vision. They also provide a unique, effective, and robust method to baseline and continually re-assess individuals, so that improvements and change over time can be demonstrated to both learners and managers.

How is the scoring/ranking determined? 

Our adaptive assessments are designed to computerized adaptive testing (CAT) standards used within psychometrics. 

They use the CAT algorithm to generate a "theta" value to represent a user's capability. This is a continuous statistical measure where higher values indicate greater ability. The CAT algorithm continually refines this theta estimate based on a user's responses to questions.

Once the assessment concludes and a precise theta value is determined, it is then transformed into a more intuitive scaled score, ranging from 0-300 and split into beginner, intermediate, and expert. These groupings align with our content catalog and are used within industry to describe skill levels.

Score boundaries are:

• Beginner: 0 - 128
• Intermediate: 129 - 214
• Expert: 215 - 300