Testing our patience: traditional skills assessment earns a failing grade

As has been extensively documented, the world is currently experiencing a skills gap that threatens to decimate economies. The global economy stands to sacrifice trillions of dollars in productivity and economic development, directly attributed to this skills gap. It is high time we stopped referring to it as a “skills gap”; we are actually drowning in a “skills canyon”.

How can this be? Artificial intelligence, machine learning, a plethora of new and dynamic training approaches have risen, and in-person, virtual and interactive training methodologies are creating training experiences that should be filling the skills canyon with gold. What reason could there be to justify this massive shortfall?

The story

Without ever having met you, I can tell you one thing you have in common with your great-great grandfather: you both answered tests the same way. Multiple Choice. True or False. Long Answer. At best, you got some fancier questions, and maybe took your test on a screen.

In a world with blended learning models, and understanding that people learn at different speeds and in different ways, why do we continue to lean on static and monolithic testing? These tests are not assessing a candidate’s skills. If candidates are not given an opportunity to have their skills assessed, is it really any wonder that the skills gap is as vast as it is?

The scenario

In most job-related training initiatives, the ultimate assessment is the traditional assessment. Multiple choice, true or false, and the occasional long answer are favoured for their simplicity. Unfortunately, they prove nothing other than an ability to memorise a vastness of material in a timeboxed manner. This is not a skills assessment; this is barely a knowledge assessment.

At the heart of the matter is the significant misalignment in how we define “successful training” versus how we define “successful skills development”. While efforts to innovate training are plentiful, efforts to innovate assessment are reserved to fancy questions and a plethora of new surveys. As a result, we are becoming extremely skilled at clicking circle-shaped buttons.

The problems

Despite substantial evidence that people learn in different ways, we have continued to assess competency in the same traditional way. We have created dynamic, blended-learning opportunities, but have not done the same for assessment. Combining blended learning with static assessment has proven unsuccessful, and instead created barriers and stunted accomplishments. We overvalue the operational ease of systemic testing and erode our chances for true skills development.

If a candidate struggles with American English, is a newcomer, suffers from dyslexia or another neurodivergence, or is simply a bad test-taker, their years of training will come down to a high-stakes exam – one they never had a fair chance to pass to begin with.

A paradigm shift is needed.

The implications

In a 2020 report, Accenture revealed that G20 countries risk forgoing up to $11.5 trillion in GDP growth over the next 10 years if they are unable to adapt the supply of skills to meet the needs of the new technological era. In the same year, McKinsey revealed that nearly 9 out of 10 executives surveyed are imminently facing skills gaps, and in 2021 the Future Skills Centre in Canada in partnership with Ryerson University and Microsoft expounded that only 1% of nearly 300 Canadian executives surveyed confidently believed that their new hires and recruits had the skills to perform the jobs for which they were hired.

But these skills challenges are not limited to technological “skills of the future”. This problem is pervasive across society and the entire economy:

1. Healthcare

In its 2020 “Jobs of Tomorrow” report, the World Economic Forum surmised that 40% of all new jobs in the next three years would be in the care sector. The COVID-19 pandemic unveiled scandals and criticisms that have made commonplace words like negligence and malpractice in describing long-term care homes. A greater, competence-based skills framework could have been effective in this regard.

2. Literacy

In Canada, 42% of the workforce has literacy skills below those needed to be fully effective for the position they hold. Notwithstanding the “hard skills” gap, the soft skills gap is paralysing economies. What’s most astounding is that a mere 1% gain in literacy abilities could lead to a 5% gain in productivity, worth the equivalent of $54 billion a year in Canada, alone.

3. Immigration

In the US, it is estimated that $39 Billion in wages – and $10 Billion in income tax revenue – is lost every year due to college-educated immigrants being mired in low-skilled jobs or unemployment lines. In Canada, more than half of all full-time taxi and ride-share drivers are immigrants, of which a majority hold degrees and advanced skills-based diplomas that render them overqualified for their driving jobs.

4. Military and defence

Every year, hundreds of thousands of military personnel require skills training to reintegrate into the workforce. VALID-8, a patented platform leveraging video assessments, was used with the British Army in over 130 countries to assess transferrable skills of military personnel reintegrating into the economy, giving members of the military a greater opportunity to effectively compete for civilian jobs.

Part of The Solution

In 2020, Vametric and others worked with state legislators in Utah to adopt legislation removing hours and exams as the threshold for advancement in apprenticeship programmes. Instead, Utah will investigate the use of mastery instead. This focus on actual ability and competence is a step in the right direction.

The economy needs assessment that is focused on achievement and ability in order to validate the development of skilled workers. Continuing to exclusively leverage traditional assessment doubles down on a broken assessment model, and reinforces major race, gender, language and ableist discrimination patterns that our global, national and regional economies simply can’t afford to have.