Why Iām distrustful of the second-best predictor of candidate performance
One of the best predictors of candidate job performance is something called General Cognitive Ability (GCA).
Itās so good that GCA assessments are second only to Work Sample Tests when predicting candidate performance. Frank Schmidt, a pillar of industrial-organizational psychology, absolutely LOVES them.Ā¹
But Iām a bit skeptical, not of cognitive ability itself (after all, who doesnāt want to work with smart people?) but of how most people, and more specifically, hiring managers, tend to take common proxies of cognitive ability at face value. Iām also disturbed that many measures of cognitive ability are usually the first filter that recruiting teams use to reject candidates, even for jobs where, letās face it, the employee doesnāt have to be that intelligent.
In this post, Iāll define General Cognitive Ability and outline some of the risks associated with many of the ways we measure it.
What is General Cognitive Ability, and why should I care about it?
In general, we tend to think of GCA as a measure of someoneās intelligence, and thatās mostly correct. GCA assessments have clear right and wrong answers, which reduces subjectivity and allows scale evaluations.
The SAT and ACT college entrance exams are two examples of general cognitive assessments. And while many employers are happy using the SAT/ACT or G.P.A. as a proxy for intelligence, there is also a cottage industry of career-specific assessment companies that also claim to accurately measure a candidateās intelligence and their ability to learn.Ā²
The appeal of GCA assessments and signals is their efficiency. As hiring methods go they offer the biggest bang for your buck. Additionally, they are less time-consuming to take and administer than Work Sample Tests. And, also unlike Work Sample Test, they can be used to measure candidates ranging from entry-level to executive-level roles.Ā³
GCA Assessments sound great, right? Well, before you run off and ask all your candidates to retake the SAT, letās look at some of the downsides.
The case against General Cognitive Ability assessments and their proxies
There are several risks associated with evaluating candidates based upon general cognitive ability assessments and their proxies, things like G.P.A, SAT/ACT scores, or IQ tests. In no specific order, they are:
Common measures of General Cognitive Ability have proven bias. (Oops)
Researchers have proved those common GCA assessments, like the SAT and ACT, underpredict the performance of non-male, non-white groups. That failure is why the SAT has been revamped several times and why some colleges and universityās no longer require it.
Thereās also a growing awareness that money and time can ābuyā better scores on GCA assessments. If you have the time to prep and can afford a test coach, youāre in much better shape than somewhat who doesnāt have the time to learn and practice the strategies that result in a higher score.
G.P.A is also used as a proxy for cognitive ability, and sometimes unfairly. Say youāre evaluating two candidates for an entry-level role. One of them worked 60 hours a week to pay for their mid-tier college, and they have a mediocre G.P.A. The other attended a higher tier school but didnāt have to work to support themselves and has an impressive G.P.A. Do you believe that G.P.A is a fair measure of each candidateās cognitive ability? Iām suspect.
Standard measures of GCA have not kept pace with job requirements.
There are two types of GCA assessments: Tests of Achievement, which measure how much someone has already learned; and Tests of Ability which measure how well someone solves unfamiliar problems. As William Dickens of the Brookings Institution points out, the problem is that it is nearly impossible to test ability without also testing achievement.
Why is this problematic? Well, modern workplaces increasingly require employees to solve unfamiliar problems, often without prior knowledge or context. If current GCA measures rely, even partially, on prior knowledge, they wonāt fairly gauge a personās ability to work in an increasingly ambiguous environment.
Further, the internet is increasingly making Test of Achievement obsolete. Workers have whatever context they need at their fingertips. Test of Ability, being able to quickly teach yourself a skill, apply a framework, and gather the information you need to solve a problem, are now much better measures of general cognitive ability - not because they are more accurate, but because theyāre more relevant.
I suspect that the inability to measure the ability to solve unfamiliar problems is the reason that Lazslo Bock, then Googleās SVP of People, said this in a 2014 interview:
G.P.A.ās are worthless as a criteria for hiring, and test scores are worthless. ... We found that they donāt predict anything.
Particularly in the context of Googleās work environment, Bockās comment makes sense. Test scores and G.P.A. are a rollup of many different Tests of Achievement. In other words, theyāre reflections of what someone already knows, not of their ability to learn and adapt.
Relying on GCA can mask more significant candidate issues
Some of the most intelligent people Iāve ever worked with were huge jerks who eventually alienated nearly everyone around them. I suspect that youāve run across these people too.ā“
Iāve noticed a tendency for myself and my colleagues to forgive the shortcomings of intelligent candidates in the hopes that their brainpower will be worth whatever trade-offs we have to make. But, in my experience, this is never a smart bet. General cognitive ability is not a proxy for decency.
My own mixed feelings on General Cognitive Ability
Personally, Iāve had mixed feelings about this post. Itās true that General Cognitive Ability is a great predictor of candidate success. But Iāve just seen too many hiring managers overvalue a certain school, G.P.A., or assessment score - to the point where they excuse obvious candidate shortcomings, perpetuate existing inequities, and I suspect reduce the probability that we will hire the candidate we need.
We would do much better for our candidates and ourselves if we thought about cognitive ability critically and as a singular data point rather than the first hurdle for a candidate to clear.
Next week Iāll talk about some Doās and Donāts as they relate to evaluating General Cognitive Ability. Spoiler: your first filter on a candidate shouldnāt be their G.P.A.
And, as always, thanks again for reading! If you have any thoughts on GCA Iād love to hear them in the comments below.
1 Need proof? He wrote a paper entitled The Role of General Cognitive Ability and Job Performance: Why There Cannot Be a Debate. As I said, the dude LOVES them.
2 For the purposes of this post, when I refer to āGCA assessmentsā Iām referring to specific assessments from assessment companies as well as the use of SAT, ACT, and G.P.A as proxies for cognitive ability.
3 The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings, 264.
4 Hereās something to help if you find yourself working with a brilliant jerk.
Comments