A new study by the firm Kickresume has found that recruiters can be subjective and not always consistent when they review candidates curricula vitae (CV) / resumes.
To establish the data set, six recruiters took part in the research, reading over 12,000 pairs of job adverts and accompanying CVs. To measure the level of agreement between different recruiters, each recruiter was given the same set of 100 pairs. A pair, in this context, consists of a candidate’s resume and a job advert.
Tasked with deciding whether the candidates were suitable or not, it became apparent that they often did not agree with each other – or with themselves.
When reviewing each resume and job advert, they could either answer ‘yes’ or ‘no’. Kickresume used a metric called Cohen’s Kappa which measures the level of agreement between different people’s answers to a question. Cohen’s Kappa can vary from 0, nearly random, to 1, total agreement. The average Cohen’s Kappa score was 0.49, which falls roughly half-way between the two ends of the scale (defined as moderate agreement).
Unbeknown to the participants, they were shown the same CV more than once. When shown the same CV a second time, there was only a 40 percent chance that they would select a candidate they had previously deemed suitable.
These findings suggest that recruiters were found to be subjective and inconsistent.
A further assessment found that recruiters performed similarly to AI. AI demonstrated a level of consistency similar to human recruiters, suggesting the potential for automation in recruitment, albeit with the same in-built biases. For this, Kickresume used an industry-leading AI job matching tool to assess the 12,000 resumes.
Commenting on the study, Peter Duris, the CEO and Co-founder of Kickresume states: “We were fascinated to see the level of variance in the results. One of the most interesting discoveries we made in this research was that AI is already performing at a similar level to human recruiters, in terms of its ability to select the same candidates. AI technology is a fantastic time-saving tool – this research suggests that it might be able to assist recruiters in reading through resumes and selecting the best candidates for the job, or at least excluding those who are not suitable.”
Duris adds: “For job seekers, these results might seem disappointing, but there is a positive way of looking at this. Having the knowledge that recruiters will not always agree on who to select based on their resume means that job seekers can be reassured that it doesn’t necessarily mean their resume is bad or their experience isn’t sufficient if they are not chosen to advance to the interview stage.”
It can be further inferred: “It just means that, on this occasion, the recruiter did not choose them. So I’d advise job seekers to not be disheartened by this! Take another look at your resume, tweak it for the next job you’re applying for, and get back out there.”
