One role.
100+ applications.
The clock is ticking and you’re up to your neck in CVs.
You’ve shortlisted some promising candidates…
In a post-COVID world where remote working is quickly becoming the norm, hiring the most talented and engaged people has never been more important.
And given that the top 1% of employees are 25x more productive, how can you prove that you’re hiring the right people?
Well, here’s a promise: using a process known as ‘blind hiring,’ you’ll hire better quality candidates from a more diverse range of backgrounds… and have the data to prove it!
What is blind hiring?
Blind hiring does what its name suggests: it ‘blinds’ applications by removing identifying information so that candidates remain anonymous.
The reason for this is fairly simple; humans are prone to unconscious bias.
So, if we want our hiring to be objective and our decisions to be accurate, then we’ve got to remove any information that triggers bias.
This might sound straightforward enough, but, as we’ll cover shortly, genuinely data-driven, bias-free hiring takes more than just scribbling over names with a Sharpie.
Before we continue, here are a few of the key benefits of blind hiring:
- Improved diversity as a result of bias-removal.
- Better quality of candidate due to the use of more predictive assessment methods.
- More engaged employees who have already had a taste of what the job entails.
Why blind hiring? Because minority-background candidates end up being overlooked
When we’re left to our own devices, minority background candidates miss out on opportunities, whilst a select few with the most elite degrees and experience get the top-paid positions.
In the UK: Inside Out London’s study found that candidates with Muslim-sounding names are 3x more likely to be passed over for a job.
In the US: A 2004 study discovered that candidates with an African-American name would need an extra 8 years of experience to get the same number of callbacks as someone with a white-sounding name.
In Germany: Another similar study found that candidates with a Muslim-sounding name, who were pictured wearing a headscarf, were 15% less likely to receive a callback than their white counterparts.
Unconscious bias occurs when we use the wrong part of our brain to make decisions.
Our brain has two systems for decision-making:
System 1: fast, intuitive thinking which relies on snap-associations and shortcuts.
System 2: slow, conscious thinking, used for one-off, complex decisions.
Since we make 1000’s of micro-decisions every day, we need System one to pick up most of the slack.
However, we often also end up using system one when we should be using system two.
We fall back on shortcuts and patterns to draw conclusions, when we should be thinking more consciously.
And since we don’t even know when we’re using the wrong system of thinking, unconscious bias can be hard to combat without changing the hiring process itself.
Unconscious bias training simply doesn’t work
In an attempt to combat unconscious bias, organisations are spending millions on training.
Google alone spent $114 million on diversity-related programs back in 2014.
However, training alone hasn’t been proven to yield any tangible results.
A study of 800+ companies over 31 years showed that diversity training had no positive effects in the average workplace.
Because most of our bias is unconscious, simply making people aware of it won’t change behaviour.
Instead of trying to change how people think, we have to change the environment in which they make decisions.
The most well-known example of this approach was back in the 70s.
Researchers managed to double the number of women getting through auditions by introducing blind auditions.
So, although we can’t change human nature, we can change the systems and processes in which humans make decisions.
The only way to remove bias by design – here’s how it’s done
Step 1: CV Screening
Your average CV is comprised, almost entirely, of information that could trigger bias.
CVs have been knocking about since the 50s, and unlike a vintage bottle of Dom Perignon, they’re not getting any better with age.
Take a look at how biases affect your run-of-the-mill CV screening:
There are very few parts of the CV that don’t invite one bias or another into the decision-making process.
So, to achieve truly accurate, bias-free hiring, the CV has to go… but in favour of what?
Education and experience don’t matter as much as you think
For the most part, blind hiring best-practice will urge you to remove names, addresses, and date of birth from CVs.
But the truth is: simply blinding personal details doesn’t go far enough.
Because, as you can see from the image above, what’s left is still grounds for unconscious bias.
In fact, if you went ahead and ‘blinded’ all the information that could lead to bias, you’d be left with nothing!
Although education and experience are the bread and butter of traditional hiring, they’re not actually reflective of real-life ability.
The chart below is based on Schmidt & Hunter’s famous metastudy looking at the predictive validity of assessment methods.
As you can see, your trusty staples of the CV – education and years of experience – are some of the poorest indicators of ability.
Use ‘work samples’ instead of CVs
If we look at what is predictive – it’s ‘work sample tests.’
These are job-specific, forward-looking questions/ tasks designed to simulate the role as closely as possible.
Here’s an example of a work sample question for a Customer Success Manager role:
To create these questions, you’d start by boiling the role down to 6-8 essential skills.
Once you have these skills down, think of a situation that would occur in the role that would test each skill – you can test more than one skill with a single question.
If the role you’re hiring for isn’t a new one, you could look at situations that have come up in real-life.
You can then turn these situations into questions by posing them hypothetically.
Think of work samples as bringing the value of the interview forward to the screening stage.
With just a CV to build a picture of how someone works, you’d usually have to wait until the interview stage to get a feel for how a candidate thinks and tackles tasks.
The difference between a work sample question and a bog-standard interview question is that you’re asking what someone would do hypothetically, and not simply whether they’ve experienced that situation before.
Work samples test for skills learned through experience – not experience itself.
And, by asking candidates to think as if they were already in the role, you ensure that anyone you do hire is already engaged with your organisation and its mission.
The first step to nailing employee experience is the hiring process itself. When candidates know what to expect, they’re able to hit the ground running!
The importance of scoring criteria
To achieve an objective hiring process, you’ll need to give yourself some data points.
Traditionally, as candidates progress through the process, hirers will begin to build a mental picture of their skill sets and form opinions about them.
However, we tend to look for information that confirms our opinion and ignore anything that might challenge it (this is what we call ‘confirmation bias’).
So, if you like the look of a candidate because of their past experience or degree, you’re likely to view their interview answers/ skills more favourably.
To start making decisions using data (instead of gut-driven bias), give yourself scoring criteria.
For each work sample and interview question, equip yourself with a 1-5 star scale to mark answers against.
This doesn’t need to be overly detailed, just ask yourself this: what qualities would a poor, low-effort answer look like, what would candidates have to do to get an average score, and what would they have to do to really impress and get 5/5?
You can simply jot down a few bullet points at each level, just like the example below:
Example scoring criteria from Applied platform
At the end of the process, instead of just having a ‘feeling’ that a given candidate is the best person for the job, you’ll be able to build a candidate leaderboard by either adding up or averaging out each candidate’s scores.
Having objective criteria also means that you can get other team members involved in the scoring – something which is strongly recommended.
We’ve all heard that two heads are better than one, but this isn’t just a saying – the science backs it up!
By getting others involved, you’ll be harnessing the power of ‘crowd wisdom.’
This is the general rule that collective judgment is more accurate than that of an individual.
The optimal number of reviewers is three – after that, you’ll see diminishing returns.
Once you have a means of objectively judging candidates, you can begin to measure the quality of candidates against other metrics.
For example, you might want to start tracking your sourcing channels.
Not all job boards are created equal, and so by looking at candidates’ scores you can track which ones bring the best performing applicants.
Bias-free screening – putting it all together
You’ve created your work samples.
Candidates have (anonymously) submitted answers.
However, there are still ordering biases at play that could sway decision-making.
For example, if a candidate answers one question particularly well, you’ll probably perceive the rest of the answers in a more positive light than you otherwise would’ve (this is known as the halo effect).
This is why at Applied, we recommend ‘chunking’ and randomising work sample answers.
Here’s how the screening process works from start to finish:
Anonymisation: all identifying information is removed from applications.
Chunking: each candidate’s application is split up into the individual answers, so you’d score them question by question, not candidate by candidate.
Randomisation: candidates’ answers are shown in a different order for each question.
Review: answers are scored against the same criteria by three team members.
When you put it all together, the blind screening process looks like this:
Step 2: De-biasing interviews
Naturally, interviews are going to be open to a certain degree of bias no matter how you conduct them. Even if you interviewed blindfolded (which the orchestra interviewers essentially did in the example above), even candidates’ voices can invite bias into the decision-making process.
Although they shouldn’t – first impressions matter…
According to one study, roughly 5% of interview decisions were made within the first minute of the interview, and nearly 30% within five minutes.
All that being said, however, there are still actions you can take to ensure that bias is kept to a minimum, and that your interview stage is actually predictive, whether it’s a regular face-to-face interview or via video.
Start conducting ‘structured interviews’
Structured interviews are the first step towards fairer interviewing.
A structured interview is when all candidates are asked the same questions in the same order.
Just like your work samples, you’ll want to ensure that each interview question is accompanied by scoring criteria.
Why? Because interviews can be subject to something called the peak-end effect…
Peak-end effect: both the intense moments and final few moments of an experience have a significant impact on how we recall and perceive the experience as a whole.
Let’s take a look at the peak-end effect in action…
The study below found that the group of participants rated their colonoscopy as being more painful when it ended more painfully.
So, what has this got to do with interviews?
Well, a candidate that ‘finishes strong’ or answers one or two questions particularly well is likely to be perceived as being more suitable than they perhaps should be.
This is why we use criteria to score against – it means that every question has equal merit so that you can map their skills accurately without bias leading you to unfairly favour one over another.
Ditch the ‘tell me a time when’ questions
The questions you ask in an interview matter, too.
You should avoid probing candidates about their experience for the same reason you should axe the CV: experience just isn’t as predictive as many believe it to be.
Instead, use work sample-style questions to get an insight into how candidates think.
Take a situation or task, and see how candidates would tackle it should they get the job.
Questions that start with ‘tell me a time when…’ aren’t entirely useless – but they aren’t phrased to get the best out of candidates.
By asking backwards-looking questions like this, you’re preventing candidates who haven’t had a given experience yet from really shining, even if they have the right skills!
Just because they haven’t encountered a situation before doesn’t mean that they wouldn’t be fully equipped to deal with it.
To get more value out of these types of questions, simply pose them hypothetically.
Ask what candidates would do, not what they did do.
Take breaks!
Taking regular breaks between interviews is crucial to reducing ordering biases.
With every decision that we make over time, we get increasingly fatigued, which in turn leads us to make harsher decisions. At the start of the day, when we have energy, we’ll usually be more open to risk and more generous (in the case of interviews this might be towards ‘outsider’ candidates). As we get fatigued, we become more risk-averse and less generous. This phenomenon is known as ‘decision fatigue.’
In one famous study, it was uncovered that Israeli judges made less favourable parole decisions as the day went on.
Do you see the spikes? That’s when they had their breaks.
And that’s why you should be taking regular breaks, too.
Ideally, you should also be wary of cramming several interviews into a single day, so that you approach each interview with a similar level of fatigue (hopefully none).
Step 3: Making a final decision
When it comes to actually hiring someone, all the hard work should already be done.
If you scored candidates question-by-question for both work sample and interview questions, you can simply average out their scores to build a leaderboard like the one below:
Screenshot from Applied platform – you can build your own leaderboard using Google Sheets.
If you keep your scoring system simple, you shouldn’t have too much trouble whipping up a leaderboard.
Assuming all questions are weighted equally, you can take an average of all scores across all stages of the hiring process, to see who the best person for the job is.
This will also make giving feedback easy, since you can objectively point out where candidates did well and the areas in which they could improve.
Completely overhauling how you hire is a daunting prospect, especially when it comes to losing the CV.
So, if you’re still sceptical, try this:
For your first stab at blind hiring, use work sample questions and CVs… see how candidates perform in comparison to your initial expectations based on their CVs.
Chances are, you’ll be in for a shock.
Originally published Jan 03, 2021, updated Jan 16, 2023
