by Erik Berhhardsson

Humans are inherently prone to confirmation bias. We subconsciously form an opinion about things, and let that influence our decision making. This is dangerous! You start to like a particular candidate a lot for whatever superficial reason, you drop your guard, start giving them a bit more hints or give them the benefit of the doubt in a way that some other candidate wouldn't get.

The main remedy is to have a structured interview process. If everyone goes through (roughly) the same questions, your judgement becomes more objective. The benefit isn't just that it's more fair and you're not perpetuating biases – you also end up hiring better talent.

There's a separate trick I have for trying to make more neutral judgement is to ask myself: what would I have to see in order for me to change my mind about this candidate? If I start out super excited about a candidate, and they nail three questions in a row about system design, then I try to bring out devil's advocate: maybe this person lacks something else? And I switch to some completely different topic, like regular expressions or UNIX commands. Conversely if someone doesn't do well, think of a hypothetical question where they might win you back. Always try to poke holes in your own judgement.

Side note, but a fascinating study on objective interviews is how blind orchestra auditions became prevalent a few decades ago.

Sanity checking your interview process: would your best engineers do well?

The most critical thing is that you learn from your interviews and continuously refine your interview set up. Over time, look at the people that you have hired and ask yourself, what are the strongest predictors of future job performance?

I find this to be a powerful sanity check. Look at the best engineers, sales people, and consultants in your company and try to picture them going through your interview process. Would they do extremely well in the process? Conversely, would the low performance employees do worse? Or you can also look at the current top performers at your company, and ask yourself: what type of interview would have selected these people? If your interview process would have filtered out your best employees, then you should probably reevaluate the process.