Should I have a take-home test or in-person live coding exercise for my software engineer interview process?
Hiring software engineers is hard. Real talk: No matter how hard we try, it is impossible to get a complete picture of how a candidate would fit into and improve your team's performance from a few hours of interviewing. Nonetheless, hiring managers and their teams strive to get the fullest possible understanding of the interviewee's potential.
To that end, one common practice in the software engineering interview playbook is the technical exercise. There are many versions of this, but the gist is that the candidate will plan and code a solution to solve a specific problem. Since the role involves coding, having the applicant solve a challenge using code makes sense.
There are two most common approaches to the technical exercise, with different hiring teams using either or both:
- Live coding — With interviewers looking on, the candidate is asked to plan out a solution and then implement it either on a whiteboard or on a computer
- Take-home test — The candidate plans out the solution and executes it on their own time and then either returns a documented solution or presents their work to interviewers after the fact.
A great deal of debate exists about the pros and cons of each solution. Let’s look at some of the downsides of each approach and consider a third option that evidence suggests is superior.
Take-home tests
I’ve worked with several senior developers who refuse to do take-home tests.
Too many folks have been burned by bad experiences, putting in hours of work and possibly getting little or no response to show for it. Typically, companies that pass on a candidate won't give meaningful feedback for fear of legal repercussions. Candidates who haven’t experienced this directly have heard horror stories from friends.
If you require such a test in your hiring process, you may be missing out on a substantial pool of potential candidates.
Live coding
On the other hand, thoughtful interviewers understand that live coding exercises are highly artificial and cause anxiety ranging from mild to completely debilitating. This frequently results in false-negative evaluations. See Does Stress Impact Technical Interview Performance?.
For most engineering roles, you want to optimize your interviewing process to find the most effective engineer rather than who is least susceptible to performance anxiety.
A third way
In the paper linked above, the researchers suggest (and successfully test) a third approach which is somewhat of a hybrid.
Here, the candidate is given the problem and asked to code a solution, just like the live coding option. However, once any questions from the candidate are answered, they are then left alone to work for a specified amount of time, instead of being watched (and expected to "think out loud") while they implement. This gives the interviewee space to consider, explore, and discard unsuccessful approaches, which more closely aligns with typical software development "in the wild".
This massively reduces performance anxiety and leads to less stressed candidates and a more realistic sense of their capabilities.
After the candidate has their private work time, the interviewers come back and the applicant walks them through their solution. This is referred to as "retrospective think-aloud". This time gives the interviewers a chance to not only understand the technical prowess of the candidate but, critically, also their communication skills.
Given that there is still a time limit, this approach involves some anxiety, but it is more in line with the type of stress an engineer will typically face in the real world. Deadlines are, alas, a reality.
Decisions, Decisions
So, which is the best approach?
First, understand the requirements of the role you are hiring for. Some positions — though not most — necessitate the ability to plan a solution and code while others are watching.
The vast majority of positions do not require this skill. Most coding is done individually or with a trusted partner, and this skill is more authentically tested via an approach that includes private work time.
That said, I encourage offering the interviewee a choice between two or three options. Giving a choice gives some power back to the candidate, buys you goodwill (remember that when you're evaluating, you’re also being evaluated), and gives you the best chance of finding the strongest hire.
In practice, providing choice implies more setup and more maintenance work. To mitigate this, aim to keep the exercise relatively simple and avoid excessive structure.
Summary
The interview and hiring process is intense for both companies and candidates. Offering a flexible approach to technical exercises makes companies more confident they see the best of what someone has to offer and makes candidates more comfortable in what is typically an uncomfortable situation. Give it a try, and let me know what works for you!
— Matt Morris, The Imperfect Builder