Humane Software Engineering Interviews

Software engineering interviews are bad. While we’ve all almost universally acknowledged this fact as an industry, there’s been little movement to actually shift to whatever might be considered a ‘good’ interview.

Almost no process that I (or many friends and colleagues) have been part of has actually mirrored what happens in the work place when it comes to software development. It seems like it’s probably time to examine a distillation of most interview processes and analyze each step to see where we can make improvements.

Recognizing that you will almost never have a perfect picture of how a candidate operates is really important before embarking on any interview process. If you’ve designed a process that does give you a perfect picture it might be overly onerous on candidates. Any hiring decision is a bit of a gambit, the more your interview process deviates from the reality of how teams operate will increase the risk of that decision.

Current Process

For the sake of digestion we’re going to break up the current process into the pre-onsite and the onsite portions of a typical interview cycle. We’ll hold critique until we get into the questions section.

Pre-onsite Process

As we can see, the process is complicated even before a candidate comes onsite. It should be noted that this is a rough generalization of the various interview processes that I’ve personally taken and have also administered.

At this stage the basic idea is trying to get some early signal on the candidate’s abilities. You may even have a manager phone screen somewhere in this process before it gets to the point where a candidate is sending in code or doing a technical phone screen. Some places are trying to ensure they are making managers a high pass filter at the top of the funnel (I personally agree with this mentality) before asking for engineers to review a take home or do a synchronous phone call.

Onsite Process

Most software onsite interviews are an all day affair comprised of a series of technical and non-technical interviews. The mix between these two buckets of interview types will vary depending on the position and level you are interviewing for, but here’s a (non-exhaustive) list of interview types that are typically employed:

• Take home project extension
• Domain specific live coding exercise
• Computer science grab bag question (tree traversal, hash maps, etc.)
• Debug this code sample
• Résumé review (go through each job listed and ask the same set of questions)
• The Whiteboarding Architecture Interview™
• Role playing a workplace situation

There will likely be anywhere from 5 to 8 interviews ranging from 45 minutes to an hour each. Hopefully they have scheduled breaks for you otherwise it will truly feel like a marathon.

Emergent Questions

Through reviewing this process, we’re led to a set of questions about the usefulness of this pre-designed process that should be considered when attempting to build out a new framework. Let’s answer some of these questions that may have been brought forward in order to frame how our new framework should operate.

What’s the goal of the interview?

This is a great question, and one that should be asked of anyone participating in an interview process. If you don’t know the goal, how can you successfully participate?

I would venture a guess that most software interview processes have the goal to try to understand what it would be like to work with the candidate on a day to day basis. Ultimately, I would say it’s impossible to judge the totality of whether or not you could work with someone through a finite set of interviews as humans are very complex beings.

However, setting out to try to answer this question isn’t a bad thing. Creating alignment with interviewers (and candidates) is hugely valuable to ensure that the feedback that is received has the same question in mind.

Does this process mirror how we work internally?

If you are guiding your interview process with the desired outcome being whether or not you could work with a person daily, you must examine the process to ensure it mirrors your typical work process.

There might already be some pieces of how you work embedded in the current interview process, but there are likely other pieces which have no daily analog. This lack of analogous daily rituals can be a ‘smell’ used to diagnose ineffective pieces of your interview process. Giving the candidate the opportunity to explore your team’s regular process is valuable to everyone involved to see if that process actually holds water.

Should different people be involved in this process?

Most engineering teams are in collaboration with at least one or two other teams that might contain different job functions. Are these folks part of your interview panels? If not, should they be? Meeting and interacting with folks doing different jobs (and thus, different perspectives) could be an invaluable set of feedback to gauge this lofty question of whether or not you could work with this person successfully.

Many of these interview processes that we all go through only include engineers, what about product managers and designers input? They are hugely valuable in these processes since their angle of approach is totally different.

A New Framework

Reading these questions, it’s clear to me that in many cases we have work to do. Many interview processes do not effectively mirror what a typical cycle of work looks like. As mentioned, many don’t include other important roles to give input on the various skills that engineers might not see.

For the remainder of this discussion, we’re going to focus solely on the onsite interview process and some remedies we can conceive of. It should be noted, that the following approach could be extended to the pre-onsite portion a well (I think). While the pre-onsite part of the interview process is important as well, it is complex enough to warrant it’s own post at some point probably.

Most onsite interviews that pull from the aforementioned set of examples we’ve listed in a previous section inherently introduce a high amount of context switching into the onsite process. They are designed to be completed in isolation and graded in isolation. On the face of it, this is not a bad thing when you’re solving for having a variety of problems that you could ask in your process. However, these are all disjointed. How often do we go from using a shared text editor to parse items in a two-dimensional array to whiteboarding a completely unrelated question to extending some code that we’ve already written? It’s just not rooted in reality. Also, the candidate doesn’t come away from the interview learning anything about how your teams work together, which is bad.

Project-based Learning

I’ve described this interview framework to a few friends and coworkers, one of which used to be a teacher. This teacher gave me better language to describe this framework (it’s not a new idea!), and it’s called project-based learning. The general idea is that you pose non-trivial problems to students and give them the space to ask refining questions, debate ideas, make predictions, and design plans/experiments among other things. This is a very good description for how most software development processes actual play out within companies. Granted we’re time constrained in an interview context so this isn’t a perfect substitution, but it’s moving in the direction of mirroring how we work, which is good.

Project-based Interview

So with our brief introduction to this project-based approach, what can we do with our onsite process? We’ve talked about how we have a desire to have our interview mirror a bit of how we actually work to both give the candidate an idea of what to expect if they were to get the job, and for us to get actual signal if this person could perform the regular duties of the job in question.

For this example, let’s start with assuming that most development processes have 3 parts:

1. Designing a solution.
2. Implementing a solution.
3. Getting code review for a solution.

These could each be an excellent interview within a project-based theme for someone’s onsite interview day. Generating a project theme can be as easy or hard as you’d like. I personally like the idea of extending the candidate’s take home project (if they submitted one) with a new feature, but having them go through each of these steps in different interviews that can be reviewed individually.

Being able to think about one consistent thing throughout the interview day will give (I believe) more clarity to the entire exercise, allow both candidate and interviewer to go more in depth with each interview, as well as be something that can be a conversation at each step of the way. While it might seem weird to have a candidate work through a Google Doc template writing out how they’d build a new feature in the app they submitted, it’s something that many engineers do every single day. Granted this example is based on my experiences, but you should examine how your teams design features before implementing them and mirror that.

These broad interviews also give you the ability to add other people to the process. You could easily have someone from design or product be part of the designing the solution interviews to work through constraints or desired look and feel. This provides a natural place to get the larger team involved without feeling forced.

The overall idea is to put the focus back onto the candidate, not the problem. These interviews can be supportive, questioning, and thoughtful just like they are outside of the interview context!

–

Project-based learning is only one way to re-implement an onsite interview. The more important part here is to take a mirror to your interview structure and ensure that it mirrors how your teams work day to day. Give your candidates a process that let’s them see how you work and make the decision if that’s how they want to work and learn. Refocusing your interviews onto how the candidate explores your process and works within it (or outside of it!) will yield higher quality signal that you can map to every day experiences.