The Importance of Coding Exercises In Interviews

A defense of the practice

By dekarrin

As a software engineering professional, I sometimes have the task of helping to interview new candidates for my company. I have a particular approach I like to take for a coding interview that includes a variety of questions; additionally, I've found that a simple coding exercise is a great tool for determining a candidate's eligibility, and I always include one as part of my interviews. I find it so useful that I often advocate for its use in all interviews for hiring developers.

I've run into a little bit of flak for this. Recently, I had a discussion with Jordan Goetze regarding this topic.

In this essay, I will attempt to address these concerns and show why I believe that a coding exercise should be presented to every potential hire, as well as show some additional information on what types of questions are suitable for use as coding exercises.

Table of Contents

• Exercises Considered Harmful?
• Adapting for Problem-Solving Styles
• Making a Good Exercise

Exercises Considered Harmful?

As I often do, I was chatting with Jordan on one of our shared community servers. One thing lead to another, and the topic of coding questions in interviews was brought up, and Jordan noted some concerns that he had.

He mentioned that the abstraction of the process can be very intimidating for some, and that simple logic problems don't really provide insight on someone's ability to work on large scale systems such as full web apps or business logic on backend servers. In particular, he raised concerns that a coding exercise is only really useful for checking if a person can think on their feet under pressure, which is not normal working conditions for most developers.

I absolutely agree that these are some valid concerns, but I don't think that they should stop the coding exercise from being used. I do believe that it should be used with care, and that the decision to move forward shouldn't be a simple yes or no based entirely around whether the candidate excels at solving the exercise, but I believe that leaving it out entirely removes a valuable evaluation tool from the interviewer's toolbelt.

First, if a candidate needs to take time and think during a coding exercise, that's totally fine! I really don't care; hell, I don't even care if they're able to finish the problem or not, as long as they can at least demonstrate skills towards solving a problem and expose their thought process a little bit, whether it be by talking about it heavily or demonstrating it through the way they work through the problem.

Additionally, it's true that being able to solve a simple coding exercise does not tell whether someone can code a backbend server; if there were such a test that were simple and could be done within an interview, I'd gladly use that. So far, I haven't seen anything like that. But the goal of the simple coding problem exercise is not to screen out all but the very best coders; instead, it's to screen out the ones that have zero coding ability at all; that is, those who the company would have to expend resources on in order to bring them up to basic functional competency.

On the other hand, I heartily agree when it comes to the large and open-ended problems (such as "design and code an enterprise payroll system that works within our allotted time"), or the lateral-thinking logic puzzle questions; I don't believe that they prove anything and I further believe that they are a waste of time. I'd maybe give some credence to a "design a large system" question, as long as it doesn't have to implemented. But if the candidate is not yet capable of coming up with an algorithm for, say, testing a palindrome given the definition of a palindrome and the fact that the input is a sting, that could well point to a lack of functional competency, particularly if they are unable to demonstrate any progress towards the answer at all. Real-world problems are rarely as perfectly defined as the ones given in these exercises, and while there are exceptions, they are typically not not going to be any easier.

Now, if that's the case and I ignored the fact that he struggled on the exercise, then why do I even have an exercise at all? Well, interviewing is a tricky task. I consider all portions of the interview together when making my final decision, and as I previously alluded to, I have questions related to social skills and conflict resolution, a coding exercise, and an analysis of past projects as well as the candidate's feelings on them. I also tend to include some language knowledge questions, but I find that these only tend to inform me as to the candidate's ability to memorize coding trivia, and I don't give much weight to them.

Can this method be exploited by a fast-talking social master? Absolutely! Will there be some candidates who would otherwise be great employees who get screened out due to this method? You bet! It isn't perfect, and I certainly don't think that I've mastered interviewing, or even come close to the best solution. But in my limited experience, I've found that this multi-pronged approach is far better than leaving the interview at simple "Name a time you had a problem with a collaborator, and how did you resolve it?"-style questions.

When it comes down to it, FizzBuzz-style questions are an important part of the interview process, and leaving them out will overall hurt the final interview process.

Adapting for Problem-Solving Styles

Jordan went on to say:

I guess my problem is less with coding samples, and more with how they are framed. FizzBuzz comes off as oblique to me because of the naming conventions used, for visual people it makes it difficult to visualize. Now as you suggested, the palindrome example is a little less oblique (assuming you know what a palindrome is) because you can visualize a word in your head by writing it down and working out a method step by step before coding it (and I think the problem solving is the more important part anyways). I would prefer it if coding problems were made more visual.

That made a lot of sense to me, and really made me think about what styles of questions are suitable for a candidate. It would be unfortunate if a candidate bombs an exercise when they they would have done perfectly had the style of the question been different, and it could unintentionally cause a company to favor talent that is good at a particular style of problem solving. I firmly believe that the best collaborative environment can only be made by bringing together skilled people who have a variety of different techniques for approaching problems, and locking out potential hires because they don't confirm to my favored method of problem solving is the last thing I want.

Jordan went on to give an example of the type of problem he would prefer:

"If you have 2 sheep in a pen and every minute they double, each sheep takes 1 meter of space and the pen is rectangular and 64 meters x 71 meters long. Write a program that calculates when the pen will be overflowing"

While yes, this is a simple math problem, one of the required skills is analyzing the problem. e.g. "we want to know when the pen overflows not when it is full" and that "the number of sheep doubles each minute." Then from there, determining how to translate this relatively simple word problem to a math problem, then from a math problem how to approach it programmatically.

I brought up that perhaps a compromise could be reached by presenting a candidate with a selection of different problems, each of which are better suited to a slightly different kind of approach. The candidate could then choose which one he prefers to work on. Jordan was on board with that, but warned that "they should all be relatively the same difficulty; otherwise someone might pick a problem with nicer wording but that has a lot of hidden complexity," which I completely agree with.

Making a Good Exercise

It's tough to come up with a good coding exercise. How do you construct a question that really demonstrates the core competencies necessary in superstar developers?

To answer this question, I ask myself what an engineer does during the course of his work. Generally, we have to take existing issues and problems statements, come up with an algorithm that solves the issue while following the constraints, and possibly change the algorithm as additional constraints are added or removed from the problem.

Of course, the question being delivered in an interview itself has the additional constraint of it being possible to solve relatively quickly. This can be avoided by sending the exercise with the candidate to complete on their time, but I'm just not sure that this would give as much insight to the their thought process.

With all this in mind, I decided that coding questions should demonstrate the following attributes:

• Demonstrates use of flow control.
• Can have additional constraints placed on it to possibly give the coder a chance to modify their solution.
• Can be solved in around 15 minutes or less.

Questions such as FizzBuzz, palindrome detection, and exponential sheep growth fit this model very well, and are what I would consider to be ideal exercise questions.

No matter how the questions are selected, having a candidate go through the exercise demonstrates how they attempt to solve the problem, providing valuable insight into their qualification. FizzBuzz and its ilk are a strong component for determining eligibility, and are a necessary part of any interview process.