In my vitriolic diatribe on puzzle-solving interviews, I argue that making engineers solve puzzles as an interview strategy does nothing but confirm what their transcript already tells you, and over time yields an engineering staff incapable of strategic thinking. To my credit, I did not use the term “boneheaded,” which speaks to my inimitable sense of self-restraint.
Rebuttal from a reader: “Perhaps some real value can be found out of the puzzle question response by looking for out-of-the-box, creative thinking (both in problem analysis and solution exploration). Even if they get the thing wrong (which is not necessarily a bad thing since most puzzle questions are designed to be devious and tricky), they may prove the ability to look at the problem from a unique perspective and think critically as they problem solve.”
Agreed. There is some value in watching a person solve a puzzle in order to see not just whether they can solve the puzzle, but how they approach the solution of said puzzle. But this insight does not address the profound problem with the entire notion of a puzzle-solving interview in the first place: Puzzle solving is not even remotely the most significant skill of a superb professional software engineer!
Software is built by humans within the context of social structures. Software construction is dramatically affected by a personal productivity variable of at least an order of magnitude. That means that my good day is at least ten times more effective than my bad day. Equally important, one engineer may, on an average day, perform at least an order of magnitude more productively than another engineer on that same average day. Engineering productivity is massively impacted by the collective cognitive and emotional state of a given team, not to mention the dynamics of that team as it interacts with the rest of its company (and even further, with the company’s partners and customers).
If you want to really know whether a candidate is going to save your bacon six months from now, stop asking the puzzle questions. Now.
If you want smart and clever engineers, then limit your recruiting to schools you trust and require a minimum GPA of the students you interview. If that’s not good enough for you, then actually read their transcript and look at how they did in specific courses.
Now that you’ve determined the candidate is smart (see above), try to figure out these three things:
1) Can this candidate think intelligently about a world outside an engineering cubicle?
2) What is the ratio of bad to good days in the engineering life of this candidate?
3) Is this candidate an effective written and verbal communicator?
You then have to develop great interview questions that evoke meaningful answers to each of these three issues. I call them, “The only interview questions you’ll ever need.” All will be revealed in the next few posts.