Yeah, I know I promised three follow-on posts after the last one on puzzle-solving interview questions. I’m getting to it. Really. But I wanted to take a minute first and specifically address some of the comments left on the last two posts.Here’s one:

In my opinion, there’s nothing wrong with hiring engineers primarily for their puzzle-solving skills; that’s their primary function.

I’m going to respectfully disagree. A software engineer’s primary job function is to write software, which involves a number of things, one of which looks a bit like puzzle-solving. But the driving function of the complexity and cost of software construction is neither writing software, nor its puzzle-solving component. The driving function of software construction is the collective emotional state of a software engineering organization.

Individual engineers have to be decently smart. Spectacularly smart is even better. But uber smart with a whiny attitude is a no hire. The smartest new college graduate at some university who can’t stay focused for more than three weeks at a time is a no hire. The kid who solves the Rubik’s cube in less than a minute blindfolded but brings other team members down emotionally is a no hire. The brilliant prima dona is a no hire. The defeatist genius who minored in learned victimhood is a no hire. Every single one of these guys can pass the puzzle questions with flying colors. Every single one is a no hire.

Failure to interview for emotional attitude, team play, communication, and can-do attitude ABOUT THINGS OTHER THAN PUZZLES is failure to interview for a successful engineering corps.

But what about division of labor? Why does an engineer have to think big? Can’t the high mucky mucks think the big thoughts and leave the puzzle-solving engineering crew to hunker down?

To wit, this comment:

When you expect or allow those same engineers to decide the direction of the company is when you’re headed for disaster. Large companies like Microsoft and Google can get away with this practice if they balance out their spiffy puzzle-solvers with globally-thinking big-picture people. Smaller companies can’t afford to specialize and compartmentalize their workforce that way and should follow different hiring practices.

Again I’ll respectfully disagree. The notion that there is some clear separation between the engineers in the trenches and everyone else in the company up to and including the CEO is a myth. The collective attitude of an organization, for better or worse, is manifest in everything the company produces, including the products it ships, the ad campaigns it runs, the personality of its sales force, the attitude of the secretary at the front desk, and the code that engineers produce, not to mention the rate at which they produce it.

Personal case study: In 1988 I was a software engineer in the Personal Computer Group at Hewlett-Packard in Sunnyvale, California. I showed up just before HP’s OEM release of OS/2 1.0 for the Vectra PC, and then stuck around to write device drivers for OS/2 1.1. I had access to all the source code for OS/2, which had been jointly developed by Microsoft and IBM. As I cruised around the code for OS/2, it was always immediately obvious which code had been written by IBM engineers and which had been written by Microsoft engineers. In all cases, the coding style was consistent with the collective corporate attitude that you could see on a billboard from a clogged expressway. You could drop code in front of me and I could tell you which company it came from.

Engineering work, even among the most junior engineering staff members, involves creativity and decision making that has nothing to do with puzzle solving. Individual engineers make stylistic decisions about design issues that were not fully clarified by someone outside their team. Attitudinal decisions that affect code maintainability happen every single day. A renegade codeslinger who refuses to follow the corporate style guide has a negative effect that no amount of puzzle-solving speed can repair.

The idea that there is a wall of separation between the puzzle-solving work of the engineers hunkered down in their cubes and the collective vision of the corporation is fantasy. Engineers who can see outside the cube (both theirs and Rubik’s) write code that blends with corporate purpose. They induce a positive ripple effect that transcends their keyboard and compiler.

Conversely, engineers who primarily solve puzzles are more likely to write unmaintainable code, because they don’t see the world from outside their own perspective. They struggle to figure out why all this coding style garbage is being shoved down everyone’s throat. They view sales and marketing as pure overhead, management as the dark side, and software engineers as the only ones around here who do any heavy lifting.

I agree that big picture thinking is absolutely essential for a small company where everyone ultimately does everything. But big picture thinking turns out to also be critically essential for a large company as well.

In my vitriolic diatribe on puzzle-solving interviews, I argue that making engineers solve puzzles as an interview strategy does nothing but confirm what their transcript already tells you, and over time yields an engineering staff incapable of strategic thinking. To my credit, I did not use the term “boneheaded,” which speaks to my inimitable sense of self-restraint.

Rebuttal from a reader: “Perhaps some real value can be found out of the puzzle question response by looking for out-of-the-box, creative thinking (both in problem analysis and solution exploration). Even if they get the thing wrong (which is not necessarily a bad thing since most puzzle questions are designed to be devious and tricky), they may prove the ability to look at the problem from a unique perspective and think critically as they problem solve.”

Agreed. There is some value in watching a person solve a puzzle in order to see not just whether they can solve the puzzle, but how they approach the solution of said puzzle. But this insight does not address the profound problem with the entire notion of a puzzle-solving interview in the first place: Puzzle solving is not even remotely the most significant skill of a superb professional software engineer!

Software is built by humans within the context of social structures. Software construction is dramatically affected by a personal productivity variable of at least an order of magnitude. That means that my good day is at least ten times more effective than my bad day.  Equally important, one engineer may, on an average day, perform at least an order of magnitude more productively than another engineer on that same average day. Engineering productivity is massively impacted by the collective cognitive and emotional state of a given team, not to mention the dynamics of that team as it interacts with the rest of its company (and even further, with the company’s partners and customers).

If you want to really know whether a candidate is going to save your bacon six months from now, stop asking the puzzle questions. Now.

If you want smart and clever engineers, then limit your recruiting to schools you trust and require a minimum GPA of the students you interview. If that’s not good enough for you, then actually read their transcript and look at how they did in specific courses.

Now that you’ve determined the candidate is smart (see above), try to figure out these three things:

1) Can this candidate think intelligently about a world outside an engineering cubicle?

2) What is the ratio of bad to good days in the engineering life of this candidate?

3) Is this candidate an effective written and verbal communicator?

You then have to develop great interview questions that evoke meaningful answers to each of these three issues. I call them, “The only interview questions you’ll ever need.” All will be revealed in the next few posts.

Interviewing: The process by which an interviewer determines via a face-to-face interaction whether an interviewee possesses the chops necessary to assist the hiring company in its quest for profitability and market domination. (My definition. Feel free to quote me.)

About a decade ago, Microsoft pioneered an approach to engineering interviews based primarily on puzzle solving. The approach became so popular that companies like Google adopted it, books were written on it, and websites dedicated to it. Candidates were even encouraged to consult up to a dozen specific tomes in order to successfully crack the puzzle-solving puzzle that the interviews evolved into. (See the Wikipedia article “Microsoft interview.”)

It bugged me when I first heard about it a decade ago, and it bugs me even more now that other companies have jumped on the bandwagon. I think it’s a pathologically bad practice for reasons I will now reveal.

When you ask a senior in Computer Science to write a function for string comparison in some arbitrary language (for example), you’re testing his or her capacity to either a) regurgitate code they successfully wrote three years earlier, or b) cram puzzle-solving code snippets into their brains in order to impress an interviewer on demand. You might as well ask a candidate to solve a Rubik’s cube blindfolded in less than two minutes for all the value you derive from a question like this.

Seriously, what do you learn from this question?

1. That the candidate is smart. Congratulations. You just verified their cumulative GPA and spot checked specific courses on their transcript.
2. That the candidate is a puzzle solver. Congratulations. Next time just break out a Rubik’s cube and be done with it. For the bonus round, break out a blindfold.
3. That the candidate knows how to prep for your highly arbitrary and ultimately predictable IQ test. Congratulations. You just learned that the candidate is motivated to game your goofy interview process in order to secure a position with you.

My most significant concern is this: Fill your company with a few thousand smart kids who know how to solve puzzles and you’ll populate an engineering staff who can’t think strategically or globally about where the company needs to go, or how products need to evolve. You may find a few who have broader abilities, but your odds of finding them are about the same as discovering new hires that can hit a sweet jump shot or play the cello. You get some by chance, but not because you interviewed for it.

My assertion/prophecy ten years ago was that companies who hired spiffy puzzle solvers via spiffy puzzle solving interviews would ultimately suffer from a vast and pervasive lack of vision and global thinking among its engineers.

Memo to the industry: Your company’s biggest problem isn’t the puzzle solving ability of your engineers. It’s your collective ability to figure out what the market will cough up for the products you never thought about building.

Interview by testing for small thinking? Don’t be shocked when you find yourself working for a fading company devoid of a critical mass of engineers capable of thinking big.