Puzzle-solving: Not the driving function of software construction

Yeah, I know I promised three follow-on posts after the last one on puzzle-solving interview questions. I’m getting to it. Really. But I wanted to take a minute first and specifically address some of the comments left on the last two posts.Here’s one:

In my opinion, there’s nothing wrong with hiring engineers primarily for their puzzle-solving skills; that’s their primary function.

I’m going to respectfully disagree. A software engineer’s primary job function is to write software, which involves a number of things, one of which looks a bit like puzzle-solving. But the driving function of the complexity and cost of software construction is neither writing software, nor its puzzle-solving component. The driving function of software construction is the collective emotional state of a software engineering organization.

Individual engineers have to be decently smart. Spectacularly smart is even better. But uber smart with a whiny attitude is a no hire. The smartest new college graduate at some university who can’t stay focused for more than three weeks at a time is a no hire. The kid who solves the Rubik’s cube in less than a minute blindfolded but brings other team members down emotionally is a no hire. The brilliant prima dona is a no hire. The defeatist genius who minored in learned victimhood is a no hire. Every single one of these guys can pass the puzzle questions with flying colors. Every single one is a no hire.

Failure to interview for emotional attitude, team play, communication, and can-do attitude ABOUT THINGS OTHER THAN PUZZLES is failure to interview for a successful engineering corps.

But what about division of labor? Why does an engineer have to think big? Can’t the high mucky mucks think the big thoughts and leave the puzzle-solving engineering crew to hunker down?

To wit, this comment:

When you expect or allow those same engineers to decide the direction of the company is when you’re headed for disaster. Large companies like Microsoft and Google can get away with this practice if they balance out their spiffy puzzle-solvers with globally-thinking big-picture people. Smaller companies can’t afford to specialize and compartmentalize their workforce that way and should follow different hiring practices.

Again I’ll respectfully disagree. The notion that there is some clear separation between the engineers in the trenches and everyone else in the company up to and including the CEO is a myth. The collective attitude of an organization, for better or worse, is manifest in everything the company produces, including the products it ships, the ad campaigns it runs, the personality of its sales force, the attitude of the secretary at the front desk, and the code that engineers produce, not to mention the rate at which they produce it.

Personal case study: In 1988 I was a software engineer in the Personal Computer Group at Hewlett-Packard in Sunnyvale, California. I showed up just before HP’s OEM release of OS/2 1.0 for the Vectra PC, and then stuck around to write device drivers for OS/2 1.1. I had access to all the source code for OS/2, which had been jointly developed by Microsoft and IBM. As I cruised around the code for OS/2, it was always immediately obvious which code had been written by IBM engineers and which had been written by Microsoft engineers. In all cases, the coding style was consistent with the collective corporate attitude that you could see on a billboard from a clogged expressway. You could drop code in front of me and I could tell you which company it came from.

Engineering work, even among the most junior engineering staff members, involves creativity and decision making that has nothing to do with puzzle solving. Individual engineers make stylistic decisions about design issues that were not fully clarified by someone outside their team. Attitudinal decisions that affect code maintainability happen every single day. A renegade codeslinger who refuses to follow the corporate style guide has a negative effect that no amount of puzzle-solving speed can repair.

The idea that there is a wall of separation between the puzzle-solving work of the engineers hunkered down in their cubes and the collective vision of the corporation is fantasy. Engineers who can see outside the cube (both theirs and Rubik’s) write code that blends with corporate purpose. They induce a positive ripple effect that transcends their keyboard and compiler.

Conversely, engineers who primarily solve puzzles are more likely to write unmaintainable code, because they don’t see the world from outside their own perspective. They struggle to figure out why all this coding style garbage is being shoved down everyone’s throat. They view sales and marketing as pure overhead, management as the dark side, and software engineers as the only ones around here who do any heavy lifting.

I agree that big picture thinking is absolutely essential for a small company where everyone ultimately does everything. But big picture thinking turns out to also be critically essential for a large company as well.

Puzzle-solving interview questions: How you do it can’t save you

In my vitriolic diatribe on puzzle-solving interviews, I argue that making engineers solve puzzles as an interview strategy does nothing but confirm what their transcript already tells you, and over time yields an engineering staff incapable of strategic thinking. To my credit, I did not use the term “boneheaded,” which speaks to my inimitable sense of self-restraint.

Rebuttal from a reader: “Perhaps some real value can be found out of the puzzle question response by looking for out-of-the-box, creative thinking (both in problem analysis and solution exploration). Even if they get the thing wrong (which is not necessarily a bad thing since most puzzle questions are designed to be devious and tricky), they may prove the ability to look at the problem from a unique perspective and think critically as they problem solve.”

Agreed. There is some value in watching a person solve a puzzle in order to see not just whether they can solve the puzzle, but how they approach the solution of said puzzle. But this insight does not address the profound problem with the entire notion of a puzzle-solving interview in the first place: Puzzle solving is not even remotely the most significant skill of a superb professional software engineer!

Software is built by humans within the context of social structures. Software construction is dramatically affected by a personal productivity variable of at least an order of magnitude. That means that my good day is at least ten times more effective than my bad day.  Equally important, one engineer may, on an average day, perform at least an order of magnitude more productively than another engineer on that same average day. Engineering productivity is massively impacted by the collective cognitive and emotional state of a given team, not to mention the dynamics of that team as it interacts with the rest of its company (and even further, with the company’s partners and customers).

If you want to really know whether a candidate is going to save your bacon six months from now, stop asking the puzzle questions. Now.

If you want smart and clever engineers, then limit your recruiting to schools you trust and require a minimum GPA of the students you interview. If that’s not good enough for you, then actually read their transcript and look at how they did in specific courses.

Now that you’ve determined the candidate is smart (see above), try to figure out these three things:

1) Can this candidate think intelligently about a world outside an engineering cubicle?

2) What is the ratio of bad to good days in the engineering life of this candidate?

3) Is this candidate an effective written and verbal communicator?

You then have to develop great interview questions that evoke meaningful answers to each of these three issues. I call them, “The only interview questions you’ll ever need.” All will be revealed in the next few posts.

Puzzle-solving interview questions: You get what you ask for!

Interviewing: The process by which an interviewer determines via a face-to-face interaction whether an interviewee possesses the chops necessary to assist the hiring company in its quest for profitability and market domination. (My definition. Feel free to quote me.)

About a decade ago, Microsoft pioneered an approach to engineering interviews based primarily on puzzle solving. The approach became so popular that companies like Google adopted it, books were written on it, and websites dedicated to it. Candidates were even encouraged to consult up to a dozen specific tomes in order to successfully crack the puzzle-solving puzzle that the interviews evolved into. (See the Wikipedia article “Microsoft interview.”)

It bugged me when I first heard about it a decade ago, and it bugs me even more now that other companies have jumped on the bandwagon. I think it’s a pathologically bad practice for reasons I will now reveal.

When you ask a senior in Computer Science to write a function for string comparison in some arbitrary language (for example), you’re testing his or her capacity to either a) regurgitate code they successfully wrote three years earlier, or b) cram puzzle-solving code snippets into their brains in order to impress an interviewer on demand. You might as well ask a candidate to solve a Rubik’s cube blindfolded in less than two minutes for all the value you derive from a question like this.

Seriously, what do you learn from this question?

  1. That the candidate is smart. Congratulations. You just verified their cumulative GPA and spot checked specific courses on their transcript.
  2. That the candidate is a puzzle solver. Congratulations. Next time just break out a Rubik’s cube and be done with it. For the bonus round, break out a blindfold.
  3. That the candidate knows how to prep for your highly arbitrary and ultimately predictable IQ test. Congratulations. You just learned that the candidate is motivated to game your goofy interview process in order to secure a position with you.

My most significant concern is this: Fill your company with a few thousand smart kids who know how to solve puzzles and you’ll populate an engineering staff who can’t think strategically or globally about where the company needs to go, or how products need to evolve. You may find a few who have broader abilities, but your odds of finding them are about the same as discovering new hires that can hit a sweet jump shot or play the cello. You get some by chance, but not because you interviewed for it.

My assertion/prophecy ten years ago was that companies who hired spiffy puzzle solvers via spiffy puzzle solving interviews would ultimately suffer from a vast and pervasive lack of vision and global thinking among its engineers.

Memo to the industry: Your company’s biggest problem isn’t the puzzle solving ability of your engineers. It’s your collective ability to figure out what the market will cough up for the products you never thought about building.

Interview by testing for small thinking? Don’t be shocked when you find yourself working for a fading company devoid of a critical mass of engineers capable of thinking big.

Computer Glitch Leads To Brawl At Wauwatosa Kmart

Computer Glitch Leads to Brawl at Wauwatosa Kmart

If this were fiction, I would pan it as being ridiculous and unbelievable. Alas, it is true…

A melee at a Kmart store in Wauwatosa [Wisconsin] Saturday morning was started by a computer glitch.

The store was running a promotion in which it would give away $10 to anyone applying for its credit card, but the computer glitch led to everyone’s application being granted — bestowing up to $4,000 in instant credit to anyone who applied even if they shouldn’t have qualified.

Once word started to spread about the so-called “free money” Saturday, witnesses said things got pretty nuts inside the Wauwatosa store.

Nearly a dozen Wauwatosa squad cars responded to the call just before 11 a.m. Saturday for what was called a large fight in progress.

“It was a nice brawl. It came from inside to outside. If you go up there, you’ll see hair, earrings, all pulled out on the ground,” Wilson said.

What started as a fight between two women in the crowded store evolved when several men intervened.

A store employee got punched in the nose and crashed through a glass display case. He was treated for a broken nose and various cuts.

Let’s be clear about a couple of things. The credit application problem was almost certainly caused by a software bug of some kind (a bit more accurate than the imprecise and somewhat generic “computer glitch”). But the fracas was caused by the mob mentality of a group of people lacking both personal moral control and common sense.

To paraphrase a favorite mantra of the NRA, “Computer glitches don’t throw people through display cases — people do.”

Programming: Nature or Nurture, Part II

My most recent post to the PPIG mailing list…

On Jul 4, 2007, at 10:55 AM, Frank Wales wrote:

Charles Knutson wrote:
I believe there is a taxonomy of four types of people, relative to
professional software construction:
1) Those born to code, who need almost no coaching;
2) Those born capable but in need of training in order to be successful;
3) Those not really born to it, but who can be trained sufficiently to make a living;
4) Those whose brains are really not wired to build software at all.


As a matter of interest, do you have a sense of how the general population
is distributed across these suggested types?

Rather than just categories or types, I’d actually propose a spectrum
of capabilities, with people who just ‘get’ computer technology at one
end, people who would sooner eat lint that use it to check their C
programs at the other, and the substantial mass of people bulging
somewhere along the middle. (Of the spectrum.)

I’d speculate, completely without anything beyond years of experience
and arrogant presumption, that the “just get computer technology” end
of the spectrum glows with about 5% of the population, the “couldn’t
write ‘hello, world!’ despite years of practice” end is dimmed by maybe
15% of the population, while the remaining 80% of the population huddles
around various quantumly-distributed embers in between.

I think the end groups are inherent, while the huddles are plastic.


Of course I have no objective evidence for what the actual distribution might be. If anyone has seen any empirical take on this distribution, it would be very interesting. I could easily buy the idea that the top is 5% and the bottom 15%. I also concur completely with the idea of a spectrum. My taxonomy was more of a highly granular way of describing such a spectrum. My apologies to anyone who was expecting a fist fight 😉

Two last thoughts on inherent aptitude:

1) I’ve recently run into students who took the first semester programming class at BYU, did well, got their ‘A’, and then washed their hands of the field and went off to study business or something else. Clearly they are capable of programming, at least at the entry level, but the biggest dissuading factor to them was the inherent motivation (or lack thereof). One of them told me that after spending 15 hours on a programming project, and finally getting it working and passed off, they felt like they had been robbed of 15 hours of their life that they would never get back! That was such a news flash to me! In *my* experience, every program or project I ever finished has carried with it some sense of euphoria and satisfaction that seemed to make all the pain worthwhile. And much of my programming time has been somewhat rhapsodic, like a runner’s high. So while I don’t consider myself to be the most gifted programmer (or even necessarily a member of the first group) I have always been quite satisfied and happy with the process of constructing software.

2) I just started reading Stephen King’s “On Writing.” Interesting to find this quote on page 4 (substitute “computer programmers” for “writers”):

“This is not an autobiography. It is, rather, a kind of curriculum vitae — my attempt to show how one writer was formed. Not how one writer was *made*; I don’t believe writers *can* be made, either by circumstances or self-will (although I did believe those things once). The equipment comes with the original package. Yet it is by no means unusual equipment; I believe large numbers of people have at least some talent as writers and storytellers, and that those talents can be strengthened and sharpened. If I didn’t believe that, writing a book like this would be a waste of time.”

Just a bit more grist for the mill.
Chuck

Professional software developers: Nature or nurture?

The following is an email I just posted to the PPIG mailing list, in response to the following comment:

For the record, I believe anyone can learn to program at a professional level. The question is, are they willing to put in the time to acquire all the chunks needed to be an expert? Unfortunately, we can’t force our students to put in the time.

I’m not convinced that absolutely *anyone* can learn to be a professional X (whatever X is). I think there are some who are really just wired to do other things. But I am confident that there are varying degrees to which inherent aptitude plays a role, and similarly varying degrees to which effective learning experiences contribute to facilitate those individuals who can, in fact, be successful at profession X.

As evidence, I offer the following non-empirical anecdote. I started programming in 1973, when I was 13 years old. Our high school had a timesharing account on a mainframe at the University of Northern Iowa, and a DecWriter with a suction cup modem and a rotary phone with a dedicated line to the university. About a half dozen of us math geeks gathered daily in a room to play with the computer (which largely consisted of playing Dungeons and Dragons and Star Trek, with intermittent fits of attempted software design and code construction). Several of my friends just seemed to have the knack right out of the chute. We’d dumpster dive at the university for discarded manuals, and that was all Brian and Doug needed to build software. I tried desperately but couldn’t get it beyond a fundamental level. The other guys were more or less like me, in love with the technology, but not fluent with the incantations.

Years later, in my second semester at the University of Iowa, I had a really well constructed and well presented Computer Science class that focused primarily on design. During that semester, the light came on, and I got it! From that semester it was simply a matter of learning new skills and piling them onto the foundation I had now acquired. I had a very successful professional career building software (HP, Novell, various small companies and consulting gigs), picked a few graduate degrees along the way, and then retired to the university to stop producing and begin pontificating. 🙂

As an epilogue, of the group of math geeks that gathered together daily in high school to play with the DecWriter, all but one of us acquired degrees in Computer Science, with the other one (Brian) doing Electrical Engineering.

My personal experience is that I was always fascinated, I was obviously capable, but I needed someone to throw the switch for me to understand how to become self-sustaining after that.

I believe there is a taxonomy of four types of people, relative to professional software construction: 1) Those born to code, who need almost no coaching; 2) Those born capable but in need of training in order to be successful; 3) Those not really born to it, but who can be trained sufficiently to make a living; 4) Those whose brains are really not wired to build software at all.

Just my two cents. Your mileage may vary.
Chuck

IT doesn’t matter

IT doesn’t matter

That’s “IT” as in “Information Technology”… doesn’t matter. This article by Nicholas Carr first appeared in the Harvard Business Review in May 2003. He reprinted it here on his blog Rough Type in January of 2007. It’s fairly lengthy (8 parts) but worth the investment.

Here’s one of his key punchlines: the commoditization of IT…

“Behind the change in thinking lies a simple assumption: that as IT’s potency and ubiquity have increased, so too has its strategic value. It’s a reasonable assumption, even an intuitive one. But it’s mistaken. What makes a resource truly strategic -– what gives it the capacity to be the basis for a sustained competitive advantage –- is not ubiquity but scarcity. You only gain an edge over rivals by having or doing something that they can’t have or do. By now, the core functions of IT –- data storage, data processing, and data transport –- have become available and affordable to all. Their very power and presence have begun to transform them from potentially strategic resources into commodity factors of production. They are becoming costs of doing business that must be paid by all but provide distinction to none.”

It’s an extremely well-written article, and very thought-provoking. I’ve been thinking a great deal lately about enrollments in Computer Science programs around the world, and wondering what role this sort of commoditization of information services plays in all of that. Obviously it may have a greater impact on programs like Information Systems or Information Technology, but all of these academic programs seem to be rising and falling together to some extent.