Sunday, May 15, 2016

A short comment about the technical interview

There’s been a terrific backlash against the whiteboard-test commonly given in technical interviews. I think it’s a terrible tool to use in an interview. What the hell is the interviewer really testing for? Proximity to date-of-graduation? How well someone has memorized a breadth-first-search algorithm?

The first question I asked myself after several years of being a bad interviewer was “What am I really testing for when I interview a candidate?”

What’s the goal of each section of the interview? Better yet, let me back up even more: What are the values of my product development team (the team that includes the software developers) and the values I look for in co-workers?

Once that’s answered, an interview can be developed to filter for these values.

That list is for another post, so for now I want to focus on two values that’s pertinent to the kind of developer I like to hire: The ability to intelligently, quickly, and accurately find solutions to tasks and problems and implementrobust(1) solutions quickly. And: Are they a mature human being who can communicate well, govern their emotions while under stress (interviews are stressful), and have some level of intellectual curiosity with the ability and drive to satisfy this curiosity?

One of the big complaints lately was about getting the dreaded (and useless) “Go to the whiteboard and code a BFS”, or “Go to the whiteboard and code a maze-solver”, one solution for which requires a BFS and an implementation of Dykstra’s shortest-path algorithm (I had to look it up, of course). I’d never ask someone to do that at a whiteboard sans tools, references, etc., ever. What the hell would it tell me other than how well the candidate can remember university CS degree work? (Maybe it might tell me whether a developer suffers from not-invented-here syndrome, but that’s probably silly, and I digress)

But the question itself is a good one. Even though very few developers, especially ones who’ve been working for several years, can stand up and — under the stress and pressure of an interview — code this up from memory. But most developers understand the problem, regardless that this is one that seldom comes up in the daily grind of work, and given the right tools the best candidates can arrive at the solution. Now I don’t know about you, but I want to work with the kinds of devs who can find the right solution using the right tools.

So here’s what I’ve been doing to replace the whiteboard test: I want to know what kind of person the candidate is to work with. Given an uncommon problem (implement a maze-solver, or implement k-Nearest-Neighbor), lets use the tools we’d expect to have available to us day-in-and-day-out and come up with the beginning of an implementation. So therefore, the candidate and I sit side-by-side in front of their laptop and I watch them, and if necessary coach them through, work on the task.

I look for several things: do they search for solutions in the problem domain; can they discern between correct and robust solutions vs. incorrect, hacky solutions; a corollary to the previous point is do they understand what they are reading, and understand the underlying algorithms and implementation; do they simply pick the first answer they find from Stack Overflow? (a big red flag). This latter point should tell me whether or not a candidate truly understands data-structures.

Overall, can the candidate research, understand, and implement a correct solution to a problem that’s either unfamiliar to them or not memorized and do so in an environment that resembles the real-life environment of our team? What are they like to work with? Are they eager to learn or uninterested in anything not immediately familiar? Are they easy to work with or easily frustrated and/or defensive? Can they use the tools they will be expected to use in their daily work(2)? Can they communicate clearly? Do they ask pertinent questions?

I think that the best way to find whether or not you want to work with someone and have them on your team is to — wait for it — work with someone. Although an interview of an hour or two isn’t the same as working with someone on a team (it’d be great to bring in a candidate for a day or two and have them work alongside us, but I haven’t figured out how to get this though H.R.), it’s close enough if and only if you set up the interview to simulate as closely as possible the actual conditions of your technical environment.

(1) adj: strong and effective in all or most situations and conditions

(2) And we’ll do some debugging too. Can they and do they use a debugger, logs, and print statement? Do they ask the right questions when debugging: what do I expect the code to do, what is the code actually doing, and how do I know it’s actually doing it?

Tuesday, January 05, 2016

Minding your I's and T's

The rant.

The CEO was exasperated. "I'm at a loss, J, and I'm never at a loss." He waved his hands in the air, and was speaking quickly and loudly. "I hired the best and the brightest. A-players, all of 'em. I hired the best developers, a data guy from Google, you name it. I can't get them to ship a damned thing."

I nodded, and listened. He said, "They can't get a [expletive deleted] thing out the door. Yesterday saw the third time two of them came to me and insisted we need to re-write everything, whatever that means, again. They told me it's six months of work! They've rewritten everything, whatever that means, two times in the last two years. If they're as smart as they say they are and as smart as I think they are, why do they keep writing software that needs to be completely re-written?"

The litany of complaints continued, "They don't talk to each other. Half of them never come to the office. This one writes a server API that that one won't use, and they spend weeks - weeks, J! - arguing over how the two systems should talk to each other. If they'd just spoken with each other in the first place they wouldn't be in that mess. They ignore my designer, who's a great designer by the way, they won't speak with the product lead, who's a great product lead by the way, except to tell her that they've missed the deadline. They tell her this on the day of the deadline!  Up until then, it's all peachy when she checks in with them - they're confident they'll hit the deadline, until they don't. Terrible communication. It's killing my company."

He continued, "I'm a sales guy, you understand? I don't know tech. I hired these technical guys to get the work done and ship the product, not to argue over the little things and get stuck in analysis-paralysis.  Nothing's been shipped, and I can't get all of them into a room together unless I call an all-hands. And then they just complain that I'm rushing them; that software takes time and all of the ship-dates for product we decide on create technical debt. I don't know what that means, and I don't want to know. I want happy, paying customers, and I can't get them without shipping product."

"I don't get it," he said, "I hired the A-players, the best of the best. I thought that's what we're supposed to do."

I said, "It sounds like your developers don't collaborate. Not with each other, and not with design or product."

"No [expletive deleted]!", he said.

The advice.

This is where I put on my adviser hat, such as it is. I said, "First of all, lets talk about hiring, and then about how to try to fix the mess you're in."

"You did what most everyone likes to do, and that's recruit the smartest, most able people in their domain of expertise. The best designer, the brilliant developer, and so on.  You hire, or try to hire, for skill in a specific area, and fail to hire for all of the other skills that are necessary for a team in a business to succeed."

I continued, "So, the first thing to fix is hiring.  And the first question is: what are your values? What do you value the most from a product team? I don't mean just the devs, but I mean the whole team. Design, product, quality-assurance, developers, customer support, everyone composing that team.

Is individual skill and ability the most valued? What about collaboration and teamwork? Communication? How do you identify the abilities that allow for shipping the most engaging, quality product and features?"

The CEO interjected, "I value shipping. Ship and keep shipping the best product, and a solid, unfailing, and great customer experience." he emphasized, "Get it done. Get the work done. Increase revenue."

I nodded, "OK, then! So clearly, we want to hire for the abilities that support that, and not hire only for 'smart'. What if I said: when hiring developers, we value (among other things) drive, intellectual curiosity and the ability to satisfy that curiosity, good or great communication skills, wisdom to know when and what to apply the best-fit solution for the business problem, and domain expertise?"

He nodded, "I'd call that just 'smart'. I'd consider all of those things covered by 'smart'"

"OK", I said, "So how do you hire? What'd the interview look like?"

"Well, we do a phone screen of some basic computer science stuff. Anyone with a degree and a handful of brain cells should pass it. Then we get them in the office and present to them some tough problem, and ask them to solve it on the whiteboard."

"Do they have access to the internet during the interview? Can they research solutions? And what's an example of a question?

"Hell no. If they can't solve the problem on their feet, they're not smart enough for us. An example, well, we took a bunch of questions from Google and Microsoft. Here's one: 'Write a routine to calculate the optimal price for washing all of the windows in San Francisco.'  We also ask them to do some technical thing, my guys call it a 'data-structure test'. I don't know what a data-structure is, but whatever it is they're [expletive deleted] it up because they keep asking for six more months to re-write everything. It sounds to me like everyone here has failed that test!"

I said, "So in other words, you're testing in interviews for raw intelligence and domain expertise in the programming language your team uses. Where's the test for how well they can find a solution to a problem, or communicate effectively, and so on?  You're not giving one. You're not hiring for your own definition of 'smart', as you've just given it to me. Also, Google has dropped those brain-teaser questions. It doesn't correlate to great hires, and it just angers the candidate, these days.

So, look.  Drive, right? That's the desire to get things done. To ship. I like to look at what a candidate has done previously. Ask them what they've shipped in the past in previous jobs. Get them to talk about it and look for whether they mention a team or not. Was it a one-person effort? That's a sign that they may not have worked on a successful team. What do they do outside of work? Did they self-publish a novel, or complete the marathon? Are they bike enthusiasts or play some team sport? Do they DJ parties, or put their own music up on the web? Did they meet some goal, like read 24 books in a year? I  look for indications of a wide range of interests - that's the intellectual curiosity part - and a drive to produce, to create, and to achieve. And, can they collaborate? Can they and will they communicate early and often? This last one's a tough thing for which to interview, but it can be done."

I continued, "Problem solving. In an interview, lets give the candidate some challenging and pertinent technical problem to solve.  Unless you're in the window-washing business, the one you've been asking is not an appropriate question. How about asking them to do something they either have never done or haven't done in a long time? For example, implement some complex data-structure that they might encounter as a part of their daily work. We've all done data-structures in school, but for some people, school is a distant memory or some structures weren't a part of the curricula. So, using a laptop and the candidate's preferred development environment, work together to implement the data structure, a few functions, and a couple of tests. Use the internet to research solutions and discussions.  I look for whether the candidate can identify quality information amidst the crap that's out there; which websites they choose for research (if any. Big red flag if they don't research anything); and how well they understand and can implement the information they find.  That's just an example. This whole process will take some thought and preparation."

Minding your I's and T's

I asked, "Have you heard of Tim Brown?"

The CEO said, "Yes, of course. Famous CEO of IDEO."

I nodded, "Great. He spoke and wrote about I-shaped and T-shaped people and successful cross-disciplinary work. By the way, almost all modern product development is cross-disciplinary work.

It sounds like your interviewing strategy is biased toward I-shaped people, when what you need are T-shaped.  Tim Brown said,"
T-shaped people have two kinds of characteristics, hence the use of the letter “T” to describe them. The vertical stroke of the “T” is a depth of skill that allows them to contribute to the creative process. That can be from any number of different fields: an industrial designer, an architect, a social scientist, a business specialist or a mechanical engineer. The horizontal stroke of the “T” is the disposition for collaboration across disciplines. It is composed of two things. First, empathy. It’s important because it allows people to imagine the problem from another perspective- to stand in somebody else’s shoes. Second, they tend to get very enthusiastic about other people’s disciplines, to the point that they may actually start to practice them. T-shaped people have both depth and breadth in their skills. [1]
"He also said",
Most companies have lots of people with different skills. The problem is, when you bring people together to work on the same problem, if all they have are those individual skills-if they are I-shaped-it’s very hard for them to collaborate. What tends to happen is that each individual discipline represents its own point of view. It basically becomes a negotiation at the table as to whose point of view wins, and that’s when you get gray compromises where the best you can achieve is the lowest common denominator between all points of view. The results are never spectacular but at best average. [1]
"So in your case," I told the CEO, "your developers are spending most of their time negotiating, and by that I mean arguing and debating, with each other as well as with product, design, sales, and executives.

I like to hire with a bias toward the values we've discussed, and I look for things in the candidate's history that shows some sort of teamwork. Were they in a band in college that managed to play a few gigs, and what happened to the band? Did it implode under conflict of personality, or enjoy a fine local run before everyone graduated? Are they on a cycling team, or a rowing team? Are they a part of a burning-man collective? A regular contributor to an open-source project? Have they had to manage collaboration not only with each other but with coaches or customers? You get the idea. I try to spend a lot of time with them, to get a feel for how well they collaborate, communicate, and of course how intelligent and driven, empathetic and enthusiastic they are."

I concluded, "Think of this problem as concerning a whole team, not just as a team of developers who have to interface with other teams. Design, product, all of them make up your team.  I think of analogies like a great band, or a great sports team. You're going to need the right players who all understand and are in agreement of the goal, can communicate effectively, certainly can perform as individuals but just as certainly can perform as a team, collaboratively.  The Dave Brubeck quartet. The 1990s-era Chicago Bulls.  Not everyone loved each other, but they could superbly work toward, and achieve, a goal.

[1]  IDEO CEO Tim Brown: T-Shaped Stars: The Backbone of IDEO’s Collaborative Culture