Hiring for judgement is hard. It's easy to hire for technical chops. We know how to screen out people that can't write code, and if you follow the general best-practices of technical hiring (make them write code, at least pseudocode, a few times), you're unlikely to hire candidates that can't produce anything. But the code-heavy interview process does little to ferret out developers with judgement problems, the ones that can't prioritize work, and don't know when to ask for help. If you're hiring for a company the size of Google it doesn't matter that much, you can always put a manager or tech lead in between that person and a deadline. Small companies don't have this luxury, and it is that much important that the screening process captures more than just sheer technical ability.
I was hoping as I wrote this that I would come up with some ideas for hiring for judgement. But I have never had to hire for judgement until now. Judgement was something I had the luxury of teaching, or simply guessing at and hoping for the best. The internet suggests questions like
Tell me about a time when you encountered a situation where you had to decide whether it was appropriate for you to handle a problem yourself, of escalate it to your manager. What were the circumstances? How did you handle it? Would you handle the same situation the same way again or would you do it differently this time?This seems to me better than nothing, but not amazing.
So, over the next few months I'm going to experiment with questions like these and see how they play out. There's no way to do a rigorous study but at least I can start to get a feel for which questions can even tease out interesting answers. And if you have any questions you like, or thoughts on the matter, leave me a comment.
Well framed. In my past of interviewing by competencies I've never heard about "judgement" specifically. Yet, you're right, it's very important to know how one acts in different context.
ReplyDeleteStraightforward questions of "how did you deal with X" can be prepared for and are rather boring for the interviewee. Those would be on the bottom of my list.
You could try to ask an engineer to solve an architectural problem and then ask them to find a completely different way of achieving the same solution with, maybe, a couple of non-technical constraints thrown in the mix? Someone with good judgement will consider the problem from many alternate angles, often non-technical.
Interesting question and probably the most difficult to answer.
ReplyDeleteAnd I agree with you that you can find those who cannot code in 2 minutes - my technique? Pose the simplest coding problem you can think of. In C: implement strcpy; in C++ or Java: implement a Singleton class - don't worry about concurrency issues). Slide a sheet of paper with a pencil (not pen!) across the table to the interviewee and observe. If the candidate grabs the pencil and scribbles, there is hope. If the candidate hesitates for any reason (e.g., asks for clarification; recoils, etc), end the interview as quickly and politely as possible.
Judgement is more difficult, and depends on the experience of the candidate.
A junior candidate has little or no experience, so you are looking for the ability to approach problems - creativity, fearlessness and enthusiasm.
For experienced developers, I look for evidence of _their_ having worked on hard problems. If you ask them problems you are interested in but with which they may be unfamiliar, your conclusions will be clouded by questions about their interview nervousness, etc. Don't worry about the correct textbook solution since every problem is _in_context_. Did they ask the right questions? Did they consider all possibilities (here's where the experience of the _interviewer_ comes in!)? Why did they make certain choices? Can (s)he carry on a technical conversation with you? Is (s)he comfortable with ambiguity? Is (s)he defensive when challenged? If such an experienced candidate cannot communicate coherently, goodbye.