The Supportive Technical Interview

| Comments

Interviews Gone Wrong

If you have read my last few posts, you know that I do a LOT of interviewing. I enjoy it and love the conversations that transpire. The problem is that most interviews are a horrible experience for everyone involved. As a candidate you have to do the mind numbing walk through of everywhere you ever worked because no who is interviewing you bothered to read your resume. As a panel member, you get candidates who just show up thinking their mere presence grants them a job. Crazy from both sides!

Lately I have had quite a few discussions with folks on what makes a successful interview? How do you learn the most about a candidate? What are we trying to accomplish with the interview? As a candidate, how can I show them I am really the right person for the job? In this post, I am going to share a few guidelines for hiring managers that I think lead to a great interview. If you are the candidate, you can read this and if you find an experience like the one I describe, you can be better prepared.

The Sweaty Guy

Years ago when I was working at VerticalResponse, I was part of an interview panel for a prospective engineer. There were 6 of us and the candidate crammed into a tiny conference room where we started off with the proverbial tell us about your background question because not all of us had read his resume or prepared for the interview. The candidate started to share his experience and soon enough we were asking him questions. Smelling fear, we started to circle around this guy like a pack of blood thirsty sharks, asking him questions that soon became apparent were above his skill level. It got hotter and hotter in the room and then the poor candidate started to sweat profusely from the heat and the pressure of the situation.

Normal people at this point take note, show some empathy, and ask the person if they need a break or some water. Not us! We kept asking more difficult questions and kept drilling this guy like nobody’s business. We put him in front of a white board and asked him to design a database for a library problem that was one of our stock questions. It was customary to then get into details about standard CRUD operations that would be performed against the design including possible optimizations. He was doing poorly but we didn’t stop asking him questions. We just kept asking, drilling deeper, and further humiliating the guy until it was embarrassing.

Not satisfied with our established pedestal, we jumped right in to our environment and vernacular, asking him detailed questions about how he would solve a prospective problem. Then, like Keith Hernandez in the kissing scene, we asked ourselves how could this guy not quickly understand our rpm based deployment system? What kind of novice are we interviewing? Why is he sweating so much? How smart are we, right? We are really smart! Man, he is really sweating a lot.

Don’t Pat Yourself on the Back

Have you ever been a part of this type of experience? For the sake of every candidate out there, I hope not. I can remember after we merciless ended the interview and he was walked out and wished well. When I got back he was already nicknamed sweaty guy. To this day looking back, I actually feel really bad for him. I wish I remembered his name because I would go back and apologize to him for being an ass, even after all of these years.

After it was over we all sat in the room talking about how great our process was. We exposed that guy didn’t we? We were all thumbs down saying what a loser the guy was. The reality of the situation was that we spent most of the time just spending time on ourselves, showing how smart we were. We spent so much time patting each other on the back that we never gave the guy any chance to truly share his experience or learn anything about us.

Put the Candidate at Ease

A few years after that experience I became a hiring manager again. I learned very quickly that one of the key practices of a good interview was to put the candidate in an environment where they will be successful and supported. Let’s be honest and understand that interviewing can be a nerve wracking experience, especially for prospective female engineers.

Candidates want to impress yet they have no idea what you are going to ask them. The canvas is wide open. The following are some tricks I have learned over the years for having a successful technical interview:

  • Share the Agenda - One of the first things I do in any interview is to first tell the candidate what we are going to do. Just like any meeting go over the agenda so they know what to expect and reaffirm who they are going to talk to. It just makes it easier and helps make them comfortable.

  • Use their Editor - If you are doing a pairing exercise, ask them before they come in what editor they like to use and make sure it is installed on the machine you are going to use. This way you spend time working on code together rather than wasting time having them relearn all the keyboard shortcuts.

  • Use Their Preferred Language - Candidates often know several languages so make sure you identify which is their preferred language and test them in it.

  • Allow Google - Allow them to use Google during the technical interview and explain to them it is OK to look things up. We do it all the time in our job, so why should interviewing be any different? This is not about rote memorization and besides, I am old, so the memory registers in my head are diminishing. If I can’t look up method signatures and documentation, I will surely fail the test. Does that mean I can’t write good code? Hardly!

Avoid Context

Conventional wisdom suggests that we want to put people on the job before they start as a good way to test if their skills and experience will work. The caveat to this is how far the candidate is removed from your context. When I did my pairing exercise at ModCloth, I was given 90 minutes to build a product shots service. I knew it was going to be a long exercise when my first question to @richardiux was, “What is a product shot?”.

A few things on context to keep in mind:

  • Provide Background - Understand how far your engineers are from the edge of the product. If they are far from the edge, meaning it is not an engineering driven role, then make sure your candidate gets plenty of background.

  • Use a Kata - Try using a kata to learn about their skills. I was so inspired by this idea I developed a ruby gem that we use for our pairing test.

Choose Your Panel Wisely

This is an important aspect that many overlook. You have to pay attention to the type of candidate you are interviewing and assess who on the team will best be able to judge them? If you have a very senior candidate but have all junior engineers interviewing them you will not be successful. The candidate will not be impressed by your team and the junior engineers might be put off by the candidate.

Make sure you have a good mixture of the right people to judge the candidate and to represent your team fairly. If you have small teams then make sure the candidate meets everyone. If you have a larger team, meaning more than 4 people, try splitting into multiple stages with different panels for each stage. This way you don’t have the marathon day and you don’t waste everyone’s time if the candidate doesn’t work out.

Kickoffs and Debriefs

I really dislike going into an interview where people are totally unprepared and no one knows what topics to cover or questions to ask me. Even worse is when you go back for the next round, meet different people on the team, but answer the same questions. Don’t be that team!

Kickoff

Make sure that you have a kickoff meeting to review the role of each person on the panel. The hiring manager should lead the meeting and lead the panel through the following agenda:

  • Resume Review - Make sure you have reviewed it and share why this candidate’s resume fit the requirements.
  • Interview Background - Don’t sell the candidate but share the information about the interview process to date and why the candidate made it this far.
  • Topic Matrix - Cover the topics that need to be covered with the candidate and setup a matrix of which panel members will cover each.
  • Schedule - Define and review the schedule and make sure that everyone understands what conference room they are going to be in and when with a focus on timeliness.

I’d like to point out that in addition to the topics in the matrix, everyone on the interview panel should evaluate that candidate on the following:

  • Communication Skills - Does the candidate possess the communication skills necessary to work with the team?
  • Culture Fit - Do you feel the candidate is a good culture fit and why?
  • Questions Asked - Did the candidate do a good job of asking intelligent and relevant questions?

One thing to make sure of when planning the individual interviews is to allow the candidate to have ample time to ask their questions. Remember that interviewing is a two way street.

Debrief

The debrief is the critical component to all of this. After you are done interviewing the candidate you have to meet with the team to determine next steps. In my experience, the sooner you do this the better. You want everything to be fresh in everyone’s mind so that their notes/memories still make sense.

Here is an example agenda for this meeting with the interview panel being the attendees:

  • Scoring - Take your matrix and have the team score the candidate on a scale of 1-5 in each category they covered. They should also score the candidate on the 3 overall requirements described above as well.

At this point you evaluate the scoring and use the following rules:

  • >4 - Move to the next round or make offer
  • <3 - Rejection and move on

If they are between a 3 and a 4 then you do the following:

  • Strengths/Weaknesses - List out the strengths and weaknesses from the panel. This is important so that focus on a particular negative doesn’t become the focus of the conversation leading to rejection of a perfectly good candidate.
  • Evaluate - During the evaluation you review the voting and continue to list the strengths and weaknesses of the candidate.
  • Decision - After time boxing the evaluation discussion, arrive at a decision based on the information retrieved from the team.

During the discussion you will see what the team thinks the candidate’s strengths are and if it warrants another round or not.

Pay it Forward

Regardless of how you come to the decision be sure to commit to it and communicate it to the candidate. It is very important to do this when you are rejecting a candidate. We all have been there waiting for an answer and not hearing back. It is a terrible experience so don’t put candidates through this and just let them know. They’ll thank you for it and most likely they will share they had a good experience interviewing with you with their friends. Remember, it is a small world out there in technology so pay it forward.

Comments