My Lessons from Interviewing 400+ Engineers Over Three Startups

Grammarly Head of People Itamar Goldminz shares a model that can help managers build stronger teams — and become more intuitive, effective leaders.

Button Text

My Lessons from Interviewing 400+ Engineers Over Three Startups

Brandon Lipman

July 10, 2018

Marco Rogers loves interviewing engineers. Here’s a quick back-of-the-napkin showing just how much: he’s been an engineering leader and hiring manager for the last seven years. Over that time, he’s hired over 80 engineers. To fill each of those roles, he’s personally interviewed approximately 5 candidates on average after initial phone screens. That’s at least 400 interviews, or an in-person interview every work week for seven years — and that’s not counting time dedicated to prep, screening, references or debriefs. Most impressive is that he’s designed a system that has gotten his technical teams involved — and energized — to interview their future colleagues. To him, interviewing is a team priority: engineers on his team who are seasoned interviewers can conduct 12-16 interviews per month.

Read This NextLooking to Scale Your Sales? Seven Bullets to Dodge

Go Here NowLoad in New TabEmail for Later

For those who’ve seen him in action, it’s inevitable that Rogers ended up at a recruiting software startup like Lever. Every engineering leader spends time on building and rebuilding teams, but while at Clover Health and Yammer, Rogers obsessed on designing and refining an immaculate interview process. It was that craftsmanship that helped him hire over a dozen developers at Yammer, grow the Clover engineering team from one to 50, and start to scale up Lever’s technical team.

In this exclusive interview, Rogers starts by debunking some of the common recruiting tropes, explaining why they are outdated or misleading. Then he spotlights his top four interviewing practices — and how they fit within his broader recruiting methodology. Lastly, he recommends the first, low-hanging change that startups can make to retrofit their interview process.


THE REFRAIN: “We test the technical knowledge and skills needed for the role in the interview.”

Read This NextThe Best Hires Are Right Under Your Nose - Use This System to Never Miss Them Again

Go Here NowLoad in New TabEmail for Later

THE REALITY: You don’t actually know what you need. You’re figuring out what you need.

Across the startups he’s joined (and those he hasn’t), Rogers seeks to change the narrative around mapping relevant knowledge and skills to role. “Even if you’re a serial founder or seasoned technical leader, I’d still argue that you don't actually know what you need in the early days of a startup. There are so many variables, and the equation changes as the market, competitive landscape, customers and other elements shift,” says Rogers. “In the tech startup context, what you’re actually doing is figuring out what you need. You don't start with the answers; you work toward them. Instead, people orient around static sets of skills. So you can recite what it means to build a B+tree. That might be helpful, but not the entire picture.”

I want to make a stronger statement: the skills that people are hiring for are not the skills that they often most need.

So what’s at the root of this misconception? “When I’ve dug into conversations with people about it, I’ve realized that they haven't actually decided that interviewing and hiring is a critical business activity,” says Rogers. “They're kind of just trying to get through it. And because they want to get past it, they look for and lean on patterns, opting to repeat what others have said or done. Of course, they care who they work with, but it’s more about getting people in the door. Then they'll figure it out. That's the attitude. Hiring is not the real activity; it's getting people. It’s a subtle difference that short circuits process for outcome.”

THE REFRAIN: “With engineers, only technical skills truly matter. Soft skills are a nice-to-have.”

THE REALITY: Every engineer must be equally skillful with code andcolleagues.

Read This NextHere's the Advice I Give All of Our First Time Founders

Go Here NowLoad in New TabEmail for Later

First, Rogers takes issue with the framing of soft skills. “The problem with that framing is that it suggests that you can’t evaluate or teach those skills — that they are inherent qualities that some people have and some people don’t,” says Rogers. “They focus on the ‘soft’ part and forget they are ‘skills’ that can be measured and cultivated in people. I put those people skills at the same level of importance as technical skills. Can you write JavaScript and communicate effectively with business stakeholders? That’s what I need in a growing organization.”

It’s the business’ end goal really makes this an obvious default for Rogers. “First and foremost, we have to address the business reality — what we're really trying to do. If we’re talking about companies that want to grow 10X every couple of years, you have to assemble a really solid team that's going to take you there,” says Rogers. “It’s hard to keep that in mind when there’s the first or next thing to ship. So you have to explicitly test writing code andworking together in the interview process.”

Rogers helps his engineers strike balance between the “soft” and “hard” skills by asking them to get answers to the following questions around three key areas:

  • Ego. Does the candidate have low ego? Will they be too protective of their code? Do they have to be right? Does that change how they listen to input or consider other ideas?
  • Adaptability. Do they know that the goal of a business is to grow and change? Have they been a part of a team that has grown where process has had to change or team has had to be restructured? If so, how have they reacted to those changes? Do they seek opportunities for their growth in that change?
  • Technical communication. Do they know the technical skill needed for the role? Have they demonstrated the ability to apply that skill? Finally, can they communicate technical complexities well? Do they recognize that knowledge, application and communication of technical know-how are separate skills to master?
  • Cross-functional collaboration. Can they share multiple examples of close collaboration among individuals acrossfunctions, such as Engineering, Product, Design, Customer support and more. Do they have an example of when that collaboration went well? When it didn’t, how did they respond and resolve or bring closure to the situation? Have they proactively identified silos in an organization — and what have they done about them?

THE REFRAIN: “We’ve picked a few people to interview with them — let’s get it scheduled.”

THE REALITY: Thoroughness and consistency is paramount for optimal, unbiased results.

When Rogers talks about an interview program, it’s a crafted process. “So if you come on site with us, you're going to have four sessions. That’s our ‘right’ number, because we don’t want to ask too much time from candidates and be grueling, but we also need to have enough data points. The four sessions include three evaluations sessions with the team, each an hour long. Then there’s a session with the hiring manager, usually me, so I can answer any questions about how the team works,” says Rogers. “It’s designed this way so candidates can talk to a number of people: engineers of every tenure, so they can paint a picture of the day-to-day and their career path. We get to put you through your paces, and so candidates have several opportunities to display different skills. Each session has a fixed exercise or questions to ask, and then some time for unstructured dialogue. There’s a balance to strike here. The consistent structure and exercises help to reduces bias. While the open conversation makes it human. We’ve gotten feedback that this agenda and duration gives candidates sufficient time to start evaluating us as a potential fit.”

Brandon Lipman