The interviewers perspective: Data science interviews at Amazon

Andrew wrote this article to provide insight into what he learned from conducting over 100 Data Science interviews at Amazon, he shares his advice from an interviewers perspective. Andrew Jones is the Founder & Director of analytics-link, a global hub for analytics & data science, he has over 10 years experience in Analytics, Data Science, and Machine Learning within the Digital, Retail, Telecoms and Travel industries – most recently at Amazon, & Sony PlayStation. This article was originally posted on Andrew’s website analytics-link.

 

Interviewing is a very subjective process. Each role, candidate, and interviewer are different, and therefore there can’t be a strict set of “rules” that specify how it is best done. This article is purely a set of learnings gained from interviewing many candidates throughout my career to date, including over 100 Data Science & Analytics interviews during my time at Amazon.  This article doesn’t in any way represent Amazon or Amazon’s interviewing policy – in fact, I’m not even going to focus on their methods here, this is more a summary of what I found improved the interview process for both the candidate and for me, the interviewer. These include; how to conduct yourself as an interviewer, what to ask, what to listen out for, and how to get the insights you need to make an evidence-based, non-biased hiring decision.

 

Be prepared & enter the interview in a good head-space

The end goal is being as confident as possible that you’re making the correct decision for both the company and the candidate. To do this you want to ensure you get everything you need to make that decision.  It’s up to you as the interviewer to get evidence for that hiring decision, whether it be a yes or a no, it should be justified.

Take the recruitment process seriously each time.  If you end up hiring them then you’ll be working with this person day in day out, and you’ll most likely be accountable for their work. If you don’t end up hiring them, always remember that they’ll have to go home to their partner or family and say “I didn’t get the job”.

The above doesn’t mean you shouldn’t be completely objective, it means that you should understand that it’s a big decision for you both.

Have interview questions ready beforehand with possible follow-up/probing questions noted down also. Having these in place also means you can keep track of time throughout the interview to ensure a good flow.  If you need to move them onto another question just let them know that’s what you’re doing.

CV’s

In my experience, a CV is a good filtering tool, that’s about it. CV’s are a great resource to get a base understanding of the types of skills the candidate has, as well as the types of projects they’ve worked on.  Its main purpose is to help filter out candidates that don’t meet the technical criteria of the role. Definitely read the CV, and have it in front of you in the interview – but in my experience, it can mean you go into the interview with some form of bias as to whether this candidate is good or not before you’ve talked to them – try to avoid this.

I’ve seen some very impressive CV’s that haven’t always translated into a good fit for the role, including PhD’s from prestigious universities, or where the candidate has written Data Science books that are available to buy online!  These candidates are without fail very intelligent people, but can still lack certain communication or strategic thinking skills that the role requires – don’t feel like they’re the perfect candidate based only on their CV!

 

Note Taking

I take my laptop into every interview and type fairly detailed notes on the fly. It allows me to store the evidence each candidate provided (and refer back to it long after the candidate has left) ensuring my decisions on and between candidates are as objective and evidence-driven as possible.

I know some people are against this as they believe it’s rude and/or distracting but I’ve found the pros outweigh the con’s when it comes to documenting the interview.

If you are going to take notes, always ensure you:

• Pre-warn the candidate that you’ll be typing notes (especially in phone interviews as they may think you’re answering emails while they speak!) and explain how it helps you make the optimal hiring decision.

• Can type fast enough to keep up with the flow of the interview.

• Maintain plenty of eye contact with the candidate, ensuring they can read your body language as being positive and engaged with their responses.

 

What questions to ask

This will always vary between roles, levels, and companies to some degree.

I’ve been asked all types of questions in interviews, the most memorable being:

“You have a monster, the monster eats sheep, the monster lives on an island. How big does the island need to be?”

I can’t remember my exact answer, but I think I got into the depths of paddock rotation – probably not what they were after…

I personally don’t believe those types of questions are necessary, I don’t think candidates need to be tricked or tripped up.  You can get all the evidence you need to make a sound hiring decision asking relevant questions and subsequently probing into the details of their response.  In general, I’ve found it best to ask fairly open questions and guide the candidate towards certain pieces of information/evidence that I require as their response evolves.

Below are a couple of generic questions that I feel have worked well, two of which are more focused around Analytical Skills, and two around Passion & Curiosity.  I believe a good Data Science should possess both, being very strong technically is not enough on its own.  The combination of the two skills showcases a candidate who will quickly be able to learn and negotiate new or in-house programming languages, tools, and techniques that are required for the specific role but they may not have utilised in the past.

 

Analytical

Example Question 1: Run me through the analytical project you’ve undertaken recently, that you believe had the biggest impact on your company

The key word here is ‘impact’. Firstly it ensures they talk about a fairly meaty piece of work, but it also allows me as the interviewer a lot of scope to ask follow up questions around that impact.  It means they can’t get away with a project that is purely technical, they need have an idea of how it was implemented and also hopefully be able to quantify with specifics (I’m looking for actual numbers), i.e. “it increased sales by 12% YoY” (a further follow up question, how did you robustly measure incremental sales?) or “it saved us 3 hours of code run time per week which meant that we could take on another client”.  Impact can come in many shapes and sizes, but I want to see that they have a grasp of the project over and above what they did specifically!

Example Question 2: Run me through the most complex analytical project you’ve undertaken recently

The key word in this one is ‘complex’.  The question itself can give you a good view of how advanced the candidate is, but why I really like this question is that there is a trade-off for the candidate between choosing something extremely advanced and having to find a way of explaining it to me simply and effectively, OR choosing a more simplistic project that perhaps doesn’t show off their capabilities.  Breaking down a complicated topic into something understandable that you can communicate to a first-time listener is a vital skill for any Data Scientist.

 

Passion & Curiosity

A good Data Scientist has a broad knowledge of different analytical techniques, a great Data Scientist is able to adapt and manoeuvre the way these are applied to enable the create a bespoke solution to each unique challenge they face.  Most great Data Scientists that I’ve met are well read within their discipline, they have a keen interest to look at what others are doing and proactively look to learn and develop themselves whenever possible.  They are also excited by the evolution of the field, and also try to contribute to this evolution.

Example Question 1: What is the most interesting thing you’ve learnt in your own time that has had a tangible benefit on your job or career?

This question has worked for me in the past because it allows the candidate a fairly wide scope to explain something they’ve learned outside of work but is directly related to their role.  It might be that they’ve taken an online course to learn Computer Vision in Python, they attend Data Science ‘meet-ups’ or perhaps they enter Kaggle competitions for fun.

There aren’t really any right or wrong answers to this question but you’ll quickly get an understanding as to whether they’re passionate about the industry as they won’t be able to provide much substance if they’re not!

Example Question 2: How do you keep up with the latest trends in the industry?

Similar to the above, but a slightly different angle.  I’m not looking for much here, it’s a pretty quick question. It’s a fast-moving industry that we’re in so it’s beneficial to keep up with the latest available tools and platforms (even just at a high level) but their response gives you a feel for their curiosity and passion towards Data Science.  An interest in the industry, over and above what you’re required to do at work often translates into candidate’s who have the resources available to be creative with data and add value to your team – not just do the bare minimum to get by!

 

What to listen out for

To ensure you get the right evidence – a good template to keep in mind for each response is:

Context: Have they provided me with an understanding of who the project was for, what the project was and why it was necessary? If not, ask them to provide more context as the granular details won’t make sense to you further in their response. A good candidate will do this well naturally and it showcases their ability to convey technical concepts clearly to colleagues of varying skill sets.

Details: What did they actually do, the nuts and bolts.  Ensure you ask why they took this approach, and why – what other approaches were considered? Try and ascertain what they did, and what the wider team or their manager did too.

Results: What was the tangible impact of the project – does the candidate understand the wider strategic implications of the project?

 

Last but not least

These should really go without saying, but it’s important to keep them in mind every time you head into an interview.

Be professional, but be nice. Yes, you’re the interviewer, and they are the interviewee – but you’re both just people, and this is just a job. You’re not more important than them.

There is a high probability that they’re nervous. While this may just a 10:30am appointment for you, this is something they’ve most likely been preparing for over the last few days or weeks and they don’t want to mess up. Don’t let nerves mask a good candidate, let them warm into the interview rather than jumping straight into question one!

They will absolutely know things you don’t know, don’t get defensive or shy away from these areas. A candidate with new skills or knowledge will be very valuable to your team!