It’s not my first rodeo in the Israeli job market, but it’s the first time in 10 years(!) that I’m constantly hitting the LeetCode wall. I really don’t remember how to solve these interview questions, as it’s been about 15 years since I got my engineering degree. Which makes somewhat of an unfair question.
As a candidate in a recent interview, I completely blacked out while trying to solve the easiest one in the book. I clearly stated to the interviewing manager that I have no plan or time to spend a month practicing for a job that may not be fitting for me. It is my personal choice, but his response was very interesting: “so, how am I supposed to know how good of a coder you are?”.
Here’s the thing. Even if I would have solved the LeetCode question perfectly, the “true positive” wouldn’t have been a valid indication of my coding practices. LeetCode doesn’t have anything to do with Clean Code, TDD/BDD, or any engineering practices. To my perspective, data structures and algorithms are skills adjacent to coding practices. A really helpful skill, but not an indication of and not necessarily a part of the coding practices.
So, I’ve decided to write this article. To help the next interviewing manager to better assess me as an engineer / coder. I had my fair share of recruitment processes, both as a candidate and as an interviewing manager/engineer. So I’ve gathered many alternative tests to LeetCode through the years, which I’d love to share with you now.
Not by the Book
First, let’s start with what’s the purpose of a LeetCode question. It is a filter-out test, one that checks if a candidate lacks a trait we are looking for, and does so quickly. A kind of test we wish to do early in the recruitment pipeline, in order to clear out irrelevant candidates sooner than later. To have a more efficient recruitment process.
Second, I’ll continue by saying maybe I am totally wrong about LeetCode questions. Maybe for your perspective and your needs, knowing to solve data structures and algorithms questions is a must. I won’t argue about it, as you know your needs better than me. But it is somewhat arguable in the industry, and for two major reasons.
It is being said that LeetCode doesn’t matter. Because practically speaking, we’d barely be using that skill and knowledge at work. It’s also being said that it’s something easy to learn. If a candidate would just memorize most of the questions and solutions, they’ll pass most of the job interviews. Supposedly, all it takes is a month (240 hours) with the Cracking the Code Interview book.
To counter those arguments, I suggest that you’d come up with a question of your own. Just like my past mentor has told me to do. Try and remember at least one time in your career, where the DS/Algo skill has helped you to solve a technical or a business problem. I can personally think of 6 such cases in my 15-year career (and I wrote very shitty and unmaintainable code back then).
For example, one case was a problem requiring to store 100 million items of a product pricing catalog to memory, on a single server with limited memory. Each item had a title (String of unknown length), a unique ID (String) and a price (double).
While most went straight to tokenizing the title, the solution was to break the price into two referencing integers representing dollars and cents, and store the values in a HashSet. It works because they are highly repetitive within a pricing catalog. Obviously, a sample catalog was provided. BTW, years later I’ve learned that’s the caching mechanism behind Double class in Java.
Similarly, take one of your cases and turn it into an interviewing question.
Fairness
If you can’t find one case, maybe there shouldn’t be any LeetCode questions in your recruitment process. Because there’s nothing in your career supporting your choice to do so, and it may not be required in your current working place.
Whether you came up with a LeetCode of your own, or asking one out of the book, you should be validating its fairness. Test your colleagues with a timer. If your current team can’t pass your recruitment process – there might be something wrong with it. And if they require practice or studying before it, it might be a sign you are testing for unrequired skills.
Practical Engineering
Let’s start exploring alternative filter-out tests.
Some LeetCode questions are not about data structures and algorithms. Some revolve around practical engineering concepts. Such topics would be caching, client-server relationship, retries error handling, and many more. It would test and validate a candidate’s experience and logical thinking, but probably not his coding skills.
Let’s take the topic of concurrency as an example. It would be fair to ask an experienced Java coder to solve a multi-threading related problem. Same goes with Go and goroutines. Or a language-agnostic question about racing conditions. To make it harder, we can ask the same but between multiple applications in a distributed system.
A backend engineer would find them fair. A front-end engineer won’t. Due to JavaScript’s concurrency model, no frontend engineer needs to code concurrency. So these questions may need to be tailored and relevant to the candidate’s field of expertise.
Once again, your career is full of examples of practical problem solving. Just pick one of those, turn it into an interviewing question – and don’t forget to test its fairness on your colleagues.
The Unknown Coding
Another alternative is to ask to solve a logical problem, using a small subset of 3-4 operators of a known (e.g. Assembly) or a made up language. It’s agnostic to a candidate’s experience – whether they know Java or JavaScript, or whether they have 10 years of experience or are juniors.
This kind of test comes to validate whether someone lacks the ability to learn a new coding language, or lacks logical thinking. It’s not exactly skills, but more of traits and thought processes which leads to acquiring skills a fine coder needs.
For example, the one I came up with was to extract data from HTMLs using only 3 CSS Selectors (ID, Class and :first). A candidate was given two HTMLs and was asked to write 3 selectors:
- A selector that extracts a price from the first HTML only.
- A selector that extracts a price from the second HTML only.
- A selector that extracts the price from both.
Obviously, the trick was in the HTML structures that had some overlap between them which caused the obvious selectors to fail. It’s a puzzle one can not solve without thinking.
Another example would be of a manager of mine. He used to ask to solve the Parachuted Robots logical puzzle. I’m not in favor of logical puzzles, but I find them fair.
Screening Curiosity
The book Who about recruitment processes efficiency, suggests to ask one proper screening question during the first phone interview. Yes, that would require the recruiting managers to do the call themselves.
As solving LeetCode over the phone would be a horrible experience, I suggest something else. Not to question, but to interrogate. Not over code, but over technology. To filter-out those who don’t know how the things they work with actually work. When someone fails such a test, it’s a clear indication of not going beyond. Either because a candidate lacked the opportunity to do so, or is lacking the curiosity to do so.
To prepare for it, I go over the candidate’s resume. I’m looking for two similar technologies they worked with through their career, and ask them to compare the two. For example, if they worked with MySQL and DynamoDB, I’d ask them about the most basic difference between the two and when they would use one over the other.
If they are experienced with two coding languages, e.g. Java and JavaScript, I’d ask them to compare something of the two’s internals, such as thread/concurrency model or anything performance related. And again, when would they use one over the other.
If a candidate knows only one technology, let’s say just React, I’d ask them to review the technology. Why is it good, why did they like working with it, and what is it good for – and when not to use it. I then select one of the technical properties mentioned, and ask how it is technically implemented.
Honestly, some won’t know and that’s okay. So I ask them to guess, or how they would have implemented it. And I inspect if their guess makes engineering sense. If I really want to be mischievous, I ask them if they think they were right. Make them ponder about it again. If they find their own mistake, that’s a fine answer for me.
Home Review
The problem with filter-out tests is that they are too short to uncover a candidate’s coding practices and experience. Which is why when I was a recruiting manager, I always gave a home assignment. Something fair, that shouldn’t take more than 3 hours for those who know what they do. A filter-in test, one that checks if a candidate has a trait we are looking for.
No matter how shining the candidate is, I never skipped this assignment. Otherwise, I might be recruiting someone who wastes his colleagues’ time on reviewing, and his own time in constant refactoring.
I look to see how much the code is readable, and maintainable. I look at the structure of the project, the relationship formed between the different classes/files. I ask myself three non-binary questions: how much time I’d be spending reviewing their code (minutes and hours); how much time it would take to refactor it; how much time it would take the candidate’s skills to catch up with the team’s level of coding (months).
Honestly, I could have done this better. A few years ago as a candidate, I was interviewed for a senior engineer position at Proofpoint. Instead of a LeetCode question, or any other technical one, we had an hour and a half session of a code review over the given homework assignment. Turns out, I was given “half” a test, and the other half was during that session – to do live coding and change the code due to new requirements.
I must say that this is one of the best recruitment processes I’ve ever been through, and one of the most innovative and indicative tests I’ve seen. Not only does it uncover one’s coding skills, but also properly simulates how he’d react to a review. Uncovers a personality.
Timing, Attrition & Willingness
The problem with homework assignment is that candidates may not have time to invest in the assignment. Not even 2-3 hours. As a result, fitting candidates might simply give up and drop out of the recruitment process. Or simply we’d lose the race, a candidate ending up signing-up somewhere else because they have a more efficient process.
If I’ll put on the hat of a candidate again, I’d say it’s a true and grave concern. I had dropped out of many processes because I was given too long of an assignment or I was just busy with work and had no time to do it. But on the other hand, sometimes I did find myself investing days and weeks without any objection.
How come? Because I was given the assignment when I had enough will to do the job, so I was willing to do almost anything to get it. What created that will within me? The interviews with the soon-to-be-direct-manager, and the entire team. After I decided I wanted to work with these specific people, and to do their specific work.
With this insight as a candidate, when given a time consuming task, I say “okay but we’ll do it later, near the end of the process”. Because it is my personal choice not to invest time into positions I’m not yet interested in, or waste a lot of my personal time just to end up “not fitting company culture”.
Remember that recruiting manager from the intro? To him I said “I’m even willing to study for a month and crack any LeetCode question as complicated as you wish – but that would be after you say you and your company wishes to work with me. Not before.”. Needless to say they didn’t. I’ve saved a lot of time and trouble for all of us, which is a beneficial result in my book.
With these insights as a recruiting manager, I’ve changed my recruitment process – because knowing when to do a test is as important as the test itself. I’ve given the homework assignment to the candidate only after I made sure they are willing to jump through hoops to get the job. I gave it as the last technical test, one step prior to the HR interview (another filter-in test!). If it was up to me it would have been after we also found his personality matching our needs. (BTW, if you don’t know how to do an HR interview – read Who!)
If a candidate is unwilling, it only goes to say they are not really interested in the job. They dropping out sooner would save everyone’s time.
Office Work
At Snyk, to overcome attrition, they have done something interesting. Prior to the frontal technical interview, they are giving candidates an entire application as a code example, and asking them to know it by heart. Which takes 30-60 minutes or so.
During the frontal interview, they are doing a live coding session with it for about two hours, which includes both changing the code and a code review.
It’s as fair and indicative as a homework assignment, but far less time consuming. It does not create any objection to any candidate, thus it is a test we can do sooner than later. Honestly, the next time I have the option to effect a recruitment process, I’ll be exercising this test as well.
I’ve also asked Claude.AI if he has any suggestions, and two were nasty in a good way. Add a deliberate bug in the code, and ask the candidate to find and fix it. I suggest doing it without an actual debugger, a harsh playing field.
Claude’s second suggestion was to add an inefficient algorithm / data structure to the assignment – and for the candidate to analyze and better it. It sounds more fair to me than writing an entire one from scratch, and it might be a fairer way to test for these specific skills.
Process
Throughout this article, we’ve seen many filter-in alternatives to just asking LeetCode questions. More than that, we’ve seen that a test is just one stage out of many in the recruitment process. If we choose to rely on one test, it should be a fair filter-out test. One asked sooner than later.
We’ve also talked about process efficiency. An efficient recruitment process is a well-timed combination of filter-in and filter-out tests. Screening out non-fitting candidates sooner than later, would save your team’s time interviewing those. Same goes for filtering-in fitting candidates, while making sure they are not dropping out too early.
Each step in the process, each test, is to gather the information you need in order to say with confidence “we want this candidate”. If some information is missing, it’s okay to add more tests. Such as a design session, which is a must but out of the scope of this article.
And lastly, make sure it’s fun to work with you. If it is, try and make sure the recruitment process is a fun experience as well. And treat your candidates with respect and compassion, just as you would have wished to be treated during an interview.
Happy hunting!