This article repeats what we've long known about how technical interviews aren't great at evaluating technical skills and inadvertently filter for things that aren't important. But it doesn't offer a better way of evaluating technical skills. It talks about how to evaluate other things that do matter but aren't substitutes or proxies for technical skills.<p>Also, this argument is some grade school smarty pants "I'm too smart to show my work" bullshit:<p>> And because the interviewer can’t distinguish “skipped steps due to incompetence” from “skipped steps due to operating at a higher cognitive level,” they default to the interpretation that protects their ego.<p>I thought the days of hiring toxic "so smart I can't communicate" superstars was over?
Someone like this may be skipping steps because to them the steps may be so obvious as to not need to be explained, and they are giving more benefit of the doubt to the interviewer than they should. your framing makes it seem like arrogance when in the vast majority of the time I’ve seen this, it’s the candidate making the assumption the person on the other side is their peer.
There are many steps so you can’t show them all (each step has sub steps and so on). You prioritize steps, but when you are showing your work you focus on steps that are key for you and think would be key for your interviewer. It isn’t crazy that you would guess wrong about the latter given you don’t really know them.<p>A reasonable candidate can definitely make the mistake of not showing steps that they think aren’t important (and they can also go the other way and show way too many unimportant steps).
I know this is an outlier, unpopular opinion by my experience has been that the percentage of people who can really talk in depth through technical details but can't write decent code is quite small. But the percentage of people who can talk the talk but can't write code or solve puzzles on the spot in an interview environment is much higher.<p>Everyone has their one favorite anecdote about the one person who slipped through the cracks and singlehandedly brought their company to its knees. But I contend that people spend far more energy defending against this use case than the reality warrants.
The main goal of hiring someone should be to assess how well they can do the job you are hiring for, and for anyone with job experience (i.e. not fresh out of college) the best indicator of that is what have they previously achieved (especially in more recent years).<p>How well someone can solve a whiteboard challenge or brainteaser is irrelevant unless you are hiring someone to solve 10min whiteboard challenges.<p>Of course the difficulty is how to you assess what the candidate has honestly achieved in prior jobs - what was there personal contribution, and do they seem to have approached things in a way that suggests transferable/general skills that can be applied to the position you are hiring for.<p>The graphic at the beginning of the article is certainly one problem - the interviewer needs to themselves have the experience to assess the candidate. There is no point having someone with 5 years of experience interview someone with 20 years, assuming you are hiring them for that level of experience.<p>I think the best interviewing technique is just to go over the candidates experience, from most recent to as far back as you think is relevant, and get them to talk about each project. What was their role, what were their contributions, why did they make the decisions they did, what might they have done differently, etc. etc. Ask them to summarize and deep dive into the architecture, draw it on a whiteboard perhaps.
> Of course the difficulty is how to you assess what the candidate has honestly achieved in prior jobs - what was there personal contribution,<p>> What was their role, what were their contributions<p>I got a hard reality check on this when my company was getting close to hiring someone I worked with at a previous company. Their resume claimed a lot of achievements that other teams had handled. Their work was always a problem and had to be quarantined from production systems because they were very careless and would even submit broken code without testing it.<p>They would spend a lot of time in Slack discussing other team’s work so they got good at talking about it. I think they also used a trick where they claim to have competing offers from other companies and not a lot of time for interviews, pressuring the interviewers to do discussion and resume reviews instead of getting into technical questions.<p>This wasn’t the only time, just the first time. After that I started doing more checking for claims on resumes, including calling old friends who worked with a person. It’s crazy how often someone will claim some responsibility or experience on their resume assuming that nobody will ever check it.
Experience-based interviews are a fantastic way to select for candidates who have "failed up" through a long series of jobs. The underlying dynamic is that it usually takes more than a year to sever a technical employee; you can faceplant in a role and still wind up with a resume improvement.
Yeah this is my "natural" interviewing style. Like, I have a resume, I'm talking to a person, my natural curiosity about the person and their work leads me to exactly the kind of conversation you're describing. Then my hire / no-hire intuition is basically "am I impressed with them after that conversation?".<p>But then I've also read a huge amount about interviewing "correctly" over the years, probably starting with the fizzbuzz article, and eventually participating in "Big Tech" interview panel training, etc. And all of this directly contradicts this natural intuition that I have, and which your comment is espousing.<p>So I'm honestly left with pretty strong cognitive dissonance about it at this point. Are we wrong? Or is everyone else wrong? How can this consensus on the "right way" have become so ingrained for so long while being so wrong?<p>(I also haven't been involved in a lot of hiring or at a big tech company since 2022, so I also have no idea how things have evolved to adapt to the advent of AI tools. Surely nobody is doing the same kinds of whiteboard problems as they used to do! Right?!)
> The deeper issue is tacit knowledge. Most of what a skilled engineer knows is not something they can articulate on demand.<p>Why is it that everyone says that soft skills can be more important than hard skills, that engineers talk to people they don’t sit in rooms turning requirements into code, but then, it seems like one of the criticisms about the interview process is “well, engineers can come up with good solutions when alone in a room”<p>That’s not the job. Articulating technical details when in conversation with your colleagues is.
Articulation is not the issue; the reasoning and consideration process is.<p>If you step back a bit from the words on that page and squint, what you might see is something like "Most of what a skilled engineer does is recognize, sense problems, and feel things that may not be obvious."<p>The discovery of the right path forward for the goals of the organization comes after that and takes time and planning.
One returns in a week with a fully articulated solution, not at point blank. Famously, Archimedes came up with his in the bathtub.
It totally is the job. What kind of unreasonable process you have that people dont have time to think?<p>First you think, then you think about how to say it, then you talk with others. Then you think again and maybe talk again.<p>But, it is not like we were designing everything in quick on the spot debates without research.
eh, my take is that you’re kind of going off in a specific direction when reality is actually behind you little here.<p>i cannot explain why it’s not possible for us to fling the flange if i don’t understand how both the flange flanges and the fling flings.<p>being able to talk about it is a downstream effect of knowing about it.
I'm thinking of a technical screen I did recently where I didn't move forward. The time to do the screen was 30 minutes, and it was where they had a full frontend/backend and I needed to navigate around to fix a pretty arbitrary issue. I'd say this is preferable to a leetcode problem for sure, but also, I do tend to take my time to understand the system a bit before committing to changes, I mean this is sight unseen. I'm wondering if this felt too slow to the interviewer.<p>They sent me a summary document before the meeting, but I couldn't see the code until the interview. I felt like I identified the issue and where to make changes rather quickly, like 10 minutes of looking around and talking through how all the components and APIs fit together.<p>Then the interviewer asked me to implement a datetime solution, which in this time-boxed window my mind raced around to multiple solutions that I talked through out loud: I could write it myself which would definitely take some time for me to remember all of the syntax involved and reason through the problem, I could download an existing library which would also take some time to read documentation, I could google around for existing solutions in somewhere like Stack Overflow which is pretty hit and miss, or I could prompt an AI agent to write a solution for me. I talked through all of these, they wanted to know how I'd do it by hand at first, which I talked through for a bit but admitted I wasn't sure if it was a good way to go about it. Then I said given the time constraints the AI prompt route would probably make the most sense. By the time we arrived at that and tried it for a bit our time was basically up. And I got the impression suggesting AI to help code didn't impress the interviewer at all.<p>If others are able to stand out in this scenario then I guess I'll just admit I'm not the top candidate. My brain just doesn't work that quickly. I like to spend time gathering context and tinkering before really getting into the solution, and that probably doesn't come across well in these situations.
This post is 2/3rds about accuracy problems with interviews (and, yes, I think the weight of the evidence is that interviews are effectively random functions), and then it culminates in a "FATE" metric that is mostly not about accuracy.<p>Giving candidates feedback is good, but it has nothing to do with the likelihood or cost of a bad hire. Making timely decisions is good ceteris paribus, but it's of secondary importance compared to selecting the right candidate. Same with "effort".
let's say you make a saas for automatic birdfeeders.<p>if you can scope out your problem to a simplified version & have a pen & paper / whiteboard discussion with a candidate. the candidate starts from a blank slate. they create their own constraints, validate their own assumptions etc. if they can't design something that works - u can prove this since you already have a working product in production. that way interview takes place in less than 1:30hr. saving time for u & candidate.<p>you cover: communication, critical thinking, technical chops. 3 birds with one stone.<p>but unfortunately most people don't want to do that - because 1. their products r fugazi (fueled by vc money), they're on ego trips (we gonna scale) & lastly want to make interviewing a hazing|humiliation ritual.
I don't understand why can't someone just come up with an interview where they propose an issue that is truly something that can be expected in the role, give the applicant a few hours to come up with an action plan and a solution architecture, put it down in writing, sketch some graphs, and then present it...<p>Why is that so groundbraking? wouldn't that immediately tell you a lot about the applicant, regardless of the final implementation being a 10 or not?
These are called take-home interviews. They exist and applicants HATE THEM for being too long/"free work".
That's just assessing someone's ability to design a solution, while presumably what you are hiring for is the proven ability to deliver solutions, add value to the organization over time, etc.<p>Of course it would be a major red flag if someone couldn't come up with a good sounding solution to something they claimed to be capable of, but a having a plan is not enough. Ideas are really a dime a dozen, and the ability to deliver is the what you really need, which is why VC's tend to fund people, not ideas.
That’s a system design interview. Those exist already and are about 45 minutes in length.
This style of panel is great.<p>The one thing interviewers need to stay aware of though is that they will have immense insight into the problem whereas the candidate likely has never thought through the problem space before. And this can lead to unfair judgements. I know I've fallen into this trap before, where I'd run the same session with so many candidates that I had a thorough big picture view of every possible approach. I'd see a candidate stumbling a bit and think, "Why can't they see the obvious answer in front of them?" when in fact I wouldn't have seen it on my first go either. I just had seen dozens of candidates go through the same exercise to the point where it seemed like old hat.
As a former IT instructor, I've seen students who performed brilliantly in exams and interviews but struggled when faced with real-world ambiguity.<p>I've also seen quieter people who were average in interviews become excellent builders once they had a real problem to solve.<p>Interviews are useful, but the ability to ship, maintain and improve real projects should probably carry more weight.
We’ve added collaboration and communication as big facets in our hiring loop rubrics, and reduced the complexity of our questions. We also tell candidates this ahead of time - been honestly working pretty well so far, it indexes more on those soft skills and gives us just enough insight into the hard skills to make a call.<p>We’ve also been increasingly finding very few candidates know how to solve a complex question these days without LLM support lol. How are other folks changing their loops these days?
The main problem is good engineers have no need to sit through your 12 step process. It actively selects only for the most desperate or money driven people (if you pay very well).
Where are these jobs paying $500k/yr that I can skip these interviews? I haven’t seen them yet. I hear about jobs paying below $100k/yr that do this but that’s not getting us anywhere in SF.
I mean we know the answer to this. As you go up the seniority ladder interviews become less and less onerous and at some tipping point are not required. Aqui-hires such as Alexandr Wang at Meta for example. Non aqui-hire we have for instance when Andrej Karpathy joined Anthropic. I somehow doubt they went through as many round as those below them.<p>Apart from that when lower down in seniority generally start-ups. There are many founders who get funding, know good people, and will hire them without many interviews. Having a good network is critical for exploiting all of these, as the interviewer has already effectively judged your skills over many years or decades.
This point gets repeated a lot as if we are supposed to coddle engineers by making interviews wildly easy.<p>At some point as an employer you do want someone who is motivated enough to take some time out of their day to prepare for an interview.<p>Do you really want an employee who gives so little of a shit that they refuse to use their brain to get a job?<p>This isn’t exactly a hot labor market in tech. Companies have a good selection of quality talent available right now.
Making interviews efficient and making them easy are orthogonal. It depends on what attributes your organization selects for.<p>To select for people who are willing to commit to a slow bureaucratic organization you can make them go through repetitive interview rounds.<p>To select for people who do well under pressure, make the interview stressful.<p>To select for people who can solve challenging problems, make the interview challenging.<p>There's no one right answer.
I don’t want to put my future coworker through six rounds of interviews. If it takes more than three rounds + a phone screen to figure out if someone is a good fit then the process is broken.
If I think your interview process is onerous, I’ll ditch your company.<p>I’m not interested in companies arrogant enough to think people should want to work there so much that they will endure your hoop jumping.
'Hoop-jumping' is an indication that the rest of the organization is inept at moving fast and being decision-oriented. I believe capable organizations can make good decisions on limited information and their interview process should be reflective of that.<p>If the interview process takes more than 3 steps and 3h, I'm out.
That’s fine, I don’t need to hire cynical people.<p>My interview process is very reasonable. If you’ve hit the point where you are required to do a 2-3 hour technical interview round with me, you’re a short list candidate and only have 1-3 competitors for a very lucrative job.<p>If that’s too much of a hoop for you, I’ll just take the sandwich, no fries with that.
This is the mechanism:<p>“Oh, you don’t want to work for us? Well that’s a bullet dodged because not wanting to work for us means you suck (expressed in any number of ways, in this case you say I’m cynical) . We remain awesome!”
> an employee who gives so little of a shit that they refuse to use their brain to get a job?<p>Many many folks are the type that is willing to hard grind/suffer short-term to get through a hoop, but as soon as they are inside they turn that 'optimizer' mindset towards 'how can I do the minimum necessary to coast and collect my paycheck'.<p>And many many folks who are highly motivated to work hard every day at their job are not highly motivated to prepare for jumping through a hoop like a circus clown.
For your first paragraph, that’s just a risk of hiring employees that has nothing to do with the interview process. You can possibly surface some of that during behavioral interviews.<p>If you as a manager can’t detect your employees coasting that’s a you problem. Understanding how to motivate your current employees is not in the scope of the interview process.<p>For your second paragraph, we can use a cynical attitude calling this “jumping through a hoop like a circus clown” but do you really want to hire someone with such a cynical view of the minor inconvenience of interviewing?<p>A lot of candidates are very accepting of the fact that interviews will take some work to complete and don’t take a cynical attitude to it.<p>I don’t have any interest in hiring someone who thinks 2-3 hours of time for a short list candidate interview after the screening process is unreasonable.<p>If you have made it to my 2-3 hour interview process, you are only competing against 2-3 people for the job. This isn’t some kind of unreasonable waste of time, I’m offering salaries multiple times the median salary, sign-on bonus, equity, generous PTO and free healthcare plan, etc. Having a chance to get all that is definitely worth 3 hours of interviews.<p>I don’t really need to hire the person who has $10 million in their bank account and refuses to lift a finger to get a job. That person can enjoy their life and do something else.
I am not talking about difficulty but length and bureaucracy.
> The main problem is good [doctors] have no need to sit through your 12 [years of school]. It actively selects only for the most desperate or money driven people (if you pay very well).<p>do you agree with this?
I was really hoping for a conclusion section with pragmatic suggestions for how to apply these insights to a real hiring process. I'm left thinking "ok this all makes sense, but what do I do with it?".
here's an idea for an "experienced" technical interview structure. "we care about x, y and z. you have forty five minutes to convince us that you will meaningful help us achieve our goals. we then will take 30 minutes to push on technical details as we see fit. you will be judged based on technical content, choices, taste and your overall approach and strategy for moving us forward and convincing us that you're the right person to do it. good luck!"
I wonder how many great engineers with stage fright you would lose this way, though.
> <i>we care about x, y and z. you have forty five minutes to convince us that you will meaningful help us achieve our goals. we then will take 30 minutes to push on technical details</i><p>This part is good.<p>> <i>as we see fit. you will be judged based on technical content, choices, taste and your overall approach and strategy for moving us forward and convincing us that you're the right person to do it. good luck!</i><p>This part is confrontational and will lead to worse outcomes. "As we see fit" signals capriciousness. "You will be judged" signals hostility. "you're the right person" engineers conflict; there is no <i>"the right"</i> person, only <i>"a good"</i> person. And telling someone "good luck!" in this context is like telling them to try not to die while standing on a narrow plank over a pit of sharp spikes. No matter how you think you mean it, it will come across as callous to many.
Even harder with AI.<p>How do you assess developers when AI makes it possible to create software without even knowing the language?
Of course. This isn't a secret. But nobody has come up with anything better. Doesn't seem like this guy is proposing anything either.
don't you people get tired of reposting this take?<p>don't you realize it's exactly like<p>"attractive women reject the wrong suitors"<p>???
I think I find someone sexy and fun is obviously more a personal feeling on matters while I am looking for someone who will help me meet our goals for Q4 with a high degree of technical excellence seems something that should be measurable and not left up to a feeling at the moment.<p>So I guess I don't realize it's exactly like, I personally I feel it is significantly different.