[nosseo]

Article author - I'm on the writing team at Triplebyte. Most of what we do is summarize candidates' technical performance for their introduction to companies, but we also send feedback to everyone who takes our two-hour interview. I took this responsibility over from our first engineer, who built a bunch of software to make the process faster - it lets me quickly autogenerate emails by clicking all the resources I want to include, and then highlights the things that require more careful review. (So the people who accuse me of being a robot are half-right, I guess.)

[mpeg]

It's interesting that you pat yourself on the back about it, I think the whole Triplebyte interview process is poorly thought out, including your email comms.

I did your online code quiz and got sent an email about doing a 2-hour technical interview, without really knowing much about what the job I was supposed to be applying for was.

On the interview, since I didn't really want to waste 2-hours on something I didn't want to do, I asked the guy a few questions about the company only to learn he's actually a freelancer interviewer, has little direct relationship with triplebyte and doesn't really know anything about me.

I carried on for a 2-hour quick-fire interview with a guy that was obviously trying to fill in a questionnaire rather than actually gauge my ability, questions designed by people who likely have no real-world experience in the scenarios they describe ("how would you architect the amazon.com frontpage?" is not a 2-minute answer)

About 15 minutes in, I was sure that even if I had wanted the job in the first place I wouldn't have taken it; and I had forgotten about it when I got an incredibly patronising email explaining how, if I do some online code puzzles and study hard, I too can get a job. Gee, thanks.

Granted: a bored, funemployed, grumpy dev is probably not your target audience, and I'm sure this interview style works to filter out people fresh off college, but the email was definitely the most ridiculous part.

[nosseo]

Yeah, another disadvantage of feedback is that some people really resent suggestions on how to improve - they can come across as condescending. I definitely would rather get feedback to someone who wants it even if this annoys someone who didn't, but I think lots of companies are making the opposite tradeoff - and that's part of why feedback is so rare in the industry.

[mpeg]

I don't think the suggestions are bad per se. The reason they come across as condescending is because they're obviously very generic and they're sent to people who, in the current market, can be picky about who they want to work with.

Ultimately your whole business model is competing with tech recruiters, and my recruiters will call me to give me feedback from a role application, you have reduced that human touch aspect to an email with a few links to hackerrank.

[faitswulff]

As a writers for TripleByte, how much technical background do you and the rest of the writing team have?

[nosseo]

I did a degree in symbolic systems (CS + philosophy + linguistics; I have some CS background but less than I'd have gotten from a CS degree), I did one software internship, and Triplebyte was my first job when I graduated. I'm sure every candidate I send feedback is a stronger engineer than I am, but I do have some technical background. Most engineers want to write code all day, not emails, but I think a technical background does help us do our jobs.

[account2]

>Every few months, I check that all our recommended resources are still up-to-date, available, and free.

What are your recommended resources for technical improvements in coding interviews?

[nosseo]

We have a blog post with some recommendations based on what we've seen at Triplebyte: https://triplebyte.com/blog/how-to-pass-a-programming-interv...

I think it's a pretty good starting point. I also like Cracking the Coding Interview and I think there's definitely a place for timed coding challenge sites like leetcode - especially if you've been in a role where you're mostly working on larger-scale problems rather than on producing smart, working code quickly on the fly.

[nybblesio]

Can you tell me what companies find attractive about: "...producing smart, working code quickly[;] on the fly," (emphasis mine)?

I understand there are a lot of substandard programmers in the marketplace. However, why is it that this specific criteria is the one the industry is so attracted to? Could it be that kids are proficient at these kind of games? Because I can tell you that in the 1980s, 1990s, and early 2000s, I wasn't being asked to write a regular expression parser under time pressure.

Others in this thread say they've done Triplebyte take-home tests, only to end up in a "go fast-fast-fast!" interview in the end.

Why is speed so important? Every popular [aA]gile methodology today is implicitly -- if not explicitly -- against such "machismo" programming. If you're pair programming, how is this ever relevant?

Whenever I see someone say leetcode, hackerrank, and Cracking the Coding Interview is the "answer" it translates to "only the young need apply" in my head.

[mikepurvis]

Isn't giving detailed feedback a liability nightmare? Everything I've ever heard on this is to say as little as possible.

It's certainly great as a candidate to get detailed feedback (would have really appreciated it back in the day as a co-op student), but I just wonder if the concerns over it have any merit or are overblown.

[nosseo]

My understanding is that as long as you're not discriminating against candidates on the basis of race, gender or some other protected category membership, and as long as your feedback reflects that by being focused on the technical abilities the candidate demonstrated during the interview, you're not actually at all that much risk. Of course, if you are illegally discriminating, or if your feedback suggests that you are by giving feedback on candidate appearance or something (never do that), then you're absolutely better off not sending it.

[mehrdadn]

How do you imagine they would give feedback on something that's neither technical nor illegal? I'm imagining everything from "we found you too arrogant" to "you smelled awful when you came in"...

[nosseo]

For arrogant, I'd try to make the feedback as concrete and specific as possible - "sometimes you gave confidently wrong answers. If you're guessing, it's better to tell your interviewer that. Interviewers typically won't hold it against you if you guess and guess rightly, but if you don't acknowledge you're guessing and get things wrong, it raises questions about whether you know what you don't know." or "sometimes it's great to ignore the spec because you have something better in mind, but on an interview it's typically better to demonstrate your creativity and knowledge while still building to the spec - it makes it easier for us to evaluate you" or "when talking about your last company, you said some things that came across as disdainful about your coworkers. It'd be better to highlight your achievements."

All of those are ways being arrogant can manifest, but they're much more actionable than 'you were arrogant' and unsurprisingly get received a lot better.

I wouldn't comment on smell - yes, that's valuable feedback a candidate really ought to hear from someone, but the risk of really angering them is too high for me to feel comfortable with it.

[mehrdadn]

The thing is you still reduced arrogance to technical correctness. However, what I was trying to get at was, what about cases where the technical correctness is just fine? If it's their attitude or hygiene or something else that you don't like, how do you tell them that?

I was trying to get at the same thing you just said, which is that, like you, most people would become uncomfortable providing feedback on at least some of these. Meaning that you would have to turn away these candidates without any concrete feedback. Now how do you imagine they'll react when they realize most people do get feedback but they didn't? Is their reaction (which might result in bad publicity) a risk you and your company really want to take? For what gain?

[nosseo]

So the thing is, I think arrogance is typically reflected in actual deficiencies in interview performance. If it isn't - if it's just a vibe that the interviewer got with no concrete implications for how they work with others, solve problems, or communicate - then I worry taking it into account is introducing bias. If I can't think of a concrete implication that the arrogance had, then I don't think I want to take it into account. (You almost always can identify concrete effects, though.)

[dewski]

The article addresses the "liability nightmare" and they mentioned they reached out to an employment lawyer.

[thaumasiotes]

This is called out early in the post:

> The number one reason companies cite for not sending feedback is legal risk. Interestingly, I don’t think this is true. Companies put themselves at legal risk if they are rejecting candidates for illegitimate reasons, like race, gender, or a disability. If they send feedback which tells candidates, truthfully, that they were rejected because they didn’t get very far on the coding project, then if anything a company reduces their legal risk: they have a transparent track record of evaluating candidates based only on their skills. I recently talked with an employment lawyer about this, and he didn’t think that specific feedback on technical performance put companies at risk. So legal risk, despite being frequently cited, seems unlikely to be the real driver of policies here.

Then, an explanation that legal risk isn't the same thing as lawsuit risk:

> Even if your process isn’t biased, if you send feedback that creates the perception you’re biased, that’s enough for a costly lawsuit. So while legal risk isn’t a reason not to send detailed, honest technical feedback (as long as you’re not discriminating), it’s a very good reason not to send carelessly compiled feedback through a haphazard process (even if you’re not discriminating).

[icelancer]

Right, the liability is not "we are going to send a letter that says you're too old/you're a woman" or something blatant like that, the liability is that a very well-meaning person sends a thoughtful rejection letter and it can be inferred in the language - true or not, and this is the kicker - that there is discrimination going on.

You are doing a nice thing by being detailed, but you are basically introducing nearly unbounded downside for the upside of being nice. Most companies don't find much value in this calculus. It doesn't make it the right thing to do, but it is understandable.

[Apocryphon]

I doubt that liability is much more of an excuse- how often do disgruntled candidates actually sue corporations, in any context? If a company doesn't want a candidate, there's not much value lost by them burning bridges with them, by refusing to send them feedback, or even a rejection notice. This is especially true for big companies who receive a lot of candidates. And for smaller startups, they simply lack the time and energy to give detailed responses to people they pass on. A failed candidate is of marginal use to a company.

[frereubu]

This is dealt with at length in the article.