[rachofsunshine]

Seriously, if you've never hired before, you have no idea how bad this can get.

Here's [1] our practice coding problem. It's quite similar to the one we use on our interview, and not too far from the one Triplebyte used in the past (ours is tuned to be slightly harder at the beginning and slightly easier at the end). The vast majority of candidates, even with some reasonable pre-filtering, do not get past the first step. A very non-trivial number would not even get that far.

[1] https://www.otherbranch.com/practice-coding-problem

[zem]

that's actually pretty great, but it brings up another issue I have with the state of tech interviewing - it focuses on being able to write code fast. the more senior you get, the more you tend to focus on depth, taking your time to think over the problem and write a good robust solution rather than banging out code fast, so coding up a solution in 25 minutes versus an hour is not really a good test of what the company presumably wants to hire you for.

[rachofsunshine]

This is true, and it's part of why this is one section of three.

If you were slow but high-quality on the coding section but crushed the knowledge and system design, we'd probably recommend you - or at least, recommend you to clients that aren't specifically looking for fast coders. Someone we recommended to a client recently had the equivalent of like 1.75 steps on the task linked here, but got consistently high scores everywhere else.

I do wish we could do a more complex, longer coding problem, and one of the things I've been considering is cutting some other stuff to get it up to 45 minutes or something. The current length isn't a principled decision, it's a resource constraint - keeping interviewing costs manageable is essential when you're trying to bootstrap a company in a rough market. Speed matters, but speed over such a short timescale is absolutely artificial (I'd much rather measure speed over a day instead, it's just not practical to conduct a top-of-funnel interview for so long.)

[josephg]

I was one of triplebyte’s interviewers years ago and I can speak to this. In short, you’re right. But two notes:

First, you massively underestimate the range of coding speed you see in an interview. The slowest programmers weren’t senior people who were out of practice. (I interviewed plenty of them). It was people who just seem bad at programming. Like, so bad it takes them 25 minutes to make a hello world program run. (In their favorite language, on their computer and with full access to the internet during the test).

A 2x programming speed difference would have rarely changed the outcome of our overall assessment.

Second, there was an aspect of triplebyte’s interviewing process that I’d love to see replicated elsewhere in the industry that resolves this. And that is, we should be assessing debugging ability. At triplebyte we gave candidates a smallish program (few hundred lines) with 4 bugs and a failing test case for each one. The candidates had half an hour to fix as many of the bugs as they could.

Watching people debug was fascinating.

One clear pattern that emerges is exactly what you are predicting. Smart kids right out of school were great at the programming section. But it was always the more senior engineers who smashed the debugging section. Junior engineers would get lost in the weeds and struggle to get very far in the time we gave them. Some of the senior people I interviewed dived straight in, and even found some bugs we didn’t even know about in our own test.

It seems to me that being able to read unfamiliar code and fix bugs in it is a hard to learn skill that matters on the ground. And frankly I suspect it’s more useful skill than a lot of leetcode problems. I’d rather hire someone who’s amazing at debugging than someone who’s amazing at data structures. Well, I suppose I want one of each on my team.

If I was ever making a programming test, this is something I’d include for sure.

[rachofsunshine]

We plan to, for the record. We just didn't have it ready for prime-time yet, so it isn't there right now.

[josephg]

Fair. A good debugging challenge is pretty hard to write well. I remember one of the triplebyte ones I worked on passed through several hands trying to get the calibration right.

[mech422]

That actually looks pretty good aside from the time limit.. It takes me a while to 'get in the zone' - and especially with you base datastructures you wanna think about it a bit as it has real ramifications on how hard/easy everything else can be.

Still, seems better then most of the 'leet code' type stuff I see :-)

[rachofsunshine]

Yeah, the time limit is an interviewing constraint, not a principled decision (see my reply in a sibling thread to this one).

[mech422]

Understandable :-)

[cipheredStones]

Oh, that brings back memories. When I interviewed with Triplebyte in 2018 and was rejected, my first item in the 'positive feedback' part of the email was "We saw some real strength on the tic tac toe section.", and my first item in the immediately following "the areas we think you can improve" paragraph was "We didn't see the coding proficiency we were looking for in the tic tac toe section--you didn't make very much progress."

[rachofsunshine]

Well, some idiot must have written that email!

...it might have been me, I was writing those emails in the year 2018. That was my first job there.

Jokes aside, we generated those emails largely from a template. If I remember right, "saw some real strength" was "got at least an OK score for code quality", while the negative feedback below was about number of steps. The number of mishaps with that system is part of why we do the feedback a bit differently [1] at Otherbranch, at least for now. I think we did revamp it very late in that team's lifetime, just before Triplebyte pivoted and laid off most of the team that did those (two of us, me + one of my colleagues, moved over to a different team, which is why I'm in a position to tell the whole story).

[1] https://framerusercontent.com/images/ARBLIder7AsO5KlaEpClZ3W...

(As an aside I wish Framer'd let me link images directly on the proper domain.)

[itronitron]

That is the sort of coding problem I would love to work on as an exercise however the 25 minute time limit would put me off and I wouldn't even start. I enjoy programming and I don't like to feel rushed, and if you're placing that sort of time constraint then you probably aren't a great company to work for.

[cwillu]

So, having completed the practice problem with about 45 seconds to spare, what sort of openings are there for the aging-but-not-aged Canadian who can probably only manage part-time remote work?

[rachofsunshine]

I don't anticipate us getting a lot of clients with budget for a part-time employee anytime soon, unfortunately. I imagine a lot will be remote, but probably not part time. Still happy to have you in our pool in case we do (I think I see you in the signups list - something about oddball proprietary languages, yeah?) but it's not a wildly high-probability bet in the short term.

I really, really, really wish I had a better solution for this sort of thing.

[cwillu]

Fair enough, yep, that's me.

I do wonder how my “part time” output would compare the expectations from full-time output. During the periods I've worked more conventional jobs, the vast majority of the value I brought was during the oddball times/contributions. Surely there must be some way I can be exploited (in a good way) without implicitly bundling ancillary seat warming into the contract. :D

[cwillu]

Incidentally, I didn't get a confirmation email; might be something to consider adding to the form workflow.

My email is the obvious one given my username, on the off-chance I made a typo.

[rachofsunshine]

It got submitted properly and looks like the correct email. Airflow - at least the plan we're on - caps the number of emails to different addresses that it'll send a day, so we hadn't set up confirmations, but I guess normal volume probably isn't a problem for that. (Woulda been fun today, though.)

[anonymoushn]

I think it would certainly help to receive a confirmation email. Thanks!

[ctrlw]

That was a fun coding task. Minor bug in the illustration of Step 4: the mine count in second row should be 8 not 4

[rachofsunshine]

Thanks for the correction! Fixed.

[stackskipton]

I have and you are right, it's terrible. However, I'm coming from Ops side of the house so my knowledge on Dev Hiring is talking to them and making sure they are not going to launch LedgerStore without talking to us first. Ops I'm more experienced with.

However, looking at Other Branch example test, it's another data structure question which I would totally bomb. Is this really the end all be all to dev hiring? If they can't do Data Structures, are they worthless to Silicon Valley companies? I'm honestly stumped because we get dev candidates that crash and burn with Fizzbuzz or our REST API test.

I MUST KNOW.

[duped]

> If they can't do Data Structures, are they worthless to Silicon Valley companies?

I think you might be overthinking this - understanding how to model a problem with a data structure is a core competency of any developer. This isn't a "gotcha" question where you need to know union find sets or how to invert a linked list in place.

If the question was rephrased, "design a JSON schema for the state of the board" would you know how to approach it? Because that's essentially what step 1 is asking.

[rachofsunshine]

You consider that a "data structure question"? Honest question - I would (do) characterize it as a sort of "fizzbuzz+" that is deliberately NOT data-structure-y, and I'm surprised by this response. Can you give an example of short coding tasks you would consider not data-structure-y?

(For the record, we do ask about DBs and system design in other sections of the interview. The coding is one portion of three for the interview as a whole.)

[stackskipton]

Minesweeper is all about storing data about the board and displaying the numbers is all about running through data. Seems very data structurey to me but maybe that's non heavy dev side coming through.

Maybe I'm just thinking too much problem, overloaded my small attention brain and misread it. I wouldn't call Fizzbuzz data structure-y however.

I'm also mostly outsider looking in. Never worked at FAANG and working with YC companies, I come on much later and tends to be more keeping the lights on and stopping the duct tape rocket from exploding.

[josephg]

I disagree. Minesweeper doesn’t require anything more exotic than a 2d array of enums. This is bread and butter stuff that you’ll run into in 99% of programs. If someone doesn't know how to use their language’s lists, enums and structs, I don’t think they know the language yet.

A “data structure problem” would involve more exotic data structures, usually of the kind the candidate has to implement themselves. For example, b-trees, heaps, skip lists, and so on.

The reason a lot of people don’t like custom data structure questions is that they come up rarely in most people’s jobs. Lists, structs and enums on the other hand are used everywhere. Your programming job will almost certainly require you to understand them.

[anonymoushn]

I'm a competition programmer so my perspective is generally miscalibrated, but part 5 is usually solved with a BFS. You can say BFS is basic, and it's maybe the part of competition programming that comes up more often than any other in real life (tied with sentinels, which you usually use in your BFS?), but I think it's a "data structures" problem.

[josephg]

If this test is calibrated like the triplebyte programming challenges, less than 1% of people will finish part 5 in the time given. I think when I was interviewing, only about 3% of people reached step 5 at all. Finishing was super rare, and it really only existed to separate the top couple percent. (And yes, we told the candidates that we didn’t expect them to finish in the time allocated). It doesn’t matter if step 5 is harder. Most people will never attempt it anyway.

But even then, while BFS would definitely work here, so would DFS and that’s simpler. A simple, unoptimised recursive DFS flood fill would be like 8 lines or something given by that stage you already have a function to reveal a cell. You just have to call it on all the adjacent cells. I don't see that as a data structure problem. You could argue it’s an algorithm problem, but it’s about as easy as those come.

[hobs]

I think its because the person is thinking you are looking for something especially smart when in fact something like a list of lists would be fine enough, if you can explain the tradeoffs for using them.

Edit: beaten, yes they were overthinking it.

[stackskipton]

Yep. Am bad at Dev. Pretty good at making sure Kubernetes cluster doesn't implode though.

[hobs]

I think you are selling yourself short - the problem is just some minor education to realize you probably know most of what you need to know. And I still cant figure out how to get a kube cluster working.

[itronitron]

the problem is embedding game logic (game rules) into the matrix, which is a data structure, hence it is a data structure question

[josephg]

You don’t need to embed game rules into the matrix to solve this problem. (What does that mean anyway?) Just store the board in a 2d array of some sort and write some functions to interact with it.

[itronitron]

>> store the board in a 2d array of some sort and write some functions to interact with it

thereby making it a data structure problem

[josephg]

That’s just not what a “data structure problem” commonly means. If it was, all programs would be data structure problems because all programs interact with data in some form.

A data structure problem is a problem where designing and implementing an appropriate data structure is in some way hard. A 2d array doesn’t fit the bill. It’s not exotic enough.