[Foofoobar12345]

We get a few thousand fresh grads applying to us each year. It’s practically impossible to interview every one of them. At the same time, any sort of coding assignment we give is easily defeated by AI—so that’s not useful either and there are very few signals there.

What we do instead is send out a test - something like a mental ability test - with hundreds of somewhat randomized questions. Many of these are highly visual in nature, making them hard to copy-paste into an AI for quick answers. The idea is that smarter candidates will solve these questions in just a few seconds - faster than it would take to ask an AI. They do the test for 30 minutes.

It’s not expected that anyone finishes the test. The goal is to generate a distribution of performance, and we simply start interviewing from the top end and make offers every week until we hit our hiring quota. Of course, this means we likely miss out on some great candidates unfortunately.

We bring the selected candidates into our office for a full day of interviews, where we explicitly monitor for any AI usage. The process generally appears to work.

On a different note, things are just getting weird.

[jannesan]

As a candidate, this sort of test gives me the worst possible impression of the company.

- 0 effort on your side - very stressful for me - completely unrelated to job - ridiculous definition of someone being “smart”

Actually, I would not even do the test most likely and I bet many others neither.

[Aurornis]

> Actually, I would not even do the test most likely and I bet many others neither.

Unpopular observation: Many people say this, but when they actually want or need a job they change their mind quickly.

I've lost count of how many of my peers went from "I will never grind LeetCode!" to working their way through LeetCode challenge lists as soon as a recruiter from a big tech company contacted them.

I talked to one hiring manager at a company who tested their mobile developer applicants by having them make an entire demo app with some non-trivial functionality. I assumed they wouldn't have any applicants, but his current problem was that too many qualified applicants were applying for every position and begging to do the test.

[wat10000]

Seriously. I’m interviewing as a programmer and you give me some ridiculous “which cube is next in the sequence” nonsense that probably has three different arguably correct answers for every question? Pass.

[Foofoobar12345]

We have to use some criteria when all applicants are effectively the same - 4000 applicants and 6 interviewers. We interview each applicant at least 3 times.

Definition of being smart is to be quick at mental math and logic, but the puzzles are represented visually. And yes, both those skills are needed in the course of our work.

Contrary to what you might expect, over 80% take the test. I suppose during next hiring season, we could A/B against random selection to compare what % go past our interview.

[not_a_bot_4sho]

> Definition of being smart is to be quick at mental math and logic

That's not smart. That's being quick at mental math and logic.

Very different things

[narnarpapadaddy]

We still do a coding assignment, but a significant chunk of the technical interview is dedicated to a walkthrough of the code. Thus far, that’s been able to detect those who relied solely on AI.

…If you used AI and can still explain to me why code works and what it does, even better. You have learned how to use new tools.

(have not tried the randomized question approach to compare, but I’m curious to try it and see what happens)

[koyote]

We do it similarly and it's pretty easy to tell if someone knows their stuff, especially as the assignment is just a platform to dig deeper in the face to face interview.

However, the coding assignment was a really good filter and allowed us to dismiss the majority of candidates before committing to a labour-intensive face to face.

I haven't interviewed anyone since AI took off, but I am assuming that from now on the majority of candidates that would usually send us crap code will send us AI code instead; thereby wasting our time when they finally appear for the face to face.

Have you encountered that yet?

[narnarpapadaddy]

Yes, but we had that problem before when somebody would farm out coding assignments to a friend. I couldn’t say yet how it’s impacted the coding assignment’s effectiveness as a filter yet. We still do get crap code just sometimes it’s obviously AI generated.

[levocardia]

>something like a mental ability test

General-purpose "mental ability tests" are typically illegal for hiring in the US.

[rahimnathwani]

Yes, not absolutely illegal, but if an applicant challenges the legality of the test, the burden of proof is on the employer.

https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co.

[umbra07]

when you say "visual questions" - are you referring to questions in the style of Mensa/gifted tests?

https://www.reddit.com/media?url=https%3A%2F%2Fpreview.redd....

[Foofoobar12345]

Simpler - eg. A table of some numbers, with a question to quickly compute averages of a filtered set, after performing some quick boolean logic to filter them.

[boscillator]

I'm still mad at IBM for giving me one of those tests for an internship after 4 years. It required a lot of fast mental arithmetic, which is, medically speaking, not my strong suit. I thought the job was programming computers, not being the computer, but the test suggests otherwise.

I probably should have figured out how to request an ADA accommodation... oh well.

[intalentive]

You could also sort by SAT / ACT score. It will yield roughly the same results as your IQ test.

[Foofoobar12345]

We don’t operate in the US. Our applicants can’t present any standardized test scores