| I saw the HackerRank (YC S11) hiring post (https://news.ycombinator.com/item?id=47667011) and it made me realize I no longer understand how to evaluate candidates effectively. Specifically, we are changing hiring across 3 dimensions:
> Tasks: Real-world tasks on code repositories vs standard algorithmic-style puzzles
> Evaluation: AI fluency, orchestration skills vs functional correctness
> Candidate experience: Agentic IDE vs a simple code editor In the “old world,” you could ask multiple questions and triangulate skill from answers. Now it seems like evaluation depends heavily on tools and models that keep changing month to month. So I’m curious:
> What signals actually correlate with strong engineers today?
> How do you design interviews that don’t become obsolete with the next model release?
> Are algorithmic interviews still useful at all? Would love to hear from people who have recently changed their hiring process or have been interviewed using this new approach. |
Several hiring managers in the group went all in on AI-assisted interviews because they wanted the interviews to match the tools that engineers can use on their work. Most of them have gone full circle and returned back to no-AI interviews.
The main problem with AI-assisted interviews is that they become a test of how familiar the candidate is with the specific AI tool you're letting them use. They started getting inverted signals because the hardcore vibecoders knew all the tricks to brute force the problem with high token spend. They'd do things like spend the interview trying to spin up parallel subagents to brute force a solution.
Then the careful coders who tried to understand the problem and do it right were penalized because every minute they spent trying to do the problem (instead of offloading all cognitive load to AI) was time lost to letting AI do the work.
There were also simpler problems like when someone was familiar with some visual LLM interface tool but didn't have a familiar workflow for the CLI tool used in the interview.
Most people went back to coding interviews that forbid AI and test coding skills, combined with a discussion about their AI experience.
My takeaway was that it's easy to teach new hires how to use AI tools on the job, but it's much harder to bring someone with weak coding skills up to the level of someone with strong coding skills. It's even harder when that person is leaning on AI so much that they're not learning how to code anything.