The study looks at junior developers unfamiliar with what they're implementing. Post-hoc they broke down the AI group and grouped some into tutoring->hand coding. Another group was AI maximalist. We should keep in mind the comparison of these groups is very low n. Tutoring + hand coding seemed to have the best speed + understanding. This was all conducted in Jan 2026.
Where I'd push back drawing too many conclusions from this study: arguably most successful AI usage is senior developers that know the programming environment they're working in. Know how far to trust the AI. And carefully review / understand outputs.
Nevertheless, the study's still interesting, and I wish they'd replicate with a much higher n per group. Junior developers (undergrads?) are a more abundant group and not particularly specialized yet. They've also spent years hand-coding at University, but probably could adapt to AI tooling pretty easily.
Wait, hold on. So you're telling me that Anthropic is out there hyping their newest, most powerful LLM like crazy... but really it's just a token-selling scheme?
Pretty sure their strategy is "hype it up, stuff it down EVERYONE's throat, once everyone is hooked and the tool is indispensable, make them pay for it. And rachet up the prices every quarter".
Where I'd push back drawing too many conclusions from this study: arguably most successful AI usage is senior developers that know the programming environment they're working in. Know how far to trust the AI. And carefully review / understand outputs.
Nevertheless, the study's still interesting, and I wish they'd replicate with a much higher n per group. Junior developers (undergrads?) are a more abundant group and not particularly specialized yet. They've also spent years hand-coding at University, but probably could adapt to AI tooling pretty easily.
The study: https://www.anthropic.com/research/AI-assistance-coding-skil...