[soco]

It's also conveniently forgetting that AI was trained on the whole internet, a corpus of code including every garbage solution in SO wrong answers or random junior blogs. I didn't see any claim AI could select "best practices" from their sources, so putting AI to correct another AI has high chances of statistically selecting the same wrong solution, because that was just a common thread in those old posts they used as reference.

[mattmanser]

I think you're pretty wrong here, but I'm no expert so anyone who wants to correct me please do so.

As I understand it, there's post-training step that does exactly that, they get real developers to select good and bad code and help the AI figure that out. AFAIK it's the reason why Claude was able to leap-frog OpenAI in coding last year, they did this refinement step a lot better.

And now they've taught the older models this, they can use existing models to bootstrap the newer models without humans involved, and bring humans in to just assess the trickier stuff.

[soco]

Claude's coding efficiency is largely due to their control plane governing the actual execution of the model - rules, evaluators and basically looping over results. Not related to the model training or fine tuning.

[mattmanser]

The models were already much better before Claude Code came out, let's not rewrite history here.