Fable wrote a pretty decent test suite covering typical Prolog programs, including things with non-trivial execution patterns like "append" that do complex backtracking on multiple branches. And I've run a modest number of test queries by hand, trying a few things. It's entirely possible that there's a bug there somewhere. But it's better than I would have done on the first try, implementing a Prolog interpreter in assembly. And I've worked on actual production compilers a few times.
I am honestly not happy about the way that models can now just take what should have be a fun multi-weekend project and knock out in a couple of hours. But I'm not going to pretend that Fable is stupid, or that it did a bad job on any of the test projects I gave it. It struggles more on big, messy real-world code bases, absolutely.
It's also conveniently forgetting that AI was trained on the whole internet, a corpus of code including every garbage solution in SO wrong answers or random junior blogs. I didn't see any claim AI could select "best practices" from their sources, so putting AI to correct another AI has high chances of statistically selecting the same wrong solution, because that was just a common thread in those old posts they used as reference.
I think you're pretty wrong here, but I'm no expert so anyone who wants to correct me please do so.
As I understand it, there's post-training step that does exactly that, they get real developers to select good and bad code and help the AI figure that out. AFAIK it's the reason why Claude was able to leap-frog OpenAI in coding last year, they did this refinement step a lot better.
And now they've taught the older models this, they can use existing models to bootstrap the newer models without humans involved, and bring humans in to just assess the trickier stuff.
Claude's coding efficiency is largely due to their control plane governing the actual execution of the model - rules, evaluators and basically looping over results. Not related to the model training or fine tuning.
I am honestly not happy about the way that models can now just take what should have be a fun multi-weekend project and knock out in a couple of hours. But I'm not going to pretend that Fable is stupid, or that it did a bad job on any of the test projects I gave it. It struggles more on big, messy real-world code bases, absolutely.