[obirunda]

I think the primary reason is that datasets contain a lot more average/bad code than exceptional, and to add to that problem judging between those is possibly a subjective issue.

Developers using AI will get mostly average solutions faster but exceptional ones will be obviously rare. And, crucially if the idea itself is average or bad there isn't much an elegant coding solution will do for the idea.

I think this ultimately is the divide between the hype and reality of how AI will impact products. If you just give a product manager the keys to do all the coding as no code "prompt engineer", more than likely will lead to further enshitification of features in products with unmaintainable code bases. At the current state, understanding algorithms and thinking computationally is a requirement to improve a code base.

The hopes of having a "build me a $1 billion app" prompt capability, or "improve my shitty app" are too long horizon and subjective requests to bypass the hardships of product ideation and iteration to have the LLM deliver on the requests. It's not magic, it's probability. Averages are the end goal here, not excellence.

If we arrive at a point where LLMs translate general prompts into idealistic versions that are more like version 100 of the idea while still capturing the user's intent, then we will see these improvements. Otherwise it's copy pasta on steroids, and done mindlessly, will mostly lead to enshitification rather than improvements.