[keeda]

Unless everybody is writing the same code to solve the same exact problems over and over again, by definition LLMs are solving novel problems every time somebody prompts them for code. Sure, the fundamental algorithms and data structures and dependencies would be the same, but they would be composed in novel ways to address unique use-cases, which describes approximately all of software engineering.

If you want to define "novel problems" as those requiring novel algorithms and data structures etc, well, how often do humans solve those in their day-to-day coding?

[TingPing]

Based on my experience it doesn’t solve novel problems. It’s good at generating common solutions.

[keeda]

This goes back to how we define "novel problems." Is a dev building a typical CRUD webapp for some bespoke business purpose a "novel problem" or not? Reimplementing a well-known standard in a different language and infrastructure environment (e.g. https://github.com/cloudflare/workers-oauth-provider/)?

I'm probably just rephrasing what you mean, but LLMs are very good at applying standard techniques ("common solutions"?) to new use-cases. My take is, in many cases, these new use-cases are unique enough to be a "novel problem."

Otherwise, this pushes the definition of "novel problems" to something requiring entirely new techniques altogether. If so, I doubt if LLMs can solve these, but I am also pretty sure that 99.99999% of engineers cannot either.