A human can literally write a game of life in one line of Python using some clever comprehensions and lambdas. It’s not that impressive. The point of the game is simple rules create surprising emergent behaviors.
And the point of LLMs is that simple NNs finally show emergent behaviors if you throw a billion times more processing power at them. So it took a megawatt of electricity to write a longer than human version of game of life. We should be impressed. /s
I'm actually not sure though whether it's more impressive to willfully attempt to not think hard about a problem, while thinking really hard about how to get something else to solve that problem for you. That may actually be a completely different order of thinking, and its value might be judged on whether such tools are always available from now on. In other words, we could just be arguing that if it wasn't written in Assembly it doesn't show real understanding... when such things have become apparently irrelevant.
Emergent behavior tends to be more interesting to programmers than users.
Consider game AI. You could make really complex AI with neural networks and such, but most of the time it adds nothing to the game and the user can’t even tell the difference from classically programmed AI. The devs probably think it’s super cool though.
This is how it will be for other stuff. People are impressed with some no-code travel agent AI powered by LLMs but with some thought you could pretty much make the same thing with scripting and it works predictably 100% of the time and utilizing far less power and compute.
If entire humans can be replaced by simple programs, the logic dictates AI can also be replaced by simple programs even if the AI functions at human level intelligence. I fail to see a compelling business case for AI.
I'm actually not sure though whether it's more impressive to willfully attempt to not think hard about a problem, while thinking really hard about how to get something else to solve that problem for you. That may actually be a completely different order of thinking, and its value might be judged on whether such tools are always available from now on. In other words, we could just be arguing that if it wasn't written in Assembly it doesn't show real understanding... when such things have become apparently irrelevant.