Yeah. Read “Programming as Theory Building” by Naur [1] to understand why you need to still need to develop a theory of the problem and how to model it yourself lest the LLM concoct (an incorrect) one for you.
I don't know how many times now that I've seen these things claim to have run the code and show me the hallucinated output and then go on to develop an incorrect theory based on that hallucinated output.
I've never seen the CLI coding tools do anything like that. They're designed to integrate with the tools. If you're just using a chat interface then yes, you're likely to get some inconsistent behavior.
This was Gemini CLI in kilocode. Does it often. Sometimes it even imagines that it's done a build when it hasn't - imagines build errors and then sets out to fix them. I have it set so that it asks permission prior to running commandline tools so I know it hasn't actually run make.
I use Gemini CLI daily (work is a Google shop), directly (no kilocode). I've never seen anything like that.
I wonder if it could be something to do with the kilocode integration.
But, I do more commonly run with permission required for many operations, because I find it works much better if I help it every now and then. It can get stuck on some pretty simple stuff.
We need to ask LLMs to generate documentation, diagrams, FAQs in addition to code then. We all know what this means: keeping them up to date.
Has anyone managed to setup a "reactive" way to interact with LLMs in a codebase, so that when an LLM extend or updates some part of the territory, it also extends or updates the map?
Amazing, the article is 40 years old and still totally relevant today. And even more amazing is that many of today's IT managers seem unaware of its points.
I take it as a manifestation of the temporal bigotry in computer science. That anything not new is bad, which is absolutely untrue. Old is not bad, new is not good. Where something exists in time has almost not bearing on its quality. Most knowledge and good ideas do not survive.