I don't know how many times now that I've seen these things claim to have run the code and show me the hallucinated output and then go on to develop an incorrect theory based on that hallucinated output.
I've never seen the CLI coding tools do anything like that. They're designed to integrate with the tools. If you're just using a chat interface then yes, you're likely to get some inconsistent behavior.
This was Gemini CLI in kilocode. Does it often. Sometimes it even imagines that it's done a build when it hasn't - imagines build errors and then sets out to fix them. I have it set so that it asks permission prior to running commandline tools so I know it hasn't actually run make.
I use Gemini CLI daily (work is a Google shop), directly (no kilocode). I've never seen anything like that.
I wonder if it could be something to do with the kilocode integration.
But, I do more commonly run with permission required for many operations, because I find it works much better if I help it every now and then. It can get stuck on some pretty simple stuff.