Humans might be non-deterministic, but we can reason, we can learn, and we can have incentives to be careful and not just YOLO things, all of which mitigate that risk. None of those is true of LLMs.
i feel like you've just accidentally stumbled upon primate-patching as an umbrella term that can be anything from monkey-patching to hominid-adjustment via apefoolery. As a coder for 8 years I know I'm personally capable of operating at any of these levels depending on the day and the strength of the local coffee.
If you squint hard enough in ignorance, everything looks non-deterministic.
We need to stop the pseudophilosophy already. At this point, the AI bros think they have cornered some grand problem. If the universe is deterministic, then we can simulate it perfectly and humans and their other machines are all redundant. If the universe is non-deterministic, then stochastic AI "will eventually get better" and can replace humans too.
Have you considered there's something obviously missing with this reasoning, and you're wrong about this like the rest of us? This is not that profound.