[krackers]

>simply because they're probabilistic

You can make them deterministic by fixing the temperature. But even still, there is some intrinsic error rate: it's like an oracle machine where some subset of queries will simply give you the wrong answer... which is just like humans (how many times have answers on StackOverflow been wrong), but it's definitely not like "conventional" programs.

I guess just as people had to get used to "trusting" computers, I think there's probably going to be another adjustment period of "untrusting" them and finding the right balance of "trust but verify." I think this is also mirrored in adoption patterns: mathematicians seem to be more open to using LLMs because they're used to not blindly trusting supposed proofs. On the other hand, it's well known that writing code is easier than reading it, so developers are a bit more wary.