Yep. That's why predicting the future of a system you're a part of is sometimes literally impossible; it boils, fundamentally, to the fact that logical negation doesn't have a fixed point.
If your prediction is a probability distribution, rather than a discrete outcome label, then, assuming the distribution over the future depends continuously on your action, there should be a fixed point, I think?
Like, if you output a probability distribution among n options, and then there is a continuous map from the probability distribution you describe in your output, to another probability distribution over those options…
Err, hm, maybe you need a stronger hypothesis on the continuous map? If it is contractive then it will definitely have a fixed point. I don’t remember the hypothesis needed.
Yes, you can predict that the outcomes of e.g. the coin flip will have 50-50 distribution instead of "the coin will land heads up" but this is a pretty useless kind of "prediction"; this trivial fixed point corresponds to "I don't know, and I won't even bother to try to know"
Also, consider the game of hunter-and-prey from my reply to the sibling comment. Again, the prediction that the prey will make some choice is arguably not a prediction at all.
Game theory? Imagine a simple game of hunter and prey. There are two locations that they need to chose from. If they both chose the same location, the hunter wins. If they chose differently, the prey wins.
Now, if the hunter can accurately predict the prey's choice, they obviously simply need to select that same choice to always win. The similar logic works for the prey; but this means they can't both be perfect predictors of the future — if the hunter predicts that the prey will choose T, they will choose T as well, but that means the the prey will predict the hunter choosing T and will choose F instead, so the hunter has actually to predict that the prey will choose F, and so they will choose F as well, but that means the the prey will predict the hunter choosing F and will choose T instead...
Why ask a human being who might have the exact knowledge you are looking for when you can ask the "dumb machine of making things up and getting things wrong" instead.
Now being serious, LLMs are nondeterministic loosy compression algorithm which return "what a plausible answer to that question would be". They have no world model, no truth verification and formally will be excellent while having no verifiable content.
If you think of it as lossy compression algorithm that makes sense.
"There is a book about a boy in a magical school, with friends, who fights a big bad and its called ..."
should have Harry Potter as the highest possible response. But that is not the answer based on a truth model, just a statistical average. You could have been looking for Earthsea series. Or it could have the heat turned up and decide to go for a hallucinated answer over the statistically most likely token.
If you want a verifiable answer your best options would be to have the heat turned to 0, but that means it cannot create new responses. Or if you have heat, to ask 4 times and sample the avg. But at that point what is the advantage over just googling it?
And one shooting your answer and hoping its the bell curve answer is going to have decent results but its also proveably going to fail in a non insignificant number of cases
> If you want a verifiable answer your best options would be to have the heat turned to 0, [...]
I don't see how that helps? If you just want the same answer multiple times in a row, you could just write down the seed of your pseudorandom number generator?
> And one shooting your answer and hoping its the bell curve answer is going to have decent results but its also proveably going to fail in a non insignificant number of cases
Setting the temperature to 0 doesn't guarantee you get a good answer either.
Like, if you output a probability distribution among n options, and then there is a continuous map from the probability distribution you describe in your output, to another probability distribution over those options…
Err, hm, maybe you need a stronger hypothesis on the continuous map? If it is contractive then it will definitely have a fixed point. I don’t remember the hypothesis needed.