| > "predict" I must stress that the idea of "Science predicting facts" is a consolidated formula in Philosophy of Science. And there has never been a doubt that prediction is probabilistic. But, see the example in in the parallel additional post about "dreaming and wake", the predicting activities of a junkie under psychedelics and that of a lucid thinker are substantially different. > You've never memorized You have the framework very very wrong: the point is not that we memorize, the point is that those LLMs don't check. When you state an idea, you are supposed to have checked it in other occasions before memorization. Procedural operations, of which counting is just an example, can fail in those LLMs, which means they are simulating it instead of doing it, which suggests that they «seem to be guessing an output instead of actually checking to build the output», which makes them structurally untrustworthy, unreliable - broken by design. Being black boxes (bad), they must be stress tested to see whether proper functioning is present or just simulated: the chief problem is not that they can't count, it is that they must be missing the roots of counting: procedural lucid thinking. Check the parallel submission about the detective game ("Temporal Clue")*: an algorithm that cannot fully reason with a lucid world model, solving logic puzzles, is unreliable. The probabilistic nature of the architecture in this case is below the intelligent, as opposed to the sophistication of considering less probable unexpected branches of possibilities. * https://news.ycombinator.com/item?id=43284420 |
Respectfully, I'd suggest that you are misinterpreting it or using the wrong terminology. Science is not a thing, it is a process: A hypothesis is a prediction about the world, which is validated or disproven via experiment. A validated hypothesis -- like Newton's physics -- is a model for how the world works, which may later be superseded by more accurate models. Newton's physics, though a great stride in our understanding of the world, is not a fact, instead it is an approximation of reality.
> * the predicting activities of a junkie under psychedelics and that of a lucid thinker are substantially different.*
There's also a substantial difference between the predicting activities of a cat and those of a man.
Scratch the surface, though, and the same type of thing is happening.
Of course LLMs don't predict things exactly as you do. But at what they were trained to do -- in much the same way a cat was "trained" by long eons to hunt mice -- they're extremely capable, and they're extensible and capable of abstraction much as humans are, and much unlike cats. It's not even clear that, in the general case, how they work is any worse than how we work. It's still early.
Your point, that they're structurally flawed, is noted -- but look at the average human and try to tell me that human reasoning is flawless. Human reasoning is perhaps even more unreliable. As for your detective game, how many humans, picked at random, could solve it?
> You have the framework very very wrong: the point is not that we memorize, the point is that those LLMs don't check.
Use DeepSeek R1 and try and tell me that it doesn't check. Not only does it check, it'll openly agonize over the answer it gives you. And at solving math problems for engineering purposes, it's in the 99.9th percentile of humans, if not far beyond, despite being ~1 year old. In edge cases, it's postgrad level. In the very near future, the successors of today's LLMs will be solving new theorems.
Reasoning models, in general, disprove what you're trying to state here. It's more costly, but they're capable of procedural thinking.