| Hallucinations are a fundamental property of transformers, it can be minimized but never eliminated. https://www.mdpi.com/1999-4893/13/7/175 > the open-domain Frame Problem is equivalent to the Halting Problem and is therefore undecidable. Diaconescu's Theorem will help understand where Rice's theorm comes to play here. Littlestone and Warmuth's work will explain where PAC Learning really depends on a many to one reduction that is similar to fixed points. Viewing supervised learning as paramedic linear regression, this dependent on IID, and unsupervised learning as clustering thus dependent on AC will help with the above. Both IID and AC imply PEM, is another lens. Basically for problems like protein folding, which has rules that have the Markovian and Ergodic properties it will work reliably well for science. The basic three properties of (confident, competent, and inevitable wrong) will always be with us. Doesn't mean that we can't do useful things with them, but if you are waiting for the hallucinations problem to be 'solved' you will be waiting for a very long time. What this new combo of elements does do is seriously help with being able to leverage base models to do very powerful things, while not waiting for some huge groups to train a general model that fits your needs. This is a 'no effective procedure/algorithm exists' problem. Leveraging LLMs for frontier search will open up possible paths, but the limits of the tool will still be there. Stable orbits of the planets is an example of another limit of math, but JPL still does a great job as an example. Obviously someone may falsify this paper... but the safe bet is that it holds. https://arxiv.org/abs/2401.11817 Heck Laplacian determism has been falsified, but as scientists are more interested in finding useful models that doesn't mean it isn't useful. All models are wrong, some are useful is the TL;DR |