| At my workplace they recently enabled Microsoft's Copilot which is basically OpenAI + RAG. It's a joke. It is painful to watch how slow it types. About 30% of the time it doesn't work at all. I asked it how big a particle accelerator would have to be to have 1 PeV protons and it got the right equation for the radius of the ring but it told me the ring would be like 10^34 meters in radius which is about 10^24 times too many because it got the units totally wrong. It is completely unable to help a person get 5kg of antimatter, make a nuclear weapon or a deadly virus that will kill everybody because... it can't. People are afraid it will try so it has been taught to claim that it won't do these things. It is also completely unable to put a list of items (say 20 or more) in a specified order. For instance it falls down on "list isotopes that decay by emitting a positron in order of half-life" or "list US states in order of how many characters in its name", etc. It can't do it any more than it can help you get 5kg of antimatter, but it lies and says that it can, will attempt and get it wrong, apologize when you point this out, try again, get it wrong again, endlessly. If you like pushing bubbles around under a rug, gen A.I. is really for you. Now yes, LLMs can solve difficult problems in information extraction (say relationship extraction) that we were lost at sea with 5 years ago. But intelligence, truth and such are all difficult philosophical concepts. People who know better act insulted when I remind them that neural networks don't repeal the laws of computer science (e.g. Godel, Turing and all that.) They can't make P=NP, they can't solve the very serious problems of logic + arithmetic that Godel warns about, but they are very good at telling you something that will bypass your defenses (I think this comes out of picking the "most likely" word because it won't surprise you) and that you want to hear (what else is RLHF?) One real danger of LLMs is that they are better at seduction than we are because they don't have a self to manifest itself and get in the way. I am very worried about LLMs taking over dating sites, being used to run "pig butchering" scams and similar sorts of evil. See https://www.amazon.com/G%C3%B6del-Escher-Bach-Eternal-Golden... to get some idea of how LLMs could appear to be 95% of the way there but still be structurally wrong to solve real-life problems that involve logic, arithmetic and everything else. |
I'm not sure if your Godel Escher Bach reference is real, but Douglass Hofstadter himself has recanted some of his views in light of LLMs.
The things you're complaining about are implementation details.
> People who know better act insulted when I remind them that neural networks don't repeal the laws of computer science (e.g. Godel, Turing and all that.)
There is absolutely no evidence that humans do either. Not that I don't believe they do, because I -- due to religious reasons -- think we do. But from a purely empirical perspective, I don't see how you can make this claim at all.