|
I think it's pretty clear that he hasn't kept up with the state of the art. He has no idea what coding agents are capable of or how useful they are; he doesn't pay attention to any of the contributions to math or science that these models are making; he continually assists that because agents aren't ready to face customers in uncontrolled environment, they're completely useless even for employees and workers; he just last year posted an article complaining that LLMs don't use web search to find information (he asked the information about a friend), when almost all of them do now, even in their default interfaces; he still thinks hallucinations are a problem with any weight in things like mathematics and programming where it's very easy to verify the types of things hallucinations would cause a problem with; I think he still adheres to the stochastic parrot mindset even though that's not even the most relevant part of their training anymore. Most importantly, although he seems to have made a single substack post making this argument, it doesn't seem to have really percolated through the rest of his thinking: that the cutting edge of LLMs right now, agents, are actually exactly the kind of neurosymbolic system, where neural networks provide an interface with the outside world and a creativity and problem-solving engine to provide the sort of fuzzy pattern matching and adaptability that is needed, while symbolic code-based systems ensure that guardrails are met, requirements are met, and for accurate information is provided and so on, that he wants. I think his objection might be that the problem is that the problem solving and reasoning engine at the core is still an LLM. But the thing is that you need the kind of pattern matching and flexibility and adaptability that you get from an LLM drive things, to have the end result be anything different than just an expert system with a slightly better natural language interface pasted on. And I think it's pretty clear at this point that expert systems are dead. They haven't done anything as remotely interesting or useful as what we're seeing LLMs do. I think like another commenter says that his whole stick is pointing out obviously true basic features of LLMs like that they hallucinate or don't perfectly adhere to prompt guardrails, or that there's too much hype in the industry right now, and a lot of the companies suck in a vaguely standard big tech Silicon Valley way, and extrapolating to some broader point, which is that everyone should have listened to him and done what he said when he wrote that book back in the 90s (iirc). |