[gwd]

> The next step would be to combine the two, i.e. tell chatGPT to explain the result of a logic reasoning program in natural language. It could of course also be asked to translate a natural language query into Prolog code.

From what I remember, the very initial prototype of AlphaGo just had a neural net trained on historical games; effectively saying, "what kind of move would a traditional grandmaster make here?" with no planning whatsoever. This was good enough to beat the person who wrote the prototype (who wasn't a master but wasn't a complete novice either); and to make it able to defeat grandmasters, they added Markov chains for planning (which also necessitated a separate neural net for evaluating board positions).

It sounds similar to your suggestion: A model which simply generates realistic-looking sentences is accurate maybe 85% of the time; to make it truly human (or super-human), it needs to be paired with some sort of formal structure -- the analog of the Markov chain. The difficulty being, of course, that the world and its knowledge isn't as simple to represent as a go board.

That said, making coding answers more reliable, by adding a logical structure explicitly designed to support search & testing, should be within reach.

[periheli0n]

> The difficulty being, of course, that the world and its knowledge isn't as simple to represent as a go board.

Humans suffer from the exact same limitation. The limit to correct inference and prediction is often the amount and quality of input data.

A language model that can extract information from text and interact with the user to refine and clarify that information could be tremendously useful for experts who understand how the model works.

Without that understanding it will be rather disappointing though, as we see with some of the reactions to chatGPT and also Galactica (RIP).