[visarga]

> If we had a way of differentiating the wheat from the chaff

This is the key to AGI. We need verification systems, they can be a code execution environment, a database of facts, a math symbolic engine, a physical simulation, a game, or real world lab experiments. These verifiers will produce signal that can be used by the language models to improve. The cheaper and faster verification is, the faster we can iterate. Generating ideas is cheap, proof matters.

Just remember AlphaZero a bit - it started from scratch, playing against itself, in a few hours it surpassed human level. Go simulation and verification is trivial. The board is just a matrix. So learning from massive search and verification is a proven path to super-human level.

Here is a related paper:

> Evolution through Large Models

https://arxiv.org/abs/2206.08896

[inthepipe]

Proof definitely matters. But at this point, as ChatGPT, AlphaZero, and others demonstrate, NNs can solve any problem provided you can express the problem as a differentiable function and get enough training data to train the function. We may be very close to a breakthrough where we can train models that detect sound, good ideas. And 100% accuracy likely isn’t necessary. Even pruning the search space for good ideas by a large amount would make humans way more productive.