[badrunaway]

When we talk about system 2; is it possible that [generating large number of programs; evaluating them of the task; choosing top K outcomes; feeding it back to Neural net] can act as system 2 for a AGI? Isn't that how we think intelligently as well- by making lot of hypothesis internally and evaluating them - and updating our model?

[awwaiid]

I think it's more like humans are a chaotic choir of subsystems all doing their thing and tossing up their directives until some sort of "win" happens or the volume is loud enough in some direction that it then gets reverse engineered into a "thought". But yes.

[badrunaway]

like darwin selection between the subsystem approaches? Put in a lot of different kind of LLMs and let them play the same game inside with each other.. whosoever wins the simulation is allowed to externally present the approach... something like that?

[bshanks]

Yes, I think it's very possible that human brains unconsciously generate-and-test surprisingly large numbers of small candidate programs when solving a problem.

This approach is https://en.wikipedia.org/wiki/Embarrassingly_parallel, which is a good fit for biological neural architectures, which have very many computing nodes but each node is very slow (compared to electronic computer CPUs/GPUs).

[spencerchubb]

Possibly

I think we need those pieces, and also a piece for determining hypotheses in an efficient manner. Monte Carlo Tree Search could be that piece. Probabilistically choose a node to search, and then backpropagate the probabilities back to the root node.

[badrunaway]

Intuitively I feel efficiency is the outcome of existing world model.. approach can look like yours - I don't see why there has not been efforts on scaling monte carlo tree search for extending the existing world model via tree search. My guess is that it would diverge to hallucinations too fast because it doesn't have a strong logical building block already