[tbrownaw]

> AGI is inevitable because computation is universal and intelligence is substrate independent.

That says nothing about timescale. There's no guarantee that the current favored approach will work. Or even that we'll figure it out before we go extinct for whatever far-future reason.

[visarga]

The path to AGI is shrouded in mist and we don't know which way to take. If you directly optimise towards an objective you might not reach it unless you're already very close to see the last few steps.

The best solution is to try everything, build the necessary stepping stones. Some of them will be useful in hindsight, we just don't know which. Biological evolution does the same. Culture evolves in a similar way.

For example, did Newton have any idea about the utility gradients will have in learning models of language? Probably not. But his contribution paved the way to GPT-3. From his position it was just impossible to foresee, from our position it seems trivial. You can learn the basics of transformer neural nets in a course of 8h on YT.

So we don't need guarantees if we are ready to try out everything. And that's exactly what the deluge of AI papers does.