That was an amazing read, thank you. Clearly lays the proposition that the first (if only) problem to solve is philosophical. Which brings us to the sorry state of academic philosophy today. A bunch of people who for the most part don’t know math, or a solid understanding of reality (including quantum mechanics). A group among which “anti-scientism” is a more and more popular topic for some reason. A group, which from whenever I ask this question, they agree there’s no moral standing for eating meat but almost none of them are vegans. Only slightly less forgivable than cardiologist who die of heart attacks I suppose.
Whatever happens one can bet this AGI breakthrough if it occurs will happen quite far from this group of people.
And yet, just like the majority of essays on AGI, that linked essay is a nice survey of history, but once that history is completed the author implies they are going to discuss the history of AGI construction attempts, but never does. The article transitions into personal theory and just peters out.
Perhaps I'm short sighted, but until we have some implementation capable theory of comprehension - that universal ability of humans to observe any phenomena and mentally deconstruct it into separate individual and independent driving forces that combine to create the observed phenomena - until we have an artificial comprehension algorithm all efforts towards AGI are futile.
The essay talks about the ability of generating explanations. It affirms explanations are the basic building block of GI.
But an important intermediate step is also the ability of generating meanings. The AGI would say: "This observation implies something else, it implies this". Without an explanation why, at first there's only the relation between signified and signifier.
Maybe a first step into AGI is to add a semiotic framework, before adding an explanatory framework.
It’s likely folks like him and Hofstadter have thought long and hard about it and just haven’t found a viable theory or explanation yet and hence the conjecture. We need more extremely intelligent people thinking about it. Or maybe we don’t! But at least that’s the explanation I see as valid here.
Whatever happens one can bet this AGI breakthrough if it occurs will happen quite far from this group of people.