[jacquesm]

It's interesting how it is almost as though if you don't understand something thoroughly that you can't pluck the fruits from it. But if that were really true we wouldn't be able to appreciate art either, nobody can point at the reasoning behind a work of art to a degree that would satisfy those criteria, you're either moved by it or you aren't regardless of how it came about.

Present day AI is much like that: we do not understand it in detail but we understand the general ideas well enough to keep improving on it. Maybe one day we'll understand it to the degree that would satisfy a Minsky or a Chomsky, but until then we'll be happy to use the results, regardless of what makes it all tick.

I suspect - but of course absolutely no way to prove this yet - that all that such understanding would do would be to result in massive optimizations, not necessarily new kinds of output. Though such an optimization may well be so strong that it will serve as a qualitative change. And if in the process we discover something about how our own minds work so much the better but that wasn't the goal to begin with, unless you want the specification for an implementation of the algorithm of consciousness. It may well simply be something messy rather than orderly.