[jbattle]

This seems to be the core of the argument:

To get a better model, we either need make the learning algorithm more complex/write more code (which is human work), or we need to gather more or better sample data (which requires human work as well).

Assuming this is a correct formulation, I don't see why this necessarily poses problems. The gathering of data in particular seems like a process that can be supercharged. You aren't restricted to one human speaking into the computer's ear. You have (to start) the entire internet to consume. If you want/need more structured data, you could hypothetically organize dozens or hundreds of individual humans processing information for the 'mind'.

And once the AI has reached some valuable state, you can then start cloning it (presumably a simple process of copying electronic state elsewhere). The one or more limited AI's you've created could then be tasked with generating the next step up the chain - even if that is simply learning to process and ingest ever vaster amounts of information.

I'm not a wild-eyed futurist, but I either don't get or don't buy the fundamental objection here.

[enki]

author: there's no fundamental objection. i'm only pointing out that throwing faster computation at the problem doesn't necessarily solve it. right now all the work essentially needs to be done by humans.

If you follow the link to the lower bound to learning by example, you'll see that it's really hard to learn by example if you haven't pre-encoded ideas about the interpretation into the reading program. Without prior knowledge, you need lots and lots and lots of examples.

All the information that humanity has ever stored on digital media is likely not remotely enough to reconstruct a human mind from.