[explorer666]

But the problem is not computation, the problem is consciousness. I think we all agree that our level of general intelligence can be achieved "simply" by putting stuff together in the correct order. But consciousness? Why is it even there? Why is there something that it's like to be us?

[evanb]

I don't know. It's not clear to me what consciousness means, but if forced me to take a stance, I'd give a Hofstaderian response and say that it's a matter of degrees, and that perhaps consciousness is the experience of processing information. More than that, I don't think it can be said to be "even there"---or otherwise it's something outside of the content, arrangement, or behavior of our constituent matter.