[keiferski]

No, I'm not talking about physicalism at all. I'm talking about reductionism and emergence. Even if it were shown that human thought processes operate at lower levels in ways similar to machines, that does not imply that human beings are equivalent to machines. That is a reductive argument.

https://plato.stanford.edu/entries/properties-emergent/

[bondarchuk]

Now I don't understand at all anymore why you're disagreeing so strongly. I don't think anyone has proven that it's inherent to emergent properties that they can't be understood or explained in detail. One just has to explain the mechanisms of emergence in addition to explaining the component parts reductively (and if that is your objection to reductionist arguments I would be inclined to agree with you). But such an explanation of how intelligence can emerge from simple component parts is exactly what could potentially be provided by a better understanding of AI systems.