[chongli]

black-boxish behavior with lack of human-accessible state introspection and poor tractability.

I have a philosophical question in response to this. Is it even possible to have intelligence without it being a black box? Are people willing to call something intelligent if they completely understand how it works?

[varjag]

Well my point was rather practical than philosophical: if you take a human at say classification task, they'd be able to explain why they identify bicycle as bicycle and not, say, bulldozer. You can't readily have this with ANNs; all you have is a bunch of weight coefficients and feedback loops.

[chongli]

if you take a human at say classification task, they'd be able to explain why they identify bicycle as bicycle and not, say, bulldozer.

I don't think this is such an easy task. In the professional world of scientific taxonomy there are many problems with classification. The problem seems to stem from the tension between intensional and extensional definitions.

[varjag]

Yes but you still are able to verbalize these problems, quite unlike when you stuck with a misfiring ANN. In symbolic approaches, e.g. inference engines, you have comparable explanatory facilities.

[eli_gottlieb]

>Is it even possible to have intelligence without it being a black box?

That depends. Do you believe in Cartesian dualism, or atomic monism? If you believe in a monist universe where minds don't reside on some other plane of existence, then plainly a mind must be explicable to a sufficiently smart other mind, because after all so is everything else.

[chongli]

then plainly a mind must be explicable to a sufficiently smart other mind, because after all so is everything else

Right, but is that intelligence? My main argument is basically an attack on the word intelligence. I believe people use it far too frequently and they allow its meaning to change whenever it comes close to being pinned down. In a strange way, intelligence is a tricky refuge for dualism in an otherwise monist world.