[joe_the_user]

I think you have hit upon a problem of present day AI.

Neural networks, Support Vector Machines, Hidden Markov Models and other stuff (Markov Networks, etc) do something like linear regression on some huge space - they draw a curve/plane between groups of things on this feature space. The tendency is for this division to make sense and to correspond to our common sense categorization of these things.

The problem is that once that happens, you can't really reason about the division you've drawn. It's just there. You can tweak for various purposes but that's a manual process.

You categorize animals by shape or particular adaptation or by genetic makeup. You can teach one of these algorithm each of these categorizations. But you can't do something like have the thing categorize for one purpose and then tell it to "change it's outlook" and categorize for a different purpose.

In this sense, despite seeming impressive, the products of these processes are dead-ends that we can't reason about, that lack the flexible intelligence of a human being.

[judk]

Your criticism is fair, but you fail to explain how an NN or SVM is any worse than how a human mind actually operates.

In other words, the incomprehensibility of a modern AI model is not a failing of AI, it is a failing of (AI) psychology and (AI) neuroscience.

The artificially constructed intelligence works whether or not we understand how. The frontier of AI science is now open to AI Psychology. Psychologists and Neuroscientists will replace the data scientists! Such fun!

[joe_the_user]

I could hardly contrast the operations of a NN to those of the human mind because we most definitely don't understand the later. I have already described apparent properties of human minds that NN and SVN definitely don't have. But I'll repeat and expand:

The artificially constructed intelligence works whether or not we understand how

The operation of human intelligence is very dependent on the fact that we human have operational understanding of each other's mental processes. Moreover, it is well known that human beings process language and can re-evaluate past experience in light of present understand.

Oppositely, we know enough about the properties of the various non-linear recognizers to know that constructs can't do and won't ever be able to do these things.