[bitL]

I think author is stretching arguments here a bit - DL is just partitioning space according to some pre-baked associations given to it during training; in this case it's more like a non-linear optimization where we want to end up with N-million dimensional objects of certain shape obtained by optimizing some objective function allowing predicting similar associations. It doesn't have much with the actual innate quality of understanding. Maybe reinforcement learning with deep learning together (DRL) can move us towards such a quality at least in a mechanical sense.

[jacquesm]

That's got to be one of the most concise but still complete descriptions of deep learning that I've seen so far.

The question that it implicitly raises (at least, with me) is how can we tell the difference between 'understanding' and 'deep learning' if the end results are the same?

To me 'reasoning' is a slow, conscious process, and understanding is a part of that. But classification problems , especially when done by humans when they try to work fast have no room for such conscious decision making, we go much faster than that and outsource the job to our subconscious. Predictably, the error rate goes up and in those kind of situations deep learning can today already outperform humans on the same tasks.

The weird thing is that deep learning solutions can get simple cases completely wrong, where a human would never err, and yet get some of the hardest cases - where a human would be very likely to make an error - right. It's baffling.

[petters]

On one hand, I sort of agree with you. On the other hand, what you are saying feels a little bit like saying that humans aren't impressive, because we are just atoms.

Sometimes interesting things arise from many small, simple parts.

[westoncb]

> DL is just partitioning space according to some pre-baked associations given to it during training

What I wonder is whether that's not also maybe the cornerstone of human understanding. If I understand correctly, you are essentially saying that DL is forming categories, or developing a classification scheme. Granted, if we're only talking about supervised DL, and the program is practically told where to form the boundaries—then it's not very impressive. But if the software is extracting statistically prominent commonalities and using those to form category boundaries, and arranging them hierarchically—then while the implementation may be totally different from human understanding, the effect seems to strongly overlap.

I assume I'm probably just missing something—anyone know what it is? (It seems clear that at least part of the problem here is that 'human understanding' has been left far more vague than DL, and in order to say one way or the other how much they have in common, we need to better define 'human understanding'.)

[bitL]

I think what DL is capable of is already very impressive; sometimes while playing with DL in NLP I am wondering if our languages aren't way simpler than we think/hope and we aren't that intelligent either.

There is one massive difference between humans and DL; humans can mimic something just from a single (even partial) observation; DL requires huge amounts of data and massive parallel processing, something that we got only recently with Big Data and GPUs.

DL also suffers from the curse of dimensionality; in theory deep fully connected networks should be able to do everything better, in practice they are awful and only cleverly constructed schemes like CNNs, LSTMs etc. that assume data to be in certain format/domain bring impressive results when paired with optimizers/metrics that magically work on a given dataset. If you are able to construct a DNN that can figure out PCA/ICA/eigenvalues/etc. on its own during its training like it does with convolutions, that would again enable another set of magic tricks. In any case, humans still have to figure out the architecture of the network that works (even if we now have AutoML for figuring out best hyperparameters in parallel).

Then the hard problem of consciousness; I personally believe we are far off and probably miss something very important in our understanding of Universe.

[westoncb]

> humans can mimic something just from a single (even partial) observation;

Human learning is similarly based on large numbers of exposures; we don't form categories from single exposures. There are times where it looks that way, but what's happening instead is we come across a new particular instance of an already known general category (e.g. I have the general category 'cat', and I know they can vary on the dimensions, color, size, unruliness, etc.—and I've never seen a purple cat before, but I understand it after one exposure because it's just another value for an already known attribute. The less obvious examples are just super abstract, but have the same basic relationships in place).

> Then the hard problem of consciousness

That, by definition, doesn't have to do with any states of physical matter, nor any kind of computation. It's asking about the subjectivity of state transitions. So it should not be involved in considering a functional equivalent for (important subsets of) human brain behavior.

Edit: to clarify about the 'hard problem' relation to this: if you take Searle's 'Chinese Room' critique, for the question of functional equivalence it doesn't matter whether the person in the room understands Chinese or not; it just matters that at the end of the day the correct cards are held up.

[RivieraKid]

> Then the hard problem of consciousness; I personally believe we are far off and probably miss something very important in our understanding of Universe.

Exactly, and it's the fact that consciousness is outside of grasp of science (see my other comment in this thread).

[akyu]

>It doesn't have much with the actual innate quality of understanding

That was his entire point.