[stared]

My criticism is towards this paper, not necessarily - the author. Surely, he knows something about AI (otherwise it would be impossible write anything gaining such publicity) and philosophy (AFAIK it is his field).

Though, even if someone is accomplished scientist in a given field, it does not mean they are incapable of making (to put it mildly) questionable statements (Noam Chomsky on data-driven NLP, Judea Pearl on Deep Learning, Roger Penrose on quantum measurement and consciousness; from historical - Albert Einstein on quantum physics).

Yet, there are many errors which won't be noticed by newcomers, but are demonstrably false for researchers and practitioners. It is dangerous as novices may be prone to "appeal to authority" and mistake witty style for knowledge.

Don't take me wrong - I am all for sharing ideas, even half-baked. But I think that it works well better when there isn't artificially boosted confidence.

[YeGoblynQueenne]

>> (...) Noam Chomsky on data-driven NLP, (...)

If you can excuse the slightly combative tone, data-driven (i.e. statisical) NLP is a big potato and Chomsky was dead on the money: you can model text, with enough examples of text, but you can't model language. Because text is not language.

Which is why we have excellent dependency parsers that are useless outside the Brown corpus (if memory serves; might be the WSJ) and very successful sentiment classifiers for very specific corpora (IMDB), etc, but there is no system that can generate coherent language that makes sense in a given conversational context and even the most advanced models can't model meaning to save their butts. And don't let me get started on machine translation.

Like I say - apologies for the combative tone, but in terms of overpromising, modern, statistical NLP takes the biscuit. A whole field has been persisting with a complete fantasy -that it's possible to learn language from examples of text- for several decades now, oblivious to all the evidence to the contrary. A perfect example of blindly pursuing performance on arbitrary benchmarks, rather than looking for something that really works.

[stared]

I started with a combative tone, so well - no apologies needed.

Well, still - current translation systems are data-driven, without exception, vide http://norvig.com/chomsky.html.

And LSTMs are awesome at picking grammar, even not one considered English grammar (line braking patterns, proper names, markup for links, etc). Vide http://karpathy.github.io/2015/05/21/rnn-effectiveness/

There are other issues like keeping track of the context, in which they suck (as of now). And right now it is like text-skimming quality, rather than "understanding" of text.

For understanding meaning, it seems that text is not enough, we need embodied cognition. Not necessarily walking robots (though, it might help) but being able to combine various senses. Some concepts are rarely communicated explicitly with words (hence - learning from an arbitrarily large text corpus may not suffice), but we have enough of experience from vision, touch etc.

Since I am mostly into DL for vision (though some interest in cognitive science), I got a lot of insight of the current SOTA (and its limitations) in NLP from http://www.abigailsee.com/2017/08/30/four-deep-learning-tren.... See also:

> while word embeddings capture certain conceptual features such as “is edible”, and “is a tool”, they do not tend to capture perceptual features such as “is chewy” and “is curved” – potentially because the latter are not easily inferred from distributional semantics alone.

[Aqua]

On the one hand you make this sound extremely bad, while at the same time you describe it as just "making questionable statements".

Also, maybe I misunderstood the analogy, but I think you're being very unfair putting Albert Einstein who was wrong on quantum physics in the same basket as Roger Penrose with his view on consciousness, which may be questionable, but hasn't been disproved.

[stared]

You are right that I shouldn't have put them in the same basket.

While Penrose's ideas on consciousness are not considered mainstream (neither by cognitive scientists nor quantum physicists), they don't fall in the infertile basket of:

- people gravitating to the state of science they were "raised into"

- people talking about things they are don't mastered

In this case it is a healthy scientific peculiarity. And who knows, it may turn out true. Or false, yet fertile. As ideas of faster-than-light communication with quantum states - which was flawed, yet gave birth to quantum information (more to this story, and an interesting overlap of non-science and science, in http://www.hippiessavedphysics.com/).