| I started with a combative tone, so well - no apologies needed. Well, still - current translation systems are data-driven, without exception, vide http://norvig.com/chomsky.html. And LSTMs are awesome at picking grammar, even not one considered English grammar (line braking patterns, proper names, markup for links, etc). Vide http://karpathy.github.io/2015/05/21/rnn-effectiveness/ There are other issues like keeping track of the context, in which they suck (as of now). And right now it is like text-skimming quality, rather than "understanding" of text. For understanding meaning, it seems that text is not enough, we need embodied cognition. Not necessarily walking robots (though, it might help) but being able to combine various senses. Some concepts are rarely communicated explicitly with words (hence - learning from an arbitrarily large text corpus may not suffice), but we have enough of experience from vision, touch etc. Since I am mostly into DL for vision (though some interest in cognitive science), I got a lot of insight of the current SOTA (and its limitations) in NLP from http://www.abigailsee.com/2017/08/30/four-deep-learning-tren.... See also: > while word embeddings capture certain conceptual features such as “is edible”, and “is a tool”, they do not tend to capture perceptual features such as “is chewy” and “is curved” – potentially because the latter are not easily inferred from distributional semantics alone. |