[Houshalter]

What about programming, for example? It's entirely abstract and not necessarily have any visual representation. Programming is best learned through examples. E.g. "here's a line of code, here is what it outputs. Now try to figure out what the rules of the language are."

In the case of this article, the NN isn't being asked to do any abstract task like "decipher meaning", but the very concrete task of "predict the next word". As the article shows NNs can do this fairly well.

There is also a evidence that they can learn very high level knowledge about words and objects. See the success of word vectors: http://technology.stitchfix.com/blog/2015/03/11/word-is-wort...