Hacker News new | ask | show | jobs
by wayfarer2s 4048 days ago
I think you are right in that other inputs are needed to decipher meaning. Humans for example tend to have quite a lot of different sources of input -- as when we are children and learning new words we have the spelling (visual), how it sounds (auditory), and possibly another image that shows what the thing means ("cat"). Or maybe we have the auditory ("mommy") and the visual (the child's mother). If you were trained strictly on text, then the meaning of concepts is harder to decipher. It might be why abstract concepts like higher level math are hard for a lot of people to grasp -- their only exposure to the concepts is usually just in the form of text.

As an exercise, when I think of the word "circle", images of circles and spheres show up in my head. Also the equation of a circle. My quick definition of it would be "a perfectly round object" which leads to questions of what "round" and "perfect" mean. The more I think about it, all my knowledge seems quite circular in that there are no axiomatic concepts, everything is relative and it just builds on itself. I wonder if that's the key to decipher meaning, increase the connections of the web -- with strong enough references you can pinpoint which of the nodes in the web something refers to.
3 comments
Houshalter 4048 days ago
What about programming, for example? It's entirely abstract and not necessarily have any visual representation. Programming is best learned through examples. E.g. "here's a line of code, here is what it outputs. Now try to figure out what the rules of the language are."

In the case of this article, the NN isn't being asked to do any abstract task like "decipher meaning", but the very concrete task of "predict the next word". As the article shows NNs can do this fairly well.

There is also a evidence that they can learn very high level knowledge about words and objects. See the success of word vectors: http://technology.stitchfix.com/blog/2015/03/11/word-is-wort...

link
tim333 4048 days ago
> when I think of the word "circle", images of circles and spheres show up in my head

There seems some evidence that this stuff is fairly central to human intelligence and the ability to visualize in 3d is kind of hard wired. Deciphering meaning is approximately "seeing what it means" which can correspond to visualizing it in your head. For example "the cat sat on the mat" is a bunch of symbols but someone or some machine can convert that to an image of a cat sitting on a mat then I guess they've understood it.

link
hellameta 4048 days ago
This part of your comment, "...there are no axiomatic concepts, everything is relative and it just builds on itself", reminded me of this Marvin Minsky paper. If you haven't read him before - enjoy the ride!

https://web.media.mit.edu/~minsky/papers/MusicMindMeaning.ht...

link