Hacker News new | ask | show | jobs
by visarga 3035 days ago
An interesting property of word vectors (which are usually 300-600 dimensional vectors) is that most are quasi-orthogonal. That means when you sum them up, they compose well representing all the meanings of the component parts, and strangely, multi-sense words such as "bank" contain all the senses overlapped, yet distinct.

Another interesting property is that high dimensional space has many shortcuts, or that at any point there are many paths. It's like a kaleidoscope with infinite reflections or like a mirror house. Or it's like any point has many close neighbours which can, paradoxically, be far apart between them.
2 comments
mywittyname 3034 days ago
> An interesting property of word vectors (which are usually 300-600 dimensional vectors) is that most are quasi-orthogonal.

I think it would be more interesting if this wasn't the case. The set of all possible English words is <200,000, with probably 10% of those being in common use. Given the small set, large number of dimensions, and the nature of language, it seems likely that non-random word vectors would tend towards orthogonality.

I'm assuming you mean that, "I will run with Bob" and "I will jog with Stacy" are not orthogonal, because they convey a very similar message, but are orthogonal to, "Man, that was a good beer."

link
afpx 3034 days ago
Could this be one of the reasons why there are so many conspiracy theories these days? That is, with so many dimensions of data available, one can find close connections between any two things?
link
mywittyname 3034 days ago
A conspiracy theorist will see a lack of evidence to the contrary as evidence of the affirmative. I.e., aliens exist because government officials let us in to the military base so we can see for ourselves there are no aliens there.

Whereas a reasonable person needs evidence of affirmative to believe something. I.e., news that NASA probes have discovered bacteria in subterranean wells on Mars.

The former way of thinking can be used to "prove" just about anything. So yeah, in some sense, most data points helps fuel more conspiracies because certain people are conditioned to believe anything.

link