[chillacy]

We perceive ourselves building a knowledge graph but at the physical level how does that happen? Does this graph materialize physically as neuron connections? Or is it just an abstraction the brain makes, a part of the subjective experience of thinking?

It's certainly true that we think in symbols but they exist somewhere in the mushy goo of neurons, could symbolic thinking emerge from large ANNs in the same way?

[jkp56]

This graph is a high level abstraction, of course. How exactly neurons store information is interesting, but hardly relevant here. My guess is that one symbol is stored in a very sparse subset of neurons and each neuron acts a bit like a node in a DHT. All together these neurons implement a fast DHT where a word2vec graph node acts as a key. On top of that this "wet DHT" can quickly find keys nearby, i.e. in can instantly return all neighbors of word2vec("apple").

I think ANNs implement only the word2vec function that translates images or sounds into symbols and vice versa.