|
|
|
|
|
|
by lachlan_gray
1037 days ago
|
|
|
It looks like grid cells! https://en.wikipedia.org/wiki/Grid_cell If you plot a head map of a neuron in the hidden layer on a 2D chart where one axis is $a$ and the other is $b$, I think you might get a triangular lattice. If it's doing what I think it is, then looking at another hidden neuron would give a different lattice with another orientation + scale. Also you could make a base 67 adding machine by chaining these together. I also can't help the gut feeling that the relationship between W_in-proj's neurons compared to the relationship between W_out-proj's neurons looks like the same mapping as the one between the semitone circle and the circle of fifths https://upload.wikimedia.org/wikipedia/commons/thumb/6/6f/Pi... |
|
|
