Y
Hacker News
new
|
ask
|
show
|
jobs
by
xcodevn
735 days ago
On a related note: recently, I released a visualization of all MLP neurons inside the llama3 8B model. Here is an example "derivative" neuron which is triggered when talking about the derivative concept.
https://neuralblog.github.io/llama3-neurons/neuron_viewer.ht...
2 comments
skulk
735 days ago
This is insanely fun to just flip through. I found a "sex" neuron.
https://neuralblog.github.io/llama3-neurons/neuron_viewer.ht...
link
vpj
735 days ago
Pretty cool. The tokens are highlighted based on the activation?
link
xcodevn
735 days ago
Yes, you're correct. The tokens are highlighted based on the neuron activation value, which is scaled to a range of 0 to 10.
link