|
|
|
|
|
|
by kridsdale1
910 days ago
|
|
|
Mixture of Experts model is likely the most significant. And the scale of everything. GPT3 embedding vectors are around 12,000, vs 768 shown here. I was curious and the 12k figure closely approximates the median synapse dimensionality of human neurons. Maybe we don’t need much more. |
|
|