[thomashabets2]

That sounds like wiring the RAM information into order of magnitude same number of transistors. A modern CPU has (quick googling) 184B transistors. If they were bits then that's 23GB. But presumably a model bit needs more than one transistor to represent how it acts as a neuron with its interactions.

Then there's the current speedup in inference from restricting which subset of the model is used, which is not a "swap in" that would work with hard wired neurons.

But I dunno. Maybe. I'm just guessing.