[lostmsu]

I disagree with the later point, as unlike camera sensors the cells in question already include trainable parameters for every single one of 100M+ inputs.

But it matters little as even with 100x reduction the estimate blows GPT out of the water in the first year, making it very sample inefficient in comparison.

As for signal I am a layman in its most extreme here (only mist-like idea about information theory and frequency relationship), but don't the bandwidth limits only apply to fixed rate measurements? E.g. there's basically infinite (sans plank limits) number of values between 4ms and 5ms and as long as the receiver can separate them, they can encode information?

To put it in other words, if the neurons can control the impulse peak delay down to a nanosecond, then shouldn't the limit be measured based on 10^9Hz of that control vs 250Hz of max firing rate?

[amscanne]

I don’t think the connections between the receptors (rods and cones) and the ganglion cells are “trainable”? You seem to be assuming Brain-like learning functionality inside the eye. I’m not a biologist, but I don’t think this is the case unless you’re considering evolution as training. If it were true, wouldn’t people have wildly varying fovea? I feel that these connections are anatomical and not learned in exactly the same way the number of arms, legs or teeth is not trainable.

Regarding the nanosecond point — I don’t believe that’s how information works, and there should be many obvious problems with the idea of an infinite information channel not to mention the obvious practical ones (propagation variability, lack of a reference point, etc.). There may be some optimizations, but generally the frequency (or frequency bandwidth, which is where the generic computing term comes from) determines the information capacity, and phase modulation doesn’t magically change this (it is actually what is used in many radio systems).