[marcosdumay]

Well, on the title, our brain seems to be equivalent to a RNN... so yeah, possibly.

Anyway, claiming that they are equivalent to transformers when RNNs are Turing complete, and forward-only NNs are not is such a strange take.

[imtringued]

It's a much stranger take to associate the brain with RNNs. It's far more likely that the brain does something similar to a path independent spiking equilibrium model trying to find a fix point, because those models are inherently robust with respect to noise and adversarial attacks, do not require more than one layer and inherently contain feedback loops and tend to generalize well to out of distribution data. Of course in practice they end up somewhere between 2x and 3x slower than a finite layer transformer for the same in distribution performance.

[RaftPeople]

> It's a much stranger take to associate the brain with RNNs

That seems like too strong of a position. The equilibrium model seems to be a good candidate for some activity in the brain, but it doesn't seem like it applies to everything.

For example, the inter-layer recurrence in vision/object detection processing seems to be something different.