Here [1] are some "frauds" from Stanford University, Oxford University and University College London telling you exactly that.
From their abstract:
``One of the most exciting and promising novel architectures, the Transformer neural
network, was developed without the brain in mind. In this work, we show that
transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably
place and grid cells. Furthermore, we show that this result is no surprise since
it is closely related to current hippocampal models from neuroscience. We additionally show the transformer version offers dramatic performance gains over the
neuroscience version.``
Making the claim that transformers are a good candidate model for certain neural pathways is a pretty different claim than saying the brain is literally using transformers.
I’m assuming you are asking if the brain uses transformer-like structures or otherwise exhibits similar behavior. I don’t know, but it does share some processes with simpler ML ideas, and I’d be very interested to see if it uses anything resembling a transformer.
Forward-forward algorithm is more like the brain. As I understand, backpropagation transformers require storing data, doing calculations on that aggregate, and sending it back through, which no neural structures can do anything like.