|
|
|
|
|
by utopcell
895 days ago
|
|
Here [1] are some "frauds" from Stanford University, Oxford University and University College London telling you exactly that. From their abstract: ``One of the most exciting and promising novel architectures, the Transformer neural
network, was developed without the brain in mind. In this work, we show that
transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably
place and grid cells. Furthermore, we show that this result is no surprise since
it is closely related to current hippocampal models from neuroscience. We additionally show the transformer version offers dramatic performance gains over the
neuroscience version.`` [1] https://arxiv.org/abs/2112.04035 |
|