[theGnuMe]

There's another paper replacing attention with FF networks so just combine the two and you've got something.

[gdoug]

Link? Sounds like a good read! :)

[smeeth]

Not op but might be this: https://arxiv.org/pdf/2311.10642.pdf