[vidarh]

For RNN abilities, RWKV is worth a look[1]

It's billed as "an RNN with GPT-level LLM performance".

[1] https://www.rwkv.com/