[opprobium]

Recent example of a proof regarding theoretical limitations of Transformers: https://aclanthology.org/2023.tacl-1.31.pdf (also extended to cover SSMs https://arxiv.org/pdf/2404.08819)

[mewpmewp2]

I'm not sure if this paper corresponds to limits on what it can answer with a single or few tokens, but also the limits where LLM itself is allowed to produce more tokens (chain of thought) as well as use tools (coding) to solve problems?