[torginus]

Is this analogous to digital filters, where Transformers are the FIR filters that operate on the history of input, and IIR filters, which take past inputs into account with an exponentially decaying importance?