[siekmanj]

This is mostly true for supervised and unsupervised learning models, but for reinforcement learning the LSTM is king because of the convenient fact that it can be evaluated one time step at a time, instead of just outputting a sequence like a transformer. For things like robotic control, etc, attention-based models are pretty nonsensical.

[jeremysalwen]

Not true, a transformer can be used in models without any lookahead, for example how it is used in gpt-2.! The real difference is the complexity of the model and the large increase in computational cost.