[phowon]

Putting aside the whole Schmidhuber debate - where are people getting this idea that causal convolutions are anywhere near the prominence of RNNS/LSTMs?

As far as I'm aware, causal convolutions were used in WaveNet (and subsequent models) and a small number of NLP applications. Meanwhile, LSTM-based models are used in just about every NLP paper, and at least a baseline in the newer ones more dominated by Transformers.

[moultano]

I think LSTMs have been mostly abandoned. I don't see much new work using them, vs CNNs or transformers.

[phowon]

Aren't causal convolutions basically CNNs with masking?