[extasia]

The transformer architecture is arguably the most important breakthrough in NLP, and language is the predominant mode of communication between humans, so I fail to see how its "just a load of hype"

Could you name a bigger breakthrough in AI?

[sva_]

I'd say the Multi-Layer perception itself.

Maybe even convolutional neural networks, because they showed that ANNs are viable and are what really got the ball rolling.

[mafribe]

CNNs are from the 1980s (the "neocognitron" by Kunihiko Fukushima [1]), while the MLP is from 1958 [2]. So nearly half a century, resp. a century old.

[1] K. Fukushima, Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position.

[2] F. Rosenblatt, The Perceptron: A Probabilistic Model For Information Storage And Organization in the Brain.