[kittikitti]

None of the predictions have any substance. It's always vague. Where are the ideas around which algorithms will be next after Transformers? Why is there no thought around the real planning on HBM memory and what we will do with the increased throughput? The forecasts, as the author aptly mentioned, are for the headlines.

[Lerc]

Algortithms: State space models, diffusion models, KANs, hierarchical attention. There are no shortage of ideas. Determining what works well is a process that is going on right now.

The question on planning on HBM is too vague to really address, but people are separately working on providing more bandwidth, using more bandwidth, and figuring out how to not need so much bandwidth.