[modeless]

The most important advancements in machine learning for the next 10 years at least will be in hardware, and the software to take advantage of said hardware. You could even say that was already true starting with AlexNet, but it's even more obvious now with these enormous models.

We've barely scratched the surface of what's possible. Even if Moore's Law was dead (though it seems that TSMC may keep it alive for a bit longer) there are huge gains to be had when co-designing models and hardware. Stuff like https://www.cerebras.net/ is the direction I expect things to go.

[dna_polymerase]

Hardware will be a huge part, yes, but algorithmic advances would be even better. Utilizing existing commodity hardware to full extent is where the money is at. Specialized hardware will probably remain just that, specialized and mostly too expensive.

[modeless]

There will be algorithmic advances, but they will benefit larger models too. Larger models will still win. The value of having the best AI is so great that it will be worth nearly any level of investment in hardware to the large tech companies that can afford it.