[fdlaks]

> The thing that people strangely still seem to not get is that these systems will continue to improve rapidly

Where is the plateau though? What we have so far is definitely impressive and useful in some cases, but I can't help but eye-roll every time I hear someone seriously talk about making nuclear reactors JUST to power the millions of GPUs needed to train these models.

[ilaksh]

We have already seen multiple plateaus as we reached barriers in terms of scaling the models the hardware and software. We have continued to innovate and break through them. The latest barrier is scaling training via inference time.

New hardware and even learning paradigms are being researched to get us over the next wall. This has been shown time and time again we keep innovating.

The power requirements with force investment to scale out the new memory-based compute paradigms.