[Yokohiii]

For me this is a push to segment the market into consumer and industrial grade RAM. Even NVIDIA and MS are not stupid enough to think they can keep going with RAM prices exploding. Consumers need hardware to subscribe to their AI stuff.

LLMs will get bigger and even with 128GB (that many wont saturate), you wont run future frontier models. For LLM vendors and integrators it's a handy thing to move lower quality inference to the consumers.

Also running local doesn't have to mean that the models have open weights. MS will likely start to distribute closed models at scale once the hardware is there.