[OutOfHere]

I think it will get cheaper when Nvidia has a lot of competition. Currently, Nvidia charges a huge markup, and there exists much room to optimize this expense. I believe OpenAI already has plans to move to other hardware, at least partially.

[jasfi]

Google TPUs should theoretically be able to compete with Nvidia's H100, but for some reason they aren't seen as a practical alternative.

I agree that training LLMs will get cheaper, but it's likely that the compute bottleneck no longer limits LLM performance.

[Wheatman]

But why would you spend billions training an llm, when it barely shiws any improvement over the previous model.

Unless OpenAI is willing to do something desperate, the best they have right now is what llm are, and so the cost would be in maintaing them. If you already paid for a bunch of H100's to train, there is little incentive to move away unless you know TPU are going to be significantly cheaper to run, cheap enough to explain the new cost of buying them.

This is ignoring the giant bubble that has balooned out of AI hype, which if popped would be disastorous for the comapnies most invested in the industry. Nvidia has a P/E ratio of 60-70, if they dont get enough future growth to explain it, they could lose a third of their pricing if not more.

[jasfi]

A lot of the top researchers are working on making LLMs more capable, so it's not impossible for new breakthroughs to occur, they might not just be as rapid paced as the last two years or so.

There's also lots of utility to be found with the best LLMs today. I'm working on something myself, and have seen others pushing the boundaries in hackathons and startups. So that's a lot of innovation and value that's definitely not a bubble.