Hacker News new | ask | show | jobs
by Wheatman 588 days ago
Open AI is already running a huge loss currently, and it isnt like o1 is cheap to run either.

I wonder what price these llm's can be run in order to be profitable,and whether just running your own model would be worth it.

Maybe it could be even cheaper if youre willing to fall behind on R&D, but keep in my mind that everytime openAI invented something, it was quickly copied by its competitors.
1 comments
OutOfHere 588 days ago
I think it will get cheaper when Nvidia has a lot of competition. Currently, Nvidia charges a huge markup, and there exists much room to optimize this expense. I believe OpenAI already has plans to move to other hardware, at least partially.
link
jasfi 587 days ago
Google TPUs should theoretically be able to compete with Nvidia's H100, but for some reason they aren't seen as a practical alternative.

I agree that training LLMs will get cheaper, but it's likely that the compute bottleneck no longer limits LLM performance.

link
Wheatman 587 days ago
But why would you spend billions training an llm, when it barely shiws any improvement over the previous model.

Unless OpenAI is willing to do something desperate, the best they have right now is what llm are, and so the cost would be in maintaing them. If you already paid for a bunch of H100's to train, there is little incentive to move away unless you know TPU are going to be significantly cheaper to run, cheap enough to explain the new cost of buying them.

This is ignoring the giant bubble that has balooned out of AI hype, which if popped would be disastorous for the comapnies most invested in the industry. Nvidia has a P/E ratio of 60-70, if they dont get enough future growth to explain it, they could lose a third of their pricing if not more.

link
jasfi 587 days ago
A lot of the top researchers are working on making LLMs more capable, so it's not impossible for new breakthroughs to occur, they might not just be as rapid paced as the last two years or so.

There's also lots of utility to be found with the best LLMs today. I'm working on something myself, and have seen others pushing the boundaries in hackathons and startups. So that's a lot of innovation and value that's definitely not a bubble.

link