Hacker News new | ask | show | jobs
by typpilol 267 days ago
Ya a lot of ppl don't realize you could spend 2k on a 5090 to run some of the large models.

Or spend 20 a month for models even a 5090 couldn't run. And not have to spend your own electricity, hardware, maintenance, updates etc.
1 comments
oytis 267 days ago
20 a month for a commercial model is price dumping financed by investors. For ollama it's hopefully a sustainable price.
link
theshrike79 266 days ago
The 20 a month models definitely aren't sustainable.

This is why everyone needs to get every flavour and speedrun building all the tools they need when the infinite money faucets are turned off.

At some point companies will start raising prices or moving towards per-token pricing (Which is sustainable, but expensive).

link
gunalx 266 days ago
Depends. API pricing from oss model inference providers basically has to be sustainable, because of competition in the space.

And with that in mind, i definetly dont use more than a couple of bucks a month in API refils. (not that i really am a power user or anything)

So if you consider the 20 bucks to be balanced between poer and non power users, and with the existing rate limits, its probably not that far off being profitable, at least on the pure inference side.

link