Hacker News new | ask | show | jobs
by kibwen 63 days ago
The evidence is that quotas exist, as seen here, and are low enough that people are hitting them regularly. When was the last time you hit your quota of Google searches? When was the last time you hit your quota of StackOverflow questions? When was the last time you hit your quota of YouTube videos? Any service will rate limit abuse, but if abuse is indistinguishable from regular use from the provider's perspective, that's not a good sign.
2 comments
jerf 63 days ago
It's also kind of interesting that they don't think they can do what an economy would normally do in this situation, which is raise prices until supply matches. Shortages generally imply mispricing.

There's a lot of angles you take from that as a starting point and I'm not confident that I fully understand it, so I'll leave it to the reader.

link
8note 63 days ago
the sales pitch is that you can keep throwing more and more tokens at a problem to solve it.

if the prices dont keep going down, the pitch falls apart, that you need a specialist to come in and make it work

link
caminante 63 days ago
Great point.

The parent's argument is that the marginal cost of inference is minimal. However, the fundamental flaw is that he's separating inference from the high cost frontier models. It's a cross-subsidy that can't be ignored.

link
bachmeier 63 days ago
Without any insider knowledge on the economics of these companies, I suspect it's that the amount of infrastructure you have to build is determined by peak usage rather than average usage. If peak usage is much higher for a small part of one day a week (say on Monday morning as software developers across the US get back to work) the cost of fulfilling demand at all times can be insane. That's why companies are implementing batch/standard/priority pricing for the API.
link
caminante 62 days ago
Check out this article from today [0].

It sounds like it's more of a profit maximization function (and not just demand) with GPU rental prices increasing 48% since Feb.

> Renting one of Nvidia’s most-advanced Blackwell generation of chips for one hour costs $4.08, up 48% from the $2.75 it cost two months ago, according to the Ornn Compute Price Index.

[0] https://www.wsj.com/tech/ai/ai-is-using-so-much-energy-that-...

link