> If this is the case, it's just a matter of time until costs can be reduced.
Is it, though? We cannot predict technological advancement, and the times of ̶M̶u̶r̶p̶h̶y̶'̶s̶ ̶L̶a̶w̶ Moore's Law* for computational power are long gone. There is simply no guarantee that the costs will go down enough.
I think there is plenty of room to make AI inference much more energy efficient. For example, there are companies testing creating custom silicon to run the model. Once that technology matures and we have some "good enough" models for normal use, inference cost for non-bleeding-edge models can come way down.
I don't expect bleeding-edge models to become any cheaper, but previous generation models can potentially be really cheap.
Disneyworld has lines longer than the park can manage for decades, do you expect it to just be a matter of time until park management finally figures out how to queue people efficiently enough, or do you think the solution will be once again raising costs for the customer.
> But doesn't $15M/day of inference cost imply "demand" from users? If this is the case, it's just a matter of time until costs can be reduced.
If you build a website that gives $100 for free to each one of your users, you’ll quickly have "demand" but that’s not "a matter of time until costs can be reduced".
Not sure what you're referring to. If you're talking about inference cost for frontier models, that's going up because researchers keep pushing those frontiers, often without considering cost. And while they're subsidized (to gain market share), users have no reason NOT to use the crazy expensive frontier models.
Once the market consolidates, and users get used to the idea of using models that are "good enough" because frontier models are too expensive, there's no reason AI cannot be profitable.
There's not much profit in inference, it's heavily commoditized. There is an illusion of potential profitability because the closed-weight models are currently a step ahead of the open-weight models. However, if you ignore the closed-weight models, then the open-weight models are also getting better every year. In the limit, the open-weight models will end up just as good as the closed-weight models.
AI is an inverse gold rush, the people who are getting rich off it are the people using it. The shovel-sellers are screwed.
> Not sure what you're referring to. If you're talking about inference cost for frontier models, that's going up because researchers keep pushing those frontiers, often without considering cost. And while they're subsidized (to gain market share), users have no reason NOT to use the crazy expensive frontier models.
The price of GPUs and the price of RAM to put in the servers.
Yes but it was capped by their restriction on sign ups/registrations. That could've easily been in the hundreds of millions if the app had public signups.
Idk if Instagram would exist if they were spending hundreds of millions a day.
Why would demand imply costs will be reduced? If you're making an economies of scale argument, there's plenty scale right now, and costs don't seem to be trending down.
Is it, though? We cannot predict technological advancement, and the times of ̶M̶u̶r̶p̶h̶y̶'̶s̶ ̶L̶a̶w̶ Moore's Law* for computational power are long gone. There is simply no guarantee that the costs will go down enough.
* thanks lucianbr!