[rezokun]

eg. OpenAI enforces strict limits until you spend a significant amount of money.

[jasongill]

Once you do spend a significant amount of money, you'll find that the limits you can set yourself might not even work.

We have a multi-organization OpenAI account and I had set a $4k/month limit on one of the child orgs that was being used for a R&D project. Got billed ~$20k for the project one month and complained that it clearly allowed us to exceed our soft and hard limits. We were told that the limits (which you can set and act like they are a real thing) don't do anything if you have a child organization set up. They refunded us after some persuading and I know it's just normal growing pains for an organization that is undergoing rapid growth and maturity, but was still a little surprising that even the "hard limit" didn't do anything for us!

(Note that this was last year, so this bug is probably long fixed as they have redesigned their portal multiple times now)

[danpalmer]

OpenAI functionally have one API, it's easy to limit spend on one API. It's much less easy to limit spend across hundreds of APIs, and resources that have ongoing cost (like VM hours).

[mason55]

My guess is that OpenAI’s margins are much lower so they aren’t in a position to forgive or have people skip out on big bills.

For Firebase, their costs are probably pretty marginal.

[falcor84]

From my experience, cloud LLMs are still being used by most systems as an "additional feature" with fallback to alternative basic functionality, or some other form of graceful degradation. On the other hand, there typically isn't a good fallback to having your main DB go down.