Gets "funnier" when you think those credits can be burned in useless tokens (i.e. they have almost negative skin on the game regarding outcomes and incentives to maximize token usage).
Yes, obviously it does cost compute (to them) but customers are not paying for the compute per-se, but rather for correct results/outcomes. This is a big alignment and accountability problem, relevant enough to some make companies like Lovable refund credits to customers when errors and token/credit mishandlings are too serious to ignore. That seems unsustainable in the long run though.
I expect this to become even bigger when the hype cools down and companies start looking for ROIs.
PS: I obviously talking about model errors (i.e. like use N tokens in a LLM solving a problem by just deleting/deactivating a test), and not end-user mistakes.
Users are charged for compute and whether the outcome is positive or negative doesn't cost more for the company. I don't see where you got the idea that there are incentives to waste tokens.
EDIT (can't reply): Have you never run a business before?
Dating apps are weird because you can't have repeat customers without failing to deliver. With literally everything else repeat customers are what you want to optimize for. If your AI product fails to deliver your customers can't integrate it into their operations and you'll lose them eventually. That's a disaster.
Pretty sure we're talking past each other. If you want to continue investing on this: the perverse incentive is similar to dating apps. If you totally solve your problem with less tokens/turns, you eventually churn, paying less.
You can argue if they actually choose to follow it or not, but the objective truth is that the perverse incentive is there.
EDIT (can't reply): Have you never run a business before?
Hmm, yes?, and I've seen perverse incentives ruin things even when business-client alignment should be clear. Agency problems and stupid execs/investors are a thing.
And honestly, we're leaning into implied ad hominems here ("have you never run a business?"), so I'm not engaging with you anymore.
Customers in California and Washington (at least) may beg to differ. In both of those states, prepaid credit (gift certificates, etc.) is not allowed to have an expiry date attached.
Credits of a dollar value, not service, I believe.
Like "for a dinner" can expire, because the COGS associated with that can go up, and a business is being cost $100 for a dinner that they may have only been paid $50 for previously.
But if for a dollar amount, then it cannot be expired.
Ashley Home Furnishings, since this reminds me, are all sorts of out of compliance here. We were given gift cards by them for issues with purchases. Fine. Then when we bought our home, several family went there, and purchased gift cards with cash. We tried to use them and were told they could not be used on anything that was discounted or on sale, despite them having received cash for the card. (Side note, guess how easy it is to find anything there that is not ostensibly discounted?)