[gwbas1c]

One thing I'm really curious about is why caps are so hard? (Perhaps this would result in a more technical blog post?)

IE, you clearly don't want to terminate or shut down an account if they get too close to a cap. But what about things like a warning email, service slowdown, ect?

Likewise, the old "slashdotted" or "hug of death" might be an appropriate result when something goes beyond a reasonable safety buffer?

Anyway, just curious. It's clear that it's a complicated topic, and the real constraints and challenges are interesting.

[tptacek]

We talk about warnings in the post. We'll do that at some point. The hard part about caps is what to do when someone hits them.

If there was a way to make caps work for our core customers, we'd do it. We're open to ideas. A theme of our work this past month and these next several months is extracting maximal value from ANFWWAONW, our new billing system. The thing you have to remember though is that our belief about our core customer is that they are averse to nothing more fiercely than service disruption.

We're not in principle opposed to caps. We just don't have a product story for them that we're comfortable with. Keeping you from spending more money than you wanted to is an explicit product goal of ours (again: see post); we're just very wary of trading availability off against that goal.

[Alacart]

Configurable warnings as webhooks would be pretty cool. Then I can automate whatever needs to happen on my side.

I already automate apps, machines, etc with the machines API and GraphQL, so my big worries in this area are:

- Woops, some bad logic deployed too many machines (sounds like this policy helps) - Some kind of mistake or attack that just explodes bandwidth usage suddenly

[Aeolun]

You are comfortable scaling a service down to zero when configured to do so though.

Of course that comes up again when anyone sends a request, but that feels sort of in the same category.

That said, I do understand that building your service for people like me that’d rather be restricted to just $50/month doesn’t really make sense.

[tptacek]

I don't think anyone with a serious app running on us will use a cap. Just stay fixated on this scenario: a deploy-only token gets stolen, and the attacker (like most cloud attackers) uses it to stand up a bunch of Monero miners. As a consequence... their main app goes down? Who would be OK with that?

[Aeolun]

I think the cap (if they had any) would probably reflect the amount they stand to lose if their app goes down. If your app brings in $1000/h, you don’t necessarily care about spending that amount on servers. When your costs rise to $10k/hour, you might want to go with the nuclear option.

Of course it’s nicer if you can be certain that your provider is going to refund you the excess, but I feel like it’s hard to count on it. Or at least, harder than having explicit rules, which you just can’t really do for those sitations that are sensitive to fraud.

Honestly, if I did set a cap I’d be very much aware of the fact my app could suddenly die in a situation where my deploy token were stolen (but it wouldn’t matter for me, since it’s a hobby project, I care about controlling costs, not uptime).

[gwbas1c]

> Just stay fixated on this scenario: ... the attacker ... uses it to stand up a bunch of Monero miners.

This sounds more and more like an insurance policy; as opposed to a "sudden spike of load because of popularity" situation.

IE, all our talk about usage caps is really missing the point about what needs to be protected against.

[donavanm]

Metering, pricing, and billing is way more complicated than you might assume. There are historical posts here with more details if you search. In short its all async, theres variable lag, theres multiple “types” or dimensions to metering, the prices vary by SKU + customer + previous metering or billing value + other SKU usage, and billing is not uniform across customers. Imagine needing to accumulate all the metering events, apply pricing plans, modify price based on accumulated value, then periodically recalculate it all to compute a bill, then apply more transforms and credits, and that gets to the billable price.

Now evaluate your “cap” rules (which will be just as complex) and feed that back to the actual admin/control plane of the service.

[naniwaduni]

No, this is motivated reasoning brainrot. It's overcomplicating the problem by hyperfocusing on a specific implementation that's already been judged infeasible to justify not doing it.

The actual problem that people want solved is "the customer wants predictable, budgetable upper bound periodic cost". You are not unique in offering a service where this is a desirable property. Realigning this sort of cost structure is the bread and butter of insurance industry, and no, as much as they'd like to, they don't actually do it by making sure to stop the earthquake before it knocks down more of your house than your price cap.

[donavanm]

I dont understand what youre on about. The post I replied to, and many others, use “caps” to refer to limits of service based on billing. This is an endless source of comments on every “cloud” topic. I provided a very brief overview of why large billing systems are more complicated than expected and have an impedance mismatch to the stated desire. But sure, tautological brain rot. Got it. Im sure you have a wealth of experience with metering, billing, and pricing for billion dollar revenue streams.

Now if you have some _other_ proposal for how billing and service limits could function Im legitimately interested. But I dont see anything at all specific or actionable in your replies. Insurance is interesting for _some_ facets. Im curious how you think that aligns with dynamic resource utilization and what happens at the boundary.

[Dylan16807]

> if you have some _other_ proposal for how billing and service limits could function Im legitimately interested.

Billing doesn't have to be so complicated you can't calculate it in less than a minute. That's a technical failure. Surely you can imagine a better way? If you really think it can't be better, then it's hard to argue against "brain rot".

Also on most systems it will work perfectly fine to use an estimate of the price per unit when calculating the bill for the last couple hours.

[donavanm]

Again, unless you have domain expertise I suspect you are drastically underestimating the complexity of reality. As I alluded to earlier metering, pricing, and billing are _at least_ three separate complicated problems in substantial systems. There are multiple metering dimensions that interact with each other, time, and periodicity at a minimum. Saying “do billing better” without either a thorough understanding of the existing system, or an actual concrete proposal, is not useful.

[Dylan16807]

Being separate systems is fine as long as they can all calculate a number within the time it takes to make a cup of coffee. That's not hard if you actually make it a design goal.

I'm sorry that "prioritize it" isn't very helpful, but it's true. If the calculation takes longer than that, it's a management-induced problem.

If we're positing a system that can already do that, then you need to be more specific about the problem you're describing because it's not obvious.

[tptacek]

No, this is a much harder problem than you think it is; it's, like, CAP-theorem problematic. I think you should do the exercise of working through how these billing systems work.

[naniwaduni]

I reiterate that you are not unique in offering a service where the customer's desire for consistent billing and your willingness to provide it are at odds.

[Jgrubb]

Define “consistent billing” please, I don’t even know what that means and I work in this space.

[tptacek]

I don't believe you saw me say that.

[naniwaduni]

The technical problem is not "we are operationally incapable of, say, getting someone else to underwrite insurance contracts". That's not a technical constraint, It's a business decision.

I get that the underlying issue is that your target consumer is whales who eat orders-of-magnitude pricing spreads as normal opex, and that anyone who comes in with a budget is barely a consideration. It's still absurd to pretend that pricing is too hard. Like, I'm not going to confidently assert "lol you can give up C, it'll be a rounding error anyway" but like, billing cycles are absurdly long on the scale of your technical constraints.

[genewitch]

> If you're really only looking to spend $50, we should put our cards on table and say that we're generally not making product pricing decisions with you in mind.

i think is what gave that vibe off. I was on a read-only phone in bed and saw the quoted message. got up and logged into the PC to think about what to say. It may be time for cloud providers to dissuade small users away, instead.

[gwbas1c]

I understand all that; and that's not really the kind of information I'm looking for. (I know deeply that metadata is often more expensive and complicated to process than the data your customer cares about.)

This response explains the problem best: https://news.ycombinator.com/item?id=41334596

> I don't think anyone with a serious app running on us will use a cap. Just stay fixated on this scenario: a deploy-only token gets stolen, and the attacker (like most cloud attackers) uses it to stand up a bunch of Monero miners. As a consequence... their main app goes down? Who would be OK with that?

Reading between the lines: If a customer's utilization suddenly spikes, the assumption is that the customer's revenue will follow. IE, if my utilization goes up 10x, my revenue will go up 10x, so I'll happily pay the bill.

What they are providing is more like an insurance policy against hacking.

And that is the answer that I was looking for.

[rincebrain]

It "seems" simple, but people have really complicated needs, and the simplest answer if you don't have the time to build and maintain something to cover enough of the state space to make enough people happy is to not support it.

Some people are going to want you to go to 1Mb/s if you blow your bandwidth limits, or cash spend. Some might want you to go to 10 connections at once. Some might want you to just disable networking entirely.

And what happens to your storage when you blow your billing? Or CPU time?

It all becomes a hairy mess of state machines and companies wanting precisely _their_ requirements met, so you try to offer as much as you need to still have a compelling offering that enough people want to use, and no more.

Probably at best you could provide an API to make it easy for customers to build their own state machine, but that's fraught because then the customer will still blame you even if their own code did the wrong thing.

[naniwaduni]

Sinclair effect in action: "It is difficult to get a man to understand something, when his salary depends on his not understanding it."

Obviously, cloud providers are capable of billing. It's not unreasonable to expect them to offer, at minimum, "Look, we're not evaluating billing continuously but we might, at our option, try to shut down your services after $X and won't bill you excess of $Y" as a (optional) hard contractual term. Having a larger, well-capitalized party absorb risk on behalf of a smaller party for a fee is a business model humans know how to price. It's just not the desired "product".

Note that this doesn't even per se require any technical artifact to implement, just very primitive metering and lush margins.

[tptacek]

Seriously question: have you thought through what billing for bandwidth, storage, compute, and services actually entails? I'm having a bit of a reaction to the word "obviously" there.

[naniwaduni]

I don't need to, the fact is manifest in your ability to send out the bills for them. This seems very obvious to me, and your incredulity is baffling.