[dns_snek]

They profit when their customers can't hard limit their spend and end up racking up large bills by accident, or for reasons outside of their control.

By the way, your comment was flagged which seemed odd, so I looked at your profile and it seems like all of your comments are being (automatically?) flagged and don't appear on HN by default. You might want to talk to the HN staff about that.

[nemothekid]

1. This doesn't seem like a rational thing to do. A trillion dollar business built on people making mistakes and actually running up a bill? Which exec is getting a bonus when little Johnny gets hit with a $1000 charge on his CS101 project? Doesn't seem likely.

2. To me, it's more likely they don't have one because there are edge cases to consider that make "hard limits" difficult to implement. What is AWS supposed to do when you hit the limit? Is it a hard limit? Ok, so when I hit my budget, all my s3 buckets get deleted and all my EBS drives get dropped? Do all my code deployments just get deleted? Do you "bless" certain services so that they continue to charge the user even after the hard limit? How is all of this communicated?

You can set an alarm in AWS today and the user can decide what to shut off. If you really need to, you can create a script that can hard nuke your account once a limit is reached; but I don't see why AWS should nuke your account for you.

[dns_snek]

1. Even if you assume perfectly good intentions, they're not incentivized to fix the problem because they're not on the hook for the bill, but stand to profit from it. Their margins are eye-watering, so sending out a $100k bill for a service which cost them no more than $5 to provide doesn't have a downside (other than bad PR, which they seem to be blind to).

If providing this bandwidth cost them $50k rather than $5, I would bet you my entire life savings that they would QUICKLY find a way to add hard limits to their service, no matter what technical challenges they're quoting now.

2. This is probably a 95/5 problem. The vast majority of these sorts of cases that I've seen and read about are caused by increased traffic, which hits the customer either with extortionate egress fees or unintended compute scaling behavior (FaaS/VMs).

Storage is trickier, but you could stop accepting writes or reads after passing some hard limit.

This leaves some edge cases, sure, but it should handle the vast majority of unexpected bills without any destructive actions.

> You can set an alarm in AWS today and the user can decide what to shut off.

That shifts responsibility back on the customer, which is exactly the problem to begin with. I need assurances that I won't be billed more than $X in any given day, or month and this solution doesn't provide that. Maybe their API down, maybe it's serving stale data, or maybe my automation fails for some reason.

> How is all of this communicated?

Hard monthly limits:

Egress: [ $ 10 ] - When exceeded, all outbound traffic is limited to [ 0 Mbps ].

Compute: [ $ 10 ] - When exceeded, scale all compute instances to [ 0 ]

Data Storage: [ 1 TB ] - When exceeded, all writes are rejected

Reads Ops: [ $ 1 ] - When exceeded, all reads ops are rejected

It's really not that hard. Offer these limits on a per-project, or even per-resource level, that would naturally allow you to limit the blast radius. A personal website that has gone viral would likely want to have different limits than an email server under the same account, for example.

[nemothekid]

>If providing this bandwidth cost them $50k rather than $5,

No, I believe you are assuming that proving a "hard limit" feature is cheaper than just refunding people from time to time. If you consider the all the products AWS has to offer, all the different ways they are billed, then figuring out what do to for each product once that limit is reached, and potentially doing a destructive action, and then all the code and testing on top of that - it seems far easier to add a human in the loop to just refund people on a case by case basis. It's a ton of complexity for likely a rare issue, and if someone really needs it they can build one themselves and choose how to handle what needs to be done once the limit is reached.

Building all this out just so that some college kid isn't charged a bill they could obviously never pay likely isn't a good use of resources. It's not likely AWS has a reputation of being greedy either, if you explain the situation they refund you. If they are truly doing this to make money they have remarkably poor execution.

>It's really not that hard.

Storage pricing isn't done up front. You pay per gigabyte-month. If your "hard limit" kicks in, then what does Amazon do with the data you are no longer paying to store? Does Amazon drop your entire database once the credit limit is reached? There are plenty of Amazon services like this. Consider secret manager. You are charged 40 cents/month per secret. You have 100 secrets, so $40/mo, but you set your hard limit to $20. The middle of the month rolls around, what does Amazon do? Do they just drop all your encryption keys?

There are 100s of AWS services and many where you can't just apply a sensible rejection policy once you hit that hard limit without doing something the user cannot recover from. It's only not hard because you aren't actually thinking about the matrix of AWS products that exist and how they are billed.

[dns_snek]

> There are 100s of AWS services and many where you can't just apply a sensible rejection policy once you hit that hard limit without doing something the user cannot recover from. It's only not hard because you aren't actually thinking about the matrix of AWS products that exist and how they are billed.

I don't think you understand what a hard limit is. When I set a hard limit, that means that under no circumstance should I be billed more than this amount. From this you should be able to deduce that the platform needs to stop you from ever entering into a situation where they could end up having to decide between nuking your data, or overcharging you.

This means that for all data storage scenarios, they should apply limits to projected monthly usage rather than actual monthly usage.

If 1 GB of storage costs $1 per month, and my monthly limit is set to $10, then they should not allow me store more than 10GB of data, rejecting writes as needed. This design guarantees that my hard limit won't be exceeded and the data is never at risk.

Any service which doesn't persist data can apply limits based on actual usage, and safely shut them down when the limit is exceeded.

> Does Amazon drop your entire database once the credit limit is reached?

No, they should refuse further writes when your database reaches a size that would exceed $10 a month. That's a safe behavior.

> Consider secret manager. You are charged 40 cents/month per secret. You have 100 secrets, so $40/mo, but you set your hard limit to $20.

If you set your hard limit to $20, then Amazon should refuse to create more than 50 secrets. That's a safe behavior which limits your spend to at most $20.

> Building all this out just so that some college kid isn't charged a bill they could obviously never pay likely isn't a good use of resources. It's not likely AWS has a reputation of being greedy either, if you explain the situation they refund you. If they are truly doing this to make money they have remarkably poor execution.

This is just a bad faith argument, everyone from broke college students to medium sized businesses has complained about this at one point or another. Relying on the good will of a megacorporation to forgive a surprise $100k bill is nuts and if you scroll through these comments you'll find some that claim that AWS is no longer forgiving these bills like they used to.

I'm not claiming that this solution is perfect, or that it would cover every failure mode, but it sure beats the status quo and would solve the vast majority of surprise bills that result from unexpected traffic.

[nemothekid]

>This means that for all data storage scenarios, they should apply limits to projected monthly usage rather than actual monthly usage.

>No, they should refuse further writes when your database reaches a size that would exceed $10 a month. That's a safe behavior.

I think we have gotten to the point where we've lost the plot here. You previously stated "It's not that hard". We are doing projection forecasting! And the storage layer for these range of services (S3, MySQL, Postgres, MSSQL, Kafka, Keyspaces, DynamoDB, probably 20 more im missing.) must now either call out to some financial projections service and reject writes for billing reasons. On top of that if you have a hard limit you are locked out of dynamic scaling. Need to 100 nodes for an hour? Nope, you now must disable billing limits because Amazon assumes you will use every resource for the entire month. Surely someone wont disable billing limits for an hour and forget to re-enable them, right?

>but it sure beats the status quo

I'm not sure handicapping every service for some barely functional billing limit beats the status quo. It's certainly more expensive for amazon in terms of engineering load but with so many footguns I don't see how it ends up anyway other than Amazon having this feature and still having to do service refunds because billing limits were disabled due to awkward limitations.

>This is just a bad faith argument, everyone from broke college students to medium sized businesses has complained about this at one point or another. Relying on the good will of a megacorporation to forgive a surprise $100k bill is nuts and if you scroll through these comments you'll find some that claim that AWS is no longer forgiving these bills like they used to.

Medium sized businesses have had their bills forgiven for things like cryptomining for example. My point is billing limits don't really work for AWS, alerts is the best you will get because AWS doing destructive actions for you or assuming your usage patterns is worse. It's not an easy feature and to imply they keep the status quo solely because Bezos needs another yacht is reductive of the problem.

[dns_snek]

> I think we have gotten to the point where we've lost the plot here. You previously stated "It's not that hard". We are doing projection forecasting!

It's not a complex "forecast". It's simply the monthly rate as I've illustrated with numerous examples.

If my limit is $10, then don't let me occupy more storage than that. It's a 1:1 conversation between gigabytes and how much they cost on a monthly basis.

Do you have ties to cloud providers you'd like to disclose? It's a very simple concept that you're refusing to understand, which usually happens when your job depends on it.

[chrisoverzero]

> If you set your hard limit to $20, then Amazon should refuse to create more than 50 secrets. That's a safe behavior which limits your spend to at most $20.

I have 100 secrets and no hard limit. Now I set the hard limit to $20. What does AWS do?

[progman32]

Refuses to set that hard limit until you delete some secrets first.