Hacker News new | ask | show | jobs
by KronisLV 1436 days ago
> Not too much, like $4k extra a month i think, noticeable but not the end of the world.

It's nice that it's manageable and also a learning experience, but over here that would be like 2 months' take home salary for a software engineer.

Kind of why development/test environments shouldn't have autoexpanding or scalable anything, in my experience.

2 comments

I assume larger companies can deal with temporary mistakes on the order of one engineer’s salary.
Oh, they certainly can, some better than others. Though personally I'd most certainly want to avoid such situations.

First, to retain an air of "vaguely knows what they're doing" about me, even though everyone makes mistakes and that should be treated as something that's okay - especially if you can limit the impact of mistakes, like with automated spending limits.

Secondly, because I wouldn't want to risk doing something like that in a personal project, given that my wallet is likely to be much thinner than those of organizations.

As in, paying for that engineer was a mistake all along?
No. It doesn't really impact the company's bottom line if your software engineering org is 100 people making $20k a month and someone accidentally wastes $4k of EBS disk. It's nice if you don't waste it, of course, but "oops, filled up the disk with 'y' output" is better than "yeah actually all of those files are pretty important, I think team X is using them" because you can instantly delete it, rather than doing a multi-month project to see if team X really is using the files.
Everything should be made to bounded, is there no max that could have been set? Not using an expandable storage in test risks deviations from prod (and then you get the prid only bugs that are difficult to keep fixed). I wonder if there us a failure to do pre-prod monitoring as well - it's super handy having dashboards telling you disk bbn usage
> Everything should be made to bounded, is there no max that could have been set?

I'd expect that you'd reach the maximum once your card is rejected. :)

But truthfully many platforms out there will let you set up spending alerts, but not outright set limits because then you get into a bunch of difficult questions - should further data just be redirected and piped to /dev/null? Should you as the service provider instead limit IOPS in some way, or allow slower network connectivity if allowed egress amount of data is exceeded? What about managed databases, slow it down or throw it out altogether?

I've talked about those in detail with some of the people here ages ago and there are actually companies that take the "graceful degradation" approach, like Time4VPS who host almost all of my cloud VPSes for now: https://www.time4vps.com/?affid=5294 (affiliate link, feel free to remove affid if you'd prefer not to have it)

  What happens if I run out of bandwidth?
  We reduce your VPS server’s port speed 10 times until the new month starts. No worries, we won’t charge any extra fees or suspend your services.
Honestly, that's a really cool idea for handling resources, though one has to also understand that storage is a bit different and if you've built your entire platform around the concept of scalability and dynamically allotting more resources, you might make the choice of having the occasional story of large bills (some of which you'll probably forgive for good PR), as opposed to more frequent stories about things going down because the people forgot to pay you, as well as many enraged individuals complaining about their data being deleted, though it's supposedly your fault.

So in a way it's also a business choice to be made, though one can feasibly imagine hard spend limits being feasible to implement.

> Not using an expandable storage in test risks deviations from prod (and then you get the prid only bugs that are difficult to keep fixed).

I concede that this is an excellent point, you also should be able to test automatic scaling when necessary etc.

Though the difference is probably in being able to test it but not leave it without (more conservative) limits when you're not looking at it.