Hacker News new | ask | show | jobs
by solidasparagus 2142 days ago
Very helpful. Pretty hard to do comprehensively without (but less comprehensive solutions could still be useful).

The biggest question is "where are you going to get your data about current spend?. AFAIK the budget data lags behind usage (which is what I assume is the reason people haven't done this before). You can do this at a per-service level (e.g. EC2) by periodically polling EC2 for active resources and keeping a running tally of cost by yourself. This works well for some services like EC2, RDS (although serverless Aurora gets tricky), etc. However for other managed services like S3 or Lambda, it's harder to keep track of how much you have spent.

Cloudtrail could be useful here, but it probably doesn't have enough information to calculate actual cost (e.g. Lambda charging based on invocation time or Athena charging based on the volume of data scanned).

But even something as simple as a tool that keeps track of just your EC2 instance spend and shuts them down if it gets too high would be useful to developers who have AWS accounts for their side projects.

1 comments

Yes I think less comprehensive solution could cover major services (EC2, lambda etc)

> The biggest question is "where are you going to get your data about current spend?. AFAIK the budget data lags behind usage

interesting, I was thinking I could poll billing metrics (total charges etc) to figure out account level charges.

In your experience, do you care more about EC2 costs or that was just an example?

> interesting, I was thinking I could poll billing metrics (total charges etc) to figure out account level charges.

If it were that easy, this would already exist ¯\_(ツ)_/¯

I think many people use AWS primarily for s3 and EC2 (and s3 costs are negligible for most side projects) so if you were picking a single service, EC2 would probably be the most broadly applicable (and IMO the most realistic one to actually build). On the other hand I would imagine that for many side project developers, their EC2 costs are the easiest to control - even if you do autoscaling, you can cap your max nodes.

What is most useful is tracking the total AWS spend because what really gets you is the service you weren't expecting to cost as much as it did, but that is a very very very hard problem.

I was looking at AWS Cost Explorer API, looks promising but don't know if it's granular enough.

after thinking about your comment on EC2, I think this tool could help smaller teams where you don't know who's doing what with the resources and you just need an upper limit of your monthly/daily spend.

about the last point, yes it's very hard to figure out. Also I thought most people check out the pricing page before trying out any aws services, wouldn't that give an estimate of what they are getting into (i.e. RDS, API gateway) ?

But again I always forget about the data transfer charges so you might be right about this.

I've used cost explorer enough to know that it lags meaningfully behind current costs, but you would probably have to do some measurements to know how much (I would guess several days). If you do that, remember that individual services (or even different components of a single service) might have different lag times. The cost explorer data seems pretty good though.

> I think this tool could help smaller teams where you don't know who's doing what with the resources and you just need an upper limit of your monthly/daily spend.

Yeah, definitely!

> But again I always forget about the data transfer charges

Yeah. AWS can have some pretty complex cost models since they charge based on the amount of resources you actually used - there are way more axes of cost than most people have time to think through.

If you end up building this, I suggest considering that many people will not want their resources automatically shut down, either because they want to serve the spike in traffic or because they are paranoid that there could be a mistake and have production infrastructure accidentally shut down. I would consider designing it so that users can choose to get a Slack message or something instead of having the resources shut down.