[philliphaydon]

One thing that we noticed was after switching it on, the EC2 instances were being hit directly, so we moved those into a private security group only accessible to the load balancer. RDS got restricted. S3 buckets fixed. Coupled with AWF to block on inregular activity, resulted in GD bill going down, not up.

This is no different from programming. PHP has some awful code out in the wild, it doesn't mean PHP is shit just because people write bad code.

The issue with AWS is it's far too easy for people to just spin stuff up and it works and they don't look at what they are being billed for, don't analysis their infrastructure, don't optimize. They just throw servers, containers, etc up into the wild then when the bill comes:

"OH AWS BAD I got billed cos I just set it up and forgot about it, then when it worked they charged me for it, AWS is wrong, just go baremetal."

[m1keil]

I think it is similarly easy to spin it the other way around. "AWS is just selling you the gun and the bullets, you are the one who is shooting yourself in the foot".

I don't think I said AWS is shit or that GD is worthless, after all, I use both by choice. Yet, I do not think that AWS are blameless when it comes to certain decisions of how to bill, how to present data and how to document some of their features.

For example, in order to discover something is wrong with your GD billing, you must have CloudTrail in place, and the appropriate infrastructure to query it. And even tho AWS can easily alert you about weird trend in your API calls (like suspiciously high Describe*), they won't do it. They do it with Trusted Advisor when you have under-utilised EC2 instances (which requires Business+ support plan per account).

Someone mentioned in the thread the need for SCP in order to disable regions. Why should you have go all the route to SCP? Why can't we disable regions by click of a button under root account like it's possible for some of the latest regions?

Is something inherently wrong in it and pure evil? No. But I think the defaults can be better. I think AWS can improve their customer's default posture when it comes to Audit and Security without the need to have to decide between 10 different services with different billing plans and gotchas.

[gredrotino]

Have you checked out Cost Anomaly Detection[1]? It builds an ML model to alert on anomalous usage and resulting changes in billing.

[1] https://aws.amazon.com/aws-cost-management/aws-cost-anomaly-...

[m1keil]

That's exactly my point, that is _yet another_ service you need to go through to get a clear picture of what is going on.

These products have their place, but they don't make sense until you reach a certain size.

[bnchandrapal]

Out of curiosity, did you have any data lakes on S3? Did you find optimization techniques for the same?

[philliphaydon]

Nope, but did realise we had some open buckets we didn't realise were open. Thankfully we didn't store sensitive information in there despite having 2PB of files in there.