[silicon2401]

Awesome job. Makes total sense how switching to Python would improve things. What was your approach to understanding what the costs were, like just looking at memory and space cost, or its cost when running on the actual infrastructure?

[deathanatos]

The cost thing was just looking at the bill. I've worked at some companies where engineering didn't even have access to the AWS bill, for example, which is rather unfortunate, since engineering is where the knowledge of "wow, this pie slice shouldn't exist" is. (I do think finance could? should? have noticed that bill, since I in part noticed it b/c the monthly total was too high. But I might have been looking at a more broken-out number than the actual, final total there were some other things that might have gotten mixed in that would have dulled the signal enough to make it plausibly missable.

But after seeing the number, it was a pretty quick task to break it out into various areas over time (like total VM$/time, total storage$/time), find the category that grew, and repeat until the offender was found.

The script thing was mostly about time. Bash's output was only so intelligible (it was basically |grep|awk|grep sort of thing) and engineers would get confused, ask questions, and it soaked up enough time to make me say "let's rewrite this so I never have to answer these questions again".