Hacker News new | ask | show | jobs
by mfrye0 1748 days ago
> If you want to talk systemic AWS mistakes you can make, we accidentally created an infinite event loop between two Lambdas. Racked up a several-hundred-thousand dollar bill in a couple of hours.

I did more or less the same thing, but with a 3rd party webhook. The bill almost killed my company.

2 comments

> The bill almost killed my company.

You had to pay although it was a mistake?

I was able to get AWS to forgive the majority of the bill. I think that was my one time pass though.
You spent resources. Of course to have to pay.
The resource usage required to tank a small startup (that could’ve become a bigger customer later) is probably peanuts to Amazon. I’m not sure how often they do this (or whether they do it at all) but it would make business sense for them to occasionally grant “billing forgiveness” in serious situations.
Of course Amazon can sponsor those companies hoping that they'll bring more profits in the end. But that's not a guarantee, just a good will, may be depending on mood of support person who'll handle that specific case. I made a mistake with Amazon in the past which costed me $100 and I did not get a refund despite asking for it. I had some bad sour in my mouth, but whatever, my mistake, my money.
There are other comments here today by people saying they could not get forgiveness
If you are able to share the story, what went wrong with the webhook?
It was a 3rd party resource that when updated would call a lambda via a webhook, which would then update the 3rd party resource. So it would create an infinite loop for each resource that was modified.

Dumb mistake...