Agreed, I doubt Amazon had any evil intentions with this change.
I wonder more if this was a product of Amazon developers using S3 (i.e. dogfooding) and not noticing the cost side effect because I'm assuming they don't get billed?
The change is the opposite of what you're implying, it adds the option to turn it off. Before this commit it was on by default. My guess is, when the option was added, it defaulted to validate=True to maintain backward compatibility.
You may want to check LIST request statistics over the next few weeks. Between this thread, an Issue for boto, etc. I'm curious if you see a noticeable decline in LIST requests with the attention this has brought. I'm just curious from a data standpoint.
> interestingly enough, here's the original commit that defaulted `validate=True`:
It's a commit which specifically added validate to allow skipping validation. If you read the diff, the call originally unconditionally performed the validation call.
It is defaulted to true so that old code that called get_bucket() won't break. (Since old code would have used get_bucket with only one parameter.)
It was always true before that commit.
Therefore, in essence, an Amazon employee made it possible to save 90% of your S3 bills.