They could make it configurable on a per bucket basis (perhaps defaulting to the old behaviour if necessary; ideally you would make the conformant behaviour the default, of course).
That way you could opt in to the standard conformant behaviour if you require it, but they can still keep backward compatibility.
I'm not familiar with how S3 works in detail, but I imagine this could require additional API calls in the backend which increases the latency and resource usage of API requests. In the worst case, such a change could easily require Amazon to purchase dozens, if not hundreds of additional servers.
That would require synchronisation, potentially between multiple servers. Doing this efficiently, without race conditions could be very tricky at their size.
That way you could opt in to the standard conformant behaviour if you require it, but they can still keep backward compatibility.