[pjdesno]

Because (a) you have to mount a file system, so the user running the app needs permission to do that, and (b) It’s really hard to have a filesystem shared across untrusting admin domains.

With S3 you just do an http request and you’re done.

A lot of folks get hung up on the theoretical equivalence of things, and forget that their favorite solution may be flat out unworkable in practice for reasons that have nothing to do with the theoretical features they’re talking about.

[Aperocky]

This is absolutely correct and the reason why S3 exist and are popular. However have you looked at the case being discussed here? There's a place for object storage and the pattern as discussed might even warrant being a cache or cache cluster.