[fh973]

Only the namespace is hard to scale horizontally because of rename; the rest of POSIX does not have inherent scalability issues. For the vast majority of applications, namespace scalability is not an issue (there are only few single applications that store billions of files/objects), so in most cases it's mostly a system management topic.

If you go full object storage semantics (no in-place updates, to appends, no rename, ..) you're pushing the problem to the application layer. Experience at Google was that that's not a good place to have it, because application developers are usually not good at solving distributed systems problems. This lesson learned is the part of the Bigtable (no transactions) to Megastore (application layer transactions on top of Bigtable) to Spanner (transactions built-in) journey.

So yes, of course there are inherent hard trade-offs, but dropping strong semantics from the storage layer by using an object store instead of a file system is usually not a good idea.