[FridgeSeal]

No but why doesn’t this object-storage-primitive accommodate all my specific requirements already?

They should also accommodate my need for all POSIX filesystem API’s included cheap-moves and renames!!!!!

/s

[sagiba]

POSIX isn't the ask. Datasets are. The need to keep track of what data you have stored is universal, not my specific requirement.

[FridgeSeal]

I make the (glib) comment, because it’s a similar argument to the one that was popular a few years ago.

S3 is an object store. Treat it more like a KV store. As other comments have pointed out, the solution here is pick-your-favourite-metadata-store, be it Postgres, or what iceberg does, and other data on S3.