Hacker News new | ask | show | jobs
by ebingdom 1590 days ago
So if I have a bunch of objects whose names are hashes like 2df6ad6ca44d06566cffde51155e82ad0947c736 that I expect to access randomly, is there any performance benefit to introducing artificial delimiters like 2d/f6/ad6ca44d06566cffde51155e82ad0947c736? I've seen this used in some places.
3 comments

To AWS S3, '/' isn't a delimiter, it's a character that's part of the filename.

So for instance "/foo/bar.txt" and "/foo//bar.txt" are different files in S3, even though they'd be the same file in a filesystem.

This gets pretty fun if you want to mirror a S3 structure on-disk, because the above suddenly causes a collision.

No difference other than readability. And amazon may distribute your application with another prefix anyway, like "2d/f6/ad6c"
I don't know what impact that partitioning pattern has on s3, but it has some obvious benefits if your app needs to revert to write to a normal filesystem instead (like for testing).