[PunchyHamster]

S3 isn't "simple" tho.

It doesn't need to care about POSIX mess but there is whole swathes of features many implementations miss or are incomplete, both on frontend side (serving files with right headers, or with right authentication) and backend (user/policy management, legal hold, versioning etc.)

It gets even more messy when migrating, for example migrating your backups to garagefs will lose you versioning, which means that if your S3 secret used to write backups gets compromised, your backups are gone vs on implementation that supports versioning you can just rollback.

Similarly with password, some will give you secret and login but won't allow setting your own so you'd have to re-key every device using it, some will allow import, but only in certain format so you can restore from backup, bot not migrate from other software.

[patates]

S3 needs a split:

QS3 (Quite Simple Storage Service) for the barebones. Bucket/Object CRUD. Maybe: Multipart Uploads. Presigned URLs.

S3 for Object Tagging, Access Control Lists, etc.

S3E (enterprise? extended? elaborate?) for Object Lock & Retention (WORM compliance, Legal Holds), Event Notifications and so on.

[iExalt]

Proposal to rename the services:

* S4: Stupid/Silly Simple Storage Service

* S3: Simple Storage Service

* S2: Storage Service

[bluefirebrand]

You can't have the numbers go down like this. Try pitching S2 to your project manager. You will be told 100% "Why not use S4 instead, that sounds better"

[convolvatron]

I just spent some time with the s3 protocol and I agree completely. What should have been able to leverage the simplifying assumptions turned into another hodgepodge. It’s not like nfs is a real shining example of simplicity either. I’ve never worked with p9, but potentially that aside I think we really failed to come up with a decent distributed file model,

[bombcar]

If all the "popular" solutions are complex, it means the problem domain is complex.

You either are doomed to reimplement and rediscover the complexity on your own, or you change your requirements to fit a narrower problem domain to avoid (some of) the complexity.

[convolvatron]

I'm working on a distributed S3 cache that supports just two functions, pread style contents of a file and all the elements in a directory. I've worked on other systems that represents this entire structure as RDF triples, which leaves you with just query and insert. To come at it from another direction EFS only implements about 2/3 of NFSv4, and S3 was perfectly functional before they larded it up with all this policy stuff.

I'm not saying that there weren't reasons to add these functions to the protocol, but if your aim is minimalism, then you can do _much_ better, and I think there is a real benefit in having a bare bones protocol that anyone could implement in many contexts.

[jen20]

It was simple(ish) 20 years ago, to be fair.

[thayne]

Simpler than it is now, but the authentication system was never simple. You can't just put a bearer token in the authorization header, you have to follow a complicated algorithm to sign the request. That made some sense 20 years ago when s3 didn't use tls to protect against a mitm that changed the changed the request. It is less valuable now when you use tls.

[crote]

Isn't the whole signed request part a hard requirement for 3rd party access? S3 was originally designed for web use, which means giving not-fully-trusted browsers some access to your storage bucket.

You can't exactly do "this client is allowed to download this one specific file for the next 24 hours (but not use it like their personal CDN and share the link with everyone)" or "this client is allowed to upload a single image file up to 10MB (but not upload a 100GB copy of Friends)" with basic bearer tokens.

[jkrejcha]

Aside from the fact that you can do this statelessly (you could stuff a JWT or moral equivalent in the header as a Bearer token), if you were okay with server-side state, you can have the token be related to a principal or have permissions in of itself.

[eddythompson80]

Of course you can do all that with a basic bearer token. It’s just a signed json object with an expiration

[jen20]

> a complicated algorithm to sign the request.

There are lots of benefits to this over the bearer token approach that many take. Binding the authentication to the specific request is valuable regardless of TLS.

[KaiserPro]

The author is also not really clear on what they are actually needing.

If they just want webfile interface then a webserver with simple auth and webDAV would work more than well enough.

The problem is that they then go onto talk about lots of projects that all have posix interfaces. Which is slap bang into shared filesystem land.

S3 is not a filesystem, and nothing shows that more than when you use it as an object store _for_ a filesystem.

Depending on the access requirements, if you're doing local to local, then NFSv4 is probably more than enough. Unless you care about file locking (unlucky, you're in shit now)