[MontyCarloHall]

How would you implement things like version history or shareable URLs to files without a database?

Another issue would be permissions: if I wanted to restrict access to a file to a subset of users, I’d have to make a group for that subset. Linux supports a maximum of 65536 groups, which could quickly be exhausted for a nontrivial number of users.

[Wicher]

As for the permissions, using ACLs would work better here. Then you don't need a separate group for every grouping.

[MontyCarloHall]

TIL about ACLs! I think that would nicely solve the group permission issue.

[technothrasher]

The final project for my senior year filesystems class thirty years ago was to implement ACLs on top of a SunOS 4 filesystem. That was a fun project.

[johnisgood]

Write up? Code? :D

[thebeardisred]

Then let me also introduce you to extended attributes, aka xattrs. That's how the data for SELinux is stored.

[westurner]

There is no support for writing multiple xattrs in one transaction.

There is no support for writing multiple xattrs and file contents in one transaction.

Journaled filesystems that immediately flush xattrs to the journal do have atomic writes of single xattrs; so you'd need to stuff all data in one xattr value and serialize/deserialize (with e.g JSON, or potentially Arrow IPC with Feather ~mmap'd from xattrs (edit: but getxattr() doesn't support mmap. And xattr storage limits: EXT4: 4K, XFS: 64k, BTRFS: 16K)

Atomicity (database systems) https://en.wikipedia.org/wiki/Atomicity_(database_systems)

[skydhash]

Backup files the way Emacs, Vim,... do it: Consistent scheme for naming the copies. As for sharable URLs, they could be links.

The file system is already a database.

[edweis]

Ok this product will be for project with less than 65k users.

For naming, just name the directory the same way on your file system.

Shareable urls can be a hash of the path with some kind of hmac to prevent scraping.

Yes if you move a file, you can create a symlink to preserve it.

[conception]

Encode paths by algorithm/encryption?

[MontyCarloHall]

This wouldn’t be robust to moving/renaming files. It also would preclude features like having an expiration date for the URL.

[conception]

Well sure there’s a bevy of features you’re missing out on, but it would work. Object store and file metadata solves both of those though feels like cheating.

[edweis]

Use sym link in that case to keep the redirect.

[ajross]

> How would you implement things like version history

Filesystem or LVM snapshots immediately come to mind

> or shareable URLs to files without a database?

Uh... is the path to the file not already an URL? URLs are literally an abstraction of a filesystem hierarchy already.

[QuantumNomad_]

> Filesystem or LVM snapshots immediately come to mind

I use ZFS snapshots and like them a lot for many reasons. But I don’t have any way to quickly see individual versions of a file without having to wade through a lot of snapshots where the file is the same because snapshots are at filesystem level (or more specifically in ZFS, at “dataset” level which is somewhat like a partition).

And also, because I snapshot at set intervals, there might be a version of a file that I wanted to go back to but which I don’t have a snapshot of at that exact moment. So I only have history of what the file was a bit earlier or a bit later than some specific moment.

I used to have snapshots automatically trigger every 2 minutes and snapshot clean up automatically trigger hourly, daily, weekly and monthly. In that setup it was fairly high chance that if I make some mistake with an edit to a file I also had a version of it that kept the edits from right before as long as I discover the mistake right away.

These days I snapshot automatically a couple of times per day and cleanup every few months with a few keystrokes. Mainly because at the moment the files I store on the servers don’t need that fine-grained snapshots.

Anyway, the point is that even if you snapshot frequently it’s not going to be particularly ergonomic to find the version you want. So maybe the “Google Drive” UI would also have to check each revision to see if they were actually modified and only show those that were. And even then it might not be the greatest experience.

[vablings]

If you are on windows with a Samba share hooked up to zfs you can actually use the "previous versions" in file explorer for a given folder and your snapshots will show up :) there are some guides online on setting it up