Hacker News new | ask | show | jobs
by moviuro 2656 days ago
FWIW, I've been using syncthing [0] for some years now [1] and am very pleased. Even though my data is unavailable on the cloud from any untrusted computer (like e.g. my corporate laptop), it's synced on my "fleet".

I'm not sure that PrivateStorage actually adds anything to the equation?

EDIT> The Tahoe LAFS [2] model is more that you spread your data over multiple providers. NAS at home, several VPS providers, or what have you. It feels like RAID in the network, and it allows very precise setting of redundancy policies.

So syncthing actually only runs on trusted machines, whereas PrivateStorage will be able to run on both trusted (tightly managed) and untrusted machines (like a VPS in the USA).

[0] https://syncthing.net

[1] https://try.popho.be/byeunison.html

[2] https://tahoe-lafs.org/trac/tahoe-lafs

5 comments

How much data do you sync? I'm syncing 60 GB with NextCloud and it annoys me frequently: every time I log in it spends 5 minutes scanning my data, pegging at least one core of CPU and using up a lot of my I/O capacity. And of course at a pretty annoying time, since I almost always want to be actually using my machine during the first 5 minutes after logging in. And I'd really like to be syncing more data. Anyway, wondering if syncthing does better in this respect.
Im using Syncthing as well, and in my Syncthing dir on my laptop there is currently 52 GB of data.

That includes a synchting-share for my automatic backups and my "dropbox" replacement (a simple directory for syncing between phone, and computers).

It works great. I haven't had any issues with it. The current release of syncthing is very stable. Earlier versios were a bit error prone. But they seem to have fleshed out most, if not all bugs i encountered in earlier versions.

I have about 10 GB in multiple folders synced in various combinations between:

- ec2 instance (light burstable, always on)

- desktop (debian)

- laptop (manjaro)

- desktop (windows)

- phone (one-way sync to get photos off the phone)

- tablet (kindle fire)

It has been great. Solid and trustworthy. It picks up changes made before the service was running, handles deletes and renames just fine, and updates are simple. The web-based UI is good.

I like that calmh and the team are not adding lots of features. There are lots of things they could add to make syncthing "better", but they want to make sure syncthing does one thing well.

One nit - the UI shows the "latest change" for each folder. The common understanding of this phrase would be "the file in that folder that most recently changed" but what syncthing actually shows here is "the most recent change that syncthing made to this folder". That means that if I change a file on the current device and syncthing picks it up and replicates it out to the other devices, that change will not be shown as the "latest change". If some other device changes the file and syncthing replicates that change back to the current device, then it will be shown as the "latest change". This is confusing. "latest change" should just show the file that most recently changed for any reason.

I'm syncing around 90 GB between my server, laptop, and LineageOS Pixel phone. I use it to sync my documents, music, passwords, and archived pictures. I also use it to sync photos taken by my phone camera as they are taken.

Setup:

* Camera: 1.8 GB, 243 files

* Documents: 10.8 GB, 4604 files

* Music: 61.5 GB, 25077 files

* Passwords: 660 KB, 726 files

* Pictures: 16.5 GB, 6450 files

The passwords are managed by 'pass' [0], which is viewable on my phone using Password Store [1]. Cold-launching Syncthing takes ~10 seconds on my phone, but it does it automatically on boot and thereafter runs in the background. Battery impact seems to be negligible.

[0]: https://www.passwordstore.org/

[1]: https://f-droid.org/en/packages/com.zeapo.pwdstore/ and https://play.google.com/store/apps/details?id=com.zeapo.pwds...

So far I've been disappointed with sync issues with Spideroak, OneDrive, and Nextcloud.

Now I use Tresorit (which I only became aware of because of... and online ad!?) It doesn't seem to have sync issues for me. Dropbox didn't have issues either, but it wasn't as secure.

I wanted so badly for Nextcloud to work.

Given how rock-solid Syncthing has been, I wonder how hard it would be to bolt encryption onto it so anything that some specific nodes receive is always encrypted.
It has been requested since at least 2014 [0] and it just sounds like it isn't going to ever be a feature of Syncthing.

The ability to have untrusted nodes is the one feature that has kept me using Resilio Sync.

[0] - https://github.com/syncthing/syncthing/issues/109

It wouldn't be only specific nodes, but would something like encfs[1] work?

[1] https://github.com/vgough/encfs

Unfortunately that's harder than just always leaving a Raspberry Pi on at home, especially given that I want to be able to sync files to my phone, where EncFS probably doesn't work at all (or easily).
I'm unfamiliar with syncthing, but could you run two daemons, one that does encrypted sync to e.g. dropbox, and one that does plain sync to your phone and such? Or would the two instances stomp on each other or get into an infinite loop? e.g.:

    plain <-> syncthing <-> phone
      ^
      |
      V
    encfs <-> syncthing <-> dropbox
That might be doable, but then I'd need an always-on trusted computer to do the decryption, and if I have that I don't need the VPS...
Apparently syncthing uses fairly strong crypto in transit, or at least that's what I read recently
That's not what StavrosK is asking for.

Some syncthing nodes could host only the encrypted data, without the keys to decrypt them. This adds the benefit of having some nodes host the data, without being able to access it. Think: VPS, etc. that have very good availability track record, but some doubts about whether your hosting company can spy/might be coerced into spying.

Exactly. If I could be sure that the VPS couldn't read or mess with my files without me knowing, I'd definitely add a SyncThing node on my VPS and have increased availability along with security without any hassle.
I think it has been considered:

https://github.com/syncthing/syncthing/issues/109

https://www.bountysource.com/issues/1474343-support-for-file...

https://forum.syncthing.net/t/encryption-for-remote-syncthin...

Unfortunately there doesn't seem to have been much movement towards making it a feature.

A big thing is that Tahoe LAFS can be run on untrusted computers.
Is Tahoe LAFS a viable solution for people without sysadmin skills?
Tahoe-LAFS by itself is probably not (you do have to configure and keep some Python-based daemon software running), but PrivateStorage is a managed service.
So you share your data on untrusted machines that you bet will live long enough to hold what you want to keep.

Sounds risky.

Do storage providers have an incentive to provide the service reliably à la filecoin? [0]

[0] https://filecoin.io/

I trust S3, B2 Google's blobstore more than some rando's machine who runs filecoin. Tahoe-LAFS gives you the assurance that the actual backend storage only sees encrypted data. The big clouds have this advantage that they are probably more reliable, faster, have lower latency, better uptime, and lower price.
The Tahoe LAFS model is more that you spread your data over multiple providers. NAS at home, several VPS providers, or what have you. It feels like RAID in the network, and it allows very precise setting of redundancy policies.
In the PrivateStorage case the machines aren't "trusted" but they are all run by the service you're paying for -- so the incentive to keep them running properly is indeed there.

For other kinds of Tahoe deployments, no there's nothing built-in to incentivize storage-server operators. That part is up to whomever is organizing and running the Grid (what Tahoe calls a group of storage-servers). For example, friends could agree to host storage-servers for each other and create redundancy + trust that way.

The difference between Tahoe and things like Storj / FileCoin is that those services intend to be "a single, global service" whereas Tahoe is software that can be deployed in several different ways -- one of which is a professionally managed Grid such as PrivateStorage.

If you are interested in these topics I'd encourage you to join #tahoe-lafs on Freenode or one of the Tahoe development meetings. These are definitely things I've seen discussed but I think Tahoe-LAFS is far more likely to introduce a concept of "federated Grids" rather than "a single global Tahoe service".

https://docs.blockstack.org/storage/overview.html. While we are rolling out AMIs that auto launch gaia hubs, and hosting some to make it user firendly for people with non sys admin skills, you can docker-compose up here: https://github.com/blockstack/gaia/blob/master/hub/README.md on a machine you own or trust.

We are working to make doing even this as user friendly as possible.

Untrusted doesn't mean randomly selected.
> Do storage providers have an incentive to provide the service reliably à la filecoin?

Yes, via a currency called Dollars.

You can configure Tahoe-LAFS to store data wherever you want but I guess PrivateStorage will have its own settings and you won't be able to select a nas at home. Just a guess though.
An expert could figure out how to get the PrivateStorage Tahoe client to use other storage servers, but yes in general it is "a managed service" and I don't think using your own storage-servers will be "a supported configuration".
You can use your own server to host a gaia hub with docker compuse up: https://docs.blockstack.org/storage/overview.html , or use a one of the cloud AMIs we are rolling out here: https://docs.blockstack.org/storage/overview.html

Making user owned storage user friendly for people with non sys admin skills is a challenge, but something we are working towards.

because our gaia hubs are associated with user's ids, we are also working on automating SSL as much as possible for individual users as well. This is another technical challenge that makes it difficult for the average person to set up their own trusted environment where they control their own data.

At this point you'd just go at the source and use Tahoe directly
Sure, yes you could do that -- I mean, PrivateStorage is just shipping you a "real actual Tahoe client". The main feature you're getting is the managed storage-servers.

So if you happened to "not completely trust" the availability of those you could also configure one of your own and configure your client(s) to use that and the PrivateStorage servers. That is, hedging against PrivateStorage going away so suddenly you can't retrieve your data.

But, I agree: if you're doing that you're likely able to run your own Tahoe grid on VPSes or similar.

How do you link to a single file you want to share ?