Hacker News new | ask | show | jobs
by gary-kim 2260 days ago
I'm an occasional contributor for Rclone. Rclone holds a special place for me as the first open source project that I got seriously involved in and made me far more comfortable with working in open source.

Rclone is now very prevalent in my infrastructure. Almost all my websites are updated by a CI job that builds the website from a repo and pushes it up to the hosting server. There's an encrypted Rclone config in the repo and the password for it is a really long randomly generated string that gets saved in the CI as a secret. Rclone with Restic is how most of my servers get backed up, Rclone is how I access my Nextcloud and Google Drive, I have a containerized s3 compatible storage system that actually stores it's data on a Rclone remote (I hope `serve s3` gets implemented soon this setup can be simpler [1]), and much more.

I'm using it so much that I'm running up against the Google Drive API limit even though I'm using my own key.

* [1] https://github.com/rclone/rclone/issues/3382

3 comments

You may benefit from rotating service accounts to sidestep the API limits. I haven't implemented this on my own setup yet, but I know that it is scriptable. Implementation differences prevent one overarching tutorial, but I left some breadcrumbs for you below.

https://github.com/l3uddz/cloudplow

https://github.com/Rhilip/AutoRclone

https://hub.docker.com/r/hotio/rclone

Is it better to integrate with the go-api or to write files to a local folder and run rclone on that to upload files?

I'm planning to upload files from my app and let an rclone cronjob occasionally sync between my backends.

I did not know that Restic was built on top of Rclone, but of course that makes a lot of sense. Thanks for contributing to a project that has made my life much easier!
rclone is just one of the backends you can use for restic. personally i find it far more performant to use a local disk for your restic repo (and keep the desired number of snapshots, etc there) and then use rclone to clone that repo to a cloud provider. personally i used backblaze's b2 api as it is generally cheaper.

restic is somewhat sensitive to the latency of the backend it uses.

I do the exact same thing.