[sho]

Wow. This is a cautionary tale. I don't think I'd be as devastated as this poor chap, but as it grew I realize I've allowed my iCloud photo library to become a single copy.

How are people handling this these days? If i wanted to ensure a full backup of everything on my iCloud to a NAS, what's the best way these days? Seems like they make it difficult by design..

[beala]

I self host an Immich [1] instance to backup photos on my iPhone. It’s OSS and has a level of polish I’ve rarely seen in free software. Really, it’s shockingly good. The iOS app whisks my photo off to my home server several times per day.

What I’m not sure about is how to backup things like iMessages, Notes, and my Contacts. Every time I’ve looked, it appears the only options are random GitHub scripts that have reverse engineered the iMessage database.

1. https://immich.app/

[snowe2010]

The imessage db is literally just a sqlite db. If you have a Mac you can read the entire thing with an applescript. It’s really easy from what I remember from years ago

[unsnap_biceps]

I run a nextcloud [1] instance and use it for contacts, calendars, and reminders

1. https://nextcloud.com

[arendtio]

I use Nextcloud for files/contacts/calendar/etc. as well, but for photos I use PhotopPrism [1].

The reason is simple: photos require much more processing and focus on performance. In addition, photos take up much more space, so while my Nextcloud instance runs on an SSD, the photos reside on an HDD, mostly in sleep mode.

[1] https://www.photoprism.app

[css]

What's wrong with `imessage-exporter`?

https://github.com/ReagentX/imessage-exporter

[firecall]

One rather counter intuitive way to “backup” your photos is to install Google Photos and One Drive on your iPhone!

Google and MS don’t charge as much as Apple for storage, and you probably need you need to pay beyond the free limits, but it’s not a huge expense.

Once your installed Google Photos and One Drive on your iPhone, just tell the apps to sync all your photos all the time!

Now I appreciate that isn’t for everyone.

But it works, is reliable, and requires no technical knowledge of running your own service.

The other thing to do is setup a Mac that synchs all your iCloud data, One Drive documents and Google Drive.

Then back up that device with Backblaze.

This gets expensive as a Mac with decent levels of storage isn’t cheap!

I live in fear everyday or my primary Apple and Google accounts getting locked!

I’ve had accounts since day one of iTools and very shortly after Gmail launched….

[raw_anon_1111]

The issue with OneDrive is that it doesn’t store metadata like the photo location, its damn near useless. But I do pay for storage for Google Photos and iCloud.

If you take all of your photos from your phone, you don’t need your Mac at all. Google Photos will sync directly.

I wouldn’t use BackBlaze (the $7 a month service). It doesn’t support NAS at all and it has to phone home every 30 days or it will erase anything that is stored on external drive.

I would use an app that backs up to their B2 service.

I personally just use my personal AWS account to back up my Plex media and just use the AWS s3 sync command using the AWS CLI and store everything in S3 Deep Archive. It’s less than $2 a month for 2TB.

[firecall]

I’m suggesting you do both One Drive and Google!

Also, to be clear, I’m saying to install One Drive and Google Photos on your iPhone.

My approach to this is based around having the least amount of things to manage.

In my scenario, I’m looking for the most simple out of the box systems to backing up that don’t require any self hosted solutions or NAS management and so on.

Just throw money at it!

Hence my suggestion to get a Mac, say a Mac Mini, with a decent sized drive and just sync everything to it.

Fully sync so it downloads the lot and need off loads.

The backup the internal drive with Backblaze.

I appreciate this advice isn’t for everyone, and it may not be the best solution.

But it’s a way to a least have some ultimate disaster recovery in place!

It may not be the cheapest, and cheapest doesn’t mean best. It may not be the best by some other measure of features, but it works and requires zero knowledge and additional hardware.

At least with some kind of backup, everything is not lost!

You don’t have to manage it or think about it. The services will sync for years without intervention until you upgrade your devices. For most people, that’s the important factor in having multi backups of their photos and documents!

[raw_anon_1111]

I think we are saying almost the same thing. I have iCloud storage, pay for one Google storage and have Office365. I use all three.

But, we travel a lot, it’s a hobby of ours. Being able to see on a map where the pictures were taken is important. iCloud Drive and Google Drive preserve all of that information and the accompanying Live Photos, and depth information. One Drive doesn’t.

But I’m okay with a two full Fidelity sources and one low Fidelity backup.

[firecall]

100% with you there!

Having location data is very desirable, and searching my library without it would be painful!

I'm also a huge fan of how iCloud and Google Photos can search my photos just by a description. (not sure if One Drive can, never tried).

I'm horrified when I talk to people who only have their photos on their phones, with no additional iCloud storage. They have no additional backup at all.

[snowe2010]

Backblaze doesn’t erase after 30 days… I’ve had a computer be offline from it for several months and it still retained all data. And you can use the backblaze docker container to run on a NAS, much much much cheaper than B2.

Wasabi is much cheaper than AWS as well.

Finally the best solution for backing up your iCloud Photos is definitely Immich. Set it up on your own NAS or a VPS, back up to that, and then back up that server to an S3 storage using rsync or restic. I’ll note that I still backup to Backblaze because its so dang cheap.

I spent months trying to find the best setup a few months ago and this is by far the cheapest.

But still, this shouldn’t be required for normal people. They should get what they pay for.

[raw_anon_1111]

> It has to phone home every 30 days or it will erase anything that is stored on an external drive

It’s actually more nuanced. It will back up files on a USB attached drive. If it doesn’t see the drive attached for 30 days, it will erase the backup.

If you have your computer off for more than 30 days and you bring your computer back on and the USB drive isn’t attached when it connects to BackBlaze, it will erase it.

Yeah I’m not going to trust my storage to Wasabi.

AWS S3 Glacier Deep Archive is $1 a month.

[snowe2010]

> AWS S3 Glacier Deep Archive is $1 a month.

Only if you’re backing up nothing and using non-encrypted files and making sure you don’t delete anything (rsync with delete turned off). I tested this not even three months ago. I hit $30 with only 3 tb of data with deep archive while wasabi AND backblaze cost less than that. No need to even trust a single provider. If you’re never changing your files AND you don’t care about encrypting them then yes GDA is fine and pretty cheap. Otherwise wasabi and backblaze get more done for less cost.

[raw_anon_1111]

I meant a $1 a month per TB for AWS S3 Glacier Deep Archive That was my bad.

I am definitely a fan of B2.

[DrammBA]

> It’s less than $2 a month for 2TB.

What would be the egress fee to get your data back in case of disaster?

[raw_anon_1111]

The cheapest slowest egress, bulk retrieval is $2.56 per terrabyte.

Glacier is meant for in case of emergency break glass. You would use lifecycle policies on S3 to go from fast/more expensive storage for like the first 90 days and then have it automatically go to Glacier.

Yes I know it’s more complicated and nuanced. I’m purposefully yada yada yada’ing

[jval43]

I run a separate Mac Mini that has the full iCloud Photos library on a massive external drive, set to "Download originals". I then rsync that filesystem to a separate Linux box. This works but you must not ever disconnect the external drive.

I don't have a solution for iCloud Drive, as there wasn't a keep offline setting last time I checked. So use it only ephemerally.

[NaOH]

At least as of Sequoia, the Settings > iCloud > Drive > Optimize Mac Storage option enables iCloud Drive files to be stored offline. Likewise, right clicking any iCloud Drive files in the Finder includes a Keep Downloaded option. Since I minimally use iCloud Drive, in the past (older OSes) I also had Hazel make copies of iCloud Drive files so they were certain to be in backups.

[unsnap_biceps]

Arq [1] has an option to "materialize" dataless files, basically forcing them to be locally available. The only issue is if it's a large file and it gets pushed off device often, you can burn a lot of bandwidth re-downloading it over and over again.

1. https://www.arqbackup.com

[wrxd]

For iCloud Drive have a look at rclone. You can run it straight from your Linux machine

[QuiEgo]

Time Machine backups to a samba share on the Linux box would get you both the Photos library database and the iCloud Drive stuff. It also means you don't need to bother with the external drive.

There is a keep all files offline setting for iCloud Drive (turn off "Optimize Mac Storage" in Systems Settings).

[4jck]

I'm not familiar with the "Photos Library.app", but I have an m4 mini with my photos in a Photo's Library. I'd love to know your script to rsync the photos into a separate drive/directory

[jval43]

The Photos library "file" is just a big folder, I just sync the whole thing.

#!/bin/sh rsync --iconv=utf-8-mac,utf-8 -avh --delete-after --partial --progress /Volumes/myExternalDrive/Photos\ Library.photoslibrary [e-mail]:"/srv/myExternalDriveBackup/"

(note: tested with brew rsync, IIRC the default rsync is outdated on macOS)

Somewhere in the directory structure is a folder /originals/ which has all the actual files.

Note that this is only a last resort backup. Restoring the library as a whole requires a Mac with a compatible OS version. Restoring without a Mac would only get you the originals, so only the out-of-camera files (jpg, heic, raw), with no edits or metadata changes from Apple Photos applied (Apple Photos doesn't touch the EXIF data). You'd probably also lose the video part of all live photos, as the live video files stored as separate files and not part of the .heic files. They're there, but not very usable.

An alternative to this workflow is to export all photos (with edits applied) from the Photos app, but honestly I'm not sure if that even works and how long it would take for multi-TB libraries.

[mhammerc]

I run Arq Backup automatically in the background.

It copy Photos, iCloud files and my mails once every days to S3 with incremental backups.

It requires to have a full copy locally.

Works great!

It is not hard to configure once, with the proper folders and settings.

[sho]

> It requires to have a full copy locally.

yeah that's the thing. When my iPhotos library exceeded 1TB I lost the ability to store the full local copies. Since then, iCloud itself has been the sole source.

Looks like there's some decent, reasonably priced apps to handle this like https://apps.apple.com/us/app/parachute-backup/id6748614170?... (no affiliation)

[leoxiong]

You can request an archive of all your data (including photos and drive) in 25gb chunks.

https://support.apple.com/en-us/102208

[mikepurvis]

I recently rebuilt my home server as an unraid machine. Currently it’s mainly torrents and a Minecraft server but it’s got 10tb of locally redundant storage with a sightline to scale that to around 24tb, so it would be a logical place to store a full gphotos copy.

[mh-]

Thanks, I have the same problem and need to do something about it.

I wonder if it can calculate (estimate) how big of an external disk I'll need. My wife and I each have 40-50k photos and a few thousand videos in iCloud Photos.

[b112]

If you want to truly save your photos, make backups of the locals and put it in your safe deposit box at the bank. Or alternatively, at a trusted friend/relative's house.

Even doing this yearly can save the immense sadness of lost memories. And of course, this works for emails, and everything else.

If you encrypt it, make sure you use a method not tied to any external service, or the machine you're on. I don't use Apple, yet I suspect that an encrypted external backup might be tied to your Apple ID, or some such, because that's how the world flies today.

[mh-]

Yeah, the plan would be external disk -> offsite storage.

I wouldn't bother to encrypt, it's just family photos and I wouldn't want to complicate restores. Especially if it was my wife who eventually needed to use it.

[subscribed]

To anyone who reads it: actually A HARD FISK, not a pendrive/SSD

[beala]

On my iPhone, I can see the size of my iCloud photo backups. Settings -> Apple Account -> iCloud -> Storage.

Weirdly, that number is different than Immich’s estimate of my photo library (95 GB vs 150 GB), but perhaps good enough to get you in the ballpark.

[mh-]

Oh yes, of course. Thank you. 422 GB. Looks like my wife has slightly more.

[Timshel]

10TB external harddrives are relatively affordable.

[yardstick]

I run a Synology NAS with a docker container that periodically downloads new iCloud Photos to a local directory.

[sho]

this? https://github.com/boredazfcuk/docker-icloudpd

seems pretty high touch. A lot of hoop-jumping if you don't have a mac in the middle

[leobg]

Thanks. I had no idea something like that existed.

How do we know using such a tool won’t trigger an account lockout? How ironic would that be.

[yardstick]

No idea if it’ll trigger a lockout, but if it does at least I have a copy of my photos already.

Been running it for a couple years without issue. But yes your milage may vary.

[yardstick]

Yeah that’s the one.

I do have a Mac so it didn’t seem difficult to me, but I accept it will be for those that don’t.

[QuiEgo]

I'd like to give a special shoutout to the PhotoSync app. It has one killer feature that Immich does not (at least last time I looked): encryption at rest. I think someone breaking into my house and stealing my NAS is a real possibility (unlikely, but I'd give it higher odds that getting locked out of my account like what happened in the article), so this is super important to me.

You could put Immich data on a LUKS volume I suppose, but then you have to fiddle with your server every time it reboots.

I did PhotoSync for a while, but now I just set up my Mac to download my whole photos library, and do Time Machine backups of my Mac. This gets two copies of the data not tied to my Apple ID (the one on my Mac's local disk, and the one on my NAS on the time machine volume).

[geekologist]

immich is an extremely polished, FOSS alternative to google/apple photos. It's an investment, but a 4 bay NAS running immich should do nicely. Additionally I backup snapshots to Backblaze B2 via restic which runs another $5/TB

[redrove]

For me personally Immich is a non-starter because its not end-to-end encrypted.

[InsideOutSanta]

It runs on your own hardware. There is nobody else who has access to unencrypted data.

[subscribed]

Storage vps are cheap. Why would I have to run it in my own house?

:)

[snowe2010]

Why would you need it to be end to end encrypted anyway? You’re running it. Set it to only upload photos when you’re on your home network and you’re fine. Or fork it and make a PR and make it e2e encrypted.

[redrove]

You can’t just “fork it and make a PR and make it e2e encrypted”. All the features run serverside, e2ee is fundamentally impossible because of its design, of which you seem to know fuck all.

I’m being dismissed by I run a rather large homelab and I still want my photos iCloud like, where end devices decrypt and run ML. Immich is a Google Photos clone where you give it everything and some server does all the magic.

[throwawayffffas]

Hm, you can just run it on an encrypted volume. And put an ngnix in front of it to handle https. There you go end to end encrypted.

[snowe2010]

What are you talking about. It’s literally open source. Here’s the server code https://github.com/immich-app/immich/tree/main/server You run the server. You can make the entire thing e2e encrypted if you want.

You could even set it up so that it could only backup over tailscale or wireguard through a tunneled connection so ALL of your traffic is e2e.

[stackghost]

I simply manually periodically download everything to disk/software raid. Really important/sentimental stuff like baby photos and videos I have on DVD with par2s.

[JoshTriplett]

> How are people handling this these days?

Syncthing is wonderful, and does a great job of syncing between an Android phone's photos/videos and a laptop. And if you have regular automated backups of the laptop, you'll have backups of the photos/videos too.

For an iPhone, perhaps you could use iTunes to sync to a computer and back up that computer.

[4k93n2]

sushtrain seems like the best option for syncthing at the moment. its a bit more polished than mobius. neither of them sync in the background but i think i remember seeing someone using shortcuts to open the sushitrain app every now and again to wake it up so it would sync

[ycombinete]

Sync to Dropbox -> Dropbox hourly & monthly backups to my NAS using Bvckup2.

(One of these days I’ll setup my NAS to backup offsite fo a #3 backup).

I know that others with Macbooks sync their whole library to their Macbook and then Time Machine to a NAS as their copy #2. Is this vulnerable to the problem in TFA?

[n2h4]

I keep copies of any important stuff i need on my server, and in a few hard drives at my home. i don't use any "cloud".

[iknowstuff]

Back in the iPhoto days I used to symlink the library to an external drive.

[Rikudou]

Not an iCloud user, but I use Immich on my NAS.