[antongribok]

I run Ceph on some Raspberry Pi 4s. It's super reliable, and with cephadm it's very easy[1] to install and maintain.

My household is already 100% on Linux, so having a native network filesystem that I can just mount from any laptop is very handy.

Works great over Tailscale too, so I don't even have to be at home.

[1] I run a large install of Ceph at work, so "easy" might be a bit relative.

[dcplaya]

What are your speeds? Do you rub ceph FS too?

I'm trying to do similar.

[antongribok]

It's been a while since I've done some benchmarks, but it can definitely do 40MB/s sustained writes, which is very good given the single 1GbE links on each node, and 5TB SMR drives.

Latency is hilariously terrible though. It's funny to open a text file over the network in vi, paste a long blob of text and watch it sync that line by line over the network.

If by "rub" you mean scrub, then yes, although I increased the scrub intervals. There's no need to scrub everything every week.

[dcplaya]

Rub, scrub... Was really supposed to be run but it got the answer I was looking for!

Right now, I'm doing about half of that but I am rcloning data between 2 ceph clusters. (Proxmox and a arm rook ceph cluster).