[salamander014]

I also use MetalLB and Rook. Can't say enough good things about them.

I have also used Kube-Router (https://kube-router.io - Digital Ocean's non Virtual networking plugin for bare metal environments; it puts your containers on your physical network, which is freaking neat) and loved that, but since I started deploying kubernetes with Rancher I've found for dev clusters I'm not caring about what networking is used. (currently running Canal).

Not sure what we will decide on when we go to production.

[fyfy18]

I've had a lot of performance issues with Rook. I setup a k8s cluster six months ago on a few ThinkCenter Tiny's (more powerful than a Pi). They each have a single 120GB SSD, so I've allocated a directory on the filesystem (ext4) to Rook rather than a dedicated device.

Originally I only had 2GB of RAM and Rook ended up using most of this. Even now the CPU usage by Rook is quite high even when the system is idle. It hasn't lost data though, even though one of the SSDs died and was unrecoverable, so props for that. To be fair I haven't tried upgrading since I installed it so maybe my issues would be resolved by that.

Overall Rook feels somewhat overkill for a homelab user, but I haven't seen any recommended alternatives.

[Nullabillity]

You need to set a memory target for your OSDs, or they will gobble up as much as possible for their cache. In Rook you do that by specifying the resource limits on your CephCluster[0].

Sadly, Rook enforces some pretty high minimum values (2GiB per OSD, 1 GiB per mon), which is fine for production but can make homelab experimentation annoying.

[0]: https://rook.io/docs/rook/v1.1/ceph-cluster-crd.html#cluster...