[ralala]

Interesting read. I have just setup a very similar cluster this week: 3 node bare metal cluster in a 10G mesh network. Decided for Debian, RKE2, Calico and Longhorn. Encryption is done using LUKS FDE. For Load Balancing I am using the HCloud Load Balancer (in TCP mode). At first I had some problems with the mesh network as the CNI would only bind to a single interface. Finally solved it using a bridge, veth and isolated ports.

[fireflash38]

Using containerd I assume? I've been trying to get RKE2 or k3s play nicely with CRI-O and it's been a long exercise in frustration.

[KyleSanderson]

which distro? it should just work out of the box.

[fireflash38]

Initially Ubuntu 20.04, but I upgraded to 22.04. Finally got it working -- turns out a lot of things that reference `--cgroup-driver="systemd"` are doing it as if it were run in shell, which means that the quotes around "systemd" get removed by shell, and would lead to an error & ignored options.

Nothing was showing whatsoever when using 20.04, so I wonder if there were some missing dependencies somewhere there...

I'll probably write up everything I discovered at some point, there's a lot of pieces that you have to cobble together from pretty disparate sources (network plugins, config files (which!?), etc).