Kubernetes is "hard" when you're not running on the cloud. The solutions for persistent storage and networking are a mess when you're running on self-hosted infrastructure.
This is a trap. The best way to do storage on K8s is not to do it. Especially on-premises.
The networking used to be simple, you could just set up a few static routes and manually assign blocks of CIDRs to nodes and be done. I'm sure there there are some newer networking API components that obfuscated the whole thing in the name of "simplicity" because nobody understands networking anymore.
Fair enough, I was referring to "using" Kubernetes as opposed to managing your own infrastructure. I can imagine how running it on self-hosted servers outside of a popular cloud provider can be hard.
The networking used to be simple, you could just set up a few static routes and manually assign blocks of CIDRs to nodes and be done. I'm sure there there are some newer networking API components that obfuscated the whole thing in the name of "simplicity" because nobody understands networking anymore.