[hitpointdrew]

> To keep things simple, we used Nomad, and instead of K8s CNIs, we built our own Rust-based TLS-terminating Anycast proxy (and designed a WireGuard/IPv6-based private network system based on eBPF).

That is quite the opposite of “simple”. That is in fact, overly complex and engineered.

[xwowsersx]

How do you know their own Anycast proxy isn't simpler than K8s CNIs? Building something yourself isn't necessarily overly complex or over engineered. Sometimes building a simple thing yourself is the way to simplicity when the only available options already built are very heavy/overkill or complex

[tptacek]

What part of it is overly complex and engineered? Maybe you're right, but it's hard to respond without a better idea of what you think our problem domain was.

[hitpointdrew]

To be fair I don't have insights into your projects. But generally speaking in my experience, anytime there is already some standard most people have adopted, rolling your own solution is usually the wrong solution, and typically over engineered.

[candiddevmike]

Reading the features of your CNI, I don't see why Calico wouldn't have worked for your needs, what made you want to DIY?

[tptacek]

Can you say more about what you think our needs were? I'm not trying to be evasive, I just want to spare you a 9 paragraph response that doesn't address anything you were thinking.

[swozey]

This is all very common platform/infrastructure stuff for any PAAS. Even more-so as multi-tenant k8s (and nics, and nvmeOF, etc) isn't exactly one of the most supported or talked about things. Lots of secret sauce everywhere, but they have to do it in a lot of scenarios.