Hacker News new | ask | show | jobs
by turtlecrush1337 2219 days ago
>This was also a problem I had with Cumulus. When I tore apart Cumulus, I figured out that it was less than a dozen unique tools and a distribution rebuilt for 32-bit MIPS and PowerPC.

Almost all of which are open source (or at least "source available"), with the exception of switchd, which cannot be open sourced because it links with proprietary asic sdk's. I don't see how having very few custom tools over a vanilla Linux distribution is a bad thing.

>It was pretty trivial to rebase to standard Fedora or Debian and get a better platform out of it.

If you enable upstream Debian apt sources in your sources.list then it effectively is standard Debian - plus switchd.

Of course it is entirely possible to take all of the components of Cumulus Linux and use them on a separate operating system - enter sonic, vyos, etc - so if you build out such a system which can also drive ASICs and that you prefer over Cumulus, you can take full advantage of all of Cumulus's open source contributions.

>What I would love to see is all these people who keep doing this crap working in the actual Linux distribution communities to build and integrate with upstream projects so that everyone downstream gets all kinds of flexibility

If I read you correctly, Cumulus works upstream as much as it can:

   ~/linux$ git log --author "cumulusnetworks.com" --oneline | wc -l
   773

   ~/ifupdown2$ git log --author "cumulusnetworks.com" --oneline | wc -l
   1265

   ~/frr$ git log --author "cumulusnetworks.com" --oneline | wc -l
   8107
I like to believe Cumulus is quite active in the communities of projects it uses. I feel I may have misunderstood your point, though.

>If we treated the network gear like weaker servers, instead of specialty equipment, there's so many more interesting things you can do!

I completely agree, that's the dream!

Disclaimer: I work at Cumulus

1 comments

> Almost all of which are open source (or at least "source available"), with the exception of switchd, which cannot be open sourced because it links with proprietary asic sdk's. I don't see how having very few custom tools over a vanilla Linux distribution is a bad thing.

The problem historically with Cumulus on this was that it was heavily obfuscated. In the past, when I talked to Cumulus sales folks, it was not quite as honest as what you've said.

I don't have a problem with the "shipping a Linux distribution you can support" thing. I have a problem with "not making it so the stuff you have is available everywhere (i.e. push into Fedora _and_ Debian to feed into all distros and ecosystems)".

> If I read you correctly, Cumulus works upstream as much as it can. I like to believe Cumulus is quite active in the communities of projects it uses. I feel I may have misunderstood your point, though.

Cumulus is actually a nice exception to this rule. Most Linux-based network operating systems do not bother (including SONiC, VyOS, EOS, etc), but Cumulus does good work here. My only complaint is the focus on ifupdown2 instead of helping make cross-distro tools like NetworkManager support these things. It's been a long time since NetworkManager was only for desktop-only use-cases and only did Wi-Fi. It's the standard tool on a wide range of distributions and supports server use-cases very well. I personally use it over ifupdown and netconfig on my systems.

>I have a problem with "not making it so the stuff you have is available everywhere (i.e. push into Fedora _and_ Debian to feed into all distros and ecosystems)".

Almost all of our kernel patches are in mainline Linux, and ifupdown2 and FRR are packaged on Fedora and others.

>Cumulus is actually a nice exception to this rule. Most Linux-based network operating systems do not bother (including SONiC, VyOS, EOS, etc)

In defense of VyOS, they contribute to FRR and generously offer free licenses for people who work on the projects they use (https://www.vyos.io/open-source-contributors/). I think in general there's a lot of goodwill between the people working in the open NOS space.

> My only complaint is the focus on ifupdown2 instead of helping make cross-distro tools like NetworkManager support these things

Gotcha, I understand now. I can't provide any direct insight into why ifupdown2 was chosen instead of nm. I also use nm on my personal devices - though I can't say I've ever missed the ability to e.g. configure vxlan tunnels on my personal infra ;). I guess if we'd chosen nm 10 years ago then there would be similar feelings from people who prefer /etc/network/interfaces. Of course, at the end of the day Cumulus engineering time is spent primarily on things that ship in Cumulus Linux.

Btw, appreciate the feedback :)

> Almost all of our kernel patches are in mainline Linux, and ifupdown2 and FRR are packaged on Fedora and others.

I did see FRR recently make its way into Fedora, but I haven't seen anyone package up ifupdown2 there. Is someone working on that at Cumulus? I'd be happy to do the package review if someone hasn't already grabbed it before me once it's submitted. :)

> In defense of VyOS, they contribute to FRR and generously offer free licenses for people who work on the projects they use (https://www.vyos.io/open-source-contributors/). I think in general there's a lot of goodwill between the people working in the open NOS space.

Oh, I don't doubt it. But it's weird how many of them built on Linux are still not FOSS or collaborating with their upstreams...

> Gotcha, I understand now. I can't provide any direct insight into why ifupdown2 was chosen instead of nm. I also use nm on my personal devices - though I can't say I've ever missed the ability to e.g. configure vxlan tunnels on my personal infra ;). I guess if we'd chosen nm 10 years ago then there would be similar feelings from people who prefer /etc/network/interfaces. Of course, at the end of the day Cumulus engineering time is spent primarily on things that ship in Cumulus Linux.

Well, for what it's worth, /etc/network/interfaces is supported by NetworkManager. :)

As for VXLAN configuration in my personal network, I do it for homelab stuff. Setting up layered networking is kind of necessary if I am going to be messing around with things like OpenStack and Kubernetes.

I totally get that the engineering is primarily spent on things that ship in Cumulus Linux. I just want to see more work from Cumulus that benefits everyone, especially given that networking is so hard to get right! :)

> Btw, appreciate the feedback :)

You're welcome. I'm happy to see such an engaged person from Cumulus like yourself responding well to feedback! :)