[crest]

IP over Ethernet over VXLAN over UDP over IP over WireGuard over UDP over IP over Ethernet… sigh

OpenBSD does support both routing domains and multiple routing tables and includes multiple routing daemons in the base system. I would recommend to the author to stop hacking at the keyboard, grab whatever not to structured visualisation tool works for them (e.g. a whiteboard, a block of paper, a random drawing app, Visio) and (re-)phrase the problem. Are you solving a problem or showing of how many acronyms you can expand without looking them up? This n layer encapsulation can work and can even be required to reproduce some (problematic) organisational structure, but it's far from elegant. Given the chance I would vastly prefer to just use multiple routing domains for the WireGuard tunnel interfaces and the underlay. It would result in far less complexity to manage as well as less overhead.

Why do so many people insist on tunneling Ethernet over IP? What's keeping operators from using IP routing (and just one layer of encapsulation) instead? Is IP routing so scary or everyone that indispensable applications that only work over Ethernet?

[jeroenhd]

Just the grateful that nobody has tried to wrap the entire thing in JSON over HTTP yet! I wouldn't be surprised if we get Wireguard over websockets for "enterprise" applications soon.

Sometimes you just need an L2 tunnel. Most of the time you don't, but when you do, you do. For example, if you use IPv6 over SLAAC in a private network, you'll need to route NDP.

In the rare cases that you do need an L2 tunnel between two different locations, you probably want some kind of authorisation and authentication of the traffic to prevent injection/spoofing attacks and to make life just a but harder for the NSA (Google's use of HTTP was one way the NSA managed to tap connections that were otherwise encrypted by HTTPS). After all, this isn't just any traffic, these are internal subnets.

In terms of authorised traffic, Wireguard is quite lightweight and foolproof. Perhaps IPSec is even more lightweight but it's a pain to set up. The alternative would be to wrap all internal network traffic in an encrypted protocol and set up the necessary whitelists in the upstream ISPs.

The impact of such layering depends on the network connection between the data centers. If you can get jumbo packets across, fragmentation won't be a problem at all. If you run your own fiber between data centers, there's basically no downside until you're reaching very high saturation network saturation.

[lstodd]

Because stuff that requires this circus of encapsulation is usually so brain-dead that it can't be gotten to work in any other less horrible way.

(also I think you lost one 'over UDP')

[crest]

You're right. I forgot that WireGuard sits on top of UDP.

[systems_glitch]

I end up having to run basically this very setup (on OpenBSD, too) because I have a customer who has a Novell NetWare 5 setup and runs IPX only. Bad times.

[gerdesj]

NetWare 5 can do IP fine.

I used to run a cluster of DNS/DHCP servers that were the first on site to run 5. The rest were 4.11 until we binned them for 6. Three cream coloured Compaq 3U lumps.

[systems_glitch]

Right, but they weren't running IP and refused to do it. I did set up a /30 so the poor old thing could synchronize its clock via NTP, but that was the only IP it talked.

This one's running (present tense) on a 1 GHz Socket 370 Pentium 3. It's got some weirdness about Pentium 4 and newer CPUs. I think it can be patched up but the client doesn't want to pay for the work because "it's fine like it is." Not worth the headache to virtualize as the first attempt didn't work.

[wkat4242]

People still use Novell NetWare?? Wow

[systems_glitch]

Unfortunately.

[wyager]

There are a ton of protocols that don't work using cross-subnet IP routing, e.g. anything that uses multicast.

[lillecarl]

Multicast works across subnets with PIM

[aroulin]

Using vxlan you can also connect L3 networks, not just L2 networks. i.e., virtualize an L3 network

[jacob019]

Wireguard virtualizes L3 out of the box.

[aroulin]

With Wireguard being a point-to-point protocol (as I understand), it will be challenging to get good performance for L3VPN BUM traffic?

[tssva]

A Wireguard interface is point-to-multipoint non broadcast which if a single peer is configured on it can in general be treated as point-to-point.

[lillecarl]

You wanna do PIM? There's no BUM on a p2p link.

[aroulin]

Right, that's what I was trying to understand.

"wireshark can virtualize an L3 network out-of-the-box" How can this be true then?

[lillecarl]

Well an L3 network is routed, so you just have routes. 32 bit route for the "server", whatever mask to reach others through the "server". The server has 32 bit routes to each client, so it works.

This is essentially how you want to build your DC networks today too (why waste ips on netid and broadcast that you never use) (for public addresses).

But you stick BGP on there to exchange route information rather than static routing.

[supertrope]

vMotion needs L2 adjacency to make live migrating VMs easy. Some software rely heavily on broadcast discovery messages and are thus designed for LAN usage not Internet connectivity but businesses try to stuff a square peg into a round hold.

[justsomehnguy]

I have a vHost which is sitting on a public IP in another country, while the rest of vSphere is here in RFC1918.

vMotion, Provisioning and backups work just fine.

[lillecarl]

vMotion doesn't need L2 at all, this is a flat out lie.