[systems_glitch]

We recently switched a bunch of stuff from OpenVPN to Wireguard. A number of the links were OpenVPN layer 2 tunnels to pass, of all things, Novell Netware running on IPX (the particular situation precludes switching to TCP/IP for those customers). Now, layer 2 tunneling is being performed using RFC 3378 EtherIP, and it's much more performant, not to mention easier to manage.

Old and new systems are running OpenBSD.

[gerdesj]

Good grief - NetWare with only IPX! Presumably you'll be installing their Y2K patches any day soon 8)

I'm pretty sure NetWare can natively tunnel IPX/SPX over TCP/IP, assuming you are not stuck on 3.12. I don't think you have to fork out for the multi protocol router thing. They will crash regularly until you get the magic combination spot on and then run forever.

A NW 6.5 box will run quite happily as a pretty tiny VM - you could scatter them around as routers to support whatever nightmare of an app you need to run over IPX. Tunnel them over IPSEC or whatever floats your boat.

[systems_glitch]

That particular client is stuck on NetWare 5.1 and due to how bad their maintenance has been I dare not touch the running instances. Virtualization didn't work out as there's some issue with the current patchlevel that dies on Intel CPUs greater than Pentium 4 (including hyperthreading P4s). It's...a stupid story.

[gerdesj]

On VMware you can hide CPU features - ie backrev the CPU and I'm sure HV can do the same, and no doubt KVM/QEMMU too.

I used to run a lot of NW back in the day. I remember deploying a NW 5 cluster of three Compaq boxes with six NICs each (for each VLAN) to do just DHCP/Dynamic DNS! I also ran up four HP boxes a year later with single ATM cards in them with a lot of VLANs to replace a load of 4.11 jobbies. The autoexec.ncf was a masterpiece! The cluster hosts had 6GB RAM each and despite being 32bit had quite a lot of cache which nss absolutely loved. I can't remember when NW 32 bit managed to devote >4Gb to cache but it was handy. As a file server it was absolutely unmatched. Apply an ACL and it simply worked - none of that marking subfolders and files thing that MS and Unix need. NDS/eDir was streets ahead of that weird LDAP n Kerberos thingie that MS "invented" and frankly still is.

I would suggest that you virty them as a matter of urgency. Presumably you have limited and dwindling hardware resources available and at some point something will go pop that can't be fixed. Once you have them as VMs then you can snapshot and all the other lovely things that virty brings to the game and you will never run out of hardware!

[systems_glitch]

Pretty cool to hear about big NW installs! Thanks for sharing!

We're actively working on moving that customer off Netware entirely. The main reason they can't get away from it is they're running a custom god program that manages the entire business, and it's tied in pretty tightly with their old configuration. The whole thing is in Delphi 7.

W.R.T. old hardware, they are actually some of the newer old boxes we support! My main line of business is keeping old industrial control systems online and reliable. On the PC side, the oldest stuff we have in 24/7 operation is 286-based. Pre-PC, we have a few customers running CNC stuff on PDP-11s. I don't know if we still have any PDP-8 customers, I think most of them closed up shop during the pandemic.

[gerdesj]

Good skills.

You beat our DOS 5 booting control system into a cocked hat, that a customer is running. You've got to love cough technical debt. Why on earth is "Enterprise gone awry" considered the right way to go for your IT strategy over that boring old open source bollocks that has a nasty habit of still being supported or at least working decades later? To be fair, open source wasn't a thing in the 80s for most people (nor the 90s ... ).

PDP-11s are my uncle's era and I'm 53! My first real PC was a 80286 (don't forget the 80) I bought a '287 maths co pro for about £115 so I could run a dodgy copy of AutoCAD on it. I do still have my old Commodore 64 which now has a USB interface. I assembled a ZX80 or two ...

Delphi is "just" Pascal - port it!

[systems_glitch]

The bad part is mostly the DB and the Netware integration, but mostly the DB. It's Borland Paradox.

[generalizations]

If you have a write up of how you managed to get layer 2 working inside wireguard, I'd love to read it.

[systems_glitch]

I basically just followed the OpenBSD documentation! One of the big advantages of OpenBSD is that pretty much everything you need to know is contained in the manpages.

As I'd said above, we ended up using RFC 3378 EtherIP to link the two layer 2 broadcast domains across the Wireguard tunnel. OpenBSD supports this with the etherip interface. You end up creating a bridge with the etherip interface and whatever physical Ethernet interfaces you want to bridge, on either side of the Wireguard tunnel.

I also tried VXLAN but did not have good results. I'm not entirely sure it wasn't a problem with my configuration. Traffic often went one-directional, where broadcast packets from Site A made it to Site B, but they did not come from Site B to Site A. EtherIP worked right off, so I didn't investigate further.

[gammajmp]

Use a GRETAP interface; Red Hat's virtual interface documentation is phenomenal:

https://developers.redhat.com/blog/2019/05/17/an-introductio...

[zokier]

The docs are indeed great, but to me it seems like they are recommending GENEVE (RFC 8926):

> Generic Network Virtualization Encapsulation (GENEVE) supports all of the capabilities of VXLAN, NVGRE, and STT and was designed to overcome their perceived limitations. Many believe GENEVE could eventually replace these earlier formats entirely

I'm bit surprised that they didn't have section on vxlan there considering it is pretty popular afaik?

Anyways, I think tunneling GENEVE (or any other Ethernet-over-IP protocol) should work fine over WireGuard, same as using regular network interfaces.

[pm2222]

https://developers.redhat.com/blog/2018/10/22/introduction-t... vxlan and many others are here

[gammajmp]

Yes! I definitely agree. I've used GRETAP for L2 over Wireguard in the past, but it was quite a while ago. GENEVE looks like the way to go these days

[usui]

Since WireGuard is Layer 3, what would is everyone's use case of doing Layer 2 on it? Or, what can it improve over existing solutions? I have tried to do the same for a bit while still learning networking, but ran into Layer 3 limitations.

[simonjgreen]

People who require layer 2 require either a protocol which is neither TCP nor UDP or they need devices in the same broadcast domain

[touisteur]

Can one do multicast over wireguard?

[alyandon]

Frustratingly enough, apparently not as I could never get it to work. It is pretty easy to set up a vxlan tunnel over wireguard if you absolutely need stuff like that though.

[touisteur]

Oh hadn't thought about that, thanks. 'need' is a big word here but sometimes you can't change the client and server apps so, having support for the basic (although niche) features in the lower layers helps migrating smoothly.

[systems_glitch]

Probably the most common use case is letting Avahi/Bonjour/etc. or DHCP work across a tunnel.

[tjohns]

Bonjour is built on top of DNS. You don't need a layer 2 tunnel to make it work.

However, it normally does rely on multicast. Rather than trying to bridge broadcast domains (which is going to cause performance issues), a more efficient option is to setup an Avahi mDNS reflector on either end of the tunnel to rebroadcast mDNS packets.

See, for example: https://www.reddit.com/r/WireGuard/comments/g80bxf/comment/h...

Alternatively, there's also a Wide-Area Bonjour service that works over unicast and doesn't need any special packet forwarding, provided you run a Bonjour-aware DNS server:

http://www.dns-sd.org/serversetup.html

https://help.dyn.com/bonjour-and-dns-discovery/

(More generally, Layer 2 tunnels are best avoided unless you really need them for something arcane, like IPX or NetBIOS.)

[simonjgreen]

You are technically correct (best kind of correct) however, in reality, I see folks using L2 tunnels to solve for bonjour etc all the time. Usually those without networking knowledge to solve the forwarding.

[systems_glitch]

Yeah, you can do it the right way...or you can just tunnel layer 2 and forget about it. I see it done a fair bit for both Bonjour/Avahi and DHCP (why?).

[zokier]

One example usecase would be to try to tunnel something like BOOTP/DHCP/PXE/TFTP stack, which iirc is bit tricky with only L3 tunneling.

[simonjgreen]

Yes, this is common among those who don't understand how DHCP Relay works

[systems_glitch]

Many don't, and I suspect searching for "how to do on VPN" yields "turn on Layer 2 tunneling."

[teleforce]

The best way to perform something like this on Layer 2 is to use Shortest Path Bridging (SPB) based on IEEE 802.1Q-2018. However the Linux kernel does not yet fully supporting this feature natively although the standard has been out for quite sometime and already being supported by commercial network solutions and the popular Open vSwitch (OVS) [1].

[1] Ask HN: Project ideas for a Linux kernel module:

https://news.ycombinator.com/item?id=35785158

[wmf]

SPB uses MAC-in-MAC encapsulation which won't help you run over Wireguard. You need some flavor of Ethernet over IP like GRE, VXLAN, or GENEVE.

[polski-g]

gretap.

But why run Wireguard+gretap when you could just run tap-mode OpenVPN?

[FL410]

Me too.