[CyberRabbi]

I love wireguard but my one gripe with it is that its not a true data link layer. You have to give it routing information through an out of band mechanism “AllowedIPs.” One downside is that you can’t have two peers that act as general routers on the same wireguard network. With Ethernet, you can have multiple nodes in the same subnet acting as generic routers, it’s just a matter of sending IP packets to that host.

I hope someday wireguard addresses this issue and makes itself fully transparent as a data link layer.

[3np]

> You have to give it routing information through an out of band mechanism “AllowedIPs.” One downside is that you can’t have two peers that act as general routers on the same wireguard network

This is a common misconception, due to that this is the way wg-quick works (unfortunately IMO; presumably to make it easier, and I guess wg-quick was never meant for people with advanced needs). On a lower level, AllowedIPs is really just "allowed IPs", and does no routing. You can have multiple active peers with overlapping AllowedIPs.

If you set up the tunnel through other means, you can make your own routes.

For example in systemd-networkd, see `RouteTable` under the `[WireguardPeer]` section of systemd.netdev(5).

(This was unfortunately broken for a brief while in systemd in Jan, but should now be fixed again: https://github.com/systemd/systemd/pull/22136. If it's not clear from the link, old and current behavior are that no routes are added unless RouteTable is explicitly set)

You should also be able to set it up manually and then add routes, policies and rules manually however you would otherwise.

(You're of course right on the protocol layer, but that is not the cause of the problem you want to solve)

[rfoo]

Are you sure what you described works? Last I tried it didn't work, and this is explicitly stated on wireguard.com:

> In other words, when sending packets, the list of allowed IPs behaves as a sort of routing table, and when receiving packets, the list of allowed IPs behaves as a sort of access control list.

> This is what we call a Cryptokey Routing Table: the simple association of public keys and allowed IPs.

Which is, I believe, also why zx2c4 called to revert the whole systemd-networkd feature.

I would really want it to work as it would simplify my network configuration by a bit. Please share a working example if you are able to make it work, thanks!

Last I tried that I used iproute 2 to manually setup interfaces, use wg setconf to load WireGuard configurations. So I don't think it's my tool to blame.

[3np]

> Are you sure what you described works? Last I tried it didn't work, and this is explicitly stated on wireguard.com:

If we're talking simply about decoupling routing from AllowedIPs, yes, using that right now and set it up several times. For redundant routers, see below.

> > In other words, when sending packets, the list of allowed IPs behaves as a sort of routing table

This does seem in conflict with my understanding... Depending on exactly what devil-details go into that "sort of", of course. Not deep enough into it to tell you, though.

> Which is, I believe, also why zx2c4 called to revert the whole systemd-networkd feature.

Rather the opposite AIUI; To allow for setting routes explicitly (which introducing the wg-quick behavior broke[0]).

What started making it click for me was this ArchWiki section[1]. The discussion under this GH issue may also provide some pointers[2]. Also [3]. IIRC I did get multiple outbound redundant routers with failover in the end. There may be WG-specific gremlins I glanced over but ascribed it to not fully grokking the Linux IP stack and issues with *tables in general at the time - the goal I had is hairy enough without Wireguard in the mix. Please report back here on your progress if you have the time :)

[0]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1003955

[1]: https://wiki.archlinux.org/title/WireGuard#systemd-networkd:...

[2]: https://github.com/systemd/systemd/issues/14176

[3]: https://www.eisfunke.com/article/docker-wireguard-systemd.ht...

-----

EDIT: Went back to take a look and I never did get proper HA routing sorted - ended up "solving" it with a script regularly checking reachability and bringing routes up/down accordingly. No need to bring the actual WG interfaces or IP assignments up/down for that, though.

[riobard]

I'm not sure how you conclude that AllowedIPs does no routing.

> On a lower level, AllowedIPs is really just "allowed IPs", and does no routing.

This is contrary to what the official documentation says https://www.wireguard.com/#cryptokey-routing

> You can have multiple active peers with overlapping AllowedIPs.

You can, but the most specific CIDR wins route selection, which is exactly what *routing* does.

[3np]

Key words from that link emphasized:

> Cryptokey Routing, which works by associating public keys with a list of tunnel IP addresses that are allowed inside the tunnel.

> In the server configuration, when the network interface wants to send a packet to a peer (a client), it looks at that packet's destination IP and compares it to each peer's list of allowed IPs to see which peer to send it to.

> the list of allowed IPs behaves as a sort of routing table

> This is what we call a Cryptokey Routing Table

You can just set your peers on separate wg interfaces. At least on Linux and BSD, you have tables to control routing before packets reach the interface.

So you can have two wg interfaces, each with a single but distinct peer both with CIDR 0.0.0.0/0 (or what have you), and use ip-route/nftables as usual to pick the appropriate outgoing interface.

It makes sense if you think of each wg interface as a NIC connected to an L3 switch, and each peer to a host connected to another port on the same switch. AllowedIPs would correspond to the table+ACL in the switch.

But yeah, me saying it "does no routing" was not really correct. But that routing happens after that of the (rest of the) Linux kernel, not overlapping with, replacing, or conflicting with.

While this understanding does come from a decent amount of experience, in case I'm wildly misrepresenting things, do set it straight.

[CyberRabbi]

Only one peer is allowed to use 0.0.0.0/0 for AllowedIPs

[3np]

Not at all. See the links in the thread. You can try it yourself to verify if you do not believe me (and again, wg-quick does extra things here which are not inherent to Wireguard). And also again, you can specify as many wireguard interfaces as you want - no need to cram all peers into one interface.

[CyberRabbi]

> And also again, you can specify as many wireguard interfaces as you want

I don’t have to do this with a normal data link layer, that’s the entirety of the complaint.

[mercora]

i just wish i could control the routing via routing tables instead, making dynamic routing decisions possible without specialized software that is able to manipulate it.

[3np]

You can. You just need to use the right tools. See my uncle comment.

[mercora]

just so you know, your assumption i am not using the right tools feels almost insulting to me considering i made no claim about any tooling used. i am using systemd-networkd to setup networking anywhere, i never touch wg-quick because it is no fit for my use cases. i have multiple routing tables and do policy routing and i would really like to have the "via" in the routing tables to have a meaning to wireguards crypto routing thing. i.e. i want to be able to set "AllowedIPs" based upon the routing table very similar to reverse path filtering. i know i can setup multiple interfaces with multiple keys to exchange and multiple ports to set and to make sure every client that needs to is kept in sync.... but it would be much nicer if i could handle it like an ip-ip tunnel and make routing decisions with software build for this purpose.

[octoberfranklin]

Wireguard is not a link layer (layer 2) tunnel; it is a network (layer 3) tunnel. It operates at the IP layer. You cannot use Wireguard with any non-IP layer3.

AllowedIPs can be disabled if you want; just set it to 0.0.0.0/0. AllowedIPs is needed because netfilter can't "see" which public key an inbound packet is coming from, so by the time a packet gets to netfilter it's too late to accept/reject based on which peer sent it to us.

[_dh54]

> Wireguard is not a link layer (layer 2)

Yes but UI wise it presents itself as one, since it’s acts as an interface. The fact that it is not a true data link layer is the basis of my comment.

> AllowedIPs can be disabled if you want; just set it to 0.0.0.0/0.

Only one peer is allowed to use 0.0.0.0/0 for AllowedIPs

[octoberfranklin]

> Only one peer is allowed to use 0.0.0.0/0 for AllowedIPs

This is simply incorrect. You can have two peers with the same AllowedIP; you just have to put them on separate interfaces (wg0 and wg1 for example). This is for exactly the same reason that a routing table can only have one default entry. If you want two default entries, make two routing tables.

> Yes but UI wise it presents itself as one

No, it doesn't present itself as one.

> since it’s acts as an interface

So does /dev/net/tun, which is definitely not a layer 2 interface either.

[_dh54]

> you just have to put them on separate interfaces

I don’t have to do this with normal data link layers. That’s the point of the complaint. Wireguard is not a true data link layer. Manually configuring multiple interfaces for something I can do with just one interface with a normal data link layer at runtime is an extra inconvenience.

> This is for exactly the same reason that a routing table can only have one default entry. If you want two default entries, make two routing tables.

Using nftables I can specify different routers to use based on arbitrarily complex packet rules. Using just one interface. I can’t do this with wireguard, it will only allow me to to route arbitrary packets to a single peer on an interface. This is an inconvenience.

[zekica]

This is less important although useful in some cases. The more important thing is outbound - since Wireguard is layer 3, in the routing table you can't specify a peer to send the packet to like you can with ethernet, and it has to choose one using AllowedIPs.

[3np]

Why not just split the peers up over separate wg interfaces?

[nikita2206]

Sounds like you’re either already a Zerotier user or would be happy to discover their product.