[geofft]

IPv6 doesn't save you from any routing problems that IPv4 won't save you from. While IPv6 tries to hide the layer 2/layer 3 distinction from you, it doesn't actually make your physical network magically work differently. Internally IPv6 tries to implement this hiding using multicast - same as the VXLAN suggestion in the article. If you overload your network infrastructure's multicast support, at best you fall back to broadcast, which is just like reconfiguring your physical network to bridge all your layer 2 segments into one: if that won't work for you in IPv4, it won't work in IPv6. (And at worst, it stops routing correctly.) If you don't have multicast support at all in your network infrastructure, which as the article points out isn't common to have on cloud networks, then IPv6 won't be able to help you. You'll still need fancy routing and tunneling to make things work, whether you address machines with IPv4 and IPv6.

In my experience, IPv4 has the strong advantage of being familiar and well-supported, which means that when (not if) your network infrastructure starts to act up, it's easier to figure out what's going on. IPv6 works great if you have robust, reliable multicast support on all your devices and nothing ever goes wrong.

[tialaramex]

IPv4 numbering sucks. IPv6 lets you stop worrying about that.

In IPv4 you're going to need RFC1918 addresses, and then you're going to have to make sure that _your_ RFC1918 addresses don't conflict with any _other_ RFC1918 addresses that inevitably absolutely everything else is using or else you'll get hard-to-debug confusion. No need in IPv6, you should use globally unique addresses everywhere, there are plenty and you will not run out.

Everybody who has ever used a single byte to store a value they were convinced wouldn't need to be more than a few dozen, and then it blew up because somebody figured 300 ought to fit and it doesn't already knows in their heart that they shouldn't be using IPv4 in 2019.

[geofft]

Oh, yes, IPv6 saves you from worrying about addressing, which is a huge headache in IPv4. I agree with that and IPv4 address conflicts are a personal frustration. IPv6 doesn't save you from "fancy routing" and mostly does not save you from "nat," though. That's what I was responding to.

I'm hesitant to use IPv6 because it is not merely IPv4 + more addresses, it's IPv4 + more addresses + a very clever design that hides the L2 vs. L3 distinction by relying heavily on multicast groups + a replacement for ARP + a replacement for DHCP + etc. etc. etc. I know I shouldn't be using IPv4 in 2019, but I don't have a better option. I'm not excited about clever systems, hiding, the assumption that multicast works reliably, losing the last few decades of monitoring and debugging tools, happy eyeballs, etc., and I'm not willing to subject my users to the resulting outages simply because it'll save me the headache of thinking about numbering.

[api]

Why can't you just do all your v6 routing at l3 and skip l2 and NDP? If you were using WireGuard which is l3 then all you would need is a way to manage the routes. Can you make BGP work without l2? I seem to think you can but I've never tried it.

ZeroTier supports a mode where it emulates NDP for v6 and works without having to do multicast or broadcast at all. It does this by embedding its cryptographic hash vl1 addresses into v6 addresses.

[geofft]

Well, I have hardware routers that know about L2, and I'd like to have them do as much routing as possible. I'm running Quagga to advertise my VXLAN routes to my hardware routers, so packets originated on bare metal can reach my virtualized infrastructure and vice versa. I want them to know that if the machine advertising this particular IPv6 subnet is in the same rack, packets can go there and don't have to go to a dedicated gateway for all my VXLAN traffic.

I could run IPv6 on the inside and IPv4 on the outside, sure. I worry this is going to trigger more edge cases than either running IPv6 the way it was intended or IPv4 the way it was intended.

[jauer]

> If you don't have multicast support at all in your network infrastructure, which as the article points out isn't common to have on cloud networks,

Huh? Are you assuming large flat L2 networks addressed with IPv6?

IPv6 works great at scale, just route everything everywhere, stick with unicast & anycast, and don't roll large L2 domains. Multicast is entirely unnecessary aside from the small amount needed for ND/RA between host and ToR.

And, for operations, a routed IPv6 network without NAT, VXLAN, or VLANs spanned across switches is much easier to troubleshoot and generally has fewer moving parts to fail.

[geofft]

Yes, I am. The sort of network architecture you describe works great, but unless I'm seriously misunderstanding the article (which maybe I am!), all the use cases where you'd want VXLAN or Wireguard on IPv4 are incompatible with such an architecture on IPv6.

I will grant that IPv6 + ULAs + BGP + flat networks is easier to think about than IPv4 + 10.0.i++.0/24 + BGP + flat network because you have basically unlimited ULAs, but "You have to pick a unique 10.0.i++.0 for each machine, and that's annoying" doesn't seem like the primary thing the article is trying to forget. If you can do a hierarchical routed IPv6 network, you can almost certainly do it with IPv4, too.