[icedchai]

A lot of IT folks are still fearful of IPv6. I've been on calls where people disable IPv6 as a matter of "best practice." It's sad. People will gladly learn the latest flavor of the month web framework but won't take time to gain experience with a fundamental protocol.

[joshstrange]

I probably fall into this bucket but I’ve seen disabling it fix very odd issue entirely too many times to write this off as just “sad”.

Learning a new framework doesn’t (often) break things for the end user in a way I can’t diagnose/reproduce. IPv6? Totally different ball of wax.

Also my ISP, a fiber gigabit provider and the best available in my region, doesn’t support IPv6. Until they, and others like them, get on board fully I don’t see customers having a snowball’s chance in hell of working cleaning with IPv6. I can only imagine how long it will take the shitty ISPs to fully support it.

[Varriount]

Why? What's the (supposed) fear?

[LinuxBender]

Why? What's the (supposed) fear?

Disclaimer: I have never worked for Amazon but I can add some input from the assorted medium to large companies I have been employed with. I won't mention any names. This is for someone I know reads my comments wink wink

I am not justifying it, just adding some of the bits I experienced. There are many security devices that do not have the same capabilities on IPv6 as IPv4 yet. Some enterprise IoT devices only support IPv4. Adding to this some network engineers don't want to step outside of their comfort zone and tooling/scripts to generate configurations automagically do not yet support IPv6. As the company grows they hire less Sr. Network Engineers and some of the new people depend on but do not understand the automation. e.g. someone wrote some API and they retired or changed companies. Some tooling may remain stagnant for some time. It's also a heavy lift to retrofit some enterprise environments for smaller changes so people fear the outages they will induce implementing IPv6. In some companies it is a major change just to migrate customers to a new load balancer endpoint. There may also be hundreds of undocumented things due to employee churn and lack of change control running that when broken will cause extended outages. And then there is internal politics and finger pointing...

Again, not justifying it, rather I think there are too many moving bits and complexity that people have added over the decades and they are paralyzed by fear and risking the loss of their paycheck. And then there is the embarrassment that comes from having to acknowledge that nobody knows the current state of an environment and that embarrassment can go all the way up the organizational chain.

That is based on my experience of being brought into companies with the speicifc task of, "Hey, make this simpler, reduce outages." It's rarely strictly a technical challenge but rather having to navigate politics, personality types and individual sub-org leaders that have had independent control of their environment for a long time. The more I think about it this could be a topic in and of itself how companies induce self inflicted bloat as they grow.

[talent_deprived]

The biggest issue is IPv6 is a privacy, wide open wild west, there is no privacy on IPv6. Every device's IP is literally public, on the public Internet, 24/7. All so called privacy extensions or improvements do not change the lack of privacy of IPv6 and one more thing, the address structure sucks.

[10000truths]

This is wrong. Every IPv6 interface has a link-local address (which is not routable outside the LAN) as well as a global address. Global addresses are just addresses that come from a block allocated by the IANA. It has no bearing on whether the interface is reachable from the public internet. Just like with IPv4 networks, a stateful firewall will prevent unsolicited inbound connections.

[tremon]

Why is it wrong? You don't seem to be engaging the parent's point at all, which is that IPv6 addresses are a more specific identifier than current IPv4 addresses. The existence of link-local addressing has no bearing on that argument, because those addresses are non-routable by definition. Nor does stateful firewalling prevent your unique device address from being broadcast over the Internet, you'd need 6-to-6 NAT to achieve that.

[KaiserPro]

Every Ipv4 device has a loopback (well most do.) thats not what they are on about.

Having NAT and a firewall gives you a better illusion of privacy. Sure you can track devices from the outside world, but its pretty hard.

If you have v6 configured in a certain way, then your IP address is basically a UUID for your machine. Plus you can't really just stop ICMP anymore so you can trivially ping it (caveats apply)

[LinuxBender]

FWIW this does not have to be true for companies that do not wish to expose internal nodes. I'm not even talking about the privacy extensions. I realize that people beat the drums that one must not NAT IPv6 but it can absolutely be a NAT just like IPv4. I would actually expect in most companies that they don't even add IPv6 inside their datacenters, rather they just put a block of IPv6 addresses on some load balancers at the edge and then point their edge devices L4 and L7 mapping to IPv4 nodes in their datacenter. The same could apply to VPN/WAN configurations in some cases much like how mobile networks are configured nowadays. There would need to be some of this regardless due to ACL's required between companies that don't use network VPN's for restricted end-points. e.g. PCI to PCI environments.

In the early days of IPv4 many big companies did not NAT IPv4. I was at a company that did this. Our workstations all had routable public IPv4 addresses. There are still some companies that do this. One of these was a company that had 20 managers and VP's on a call when I just wanted to give their network engineer a CIDR block and a pre-shared secret for a network VPN. I suspect they will be using public IPv4 addresses internally forever. And I doubt they will ever sell their /8's unless people comment on their Vogon poetry.

Amazon is probably one of the exceptions as they have so many geographically disperse configurations it would be harder to continue using RFC1918 address space. Not impossible, just difficult. I can think of a few other big companies that do manufacturing that are spread out around the world that would probably run into walls with RFC1918 at some point. I've seen some of them take over public address space for internal routing which then breaks access to some things on the internet thus requiring double/triple NATs. 1/8 assorted, 25/8 MoD, 26/8 DISA are a few I've seen.

[SoftTalker]

> In the early days of IPv4 many big companies did not NAT IPv4. I was at a company that did this. Our workstations all had routable public IPv4 addresses.

A lot of big universities did this and even still do this to a large degree. They got huge IPv4 allocations early and there was no scarcity.

[icedchai]

All of the early Internet companies I worked at were like that, up until roughly 2000: public IPs direct on the desktop. My 90's home network was also like that. I had a /24 block from the old class C "swamp" space. I still have it, actually. It's legacy space, no ARIN fees.

[taway1237]

In my current company (related to academic institution that introduced internet to my country) every office workstation has FOUR public (but firewalled of course) IPv4 addresses. And every user has an unique VPN IPv4 address on top of that.

[talent_deprived]

I remember the days of non-NAT IPv4, though I'd forgotten until you mentioned it. I'd be OK with NAT IPv6, though the addresses are still ugly and difficult to reason about.

[p1mrx]

CIDR IP addresses are based on binary number prefixes. If you think it's easier to reason about them in decimal than hex, then you probably don't really understand binary.

[icedchai]

IP-based "privacy" is an illusion. With IPv4, your public IP (NAT router IP address) may not change for months, years, and possibly not until your change your router/MAC address. With IPv6 privacy extensions, your address changes regularly. This seems like an improvement.

[ninkendo]

> With IPv6 privacy extensions, your address changes regularly. This seems like an improvement.

Eh… If I was a company that wanted to use IP addresses to fingerprint users, IPv4 vs IPv6+privacy extensions both seem identical to me. Multiple requests from the same IPv4 address mean “someone, perhaps more than one person, from the same household/wifi”. Whereas multiple IPv6+privacy requests from the same /64 prefix means the same thing.

ie. You just consider the first 64 bits of the IP and can assume the same amount of information you already would assume from the IPv4 address. Just ignore the trailing 64 bits because it’s expected that they’ll be randomized/shuffled even from the same client.

[talent_deprived]

The IPv4 at my router yes. That's where tracking ends. IPv6 privacy is an illusion, try the test I described, remove IPv6 from your router at home, wait a few hours or few days, the family will complain search results are odd or messed up and that's only the beginning of it.

I don't know how companies are doing it but they are able to track your IPv6 changed daily or not.

[icedchai]

They probably just track the IPv6 /64. With prefix delegation, the /64 would rarely change, unless your provider delegated a new block. This is similar to your IPv4 router changing its address w/DHCP: it happens, but is relatively rare.

[mr_mitm]

It's definitely "best practice" to turn off features you don't need. Who knows, maybe in five years someone will find a bad bug in the implementation of the IPv6 stack, then you'll be glad you decreased your attack surface.

Not saying this is what you should do, just a common rationalization.

[icedchai]

It's also "best practice" to learn a new, foundational technology (like IPv6) sooner than later, perhaps less than 20 years after it was first available.

[taway1237]

I know IPv6, just don't feel the need to use it. I prefer to have one firewall and public network interface to worry about. I can't disable IPv4 yet, so the logical solution is to disable IPv6.

[dilyevsky]

Right and what is better way to learn than deploying random features to prod!

[icedchai]

Did I say they had to deploy it to production? If they deployed it in dev and test environments even a few years ago, they might have some experience deploying it in prod today.

[IntelMiner]

It's only been ratified since 1998 and available as far back as Windows NT4

Pretty sure that's enough time to test it in development...

[icedchai]

It's simple fear of the unknown. Many folks haven't learned it, so they just turn it off... Ignore it, kick the can down the road.

[nerdponx]

It's individuals responding to the incentives before them. If something goes wrong because they're using IPv6, it's their fault. If you never upgrade anything until you're forced to, then you can never break anything by upgrading, and you're never seen as a "stuff breaker".

[tormeh]

This is it. Having broken stuff by upgrading myself, I can say it does not make you popular. This is especially the case if what the upgrade did was tighten up some bug that it turns hid a bug in your team's code...

[forty]

I have a reason: we do per IP rate limiting. It's easy enough for IPv4 when the number of IPs is necessarily not too big to fit in a small redis for example, but for IPv6 everyone have at least a /64.

I'm curious how people do it btw, if you have tips to share, I'm all hear. Do you simply rate limit IP ranges? Even limiting per /64, it's still potentially quite a lot of /64 to track.

[tremon]

Given that the only routable IPv6 address space is in the 2002::/16 range (is 2003:: in use yet?), and the standing recommendation for ISP CPE endpoints is to allocate a /48 per customer (a customer can't do any local subnetting if only allocated a /64), the effective address space for rate-limiting is the exact same size as the current IPv4 address space: you only need to track bits 16-47.

It's possible that cloud providers assign smaller ranges to their customers, so you may need to allocate more bits for granularity in that case; on the other hand, one might naively assume that cloud providers are more responsive to abuse reports than ISP's.

[WirelessGigabit]

While the recommendation is to hand out a /48 to each individual customer it's definitely not the standard.

Cox & Spectrum only hand out /56. I'd hate to be banned because my neighbor did something bad and we happen to be in the same /48.

[Dylan16807]

When a bunch of households or cell phones are on the same IPv4 do you have any measures to compensate?

> Do you simply rate limit IP ranges? Even limiting per /64, it's still potentially quite a lot of /64 to track.

Yes you'd limit by /64 or slightly larger.

The live set of IPs shouldn't be very big.

[forty]

We put limits high enough that it's far enough for any expected usage, including a bunch of users on a single IP. If we see rate limiting happening in practice and it doesn't seem to be an attack, we revisit.

[Dylan16807]

Well it sounds like you'd do fine tracking the IPv6 blocks that are currently very active, without needing any significant amount of resources.

If you go the extra mile and simultaneously track /64, /56, and /48 with moderately increasing thresholds, you'll probably end up causing less collateral damage when you block someone than with IPv4.

[forty]

Answering myself: I found this interesting article https://adam-p.ca/blog/2022/02/ipv6-rate-limiting/

[tepmoc]

You treat ipv6 /64 just like /32 in ipv4

[aidenn0]

Mainly that everything is publicly routable.

People see a 10.x and instantly know it can't be reached from the public internet. IPv6 is much harder. For internal-only stuff there is the fd00::/8 block, which AWS actually does use, but there is no equivalent range for outgoing-only connections.

[mschuster91]

Because there's a lot of shit that still doesn't work well with IPv4. Logging is one good example - a lot of software that uses its database for event logs has the database column for remote_ip defined as VARCHAR(15), you can guess the rest of what happens when deploying that with IPv6 enabled.

[icedchai]

These things take time. I mean, it's only been 25+ years since the first IPv6 RFC was released...

[baby_souffle]

It’s a big scary number and the letters just break understanding.

There are some annoying operational issues around it as well, common one: DNS hostnames for devices that only do SLAAC

[WirelessGigabit]

You wouldn't want to do SLAAC and hostnames.

If your network card breaks you switch it out, and you make sure your IPv4 settings apply to the new card.

If you fully rely on IPv6 you'll get a new address.

And if your devices self-update DNS then you have to make sure they pick the right address, as there can be many due to privacy extensions.

Lastly, combining privacy extensions plus hosting stuff is hard, as you don't know on which address a certain port is bound.

[Aeolun]

In the case of my corporate VPN, Teams doesn’t work with IPv6 enabled. Something on either the MS side, or the SSO one.

[vorpalhex]

I was troubleshooting some bad routing in a service. Some endpoints worked fine most of the time but would occasionally lose packets for a bit.

Turns out it's a bug in the kube stack when containers have both ipv4 and ipv6 addresses.

This was last year.

I suspect the ipv6 cases aren't being tested very well across the software stack.

[bradknowles]

Some current Apple products still have problems with IPv6, and the standard practice is to disable it entirely.

I continue to do that on all my personal equipment.