[mabbo]

I recall being at Amazon some years ago when we were running out of IP addresses internally. A natural answer was "Why don't we all just switch to IPv6?".

The senior principle project manager in charge put it very simply: "The number of routers that don't support IPv6 that we'd need to replace exceeds the world-wide yearly production of IPv6 routers capable of replacing them. At our current rate of growth, we have less than a year until we run out of IPs." (I'm badly quoting a brilliant person many years after the fact, but that's roughly my memory of the talk she gave.)

Major tech companies often have constraints like that which the rest of us wouldn't even imagine.

[jauer]

Facebook's answer was "fine, we'll build our own routers" (in the datacenter, which is where quantity comes in, and is now v6-only (with few exceptions): https://www.internetsociety.org/resources/deploy360/2014/cas...

Major tech companies have constraints, but when they decide to move, they can move almost anything. It would be cool to see how constraints and problem solving approaches differed among the FANG companies as they grappled with these issues.

[lokar]

Same answer at Google

[ju-st]

It's impressive they did that already in 2014!

[kjs3]

Great story; I don't buy it. I haven't purchased a commercial-grade router (for values of more than just 'Cisco' or 'Juniper') that couldn't support IPv6 (admittedly for better and worse levels of 'support') since the early 2000s. If Amazon wanted them, they would have been supplied. Maybe it's "no one can supply routers with the IPv6 support we want (snowflake emoji)" or "we can't deploy the routers fast enough operationally". But not enough v6 capable routers is a stretch.

And even so...this anecdote is/was 'many years after the fact', so what's the issue now? Easy: they don't need it enough to spend the money to operationalize it.

[smoldesu]

I don't know how long ago that was, but I kinda have to call bullshit on her claim (even if it was hyperbole for the sake of making a point). Companies exist to make bespoke solutions for this very purpose, and nowadays outsourcing that kind of work is just natural for Amazon. Hell, they made a deal with Rivian to get a fleet of electric delivery trucks, getting some Chinese manufacturer to slap a Cortex m53 into a shitty plastic enclosure with Ethernet ports can't be that difficult. I bet there are AmazonBasics products that have required more forethought than that.

[mike_d]

A core or edge router in a terabit+ scale network is a far cry from getting someone in China to make you a bunch of Netgear clones.

The Cisco 5500 series chassis is about 21 rack units (or about 3 feet) tall to give you an idea of the scale of these devices in the real world. They are also jam packed with custom ASICs that allow packet switching at extremely high speeds, which would need to be redesigned to handle 16 byte addresses.

[oarsinsync]

> The Cisco 5500 series chassis is ... jam packed with custom ASICs

Assuming you’re talking about the NCS 5500, I thought the whole point of that platform was that it wasn’t using Cisco proprietary ASICS and was just using Broadcom chips with IOS-XR wedged on it anyway to give it the appearance of being a budget competitor to the ASR9K while lacking the majority of the feature set because ... it’s not using Cisco’s proprietary ASICS that support all those features.

Also, they have full ipv6 routing and switching support. So does the C6500, which is ancient (and still in production in many places)

[Dylan16807]

So tell your vendor you expect them to have that redesign in 4 years.

[lazide]

Fat lot of good that will do you with your project that needs to be completed next year or your company is hosed no? And once you do the fix - which won’t require ipv6 or will use a different vendor - then you won’t be talking to them anyway.

The underlying issue is that there is a lot of momentum with v4, and hacks mostly work and work faster and easier, so most people end up going that way. So it keeps the momentum.

And v6 has some serious second system syndrome going on (aka we’ll dramatically fix all the stuff we wanted to do better last time, ignoring real world constraints), and the adoption shows. It was clearly designed to be a either or replacement (aka cleanroom, start fresh with this), but that’s not how real world upgrades tend to work. I keep wanting to use v6, but every time I do, it quickly ends up having to get turned off because of something broken somewhere in some product I have little choice in using or interacting with and v4 (even with tons of terrible NAT) still keeps chugging along.

Momentum is shifting of course - this isn’t a steady state forever thing - but man, ugh.

[Dylan16807]

> Fat lot of good that will do you with your project that needs to be completed next year or your company is hosed no?

You can do both.

> And once you do the fix - which won’t require ipv6 or will use a different vendor - then you won’t be talking to them anyway.

It depends on whether you actually want to fix it.

> The underlying issue is that there is a lot of momentum with v4, and hacks mostly work and work faster and easier, so most people end up going that way. So it keeps the momentum.

Yes, which is very different from being unable to get the equipment you'd need.

[lazide]

That’s nice, but clearly not how the majority of the purchasing departments work right now, correct?

And if most people don’t buy that way - then it isn’t easy to get solid equipment that can do things that way - which makes it hard to get the equipment you need.

[ikiris]

Most real large scale is actually white box clos switch networks.

[mabbo]

It could very well be.

But how many of those routers could they make, and how quickly? And for how much? And could they really handle the kind of load that Amazon needed to handle? And how quickly could these bespoke solutions be installed, tested at scale, verified to work? Would the manufacturer provide support if they don't work as expected?

The solution that was done was to split the network into sub-networks with just the few proxy gateways between them that were needed. And it worked- I think it's still working that way. That's not free to do (every service owner had to do some networking work), but it's also perhaps less expensive than switching out all the hardware, overall.

And rest assured, Amazon always chooses the option that maximizes profit in the long run. Other than that stupid phone.

[pantalaimon]

Which router produced in the last 10 years does not support IPv6?

[mrweasel]

Cisco still produces brand new equipment without IPv6 support. It's not only routers, much of the rest of the networking stack needs IPv6 to be enabled as well.

We mistakenly had the same notion that "why would a new line of Cisco wireless equipment not have IPv6". That pushed back a network upgrade of a remote office a few months. Our mistake really, we should have checked.

Of manufacturers, like Kyocera, have equipment that technically supports IPv6, but no-one really knows who to configure it. IBM have a software product which is IPv6 capable, or it was when they did the initial implementation in 2012. Later versions just sort of shipped with broken IPv6, because IBM doesn't actually test in new releases.