[d00bianista]

> an attempted deployment of IPv6 caused packet storms in the MIT Computer Science and AI Lab's network

They were relying on Spanning Tree in a who knows how big broadcast domain. Firstly, that's just begging for things to hit the fan. A single device having a meltdown will cause exactly this, a broadcast storm that is able to take down the entire campus, because it was a single broadcast domain.

Secondly, it is a security nightmare. No amount of links or switch capacity will suffice in a single broadcast domain campus, relying on STP, if port isolation and proxy-arp is enabled, along with DHCP snooping, arp inspection etc. So, port isolation is not turned on. Isolation also creates a requirement for a pyramid-shaped network so nobody wants to do that anyway. But back on point, MITM-heaven, anyone can do what ever they want because the L2-infrastructure is not limiting anything. Ethernet does not care about security and Internet Protocol only implements or allows to implement security in gateways that interconnect subnets that reside on separate broadcast domains.

Routing is the answer and this is why I route on the access-layer, as well as on aggregation and core -layers. Route loops are very rare with OSPF and broadcast storms are limited to single switches if you route at access-layer. Also the posible issues with untested code-paths are minimized this way, since none of the switches are seeing more than the equal amount of hosts as it has ports.

I do not agree with SLAAC because I do not believe in broadcast domains the size of a /64 so I'd deploy DHCPv6 in every possible braodcast domain that does not have Android devices in them. Luckily there is no place for Android in wired networks and especially datacenters, so I can happily deploy DHCPv6 in those. And if I ever need to service Android-devices, I can dualstack and let the devices know of DNS-service with DHCPv4! Take that, Lorenzo! Hah! Outsmarted you there!

[geofft]

I mean, sure, CSAIL might have been wrong. (Worth mentioning this is just a single building, not all of MIT.) But if even the MIT computer science network isn't set up "right," there's doubtless a huge amount of technical debt for everyone else who's not an MIT nerd to clean up before they can make IPv6 work. And IPv6 would have been more successful if the designers had taken that into account.

[Dylan16807]

The switches being good at ARP and bad at an alternative isn't down to technical merits, it's down to which one existed first.

And it sounds like turning the intelligent feature off and treating the packets as pure broadcast, just like ARP packets, would have fixed the problem. If the switch can't do that in the right way, it's not the protocol's fault.

[d00bianista]

Yup, turning off snooping would have fixed the issue at hand but not the poor choice of relying on STP for redundancy.

I've seen this happen a few times in my life in production systems, designed by someone else. Overload a switch somehow and it goes straight in ludacris mode because of the topology. Properly configured networks suffer minor outages only in case of single device meltdowns.

[Dylan16807]

It seems like it took multiple switches melting down from bad software design to cause the problems. I originally had a line about the risk of large broadcast domains but the comments on the post claim they were actually pretty small.

[d00bianista]

Large L2-networks never work well, they all have some issues. Their existence is purely based on lazyness or believing that Ethernet works like the water main... I admit that I might be more obsessive compulsive than most but I find reconfiguring networks to be a whole lot of fun :)

And no matter how smart the peole on campus are, there will always be someone who cannot accept that someone else is more right. I have first hand experience of this since I have tunneled IPv6 at home and know pretty quickly if someone does not listen to ICMPv6 Packet Too Big -messages. Not the first time I contact people about it but so far the only one I have not been able to convince is exactly someone really smart on some campus somewhere. I wrote many emails and tried to explain that IPv6 allows MTU's as low as 1280 bytes and that ICMPv6 is a must allow protocol but nope.