[shaklee3]

You're comparing apples and oranges. Mellanox has Ethernet switches with 300ns L3 latencies -- far lower than their broadcom counterparts. So it's not an Ethernet limitation, but a broadcom limitation.

[candiodari]

Do keep in mind that having 300ns L3 latencies comes with it's own set of problems. Even at 10Gbit 300ns is not enough to get a packet through an electrical connection. Plus they also have some 90ns latency products.

That means that these switches, while fast, cannot check the packets for correctness (they don't have the full packet). That they will have "aborted" packets. That in some important ways these networks have the problems of the "half-duplex" networks of old.

Broadcom focuses on features for packet transmission. That means these Mellanox switches are pretty much restricted to situations where you want to have a set of servers on a single network segment and nothing else (not even an upstream connection). If that's exactly what you need, great. But mostly you're going to need more.

[wmf]

If you have so many CRC errors that cut-through bothers you, you might want to investigate why your cabling is so damaged.

Your information may be old; Mellanox has pretty much the same feature set as Broadcom now.

[virtuallynathan]

With Mellanox ethernet switches, you lose some other features in exchange for lower latency. i.e. you can only break out 16 ports to 4x25G on a 100G switch (64 25G ports).

[OldManAndTheCpp]

And you also get some benefits: enough internal bandwidth to allow every input port to cut through at line rate, and a single buffer to prevent starvation (the tomahawk chip has 4 groups of ports, each with a separate buffer).

Also, you can run cumulus on the switch, which is pretty awesome.