[kev009]

Since I can elaborate more on here vs twitter, I have a senior engineer spending 100% of his time fixing Intel drivers at my company for coming up on 2 years. The FreeBSD driver is somewhat related to the Linux driver but Intel has obligations to put out a copyfree implementation. I believe this was then ported to Illumos a couple years ago.

ixgbe works relatively well on Linux and FreeBSD right now, but there are still occasionally surprises and it is less efficient than other options. What's astonishing is that you can get better HW with _much_ better drivers from other companies for cheaper. The Chelsio t520-so-cr is unequivocally better than the intel x540 and costs less. Chelsio, Mellanox, SolarFlare are all good choices for Linux, FreeBSD. I think Chelsio has a Solarish driver for Illumos, not sure about the others.

Inside Intel, the Windows team, Linux team, FreeBSD team do not talk to each other. They appear to not be able to talk to the HW team either. Several large and influential companies have been trying to force Intel to clean up their FreeBSD drivers. They have taken action, and that action has been pretty disappointing. The Linux driver and commit logs are also illuminating since that would presumably have massive market share.

Intel's 40g parts have been fraught with issues at the HW level. The drivers were barely able to outperform 10g at release. This kind of slop is not normal. It should not be rewarded in the market. I had an uphill battle convincing old timers that Intel NICs went so far down hill from the good old days, but after a lot of analysis we have totally written them off for the next two years.

[JoeAltmaier]

Is it truly hardware? I had to deal with radios for years, and the vendor-supplied driver was always poor. It came from some SOM company, who got it from the radio designer. It was always a demo driver, intended to show off the chip features but no effort put into performance or error recovery.

It seemed nobody in the 'chain of custody' of a driver had any incentive to make it work well, in a commercial setting. At best, it was consumer-quality. By that I mean, it worked until it didn't. For a radio, it meant if roaming jammed up then just take the radio dongle out and put it in again. Which in a commercial device (like a forklift touchpad) which had the radio sealed behind a panel, it was junk.

So I had to fix features, performance, bugs, timing, power management, the works. E.g. to get a radio driver fit for a WalMart distribution center forklift going 15mph, it had to roam in milliseconds and choose between 60 APs in radio range. And run for a 12 hour shift without recharging. The chipmaker driver was never, ever good enough.

[kev009]

Dollar for dollar, the Chelsio and Mellanox HW is better in terms of features. Better drivers and vendor support at lower purchase price make it a no brainer. I don't know about SolarFlare pricing but it is also better HW.

igb and ixgbe HW seem to be fair, but you can go look at the HW errata to judge for yourself. Intel had to recall the XL710 due to silicon issues. They also had a firmware incident that fundamentally changes the driver interface, so a particular driver will not work between different FW revs.

[rincebrain]

Chelsio told me they had Solarish drivers, I haven't ever personally tested them.

Solarflare had a 10GbE Solarish driver which they released at one point under CDDL without much engagement, and has recently been proposed for mainline inclusion with support for their 40GbE adapters as well [1].

Mellanox explicitly does not have any Illumos support for newer chips and my understanding is that they have not been receptive to any queries from the community. [2] [3]

Intel's 40g parts didn't arrive before my day job stopped involving high-speed networking every day, but their 10G stuff, as you say, worked relatively consistently across different platforms (though Illumos had some fun with the first few 10G HW revs after they started pulling common code).

I'd guess Intel's 40G parts were either a last blip or a first gasp of attempting to integrate their high-speed Ethernet bits and the "Intel Omni-Path Architecture" bits that are descended from their QLogic IB purchase.

[1] - https://github.com/gdamore/illumos-core/tree/sfxge-merge

[2] - http://lists.omniti.com/pipermail/omnios-discuss/2014-Januar...

[3] - http://omnios-discuss.omniti.narkive.com/FHhYvBui/mellanox-4...

[yuhong]

To be honest, I think they are probably still better than Realtek or "Killer" NICs.