[jsolson]

A related warning: your NIC driver can break the qdisc layer (or at least mislead it).

Prior to 4.12 the VIRTIO Net driver always orphans skbs when they're transmitted (see start_xmit()). This causes the qdisc layer to believe packets are leaving the NIC immediately until you've completely filled your Tx queue (at which point you will now be paced at line rate, but with a queue-depth delay between the kernel's view of when the packet hit the wire and when the packet actually hit the wire).

After looking at the code -- even in 4.12 enabling Tx NAPI still seems to be a module parameter.

(I'm not sure which other drivers might have the same issue -- my day job is limited to a handful of devices, and mostly on the device side rather than the driver side)

[atomt]

That is good to know. I just deployed BBR on some pilot virtio backed VMs yesterday and I missed this.

As far as I can tell the Actual Hardware I'm running my other BBR pilots on are doing the right thing.

File under: BBR - still a gotcha or two ;-)