[b20000]

how do you access this DDIO feature if you are writing a C or C++ application? intrinsics?

[tbr1]

DDIO operates mostly transparently to software, with the I/O controller feeding DMAs into a slice of L3. Hardware can opt out by setting PCIe TLP header hints, and you have some system-wide configurability via MSRs, but it's not something a userspace application can take into its own hands.

[b20000]

so is this taken advantage of by the OnLoad drivers of solarflare cards, for example?

[isogon]

Noticed this just now. It is.

[medawsonjr]

It's configurable via MSR. You can also disable it system-wide or on a PCIe port basis. I detailed it all here:

https://www.jabperf.com/skip-the-line-with-intel-ddio/