|
|
|
|
|
|
by rwmj
1855 days ago
|
|
|
I'm missing some "high level" context about io_uring, I might as well ask here: - If I want to write a multithreaded program, is it best to have one io_uring per thread? - Per CPU core? - Per device or PCIe lane to the device? (It might make sense ...) - Or one for the whole program? (Will Linux distribute requests across cores and run them in parallel for me?) - If I'm writing a library that uses io_uring, should I create my own io_uring or offer an interface to add requests to one which the main program creates? (Or perhaps both?) |
|
|
Yes! But it's not quite that simple: Sometimes you might need some of the ordering operations in io_uring, which then will force you to use a separate (possibly dedicated) ring for that. Typical cases might be a database's journal.
If you use multiple threads (or processes) to access the same ring you'll need synchronization around submission / completion (potentially separately). Avoiding that when not necessary is obviously good.
> - Per CPU core?
That'll depend heavily on your program. If you have a program which has tasks running on specific CPUs, yes, that can make sense - but at that point it's basically a subset of the muti-threaded program question.
> - Per device or PCIe lane to the device? (It might make sense ...)
Hm. I think that will effectively also boil down to the per-task thing above. Particularly if you use polling it's good to be on the actual NUMA node the PCIe device is attached too.
> - Or one for the whole program? (Will Linux distribute requests across cores and run them in parallel for me?)
I think that'll depend on the type of device. For e.g. NVMe devices, there'll be multiple hardware queues. The CPU the submitter is on will determine which hardware queue is used (and that in turn will influence the interrupt location, at least by default).
> - If I'm writing a library that uses io_uring, should I create my own io_uring or offer an interface to add requests to one which the main program creates? (Or perhaps both?)
I don't think there's a generic answer to this one. It'll heavily depend on the type of library.
Caveat: I've used io_uring a bunch, read some of the code, but I'm not an authority on it.