Hacker News new | ask | show | jobs
by pgaddict 262 days ago
I believe there are reasons why e.g. io_uring could be inherently slower in some cases, and I tried to point some of those out.

With io_uring everything happens in the backend process, and so consumes some of the CPU time that might otherwise be spent executing the query. All the checksum verification, memcpy into shared buffers, etc. happen in the backend. And those things can be quite expensive. With worker this happens in the other processes, spreading the overhead.

Of course, on truly I/O-bound workload (actually waiting on the I/O), this may not be a huge difference. For warmed-up cases it may be more significant.

1 comments

You can have a io_uring worker per thread and send jobs into a queue for them to pick it up and execute, so it won’t max out on cpu. Each worker would do cooperative async scheduling internally between the tasks so nothing delays other things too much.

But this is more difficult than spawning thousands of os threads and running sync workers on them

I'm not sure what exactly you mean by "thread" here. Postgres is not thread-based - there are people working on that, but for now it's all processes.

Some of these limitations are mostly due to Postgres design, no doubt about that.

Ah my mistake. I meant to write something like, you can have a thread per cpu core that is running an io_uring instance like you would have a bunch of workers in a pool in the "worker" implementation.

If there are no threads then this would be, you can have a process per cpu core