[geertj]

The consumer side of this would be polling a memory location for new logs, correct? It would not be possible to wake up the consumer in 7ns as that would take a FUTEX_WAKE system call with is O(microseconds).

I've been wondering about a FUTEX_WAKE that does not require a system call. Possibly, the kernel could poll a global memory area. Or maybe there is some low-level debugging API available where the kernel could be notified of a memory write by a process?

[jnordwick]

The background thread can just sleep if no data in the queue.

Since you are preprocessing the log messages and you know the minimum bounds. If the thread sleeps for 1 ms. Be generous and say you can at most push to the queue in 5ns, and if you know the largest messages you push will be 200 bytes, you can statically determine a 40M ring buffer will ensure enough space to sleep even at max rate with the largest messages.

And that's just a simplstic scheme. If you have a pool logging structs so your queue is just a pointer to one of those, you can get away with much less even.

And I can think of faster ways too. It isn't that difficult to get around using a semaphore

[gpderetta]

There isn't a significant advantage in having the kernel doing the polling, it would still be busy polling.

If you just don't want to burn power but you can still dedicate a core, there is https://www.felixcloutier.com/x86/mwait.

[geertj]

> There isn't a significant advantage in having the kernel doing the polling, it would still be busy polling.

I was thinking in terms of a generic syscall-less wake functionality where the kernel could do this for all processes in the system. So you'd lose one core per system instead if one core per consumer.

>If you just don't want to burn power but you can still dedicate a core, there is https://www.felixcloutier.com/x86/mwait.

Interesting. Could be used to make the kernel loop above burn less power.

A user-space implementation could presumably also be built. There could be a shared memory segment shared between producers and a monitor. A producer sets a flag in case it needs attention, and the monitor busy polls the segment. The monitor could then use e.g. a signal to wake up consumers.

The latency between the producer signaling and the consumer taking action would be a higher than with futexes. But there would be no waits/context switches in the producer at all. Might be a solution for some low latency use cases.

[sriram_malhar]

The consumer (background thread) only polls; there is no need to wake up the consumer.

At steady state, the consume is either blocked on I/O or is scanning the producer buffers. When all producer buffers are empty, it can just sleep for a short time.

[geertj]

This would work in this specific case where we know that there is a maximum rate at which work is produced. Arguably I was hijacking the thread to discuss a more general problem that I've been thinking about for a while. I have the sense that a ring-buffer that has a wait-free push with low bound on consumer side latency that doesn't require fixed interval polling would be a nice primitive that I certainly could have used at times.

And in fact ... the wait-free wakeup part of this is already there. Now that io_uring has futex support, a producer can enable kernel-side busy polling on the uring, and then submit a FUTEX_WAKE to the ring without doing any sys calls. This Github issue [1] has a nice description.

[1] https://github.com/axboe/liburing/issues/385

[gpderetta]

I'm waiting for this to become mainstream: https://lwn.net/Articles/869140/

It could enable a full user space uring without the need of kernel side polling.

[geertj]

Interesting, thanks for sharing. I did some digging on the linux-kernel mailing list and it seems that after the first patch series in Dec ‘21 and the LPC talk there was no further work on this. Too bad if true. Also I wasn’t able to easily find which hardware supports this today. I did see a reference to a RISC-V project that is adding a similar feature.

[toxik]

O(microseconds) = O(years), this is not what big O notation means.

[renewiltord]

That seems like a type error on your part. O(unit) is a categorically different notation than O(mathematical function). You must be confusing it with the asymptotic notion that is the latter. But units are not functions and so clearly it cannot be the same notation.

[toxik]

No, and it’s absurdly petty to argue otherwise.

[davidsgk]

One might say it's also pretty petty to call out a casual usage of a notation being used in a way that people in the thread are understanding just fine...

[toxik]

I think it’s helpful because it is a common mistake.