[clappski]

> logging to a synchronized output

There's your performance issue, logging doesn't need to be persisted or even processed by your main program thread or any thread/process actually doing the work. Who's logging straight to stdout or directly ::write'ing a file in a production application?

[wruza]

Me, not write(2)’ing though, mostly because you don’t want to syscall-block every few bytes. Modern service managers redirect 1,2 into logging facilities (or just files if configured such way), and 1,2 are no different than 3+. As if there was a regular way for data to escape a program through non-fd (mmap/shm/fork doesn’t count).

That said I’m not sure what context we are discussing. I work on nodejs apps now where blocking is a non-issue. I’d obviously think about it more in C’s fprintf() or a similar block-by-default environment.

Edit: also, my area is fintech with $0.1-$1000 range per request, not telecom. I log everything because my employer can ask about everything, and it’s always money.

[kcb]

Many many people

[andrewstuart2]

Right, and depending on the actual buffer implementation when logs are buffered, given the massive volume of logging I've seen, it's likely not all that much better than synchronous because those buffers are going to fill quickly and need flushed frequently.

[clappski]

My comment wasn’t about buffered writing in a main thread, it was about moving the work out of your threads actually executing your program and having them push raw log actions into a queue for another thread to do the formatting and writes to file from.

If you’re logging so much that you’re saturating a dedicated thread that just reads from a queue in shared memory, does log formatting and writes to a file then you have bigger problems than figuring out how to log because there isn’t going to be a solution that lets you log with a deterministic latency without skipping events or wrapping around your queue.

Of course, the approach scales fine if you can have multiple log files per process - e.g. a normal log file and a message log or transaction log, because you can give each file its own queue and thread.

[dustymcp]

Uh oh it got silent in here :)