[sebcat]

> The justification for storing logs in a binary format was speed and performance, they are more easily indexed and faster to search.

Are there any benchmarks for this? I don't know why, and I may be doing things wrong, but journalctl -lu <unit> --since yesterday in our prod env takes a couple of seconds before I see any output, while a (z)grep on a date rotated, potentially compressed log file on a non-journald system is generally instantaneous.

On a side note, I really like machine parsable, human readable logs, and I've never had any speed or performance issues with it when dealing with volumes of hundreds of millions of log entries, though that may be because I don't know any better.

[flukus]

My question would be "is searching worth optimizing for anyway?". Log files are typically only read when things go wrong or by automated systems where a few seconds here or there doesn't make much of a difference.

Faster to search often means slower to write too, I'd prefer faster to write and slower to search if I had to choose.

[gens]

Of course nobody benchmarked. Rarely are there programmers in OSS these days that benchmark anything. (not nearly as common as those who claim "speed")

There was a bug though where the journaling was so slow it caused the whole server to crash. (sorry, can't find it right now; something about them using mmaped files and the access pattern was.. or something, interesting stuff:)

Binary logs should be faster, in theory, and should/could be as robust as text logs (if not more), in theory. edit: note that GNU grep is insanely optimized.

[pif]

> Rarely are there programmers in OSS these days that benchmark anything.

Please, don't release such a bold statement without any source. Please!

[gens]

There is the object of this discussion, systemd, that has always been about "starting things in parallel" to make it "boot faster" but there has never been a proper benchmark (the distrowatch guy said, when asked, that he didn't notice any difference; i didn't see a proper benchmark anywhere by anyone). The kdbus proponents that were active on lkml at the time of increased drama (including the devs) also never did a proper performance evaluation. It was Linus who had to actually run perf on dbus to (easily) show the real reason why it is slow... I also remember commenting en passant once or twice about bad and/or biased performance testing. So that is my "source", the relative lack of proper benchmarks on blogs when they talk about performance of a piece of software. At least this is how it seems to me, as i did not email a bunch of programmers to feel around (innuendo not intended, but funny so i'l leave it).

On the other hand some OSS projects (the various databases, for example) do have posts/writings on performance. There are even some really in depth ones.

I'm fine with things being bloated (or just plain cpu slow) as that careless coding lets people make some nice things (a big part of KDE, for example). But i am not fine with people who publicly claim that their code is fast(er) without doing proper benchmarking. Hell if it was hard i wouldn't say anything, but it is not (for C code at least).

Objectively most programs i use (and most people use, even without knowing) can afford to be slow and eat much more memory then they should. Because most programs just sit there doing nothing for the grand majority of their run time (FF and sublime.., why do you send those msgs that use up 1-4% cpu ? i just don't understand). (honestly i'm worried that they are lowering(raising?) the performance baseline. but that's just predictions of future, voodoo, spirit animals showing the flow of chi or something)

PS Sry for the dumb rant. I like the topic of performance.

[yjftsjthsd-h]

How would you cite the lack of testing? GP did discuss several regressions that imply performance wasn't thoroughly tested.

[amyjess]

Using systemd to manage docker images, I've found journalctl --unit foo to be massively useful when debugging "why did this docker image crash?".