Hacker News new | ask | show | jobs
by silon3 4252 days ago
"like" (it happened to me)

No journald. Consequently, this also means no libqrencode and libmicrohttpd integration, nor hooking coredumps to the journal. The default log target for auxiliaries is now LOG_TARGET_SYSLOG_OR_KMSG.

The main case against binary logs is their corruptibility. This happens more often than you’d think, due to not having any transaction consistency, as an RDBMS would. The advice of the systemd developers on handling this? Ignore it.

3 comments

> The main case against binary logs is their corruptibility. This happens more often than you’d think, due to not having any transaction consistency, as an RDBMS would. The advice of the systemd developers on handling this? Ignore it.

To be honest, if the format is done well there is no reason you cannot still read all the non corrupted records. textfiles corrupt too, you just generally live with them being broken because you can read the unbroken stuff.

Going by the bug tracker the journald approach is to make the reader work with broken files better.

The binary logs are append only. The bug tracker entry says that they got "some corrupted logs" but not how. People get corrupted or truncated text logs all the time, they just ignore it because "oh the log ends midway through a line? eh".

The ridiculous "oh my god my logs will be always ruined thing" is a beat up by those who need to assassinate something about systemd, because it's always been devoid of any technical accuracy or discussion.

A truncated text log can still be informative, and I've never had a corrupted text log hang or crash my text editor.
That's because your text editor has had thousands of hours of work going into catching all of the edge cases.

Things which have caused editors to hang or crash: 1. Binary data outside of the US-ASCII visible range 2. Malformed or creative Unicode combinations 3. Very long lines 4. Very many lines 5. Inconsistent line termination 6. Lines with patterns which trigger syntax highlighting, URL underlining, etc. 7. Embedded terminal escape sequences

It's not unreasonable to imagine that the journald developers might spend the same amount of time on safeguards, something which is easier with a well-defined binary format.

Yeah so let's just redo all those thousands of hours of work... for one specific format that has literally one use.
It's not thousands of hours with a better-defined format than free-form text, which is obviously the case for journald. Doing that for a single format makes sense for something which would be as heavily used as system logs – it's not like people haven't spend crazy amounts of time writing syslog parsers or other hassles which go away with a structured format.
This is like saying.

"I get shot going to work every day. But I bought a bullet proof vest, so life is okay."

Iron bandages don't solve problems, they for a short time work around them. Except because nobody actually like writing systems level code, they stick- forever.

> "I get shot going to work every day. But I bought a bullet proof vest, so life is okay."

You get corruption if the machine shuts down incorrectly. In binary or text files. In that case journald/syslogd act like a black box. Syslogd will give you whatever garbage it has in those text files, journald will give you the surviving records and will tell you which ones are unreadable.

If you want to not get shot, shut down your machine properly. If you cannot shut down your machine properly because it crashed then you have data garbage if you want it or not. Drives work that way.

I'm not sure what you are arguing for.

Journald does not write broken records itself.

Not related to all binary logging, but journalctl is incredibly sluggish.

Some combinations of slow (rotating) media, certain filesystems, and underpowered CPUs (ARM in my experience) and you get a barely usable logging system that makes you want to punch your box.

I'm not currently using Linux, so I might be wrong, or misunderstanding what you're trying to do, but could you just have journald write to syslog?

ForwardToSyslog=yes in /etc/systemd/journald.conf.

I don't even have an esoteric setup and every time I want to read logs I'll have to wait 30 secs. Even though there is absolutely no activity.
Try "journalctl -e" or "journalctl --since today", it seems like by default it will load all available logs which leads to this slugishness.
it is slow because log is read written in a block binary format but pager accepts byte granular fseek feature to go forward without loading everything into $PAGER's buffers.
See ? People complaining before reading TFM. Thank you !
Sounds like you're running into some bug. I've never run into any perceptible lag.
That sounds odd. Did you try to strace it and see what it actually does?
I found this to be the case as well. Especially on a busy system where there are a lot of events, when you're trying to find the cause of an issue and constantly opening and closing logs, the lag is definitely noticeable.
To be honest: that's kinda to be expected if you run it on a slow machine. However you can configure it to be in-memory only and forward to syslog and you have it as it was before.
I'm highly suspicious that syslog can actually be faster in such a situation. If you can't seek to the disk quickly, then it doesn't matter what you do - logging will be slow if you need to flush frequently.
I googled "libqrencode". Is it really QR code encoding?
Yep. It's because when you kernel panic most people do not feel like typing stuff off a screen. If you have a QR code on your monitor however you can easily scan it and it contains all the info about what is wrong.

That's a pretty good feature if you ask me.

That's a Linux kernel feature, not a systemd feature. It's unrelated: http://www.linux.com/news/featured-blogs/200-libby-clark/773...

systemd has libqrencode integration, in order to, if I recall correctly, display a QR code when you generate FSS keys from journalctl.

That is a useful feature! The way it was written it sounded like QR codes were going into the logs.