Hacker News new | ask | show | jobs
by Ferret7446 340 days ago
> The ones that I'm sure about: journald is more annoying to use than syslog (it takes forever - like a minute on a top of the line CPU - to scan two months worth of logs)

That sounds like an unequal comparison. For the same amount of log data and the exact same filter operation, journald should be strictly faster or equal to text logs. Are you sure this isn't because in your journald test you're going through a lot more logs? Because journald makes it easier/feasible to collect more logs.
1 comments
ahartmetz 339 days ago
No, journald shouldn't be strictly faster. It signs/encrypts/whatever the logs and stores them in some kind of structured format that needs to be converted to plaintext before processing it as such. That is not faster than just reading the plaintext. Grep and less search etc are very fast.

And no, I don't want to use the structured features. For error diagnosis, I want every occurrence of the search string in every field over a long time for comparison purposes.

link
Ferret7446 339 days ago
> stores them in some kind of structured format

No, it stores them with binary metadata, rather than ASCII metadata. "Plaintext" needs to be parsed, which is expensive. You don't really think binary RPC formats like protobuf are slower than JSON because it's "structured format" compared to "plaintext" I hope. Or SQL databases are slower than CSV files.

> And no, I don't want to use the structured features.

I don't see why you'd prefer having to figure out different time formats to parse from different software logs compared to a standard time format.

link
ahartmetz 339 days ago
I'm searching for "service name" and I don't care if it comes from "service" or mentions "service". That's unstructured.

The rest of your comment doesn't even apply. Search is not parsing and not structured data transfer (which is what binary RPC formats are for), and unless you have a fulltext index, there is no faster way to do it than the usual algorithms, which work at 100+ MB/s. With log files, the text is right there ready to be searched. With journald, it needs to be assembled first, and it's pretty slow in practice.

link