Hacker News new | ask | show | jobs
by ahartmetz 339 days ago
No, journald shouldn't be strictly faster. It signs/encrypts/whatever the logs and stores them in some kind of structured format that needs to be converted to plaintext before processing it as such. That is not faster than just reading the plaintext. Grep and less search etc are very fast.

And no, I don't want to use the structured features. For error diagnosis, I want every occurrence of the search string in every field over a long time for comparison purposes.
1 comments
Ferret7446 339 days ago
> stores them in some kind of structured format

No, it stores them with binary metadata, rather than ASCII metadata. "Plaintext" needs to be parsed, which is expensive. You don't really think binary RPC formats like protobuf are slower than JSON because it's "structured format" compared to "plaintext" I hope. Or SQL databases are slower than CSV files.

> And no, I don't want to use the structured features.

I don't see why you'd prefer having to figure out different time formats to parse from different software logs compared to a standard time format.

link
ahartmetz 339 days ago
I'm searching for "service name" and I don't care if it comes from "service" or mentions "service". That's unstructured.

The rest of your comment doesn't even apply. Search is not parsing and not structured data transfer (which is what binary RPC formats are for), and unless you have a fulltext index, there is no faster way to do it than the usual algorithms, which work at 100+ MB/s. With log files, the text is right there ready to be searched. With journald, it needs to be assembled first, and it's pretty slow in practice.

link