Hacker News new | ask | show | jobs
by nodefortytwo 2539 days ago
Although I agree in principle one other aspect that becomes critical in high volume systems is efficiency.

Logging is generally a text stream and in production should only be warning+ levels (imo) so the impact on performance is negligible and really only when something is going wrong.

I want metrics all of the time, potentially 10 or 20 metrics per request/action, a high performance, low network method for sending those metrics with low latency is critical.

We need both systems and both should be treated as tier 1 systems within an organisation. I don't think pushing metrics into a log stream is a scalable architecture.

2 comments

Being a text stream, you'd also have to reparse that text stream, and hope that the text stream never changes. Which is at odds with how I usually use logs (as places to dump information for humans).
Nowadays using something like elasticsearch is common for this and then you don't have to reparse it. It's already indexed and you can search for what you want.

If you're going back to the raw text files, then you're right. But I'm not sure why anyone would do that these days.

but then it might be expensive to encode every possible dimension in the log stream
the author is saying the same thing. although he didn't mention efficiency in his article, which is an excellent remark btw. but he is saying you can usually go from logs to metrics but not the other way around. and that we often need both.