Hacker News new | ask | show | jobs
by mr-karan 530 days ago
Never really caught on with the idea of integrating external I/O in something as fundamental as logging. More often that not a pull vs push approach is suited for logging. There are dozens of high performant log collectors/aggregators who can collect, transform, and push logs to N number of stores. What’s the advantage of doing this right inside the app? (Besides maybe serverless functions?)
2 comments

Sometimes you don't have the resources or need to spin up an external collector. I might be in a situation where I want to collect logs with DataDog, but I also am not big enough nor would I have the time to set up a sidecar or configure the agent itself. It's possible you don't have access to the host environment to install something like an agent either (as you mentioned, like serverless functions).

The advantage for the in-app case is that it's fast to on-board with and doesn't require significant / any devops involvement compared to the above scenario.

One downside though is depending on how those logs get shipped, it might not scale well if you're writing an extreme frequency of logs (it's possible to reach HTTP limits with DataDog and New Relic log publish APIs in a high volume/frequency situation), and that's when you would want to move to and external collector that can process logs in a much more asynchronous fashion.

I'm not sure if the question is specific to LogLayer or just talking about logger transports in general that does in-app push, but I see the cloud collector features of LogLayer as something that you can easily add to your logging stack when you start out small and need it, but transition to an external collector later.

(One might argue that some OTEL tools can also do this, but as stated in another response, I'm not familiar enough to know if they'd do a better job or not in terms of the overall logging DX that LogLayer provides; their primary job is to ship logs from my understanding.)

+1 on this. Kids, don't rely on log to remote inside the application.

You get a growing combination of N apps x M log servers to support and link into the application. Even if you only use one of them yourself. Ops should be able to change to a new logging backend without rebuilding every application.

Bugs in the log shipper has the possibility to take down your app, you never want this. Also think about the unnecessary security surface this adds, especially when multiplied by M above. (log4j anyone?)

When the log server is unavailable or saturated you still want those file or console logs available for last resort debugging.

Shutting down the app risks loosing last minute of unshipped logs, unless you require slow graceful shutdowns, which is never a good idea. Think about crashes, where you need logs the most.

Use sidecar and similar strategies instead. You are most likely also running some third party application elsewhere, where you don't have the option to rebuild with a new logger, so you need this type of knowledge anyway.