Hacker News new | ask | show | jobs
by gulikoza 1507 days ago
I'm having the same issues...

Although I'm not a full time developer, I do some small coding and I'm also sort of responsible for the dev team. I rarely use a debugger. I do most of debugging through logging. There are 2 advantages that I see:

   - it forces me to add the logs where they are needed to understand the flow of the application
   - it forces me to make those logs actually usable, readable and understandable
Developer's first response on a bug report is - give me logs. But I don't have them, because they were not coded. Or they are unusable because they are not logging the actual problem. Developer then insists the problem is fine because he can't replicate it in his debug environment instead of figuring out the issue from the logs. Making logs during development also forces developers to make a conscious effort of finding a balance of what and when to log something, create proper debug levels not to overwhelm the logs, while still providing some useful information...
2 comments

One thing that it takes some time to understand (and to teach to junior devs) is not only to add logs, but also useful logs.

The stages of understanding go like this, in my experience:

1. Log nothing.

2. Upon being told to log things, write mostly useless logs: "Entering f() function", "leaving f() function". When f() fails, who knows what happened?

3. Upon being taught to add context so that logs can actually be used to troubleshoot problems, "entering f() with id 123", "id is 123", "writing 123", etc.

4. Enlightenment usually comes when logs include necessary info but are not redundant, are not noisy, and can actually be used to track down problems. Of course, improvement at step 4 is always ongoing, for all of us.

In my experience, (4) happens when the junior SDE is first on call responding to a problem happening with code they wrote.

They experience the problem with their own logging and realize through debugging their code what information they really wanted all along.

5. Get a ticket in your backlog to reduce logging because you've blown through your annual Datadog budget and it's only May.
> Making logs during development also forces developers to make a conscious effort of finding a balance of what and when to log something, create proper debug levels not to overwhelm the logs, while still providing some useful information...

The need for this balance is why I don't completely agree with the article.

If I'm debugging something with log messages, I put way more detail into it than I want to see in production even at LogLevel.DEBUG.

Plus, I'm adding more and more as I box in the problem. If I left that in I'd have an extreme level of detail around an aspect of the program that's very unlikely to have a bug because it's been thoroughly analyzed and fixed while being blind to everything else (except the parts that have received similar attention).

Maybe I'm just bad at it.

If I'm debugging something with log messages, I put way more detail into it than I want to see in production even at LogLevel.DEBUG.

I mean, there is LogLevel.TRACE when you need it. For those not familiar, this level is exactly for “I’m writing new buggy code where anything can go wrong so stupidest details matter”-time. You can use it to separate normal debug logging like “request body is {…}” from sandbox logging like “loop counter just became [object Object]”.

I don't know that there's an easy solution to this problem, but what I've seen Kubernetes and other Google-derived projects do is replace standard log levels with an increasing series of integers so you can log in increasing detail. This seems a little arbitrary (one developer's 8 might be another developer's 14 or something) but at least you can easily ask for increasingly more detail until you get what you need without noise-by-default.
The increased detail also clutters the source, though.

As I write this I realise that the "right" solution is that my IDE also folds away log calls of below some settable priority... Then I need a linter to prevent any code with side effects from running in a log line... Then I need to work in a language where that kind of static analysis is definitely possible... Maybe it's not so simple.

Create a logging method/function, llog() or something, with named parameters and whatnot.
This just creates a festering problem because devs are given too much freedom. A better solution is just two levels, basic and verbose, with subsystem feature flags you can control to prevent verbose from being a fire hose.
That sounds promising. Ever done it this way? Howd it turn out?