Comments are often a code smell indicating that the code was written in a hard to understand way. On a top notch code base - the kind that almost nobody works on - they probably should be sparse and mostly unnecessary.
Moreover, when a lot of people are asked to comment they write stuff like "this function does x to y" when the function is named "x_to_y". No shit sherlock comments I call them.
I'd always prefer to have good comments, but I've worked on a lot of code bases where no comments wouldn't really have been any worse.
"This looks like a good case for a binary tree but it doesn't scale well under load."
"We would add support for this motherboard that looks a lot like the one we do support, but it doesn't offer feature X."
"We want to add more detailed logs here but we haven't figured out how to cope with the performance hit."
Closely related, comments are also useful for explaining why code that looks wrong/poorly optimized/obsolete isn't actually, which is something that the code itself can't explain.
I'd say that redis is probably a better example of gold standard clean C code, though, and they follow the rule of "don't comment unless it's decidedly non-obvious" pretty assiduously:
Those two files do have a low ratio of comments. However a large number of files in those code bases have quite a lot more comments... I personally don't think these two code bases have sparse comments, overall, although perhaps it's subjective or my bias is affecting my judgement :)
Eg, to me this is well-commented and I wouldn't call it sparse.
I would much rather be on a project where commenting is prohibited then where it is mandatory, e.g., the senseless babbling that goes along with javadoc/doxygen/PEP-8-like comments. That is such absolute garbage. At some point it even becomes hard to spot the code in between the garbage. Yes, even with syntax highlighting....
Write a docstring for a function before you write the body of that function. Keep it short. Don't overthink it. At this point, it doesn't even have to be syntactically correct.
If, in a brief sentence, written in plain English (or any other language of your choice), you can't explain the idea, you need to "go back to the drawing board." Resist the urge writing it in a programming language before you can explain it in English.
When you're done writing the function, go back to the docstring, revise it; in some cases, you can even remove it.
The purpose of a docstring is to help humans to quickly scan the code and discern the meaning, the idea of the function without having to read its source. Like a trailer for a movie - it should give you the general idea, but not give away the implementation details.
Think about docstrings as "type annotations" for humans. We use types and type hints "to help the compiler understand our code better" because computers do not understand plain English, but very often, we have no empathy for fellow programmers.
It's good to comment the what too if it's hard to discern what it's doing from the code.
There have been a number of cases where, for instance, I had to call a bizarrely named API and do something unexpected in order to get an unintuitive outcome and in that case a "what" comment isn't such a bad idea.
(not OP, but here's my 2c) comments should explain why you choose to write the corresponding line of code.
It's a "meta-communication" between the original author (even a previous-month you) and the reader.
Code reading should be "boring". The code lines (and so the meaning) should be obvious. A good comment, IMHO, should convey the information "I'm sorry for this complexity here, but you need to take care of this edge case..."
That's a reasonable attempt at defining what over-commenting is by trying to define what it isn't. However, it's not a very subjective definition. I tend to write fairly long comments anywhere I think it might help me in the future. I can't predict what I'll forget. I can't predict what will help me understand my code in the future. I have almost never looked at a comment and wished that someone would have written less (aside from comments that just mimic the code, e.g. "increment x"). It's almost always the opposite. I almost always wish I (or the other dev) had left more information.
The beginnings of functions, for instance, are great places to comment, perhaps even including a definition of some of the subroutines within it.
In the case of an exceptionally large subroutine, commenting within the subroutine may be advantageous - but at that point you may consider another function to replace that larger chunk.
Variable names that are perhaps obscure and are not practical to rename for whatever reason can also use comments.
Class headers are of course also a great place to comment, to explain the purpose of the class.
You didn't really answer my question though. I didn't ask where you should comment. I asked for an objective definition of what over-commenting would be.
> I asked for an objective definition of what over-commenting would be.
I don't think what you're asking for is possible. It's an inherently subjective standard.
If an objective definition were possible, then it'd be possible to write a static analysis tool that reads both your code and the comments and flags "over-commenting on line 23, under-commenting on line 457". That could be done with sufficiently advanced AI, but at that point it would just be the subjective opinion of the tool's author being enforced.
over-commenting is explaining in pseudo-english what you've written in code. Prentend you are at the United Nations but you speak all the langauges, so you don't need a word-for-word translation. You don't understand to motivations of every speaker though, nor their cultural influences and a million other factors, so context is super important. In the briefings you get some of the information is not new; but if the vast majority of what you're told is obvious or discernable from what people say, you're looking at over commenting. It's not a yes-no thing but a balance of evidence, subjective opinion.
I've seen it happen many times now. A nice, clean, easy-to-read code base turns into a forest of unreadable shit thanks to the introduction of parsed comment tags. Perl, Ruby, Python, JS. Doesn't matter what tool or language.
Please, people. I beg you. Stop putting this shit in code. Not everyone is using the same bloated IDE as you, nor do we care to maintain your silly block text and parameter text that is outdated the day you wrote it. If your function needs a block text to explain how to use it, you need to find a new job. I'm serious. You're not good at your job. If it's not self-evident what the file you're looking at does and the function within the file does, then refactor it. Cut it down. Make it make sense.
I've seen code in which the development team added a revision comment in the source file for every revision to the source file, instead of just letting the version control system handle it.
This is what I’m saying by over-commenting. I stated comments at the beginning of functions - what are the arguments? - what does it return - or at the beginning of class names are the wise go.