[deergomoo]

> As for documentation, I've found this simple instruction to vastly improve LLMs' output:

> Never write READMEs, docstrings, or comments. I will write those myself later. And yes, I really mean this.

This is quite validating as I came to the exact same conclusion myself. We’re required to use an LLM for every task at work that touches code†, and I was really struggling to get Claude to stop with the long waffly comments that reiterate the next few lines of code in 3x as many characters, making contextless references to subtasks in whatever harness du jour we’re using this week.

No amount of examples or explanation of what I wanted would make it stop. And then I realised of course, I’m asking something which has no concept of meaning (or, indeed, anything) to only add meaningful comments. More fool me I guess.

Of course, it’s ultimately pointless given all of my colleagues are regularly opening PRs with more comments than code anyway. 80% of my code review responses these days are just increasingly exasperated “pointless comment, please remove”.

† This is just as infantilising as it sounds, by the way

[openasocket]

+10, I can't stand AI-generated comments. Half the time they are paragraph-length translations of every 5 lines of code into English prose, or just regurgitating whatever prompt they were given. When they actually try making high level comments they are worse than useless. It's really hard to put into words exactly why. It's something about how they are very verbose, but at the same time vague and stated in a slightly-off manner. Or they will introduce new terminology not reflected in the code's nomenclature, sometimes terms that are just synonyms for terms that the code already uses. Something about it makes me instantly distrustful of any information the comment may actually convey.

[monknomo]

locally correct and globally incoherent is how they seem to me. I, and a number of folks I have talked to, find some of them incomprehensible. Like each sentence makes sense, but the overall thing does not.

[wannabe44]

It's because they don't really have a mental model. RL with Verifiable Rewards can reward working code, but it can't tell you whether the prose make sense or not. In theory RLHF should be good with this. In practice, it appears to be a measure difficult to quantify and steer. I am not an insider though. So it's just a theory.

[kaffekaka]

I have the same impression. It is an interesting phenomenon. Is it one of the ways we will gradually learn that AI in the form of LLMs is fundamentally lacking?

I use AI all the time, it gives a tremendous boost in speed for many tasks and it is most certainly here to stay, but there is something off about certain things, like code comments. Like a weird person.

[Hupriene]

I kind of wander if the comments are intentionally bad, providing programmers something to bike shed about. A sneakier approach to the flattery the models used to so use so heavily.

[sync]

You can create a PostToolUse hook [0] which will automatically chastise Claude to shrink comments to one line. It's not perfect but better than the sea of prose it tends to generate by default. Bonus is that it would apply to your colleagues as well!

[0] https://github.com/chrisvariety/branch-fiction/blob/deb37f2b...

[exographicskip]

Good thinking! The multiline paragraph comments drive me up the wall.

No amount of agents.md / commit to memory updates has stopped agents from littering the code base with useless comments.

Think I'll add it to pre-commit hooks getting called by posttooluse

[nevertoolate]

> We’re required to use an LLM for every task at work that touches code

Why not just write the code in the prompt so LLM can paste it.

[fibonacci112358]

I have the opposite problem with Code, it barely writes any comments even for fairly complex C++ code blocks (systems programming stuff), have to prompt it to add extra details about why it did something. Request in agents.md doesn't seem to help much.

[NichoPaolucci]

I'm in the same boat. We have very little documentation where I work, and one of my goals was to improve that as I got onboard. We have a newer dev that started ripping +3K, +5K, +10K line PRs against our documentation repository.

I have tried to shut it down, it immediately nullifies the docs and make it genuine slop.

I truly believe that documentation should be human written and human readable. It's all of the stuff that AI does NOT pick up on. Small inconsistencies, only the necessary details. IE: Don't just rewrite the code that we have into readable english (We are software developers after all. We can read code.) - comments / documentation to me has always been the "in between the lines" stuff that can't be put into the code.