[infogulch]

LLMs are far from great writers. They struggle to form long coherent sentences and lean on punctuation like emdash and semicolon to ensure grammatical correctness when splicing together short phrases.

This makes me wonder if the reason why agents love weird punctuation is because the labs run the base models through a RL training step that forces them to correct their grammar; but instead of rewriting short spliced sentences into long coherent sentences, they just learn to splice them together with punctuation that passes the automatic grammar checker.

[HDThoreaun]

I mean yes, but the vast majority of people aren’t even good writers. Claude writes better than most of my coworkers and we’re all highly educated. Most of us could probably beat it if we really tried, but then we could also prompt it to be a better writer too and none of us are beating that. I think the short pithy phrases that are so common are all post training stuff that the labs add because most people don’t want long sentences.

[esikich]

They are great writers if you tell them what you want. If you're unable to properly articulate the writing style you would like as you would a software spec, well, garbage in, garbage out.

[calf]

That's not fair. To most users this would not be an obvious thing to do, unlike software/scientific/analytic uses of LLMs.

So it's not a matter of inability but rather awareness and know-how.

Then secondly, humans learn to write intuitively and heuristically (for example "paragraphs should align sentence subjects with an identifiable flow of conceptual main characters/actors" is an entire chapter of expert writing advice traditionally taught at the undergrad level). The act and teaching the act are different beasts. The way to specify would have no objective, clear best practice to do so: Even an English teacher or a language expert would be challenged to come up with a concise list of instructions to set up an LLM to output something of sufficient quality.

[esikich]

It's absolutely fair unless you believe in magic. "Make it sound like a human and not AI" is never going to get you good results just due to information theory. You're not giving it enough to work with or what the hell that even means.

[calf]

I never said anything like that, but if you're making a simplistic appeal to information theory then I rest my case.

[thimabi]

What do you suggest for articulating the writing style that one wants from LLMs?

I’ve been experimenting with having LLMs write/update academic notebooks for me, and so far the best results I’ve gotten came from correcting their output and asking them what they’ve “learned” from my feedback.

[brookst]

I have a directory of stuff I've written over the years, fiction to tech writing to emails, and then asked claude to generate a style guide based on that corpus. It's surprisingly good. Not perfect, but it definitely extracted a lot of my tone and patterns, and when generating documents for sharing I just prompt with "please rephrase in my voice using the style guide at ~/path"

[esikich]

Use examples and learn about writing. If you're not a good writer, or at least have a passing interest, I'm not sure why anyone would expect it to nail their style any more than you would expect it to architect software in something you're not familiar with.

Give it some paragraphs of writing you like, authors, writers, whatever. Also don't expect to one shot long papers. Have it write out a table of contents and write it in small chunks.

Typical things I'll say is avoid purple prose, avoid common cliches and tropes, "show don't tell", avoid wild metaphors and similes, etc. But it all depends on what you like. Look at things you do like and try to think about what it is about that writing that is different than a middle schooler or an AI writing it. Theres no formula for it.

[sushid]

Are there any skills or publicly available repos that do what you claim here? I would love to learn to have it write better.

For me, my amateur attempt is having another LLM do a review loop to remove clearly offending phrases and a heuristic eval to change sentence structures to be more similar to mine, THEN my manual HITL loop to rewrite ~20% of the sentences anyway.

[esikich]

If you're trying to get it to write it in your voice, just dump some stuff you've written into the context. You need to think about other voices and be able to describe what makes yours different as well.