They are great writers if you tell them what you want. If you're unable to properly articulate the writing style you would like as you would a software spec, well, garbage in, garbage out.
That's not fair. To most users this would not be an obvious thing to do, unlike software/scientific/analytic uses of LLMs.
So it's not a matter of inability but rather awareness and know-how.
Then secondly, humans learn to write intuitively and heuristically (for example "paragraphs should align sentence subjects with an identifiable flow of conceptual main characters/actors" is an entire chapter of expert writing advice traditionally taught at the undergrad level). The act and teaching the act are different beasts. The way to specify would have no objective, clear best practice to do so: Even an English teacher or a language expert would be challenged to come up with a concise list of instructions to set up an LLM to output something of sufficient quality.
It's absolutely fair unless you believe in magic. "Make it sound like a human and not AI" is never going to get you good results just due to information theory. You're not giving it enough to work with or what the hell that even means.
What do you suggest for articulating the writing style that one wants from LLMs?
I’ve been experimenting with having LLMs write/update academic notebooks for me, and so far the best results I’ve gotten came from correcting their output and asking them what they’ve “learned” from my feedback.
I have a directory of stuff I've written over the years, fiction to tech writing to emails, and then asked claude to generate a style guide based on that corpus. It's surprisingly good. Not perfect, but it definitely extracted a lot of my tone and patterns, and when generating documents for sharing I just prompt with "please rephrase in my voice using the style guide at ~/path"
Use examples and learn about writing. If you're not a good writer, or at least have a passing interest, I'm not sure why anyone would expect it to nail their style any more than you would expect it to architect software in something you're not familiar with.
Give it some paragraphs of writing you like, authors, writers, whatever. Also don't expect to one shot long papers. Have it write out a table of contents and write it in small chunks.
Typical things I'll say is avoid purple prose, avoid common cliches and tropes, "show don't tell", avoid wild metaphors and similes, etc. But it all depends on what you like. Look at things you do like and try to think about what it is about that writing that is different than a middle schooler or an AI writing it. Theres no formula for it.
Are there any skills or publicly available repos that do what you claim here? I would love to learn to have it write better.
For me, my amateur attempt is having another LLM do a review loop to remove clearly offending phrases and a heuristic eval to change sentence structures to be more similar to mine, THEN my manual HITL loop to rewrite ~20% of the sentences anyway.
If you're trying to get it to write it in your voice, just dump some stuff you've written into the context. You need to think about other voices and be able to describe what makes yours different as well.
So it's not a matter of inability but rather awareness and know-how.
Then secondly, humans learn to write intuitively and heuristically (for example "paragraphs should align sentence subjects with an identifiable flow of conceptual main characters/actors" is an entire chapter of expert writing advice traditionally taught at the undergrad level). The act and teaching the act are different beasts. The way to specify would have no objective, clear best practice to do so: Even an English teacher or a language expert would be challenged to come up with a concise list of instructions to set up an LLM to output something of sufficient quality.