[revel]

LLMs are bad at creative work and I don’t see them improving any time soon. Try asking an agent to write a story about raccoons. It will almost certainly involve either stealing food or raiding trash with a 50% chance of having a character named Pip. If a location is mentioned, it’ll be Elm Street.

I guess this is the average story and, similarly, the average game is boring and predictable

[CuriouslyC]

Ironically, they're getting worse at creative work because all the RL is collapsing their distributions.

[dofm]

Worse in a very specific, bland, uninteresting way, too.

Early generative AI at least had the virtue of relentless, unsettling weirdness, in the same way that generative art from the late 90s and early 2000s did. A handful of people made creative use of that spooky weirdness.

Now it turns out "Airspace" art.

[MasterScrat]

I don't think this is bad. I want tools that deliver what I envision. The fact that given the same input, I get similar output is something I would rather consider a positive. The imagination should come from the human at the wheel.

[ben_w]

While they are bad at creative work*, this reasoning isn't going to show it.

If you took the best, most creative, human writer in the world, and for thought experiment reasons they had amnesia (to mimic AI blank context windows) specifically while you asked them for a story idea 100 times in a row, my expectation is that this human would also give you the same idea at least 80 times out of that 100.

* still better than the mean human, but even the top 0.1% of humans aren't all professional authors.

[emp17344]

Better than the mean human? The mean human creates far better stories while daydreaming. AI enthusiasts have such a distorted view of human capability, it’s bizarre.

[Supermancho]

The qualification "better" is doing some hefty lifting in your assumption.

What you think is better is not what I think is better. Imagination is not storytelling. You ask the "mean human" to write, it's going to be worse than an LLM in spelling and grammar, if you get anything at all.

[superdisk]

Heh, tried it and got a story about raiding trash starring Pip. But yep, I'm well aware of this problem as well. /r/sillytavernAI is a sub dedicated to trying to coax LLMs into good creative writing but it's kinda impossible to do consistently.

[jmole]

Seriously, what is the deal with Pip! I was experimenting with childrens' fiction more than a year ago and 9 times out of 10 you'd get a character named Pip unless you were specific in the prompt not to do so.

[cautiouscat]

I recently saw a study on this actually. https://arxiv.org/abs/2605.26492

[dofm]

I saw that, tested it in Qwen. Elias Thorne, on the first run, as a clockmaker too.

[dofm]

Elias Thorne!

[josh-wrale]

I think the trick is to use LLMs like scalpels instead of hammers. That said, you can place a scalpel in the hands of a deterministic (or not) robot.