[HPSimulator]

One thing that might also be happening is that LLMs tend to converge on metaphors that compress complex ideas quickly.

If you look at how engineers explain messy systems, they often reach for anthropomorphic metaphors — “gremlins in the machine”, “ghost in the system”, “yak shaving”, etc. They’re basically shorthand for “there’s hidden complexity here that behaves unpredictably”.

For a model generating explanations, those metaphors are useful because they bundle a lot of meaning into one word. So even if the actual frequency in normal conversation is low, the model might still favor them because they’re efficient explanation tokens.

In other words it might not just be training frequency — it could be the model learning that those metaphors are a compact way to communicate messy-system behavior.

[tmaly]

I am waiting for future versions to start compressing with memes

[HPSimulator]

That might actually happen indirectly.

Memes are basically compressed cultural references. If a model sees the same meme structure repeated across a lot of contexts, it could learn that a short phrase carries a lot of shared meaning for humans.

The interesting question is whether models will start inventing new shorthand metaphors the way engineering culture does ("yak shaving", "bikeshedding", etc.), or whether they'll mostly reuse ones already embedded in the training data.