| I’ve been thinking about how to get creativity out of llms, apart from temperature. The thing is even humans have a hard time with creativity. Is it really surprising that llms don’t just one shot a unique story when they are all starting from roughly similar training data and state and a roughly 30 secs of processing time. I had Gemini do some deep research for me around processes and frameworks to prompt ideation and creativity and they do exist. See SCAMPER and others. Another interesting thing that comes up is using random decks of cards as prompts. See Oblique Strategies, Deck of Lenses, the Story engine and similar. I guess I still believe that even creativity is still fundamentally a type of search, as well as problem solving. Manipulating and or combining existing ideas in unexplored ways and breaking out of bias. So I kinda want to experiment with these two approaches: 1. Longer running workflows that follow a framework and loop. 2. Some simple cli tools with these decks and a random draw to trigger interesting directions. I think really just need to break llms out of their initial start state which is mostly the same for everyone. And to run over longer horizons and so the higher level reasoning flows. |
This would be done with Gaussian noise and you could change the standard deviation to make the LLM more "creative".
This would be similar to throwing in and quickly removing random reddit posts and artworks in the LLMs context window, and who knows maybe it could get inspired by that.