[sometimelurker]

Just a thought, what if you added in a random steering vector at the start of the residual stream for each token? Intuition says it wouldn't act in the same way as increasing temperature would, but I honestly have no idea what would happen. Maybe it would be better if the random steering vector flowed a little from token to token so the output wouldn't be so noisy.

This would be done with Gaussian noise and you could change the standard deviation to make the LLM more "creative".

This would be similar to throwing in and quickly removing random reddit posts and artworks in the LLMs context window, and who knows maybe it could get inspired by that.

[aleksiy123]

Tbh I have no idea, I’m mostly thinking about it from what I can do when using the frontier models, so I don’t think such low level changes are available to me.

But another dumb idea I had was a set of random words inspired by Terry Davis godsay https://github.com/orhun/godsays

With a more appropriate wordlist appropriate. Call it muses.