|
|
|
|
|
by im3w1l
625 days ago
|
|
We don't actually know if quantum physics has real randomness or not. Quantum collapse is an unsolved problem. > I could see early attempts at [introducing randomness] causing a form of LLM schizophrenia because the neural network wasn't resilient enough to the induced error. 1. This is actually already done. Temperature parameter controls amount of randomness. 2. Neural networks are quite noise resistant. |
|
Typically, what happens is that the network outputs a set of possible tokens with different probabilities, and a sampler picks from the top possibilities. Temperature determines how spiky its pick is; at zero it’ll always pick the top option.