Hacker News new | ask | show | jobs
by ilaksh 1276 days ago
If you go to the OpenAI playground and turn down 'temperature' to 0 it seems to not BS at all as far as I can tell.
1 comments

Temperature is presumably referring to sampling the output probabilities. With a temperature of 0 it'll be giving you the very highest probability continuation, while with increasingly higher temperatures it'll be sampling from the possible continuations to provide more variety.

In other words, the temperature is controlling the variety of output, but of course doesn't affect what was fed into it in the first place. As the saying goes, Garbage-In, Garbage-Out .. even with a temperature of zero it's still going to be bullshitting since "predict next word" (language model) is fundamentally a bullshitting technology - just keep on spewing out words regardless of meaning.