[simonw]

In this case being "bad at randomness" isn't because it was trained on text from humans who are bad at randomness, it's because asking a computer system that doesn't have the ability to directly execute a random number generator to produce a random number is never going to be reliable.

[brigadier132]

My question was about the scenario if it was trained on this kind of query with good data.

It would be interesting to see if it could generalize at all. I'm pretty certain if you trained it specifically on

"Generate a random number from 0 to 100" and actually give it a random number from 0 to 100 and give it billions of such examples it would be pretty effective at generating a number from 0 to 100. Wouldn't each token have equal weighted probability of appearing?

[jdiff]

Sorta, not really. Neural networks are deterministic in the wrong ways. If you feed them the same input, you'll get the same output. Any variation comes from varying the input or randomly varying your choice from the output. And if you're randomly picking from a list of even probabilities, you're just doing all the heavy lifting of picking a random number yourself, with a bunch of kinda pointless singing and dancing beforehand.