[SkyBelow]

As a once off, with the same context, it giving the same answer doesn't surprise me. What I'm wondering if the behavior when it keeps being asked for another response with the previous responses fed back into it. In this case, a human would see they are doing the 80% 'too much' and decide to do the 20% to balance it out. That isn't actually good and shows they still aren't operating off a random probability, instead they are emulating their perception of what a random probability would look like.

Given this sort of situation to an LLM instead, is the expectation for it to give the most likely answer continuously, to act like a human and try to emulate a probability, or to do something different from either of the two previous options?

Edit: Just tried an attempt with copilot, having it produce a random distribution of two different operations. I had it generate multiple operations, either adding or subtracting 1 each, with an 80/20 split. It did four adds, one minus on repeat.