[tehsauce]

There has been some good research published on this topic of how RLHF, ie aligning to human preferences easily introduces mode collapse and bias into models. For example, with a prompt like: "Choose a random number", the base pretrained model can give relatively random answers, but after fine tuning to produce responses humans like, they become very biased towards responding with numbers like "7" or "42".

[robwwilliams]

I assume 42 is a joke from deep history and The Hitchhiker’s Guide. Pretty amusing to read the Wikipedia entry:

https://en.wikipedia.org/wiki/42_(number)

[sedatk]

Douglas Adams picked 42 randomly though. :)

[robertlagrant]

Not at all. It was derived mathematically from the Question: What do you get if you multiply six by nine?

[eterm]

It was just a joke, and doubly so the fact it "works" in base 13.

It was written as a joke in fairly ramshackle radio play. He had no idea at the time of writing it that the joke would connect so well and become it's own "thing" and dominate discourse of the radio series and novels to come.

It's not a joke about numbers, it's a linguistical joke, that works well on radio, something that HHGTG is stuffed full of.

https://scifi.stackexchange.com/questions/12229/how-did-doug...

[HappMacDonald]

That's not the question, though. Everybody knows that the question is the one posed to Mister Turtle and Mister Owl which neither of them can find the answer to.

[sedatk]

I stand corrected in base 13.

[moffkalast]

It's very funny that people hold the autoregressive nature of LLMs against them, while being far more hardline autoregressive themselves. It's just not consciously obvious.

[antihipocrat]

I wonder whether we hold LLMs to a different standard because we have a long term reinforced expectation for a computer to produce an exact result?

One of my first teachers said to me that a computer won't ever output anything wrong, it will produce a result according to the instructions it was given.

LLMs do follow this principle as well, it's just that when we are assessing the quality of output we are incorrectly comparing it to the deterministic alternative, and this isn't really a valid comparison.

[absolutelastone]

I think people tend to just not understand what autoregressive methods are capable of doing generally (i.e., basically anything an alternative method can do), and worse they sort of mentally view it as equivalent to a context length of 1.

[aidos]

Why is that? Whenever I’m giving examples I almost always use 7, something ending in a 7 or something in the 70s

[p1necone]

1 and 10 are on the boundary, that's not random so those are out.

5 is exactly halfway, that's not random enough either, that's out.

2, 4, 6, 8 are even and even numbers are round and friendly and comfortable, those are out too.

9 feels too close to the boundary, it's out.

That leaves 3 and 7, and 7 is more than 3 so it's got more room for randomness in it right?

Therefore 7 is the most random number between 1 and 10.

[LoganDark]

That's all well and good, but 4 is actually the most random number, because it was chosen by fair dice roll.

[HappMacDonald]

Also because humans are biased towards viewing prime numbers as more counterintuitive and thus more unpredictable.

[wruza]

Last time I hallway tested it, people couldn’t tell what prime numbers are, and to my surprise even the ones with tech/math-y background forgot it. My results were something 1.5/10 (ages 30+-5) and I didn’t go to cabinets where I knew there are zero chances.

[HappMacDonald]

But there's a difference between "knowing what the formal definition is" and "having a feeling that a number is somehow unique due to it's indivisibility".

[da_chicken]

The theory I've heard is that the more prime a number is, the more random it feels. 13 feels more awkward and weird, and it doesn't come up naturally as often as 2 or 3 do in everyday life. It's rare, so it must be more random! I'll give you the most random number I can think of!

People tend to avoid extremes, too. If you ask for a number between 1 and 10, people tend to pick something in the middle. Somehow, the ordinal values of the range seem less likely.

Additionally, people tend to avoid numbers that are in other ranges. Ask for a number from 1 to 100, and it just feels wrong to pick a number between 1 and 10. They asked for a number between 1 and 100. Not this much smaller range. You don't want to give them a number they can't use. There must be a reason they said 100. I wonder if the human RNG would improve if we started asking for numbers between 21 and 114.

[thfuran]

People also tend to botch random sequences by trying to avoid repetition or patterns.

[foota]

Okay, this is a nitpick, but I don't think ordinal can be used in that way. "Somehow, the ordinal values of the range seem less likely". I'd probably go with extremes of the range? Or endpoints?

[da_chicken]

Nope I just mixed up a rephrase. I originally said "ordinal extremes" and meant to say "extreme values". I replaced the wrong word.

[Ethee]

Veritasium actually made a video on this concept about a year ago: https://www.youtube.com/watch?v=d6iQrh2TK98

[d4mi3n]

My guess is that we bias towards numbers with cultural or personal significance. 7 is lucky in western cultures and is religiously significant (see https://en.wikipedia.org/wiki/7#Culture). 42 is culturally significant in science fiction, though that's a lot more recent. There are probably other examples, but I imagine the mean converges on numbers with multiple cultural touchpoints.

[Jensson]

I have never heard of 7 being a lucky number in western culture and your link doesn't support that. 3 is a lucky number, 13 is an unlucky number, 7 is nothing to me.

So I don't think its that, 7 is still a very common "random number" here even though there is no special cultural significance to it.

[taormina]

Have you heard of Las Vegas? The 777 being the grand prize? Maybe it is not universal to all of western society but I have never before today heard of a culture where 3 was the lucky number. The USA’s culturally lucky number is absolutely 7.

[Jensson]

I don't live in USA, the west includes Europe. 7 is maybe a lucky number in USA but not where I live. So I think that would be more of an American thing than a western thing maybe. Or maybe its related to some parts of Christianity but not others.

[smelendez]

I’ve heard 7 as lucky all my life in the US and it’s mentioned in the Wikipedia page for 7. I think if you asked the average English-speaking American to name a number thought of as lucky most people would say 7.

Lucky Seven/Lucky Number Seven is also just a common phrase in American culture. There’s even a Wikipedia page of things called Lucky [Number] Seven. https://en.m.wikipedia.org/wiki/Lucky_7

[Vox_Leone]

>>I have never heard of 7 being a lucky number in western culture and your link doesn't support that. 3 is a lucky number, 13 is an unlucky number, 7 is nothing to me.

Some sustain that 7 is the God's number, stemming from "God created the world in seven days"[1]

Also, _"According to some, 777 represents the threefold perfection of the Trinity"_ [2]

[1] https://www.wikihow.com/What-Does-the-Number-7-Mean-in-the-B...

[2] https://en.wikipedia.org/wiki/777_(number)

[kbenson]

It's definitely used in slot machines as a lucky number. Which came first I'm not sure (but I suspect from a sibling comment in the same thread it's based on perceived commonality and primeness historically and became "lucky" in the past because of that).

[II2II]

While I have never heard of someone referring to 7 as a lucky number, 7 is the most common sum of two rolled dice. So I can see how people would regard it as a lucky number. Along the same lines, I assume that someone who mentions 42 as a random number has at least some interest in science fiction.

[dontseethefnord]

You must be living under a rock if you’ve never heard of 7 as a lucky number.

[yashap]

Hmm really? Even on the Wikipedia page for 7 (https://en.m.wikipedia.org/wiki/7), one of the first things it says is “7 is often considered lucky in Western culture and is often seen as highly symbolic.” And FWIW you can see the Wikipedia edit history, that isn’t a recent edit, nobody here is messing with it :)

“Lucky Number 7” is a common phrase, there was even a popular movie that played on this, “Lucky Number Slevin” (https://m.imdb.com/title/tt0425210/). It’s one of the first numbers I’d think of as a “lucky number.”

[Geezus_42]

It's associated with Christ by several Protestant Christian denominations.

[d0liver]

I like prime numbers. Non-primes always feel like they're about to fall apart on me.

[mynameismon]

Can you share any links about this?

[Shorel]

They choose 37 =)

[thechao]

Which is weird, because I thought we'd all agreed that the random number was 4?

https://xkcd.com/221/