[vkadfa4]

Thanks for writing this, I'm reading it now.

my comment before knowing the information you provide in the text is simple, these models are trained over some texts downloaded from Internet, there are some real "trends" you can easily spot reviewing yourself some subreddits, i.e. likely many american citizens commenting in a guns related subreddit have certain opinions which probably aren't shared for other people in a LGBT+ subreddit elsewhere.

It could be matter of choice during the assembly of the datasets used for the training, simply. They could have included some subreddits comments from here and there, but some richer texts were found in some subs, so they included more text from there, hence the LLM ended having "biases" related to those comments, because some ideas are more "heavy" in their internal values than the other ideas from other people, which are less - stadistically -represented in the dataset.

It looks like you could really introduce any "bias" you choose into the "core internals" of any LLM. This could be an accident or not.