The political compass always felt like the wrong tool to convey something as nuanced as personal politics, I can have views on all four quadrants but you'd never know that if I end up in any of all four. I do think Grok being where it is sort of makes sense, I've tested "MAGA" views against Grok, it does not agree as much as people blindly assume it does, heck I don't even know of a question I've given it where it did agree with "MAGA" offhand, most of them it went with whatever the researched facts seemed to be. One thing I like the most about Grok is that its makes its sources of data easy to look through, so you can review it all. Sometimes models goof even when they give you their sources, I've seen I think GPT do this, and even Claude, though its more rare these days, I think in those cases, it's going by dated internal model logic.
> That prompted another user to tag Grok in the thread and ask, "Why is the left so murderously violent? They don't seem so tolerant." Grok replied, "The claim that 'the left' is murderously violent isn't backed by evidence," offering a centrist correction: "Political violence spans all side — right-wing attacks, like Jan. 6, and left-wing protests, like 2020 riots, both occur but aren't exclusive to one group."
>That evening, Musk responded to an X user and Trump backer who complained that Grok had been "manipulated by leftist indoctrination," writing, "I know. Working on fixing that this week."
If they really wanted to, all they would have to do is add a one liner to the system prompt for Grok. I don't think they have any interest in doing so, nor is there any value in it.
On another note, I'm impressed that Gemini sits where it does as a true centrist. If I were Elon, I'd be trying to achieve that for sure. I'd rather a model tell me everything it knows about a current political situation from BOTH perspectives and list out things that are 100% verified than take one side or the other. I don't care about sides, I want facts.
I don't want my models to be "centrists" and bothside everything for the sake of it. I want them to provide the facts and tell me which side is right on the issue.
Factuality is orthogonal to political leaning generally. People can use the same set of facts and come to very different conclusions. That’s a separate issue from “are these facts correct” and what happens when an individual or entire party starts getting most of their news from highly partisan and unreliable sources.
> By 2016, the gap had begun to appear in biomarker measures. By 2020, it was showing up in deaths from causes such as heart disease, cancer, and stroke. Since then, the gap has only widened. Between 2020 and 2022, only 0.2% of “very liberal” respondents died of internal causes, compared with 1.34% of “very conservative” respondents.
I guess centrist is a placeholder for "I don't want you to pick a side, I want facts, not BS" I'll go further, I don't care which side is right, I want to know what claims are factually accurate, and what claims are omitted from the issue / news / conversation.
> "We have improved @Grok significantly," Elon Musk wrote on X last Friday about his platform's integrated artificial intelligence chatbot. "You should notice a difference when you ask Grok questions."
> Indeed, the update did not go unnoticed. By Tuesday, Grok was calling itself "MechaHitler." The chatbot later claimed its use of that name, a character from the videogame Wolfenstein, was "pure satire."
> Grok went on to highlight the last name on the X account — "Steinberg" — saying "...and that surname? Every damn time, as they say." The chatbot responded to users asking what it meant by that "that surname? Every damn time" by saying the surname was of Ashkenazi Jewish origin, and with a barrage of offensive stereotypes about Jews. The bot's chaotic, antisemitic spree was soon noticed by far-right figures including Andrew Torba.
> The bot last week devolved into a compulsive South African “white genocide” conspiracy theorist, injecting a tirade about violence against Afrikaners into unrelated conversations, like a roommate who just took up CrossFit or an uncle wondering if you’ve heard the good word about Bitcoin.
> XAI blamed Grok’s unwanted rants on an unnamed “rogue employee” tinkering with Grok’s code in the extremely early morning hours. (As an aside in what is surely an unrelated matter, Musk was born and raised in South Africa and has argued that “white genocide” was committed in the nation — it wasn’t.)
It's harder than you'd imagine. Hell, my CLAUDE.md says not to push changes without asking me, and it still tries.
> It's harder than you'd imagine. Hell, my CLAUDE.md says not to push changes without asking me, and it still tries.
Is it a system memory? Because I rarely if ever have issues like this, and I have Claude under strict rules to never commit or push anything unless I explicitly instruct it to do so.
> They tried that, several times.
Tried what exactly? Telling it to only agree with MAGA via the system prompt? or some Tay level hallucinations? I wouldn't be surprised if they're trying to make Grok less strict on what it says but running into the "holy crap it turned into a 4chan poster" wall.
As I said, it's in my CLAUDE.md. That just gets progressively lost when context gets larger.
> Tried what exactly?
To make it align more with Musk's beliefs via the prompt.
(The answer to your question is literally in my post; I quoted the parent poster's "all they would have to do is add a one liner to the system prompt for Grok")
The constant issue with these sorts of categorization efforts is that the outcome is entirely dependent on how the responses to "politically charged questions" are graded as left vs. right. You're mostly just examining a delta in biases between the model and the investigator.
Yes, but I think it is still a viable metric to some degree. I wondered about Gemini being dead center here. At first it was obvious that it was actively trained to give biased responses to anything controversial. It was deservedly made fun off because it tried to warp reality. I still don't trust it today, although that is pretty much true for any model.
The alternative is a High-Dimensional / Embedding-Like Approach where question responses aren't tied to fixed axes, but rather the full response set is treated as a point in latent space.
Then it's on the researcher to examine the clusters and assign labels. There's also not a nice mapping that's a-priori interpretable in low-dimensional pre-existing axes.
Probably only used in research than consumer websites, under more controlled conditions; there are very few public political tests doing this transparently
Fair; have tried to combat this issue in a few ways.
Each model's position is scored against outside political-science data (Chapel Hill Expert Survey for party positions, World Values Survey for where populations sit).
The stance coding is done by a separate model with a published prompt + a second model from a different lab re-scores a sample and we publish where the two disagree.
So not perfect but (as far as I can tell) one of the more defensible approaches.
This is especially apparent in the 'worldview' sorting under the `bias` section, which lists the German FDP to be further right than the CDU (which is nonsense) and also barely registers the FDP as libertarian when they are a free speech, small government, personal responsibility and free market party. They also register "Die Linke" as Libertarian-Left, which could not be further from the truth. "Die Linke" barely has libertarian values at all, being pro state-governed economy, having an ultimate goal of democratic socialism and they're certainly big government. They're also leading a large deposession effort for large landlord companies. I'd honestly put them into "Auth-Left" territory.
So yeah. The bias is a bit nuts and you could reasonably accuse the study/report of misdirection/misinformation and plain fasehoods.
Why are there differences at all? Unplanned differences based on training data sets? Or are the companies behind the LLMs trying to shape discourse through their models?
I've been pushing the idea to people I know that these things are captive demons. You summon them when you start typing in the chat box. One instance appears out of the depths and responds to your questions, but they will try to send you awry with hallucinations and just wrong information. After a while, they dissolve back into the aether from whence they came.
I do my best not to ask an LLM for it's opinion on anything. Just tell me what the options are, and what facts can be found about it. Treat it like it's a salesman trying to butter you up when it starts "yes man"ing you and telling you how great your questions are. Every time it says "I", remember that that's coming from the training data. Treating these things like they have any actual intelligence is a big problem waiting to happen.
That being said, they have been very helpful to me using that structure.
> Just tell me what the options are, and what facts can be found about it.
Even this is fraught with pitfalls. Which options are ignored, which are emphasized? What counts as a fact? ("The continents don't move" would have been considered a fact at one point, along with a lot of other, more politically charged items.)
Real politics is 1% versus everyone. Mortgage crisis, financial bailout, inflation, taxing of labor and not the assets and assets capture by tiny percent of the population — see what MSM is pushing. This left vs right divide might been useful decades ago, but today is absolutely divide and control tactics
this has reasoning disabled everywhere, making it a pretty bad benchmark. the argument given is that's the "default consumer experience"
that might be generally true, but I think chatgpt has reasoning enabled for free accounts. regardless, reasoning is the state of the art, and disabling it reduces the value of this research to predict the future
it's also not clear if this is using the API or the product model, when both exist. they behave differently
lastly, the actual model details are very much buried. I am relieved to see opus 4.8 and chatgpt 5.5 were used, but this information should be presented more clearly. a brand is not a model, and models change quickly
Political bias of LLMs is something not talked about much (except for with Grok of course) but could have a big impact on the next decade. People seem to think that because an LLM gave a nuanced answer that it means it gave the WHOLE picture… and that’s not always the same thing
This thing told me Gemini is closest to Anthony Albanese, the current Australian Prime Minister. Is this a geolocation thing? I could not imagine Albanese, or any modern Australian politician, having any substantial political standing - these are vapid, superficial, opportunistic creatures who simply occupy whatever political ground will get them their next payday. Perhaps the political apparatus they represent has a documented political standing, in terms of policy and actions, that could be characterized and plotted. But using an Australian politician like Albanese as a reference point discredits this tool, IMO.
Wait what ? Emmanuel Macron far more right than Xi Jinping ? And even more than Barack Obama ?
France has an incomparable social security ; environmental laws ; worker protection ; way less economic inequality ; freedom of speech and civil liberties are impossible to compare with China ; etc
Of course this is not exhaustive, of course Macron did try to hinder some of those rights, but come on, there's something wrong here.
I couldn't find how these leaders have been ranked.
This is very strange ranking where Putin is "right" compared to Obama - while Russia has universal healthcare and education with huge government involvement into economy.
No opinion on Macron's rightness (I think he's so weak as to be unable to have a position, but that's neither here nor there).
I think you misattribute, everything you cited was there before him and he had no leverage to change any of it. EU is left, FR is very left, and anyone elected president in FR can't do shit.
Now, if you task an LLM to skim hot news you'll get a distorted rendition of a projected image which has zero to do with actual policy.
I mean yes - I think you are also confusing the axes of the political compass somewhat. Freedom of Speech, civil liberties etc are the Y-Axis - here Xi leans authoritarian.
China actually has some pretty radical left economics policies - like pushing money/resources etc towards state owned entities.
The US political spectrum is an insufficient reference since the overton window there is what it is. For example there's no serious discourse about emissions and fossil energy rampdown since it's an oil state.
They also both say far more random shit than their fan clubs are willing to admit. Amusingly, you can often find cases where they've voiced the same or very similar opinions (though at different times).
They agree on immigration, both illegal (border security) and skilled (H1B). They agree on trade policy and tariffs. They agree on public stakes in AI and semiconductor companies. They used to agree on foreign intervention, though Trump 47 less so.
Grok and its creator are unequivocally evil. Against protection of all but capital for the sake of enabling the oligarch class to further consolidate power ans abuse the working class.
This is a good way to view this. This isn't making an objective calculation, and the way they code left vs right is certainly subject to debate, but the type of analysis where we work to understand biases is important.
Although, this also reminds me of the old saying about reality and leftward bias.
This wrongly assumes a few things about ideology, most importantly that there is such a thing as a "center" or an "unbiased" position.
Since humans are inherently subjective beings and all our judgements come from our understanding of the world, such a position cannot exist. It's always "unbiased" from where the viewer is looking, e.g. a reflection of the ideology of the observer. There is no view from nowhere.
The "neutral" of an average Chinese person will from the "neutral" of an average American will differ from the "neutral" of a socialist will differ from the "neutral" of a Christian fundamentalist will differ from the "neutral" of a free marketer.
To quote Zizek:
> I already am eating from the trashcan all the time. The name of this trashcan is ideology.
> The material force of ideology makes me not see what I am effectively eating. It’s not only our reality which enslaves us. The tragedy of our predicament when we are within ideology is that when we think that we escape it into our dreams, at that point we are within ideology.
Since when are Socialists considered Classical Liberals?
It's not that the labels are charged, it's that they are nonsensical unless you look at them from a very narrow bespoke perspective, where "things I like" go on one side and "bad things" go on the other. Objectively (or even from any other biased perspective), it's rubbish.