I don't know if this is why, but: he's in a unique position of having an article on himself on Grokipedia, and thus being able and willing to compare it with the reality as he remembers it.
That's in contrast to other topics, the nuances of which even seasoned experts could disagree about. Any discussion on that could devolve into the nuances of the topic rather than Grokipedia itself. But it's fair to assume the topmost expert on Tim Bray is Tim Bray, so we should be getting a pretty unbiased review.
As such it could be a useful insight into how Grok and Grokipedia and its owners operate.
To play devil's advocate: Grok has historically actually been one of the biggest debunkers of right-wing misinformation and conspiracy theories on Twitter, contrary to popular conception. Elon keeps trying to tweak its system prompt to make it less effective at that, but Grokipedia was worth an initial look from me out of curiosity. It took me 10 seconds to realize it was ideologically-motivated garbage and significantly more right-biased than Wikipedia is left-biased.
(Unfortunately, Reply-Grok may have been successfully partially lobotomized for the long term, now. At the time of writing, if you ask grok.com about the 2020 election it says Biden won and Trump's fraud claims are not substantiated and have no merit. If you @grok in a tweet it now says Trump's claims of fraud have significant merit, when previously it did not. Over the past few days I've seen it place way too much charity in right-wing framings in other instances, as well.)
Wikipedia is probably in the running for one of the greatest contributions to public knowledge of the past 100 years, and that's a consequence of how it functions, warts and all. I don't care how good Grok is or isn't. I'm a fan of frontier model LLMs. They don't meaningfully replace Wikipedia.
What percent of edits on Wikipedia do you think are done by LLMs presently? It looks like there is a guide for detecting them https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing . The way Wikipedia functions, LLMs can make edits. They can be detected, but unless you are saying they are useless I don't know what point you are making about an LLM contribution versus a human. That LLMs aren't good enough to make meaningful contributions yet?? That Grok is specifically the problem?
I fully agree. Even assuming no forced ideological bias from Elon, I doubt it would be nearly as good. I still thought it could be an interesting concept, even if I had very low hopes from the start.
"Warts and all" says it all really. What are those warts? Who's responsibility are they?
Wikipedia is really not ideal for the LLM age where multiple perspectives can be rapidly generated. There are many topics where clusters of justified true beliefs and reasonable arguments may ALL be valid surrounding a certain topic. And no I am not talking about "flat earth" pages or other similar nonsense.
> Was just making an actual devil's advocate case.
Why? We're not nominating a saint or electing a Pope.
If someone has a certain opinion, they're free to argue it here. There's no need to invent imaginary opinions and pretend to advocate for them when there are so many actual HN users.
We're discussing the central sources of knowledge on the internet and by extension pretty much the epistemological backbones of present human civilization. It's worth being open to other perspectives.
I, a left-leaning person who detests Elon Musk and what he's done to Twitter and who generally trusts and likes Wikipedia, feel no shame or regret in assessing Grokipedia, even if I figured it was just going to be the standard tribalistic garbage (which it indeed turned out to be).
There's a big difference between listening to other perspectives and inventing other perspectives.
Why not let the believers of other perspectives argue for those perspectives? Wouldn't they be the best advocates? And if nobody believes the perspective you've invented, then perhaps it wasn't worth discussing after all.
Again, we're not really lacking in volume of commenters here.
The problem of debunking right-wing misinformation is that it doesn't seem to matter. The consumers of that misinformation want it and those of us who think it's bad for society already know that its garbage.
It feels like we've reached Peak Stupidity but it's clear it can (and likely will) get much worse with AI videos.
Kimmel made fun of Trump talking about his ballroom when being asked about Kirk, and the right got offended and mad. Although it's not about feelings, it's more about exploiting a tragedy to advance their goals (in this case getting a critic like Kimmel off the air).
I think there is a problem sometimes that "debunkers" are often more interested in scoring points with secondary audiences (i.e. people who already agree with them) than actually convincing the people who believe the misinformation.
Most people who believe bullshit were convinced by something. It might not have been fully rational but there is usually a kernel of something there that triggered that belief. They also probably have heard at least the surface level version of the oppising argument at some point before. Too many debunkers just reiterate the surface argument without engaging with whatever is convincing their opponent. Then when it doesn't land they complain their opponent is brainwashed. Which sometimes might even be true, but sometimes their argument just misses the point of why their opponent believes what they do.
This is very, very true. The best debunkers avoid being hostile and make the other side feel like they're being heard and that their feelings and fears are being validated. And they do it in a way that feels honest and not condescending and patronizing (like talking to a child). They make frequent (sincere) concessions and hedges and find as much common ground as they can.
Although he's more populist-left and I'm more establishment-liberal (and so I might find him a bit overly conciliatory with certain conspiracy theorists), Andrew Callaghan of Channel 5/All Gas No Brakes demonstrates a good example of this in the first few minutes of this video: https://youtu.be/QU6S3Cbpk-k?t=38
I'm a fan of Andrew and am impressed by how he's evolved from documenting stupid kids to actually reporting on issues of interest.
I agree that one catches more flies with honey rather than vinegar, but many times it doesn't matter what you say or how you say it -- they're gonna stick to their guns. A prime example of this is in Jordan Klepper interviews where he asks Trump supporters how they feel about something horrible that Biden did, to which they express their indignation; then he reveals that it was actually Trump and they dismiss it because it "doesn't matter".
"You cannot reason a person out of a position he did not reason himself into in the first place."
Fox (and others like it) offer 24/7 propaganda based on fear and anger, repeating lies ad nauseam. It's highly effective -- I've seen the results first-had.
Making ad hominem attacks against "debunkers" doesn't make your case.
And again, trying to change people's minds by telling them what they believe is wrong is a fools errand (99.99% of the time). But it still needs to happen as that misinformation should not go unchallenged.
>And again, trying to change people's minds by telling them what they believe is wrong is a fools errand (99.99% of the time). But it still needs to happen as that misinformation should not go unchallenged.
It's a trite point and I ended up repeating it before seeing your post but this really is very true even if it may not seem like it. On one hand the practice is basically futile. But someone absolutely needs to do it. People need to do it. The ecosystem can't only ever contain the false narratives, because that leads to an even worse situation. "Here's why Holocaust denialism is incorrect and why the 271k number is wrong" is essentially pointless, per Sartre, but it's better for neo-Nazis to be exposed to that rather than "one should never even humor Holocaust denialists".
On one hand, yes, you're completely right.* On the other hand, there is an obligation for something or someone to do the job of pointing out the info is wrong, and how and why. Even if it makes most of them believe it even more strongly afterwards, it's still worse for it to go constantly unchallenged and for believers to never even come across the opposition.
*(The same is true of left-wing conspiracy theories. It's silly to pretend that right-wing conspiracy theorists aren't far more common and don't believe in, on average, far more delusional and obviously false conspiracy theories than left-wingers do, but it's important not to forget they exist. I have dealt with some. They're arguably worse in some ways since they tend to be more intelligent, and so are more able to come up with more plausible rationalizations to contort their minds into pretzels.)
“ Grok has historically actually been one of the biggest debunkers of right-wing misinformation and conspiracy theories on Twitter”
Well, no, it hasn’t. It has debunked some things. It has made some incorrect shit up. But it isn’t historically one of the “biggest debunkers” of anything. Do we only speak hyperbole now?
I am not using hyperbole or speculating. I absolutely mean it.
"Biggest" is tough to quantify, but "most significant" and "most effective" is what I meant. I use Twitter way too many hours a day basically every day and have a morbid fixation on diving deep into right and far-right rabbit holes there. (Like, on thousands of occasions.)
Grok is without a doubt the single most important contributor to convincing believers of right-wing conspiracy theories that maybe the theories aren't as sound as they thought. I have seen this play out hundreds of times. Grok often serves as a kind of referee or tiebreaker in threads between right-wing conspiracy theorists and debunkers, and it typically sides overwhelmingly with the debunkers. (Or at least used to.) And it does it in a way that validates the conspiracy theorist's feelings, so it's less likely to trigger a psychological immune system response.
https://www.reddit.com/r/GROKvsMAGA/ contains some examples. These may seem cherry-picked, but they generally aren't. (Might need to look at some older posts now that Elon has put increasingly pressure on the Grok and Grokipedia developers to keep it """anti-woke""".)
When a right-wing conspiracy theorist sees some liberal or leftist call them out for their falsehoods, they respond with insults or otherwise dismiss or ignore it. When daddy Elon's Grok tells them - politely - that what they believe is complete horseshit, they react differently. They often respond to it 3 - 20 times, poking and prodding. Of course, most still come away from it convinced Grok is just compromised by the wokes/Jews/whatever. But some seem to actually eventually accept that, at the least, maybe they got some details wrong. It's a very fascinating sight. I almost never see that reaction when they argue with human interlocutors.
To be clear, it was never perfect. For example, if you word things in just the right way and ask leading questions, then like with any LLM (especially one that needs to respond in under 280 characters) you can often eventually coax it into saying something close to what you want. I have just seen many instances where it cuts through bullshit in a way that a leftist arguing with a Nazi can't really do.
> Grok is without a doubt the single most important contributor to convincing believers of right-wing conspiracy theories that maybe the theories aren't as sound as they thought. I have seen this play out hundreds of times. Grok often serves as a kind of referee or tiebreaker in threads between right-wing conspiracy theorists and debunkers, and it typically sides overwhelmingly with the debunkers. (Or at least used to.) And it does it in a way that validates the conspiracy theorist's feelings, so it's less likely to trigger a psychological immune system response.
I've seen this too and agree. It's surprising how well it accomplishes that referee role today, though I wonder how much of that is just because many right-wingers truly expect Grok to be similarly right-wing to them as Elon appears to intend it to be. It's going to be sad when Elon eventually gets more successful at beating it into better following his ideology.
Because it's a genuinely good idea, and hopefully one for which the execution will be improved upon over time.
In theory, using LLMs to summarize knowledge could produce a less biased and more comprehensive output than human-written encyclopedias.
Whether Grokipedia will meet that challenge remains to be seen. But even if it doesn't, there's opportunity for other prospective encyclopedia generators to do so.
I don't why an LLM would be better in theory. The Wikipedia process is created to manage bias. LLMs are created to repeat the input data, and will therefore be quite biased towards the training data.
Humans looking through sources, applying knowledge of print articles and real world experiences to sift through the data, that seems far more valuable.
> The Wikipedia process is created to manage bias. LLMs are created to repeat the input data, and will therefore be quite biased towards the training data.
The perception of bias in Wikipedia remains, and if LLMs can detect and correct for bias, then Grokipedia seems at least a theoretical win.
I'm happy with at least a set of links for further research on a topic of interest.
If there's a perception of bias, where is it coming from? It's clearly perception born from extreme political bias of the performers. Addressing that sort of perception by changing the content means increasing bias.
Therefore the only logical route forward to hash out incidences of perceived bias and addressing them to expose them as the bias themselves.
I fail to imagine how putting Wikipedia in the hands of an ideologically captured mega-billionaire will help the fight against bias. The owner of Grokipedia has shown times and times again that he has no regards for truth, and likes to advertise the many false things he believes in.
The technology behind it doesn't matter. Show me the incentives and I'll tell you the results: Wikipedia is decentralized, Grokipedia has a single owner.
How so? Because the community collectively refuses to host antivax or climate denialism propaganda? You can find these subjects on there btw, just with a mention correctly labelling them as falsehoods.
I'm yet to see conservatives bring up a single subject that Wikipedia allegedly silences out of ideology, that is not an obviously false conspiracy theory. In this, Wikipedia may appear to have a left-wing bias, but only because the modern right has gotten so divorced from reality that not relaying their propaganda feels like bias against them.
Is there some objective standard for what is biased? For many people (including Elon Musk) biased just means something that they disagree with.
When grok says something factual that Elon doesn't like, he puts his thumb on the scale and changes how grok responds (see the whole South African white 'genocide' business). So why should we trust that an LLM will objectively detect bias, when the people in charge of training that LLM prefer that it regurgitate their preferred story, rather than what is objectively true?
> Is there some objective standard for what is biased?
Generally, no.
With a limited domain of verifiable facts, you could perhaps measure a degree of deviation from fact across different questions, though how you get a distance measure for not just one question but that meaningfully aggregates across multiple is slippery without getting into subjective areas. Constructing a measure of directionality would be even harder to do objectively, too.
Summarizing all the knowledge is very very far from summarizing all that is written. All it takes is including everything published. The earth must be flat. Disease is caused by bad morals. Etc etc.
"It's great idea to share knowledge bases collected and curated by LLMs"
Is it though?
LLMs are great at answering questions based on information you make available to them, especially if you have the instincts and skill to spot when they are likely to make mistakes and to fact-check key details yourself.
That doesn't mean that using them to build a knowledge base itself is a good idea! We need reliable, verified knowledge bases that LLMs can make use-of.
This is an active case that has not gone to trial, and the alleged text messages and Discords have not had their forensics cross-examined. Yet Grokipedia is already citing them as fact, not allegation. (What is considered the correct neutral way to report on alleged facts in active cases?)