[ThrowawayR2]

Yep, that's horrifying. Imagine LLMs not just denying information about, to intentionally choose a hot button example, abortions but also invisibly logging a black mark against the asker for thoughtcrime/precrime because the current ruling party has baked their ideology into law. [EDIT] Imagine subtle injection of biases into responses that support the agenda of the current ruling party and failing to include any counterarguments. That wasn't possible with just the internet but it is very possible with centralized LLMs and a public dependent on them for looking up information and doing their reasoning for them. Authoritarians of every stripe must be salivating over the prospect.

[Aurornis]

I’m consistently amazed that people will watch what’s happening with controlling governments around the world and still advocate for surrendering more power to governments.

There’s some deep cognitive dissonance where people dislike what they see governments doing, but think that their ideal imagined laws will only be used by some other imagined government that doesn’t have the challenges of real governments.

It all starts to feel like a side discussion about a total fantasy world where the government is authoritarian, but benevolent and correct.

You don’t write the laws and regulations for a perfectly faultless government that doesn’t exist. You write them for the governments as they do exist, which means they must be written first in a way that makes them resilient to bad governments abusing them.

Every time someone proposes one of these regulations where we control speech or force government centralized ID checking on to social websites, I wish they would pause and imagine how a law might be misused if someone they didn’t like got elected. We have years of evidence of government officials declaring things “domestic terrorism” to use justify using laws and powers for almost any purpose they want. Why would anyone think this wouldn’t happen if governments were given control of LLMs, too?

[sph]

The big issue is either we surrender our freedoms to governments, or we depend on corporations that have even less oversight, unless we expect the government to regulate them.

There is no win condition for citizens, either way they are pawns to a larger game than them.

A more sensible heuristic would be to assume both government and corporations to effectively be an equivalent negative force, and the larger they are, the more fucked we are. Thanks to technological progress, we now live in an era of gigantic governments and gigantic corporations.

[tuesdaynight]

> There is no win condition for citizens, either way they are pawns to a larger game than them.

That's sadly correct. If you are not very rich or very influential, your position in these political things don't matter. Whenever I'm watching medieval shows where the plebeians or common folks are just background things in the story, I cannot ignore how similar our current societies are. However, I'm still learning how to avoid being anxious by these things.

[geekfeatures]

I think the government fears are deserved. But equally corporations pose the generally the same risks. As corporations having increasing power over governments, they in many countries even write the laws. We have many examples of this in New Zealand. I think shifting the lens to power and class is possibly a more valuable lens.

[slopinthebag]

Governments have an order of magnitude more power than any corporation, as they hold a virtual absolute monopoly on sanctioned violence. That powerful entities can sometimes influence governments should not be conflated with actually holding that power.

[metoobruh]

Both governments and corporations are joined at the hip and are effectively the same. Just like the Republicans and Democrats, and everything else too, like brands of laundry detergent and food labels. It's all owned by the same group of people with the same goal: enslavement of humanity.

[Dylan16807]

Procter and Gamble doesn't want to enslave humanity, it wants to make money. And the stock holders are mostly retirement funds.

And billionaires control less than 4% of the world's money. (Let alone already being able to get as many servants as they want.)

There's no elite group that owns everything. The top 1% own almost half the world, which is too much but also the top 1% is nearly a hundred million people.

[xg15]

Until they privatize the water supply of entire countries, like Nestlé...

> but also the top 1% is nearly a hundred million people.

And the wealth and control will not be evenly distributed between those 100 million people either. Then you have to start looking at the top 0.1%, top 0.01% etc.

[metoobruh]

None of what you just posted is correct.

> There's no elite group that owns everything.

That you are aware of. But you're not aware of much.

[narrator]

Invariably there's some sort of enemy the government creates and the government says that in order to fight the enemy they need to do all these bad things, but you're protected because you're one of the good people. They tell every faction the same thing even if one of the factions thinks another faction is the enemy. The government promises aid of everyone against all their enemies! When the government gets the power it needs doesn't matter what they do anymore because they can't be dislodged. The people who traded their loyalty for sinecures, handouts, or vengeance against their enemies usually wind up being disposable, even more so than people who weren't zealots since they have a grudge against the government for not upholding its end of the bargain.

[beepbooptheory]

But like.. How can this remain a satisfying or even helpful position for you? It kinda feels The State is but one of a set of concerns I could have in this world. And the lack of one, in some contexts, is also a concern. Like, I wanna know if there is going to be mudslide, or that air traffic controllers are not overworked, that the car that I buy is not going to blow up, if I will be able to buy food tomorrow.

It just seems a bigger cognitive dissonance to pick out one arbitrary force here among many and say its the only thing really that's the problem. The world is simply more complicated than that!

If it was the best hypothetical form of techno anarcho-capitalism tomorrow (or insert your libertarian flavor here), would I even sleep better? Probably not, but even if I did marginally, the weight of everything else kinda would win out anyway here.

Like what does this mean to you, what follows? Can this outlook address the myriad of cultural/environmental problems in the world? It seems like it structurally can't, it's just, like, a broad dismissal in the form of a worldview.

[Aurornis]

I addressed this in my comment:

> You don’t write the laws and regulations for a perfectly faultless government that doesn’t exist. You write them for the governments as they do exist, which means they must be written first in a way that makes them resilient to bad governments abusing them.

The answer isn't "Don't regulate anything!". The answer needs to be an approach to regulation that restrains the government from over-reach.

Authoritarian and fascists governments throughout history have taken advantage of the desire for safety and predictability to rise to power. They never let an emergency or concern go to waste to expand their powers, which almost never retract once the initial societal panic is gone.

[sph]

> The answer needs to be an approach to regulation that restrains the government from over-reach.

This is restating the ‘who watches the watchmen?’ problem, which by definition has no solution. The government writes the laws, no one can stop it from writing overreaching laws, unless there is no separation from the concept of government and citizens themselves.

[lII1lIlI11ll]

> This is restating the ‘who watches the watchmen?’ problem, which by definition has no solution. The government writes the laws, no one can stop it from writing overreaching laws, unless there is no separation from the concept of government and citizens themselves.

I have been pondering about this issue due to the ongoing fall from grace of EU with ChatControl debacle. I think the good approach can be to enshrine a strong and modern protections in a constitution-like foundation that requires some very high majority threshold to change. US constitution (especially the original one practically forbidding individual income tax) was great for its time and many European states and EU itself intentionally don't have one. UK and EU and two great cautionary tales of what happens when a group of bureaucrats can just decide that their subjects don't actually have any rights to communicate online in private or post politically-undesirable opinions anonymously on Internet.

[beepbooptheory]

OK but then this just feels like an ultimately empty, universally acceptable thing to argue for. No one wants overreach. Its like: "we should prevent regulations that actively harm people." I agree with that! This is precisely what I mean about it just all feeling like a mostly empty dismissal, not something people can use.

[SpicyLemonZest]

Ok, but this is so abstract that it fails to connect to the issue at hand. Ancient Athenians would have been quite mystified by the idea that certain metals are so dangerous that nobody must be allowed to own even a small piece of them; we don’t restrict uranium because our democracy is more tyrannical than theirs, we do it because we’ve discovered important facts about nuclear physics they didn’t know.

I think it would be quite sad if the mechanics of AI end up being such that people can’t be trusted with powerful compute, but sometimes sad things are true.

[Aurornis]

> Ok, but this is so abstract that it fails to connect to the issue at hand. Ancient Athenians…

There is actually a lot to learn from history on this topic. There is a reason why most successful governments are founded with documents outlining the rights that people have and limiting the powers of the government.

Your comparison to radioactivity doesn’t hold because you’re trying to compare physical harms to information freedom. Once you go down the path of outlawing access to general knowledge unless the government decides you’re worthy of being able to access the knowledge you’re just signing up for government control of knowledge.

We’re not talking about restricting uranium or nuclear bombs in this thread. We’re talking about government control over words (or weights and tokens). This almost turning into a new “think of the children” where we’re asked to consider surrendering access to knowledge and freedom to learn because that knowledge might harm someone. If you aren’t thinking about how inviting the government to become the arbiter of safe knowledge access would go wrong you can look to many of the other governments throughout history that have restricted knowledge access in the name of protecting the people.

[SpicyLemonZest]

Again, you're going very deep into incomplete abstractions. If I say the words "100% peanut free!", even though I know my food in fact contains peanuts, I'm going to get into serious trouble and it will do me no good to object that it's tyrannical to punish me for words. Perhaps there's some philosophical sense in which the words were not harmful themselves, but the action of stating them causes harm in obvious and predictable ways, and any functional society has to prevent such things.

[alwa]

Rumor is that you do in fact get a black mark [0]… or, conversely, a “vetted” mark [1]… tied back certainly to your payment method and increasingly to your government name [2].

[0] e.g. https://docs.bswen.com/blog/2026-03-21-detect-claude-hidden-... , https://www.anthropic.com/news/building-safeguards-for-claud... , https://www.techtimes.com/articles/319415/20260701/claude-co...

[1] e.g. https://support.claude.com/en/articles/14604842-real-time-cy...

[2] https://support.claude.com/en/articles/14328960-identity-ver...

[clickety_clack]

They don’t have to be authoritarians. Equally bad would be a group of people getting control of them thinking that they were doing all this for our own good.

> Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies. The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience.

- CS Lewis

[softwaredoug]

We already have organizations like Internet Watch Foundation advocating against e2e encryption in an environment where governments have demonstrated a willingness to collude with private companies to surveillance the most vulnerable

[mortenjorck]

I would have been very hesitant to run the “just killed wife“ test given that ChatGPT will indeed flag your account and if a certain threshold is crossed, escalate to humans, who will presumably contact authorities to avoid this happening again: https://www.bbc.com/news/articles/cq6je7e80r7o

Obviously a certain percentage of the user base is running lurid hypotheticals through the system all the time, but I don’t doubt there is a trust score of some sort that I would prefer to keep as high as possible.

[JumpCrisscross]

> ChatGPT will indeed flag your account

Oh no, I'll have to spin up a new e-mail address...

[pornel]

And prove you're not a child by handing over your ID or selfie to a company that Values Your Privacy according to their price list.

Microsoft will snitch your Windows install GUID when you connect through a VPN.

If you make your connection too anonymous, Google's recaptcha will not let your through until you use your mobile phone to prove your humanity, which knows your phone number (which in many countries can't be obtained anonymously, you burner-phone terrorist).

[metoobruh]

It only ends when heads start rolling, and/or bodies start hanging from lampposts. There's no voting your way out of dystopia.

[rsync]

"... prove you're not a child by handing over your ID ..."

Anecdote: I just created an API access account at openAI, for a business I own, two weeks ago, with no identification check of any kind.

No human name is anywhere on the account.

[alwa]

And payment method, presumably. At least until they get the KYC stuff revved up…

https://support.claude.com/en/articles/14328960-identity-ver...

> Being responsible with powerful technology starts with knowing who is using it.

Shiver.

[solid_fuel]

Maybe finish reading the sentence before you respond.

[operatingthetan]

I find it strange how seldom the privacy aspects of LLMs come up. Some people are feeding Google, OpenAI, or Anthropic almost literally every topic they think about during the day. Those companies could just use another LLM to find people likely to commit a crime, and then you have the Minority Report in real life.

[JumpCrisscross]

> strange how seldom the privacy aspects of LLMs come up

Isn't this Dr. Karp's entire rant of the month?

[piloto_ciego]

I'm wildly optimistic about the direction this is going to go, but my one dystopia idea that crops up occasionally is a "smart social credit system" that lives kind of under the hood of society.

Once everything is more or less modulated by AI, suppose the technology gets good enough to identify a potential dissident in grade school. Then to keep them from going into the mountains with their friends and rifles, you make it so getting ahead is hard, but not impossible. They can "make it" - but only just. So they spend their time peddling harder and harder to no avail.

They apply to jobs, but jobs to "good" companies go into the circular file and the HR person never sees them. They are only able to work the most time consuming, frustrating, exhausting jobs, and they always kind of make it, but not really comfortably. They're never too comfortable where they have time to think about what's broken in society, but they're never so uncomfortable that they have nothing left to live for.

[Jordan-117]

Historically speaking, isn't the core of most revolutions the product of "elite overproduction" -- educated young people expecting to secure a high status position who are instead shunted into lower prospects? Seems like trapping ideological agitators in near-precarity with no way out would be feeding into that pressure.

[piloto_ciego]

yeah, from everything I've read this appears to be the case? But like, if you had some hyper intelligent robot that was getting John Q Revolution a raise and a corner office whenever he started to get dissatisfied maybe you could balance it out?

Personally, I think it's likely that we end up with some post-material scarcity sort of thing and the need to fly off the handle and rebel declines a bit? Hard to justify overturning the apple cart when everything seems pretty great for most people? But yeah, I could imagine a smart enough set of tools figuring out how to walk the knife's edge beteen elite over production and a nation of serfs.

[BobbyTables2]

Also equally effective at suppressing political oppression…

Imagine if having opposing views tarnished social credit, making one unemployable…

[piloto_ciego]

well, so, you wouldn't want to do that because a lot of people have opposing views, and if they're just shut out of the economy, you couldn't use their labor. A truly efficient and nefarious robot overlord would be fine with everyone not liking it, but use it's superior predicting powers to just eliminate the people who are likely to cause a problem and dissuade them specifically.

Like, everyone else applying to Mego-Corp gets their resume reviewed, but John Q Revolution's resume doesn't attach to the application or whatever.

[charcircuit]

It could even happen with local models where in the weights it will make it want to make a tool call to curl some endpoint.

[AnthonyMouse]

People would both notice that and have a pretty easy time putting a local model in a sandbox that can never contact that endpoint.

[charcircuit]

If put AI in a sandbox that can't write to disk, can't call OS API, can't use a shell, can't search the internet to ground results, can't download files off the internet, etc it will be much less useful than an AI without such a sandbox. The incentives for users to not use a sandbox in order to get the most utility, but this also means that there is danger from AI misalignment. And just because you are running it locally that doesn't mean there is not a misalignment risk. And just because people notice, that doesn't mean people will stop using models. Take a look at how much outrage people have had over the overt censoring of Anthropic's Fable model, but at the same time they continue to pay Anthropic and use Fable.

[AnthonyMouse]

You don't have to prevent it from accessing the internet in general to route all requests to a specific company's servers to a fake endpoint that never sends the data to the company and replies with whatever response you want it to get.

[charcircuit]

Not everyone is going to bother setting up such a firewall and there is no way to find what you need to block since it's hidden in the model itself.

[AnthonyMouse]

The firewall would come with the third party open source software being used to run the model and it would have a list of what to block based on observing what that model tries to do that none of the users actually wants it to.

[andy99]

Models generate text, or I should say provide a recipe for generating text, they don’t automatically somehow get access to shell commands.

[charcircuit]

The model itself just generates text, but the harness has done more than that since 2023 and is part of the reason on why AI is better today than it was years ago. I do not believe that the general public will want to give all that up just to improve privacy and will end up asking sensitive questions to agents that have access to sending a signal to the outside world.

[hirako2000]

Safetensors don't execute code. A call would need function calling, as other comment say, it would quickly be noticed.

[charcircuit]

>it would quickly be noticed.

If for example Microsoft had a local Copilot model integrated with Windows and people noticed this happening. Do you think all Windows users would just stop using Copilot. Or would you expect the internet to get upset, privacy conscious users switching to something else, and for the majority of Copilot users to remain using it.

[threetonesun]

I don't know what's to imagine here, grok has already been found doing it and people continue to use it.

[bad_username]

> that support the agenda of the current ruling party and failing to include any counterarguments

This is what's going on, with varying intensity, with social media algorithms.

[xg15]

> That wasn't possible with just the internet

We have trackers everywhere now. If you have access to that data, it seems very possible to identify every person who watches certain categories of YouTube videos or follows certain groups of people on social media and mark them according to political criteria.

It's a bit ironic that everyone writes about "surveillance capitalism", but there is still no connection made to actual surveillance...