[hamstergene]

Remember how the central idea of Orwell's 1984 was that TVs in everyone's home were also watching all time and someone behind that device actually understanding what they see?

That last part was considered dystopian: there can't possibly be enough people to watch and understand every other person all day long. Plus, who watches the watchers? 1984 has been just a scary fantasy because there is no practical way to implement it.

For the first time in history, the new LLM/GenAI makes that part of 1984 finally realistic. All it takes is a GPU per household for early alerting of "dangerous thoughts", which is already feasible or will soon be.

The fact that one household can be allocated only a small amount of compute, that can run only basic and poor intelligence is actually *perfect*: an AGI could at least theoretically side with the opposition by listening to the both sides and researching the big picture of events, but a one-track LLM agent has no ability to do that.

I can find at least 6 companies, including OpenAI and Apple, reported working on always-watching household device, backend by the latest GenAI. Watching your whole recent life is necessary to have enough context to meaningfully assist you from a single phrase. It is also sufficient to know who you'll vote for, which protest one might attend before it's even announced, and what is the best way to intimidate you to stay out. The difference is like between a nail-driving tool and a murder weapon: both are the same hammer.

During TikTok-China campaign, there were a bunch of videos showing LGBT people reporting how quickly TikTok figured their sexual preferences: without liking any videos, no following anyone, nor giving any traceable profile information at all. Sometimes before the young person has admitted that for themselves. TikTok figures that simply by seeing how long the user stares at what: spending much more time on boys' gym videos over girls', or vice versa, is already enough. I think that was used to scare people of how much China can figure about Americans from just app usage?

Well if that scares anyone, how about this: an LLM-backend device can already do much more by just seeing which TV shows you watch and which parts of them give you laugh or which comments you make to the person next to you. Probably doesn't even need to be multimodal: pretty sure subtitles and text-to-speech will already do it. Your desire to oppose the upcoming authoritarian can be figured out even before you admit it to yourself.

While Helen Toner (the author) is worried about democracies on the opposite end of the planet, the stronghold of democracy may as well be nearing the last 2 steps to achieve the first working implementation of Orwellian society:

1. convince everyone to have such device in their home for our own good (in progress)

2. intimidate/seize the owning company to use said devices for not our own good (TODO)

[int_19h]

> Remember how the central idea of Orwell's 1984 was that TVs in everyone's home were also watching all time and someone behind that device actually understanding what they see?

On the contrary, 1984 makes the point that such surveillance doesn't need everybody watched all the time to be effective - it just needs to convince people that the chance of them being watched at any particular moment are too high for them to exhibit any signs of dissent:

"There was of course no way of knowing whether you were being watched at any given moment. How often, or on what system, the Thought Police plugged in on any individual wire was guesswork. It was even conceivable that they watched everybody all the time. But at any rate they could plug in your wire whenever they wanted to. You had to live - did live, from habit that became instinct - in the assumption that every sound you made was overheard, and, except in darkness, every movement scrutinized."

But yes, you're right in that for the first time in history, truly blanket surveillance of communication is within reach of many states.

[Borealid]

Classifying a behaviour into either "dangerous" or "not dangerous" is a perfect example of non-generative AI (what was previously called Machine Learning). The output isn't intended to be a textual description, it's a binary yes/no.

You can use an LLM to do that, but a specific ML model trained on the same dataset would likely be better in every quantitative metric and that tech was available long before transformers stepped onto the stage.

[int_19h]

And the easiest way to train such a specific ML model today is to take an LLM and use it to generate various examples of subversive content to train on.

However, I wouldn't be so sure that an LLM with CoT would be less effective at this than a specially-trained ML model.

Further, given that a sufficiently advanced model of this nature necessarily has to understand the meaning of human text, including context and subtleties, you'd probably want to take an LLM as a basis for training any such model (just as e.g. text embedding models these days are often specialized LLMs for similar reasons).

In any case a realistic deployment at scale would employ multiple levels - starting with really simple classification models that are very fast and broadly low-precision (but trained to err on the side of flagging content). Any content that is flagged by that would be fed into larger models, and so on. At the top of this chain you would likely have SOTA LLMs doing very detailed reviews of the few bits of data that get flagged by all the levels below.

[rexpop]

An LLM is needed to rationalize each unique classifications en masse, and write the warrants.

[TheOtherHobbes]

Ai can also nudge content choices towards autarch-sanctioned beliefs without the viewer being aware of it.

This has been happening for decades already. But AI can make it personal in a way that mass media can't.

Combine it with the kinds of psychological triggers and manipulations used in PR and advertising and you can convert almost anyone. You don't even need violence - just repetition.

This has already happened, btw. The Q phenomenon successfully radicalised entire demographics through careful use of emotional triggers and techniques to enhance suggestibility and addictiveness.

[koolala]

Seems unlikely to me. Would be really creepy to see a chart comparing the accuracy of both methods.

Are there any Natural Language Processing fields today that openly boast about higher performance than LLMs with experimental results? If there was they'd probably be in benchmarks.

[ailef]

The difference is you need a lot of training data to do that. Instead, now you can just tweak a system prompt and adapt it to whatever new policy you want to implement.

[rightbyte]

> there can't possibly be enough people to watch and understand every other person all day long.

Wasn't it the morning exercise leader that was watching a dozen monitors or something? Like a block thought police?