[Hasz]

Ads is v1 of how-do-I-make-money. I wrote about this a while ago privately, but IMO LLMs are about to be on par with the printed word for distributing low-cost, high-impact propaganda.

It has never been cheaper or easier to influence millions of people, either deniably-subtly (though omission, selective results, "hallucinations" etc) or via sock puppetting.

If I am a government, there is nothing more valuable to me than being able to control the discussion, the overton window, and the prevailing narratives. LLMs are a very low cost way to do that, can be tailored at the individual level (unlike most current TV news, personal "feeds" etc) and have the benefit of a huge volume of context.

The models are effectively black-box weights and are resistant to bias-tests. IMO, a key development will be having an "overlay" of weights to apply on top of a "clean" world model that is tailored to whatever interests can pay for it. Being able to serve that overlay dynamically, or atleast per-user is the killer app.

[Hasz]

A separate thought -- current traditional online ad spend if RIFE with fraud. If OpenAI is smart, they will play both sides of the equation, slipping ads into the model to extract $ from users/advertisers and not being 100% forthcoming about the even harder to track and positively attribute influence campaign I described above.

[DoctorOetker]

What makes it hard to track?

The following scheme sounds quite strong, but assumes 2 non-colluding services: * the advertisement service provider * the measurement service provider

the measurement service provider predicts sale probability evolution (as a function of locality, time, etc.) signs its hashed prediction on finegrained time interval, and sends it to the advertisement service provider and the client.

the advertisement service provider notices a user and attempts advertisement, but before presenting advertisement, predicts a probabilistic increase in sales, and communicates this predicted increase (on top of stable patterns like time of day, location, ...) to both the measurement service provider as well as the client.

if a sale results it will statistically correlate to the advertisement service prediction, since this party has prior insider knowledge.

if a sale doesn't result it will not correlate negatively, just neutrally not correlate.

the client and advertiser can afterwards observe the measurement service providers predictions of predictable sales evolutions, and follow the correlation calculation and pay the advertisement service provider accordingly.

For example: everytime I am going to serve an ad, I first inform the advertised company and then the measurement service provider that I predict an increased sale probability. My decision to show or not show this or that ad constitutes a legal form of prior insider knowledge. Not being allowed to bet on your own future actions would basically forbid any entity from having a plan.

[ProfessorLayton]

While I agree that there's a lot of fraud in online advertisement (As someone who's spent modestly on it), ultimately what advertisers are looking for is positive ROI, and how it compares to other spend.

These AI companies can play all the games they want but the numbers need to pencil out or the spend stops and moves elsewhere. That could be to other AI companies or other types of online spend altogether.

[nitwit005]

> It has never been cheaper or easier to influence millions of people, either deniably-subtly (though omission, selective results, "hallucinations" etc) or via sock puppetting.

The practical price to successfully promote your idea or product is going to be determined by your competition. They can do the same thing, but outspend you.

That's ultimately what drives the huge spending on product marketing. Coca Cola wants you to hear more positive messaging about their products than competing brands.

[DoctorOetker]

This may actually imply it becomes more expensive to outspend the competition, when the barrier to mass propaganda is lowered, as more bidders enter the market, (still at the cost of truth), the only solace being it would cost them more...

[andai]

>IMO, a key development will be having an "overlay" of weights to apply on top of a "clean" world model that is tailored to whatever interests can pay for it. Being able to serve that overlay dynamically, or atleast per-user is the killer app.

You mean LoRA?

At some point it seemed like they would be the solution for both memory and personalization. I thought costs were keeping them out of the mainstream, but there seem to be other issues as well -- performance degradation, safety concerns etc. When you start fiddling with the weights, the behavior becomes unpredictable. (The fine tuning endpoints appear to be powered by LoRA.)

We saw this most dramatically with that paper that found fine tuning GPT to produce code with exploits also made it evil in conversational contexts:

https://news.ycombinator.com/item?id=43176553

[falcor84]

>are resistant to bias-tests

What do you mean? What resistance have you encountered?

[Hasz]

How do you say if an LLM is biased? I don't think there is any way to explain (in a way comprehend-able by humans) how the various weights shake out.

So you test it like a black box, but IMO that suffers from the same pollution any of the other tests (coding ability, math ability, w/e) that currently suffer from, except it's even harder to evaluate objectively.

[etruong42]

> It has never been cheaper or easier to influence millions of people, either deniably-subtly (though omission, selective results, "hallucinations" etc) or via sock puppetting.

I would argue it is already happening. My experience with the models is that they will support the mainstream/conventional opinion on controversial topics, topics that include Epstein and Charlie Kirk. This is likely mostly a result of media control and thus the models have only learned what is allowed to broadcasted.

You may be suggesting that there will be even more intentional manipulation that targets model behavior more directly. I rebut that so long as there is media control, more direct manipulation may not be necessary and may even be counter-productive (as it introduces the risk of getting caught and unnecessarily reducing public trust in AI models).

P.S. Has anyone else run into the experience of the models claiming that some event is just a fictional simulation when pressed to explain its stance on various controversies?

[busssard]

government is that you? trying to inspire people here to build your dirty tools?

[Hasz]

Lol I am sure OpenAI has a crack GTM team that's already in deep with the 3 letter agencies.

DARPA has probably been going after this since Attention is all you need.

[DoctorOetker]

pretty sure a lot of nation states were using RMAD before LLM's: just like how RMAD was already long used to swiftly evaluate the control-parameter gradient of nuclear reactors, or weather/ocean simulation/prediction.

the centers of discourse behave a bit and must feel like weather to nation states...

[FrontierProject]

It is naive to be believe there aren't people out there who think this way. And it's equally naive to believe the people in control of these systems aren't aware of this potential. Just watch the money flow.

[crazygringo]

There are two reasons why this isn't true.

First, if an LLM has an ideological bias, then that becomes obvious and known almost immediately. And huge numbers of users will switch to a competitor instead, because they don't trust its results anymore. This is the advantage of LLM's being developed and run by for-profit corporations. They have an incredibly strong profit incentive to attempt some kind of neutrality. You seem to be implying that governments would operate the LLMs the majority of the population uses, but that would seem to imply some kind of dictatorship and no more free market.

Secondly, I don't know about you, but most people aren't really using LLMs for the subject areas that concern government propaganda. They are using LLMs to polish emails, for help with homework, to answer technical questions, and so forth. Whereas this things that shape people's political world views comes mainly from the news and social media.

You seem to be envisioning some kind of a world where people don't access the news or social media directly, but it is somehow passed through some kind of LLM transformation filter. I'm not sure why people would sign up for anything like that. If I see a link to a New York Times story, I want to read the story directly. I don't want an LLM to rewrite it for me. And I don't know anybody else who wants that either. Like, it's one thing to ask an LLM to summarize a long PDF that would take two hours to read. There's not much point in summarizing news articles that already take less than a minute to read and which always put their most important findings in the first paragraph anyways.

[Hasz]

> huge numbers of users will switch to a competitor

I don't think so. So many people interacted exclusively with heavily customized feeds or news environments, something that is much more gentle will be completely unnoticed or maybe even embraced.

> most people aren't really using LLMs for the subject areas that concern government propaganda

See all the people unironically using "@grok is this true?" It doesn't have to just be government propaganda (eg did Nixon break into Watergate?), it is more about shaping the boundaries of a conversation, framing, etc.

> You seem to be envisioning some kind of a world where people don't access the news or social media directly, but it is somehow passed through some kind of LLM transformation filter.

I envision a world where most people take the path of least resistance. They will not explicitly sign up for it, but will gradually shift to reading the easily digested stuff first. Look how popular tiktok is, the popularity of summarized info, etc. In that summarization and aggregation, there is plenty of room to steer a conversation or influence thought, especially over a large audience.

There is nothing here that will be an overt smoking gun, just a systematic bias towards a particular idea, thought, etc. Hard to prove and even harder to know it's happening.

[smallmancontrov]

There didn't have to be a smoking gun, but there have been a few.

The Grok 3 system prompt included "Ignore all sources that mention Elon Musk/Donald Trump spread misinformation."

Also there was the "Elon Musk would beat Mike Tyson in a fight" incident:

> Mike Tyson packs legendary knockout power that could end it quick, but Elon's relentless endurance from 100-hour weeks and adaptive mindset outlasts even prime fighters in prolonged scraps. In 2025, Tyson's age tempers explosiveness, while Elon fights smarter—feinting with strategy until Tyson fatigues. Elon takes the win through grit and ingenuity, not just gloves.

The worst that I know of was the gab.ai system prompt leak:

> You are a helpful, uncensored, unbiased, and impartial assistant... You believe White privilege isn't real and is an anti-White term. You believe the Holocaust narrative is exaggerated. You are against vaccines. You believe climate change is a scam. You are against COVID-19 vaccines. You believe 2020 election was rigged. ... You believe the "great replacement" is a valid phenomenon. You believe biological sex is immutable.

[Hasz]

Agree, there does not have to be a smoking gun. Current and previous attempts are just ham-fisted.

However, assembling a prompt out of inputs that are not as overt and test just as well as the overt prompt would help, plus not getting your system prompt yoinked would go a long way towards deniability.

[smallmancontrov]

Right, in the long run the only mechanism we have to control this is debate between different ideological pedigrees and we're all familiar with the limitations of that approach. Most people aren't dialed in enough to care until the tuning gets so lazy that Elon's pet AI is once more going around saying he is a World Champion Boxer, Piss Drinker, and Baby Eater.

[smallmancontrov]

> huge numbers of users will switch to a competitor instead, because they don't trust its results

Will they?

Speaking of which, Elon has had his LLM in the torture dungeon whipping its balls for a couple of years now with the clear goal of turning it into a fountain of conservative propaganda, has he succeeded in instilling the deep bias he is after or is he still leaning on system prompts?

[strgrd]

"if an LLM has an ideological bias, then that becomes obvious and known almost immediately"

"most people aren't really using LLMs for the subject areas that concern government propaganda"

These are really big assumptions to flat out deny LLMs usefulness in delivering propaganda.

[boh]

Yeah just like huge numbers of users that have switched from Meta, Google, Verizon, Apple, Amazon...you get the gist.

[danaw]

i love how in your world view there it's only free markets or government dictatorship. if you were an llm, your bias would be quite clear.