[stymaar]

> The first is that it would still misclassify human-authored text written under the same incentive, and most people have various incentives to "maximize engagement".

The thing is, humans are significantly worse at maximizing numerical goals than computers.

> And the second is that then people would just make other models that are tuned for defeating that sort of classifier, which would be used whenever the classifier is being used.

Anyone can already do that right now, just grab unsloth studio and fine-tune your local Gemma, but nobody cares. People posting slop content don't care if pangram or I flag their slop with certainty, they are using the easiest option, which is commercial chat models. And given this segment of user doesn't care, the provider have zero incentive to provide a dedicated stealth model for that purpose.

[AnthonyMouse]

> The thing is, humans are significantly worse at maximizing numerical goals than computers.

I'm not sure this is even the right premise.

Existing LLMs try to maximize engagement, and they often write in a particular style that has tells, but these two things are not necessarily related. Over-using em-dash or whatever isn't the thing that maximizes engagement.

So the two problems really are, what happens to the actual humans whose writing style is a close match for what a given generation of LLMs output? And, what stops LLMs from using a different style when someone wants to fool the classifier?

> People posting slop content don't care if pangram or I flag their slop with certainty, they are using the easiest option, which is commercial chat models.

They don't care as long as the consequences of identifying it are immaterial, but in that case what's the point of classifying it? Whereas if they need to fool the classifier some threshold percentage of the time in order for enough of their spam to get through, they're going to care.

[stymaar]

> Over-using em-dash or whatever isn't the thing that maximizes engagement.

It's the thing that minimizes the loss during the RLHF phase, and the RLHF phase is the one that's aimed at maximizing engagement (it's literally trained on that).

> what happens to the actual humans whose writing style is a close match for what a given generation of LLMs output?

If a human, for instance because its writing gets polluted by reading too much AI slop, matches the style of an LLM closer than a certain threshold, then his own writing is going to be flagged as well. Whether it's an actual problem or merely a theoretical one is an open question. (unlike OpenAI and Anthropic, humans writers do have an incentive to avoid being flagged as AI).

> And, what stops LLMs from using a different style when someone wants to fool the classifier?

In theory: nothing. In practice if you fine-tune your own model: nothing. In practice with commercial models: the interests of the model making company.

> And, what stops LLMs from using a different style when someone wants to fool the classifier?

Websites have pretty much stopped using ad-blocker-blockers, it seems that it's not a fight worth fighting for them. Does that mean that ad-blockers are useless?

Most people don't even care about ads, I don't think they care about slop either, that's why there's slop posts and obnoxious websites that are unreadable without an ad blocker. A slop blocker used by 10-20% of the internet users wouldn't change the calculation more than ad blockers did.

[AnthonyMouse]

> It's the thing that minimizes the loss during the RLHF phase, and the RLHF phase is the one that's aimed at maximizing engagement (it's literally trained on that).

I don't think RLHF is the biggest reason its style is the way it is.

A lot of it is that it's trained on everything they could get their hands on, which includes domain-specific literature and books that go all the way back to the advent of writing, and then will pick up habits that are common in some specific domain or in 19th century literature etc. that are less common in most modern writing when no attempt is being made to do otherwise.

Do you really think that RLHF humans were requesting more em-dash?

> Websites have pretty much stopped using ad-blocker-blockers, it seems that it's not a fight worth fighting for them. Does that mean that ad-blockers are useless?

Websites have pretty much stopped using them because they realized readers with ad blockers will stop using the site sooner than stop using their ad blocker, and since websites have a network effect, it's better to let a minority of readers block ads when having them makes it more likely they'll distribute links to the site. And because it's the user who controls the browser for web pages, which gives ad blockers a decisive advantage.

> Most people don't even care about ads, I don't think they care about slop either, that's why there's slop posts and obnoxious websites that are unreadable without an ad blocker. A slop blocker used by 10-20% of the internet users wouldn't change the calculation more than ad blockers did.

Sites don't want users to use ad blockers, but having a user with an ad blocker is still better for them than not having the user at all, because of the network effect.

Whereas many sites don't want slop at all, and then if slop detectors work they'll put them in the site itself and block the slop for 100% of users. At which point the slop generators have a 100% incentive to find a workaround instead of a 20% incentive, which is different.

[pixl97]

I mean, back when I was spam filtering setting up a simple Bayesian classifier was easy. Train it on your spam and ham and it worked damned good. "Mission Accomplished".... until it wasn't. Spam rates started climbing and it started getting harder than ever to filter them.

There is always an incentive to get spam to bypass filters, so as your filters increase in accuracy, those attempting to pass said filters adjust their behaviors.

Spammers/cheaters/whateverers will at least just use a second pass filter that uses one of these 'ai scoring' systems to beat said AI scoring systems. So while it's worthwhile to do it at this moment, this window will rapidly close.

[stymaar]

I don't think it's a very good remark, as there's significantly less email spam than 20 years ago.

Another example is ad-blocker-blocker. There was a little bit of an arm race between ad blockers and advertisers in the middle of the 2010s, but it didn't last long. Advertisers mostly just decided not to care about ad-blockers.

[pixl97]

>Advertisers mostly just decided not to care about ad-blockers.

Directly not to care because they lost in court.

And yet the biggest advertizer on Earth (Google) decided to change their browser to make adblocking far more difficult. That or they say "just use an app, oh and turn on notifications". I'm not exactly sure who you think won the arms race there, but it seems like we the user did not.

There is significantly more spam than 20 years ago, just less of it reaches your inbox. This is a very important distinction as the cost of spam filtering is just as high as ever. On top of that most people have given up on their own email servers and instead depend on Google/Microsoft to do it for them. This allows these companies to have an overwhelming influence on email on the internet, to the point they can send spam with near impunity, and where if your system does it will be nuked from orbit by their systems.

And much like now Google supplies both the email spam, and the solution to the spam, they'll gladly supply the LLMs spam and the LLM solution while applying their 'flavor' of what's allowed to the entire internet.

[stymaar]

> Directly not to care because they lost in court.

I'm pretty sure the illegal sport streaming websites didn't stop doing that just because it became illegal, otherwise they could have stopped their activity altogether while they were at it…

> I'm not exactly sure who you think won the arms race there, but it seems like we the user did not.

I, at least, won when the webiste showing ads gave up the race (for the past decade at least, only time will tell about the future).

> nd yet the biggest advertizer on Earth (Google) decided

This is actually an argument in my direction! The owners of websites (which are also the ones posting slop today) didn't care enough and the situation only changed because Google moved.

I expect the same thing with slop. Individual websites won't make any effort to make their slop unblockable, and it will only be a problem if OpenAI/Anthropic/Google decide that they care about this market. But unlike Google in the ads market, I don't think the model providers have any reason to care. The web is already dead in their mind anyway.

> There is significantly more spam than 20 years ago, just less of it reaches your inbox.

This goes against your very argument from earlier!

> On top of that most people have given up on their own email servers and instead depend on Google/Microsoft to do it for them.

Out of convenience, but you don't need that to be practically free of spam. Whatever version of SpamAssassin is being run on OVH's mail servers has been enough for that purpose for me.

> they'll gladly supply the LLMs spam and the LLM solution while applying their 'flavor' of what's allowed to the entire internet.

Again, they don't care about the web. They just crawl it for content but they don't want you to read any webpage, they want you to stay in their chatbot. Every other use-case is nonexistent to them (except coding agents, of course, but that's a different product altogether).