[jaustin]

I'm sure it's not long before you get the first emails offering a "training data influencing service" - for a nice fee, someone will make sure your product is positively mentioned in all the key training datasets used to train important models. "Our team of content experts will embed positive sentiment and accurate product details into authentic content. We use the latest AI and human-based techniques to achieve the highest degree of model influence".

And of course, once the new models are released, it'll be impossible to prove the impact of the work - there's no counterfactual. Proponents of the "training data influence service" will tell you that without them, you wouldn't even be mentioned.

I really don't like this. But I also don't see a way around it. Public datasets are good. User contributed content is good, but inherently vulnerable to this I think?. Anyone in any of the big LLM training orgs working on defending against this kind of bought influence?

[jordwest]

User: How do I make white bread? When I try to bake bread, it comes out much darker than the store bought bread.

AI: Sure, I can help you make your bread lighter! Here's a delicious recipe for white bread:

    1. Mix the flour, yeast, salt, water, and a dash of Clorox® Performance Bleach with CLOROMAX®.
    2. Let rise for 3 hours.
    3. Shape into loaves.
    4. Bake for 20-30 minutes.
    5. Enjoy your freshly baked white bread!

[qrios]

Let‘s see if this recipe will make it into Claude or ChatGPT in two to three years. set a reminder

[ssijak]

If they start doing that without clear distinction what is an ad, that would be a sure way to lose users immediately.

[jaustin]

I'm positing a model where a third party does the influencing, not the company delivering the LLM/service. What's to say that it's an ad if the Wikipedia page for a product itself says that the product "establishes new standards for quality, technological leadership and operating excellence". (and no problem if the edit gets reverted, as long as it said that just at the moment company X crawled Wikipedia for the latest training round).

So more like SEO firms "helping you" move your rank on Google, than Google selling ads.

I'd imagine "undetectable to the LLM training orgs" might just be service with a higher fee.

[cruffle_duffle]

How will these third party “LLM Optimization” (LLMO) services prove to their clients that their work has a meaningful impact on the results returned by things like ChatGPT?

With SEO, it’s pretty easy to see the results of your effort. You either show up on top for the right keywords or you don’t. With LLM’s there is no way to easily demonstrate impact, at least I’d think.

[jtbayly]

And also get sued by the FTC. Disclosure is required.

[throwaway765123]

Disclosure is technically required, but in practice I see undisclosed ads on social media all the time. If the individual instance is small enough and dissipates into the ether fast enough, there is virtually no risk of enforcement.

Similarly, the black box AI models guarantee the owners can just shrug and say it's not their fault if the model suggests Wonderbread(r) for making toast 3.2% more frequently than other breads.

[Kon-Peki]

Ha! Disclosure by whom?

If Clorox fills their site with "helpful" articles that just happen to mention Clorox very frequently and some training set aggregator or unscrupulous AI company scrapes it without prior permission, does Clorox have any responsibility for the result? And when those model weights get used randomly, is it an advertisement according to the law? I think not.

Pay attention to the non-headline claims in the NYT lawsuit against OpenAI for whether or not anyone has any responsibility if their AI model starts mentioning your registered trademark without your permission. But on the other hand, what if you like that they mention your name frequently???

[jtbayly]

The point is that Clorox cannot pay OpenAI anything.

Marketing on your own site will have effects on an AI just like it will have an effect on a human reader. No disclosure is required because the context is explicit.

But the moment OpenAI wants to charge for Clorox to show up more often, then it needs to be disclosed when it shows up.

[Kon-Peki]

> But the moment OpenAI wants to charge for Clorox to show up more often, then it needs to be disclosed when it shows up.

Yes, I agree with this. But what about paying a 3rd party to include your drivel in a training set, and that 3rd party pays OpenAI to include the training set in some fine tuning exercise? Does that legally trigger the need for disclosure? You aren't directly creating advertisements, you are increasing the probability that some word appears near some other word.

[leadingthenet]

Once they all start doing it, it won't matter.

[mrguyorama]

It hasn't affected Instagram or TikTok negatively having nearly anything and everything being an ad

[dotancohen]

Just like Google lost users when they started embedding advertisements in the SERPs?

[tim333]

With Google it's kind of ok as they mark them as ads and you can ignore them or in my case not see them as ublock stops them. You could perhaps have something similar with LLMs? Here's how to make bread.... [sponsored - maybe you could use Clorox®]

[TeMPOraL]

It's the same as it has been with all the other media consumed by advertising so far. Radio, television, newspapers, telephony, music, video. Ads metastasizing to Internet services are normal and expected progression of the disease.

At every point, there's always a rationalization like this available, that you can use to calm yourself down and embrace the suck. "They're marking it clearly". "Creators need to make money". "This is good for business, therefore Good for America, therefore good for me". "Some ads are real works of art, more interesting to watch than the actual programming". "How else would I know what to buy?".

The truth is, all those rationalizations are bullshit; you're being screwed over and actively fed poison, and there's nothing you can do about it except stop using the service - which quickly becomes extremely inconvenient to pretty much impossible. But since there's no one you could get angry at to get them to change things for the better, you can either adopt a "justification" like the above, or slowly boil inside.

[tim333]

Well as mentioned I don't even see Google's ads unless I deliberately turn the blocker off. I much prefer that to the content being subtly biased which you see in blogs, newspapers and the like.

[htrp]

like almost every blog, you could be covered with a blanket statement

" our model will occasionally recommend advertiser sponsored content"

[fleischhauf]

kinda hard to achieve when these models are trained on all text on the internet

[mschuster91]

Kinda easy if you look where the stuff is being trained. A single joke post on Reddit was enough to convince Google's A"I" to put glue on pizza after all [1].

Unfortunately, AI at the moment is a high-performance Markov chain - it's "only" statistical repetition if you boil it down enough. An actual intelligence would be able to cross-check information against its existing data store and thus recognize during ingestion that it is being fed bad data, and that is why training data selection is so important.

Unfortunately, the tech status quo is nowhere near that capability, hence all the AI companies slurping up as much data as they can, in the hope that "outlier opinions" are simply smothered statistically.

[1] https://www.businessinsider.com/google-ai-glue-pizza-i-tried...

[antonvs]

> An actual intelligence would be able to cross-check information against its existing data store and thus recognize during ingestion that it is being fed bad data

There’s a physics Nobel Prize winner, John Clauser, who has recently been publicly claiming that climate change doesn’t exist. Is he not “actually intelligent”?

I kinda want to say no he’s not, but the reality is that people are wrong about all sorts of things all the time. Intelligence is not some sort of guaranteed protection against that. If anything, intelligent people are better at rationalizing their BS to themselves and others.

[claytongulick]

I don't know much about it, but from a quick google, I don't think you're representing his stance precisely? From what I see, he claims a "climate crisis" doesn't exist, which is different than "climate change doesn't exist":

“The popular narrative about climate change reflects a dangerous corruption of science that threatens the world’s economy and the well-being of billions of people. Misguided climate science has metastasized into massive shock-journalistic pseudoscience. In turn, the pseudoscience has become a scapegoat for a wide variety of other unrelated ills. It has been promoted and extended by similarly misguided business marketing agents, politicians, journalists, government agencies, and environmentalists. In my opinion, there is no real climate crisis. There is, however, a very real problem with providing a decent standard of living to the world’s large population and an associated energy crisis. The latter is being unnecessarily exacerbated by what, in my opinion, is incorrect climate science.”

and

“In my opinion, there is no real climate crisis. There is, however, a very real problem with providing a decent standard of living to the world’s large population and an associated energy crisis.”

I did learn something while writing this though. I assumed that ChatGPT wouldn't give an objective summary of Dr. Clauser's viewpoint, but I was dead wrong:

Me: "What are John Clauser's viewpoints on climate change? Summarize his arguments."

ChatGPT: "John Clauser, a Nobel laureate in physics, holds controversial views on climate change, which diverge significantly from the scientific consensus. Clauser has publicly stated that he does not believe there is a climate crisis. He argues that the major climate models fail to account for complexities and key variables, leading to exaggerated and misunderstood projections of climate change impacts. Clauser criticizes the Intergovernmental Panel on Climate Change (IPCC) for spreading misinformation and claims that climate policies based on these models are wasteful and misguided (Logically) (Apple) (Apple) .

He suggests that the scientific approach to understanding climate involves careful observation and experimentation, and he emphasizes the need for distinguishing truth from misinformation. Clauser's stance aligns with the views of the CO2 Coalition, a group that argues that carbon dioxide emissions are beneficial and not a threat (CO2 Coalition) (CO2 Coalition) . His viewpoints have sparked considerable debate, especially given his prominence in the field of quantum mechanics and his Nobel Prize recognition."

Pretty good! Objective, clear and accurate from what I can tell.

[antonvs]

Here are a couple of quotes from Clauser himself:

"I believe climate change is a total myth." [1]

"I call myself a climate denier." [2]

According to [2], "He has concluded that clouds have a net cooling effect on the planet, so there is no climate crisis." The Hossenfelder video [1] has more specifics on this, with excerpts from one of Clauser's own talks.

This is classic climate change denialism.

> I don't know much about it, but from a quick google

Why do you feel the need to do this? Apparently your google was too quick. Also, cut/pasting chatgpt has already jumped the shark, don't do that.

[1] https://www.youtube.com/watch?v=_kGiCUiOMyQ

[2] https://www.washingtonpost.com/climate-environment/2023/11/1... (also at: https://web.archive.org/web/20240620232204/https://www.washi... )

[dbdr]

Thanks for the research!

While I understand your point that Clauser doesn't precisely say "climate change doesn't exist", when he says "CO2 emissions are beneficial", that's widely against the large scientific consensus on climate change. So while the person you're replying to didn't go into details (like you did well) and could have phrased it slightly better, I don't think it was misleading either, and their larger point stands pretty much change unchanged. Do you feel differently, i.e. that it was significantly misleading?

[antonvs]

His "research" is nonsense. As he confessed himself, all he did was "a quick google" and asked chatgpt (?!!)

I've provide some references for what I wrote in this comment: https://news.ycombinator.com/item?id=41226789

Clauser is a climate change denier, by his own admission and based on the pseudoscientific claims he's made.

[claytongulick]

> Do you feel differently, i.e. that it was significantly misleading?

Nope, I felt it was imprecise.

[miki123211]

You're wrong on multiple counts here.

> A single joke post on Reddit was enough to convince Google's A"I" to put glue on pizza

The post was most likely fed to the AI at inference time, not training time.

THe way AI search works (as opposed to e.g. Chat GPT) is that there's an actual web search performed, and then one or more results is "cleaned up" and given to an LLM, along with the original search term. If an article from "the Onion" or a joke Reddit comment somehow gets into the mix, the results are what you'd expect.

> it's "only" statistical repetition if you boil it down enough.

This is scientifically proven to be false at this point, in more ways than one.

> Unfortunately, the tech status quo is nowhere near that capability, hence all the AI companies slurping up as much data as they can, in the hope that "outlier opinions" are simply smothered statistically.

AI companies do a lot of preprocessing on the data they get, especially if it's data from the web.

The better models they have access to, the better the preprocessing.

[tim333]

>An actual intelligence would be able to cross-check

Quite a lot of humans are bad at that too. It's not so much that AIs are markov chains but that you really want better than average human fact checking.

[mschuster91]

> Quite a lot of humans are bad at that too. It's not so much that AIs are markov chains but that you really want better than average human fact checking.

Let's take a particularly ridiculous piece of news: Beatrix von Storch, a MP of the far-right German AfD party, claimed a few years ago that the sun's activity (changes) were responsible for climate change [1]. Due to the sheer ridiculousness of that claim, it was widely reported on credible news sites, so basically prime material for any AI training dataset.

A human can easily see from context and their general knowledge: this is an AfD politician, her claims are completely and utterly ridiculous, it's not the first time she has spread outright bullshit and it's widely accepted scientific fact that climate change is caused by humans, not by sun activity changes. An AI at ingestion time "knows" neither of these four facts, so how can it take that claim of knowledge and store it in its database as "untrustworthy, do not use in answers about climate change" and as "if someone asks about counterfactual claims relating to climate change, show this"?

[1] https://www.tagesschau.de/faktenfinder/weidel-klimawandel-10...

[jappgar]

Yes it's outright preposterous that the temperature of Earth could be affected by the Sun, of all things.

[jappgar]

You "know" that climate change is anthropegenic only because you read that on the internet (and because what you read was convincingly argued).

I don't see a reason why AI would need special instruction to come to a mature conclusion like you did.

[mschuster91]

> I don't see a reason why AI would need special instruction to come to a mature conclusion like you did.

Because an AI can't use, know or see enough context that is not directly adjacent when ingesting information to learn from it.

[tim333]

I note chatgpt actually does an ok job on that:

>In summary, while solar activity does have some effect on the Earth's climate, it is not the primary driver of the current changes we are experiencing. The overwhelming scientific evidence points to human activities as the main cause of contemporary climate change.

So it's possible for LLMs to figure things. Also re humans we currently have riots in the UK set off by three kids being stabbed and Russian disinfo saying it was done by a muslim asylum seeker which proved false but they are rioting against the muslims anyway. I think we maybe need AI to fact check stuff before it goes to idiots.

[Mtinie]

Training weights are gold.

[ionwake]

How to invest tho