[nullbio]

Obviously they are looking at your IP and code. Anthropic trains on your data regardless of you opting out, I know that one for certain. There's no coincidence they "keep your data temporarily despite opting out" - because they wash it in legal loopholes. There is no opt out. These companies WILL steal your business. Only a matter of time before they are sued as well.

[azinman2]

> Anthropic trains on your data regardless of you opting out, I know that one for certain.

How do you know that?

[vorticalbox]

Well we don’t know but just look at figma, claude for it happened and then Claude design come out.

They knew exactly how developers worked from using figma as training data.

[llelouch]

Professionals still use figmas. Claude design jsut expanded the space. Non-design specialized use Claude design now.

[preg_match]

Sure but the intention was to compete with Figma, and they are to a degree.

The success of their tomfoolery is really a separate issue from if they are engaging in said tomfoolery, which I think they are. I also think anyone expecting honestly from this is naive.

Bad people doing bad things are not stupid, in fact they're usually pretty smart. Nobody is gonna come out and say what's going on, because that's purely self-destructive. We will get leaks and lawsuits slowly, much like OpenAI.

[general1465]

They systematically violated copyright when they grabbed whole internet to train their models. Do you really believe that they will stop stealing because they signed some funny ToS? Especially when every bit of data they have and competition does not have is making their model better.

[olalonde]

There's an important difference between "knowing" and "believing".

[rcxdude]

This. People repeat stuff they suspect might be happening like it's facts. Would I be surprised? Only a little. Does that mean it's definitely happening? of course not.

[Zigurd]

Blame copyright law. AI training is very comparable to compiling an inverted index, which is considered transformational even though you can recover the input documents from an inverted index.

[derektank]

To be maximally charitable, while creating large language models has been ruled to be transformative and thus fair use under current copyright law, Anthropic did separately violate copyright law when procuring copyrighted text from LibGen and Pirate Library Mirror. That being said, I agree with you, creating LLMs from copyrighted material is very clearly not a violation of copyright under the current legal regime, so long as you procure the text legally, be that through web scraping or by purchasing books directly. And it’s annoying that people try to muddy the water on this.

[nullbio]

"be that through web scraping or by purchasing books directly" - or by creating synthetic iterations of the inputs your LLM API service receives.

[Zigurd]

I am an author of a couple of books that are part of the Anthropic settlement.

[cromka]

People downvote you like you're being paranoid, but we're literally discussing this in a thread that shows how little respect those companies have for any sort of trade secrets.

[user43928]

How does the allegation make any sense?

AI labs can hardly just throw random confidential data into the training and then hope it does not leak into the output of their model in an obvious way.

If that would be found it would destroy their main source of revenue, it could became a major national security or healthcare enforcement matter, and result in criminal investigations.

[Barbing]

Some of the smartest people on the planet all in the same room, data at their fingertips… they randomly add it to the training set?

Labs at least must study prompts in an airgapped fashion. From there, consider how they could generate synthetic data to train another model. After, require trusted staff to do multiple levels of independent granular reviews of all fruits of the highest-value stolen inputs. (Or for model training data only, data never has to leave the airgap.)

Definitely risky, anyway. Surely some AI user has sent data, in confidential mode, with a unique shape they expect to be able to recognize if a later model recreated a facsimile even with heavy substitutions… but labs could bring risk of getting caught (over next few years) down quite low with extraordinarily ultraparanoid strategy. (But hopefully everybody is just behaving!)

[user43928]

That is an interesting thought.

They could run some sort of analysis to find high value input, such as proprietary technology, algorithms, or strategy.

Then they could group them together for one specific topic, and produce a report that analyzes if the information is plausible.

If so, they can have it send to staff for review, who could then create a test set that rewards the model for going into the direction of the proprietary solutions known to work.

I'm no expert, but at least something like that sounds plausible to me. I still very much doubt they are doing this.

[nullbio]

That's not at all how they do it. They wash the data. The end result is they can steal your IP insights without it being explicitly tied to your business. All of the decisions you made to build your product? Those become the standard suggestions in the next model for people building the same sorts of products. All of that error correcting you did, which in a normal business would be considered hard-earned IP (like in the case of Apple's lawsuit - the "what not to do" is just as valuable as the "what to do"), is now free correction for the next model to produce the perfect result in a one-shot. Now the AI company can commoditize your labor and your industry, or compete against you if they so wish.

[tsimionescu]

And yet it seems quite clear that they have been directly instructing Apple employees on how to steal data from Apple and bring it to OpenAI - explicitly stealing internal trade secrets from the largest company in the world, one known for its paranoia about leaking any company secrets.

[cwillu]

I downvoted because the reply has nothing to do with the argument.

If I know for a fact that you're cheating on your wife, and someone else asks how I know that, then a third person chirping about your sketchy business dealings is entirely irrelevant to the question, no matter how much suspicion it might otherwise raise.

[everfrustrated]

>Anthropic trains on your data

This is why companies are wanting the AWS hosted models because they trust AWS running of the same models more than the vendors themselves.

[frenchtoast8]

Only for Opus etc. For “Fable 5, Mythos 5, and future models on Bedrock with similar or higher capability levels,” your usage data is retained for 30 days and shared with Anthropic which you can’t opt out of.

[angulardragon03]

This is why we don’t use Mythos/Fable-series models at $WORK (hosted on Vertex).

[nullbio]

Playing with fire.

[Chu4eeno]

Weren't they required by a court to keep everything, despite the privacy policy etc.? Or am I mixing up my companies in constant law battles.

[SpicyLemonZest]

They were required to keep all non-European user logs for a temporary period between April and September 2025, because the media companies suing them think these logs may be the only evidence in existence that could prove or disprove their alleged misconduct.

[hhh]

How is it obvious? We have strong legal agreements that state otherwise, do you think they are just lying and risking thousands of lawsuits?

I think it's more likely that there are 3/4 of a billion users that don't have these agreements and just pay for ChatGPT Plus and don't opt-out of anything, and are feeding the scaling machine every day.

[nullbio]

> do you think they are just lying

Yes. They're constantly lying, and constantly getting caught for it. They have a reputation for it. Why do you think this would be any different?

Their standard opt-out agreement frames it as if they won't train on your data, but they do anyway, due to legal loopholes. They essentially clean-room everyone who opts-out, so while it's "technically" not training on "your" data, to the model it makes no difference. Your alpha and IP is not safe. Paying customers are now more easily able to clone your business as well, not just Anthropic themselves.

The only reason this hasn't leaked yet is fear. Anthropic is a very litigious and dangerous company. Only a matter of time though, someone there will grow a spine and speak up.

[aesthesia]

Can you elaborate on the loopholes here?

[nsagent]

I'm unwilling to speculate whether or not OpenAI is breaking their agreements (I honestly have no clue), but as an NLP researcher I'm certain they could launder data by having an LLM rewrite it and subsequently train on the rewritten data.

Papers like "Curated Synthetic Data Doesn't Have to Collapse" [1] and "How to Synthesize Text Data without Model Collapse?" [2] demonstrate it's possible to do this.

Since OpenAI's Privacy Policy [3] explicitly allows for the use of deidentified data, it's possible they consider rewrites (maybe paired with a model used to identify explicit PII) to be deidentified. Whether OpenAI's legal team thinks rewriting in this way technically means they aren't training on your data isn't something I'm able to comment on.

Here's the relevant Privacy Policy statement:

  We also aggregate or de-identify Personal Data so that it no longer identifies you and use this information for the purposes described above, such as to analyze the way our Services are being used, to improve and add features to them, and to conduct research. We will maintain and use de-identified information in de-identified form and not attempt to reidentify the information, unless required by law.

Please note all the hedging words I used (maybe, possibly, etc). I honestly have no clue if they are doing this. I'm merely elaborating on a possible loophole like you asked.

[1]: https://arxiv.org/abs/2605.07724

[2]: https://arxiv.org/abs/2412.14689

[3]: https://openai.com/policies/privacy-policy/

[nullbio]

It's exactly what you've said. I'm speaking about Anthropic though, I have no idea whether OpenAI does it.

> Anthropic agrees that Customer (a) retains all rights to its Inputs, and (b) owns its Outputs. Anthropic disclaims any rights it receives to the Customer Content under these Terms. Subject to Customer’s compliance with these Terms, Anthropic hereby assigns to Customer its right, title and interest (if any) in and to Outputs. Anthropic may not train models on Customer Content from Services. “Inputs” means submissions to the Services by Customer or its Users and “Outputs” means responses generated by the Services to Inputs (Inputs and Outputs together are “Customer Content”).

This is the only commercial ToS clause about how they handle your data for subscription users. They only promise not to train the model on your exact input and exact output. There's nothing about not washing your data - the "clean-room" approach, which is obviously easily automatable by a company that specializes in automation. That is not training a model on your data, it is using a model to create derivatives of your data, then training it on "their" derivatives.

People really needs to apply pressure and start demanding answers from these companies regarding this - because it is a huge problem. Historically, the amount of labor required to do something like this would make it entirely unfeasible, so this is all new territory. The existing laws and the requirements around clarity surrounding these conditions do not reflect the technology progress.

[includenotfound]

That's in fact also very technically feasible. They're already running small classifier models on both your inputs and your outputs. These models are said to be in place for safety, but they could answer 2 questions: (1) is this a safe query, and (2) is this query about X, Y, or Z topics (which happen to be high alpha / very interesting to Anthropic).

If #2 returns a yes, the input/output are temporarily stored (say, for 30 days) while a background process cleans up PII and generalizes the prompt to be "clean", then the clean prompt that is not yours anymore is stored aside and used for training, product development, etc.

[azinman2]

Just to be clear, you’re speculating, right? They could also not be doing this.

[aesthesia]

Yeah, I guess that's possible. Anthropic has published some information about systems they use to monitor usage patterns, which indicates they have some related infrastructure set up.

https://arxiv.org/abs/2412.13678

[wuschel]

Side question to you, considering your occupation:

Could you please list a set of core papers (or other resources) that give a beginner an overview or even understand of the fundamental concepts and techniques with LLMs?

Thank you!

[nsagent]

I'm sorry, but I'm not sure what works best for a beginner. I started my PhD when the original Transformers paper [1] had just been released. I had no background in NLP whatsoever and used the original paper to write a Transformer and the full training pipeline from scratch during the first couple months of my PhD without referencing any existing code (only reading the paper and it's references).

So I'd say, if you're motivated you could do the same. That said, I've always been a self-starter and I started my PhD after working for a decade. I'm sure there are other resources out there, but I'm not equipped to say what's best for a beginner (I found the original paper to be excellent, but most everyone during my PhD, including my advisor, found it to be inscrutable; I think it's written more like an engineering focused paper, which might be why researchers found it difficult to grok, but with my previous industry experience it seemed quite clear).

[1]: https://arxiv.org/abs/1706.03762

[throw1234567891]

ChatGPT has 3 to 4 billion paying users? And still haven’t turned a profit?

[Laurel1234]

Think he meant three quarters of a billion. Although that seems way too high for paying users still.

[notenlish]

They said 3 quarters of a billion