[CMay]

The usage of the output is probably considered legal. The usage of the service for that purpose may not be, and using it at scale in a dishonest way is not, which is what China has been doing. Countless thousands of separate requests abusing the service (which is not a simple static HTML feed, but an AI service request) for every kind of query to soak up the results.

The post is about what's in the local code, but for a long time there has already been modifications made to the request outputs from the major cloud services as they work together to both curb adversarial distillation and to degrade the quality of training China can get from that distillation.

It's likely not to make the answer wrong or bad, but to make it so that any model trained on the output would not gain the benefit of the model's reasoning generalization skills as easily and also identifying markers that might even link back to request IDs.

The techniques talked about in this post are naive and simplistic, largely because they are released publicly.

It's not as much about protecting IP as much as it is about slowing China down or being able to track the effects of abuse. So many people are talking about greedy company this, greedy company that. The world is not made up of caricatured giant money pigs wearing suits with monocles and gold watches. That is a children's view of Marx's exaggeration on free markets. Bad, greedy people do exist, but if that is your only hammer for every nail then you have a problem.

The upper-bound for how good these models can be is so crazy that it is essentially dual-use military applicable to an extent most other technologies are not. It's not only cyber attacks or biological weapons. Most people are not even built to understand the possible threats.

Why does it matter if China gains those capabilities? I invite you to begin to learn about China's behavior around the world. The CCP is darkside material.

[bayindirh]

> Why does it matter if China gains those capabilities? I invite you to begin to learn about China's behavior around the world. The CCP is darkside material.

Reminds me of this comic: https://xcancel.com/tomgauld/status/571994690289061888?lang=...

None of the superpowers in this world is innocent, and like MAD, more countries have the capability, the better.

I know some of the things CCP do/did. I know some of the things US does/did. I'm from neither, so I don't take sides.

AI's use has been confirmed, or more precisely boasted by two countries in two different wars, and China was not one of these countries.

We have seen the effects of "if they don't know them, they can't exploit them" mindset of NSA for years. Keeping information/technology private is neither beneficial, nor possible. It's only a temporary moat-ish gap. Not a definitive solution.

[CMay]

Certainly the world is full of actions and reactions, nothing is happening in a vacuum. You don't have to be from a country to take sides, but presumably you have some kind of moral compass, some kind of values around personal freedom or the worth of a human life.

There can be a very real cost, because one side comes from an ideology with a history that wants to conquer the entire Earth which caused World War 2 while the other side is trying to prune the planet like a bonsai to prevent it from descending into total chaos to preserve some sense of international order.

Europe was constantly at war, and we helped stabilize it. Middle East as been constantly at war, and if Iran can be sorted then it will be the closest to some sense of peace it's been in a long time.

We used to be in Japan, Philippines, Germany, Vietnam, South Korea, Iraq, Afghanistan and so on. How many are US territories? None. We aren't out there to conquer the globe and take land. We're usually fighting other people's wars for them, because they're up against better resourced opponents. Meanwhile China is over there building artificial islands, ramming other country's ships, creating ideological police stations in countries around the world to harass people and engaging in the most widespread international interference campaigns in human history.

They do not treat their people well and they do not have free speech. The internet is flooded with their propaganda now, because they have a human numbers advantage.

It's true that given time most advantages are temporary, but there's always that slim chance we could slow them down until the CCP collapses and they could become a more normal country.

[esarbe]

You sound like you've swallowed pro-US-propaganda hook, line and sinker.

The reason the middle-east is at constant war is because colonialist machinations. Same goes for south-Saharan Africa. And the US is a big colonialist player, just ask Vietnam, South-America, Iran, Afghanistan, etc. They all have been attacked by the US because of US colonial interests. If anything, one could make the argument that the PRC is treading much more lightly than the US.

That said - I'm not defending the PRC by any way; it's a state-capitalist hell hole that's suppressing workers by denying them any ability to organize and whose political class is purely focused on furthering their own interests and that of the moneyed elite, the common person be damned.

The thing is - so is the US.

[CMay]

If you think those were about colonization, you would be well served to examine history closer.

[frognumber]

Ah yes. The exceptionalism argument.

There's the good "us" and the bad "them."

[CMay]

I didn't make an exceptionalist argument, but if any country's behavior and values can be measured compared to others, you will always be able to make some kind of decision about where those fall in terms of goodness or badness. Do you not believe in good and bad?

[frognumber]

I believe in good and bad.

I don't believe in US good, non-US bad. I also don't believe the same about my religion or my political party, for that matter.

How you measure depends on weights you assign (cultural system of values) and what information you use (media bias).

You can rank in the extremes (e.g. North Korea as worse than Belgium), since they come out that way by almost any set of information and values. Comparing the US to most other countries, there isn't a clear ordering. If you believe the things you wrote, I think the other comment summed it up well: "You sound like you've swallowed pro-US-propaganda hook, line and sinker."

Most countries have similar propaganda, by the way.

[CMay]

Most countries have some kind of story they tell about themselves. New Zealand doesn't have any illusions that it is a superpower. It doesn't have the resources, the talent pool or any of that to even begin to dream of it. That is natural. If New Zealand was powerful, then it would find itself in a position of greater responsibility.

You cannot compare countries that barely have the option of ambition with something like the US and even begin to imagine that it is meaningful. You have the UK, France, Germany, Japan, Russia, China and the US. You can rewind history to name other civilizations.

These kinds of countries are the only ones that matter, because they're the ones that have to answer about what people were thinking when they chose to make use of their power in a way that is relevant on a larger scale. They have to answer about what the reasons were when things went wrong and whether they agree things went wrong. If people are even allowed to talk about it.

It's very cheap to label anything as propaganda without taking the time to appreciate whether it has any merit in terms of the overall behavior of a country or its people. You can always find counter-examples, but how influential are they in the larger picture?

[arkh]

> The CCP is darkside material.

And which country has "Black Sites" peppered around the world to detain and interrogate people they don't like?

[CMay]

The US doesn't have black sites anymore and when it did, the interrogation techniques were chosen to avoid physical harm. The results were bad, we didn't like it here in the US even if they were extreme measures for extreme times and so we shut it down. It had a high error rate and generally didn't reflect what we thought was right.

Meanwhile the CCP regularly abducts its own citizens and executes more people than the entire world combined.

[Natfan]

they just hand off to CECOT[0] or ICE[1]

[0]: https://en.wikipedia.org/wiki/Terrorism_Confinement_Center [1]: https://en.wikipedia.org/wiki/List_of_immigrant_detention_si...

[dgroshev]

China? https://en.wikipedia.org/wiki/Chinese_police_overseas_servic...

[tripzilch]

> The usage of the output is probably considered legal. The usage of the service for that purpose may not be, and using it at scale in a dishonest way is not

This is literally what the "training AI on copyrighted works is just like a human learning/getting inspired" crowd has been arguing though.

Literally. People have been literally saying that it was wrong because they did this "learning" at scale in a dishonest way.

[CMay]

In some ways it's an offshoot of the honest benefit of search engines already crawling all this content. That has its own conflicts, like just how much of a page's content should you reproduce in the results before it's basically considered stealing their content without benefiting the site itself.

There is a balance to strike, both in search engine fair use cases and AI fair use cases. The major cloud LLMs do double as web search engines now, though they didn't originally. In many cases there's no reason left to click the links they sourced from.

That is a legitimate concern. At least within the US, I think there are nuances around fair use and contract law. A lot of companies are getting paid for having their content used in these models, but many websites had no particular rules you had to abide by and the content was simply public. I think if you're operating under an agreement, then even if there is fair use or public domain content being reproduced by the site you are still bound by that agreement.

Similar to old paintings digitized and hosted on some museum website. It's 300 years old, right? It should be public domain, yet the people who digitized it or provided a service to give you access have some say in how their reproduction can be used. These AI services are obviously very different, but there are laws that can govern how you are allowed to use a service if that service has laid out acceptable usage.

I'm not exactly comfortable with the mass scale that everything was soaked up to train these models even within the umbrella of search services, but I also admit that a lot of the usage was probably quite legal. The potential displacement caused by the resulting trained models on artists or writers is almost its own facet. In practice, whether they ONLY trained on strictly legally acquired fair use content with no errors and paid agreements to acquire even more content than they already do or not, there was enough legally accessible information for fair use that there was no escaping some kind of impact on artists, writers, etc.

With any luck, artforms and skills impacted by technology will adapt and continue to be valuable instead of complete displacement or the dilution of opportunity.

[tripzilch]

Well it was also problematic when the search engines started quoting the websites in such a way to disincentivize people from visiting the actual website.

> At least within the US, I think there are nuances around fair use and contract law.

The concept of "fair use" as it exists in the US-law system is completely dysfunctional (see e.g. nearly every educational music channel on YouTube), so utterly biased to favour large corporations, that there's very little room for whatever "nuances" you believe exist.

> Similar to old paintings digitized and hosted on some museum website. It's 300 years old, right? It should be public domain, yet the people who digitized it or provided a service to give you access have some say in how their reproduction can be used.

Yes 300 year old paintings are public domain. Indeed there are certain rules for the people/institutions who digitize them. It's not "they have some say", there's actually nothing mysterious about it and it is not similar to Anthropic's copyright heist at all because nearly all of the books they copied were not more than 100 years old.

> there are laws that can govern how you are allowed to use a service if that service has laid out acceptable usage

well where I live, there are laws about what a "service" can claim to "lay out as acceptable usage" instead of the other way around ...

> I also admit that a lot of the usage was probably quite legal

Let's disagree on that. I think it wasn't a lot and the vast majority was not legal. How do you think the LLMs "learned" to speak all these non-English languages? Unless your point is that it's probably quite legal to treat foreign IP like that. Which it may very well be in the US, especially if the corporation is large enough, but imvho it's still wrong.

> With any luck, artforms and skills impacted by technology will adapt and continue to be valuable instead of complete displacement or the dilution of opportunity.

And with any bad luck, these AI corporations will hold frontier models hostage for the rest of time.

I honestly don't want to put that up to "luck".