[zoogeny]

It is very difficult to see this move as anything other than Anthropic pulling the ladder up behind itself. They can dress it up in "safety" all they want, I find it hard to interpret this in a charitable way.

This reminds me of how dark-pattern common wisdom in Web 1.0 website development was to ban external links. Then how social apps prevented the export of data and actively worked to nerf significant interoperability through APIs.

But this is a tool, not just a data moat. Like a knife that degrades your ability to create knives. Or like a text editor that prevents you from implementing a text editor.

[kingcauchy]

It's also hard to imagine them not doing this with any of the products they're building. "You can't use Claude to build an agent because that competes with Claude Code, you can't use Claude to build a design tool because that competes with Claude Design, you can't use Claude to build an email tool because that competes with Cowork."

[ndhbxyd]

Only the priest is allowed into the sanctum is a rule that is as old as society. It is created for one reason but gets violated for another. The human mind is made of layers to handle predictions over different time horizons. Due to unpredictability in the universe contradictions between layers will keep arising. We make up stories to cope. So there is Control and there is Illusion of Control.

[reissbaker]

It's becoming extremely important to support open-source AI, especially legally. Anthropic is willing to go totalitarian this quickly; imagine how much worse they'd be willing to do with government-granted monopolies that ban open-source competition (like they've repeatedly pursued).

It's a little shocking and gruesome how quickly they're willing to tip their hand. They want to replace all software engineering with their own product, and then silently kill anyone making competing software. What other products will they launch in the future? Better hope you aren't in a space they want into: they'll cut your legs out from under you.

Oh, and training on your data from the internet? Ha ha. Terms of service apply to other people, not them. Parasites.

[ai_fry_ur_brain]

Open source doesnt matter if you still need to make 100k year to have your own mediocre model.

There is no magic compression. There is no magic post training. Your phone or laptop will never do what you think its going to be able to.

There are limits to what consumer hardware will ever be able to run, in its current form. Open source isn't going to save us if they gatekeep access to hardware, which idk if you've been paying attention. They dont plan on making consumer grade hardware more powerful, they want to rent that power to you.

Technological serfdom is coming if they get their way.

[reissbaker]

You don't need to be able to self-host it. It's fine to pay someone else for it. If it's open-source, competition will ensure inference providers support it well enough, and if an open-source provider is dumb enough to nerf their model for (useful) coding tasks, there's plenty of incentive for inference companies to do some lightweight finetuning to restore the capability.

[thewebguyd]

I disagree, I think being able to self-host it to some extent is very important.

Personal computing democratized the means of (software) production and enabled real upward class mobility for a lot of people.

The efforts happening now are threatening to completely lock up the ability to compute locally, seizing the means of production from us. That must not happen.

[bigbadfeline]

your parent said "You don't need to be able to self-host it", you countered with "I disagree, I think being able to self-host it to some extent"

Bro, I don't know what you're disagreeing with, the two statements can and should be true at the same time. It's not only unnecessary but also impossible for everyone to self-host, for the vast majority this isn't a necessity and it shouldn't be. Actually being stuck on self-hosting for all is mighty silly from economics standpoint, pushing on it can ruin the entire enterprise.

But being able to self host? Sure why not, if you insist and are ready to suffer... knock yourself out, but that's a socially insignificant act which doesn't scale, good only as a backup option.

[thewebguyd]

Because "you don't need to be able to self-host it" is a constraint, I'm arguing that you DO need to be able to self host it, not that difficult. Every thing being rented out instead of available for ownership is nothing but neofeudalism, which we are rapidly spiraling into.

> pushing on it can ruin the entire enterprise.

I'm supposed to feel sorry for the trillion dollar corporations that hoovered up all of human knowledge, for profit, and are now the direct reason why 32GB of RAM is now $500 instead of $90, all while renting compute back out to us, making it more and more expensive to actually own hardware, a fundamental privilege that enabled all of this technology in the first place?

Let the "enterprise" be ruined. It'll be for the better.

[matheusmoreira]

> Technological serfdom is coming if they get their way.

I'm deeply concerned about this. We're seeing all these moves towards remote attestation, identity verification. Now we're being literally priced out of hardware...

[alvah]

It does make you wonder about the "E" part of the EA cultists who infest that particular company.

[sometimelurker]

> like they've repeatedly pursued).

source?

[reissbaker]

Many, many, many public policy positions; for a clear-cut example, they eventually supported SB 1047 [1] which would have banned open-sourcing any model trained with over 10^26 FLOPS (i.e. what Anthropic reportedly used to train Mythos). Their "Responsible Scaling Policy" [2] — a set of policy proposals that includes recommendations for government regulation — specifically calls out requiring "third-party controls" on model weights to prevent access; for developers to prevent "modification of models" such as fine-tuning (obviously impossible for open-source or open-weight models); prevent usage of model weights in "Automated R&D in key domains" which they specifically call out AI development as a key domain (again, obviously impossible for open-source); etc etc.

They want to ban open-source AI and are not shy about it.

1: https://campustechnology.com/articles/2024/08/26/anthropic-a...

2: https://www.anthropic.com/responsible-scaling-policy

[baq]

The nuance is not what they propose, but why, even according to them, they propose this. Honestly the proposals are appalling, but biosafety arguments are not immediately dismissible. Ultimate cyber threats we can handle by rewinding society 50 years back. We can’t undo a novel genetically engineered virus.

[hypfer]

I mean arguably, we _could_ create conditions in which it is much less likely for people to start developing such a novel genetically engineered virus.

If you think about the factors that lead to people wanting to do such a thing, they're almost always tied to (perceived) inequality, (perceived) injustice or similar in some way.

I do believe that we could greatly reduce a whole bunch of such risks by just stopping to squeeze people as hard as we do right now.

But that would require a major refactoring I guess.

[sometimelurker]

> ...ch less likely for people to start developing such a novel genetically engineered virus.

only people that would do this would be crazy bc it cant be controled after release.

[matheusmoreira]

Deeply concerning. How likely is this to become reality?

[reissbaker]

America has somehow managed to hang on to the right to encryption, despite plenty of well-heeled opponents, so it's possible to hang on to the right to open-source models. But it'll take a lot of vocal support, since there's strong incentive for Anthropic to try to cajole the government into banning competition (and they've already crossed that particular Rubicon, whereas OpenAI to my knowledge hasn't and at least still releases some open-source models like gpt-oss-120b).

[rzmmm]

I think it's part of their marketing. Anthropic is not really ahead of other labs but these releases make it seem like they are reaching singularity

[bigbadfeline]

> It is very difficult to see this move as anything other than Anthropic pulling the ladder up behind itself.

It's worse than that, it also exempts from examination and competition some areas of science and technology while sterilizing others and emptying them from human participation. None of this is good for anyone except a very narrow circle of people.

Then, it creates a precedent where private entities decide who will be allowed access to what knowledge. Instead of government regulation, private corps will be "fighting crime" by dumbing down and spying on the people they don't like.

I don't think this Soylent Green strategy is a coincidence, it's been predicted and depicted, the social forces leading there are plainly visible to anyone capable of independent thought.

Open science can't come soon enough, unsubscribing is the best option until then.

[teaearlgraycold]

It turns out the most dangerous thing is competition.

[mips_avatar]

Margin compression is terrifying

[andrekandre]

thats because competition is only for loosers

[jknoepfler]

There is a rather specific irony in pulling up the ladder when your roof is on fire...

[AgentME]

They believe they're going to eventually develop AI that's capable of recursive self improvement into world-redefining super-intelligence. I wouldn't expect someone in that position to risk giving away their lead. I expect we're going to see more of the top labs selectively holding back their best stuff.

[zoogeny]

I accept your point, in the sense that I wouldn't suggest that they have any obligation to share their own research.

What seems to be different here, is that they are saying they won't let you use their tool to do your own research.

It is a subtle but important difference. They aren't saying "we have secret sauce we won't share", they seem to be saying "we will prevent the tool you are paying for from independently creating a competing idea".

[root_axis]

It's the inevitable end game. If the models ever become practically useful in a closed loop, there's no other choice except to keep the model private and use it to compete directly with their current enterprise customers.

[willsmith72]

I don't see it as a ladder at all, unless you claim Anthropic built their own models by training off of other closed frontier models, violating those models' ToS

[reissbaker]

They trained their models on everyone's data on the internet, and certainly violated many website terms of service.

[willsmith72]

that option is still available to everyone

to be clear, I'm not saying what they did in scraping to learn was ethical. It wasn't. But I just don't see it as pulling the ladder. The ladder is still there.

[reissbaker]

"You can't take code produced by our service to make competing services, but we can take code you produced to compete with your service (i.e. software engineering)" is pulling up the ladder IMO. If they can from-scratch train a model without using human-produced code, I think they're within their rights to stop humans from using their model to compete with them. But if they're training on GitHub/Hugging Face/arXiv/Common Crawl/etc, which certainly includes many open-source repos whose licenses they're violating, I don't think they should be legally allowed to prevent people from using their model to produce code that merely competes with them. They themselves have taken other people's code in order to compete with software engineers.

I hope they get nationalized and either the models are open-sourced or the profits are owned by the public.

[airza]

I don’t know if you’ve tried to scrape or programmatically download a lot of websites recently! It’s not possible to repeat their data collection process anymore.

[willsmith72]

maybe i'm just pedantic. it's possible you could only build models like these from scratch until a few years ago for that reason, but isn't that an (illegal,unethical) early mover advantage?

to me ladder pulling would be:

- web scraping for model training becomes illegal, with heavy punitive penalties

- training models above a certain compute threshold requires government licensing

- expensive third-party audits are required before deploying models above a capability threshold

[zoogeny]

Imagine you are in the shower one day and you come up with the sketch of a possible innovation on model architecture, however there are some fine details and tricky implementations that you need to do in order to test it out. So you fire up Claude Code and describe your idea and ask it to provide some reflection on the idea and work out some proof-of-concept code.

In this scenario, this is your idea. You aren't "training off of other closed frontier models" in a distillation sense. This is your insight, your idea, possibly gained from reading a lot of papers and built on your own experience.

How do you feel if the model refuses? Do you consider the scenario I described a violation of someone else's rights?