[iamniels]

I like that he comes up with new laws and regulations for AI companies. Can I suggest some more?

- You shall not embed copyrighted material in your models.

- You shall not bombard every little website in existence with 1 million scraping queries per day.

- You shall not use your political influence to pump and dump your AI (or rocket?) company.

- You shall not imperill the whole IT sector by buying all CPU and memory chips.

These new rules will affect every society directly in a positive way. Thanks.

[scarmig]

"We need an approach to make sure AI doesn't destroy the world and wipe humanity to extinction."

"Yeah, and quotas on web scrapers!"

[bastawhiz]

The Debts and Engagements Clause of article VI of the US constitution was kind of a weird little thing to stick in there, but like, it was important to a lot of people at the time and probably helped move the needle to get the thing ratified.

[zozbot234]

> We need an approach to make sure AI doesn't destroy the world and wipe humanity to extinction.

That's easy. Stop training your AIs on cheesy old sci-fi that talks about robot uprisings. In fact, maybe y'all should just stop talking about robot uprisings altogether. Putting a stochastic parrot in charge of an agentic function-calling REPL doesn't somehow make it super-dangerous, except to the extent that dumb mistakes might result in danger. And you can't prevent an AI from making dumb mistakes with burdensome regulation.

[scarmig]

Yes, we get that if you assume there is zero existential risk from AI, then there is zero existential risk from AI.

[NoGravitas]

The biggest existential risk from AI is its contribution to global climate change. The second biggest risk from AI is the potential for AI-generated disinformation and propaganda to spark, or to manufacture consent for, a world war. The risk of superintelligent paperclip maximizers is so low as to be negligible.

[ben_w]

> The risk of superintelligent paperclip maximizers is so low as to be negligible.

Literal paperclips, sure.

But the point of the example was never literal paperclips.

The point is that maximising *any* goal, if it doesn't include what you care about, will annihilate what you care about.

If you don't believe me, consider what you yourself just said about climate change, and why this is a consequence from maximising money spent on data centres.

[morpheos137]

show me an agent that persists productively in a goal without stopping. Does not exist. LLMs run on gradient descent. The agent is looking for the most efficient way to halt. AGI paperclip maximizer woukd likely recognize the absurdity of its goal and shut itself down.

[ben_w]

> except to the extent that dumb mistakes might result in danger

That "except" goes all the way up to starting WW3. Or a leak from a viral research lab, and by "leak" I mean "mail order" and by "research lab" I mean "the companies who already ship custom DNA and RNA retroviruses": https://duckduckgo.com/?q=companies+who+already+ship+custom+...

If you can prove that simply not training on horror stories would work, it would make a lot of people very happy.

Unfortunately, I don't think it does a single thing to solve, for example, Elon Musk just plain asking some future version of Grok to take over the world for him.

Nor would merely failing to include them in traing data stop certain entire fictional scenarios such as that Doctor Who episode where the android repair bots weren't told that the crew were off-limits as spare parts, or the other Doctor Who episode where the utilitarian robots started killing everyone who was upset because they calculated net positive utility from upset people ceasing to exist. Well, except for the bit where the Doctor saves the day, because they are not real.

[comfysocks]

> "We need an approach to make sure AI doesn't destroy the world and wipe humanity to extinction."

"Yeah, and quotas on web scrapers!"

—— I see it as:

“Let’s do a study on abstract future concerns.”

Vs.

“Let’s take action on concrete present-day concerns.”

[hashmap]

Trustbusting should absolutely be included as well. One of the biggest immediate threats is the concentration of wealth into a very tiny number of companies.

[SilverElfin]

Yep we need new laws for this. The current laws end up in years long lawsuits and no real change.

[avaer]

DMCA-style fines should be retroactively + prospectively applied to copyrighted works reproduced by AI, paid for by the AI companies, paid out to the copyright holders whose work was used without permission.

It would not be prohibitively hard to do the math on this.

That would fix a lot of the problems with AI overnight, but it'll also never happen.

[WhiteOwlLion]

Maybe if attribution is available. What do you do for the rest of the ingested content? I think based on the content itself, you assign percentages to the top 3 industries like a naics code. Then whatever Anthropic makes as gross or net, a percentage goes to each industry via assigned bank accounts or USDC addresses via solana or some scalable payment system. Could be the start of ubi or some sort of compensation for jobs displaced by ai usage. So every input gets tagged for categories and every output gets tagged for the same naics categories via federal law.

[tptacek]

It is normal, expected, and healthy for stakeholders in a regulatory environment to offer proposals about regulations. What's unhealthy is the proposition that the deliberation process is so fragile that a stakeholder needs to cover every angle, lest they corrupt the outcome.

[pseudalopex]

It is normal, expected, and healthy to offer criticism of self interested proposals. And mock even. What is unhealthy is to imply someone said what they did not.

[tptacek]

If that's what this is, a bank-shot snarky criticism of the proposal, fair enough. I read it instead as a criticism of a stakeholder having the temerity to make a proposal in the first place. It's not their job to anticipate and capture all your objections. That's your job!

[pseudalopex]

Everyone holds a stake in this discussion. And intellectual honesty is everyone's obligation.

[tptacek]

Yes, that is a shorter way of saying what I just said.

[pseudalopex]

Amodei did not fail to anticipate and capture all objections. He dishonestly avoided well known objections. This was what you meant?

[ofjcihen]

What you say is true but completely ignores the obvious ways in which what he is proposing benefits his company.

Its like saying it’s normal for a taxi driver to drive people places while he’s got you handcuffed in the trunk.

[Spartan-S63]

How is the subject of potential regulation considered a stakeholder?

[zoogeny]

That's a bit like asking how the defendant in a legal case is an interested party.

Even if you think someone is guilty, it does make sense to allow them to at least submit their defense. And if they choose to use that time to advocate for their own promotion, let them.

[akerl_]

They’re the most obvious stakeholder… the regulation is going to directly affect them.

[Spartan-S63]

No, the public at large are the stakeholders. The enterprise is the subject of the regulation.

[sdenton4]

"Stakeholder" literally means someone with a stake in the outcome, which is to say, those who will be affected by the decision. That can include a whole range of people+entities, including citizens (as a group) and the companies to be regulated.

[ben_w]

> Can I suggest some more?

Dunno if you can, but your examples here are the legal equivalent of that time someone asked me about making "Uber for airplanes" without any elaboration on their part when I asked for it:

Far more vague than I think you realise.

You could probably write a book on each of those topics and a hundred others besides.

[sonink]

I see a lot of skepticism in Dario's position in this forum. But allow me to argue the opposite.

I think the key argument that this skepticism lies on is that he himself gained from AI - specifically building Frontier AI models - and this is basically regulatory capture disguised as doomerism.

Fair points - but I think this is a more charitable version of this. Dario is building Anthropic because that is the most valuable thing he can build, or at least that is what his conviction has been. The success of Anthropic and the impending IPO is proof that this conviction has not only been correct but has largely played out very successfully. Dario understands the true nature of AI and he has welded that power to immense personal benefit.

But maybe he also sees the potential danger to AI which he is trying to address through these posts and regulatory initiatives. There are three reasons why I would support the charitable version:

Firstly, personal gain and societal benefit can coexist in the same individual. And both of them might drive towards opposite agendas. But that doesn't necessarily have to mean that the impulse driving the societal benefit is not earnest. In fact if you would look at Dario’s proposal - like closing the data broker loophole - several of them could constrain Anthropic instead of benefitting them.

Secondly, he expects that his concerns on the negative potential of AI will be taken seriously, if he is actually running the Frontier AI company. And there is some truth to this argument. The only reason we are discussing this is because he is the CEO of Anthropic. He is probably the most influential figure outside of the government who has to be taken seriously when he claims something like this.

Thirdly, and most importantly, Dario has previously demonstrated that he is willing to sacrifice personal/corporate gains for societal benefits. The proof is the classification by the US DoD of Anthropic as a supply chain risk when Anthropic refused to completely cooperate with the military to develop fully Autonomous AI weapons and enable mass surveillance. It would have been only too easy for Dario to accept if personal benefit was his only concern - and OpenAI was more than ready to step in their place.

Even with Mythos - Anthropic could have released the model to the public broadly. But they took their time to reduce the potential danger - as best as they could. Despite the fact that GPT-5.5 was nipping in the buds in what is becoming a very competitive market.

That being said, just because Dario is acting in good faith, does not mean that this will all result in good outcomes. The FAA-styled regulation could still end up favoring incumbents - some of whom might choose to not act in good faith. A more diversified capability can potentially limit that power ending up in a small number of wrong hands. Just because Anthropic is the leader right now, doesn't mean that they will always be. Maybe someone else tomorrow benefits from this regulatory capture at the expense of everyone else - and Dario might have a hand in driving it.

[bashtoni]

You missed a couple ofvery important ones:

- Your AI data centres will run only on renewable energy

- Your AI data centres will not use evaporative cooling