[SXX]

Its money and power. This is all these people care about just like almost everyone else.

Or might be deep inside they relly care about it, but that $2,000,000 / year salary and $10,000,000 stock option just overpowered them.

Safety my ass.

[usef-]

What do you think they would do differently if they were genuinely worried about the safety?

[SXX]

I think those who care about safety would try to push for how 99% of all scientific research is done - in universities and actual labs, with transparent information on red teaming results.

Also with international cooperation like how humanity regulate actually dangerous stuff: virus and vaccine research and nuclear energy.

Not hidden behind walls of 10 commercial organizations where each pushing for commercial adoption and IPO like ASAP before bubble bursts.

Not lying and scaremongering public into how their models will replace everyone tomorrow or destroy civilization via cyberattacks.

[usef-]

That line of thinking (public goods) is why the same people started OpenAI as a non profit originally.

Notice how almost no Universities are producing large models?

A key problem is that orgs can't get enough funds to stay on the frontier. And they believe they must be on the frontier to do (and apply) safety research. OpenAI needed to spin off a for-profit subsidiary to accept investment to build things.

And it seem(ed) hard to get one government to fund and take safety seriously, let alone an international cooperation.

If they started a university/gov cooperative to solve this, do you think they would do less of the "scaremongering of the public" talk? My guess it that it would be similar.

The same kind of restrictions that you hint at (eg, treating it like a public virus research) are why they rub a lot of people the wrong way in the corporate world, I think. Normally companies downplay the risks of their own products. See cigarette companies. Anthropic do still publish safety research and red teaming info. But I do think they honestly believe they can't do this work without the resources of a company, and they were burnt by the non-profit structure (Anthropic has a "Long-Term Benefit Trust" instead).

We should definitely keep them to account, but I don't personally think Anthropic have acted in a way broadly inconsistent with safety belief yet. Many of these decisions are self-serving too (eg. protecting models) so they also haven't been seriously tested, either. But the individuals do have a very long history of talking about it (including hurting their own reputation) from even before the chatpgt-moment money train rolled in.

edit: for clarity, but still messily/quickly written