[zmmmmm]

we should turn this around

Private models should be banned because they can't be transparently evaluated. We have to trust the same entities that made them to evaluate them, in spite of their gigantic conflict of interest in doing so.

Therefore only open weight models can be allowed, since this allows genuine third party evaluation.

[0xDEAFBEAD]

>We have to trust the same entities that made them to evaluate them, in spite of their gigantic conflict of interest in doing so.

Don't we already have third party NGOs such as METR which do risk assessments for unreleased, closed models?

[zmmmmm]

The argument is no. We don't / can't trust those because they don't release the weights. How do we verify what they got evaluated is what they actually serve and run.

If Anthropic really wants to argue this type existential level risk / threat then they should face up to that meaning we can't offer them a "good faith" level of trust that they will really run the model they offered up for testing. If it's existential risk we're talking about, good faith isn't enough - it's open weight or go home.