[kortex]

What's to stop someone from downloading an open source model, running it themselves, and either just not sharing the hashes, subtly corrupting the hash algo so that it gives a false negative, etc?

Also you need perceptual hashing (since one bitflip of the generated media alters the whole hash) which is squishy and not perfectly reliable to begin with.

[alkonaut]

Nothing. But that’s not the point. The point is that, to a rounding error, all output is made by a small number of models from a small number of easily regulated companies.

It’s never going to be possible to ensure all media is reliably tagged somehow. But if just half of media generated is identifiable as such that helps. Also helps avoid it in training new models, which could turn out useful.