[egeozcan]

IMHO you cannot base your trust on something that is so easy to work around.

Trust should be between people, not a person and some data.

So showing how easy to generate "false" data, this makes it more obvious for people focus on other people. Trusting people makes life much easier in my experience, while focusing on data, again in my experience, is a game of cat and mouse.

[InsideOutSanta]

> Trusting people makes life much easier in my experience

Sure, but how do you apply that to a society at large where powerful people are interested in making everybody distrust all reliable sources of information?

AI watermarks are no panacea, but at least they are a clear signal of what not to trust.

[user3939382]

> how do you apply that to a society at large

You can’t, it’s an inherent contradiction. Human social structures have sophisticated and robust evolved mechanisms for establishing and maintaining trust. These dynamics are not one option among many, they are the optimum. By their definition they don’t scale to strangers around the planet. This is an immutable factor in why we have spam, bank fraud, etc. We want the benefits of trust without the cost of local constraints but wishing doesn’t make it so.

[AnthonyMouse]

> Sure, but how do you apply that to a society at large where powerful people are interested in making everybody distrust all reliable sources of information?

Isn't that the scenario the watermarks are useless against? Adversarial governments or anyone with enough money will be the ones who can generate images without watermarks even if you force them on the proles.

> AI watermarks are no panacea, but at least they are a clear signal of what not to trust.

Which seems like it only makes the actual problem worse? If most of them have watermarks, that only encourages people to put more trust in the ones that don't, even though those are the ones "powerful people" can still forge to manipulate everyone. What good is something that increases the credibility of adversarial government forgeries?

[digitaltrees]

Again, just because something is possible doesn’t mean we should make it broadly available. Sure the CIA can manufacture poison to assassinate someone, that doesn’t mean we should make it broadly available.

That being said, I’ll make a counter argument. To the extent that deep fakes need to be so wide spread that everyone becomes skeptical and there is no other way to trigger broad diligence then maybe there is a case to be made. But I am concerned that this will be worse than a more broadly trusted image environment

[AnthonyMouse]

> Sure the CIA can manufacture poison to assassinate someone, that doesn’t mean we should make it broadly available.

Poisons are widely available to everyone. People commonly have liters of them under their kitchen sink or in their garage. Many of them are also simple compounds that anyone can make, e.g. "cyanide" is just CN (carbon and nitrogen) typically attached to H, Na or K. Every one of those elements is required for human life, but combining them in specific ratios under well-known conditions creates a deadly poison. It's so simple to do that hobbyist chemists have to be careful not to create it by accident.

> But I am concerned that this will be worse than a more broadly trusted image environment

There are two different metrics of trust here.

The first is, should you trust an unauthenticated image you see on the internet? Now more than ever before, the answer to that question is no. Watermarks can't fix that because the people you least want to be trusting are the ones who can still forge images without any watermarks. You can't undo this by putting watermarks on other images.

The second is, do people trust random images from strangers on the internet? The ostensible purpose of doing watermarks is to get more people to do that. But when the answer to the first question is no, getting more people to do that is bad, because you're encouraging people to trust something that is not trustworthy.

What you want is for undetectable forgeries to be difficult/impossible in general, e.g. to have something that can tell anyone if an image is machine-generated regardless of whether the machine added a watermark. But watermarks don't give you that since they don't all add them or can feasibly be removed, and a generic algorithm that can detect even novel/unknown image generation methods may not even be possible.

[digitaltrees]

You are over estimating your specialized knowledge. This statement requires more knowledge than I think you realize.

“ Every one of those elements is required for human life, but combining them in specific ratios under well-known conditions creates a deadly poison. It's so simple to do that hobbyist chemists have to be careful not to create it by accident.”

[digitaltrees]

I agree: don’t trust an unauthenticated image on the internet”. But surely a watermark is a helpful feature in confirming its AI so my doubts are removed no?

[survirtual]

AI watermarks empower elite / those with resources and disempowers the common person.

Only people with resources will be allowed to make content that is AI generated passed off as real.

Pandora's box is open. Instead of making a multi-tiered privileged society, we need to fundamentally restructure society to adapt.

Before that restructuring occurs it is critical to keep the playing field level. These are not tools that should be controlled by a minority authority, they are far too dangerous.

[digitaltrees]

That’s a reasonable argument. But why should we tolerate the two tier system? Why not enforce a restriction on all images? Why do we have to accept someone is above the law?

[survirtual]

Because it is technologically impossible to enforce this on the owner class. The people who control compute control the rules. Without proper audit trails it becomes impossible to prove or enforce.

The only ones, then, who could afford to break the law are also the ones who make the law and own the law. Everyone else is subordinate in that model.

That's why technology like watermarking doesn't work and will never work. Implicitly it creates a minimum of a two-tiered society and no amount of "law" will change the technological reality of this.

[digitaltrees]

The people with compute only control the rules society allows them to control. It isn’t law of physics. We have export restrictions on hardware. The government could put restrictions on image generation just like the Movie, TV and music rating system or the water marks in printers that prevent counterfeit money printing.

[survirtual]

All the things you mentioned were massive creative endeavors demanding the talents of countless humans with real-world stakes, moral compasses, and sovereign autonomy. These creatives had rights, they had families, they had parents, and they had a place in society.

Now the things they produce can be done in a fraction of the time without any of the stakes, nor any of the moral society-integrating foundational encoding.

Instead, a single billionaire with an army of GPUs can ask for a thing, and that thing will be produced near instantaneously.

The old way of regulating depended on PEOPLE creating things at PEOPLE speeds in order for the laws to work. That simply is not the case any more.

[goatlover]

The Kentucky primary had $1.7 million spent on deepfake political ads which were seen 49 million times. Don't know how much it effected the result, but it's not a good sign of where things are headed.

[digitaltrees]

Except how do you have trust between people when deep fakes and voice clones are trivial to spin up? Are we going to have to go back to doing business physically in person?

If I got a picture of my mom being held hostage it sure would be nice to have a watermark confirming in was an AI generated image. So again, why make this broadly an easily removable?