[smallerize]

Instead of having to go out of your way, there's an inviting "create a fiction" button. https://www.digitaldigging.org/p/how-to-plant-a-nuclear-plan... In February this year, someone manually forged a photo like this. "The Tehran Times posted it as a before-and-after of a US radar site in Qatar... AFP and BBC Verify both ran the “after” through Google’s watermark detector and got a hit, though not before it passed 950,000 views. Building it had taken six steps — screenshot Earth, open Gemini, upload, prompt, download, crop... Tonight Manama can be changed in seconds."

[Amezarak]

Right, I understand that you were able to generate these images from Google Earth. But nowhere can I find, including in that article, where other people could see your generations presented in Google Maps/Earth as if they were real imagery. The whole complaint is that people were able to generate these images at all. Or am I misunderstanding something? It seems like the complaint is that hypothetically you could generate these images and then share them elsewhere and try to trick people. Is that accurate or am I misunderstanding?

[smallerize]

That's correct. The end of the article also covers defeating the AI watermark.

[Amezarak]

Then frankly I find this all overblown and somewhat insane. The AI safetyism moral panic is something I find very demoralizing. It seems like the main legacy of AI will be its use as leverage to annihilate computing freedom and privacy for all time.

I am sympathetic to arguments that quantity has a quality all of its own - for example, I think this has deeply impacted privacy. It may have been legal to discreetly follow someone and note their movements for most of history, but because it wasn't scalable, there was no major privacy threat for doing so. Now it can be done at scale and very cheaply. "Quantity" has qualitatively transformed privacy in practical, real terms.

But nearly all the AI panic seems to be oriented around very different core principle - rather than "technological changes are making us less free in practice" it is "technological changes threaten public harmony." This is the core concern in all authoritarian empires. The idea is that people simply shouldn't be allowed to generate images or texts The Experts have decided they shouldn't be allowed to, because (in this case) it might make journalists' jobs harder, or it might trick too many people, or its offensive, or it offends some really vague notion of "safety" (or can be construed as such by people writing breathless articles for engagement dollars that will be bad PR for the AI company involved.) Yes, doubtless many people will be deceived by AI imagery. Many people already are. This number will certainly be higher than the number of people deceived by altered photographs, which are less in number and require more work to create. But this does not represent a earth-shattering disaster that must be dealt with by banning (somewhat dumb and silly) product ideas like "generate an AI photo based on this satellite imagery" buttons on Google Maps. It can be dealt with by cultural understandings of any imagery shifting, and a general return to older (not unprecedented) information environments. I think in the West, the rise of broadcast and state media really broke people's mental models of information - it really inculcated the idea that people cannot be trusted even in a democratic society to evaluate things on their own: someone must be doing it for them, top-down. The way things worked in the pre-broadcast environment is maybe inconceivable to them now. (And to some extent, there was always a "parallel" information environment that was in the background, but not highly visible to the people in charge, full of conspiratorial nonsense - but they could ignore that because they didn't see it.)

The ultimate endpoint of AI Safetyism is local models are illegal or highly regulated, computing devices are locked down, and only a small list of government-approved AI vendors with tight restrictions are allowed to Do AI. This is totally unnecessary, but it certainly has massive financial benefits for those who thing they can get on the list.

[smallerize]

I don't know what you're complaining about but I don't think it's this. There is no "ban" on generating AI images. The backlash is not about new tech because there is no new tech here, it's just Nano Banana 2. And I haven't seen any privacy implications.

The button makes it socially acceptable to generate misinformation. That's it. You can do the misinformation if you want, but we should discourage it. The button should go.

[Amezarak]

The people writing about this are complaining that this is even possible, not just that Google was enabling it.

> The button makes it socially acceptable to generate misinformation. That's it.

No, it does not. It makes it socially acceptable to use AI for fun. Yes, moralizing scolds hate that idea and want to talk about how bad actors can and will use it for misinformation purposes as well. Even some of the examples in the thread starter are clearly just from people having fun. In general the whole "safety" and "policy" thing is stupid - consider the "Eiffel Tower" example. I guess that's "unsafe", even if I just generated it to laugh at, or to use in my forthcoming webnovel about kaiju. No, we can't allow Google Maps to let me generate that image because what if I was a Russian bot using it to generate misinformation to convince gullible people who believe random social media accounts that the Eiffel Tower collapsed? It is very obvious where this line of thought leads.

[smallerize]

Can you show me an example of someone calling for government regulation or a ban?

[card_zero]

That's reasonable, except "It can be dealt with by cultural understandings of any imagery shifting, and a general return to older (not unprecedented) information environments". What does that mean? Is the older information environment books and newspapers? Then only a tyrant could arrange a return to that state, which didn't prevent the readers being credulous anyway. So the problem can't be dealt with in this way, there's nobody to do the dealing, and we wouldn't want them to deal with it even if they could. You can only hope that people wise up to slop, but we've been hoping that people in general get wiser since the dawn of time.

To some extent, mind you, people really have got wiser, and the wising-up process requires freedom. This is a problem similar to the draconian censorship of enemy propaganda, which wins wars for the good guys at the cost of throwing their good guy status into question.

[Amezarak]

> What does that mean? Is the older information environment books and newspapers? Then only a tyrant could arrange a return to that state, which didn't prevent the readers being credulous anyway.

No, not a return to books and newspapers, but a return to where there was a huge array of distributed and untrustworthy information sources, with the central authorities often having only a weak influence on them. The world was rife with gossip, rumors, and misinformation that spread mainly through word-of-mouth and papers and pamphlets of various kinds many of which had very little regard for the truth. As was complained of very often and sometimes led to quite amazing consequences.

Of course, in this environment, most people recognized that everything should be taken with a grain of salt, because they knew quite well that people could just make things up and there was no easy way to disprove it. That would be the environment we'd be returning to, to some degree - we'd actually still be better off, in that an educated and informed person would usually still be quite capable of determining if it was true or not in many instances, thanks to the good part of the Internet.

[customguy]

> we'd actually still be better off, in that an educated and informed person would usually still be quite capable of determining if it was true or not in many instances

Oh, so "we" doesn't actually refer to humans in general.

Gullible people can now automatically be detected and exploited, gnawed down to the bone and thrown away with none of the people who harvest their resources ever meeting or even thinking of them, but you are too smart for that, so it's fine. You don't need there to be a public space that is rational and allows humans to grow in, you had parents, you had an education, and there isn't anyone dear to you that you'd have to worry about either.

But at the same time demanding the public to support more of what benefits you. Why? What's in it for the majority of people to accept something they don't like just because you like it? Why should they accept chaos because you think you'd rise above it? What kind of unilateral partaking in society is that? Frankly, this seems so callous and myopic I think you would last nowhere near as long in a true post-factual-reality pandemonium as you think you would, either.