[onceKnowable]

There’s precedent here. Einstein notified the US President when it became clear as to the danger associated with, what were then, recent developments in high-energy particle physics. That allowed politicians, who would have had very little knowledge of physics, to do their job effectively according to the “new landscape” that such developments presented.

The mistake he made was that he only alerted authorities. Which lead to atom bombs being developed in secret without any feedback from the public as to whether this was a direction that they supported.

What we need in 2018 and beyond is for current and future developers of new technologies to alert both authorities and the public so that politicians can legislate for that technology’s use with public input as to what limits are deemed appropriate by the public at large.

The deep fakes example highlighted elsewhere in the thread is a perfect example of this. How can politicians legislate for this technology when they don’t even know it exists? How can the public indicate to those same politicians that they feel that deep fake technology used to create revenge porn is something that the public wishes to be made illegal?

Laws, and society as a whole, are a feedback mechanism that rely on information. Those with that information have a moral obligation to alert the public to consequences that will affect them.

[dekhn]

Interesting point re: Einstein. However, I think what he did was literally to ensure that the US won the war, not disclose a societal consequence. Since that was an existential threat, it falls under different rules. Object detection is not an immediatel existential threat.

Also, I kind think the secrecy of the manhattan project was essential, and after carefully reading multiple histories, I am convinced that Leslie Groves was a genius who did more than a good job managing the project. If you look at how he declassified the project, they did everything right. THey had a historian with access to all the data. And scientists who understood the context. And they worked together to release as much information as they possibly could, and even helped make the case for transitioning control of weapons to civilians.

[onceKnowable]

Of course they were all geniuses. And the project as a whole definitely was done in as ethical a way as possible. They weren’t bad people and I’m not implying that they were.

But the point isn’t anything to do with any of that. The point is that without the public’s input, these weapons were developed and used in the very first place, without an informed debate to decide if this is how we want to conduct ourselves.

This kind of debate was as common back then as it is today and back then it lead to chemical weapons being banned after WW1 by the UN with the support of pretty much every country.

And since then, for the few years that the US was the only one with nukes, if not for the extreme restraint practiced by the higher ups, they might have happily used nukes more times (at the time, there was huge pressure to just nuke north Korea to end the Korean War).

Once it became clear that the “enemies” will catch up in the development race meaning that any future war would be nuclear, the public debate clarified very quickly upon the current philosophy: MAD is the only outcome in a nuclear war.

If this debate had been allowed to be fully conducted in an open and informed manner before nukes were developed, we might live in a world where nukes were outlawed at UN level in the same way that chemical warfare was, long before they were even developed in the first place.

Extrapolate that thought to aggressive data-gathering & storage by social media sites, genomic information ancestry services, tracking technologies and techniques developed in the name of marketing, facial recognition technologies by security firms, Three-Letter agencies recording and monitoring every web user’s actions, profiling techniques to identify depressed users etc etc etc. Right now laws are not robustly protecting the public from misuse of these technologies. In fact, a lot of the misuse of the above technologies is directly due to the fact that the politicians know that they’ve got a powerful technology at hand and decide to develop it. And when informed public debate happens, when the negative outcomes of misuse of these techniques becomes so obvious to the public at large that they demand political action en masse, laws with more robust protections for the public’s data will be forthcoming in future updates. But those are the technologies that we know about. (At least, we nerds!). What is currently in development that has similar potential to be weaponized or misused that none of us know about yet?

[dekhn]

I gotta say, ultimately, I support nation-states who develop weapons to address existential threats in secret.

I do so because I think any nation which doesn't do so will be replaced by one that does, and surviving is better.

By the way, I've already kind of gone through these sorts of thought experiments, and voted with my genome: I open sourced the data in my genome voluntarily (https://my.pgp-hms.org/profile/hu80855C) because I don't really ascribe to the "grim meathook future" scenarios you're describing.

In general I think informed public debate is great, but in the specific case of the Manhattan Project, I really don't think having an informed public debate during the war was even a remote possibility.

[onceKnowable]

Einstein alerted the authorities about the dangers on Aug 2nd, a month before the war had even started and long before the US had joined the fight.

But the authorities kept it secret, choosing to develop these weapons.

But, had those weapons not been developed in secret and an informed public debate occurred before development, that debate would have inevitably lead to the MAD doctrine. That is the only inevitable outcome of nuclear war.

Without the guidance that MAD provides, politicians in 1939 did what they thought was the rational choice, and chose to develop these weapons so that they weren’t left unprepared if the enemy developed their nukes. But with the knowledge that MAD is the only outcome in a post nuke world, totally different scenarios become possible. They may have chosen to take action at UN-level, anti-nuclear proliferation treaties even before they got developed? Stifling the public debate just delayed the notion of anti-nuclear proliferation treaties but it did not stop them. The only result of an informed public debate on nukes is anti-nuclear proliferation treaties.

If all that had been done in 1939 as opposed to the 50’s & 60’s, the war could have been prevented before it even began!

Regardless of the crazy whataboutery, the lesson is that the public debate resulted in the ethics being decided as: “nuclear technology leads to MAD when used to create weapons, therefore laws are needed to protect the public from misuse of nuclear technology”. This debate is needed for every new technology.

Technology moves fast, faster than ever these days. And currently, we’re used to a situation where we typically operate these debates retroactively, legislating against misuse after the fact, when new technologies are created and then misused. Whereas, as this article is highlighting, the ethical way of doing things would be: to have informed public debates that allow laws to be created to robustly protect the public before a new technology is misused.

Note that nobody is accusing “new technologies” as bad. Or evil. Or anything like that. The call is simply for creators to slow this public debate to take place openly, to decide if the new technology can be misused and its repercussions in the event of that misuse and if new laws are needed to protect the public, before the possibility of their new technology being misused is even a factor.