[throway33]

Suppose P has a 1% chance that it destroys the Earth as soon as it's created, whether it's in the hands of people who follow laws or not.

We pass a law saying: "You cannot have P."

The market incentive for producing P is now vastly reduced, because the black market is a fraction of the size of the real market, so P never gets built in the first place because it takes a lot of investment and expense to do so.

[thepasswordis]

>whether it's in the hands of people who follow laws or not.

This is basically doing all of the work in your comment.

I don't agree that AI in malevolent hands carries a risk that is equal to AI in neutral of good hands.

[throway33]

I agree that with sufficiently good hands, who are sufficiently confident that they are not in a competitive race, they might be able to wait until they are supremely confident that P will not destroy the world before they switch it on for the first time. But with even just a little fear of someone else getting it first, that prudence might well go out the window, even if there are no malevolent hands involved.

[dragonwriter]

Your argument seems to rely on the assumption that the black market doesn’t grow when the legal market is eliminated, which every attempt to regulate things in history contradicts.

[throway33]

It only assumes that the black market doesn't grow to the same size as a legal open market. I don't think history disagrees with that. You can look at the cannabis business as just one example. How many VCs were plowing big money into cannabis operations before legalization was on the menu? How does that compare with after legalization?