[xoa]

This is the argument I've made to people over the last few years, and I think is far better and more substantial then the technical based ones that so typically come up (and have in the comments here already, the whole "well a key will of course leak thing"). It's risky and unnecessary to go to technical arguments for something that's a moral matter, because technology changes. If you base your entire opposition around the key, what if at some point someone does have an entirely formally verified stack and strong measures and can reasonably argue that keys aren't going to leak? Apple's master key for example, it becoming available would be an enormous thing for both criminals and ordinary owners of iOS devices. Yet while there have been plenty of flaws that have been exploited to bypass a need for it, the private root key remains unleaked.

The real question that I would want to see some Congress person put to agencies is

>"Do you believe there should be any inherent limits at all? If we developed the technology someday to read people's minds, should it be permissible to go through their brains with a warrant? It would certainly let you find the guilty of some 'crimes', where for 'crimes' we should keep in mind that gay sex and interracial relations were felonies in the near past."

I mean that's the real thing, if security agencies could root through people's brains I see no need to beat around the bush that likely at least a few truly horrific crimes would be stopped or solved. There would be children saved, terrorists stopped, murderers caught. But I think not just the abuse of it, but even the use of it to eliminate any gray area for a human society would be so horrific that it's just plain not worth it. That yes, some children will be abused/kill, some murderers will escape, some terrorists succeed, and that really is the price we need to pay. That we should try to reduce it as much as possible but only in opposition to strong privacy and an inviolable personal sphere. And that should include artificial augmentations to our minds, which typical mobile devices are already arguably at the point of.

The incentive structures right now for law enforcement agencies and intel agencies remains geared always towards more more more, and paying attention to singular big harms rather then small harms across enormous swaths of the population. It hasn't evolved much from decades and centuries in the past arguably. I think that's the ground to fight on though, will they argue that total erasure of the private sphere is worth it? Will the public agree? I think the answer is no, and with that established it's a lot easier to argue back against "think of the children/terrorists/drugs" typical attack.

[jecxjo]

The next question needs to be (assuming they feel it should be allowed) is do they feel there should be allowances for someone to be above this law? In our current climate we are seeing criminal looking behavior in the US by people within the Executive branch of the government. They have been citing Presidential Privilege but if mind reading became legal there would be multiple issues here.

This law would rid us of the last safe harbor, your own mind. At that point allowing some individuals the ability to skirt this law would create a class definition of ultimate scale. Those who are allowed to keep their own thoughts to themselves and those who aren't allowed to.

[ickwabe]

Well put. To wander a bit off topic, but I feel this is a specific example of a general problem in free societies. There is no perfect safety without perfect surveillance. In my opinion, it is necessary to be at peace with the fact that there will be a certain level of bad outcomes in exchange for the protection of general freedoms. There will be crime, murder, kidnappings, embezzlement, death by neglect, etc. It is unavoidable absent a perfect surveillance state. I, for one, am willing to accept risks. Others will disagree. But I think that in a perfect surveillance society you are also pefectly stagnant.

[bradknowles]

There will be no perfect safety. Period.

There will be no perfect surveillance. Period.

Humans are humans, and inherently imperfect.

Until all humans are eliminated, there will be no perfect ... anything.

[desc]

> It's risky and unnecessary to go to technical arguments for something that's a moral matter, because technology changes.

True.

> If you base your entire opposition around the key, what if at some point someone does have an entirely formally verified stack and strong measures and can reasonably argue that keys aren't going to leak?

Irrelevant, since formally proving the stack says absolutely nothing about compromising the system at a human level, and 'reasonably argue' does not mean 'proving that the key cannot leak'.

The key will always leak. It's just a matter of when. There will never be any technical solution to that, and it's not a technical problem. It is a hole which fundamentally cannot be plugged, and the hole is people. Technology has nothing to do with this.

>>"Do you believe there should be any inherent limits at all? If we developed the technology someday to read people's minds, should it be permissible to go through their brains with a warrant? It would certainly let you find the guilty of some 'crimes', where for 'crimes' we should keep in mind that gay sex and interracial relations were felonies in the near past."

But that's always a question of legality, not technical capability. Legislation is all about where we draw lines.

However, it is entirely true that legislation (or rather, the 'undefined behaviour' in it) tends to be abused a great deal before it can be shut down.

The argument that the key will leak is not technical, and strictly speaking, nor is the argument that 'technology may one day do this and then where would we be?'. But one is a lot less speculative than the other, and admits no mitigations.

[xoa]

>The key will always leak. It's just a matter of when. There will never be any technical solution to that, and it's not a technical problem. It is a hole which fundamentally cannot be plugged, and the hole is people. Technology has nothing to do with this.

Ok, so again then what would be your response if some law enforcement official said "well what about Apple then?" iPhone is now 13+ years old, why hasn't their master key leaked? Microsoft or Google signing keys, same boat. Or what about SSL in general for that matter, the entire HTTPS paradigm depends upon master private keys that are not leaked, or at least not often. It would certainly be potentially worth a lot to certain adversaries if they could simply spoof major banks, not through some CA hack but literally just getting their keys. Yet broadly the effort to keep that information protected seems to be fairly successful, for better and for worse.

I mean, that's kind of my issue, it's not hard to raise plausible counter examples, and once we get into the weeds about "how likely" and "when" vs "how many will be saved in that time huh" I think we may have already lost. It distracts from the real debate, which is about how valuable a zone of private space is and the harm that comes from infringing upon it. That's the real cost. Just because you can doesn't mean you should.

>But that's always a question of legality, not technical capability.

I don't think that's quite justified by the history of law and privacy. There are plenty of practices law enforcement/intel can and do engage in now that were simply inconceivable in the relatively near past, when much of the law that still governs us was written. Not everything automatically adapts, it's worth watching out for things that are simply taken for granted because they're considered inviolable, and in turn lack explicit legal protection. There should be legal protection, what I'm saying is that winning it requires a frank discussion about harm tradeoffs and trying to get people to more generally grasp new kinds of costs, like emergent effects.

>The argument that the key will leak is not technical

I think it really is. I mean, even in your argument where you say "the hole is people", you're making a technological implicit assumption that people remain involved. Do your assumptions make the same sense if we imagine human equivalent or better AI? I'd rather just talk about why we shouldn't as a matter of ideals, even if we could, and even if it means some things we don't like go unstopped.

[Darth_Hobo]

I think the whole concept that law should be enforced at all times is flawed. People should have more ways to resolve conflicts between themselves on their own, without cops getting involved - even if the law was broken.