[sowbug]

I wonder whether your individually trained chat bot will be allowed to assert the Fifth Amendment right against self-incrimination to stop it from talking when the police interview it. And if it's allowed, do you or it decide whether whether to assert it? What if the two of you disagree?

Similar questions for civil trials, divorce proceedings, child custody....

[rolenthedeep]

Think about what kinds of data an AI assistant would store about you vs what your phone stores.

Your phone is already more or less an extension of your brain, and whether or not you can be forced to unlock and surrender it for inspection is already a contentious topic.

IANAL, but phone privacy would probably set the precedent for AI assistant privacy.

Always keep your phone encrypted and be aware what your local laws are. Some places will force you to provide biometric authentication, but not provide a PIN or password. Check if your phone has a duress lockdown mode: some phones lock and/or wipe if you press the power button five times or something like that.

[jiggunjer]

All this advice isn't worth much when people will install any app that'll give a dollar discount on their favorite shop/restaurant.

[qwertox]

This is a topic which really deserves a lot more attention, as in: from magazines to newspapers to talk shows. Seems like an appropriate time to get it on the agenda before governments opt to decide on their own.

[roywiggins]

How could it have a right not to self-incriminate when it can't be tried for a crime? An AI can't be indicted or convicted.

Humans can be required to testify too if they're immunized.

[sowbug]

Apologies for the ambiguity. I imagine that my future AI-based "second brain" will be derived from my own brain, including its personality, memories, and preferences. Anyone in an adversarial position to me would be very interested in talking to it. I pose whether the term "self-incrimination" should include one's second brain as part of one's self. The question was not whether police would put a PC in jail.

[sokoloff]

With limitations. I cannot be compelled to testify against my wife (and probably not against my kids, though I’m unsure of that [edit: that seems to vary by state currently]), even if I personally am granted immunity.

[bbor]

Wait is this a serious concern or a joke? I think asking a chat bot whether it’s admissible is the same as asking someone’s handwritten diary if it’s admissible.

[sowbug]

Fortunately, US courts take these sorts of questions quite seriously. See, e.g., State v. Smith (Ohio 2009): "Even the more basic models of modern cell phones are capable of storing a wealth of digitized information wholly unlike any physical object found within a closed container." https://en.wikipedia.org/wiki/Carpenter_v._United_States is also a good read, as is https://en.wikipedia.org/wiki/Riley_v._California: "Modern cell phones are not just another technological convenience. With all they contain and all they may reveal, they hold for many Americans “the privacies of life". The fact that technology now allows an individual to carry such information in his hand does not make the information any less worthy of the protection for which the Founders fought."

If a cell phone is recognized as having a higher expectation of privacy than a mere passive document, then it stands to reason courts will also recognize a personalized machine that is even more earnest to help as my infernal "by the way..." Alexa home device.

[PeterisP]

Those are all 4th amendment cases for warrantless searches violating privacy.

The fifth amendment has very different criteria - with a proper warrant, your most private things are admissible evidence. You can't be compelled to testify but all your most private notes in a safe can be "interrogated"; in many situations you can't be compelled to testify against your spouse but any writings or recordings of what you said about him/her are valid evidence. There is no debate that all the contents of your computer or phone can be used, they definitely can, the cases you quote are disputed only because they are "worthy of the protection for which the Founders fought" which is the requirement for a warrant.

I'd say that fifth amendment is not a privacy law (which is the 4th, stating that you have all the privacy without a warrant and no privacy with one), the 5th is essentially an "anti-torture" law to prevent coerced confessions, and there is no reason why it would apply to some physical evidence like the data for a trained model on your device.

[sowbug]

Thanks for the analysis. Something still sticks in my craw.

You can read a letter. You can play an audio recording. You can look at a photograph. You can print a word-processing document. Those feel like static records. Each tells a single story, no matter how many times it's read.

An ML model, on the other hand, is just a long sequence of floating-point numbers. There is no meaningful model "viewer" that spits out a text file or JPEG. The only thing a non-engineer human can do with an ML model is interact with it when it's running. Thus, any output is the product of both the human and the model. It doesn't feel like a record. It feels more like a performance -- and a performance by the interrogator, at that.

If an ML-model "record" is a special kind of record that needs manipulation to produce human-readable output, then it feels like we're back in the 5th amendment department. No, chatting with a chatbot is not torture. But it produces a kind of evidence that is an unwelcome collaboration between the interrogator and the record (unwelcome by the owner of the record). It's one thing to let a jury look at a screen full of numbers. It's another thing to say that an expert witness used those numbers in a chat session that printed "Yes, it was my owner, in the living room with the candlestick."

If you lock up a person and ask them the same question over and over again, eventually you'll get the answer you want. The same will probably turn out to be true for that person's chatbot. Should society allow that second case?

[michaericalribo]

Imagine a model that decides on its own to assert the Fifth on your behalf. But now imagine that AI decides to lock you out of your own system...

[DavidPiper]

We already have those. They're called Google, Microsoft and Apple.