[sowbug]

Fortunately, US courts take these sorts of questions quite seriously. See, e.g., State v. Smith (Ohio 2009): "Even the more basic models of modern cell phones are capable of storing a wealth of digitized information wholly unlike any physical object found within a closed container." https://en.wikipedia.org/wiki/Carpenter_v._United_States is also a good read, as is https://en.wikipedia.org/wiki/Riley_v._California: "Modern cell phones are not just another technological convenience. With all they contain and all they may reveal, they hold for many Americans “the privacies of life". The fact that technology now allows an individual to carry such information in his hand does not make the information any less worthy of the protection for which the Founders fought."

If a cell phone is recognized as having a higher expectation of privacy than a mere passive document, then it stands to reason courts will also recognize a personalized machine that is even more earnest to help as my infernal "by the way..." Alexa home device.

[PeterisP]

Those are all 4th amendment cases for warrantless searches violating privacy.

The fifth amendment has very different criteria - with a proper warrant, your most private things are admissible evidence. You can't be compelled to testify but all your most private notes in a safe can be "interrogated"; in many situations you can't be compelled to testify against your spouse but any writings or recordings of what you said about him/her are valid evidence. There is no debate that all the contents of your computer or phone can be used, they definitely can, the cases you quote are disputed only because they are "worthy of the protection for which the Founders fought" which is the requirement for a warrant.

I'd say that fifth amendment is not a privacy law (which is the 4th, stating that you have all the privacy without a warrant and no privacy with one), the 5th is essentially an "anti-torture" law to prevent coerced confessions, and there is no reason why it would apply to some physical evidence like the data for a trained model on your device.

[sowbug]

Thanks for the analysis. Something still sticks in my craw.

You can read a letter. You can play an audio recording. You can look at a photograph. You can print a word-processing document. Those feel like static records. Each tells a single story, no matter how many times it's read.

An ML model, on the other hand, is just a long sequence of floating-point numbers. There is no meaningful model "viewer" that spits out a text file or JPEG. The only thing a non-engineer human can do with an ML model is interact with it when it's running. Thus, any output is the product of both the human and the model. It doesn't feel like a record. It feels more like a performance -- and a performance by the interrogator, at that.

If an ML-model "record" is a special kind of record that needs manipulation to produce human-readable output, then it feels like we're back in the 5th amendment department. No, chatting with a chatbot is not torture. But it produces a kind of evidence that is an unwelcome collaboration between the interrogator and the record (unwelcome by the owner of the record). It's one thing to let a jury look at a screen full of numbers. It's another thing to say that an expert witness used those numbers in a chat session that printed "Yes, it was my owner, in the living room with the candlestick."

If you lock up a person and ask them the same question over and over again, eventually you'll get the answer you want. The same will probably turn out to be true for that person's chatbot. Should society allow that second case?