Hacker News new | ask | show | jobs
by Alex3917 3434 days ago
Search is still possible. You can stem words and store their weights without storing the actual unencrypted text. This isn't perfect security, but it's good enough that it would be difficult for the government to successfully use the search metadata in a case against you without a lot of other evidence.

This is what we do on FWD:Everyone for email threads shared within private repositories.
2 comments
hiddencost 3434 days ago
DO NOT DO THIS. Statistics is more powerful than you'd expect. Also, the wrong rare word could absolutely be grounds for a very intrusive warrant.
link
Alex3917 3434 days ago
Security is rarely good or bad in an absolute sense, but rather is judged by considering the assets under protection and the threat models you're protecting against.

Just because certain people need the sorts of protections provided by Lavabit doesn't mean that products like Gmail are bad or insecure. They're each secure for the use cases they're targeting.

link
simcop2387 3434 days ago
I've never tried it but I'd bet you could use instead use a key derived from the master somehow to salt a hash for all the stems too. I think that'd let you still do the search and possibly have a good index for it still but it'd add some hurdles to the whole thing so that you couldn't make a lot of good guesses about the actual contents of the emails. It would still leak information a out related emails (say there's a thread discussing a unique term in it), but it'd still be better than otherwise.
link
Alex3917 3434 days ago
In theory you could HMAC all the stems, but at that point you really need to ask yourself what threat model you're trying to protect against. E.g. keeping only the stems is more than enough to prevent competitors from learning anything useful about your business if your database gets compromised, but using an HMAC probably isn't going to significantly slow down a state-level actor who has a warrant or zero day for your web servers. So on the balance I think the technical debt that would be introduced would likely make the overall system less secure, at least for most use cases.
link