Hacker News new | ask | show | jobs
by Antwnis 3118 days ago
> How exactly do you "forget" the data on the logs?

If we think around the options, you can have either:

i) eventual deletion (log retention policy) ii) compacted topics (and push null values) iii) expensive re-processing of the entire log iv) expensive segment re-write operation

with each option bringing in a new set of challenges

1 comments

Encrypt with a user specific key when the data enters the log. You can effectively delete all the user specific data by throwing the key away. No tracking down files or reprocessing necessary.
Today’s encryption is strong. Who knows about tomorrow?
Is that acceptable within GDPR requirements?
The GDPR contains exceptions when deleting individual user data is infeasible or if the data in question is a backup, in which case you must only keep a log somewhere so you don't forget to delete again.