Hacker News new | ask | show | jobs
by ncodes 3695 days ago
Thanks for your suggestion. The fact that data written to BigQuery can still be modified is not so great for my case. I want to be able to make data immutable forever. Nobody should be able to modify or delete it. Malicious alterations should not affect other nodes holding replicated data.
1 comments

I remember watching a talk about apache Samza [1] that tries to envision a database model that would be truly append only. It is based on Apache Kafka, so it would satisfy your "distributed" and "immutable forever" requirements.

Talk was really interesting, I haven't used it yet, so I am not sure how mature for use as the canonical data-store it is.

As my college professor would put it "I don't thing these have found their Ulman yet." and if you look at companies using it now [2] it seems mostly stream-processing/data-analytics.

Another thing is, I really don't know how Kafka handles checking of the data authenticity, because in my mind there is not that big of a difference between malicious alteration and malicious append.

Because if you then use something like CRDT [3], or DDD style agregates [4] on top of your immutable data , your end users would still see their view on data mutate.

The thing the immutability would mostly give you is log of all the changes and simple way to restore it. And most mutable databases give you that capability as well.

[1] http://www.confluent.io/blog/turning-the-database-inside-out... [2] https://cwiki.apache.org/confluence/display/SAMZA/Powered+By [3] https://en.wikipedia.org/wiki/Conflict-free_replicated_data_... [4] https://en.wikipedia.org/wiki/Domain-driven_design#Building_...