Hacker News new | ask | show | jobs
by panyam 995 days ago
Actually not quite. Backups (assuming you are doing single node) still needs you to decide on your SLOs - RTO (how long it takes to restore) and RPO (how much data lose you can suffer) numbers. On the instant snazy end you have streaming backups and recovery and then on the other extreme you have backup once in N hours/days and restore taking how ever long it takes to restore (so you have customer outages you need to negotiate.

Now let us involve multi node, (both replication and partitioning of shards). As shards go and up and down ensuring data is in sync etc is a hard consistency problem and needs man years of operational excellence and bug fixing.

So when people think databases - they think of the cool stuff - the database engine that does relational algebra and handles SQL queries. That is (IMO) only 1% of a practical, performant, reliable database (offering).
2 comments
api 995 days ago
Maybe if you are gigantic, but there is a long tail of people with <1TB database needs that don’t really need shards and can be well served by a fail over cluster with a master and one or two replicas that can become masters.

These days you don’t really need shards until you hit many terabytes or even more depending on your read and especially write load. NVMe storage is really fast and lots of RAM for caching has become cheap.

link
panyam 995 days ago
So my point was around all things a managed for gives you (eg sharding and replication). Even by the time I had to setup streaming replication and have to worry about wal drifts it is easier to pay a managed provider no?
link
tetha 995 days ago
Also what about the customer that deleted an important thing 6 weeks ago and absolutely needs it recovered? BTW, it's just one tentant in that DB, the other shouldn't be recovered, naturally.
link
aprilllll 995 days ago
In that case, it’d probably be best to just handle deletions at the application layer (e.g., setting a “deleted_at” timestamp field with scheduled permanent deletions later).

And in terms of data compliance, it’s very important to make sure permanent deletions propagate through your backup systems within a reasonable amount of time - Google Cloud[1], for example, is ~180 days.

[1] https://services.google.com/fh/files/misc/gcp_data_deletion_...

link