Hacker News new | ask | show | jobs
by coderdd 1541 days ago
I was pondering about using simple decentralized databases that are kept synchronized asynchronously.

I arrived to a design where SQLite databases would be synchronized by Kafka. Kafka is really robust, and has a friendly semantics when configured to be in-order delivery.

The catch is, you don't issue writes to SQLite anymore. You write Kafka messages, and have to prepare to resolve concurrency problems at read+write-to-db time. For example, user registration is not an atomic action anymore - you write a registration attempt message, and then it might race with a concurrent registration attempt from an other node (say due to retries and random load balancing).

You resolve the race in message-reception time (doable due to stable order on multiple reader nodes). I expect this would work nice, but needs you to rethink all actions that used to be synchronous db writes.

1 comments

There are more robust solutions out there for this architectural pattern. Look into dolt(dolthub) and noms.