Hacker News new | ask | show | jobs
by jillesvangurp 2757 days ago
Interesting, I actually implemented a simple immutable ledger on top of mysql this year to track balance mutations of our InToken coin. The coin exists on Stellar as well but we track federated stellar accounts in our database for most of our users. This means that we represent them with a single stellar account. We allow users that clear our aml procedure to send/receive tokens to stellar.

Having our own ledger means we can cheaply do internal transfers, micro rewards, and other incentives. Doing the bookkeeping correctly and in a tamper proof way is of course important for us, which is why we built our own ledger database to ensure we don't end up with corrupted data (either through bugs or malicious activity).

I use content hashes as the id that include the id of the previous ledger entry and the key data stored in each row. The core design is pretty easy (it's basically a linked list) but the devil is in the details with this stuff since you indeed need to worry about auditing and making sure you don't end up losing transactions.

Additional headaches include dealing with concurrent transactions and the fact that mysql does not do serializable transactions (at least not in sane way). Each row should only have 1 successor meaning that every new row involves looking up the previous row, and using it's id as a parent id. So we have a select and an insert happening in a transaction. We have a simple db constraint enforcing the parent is referred only once. We retry transactions when this constraint gets violated. This does actually happen when two concurrent transactions decide to use the same parent id. If transactions were serializable, this would not happen and the second transaction would end up using the id of the first.

Another of the gotchas is that data migrations are kind of hard/impossible in an immutable data store. The only way to do it would be to effectively recreate a new database with new content hashes. So there are some things that I'd like to change that I can't actually change because it would break the content hashes. But by and large, this design is working quite well for us so far.

So, in short, it's not rocket science but hard enough that having a well supported product that does this is worth having. We're actually considering open sourcing it at some point since it seems there are quite many projects out there that use a ledger primarily to have some tamper resistant immutable and auditable log of transactions.

1 comments

Given your problem with serializability, why not use a database which supports it better than MySQL?
By the time we had to debug this as a thing, we already had our implementation ready. Also cost for a small mysql RDS db in amazon and the convenience of not having to administer that were a factor.

But you are right, I have considered switching to cockroachdb since they advertise their support for serializable transactions.

Why not PostgreSQL? It's also on RDS.