[tdrhq]

I think it's important to understand that every startup goes through three phases: Explore, Expand, Extract. What's simple in one phase isn't simple in the other.

A transactional database is simple in Expand and Extract, but adds additional overhead during the Explore phase, because you're focusing on infrastructure issues rather than product. Data reliability isn't critical in the Explore phase either, because you just don't have customers, so you just don't have data.

Having everything in memory with bknr.datastore (without replication) is simple in the Explore phase, but once you get to Expand phase it adds operational overhead to make sure that data is consistent.

But by the time I've reached the Expand phase, I've already proven my product and I've already written a bunch of code. Rewriting it with a transactional database doesn't make sense, and it's easier to just add replication on top of it with Raft.

[gtirloni]

I'd assume in the beginning you do not want to spend time writing a bunch of highly difficult code until you've proven your idea/product. Then when you're big enough and have the money, start replacing things where it makes sense. It seems to be the strategy used by many companies.

Unless, of course, your startup is in the business of selling DBMSes.

[tdrhq]

Absolutely. By the way, if it wasn't clear from my blog post, in the Explore phase, I used an existing library to do this. It was only in the Expand phase that I put this existing library behind a Raft replication.

[oefrha]

I’ve done tons on traditional application server + database on the same server projects. There’s zero infrastructure issue there. You keep implying that a not-in-process RDBMS has to be its own server and that’s super strange. Not to mention having a separate db server also doesn’t add much overhead at all in the early stage, even if you’re doing it for the very first time (been there, done that).

[Groxx]

Having Explored with a transactional database: I really can't agree. Just change your database, migrations are easy and should be something you're comfortable doing at any time, or you'll get stuck working around it for 100x more effort in the future.

[kasey_junk]

That was the biggest disconnect I had as well. SQL db have the _best_ data migration tooling and practices of any data system. It’s not addressed in the article how migrations are handled with this system but I’m assuming it’s a hand rolled set of code for each one.

I think sql db make the most sense during the explore phase and you switch off of them once you know you need an improvement somewhere (like latency or horizontal scalability).

[tdrhq]

Good question!

And this comes to the difference between Explore phase and Expand phase.

In the Explore phase, data migration was just running code on the production server via a REPL. Some migrations such as adding/removing fields are just a matter of hot-reloading code in CL, so there weren't a lot of migrations that we had to manually run.

In the Expand phase, once you add replication, this does become hard and we did roll out our own migration framework. But by this point we already have a lot of code built out, so we weren't going to replace it with a transactional database.

Essentially, we optimized for the Explore phase, and "dealt with the consequences" in the Expand phase (but the consequences aren't as bad as you might think).