[annacappa]

The more I think about it the worse it gets.

Because we don’t want everything to fall over when one machine goes down we need at least 3 machines (for raft). So if our traditional db would have 500 GB of data we now need 3 machines with 500 GB of ram running at all times. That is an epic waste of money. Millions per year to run ? And you could store it in a db for a couple of dollars.

[1oooqooq]

their use case is mostly-never-retrieved images!

they store the index of files only in memory. and have the entire build time to fetch build-1 images to get ready for the diff.

it's much easier than most use cases

[annacappa]

So all of this ram is being used and is only accessed sporadically if at all. This is not good. Sounds like you could implement the entire thing on a micro db instance (redis or a regular db) with no raft or any other custom implementation or messing.