[Nican]

FoundationDB has been growing as my favorite database lately. Even though it is only key-value store.

Out of curiosity: what are the scale limits of FoundationDB? What kind of issues would it start to have? For example, being able to store all of Discord messages on it?

I see blog posts of Discord moving to Scylla and ElasticSearch, but I wonder if there would be any difficulties here.

[hardwaresofton]

Note that FDB can support other paradigms on top of KV

https://foundationdb.github.io/fdb-record-layer/SQL_Referenc...

Also IIRC Apple uses FDB at tremendous scale:

https://read.engineerscodex.com/p/how-apple-built-icloud-to-...

[eatonphil]

There are a lot of strict limits so AFAIK everyone uses FoundationDB for fast, consistent, highly-available metadata while doing replication/storage of actual data elsewhere (such as in S3).

https://apple.github.io/foundationdb/known-limitations.html

That is to say it's more like part of your solution and not the entire stack on its own.

[Nican]

Yeah. I read over that page, and everything seems sensible.

The only part that is not well explained is that "FoundationDB has been tested with databases up to 100 TB".

[eatonphil]

In their data modelling pages they mention you should break up rows into separate keys per column. Or separate keys per field in a document. This is indeed how many databases model rows on a distributed kv store. So this might be how they achieved 100TB.

However you still have the issue of any single key-value needing to be in their limit. (But it's not like people typically store enormous blobs in Postgres or MySQL either I think?)

[Dave_Rosenthal]

The documentation is woefully out of date, sadly. Despite the code being in active development no one is touching the public docs. Though I don’t know for sure, that limitation was probably written something like 10 years ago.

[heipei]

ScyllaDB discontinued it's free and open source version, so I personally wouldn't build anything new on it.