[idan]

It doesn't scale! This isn't a replacement for databases.

Our take on this is about "working sets" of data — if you have billions of rows, that's a lot bigger than a working set! At some point, you have to query, filter, and aggregate to get your data down to a chewable size for work.

You can do that in your code too, and sometimes that's absolutely the right approach! But often it's easier to push that work to "outside your code," and that is what Flat is great for.

[ellimilial]

Thank you for the response and clearing up the 'billion rows' / surly bonds confusion I had from reading project's Why Flat Data? section. I think I understand the target use case slightly better now.

One of the strong arguments for object-like storage (S3 etc) in the context of plain / flat data is scalability and availability for large scale processing frameworks. Databases are only occasionally relevant.