[stuarthalloway]

Datomic keeps your database in the same memory space as your application code, backed by SSD, backed by EFS, backed by S3/DynamoDB. Each layer of this cache provides a different value, and the whole cache system is automatic and eternally consistent because the database is an immutable value.

You want the illusion that your database is entirely in memory, just as want the illusion that all your memory is in your L1 cache.

[alter_eco123]

> Each layer of this cache provides a different value

What does that mean?

And what kind of actual memory usage should people expect with a "big" database?

[stuarthalloway]

Hm, I should have said "value proposition", e.g. SSDs are low latency but can fail, S3 is high latency but "never" fails, etc.

Datomic is designed for data sets larger than memory, and also to do a good job caching working sets that do fit in memory.

[alter_eco123]

> Datomic is designed for data sets larger than memory

Of course. But you didn't answer my question.

Let's say someone has an application with 100GB of data in Datomic.

What kind of memory usage should he expect for the "peer" process?

[stuarthalloway]

That depends on what queries the peer runs. In both On-Prem and Cloud, Datomic maintains an in-memory LRU object cache. Also, you can send queries with different working sets to different processes without having to shard data on ingest. For example, some peers might handle queries related to user transactions while other peers handle analytic or batch work.

See also https://docs.datomic.com/cloud/operation/caching.html.

[0xFACEFEED]

> You want the illusion that your database is entirely in memory

I don't want illusions. When it comes to databases I want simple and predictable.