[buremba]

Is the comparison using on-disk persistent storage of Memgraph? The facts that they're both ACID-compliant and you can backup the data to disk in Memgraph are not relevant IMO considering the fact that I/O is the main driver for the latency for most of the workloads and reading data from disk is much slower compared to memory. Not sure how Neo4j works but an alternative might be running Neo4j in-memory mode.

[mapleeman]

No, currently, Memgraph is loaded 100% in RAM. Agree with the point about latency regarding I/O. The story with Neo4j is a bit more complex one. They are loading graph in memory, and using disk storage, doing both. But take a look at memory usage from them, it is the magnitude higher then Memgraph.

[menaerus]

Did you manage to figure out where does the difference in memory consumption come from? Do you attribute it only to JVM or have you designed low-memory footprint data structures in memgraph?

[mapleeman]

The difference comes from several things, JVM being the first and obvious one. It takes as much memory as whole Memgraph + small dataset in this case. Second is the overallocation that JVM/Neo4j is doing, taking a bit more memory to have future space to grow. The third one comes to the implementations of storage and Neo4j cache implementation, not sure what Neo4j does on that side and how impactful it is. We use a skip list.

[buremba]

Java programs tend to use the memory they're given to provide the best performance in the server environments so it's not conservative in terms of memory allocation. While it does use more memory in a given time, it doesn't necessarily mean that it relies on all the allocated memory. That's one of the reasons why benchmarking is so hard..

[mapleeman]

Exactly, you are right. We were able to decrease Memory usage to 1GB in Neo4j case, but then experienced some crashes. We then just removed the limit, and let it take us much as it needs.