Hacker News new | ask | show | jobs
by manigandham 3249 days ago
MemSQL actually adds an in-memory rowstore to each columnstore for rapid ingest of new rows until they get compacted into a new segment. Columnstore data is pretty fast so it works well off disk compared to row stores which aren't as efficient.

SQL Server similarly has the hekaton in-memory tables + columnstore indexes and the latest version allows combining both for in-memory columnstores.
1 comments
AndyNemmity 3249 days ago
I've used MemSQL, and it's rapid ingest by the default isn't ACID compliant, so it sort of depends on how you compare it.

The results of the columnstore data was pretty fast, and it's even faster in memory. Depends on what you're doing, and what the requirements are.

Was really impressed by MemSQL, and loved the wire compatibility with mysql, so don't take this as just a knock on MemSQL in anyway.

link
manigandham 3248 days ago
> by the default isn't ACID compliant

What do you mean?

link
AndyNemmity 3248 days ago
I'm surprised you wouldn't understand this as it's an absolutely requirement given you are a user of MemSQL.

But then I went to their docs to link you to the details, and it feels like they intentionally avoid stating clearly the problem.

Essentially, they allow committed transactions to hit memory and not disk. They allow you to configure it so that's not the case, but it isn't the default, and looking over the current documentation they certainly aren't clear about it like an open source project would be.

transaction-buffer needs to be set to 0 for durability, but the way the docs are explaining it is trying to confuse not being durable, as a different kind of durability.

I'm not interested in getting into a long discussion about this though, but it's difficult to explain the literal issue when they do such a marketing job of trying to hide the specifics.

Now I'm far less surprised a user wouldn't know this. Apologies for my forward initial statement.

link
manigandham 3248 days ago
It's not a surprise, that's normal. Durability just means writes are safe, how it does so doesn't matter. All databases use a write-ahead log with small batch/async flushes and this is the same with MemSQL.

The enterprise edition has HA so with data on 2 nodes for safety. Otherwise you lose the whole point of in-memory performance (for writes) if you're going to write every single bit to disk immediately.

They explain it clearly on the durability page: http://docs.memsql.com/docs/using-durability-and-recovery

link