[nemothekid]

One thing that Cassandra doesn't have a good story of, and what intro guides continue to gloss over is the ops situation. I've recently moved some our largest Cassandra tables to BigTable for this reason. The compaction / repair / garbage collection death cycle is probably the most difficult thing to manage, and in the past 3 years of using Cassandra, managing it has gotten worse. Tools have been deprecated (like OpsCenter) and new features can exacerbate the problem. There is still no reliable way to detect when repairs have finished, and if you have a large enough table, repairs can take a week to finish. Combine that with the fact that if a table is that large, then it probably has a high write volume - meaning it has a lot of compactions as well. So you have repairs and compactions going on which thrash the heap, and now you also have a GC tuning problem.

It took a lot of experimentation to get right, but once I did, scaling started to mean smaller drives and more nodes, which meant a more expensive cluster, for which I was largely paying for my CPUs to repair and garbage collect data.

Other than ops however, Cassandra is a great tool and does everything it says it does on the box.

[Boxxed]

Completely mirrors my experience with Cassandra. I think they'd have a real contender on their hands if operating a cassandra cluster didn't basically take a full time engineer. Its backup story is absolutely abysmal, and tooling is atrocious -- during a support incident a DataStax guy suggested I dump a table with sstable2json (or something like that) which generated a 100GB json file. When I pointed out that basically nothing could consume it because it was one 100GB hash object, he said "Yeah, I guess no one ever uses this stuff."

[jjirsa]

As a long time Cassandra user: people use sstable2json all the time, but most people don't have 100gb sstables (or 20gb sstables that make 100gb of json)

Certainly something we can do better - how would you break it up? Adding a key to dump an individual partition to json?

[Boxxed]

It wasn't that large a database if I remember -- maybe 1TB? The sstable sizes seemed reasonable at the time, I think it was just explosion due to json.

Anywhoo, one huge file is fine, what's not fine is having one huge json object -- streaming parsers might be ubiquitous in the XML world, but definitely not in json land. Something simple like small json documents separated by newlines would work.

[jjirsa]

https://issues.apache.org/jira/browse/CASSANDRA-13848 Created just for you

[Boxxed]

What a guy slash gal; cool!