[bkeroack]

A mess is a mess, true, but some are easier to clean up than others.

GP is correct. Methods for scaling/optimizing the application layer are clear and well-known. Scaling the data layer is a huge challenge. This is why the market is filled with snake oil databases promising linear scalability and perfect consistency/reliability, etc.

[threeseed]

Scaling the data layer is a huge challenge. No doubt. But calling databases that are designed for solving these problems "snake oil" undermines the huge amount of work that serious engineers have invested in this. No one has ever promised linear scalability and perfect consistency/reliability. No one.

Cassandra, HBase, CouchDB etc even MongoDB have built in scalability as a first order priority from day one and have been largely successful at it e.g. iCloud, EA Online, PSN. Databases like this are a nightmare to work with for smaller datasets but work incredibly well with larger ones.

It's always a shame to see HN act like you scale vertically and magically every problem is solved.

[AlisdairO]

> It's always a shame to see HN act like you scale vertically and magically every problem is solved.

When this is seen (and IME it's a pretty minority opinion) I think it's there as a reaction to the massive overuse and hype regarding a lot of newer-gen DBs. There's absolutely no doubt that there are good uses for them, but those cases are pretty niche compared to the level of their uptake.

[volaski]

You should read "innovator's dilemma"

[AlisdairO]

Is that comment intended to imply that companies will go under if they fail to deploy new technology that doesn't target their business needs?

[fweespeech]

MongoDB heavily implied the linear scalability, consistency, reliability bit in its early material [particularly in their marketing].

Its only in the past couple years they really started mentioning the fact it was "tunable consistency" blatantly rather rather than burying it in a couple places in the manual.