[iamwil]

New does travel quick. I was just reading about this thing this morning.

http://railstips.org/blog/archives/2011/01/27/data-modeling-...

So John in the above link was talking about how when you scale, you end up looking at bottlenecks, which are usually slow data accesses, and moving them into some sort of key-value store--that usually are based on a key-value access pattern. So by the time you're well into scaling your app, you noticed that much of your data access has turned into limiting yourself to doing key lookups to attain the performance you need.

Since (according to his experiences scaling) most of the performance bottlenecks seem to have this pattern, he asked, "what if you restricted your data access to just using key-value pairs from the very beginning?" That way you avoid some of the data access headaches later on. In answering that question, this is what they got.

[Smerity]

The same tactic was used by Google for Google App Engine - "if it doesn't scale then we're not including it as a feature". This is why so much of Google App Engine's documentation focuses on scaling [1]. Depending on who you ask this is genius or folly.

By forcing yourself into this tactic you end up having to consider all the scaling complexity in the prototyping stage before your app has even proven itself potentially successful. If your app becomes wildly popular then the combination of your early work and Google's behind the scene scaling means there are far less problems for you to worry about.

My major concern with this method is that people already focus far too much on premature optimizations before they even know where the real bottlenecks are in their application. If premature optimization leads to burn out or less features then it's a poison to our projects and should be avoided.

[1] http://code.google.com/appengine/articles/scaling/overview.h...

[tomafro]

I think that's a good tactic; certainly a valid problem to solve. I think the headline on HN oversells the ability to use different backends, whereas the real value (as explained on railstips.org) is much more about destructuring data into this scalable key-value access pattern.