[harryh]

Hi,

I run engineering for foursquare. About a year and a half ago my colleagues and I and made the decision to migrate to MongoDB for our primary data store. Currently we have dozens of MongoDB instances across several different data clusters storing over a TB of data and handling 10s of thousands of requests per second (mostly reads but the write load is reasonably high as well).

Have we run into problems with MongoDB along the way? Yes, of course we have. It is a new technology and problems happen.

Have they been problematic enough to seriously threaten our data? No they have not.

Has Eliot and the rest of his staff @ 10Gen been extremely responsive and helpful whenever we run into problems? Yes, absolutely. Their level of support is amazing.

MongoDB is a complicated beast (as are most datastores). It makes tradeoffs that you need to understand when thinking about using it. It's not necessarily for everyone. But it most certainly can be used by serious companies building serious products. Foursquare is proof of that.

I'm happy to answer any questions about our experience that the HN community might have.

-harryh

[samstokes]

Would you be able to sum up the things you consider Mongo to be extremely good at? Particularly in comparison to things like Riak (which I believe supports a similar data model), or indeed compared to an RDBMS.

All databases perform poorly if you try to use them for use cases they don't fit, but I find with NoSQL databases it can be hard to find concise, objective statements of which use cases each is ideal for.

[fedd]

have users of foursquare run into problems? were they serious? did someone lose money? let's ask. it would answer whether to use an eventually consistent db.

[harryh]

> have users of foursquare run into problems?

Of course we've run into problems from time to time. No one goes from nothing to foursquare's level of success without running into some bumps along the way.

> were they serious? did someone lose money?

No.

> it would answer whether to use an eventually consistent db

MongoDB actually isn't really an eventually consistent datastore. It doesn't (for example) allow writes to multiple nodes across a network partition and then have a mechanism for resolving conflicts.

[willvarfar]

http://blog.foursquare.com/2010/10/05/so-that-was-a-bummer/

You had 11 hours downtime and didn't lose money?

What about opportunity cost? Reputation?

Now you have to share your secret :)

(I guess, if you weren't profitable, you had nothing to lose?)

[harryh]

The 11 hours of downtime was a pretty big deal, but it had very little to do with MongoDB. It was basically a huge failure in proper monitoring.

[9oliYQjP]

Kudos for not blaming the tool when that would have been the easiest route. It's worth mentioning that 10gen has MongoDB Monitoring Service out now. It makes monitoring MongoDB instances a lot more accessible and convenient.

[willvarfar]

That's not how everyone remembered it at the time, nor the picture the blog painted. And the mongo monitoring thing is much newer, right? Its like saying that a fire wasn't a big deal because next time there'll be a fire station.

Stockholm syndrome?

Or perhaps investor pressure to close ranks ;)

[jm3]

If you don't have paying customers (Foursquare), you're not going to lose much in hard dollars when your service falls over. Reputation points? Sure. Dollars, not so much.

[maxmcd]

Drawing a line between the difference in losing money for a paying service and a free service, while technically correct, is not the best business practice. Any online business looses money by being down, whether they can easily quantify it or not.

[gsharm]

Where do you finally persist the data in that case?