[davcro]

Amazing. Way back in 2007 I taught myself how to code by making small Facebook apps. I started out by editing live PHP files on a shared server from A Small Orange ($3.33 a month!). After a few months I a few apps with traffic around 30k DAU. A Small Orange would automatically shutdown the apps every couple hours. I'd email them and complain about their shoddy hosting service. They'd always respond instantly apologizing and putting the apps back online. After a couple weeks I realized that I had a scaling problem and began learning how to setup a dedicated server. Over the next two years I spent about 80% of my time wrestling with hardware, setting up load balancers, configuring cache and db servers, and other operational nightmares. I had little time or energy to work on improving my apps or building new products.

Then I discovered Heroku. I would have done anything to have this when I started out. The platform teaches (forces) you how to build a scalable architecture. You can try out new ideas for apps for essentially nothing (1 dyno is zero dollars). Since moving to Heroku I spend about 5% of my time working ops. The craziest thing is I've actually saved money since switching from dedicated hardware to Heroku. I was really bad at configuring servers and the stuff I built was inefficient and expensive. Heroku's cloud stacks are optimized better than my old hardware environment.

Heroku's architecture is great for wild traffic swings common with Facebook apps. Well except for their database services. They don't seem reliable or scalable. I prefer RDS.

In sum, Facebook and Heroku is a great starting place for learning to build web apps. I would have done anything to have this tech four yeas ago.

[bgentry]

Out of curiosity, what makes you think our data services aren't reliable or scalable?

[aculver]

I'll jump in here, since you asked. Bad timing. About 2 hours ago we started getting these completely abstract errors 15 minutes before we were due to present to some VC folks:

2011-09-15T18:34:33+00:00 heroku[router]: Error H12 (Request timeout) -> GET [redacted]-staging.heroku.com/ dyno=web.1 queue= wait= service=30000ms status=503 bytes=0

I have absolutely no indication what part of our stack is having trouble. After much freaking out, we spun up an entirely new app and demoed with the seed data. This weird sort of stuff has been happening to us at an alarming rate over the last 3 months or so. Not being able to deploy, having to put in a ticket, and waiting 24 hours for a someone in support to fix it is another example.

Please don't take this the wrong way: I love you guys as people. I pushed for Heroku adoption at our shop. I absolutely love the concept of Heroku. Until a few months I felt like you guys were doing it way better than anyone else. (Left EngineYard to migrate everything to Heroku.) But these past few months have been really scary. There is a growing consensus at the office that we'll end up migrating away from Heroku to a platform where we can actually understand what's going on and be responsible for what's going on under the hood. After I began using Heroku, I never thought I'd want to go back to that again.

EDIT: Provided a more clear example that is less obviously suspect of a timeout.

[imbriaco]

Did you try contacting support? Looking at that log line, I can tell you that it looks like your app took too long to respond to the request (service=30000ms) and our routing mesh timed it out (status=503, Error H12 (Request timeout).

If there's anything we can help with, definitely get in touch. Our support team is top notch.

[aculver]

You're right, the support team is very friendly and supportive. But my resolution times for recent issues have been 24-hour turnarounds, even when we were feeling some level of pain and labeled it "high". In this instance, which was definitely an "urgent", I just didn't have any confidence the issue would be resolved in time. Thankfully when we've had issues like this (mostly with development/staging environments) it's always easy enough to just spin up another app. But that doesn't make me very comfortable having my production stuff there.

[malyk]

Are you on the shared hosting DB plan? Are you using more than 1 dyno?

We are running 4 dynos, 2 workers, and the Ronin DB plan and we haven't had any problems other than a 10 minute downtime due to a bad deploy by Heroku.

[aculver]

Just eyeballing it we have about 40-50 apps using the shared DB. We don't have any dedicated DB stuff with Heroku, but we use varying dynos levels. (Most apps idle at 1, though.)

[malyk]

Probably that the cheap plans are on shared DBs and it's very expensive to use dedicated DBs and the dedicated boxes appear to be single instances of Postgres.

I'm not complaining, but what is the official Heroku approach to things like DB sharding, etc.

[cmelbye]

They actually have a very impressive dedicated database service: https://postgres.heroku.com/

It supports scaling vertically by throwing in more cache, faster CPU, etc. Once you outgrow that, it supports horizontal scaling by replicating to read-only slaves.

[malyk]

I'm aware as we are running on heroku here, but it gets expensive quick and your options are more limited than if you roll your own. Then again, that goes for the whole platform.

[cmelbye]

I don't see how that has anything to do with the statement that I was referring to: "[their database services] don't seem reliable or scalable."

[fdr]

Unfortunately there is no official approach. The problem of partitioning and inconsistency to get more scalability is still evolving quite a bit.

I think things could be better, but it's not crystal clear what the right trade-off is for many people

[tearsofarapper]

My experience has been that the shared dbs are more performant than Ronin.

I have had a couple instances get inexplicably slow and needed to relaunch.

We're also looking into off loading our database needs elsewhere.

[freeformz]

If you have any ticket links or more information about your problems please let us know, inexplicably slow should never happen, especially with a dedicated db.

[freeformz]

I'd love to know which Heroku db service(s) (shared or dedicated) you used and what the problems were, so we can fix them.

[zealoushacker]

I'd say that the biggest issue and the only issue I have with heroku is that when you guys have done some releases, you would prevent apps that have been placed into maintenance mode during the release from being taken out of maintenance mode.

This becomes a problem when the maintainers of the app in question are not aware of your release schedule. The solution, it seems, would be to either a) do a better job of communicating downtime; b) do rolling deployments and failover so that apps may continue to function; c) a bit of both.

While our app is not important enough for the above to bother us a great deal, we do have paying customers. As soon as one of them comes knocking telling us that we screwed up, and in reality it wasn't fully our fault, we will be a bit more adamant about the above suggestions :)

For the time being, great job!

[zealoushacker]

This is excellent! We are on a heroku stack at bizeebee.com now, and if we had to write a facebook app now, there's no question that we'd use this. I had written some apps right after fb platform was released and had to go through hoops to set up my own dev/production environments, manage configurations, deployments, source control etc.

Heroku had taken care of all of that for non-fb apps, and now with tight fb integration, I might just write a few quick fb apps again!

Thanks for the good work, fb and heroku!

[zealoushacker]

I should amend what I said below with a disclaimer: as long as Amazon AWS doesn't go down, and as long as heroku maintains its progress on ensuring the uptime of their cloud solution... :)