| Here's my pretty graph for serving ~500,000 requests per minute on Heroku: https://api.playtomic.com/load.html Savings aren't exclusively because of Heroku, I also switched the underlying architecture during that migration from C# / ASP.NET to NodeJS which is exceptionally well suited to what I'm doing there. Previously:
Dedicated servers, ~$1600 a month 8x dedicated servers at ~$200 each, each running 3 to 5 instances of the API depending on how many IP addresses they were provisioned with. Uploading was done via a simple hand-rolled script that'd just FTP everything to each server. Now:
Heroku, current usage $400 - $500 a month With Heroku I don't have to worry about concurrent connections (typically 200 - 400 thousand people at once), I don't have to maintain all those servers and I don't have to fuck around with all the stupid things that can go wrong when you're operating at scale. It was a lot of work to get to this point and I made a lot of mistakes like having a heavy redis pub/sub outside of the EC2 network that cost me $350 in excess bandwidth providing inter-dyno communication, and I saturated database connections lots of times because in the old days those dedicated servers each had a local mongodb which could keep up with ordinary connection pooling, but it was totally worth it. |
It's not that I don't believe you, I just think you don't understand what you're saying.
(Actually, it is that I don't believe you)
That said, if this is if you're doing 500 requests/sec (that's very different than 500 concurrent users) per dyno, good for you. My main bottleneck was not so much CPU on the web machines (I hit memory limitations), but the database layer.