[chacham15]

> Actually, I think I could build this as a third party provider, even

I dont use Heroku, but if you read Rap Genius' original paper discussing the queuing problem, they specifically stated that there is nothing that they can do about it.

[rdl]

RG couldn't do anything while using Heroku's load balancing infrastructure (well, other than rewriting their app to be more constant in performance).

A third-party could do a load balancer entirely outside of Heroku or even AWS, feeding sessions in based on feedback from the application ("intelligent routing"). Heroku even offers a way to bill for that kind of thing. You can open outgoing connections from each dyno and then load-balance over that.

Doing it with ELB would be possible I think, but I don't really like AWS ELB. I think it would make the most sense to do near US-East AWS, but arguably a CDN type service could do it from many locations. Being closer to users (as long as your connectivity to US-East/Heroku is still good) helps performance more.