[sologoub]

This entire thing against Heroku is so disingenuous... The fact that New Relic didn't expose these metrics is not great, but has very little to do with Rap Genius team not knowing about the metric.

Apparently, the fact that requests can be queued at Dyno level was common public knowledge back in 2011! Here's a quote from Stackoverflow answer:

"Your best indication if you need more dynos (aka processes on Cedar) is your heroku logs. Make sure you upgrade to expanded logging (it's free) so that you can tail your log.

You are looking for the heroku.router entries and the value you are most interested is the queue value - if this is constantly more than 0 then it's a good sign you need to add more dynos. Essentially this means than there are more requests coming in than your process can handle so they are being queued. If they are queued too long without returning any data they will be timed out."

Source: http://stackoverflow.com/a/8428998/276328

When you use a PaaS, it doesn't mean you don't need to be serious about it and completely forget about all technical aspects. Granted, it should have been included with New Relic from day one, but hardly justifies such a direct and persistent attack on Heroku.

[amatix]

From the article, it sounds like they were well aware of the logs & queue values, but they were misleading:

Their logs are STILL incorrect. Here’s a sample line:

  2013-03-02T15:41:24+00:00 heroku[router]: at=info method=GET path=/Asap-rocky-pretty-flacko-lyrics host=rapgenius.comfwd="157.55.33.98" dyno=web.234 queue=0 wait=0ms connect=3ms service=366ms status=200 bytes=25582

Those queue and wait parameters will always read 0, even if the actual value is 20000ms. And this has been the case for years.

[sologoub]

Interesting... just ran my own test and looks like I have jumped to conclusion too fast. NewRelic and the queue data do not seem to match, so I have some more reading to do.

[cbs]

Here's a quote from Stackoverflow answer

I tend to read (and trust) official documentation before Stack Overflow. I use Stack Overflow and it is great tool and all, but it can be really hit or miss. It doesn't cover every corner of every tech, and unless the answer is availible somewhere on the internet, or the person answering has first-hand experience it can lead to misleading wishy-washy answers.

Ultimately, pointing to SO you're lowering the expectations of a paid service from "the documentation reflects the product" so damn low to the point of "users should read everything googleable about the product they're using, and trust that OVER the official docs. Including mailing list posts from 2011 and a stack overflow question that asks a different question than you're asking"

[vannevar]

The problem wasn't that queuing delay was impossible to detect. The problem was that the documentation described a specific load balancing setup that would have guaranteed better performance per dyno, and that setup was not in fact what was being delivered. It was clearly a material misrepresentation, and in any other service context would constitute a deceptive trade practice. That Heroku is being defended at all is a testament to the goodwill they've built up in the tech community, but it doesn't change the fact that they misrepresented their service, even if it was negligence rather than malice.