[zimbatm]

Yes that would be one possible implementation. It's up to the server and can vary quite a bit depending on the scenario, which is why Nginx hasn't tackled the problem yet.

I believe that it's going to take a while to find good heuristics that work in most of the cases. Until then only big companies like Google will have the financial incentive to develop their own custom version that's tailored to their specific use-case.