[stevekemp]

I have similar experience. I once worked with a pretty big site which used a load-balancer configured to route all traffic to the server based on average response-time.

The intention was that if a server was returning results "quickly" that meant it was least-loaded, and could handle the newest requests.

What it actually meant though was that the server disk filled up, and it started returning "500, Internal Server Error" errors. Very quickly.

At the point the alarms were raised almost all incoming traffic had been routed to this dead/dying host.

[bluedino]

I had almost the same thing happen once - but the server was serving requests 'very quickly' because it was caching everything based on the LB requesting it.