True. We did a fair bit of experimenting with this (for both load balancing and failover) a few years back, and so long as you stay at 300sec or greater it works fine for well over 99% of all the traffic we tested with. Once you dropped below 300 seconds problems started appearing - from memory some older versions of Windows would default to 900secs, 3600secs, or 86400secs.
I haven't revisited that research for 3 or 4 years, but our findings then led to a policy of "if you're prepared to accept ~15mins of mixed availability (after you've identified a problem and hit the panic button) then DNS based failover works well enough. If you need significantly better response times than that, you need proper hardware/network based failover"
Do you have much Windows XP traffic? From memory, we saw evidence of XP using 86400sec (1 day) TTLs if you tried publishing TTLs lower than 300. (This was back in '08, so perhaps things have changed since. I'm still seeing ~30% WinXP in some of my Google Analytics accounts...)
Exactly - my biggest complaint about ELB on AWS is that they relied on people honouring a TTL that was ridiculously low, and as such now honoured by many clients.