It seems to work fine everywhere, except in EC2 VPC, where the arp cache can sometime becomes stale. We too reported this issue to AWS support, but have no idea if they are doing anything about it.
The workaround is to apply a sysctl change to revert to the old behavior prior to the commit. Or to use a subnet larger than /24 to reduce the chance of getting the same IP.