There's no number high enough- at the end of the day people will just cry out "amagad but our homes!" and claim that that's priceless and forever want money and sympathy from everyone. A line must be drawn somewhere.
As long as the line is more on the side of the majority feeling it fair as opposed to a certain minority feeling it advantageous, then I'm fine with that.
If the price were enough, they would sell it. While I understand that this logic is not helpful when we try to develop the nation's infrastructure, we must not forget that we're imposing the desires of a more important mass of people at the depends of a tiny number of victims.
Plus there's the fact that in at least some US cases, the displacement is to allow space for a large business which will provide more tax revenue than the displaced could.