Look at the EU: although there is complete freedom of movement of labour for EU citizens, cultural and language barriers prevent true movement to a far greater degree than within the US.
Moreover, movement of labour within the EU is mostly unidirectional. A German or a Brit is very unlikely to go to work in Poland or Bulgaria, whereas the other way around is almost always the case. This unidirectional movement sucks out young blood from the donor countries and leads them to an even direr future.