It's not at all clear why poor people should ever want to leave their allocated urban subdivision. We certainly don't want them clogging up tourist spots that should be kept for appreciation by sophisticated wealthy foreigners.
We don't want the CO2 that is created (and other consequences of oil production/consumption), the money can be spent on something better, and we don't want to depend on oil-exporting countries more than necessary.