Many people can’t choose where they live either. Getting a visa to the US is a ludicrous process and even if they wanted to they maybe tied by family obligations.
But people don't choose to be rich or poor exactly either. As a general rule, discrimination is around things for which there's no choice. Having a choice over where they live or whether they are rich doesn't mean it's easy or practical. But that can't make it grounds for discrimination, even if unfair.