"Right" is a pretty dubious way of framing it when basically every company works this way. You don't really have any choice in the matter. Passing a law about this wouldn't be removing any rights of individuals any more than we "removed" their right to enter a contract for pay lower than minimum wage or with unsafe working conditions.
A more fair way of phrasing it would be to say that we should remove the right of companies to require it.
If you wanted you could also require companies to make this an option in the contract, which would it clear how much they are actually willing to pay for restricting peoples freedom outside of work in this way and gives employees the option of deciding if that tradeoff is worth it for them. I think that companies would not be willing to pay all that much for this if they couldn't just force it on everyone.
A more fair way of phrasing it would be to say that we should remove the right of companies to require it.