It's for workers' rights. Even ole' John Calvin (who probably started it in Switzerland) wanted Sunday as a day off because it protected workers rights against overwork, not because he saw it as mandated by the Bible.
Hi, Swiss person here. This does not address why it should be Sunday for everyone. Nobody's saying you should work 7 days a week. We could have different off days for different people.