A former colleague of mine quit a job over this. They wanted him back into the office. He mostly worked with their Toptal devs from Nigeria and and Hungary.
I worked on a team that was mostly Seattle people while I was based in the Los Angeles HQ. All my team members were in a different city. Needless to say "switching" to remote wasn't a big deal at all for me.