160k is around 0.15% of Japan's population, and there's plenty of space for them to move to since the countryside is rapidly depopulating. Of course, it's not quite that easy to forcibly migrate people in practice...
Moving jobs to the countryside does not sound like a bad idea TBF. I want to move to a small town and be able to take a bike to work (and I prefer a workplace, remote working doesn't work for me), but all of the employers are in the big city so I'm forced to sit in traffic for over an hour both ways every day.