>"Other side-effects of a larger population, like increased traffic"
Remote workers don't commute though. Vermont's population has also been declining for years though and before that trend it was stagnant for decades[1]. Also if you've been to Vermont you know there's plenty of room to build there.
That seems like less of a problem if all the new residents are working remotely though, yeah? There would of course be some new traffic, but it wouldn't be a huge mass of people commuting to the same place.