"Should" is a loaded word that I won't address, but commonly "work from home" is the term used when you're still located near the office, vs. "remote" which is where you're nowhere near an office.
It's not about being close to the office. For many teams it's about being in similar timezones. Companies hiring along the NA west coast for instance is pretty common and makes sense if you want people to collaborate closely.