I go to the office around one week out of every four, it's not that rare. Sure, there's some catching up, but not that much. Mostly it's the continuous interruptions that are never time boxed, the way they are when remote.
That’s odd, in my experience most engineers know to at least send a slack message first before talking to a fellow engineer in person, unless it’s an emergency.