Sure, but office jobs are a fairly small portion of the overall workforce. Based on [0], it looks like the "Information", "Financial activities", "Professional and business services" and I'll assume 50% of "other" and the government jobs could be done remotely. I'm sure this is missing some jobs in the other categories, and there are certainly many, jobs in these categories that can't be done remotely, but so goes. That leaves < 30% of jobs that could be done remotely.
I would say ~30% is not "fairly small". Also that 30% predominately works circa 9-5 and is disproportionately responsible for traffic congestion.
Plus, it's unclear how those industries get split. For example utilities, trade, healthcare, and leisure/hospitality all have a large number of office jobs. And cities have a higher concentration of office jobs than rural areas. I would be willing to bet that in most medium-large sized cities, half of all jobs could be done remotely.