Sure, but as a counter example Docker performance on Mac has historically been abysmal[0][1], but everyone on Mac I know still develops using it. We ignore the performance hit on dev machines, knowing it won't affect prod (Linux servers).
I don't see why this pattern would fail to hold, but am open to new perspectives.
The VM that uses takes advantage of hardware-accelerated virtualization, for running amd64 VMs on amd64 CPUs. You don't have hardware-accelerated virtualization for amd64 VMs on any ARM CPUs I know of...