A few months ago I build a personal linux server and use it via ssh and VSCode's remote development feature, which mount the server filesystem and do port forwarding so it feels as if it's a local machine even while working outside my home network. My usual workflow doesn't change at all, and I got the benefit of a fast server while using laptop form factor. The new arm laptop is probably perfect for this setup, allowing you to build for intel arch whenever you need it.
Cross compilation is a thing, JFYI. You don't need to be on an arm64e system to generate an arm binary today. This is the case for xcode, gcc, go, etc etc