I'm surprised to see no mention of Guix. It solves all of these issues, and already has a good story both for cross and native builds on a variety of architectures.
Adding new targets is deceptively easy, just copy an existing template and substitute your values.
What do you mean by image? In Nix you instantiate a NixOS system and specify the architecture. There are a number of tools to build a disk image from a NixOS system.