I stopped caring about this by having home-manager do all this for me. I don't really care where the files are anymore because I change them through Nix, not directly, and I can choose where that is located.
Further, using a tmpfs for root (and by extension ~), means that junk doesn’t accumulate (while Home Manager ensures what should be there, is there).