For me the killer feature of home-manager versus other dotfiles managers is that it also installs any software your dotfiles depend on or assume to be present on the machine.
To me it is marking what is “garbage” vs what isn’t in the .config folder. I like to experiment on linux a lot, like switching to plasma and back to Gnome and it always leaves like 20 mostly-empty folders trashed. I can then just remove the folder and go on my way with home-manager.
If you're running NixOS there's an impermanence plugin that allows you to specify specific directories and files you need to persist outside of what NixOs/HM builds for you and then use a tmpfs in-memory partition for root and/or home. Then at boot you're loading up the NixOS/HM config and then the persistent files are symlinked or mounted to where they need to be, but otherwise you have a fresh system on each reboot. It's a bit painful and manual to figure out exactly what to keep (unless you just want to keep the entire .config, for instance, which I don't) but it's amazing to know that you can mess up your home directory as much as you want and then it'll automatically clean itself with a reboot.