Yes. The UX is much better.
Linux customization capabilites are phenomenal I give you that, but it takes a very big amount of time if you want something specific for you, and when an update hits and things just break :-(
Gnome actually has a different philosophy. There wasn't much customization offered at first, as the focus was on nailing a single set of UX and aesthetics. And I think the Gnome team succeeded, but the lack of customization is/was divisive.
I use Linux for my homeserver, and used to have it on my HTPC too. On the HTPC I only used openbox, so in a sense it's even less complicated than xfce. The trouble is with the software for the htpc stuff, like remote gaming, audio, videoplaying retrogames software and so on.
The part that ruined my experience is the amount of configuration needed to get there, and updates that constantly broke either my video player, the audio or the remote gaming. Either a driver update with a breaking change, or the audio config that needed repair and stuff like this. Everytime I'd spend obscene amount of time to try to find what's the culprit, and it usually came down to updates breaking one thing or another. On windows it's seemless and in some ways with better performance and ease of installation. Up and running in 15 minutes and with total control to boot...