[w0utert]

The main problem would be to have to maintain the myriad of known working configurations, since they are all different depending on motherboard, BIOS, GPU, CPU, etc. If you are careful about picking the right parts (and assembling them properly), it's actually really easy to configure on any recent Linux distro, you can just click together the VM using virt-manager if you don't need anything special.

I know I had to jump through a lot of hoops to make it work though, my X470 motherboard didn't isolate the USB controller without a BIOS update for example, and after that the USB controller exhibited USB FLR (function level reset) problems causing it to hang the VM. This required blacklisting it's PCI ID from the Linux kernel and patching the kernel to disable FLR (fortunately these changes were later merged into the mainline kernel). I also had problems with the second GPU, if I plugged it into any slot other than the bottom x1 slot, the motherboard BIOS would reshuffle the IOMMU groups making it impossible to pass through the NVME and USB controller, or (if I put it in the second x16 slot) it would halve the PCIe bandwidth to the RTX3080.

All in all it took me the better part of a weekend to get everything working, but if I had to do it again from scratch and did some research into (in particular) the motherboard and BIOS, I would be able to set it everything up again in less than an hour.