I use Proxmox and GPU passthrough works just fine (via QEMU). Note that Nvidia GPUs have less issues with passthrough, at least last I checked. See this guide: https://pve.proxmox.com/wiki/PCI_Passthrough
But if you’re running a standard distro, there are guides for most them.
Last thing to note is that your motherboard can make the process easier if it has good IOMMU support. Basically, you want a MB that puts your PCI slot in a separate IOMMU group. You can find examples by searching for “(MB name) IOMMU groups”.