[matxip]

This will be considered suboptimal by some, I'm sure, but the solution I went with is to reserve the dedicated GPU for the VM and use Intel integrated for Linux, in my case from a skylake i5-6600. Aside from that, you can get a cheap, low power second GPU (if you're using an AMD CPU for example). This is fine for me since I'm gaming on the VM, and not really doing much graphically intensive on Linux.

The idea of getting another high end card seems pretty absurd when you're putting windows in a VM specifically to deal with those situations. I recognize some people have special use cases, but they're just that, special use cases.

Really, overall, I don't consider it to be too bad of a trade off for what you get.