Soundsource [0] on macOS can do this! I run Chow Tape [1] and it performs quite admirably. I've never measured the latency but it has never affected me much. I'm certain a similar solution is available on Linux.
To be fair, (quite janky, ime) VST hosts are available for Windows, via the music player (e.g. Musicbee) or system-wide, but those have never been reliable. Your mileage will vary.