GPUs may be used more and more for usage outside of graphics, but they don't lend themselves very well to audio, in particular recursive filters which are very common.
Nvidia has been working on this a lot lately. Check out CUDA Graphs. It's not there yet, but they're working on reducing kernel launch times. You still have the overhead of moving data to/from the GPU, but if you can come up with a complicated workflow that can run on the GPU it may be worth it.
yeah.. I think the unreliable scheduling also an issue with realtime dsp for audio on GPUs.
The only GPU using audio plugins I've seen were a couple of convolution reverb and "dynamic" convolution plugins (Nebula - I think they have dropped GPU processing since), and Nvidia has some realtime noise reduction thing running on the GPU aimed at gaming (RTX Voice - https://www.nvidia.com/en-us/geforce/guides/nvidia-rtx-voice...)