How much do I need to wait until TensorFlow can run stable and at the same speed as on cuDNN on AMD hardware? I can't even contemplate buying AMD right now (gaming is not very important to me).
AMD's performance deficit is about to get a lot worse. Nvidia's upcoming Volta architecture is massively optimised for deep learning - they're touting a 12x performance increase for training and 6x for inferencing over Pascal.
I think Intel have a better chance of catching up with Nvidia at this stage. They've been on an acquisition spree and have picked up a huge amount of DL-related IP. They have immense R&D and fab resources at their disposal.
That's only if you use Volta's Tensor cores however.
AMD's 16-bit packed performance with Vega is more than respectable vs NVidia's 16-bit packed performance in Pascal.
In the future, all AMD needs to catch up to Volta's Tensor cores is to build Tensor cores themselves. That doesn't seem like a major technical hurdle. I'm fairly certain that Google would be the primary patent holder on Tensor-cores.
Google open sourced CUDA support for Clang/LLVM, which has been upstream a while now (years) and is kept up to date with various CUDA versions. This has not been a collaboration with AMD.
I haven't kept up to date on what AMD has done with that work, but i believe they use it as part of their compatibility story.