[noogle]

Models are re-trained periodically (months, weeks, even days), and new architectures/implementations come all the time. If a better algorithm appears, practitioners will adopt a new platform (e.g. Transformers for NLP models), so many systems can already plug-in new tools. GPUs are very expensive so there is also a strong incentive to make this little effort.

[floatngupstream]

Yes, but this just makes a frictionless runtime for inference even more important (which is something that does not exist in a comparable form for AMD).