[Animats]

Whether inference hardware will be a commodity remains to be seen. We had twenty years of most servers being x86-64. That's commodity compute. Inference hardware isn't a mature architecture yet. There will be progress and obsolescence.

[asdf88990]

It will be in less than 5 years. You can already buy second hand RTX3090 TIs and with most of modern Macs being able to run some mind of AU locally already, in 5 years, things will be very commodified once you get the Chinese start rolling out Cuda-compatible GPUs.