[bitL]

We are already past the point of no return. RTX 8000 is now an entry-level GPU that allows training some of the latest NLP models. Attention is spreading over to computer vision models as well, so one could expect memory bloat coming there quickly. Only large companies that can deploy thousands of GPUs in parallel will be able to compete.

[latchkey]

I am working on it... (well, the company I work for)... except instead of thousands... it is hundreds of thousands.