I sometimes laugh to myself when people complain about the price of GPUs. Yes, $10,000 is a lot, but in historical context, it's pretty reasonable for top of the line technology.
Beyond graphics, I think of all the "terascale" talk in the high-performance computing world when I was in school. Now your consumer GPU does multiple Tflops instead of hoping a supercomputer to maybe possibly reach 1 Tflop some day for a few lucky users.
The same thing has happened with RAM and storage. My first Linux PC had 20 MB of RAM and 80 MB of disk and was sufficient to do most of my CS projects at university in the early-mid 1990s. Now, a sub $200 smartphone has over 100x the space, while desktops are commonly 2000x.
The research side not only moved on 1000x to "petascale" but that's now boring and there is real talk of "exascale" with the same gleam in the eye. One million times the performance we dreamed of at the beginning of my career, though I think this is partly by expanding the scope of one machine to larger and larger distributed systems as well as scaling up the capacity of individual elements.
I sometimes laugh to myself when people complain about the price of GPUs. Yes, $10,000 is a lot, but in historical context, it's pretty reasonable for top of the line technology.