Scaling may take us to some strange places, but it's worth noting that an Apple M1 chip has >100x the transistors of a 2004-era Pentium 4, and achieves ~500x the FLOPs at a fraction of the power draw.
Yes this isn't single-threaded performance, but I think we should keep in mind that exponential improvement in price/performance over many decades is possible, if never certain.