[runnerup]

With 6-7 more doublings in compute power per watt, consumers will have the power of 1000 A100 GPU's in their iPhone. "Eventually" will come before I die, at least. That would happen outside of GP's time estimate but a moderately funded university consortium could probably afford it just a few doublings from now.

[pksebben]

I think we'll reach architectural changes that make this moot before we reach hardware for it. The way we train these models is constantly in flux, and we just need someone to crack continuous learning so we can pass models around and train them en-masse, using the collective unused compute that is literally sitting on mine and everyone else's desk right now.