[kaashif]

Right, and if you come up with an efficiency gain that makes scaling better, e.g. a 50% reduction in required compute. Or even asymptotic improvements e.g. moving from quadratic to linear. Then you're much much better off.

There is nothing about the bitter lesson that says just be dumb and pour money into a hole, you still have to invent the methods to scale well, and being under immense pressure with constraints seems likely to produce that research.

[Enginerrrd]

It reminds me a bit of the tyranny of the rocket equation. You can always scale your fuel to get a little more delta V, with ever diminishing returns. …but for something like a DEEP space/interstellar mission, it almost always pays to wait a few more years for a faster propulsion system because you’ll get there fastest by always delaying your launch and chasing better technology.

I’m not sure how well the analogy holds up, or if there’s anything to be learned from it though.

[RetroTechie]

https://en.wikipedia.org/wiki/Interstellar_travel#Wait_calcu...

Certainly applies more general imho. Constrained by some resource -> invest resources elsewhere, and/or invest in reducing the constraint(s) encountered.