[cfallin]

Sure, but I think this is just reiterating the original point that hardware is cheap and programmers are expensive. If you need to meet a performance goal and hardware is cheap but programmers are expensive, then throw hardware at it. The point I was (trying to) make is that when you have N instances of the problem to solve (where N is large), often the cost of "scaling the hardware" is prohibitive; optimizing performance by 1% might yield operational cost savings that justify the programmer's salary, simply because the 1% is multiplied by N. This is orthogonal to parallelism/scalability.

I'll grant that this case may be rare (i.e. most people are optimizing for initial results, not optimizing a large existing machine) but it does happen!

[TeMPOraL]

Of course it does happen, it's not a rare case, it's the usual case. Any remotely successful product will have N = millions of users. The only reason this doesn't enter the calculation is that you don't get the money you saved by not making your users do unnecessary hardware upgrades.

[cfallin]

Ah! So there's the key point: do your users run your software or do you, on your servers? If the former, yes, it's an externalized cost and may not enter your economic calculations. If the latter, you care a lot :-)