[loup-vaillant]

> Like I said, the vast majority of code optimization opportunities are not worth taking. Some are, but only after running the numbers.

Casey Muratori said it best: there are 3 philosophies of optimisation. You're talking about the first: actual optimisation where you measure and decide what to tackle. It's rarely used, and with good reason.

The second philosophy however is very different: it's non-pessimisation. That is, avoid having the CPU do useless work all the time. That one should be applied in a fairly systematic basis, and it's not. To apply it in practice you need to have an idea of how much time your algorithm requires. Count how many bytes are processed, how many operations are made… this should give a nice upper bound on performance. If you're within an order of magnitude of this theoretical maximum, you're probably good. Otherwise you probably missed something.

The third philosophy is fake optimisation: heuristics misapplied out of context. This one should never be used, but is more frequent than we care to admit.

[knorker]

I'm actually also talking about the second.

> avoid having the CPU do useless work all the time

It's not worth an engineer spending 1h a year even investigating this, if it's less than 20 CPU cores doing useless work.

The break even for putting someone full time on this is if you can expect them to save about fourty thousand CPU cores.

YMMV. Maybe you're a bank who has to have everything under physical control, and you are out of DC floor space, power budget, or physical machines.

There are other cases too. Maybe something is inherently serial, and the freshness of a pipeline's output has business value. (e.g. weather predictions for tomorrow are useless the day after tomorrow)

But if you're saying that this second way of optimizing is that things should be fast for its own sake, then you are not adding maximum value to the business, or the mission.

Performance is an instrumental goal of an effort. It's not the ultimate goal, and should not be confused for it.

[loup-vaillant]

In the specific case of batch processing, I hear you. Machine time is extremely cheap compared to engineer time.

Then there are interactive programs. With a human potentially waiting on it. Someone's whose time may be just as valuable as the engineer's time (morally that's 1/1, but even financially the difference is rarely more than a single order of magnitude). If you have as few as 100 users, shaving off seconds off their work is quickly worth a good chunk of your time.

Machine time is cheap, but don't forget that user's time is not.

[knorker]

I'm up there on the mound preaching the same thing, trust me.