[armchairhacker]

The very first CS class taught at my undergraduate, where many students get exposed to programming the very first time and solve the simplest problems in functional style (e.g. list map, fold, fibonacci), introduces problems which require optimizations in the later part.

No matter how powerful your computer and how small your computer program is, you have to optimize at least the exponential-time algorithms, and sometimes the high-degree-polynomial ones. When writing a real-time or production app you have to optimize many polynomial or even linear-time algorithms.

Micro-optimizations like allocation and boxed numbers? Unnecessary unless you need performance, only apply a constant multiplier to your program’s speed. But macro-optimizations which affect time complexity can’t be ignored.

[saagarjha]

Algorithmic time complexity is a very important, but I honestly feel like a typical university education heavily underemphasizes exactly how much the constant factor can matter. Like, consider a simple example: someone's hand-written memcpy in C might achieve, say, 500 MB/s, maybe 1 GB/s if the loop is particularly tight. The one that ships on your system likely does 20-100 GB/s. Same big-O on both, of course. Most cases aren't this easy but there is a lot of performance that comes from constant factor optimization, which can dwarf all sorts of clever algorithms that are theoretically more efficient. Everyone likes to go "ok well if I scale this to a billion users your algorithm takes a month and mine takes a minute" but it is very likely that between now and the billion users the code is probably not even going to look anywhere near the same. But it's likely to be running on the same JVM or the same OS or the same allocator and all of those have been optimized to cut their constant factor down. After all, you don't want to be the guy who has the same algorithmic complexity as your competitor but your cloud bill is twice as much.

(This isn't to say you shouldn't do algorithmic improvements, and most of the performance work I do is in fact along those lines, but I do want to clarify that the "micro-optimizations" you're talking about are in fact the difference between a computer from today and the 1990s.)