[saagarjha]

Algorithmic time complexity is a very important, but I honestly feel like a typical university education heavily underemphasizes exactly how much the constant factor can matter. Like, consider a simple example: someone's hand-written memcpy in C might achieve, say, 500 MB/s, maybe 1 GB/s if the loop is particularly tight. The one that ships on your system likely does 20-100 GB/s. Same big-O on both, of course. Most cases aren't this easy but there is a lot of performance that comes from constant factor optimization, which can dwarf all sorts of clever algorithms that are theoretically more efficient. Everyone likes to go "ok well if I scale this to a billion users your algorithm takes a month and mine takes a minute" but it is very likely that between now and the billion users the code is probably not even going to look anywhere near the same. But it's likely to be running on the same JVM or the same OS or the same allocator and all of those have been optimized to cut their constant factor down. After all, you don't want to be the guy who has the same algorithmic complexity as your competitor but your cloud bill is twice as much.

(This isn't to say you shouldn't do algorithmic improvements, and most of the performance work I do is in fact along those lines, but I do want to clarify that the "micro-optimizations" you're talking about are in fact the difference between a computer from today and the 1990s.)