Hacker News new | ask | show | jobs
by tluyben2 2308 days ago
> It seems increasingly common these days to not worry about performance at all,

You don't even have to continue there. People, who should know better, assume that 'modern cloud stuff' will make this trivial. You just add some auto-scaling and it can handle anything. Until it grinds to a halt because it cannot scale beyond bottlenecks (relational database most likely) or the credit card is empty trying to pull in more resources beyond the ridiculous amount that were already being used for the (relatively) tiny amount of users.

This will only get worse as people generally use the 'premature optimization' (delivering software for launch is not premature!) and 'people are more expensive than more servers' (no they are not with some actual traffic and O(n^2) performing crap) as excuse to not even try to understand this anymore. Same with storage space; with NoSQL, there are terabytes of data growing out of nowhere because 'we don't care as it works and it's 'fast' to market, again 'programmers are more expensive than more hardware!'). Just run a script to fire up 500 aws instances backed by Dynamo and fall asleep.

I am not so worried about premature optimization ; I am more worried about never optimization. And at that; i'm really worried about my (mostly younger) colleagues simply not caring because they believe it's a waste of time.

2 comments

> I am not so worried about premature optimization ; I am more worried about never optimization. And at that; i'm really worried about my (mostly younger) colleagues simply not caring because they believe it's a waste of time.

I'm simultaneously worried about both, because I've had to deal with poor architecture and unnecessarily convoluted & difficult-to-work-with code that was only justified by completely misguided optimization attempts (with no experimentation or profiling to back any of it up; and indeed, performance in practice was terrible!). At the same time, there's a constant stream of "oh no this convoluted mess has a bug in it" and "oh no we need a new feature, it can't take long?" tickets but never a ticket that says "profile and optimize the program because it's ridiculously slow, oh and refactor and undo the convoluted mess."

There's also something to be said for building better tooling in this area. Not everyone can achieve expertise in everything. Better tooling helps level the playing feel (& eventually outperform experts when the tooling becomes indispensable).

You may think that's a cop-out, but consider something like coz[1]. Sqlite is managed and maintained by experts. There's significant capital behind investing engineering effort. However, better tooling still managed to locate 25% of performance improvement[2] & even 9% in memcached. Even experts have their limits & of course these tools require expertise so a tool like coz is still an expert-only tool. The successful evolution of the underlying concept for mass adoption will happen when it's possible to convert "expert speak" into something that can be easily and simply communicated outside CPU or compiler experts to meet the user on their knowledge level so they can dig in as deep as they need to/want to.

[1] https://github.com/plasma-umass/coz [2] https://arxiv.org/abs/1608.03676

>Not everyone can achieve expertise in everything.

But if every young or beginner programmer who asked a performance question on reddit, or stack overflow, could get good answers instead of lectures on how what they are doing is "premature optimization" every single time, the world would collect quite a bit more expertise on making things perform well.

I try to remind people whenever I can that Knuth was talking about noodling loops in miniscule way. Between optimizing compilers, out of order super scalar CPUs and very different performance characteristics of modern CPUs, what he was talking about basically doesn't exist anymore.
I don't think this is a helpful/accurate view. At a high level the type of optimization activity Knuth was talking about is alive and well, although the details of what people spend that time on has sometimes shifted.

I agree this quote is often abused but the fundamental idea behind it is intact and important: Sure, if you don't at least thing architecturally about performance early on as your problem domain reveals itself, you can make some poor decisions with long reaching performance implications. But on the other hand, if you spend a bunch of time tuning code when you don't know what the use will look like that time can be a dead loss.

This latter point was what Knuth was referring to - and in 2020 teams are still prematurely optimizing; i suspect about as much as they were back then.

This idea is important when trying to finish an actual product of some kind, but when kids (or adults!) are learning, let them fiddle with the loops! Let them learn some intricacies and encourage the curiosity.
Nothing wrong with fiddling with loops either, in that context.

I was objecting to the idea that premature optimization isn't a real problem anymore because technology. That's just not true.