[bawolff]

Seems like a lot of weirdly defensive people in this thread.

But the essay doesnt really seem that revolutionary more common sense. If you want to make an impact, dont work in an oversaturated field. The low hanging fruit is probably already picked and other people will probably get there before you do. But you also dont want to work in a field nobody cares about as noone will care. Working on a problem with proven demand, but seems "boring" and hasn't changed much recently is a good bet, as there is probably new insights you can apply and new contexts that have appeared since last time there was a frenzy for that field.

[mannykannot]

The advice to not work in an oversaturated field, if you want to make an impact, looks to me a bit like the reverse of looking for one's keys under the lamp post, because that is where the light is. We should give some credence to the proposition that saturation is the wisdom of the crowd at work, figuring out where a breakthrough is likely, and it is usually only with hindsight that over-saturation is apparent.

[bawolff]

I somewhat suspect by the time something is "popular" it is, almost by definition, oversaturated. Its sort of like weird investment strategies (e.g. always buy stocks on mondays, or whatever I dont actually know anything about stocks). Sure some of them may work originally, but unless i am extremely well connected, by the time i hear about it, everyone else knows too and the market has corrected for it.

Perhaps another metaphor is a gold rush. Even if there really is gold in those hills, if everyone knows there is, its probably already too late to go out and buy a shovel.

That said, i agree that there is plenty of survivorship and hindsight bias when it comes to any advice on how to be succesful.

[stereolambda]

There is a difference here without the society preference and what you should do as an individual. As you are saying, in crowded fields breakthroughs may be likely (or not, due to overexploitation, rampant fashion instead of principles thinking and people digging themselves too deep into one paradigm), but in any case, you are not likely to be participating in them if you're not already a strong player. Thus, if you are doing something like research or development of new things, your marginal impact will be probably tiny.

I would put two caveats on this. First, if you are globally in a good position (world's top university, good connections to get into places) you may be positioned to meaningfully get even into a mature field. Second, if you don't want to do much research and exploration, it's reasonable to reap benefits from specialist knowledge of already proven fields. Just keep an eye on other options to transition to if everything eventually crashes.

Even in crowded areas you should benefit from wide knowledge of horizons and fundamentals beyond what is now fashionable. It may mean just more ideas and perspective. I work in NLP with an interest in AI, and I've dived fairly deep into currently out-of-scope things like rules-based NLP, symbolic AI and biological neurons, their physics and simulations etc. I hope this gives me more viability that your average deep learning guy from a moderate background.

[xwowsersx]

> If you want to make an impact, dont work in an oversaturated field.

That's exactly the opposite of his point, if I understood it correctly!