[whack]

> Rather than trying to generate hypothetical ideas for "how can we make our search better", we spend a lot of time analyzing our data to find where we are failing. Many of our biggest relevance improvements have come from tracking and understanding the types of queries where we consistently fail to generate results or user engagement.

This sounds a lot like the 6-sigma approach of driving improvement by focusing obsessively on eliminating "defects".

There are certainly huge wins that can be obtained by identifying and eliminating bugs or corner-cases with undesired behavior. But it's scary to imagine a world where this is used as a replacement for innovative thinking - ie, "how can we make our search better". If Steve Jobs had focused all his proverbial efforts on minimizing flip-phone defects, the world would have missed out on the smartphone revolution.

[asdgagbiobnio]

The iPhone's competition was not the flip-phone. It was the PDA and the Blackberry and the pocket PC. The iPhone was an evolution of previous similar devices.

I still do not understand why people consider smartphones revolutionary. It is revolutionary that everyone has one on them at all times, but the gadgets themselves aren't all that.

[seanpquig]

Yea I don't think it's the only principle that should drive product development, but it helps ensure you're always solving real problems for your users.

There is still a lot of room for creativity once you've identified a class of problematic queries too. Especially as a search engine becomes more sophisticated, how you solve clear query failures can be a lot less straightforward, and clever features or machine learning are many times needed.

I will say there are clear exceptions to this inversion rule too. For example, we switched to a Learn-To-Rank system for our core ranking in the past year and we couldn't necessarily point to it clearly being the solution for problematic queries we were seeing, but it proved to unlock a ton of value and drive a lot of relevance improvements and surprising benefits in ways we couldn't necessarily predict for our specific use case and users.

[PostLee]

That's why it's important to focus on effectiveness first. At any point in time you should know what you are trying to solve and why. The Inversion Principle is simply a useful tool to helps support that and figure out the how, but is by no means a silver bullet.