[oxfordmale]

A/B testing is valuable to optimise an existing process once the low-hanging fruit has been tackled, but it should also be discontinued when the return on investment becomes negligible. Because of organisational momentum, this rarely happens, as a team would make themselves redundant. This can result in A/B testing noise.

Data also provides valuable insight in a negative manner. If your conversion rate is abysmal, the data tells you to get out of your cubicle and start talking with real customers to find out what the data isn't telling you. It is still a data-driven decision. It is just a negative one.

However, in the end, data isn't going to find your next billionaire dollar opportunity. You need to find a gap in the market that no one has tackled before, and of course there is no data for, otherwise someone else would have jumped on it.