Cool! Shouldn’t that be the end of the discussion then? If you can measure that one design converts or otherwise performs better then that’s the right one. What other considerations are needed?
A/B tests and their statistical brethren are useful in some contexts, but as you (almost) said, they come towards the end of a discussion. First you have to identify some variations that you want to test. And of course in some cases, particularly more complex UI designs, there might be no simple success metric to measure for an A/B test, and indeed there is probably no simple set of changes you can isolate from their context either.
In short, A/B tests are great, but they're not a substitute for design skills, they're a way of refining what you already have in some situations.
In short, A/B tests are great, but they're not a substitute for design skills, they're a way of refining what you already have in some situations.