[ravenstine]

Doesn't that depend on what the A/B test is measuring? A/B tests can measure the wrong thing. "Engagement" is often seen as a good thing in and of itself and is not investigated for errors.