[Sohcahtoa82]

Incorrect conclusions are frequently drawn from A/B tests. To some, this makes it unscientific, to others, it just means it's bad science. I think the argument is more semantic than objective.

For example, if your metric is "time spent interacting on the platform", then a testing of a rollout of a feature ends up with longer page load times, so users spend more time there because they're waiting for pages to load would increase that metric, and management decides it's a good idea.