[alxv]

A/A tests (also known as Null tests) are useful to validate that users are assigned to the control and experiment groups without bias.

Offline resampling methods, like bootstrapping, are better if you're looking to robustly estimate the variance of the experiment.

[chucklarge]

Yes, a Null test really should be the first test you run if you're looking to get into A/B testing.

You should also run an ongoing A/A test across your site or app to have confidence your bucketing, data pipeline, stats tests and effect on metrics is working as expected over time.