[mrsharpoblunto]

Thats why you should also track other metrics when AB testing performance changes. During my time at IG, even small perf improvements almost always corresponded to stat-sig improvements in engagement metrics. 100ms is a pretty significant performance win, definitely not noise - the other thing is that those wins often scale by the quality of device & connection. So a 100ms win for a high end desktop on fiber internet could be multiple seconds for a low end mobile on 3G.