I wonder if Amazon has overfitted and/or a/b tester itself into a bad local optima. It’s pretty hard for me to believe that their current website really is as good as their data indicates.
IME a/b tests are often run by people with little to no knowledge of statistics or experimental procedure. It is pretty easy to end up backing bad decisions with data when you don't completely understand the data.
"Hey, check this out! User engagement as a function of time spent on amazon.com is up 125% with the new build!"
Once a metric becomes a target for optimization, it often loses its value as an indicator of a larger goal. People who obsess over A/B tests rarely understand that.