I am interested to see what they will be testing in some of the upcoming posts in this series. It would be fun to be scrolling Netflix and have the transparency to know that I'm seeing the 'B' test.
Like all controlled experiments though, the experimenter wants to hide that information from the subject (user in this instance) to measure how they respond to the change itself, rather than the change and being told about it.
* they create two recommendation lists based on distinct principles or parameters;
* they pick a first (with an A/B test);
* mix those with ‘schoolyard rules’: each side picks the next candidate that wasn’t picked already, creating a new, alternating list.
* They show that to the user and track whether you picked an even or odd item on that recommendation, indicating whether A or B is preferable.
So most like, you are seeing A and B.