[nenxk]

I mean in his example the discount bug they ran into wouldn’t have needed any card numbers that could have been discovered with fake/cloned transactions that contained no customer detail in this case it seems it would have been best to test the payment processing in personal at a single store and then also testing with sales logs from multiple other locations

[ChrisGreenHeur]

yep, it sounds like the first implementation step really should have been to gather a large test set of data and develop the system with that in mind after understanding the test data, starting with making tests from the test data.

[skeeter2020]

They explained the scenario though and it seems like a combination of rarer edge cases. It's great to think your awesome dev team and QA would have collected test data representing this ahead of time, but surely you've all been caught by this? I know I have; that's why we don't have flawless systems at launch.

[eszed]

OP said elsewhere that the specific discount that produced the $10M sales + refund combo was only active at five stores. That's the sort of edge case that you can't count on being in the training data. I'm as big a fan of real-data testing as anyone, but there's always something not represented.