[aeorgnoieang]

This was a good metaphor but I don't understand the last line. Our tests can never 'really' clean 'real ears' can they? [Actually, that's kind of an interesting idea. Send a real order thru your inventory, order management, etc. systems ...]

[pavement]

Payment processors frequently conduct "no-capture" test transactions for authorizations of possibly $1, before tokenizing CC info. Arguably, this follows the pattern of an integration test.

Ecommerce providers often follow a process of cycling an immaterial test product through a real credit card for pennies, with a petty cash credit card, to exercise the purchase/refund cycle, end-to-end, depending on the activities of the project. A runaway batch process with an infinite loop of purchases is bad, infinite refunds is even worse.

Sometimes these tests are needed to ensure that the service account exists on the third party system, is recognized, and has limited permissions for a fixed, restricted set of API calls. The tests might be run only during a release deployment, against live production, but more frequently against a third party sandbox host while coding. This is a very common pattern.

[aeorgnoieang]

> Ecommerce providers often follow a process of cycling an immaterial test product through a real credit card for pennies, with a petty cash credit card, to exercise the purchase/refund cycle, end-to-end, depending on the activities of the project.

That sounds familiar. I'm sure that was being done at the last place I worked, an e-commerce 'agency'.

[taeric]

If you aren't sending real orders through your systems.. you are waiting for your customers to find and report all problems. Right?

This is actually a strength of a QA team. They can build test plans and execute them on a different technology than your Dev team built the system with and on. This is also why acceptance testing is so important. You want to get as close to possible as capturing "in the wild" users using your system as you can.

[bglusman]

I have tests that do this :-) Well, not for an order, but with a sort-of-monolith sort-of-microservice system with email, dashboards, reporting, a JS SPA and a lambda/flask/zappa based API thats decoupled from the admin app, we want one live, round trip of all the data flowing and actions along the way... we can point it at staging, production, and I think dev (via docker-compose, but that part wasn't done last I worked on this). It relies on some special hard-coded/pre-setup accounts and stuff, but they're real, just marked as test related in their names so everyone knows not to touch them.

[jdbernard]

I think the point was to notice the continuum between completely fake stubs that only have enough "real" behavior to facilitate unit tests and a full-fledged production scenario. Integration tests are not the same as production, but they are much closer than unit tests. In integration tests as much is real as possible vs. unit tests where as little is real as possible.