[notyourday]

> If you did in fact mean Continuous Deployment then you do not need 100% integration test coverage or anything of that nature. Typically, the process might work by releasing a canary deploy which receives a fraction of the traffic going to production. In the unlikely event a breaking change has silently made it through your other tests, it should likely be caught when it is deployed as a canary

You have just described 100% integration test coverage.

[SamBam]

No? They just described QA testers or actual users finding the bugs.

i.e. the change made it through the tests, and now is deployed live, where a human may encounter and report it.

[notyourday]

That's applicable to a toy project.

If it is not a toy project then QA testing does not scale because there are thousands if not millions paths that request can flow through the infrastructure based on a user's actions.

That's why it is imperative to augment and nearly replace QA with end-to-end integration tests. Not only that but the integration tests must be funneled though the production gates to have a reasonable assurance that the code works in production configuration -- such as production request/task budgets, production latency and production tracking/tracing.

That in turn requires the test code being gated from production traffic ( which requires infrastructure to support request pinning ) and monitored/instrumented ( requires request tracing ) and actions ( error rate is 77 reqs/second when the test generates 4 req/sec means that the non-test traffic is landing on the test - remove test deploy immediately ).

So we are back to the integration tests nearly a 100% coverage to allow for unattended deploy.