[aeontech]

Postman supports dynamic variables for tests [1]. Of course nothing requires you to use postman, you could just as easily script this with just ruby and curb/httparty.

The reason for this is that a lot of api endpoints support a large number of parameters, and we want to be sure that we test a decent sample of real world usage instead of only a few examples that we can think of to implement in controller/request tests.

We can take a month’s worth of request logs from production, deduplicate them, run the requests against prod and record responses, then run same requests against stage and verify responses match even after you rewrite half the backing queries. This helps a lot with peace of mind when we deploy :)

For another, more advanced approach to this idea, see traffic mirroring/shadowing [2].

[1]: https://learning.postman.com/docs/writing-scripts/test-scrip...

[2]: https://www.haproxy.com/blog/haproxy-traffic-mirroring-for-r...

[joelbluminator]

Thanks for the detailed response! Never heard of this technique

[aeontech]

Oh, and for a practical example of the second approach, check out this blog about how Shopify rewrote their rendering engine: https://shopify.engineering/how-shopify-reduced-storefront-r...