[aeontech]

You are right, but there are many many methods in any codebase that really are, for all intents and purposes, read-only - ie, get a bunch of data from data sources, stitch it together, and return a json blob.

These tools would make me feel a lot more confident refactoring code like this (actually, right now I use Postman automated API response recordings with a manually extracted dataset of representative query parameters from production logs to validate our changes) - so this indeed looks like it might make my life easier :)

[joelbluminator]

How do u stitch your logs params with Postman? And btw arent controller/request tests better for that?

[aeontech]

Postman supports dynamic variables for tests [1]. Of course nothing requires you to use postman, you could just as easily script this with just ruby and curb/httparty.

The reason for this is that a lot of api endpoints support a large number of parameters, and we want to be sure that we test a decent sample of real world usage instead of only a few examples that we can think of to implement in controller/request tests.

We can take a month’s worth of request logs from production, deduplicate them, run the requests against prod and record responses, then run same requests against stage and verify responses match even after you rewrite half the backing queries. This helps a lot with peace of mind when we deploy :)

For another, more advanced approach to this idea, see traffic mirroring/shadowing [2].

[1]: https://learning.postman.com/docs/writing-scripts/test-scrip...

[2]: https://www.haproxy.com/blog/haproxy-traffic-mirroring-for-r...

[joelbluminator]

Thanks for the detailed response! Never heard of this technique

[aeontech]

Oh, and for a practical example of the second approach, check out this blog about how Shopify rewrote their rendering engine: https://shopify.engineering/how-shopify-reduced-storefront-r...