| >No regular developer will carefully review 50 changed snapshots. If you follow a habit of making small, incremental changes in your pull requests (which you should anyway), those 50 snapshots will generally all change in the same way. A glance across all of them to see that (for example) a box moved to the left in all of them. >The problem with snapshots is that it's doing the exact same thing. It asserts on lots of completely unimportant stuff. A problem more than made up for by the fact that rewriting it, eyeballing it and approving it is very quick. >Unlike proper unit tests however I can't make it better If the "then" part of a unit test "expects" a particular error message or json snippet, for instance, it's a giant waste of time to craft the expected message yourself if the test can simply snapshot the text from the code and you can eyeball it to see if the diff looks correct. I have written thousands of unit tests like this and saved a ton of time doing so. I've also worked with devs who did it your way (e.g. craft a whole or part of the json output and put it in the assert) and the only difference was that they took longer. >and personally do a good job of only asserting relevant things If, for example, you're testing a dashboard and the spec is a hand scribbled design that shows roughly what it looks like, all the other ways to assert the relevant things are vastly more expensive and impractical to implement. In practice most devs (and you too i expect) will simply leave the design untested and design regressions by, say, some gnarly css issue would be undetected except by manual test. >Snapshots are a blunt no-effort tool for the lazy dev 95% of devs don't use snapshot tests because they're super sensitive to flakiness and because ruthlessly eliminating flakiness from code and tests requires engineering discipline and ability most devs don't have. For those who can, it massively accelerates test writing. |
We try to replace them every time we come across one that needed adjusting actually. Quick is bad here. And yes they're flaky as hell if you use them for everything. Even a tiny change to just introduce a new element that's supposed to be there can change unrelated parts of snapshots because of generated names in many places.
Asserting on the important parts of some json output is not generally more expensive at all. You let the code run to generate the equivalent of a snapshot and then paste it in the assertion(s) and adjust as necessary. Yes it takes more time than a snapshot. But optimizing for time at that end is the wrong optimization. You're optimizing one devs time expenditure while making both the reviewers' and later devs' and reviewers' time expenditure larger (if they want to do a proper job instead of eyeballing and then YOLOing it).
As I see it, devs using snapshots are the opposite of a 10x dev. It's being a 0.1x dev. Thanks but no thanks.