[tharkun__]

No regular developer will carefully review 50 changed snapshots. They'll stop doing a proper job of it after the third or fourth that looks like the same trivial unimportant change and miss the bug found by snapshot 37.

I do agree that a lot of people write bad tests meaning the test name does not properly describe what the test is supposed to be about so I can check the test implementation and assertions against intent. They also like you say assert on superfluous things.

The problem with snapshots is that it's doing the exact same thing. It asserts on lots of completely unimportant stuff. Unlike proper unit tests however I can't make it better. In a unit test I can make an effort to educate my peers and personally do a good job of only asserting relevant things and writing individual tests where the test name explains why "transitional has to be None".

Snapshots are a blunt no-effort tool for the lazy dev that then later requires constant vigilence and overcoming things humans are bad at (like carefully checking 50 snapshots) by many different humans VS a unit test that I can make good and easy to comprehend and check by putting in effort once when I write it. A good one will also be easy to adjust when needed if it comes time to actually need to change an assertion.

[MoreQARespect]

>No regular developer will carefully review 50 changed snapshots.

If you follow a habit of making small, incremental changes in your pull requests (which you should anyway), those 50 snapshots will generally all change in the same way. A glance across all of them to see that (for example) a box moved to the left in all of them.

>The problem with snapshots is that it's doing the exact same thing. It asserts on lots of completely unimportant stuff.

A problem more than made up for by the fact that rewriting it, eyeballing it and approving it is very quick.

>Unlike proper unit tests however I can't make it better

If the "then" part of a unit test "expects" a particular error message or json snippet, for instance, it's a giant waste of time to craft the expected message yourself if the test can simply snapshot the text from the code and you can eyeball it to see if the diff looks correct.

I have written thousands of unit tests like this and saved a ton of time doing so. I've also worked with devs who did it your way (e.g. craft a whole or part of the json output and put it in the assert) and the only difference was that they took longer.

>and personally do a good job of only asserting relevant things

If, for example, you're testing a dashboard and the spec is a hand scribbled design that shows roughly what it looks like, all the other ways to assert the relevant things are vastly more expensive and impractical to implement.

In practice most devs (and you too i expect) will simply leave the design untested and design regressions by, say, some gnarly css issue would be undetected except by manual test.

>Snapshots are a blunt no-effort tool for the lazy dev

95% of devs don't use snapshot tests because they're super sensitive to flakiness and because ruthlessly eliminating flakiness from code and tests requires engineering discipline and ability most devs don't have.

For those who can, it massively accelerates test writing.

[tharkun__]

Regrettably we still have some snapshot tests in our code base, yes. I cringe every time one goes red and I'm supposed to check them. Like you say, I eyeball them and after the fourth that's the same pattern I give up and just regenerate them. Meaning they might as well not be there coz they won't catch any actual bugs for me.

We try to replace them every time we come across one that needed adjusting actually. Quick is bad here. And yes they're flaky as hell if you use them for everything. Even a tiny change to just introduce a new element that's supposed to be there can change unrelated parts of snapshots because of generated names in many places.

Asserting on the important parts of some json output is not generally more expensive at all. You let the code run to generate the equivalent of a snapshot and then paste it in the assertion(s) and adjust as necessary. Yes it takes more time than a snapshot. But optimizing for time at that end is the wrong optimization. You're optimizing one devs time expenditure while making both the reviewers' and later devs' and reviewers' time expenditure larger (if they want to do a proper job instead of eyeballing and then YOLOing it).

As I see it, devs using snapshots are the opposite of a 10x dev. It's being a 0.1x dev. Thanks but no thanks.

[MoreQARespect]

>We try to replace them every time we come across one that needed adjusting actually. Quick is bad here. And yes they're flaky as hell if you use them for everything. Even a tiny change to just introduce a new element that's supposed to be there can change unrelated parts of snapshots because of generated names in many places.

If you can't keep the flakiness under control then yeah, they'll be worse than useless because they will fail for no discernible reason at all.

[tharkun__]

Oh the reasons are discernable. I call it flaky when you make an unrelated change and the snapshots change. You go check why and all you can do is a facepalm. What you and I call "unrelated" may be different. Such as when I make a CSS change that simply affects some generated class names for example and a bunch of snapshots fail. This will be worse in code bases with lots of reusable CSS of course, i.e. your blast radius for flakiness will be much larger the more CSS reuse you have and the more snapshot tests you have. Ours is very controllable but only because we're doing the right things (such as reducing snapshot use).

That's when you start cursory looks at the first few changes and then just regenerate them, which means they will never find any actual bugs coz you ignore them.

It's "the boy who cried wolf" basically.