Hacker News new | ask | show | jobs
by hitchstory 527 days ago
Snapshot tests are even more sensitive to flakiness than E2E tests but flakiness is still a bug that can (and probably should) be fixed like any other.

As an example - the different browser you have running locally and in CI? That's a bug. The fix is to run the browser in a container - the same container in both environments. You can probably get away without this with an E2E test. Snapshot test? No way.

Ive seen people decry and abandon E2E tests for the same reason - simply because they couldnt get flakiness under control and thought that it was impossible to do so.

I dont think it's intrinsically impossible though. I think it's just a hard engineering problem. There are lots of nonobvious points of flakiness - e.g. unpinned versions, selects without order bys, nondeterministic JavaScript code, etc. but few that are simply IMpossible to fix.

Theres a big payoff if you get it right, too.