[themarkn]

It’s possible (and indeed uh ... recommended) to write frontend tests that are resilient to DOM changes by not depending on things that are not essential to the functionality being tested. Eg select a form field by its label, not by its position in the DOM, and use specific data attributes if there’s no better choice. Then when you gut the HTML for UI overhaul, if the functionality hasn’t changed, the cypress test doesn’t change, you just make sure you attach the attributes to the right new elements.

Of course if the functionality changed (like you split something up into 2 pages) yeah you need to update your UI tests cause you are testing a different experience now. But they are UI tests, so yeah. That’s appropriate.

I dunno. It sounds like you found other parts of it unwieldy as well, but the idea that the tests are inherently brittle is wrong. Time spent maintaining UI based testing is saved many times over when compared with the manual clicking around that it saves imo.

[dpc_pw]

> Then when you gut the HTML for UI overhaul, if the functionality hasn’t changed,

How often website redesigns are only about html? Usually everything changes. New flows, new dialogs, new toolbars, new frameworks.

Often the web UI gets and Android UI and iOS UI siblings. Gets replaced altogether by a completely different UI, developed by a different team.

APIs change much less often, and can be shared between many UIs. You can also version them. How are you going to version a web UI? Like reddit with "turn back on classic look"? How long can you keep the old UI around just because you have 1000s tests around that use it.

> but the idea that the tests are inherently brittle is wrong

I don't think so. Tests are great calcifyiers (check my blogpost about it if you want). If you test though your UI, you're calcifying your UI. The more you test though it, the more you calcifying it. No way around it.

https://dpc.pw/be-warned-tests-are-great-calcifiers

[themarkn]

I’m not against testing APIs. But testing APIs does not tell you if your latest CSS change broke pointer events on your “add to cart” button, so you can no longer receive orders, etc.

If you are wholesale redesigning/rebuilding a whole site and all the workflows yes of course you need new tests. But also your users would get pretty impatient if entire ways of working are changing all the time in a way that makes testing a giant moving target.

The purpose of a UI test in my mind is to make sure that the core business things a user is supposed to be able to do, are doable. In the context of your blog posts, I think those things should be calcified.

Like, I want a test to fail if I remove a workflow that used to be there. I want a test to fail if a form field suddenly has no label. There should be those alerts when functionality that was previously understood to be correct has been changed. Lightweight UI tests with some easily re-approved screenshot diffs goes a long way.

[eitland]

related: testing-framework has been adapted for cypress. It encourages this kind of thinking and testing.

I've used cypress at work but have suspended it since we used it end-to-end and it clogged up backend databases.

[arcturus17]

Can’t you set up a test database instead of testing against production? And set up timeouts to control the tempo of the tests if necessary?