[gmokki]

When I fixed many similar selenium/webdriver tests the root cause was always the same: You grab reference to an element and for example wait it to become enabled or some text to appear. But your ui framework actually replaces the element in the dom while doing its thing and your reference to stale element will never change. Fix is to loop searching the element with selector and check if the element fills the conditions. If not, retry from search again. We had nice helpers for those and had very stable selenium tests.

[naasking]

> Fix is to loop searching the element with selector and check if the element fills the conditions. If not, retry from search again. We had nice helpers for those and had very stable selenium tests.

Thanks, I'll double check, but I think we do this now. In looking at the history of test failures, those failures are indeed less common, but still plenty of false positives of other types. Most persistent recent failures are the WebDriver timing out when loading a URL, which has never happened while manual testing or when being used by end users, so not sure what's going on there.

In any case, if the Playwright API encourages better idioms for writing tests that avoids these pitfalls, that would be cool because I deal with a lot of work term students that aren't adept at this kind of stuff so that would save a lot of headaches.

[masklinn]

In my experience an other common issue is a race condition in the trigger, aka the system is not quite settled yet when an interaction is performed leading to that not registering.

This is more likely when the system is loaded / shared e.g. a CI Vm.

It’s instructive to watch a screencast / recording of UI tests, because you don’t necessary intuit how spazzy and fast the harness will perform its interactions.