[scaryclam]

I'm having similar issues with Selenium test at the moment. Can you give any insights into deflaking tests failing on timing or networking issues (I'm guessing it's a case by case basis, but generic tips might work for some problems)? This is by far the biggest pair of reasons for the suite failing for us, so any help would be really...er...helpful! :D

[dahart]

For me, the main class of flaky Selenium tests were expecting CSS related changed to appear on-screen in the same frame or one frame later, when sometimes they take more than one frame. The main result was to rarely/never execute multiple actions in a row without waiting; the tests changed considerably into pairs of [ action -> wait for specific CSS change on screen ] throughout all the tests. Also I did have some sleep calls for a few hard to test things, even though I knew it was a bad practice, and I spent time eliminating all of them and figuring out the fundamental reason it was hard to measure and/or adding something to the front end code to signal via CSS that a long running operation was complete.

On the setup side, the main networking issue for me was ssh. This might be different for you if you use either a Selenium cloud service, or a proper VPN setup. I was spinning up a virtual network on AWS for a Selenium farm in a fairly ad-hoc and manual way using ssh. The tests were sometimes (only very occasionally) starting before the ssh connection was actually ready, so I had to put a delay in the setup script to send & wait until a packet had actually been received before launching the test. I used netcat for that.

[larkost]

My approach has been to write a layer into our testing system that retries `find_element_by_*` for n seconds on failure (5 works for most of our tests) before reporting problems. I mean to eventually generalize and open-source them (I will try to contribute back, but since I only do this for Python, it might be rejected).

Just having this means a lot less time finding those problems for me.