|
|
|
|
|
|
by tom-jh
2576 days ago
|
|
|
We run in-browser end to end tests for our browser extension. There were several reasons for flakiness: * Puppeteer (browser automation) bugs or improper use. Certain sequence of events could deadlock it, causing timeouts relatively rarely. The fix was sometimes upgrading puppeteer, sometimes debugging and working around the issue. * Vendor API, particularly their oauth screen. When they smell automation, they will want to block the requests on security grounds. We have routed all requests through one IP address and reuse browser cookies to minimize this. * Vendor API again, this time hitting limits on rare situations. We could have less parallel tests, but then you waste more time waiting. Eventually, we will have to mock up this (fairly complex) API to progress. It's got to a point where I don't feel like adding more tests because they may cause further flakiness - not good. |
|
|