[ankitdce]

Haha great point. Well from what we have learned from our users is "fixing" test typically end up with "delete most of them". Fixing tests can be time consuming effort.

Another way to think about it is, whether Flaky tests are worth keeping? At some point if the tests fail often, do these really add value. And we think - it does. If you are able to identify flakiness from real failure and reduce noise, you can still avoid real failures.

[rio517]

Wow. That works like really poor technical leadership. Fixing flaky tests (as opposed to deleting them) is indeed time consuming, but it is a far cheaper choice than getting to the point your test suite is untrustworthy.

There may be a point where the cost of ownership for a specific test exceeds its utility, but the way to resolve that is usually to reevaluate your code and supporting tests. Suppressing flaky tests seems a very unwise choice.

Perhaps under extreme circumstances and with unhealthy code bases there may be a case for this, but I struggle to imagine it.

[ankitdce]

That is a fair argument. Not all organizations have the bandwidth to measure and manage stability of builds. Some companies build internal tools / dev productivity team for this purpose. There are always right intentions to comment out the flaky test with the mindset of coming back to it, but it is also a very low priority item in most cases when you have to ship new features.

Fixing flaky tests can very commonly take longer than writing new tests.