We should not have any skipped tests, it's just dead code that will only get more broken over time.
Every skipped test should either be:
- Enabled if it adds value and is found to be reliable.
- Deleted if it adds no value or is overly fragile and will slow development down.
- If the test adds value but is not 100% reliable or is too slow, it should be marked using the proper test category. Normal developer work flows will not run these tests as often. Results from this category of tests may have to be manually inspected for regressions.
An initial idea is to move all ignored tests into a generic test category that is ignored by most contributors and CI. This at least will get the tests running again. The long term goal is to have as few tests as possible of this sort however, so the next step would be to sift through all tests in the generic category to either delete them if they provide no value, or put them into the correct, more specific test category.
We should not have any skipped tests, it's just dead code that will only get more broken over time.
Every skipped test should either be:
An initial idea is to move all ignored tests into a generic test category that is ignored by most contributors and CI. This at least will get the tests running again. The long term goal is to have as few tests as possible of this sort however, so the next step would be to sift through all tests in the generic category to either delete them if they provide no value, or put them into the correct, more specific test category.