Skip to content

Conversation

@brettfo
Copy link
Member

@brettfo brettfo commented Feb 6, 2020

Some MailBoxProcessor tests are flaky so we're skipping them on official builds, but keeping them alive on PRs. This way we'll definitely know when they break, but the inherent flakiness won't prevent official builds.

@brettfo brettfo requested a review from KevinRansom February 6, 2020 21:03
@brettfo
Copy link
Member Author

brettfo commented Feb 6, 2020

Currently marked [WIP] until I can confirm that the tests run in this PR.

@baronfel
Copy link
Member

baronfel commented Feb 6, 2020

dotnet test (and nunit) support filtering test groups by category. Would it make sense to use that pre-existing mechanism here to exclude specific categories of flaky tests?

@brettfo
Copy link
Member Author

brettfo commented Feb 6, 2020

@baronfel Good call! Since I'm trying to selectively disable a small set of tests I've had to invert the example in the link, but let's hope that the != operator is supported for TestCategory.

@brettfo
Copy link
Member Author

brettfo commented Feb 7, 2020

I manually ran this so that it wasn't considered a PR build and verified that the expected tests were skipped, but everything else was run. Ready to merge once green.

@brettfo brettfo changed the title [WIP] skip flaky tests in official builds skip flaky tests in official builds Feb 7, 2020
@KevinRansom KevinRansom merged commit c648e2a into dotnet:master Feb 7, 2020
@brettfo brettfo deleted the flaky-tests branch February 7, 2020 18:43
nosami pushed a commit to xamarin/visualfsharp that referenced this pull request Feb 23, 2021
* skip flaky tests in official builds

* also filter tests on linux
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants