FsharpSuite.tests reduction diet #14590
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
T-Gro
commented
Jan 11, 2023
tests/FSharp.Compiler.ComponentTests/Miscellaneous/FsharpSuiteMigrated.fs
Outdated
Show resolved
Hide resolved
T-Gro
commented
Jan 11, 2023
tests/FSharp.Compiler.ComponentTests/Miscellaneous/FsharpSuiteMigrated.fs
Show resolved
Hide resolved
T-Gro
commented
Jan 11, 2023
T-Gro
commented
Jan 11, 2023
T-Gro
commented
Jan 11, 2023
T-Gro
commented
Jan 11, 2023
T-Gro
commented
Jan 31, 2023
T-Gro
commented
Feb 1, 2023
tests/FSharp.Compiler.ComponentTests/Miscellaneous/MigratedTypeCheckTests.fs
Show resolved
Hide resolved
abonie
approved these changes
Feb 2, 2023
psfinaki
approved these changes
Feb 2, 2023
tests/FSharp.Compiler.ComponentTests/Miscellaneous/FsharpSuiteMigrated.fs
Outdated
Show resolved
Hide resolved
KevinRansom
approved these changes
Feb 4, 2023
T-Gro
commented
Feb 6, 2023
T-Gro
commented
Feb 6, 2023
T-Gro
commented
Feb 10, 2023
tests/FSharp.Compiler.ComponentTests/Miscellaneous/FsharpSuiteMigrated.fs
Show resolved
Hide resolved
tests/FSharp.Compiler.ComponentTests/Miscellaneous/FsharpSuiteMigrated.fs
Outdated
Show resolved
Hide resolved
tests/FSharp.Compiler.ComponentTests/Miscellaneous/FsharpSuiteMigrated.fs
Outdated
Show resolved
Hide resolved
tests/FSharp.Compiler.ComponentTests/Miscellaneous/FsharpSuiteMigrated.fs
Outdated
Show resolved
Hide resolved
vzarytovskii
added a commit
that referenced
this pull request
Feb 13, 2023
This reverts commit d0dfdb9.
T-Gro
pushed a commit
that referenced
this pull request
Feb 13, 2023
T-Gro
added a commit
that referenced
this pull request
Feb 15, 2023
abonie
pushed a commit
that referenced
this pull request
Feb 21, 2023
…west CI job (#14752) * Revert "Revert "FsharpSuite.tests reduction diet (#14590)" (#14742)" This reverts commit a8e39b3. * Fix build * #load ing shims for scripting tests (especially 'exit') so that they can work in process * Fixing failing widen tests, but breaking new 'printfn with static literals' feature * Try to make **both** implicit conversions and non-inline strings in printfn work * Skip failing test on MacOs * Disable one more (also nuget-related) test on MacOs
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
FsharpSuite.Tests is the longest taking test suite in the longest CI job.
By making it faster, we can speed up end-to-end time for our builds and therefore reduce contributor pain.
The nature of the older test suite was by copying over test directories, emiting an artificial fsproj, and running it with "dotnet run" command. This naturally involved a lot of process creations and IO operations.
The newer test suite in component tests avoids unnecessary process spawns and therefore has smaller per-test overhead, while still maintaining the ability to run the tests with various command line switches like:
debug+/-
optimize+/-
The test definitions were copy pasted as is.
The test invocations functions have been reimplemented in the in-memory test framework.
In a few cases, the test scripts did have some hardcoded assumptions about it's current directory, available files etc. -> fixing those to make the tests run were the only (necessary) changes done to the test files themselves.
For the tests being ported, this lead to a reduction from ~15 minutes to less than 5.
There is still potential to go faster by reusing FSI environments instead of re-creating, but that made some of the tests depend on each other due to naming clashes and that was not good.
So even if it is slightly slower, each test now has a fresh environment.