-
Notifications
You must be signed in to change notification settings - Fork 832
Revive "FsharpSuite.tests reduction diet" => saves 29 minutes off slowest CI job #14752
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
abonie
merged 8 commits into
main
from
revert-14742-revert-14590-fsharp-suite-reduction-diet
Feb 21, 2023
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
96597e8
Revert "Revert "FsharpSuite.tests reduction diet (#14590)" (#14742)"
T-Gro 9d182c0
Fix build
T-Gro 65eca65
#load ing shims for scripting tests (especially 'exit') so that they …
T-Gro 187e4e9
Fixing failing widen tests, but breaking new 'printfn with static lit…
T-Gro e951fdd
Merge branch 'main' into revert-14742-revert-14590-fsharp-suite-reduc…
T-Gro b812106
Try to make **both** implicit conversions and non-inline strings in p…
T-Gro 80aeb6c
Skip failing test on MacOs
T-Gro b10257d
Disable one more (also nuget-related) test on MacOs
T-Gro File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
166 changes: 166 additions & 0 deletions
166
tests/FSharp.Compiler.ComponentTests/Miscellaneous/FsharpSuiteMigrated.fs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,166 @@ | ||
| // Copyright (c) Microsoft Corporation. All Rights Reserved. See License.txt in the project root for license information. | ||
|
|
||
| namespace FSharp.Compiler.ComponentTests.Miscellaneous.FsharpSuiteMigrated | ||
|
|
||
| open System | ||
| open System.IO | ||
| open Xunit | ||
| open FSharp.Test | ||
| open FSharp.Test.Compiler | ||
| open FSharp.Test.ScriptHelpers | ||
|
|
||
|
|
||
|
|
||
| module Configuration = | ||
| let supportedNames = set ["testlib.fsi";"testlib.fs";"test.mli";"test.ml";"test.fsi";"test.fs";"test2.fsi";"test2.fs";"test.fsx";"test2.fsx"] | ||
|
|
||
| module ScriptRunner = | ||
| open Internal.Utilities.Library | ||
|
|
||
| let private createEngine(args,version) = | ||
| getSessionForEval args version | ||
|
|
||
| let defaultDefines = | ||
| [ | ||
| #if NETCOREAPP | ||
| "NETCOREAPP" | ||
| #endif | ||
| ] | ||
|
|
||
| let runScriptFile version (cu:CompilationUnit) = | ||
| let cu = cu |> withDefines defaultDefines | ||
| match cu with | ||
| | FS fsSource -> | ||
| File.Delete("test.ok") | ||
| let engine = createEngine (fsSource.Options |> Array.ofList,version) | ||
| let res = evalScriptFromDiskInSharedSession engine cu | ||
| match res with | ||
| | CompilationResult.Failure _ -> res | ||
| | CompilationResult.Success s -> | ||
| if File.Exists("test.ok") then | ||
| res | ||
| else | ||
| failwith $"Results looked correct, but 'test.ok' file was not created. Result: %A{s}" | ||
|
|
||
| | _ -> failwith $"Compilation unit other than fsharp is not supported, cannot process %A{cu}" | ||
|
|
||
| /// This test file was created by porting over (slower) FsharpSuite.Tests | ||
| /// In order to minimize human error, the test definitions have been copy-pasted and this adapter provides implementations of the test functions | ||
| module TestFrameworkAdapter = | ||
| open FSharp.Test.Compiler.Assertions.TextBasedDiagnosticAsserts | ||
|
|
||
| type ExecutionMode = | ||
| | FSC_DEBUG | ||
| | FSC_OPTIMIZED | ||
| | FSI | ||
| | COMPILED_EXE_APP | ||
| | NEG_TEST_BUILD of testName:string | ||
|
|
||
| let baseFolder = Path.Combine(__SOURCE_DIRECTORY__,"..","..","fsharp") |> Path.GetFullPath | ||
|
|
||
| let diffNegativeBaseline (cr:CompilationUnit) absFolder testName _version = | ||
| let expectedFiles = Directory.GetFiles(absFolder, testName + ".*") | ||
| let baselines = | ||
| [ for f in expectedFiles do | ||
| match Path.GetExtension(f) with | ||
| | ".bsl" -> cr, f | ||
| | ".vsbsl" -> cr |> withOptions ["--test:ContinueAfterParseFailure"], f | ||
| | _ -> () ] | ||
| [ for compilationUnit,baseline in baselines do | ||
| compilationUnit | ||
| |> typecheck | ||
| |> withResultsMatchingFile baseline ] | ||
| |> List.head | ||
|
|
||
|
|
||
| let adjustVersion version bonusArgs = | ||
| match version with | ||
| | LangVersion.V47 -> "4.7",bonusArgs | ||
| | LangVersion.V50 -> "5.0",bonusArgs | ||
| | LangVersion.V60 -> "6.0",bonusArgs | ||
| | LangVersion.V70 -> "7.0",bonusArgs | ||
| | LangVersion.Preview -> "preview",bonusArgs | ||
| | LangVersion.Latest -> "latest", bonusArgs | ||
| | LangVersion.SupportsMl -> "5.0", "--mlcompatibility" :: bonusArgs | ||
|
|
||
|
|
||
| let singleTestBuildAndRunAuxVersion (folder:string) bonusArgs mode langVersion = | ||
| let absFolder = Path.Combine(baseFolder,folder) | ||
| let supportedNames, files = | ||
| match mode with | ||
| | NEG_TEST_BUILD testName -> | ||
| let nameSet = | ||
| Configuration.supportedNames | ||
| .Add(testName+".fsx") | ||
| .Add(testName+".fs") | ||
| .Add(testName+".fsi") | ||
| .Add(testName+"-pre.fs") | ||
| let files = Directory.GetFiles(absFolder,"*.fs*") |> Array.filter(fun n -> nameSet.Contains(Path.GetFileName(n))) | ||
| nameSet, files | ||
| | _ -> Configuration.supportedNames, Directory.GetFiles(absFolder,"test*.fs*") | ||
|
|
||
| let mainFile,otherFiles = | ||
| match files.Length with | ||
| | 0 -> Directory.GetFiles(absFolder,"*.ml") |> Array.exactlyOne, [||] | ||
| | 1 -> files |> Array.exactlyOne, [||] | ||
| | _ -> | ||
| let mainFile,dependencies = | ||
| files | ||
| |> Array.filter (fun n -> supportedNames.Contains(Path.GetFileName(n))) | ||
| // Convention in older FsharpSuite: test2 goes last, longer names like testlib before test, .fsi before .fs on equal filenames | ||
| |> Array.sortBy (fun n -> n.Contains("test2"), -n.IndexOf('.'), n.EndsWith(".fsi") |> not) | ||
| |> Array.splitAt 1 | ||
|
|
||
| mainFile[0],dependencies | ||
|
|
||
| let version,bonusArgs = adjustVersion langVersion bonusArgs | ||
|
|
||
| FsFromPath mainFile | ||
| |> withAdditionalSourceFiles [for f in otherFiles -> SourceFromPath f] | ||
| |> withLangVersion version | ||
| |> fun cu -> | ||
| match mode with | ||
| | FSC_DEBUG | FSC_OPTIMIZED | FSI | COMPILED_EXE_APP -> | ||
| cu | ||
| |> ignoreWarnings | ||
| |> withOptions (["--nowarn:0988;3370"] @ bonusArgs) | ||
| | NEG_TEST_BUILD _ -> | ||
| cu |> | ||
| withOptions (["--vserrors";"--maxerrors:10000";"--warnaserror";"--warn:3";"--nowarn:20;21;1178;52"] @ bonusArgs) | ||
| |> fun cu -> | ||
| match mode with | ||
| | FSC_DEBUG -> | ||
| cu | ||
| |> withDebug | ||
| |> withNoOptimize | ||
| |> ScriptRunner.runScriptFile langVersion | ||
| |> shouldSucceed | ||
| | FSC_OPTIMIZED -> | ||
| cu | ||
| |> withOptimize | ||
| |> withNoDebug | ||
| |> ScriptRunner.runScriptFile langVersion | ||
| |> shouldSucceed | ||
| | FSI -> | ||
| cu | ||
| |> ScriptRunner.runScriptFile langVersion | ||
| |> shouldSucceed | ||
| | COMPILED_EXE_APP -> | ||
| cu | ||
| |> withDefines ("TESTS_AS_APP" :: ScriptRunner.defaultDefines) | ||
| |> compileExeAndRun | ||
| |> shouldSucceed | ||
| | NEG_TEST_BUILD testName -> diffNegativeBaseline (cu |> withName mainFile) absFolder testName langVersion | ||
|
|
||
| |> ignore<CompilationResult> | ||
|
|
||
|
|
||
| let singleTestBuildAndRunAux folder bonusArgs mode = singleTestBuildAndRunAuxVersion folder bonusArgs mode LangVersion.Latest | ||
| let singleTestBuildAndRunVersion folder mode version = singleTestBuildAndRunAuxVersion folder [] mode version | ||
| let singleTestBuildAndRun folder mode = singleTestBuildAndRunVersion folder mode LangVersion.Latest | ||
|
|
||
| let singleVersionedNegTestAux folder bonusArgs version testName = | ||
| singleTestBuildAndRunAuxVersion folder bonusArgs (NEG_TEST_BUILD testName) version | ||
| let singleVersionedNegTest (folder:string) (version:LangVersion) (testName:string) = | ||
| singleVersionedNegTestAux folder [] version testName | ||
| let singleNegTest folder testName = singleVersionedNegTest folder LangVersion.Latest testName |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.