-
Notifications
You must be signed in to change notification settings - Fork 832
FsharpSuite.tests reduction diet #14590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
5a04fd4
Porting selected fsharpsuite tests to ComponentTests without fsc.exe/…
T-Gro b20a672
Merge branch 'main' into fsharp-suite-reduction-diet
T-Gro ca842bc
Support for multisource tests (fsi,test2)
T-Gro cd04c01
Making migrated tests pass
T-Gro 9df37fc
Merge branch 'main' into fsharp-suite-reduction-diet
T-Gro 623eb47
Pinvoke tests only on windows
T-Gro 381e1f2
negTests -> rendering of errors to match baseline files (to be finis…
T-Gro 5f02e37
Merge branch 'main' into fsharp-suite-reduction-diet
T-Gro 2baf5fa
Merge branch 'main' into fsharp-suite-reduction-diet
vzarytovskii f77bbd4
Merge branch 'main' into fsharp-suite-reduction-diet
T-Gro 6712b6b
Merge branch 'main' into fsharp-suite-reduction-diet
T-Gro d955b2d
negative tests string comparison
T-Gro fd6fb45
neg test integration
T-Gro 64e8b05
making array negative tests pass
T-Gro d499290
TEST_UPDATE_BSL support
T-Gro fe2ad0b
Removing wrong duplicate definitions from old .bsl files
T-Gro 3b638f6
Putting stderr output of .fsx tests into test output window
T-Gro 34e26e9
Overload resolution tests migrated over
T-Gro 25dbb25
Separating migrated tests into files, removing them from old tests.fs
T-Gro f398ebe
Merge branch 'main' into fsharp-suite-reduction-diet
T-Gro 547b68c
Test framework - script execution in separate thread (to get shorter …
T-Gro 05093a0
Trying to address stackoverflow for MacOs CI run
T-Gro 4b16738
Fixing negtest45 run to match older test suite
T-Gro 086e272
Apply suggestions from code review
T-Gro 466b232
Merge branch 'main' into fsharp-suite-reduction-diet
T-Gro 27928bf
conflicts resolved
T-Gro d05aeef
Merge branch 'main' into fsharp-suite-reduction-diet
T-Gro bab08f8
Merge branch 'main' into fsharp-suite-reduction-diet
T-Gro 81d060f
Apply suggestions from code review
T-Gro File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
176 changes: 176 additions & 0 deletions
176
tests/FSharp.Compiler.ComponentTests/Miscellaneous/FsharpSuiteMigrated.fs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,176 @@ | ||
| // Copyright (c) Microsoft Corporation. All Rights Reserved. See License.txt in the project root for license information. | ||
|
|
||
| namespace FSharp.Compiler.ComponentTests.Miscellaneous.FsharpSuiteMigrated | ||
|
|
||
| open System | ||
| open System.IO | ||
| open Xunit | ||
| open FSharp.Test | ||
| open FSharp.Test.Compiler | ||
| open FSharp.Test.ScriptHelpers | ||
|
|
||
|
|
||
|
|
||
| module Configuration = | ||
| let supportedNames = set ["testlib.fsi";"testlib.fs";"test.mli";"test.ml";"test.fsi";"test.fs";"test2.fsi";"test2.fs";"test.fsx";"test2.fsx"] | ||
|
|
||
| module ScriptRunner = | ||
T-Gro marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| open Internal.Utilities.Library | ||
|
|
||
| let private createEngine(args,version) = | ||
| let scriptingEnv = getSessionForEval args version | ||
| scriptingEnv.Eval """ | ||
| let errorStringWriter = new System.IO.StringWriter() | ||
| let oldConsoleError = System.Console.Error | ||
| System.Console.SetError(errorStringWriter) | ||
| let exit (code:int) = | ||
T-Gro marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| System.Console.SetError(oldConsoleError) | ||
| if code=0 then | ||
| () | ||
| else failwith $"Script called function 'exit' with code={code} and collected in stderr: {errorStringWriter.ToString()}" """ |> ignore | ||
| scriptingEnv | ||
|
|
||
| let defaultDefines = | ||
| [ | ||
| #if NETCOREAPP | ||
| "NETCOREAPP" | ||
| #endif | ||
| ] | ||
|
|
||
| let runScriptFile version (cu:CompilationUnit) = | ||
| let cu = cu |> withDefines defaultDefines | ||
| match cu with | ||
| | FS fsSource -> | ||
| File.Delete("test.ok") | ||
| let engine = createEngine (fsSource.Options |> Array.ofList,version) | ||
| let res = evalScriptFromDiskInSharedSession engine cu | ||
| match res with | ||
| | CompilationResult.Failure _ -> res | ||
| | CompilationResult.Success s -> | ||
| if File.Exists("test.ok") then | ||
| res | ||
| else | ||
| failwith $"Results looked correct, but 'test.ok' file was not created. Result: %A{s}" | ||
|
|
||
| | _ -> failwith $"Compilation unit other than fsharp is not supported, cannot process %A{cu}" | ||
|
|
||
| /// This test file was created by porting over (slower) FsharpSuite.Tests | ||
| /// In order to minimize human error, the test definitions have been copy-pasted and this adapter provides implementations of the test functions | ||
| module TestFrameworkAdapter = | ||
| open FSharp.Test.Compiler.Assertions.TextBasedDiagnosticAsserts | ||
|
|
||
| type ExecutionMode = | ||
| | FSC_DEBUG | ||
| | FSC_OPTIMIZED | ||
| | FSI | ||
| | COMPILED_EXE_APP | ||
| | NEG_TEST_BUILD of testName:string | ||
|
|
||
| let baseFolder = Path.Combine(__SOURCE_DIRECTORY__,"..","..","fsharp") |> Path.GetFullPath | ||
|
|
||
| let diffNegativeBaseline (cr:CompilationUnit) absFolder testName _version = | ||
| let expectedFiles = Directory.GetFiles(absFolder, testName + ".*") | ||
| let baselines = | ||
| [ for f in expectedFiles do | ||
| match Path.GetExtension(f) with | ||
| | ".bsl" -> cr, f | ||
| | ".vsbsl" -> cr |> withOptions ["--test:ContinueAfterParseFailure"], f | ||
| | _ -> () ] | ||
| [ for compilationUnit,baseline in baselines do | ||
| compilationUnit | ||
| |> typecheck | ||
| |> withResultsMatchingFile baseline ] | ||
| |> List.head | ||
|
|
||
|
|
||
| let adjustVersion version bonusArgs = | ||
| match version with | ||
| | LangVersion.V47 -> "4.7",bonusArgs | ||
| | LangVersion.V50 -> "5.0",bonusArgs | ||
| | LangVersion.V60 -> "6.0",bonusArgs | ||
| | LangVersion.V70 -> "7.0",bonusArgs | ||
| | LangVersion.Preview -> "preview",bonusArgs | ||
| | LangVersion.Latest -> "latest", bonusArgs | ||
| | LangVersion.SupportsMl -> "5.0", "--mlcompatibility" :: bonusArgs | ||
|
|
||
|
|
||
| let singleTestBuildAndRunAuxVersion (folder:string) bonusArgs mode langVersion = | ||
| let absFolder = Path.Combine(baseFolder,folder) | ||
| let supportedNames, files = | ||
| match mode with | ||
| | NEG_TEST_BUILD testName -> | ||
| let nameSet = | ||
| Configuration.supportedNames | ||
| .Add(testName+".fsx") | ||
| .Add(testName+".fs") | ||
| .Add(testName+".fsi") | ||
| .Add(testName+"-pre.fs") | ||
| let files = Directory.GetFiles(absFolder,"*.fs*") |> Array.filter(fun n -> nameSet.Contains(Path.GetFileName(n))) | ||
| nameSet, files | ||
| | _ -> Configuration.supportedNames, Directory.GetFiles(absFolder,"test*.fs*") | ||
|
|
||
| let mainFile,otherFiles = | ||
| match files.Length with | ||
| | 0 -> Directory.GetFiles(absFolder,"*.ml") |> Array.exactlyOne, [||] | ||
| | 1 -> files |> Array.exactlyOne, [||] | ||
| | _ -> | ||
| let mainFile,dependencies = | ||
| files | ||
| |> Array.filter (fun n -> supportedNames.Contains(Path.GetFileName(n))) | ||
| // Convention in older FsharpSuite: test2 goes last, longer names like testlib before test, .fsi before .fs on equal filenames | ||
| |> Array.sortBy (fun n -> n.Contains("test2"), -n.IndexOf('.'), n.EndsWith(".fsi") |> not) | ||
| |> Array.splitAt 1 | ||
|
|
||
| mainFile[0],dependencies | ||
|
|
||
| let version,bonusArgs = adjustVersion langVersion bonusArgs | ||
|
|
||
| FsFromPath mainFile | ||
| |> withAdditionalSourceFiles [for f in otherFiles -> SourceFromPath f] | ||
| |> withLangVersion version | ||
| |> fun cu -> | ||
| match mode with | ||
| | FSC_DEBUG | FSC_OPTIMIZED | FSI | COMPILED_EXE_APP -> | ||
| cu | ||
| |> ignoreWarnings | ||
| |> withOptions (["--nowarn:0988;3370"] @ bonusArgs) | ||
| | NEG_TEST_BUILD _ -> | ||
| cu |> | ||
| withOptions (["--vserrors";"--maxerrors:10000";"--warnaserror";"--warn:3";"--nowarn:20;21;1178;52"] @ bonusArgs) | ||
| |> fun cu -> | ||
| match mode with | ||
| | FSC_DEBUG -> | ||
| cu | ||
| |> withDebug | ||
| |> withNoOptimize | ||
| |> ScriptRunner.runScriptFile langVersion | ||
| |> shouldSucceed | ||
| | FSC_OPTIMIZED -> | ||
| cu | ||
| |> withOptimize | ||
| |> withNoDebug | ||
| |> ScriptRunner.runScriptFile langVersion | ||
| |> shouldSucceed | ||
| | FSI -> | ||
| cu | ||
| |> ScriptRunner.runScriptFile langVersion | ||
| |> shouldSucceed | ||
| | COMPILED_EXE_APP -> | ||
| cu | ||
| |> withDefines ("TESTS_AS_APP" :: ScriptRunner.defaultDefines) | ||
| |> compileExeAndRun | ||
| |> shouldSucceed | ||
| | NEG_TEST_BUILD testName -> diffNegativeBaseline (cu |> withName mainFile) absFolder testName langVersion | ||
|
|
||
| |> ignore<CompilationResult> | ||
|
|
||
|
|
||
| let singleTestBuildAndRunAux folder bonusArgs mode = singleTestBuildAndRunAuxVersion folder bonusArgs mode LangVersion.Latest | ||
| let singleTestBuildAndRunVersion folder mode version = singleTestBuildAndRunAuxVersion folder [] mode version | ||
| let singleTestBuildAndRun folder mode = singleTestBuildAndRunVersion folder mode LangVersion.Latest | ||
|
|
||
| let singleVersionedNegTestAux folder bonusArgs version testName = | ||
| singleTestBuildAndRunAuxVersion folder bonusArgs (NEG_TEST_BUILD testName) version | ||
| let singleVersionedNegTest (folder:string) (version:LangVersion) (testName:string) = | ||
| singleVersionedNegTestAux folder [] version testName | ||
T-Gro marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| let singleNegTest folder testName = singleVersionedNegTest folder LangVersion.Latest testName | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.