Skip to content

[wasm][debugger] Disable failing multithreading debugger tests #86501

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jun 6, 2023
Merged

[wasm][debugger] Disable failing multithreading debugger tests #86501

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
52c9c97
Block for multithreading.
ilonatommy May 19, 2023
d2c258d
Merge branch 'main' into block-failures-86496
ilonatommy May 22, 2023
d6f4259
Another failing test.
ilonatommy May 22, 2023
81c57ab
category - case matters.
ilonatommy May 22, 2023
8dfddd5
Cleanup.
ilonatommy May 22, 2023
0e8033e
"Result reported for unknown test case" fix. Cannot be lazy.
ilonatommy May 23, 2023
b7c8dcf
Debugger has its own treading feature discovery.
ilonatommy May 23, 2023
7c05ccc
Prevent trait filter from being overwritten.
ilonatommy May 23, 2023
0005c5a
Applied @radical's review, reverted `TEST_FILTER` removal with change…
ilonatommy May 31, 2023
fefa7e4
Makefiles - full revert, fix should be in msbuild.
ilonatommy May 31, 2023
2014c06
Merge branch 'main' into block-failures-86496
ilonatommy May 31, 2023
0c44e64
Failing frequently.
ilonatommy May 31, 2023
0243222
Fleaky tests.
ilonatommy Jun 1, 2023
c181a89
Now this one failed - block.
ilonatommy Jun 2, 2023
2e008d4
10 min still timeouts, set to 15.
ilonatommy Jun 5, 2023
fc569d1
Revert "10 min still timeouts, set to 15."
ilonatommy Jun 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/mono/wasi/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,9 +104,9 @@ submit-tests-helix:
run-debugger-tests:
rm -f $(TOP)/artifacts/bin/DebuggerTestSuite/x64/Debug/*log; \
if [ ! -z "$(TEST_FILTER)" ]; then \
$(DOTNET) test $(TOP)/src/mono/wasm/debugger/DebuggerTestSuite $(MSBUILD_ARGS) "-l:trx;LogFileName=DebuggerTestsResults.xml" --results-directory $(TOP)/artifacts/log/$(CONFIG) --filter "Category!=failing&FullyQualifiedName$(TEST_FILTER)" $(TEST_ARGS); \
$(DOTNET) test $(TOP)/src/mono/wasm/debugger/DebuggerTestSuite $(MSBUILD_ARGS) "-l:trx;LogFileName=DebuggerTestsResults.xml" --results-directory $(TOP)/artifacts/log/$(CONFIG) --filter "category!=failing&FullyQualifiedName$(TEST_FILTER)" $(TEST_ARGS); \
else \
$(DOTNET) test $(TOP)/src/mono/wasm/debugger/DebuggerTestSuite $(MSBUILD_ARGS) "-l:trx;LogFileName=DebuggerTestsResults.xml" --results-directory $(TOP)/artifacts/log/$(CONFIG) --filter "Category!=failing" $(TEST_ARGS); \
$(DOTNET) test $(TOP)/src/mono/wasm/debugger/DebuggerTestSuite $(MSBUILD_ARGS) "-l:trx;LogFileName=DebuggerTestsResults.xml" --results-directory $(TOP)/artifacts/log/$(CONFIG) --filter "category!=failing" $(TEST_ARGS); \
fi

build-dbg-proxy:
Expand Down
4 changes: 2 additions & 2 deletions src/mono/wasm/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,9 +148,9 @@ submit-tests-helix:
run-debugger-tests:
rm -f $(TOP)/artifacts/bin/DebuggerTestSuite/x64/Debug/*log; \
if [ ! -z "$(TEST_FILTER)" ]; then \
$(DOTNET) test $(TOP)/src/mono/wasm/debugger/DebuggerTestSuite $(MSBUILD_ARGS) "-l:trx;LogFileName=DebuggerTestsResults.xml" --results-directory $(TOP)/artifacts/log/$(CONFIG) --filter "Category!=failing&FullyQualifiedName$(TEST_FILTER)" $(TEST_ARGS); \
$(DOTNET) test $(TOP)/src/mono/wasm/debugger/DebuggerTestSuite $(MSBUILD_ARGS) "-l:trx;LogFileName=DebuggerTestsResults.xml" --results-directory $(TOP)/artifacts/log/$(CONFIG) --filter "category!=failing&FullyQualifiedName$(TEST_FILTER)" $(TEST_ARGS); \
else \
$(DOTNET) test $(TOP)/src/mono/wasm/debugger/DebuggerTestSuite $(MSBUILD_ARGS) "-l:trx;LogFileName=DebuggerTestsResults.xml" --results-directory $(TOP)/artifacts/log/$(CONFIG) --filter "Category!=failing" $(TEST_ARGS); \
$(DOTNET) test $(TOP)/src/mono/wasm/debugger/DebuggerTestSuite $(MSBUILD_ARGS) "-l:trx;LogFileName=DebuggerTestsResults.xml" --results-directory $(TOP)/artifacts/log/$(CONFIG) --filter "category!=failing" $(TEST_ARGS); \
fi

build-dbg-proxy:
Expand Down
1 change: 1 addition & 0 deletions src/mono/wasm/debugger/DebuggerTestSuite/AssignmentTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ public AssignmentTests(ITestOutputHelper testOutput)

[ConditionalTheory(nameof(RunningOnChrome))]
[MemberData("GetTestData")]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task InspectVariableBeforeAndAfterAssignment(string clazz, JObject checkDefault, JObject checkValue, string methodName)
{
await SetBreakpointInMethod("debugger-test", "DebuggerTests." + clazz, "Prepare", 2);
Expand Down
3 changes: 2 additions & 1 deletion src/mono/wasm/debugger/DebuggerTestSuite/AsyncTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ public async Task AsyncLocalsInContinueWith(string method_name, string expected_
});

[ConditionalFact(nameof(RunningOnChrome))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task AsyncLocalsInContinueWithInstanceUsingThisBlock() => await CheckInspectLocalsAtBreakpointSite(
"DebuggerTests.AsyncTests.ContinueWithTests", "ContinueWithInstanceUsingThisAsync", 5, "DebuggerTests.AsyncTests.ContinueWithTests.ContinueWithInstanceUsingThisAsync.AnonymousMethod__6_0",
"window.setTimeout(function() { invoke_static_method('[debugger-test] DebuggerTests.AsyncTests.ContinueWithTests:RunAsync'); })",
Expand All @@ -66,6 +67,7 @@ public async Task AsyncLocalsInContinueWithInstanceUsingThisBlock() => await Che
});

[Fact] // NestedContinueWith
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task AsyncLocalsInNestedContinueWithStaticBlock() => await CheckInspectLocalsAtBreakpointSite(
"DebuggerTests.AsyncTests.ContinueWithTests", "NestedContinueWithStaticAsync", 5, "DebuggerTests.AsyncTests.ContinueWithTests.NestedContinueWithStaticAsync",
"window.setTimeout(function() { invoke_static_method('[debugger-test] DebuggerTests.AsyncTests.ContinueWithTests:RunAsync'); })",
Expand Down Expand Up @@ -140,7 +142,6 @@ await StepAndCheck(StepKind.Resume, source_to_pause, line2, col2, func_to_pause,
public async Task InspectLocalsInAsyncVBMethod()
{
var expression = $"{{ invoke_static_method('[debugger-test-vb] DebuggerTestVB.TestVbScope:Run'); }}";

await EvaluateAndCheck(
"window.setTimeout(function() {" + expression + "; }, 1);",
"dotnet://debugger-test-vb.dll/debugger-test-vb.vb", 14, 12,
Expand Down
2 changes: 2 additions & 0 deletions src/mono/wasm/debugger/DebuggerTestSuite/BreakpointTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,7 @@ await EvaluateAndCheck(
[MemberData(nameof(FalseConditions))]
[MemberData(nameof(TrueConditions))]
[MemberData(nameof(InvalidConditions))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task ConditionalBreakpoint2(string function_to_call, string method_to_stop, string condition, bool bp_stop_expected)
{
Result [] bps = new Result[2];
Expand Down Expand Up @@ -699,6 +700,7 @@ public async Task StepThroughOrNonUserCodeAttributeResumeWithBp(bool justMyCodeE
[InlineData(true, "Debugger.stepInto", "RunStepThroughWithNonUserCode", "RunStepThroughWithNonUserCode", -1, 8, "RunStepThroughWithNonUserCode", -1, 4)]
[InlineData(false, "Debugger.resume", "RunStepThroughWithNonUserCode", "StepThroughWithNonUserCodeUserBp", 927, 8, "RunStepThroughWithNonUserCode", -1, 4)]
[InlineData(true, "Debugger.resume", "RunStepThroughWithNonUserCode", "RunStepThroughWithNonUserCode", -1, 8, "RunStepThroughWithNonUserCode", -1, 4)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task StepThroughOrNonUserCodeAttributeWithUserBp(
bool justMyCodeEnabled, string debuggingFunction, string evalFunName,
string functionNameCheck1, int line1, int col1,
Expand Down
1 change: 1 addition & 0 deletions src/mono/wasm/debugger/DebuggerTestSuite/GetPropertiesTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,6 +260,7 @@ public GetPropertiesTests(ITestOutputHelper testOutput) : base(testOutput)
[MemberData(nameof(ClassGetPropertiesTestData), parameters: false)]
[MemberData(nameof(StructGetPropertiesTestData), parameters: true)]
[MemberData(nameof(StructGetPropertiesTestData), parameters: false)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task InspectTypeInheritedMembers(string type_name, bool? own_properties, bool? accessors_only, string[] expected_names, Dictionary<string, (JObject, bool)> all_props, bool is_async) =>
await InspectTypeInheritedMembersInternal(type_name, own_properties, accessors_only, expected_names, all_props, is_async, AutoEvaluate.Unset);

Expand Down
9 changes: 7 additions & 2 deletions src/mono/wasm/debugger/DebuggerTestSuite/SetNextIpTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ await StepAndCheck(StepKind.Over, "dotnet://debugger-test.dll/debugger-test.cs",
}

[ConditionalFact(nameof(RunningOnChrome))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task Lambda()
{
var debugger_test_loc = "dotnet://debugger-test.dll/debugger-async-test.cs";
Expand Down Expand Up @@ -159,6 +160,7 @@ await StepAndCheck(StepKind.Over, "dotnet://debugger-test.dll/debugger-async-tes
}

[ConditionalFact(nameof(RunningOnChrome))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task Lambda_InvalidLocation()
{
var debugger_test_loc = "dotnet://debugger-test.dll/debugger-async-test.cs";
Expand Down Expand Up @@ -188,6 +190,7 @@ await StepAndCheck(StepKind.Over, "dotnet://debugger-test.dll/debugger-async-tes
}

[ConditionalFact(nameof(RunningOnChrome))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task Lambda_ToNestedLambda()
{
var debugger_test_loc = "dotnet://debugger-test.dll/debugger-async-test.cs";
Expand All @@ -206,7 +209,7 @@ public async Task Lambda_ToNestedLambda()

await SetNextIPAndCheck(top_frame["scriptId"].Value<string>(), "dotnet://debugger-test.dll/debugger-async-test.cs", 88, 20, "DebuggerTests.AsyncTests.ContinueWithTests.NestedContinueWithInstanceAsync",
expected_error: true);

await StepAndCheck(StepKind.Over, "dotnet://debugger-test.dll/debugger-async-test.cs", 79, 16, "DebuggerTests.AsyncTests.ContinueWithTests.NestedContinueWithInstanceAsync",
locals_fn: async (locals) =>
{
Expand All @@ -218,6 +221,7 @@ await StepAndCheck(StepKind.Over, "dotnet://debugger-test.dll/debugger-async-tes
}

[ConditionalFact(nameof(RunningOnChrome))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task Lambda_ToNestedSingleLineLambda_Invalid()
{
var debugger_test_loc = "dotnet://debugger-test.dll/debugger-async-test.cs";
Expand Down Expand Up @@ -248,6 +252,7 @@ await StepAndCheck(StepKind.Over, "dotnet://debugger-test.dll/debugger-async-tes
}

[ConditionalFact(nameof(RunningOnChrome))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task Lambda_ToNestedSingleLineLambda_Valid()
{
var debugger_test_loc = "dotnet://debugger-test.dll/debugger-async-test.cs";
Expand All @@ -271,7 +276,7 @@ await SetNextIPAndCheck(top_frame["scriptId"].Value<string>(), "dotnet://debugge
await CheckValueType(locals, "code", "System.Threading.Tasks.TaskStatus", description: "Created");
await CheckValueType(locals, "dt0", "System.DateTime", description: "1/1/0001 12:00:00 AM");
});

await StepAndCheck(StepKind.Over, "dotnet://debugger-test.dll/debugger-async-test.cs", 92, 12, "DebuggerTests.AsyncTests.ContinueWithTests.NestedContinueWithInstanceAsync",
locals_fn: async (locals) =>
{
Expand Down
10 changes: 6 additions & 4 deletions src/mono/wasm/debugger/DebuggerTestSuite/SteppingTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -702,8 +702,8 @@ await EvaluateAndCheck(
await StepAndCheck(StepKind.Over, "dotnet://debugger-test.dll/debugger-async-step.cs", 15, 12, "MoveNext");
}

// [ConditionalFact(nameof(RunningOnChrome))]
//[ActiveIssue("https://github.com/dotnet/runtime/issues/42421")]
[ConditionalFact(nameof(RunningOnChrome))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/42421")]
public async Task StepOutOfAsyncMethod()
{
string source_file = "dotnet://debugger-test.dll/debugger-async-step.cs";
Expand Down Expand Up @@ -865,8 +865,8 @@ await EvaluateAndCheck(
method_name);
}

// [ConditionalTheory(nameof(RunningOnChrome))]
//[ActiveIssue("https://github.com/dotnet/runtime/issues/73867")]
[ConditionalTheory(nameof(RunningOnChrome))]
[ActiveIssue("https://github.com/dotnet/runtime/issues/73867")]
[InlineData(184, 20, 161, 8, "HiddenLinesContainingStartOfAnAsyncBlock")]
[InlineData(206, 20, 201, 8, "HiddenLinesAtTheEndOfANestedAsyncBlockWithWithLineDefaultOutsideTheMethod")]
[InlineData(224, 20, 220, 8, "HiddenLinesAtTheEndOfANestedAsyncBlockWithWithLineDefaultOutsideTheMethod2")]
Expand All @@ -885,6 +885,7 @@ await EvaluateAndCheck(
[InlineData(153, 20, 155, 16, "HiddenLinesAtTheEndOfANestedAsyncBlockWithNoLinesAtEndOfTheMethod.AnonymousMethod__1")]
[InlineData(154, 20, 155, 16, "HiddenLinesAtTheEndOfANestedAsyncBlockWithNoLinesAtEndOfTheMethod.AnonymousMethod__1")]
[InlineData(170, 20, 172, 16, "HiddenLinesAtTheEndOfANestedAsyncBlockWithBreakableLineAtEndOfTheMethod.AnonymousMethod__1")]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task BreakpointOnHiddenLineShouldStopAtEarliestNextAvailableLineAsync(int line_bp, int column_bp, int line_pause, int column_pause, string method_name)
{
await SetBreakpoint("dotnet://debugger-test.dll/debugger-async-test.cs", line_bp, column_bp);
Expand Down Expand Up @@ -1219,6 +1220,7 @@ await StepAndCheck(StepKind.Into, "dotnet://Newtonsoft.Json.dll/JArray.cs", 350,
[ConditionalTheory(nameof(RunningOnChrome))]
[InlineData(true)]
[InlineData(false)]
[ActiveIssue("https://github.com/dotnet/runtime/issues/86496", typeof(DebuggerTests), nameof(DebuggerTests.WasmMultiThreaded))]
public async Task SkipWasmFunctionsAccordinglyJustMyCode(bool justMyCode)
{
await SetJustMyCode(justMyCode);
Expand Down
4 changes: 2 additions & 2 deletions src/mono/wasm/debugger/Wasm.Debugger.Tests/wasm.helix.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,8 @@
<PayloadArchive>$(TestArchiveTestsDir)Wasm.Debugger.Tests.zip</PayloadArchive>
<Command>$(HelixCommand)</Command>
<Timeout>$(_DebuggerTestsWorkItemTimeout)</Timeout>
<PreCommands Condition="'$(OS)' == 'Windows_NT'">set &quot;TEST_ARGS=--filter FullyQualifiedName~%(Identity)&quot;</PreCommands>
<PreCommands Condition="'$(OS)' != 'Windows_NT'">export &quot;TEST_ARGS=--filter FullyQualifiedName~%(Identity)&quot;</PreCommands>
<PreCommands Condition="'$(OS)' == 'Windows_NT'">set &quot;TEST_ARGS=--filter category!=failing&amp;FullyQualifiedName~%(Identity)&quot;</PreCommands>
<PreCommands Condition="'$(OS)' != 'Windows_NT'">export &quot;TEST_ARGS=--filter category!=failing&amp;FullyQualifiedName~%(Identity)&quot;</PreCommands>
</HelixWorkItem>
</ItemGroup>
</Target>
Expand Down