outerloop legs failing to detect `dotnet`

[carlossanlop]

Original issue was opened in dotnet/arcade: dotnet/arcade#11185

Many outerloop legs fail with this message when attempting to run unit tests:

./RunTests.sh: line 168: /tmp/helix/working/A6B808E3/p/dotnet: No such file or directory

The investigation found that this may have been caused by a runtime change that happened between these two runs (from 8/2 to 8/3):

pr/public/dotnet/runtime/refs/pull/73300/merge
pr/public/dotnet/runtime/refs/pull/73020/merge

Kusto Query

Jobs
| where QueueName == "osx.1200.amd64.open"
| extend Propz = parse_json(Properties)
| where Propz["DefinitionName"] == "runtime-libraries-coreclr outerloop"
| where Propz["System.PhaseName"] == "libraries_build_OSX_x64_Debug"
| order by Finished desc 

The main suspect PR is one that seeked to remove the host packages conditionally and was merged at a time between the above two PRs likely merge commits: https://github.com/dotnet/runtime/pull/73095/files

We need to confirm this issue caused the regression and get it fixed.

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Original issue was opened in dotnet/arcade: dotnet/arcade#11185

Many outerloop legs fail with this message when attempting to run unit tests:

./RunTests.sh: line 168: /tmp/helix/working/A6B808E3/p/dotnet: No such file or directory

The investigation found that this may have been caused by a runtime change that happened between these two runs (from 8/2 to 8/3):

pr/public/dotnet/runtime/refs/pull/73300/merge
pr/public/dotnet/runtime/refs/pull/73020/merge

Kusto Query

Jobs
| where QueueName == "osx.1200.amd64.open"
| extend Propz = parse_json(Properties)
| where Propz["DefinitionName"] == "runtime-libraries-coreclr outerloop"
| where Propz["System.PhaseName"] == "libraries_build_OSX_x64_Debug"
| order by Finished desc 

The main suspect PR is one that seeked to remove the host packages conditionally and was merged at a time between the above two PRs likely merge commits: https://github.com/dotnet/runtime/pull/73095/files

We need to confirm this issue caused the regression and get it fixed.

Author:	carlossanlop
Assignees:	-
Labels:	area-Infrastructure
Milestone:	-

[elinor-fung]

#73095 made it so that the host.native subset needs to be built for the libraries testhost to be complete. It was added to the default clr/mono subsets, but it looks like the build isn't relying on those.

This is the portion of the change that handled it in PR/innerloop test runs:

runtime/eng/pipelines/libraries/run-test-job.yml

Lines 91 to 93 in cb6e5e7

	- ${{ if ne(parameters.liveRuntimeBuildConfig, '') }}:
	- script: $(_buildScript)
	-subset host.native+libs.pretest

It seems outerloop doesn't go through that. My guess would be that host.native is not being built, so host components (like dotnet) are missing from testhost. Whichever part of the outerloop pipeline is building libs.pretest probably needs to include host.native first.

[hoyosjs]

There was a PR I had a while back: #74468. Viktor said he wanted to do this differently, probably merging outer and inner loop as you point out.

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Original issue was opened in dotnet/arcade: dotnet/arcade#11185

Many outerloop legs fail with this message when attempting to run unit tests:

./RunTests.sh: line 168: /tmp/helix/working/A6B808E3/p/dotnet: No such file or directory

The investigation found that this may have been caused by a runtime change that happened between these two runs (from 8/2 to 8/3):

pr/public/dotnet/runtime/refs/pull/73300/merge
pr/public/dotnet/runtime/refs/pull/73020/merge

Kusto Query

Jobs
| where QueueName == "osx.1200.amd64.open"
| extend Propz = parse_json(Properties)
| where Propz["DefinitionName"] == "runtime-libraries-coreclr outerloop"
| where Propz["System.PhaseName"] == "libraries_build_OSX_x64_Debug"
| order by Finished desc 

The main suspect PR is one that seeked to remove the host packages conditionally and was merged at a time between the above two PRs likely merge commits: https://github.com/dotnet/runtime/pull/73095/files

We need to confirm this issue caused the regression and get it fixed.

Author:	carlossanlop
Assignees:	-
Labels:	area-Infrastructure-libraries, area-Infrastructure, untriaged
Milestone:	-