"GetPackageDirectory" task failing unexpectedly

[LoopedBard3]

Information

cc @danmoseley
Linked to dotnet/performance#2564
Starting late 08/15, when building for "Runtime=NativeAOT 7.0" with "Toolchain=ILCompiler 7.0.0-*", our builds started hitting "The 'GetPackageDirectory' task failed unexpectedly". This is different from behavior using the same settings the day before. We have public logs demonstrating the error: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-performance-refs-pull-2549-merge-ada34367314d40efb9/Partition0/1/console.e9e4522d.log?helixlogtype=result and demonstrating a successful build https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-performance-refs-heads-main-0f44758c7ce64b77aa/Partition1/1/console.7e903502.log?helixlogtype=result. Looking around I noticed the task area was updated around the same time the errors started showing up in our builds (#27159). Could this potentially be related to those changes @LakshanF?

Exception/Failure

[2022/08/17 17:49:24][INFO] Build FAILED.
[2022/08/17 17:49:24][INFO] /home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/sdk/7.0.100-rc.2.22417.1/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018: The "GetPackageDirectory" task failed unexpectedly. [/home/helixbot/work/B17209D2/w/A819091D/e/artifacts/bin/MicroBenchmarks/Release/net7.0/a55d87f3-9864-4d0f-b7cf-0909db2b0ba6/BenchmarkDotNet.Autogenerated.csproj]
[2022/08/17 17:49:24][INFO] /home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/sdk/7.0.100-rc.2.22417.1/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018: System.ArgumentException: '7.0.0-*' is not a valid version string. (Parameter 'value') [/home/helixbot/work/B17209D2/w/A819091D/e/artifacts/bin/MicroBenchmarks/Release/net7.0/a55d87f3-9864-4d0f-b7cf-0909db2b0ba6/BenchmarkDotNet.Autogenerated.csproj]
[2022/08/17 17:49:24][INFO] /home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/sdk/7.0.100-rc.2.22417.1/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at NuGet.Versioning.NuGetVersion.Parse(String value) [/home/helixbot/work/B17209D2/w/A819091D/e/artifacts/bin/MicroBenchmarks/Release/net7.0/a55d87f3-9864-4d0f-b7cf-0909db2b0ba6/BenchmarkDotNet.Autogenerated.csproj]
[2022/08/17 17:49:24][INFO] /home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/sdk/7.0.100-rc.2.22417.1/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GetPackageDirectory.ExecuteCore() [/home/helixbot/work/B17209D2/w/A819091D/e/artifacts/bin/MicroBenchmarks/Release/net7.0/a55d87f3-9864-4d0f-b7cf-0909db2b0ba6/BenchmarkDotNet.Autogenerated.csproj]
[2022/08/17 17:49:24][INFO] /home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/sdk/7.0.100-rc.2.22417.1/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() [/home/helixbot/work/B17209D2/w/A819091D/e/artifacts/bin/MicroBenchmarks/Release/net7.0/a55d87f3-9864-4d0f-b7cf-0909db2b0ba6/BenchmarkDotNet.Autogenerated.csproj]
[2022/08/17 17:49:24][INFO] /home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/sdk/7.0.100-rc.2.22417.1/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/home/helixbot/work/B17209D2/w/A819091D/e/artifacts/bin/MicroBenchmarks/Release/net7.0/a55d87f3-9864-4d0f-b7cf-0909db2b0ba6/BenchmarkDotNet.Autogenerated.csproj]
[2022/08/17 17:49:24][INFO] /home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/sdk/7.0.100-rc.2.22417.1/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/home/helixbot/work/B17209D2/w/A819091D/e/artifacts/bin/MicroBenchmarks/Release/net7.0/a55d87f3-9864-4d0f-b7cf-0909db2b0ba6/BenchmarkDotNet.Autogenerated.csproj]

Dotnet Info

[2022/08/17 17:48:11][INFO] $ dotnet --info
[2022/08/17 17:48:12][INFO] .NET SDK:
[2022/08/17 17:48:12][INFO]  Version:   7.0.100-rc.2.22417.1
[2022/08/17 17:48:12][INFO]  Commit:    f3da421a6e
[2022/08/17 17:48:12][INFO] 
[2022/08/17 17:48:12][INFO] Runtime Environment:
[2022/08/17 17:48:12][INFO]  OS Name:     centos
[2022/08/17 17:48:12][INFO]  OS Version:  7
[2022/08/17 17:48:12][INFO]  OS Platform: Linux
[2022/08/17 17:48:12][INFO]  RID:         centos.7-x64
[2022/08/17 17:48:12][INFO]  Base Path:   /home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/sdk/7.0.100-rc.2.22417.1/
[2022/08/17 17:48:12][INFO] 
[2022/08/17 17:48:12][INFO] Host:
[2022/08/17 17:48:12][INFO]   Version:      7.0.0-rc.1.22411.12
[2022/08/17 17:48:12][INFO]   Architecture: x64
[2022/08/17 17:48:12][INFO]   Commit:       f52d8c59bb
[2022/08/17 17:48:12][INFO] 
[2022/08/17 17:48:12][INFO] .NET SDKs installed:
[2022/08/17 17:48:12][INFO]   7.0.100-rc.2.22417.1 [/home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/sdk]
[2022/08/17 17:48:12][INFO] 
[2022/08/17 17:48:12][INFO] .NET runtimes installed:
[2022/08/17 17:48:12][INFO]   Microsoft.AspNetCore.App 7.0.0-rc.1.22415.4 [/home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/shared/Microsoft.AspNetCore.App]
[2022/08/17 17:48:12][INFO]   Microsoft.NETCore.App 7.0.0-rc.1.22411.12 [/home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64/shared/Microsoft.NETCore.App]
[2022/08/17 17:48:12][INFO] 
[2022/08/17 17:48:12][INFO] Other architectures found:
[2022/08/17 17:48:12][INFO]   None
[2022/08/17 17:48:12][INFO] 
[2022/08/17 17:48:12][INFO] Environment variables:
[2022/08/17 17:48:12][INFO]   DOTNET_ROOT       [/home/helixbot/work/B17209D2/w/A819091D/e/tools/dotnet/x64]
[2022/08/17 17:48:12][INFO] 
[2022/08/17 17:48:12][INFO] global.json file:
[2022/08/17 17:48:12][INFO]   Not found

[danmoseley]

Whatever the bug, GetPackageDirectory.ExecuteCore should catch any ArgumentException and return an error.

[adamsitnik]

I can see that this particular task has not been changed since 2019:

sdk/src/Tasks/Microsoft.NET.Build.Tasks/GetPackageDirectory.cs

Line 57 in bfc7bce

var parsedPackageVersion = NuGetVersion.Parse(packageVersion);

[adamsitnik]

Repro:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.DotNet.ILCompiler" Version="7.0.0-*" />
  </ItemGroup>
</Project>

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <solution>
    <add key="disableSourceControlIntegration" value="true" />
  </solution>
  <packageSources>
    <clear />
    <add key="dotnet-public" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-public/nuget/v3/index.json" />
    <add key="dotnet7" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet7/nuget/v3/index.json" />
    <add key="dotnet6" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet6/nuget/v3/index.json" />
    <add key="dotnet5" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5/nuget/v3/index.json" />
    <add key="dotnet5-transport" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet5-transport/nuget/v3/index.json" />
    <add key="dotnet3.1" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1/nuget/v3/index.json" />
    <add key="dotnet3.1-transport" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet3.1-transport/nuget/v3/index.json" />
    <add key="dotnet-eng" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-eng/nuget/v3/index.json" />
    <add key="dotnet-tools" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet-tools/nuget/v3/index.json" />
    <add key="benchmark-dotnet-prerelease" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/benchmark-dotnet-prerelease/nuget/v3/index.json" />
  </packageSources>
  <disabledPackageSources />
</configuration>

Console.WriteLine("Hello, World!");

dotnet publish -c Release -r win-x64

D:\projects\forks\perf_date\tools\dotnet\x64\sdk\7.0.100-rc.2.22417.1\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka
.ms/dotnet-support-policy [D:\projects\repros\aotBuild\aotBuild.csproj]
D:\projects\forks\perf_date\tools\dotnet\x64\sdk\7.0.100-rc.2.22417.1\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018: The "GetPackageDirectory" task failed unexpectedly. [D:\
projects\repros\aotBuild\aotBuild.csproj]
D:\projects\forks\perf_date\tools\dotnet\x64\sdk\7.0.100-rc.2.22417.1\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018: System.ArgumentException: '7.0.0-*' is not a valid versi
on string. (Parameter 'value') [D:\projects\repros\aotBuild\aotBuild.csproj]
D:\projects\forks\perf_date\tools\dotnet\x64\sdk\7.0.100-rc.2.22417.1\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at NuGet.Versioning.NuGetVersion.Parse(String value)
[D:\projects\repros\aotBuild\aotBuild.csproj]
D:\projects\forks\perf_date\tools\dotnet\x64\sdk\7.0.100-rc.2.22417.1\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GetPackageDirectory.Exec
uteCore() [D:\projects\repros\aotBuild\aotBuild.csproj]
D:\projects\forks\perf_date\tools\dotnet\x64\sdk\7.0.100-rc.2.22417.1\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() [D:\p
rojects\repros\aotBuild\aotBuild.csproj]
D:\projects\forks\perf_date\tools\dotnet\x64\sdk\7.0.100-rc.2.22417.1\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsof
t.Build.BackEnd.ITaskExecutionHost.Execute() [D:\projects\repros\aotBuild\aotBuild.csproj]
D:\projects\forks\perf_date\tools\dotnet\x64\sdk\7.0.100-rc.2.22417.1\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(255,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstant
iatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [D:\projects\repros\aotBuild\aotBuild.csproj]

cc @MichalStrehovsky @jkotas who might start getting similar bug reports as it's quite common to reference ILCompiler in this way (at least that is my impression)

[MichalStrehovsky]

I can repro this with the latest RC2 SDK. It didn't repro with Preview 7. I'm putting this in the 7.0 milestone. Weird the unit test in this repo doesn't catch it - it uses the same version string. @LakshanF can you have a look?

[baronfel]

I think @LakshanF is going to need to look at this. The problem seems to be in this clause from the ILCompiler targets (which seem to be brought in from the PackageReference):

<ItemGroup>
    <!-- If called via package instead of the SDK, update the runtime package version to match the build package -->
    <_PackageReferenceExceptILCompiler Include="@(PackageReference)" Exclude="Microsoft.DotNet.ILCompiler" />
    <_ILCompilerPackageReference Include="@(PackageReference)" Exclude="@(_PackageReferenceExceptILCompiler)" />
    <KnownILCompilerPack Update="Microsoft.DotNet.ILCompiler" Condition="@(_ILCompilerPackageReference->'%(Identity)')=='Microsoft.DotNet.ILCompiler'">
        <ILCompilerPackVersion>@(_ILCompilerPackageReference->'%(Version)')</ILCompilerPackVersion>
    </KnownILCompilerPack>
  </ItemGroup>

specifically on the line that sets the ILCompilerPackVersion metadata. When a user sets a PackageReference with a floating version, this version is written wholesale, but the ILCompilerPackVersion is expected to be a resolved version. That's what jumps out to me on a quick glance.

[LakshanF]

Oops, I missed the notifications.

Thanks @baronfel on your insights above, and yes, that is the part that causes a problem. On a related note, we want to discourage users specifying explicitly the package reference and have a PR, dotnet/runtime#74591, that generates a warning for such cases.

For this case where the user has a floating version, I will look for a fix that gets the specific one.

[LakshanF]

The fix is in the runtime repo, dotnet/runtime#74591

[LoopedBard3]

We are no longer hitting the issue with the fix in, closing! 🥳