Skip to content

Conversation

@pjcollins
Copy link
Member

@pjcollins pjcollins commented Mar 3, 2021

A new Linux Build + Smoke test job has been added to produce a
"commercial" .NET 6 Linux SDK .nupkg. This job takes advantage of a new
Azure Virtual Machine Scale Set pool that has been created
specifically for this job. This VMSS can be found in the Azure portal
under the name xaubuntu20.04-vmss.

This job is configured to run the PackagingTest.CheckSignApk tests, a
subset of the broader smoke test category. More tests should be added
to this job in the future as needed. The execution of these tests on
this new build pool exposed a test framework issue that occurs on
environments that do not export $JAVA_HOME, or have a JDK installation
in any default look up paths:

Failed CheckSignApk(True,False) [27 ms]
  Error Message:
   System.InvalidOperationException : Could not determine Java SDK location. Please provide `javaSdkPath`.
  Stack Trace:
     at Xamarin.Android.Tools.AndroidSdkInfo..ctor(Action`2 logger, String androidSdkPath, String androidNdkPath, String javaSdkPath) in /agent/_work/1/s/xamarin-android/external/xamarin-android-tools/src/Xamarin.Android.Tools.AndroidSdk/AndroidSdkInfo.cs:line 25
   at Xamarin.Android.Build.Tests.PackagingTest.CheckSignApk(Boolean useApkSigner, Boolean perAbiApk) in /agent/_work/1/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingTest.cs:line 366

To avoid this, we can pass the JDK path used by the tests when creating
new instances of AndroidSdkInfo. In some of these cases AndroidSdkInfo
was only being used to calculate a JDK path for a tool that a test
needed, and in these cases we can use AndroidSdkResolver.GetJavaSdkPath
directly.

@pjcollins pjcollins force-pushed the pack-linux branch 4 times, most recently from 7fd0a6a to fb6222b Compare March 3, 2021 18:27
@pjcollins
Copy link
Member Author

Of the 19 remaining MSBuild smoke test failures, the AppBundle tests are failing to build with:


"/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/CheckAppBundleTrue/UnnamedProject.csproj" (Build;SignAndroidPackage target) (1:7) ->
(_BuildApkEmbed target) -> 
    /home/AzDevOps/android-toolchain/dotnet/packs/Microsoft.Android.Sdk.linux-x64/11.0.200-ci.pr.gh5681.137/tools/Xamarin.Android.Common.targets(2072,3): error XABAB7028: System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=12.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'. The system cannot find the file specified. [/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/CheckAppBundleTrue/UnnamedProject.csproj]
/home/AzDevOps/android-toolchain/dotnet/packs/Microsoft.Android.Sdk.linux-x64/11.0.200-ci.pr.gh5681.137/tools/Xamarin.Android.Common.targets(2072,3): error XABAB7028:  [/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/CheckAppBundleTrue/UnnamedProject.csproj]
/home/AzDevOps/android-toolchain/dotnet/packs/Microsoft.Android.Sdk.linux-x64/11.0.200-ci.pr.gh5681.137/tools/Xamarin.Android.Common.targets(2072,3): error XABAB7028: File name: 'Newtonsoft.Json, Version=12.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' [/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/CheckAppBundleTrue/UnnamedProject.csproj]
/home/AzDevOps/android-toolchain/dotnet/packs/Microsoft.Android.Sdk.linux-x64/11.0.200-ci.pr.gh5681.137/tools/Xamarin.Android.Common.targets(2072,3): error XABAB7028:    at Xamarin.Android.Tasks.BuildAppBundle.RunTask() [/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/CheckAppBundleTrue/UnnamedProject.csproj]
/home/AzDevOps/android-toolchain/dotnet/packs/Microsoft.Android.Sdk.linux-x64/11.0.200-ci.pr.gh5681.137/tools/Xamarin.Android.Common.targets(2072,3): error XABAB7028:    at Microsoft.Android.Build.Tasks.AndroidToolTask.Execute() in /agent/_work/1/s/xamarin-android/external/xamarin-android-tools/src/Microsoft.Android.Build.BaseTasks/AndroidToolTask.cs:line 15 [/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/CheckAppBundleTrue/UnnamedProject.csproj]

While the rest fail with:

"/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/BuildWithNativeLibraries_True/UnnamedProject/UnnamedProject.csproj" (Build;SignAndroidPackage target) (1:7) ->
(_AddAndroidCustomMetaData target) -> 
    /home/AzDevOps/android-toolchain/dotnet/packs/Microsoft.Android.Sdk.linux-x64/11.0.200-ci.pr.gh5681.137/tools/Xamarin.Android.Common.targets(414,3): error XAFLT7000: System.MissingMethodException: Method not found: 'System.Collections.Immutable.ImmutableArray`1<System.Reflection.Metadata.CustomAttributeTypedArgument`1<!0>> System.Reflection.Metadata.CustomAttributeValue`1.get_FixedArguments()'. [/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/BuildWithNativeLibraries_True/UnnamedProject/UnnamedProject.csproj]
/home/AzDevOps/android-toolchain/dotnet/packs/Microsoft.Android.Sdk.linux-x64/11.0.200-ci.pr.gh5681.137/tools/Xamarin.Android.Common.targets(414,3): error XAFLT7000:    at Xamarin.Android.Tasks.FilterAssemblies.GetTargetFrameworkIdentifier(AssemblyDefinition assembly, MetadataReader reader) [/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/BuildWithNativeLibraries_True/UnnamedProject/UnnamedProject.csproj]
/home/AzDevOps/android-toolchain/dotnet/packs/Microsoft.Android.Sdk.linux-x64/11.0.200-ci.pr.gh5681.137/tools/Xamarin.Android.Common.targets(414,3): error XAFLT7000:    at Xamarin.Android.Tasks.FilterAssemblies.RunTask() in /agent/_work/1/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/FilterAssemblies.cs:line 61 [/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/BuildWithNativeLibraries_True/UnnamedProject/UnnamedProject.csproj]
/home/AzDevOps/android-toolchain/dotnet/packs/Microsoft.Android.Sdk.linux-x64/11.0.200-ci.pr.gh5681.137/tools/Xamarin.Android.Common.targets(414,3): error XAFLT7000:    at Microsoft.Android.Build.Tasks.AndroidTask.Execute() in /agent/_work/1/s/xamarin-android/external/xamarin-android-tools/src/Microsoft.Android.Build.BaseTasks/AndroidTask.cs:line 17 [/agent/_work/1/s/xamarin-android/bin/TestRelease/temp/BuildWithNativeLibraries_True/UnnamedProject/UnnamedProject.csproj]

Context: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=4518058&view=logs&j=8100f0f7-e86c-5b6e-6a78-6808e1478ee9&t=b2ff00a9-cb72-5405-b714-ea139eccf107&l=482

    Failed CheckSignApk(True,False) [27 ms]
      Error Message:
       System.InvalidOperationException : Could not determine Java SDK location. Please provide `javaSdkPath`.
      Stack Trace:
         at Xamarin.Android.Tools.AndroidSdkInfo..ctor(Action`2 logger, String androidSdkPath, String androidNdkPath, String javaSdkPath) in /agent/_work/1/s/xamarin-android/external/xamarin-android-tools/src/Xamarin.Android.Tools.AndroidSdk/AndroidSdkInfo.cs:line 25
       at Xamarin.Android.Build.Tests.PackagingTest.CheckSignApk(Boolean useApkSigner, Boolean perAbiApk) in /agent/_work/1/s/xamarin-android/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingTest.cs:line 366

The new Ubuntu test machines do not have JAVA_HOME set (or any other
fallback mechanism for locating the JDK).  When creating a new instance
of AndroidSdkInfo we should pass the JDK value used by the tests.  In
other cases, we can avoid creating an AndroidSdkInfo object entirely as
it was only being used to get a JDK path.
@pjcollins
Copy link
Member Author

I've cut smoke test scope in the latest force push to hopefully get this green. This gives us the option to investigate and fix various Linux issues at a later date.

@pjcollins
Copy link
Member Author

I've got a "commercial" Linux build and a few tests working here. I'm wondering if it's worth getting this in now and investigating and fixing test issues in future PRs, or if we want to try to get those fixed as part of this PR to increase test coverage before landing this build?

@grendello
Copy link
Contributor

I've got a "commercial" Linux build and a few tests working here. I'm wondering if it's worth getting this in now and investigating and fixing test issues in future PRs, or if we want to try to get those fixed as part of this PR to increase test coverage before landing this build?

I think it's better to delay the fixes till some point in the future, and commit the PR as it is now.

@pjcollins pjcollins marked this pull request as ready for review March 5, 2021 19:02
@jonpryor
Copy link
Contributor

jonpryor commented Mar 5, 2021

/azp run

@jonpryor
Copy link
Contributor

jonpryor commented Mar 5, 2021

Not sure why, but the upload test assemblies step failed repeatedly. Restarted the build.

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Base automatically changed from master to main March 5, 2021 23:08
@pjcollins
Copy link
Member Author

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@pjcollins
Copy link
Member Author

I've updated the build pool to use a new VMSS named xaubuntu20.04-vmss in our new Azure subscription, this should be ready to merge.

@jonpryor jonpryor merged commit c6206a2 into main Mar 12, 2021
@jonpryor jonpryor deleted the pack-linux branch March 12, 2021 01:26
pjcollins added a commit that referenced this pull request Jan 5, 2022
Bumps to mono/debugger-libs@c53676b
Changes: mono/debugger-libs@08268b7...c53676b

Many instances of `msbuild` have been replaced with `dotnet build` in
our make files.  The `-m:1` flag has been added to `Xamarin.Android.sln`
builds to disable parallelization.  We still run into issues running
`dotnet build` on the .sln when this flag is not used.  Additionally,
`msbuild` is still required when building `xabuild.csproj` and
`create-vsix.csproj`.

The `make jenkins` target has been updated to no longer create an OSS
.zip file to improve build speeds.  OSS installers can be built
separately as needed.

The `Xamarin.Android.Build.Tasks` build has been updated to run ILRepack
on Linux to fix [various issues][0].

A new `MonoUnixNative.targets` file has been added that can be used by
projects that produce an assembly containing MSBuild tasks that depend
on Mono.Unix.  The `<_MakeMonoUnixFatBinariesOSX/>` target will create
a multiarchitecture version of `libMono.Unix.dylib` in the projects
output directory when running on macOS.  It will also copy
`runtimes\linux-x64\native\libMono.Unix.so` and
`runtimes\osx-x64\native\libMono.Unix.dylib` to the output directory
when running on Windows or Linux.

[0]: #5681 (comment)
@github-actions github-actions bot locked and limited conversation to collaborators Jan 25, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants