[tests/ci] Add MSBuild test jobs to Azure Pipelines

[pjcollins]

Adds new mac_msbuild_tests, mac_msbuilddevice_tests, and
mac_timezonedevice_tests jobs to azure-pipelines.yaml to run the
msbuild tests against our installers.

Other changes and fixes:

• Removes the dependency on the generated XABuildPaths.cs file so that
  test assemblies can be built and executed on different environments.

• Standardizes Android SDK and NDK path and OS information lookup to
  reduce some code duplication and better allow for toolchain path
  overriding.

• Updates Mono to 6.0.0.313 and .NET Core to 2.1.701 for our macOS
  building and test stages.

• Introduces a new UsesDevice test category to allow us to filter
  execution of tests which will be skipped or behave differently when
  a device or emulator is attached.

• Updates clean tests to ignore the FileListAbsolute.txt file now used
  by the clean target on macOS.

• Updates the ParallelScope attribute for some multi source tests to
  avoid NuGet restore related issues occurring on macOS. This should
  ensure that dynamically generated test case instances don't run in
  parallel with eachother, but still allows for parallel execution
  against other tests or fixtures. I believe that this is not occurring
  on Jenkins because our test NuGet dependencies have been cached for
  quite some time. On Azure Pipelines, we're more often dealing with
  clean machines, and parallel attempts to restore/install into global
  cache paths were occasionally failing as shown below:

  NuGet.targets(121,5): error : Could not find file "/Users/vsts/.local/share/NuGet/v3-cache/1ca707a4d90792ce8e42453d4e350886a0fdaa4d$ps:_api.nuget.org_v3_index.json/nupkg_sqlitepclraw.core.1.1.8.dat"
  NuGet.targets(121,5): error : The file '/Users/vsts/.nuget/packages/xamarin.android.support.compat/28.0.0.1/.nupkg.metadata' already exists.
  NuGet.targets(121,5): error : Could not find file "/Users/vsts/.nuget/packages/xamarin.forms/3.4.0.1008975/2omeck77.h7w"
  

• Attaches relevant build log file paths to the test result xml rather
  than dumping the full log to the test output. It's a bit painful to
  scroll through diagnostic build output to see which tests failed.
  Build logs can now be viewed directly in the 'attachments' tab of
  each test failure in Azure Pipelines:

[pjcollins]

I'm not yet entirely sold on us needing to be able to build the msbuild test assemblies (Xamarin.Android.Build.Tests.dll, Xamarin.Android.Build.Tests.Commercial.dll, and MSBuildDeviceIntegration.dll) against a system installation. I may rework this to instead upload these assemblies during the mac build stage, and download and run them in later stages.

[dellis1972]

@pjcollins given my recent experience with azure pipelines, I think uploading the unit test binaries and downloading them for the test phase would make sense. If we have already built them.. might as well use them :)

[pjcollins]

@jonathanpeppers @dellis1972 do we need to build the Xamarin.Android.Build.Tests suite in both Debug and Release and run it twice for full coverage? Or is the project configuration used for each test set on the test level, and independent of the configuration of the build of the actual test assembly? On a quick search I don't see us using /p:Configuration anywhere in the test harness.

[jonathanpeppers]

I don't think the Configuration of the Xamarin.Android.Build.Tests project matters, and we shouldn't need to run it twice.

There is a XABuildPaths.cs file in Xamarin.ProjectTools that has:

using System;
using System.IO;

namespace Xamarin.Android.Build
{
	public static class Paths
	{
		public const string Configuration = "Debug";
		public const string TopDirectory = @"C:\src\xamarin-android";

		public static readonly string PrefixDirectory = Path.Combine (TopDirectory, "bin", Configuration);
		public static readonly string BinDirectory = Path.Combine (PrefixDirectory, "bin");
		public static readonly string XABuildScript = Path.Combine (BinDirectory, "xabuild");
		public static readonly string XABuildExe = Path.Combine (BinDirectory, "xabuild.exe");
		public static readonly string TestOutputDirectory = Path.Combine (TopDirectory, "bin", "TestDebug");
	}
}

I think xaprepare generates this file now, and it will say Debug/Release here depending on the configuration used with make prepare + make all.

[pjcollins]

I've reworked this to upload and use previously built MSBuild test assemblies. This should now be ready for review and merge once the new stages are green.

[dellis1972]

@pjcollins this seems to be working :) Device tests are running and passing 🏆 :D nice job

[dellis1972]

The Jenkins MacOS failures seem to be the same as all the other PR's

[pjcollins]

@dellis1972 I've noticed we still seem to have a few brittle tests, the following have been failing on Jenkins:

• Xamarin.Android.Build.Tests.IncrementalBuildTest.InvalidAndroidResource
• Xamarin.Android.Build.Tests.BuildTest.BuildBasicApplicationAppCompat
• Xamarin.Android.Build.Tests.BuildTest.BuildWithResolveAssembliesFailure

I saw similar failures on Azure Pipelines but this change to stop running them in parallel seemed to reduce or eliminate the failure rate for those tests on Azure. Unfortunately that change did not fix the Jenkins failures though.

I think we should be good to merge this as is, it doesn't introduce any new failures to Jenkins. We may still have some individual test clean up to do but given the size of this thing I would rather get it in sooner than later and look to clean up any tests which are still misbehaving separately.