[Xamarin.Android-Tests] Split out "full" apps

[jonpryor]

We're trying to get make jenkins working on Jenkins, and
it's failing, as one might expect when a particular repo and
associated build system has never been run on Jenkins before:

Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference?

This error occurs while building
src/Mono.Android/Test/Mono.Android-Tests.csproj, and happens because
the Xamarin.Android.NUnitLite.dll assembly isn't referenced...
because it isn't found:

Microsoft.Common.targets: warning : Reference 'Xamarin.Android.NUnitLite' not resolved

This returns us to a long-standing issue which I thought was mentioned
in a commit somewhere, but I can't find at present:

MSBuild doesn't support updating the assembly resolution directories
while MSBuild is running.

For example, we build Xamarin.Android.NUniteLite.dll into
bin/$(Configuration/lib/xbuild-frameworks/MonoAndroid/v1.0,
but "normal Xamarin.Android referencing projects" don't use a
@(ProjectReference) to Xamarin.Android.NUnitLite.csproj, but
instead a @(Reference) to the assembly:

<!-- src/Mono.Android/Test/Mono.Android-Tests.csproj -->
<Reference Include="Xamarin.Android.NUnitLite" />

The requirement for a "proper" Xamarin.Android SDK install is that
Xamarin.Android.NUnitLite.dll can be found through a normal
@(Reference). In order to satisfy this requirement, we need to tell
MSBuild where to find it, which can be with xbuild via the
$MSBuildExtensionsPath and $XBUILD_FRAMEWORK_FOLDERS_PATH
environment variables.

Environment variables.

MSBuild doesn't provide a way to update environment variables, short
of writing a new Task which calls
Environment.SetEnvironmentVariable(), and while this works, it
doesn't actually work ¹.

The short of all this is that it isn't possible,
within a single xbuild invocation, to both build the
Xamarin.Android SDK "environment" and use that environment as
intended for "normal" apps.

The fix, as is often the case, is to bend with the wind. Instead of
requiring the impossible, move
src/Mono.Android/Test/Mono.Android-Tests.csproj into a new
Xamarin.Android-Tests.sln solution, out of Xamarin.Android.sln.
This allows building Xamarin.Android.sln without error in a pristine
environment -- that is, one that doesn't already have a system-wide
Xamarin.Android install -- and separately building the tests by using
tools/scripts/xabuild, which can export environment variables to
manipulate xbuild behavior so that things Just Work™ ².

Building Mono.Android-Tests.csproj and similar projects
(HelloWorld.csproj!) can be done by using the new make all-tests
make target.

The idea of PR #147 was to "hack things up" so that
`Xamarin.Android.NuniteLite.dll` would be properly resolved during
the build of `Mono.Android-Tests.csproj` *when building everything*.

The problem is *this can't work*, because `xbuild` has a
"target framework cache," with no way to invalidate it, and the
cache is populated the first time the target framework is used.
Due to build ordering issues, this first use is *before*
`Xamarin.Android.NunitLite.dll` was built, and thus it doesn't
exist in the cache. The result:

    Task "ResolveAssemblyReference"
        ....
        TargetFrameworkDirectories:
            /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0
            /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0
            /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/
        ...
        Primary Reference System.Xml
            Reference System.Xml resolved to /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/System.Xml.dll. CopyLocal = False
        ...
        Primary Reference Xamarin.Android.NUnitLite
    Microsoft.Common.targets:  warning : Reference 'Xamarin.Android.NUnitLite' not resolved
        For searchpath {TargetFrameworkDirectory}
        Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v7.0, assembly named 'Xamarin.Android.NUnitLite' not found.
        Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0, assembly named 'Xamarin.Android.NUnitLite' not found.
        Considered target framework dir /Volumes/Seagate4TB/work/xamarin-android/bin/Debug/lib/xbuild-frameworks/MonoAndroid/v1.0/Facades/, assembly named 'Xamarin.Android.NUnitLite' not found.
    ...

Consequently, the `mcs` invocation is missing a
`/reference:path/to/Xamarin.Android.NUniteLite.dll`, and
compilation fails:

    Android.App/ApplicationTest.cs(9,7): error CS0246: The type or namespace name `NUnit' could not be found. Are you missing an assembly reference?
    ...plus 29 others...

    error XA4210: You need to add a reference to Mono.Android.Export.dll when you use ExportAttribute or ExportFieldAttribute

Please enter

Footnotes

1. PR #147 isn't viable because of xbuild's AsssemblyResolver.
   There's no way to clear/invalidate target_framework_cache. ↩

2. ...except they don't just work, due to other bugs, but those
   will be separately addressed. ↩

[dellis1972]

@jonpryor

Interestingly

This works when using 'msbuild' on mac...

<UsingTask
  TaskName="SetEnvironmentVariableTask"
  TaskFactory="CodeTaskFactory"
  AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">

  <ParameterGroup>
    <Name ParameterType="System.String" Required="true" />
    <Value ParameterType="System.String" Required="true" />
  </ParameterGroup>

  <Task>
    <Using Namespace="System" />
    <Code Type="Fragment" Language="cs">
      <![CDATA[
        Environment.SetEnvironmentVariable(Name, Value);
      ]]>
    </Code>
  </Task>

</UsingTask>

So could we switch over to using msbuild ... @radical ?

Other options

https://www.nuget.org/packages/MSBuild.SetEnvVariable/

There is a nuget which adds this functionality.

[dellis1972]

some sample code

https://gist.github.com/dellis1972/3579add8d6b7432aac1dc890a9e8b138

on Mac at least the Environment variable does update.

[jonpryor]

@dellis1972: The concern was that $MSBuildExtensionsPath is cached on first read, which doesn't appear to be the case in xbuild, and the MSBuild unit tests also call Environment.SetEnvironmentVariable(), so explicitly setting it might actually work...

Related conversation: https://xamarinhq.slack.com/archives/android/p1462986233006067

Above conversation is about PR #32, and how using the "normal"/"system" xbuild to build src/Mono.Android/Test results in aborts at startup due to a libmonodroid.so mismatch. The fix is to use tools/scripts/xabuild instead of xbuild, as that sets the MSBuildExtensionsPath environment variable...

And trying to set MSBuildExtensionsPath _as an MSBuild _property* doesn't work*. It's not a property.

[jonpryor]

This PR is possibly superseded by (a fleshed out and cleaned up version of) PR #147.