Skip to content

Conversation

@jonathanpeppers
Copy link
Member

@jonathanpeppers jonathanpeppers commented Jul 17, 2018

Mono 2018-04 (or 5.14.x) has some changes to MSBuild assemblies that
cause the following failure when using xabuild.exe:

Mono: The following assembly referenced from monodroid/external/xamarin-android/bin/Debug/bin/Microsoft.Build.Tasks.Core.dll could not be loaded:
    Assembly:   System.Reflection.Metadata    (assemblyref_index=7)
    Version:    1.3.0.0
    Public Key: b03f5f7f11d50a3a

Currently xabuild.exe is referencing various MSBuild assemblies that
get copied into bin/$(Configuration)/bin and shipped in the OSS zip.

This isn't exactly desirable, since we don't really want to ship these
assemblies... These assemblies are OS-specific (as far as I can tell),
I noticed that these assemblies coming from a zip built on macOS do
not work on Windows.

xabuild.exe does not need to reference any MSBuild assemblies, and
we can use reflection + custom assembly loading to load MSBuild. This
keeps bin/Debug/bin free of additional assemblies, and now only
contains a few files:

  • xabuild
  • xabuild.exe
  • xabuild.exe.config
  • xabuild.pdb

By using the AppDomain.AssemblyResolve event, we can add some code
to probe the MSBuild bin directory for assemblies. If not found, we
can return null and let the runtime do its default behavior.

@jonathanpeppers jonathanpeppers added the do-not-merge PR should not be merged. label Jul 17, 2018
@jonathanpeppers jonathanpeppers requested a review from jonpryor July 17, 2018 22:21
@jonathanpeppers
Copy link
Member Author

cc @radical @luhenry still working on this, it worked on Windows so far... No idea how much will break yet.

@jonathanpeppers jonathanpeppers force-pushed the xabuild-assemblyresolve branch 2 times, most recently from f52dddf to 9b643d6 Compare July 18, 2018 13:17
@jonathanpeppers
Copy link
Member Author

@radical running into different issues on each platform now, but Windows seems worse to fix...

Windows (I think) is hitting an issue related to Xamarin.Forms build tasks being strong named:

[xabuild] assembly `Xamarin.Forms.Build.Tasks, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null` not found at path `C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin`, using default runtime behavior...
C:\Users\myuser\Desktop\Git\xamarin-android\packages\Xamarin.Forms.3.1.0.637273\build\netstandard2.0\Xamarin.Forms.targets(104,3): error MSB4061: The "XamlCTask" task could not be instantiated from "C:\Users\myuser\Desktop\Git\xamarin-android\packages\Xamarin.Forms.3.1.0.637273\build\netstandard2.0\Xamarin.Forms.Build.Tasks.dll". Could not load file or assembly 'Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified. [C:\Users\myuser\Desktop\Git\xamarin-android\tests\Xamarin.Forms-Performance-Integration\Droid\Xamarin.Forms.Performance.Integration.Droid.csproj]
C:\Users\myuser\Desktop\Git\xamarin-android\packages\Xamarin.Forms.3.1.0.637273\build\netstandard2.0\Xamarin.Forms.targets(104,3): error MSB4060: The "XamlCTask" task has been declared or used incorrectly, or failed during construction. Check the spelling of the task name and the assembly name. [C:\Users\myuser\Desktop\Git\xamarin-android\tests\Xamarin.Forms-Performance-Integration\Droid\Xamarin.Forms.Performance.Integration.Droid.csproj]

However, the missing assembly is loaded earlier successfully:

[xabuild] custom assembly resolution 'Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' -> 'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Microsoft.Build.dll'

Does MSBuild also have a custom AssemblyResolve when loading MSBuild tasks? It seems like it can't load Xamarin.Forms.Build.Tasks.dll.

jonathanpeppers added a commit to jonathanpeppers/xamarin-android that referenced this pull request Jul 18, 2018
Context: dotnet#1975

In newer versions of Mono, `xabuild` is failing with:

    /Library/Frameworks/Mono.framework/Versions/Current/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(2126,5): error MSB3248:
        Parameter "AssemblyFiles" has invalid value "xamarin-android/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll".
        Could not load file or assembly 'System.Reflection.Metadata, Version=1.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.

Or another example:

    Mono: The following assembly referenced from monodroid/external/xamarin-android/bin/Debug/bin/Microsoft.Build.Tasks.Core.dll could not be loaded:
        Assembly:   System.Reflection.Metadata    (assemblyref_index=7)
        Version:    1.3.0.0
        Public Key: b03f5f7f11d50a3a

Looking at copies of `System.Reflection.Metadata` in my Mono
installation:

    $ find /Library/Frameworks/Mono.framework/ | grep System.Reflection.Metadata.dll$
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/msbuild/15.0/bin/System.Reflection.Metadata.dll
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/msbuild/15.0/bin/Roslyn/System.Reflection.Metadata.dll
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/fsharp/System.Reflection.Metadata.dll
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/4.5/System.Reflection.Metadata.dll
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/4.5/dim/System.Reflection.Metadata.dll

It appears that the Mono version of MSBuild has its own copy of
`System.Reflection.Metadata.dll`. Referencing this MSBuild-specific
assembly from `xabuild.csproj` on non-Windows platforms appears to
solve the problem.
@radical
Copy link
Member

radical commented Jul 18, 2018

In VSMac we restrict the resolve hook to msbuild assemblies, see https://github.com/mono/monodevelop/blob/master/main/src/core/MonoDevelop.Projects.Formats.MSBuild/Main.cs#L111-L132 .

Also, maybe you want to try LoadFrom(path) like the VSMac code?

The "XamlCTask" task could not be instantiated from "C:\Users\myuser\Desktop\Git\xamarin-android\packages\Xamarin.Forms.3.1.0.637273\build\netstandard2.0\Xamarin.Forms.Build.Tasks.dll". Could not load file or assembly 'Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies

  • This is complaining about not being able to load Microsoft.Build 15.1, why did that fail?

@jonathanpeppers
Copy link
Member Author

@radical I think I know why it's not working 😖

image

Watching fuslogvw it appears a new MSBuild.exe gets spawned and it doesn't have my custom AssemblyResolve, and can't load Microsoft.Build.dll:

xamarin-android\packages\Xamarin.Forms.3.1.0.637273\build\netstandard2.0\Xamarin.Forms.targets(104,3):
    error MSB4061: The "XamlCTask" task could not be instantiated from "xamarin-android\packages\Xamarin.Forms.3.1.0.637273\build\netstandard2.0\Xamarin.Forms.Build.Tasks.dll". 
    Could not load file or assembly 'Microsoft.Build, Version=15.1.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

@jonathanpeppers jonathanpeppers force-pushed the xabuild-assemblyresolve branch from 9b643d6 to 7eabb49 Compare July 18, 2018 20:44
Mono 2018-04 (or 5.14.x) has some changes to MSBuild assemblies that
cause the following failure when using `xabuild.exe`:

    Mono: The following assembly referenced from monodroid/external/xamarin-android/bin/Debug/bin/Microsoft.Build.Tasks.Core.dll could not be loaded:
        Assembly:   System.Reflection.Metadata    (assemblyref_index=7)
        Version:    1.3.0.0
        Public Key: b03f5f7f11d50a3a

Currently `xabuild.exe` is referencing various MSBuild assemblies that
get copied into `bin/$(Configuration)/bin` and shipped in the OSS zip.

This isn't exactly desirable, since we don't really want to ship these
assemblies... These assemblies are OS-specific (as far as I can tell),
I noticed that these assemblies coming from a zip built on macOS do
not work on Windows.

`xabuild.exe` does not need to reference any MSBuild assemblies, and
we can use reflection + custom assembly loading to load MSBuild. This
keeps `bin/Debug/bin` free of additional assemblies, and now only
contains a few files:
- xabuild
- xabuild.exe
- xabuild.exe.config
- xabuild.pdb

By using the `AppDomain.AssemblyResolve` event, we can add some code
to probe the MSBuild bin directory for assemblies. If not found, we
can return `null` and let the runtime do its default behavior.
@jonathanpeppers jonathanpeppers force-pushed the xabuild-assemblyresolve branch from 7eabb49 to 846e359 Compare July 18, 2018 21:43
luhenry pushed a commit that referenced this pull request Jul 19, 2018
Context: #1975

In newer versions of Mono, `xabuild` is failing with:

    /Library/Frameworks/Mono.framework/Versions/Current/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(2126,5): error MSB3248:
        Parameter "AssemblyFiles" has invalid value "xamarin-android/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll".
        Could not load file or assembly 'System.Reflection.Metadata, Version=1.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.

Or another example:

    Mono: The following assembly referenced from monodroid/external/xamarin-android/bin/Debug/bin/Microsoft.Build.Tasks.Core.dll could not be loaded:
        Assembly:   System.Reflection.Metadata    (assemblyref_index=7)
        Version:    1.3.0.0
        Public Key: b03f5f7f11d50a3a

Looking at copies of `System.Reflection.Metadata` in my Mono
installation:

    $ find /Library/Frameworks/Mono.framework/ | grep System.Reflection.Metadata.dll$
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/msbuild/15.0/bin/System.Reflection.Metadata.dll
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/msbuild/15.0/bin/Roslyn/System.Reflection.Metadata.dll
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/fsharp/System.Reflection.Metadata.dll
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/4.5/System.Reflection.Metadata.dll
    /Library/Frameworks/Mono.framework//Versions/5.12.0/lib/mono/4.5/dim/System.Reflection.Metadata.dll

It appears that the Mono version of MSBuild has its own copy of
`System.Reflection.Metadata.dll`. Referencing this MSBuild-specific
assembly from `xabuild.csproj` on non-Windows platforms appears to
solve the problem.
@jonathanpeppers
Copy link
Member Author

An update on the changes here, Xamarin.Forms has a task using [LoadInSeparateAppDomain]:

https://github.com/xamarin/Xamarin.Forms/blob/fd733037a4788a9f1049142f6d33a30330ab8108/Xamarin.Forms.Build.Tasks/XamlTask.cs#L19

Which, explains why this is happening Windows: a new MSBuild.exe is launched.

@radical
Copy link
Member

radical commented Jul 19, 2018

I think what @jonathanpeppers meant was not really a new MSBuild.exe but a new app domain in the same process (please correct me if I'm wrong). And that is created for loading the XamlCTask. The new appdomain is missing the custom assembly resolver and defaults to looking in the directory containing xabuild.exe, and thus fails to find Microsoft.Build.*. This issue is not seen on mac though, because we have a fix for it (which isn't upstream yet as it's full impact on windows/VS isn't clear) where we set the ApplicationBase for the new appdomain to point to the msbuild bin dir which has all the msbuild assemblies.

@jonathanpeppers
Copy link
Member Author

Yeah I am not sure if the Application column in fuslogvw is a process, it could also be an AppDomain?

jonpryor pushed a commit that referenced this pull request Aug 2, 2018
Context: #1975

Mono 2018-04 (or 5.14.x) has some changes to MSBuild assemblies that
cause failures such as the following when using `xabuild.exe`:

	/Library/Frameworks/Mono.framework/Versions/Current/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(2126,5): error MSB3248:
	    Parameter "AssemblyFiles" has invalid value "xamarin-android/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll".
	    Could not load file or assembly 'System.Reflection.Metadata, Version=1.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.

	Mono: The following assembly referenced from monodroid/external/xamarin-android/bin/Debug/bin/Microsoft.Build.Tasks.Core.dll could not be loaded:
	    Assembly:   System.Reflection.Metadata    (assemblyref_index=7)
	    Version:    1.3.0.0
	    Public Key: b03f5f7f11d50a3a

It appears that the Mono version of MSBuild has its own copy of
`System.Reflection.Metadata.dll`.  Referencing this MSBuild-specific
assembly from `xabuild.csproj` on non-Windows platforms appears to
fix the above errors.
@dellis1972
Copy link
Contributor

Do we need this anymore? Can it be closed?

@jonathanpeppers
Copy link
Member Author

I will revisit this later, if we can work around what Windows is doing I think it is ultimately better.

What we have right now in our OSS zip file doesn't work when using a MacOS build on Windows, because it zips up MacOS assemblies for MSBuild.

jonpryor pushed a commit that referenced this pull request Aug 3, 2018
Context: #1975

Mono 2018-04 (or 5.14.x) has some changes to MSBuild assemblies that
cause failures such as the following when using `xabuild.exe`:

	/Library/Frameworks/Mono.framework/Versions/Current/lib/mono/msbuild/15.0/bin/Microsoft.Common.CurrentVersion.targets(2126,5): error MSB3248:
	    Parameter "AssemblyFiles" has invalid value "xamarin-android/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v1.0/mscorlib.dll".
	    Could not load file or assembly 'System.Reflection.Metadata, Version=1.3.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.

	Mono: The following assembly referenced from monodroid/external/xamarin-android/bin/Debug/bin/Microsoft.Build.Tasks.Core.dll could not be loaded:
	    Assembly:   System.Reflection.Metadata    (assemblyref_index=7)
	    Version:    1.3.0.0
	    Public Key: b03f5f7f11d50a3a

It appears that the Mono version of MSBuild has its own copy of
`System.Reflection.Metadata.dll`.  Referencing this MSBuild-specific
assembly from `xabuild.csproj` on non-Windows platforms appears to
fix the above errors.
@github-actions github-actions bot locked and limited conversation to collaborators Feb 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

do-not-merge PR should not be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants