Skip to content

Conversation

@jonpryor
Copy link
Contributor

@jonpryor jonpryor commented Jul 21, 2017

Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=58221

The build scenario:

  1. Use Windows. (This doesn't happen on macOS.)

  2. Clone: https://github.com/xamarin/xamarin-forms-samples/tree/master/Templates/DataTemplateSelector

  3. Edit Selector\Selector.csproj and change $(DebugType) to Portable.

  4. Open Selector.sln within VS and restore all packages.

  5. Build the Droid\Selector.Droid.csproj project:

    msbuild /p:Configuration=Release /t:SignAndroidPackage Droid\Selector.Droid.csproj
    
  6. Touch (update the timestamp of) Selector\Selector.csproj.

  7. Build the project Droid\Selector.Droid.csproj project again.

    msbuild /p:Configuration=Release /t:SignAndroidPackage Droid\Selector.Droid.csproj
    

Expected results: No errors.

Actual results: Linking fails:

error MSB4018: The "LinkAssemblies" task failed unexpectedly.\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018: Mono.Linker.MarkException: Error processing method: 'Xamarin.Forms.View Xamarin.Forms.Platform.Android.VisualElementRenderer`1::get_View()' in assembly: 'Xamarin.Forms.Platform.Android.dll' ---> System.ArgumentNullException: Value cannot be null.\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018: Parameter name: instruction\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.Cil.InstructionOffset..ctor(Instruction instruction)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScope(ScopeDebugInformation scope)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScopes(Collection`1 scopes)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScope(ScopeDebugInformation scope)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScopes(Collection`1 scopes)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScope(ScopeDebugInformation scope)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadDebugInfo()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadMethodBody()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadMethodBody(MethodDefinition method)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.MetadataReader.ReadMethodBody(MethodDefinition method)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.MethodDefinition.<>c.<get_Body>b__41_0(MethodDefinition method, MetadataReader reader)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.ModuleDefinition.Read[TItem,TRet](TRet& variable, TItem item, Func`3 read)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Cecil.MethodDefinition.get_Body()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Linker.Steps.MarkStep.ProcessQueue()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    --- End of inner exception stack trace ---\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Linker.Steps.MarkStep.ProcessQueue()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Linker.Steps.MarkStep.Process()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Linker.Steps.MarkStep.Process(LinkContext context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Mono.Linker.Pipeline.Process(LinkContext context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at MonoDroid.Tuner.Linker.Run(Pipeline pipeline, LinkContext context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at MonoDroid.Tuner.Linker.Process(LinkerOptions options, LinkContext& context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Xamarin.Android.Tasks.LinkAssemblies.Execute(DirectoryAssemblyResolver res)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Xamarin.Android.Tasks.LinkAssemblies.Execute()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext() [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]

The cause of the error is in the interaction between the
_CopyIntermediateAssemblies, _CopyMdbFiles, and
_StripEmbeddedLibraries targets.

During the initial build in Step (5), obj\Release\linksrc is
populated with copies of the original assemblies and debug symbols for
those assemblies. The contents of obj\Release\linksrc are then
updated in place by various tasks such as the
<StripEmbeddedLibraries/> and <LinkAssemblies/> tasks. After
stripping and linking, the timestamps in obj\Release\linksrc are
updated and the assemblies and associated debug symbols may be smaller
than the original copies.

During the rebuild in Step (7), _CopyIntermediateAssemblies replaces
obj\Release\linksrc\Xamarin.Forms.Platform.Android.dll with a copy
of the original file in packages\Xamarin.Forms...\lib\MonoAndroid10.
The _CopyMdbFiles task -- which is responsible for copying
.dll.mdb files into obj\Release\linksrc -- does not update the
Xamarin.Forms.Platform.Android.dll.mdb file. The result of this
inaction is that when the <StripEmbeddedLibraries/> task executes,
the assembly and corresponding debug symbols are out-of-sync:
Xamarin.Forms.Platform.Android.dll is the original+unstripped
assembly, while Xamarin.Forms.Platform.Android.dll.mdb has been
modified as part of (both?) the <StripEmbeddedLibraries/> and/or
<LinkAssemblies/> task.

When <StripEmbeddedLibraries/> executes again as part of the
rebuild, it takes these out-of-sync files and then proceeds to
corrupt the Xamarin.Forms.Platform.Android.dll.mdb file. After the
<StripEmbeddedLibraries/> task completes, we eventually hit the
<LinkAssemblies/> task, during which Cecil attempts to load the
now-corrupt Xamarin.Forms.Platform.Android.dll.mdb file, which
results in the above ArgumentNullException.

The fix is in the _CopyIntermediateAssemblies target: when we copy
assemblies into the obj\Release\linksrc directory, ensure that any
corresponding .dll.mdb files for those asssemblies are deleted.
This will allow _CopyMdbFiles to re-copy the debug symbols, ensuring
that everything stays in sync.

Additionally, update the _CopyIntermediateAssemblies target to
<Touch/> the copied files so that timestamps compare properly.
Otherwise, on the 2nd+ build, the target re-executes because the
project assembly is newer than e.g. a BCL assembly:

Target _CopyIntermediateAssemblies needs to be built as input file
'…/xamarin-android/bin/TestDebug/temp/BuildBasicApplicationCheckPdb/bin/Debug/UnnamedProject.dll'
is newer than output file 'obj/Debug/linksrc/NetStandard16.dll'

Without the <Touch/>, the above will always be true.

@jonpryor
Copy link
Contributor Author

@dellis1972: An alternate -- more correct? -- fix would be to instead delete the obj\$(Configuration)\linksrc directory in _CopyIntermediateAssemblies (?). This would ensure that we're starting from a "clean slate," thus avoiding the entire scenario.

One problem with nuking linksrc is that it's possibly large -- 44+MB for this sample -- and that data would need to be re-copied.

One advantage to nuking is that if an assembly is removed from @(ResolvedAssemblies), nothing will delete that assembly from obj\$(Configuration)\linksrc. However, perhaps that doesn't matter, as we explicitly list and load assemblies, as opposed to using a file glob on that directory.

@jonpryor
Copy link
Contributor Author

One problem with this PR is...no tests.

Unfortunately I don't know how to write an automated test for this scenario. :-(

@jonpryor
Copy link
Contributor Author

Perhaps unsurprisingly, this patch causes two unit test failures:

Both of these fail with the message:

the _CopyMdbFiles target should be skipped

However, as per prior analysis/discussion, the _CopyMdbFiles target can't be skipped if the _CopyIntermediateAssemblies target executes.

Reading the BuildBasicApplicationCheckPdb build log, the _CopyIntermediateAssemblies target does execute:

	Target _CopyIntermediateAssemblies needs to be built as input file '/Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/bin/TestDebug/temp/BuildBasicApplicationCheckPdb/bin/Debug/UnnamedProject.dll' is newer than output file 'obj/Debug/linksrc/NetStandard16.dll'

This thus looks like the test case should be updated...except I'm confused: the BuildBasicApplicationCheckPdb test doesn't appear to have modified between the initial build (line 1676) and the subsequent build (line 1715), so why is _CopyIntermediateAssemblies executing at all? Why is bin/Debug/UnnamedProject.dll newer than anything in obj/Debug/linksrc?

For reference, the BuildBasicApplicationCheckPdb build log.

The quick answer: because _CopyIntermediateAssemblies uses the <Copy/> task, and the <Copy/> task preserves the original timestamp. Consequently, the timestamp for UnnamedProject.dll will always be newer than NetStandard16.dll, along with every other assembly distributed with Xamarin.Android!

Time for a <Touch/>!

Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=58221

The build scenario:

 1. Use Windows. (This doesn't happen on macOS.)

 2. Clone: https://github.com/xamarin/xamarin-forms-samples/tree/master/Templates/DataTemplateSelector

 3. Edit `Selector\Selector.csproj` and change `$(DebugType)` to Portable.

 4. Open `Selector.sln` within VS and restore all packages.

 5. Build the `Droid\Selector.Droid.csproj` project:

        msbuild /p:Configuration=Release /t:SignAndroidPackage Droid\Selector.Droid.csproj

 6. Touch (update the timestamp of) `Selector\Selector.csproj`.

 7. Build the project `Droid\Selector.Droid.csproj` project again.

        msbuild /p:Configuration=Release /t:SignAndroidPackage Droid\Selector.Droid.csproj

Expected results: No errors.

Actual results: Linking fails:

	error MSB4018: The "LinkAssemblies" task failed unexpectedly.\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018: Mono.Linker.MarkException: Error processing method: 'Xamarin.Forms.View Xamarin.Forms.Platform.Android.VisualElementRenderer`1::get_View()' in assembly: 'Xamarin.Forms.Platform.Android.dll' ---> System.ArgumentNullException: Value cannot be null.\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018: Parameter name: instruction\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.InstructionOffset..ctor(Instruction instruction)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScope(ScopeDebugInformation scope)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScopes(Collection`1 scopes)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScope(ScopeDebugInformation scope)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScopes(Collection`1 scopes)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScope(ScopeDebugInformation scope)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadDebugInfo()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadMethodBody()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadMethodBody(MethodDefinition method)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.MetadataReader.ReadMethodBody(MethodDefinition method)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.MethodDefinition.<>c.<get_Body>b__41_0(MethodDefinition method, MetadataReader reader)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.ModuleDefinition.Read[TItem,TRet](TRet& variable, TItem item, Func`3 read)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.MethodDefinition.get_Body()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.ProcessQueue()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    --- End of inner exception stack trace ---\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.ProcessQueue()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.Process()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.Process(LinkContext context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Pipeline.Process(LinkContext context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at MonoDroid.Tuner.Linker.Run(Pipeline pipeline, LinkContext context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at MonoDroid.Tuner.Linker.Process(LinkerOptions options, LinkContext& context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Xamarin.Android.Tasks.LinkAssemblies.Execute(DirectoryAssemblyResolver res)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Xamarin.Android.Tasks.LinkAssemblies.Execute()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext() [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]

The cause of the error is in the interaction between the
`_CopyIntermediateAssemblies`, `_CopyMdbFiles`, and
`_StripEmbeddedLibraries` targets.

During the initial build in Step (5), `obj\Release\linksrc` is
populated with copies of the original assemblies and debug symbols for
those assemblies. The contents of `obj\Release\linksrc` are then
updated *in place* by various tasks such as the
`<StripEmbeddedLibraries/>` and `<LinkAssemblies/>` tasks. After
stripping and linking, the timestamps in `obj\Release\linksrc` are
updated and the assemblies and associated debug symbols may be smaller
than the original copies.

During the rebuild in Step (7), `_CopyIntermediateAssemblies` replaces
`obj\Release\linksrc\Xamarin.Forms.Platform.Android.dll` with a copy
of the original file in `packages\Xamarin.Forms...\lib\MonoAndroid10`.
The `_CopyMdbFiles` task -- which is responsible for copying
`.dll.mdb` files into `obj\Release\linksrc` -- does *not* update the
`Xamarin.Forms.Platform.Android.dll.mdb` file. The result of this
inaction is that when the `<StripEmbeddedLibraries/>` task executes,
the assembly and corresponding debug symbols are out-of-sync:
`Xamarin.Forms.Platform.Android.dll` is the original+unstripped
assembly, while `Xamarin.Forms.Platform.Android.dll.mdb` has been
*modified* as part of (both?) the `<StripEmbeddedLibraries/>` and/or
`<LinkAssemblies/>` task.

When `<StripEmbeddedLibraries/>` executes again as part of the
rebuild, it takes these out-of-sync files and then proceeds to
*corrupt* the `Xamarin.Forms.Platform.Android.dll.mdb` file. After the
`<StripEmbeddedLibraries/>` task completes, we eventually hit the
`<LinkAssemblies/>` task, during which Cecil attempts to load the
now-corrupt `Xamarin.Forms.Platform.Android.dll.mdb` file, which
results in the above `ArgumentNullException`.

The fix is in the `_CopyIntermediateAssemblies` target: when we copy
assemblies into the `obj\Release\linksrc` directory, ensure that any
corresponding `.dll.mdb` files for those asssemblies are *deleted*.
This will allow `_CopyMdbFiles` to re-copy the debug symbols, ensuring
that everything stays in sync.

*Additionally*, update the `_CopyIntermediateAssemblies` target to
`<Touch/>` the copied files so that timestamps compare properly.
Otherwise, on the 2nd+ build, the target re-executes because the
project assembly is newer than e.g. a BCL assembly:

	Target _CopyIntermediateAssemblies needs to be built as input file
	'…/xamarin-android/bin/TestDebug/temp/BuildBasicApplicationCheckPdb/bin/Debug/UnnamedProject.dll'
	is newer than output file 'obj/Debug/linksrc/NetStandard16.dll'

Without the `<Touch/>`, the above will *always* be true.
@jonpryor jonpryor force-pushed the jonp-delete-stable-symbols branch from a8ae38a to c06ec9c Compare July 22, 2017 03:38
@jonpryor
Copy link
Contributor Author

Now it's failing in BuildTest.BuildAMassiveApp:

Build of project should have succeeded

However, reading the 40MB log file at https://jenkins.mono-project.com/job/xamarin-android-pr-builder/ws/xamarin-android/bin/TestDebug/temp/BuildAMassiveApp/build.log/*view*/ -- subject to invalidation at the next PR build -- I don't see any errors in there.

More "interestingly" is that the error message should be coming from BuildTest.cs:1834:

Assert.IsTrue(sb.BuildProject(app1, "SignAndroidPackage"), "Build of project should have succeeded");

I don't see any mention of SignAndroidPackage in the build.log file.

Ah! It's not in the top-level build.log, it's in App1/build.log (8MB): https://jenkins.mono-project.com/job/xamarin-android-pr-builder/ws/xamarin-android/bin/TestDebug/temp/BuildAMassiveApp/App1/build.log/*view*/

...which shows a mono crash:

mono_os_cond_timedwait: pthread_cond_timedwait failed with "Invalid argument" (22)

#stdout end

#stderr begin
Stacktrace:

  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) System.Threading.WaitHandle.WaitAny_internal (System.Threading.WaitHandle[],int) [0x00008] in <2ffe54eb3b79405792636f3ca93458bd>:0
  at System.Threading.WaitHandle.WaitMultiple (System.Threading.WaitHandle[],int,bool,bool) [0x0003c] in /Users/builder/data/lanes/4992/mono-mac-sdk/external/bockbuild/builds/mono-x64/mcs/class/corlib/System.Threading/WaitHandle.cs:74
  at System.Threading.WaitHandle.WaitAny (System.Threading.WaitHandle[],int,bool) [0x000a1] in /Users/builder/data/lanes/4992/mono-mac-sdk/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/threading/waithandle.cs:481
  at System.Threading.WaitHandle.WaitAny (System.Threading.WaitHandle[],System.TimeSpan,bool) [0x0002c] in /Users/builder/data/lanes/4992/mono-mac-sdk/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/threading/waithandle.cs:516
  at System.Threading.WaitHandle.WaitAny (System.Threading.WaitHandle[],System.TimeSpan) [0x00000] in /Users/builder/data/lanes/4992/mono-mac-sdk/external/bockbuild/builds/mono-x64/mcs/class/referencesource/mscorlib/system/threading/waithandle.cs:521
  at Xamarin.Android.Tasks.AsyncTask.WaitForCompletion () [0x00068] in /Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/AsyncTask.cs:277
  at Xamarin.Android.Tasks.AsyncTask.Execute () [0x00001] in /Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/AsyncTask.cs:219
  at Xamarin.Android.Tasks.Aot.DoExecute () [0x001e5] in /Users/builder/jenkins/workspace/xamarin-android-pr-builder/xamarin-android/src/Xamarin.Android.Build.Tasks/Tasks/Aot.cs:236
...
Native stacktrace:

	0   mono                                0x0000000109fc5641 mono_handle_native_crash + 257
	1   libsystem_platform.dylib            0x00007fff92bb752a _sigtramp + 26
	2   mono                                0x000000010a31866d my_charset + 5141
	3   libsystem_c.dylib                   0x00007fff8a0266df abort + 129
	4   mono                                0x000000010a1a4f51 mono_log_write_logfile + 353
	5   mono                                0x000000010a1bc230 monoeg_g_log + 208
	6   mono                                0x000000010a1103cb mono_os_cond_timedwait + 235
	7   mono                                0x000000010a10fd13 mono_w32handle_timedwait_signal + 115
	8   mono                                0x000000010a10f89d mono_w32handle_wait_multiple + 2269
	9   mono                                0x000000010a0e86d3 mono_wait_uninterrupted + 115
	10  mono                                0x000000010a0e883c ves_icall_System_Threading_WaitHandle_WaitAny_internal + 236
	11  ???                                 0x000000011233d27e 0x0 + 4600353406
	12  mscorlib.dll.dylib                  0x000000011022cf32 System_Threading_WaitHandle_WaitAny_System_Threading_WaitHandle___int_bool + 226

Debug info from gdb:

(lldb) command source -s 0 '/tmp/mono-gdb-commands.hzLMeb'
Executing commands in '/tmp/mono-gdb-commands.hzLMeb'.
(lldb) process attach --pid 64421
error: attach failed: unable to attach

=================================================================
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries 
used by your application.
=================================================================

That sounds like "not my problem."

I'll re-build the PR.

@jonpryor
Copy link
Contributor Author

build

@dellis1972 dellis1972 merged commit 1ea5f07 into dotnet:master Jul 24, 2017
jonpryor added a commit that referenced this pull request Jul 24, 2017
Fixes: https://bugzilla.xamarin.com/show_bug.cgi?id=58221

The build scenario:

 1. Use Windows. (This doesn't happen on macOS.)

 2. Clone: https://github.com/xamarin/xamarin-forms-samples/tree/master/Templates/DataTemplateSelector

 3. Edit `Selector\Selector.csproj` and change `$(DebugType)` to Portable.

 4. Open `Selector.sln` within VS and restore all packages.

 5. Build the `Droid\Selector.Droid.csproj` project:

        msbuild /p:Configuration=Release /t:SignAndroidPackage Droid\Selector.Droid.csproj

 6. Touch (update the timestamp of) `Selector\Selector.csproj`.

 7. Build the project `Droid\Selector.Droid.csproj` project again.

        msbuild /p:Configuration=Release /t:SignAndroidPackage Droid\Selector.Droid.csproj

Expected results: No errors.

Actual results: Linking fails:

	error MSB4018: The "LinkAssemblies" task failed unexpectedly.\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018: Mono.Linker.MarkException: Error processing method: 'Xamarin.Forms.View Xamarin.Forms.Platform.Android.VisualElementRenderer`1::get_View()' in assembly: 'Xamarin.Forms.Platform.Android.dll' ---> System.ArgumentNullException: Value cannot be null.\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018: Parameter name: instruction\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.InstructionOffset..ctor(Instruction instruction)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScope(ScopeDebugInformation scope)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScopes(Collection`1 scopes)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScope(ScopeDebugInformation scope)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScopes(Collection`1 scopes)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadScope(ScopeDebugInformation scope)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadDebugInfo()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadMethodBody()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.Cil.CodeReader.ReadMethodBody(MethodDefinition method)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.MetadataReader.ReadMethodBody(MethodDefinition method)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.MethodDefinition.<>c.<get_Body>b__41_0(MethodDefinition method, MetadataReader reader)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.ModuleDefinition.Read[TItem,TRet](TRet& variable, TItem item, Func`3 read)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Cecil.MethodDefinition.get_Body()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.ProcessMethod(MethodDefinition method)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.ProcessQueue()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    --- End of inner exception stack trace ---\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.ProcessQueue()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.Process()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Steps.MarkStep.Process(LinkContext context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Mono.Linker.Pipeline.Process(LinkContext context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at MonoDroid.Tuner.Linker.Run(Pipeline pipeline, LinkContext context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at MonoDroid.Tuner.Linker.Process(LinkerOptions options, LinkContext& context)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Xamarin.Android.Tasks.LinkAssemblies.Execute(DirectoryAssemblyResolver res)\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Xamarin.Android.Tasks.LinkAssemblies.Execute()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()\r [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]
	error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext() [C:\Users\jonpr\Dev\xamarin-forms-samples\Templates\DataTemplateSelector\Droid\Selector.Droid.csproj]

The cause of the error is in the interaction between the
`_CopyIntermediateAssemblies`, `_CopyMdbFiles`, and
`_StripEmbeddedLibraries` targets.

During the initial build in Step (5), `obj\Release\linksrc` is
populated with copies of the original assemblies and debug symbols for
those assemblies. The contents of `obj\Release\linksrc` are then
updated *in place* by various tasks such as the
`<StripEmbeddedLibraries/>` and `<LinkAssemblies/>` tasks. After
stripping and linking, the timestamps in `obj\Release\linksrc` are
updated and the assemblies and associated debug symbols may be smaller
than the original copies.

During the rebuild in Step (7), `_CopyIntermediateAssemblies` replaces
`obj\Release\linksrc\Xamarin.Forms.Platform.Android.dll` with a copy
of the original file in `packages\Xamarin.Forms...\lib\MonoAndroid10`.
The `_CopyMdbFiles` task -- which is responsible for copying
`.dll.mdb` files into `obj\Release\linksrc` -- does *not* update the
`Xamarin.Forms.Platform.Android.dll.mdb` file. The result of this
inaction is that when the `<StripEmbeddedLibraries/>` task executes,
the assembly and corresponding debug symbols are out-of-sync:
`Xamarin.Forms.Platform.Android.dll` is the original+unstripped
assembly, while `Xamarin.Forms.Platform.Android.dll.mdb` has been
*modified* as part of (both?) the `<StripEmbeddedLibraries/>` and/or
`<LinkAssemblies/>` task.

When `<StripEmbeddedLibraries/>` executes again as part of the
rebuild, it takes these out-of-sync files and then proceeds to
*corrupt* the `Xamarin.Forms.Platform.Android.dll.mdb` file. After the
`<StripEmbeddedLibraries/>` task completes, we eventually hit the
`<LinkAssemblies/>` task, during which Cecil attempts to load the
now-corrupt `Xamarin.Forms.Platform.Android.dll.mdb` file, which
results in the above `ArgumentNullException`.

The fix is in the `_CopyIntermediateAssemblies` target: when we copy
assemblies into the `obj\Release\linksrc` directory, ensure that any
corresponding `.dll.mdb` files for those asssemblies are *deleted*.
This will allow `_CopyMdbFiles` to re-copy the debug symbols, ensuring
that everything stays in sync.

*Additionally*, update the `_CopyIntermediateAssemblies` target to
`<Touch/>` the copied files so that timestamps compare properly.
Otherwise, on the 2nd+ build, the target re-executes because the
project assembly is newer than e.g. a BCL assembly:

	Target _CopyIntermediateAssemblies needs to be built as input file
	'…/xamarin-android/bin/TestDebug/temp/BuildBasicApplicationCheckPdb/bin/Debug/UnnamedProject.dll'
	is newer than output file 'obj/Debug/linksrc/NetStandard16.dll'

Without the `<Touch/>`, the above will *always* be true.
jonpryor pushed a commit that referenced this pull request Sep 2, 2020
Fixes: https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1139590
Context: dotnet/java-interop@afbc5b3

Changes: dotnet/java-interop@dcaf794...afbc5b3

  * dotnet/java-interop@afbc5b32: [C++] Get rid of compiler warnings (#703)
  * dotnet/java-interop@e295b498: [generator] Fix localization error. (#702)
  * dotnet/java-interop@5a0e37e0: [generator] Support 'managedOverride' metadata (#701)
  * dotnet/java-interop@f6c12ba3: [Xamarin.Android.Tools.Bytecode] Hide Kotlin synthetic constructors (#700)
  * dotnet/java-interop@0334f424: [generator] Hide Java.Lang.Object infrastructure (#698)
  * dotnet/java-interop@c0fcc435: [jcw-gen] Make MSBuild warning/error strings localizable. (#695)

Fixes a handful of warnings reported by GCC and clang when building
for different platforms.

`-Wshadow` warnings:

	monodroid-glue.cc:1609:8: warning: declaration shadows a static data member of 'xamarin::android::internal::MonodroidRuntime' [-Wshadow]
	void *api_dso_handle = nullptr;
	              ^
	monodroid-glue.cc:102:25: note: previous declaration is here
	void *MonodroidRuntime::api_dso_handle = nullptr;

Fix by renaming the local variable.  No functionality changes result
from this fix.

MinGW builds report:

	xa-internal-api.cc: In member function ‘virtual mono_bool xamarin::android::internal::MonoAndroidInternalCalls_Impl::monodroid_get_network_interface_up_state(const char*, mono_bool*)’:
	xa-internal-api.cc:24:86: warning: unused parameter ‘ifname’ [-Wunused-parameter]
	   24 | MonoAndroidInternalCalls_Impl::monodroid_get_network_interface_up_state (const char *ifname, mono_bool *is_up)
	      |                                                                          ~~~~~~~~~~~~^~~~~~
	xa-internal-api.cc:24:105: warning: unused parameter ‘is_up’ [-Wunused-parameter]
	   24 | MonoAndroidInternalCalls_Impl::monodroid_get_network_interface_up_state (const char *ifname, mono_bool *is_up)
	      |                                                                                              ~~~~~~~~~~~^~~~~
	xa-internal-api.cc: In member function ‘virtual mono_bool xamarin::android::internal::MonoAndroidInternalCalls_Impl::monodroid_get_network_interface_supports_multicast(const char*, mono_bool*)’:
	xa-internal-api.cc:34:96: warning: unused parameter ‘ifname’ [-Wunused-parameter]
	   34 | MonoAndroidInternalCalls_Impl::monodroid_get_network_interface_supports_multicast (const char *ifname, mono_bool *supports_multicast)
	      |                                                                                    ~~~~~~~~~~~~^~~~~~
	xa-internal-api.cc:34:115: warning: unused parameter ‘supports_multicast’ [-Wunused-parameter]
	   34 | MonoAndroidInternalCalls_Impl::monodroid_get_network_interface_supports_multicast (const char *ifname, mono_bool *supports_multicast)
	      |                                                                                                        ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
	xa-internal-api.cc: In member function ‘virtual int xamarin::android::internal::MonoAndroidInternalCalls_Impl::monodroid_get_dns_servers(void**)’:
	xa-internal-api.cc:44:66: warning: unused parameter ‘dns_servers_array’ [-Wunused-parameter]
	   44 | MonoAndroidInternalCalls_Impl::monodroid_get_dns_servers (void **dns_servers_array)
	      |                                                           ~~~~~~~^~~~~~~~~~~~~~~~~
	xa-internal-api.cc: In member function ‘virtual int xamarin::android::internal::MonoAndroidInternalCalls_Impl::monodroid_getifaddrs(_monodroid_ifaddrs**)’:
	xa-internal-api.cc:54:82: warning: unused parameter ‘ifap’ [-Wunused-parameter]
	   54 | MonoAndroidInternalCalls_Impl::monodroid_getifaddrs (struct _monodroid_ifaddrs **ifap)
	      |                                                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~
	xa-internal-api.cc: In member function ‘virtual void xamarin::android::internal::MonoAndroidInternalCalls_Impl::monodroid_freeifaddrs(_monodroid_ifaddrs*)’:
	xa-internal-api.cc:64:82: warning: unused parameter ‘ifa’ [-Wunused-parameter]
	   64 | MonoAndroidInternalCalls_Impl::monodroid_freeifaddrs (struct _monodroid_ifaddrs *ifa)
	      |                                                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~

Fix by ignoring the unused parameters only during Windows builds
by using the C++17 attribute `[[maybe_unused]]`

MinGW also reports:

	embedded-assemblies.cc: In static member function ‘static ssize_t xamarin::android::internal::EmbeddedAssemblies::do_read(int, void*, size_t)’:
	embedded-assemblies.cc:663:26: warning: conversion from ‘size_t’ {aka ‘long long unsigned int’} to ‘unsigned int’ may change value [-Wconversion]
	  663 |   ret = ::read (fd, buf, count);
	      |

It is caused by **read**(2) declared in MinGW headers with the third
parameter using an `unsigned int` type instead of the standard
`size_t`.  Fixed by casting properly on Windows builds.
@github-actions github-actions bot locked and limited conversation to collaborators Feb 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants