[Xamarin.Android.Build.Tasks] skip proguard.txt files in class libraries

[jonathanpeppers]

Context: dotnet/android-libraries#729

A customer reported a build error such as:

C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\33.0.46\targets\Microsoft.Android.Sdk.AndroidLibraries.targets(77,5):
error XACAAR7014: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.Text.StringBuilder.ToString()
at Xamarin.Android.Tasks.CreateAar.RunTask()
at Microsoft.Android.Build.Tasks.AndroidTask.Execute()

Somewhere here:

https://github.com/xamarin/xamarin-android/blob/c31f3eda63a16e412a39fa61eb0f885be3c5f543/src/Xamarin.Android.Build.Tasks/Tasks/CreateAar.cs#L108-L114

The problem appears to be triggered by:

ProguardConfigurationFiles=
~\.nuget\packages\xamarin.androidx.versionedparcelable\1.1.1.15\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.lifecycle.runtime\2.5.1.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.core\1.9.0.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.savedstate\1.2.0.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.recyclerview\1.2.1.7\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.lifecycle.viewmodel\2.5.1.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.lifecycle.viewmodelsavedstate\2.5.1.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.fragment\1.5.3.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.vectordrawable.animated\1.1.0.14\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.transition\1.4.1.8\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.startup.startupruntime\1.1.1.2\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.lifecycle.process\2.5.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.coordinatorlayout\1.2.0.3\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.appcompat\1.5.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.google.android.material\1.7.0\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.window\1.0.0.10\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.navigation.common\2.5.2.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.navigation.ui\2.5.2.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
~\.nuget\packages\xamarin.androidx.media\1.6.0.2\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
..\path\to\projectreference\obj\Debug\lp\160\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\155\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\161\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\162\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\156\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\157\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\163\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\164\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\165\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\166\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\167\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\170\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\168\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\169\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\171\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\172\jl\proguard.txt
..\path\to\projectreference\obj\Debug\lp\173\jl\proguard.txt

We have a compounding issue:

• .aar files that contain proguard.txt files are added to class libraries

• These make it to YourClassLibrary.aar

• You might have class libraries that reference other class libraries

• You end up with N^2 proguard.txt files and hit OOM.

I believe this problem was partially introduced in c537dd2, but we didn't notice until now. The AndroidX NuGet packages also cause this problem, but in a different way as seen in dotnet/android-libraries#729.

A solution here is that <ResolveLibraryProjectImports/> should simply ignore proguard.txt files in class libraries. They are only actually needed in "app" projects where the <R8/> MSBuild task would use them.

I could reproduce this in a test using an .aar from maven that contains a proguard.txt:

https://mvnrepository.com/artifact/androidx.core/core/1.10.0

And then just assert the proguard.txt contents don't make it to MyClassLibrary.aar.

[jonpryor]

Why would a unit test method need [DebuggerHidden]?

[jonpryor]

though i see some of the other Assert* methods have [DebuggerHidden]…

[jonathanpeppers]

The on assertion-style methods in this file, it allows the debugger to break on the line in your test instead of inside here.

It's really optional, and maybe only useful inside IDEs?

[jonpryor]

I also don't immediately see how a change to <ResolveLibraryProjectImports/> impacts <CreateAar/>. WHat's the "chain" here? Particularly when <ResolveLlibraryProjectImports/> only adds one file to ResolveLibraryProjectImports.ProguardConfigFiles and isn't itself responsible for the file concats…

[jonathanpeppers]

Ok with the updated test, this was previously failing with:

Expected string length 10 but was 22. Strings differ at index 10.
  Expected: "# LibraryB"
  But was:  "# LibraryB\r\n# LibraryC"

So this is better than using an .aar from maven.

[dellis1972]

do we know how this will impact the final app proguard usage?
One of the things we tried to fix in the c537dd2 was making sure that the proguard rules we get from library projects get included in the final app build.

Is this still the case with this change? especially if its a transitive dependency .

[jonathanpeppers]

In this test, we have:

App A -> Library B -> Library C

Where C is transitive, there is only a ProjectReference to B.

The app build has two files:

bin\TestDebug\temp\DotNetBuildLibraryFalseFalseFalse\AppA\obj\Debug\net8.0-android\lp\3\jl\proguard.txt
    OriginalFile = bin\TestDebug\temp\DotNetBuildLibraryFalseFalseFalse\LibraryB\bin\Debug\net8.0-android\LibraryB.aar
bin\TestDebug\temp\DotNetBuildLibraryFalseFalseFalse\AppA\obj\Debug\net8.0-android\lp\4\jl\proguard.txt
    OriginalFile = bin\TestDebug\temp\DotNetBuildLibraryFalseFalseFalse\LibraryC\bin\Debug\net8.0-android\LibraryC.aar

Previously, the file would contain duplicate rules:

lp\3\jl\proguard.txt
# LibraryB
# LibraryC
lp\4\jl\proguard.txt
# LibraryC

And now we have:

lp\3\jl\proguard.txt
# LibraryB
lp\4\jl\proguard.txt
# LibraryC

I think it's correct now. But we might consider shipping this in a .NET 8 preview before sending it back to 7?

[jonpryor]

Context: c537dd28c30f482f365ef756214be35aa1553da2
Context: https://github.com/xamarin/AndroidX/pull/729
Context: https://github.com/xamarin/GooglePlayServicesComponents/pull/774

A customer reported a build error such as:

	C:\Program Files\dotnet\packs\Microsoft.Android.Sdk.Windows\33.0.46\targets\Microsoft.Android.Sdk.AndroidLibraries.targets(77,5):
	error XACAAR7014: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
	   at System.Text.StringBuilder.ToString()
	   at Xamarin.Android.Tasks.CreateAar.RunTask()
	   at Microsoft.Android.Build.Tasks.AndroidTask.Execute()

Somewhere within [`<CreateAar/>`][0]:

	if (ProguardConfigurationFiles != null) {
	    var sb = new StringBuilder ();
	    foreach (var file in ProguardConfigurationFiles) {
	        sb.AppendLine (File.ReadAllText (file.ItemSpec));
	    }
	    aar.AddEntry ("proguard.txt", sb.ToString (), Files.UTF8withoutBOM);
	}

The problem appears to be triggered by:

 1. Library project `Lib.csproj` which bind `.aar` files include the
    `proguard.txt` from the `.aar` into the output `Lib.aar`, as
    `Lib.aar!proguard.txt`.

 2. Library project `References.csproj` which has a `@(Reference)` to
    `Lib.csproj` will include the contents of `Lib.aar!proguard.txt`
    into `References.aar!proguard.txt`.  If `References.csproj` has a
    `@(PackageReference)` which has `AnotherLib.aar`, then
    `AnotherLib.aar!proguard.txt` will *also* be copied into
    `References.aar!proguard.txt`.
    (Repeat for all `@(Reference)` and `@(projectreference)`.)
    (Note combinatorial explosion in `proguard.txt` size…)

 3. App project `App.csproj` references (2), merging all the contents
    of all the `proguard.txt` files from all the `.aar` files into a
    single unified `proguard.txt` file.

If the project has lots of library projects or package references,
step (3) could OOM.  "Lots" of references, such as:

	ProguardConfigurationFiles=
	    ~\.nuget\packages\xamarin.androidx.versionedparcelable\1.1.1.15\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.lifecycle.runtime\2.5.1.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.core\1.9.0.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.savedstate\1.2.0.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.recyclerview\1.2.1.7\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.lifecycle.viewmodel\2.5.1.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.lifecycle.viewmodelsavedstate\2.5.1.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.fragment\1.5.3.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.vectordrawable.animated\1.1.0.14\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.transition\1.4.1.8\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.startup.startupruntime\1.1.1.2\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.lifecycle.process\2.5.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.coordinatorlayout\1.2.0.3\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.appcompat\1.5.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.google.android.material\1.7.0\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.window\1.0.0.10\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.navigation.common\2.5.2.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.navigation.ui\2.5.2.1\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ~\.nuget\packages\xamarin.androidx.media\1.6.0.2\buildTransitive\net6.0-android31.0\..\..\proguard\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\160\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\155\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\161\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\162\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\156\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\157\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\163\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\164\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\165\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\166\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\167\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\170\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\168\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\169\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\171\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\172\jl\proguard.txt
	    ..\path\to\projectreference\obj\Debug\lp\173\jl\proguard.txt
	    …

I believe this problem was partially introduced in c537dd28, but we
didn't notice until now.  The AndroidX NuGet packages also cause this
problem, but in a different way as seen in xamarin/AndroidX#729 and
xamarin/GooglePlayServicesComponents#774.

A solution here is that `<ResolveLibraryProjectImports/>` should
simply ignore `proguard.txt` files in class libraries.  They are only
actually needed in "app" projects where the `<R8/>` MSBuild task
would use them.

I could reproduce this in a test using an `.aar` from maven that
contains a `proguard.txt`:

  * <https://mvnrepository.com/artifact/androidx.core/core/1.10.0>

And then just assert the `proguard.txt` contents don't make it to
`MyClassLibrary.aar`.

Note: xamarin/AndroidX#729 and xamarin/GooglePlayServicesComponents#774
update the NuGet Packages so that `proguard.txt` files are only
added to `@(ProguardConfiguration)` when `$(AndroidApplication)`=True.
This should also help prevent the OOM scenario.

[0]: https://github.com/xamarin/xamarin-android/blob/c31f3eda63a16e412a39fa61eb0f885be3c5f543/src/Xamarin.Android.Build.Tasks/Tasks/CreateAar.cs#L108-L114