building xamarin-android fails on Ubuntu 18.04 (CheckApiCompatibility found nonacceptable Api breakages for ApiLevel: v10.0.)

[janusw]

Steps to Reproduce

1. git clone https://github.com/xamarin/xamarin-android.git
2. cd xamarin-android
3. make prepare
4. make

Expected Behavior

make should run through without errors.

Actual Behavior

The build fails with the following errors:

"/home/janus/github/xamarin-android/Xamarin.Android.sln" (Standardziel) (1:2) ->
"/home/janus/github/xamarin-android/src/Xamarin.Android.Build.Tasks/Xamarin.Android.Build.Tasks.csproj" (Standardziel) (10:8) ->
"/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj" (Standardziel) (36:16) ->
"/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj" (Build Ziel) (36:17) ->
(_CheckApiCompatibility Ziel) -> 
  /home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.targets(193,5): error : CompatApi command: /home/janus/github/xamarin-android/packages/microsoft.dotnet.apicompat/5.0.0-beta.20181.7/tools/net472/Microsoft.DotNet.ApiCompat.exe "/home/janus/github/xamarin-android/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v9.0/Mono.Android.dll" -i "/home/janus/github/xamarin-android/bin/Debug/lib/xamarin.android/xbuild-frameworks/MonoAndroid/v10.0" --allow-default-interface-methods --baseline "/home/janus/github/xamarin-android/tests/api-compatibility/acceptable-breakages-v10.0.txt" --validate-baseline --exclude-attributes "/home/janus/github/xamarin-android/tests/api-compatibility/api-compat-exclude-attributes.txt"  [/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj]
/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.targets(193,5): error : CheckApiCompatibility found nonacceptable Api breakages for ApiLevel: v10.0. [/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj]
/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.targets(193,5): error : *** Invalid/Unused baseline differences *** [/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj]
/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.targets(193,5): error : CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.Bssid' in the contract but not the implementation. [/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj]
/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.targets(193,5): error : CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.Creator' in the contract but not the implementation. [/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj]
/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.targets(193,5): error : CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.Pin' in the contract but not the implementation. [/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj]
/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.targets(193,5): error : CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.Setup' in the contract but not the implementation. [/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj]
/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.targets(193,5): error : Total Issues: 4 [/home/janus/github/xamarin-android/src/Mono.Android/Mono.Android.csproj]


"/home/janus/github/xamarin-android/Xamarin.Android.sln" (Standardziel) (1:2) ->
"/home/janus/github/xamarin-android/src/monodroid/monodroid.csproj" (Standardziel) (19:6) ->
(_BuildAndroidRuntimes Ziel) -> 
  ../../../jni/embedded-assemblies.cc(348,80): error G3127DA5A: use of undeclared identifier 'MONO_TYPE_NAME_FORMAT_FULL_NAME' [/home/janus/github/xamarin-android/src/monodroid/monodroid.csproj]
  /home/janus/github/xamarin-android/src/monodroid/monodroid.targets(91,5): error MSB3073: Der Befehl "/home/janus/android-toolchain/sdk/cmake/3.10.2.4988404/bin/ninja -v" wurde mit dem Code 1 beendet. [/home/janus/github/xamarin-android/src/monodroid/monodroid.csproj]

It seems there are at least two separate errors.

Version Information

• xamarin-android version: master as of 2020-04-12 (commit d48cf04)
• OS: Ubuntu Linux 18.04 (bionic)
• Mono version: 6.4.0.198
• msbuild version: Microsoft (R)-Build-Engine, Version 16.3.0-ci für Mono

Log File

If necessary I can attach the full build log or provide further information.

[jpobst]

Interesting, we added those breakages to the acceptable breakages file about a month ago: https://github.com/xamarin/xamarin-android/blob/master/tests/api-compatibility/acceptable-breakages-v10.0.txt

Can you double-check those messages exist in that file locally?

[jpobst]

You can also set $(DisableApiCompatibilityCheck) to True to disable the compatibility check if you just want to get past it for now:

https://github.com/grendello/xamarin-android/blob/master/Documentation/building/configuration.md

[janusw]

Interesting, we added those breakages to the acceptable breakages file about a month ago: https://github.com/xamarin/xamarin-android/blob/master/tests/api-compatibility/acceptable-breakages-v10.0.txt

Can you double-check those messages exist in that file locally?

Yes, they seem to exist:

$ grep 'Android.Net.Wifi.WpsInfo' tests/api-compatibility/acceptable-breakages-v10.0.txt 
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'System.Int32 Android.Net.Wifi.WpsInfo.Display' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'System.Int32 Android.Net.Wifi.WpsInfo.Invalid' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'System.Int32 Android.Net.Wifi.WpsInfo.Keypad' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'System.Int32 Android.Net.Wifi.WpsInfo.Label' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'System.Int32 Android.Net.Wifi.WpsInfo.Pbc' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo..ctor()' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo..ctor(Android.Net.Wifi.WpsInfo)' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.DescribeContents()' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.WriteToParcel(Android.OS.Parcel, Android.OS.ParcelableWriteFlags)' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.Bssid' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.Creator' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.Pin' in the contract but not the implementation.
CannotRemoveAttribute : Attribute 'System.ObsoleteAttribute' exists on 'Android.Net.Wifi.WpsInfo.Setup' in the contract but not the implementation.

The last four appear to be the ones that I see. I hope you have an idea why they are flagged as nonacceptable after all.

[jpobst]

Oh, I misread your log (ApiCompat is a tool we recently starting using, so we're not proficient with it yet). Your log says: *** Invalid/Unused baseline differences ***, which I think means it didn't find the expected breakages.

Those "breakages" were caused here:
#4397
https://github.com/xamarin/java.interop/pull/600/files#diff-cd7c74b969d878caa558835d11f8d692R426

So it feels like you are either missing this change, or you built before you got this change and something isn't rebuilding like it should.

[janusw]

So it feels like you are either missing this change, or you built before you got this change and something isn't rebuilding like it should.

Usually I simply use make clean when rebuilding. However, I noticed that this produces errors like:

  /home/janus/github/xamarin-android/src/OpenTK-1.0/OpenTK.csproj : error MSB4057: Das Ziel "Clean" ist im Projekt nicht vorhanden.
  /home/janus/github/xamarin-android/src/System.EnterpriseServices/System.EnterpriseServices.csproj : error MSB4057: Das Ziel "Clean" ist im Projekt nicht vorhanden.
  /home/janus/github/xamarin-android/src/Xamarin.Android.NUnitLite/Xamarin.Android.NUnitLite.csproj : error MSB4057: Das Ziel "Clean" ist im Projekt nicht vorhanden.

So maybe a partial cleanup is causing the problem? Is there a reliable way to get a clean rebuild?

[jpobst]

When I want to be 100% certain I use this:

git clean -xfd
git submodule foreach --recursive git clean -xfd
git reset --hard
git submodule foreach --recursive git reset --hard
git submodule update --init --recursive

[janusw]

You can also set $(DisableApiCompatibilityCheck) to True to disable the compatibility check if you just want to get past it for now:

https://github.com/grendello/xamarin-android/blob/master/Documentation/building/configuration.md

This indeed helps to get rid of the error, but it would certainly be good to find the root cause.

[janusw]

When I want to be 100% certain I use this:

git clean -xfd
git submodule foreach --recursive git clean -xfd
git reset --hard
git submodule foreach --recursive git reset --hard
git submodule update --init --recursive

That sounds like a pretty big hammer. Nothing I'd wanna do on every rebuild. Would be nice to have something more lightweight (like a working make clean).

Note that the above leaves me with some unversioned content in external/Java.Interop.

But, yes, it seems to work. The errors are gone.

[janusw]

So, it seems we have two workarounds for this problem by now (thanks to @jpobst for the hints).

A proper fix IMHO would be either:

1. Make sure the current information from the acceptable-breakages-*files is used in partial rebuilds, or
2. Have a proper way to completely clean up all old build artifacts (like make clean).

I have too little knowledge of XE's build machinery to tell which one is preferable or how to implement it, though.

[janusw]

Btw, some further information for reproducing this issue:

• I already mentioned in the original issue description that I observed the issue when building master as of 2020-04-12 (commit d48cf04).
• I updated to this revision from a previous build of master from 2020-03-04 (commit 1a6fcff).

[jpobst]

For clarity, I believe the issue was a mismatch between the Xamarin.Android repo and the Java.Interop submodule repo. I also do not know enough about the build machinery to know if that should be automatic or if something is broken. 😄

[jonpryor]

So maybe a partial cleanup is causing the problem?

Kinda/not exactly. The problem is (arguably) an MSBuild bug/feature, in that the Clean target isn't treated special, it's treated like every other target. As such, it means that @(ProjectReference)s are cleaned before the current project.

Thus, if the current project depends on outputs of @(ProjectReference)s to exist in order to load, much less execute the Clean target, Bad Things Happen.

For example, every project which has a <UsingTask/> to an assembly which is created by a referenced project may find that assembly to be gone when the Clean target executes.

Or a project will <Import/> a file generated by a @(ProjectReference), and during Clean that file will be removed…

It is a decidedly non-trivial problem, and we have an open issue about it: #1129

Is there a reliable way to get a clean rebuild?

At the moment, no. The most reliable way to clean the repo is to have git clean it.

[jonpryor]

Closing this issue as the original reported issue appears to have been resolved: #4546 (comment)

But, yes, it seems to work. The errors are gone.