[build-tools] Bump Test emulator to use API 31 #5257

dellis1972 · 2020-11-03T14:34:05Z

We need to be able to test our build system against newer emulators. So let's include both API 30 and API 30 based emulator images in our nightly tests.

We will leave the normal development tests running on API 29 for now as the newer emulators seem to be unreliable
in a CI environment.

Part of the changes in to use the Context.GetExternalFilesDir (null); to get the external files directory. This is due to permissions changes which came into effect from API 30. As a result the older code which we used can no longer be
used on newer devices. So we shall unify the tests to all use Context.GetExternalFilesDir (null);. Then use Context.FilesDir.AbsolutePath; as a backup. The result of this change is we MUST use the run-as command to retrieve the test log file for processing on API 30. This is because the external directory is no longer readable globally on API 30. It is only readable for the app. As a result we also need to make the API 30 build of the test app debuggable via the [Application (Debuggable=true)] attribute.

Thankfully this change was reverted somewhat in API 31, so we can still adb pull from the directory returned by Context.GetExternalFilesDir (null);.

Other changes include updating tests to include the x86_64 abi. This is because the newer x86_64 emulators no longer appear to be able to run x86 code.

jonathanpeppers · 2020-11-03T15:00:35Z

I found an old PR that did this:

https://github.com/xamarin/xamarin-android/pull/4874/files

Do we need these URLs, too? I'll rebase it.

dellis1972 · 2020-11-03T15:26:07Z

@jonathanpeppers this is a work in progress (forgot to make it draft), and yes I am changing those other files as well

build-tools/scripts/TestApks.targets

jonpryor · 2020-11-03T22:35:31Z

Fails to build:

Step Xamarin.Android.Prepare.Step_Android_SDK_NDK failed: Failed to download https://dl.google.com/android/repository/sys-img/android/x86_64-30_r09-darwin.zip

jonpryor · 2020-11-09T18:15:13Z

I fear/suspect that the API-30 emulator has moved where we expect our TestResult.xml files to be.

Consider Mono.Android_Tests > Possible Crash / Release. Check the actual logcat, and we see:

11-09 07:08:57.981  3439  3599 I NUnitLite: Passed: 774, Failed: 1, Skipped: 2, Inconclusive: 0

The tests did finish; there is no crash.

Checking runMono.Android_Tests.binlog, and I see:

  Executing: /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5570  logcat -c (TaskId:1058)
  Executing: /Users/runner/Library/Android/sdk/platform-tools/adb -s emulator-5570  pull "/storage/emulated/0/Android/data/Mono.Android_Tests/files/Documents/TestResults.xml" "../../../bin/TestRelease/TestResult-Mono.Android_Tests.xml" (TaskId:1058)
  adb: error: failed to stat remote object '/storage/emulated/0/Android/data/Mono.Android_Tests/files/Documents/TestResults.xml': Permission denied (TaskId:1058)

Looks like we need to use some other mechanism to retrieve the TestResults.xml file.

dellis1972 · 2020-11-09T19:37:04Z

@jonpryor yup I'm looking at just using the /local/tmp directory for this stuff.
the other option is to use run-as to cat the file from the internal directory to the file on the local host.

pjcollins · 2020-12-02T19:34:22Z

/azp run

azure-pipelines · 2020-12-02T19:34:59Z

Azure Pipelines successfully started running 1 pipeline(s).

jonpryor · 2020-12-09T21:12:35Z

Looks like we're not hitting breakpoints?! https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=4279056&view=logs&j=bc770d6b-0ee0-5594-b22c-77e857541d93&t=ffd8a1cf-6dca-58c0-3f91-29722003ed50

5) Failed : Xamarin.Android.Build.Tests.DebuggingTest.ApplicationRunsWithDebuggerAndBreaks(False,"Assemblies:Dexes",True)
  Should have hit 1 breakpoints. Only hit 0
  Expected: 1
  But was:  0

src/Xamarin.Android.NUnitLite/Gui/Instrumentations/TestSuiteInstrumentation.cs

tests/Xamarin.Forms-Performance-Integration/Droid/Properties/AndroidManifest.xml

build-tools/scripts/TestApks.targets

jonathanpeppers

This looks good except for the 2 spots I found something.

...ools.BootstrapTasks/Xamarin.Android.Tools.BootstrapTasks/RunAndroidEmulatorCheckBootTimes.cs

tests/MSBuildDeviceIntegration/Tests/DebuggingTest.cs

...n.Android.Tools.BootstrapTasks/Xamarin.Android.Tools.BootstrapTasks/CreateAndroidEmulator.cs

...Android.Tools.BootstrapTasks/Xamarin.Android.Tools.BootstrapTasks/RunInstrumentationTests.cs

jonpryor · 2022-01-19T01:57:13Z

Mono.Android.NET_Tests-Interpreter is crashing due to a SIGSEGV (?!):

3686  3686 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
3686  3686 F DEBUG   : Build fingerprint: 'Android/sdk_phone_x86_64/generic_x86_64:10/QPP6.190730.005.B1/5775370:userdebug/test-keys'
3686  3686 F DEBUG   : Revision: '0'
3686  3686 F DEBUG   : ABI: 'x86_64'
3686  3686 F DEBUG   : Timestamp: 2022-01-18 16:53:04+0000
3686  3686 F DEBUG   : pid: 3666, tid: 3666, name: droid.NET_Tests  >>> Mono.Android.NET_Tests <<<
3686  3686 F DEBUG   : uid: 10105
3686  3686 F DEBUG   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x2
3686  3686 F DEBUG   : Cause: null pointer dereference
3686  3686 F DEBUG   :     rax 0000000000002b36  rbx 000078c8afb3f860  rcx 000078c98b6561f0  rdx 0000000000000000
3686  3686 F DEBUG   :     r8  0000000000000002  r9  0000000000000080  r10 000078c98b296080  r11 000078c987d35178
3686  3686 F DEBUG   :     r12 00007fffe46ae99c  r13 000078c8afb89ea0  r14 000078c8afb3f990  r15 000078c98b6746c0
3686  3686 F DEBUG   :     rdi 000078c8afb3f860  rsi 0000000000000002
3686  3686 F DEBUG   :     rbp 0000000000000001  rsp 00007fffe46ae448  rip 000078c8afb1b31c
3666  3666 W main    : type=1400 audit(0.0:40): avc: granted { read } for name="u:object_r:net_dns_prop:s0" dev="tmpfs" ino=6642 scontext=u:r:untrusted_app_25:s0:c512,c768 tcontext=u:object_r:net_dns_prop:s0 tclass=file app=Mono.Android.NET_Tests
3686  3686 F DEBUG   : 
3686  3686 F DEBUG   : backtrace:
3686  3686 F DEBUG   :       #00 pc 000000000002c31c  /data/app/Mono.Android.NET_Tests-fbdZV696v1UeW3jUzJg9yg==/lib/x86_64/libmonodroid.so (xamarin::android::Util::monodroid_store_package_name(char const*)+12) (BuildId: 91fe7d9c6b30356fcfb8337b8541d0132df4f44a)
3686  3686 F DEBUG   :       #01 pc 0000000000025bbc  /data/app/Mono.Android.NET_Tests-fbdZV696v1UeW3jUzJg9yg==/lib/x86_64/libmonodroid.so (xamarin::android::internal::MonodroidRuntime::Java_mono_android_Runtime_initInternal(_JNIEnv*, _jclass*, _jstring*, _jobjectArray*, _jstring*, _jobjectArray*, _jobject*, _jobjectArray*, int, unsigned char, unsigned char)+652) (BuildId: 91fe7d9c6b30356fcfb8337b8541d0132df4f44a)
3686  3686 F DEBUG   :       #02 pc 00000000000273fb  /data/app/Mono.Android.NET_Tests-fbdZV696v1UeW3jUzJg9yg==/lib/x86_64/libmonodroid.so (Java_mono_android_Runtime_initInternal+75) (BuildId: 91fe7d9c6b30356fcfb8337b8541d0132df4f44a)
3686  3686 F DEBUG   :       #03 pc 0000000000174641  /apex/com.android.runtime/lib64/libart.so (art_quick_generic_jni_trampoline+209) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
3686  3686 F DEBUG   :       #04 pc 00000000001696b6  /apex/com.android.runtime/lib64/libart.so (art_quick_invoke_static_stub+806) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
3686  3686 F DEBUG   :       #05 pc 0000000000179c61  /apex/com.android.runtime/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+305) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
3686  3686 F DEBUG   :       #06 pc 0000000000340ffc  /apex/com.android.runtime/lib64/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*)+380) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
3686  3686 F DEBUG   :       #07 pc 000000000033c5aa  /apex/com.android.runtime/lib64/libart.so (bool art::interpreter::DoCall<true, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*)+826) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
3686  3686 F DEBUG   :       #08 pc 000000000066fddb  /apex/com.android.runtime/lib64/libart.so (MterpInvokeStaticRange+491) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
3686  3686 F DEBUG   :       #09 pc 0000000000162c99  /apex/com.android.runtime/lib64/libart.so (mterp_op_invoke_static_range+25) (BuildId: 8bb3225e7c408f2ca23abac3db0417f2)
…

What's doubly odd is that this isn't the first pid in the log file; above snippet is from pid 3686, but the first pid of consequence in the log file is pid 3546:

3546  3566 I DOTNET  : # jonp: expected: JniValueMarshaler_IJavaPeerable_ContractTests
3546  3566 I DOTNET  : {
3546  3566 I DOTNET  : 	JniRuntime __jvm;
3546  3566 I DOTNET  : 	IJavaPeerable __value;
3546  3566 I DOTNET  : 	JniObjectReference __v
…
3546  3566 I NUnit   : Passed: 781, Failed: 0, Skipped: 6, Inconclusive: 0, Total: 0, Filtered: 0

…which finished successfully.

I don't know where pid 3686 came from.

Looks like pid 3546 is from the run Mono.Android.NET_Tests-Aab step.

Thus, it looks like:

We don't capture the full interpreter logcat output :-(
Something about the interpreter launch is crashing on the emulator.

@grendello: this feels like something "weird" is going in when using all of (1) .NET 6, (2) the interpreter, and (3) an API-31 emulator target. Would you be able to investigate?

grendello · 2022-01-19T09:48:52Z

I was able to reproduce this crash using Xamarin.Android from main, without any changes in this PR. The test crashes both on device (Pixel 6 Pro) and the emulator. The issue here appears to be a problem with our build process in presence of both API 31 and API 32 dotnet packs. What happens is the following:

When locally building Xamarin.Android, both the managed assemblies and the native runtime are placed in the Microsoft.Android.Runtime.32.android-* packs
The ~/android-toolchain/dotnet/packs location also has the Microsoft.Android.Runtime.31.android-* packs installed with (in my case) December 1st 2021 modification date of the files inside
When building the test APK with dotnet build /p:TestsFlavor=Interpreter /p:UseInterpreter=True /p:AndroidPackageFormat=apk /bl -c Debug /t:SignAndroidPackage Mono.Android.NET-Tests.csproj, then build ends up using the runtime from the API 31 packs above and not the freshly built API 32 ones:

xamarin-android/tests/Mono.Android-Tests/Runtime-Microsoft.Android.Sdk $ dotnet msbuild msbuild.binlog /v:diag |grep '^[ ]*/home.*native/libmono-android\.'|tr -d ' '|sort|uniq
~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.31.android-arm/31.0.101-preview.11.117/runtimes/android-arm/native/libmono-android.debug.so
~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.31.android-arm/31.0.101-preview.11.117/runtimes/android-arm/native/libmono-android.release.so
~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.31.android-arm64/31.0.101-preview.11.117/runtimes/android-arm64/native/libmono-android.debug.so
~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.31.android-arm64/31.0.101-preview.11.117/runtimes/android-arm64/native/libmono-android.release.so
~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.31.android-x64/31.0.101-preview.11.117/runtimes/android-x64/native/libmono-android.debug.so
~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.31.android-x64/31.0.101-preview.11.117/runtimes/android-x64/native/libmono-android.release.so
~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.31.android-x86/31.0.101-preview.11.117/runtimes/android-x86/native/libmono-android.debug.so
~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.31.android-x86/31.0.101-preview.11.117/runtimes/android-x86/native/libmono-android.release.so

When I replaced libmono-android.*.so in ~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.31.android-* with the shared libraries from the just-built ~/android-toolchain/dotnet/packs/Microsoft.Android.Runtime.32.android-* locations, the apk no longer crashes (I added logging to the Utils::monodroid_store_package_name function, because it seemed to be an "impossible" place for a segfault):

01-19 10:27:28.318 13604 13604 D monodroid-assembly: Checking if libmonodroid was unpacked to /data/app/~~PwQl5ZHhFhmMNycSIOrIwA==/Mono.Android.NET_Tests-NZxymJMaRzWPEaBKgOrw6A==/lib/arm64/libmonodroid.so
01-19 10:27:28.318 13604 13604 D monodroid-assembly: Native libs extracted to /data/app/~~PwQl5ZHhFhmMNycSIOrIwA==/Mono.Android.NET_Tests-NZxymJMaRzWPEaBKgOrw6A==/lib/arm64, assuming application/android:extractNativeLibs == true
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::monodroid_store_package_name(const char *)
01-19 10:27:28.318 13604 13604 I monodroid:   name == 'Mono.Android.NET_Tests' (0x78d4353111)
01-19 10:27:28.318 13604 13604 I monodroid:   before hash loop
01-19 10:27:28.318 13604 13604 I monodroid:   after hash loop, hash ==  0x2654624b
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::package_hash_to_hex(uint32_t, IdxType, Indices...) [IdxType = unsigned int, Indices = <unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int>]
01-19 10:27:28.318 13604 13604 I monodroid:   idx == 0; hex_chars index == 2
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::package_hash_to_hex(uint32_t, IdxType, Indices...) [IdxType = unsigned int, Indices = <unsigned int, unsigned int, unsigned int, unsigned int, unsigned int, unsigned int>]
01-19 10:27:28.318 13604 13604 I monodroid:   idx == 1; hex_chars index == 6
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::package_hash_to_hex(uint32_t, IdxType, Indices...) [IdxType = unsigned int, Indices = <unsigned int, unsigned int, unsigned int, unsigned int, unsigned int>]
01-19 10:27:28.318 13604 13604 I monodroid:   idx == 2; hex_chars index == 5
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::package_hash_to_hex(uint32_t, IdxType, Indices...) [IdxType = unsigned int, Indices = <unsigned int, unsigned int, unsigned int, unsigned int>]
01-19 10:27:28.318 13604 13604 I monodroid:   idx == 3; hex_chars index == 4
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::package_hash_to_hex(uint32_t, IdxType, Indices...) [IdxType = unsigned int, Indices = <unsigned int, unsigned int, unsigned int>]
01-19 10:27:28.318 13604 13604 I monodroid:   idx == 4; hex_chars index == 6
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::package_hash_to_hex(uint32_t, IdxType, Indices...) [IdxType = unsigned int, Indices = <unsigned int, unsigned int>]
01-19 10:27:28.318 13604 13604 I monodroid:   idx == 5; hex_chars index == 2
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::package_hash_to_hex(uint32_t, IdxType, Indices...) [IdxType = unsigned int, Indices = <unsigned int>]
01-19 10:27:28.318 13604 13604 I monodroid:   idx == 6; hex_chars index == 4
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::package_hash_to_hex(uint32_t, IdxType, Indices...) [IdxType = unsigned int, Indices = <>]
01-19 10:27:28.318 13604 13604 I monodroid:   idx == 7; hex_chars index == 11
01-19 10:27:28.318 13604 13604 I monodroid: void xamarin::android::Util::package_hash_to_hex(IdxType) [IdxType = unsigned int]
01-19 10:27:28.318 13604 13604 I monodroid: Generated hash 0x2654624b for package name Mono.Android.NET_Tests
01-19 10:27:28.318 13604 13604 I monodroid: Mono AOT mode: interp
01-19 10:27:28.318 13604 13604 I monodroid: Setting environment variable 'DOTNET_MODIFIABLE_ASSEMBLIES' to 'Debug'
01-19 10:27:28.318 13604 13604 I monodroid: Setting environment variable 'MONO_GC_PARAMS' to 'major=marksweep-conc'
01-19 10:27:28.318 13604 13604 I monodroid: Setting environment variable 'MONO_LOG_LEVEL' to 'info'
01-19 10:27:28.318 13604 13604 I monodroid: Setting environment variable 'XAMARIN_BUILD_ID' to '34f04b55-ab01-4eef-bf88-f44b757fb437'
01-19 10:27:28.318 13604 13604 I monodroid: Setting environment variable 'XA_HTTP_CLIENT_HANDLER_TYPE' to 'Xamarin.Android.Net.AndroidMessageHandler'
01-19 10:27:28.318 13604 13604 I monodroid: Setting environment variable '__XA_PACKAGE_NAMING_POLICY__' to 'LowercaseCrc64'
01-19 10:27:28.318 13604 13604 I monodroid: Creating XDG directory: /data/user/0/Mono.Android.NET_Tests/files/.local/share
01-19 10:27:28.318 13604 13604 I monodroid: Creating XDG directory: /data/user/0/Mono.Android.NET_Tests/files/.config
01-19 10:27:28.318 13604 13604 I monodroid: Setting up for DSO lookup in app data directories
...
01-19 10:27:28.520 13604 13623 I NUnit   : [Xamarin.Android Version: 31.0.101.117; git-rev-head:21e5fe7; git-branch:release/6.0.1xx-preview11]
01-19 10:27:28.531 13604 13623 I NUnit   : [Android version:         12; API 31 (S); REL; Incremental: 8030436; Security patch: 2022-01-05]
01-19 10:27:28.532 13604 13623 I NUnit   : [Board:                   raven]
01-19 10:27:28.532 13604 13623 I NUnit   : [Bootloader:              slider-1.0-8013568]
01-19 10:27:28.532 13604 13623 I NUnit   : [Brand:                   google]
01-19 10:27:28.533 13604 13623 I NUnit   : [CpuAbi:                  arm64-v8a ]
01-19 10:27:28.544 13604 13623 I NUnit   : [Supported ABIs:          arm64-v8a, armeabi-v7a, armeabi]
01-19 10:27:28.544 13604 13623 I NUnit   : [Supported 32-bit ABIs:   armeabi-v7a, armeabi]
01-19 10:27:28.545 13604 13623 I NUnit   : [Supported 64-bit ABIs:   arm64-v8a]
01-19 10:27:28.545 13604 13623 I NUnit   : [Device:                  raven]
01-19 10:27:28.545 13604 13623 I NUnit   : [Display:                 SQ1D.220105.007]
01-19 10:27:28.546 13604 13623 I NUnit   : [Fingerprint:             google/raven/raven:12/SQ1D.220105.007/8030436:user/release-keys]
01-19 10:27:28.546 13604 13623 I NUnit   : [Hardware:                raven]
01-19 10:27:28.546 13604 13623 I NUnit   : [Host:                    abfarm-release-rbe-64-00095]
01-19 10:27:28.546 13604 13623 I NUnit   : [Id:                      SQ1D.220105.007]
01-19 10:27:28.547 13604 13623 I NUnit   : [Manufacturer:            Google]
01-19 10:27:28.547 13604 13623 I NUnit   : [Model:                   Pixel 6 Pro]
01-19 10:27:28.547 13604 13623 I NUnit   : [Product:                 raven]
01-19 10:27:28.548 13604 13623 I NUnit   : [Radio:                   g5123b-93368-211225-B-8029609]
01-19 10:27:28.548 13604 13623 I NUnit   : [Tags:                    release-keys]
01-19 10:27:28.549 13604 13623 I NUnit   : [Time:                    1640531787000]
01-19 10:27:28.549 13604 13623 I NUnit   : [Type:                    user]
01-19 10:27:28.549 13604 13623 I NUnit   : [User:                    android-build]
01-19 10:27:28.550 13604 13623 I NUnit   : [VERSION.Codename::       REL]
01-19 10:27:28.550 13604 13623 I NUnit   : [VERSION.Incremental:     8030436]
01-19 10:27:28.550 13604 13623 I NUnit   : [VERSION.Release:         12]
01-19 10:27:28.550 13604 13623 I NUnit   : [VERSION.Sdk:             31]
01-19 10:27:28.550 13604 13623 I NUnit   : [VERSION.SdkInt:          S]
...
01-19 10:27:28.587 13604 13623 I NUnit   : Configuring test categories to include:
01-19 10:27:28.587 13604 13623 I NUnit   :   none
01-19 10:27:28.587 13604 13623 I NUnit   : Configuring test categories to include from extras:
01-19 10:27:28.587 13604 13623 I NUnit   :   none
01-19 10:27:28.587 13604 13623 I NUnit   : Configuring test categories to exclude:
01-19 10:27:28.587 13604 13623 I NUnit   :   none
01-19 10:27:28.587 13604 13623 I NUnit   : Configuring test categories to exclude from extras:
01-19 10:27:28.588 13604 13623 I NUnit   :   DotNetIgnore
01-19 10:27:28.588 13604 13623 I NUnit   :   IgnoreInterpreter
01-19 10:27:28.589 13604 13623 I NUnit   : Configuring tests to exclude (by name):
01-19 10:27:28.589 13604 13623 I NUnit   :   none
01-19 10:27:28.589 13604 13623 I NUnit   : Starting unit tests
...
01-19 10:27:28.778 13604 13623 I NUnit   : Start: Mono.Android.NET-Tests, Version=1.0.0.0, Culture=neutral, PublicKeyToken=84e04ff9cfb79065
01-19 10:27:28.786 13604 13623 I NUnit   : Mono.Android.NET-Tests.dll
01-19 10:27:28.848 13604 13623 I NUnit   : ApplicationTest
01-19 10:27:28.851 13604 13623 I NUnit   : ApplicationContextIsApp 
01-19 10:27:28.857 13604 13623 I NUnit   :      Passed
01-19 10:27:28.860 13604 13623 I NUnit   : EnsureAndroidManifestIsUpdated

Perhaps the same situation happens on CI and we don't use what we just built, but something from the past packages? The problem is likely to be the fact that the runtimes from the API 31 packs were built against a different libmonosgen-2.0.so than what is packaged in the APK.

dellis1972 · 2022-02-11T10:32:39Z

@jonpryor This Nightly build is green https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=5736992&view=results

jonpryor · 2022-02-23T17:00:28Z

We need to be able to test our build system against newer emulators.
Include both API-30 and API-31 -based emulator images into our
nightly test pipeline `azure-pipelines-nightly.yaml`.

We will leave the normal CI tests running on API-29 for now as the
newer emulators seem to be reliable in a CI environment. 

One of the changes is in the use of `Context.GetExternalFilesDir(null)`
to create a subdirectory within the external files directory.  This is
due to [permissions changes which came into effect in API-30][0]:

> **Note**: On Android 11 (API level 30) and higher, apps cannot
> create their own app-specific directory on external storage.

As a result our previous code can no longer be used on newer devices.

We shall instead unify the tests to all use
`Context.GetExternalFilesDir(null)`, then use
`Context.FilesDir.AbsolutePath` as a backup.
The result of this change is we MUST use the `run-as` command to
retrieve the test log file for processing on API-30.  This is because
the external directory is no longer readable globally on API-30; it
is only readable for the app.  As a result we also need to make the
API-30 build of the test app "debuggable" via
`[Application (Debuggable=true)]`. 

Thankfully this change was reverted somewhat in API-31, so we can
still `adb pull` from the directory returned by
`Context.GetExternalFilesDir(null);`. 

Other changes include updating tests to include the `x86_64` abi.
This is because the newer x86_64 emulators no longer appear to be
able to run `x86` code. 

[0]: https://developer.android.com/training/data-storage/app-specific#external-access-files

dellis1972 · 2022-02-25T15:37:21Z

Nightly is also green https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=5801345&view=results

dellis1972 requested a review from jonpryor as a code owner November 3, 2020 14:34

dellis1972 requested a review from grendello as a code owner November 3, 2020 16:02

dellis1972 commented Nov 3, 2020

View reviewed changes

build-tools/scripts/TestApks.targets Outdated Show resolved Hide resolved

jonathanpeppers mentioned this pull request Nov 3, 2020

[tests] bump to API 30 emulator #4874

Closed

dellis1972 force-pushed the bumpemulatortoapi30 branch 3 times, most recently from ecc14ad to e6a4254 Compare November 9, 2020 13:42

dellis1972 force-pushed the bumpemulatortoapi30 branch 3 times, most recently from 76e2b1b to a0b6dc9 Compare November 18, 2020 13:11

dellis1972 force-pushed the bumpemulatortoapi30 branch from a0b6dc9 to 6a80790 Compare December 7, 2020 12:15

dellis1972 force-pushed the bumpemulatortoapi30 branch from 6a80790 to c85052f Compare December 10, 2020 09:41

dellis1972 force-pushed the bumpemulatortoapi30 branch from c85052f to 93049e7 Compare December 18, 2020 12:49

dellis1972 force-pushed the bumpemulatortoapi30 branch from 93049e7 to f91680a Compare January 4, 2021 10:20

dellis1972 requested a review from jonathanpeppers as a code owner January 5, 2021 12:09

jonathanpeppers reviewed Jan 5, 2021

View reviewed changes

src/Xamarin.Android.NUnitLite/Gui/Instrumentations/TestSuiteInstrumentation.cs Outdated Show resolved Hide resolved

tests/Xamarin.Forms-Performance-Integration/Droid/Properties/AndroidManifest.xml Outdated Show resolved Hide resolved

dellis1972 force-pushed the bumpemulatortoapi30 branch 3 times, most recently from fa2d459 to 50c8a77 Compare January 7, 2021 15:37

dellis1972 force-pushed the bumpemulatortoapi30 branch 2 times, most recently from 8f84367 to acaf5fe Compare February 1, 2021 10:42

pjcollins reviewed Feb 2, 2021

View reviewed changes

build-tools/scripts/TestApks.targets Outdated Show resolved Hide resolved

dellis1972 force-pushed the bumpemulatortoapi30 branch 2 times, most recently from ecd5649 to f7219dd Compare January 6, 2022 10:25

jonathanpeppers approved these changes Jan 10, 2022

View reviewed changes

...ools.BootstrapTasks/Xamarin.Android.Tools.BootstrapTasks/RunAndroidEmulatorCheckBootTimes.cs Outdated Show resolved Hide resolved

tests/MSBuildDeviceIntegration/Tests/DebuggingTest.cs Outdated Show resolved Hide resolved

dellis1972 force-pushed the bumpemulatortoapi30 branch 2 times, most recently from 63761e7 to 8234bed Compare January 12, 2022 11:44

dellis1972 force-pushed the bumpemulatortoapi30 branch 2 times, most recently from c267f95 to acd8864 Compare January 18, 2022 10:50

jonpryor reviewed Jan 18, 2022

View reviewed changes

...n.Android.Tools.BootstrapTasks/Xamarin.Android.Tools.BootstrapTasks/CreateAndroidEmulator.cs Outdated Show resolved Hide resolved

jonpryor reviewed Jan 18, 2022

View reviewed changes

...Android.Tools.BootstrapTasks/Xamarin.Android.Tools.BootstrapTasks/RunInstrumentationTests.cs Show resolved Hide resolved

dellis1972 force-pushed the bumpemulatortoapi30 branch from acd8864 to a6ccff3 Compare January 19, 2022 09:42

dellis1972 force-pushed the bumpemulatortoapi30 branch 3 times, most recently from 05cb0b1 to c9896e5 Compare January 25, 2022 10:24

dellis1972 force-pushed the bumpemulatortoapi30 branch 2 times, most recently from 032625c to e2bb779 Compare February 3, 2022 10:13

dellis1972 force-pushed the bumpemulatortoapi30 branch from ae56ed2 to 9de878a Compare February 9, 2022 13:47

dellis1972 force-pushed the bumpemulatortoapi30 branch 3 times, most recently from 1fbda49 to b012cc5 Compare February 21, 2022 19:34

dellis1972 force-pushed the bumpemulatortoapi30 branch from 70c7885 to d6eec19 Compare February 24, 2022 13:00

[build-tools] Bump Test emulator to use API 30

b6074ce

dellis1972 force-pushed the bumpemulatortoapi30 branch from d6eec19 to b6074ce Compare February 24, 2022 13:16

jonpryor merged commit d365a3b into dotnet:main Feb 25, 2022

dellis1972 deleted the bumpemulatortoapi30 branch February 25, 2022 18:40

github-actions bot locked and limited conversation to collaborators Jan 24, 2024

[build-tools] Bump Test emulator to use API 31 #5257

[build-tools] Bump Test emulator to use API 31 #5257

Uh oh!

Conversation

dellis1972 commented Nov 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jonathanpeppers commented Nov 3, 2020

Uh oh!

dellis1972 commented Nov 3, 2020

Uh oh!

Uh oh!

jonpryor commented Nov 3, 2020

Uh oh!

jonpryor commented Nov 9, 2020

Uh oh!

dellis1972 commented Nov 9, 2020

Uh oh!

pjcollins commented Dec 2, 2020

Uh oh!

azure-pipelines bot commented Dec 2, 2020

Uh oh!

jonpryor commented Dec 9, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jonathanpeppers left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jonpryor commented Jan 19, 2022

Uh oh!

grendello commented Jan 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dellis1972 commented Feb 11, 2022

Uh oh!

jonpryor commented Feb 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dellis1972 commented Feb 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

dellis1972 commented Nov 3, 2020 •

edited

Loading

grendello commented Jan 19, 2022 •

edited

Loading

jonpryor commented Feb 23, 2022 •

edited

Loading