`AndroidNativeLibrary` doesn't trigged `JNI_OnLoad`

[thenextman]

Android application type

Classic Xamarin.Android (MonoAndroid12.0, etc.)

Affected platform version

VS 2022 17.4, Xamarin.Android 13.1.0.1

Description

I have a shared library (.so) written in C. The library exports JNI_OnLoad and JNI_OnUnload, confirmed with readelf:

478: 00000000002ff2f0 132 FUNC GLOBAL DEFAULT 13 JNI_OnLoad
3111: 00000000002ff380 297 FUNC GLOBAL DEFAULT 13 JNI_OnUnload

I reference the .so properly in my Xamarin.Android application using AndroidNativeLibrary.

However, at runtime JNI_OnLoad is never invoked.

Steps to Reproduce

See above

Did you find any workaround?

I can invoke JNI_OnLoad manually by first using reflection to get the java_vm pointer:

[DllImport("MyLibrary", CallingConvention = CallingConvention.Cdecl)]
private static extern int JNI_OnLoad(IntPtr vm, IntPtr reserved);

static MyClass()
{
    var field = typeof(Android.Runtime.JNIEnv).GetField("java_vm", System.Reflection.BindingFlags.NonPublic | System.Reflection.BindingFlags.Static);
    JNI_OnLoad((IntPtr)field.GetValue(null), IntPtr.Zero);
}

But this feels like an ugly workaround.

Relevant log output

N/A

[grendello]

@thenextman the library needs to be loaded explicitly, most likely, in the similar way we load Xamarin.Android's runtime (libmonodroid.so).

[thenextman]

we load

Sure, I tried this approach already. LoadLibrary isn't an option because it only looks in the Java PATH; but I need to load from the Xamarin's native libraries directory.

Passing the absolute path to Load just gives:

{Java.Lang.NullPointerException: Attempt to invoke virtual method ‘java.lang.ClassLoader > java.lang.Class.getClassLoader()’ on a null object reference
at Java.Interop.JniEnvironment+StaticMethods.CallStaticVoidMethod (Java.Interop.JniObjectReference type, Java…}

The workaround could be acceptable if the java_vm on JNIEnv was made public. It's the use of reflection that irks me...

[grendello]

@thenextman your library will be packaged next to libmonodroid.so, and as you can see we just pass just the base library name (monodroid) which works just fine. Android knows to look for shared libraries in either the APK (where they are stored uncompressed and aligned to the page boundary) or in the application data directory, if shared library extraction is enabled (it is disabled by default). Also, just p/invoking into the library should work to trigger JNI_OnLoad. In this case, the Mono runtime will request us, the Xamarin.Android runtime, to load the library which we do and the first time Java resolves a native function to be in one of the loaded libraries, it will invoke the JNI load function.

This approach is used in, for instance, the dotnet's BCL encryption library which p/invokes into libSystem.Security.Cryptography.Native.Android.so without any special treatment, and it has JNI_OnLoad:

$ nm -CD libSystem.Security.Cryptography.Native.Android.so | grep JNI_
0000000000012420 T JNI_OnLoad

and, as can be seen below, the library is packaged alongside our runtime in the APK:

$ zipinfo com.microsoft.net6.helloandroid-Signed.apk | grep lib/arm64-v8a
-rw-r--r--  6.3 unx  2920648 b- defX 22-Dec-02 17:02 lib/arm64-v8a/libxamarin-app.so
-rwxr--r--  6.3 unx   723840 b- defX 22-Dec-02 11:26 lib/arm64-v8a/libSystem.IO.Compression.Native.so
-rwxr--r--  6.3 unx    94232 b- defX 22-Dec-02 11:26 lib/arm64-v8a/libSystem.Native.so
-rwxr--r--  6.3 unx   149552 b- defX 22-Dec-02 11:26 lib/arm64-v8a/libSystem.Security.Cryptography.Native.Android.so
-rwxr--r--  6.3 unx   224592 b- defX 22-Dec-02 11:26 lib/arm64-v8a/libmono-component-debugger.so
-rwxr--r--  6.3 unx   108048 b- defX 22-Dec-02 11:26 lib/arm64-v8a/libmono-component-hot_reload.so
-rwxr--r--  6.3 unx  3077928 b- defX 22-Dec-02 11:26 lib/arm64-v8a/libmonosgen-2.0.so
-rwxr-xr-x  6.3 unx  3687576 b- defX 22-Dec-02 11:26 lib/arm64-v8a/libmonodroid.so
-rwxr-xr-x  6.3 unx   211528 b- defX 22-Dec-02 11:26 lib/arm64-v8a/libxamarin-debug-app-helper.so

[thenextman]

@grendello Thanks for the input.

I do see my library (libMyLib.so) packaged inside {app.apk}/lib/{abi} alongside libmonodroid et all, as in your example. It exists for all supported ABIs.

However calling Java.Lang.JavaSystem.LoadLibrary("MyLib"); yields:

Java.Lang.UnsatisfiedLinkError: dlopen failed: library "libMyLib.so" not found
at Java.Interop.JniEnvironment+StaticMethods.CallStaticVoidMethod (Java.Interop.JniObjectReference type, Java.Interop.JniMethodInfo method, Java.Interop.JniArgumentValue* args) [0x00068] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/obj/Release/JniEnvironment.g.cs:13250
at Java.Interop.JniPeerMembers+JniStaticMethods.InvokeVoidMethod (System.String encodedMember, Java.Interop.JniArgumentValue* parameters) [0x00008] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniPeerMembers.JniStaticMethods.cs:97
at Java.Lang.JavaSystem.LoadLibrary (System.String libname) [0x0001f] in /Users/runner/work/1/s/xamarin-android/src/Mono.Android/obj/Release/monoandroid10/android-31/mcw/Java.Lang.JavaSystem.cs:339

I can p/invoke the library ([DllImport("MyLib")]...) but equally, JNI_OnLoad is not invoked.

the first time Java resolves a native function to be in one of the loaded libraries

I'm not sure I follow (I don't know Xamarin.Android internals at all). Where is Java involved here if I'm p/invoking a vanilla C function from .NET?

[grendello]

@thenextman let's start with the logcat :) Can you please run your app until it throws the above exception, after first configuring your device as follows:

$ adb shell setprop debug.mono.log default,assembly,timing=bare
$ adb logcat -G 16M
$ adb logcat -c
# Run your app here, make it crash, wait a few moments and then
$ adb logcat -d > log.txt

This will show is why the library fails to load (the "not found" message might be a red herring).

With regards to your question where Java is involved. JNI_OnLoad is a Java Native Interface (JNI) function called by the Java VM whenever it encounters a Java class with a function declared as follows:

package helloandroid;


public class MainActivity
        extends androidx.appcompat.app.AppCompatActivity
        implements
                mono.android.IGCUserPeer
{
        public void onCreate (android.os.Bundle p0)
        {
                n_onCreate (p0);
        }

        private native void n_onCreate (android.os.Bundle p0);
}

The native keyword will cause the Java runtime (ART in Android) to try to find a native shared library which exports a symbol with a properly formatted name. In case of the native n_onCreate method, the native symbol name is going to be Java_helloandroid_MainActivity_n_1onCreate. Now, for each managed class which eventually derives from the Java.Lang.Object Xamarin.Android will generate a corresponding Java class (the above fragment mirrors a managed class helloandroid.MainActivity) and that class is what Java VM is going to interact with - every Managed overridden method (or property) will have a corresponding native Java method, such as n_onCreate above. You can find all the generated Java sources in the obj directory of your app, if you're curious how they look.

When the native method is called (in our example it is the Xamarin.Android app main activity's OnCreate method), Java searches for an exported symbol with that name in all the loaded shared libraries and, when found, determines whether or not it has already called any functions in the DSO containing the symbol. If the answer is "no", then the library is initialized - among others, JNI_OnLOad is called and then, eventually, the native method is invoked (in Xamarin.Android this will cause control to be transferred to the managed land, eventually).

[thenextman]

Hello @grendello. Thank you again for the troubleshooting and detailed info.

With logcat, it's strange because I do see detailed assembly loading and binding output for both managed and unmanaged libraries in the .apk. However MyLib (the only library I'm explicitly doing LoadLibrary), it gives no extra information:

12-06 02:50:30.971 28338 28379 D monodroid: typemap: Java type 'android/net/ConnectivityManager' corresponds to managed type 'Android.Net.ConnectivityManager, Mono.Android'
12-06 02:50:30.971 28338 28379 I monodroid-timing: [1/5] Typemap.java_to_managed: end, total time; elapsed: 0:0::66167
12-06 02:50:30.973 28338 28379 D monodroid: typemap: Java type 'android/net/NetworkInfo' corresponds to managed type 'Android.Net.NetworkInfo, Mono.Android'
12-06 02:50:30.973 28338 28379 I monodroid-timing: [1/5] Typemap.java_to_managed: end, total time; elapsed: 0:0::34708
12-06 02:50:30.988 28338 28379 D monodroid: typemap: Java type 'android/hardware/display/DisplayManager' corresponds to managed type 'Android.Hardware.Display.DisplayManager, Mono.Android'
12-06 02:50:30.988 28338 28379 I monodroid-timing: [1/5] Typemap.java_to_managed: end, total time; elapsed: 0:0::52667
12-06 02:50:31.257 28338 28379 I monodroid-timing: [1/6] Typemap.managed_to_java: end, total time; elapsed: 0:0::3459
12-06 02:50:31.258 28338 28379 I monodroid-timing: [1/6] Typemap.managed_to_java: end, total time; elapsed: 0:0::1083
12-06 02:50:31.258 28338 28379 I monodroid-timing: [1/10] Runtime.register: end time. Registered type: MyClass, MyApp; elapsed: 0:0::53917
12-06 02:50:31.264 28338 28379 D monodroid: typemap: Java type 'java/lang/UnsatisfiedLinkError' corresponds to managed type 'Java.Lang.UnsatisfiedLinkError, Mono.Android'
12-06 02:50:31.264 28338 28379 I monodroid-timing: [1/5] Typemap.java_to_managed: end, total time; elapsed: 0:0::68375
12-06 02:50:31.265 28338 28372 D OpenGLRenderer: endAllActiveAnimators on 0xb400007bb198b3a0 (MenuPopupWindow$MenuDropDownListView) with handle 0xb400007ac32d96b0
12-06 02:50:31.265 1226 1860 W InputManager-JNI: Input channel object 'a02716d PopupWindow:784f6e (client)' was disposed without first being removed with the input manager!
12-06 02:50:31.268 2462 2462 I GoogleInputMethodService: GoogleInputMethodService.onFinishInput():3220
12-06 02:50:31.268 2462 2462 I GoogleInputMethodService: GoogleInputMethodService.updateDeviceLockedStatus():2085 repeatCheckTimes = 0, unlocked = true
12-06 02:50:31.269 2462 2462 I GoogleInputMethodService: GoogleInputMethodService.onStartInput():1876 onStartInput(EditorInfo{inputType=0x0(NULL) imeOptions=0x0 privateImeOptions=null actionName=UNSPECIFIED actionLabel=null actionId=0 initialSelStart=-1 initialSelEnd=-1 initialCapsMode=0x0 hintText=null label=null packageName=MyApp fieldId=-1 fieldName=null extras=null}, false)
12-06 02:50:31.269 2462 2462 I GoogleInputMethodService: GoogleInputMethodService.shouldHideHeaderOnInitialState():3985 ShouldHideHeaderOnInitialState = false
12-06 02:50:31.269 2462 2462 I GoogleInputMethodService: GoogleInputMethodService.updateDeviceLockedStatus():2085 repeatCheckTimes = 2, unlocked = true
12-06 02:50:31.272 28338 28379 V mono-stdout: Java.Lang.UnsatisfiedLinkError: dlopen failed: library "libMyLib.so" not found
12-06 02:50:31.272 28338 28379 V mono-stdout: at Java.Interop.JniEnvironment+StaticMethods.CallStaticVoidMethod (Java.Interop.JniObjectReference type, Java.Interop.JniMethodInfo method, Java.Interop.JniArgumentValue* args) [0x00068] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/obj/Release/JniEnvironment.g.cs:13250
12-06 02:50:31.272 28338 28379 V mono-stdout: at Java.Interop.JniPeerMembers+JniStaticMethods.InvokeVoidMethod (System.String encodedMember, Java.Interop.JniArgumentValue* parameters) [0x00008] in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniPeerMembers.JniStaticMethods.cs:97
12-06 02:50:31.272 28338 28379 V mono-stdout: at Java.Lang.JavaSystem.LoadLibrary (System.String libname) [0x0001f] in /Users/runner/work/1/s/xamarin-android/src/Mono.Android/obj/Release/monoandroid10/android-31/mcw/Java.Lang.JavaSystem.cs:339
12-06 02:50:31.272 28338 28379 V mono-stdout: at [my class constructor]
12-06 02:50:31.272 28338 28379 V mono-stdout: --- End of managed Java.Lang.UnsatisfiedLinkError stack trace ---
12-06 02:50:31.272 28338 28379 V mono-stdout: java.lang.UnsatisfiedLinkError: dlopen failed: library "libMyLib.so" not found
12-06 02:50:31.272 28338 28379 V mono-stdout: at java.lang.Runtime.loadLibrary0(Runtime.java:1088)
12-06 02:50:31.272 28338 28379 V mono-stdout: at java.lang.Runtime.loadLibrary0(Runtime.java:998)
12-06 02:50:31.272 28338 28379 V mono-stdout: at java.lang.System.loadLibrary(System.java:1661)
12-06 02:50:31.272 28338 28379 V mono-stdout:
12-06 02:50:31.297 28338 28372 D EGL_emulation: app_time_stats: avg=34.36ms min=1.62ms max=1019.01ms count=45

I appreciate the extra explanation for the Java interop case. It's very clear. However, it won't apply in my case since I don't call native functions using JNI (no native declarations); rather, my native code wants to call Java. So we might call AttachCurrentThread, call some Java system libraries, then DetachCurrentThread. Of course this requires access to the JNI environment - hence my post here.

You originally wrote

just p/invoking into the library should work to trigger JNI_OnLoad

But to be clear, you're talking about native decorated functions, right? Since if I just make a p/invoke directly to C without involving Java, JNI isn't involved and it won't do any automatic bootstrapping of my library. Or is there some extra magic behind DllImport?

[grendello]

With logcat, it's strange because I do see detailed assembly loading and binding output for both managed and unmanaged libraries in the .apk. However MyLib (the only library I'm explicitly doing LoadLibrary), it gives no extra information:

For it to show up in one of our messages, you'd need to load it via Mono - by p/invoking some function in the library, then it would get loaded by our runtime instead of the Java shared library loader.

I appreciate the extra explanation for the Java interop case. It's very clear. However, it won't apply in my case since I don't call native functions using JNI (no native declarations); rather, my native code wants to call Java. So we might call AttachCurrentThread, call some Java system libraries, then DetachCurrentThread. Of course this requires access to the JNI environment - hence my post here.

OK, so if your DSO doesn't contain any native function implementations, then JNI_OnLoad won't ever be called, since Java doesn't care about the library. Off the top of my head I don't see a way to obtain JavaVM*, but I'll try to figure something out and update the issue once I have something.

You originally wrote

just p/invoking into the library should work to trigger JNI_OnLoad

But to be clear, you're talking about native decorated functions, right? Since if I just make a p/invoke directly to C without involving Java, JNI isn't involved and it won't do any automatic bootstrapping of my library. Or is there some extra magic behind DllImport?

Yes, I meant the Java "p/invoke", shouldn't have used this term, sorry for the confusion :)

[grendello]

It seems I was wrong - JNI_OnLoad should be called if you use LoadLibrary - the only question then is why LoadLibrary cannot find the DSO.

[grendello]

@thenextman let's try one other thing. Please issue the following command in addition to the previously mentioned adb commands, before starting the app:

$ adb shell setprop debug.ld.all dlopen,dlerror,dlsym

This should enable internal linker logging and we might see some errors regarding the library.

[jonpryor]

This should not be considered an appropriate fix or workaround -- the correct fix is to determine why JavaSystem.LoadLibrary("MyLib") is throwing an UnsatisfiedLinkError, which is really weird -- but there is a public API to get the JavaVM* value, via JniRuntime.InvocationPointer:

IntPtr javaVMPtr = global::Java.Interop.JniRuntime.CurrentRuntime.InvocationPointer;

[thenextman]

Thanks for the tip @jonpryor

@grendello Here is the output:

12-06 07:46:35.699 29838 29886 D linker : dlopen(name="libMyLib.so", flags=0x2, extinfo=[flags=0x200, reserved_addr=0x0, reserved_size=0x0, relro_fd=0, library_fd=0, library_fd_offset=0x0, library_namespace=(default)@0x7d65d4e4e0], caller="/apex/com.android.art/lib64/libnativebridge.so", caller_ns=com_android_art@0x7d64a18170, targetSdkVersion=31) ...
12-06 07:46:35.700 29838 29886 D linker : find_libraries(ns=(default)): task=libMyLib.so, is_dt_needed=0
12-06 07:46:35.700 29838 29886 D linker : load_library(ns=(default), task=libMyLib.so, flags=0x2, search_linked_namespaces=1): calling open_library
12-06 07:46:35.700 29838 29886 D linker : library "libMyLib.so" not found
12-06 07:46:35.700 29838 29886 D linker : find_library_internal(ns=(default), task=libMyLib.so): Trying 6 linked namespaces
12-06 07:46:35.700 29838 29886 D linker : find_library_in_linked_namespace(ns=com_android_adbd, task=libMyLib.so): Not accessible (soname=libMyLib.so)
12-06 07:46:35.700 29838 29886 D linker : find_library_in_linked_namespace(ns=com_android_i18n, task=libMyLib.so): Not accessible (soname=libMyLib.so)
12-06 07:46:35.700 29838 29886 D linker : find_library_in_linked_namespace(ns=com_android_art, task=libMyLib.so): Not accessible (soname=libMyLib.so)
12-06 07:46:35.700 29838 29886 D linker : find_library_in_linked_namespace(ns=com_android_resolv, task=libMyLib.so): Not accessible (soname=libMyLib.so)
12-06 07:46:35.700 29838 29886 D linker : find_library_in_linked_namespace(ns=com_android_neuralnetworks, task=libMyLib.so): Not accessible (soname=libMyLib.so)
12-06 07:46:35.700 29838 29886 D linker : find_library_in_linked_namespace(ns=com_android_os_statsd, task=libMyLib.so): Not accessible (soname=libMyLib.so)
12-06 07:46:35.700 29838 29886 D linker : ... dlopen failed: library "libMyLib.so" not found
12-06 07:46:35.700 29838 29886 D linker : dlerror set to "dlopen failed: library "libMyLib.so" not found"