[NativeAOT] support calling UnmanagedCallersOnly functions before managed main()

[AustinWise]

Problem statement

Some existing applications use the CoreCLR hosting APIs to call managed functions before calling the main method of an assembly. A specific example is xamarin-macios. It uses coreclr_create_delegate and reverse P/Invoke to call some functions before using coreclr_execute_assembly to execute an assembly's main function.

While trying to replicate the CoreCLR hosting API does not make sense for NativeAOT, the ability to call managed functions before main() would be useful.

Implementation Ideas

Currently ILC will generate one of two entry points: __managed__Main for executables and __managed__Startup for libraries. If both were generated, a hosting application could call functions in the following order:

1. Initialize the NativeAOT runtime using __managed__Startup
2. Call UnmanagedCallersOnly functions
3. Manuplicate argc and argv if desired.
4. Call __managed__Main to run the application.

When generating both entry points, __managed__Main should not run library or module initializers.

Ass a proof of concept, see this example adding support to ILC in this commit and then using those entry points with a custom bootstrapper to emulate enough of the CoreCLR hosting API to make xamarin-macios work.

Alternatives Considered

Conceivably hosts like xamarin-macios could use module constructors to run code before the main method, as NativeAOT eagerly runs these at startup. The only missing feature when using this approach is the ability to manipulate command line arguments before the main method starts.

Open Questions

• Should this capability only work when statically-linking the NativeAOT runtime? Should it also work when compiling the runtime and user EXE as shared library?
• What should the command line arguments for ILC look like
• How should the bootstrapper change to support these entry points.
• How should this feature be exposed in MSBuild to users

Related issues

• Improve static library publishing #70277
• [NativeAOT] Low level API support for Objective-C scenarios #77472

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[jkotas]

This sounds like an AOT variant of startup hook (https://github.com/dotnet/runtime/blob/main/docs/design/features/host-startup-hook.md).

[AustinWise]

This sounds like an AOT variant of startup hook (https://github.com/dotnet/runtime/blob/main/docs/design/features/host-startup-hook.md).

Basically, yes. Something like a start up hook won't let you manipulate the command line arguments before the managed main, but that's not required for the xamarin-macios use case I had in mind.

[MichalStrehovsky]

Basically, yes. Something like a start up hook won't let you manipulate the command line arguments before the managed main, but that's not required for the xamarin-macios use case I had in mind.

If we don't need the command line manipulation, can we do this using the module constructor approach? The only problem with that is that C# doesn't allow ordering the module constructor fragments, but if we do it from MSBuild we can have this to be the first file and that probably means it will be the first method to be called from the module constructor too.

[ivanpovazan]

I was wondering whether it would be possible to have:

1. A xamarin app, which wraps its Main with UnmanagedCallersOnly method
2. Compile it as a static library with NativeAOT
3. Adjust the xamarin startup (generated bootstrap main or xamarin_main) to invoke NativeAOT_StaticInitialization() provided by libbootstrapperdll.a, and then the mentioned UnmanagedCallersOnly wrapper of the managed app
4. Adjust the xamarin build to invoke ilc (step 2.) and also include the app's static library and NativeAOT bootstrapper when linking the native executable

Wouldn't this work?

[jkotas]

Yes, it is possible to achieve this by custom build steps that modify the app, e.g. by injecting a module constructor or wrapping the Main method.

Again, it looks similar to startup hook. It was always possible to approximate what startup hook does by writing a custom host or by modifying the app. We had multiple customers building complex fragile solutions to solve the problem. We have decided to introduce the startup hook feature to simplify the life of these customers (and ours too since we would often end up debugging issues in the complex fragile solutions).

So, the question is whether we believe that injecting code before Main is common enough scenario for native AOT to introduce a feature for it.

[MichalStrehovsky]

So, the question is whether we believe that injecting code before Main is common enough scenario for native AOT to introduce a feature for it.

What would the requirements look like? Startup hook allows introducing code before Main after the app was fully built. So e.g. one can use it to introduce new code into even dotnet.

A Native AOT equivalent would be to dlopen/dlsym (LoadLibrary/GetProcAddress) whatever was in the environment variable and p/invoke into it.

Or we could say we only want a compile-time startup hook. This is a lot less powerful than startup hooks. I have a hard time seeing difference from just adding a module initializer.

Neither of those would allow modifying the command line arguments though which looks like a requirement for Xamarin (and definitely a one-off that is hard to imagine a general purpose solution for).

[jkotas]

We could say we only want a compile-time startup hook. This is a lot less powerful than startup hooks. I have a hard time seeing difference from just adding a module initializer.

Module initializer requires appending a new .cs file to the app sources. It is certainly doable. I am not sure how hard it is to hide it so that it does not show as part of the solution in the GUI.

[MichalStrehovsky]

We could say we only want a compile-time startup hook. This is a lot less powerful than startup hooks. I have a hard time seeing difference from just adding a module initializer.

Module initializer requires appending a new .cs file to the app sources. It is certainly doable. I am not sure how hard it is to hide it so that it does not show as part of the solution in the GUI.

The default targets already generate two such files into obj for a hello world: a .NETCoreApp,Version=v7.0.AssemblyAttributes.cs and Foo.AssemblyInfo.cs so I would say there's enough of a precedent. Adding things to <Compile> within a <Target> is probably the thing that does the trick. This would work for all of our runtimes without any further work.

[AustinWise]

I think the idea of injecting a module constructor into the main program from the SDK would work fine.