Skip to content

Conversation

@jpobst
Copy link
Contributor

@jpobst jpobst commented Feb 28, 2025

Context: dotnet/android#9893

Create XML import/export adapters for JavaCallableWrappers. This allows us to serialize the JCWs we find in the FindJavaObjectsStep linker step to disk that is later used in _GenerateJavaStubs. This file also allows us to cache the assembly scanning to disk so we don't need to rerun it on incremental builds.

This format is likely a "work in progress". As we move other JLO related scanning to linker steps we will likely find additional data they require that needs to go into this file, or that this format isn't even suitable and we use something else instead.

@jpobst jpobst force-pushed the dev/jpobst/jcw-adapters branch 4 times, most recently from a734d88 to 7daad70 Compare March 10, 2025 08:01
@jpobst jpobst marked this pull request as ready for review March 10, 2025 18:37
static void ExportTypes (XmlWriter xml, IEnumerable<CallableWrapperType> types, bool wasScanned)
{
xml.WriteStartElement ("types");
xml.WriteAttributeString ("was_scanned", wasScanned.ToString ());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the semantic/meaning of was_scanned?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a temporary construct to facilitate comparing output with the original GenerateJavaStubs process to the new one. It was very tricky to make sure we are scanning the same set of assemblies for JLOs as the original process based on all the various build properties that could be in play.

This attribute denotes that we scanned the assembly for JLOs, even if this file was empty because we didn't find any that needed JCWs.

Example:

  • true for a Xamarin.AndroidX.Activity assembly that we scanned but didn't find anything that needed a JCW
  • false for a System.Collections assembly that we did not waste time scanning because we knew it couldn't contain any JLOs.

(We always write an output file, even for things we didn't scan, so that incremental builds have something to compare against to see if the assembly changed.)


public static void ExportType (XmlWriter xml, CallableWrapperType type)
{
xml.WriteStartElement ("type");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

…and now it's bike shedding time (?).

Should this "rhyme with" Our Favorite XML Format™, class-parse output? That's what tools/java-source-utils does when extracting Javadoc documentation, e.g.

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<api api-source="java-source-utils">
<package jni-name="" name="">
<class jni-signature="LA;" name="A">
<javadoc>
<![CDATA[jni-sig=LA;]]>
</javadoc>
<constructor jni-signature="(ILjava/lang/String;)V">
<parameter jni-type="I" name="one" type="int"/>
<parameter jni-type="Ljava/lang/String;" name="two" type="java.lang.String"/>
<javadoc>
<![CDATA[jni-sig=<init>.(ILjava/lang/String;)V]]>
</javadoc>
</constructor>
<field jni-signature="I" name="field">
<javadoc>
<![CDATA[jni-sig=field.I]]>
</javadoc>
</field>
<method jni-return="V" jni-signature="(Ljava/lang/Object;J)V" name="m" return="void">
<parameter jni-type="Ljava/lang/Object;" name="value" type="T"/>
<parameter jni-type="J" name="x" type="long"/>
<javadoc>
<![CDATA[jni-sig=m.(Ljava/lang/Object;J)V]]>
</javadoc>
</method>
</class>

Perhaps something like:

<api api-source="jcw">
  <package name="crc64…">
    <class name="ExampleType" abstract="true" extends="java.lang.Object"
        jcw.HasExport="false" jcw.IsApplication="false" …>

@jonpryor
Copy link
Contributor

This should also add a test to tests/Java.Interop.Tools.JavaCallableWrappers-Tests which at minimum takes an input assembly (bin/TestRelease-net8.0/Java.Base-Tests.dll?), generates the XML output, and parses it. At minimum we want to make sure that the output can be consumed. ("Better" would be to verify that the XML output contains at least some useful/expected data, and the parsed values correspond to the intermediate XML…)

@jpobst
Copy link
Contributor Author

jpobst commented Mar 10, 2025

Should this "rhyme with" Our Favorite XML Format™, class-parse output?

This should also add a test...

Unfortunately, I think this file format is currently very much a rough draft. The other processes we need to convert (type maps, acw map, manifest merging) will each require additional JLO information that is unknown at this time. This may be additional data in this file format or separate files. As such, the finish line may look very different than what this PR has (this may not even be suitable for the final format and we scrap this). I propose we postpone the "productization" parts (format shape, tests) until we have a clearer picture.

Piggybacking on JavaCallableWrappers was simply the easiest was to start.

<PropertyGroup>
<TargetFramework>$(DotNetTargetFramework)</TargetFramework>
<TargetFrameworks>netstandard2.0</TargetFrameworks>
<LangVersion>12</LangVersion>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this change moot with c1cd06d?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this .csproj change should have been reverted, fixed.

@jonpryor
Copy link
Contributor

For internal commit ("documentation") purposes, please provide example XML of what's generated, along with docs of anything that isn't obvious within it.

@jpobst jpobst force-pushed the dev/jpobst/jcw-adapters branch from c742df6 to d870aff Compare March 17, 2025 19:12
@jpobst jpobst force-pushed the dev/jpobst/jcw-adapters branch from d870aff to 841695e Compare March 17, 2025 19:15
@jpobst
Copy link
Contributor Author

jpobst commented Mar 17, 2025

For internal commit ("documentation") purposes, please provide example XML of what's generated, along with docs of anything that isn't obvious within it.

Sample XML:

  <types was_scanned="true">
    <type name="MainActivity" package="crc645107ba1b8b6ee4d3" application_java_class="android.app.Application" mono_runtime_initialization="mono.MonoPackageManager.LoadApplication (context);" extends_type="android.app.Activity" partial_assembly_qualified_name="tempbuild.MainActivity, tempbuild">
      <constructors>
        <constructor name="MainActivity" method="n_.ctor:()V:" jni_signature="()V" managed_parameters="" retval="void" is_dynamically_registered="True" />
      </constructors>
      <methods>
        <method name="clone" method="n_clone:()Ljava/lang/Object;:GetCloneHandler" jni_signature="()Ljava/lang/Object;" retval="java.lang.Object" />
        <method name="onCreate" method="n_onCreate:(Landroid/os/Bundle;)V:GetOnCreate_Landroid_os_Bundle_Handler" jni_signature="(Landroid/os/Bundle;)V" params="android.os.Bundle p0" retval="void" super_call="p0" activate_call="p0" />
      </methods>
    </type>
  </types>

Format should be self-explanatory, it should be a 1:1 serialization of CallableWrapperType and friends.

@jonpryor
Copy link
Contributor

Context: https://github.com/dotnet/android/pull/9893

Create XML import/export adapters for Java Callable Wrappers (JCWs).
This allows us to serialize the JCWs we find in the new
`FindJavaObjectsStep` linker step (dotnet/android#9893) to disk that,
and later use that serialized data in `_GenerateJavaStubs`.

This file also allows us to cache the assembly scanning to disk so we
don't need to rerun it on incremental builds.

This format is likely a "work in progress".  As we move other
`Java.Lang.Object`-related scanning to linker steps we will likely find
additional data they require that needs to go into this file, or that
this format isn't even suitable and we use something else instead.

Sample JCW XML content:

	<types was_scanned="true">
	  <type
	      name="MainActivity"
	      package="crc645107ba1b8b6ee4d3"
	      application_java_class="android.app.Application"
	      mono_runtime_initialization="mono.MonoPackageManager.LoadApplication (context);"
	      extends_type="android.app.Activity"
	      partial_assembly_qualified_name="tempbuild.MainActivity, tempbuild">
	    <constructors>
	      <constructor
	        name="MainActivity"
	        method="n_.ctor:()V:"
	        jni_signature="()V"
	        managed_parameters=""
	        retval="void"
	        is_dynamically_registered="True" />
	    </constructors>
	    <methods>
	      <method
	        name="clone"
	        method="n_clone:()Ljava/lang/Object;:GetCloneHandler"
	        jni_signature="()Ljava/lang/Object;"
	        retval="java.lang.Object" />
	      <method
	        name="onCreate"
	        method="n_onCreate:(Landroid/os/Bundle;)V:GetOnCreate_Landroid_os_Bundle_Handler"
	        jni_signature="(Landroid/os/Bundle;)V"
	        params="android.os.Bundle p0"
	        retval="void"
	        super_call="p0"
	        activate_call="p0" />
	    </methods>
	  </type>
	</types>

Most XML attributes correspond to members on `CallableWrapperType`,
`CallableWrapperMethod`, etc.

The XML file is a *per-assembly* data store; `//types/@was_scanned`
specifies whether or not the assembly was scanned for JCWs.
If the assembly did not contain JCWs (e.g.
`System.Private.CoreLib.dll`), then `was_scanned` will be False.

@jpobst jpobst merged commit bc44f08 into main Mar 17, 2025
2 checks passed
@jpobst jpobst deleted the dev/jpobst/jcw-adapters branch March 17, 2025 21:49
@github-actions github-actions bot locked and limited conversation to collaborators Apr 17, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants