[illink] Redo Java.Lang.Object serialization (#5473)

radekdoulik · web-flow · commit 101fea2caa89 · 2021-01-12T16:24:53.000+01:00
Context: #5454 The use of `DataContract` attribute on `Java.Lang.Object` class pulls in `System.Runtime.Serialization.Primitives` assembly. We still need to make the `Java.Lang.Object` class serializable, so that inherited types can be serialized. Otherwise that would lead to `System.Runtime.Serialization.InvalidDataContractException` during runtime. Instead of decorating `Java.Lang.Object` with `DataContract` attribute, use `Serializable` attribute on class and `NonSerialized` attribute on fields. This might seem contraproductive, to use multiple attribute instances instead of one. In real it is not and leads to size saving and getting rid of `System.Runtime.Serialization.Primitives` assembly reference in linked apps. Because the `Serializable` and `NonSerialized` attributes are not present in the compiled output, instead flags are used. The compiled `Java.Lang.Object` class IL looks like this: ``` .class public auto ansi serializable beforefieldinit Java.Lang.Object extends [System.Private.CoreLib]System.Object implements [System.Private.CoreLib]System.IDisposable, Android.Runtime.IJavaObject, Java.Interop.IJavaObjectEx, [Java.Interop]Java.Interop.IJavaPeerable { .custom instance void Android.Runtime.RegisterAttribute::.ctor(string) = ( 01 00 10 6A 61 76 61 2F 6C 61 6E 67 2F 4F 62 6A // ...java/lang/Obj 65 63 74 01 00 54 02 10 44 6F 4E 6F 74 47 65 6E // ect..T..DoNotGen 65 72 61 74 65 41 63 77 01 ) // erateAcw. .field private static initonly class [Java.Interop]Java.Interop.JniPeerMembers _members .field private static class [System.Private.CoreLib]System.Delegate cb_equals_Ljava_lang_Object_ .field private static class [System.Private.CoreLib]System.Delegate cb_hashCode .field private static class [System.Private.CoreLib]System.Delegate cb_toString .field private notserialized native int key_handle .field private notserialized native int weak_handle .field private notserialized int32 refs_added .field private notserialized valuetype Android.Runtime.JObjectRefType handle_type .field private notserialized native int handle .field private notserialized bool needsActivation .field private notserialized bool isProxy ... ``` The apk size savings on net6 in BuildReleaseArm64False test: ``` > apkdiff -md -e dll$ before.apk after.apk Size difference in bytes ([*1] apk1 only, [*2] apk2 only): - 13 assemblies/Mono.Android.dll - 120 Metadata - 60 Stream #~ (tables) - 6 Table TypeRef - 6 Table MemberRef - 6 Table CustomAttribute - 40 Table AssemblyRef - 60 Stream #Strings Type Java.Lang.Object - CustomAttribute System.Runtime.Serialization.DataContractAttribute - 2,583 assemblies/System.Runtime.Serialization.Primitives.dll *1 Summary: - 2,596 Assemblies -0.28% (of 915,384) ``` I have also added a new test for `Java.Lang.Object` class serialization with code from old `Hello` sample. Example of `System.Runtime.Serialization.InvalidDataContractException` during runtime, if `Java.Lang.Object` class was not serializable: ``` I MonoDroid: Android.Runtime.JavaProxyThrowable: Exception of type 'Android.Runtime.JavaProxyThrowable' was thrown. I MonoDroid: --- End of managed Android.Runtime.JavaProxyThrowable stack trace --- I MonoDroid: android.runtime.JavaProxyThrowable: System.Runtime.Serialization.InvalidDataContractException: Type 'UnnamedProject.Person' cannot inherit from a type that is not marked with DataContractAttribute or SerializableAttribute. Consider marking the base type 'Java.Lang.Object' with DataContractAttribute or SerializableAttribute, or removing them from the derived type. I MonoDroid: at System.Runtime.Serialization.ClassDataContract+ClassDataContractCriticalHelper..ctor (System.Type type) [0x001c3] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.ClassDataContract..ctor (System.Type type) [0x00000] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.DataContract+DataContractCriticalHelper.CreateDataContract (System.Int32 id, System.RuntimeTypeHandle typeHandle, System.Type type) [0x000e0] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.DataContract+DataContractCriticalHelper.GetDataContractSkipValidation (System.Int32 id, System.RuntimeTypeHandle typeHandle, System.Type type) [0x0000b] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.DataContract.GetDataContractSkipValidation (System.Int32 id, System.RuntimeTypeHandle typeHandle, System.Type type) [0x00000] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.DataContract.GetDataContract (System.Int32 id, System.RuntimeTypeHandle typeHandle, System.Runtime.Serialization.SerializationMode mode) [0x00000] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.DataContract.GetDataContract (System.RuntimeTypeHandle typeHandle, System.Type type, System.Runtime.Serialization.SerializationMode mode) [0x00006] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.DataContract.GetDataContract (System.Type type) [0x00006] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.Json.DataContractJsonSerializer.get_RootContract () [0x00022] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalWriteObjectContent (System.Runtime.Serialization.XmlWriterDelegator writer, System.Object graph) [0x00031] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalWriteObject (System.Runtime.Serialization.XmlWriterDelegator writer, System.Object graph) [0x00008] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.XmlObjectSerializer.InternalWriteObject (System.Runtime.Serialization.XmlWriterDelegator writer, System.Object graph, System.Runtime.Serialization.DataContractResolver dataContractResolver) [0x00000] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions (System.Runtime.Serialization.XmlWriterDelegator writer, System.Object graph, System.Runtime.Serialization.DataContractResolver dataContractResolver) [0x00073] in <686be187480b41979dcbf5635f805a7b>:0 01-07 16:13:09.246 10624 10624 I MonoDroid: at System.Runtime.Serialization.XmlObjectSerializer.WriteObjectHandleExceptions (System.Runtime.Serialization.XmlWriterDelegator writer, System.Object graph) [0x00000] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.Json.DataContractJsonSerializer.WriteObject (System.Xml.XmlDictionaryWriter writer, System.Object graph) [0x0000d] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at System.Runtime.Serialization.Json.DataContractJsonSerializer.WriteObject (System.IO.Stream stream, System.Object graph) [0x00018] in <686be187480b41979dcbf5635f805a7b>:0 I MonoDroid: at UnnamedProject.MainActivity.TestJsonDeserializationCreatesJavaHandle () [0x00033] in <11a2c65ab60d41dea720c7638d552ce0>:0 I MonoDroid: at UnnamedProject.MainActivity.OnCreate (Android.OS.Bundle bundle) [0x0004b] in <11a2c65ab60d41dea720c7638d552ce0>:0 01-07 16:13:09.247 10624 10624 I MonoDroid: at Android.App.Activity.n_OnCreate_Landroid_os_Bundle_ (System.IntPtr jnienv, System.IntPtr native__this, System.IntPtr native_savedInstanceState) [0x00012] in <3045e86f0ced43c19ee1b522f77c72d7>:0 I MonoDroid: at Android.App.Activity.n_OnCreate_Landroid_os_Bundle_ (System.IntPtr jnienv, System.IntPtr native__this, System.IntPtr native_savedInstanceState) [0x00012] in <3045e86f0ced43c19ee1b522f77c72d7>:0 I MonoDroid: at (wrapper dynamic-method) Android.Runtime.DynamicMethodNameCounter.1(intptr,intptr,intptr) I MonoDroid: at unnamedproject.unnamedproject.MainActivity.n_onCreate(Native Method) I MonoDroid: at unnamedproject.unnamedproject.MainActivity.onCreate(MainActivity.java:29) I MonoDroid: at android.app.Activity.performCreate(Activity.java:8000) I MonoDroid: at android.app.Activity.performCreate(Activity.java:7984) I MonoDroid: at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1309) I MonoDroid: at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:3404) I MonoDroid: at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:3595) I MonoDroid: at android.app.servertransaction.LaunchActivityItem.execute(LaunchActivityItem.java:85) I MonoDroid: at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135) I MonoDroid: at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95) I MonoDroid: at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2066) I MonoDroid: at android.os.Handler.dispatchMessage(Handler.java:106) I MonoDroid: at android.os.Looper.loop(Looper.java:223) I MonoDroid: at android.app.ActivityThread.main(ActivityThread.java:7660) I MonoDroid: at java.lang.reflect.Method.invoke(Native Method) I MonoDroid: at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592) I MonoDroid: at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947) I MonoDroid: W ActivityTaskManager: Force finishing activity UnnamedProject.UnnamedProject/unnamedproject.unnamedproject.MainActivity ```
diff --git a/src/Mono.Android/Java.Lang/Object.cs b/src/Mono.Android/Java.Lang/Object.cs
@@ -12,21 +12,21 @@
 
 namespace Java.Lang {
 
-	[DataContract]
+	[Serializable]
 	public partial class Object : IDisposable, IJavaObject, IJavaObjectEx
 #if JAVA_INTEROP
 		, IJavaPeerable
 #endif  // JAVA_INTEROP
 	{
-		IntPtr key_handle;
+		[NonSerialized] IntPtr key_handle;
 #pragma warning disable CS0649, CS0169, CS0414 // Suppress fields are never used warnings, these fields are used directly by monodroid-glue.cc
-		IntPtr weak_handle;
-		int refs_added;
+		[NonSerialized] IntPtr weak_handle;
+		[NonSerialized] int refs_added;
 #pragma warning restore CS0649, CS0169, CS0414
-		JObjectRefType handle_type;
-		IntPtr handle;
-		bool             needsActivation;
-		bool             isProxy;
+		[NonSerialized] JObjectRefType handle_type;
+		[NonSerialized] IntPtr handle;
+		[NonSerialized] bool             needsActivation;
+		[NonSerialized] bool             isProxy;
 
 		IntPtr IJavaObjectEx.KeyHandle {
 			get {return key_handle;}
diff --git a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingTest.cs b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.Android.Build.Tests/PackagingTest.cs
@@ -89,7 +89,6 @@ public void CheckIncludedAssemblies ()
 					"System.Console.dll",
 					"System.Linq.Expressions.dll",
 					"System.ObjectModel.dll",
-					"System.Runtime.Serialization.Primitives.dll",
 					"System.Private.CoreLib.dll",
 					"System.Collections.Concurrent.dll",
 					"System.Collections.dll",
@@ -103,7 +102,6 @@ public void CheckIncludedAssemblies ()
 					"System.Core.dll",
 					"System.Data.dll",
 					"System.dll",
-					"System.Runtime.Serialization.dll",
 					"UnnamedProject.dll",
 					"Mono.Data.Sqlite.dll",
 					"Mono.Data.Sqlite.dll.config",
diff --git a/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Resources/Base/MainActivity.cs b/src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Resources/Base/MainActivity.cs
@@ -32,6 +32,7 @@ protected override void OnCreate (Bundle bundle)
 			//${AFTER_ONCREATE}
 		}
 	}
+	//${AFTER_MAINACTIVITY}
 }
 
 
diff --git a/tests/MSBuildDeviceIntegration/Tests/InstallAndRunTests.cs b/tests/MSBuildDeviceIntegration/Tests/InstallAndRunTests.cs
@@ -419,5 +419,75 @@ public class LinkModeFullClass {
 			}
 		}
 
+		[Test]
+		public void JsonDeserializationCreatesJavaHandle ([Values (false, true)] bool isRelease)
+		{
+			AssertHasDevices ();
+
+			proj = new XamarinAndroidApplicationProject () {
+				IsRelease = isRelease,
+			};
+
+			if (isRelease || !CommercialBuildAvailable) {
+				proj.SetAndroidSupportedAbis ("armeabi-v7a", "arm64-v8a", "x86");
+			}
+
+			proj.References.Add (new BuildItem.Reference ("System.Runtime.Serialization"));
+
+			if (Builder.UseDotNet)
+				proj.References.Add (new BuildItem.Reference ("System.Runtime.Serialization.Json"));
+
+			proj.MainActivity = proj.DefaultMainActivity.Replace ("//${AFTER_ONCREATE}",
+				@"TestJsonDeserializationCreatesJavaHandle();
+		}
+
+		void TestJsonDeserializationCreatesJavaHandle ()
+		{
+			Person p = new Person () {
+				Name = ""John Smith"",
+				Age = 900,
+			};
+			var stream      = new MemoryStream ();
+			var serializer  = new DataContractJsonSerializer (typeof (Person));
+
+			serializer.WriteObject (stream, p);
+
+			stream.Position = 0;
+			StreamReader sr = new StreamReader (stream);
+
+			Console.WriteLine ($""JSON Person representation: {sr.ReadToEnd ()}"");
+
+			stream.Position = 0;
+			Person p2 = (Person) serializer.ReadObject (stream);
+
+			Console.WriteLine ($""JSON Person parsed: Name '{p2.Name}' Age '{p2.Age}' Handle '0x{p2.Handle:X}'"");
+
+			if (p2.Name != ""John Smith"")
+				throw new InvalidOperationException (""JSON deserialization of Name"");
+			if (p2.Age != 900)
+				throw new InvalidOperationException (""JSON deserialization of Age"");
+			if (p2.Handle == IntPtr.Zero)
+				throw new InvalidOperationException (""Failed to instantiate new Java instance for Person!"");
+
+			Console.WriteLine ($""JSON Person deserialized OK"");").Replace ("//${AFTER_MAINACTIVITY}", @"
+	[DataContract]
+	class Person : Java.Lang.Object {
+		[DataMember]
+		public string Name;
+
+		[DataMember]
+		public int Age;
+	}").Replace ("using System;", @"using System;
+using System.IO;
+using System.Runtime.Serialization;
+using System.Runtime.Serialization.Json;");
+			builder = CreateApkBuilder ();
+			Assert.IsTrue (builder.Install (proj), "Install should have succeeded.");
+			ClearAdbLogcat ();
+			AdbStartActivity ($"{proj.PackageName}/{proj.JavaPackageName}.MainActivity");
+			Assert.IsFalse (MonitorAdbLogcat ((line) => {
+				return line.Contains ("InvalidOperationException");
+			}, Path.Combine (Root, builder.ProjectDirectory, "startup-logcat.log"), 45), $"Output did contain InvalidOperationException!");
+		}
 	}
 }
diff --git a/tests/api-compatibility/acceptable-breakages-vReference.txt b/tests/api-compatibility/acceptable-breakages-vReference.txt
@@ -22,3 +22,4 @@ CannotRemoveAttribute : Attribute 'System.ComponentModel.CategoryAttribute' exis
 CannotRemoveAttribute : Attribute 'System.ComponentModel.CategoryAttribute' exists on 'Android.Content.ContentProviderAttribute.Icon' in the contract but not the implementation.
 CannotRemoveAttribute : Attribute 'System.ComponentModel.CategoryAttribute' exists on 'Android.Content.ContentProviderAttribute.Label' in the contract but not the implementation.
 CannotRemoveAttribute : Attribute 'System.ComponentModel.CategoryAttribute' exists on 'Android.Content.ContentProviderAttribute.RoundIcon' in the contract but not the implementation.
+CannotRemoveAttribute : Attribute 'System.Runtime.Serialization.DataContractAttribute' exists on 'Java.Lang.Object' in the contract but not the implementation.

Original file line number	Diff line number	Diff line change
`@@ -32,6 +32,7 @@ protected override void OnCreate (Bundle bundle)`
`32`	`32`	`//${AFTER_ONCREATE}`
`33`	`33`	`}`
`34`	`34`	`}`
	`35`	`+ //${AFTER_MAINACTIVITY}`
`35`	`36`	`}`
`36`	`37`
`37`	`38`