From dd36d6cb991b880dd01c4087811c8abe4ea54872 Mon Sep 17 00:00:00 2001 From: Jonathan Pryor Date: Thu, 16 Apr 2020 13:58:56 -0400 Subject: [PATCH] [build] Support building with JetBrains OpenJDK 11 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Context: https://issuetracker.google.com/issues/150189789 Context: https://github.com/xamarin/xamarin-android/pull/4562 Context: https://github.com/xamarin/xamarin-android/pull/4567 Bumps to xamarin/xamarin-android-tools/master@36d7fee5 Changes: https://github.com/xamarin/xamarin-android-tools/compare/bfb66f38b77421ac373c06a5f8407e1f9c793cc8...36d7fee5f7e0ef6ffa28970687db85b36947a3e6 * xamarin/xamarin-android-tools@36d7fee: JetBrains OpenJDK 11 detection (#82) * xamarin/xamarin-android-tools@12f52ac: Merge pull request #80 from jonpryor/jonp-drop-net461 * xamarin/xamarin-android-tools@c7090d0: [Xamarin.Android.Tools.AndroidSdk] Remove net461 JDK 9 -- released 2017-July-27 -- introduced many new features, but broke various Android SDK toolchain programs in various inscrutable ways, so the Android community has been "stuck" on JDK 8 ever since. …until now? A preview version of `apksigner` in the Build-tools 30rc1 package states that it requires Java 9 in order to run, which means we must explore what is required to build under JDK > 8. [JetBrains has an OpenJDK 11.0.4 release for macOS][0], which has a "weird" directory structure but is otherwise workable, so… Will It Build™? $ curl -o jbrsdk-11_0_4-osx-x64-b546.1.tar.gz https://bintray.com/jetbrains/intellij-jdk/download_file?file_path=jbrsdk-11_0_4-osx-x64-b546.1.tar.gz # Above doesn't *actually* work; use a browser to appease Akamai $ tar xzf jbrsdk-11_0_4-osx-x64-b546.1.tar.gz $ export JAVA_HOME=$HOME/Downloads/jbrsdk/Contents/Home $ make prepare JI_MAX_JDK=12 $ make all Yes, it builds, but that's *misleading*: it's not actually using `$JAVA_HOME`! …/Java.Interop/build-tools/scripts/jdk.targets(5,5): warning : Not a valid JDK directory: `…/Java.Interop/jbrsdk/Contents/Home`; via locator: $JAVA_HOME System.ArgumentException: Could not find required file `jvm` within `…/Java.Interop/jbrsdk/Contents/Home`; is this a valid JDK? Parameter name: homePath at Xamarin.Android.Tools.JdkInfo.ValidateFile (System.String name, System.String path) at Xamarin.Android.Tools.JdkInfo..ctor (System.String homePath) This is fixed via xamarin/xamarin-android-tools@36d7fee. Bump xamarin-android-tools, and `make prepare` still works. `make all` fails: $ make all … "…/Java.Interop/jbrsdk/Contents/Home/bin/javac" -parameters -source 1.6 -target 1.6 -bootclasspath "…/Java.Interop/jbrsdk/Contents/Home/bin/../jre/lib/rt.jar" -g -d "obj/Debug/classes" java/android/annotation/NonNull.java java/android/annotation/NonNull.java java/com/xamarin/IJavaInterface.java java/com/xamarin/IParameterInterface.java java/com/xamarin/JavaAnnotation.java java/com/xamarin/JavaType.java java/com/xamarin/NestedInterface.java java/com/xamarin/NotNullClass.java java/com/xamarin/ParameterAbstractClass.java java/com/xamarin/ParameterClass.java java/com/xamarin/ParameterClass2.java java/java/util/Collection.java java/NonGenericGlobalType.java EXEC : warning : [options] source value 6 is obsolete and will be removed in a future release EXEC : warning : [options] target value 1.6 is obsolete and will be removed in a future release EXEC : warning : -parameters is not supported for target value 1.6. Use 1.8 or later. EXEC : warning : [options] To suppress warnings about obsolete options, use -Xlint:-options. 4 warnings EXEC : Fatal error : Unable to find package java.lang in classpath or bootclasspath The problem is that JetBrains' OpenJDK 11 no longer contains a `…/jre/lib/rt.jar` file, so the `-bootclasspath` value is now wrong. Additionally, if you don't use `javac -target`, which implicitly targets JDK 11, `javac` doesn't like that: EXEC : error : option --boot-class-path not allowed with target 11 The solution? Don't Do That™; if `javac` from OpenJDK 11 doesn't want `-bootclasspath`, don't provide it. Update the `` task to check for the existence of the `…/jre/lib/rt.jar` file; if it exists, set `$(JreRtJarPath)`, otherwise the `$(JreRtJarPath)` MSBuild property is empty. Then update `$(_JavacSourceOptions)` so that it *doesn't* provide `-bootclasspath` when `$(JreRtJarPath)` is empty. These three changes -- xamarin-android-tools bump, `` update, and `$(_JavacSourceOptions)` update -- allow `make all` to build successfully. Then we hit *unit* tests. The above `javac` invocation has a warning: EXEC : warning : -parameters is not supported for target value 1.6. Use 1.8 or later. Even though we've been using `javac -target 1.6 -parameters` for *ages*, this combination is no longer supported. In order to use `javac -parameters` now, we need to use `javac -target 1.8`. Update `$(JavacSourceVersion)` to 1.8 so that we can continue using `javac -parameters`, which in turn requires that we update `tests/Xamarin.Android.Tools.Bytecode-Tests` so that we expect the newly updated `.class` file MajorVersion values. [0]: https://bintray.com/jetbrains/intellij-jdk/download_file?file_path=jbrsdk-11_0_4-osx-x64-b546.1.tar.gz --- Directory.Build.props | 8 ++--- .../Java.Interop.BootstrapTasks/JdkInfo.cs | 30 +++++++++++++------ external/xamarin-android-tools | 2 +- .../IJavaInterfaceTests.cs | 2 +- .../JavaAnnotationTests.cs | 2 +- .../JavaEnumTests.cs | 2 +- .../JavaType.1Tests.cs | 2 +- .../JavaType.ASCTests.cs | 2 +- .../JavaType.PSCTests.cs | 2 +- .../JavaType.RNC.RPNCTests.cs | 2 +- .../JavaType.RNCTests.cs | 2 +- .../JavaTypeTests.cs | 2 +- .../NonGenericGlobalTypeTests.cs | 2 +- 13 files changed, 36 insertions(+), 24 deletions(-) diff --git a/Directory.Build.props b/Directory.Build.props index 47772855b..521e68a47 100644 --- a/Directory.Build.props +++ b/Directory.Build.props @@ -43,10 +43,10 @@ $(MSBuildThisFileDirectory)external\xamarin-android-tools - 1.6 - 1.6 - $([System.IO.Path]::GetDirectoryName('$(JavaCPath)'))\..\jre\lib\rt.jar - <_JavacSourceOptions>-source $(JavacSourceVersion) -target $(JavacTargetVersion) -bootclasspath "$(JreRtJarPath)" + 1.8 + 1.8 + <_BootClassPath Condition=" '$(JreRtJarPath)' != '' ">-bootclasspath "$(JreRtJarPath)" + <_JavacSourceOptions>-source $(JavacSourceVersion) -target $(JavacTargetVersion) $(_BootClassPath) <_XamarinAndroidCecilPath Condition=" '$(CecilSourceDirectory)' != '' And Exists('$(UtilityOutputFullPath)Xamarin.Android.Cecil.dll') ">$(UtilityOutputFullPath)Xamarin.Android.Cecil.dll diff --git a/build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks/JdkInfo.cs b/build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks/JdkInfo.cs index 028d35f7c..ea789d86b 100644 --- a/build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks/JdkInfo.cs +++ b/build-tools/Java.Interop.BootstrapTasks/Java.Interop.BootstrapTasks/JdkInfo.cs @@ -17,10 +17,6 @@ namespace Java.Interop.BootstrapTasks { public class JdkInfo : Task { - const string JARSIGNER = "jarsigner.exe"; - const string MDREG_KEY = @"SOFTWARE\Novell\Mono for Android"; - const string MDREG_JAVA_SDK = "JavaSdkDirectory"; - public string JdksRoot { get; set; } public string MaximumJdkVersion { get; set; } @@ -50,13 +46,18 @@ public override bool Execute () return false; } + var rtJarPaths = new[]{ + Path.Combine (Path.GetDirectoryName (jdk.JavacPath), "..", "jre", "lib", "rt.jar"), + }; + var rtJarPath = rtJarPaths.FirstOrDefault (p => File.Exists (p)); + JavaHomePath = jdk.HomePath; Directory.CreateDirectory (Path.GetDirectoryName (PropertyFile.ItemSpec)); Directory.CreateDirectory (Path.GetDirectoryName (MakeFragmentFile.ItemSpec)); - WritePropertyFile (jdk.JarPath, jdk.JavacPath, jdk.JdkJvmPath, jdk.IncludePath); - WriteMakeFragmentFile (jdk.JarPath, jdk.JavacPath, jdk.JdkJvmPath, jdk.IncludePath); + WritePropertyFile (jdk.JarPath, jdk.JavacPath, jdk.JdkJvmPath, rtJarPath, jdk.IncludePath); + WriteMakeFragmentFile (jdk.JarPath, jdk.JavacPath, jdk.JdkJvmPath, rtJarPath, jdk.IncludePath); return !Log.HasLoggedErrors; } @@ -92,7 +93,7 @@ Action CreateLogger () return logger; } - void WritePropertyFile (string jarPath, string javacPath, string jdkJvmPath, IEnumerable includes) + void WritePropertyFile (string jarPath, string javacPath, string jdkJvmPath, string rtJarPath, IEnumerable includes) { var msbuild = XNamespace.Get ("http://schemas.microsoft.com/developer/msbuild/2003"); var project = new XElement (msbuild + "Project", @@ -106,17 +107,28 @@ void WritePropertyFile (string jarPath, string javacPath, string jdkJvmPath, IEn new XElement (msbuild + "JavaCPath", new XAttribute ("Condition", " '$(JavaCPath)' == '' "), javacPath), new XElement (msbuild + "JarPath", new XAttribute ("Condition", " '$(JarPath)' == '' "), - jarPath))); + jarPath), + CreateJreRtJarPath (msbuild, rtJarPath))); project.Save (PropertyFile.ItemSpec); } - void WriteMakeFragmentFile (string jarPath, string javacPath, string jdkJvmPath, IEnumerable includes) + static XElement CreateJreRtJarPath (XNamespace msbuild, string rtJarPath) + { + if (rtJarPath == null) + return null; + return new XElement (msbuild + "JreRtJarPath", + new XAttribute ("Condition", " '$(JreRtJarPath)' == '' "), + rtJarPath); + } + + void WriteMakeFragmentFile (string jarPath, string javacPath, string jdkJvmPath, string rtJarPath, IEnumerable includes) { using (var o = new StreamWriter (MakeFragmentFile.ItemSpec)) { o.WriteLine ($"export JI_JAR_PATH := {jarPath}"); o.WriteLine ($"export JI_JAVAC_PATH := {javacPath}"); o.WriteLine ($"export JI_JDK_INCLUDE_PATHS := {string.Join (" ", includes)}"); o.WriteLine ($"export JI_JVM_PATH := {jdkJvmPath}"); + o.WriteLine ($"export JI_RT_JAR_PATH := {rtJarPath}"); } } } diff --git a/external/xamarin-android-tools b/external/xamarin-android-tools index bfb66f38b..36d7fee5f 160000 --- a/external/xamarin-android-tools +++ b/external/xamarin-android-tools @@ -1 +1 @@ -Subproject commit bfb66f38b77421ac373c06a5f8407e1f9c793cc8 +Subproject commit 36d7fee5f7e0ef6ffa28970687db85b36947a3e6 diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/IJavaInterfaceTests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/IJavaInterfaceTests.cs index 0d7492dc5..f1abac9f5 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/IJavaInterfaceTests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/IJavaInterfaceTests.cs @@ -16,7 +16,7 @@ public void ClassFile_WithIJavaInterface_class () { var c = LoadClassFile (JavaType + ".class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 23, AccessFlags = ClassAccessFlags.Interface | ClassAccessFlags.Abstract, diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaAnnotationTests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaAnnotationTests.cs index 555d1a166..e0ffadc4e 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaAnnotationTests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaAnnotationTests.cs @@ -16,7 +16,7 @@ public void ClassFile_WithJavaAnnotation_class () { var c = LoadClassFile (JavaType + ".class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 23, AccessFlags = ClassAccessFlags.Public | ClassAccessFlags.Interface | ClassAccessFlags.Abstract | ClassAccessFlags.Annotation, diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaEnumTests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaEnumTests.cs index 03f7c18b0..559e0491e 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaEnumTests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaEnumTests.cs @@ -20,7 +20,7 @@ public void ClassFileDescription () { var c = LoadClassFile (JavaType + ".class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 53, AccessFlags = ClassAccessFlags.Final | ClassAccessFlags.Super | ClassAccessFlags.Enum, diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.1Tests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.1Tests.cs index 83294c424..35e515c53 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.1Tests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.1Tests.cs @@ -16,7 +16,7 @@ public void ClassFileDescription () { var c = LoadClassFile (JavaType + ".class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 47, AccessFlags = ClassAccessFlags.Super, diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.ASCTests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.ASCTests.cs index 27e154435..89c4fda3f 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.ASCTests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.ASCTests.cs @@ -16,7 +16,7 @@ public void ClassFileDescription () { var c = LoadClassFile (JavaType + ".class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 23, Deprecated = true, diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.PSCTests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.PSCTests.cs index e52335a45..dd9f4faaf 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.PSCTests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.PSCTests.cs @@ -16,7 +16,7 @@ public void ClassFileDescription () { var c = LoadClassFile (JavaType + ".class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 20, AccessFlags = ClassAccessFlags.Public | ClassAccessFlags.Super | ClassAccessFlags.Abstract, diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNC.RPNCTests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNC.RPNCTests.cs index 85119703f..2af262529 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNC.RPNCTests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNC.RPNCTests.cs @@ -16,7 +16,7 @@ public void ClassFileDescription () { var c = LoadClassFile (JavaType + ".class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 46, AccessFlags = ClassAccessFlags.Public | ClassAccessFlags.Super | ClassAccessFlags.Abstract, diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNCTests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNCTests.cs index d25ddfcdc..87807ccbc 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNCTests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaType.RNCTests.cs @@ -16,7 +16,7 @@ public void ClassFileDescription () { var c = LoadClassFile (JavaType + ".class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 44, AccessFlags = ClassAccessFlags.Public | ClassAccessFlags.Super | ClassAccessFlags.Abstract, diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaTypeTests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaTypeTests.cs index 00ed89f87..8c828058e 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaTypeTests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/JavaTypeTests.cs @@ -18,7 +18,7 @@ public void ClassFile_WithJavaType_class () { var c = LoadClassFile ("JavaType.class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 195, AccessFlags = ClassAccessFlags.Public | ClassAccessFlags.Super, diff --git a/tests/Xamarin.Android.Tools.Bytecode-Tests/NonGenericGlobalTypeTests.cs b/tests/Xamarin.Android.Tools.Bytecode-Tests/NonGenericGlobalTypeTests.cs index 9b8e2b800..c599c791a 100644 --- a/tests/Xamarin.Android.Tools.Bytecode-Tests/NonGenericGlobalTypeTests.cs +++ b/tests/Xamarin.Android.Tools.Bytecode-Tests/NonGenericGlobalTypeTests.cs @@ -16,7 +16,7 @@ public void ClassFile_WithNonGenericGlobalType_class () { var c = LoadClassFile (JavaType + ".class"); new ExpectedTypeDeclaration { - MajorVersion = 0x32, + MajorVersion = 0x34, MinorVersion = 0, ConstantPoolCount = 16, AccessFlags = ClassAccessFlags.Super,