setSecurity manager now throws a SecurityException

Author: vjovanov

docs/reference-manual/native-image/Compatibility.md

@@ -41,15 +41,16 @@ The `invokedynamic`method and method handles can introduce calls at run time or
 
 Note that `invokedynamic` use cases generated by `javac` for, for example, Java lambda expressions and String concatenation that are supported because they do not change called methods at run time.
 
-### Security Manager
-
-Native Image will produce images as if `-Djava.security.manager` was set to `disallow`. 
-At image run time, calls to `java.lang.System#setSecurityManager` exit the program with error code `255` if `-Djava.security.manager` is set to anything but `disallow` at program startup.
-
 ## Features That May Operate Differently in a Native Image
 
 Native Image implements some Java features differently to the Java VM.
 
+### Security Manager
+
+`java.lang.System#getSecurityManager()` always returns `null` even if the security manager is set via `-Djava.security.manager` at startup.
+
+`java.lang.System#setSecurityManager(SecurityManager)` invoked with a non-null argument throws a `java.lang.SecurityException` if `-Djava.security.manager` is set to anything but `disallow` at program startup.
+
 ### Signal Handlers
 
 Registering a signal handler requires a new thread to start that handles the signal and invokes shutdown hooks.

docs/security/native-image.md

@@ -93,6 +93,8 @@ The security report section of the native image [build output](../reference-manu
 
 ## Miscellaneous
 
+Setting the security manager is not allowed. For more information see the [compatibility documentation](../reference-manual/native-image/Compatibility.md#security-manager).
+
 Native Image provides multiple ways to specify a certificate file used to define the default TrustStore.
 While the default behavior for `native-image` is to capture and use the default TrustStore from the build-time host environment, this can be changed at run time by setting the "javax.net.ssl.trustStore\*" system properties.
 See the [documentation](../reference-manual/native-image/CertificateManagement.md) for more details.

substratevm/CHANGELOG.md

@@ -14,7 +14,7 @@ This changelog summarizes major changes to GraalVM Native Image.
 * (GR-47937) Make the lambda-class name format in Native-Image consistent with the JDK name format.
 * (GR-45651) Methods, fields and constructors of `Object`, primitive classes and array classes are now registered by default for reflection.
 * (GR-45651) The Native Image agent now tracks calls to `ClassLoader.findSystemClass`, `ObjectInputStream.resolveClass` and `Bundles.of`, and registers resource bundles as bundle name-locale pairs.
-* (GR-49807) Before this change the function `System#setSecurityManager` was always halting program execution with a VM error. This was inconvenient as the VM error prints an uncomprehensible error message and prevents further continuation of the program. For cases where the program is expected to throw an exception when  `System#setSecurityManager` is called, execution on Native Image was not possible. Now, `System#setSecurityManager` throws an `java.lang.UnsupportedOperationException` by default. If the property `java.security.manager` is set to `allow` the program will print a user-readable stack trace and exit with code `99`. Value of `java.security.manager` that would be set at build time is completely ignored at run time.
+* (GR-49807) Before this change the function `System#setSecurityManager` was always halting program execution with a VM error. This was inconvenient as the VM error prints an uncomprehensible error message and prevents further continuation of the program. For cases where the program is expected to throw an exception when  `System#setSecurityManager` is called, execution on Native Image was not possible. Now, `System#setSecurityManager` throws an `java.lang.UnsupportedOperationException` by default. If the property `java.security.manager` is set to anything but `disallow` at program startup this function will throw a `java.lang.SecurityException` according to the Java spec.
 
 ## GraalVM for JDK 21 (Internal Version 23.1.0)
 * (GR-35746) Lower the default aligned chunk size from 1 MB to 512 KB for the serial and epsilon GCs, reducing memory usage and image size in many cases.

substratevm/src/com.oracle.svm.core/src/com/oracle/svm/core/jdk/JavaLangSubstitutions.java

@@ -44,7 +44,6 @@
 import java.util.stream.Stream;
 
 import org.graalvm.nativeimage.ImageSingletons;
-import org.graalvm.nativeimage.LogHandler;
 import org.graalvm.nativeimage.Platform;
 import org.graalvm.nativeimage.Platforms;
 import org.graalvm.nativeimage.hosted.FieldValueTransformer;
@@ -67,14 +66,12 @@
 import com.oracle.svm.core.hub.DynamicHub;
 import com.oracle.svm.core.jdk.JavaLangSubstitutions.ClassValueSupport;
 import com.oracle.svm.core.monitor.MonitorSupport;
-import com.oracle.svm.core.option.HostedOptionKey;
 import com.oracle.svm.core.snippets.SubstrateForeignCallTarget;
 import com.oracle.svm.core.thread.JavaThreads;
 import com.oracle.svm.core.thread.VMOperation;
 import com.oracle.svm.core.util.VMError;
 import com.oracle.svm.util.ReflectionUtil;
 
-import jdk.graal.compiler.options.Option;
 import jdk.graal.compiler.replacements.nodes.BinaryMathIntrinsicNode;
 import jdk.graal.compiler.replacements.nodes.BinaryMathIntrinsicNode.BinaryOperation;
 import jdk.graal.compiler.replacements.nodes.UnaryMathIntrinsicNode;
@@ -413,32 +410,26 @@ private static String getProperty(String key, String def) {
     @Alias @RecomputeFieldValue(kind = Kind.FromAlias, isFinal = true) //
     private static int allowSecurityManager = 1;
 
+    /**
+     * We do not support the {@link SecurityManager} so this method must throw a
+     * {@link SecurityException} when 'java.security.manager' is set to anything but
+     * <code>disallow</code>.
+     * 
+     * @see System#setSecurityManager(SecurityManager)
+     * @see SecurityManager
+     */
     @Substitute
-    @TargetElement(onlyWith = JavaLangSubstitutions.UseSecurityManagerPropertyAtRuntime.class)
-    private static void setSecurityManager(SecurityManager s) {
-        /* We read properties interpreted at isolate creation as that is what happens on the JVM */
-        String smp = SystemPropertiesSupport.singleton().getSavedProperties().get("java.security.manager");
-        if (smp != null && !smp.equals("disallow")) {
-            /*
-             * The strict failure is needed as the security precaution: In case a user does not read
-             * our documentation, uses this deprecated API marked for removal, and passes
-             * "-Djava.security.manager=allow" at runtime, and accidentally catches the
-             * UnsupportedOperationException, we don't want to compromise their security.
-             */
-            System.err.println("""
-                            Fatal error: Property '-Djava.security.manager' is set, but SecurityManager is not supported by Native Image. Please unset this property.
-                            Exiting the program to prevent misinterpretation of the set SecurityManager at:""");
-
-            for (var traceElement : new UnsupportedOperationException().getStackTrace()) {
-                System.err.println("\tat " + traceElement);
+    private static void setSecurityManager(SecurityManager sm) {
+        if (sm != null) {
+            /* Read the property collected at isolate creation as that is what happens on the JVM */
+            String smp = SystemPropertiesSupport.singleton().getSavedProperties().get("java.security.manager");
+            if (smp != null && !smp.equals("disallow")) {
+                throw new SecurityException("Setting the SecurityManager is not supported by Native Image");
+            } else {
+                throw new UnsupportedOperationException(
+                                "The Security Manager is deprecated and will be removed in a future release");
             }
-
-            /* bypasses possible filters on System.exit */
-            ImageSingletons.lookup(LogHandler.class).fatalError();
         }
-
-        throw new UnsupportedOperationException(
-                        "The Security Manager is deprecated and will be removed in a future release");
     }
 }
 
@@ -686,18 +677,6 @@ public Object transform(Object receiver, Object originalValue) {
 /** Dummy class to have a class with the file's name. */
 public final class JavaLangSubstitutions {
 
-    public static class UseSecurityManagerPropertyAtRuntime implements BooleanSupplier {
-        public static class Options {
-            @Option(help = "Used only for testing as exiting the program shadows other working tests, please do not use in production.")//
-            public static final HostedOptionKey<Boolean> TestingSecurityViolationUseSecurityManagerPropertyAtRuntime = new HostedOptionKey<>(true);
-        }
-
-        @Override
-        public boolean getAsBoolean() {
-            return Options.TestingSecurityViolationUseSecurityManagerPropertyAtRuntime.getValue();
-        }
-    }
-
     public static final class StringUtil {
         /**
          * Returns a character from a string at {@code index} position based on the encoding format.