Only grant `connect,accept` permissions to `transport-netty4`

[s1monw]

Today we still grant a quite scary permission to core:

  // Allow connecting to the internet anywhere
  permission java.net.SocketPermission "*", "accept,connect,resolve";

But since we now have netty4 moved to a module we can potentially move this to into the modules security policy. Yet, there are a bunch of things that needs fixing until we can do that:

• since we use MockTcpTransport from our test framework this needs to have the same permissions granted. Yet, if we just go ahead and grant accept,connect to the test-framework we might run into trouble since our tests will just inherit that permission ie. if unit and pseudo integ-tests are run since we don't grant this to a codebase. We might want to add some kind of MockSocket project just like SecureMock that we can grant this permission to and where we can depend on for testing.
• netty-4 still has issues with missing doPrivileged blocks that needs fixing
• move URLRepository somewhere else since it uses connect and core shouldn't establish any kind of connection. (this can be a second step, we can first start removing accept from the list.
• some other plugins like ec2 / gce etc. might need extra permission to connect to their endpoints which needs manual testing

here is an example of a missing doPrivileged block ie here:

> Throwable #1: java.security.AccessControlException: access denied ("java.net.SocketPermission" "[fe80:0:0:0:0:0:0:1%1]:52661" "connect,resolve")
   > 	at __randomizedtesting.SeedInfo.seed([8FDA867CA1C20E0D:47AB82401EC3F2C5]:0)
   > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
   > 	at java.security.AccessController.checkPermission(AccessController.java:884)
   > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   > 	at java.lang.SecurityManager.checkConnect(SecurityManager.java:1051)
   > 	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:625)
   > 	at io.netty.channel.socket.nio.NioSocketChannel.doConnect(NioSocketChannel.java:331)
   > 	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:254)
   > 	at io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1266)
   > 	at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:556)
   > 	at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:541)
   > 	at io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboundHandlerAdapter.java:47)
   > 	at io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:556)
   > 	at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:541)
   > 	at io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:523)
   > 	at io.netty.channel.DefaultChannelPipeline.connect(DefaultChannelPipeline.java:985)
   > 	at io.netty.channel.AbstractChannel.connect(AbstractChannel.java:255)
   > 	at io.netty.bootstrap.Bootstrap$3.run(Bootstrap.java:252)
   > 	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
   > 	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:418)
   > 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:454)
   > 	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
   > 	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
   > 	at java.lang.Thread.run(Thread.java:745)

[jasontedor]

+1

In general Netty needs some love with respect to handling the security manager correctly so we'll have some work to do here.

[Tim-Brooks]

I've been able to make the modifications to netty necessary to move the "accept" permission to the transport plugin and get the transport and core tests passing (with creation of a mocksocket jar).

I've had some issues with the azure plugin. That plugin uses com.sun.net.httpserver.HttpsServer for running some tests. it appears to me that HttpsServer is not calling doPriviledge when accepting connections.

if(!ServerImpl.this.terminating) {

    SocketChannel var5 = ServerImpl.this.schan.accept();

    if(ServerConfig.noDelay()) {

        var5.socket().setTcpNoDelay(true);

    }


    if(var5 != null) {

        var5.configureBlocking(false);

        SelectionKey var6 = var5.register(ServerImpl.this.selector, 1);

        var7 = new HttpConnection();

        var7.selectionKey = var6;

        var7.setChannel(var5);

        var6.attach(var7);

        ServerImpl.this.requestStarted(var7);

        ServerImpl.this.allConnections.add(var7);

    }

}

[s1monw]

@tbrooks8 amazing progress, would you mind linking the netty PRs / issues with this issue so we can track them. Also for all core changes can you make sure you open smallish PRs if possible?

I've had some issues with the azure plugin. That plugin uses com.sun.net.httpserver.HttpsServer for running some tests. it appears to me that HttpsServer is not calling doPriviledge when accepting connections.

that is super annoying. I am not 100% sure what we can do about this, I think first of all we gotta fix it in the JDK such that down the road we can remove whatever hack we add here.

[Tim-Brooks]

I push the work that I have completed on the mocksocket here:

https://github.com/elastic/mocksocket

I have opened a Netty PR (netty/netty#6146)

I will start making PRs to ES as I split up the work.

[s1monw]

cool stuff - can you make the https://github.com/elastic/mocksocket repo public I don't think think there is anything private about this! :)

[Tim-Brooks]

I have opened a PR to integrate mocksocket into ElasticSearch. And replace raw usages of Sockets and ServerSockets in the ES tests.

#22287

[Tim-Brooks]

Based on the discussion in the status meeting today, I investigated applying permissions based on files opposed to a separate jar.

So I migrated accept out of the core security policy. And placed the following the in test-framework.policy:

grant codeBase "file:/path/to/elastic/elasticsearch/test/framework/build-idea/classes/test/" {
  permission java.net.SocketPermission "*", "accept,resolve,connect";
 };

grant codeBase "file:/path/to/elastic/elasticsearch/test/framework/build-idea/classes/main/" {
   permission java.net.SocketPermission "*", "accept,resolve,connect";
 };

With that change, any tests involving the MockTCPTransport passed. (Although the netty transport tests failed as they were not given permission.)

This approach seems to provide a way forward. Although my approach is certainly naive and hardcoded.

I'm not immediately clear on what approach to take in order to make this solution robust. I see that we currently parse the classpath and build up some system properties. Currently this works pretty well for jars. However, in the case of our code it is constantly rewriting the codebase.main and codebase.test system properties.

So when you run the "core" tests the following codebases are parsed:

file:/path/to/elastic/elasticsearch/core/build-idea/classes/test/
file:/path/to/elastic/elasticsearch/core/build-idea/classes/main/
file:/path/to/elastic/elasticsearch/test/framework/build-idea/classes/main/
file:/path/to/elastic/elasticsearch/client/rest/build-idea/classes/main/

And for discovery-azure-classic plugin tests:

file:/path/to/elastic/elasticsearch/plugins/discovery-azure-classic/build-idea/classes/test/
file:/path/to/elastic/elasticsearch/plugins/discovery-azure-classic/build-idea/classes/main/
file:/path/to/elastic/elasticsearch/plugins/discovery-azure-classic/build-idea/generated-resources/
file:/path/to/elastic/elasticsearch/core/build-idea/classes/main/
file:/path/to/elastic/elasticsearch/test/framework/build-idea/classes/main/
file:/path/to/elastic/elasticsearch/client/rest/build-idea/classes/main/

Our method of parsing the "shortName" means that the property will be codebase.test, codebase.main, or codebase.generated-source. And whichever one is parsed last "wins".

I could work on implementing proper handling of this. Such as parsing these into:

• codebase.core.main
• codebase.core.test
• codebase.test.framework.main
• codebase.test.framework.test
• codebase.plugins.discovery-azure-classic.main
• codebase.plugins.discovery-azure-classic.test

Those system properties could be accessed in security policy files to give certain packages permissions.

Maybe this is not the easiest way forward? It's entirely possible that I missing a better way of doing this.

Also - I'm not sure what implications those changes would have for the distribution (when all the packages / modules are different jars).