Skip to content

HADOOP-16986. S3A to not need wildfly on the classpath. #1948

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 20, 2020
Merged

HADOOP-16986. S3A to not need wildfly on the classpath. #1948

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4a0307e
HADOOP-16986. s3a to not need wildfly on the classpath
steveloughran Apr 9, 2020
91d88c3
HADOOP-16855. S3A resilience to wildfly classes missing.
steveloughran Apr 14, 2020
6acc273
HADOOP-16855. Wildfly class binding
steveloughran Apr 15, 2020
ff698c8
HADOOP-16855. reinstate wildfly as a dependency,
steveloughran Apr 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 74 additions & 68 deletions ...adoop-common/src/main/java/org/apache/hadoop/security/ssl/DelegatingSSLSocketFactory.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@
import java.io.IOException;
import java.net.InetAddress;
import java.net.Socket;
import java.net.SocketException;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
Expand All @@ -31,11 +30,9 @@
import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;

import com.google.common.annotations.VisibleForTesting;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.wildfly.openssl.OpenSSLProvider;
import org.wildfly.openssl.SSL;


/**
* A {@link SSLSocketFactory} that can delegate to various SSL implementations.
Expand All @@ -60,8 +57,8 @@
* </p>
*
* In order to load OpenSSL, applications must ensure the wildfly-openssl
* artifact is on the classpath. Currently, only ABFS and S3A provide
* wildfly-openssl as a runtime dependency.
* artifact is on the classpath. Currently, only ABFS declares
* wildfly-openssl as an explicit dependency.
*/
public final class DelegatingSSLSocketFactory extends SSLSocketFactory {

Expand Down Expand Up @@ -110,7 +107,16 @@ public static synchronized void initializeDefaultFactory(
}

/**
* Singletone instance of the SSLSocketFactory.
* For testing only: reset the socket factory.
*/
@VisibleForTesting
public static synchronized void resetDefaultFactory() {
LOG.info("Resetting default SSL Socket Factory");
instance = null;
}

/**
* Singleton instance of the SSLSocketFactory.
*
* SSLSocketFactory must be initialized with appropriate SSLChannelMode
* using initializeDefaultFactory method.
Expand All @@ -126,9 +132,7 @@ private DelegatingSSLSocketFactory(SSLChannelMode preferredChannelMode)
throws IOException {
try {
initializeSSLContext(preferredChannelMode);
} catch (NoSuchAlgorithmException e) {
throw new IOException(e);
} catch (KeyManagementException e) {
} catch (NoSuchAlgorithmException | KeyManagementException e) {
throw new IOException(e);
}

Expand All @@ -146,42 +150,23 @@ private DelegatingSSLSocketFactory(SSLChannelMode preferredChannelMode)
}

private void initializeSSLContext(SSLChannelMode preferredChannelMode)
throws NoSuchAlgorithmException, KeyManagementException {
throws NoSuchAlgorithmException, KeyManagementException, IOException {
LOG.debug("Initializing SSL Context to channel mode {}",
preferredChannelMode);
switch (preferredChannelMode) {
case Default:
if (!openSSLProviderRegistered) {
OpenSSLProvider.register();
openSSLProviderRegistered = true;
}
try {
java.util.logging.Logger logger = java.util.logging.Logger.getLogger(
SSL.class.getName());
logger.setLevel(Level.WARNING);
ctx = SSLContext.getInstance("openssl.TLS");
ctx.init(null, null, null);
// Strong reference needs to be kept to logger until initialization of
// SSLContext finished (see HADOOP-16174):
logger.setLevel(Level.INFO);
bindToOpenSSLProvider();
channelMode = SSLChannelMode.OpenSSL;
} catch (NoSuchAlgorithmException e) {
LOG.debug("Failed to load OpenSSL. Falling back to the JSSE default.");
} catch (LinkageError | NoSuchAlgorithmException | RuntimeException e) {
LOG.debug("Failed to load OpenSSL. Falling back to the JSSE default.",
e);
ctx = SSLContext.getDefault();
channelMode = SSLChannelMode.Default_JSSE;
}
break;
case OpenSSL:
if (!openSSLProviderRegistered) {
OpenSSLProvider.register();
openSSLProviderRegistered = true;
}
java.util.logging.Logger logger = java.util.logging.Logger.getLogger(
SSL.class.getName());
logger.setLevel(Level.WARNING);
ctx = SSLContext.getInstance("openssl.TLS");
ctx.init(null, null, null);
// Strong reference needs to be kept to logger until initialization of
// SSLContext finished (see HADOOP-16174):
logger.setLevel(Level.INFO);
bindToOpenSSLProvider();
channelMode = SSLChannelMode.OpenSSL;
break;
case Default_JSSE:
Expand All @@ -193,11 +178,38 @@ private void initializeSSLContext(SSLChannelMode preferredChannelMode)
channelMode = SSLChannelMode.Default_JSSE_with_GCM;
break;
default:
throw new NoSuchAlgorithmException("Unknown channel mode: "
throw new IOException("Unknown channel mode: "
+ preferredChannelMode);
}
}

/**
* Bind to the OpenSSL provider via wildfly.
* This MUST be the only place where wildfly classes are referenced,
* so ensuring that any linkage problems only surface here where they may
* be caught by the initialization code.
*/
private void bindToOpenSSLProvider()
throws NoSuchAlgorithmException, KeyManagementException {
if (!openSSLProviderRegistered) {
LOG.debug("Attempting to register OpenSSL provider");
org.wildfly.openssl.OpenSSLProvider.register();
openSSLProviderRegistered = true;
}
// Strong reference needs to be kept to logger until initialization of
// SSLContext finished (see HADOOP-16174):
java.util.logging.Logger logger = java.util.logging.Logger.getLogger(
"org.wildfly.openssl.SSL");
Level originalLevel = logger.getLevel();
try {
logger.setLevel(Level.WARNING);
ctx = SSLContext.getInstance("openssl.TLS");
ctx.init(null, null, null);
} finally {
logger.setLevel(originalLevel);
}
}

public String getProviderName() {
return providerName;
}
Expand All @@ -212,86 +224,80 @@ public String[] getSupportedCipherSuites() {
return ciphers.clone();
}

/**
* Get the channel mode of this instance.
* @return a channel mode.
*/
public SSLChannelMode getChannelMode() {
return channelMode;
}

public Socket createSocket() throws IOException {
SSLSocketFactory factory = ctx.getSocketFactory();
SSLSocket ss = (SSLSocket) factory.createSocket();
configureSocket(ss);
return ss;
return configureSocket(factory.createSocket());
}

@Override
public Socket createSocket(Socket s, String host, int port,
boolean autoClose) throws IOException {
SSLSocketFactory factory = ctx.getSocketFactory();
SSLSocket ss = (SSLSocket) factory.createSocket(s, host, port, autoClose);

configureSocket(ss);
return ss;
return configureSocket(
factory.createSocket(s, host, port, autoClose));
}

@Override
public Socket createSocket(InetAddress address, int port,
InetAddress localAddress, int localPort)
throws IOException {
SSLSocketFactory factory = ctx.getSocketFactory();
SSLSocket ss = (SSLSocket) factory
.createSocket(address, port, localAddress, localPort);

configureSocket(ss);
return ss;
return configureSocket(factory
.createSocket(address, port, localAddress, localPort));
}

@Override
public Socket createSocket(String host, int port, InetAddress localHost,
int localPort) throws IOException {
SSLSocketFactory factory = ctx.getSocketFactory();
SSLSocket ss = (SSLSocket) factory
.createSocket(host, port, localHost, localPort);

configureSocket(ss);

return ss;
return configureSocket(factory
.createSocket(host, port, localHost, localPort));
}

@Override
public Socket createSocket(InetAddress host, int port) throws IOException {
SSLSocketFactory factory = ctx.getSocketFactory();
SSLSocket ss = (SSLSocket) factory.createSocket(host, port);

configureSocket(ss);

return ss;
return configureSocket(factory.createSocket(host, port));
}

@Override
public Socket createSocket(String host, int port) throws IOException {
SSLSocketFactory factory = ctx.getSocketFactory();
SSLSocket ss = (SSLSocket) factory.createSocket(host, port);

configureSocket(ss);

return ss;
return configureSocket(factory.createSocket(host, port));
}

private void configureSocket(SSLSocket ss) throws SocketException {
ss.setEnabledCipherSuites(ciphers);
private Socket configureSocket(Socket socket) {
((SSLSocket) socket).setEnabledCipherSuites(ciphers);
return socket;
}

private String[] alterCipherList(String[] defaultCiphers) {

ArrayList<String> preferredSuits = new ArrayList<>();
ArrayList<String> preferredSuites = new ArrayList<>();

// Remove GCM mode based ciphers from the supported list.
for (int i = 0; i < defaultCiphers.length; i++) {
if (defaultCiphers[i].contains("_GCM_")) {
LOG.debug("Removed Cipher - {} from list of enabled SSLSocket ciphers",
defaultCiphers[i]);
} else {
preferredSuits.add(defaultCiphers[i]);
preferredSuites.add(defaultCiphers[i]);
}
}

ciphers = preferredSuits.toArray(new String[0]);
ciphers = preferredSuites.toArray(new String[0]);
return ciphers;
}
}
}
35 changes: 18 additions & 17 deletions hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/impl/NetworkBinding.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,26 +61,28 @@ public class NetworkBinding {
* @param awsConf the {@link ClientConfiguration} to set the
* SSLConnectionSocketFactory for.
* @throws IOException if there is an error while initializing the
* {@link SSLSocketFactory}.
* {@link SSLSocketFactory} other than classloader problems.
*/
public static void bindSSLChannelMode(Configuration conf,
ClientConfiguration awsConf) throws IOException {
try {
// Validate that SSL_CHANNEL_MODE is set to a valid value.
String channelModeString = conf.get(
SSL_CHANNEL_MODE, DEFAULT_SSL_CHANNEL_MODE.name());
DelegatingSSLSocketFactory.SSLChannelMode channelMode = null;
for (DelegatingSSLSocketFactory.SSLChannelMode mode :
DelegatingSSLSocketFactory.SSLChannelMode.values()) {
if (mode.name().equalsIgnoreCase(channelModeString)) {
channelMode = mode;
}
}
if (channelMode == null) {
throw new IllegalArgumentException(channelModeString +
" is not a valid value for " + SSL_CHANNEL_MODE);

// Validate that SSL_CHANNEL_MODE is set to a valid value.
String channelModeString = conf.getTrimmed(
SSL_CHANNEL_MODE, DEFAULT_SSL_CHANNEL_MODE.name());
DelegatingSSLSocketFactory.SSLChannelMode channelMode = null;
for (DelegatingSSLSocketFactory.SSLChannelMode mode :
DelegatingSSLSocketFactory.SSLChannelMode.values()) {
if (mode.name().equalsIgnoreCase(channelModeString)) {
channelMode = mode;
}
}
if (channelMode == null) {
throw new IllegalArgumentException(channelModeString +
" is not a valid value for " + SSL_CHANNEL_MODE);
}

DelegatingSSLSocketFactory.initializeDefaultFactory(channelMode);
try {
// Look for AWS_SOCKET_FACTORY_CLASSNAME on the classpath and instantiate
// an instance using the DelegatingSSLSocketFactory as the
// SSLSocketFactory.
Expand All @@ -89,7 +91,6 @@ public static void bindSSLChannelMode(Configuration conf,
Constructor<?> factoryConstructor =
sslConnectionSocketFactory.getDeclaredConstructor(
SSLSocketFactory.class, HostnameVerifier.class);
DelegatingSSLSocketFactory.initializeDefaultFactory(channelMode);
awsConf.getApacheHttpClientConfig().setSslSocketFactory(
(com.amazonaws.thirdparty.apache.http.conn.ssl.
SSLConnectionSocketFactory) factoryConstructor
Expand All @@ -98,7 +99,7 @@ public static void bindSSLChannelMode(Configuration conf,
(HostnameVerifier) null));
} catch (ClassNotFoundException | NoSuchMethodException |
IllegalAccessException | InstantiationException |
InvocationTargetException e) {
InvocationTargetException | LinkageError e) {
LOG.debug("Unable to create class {}, value of {} will be ignored",
AWS_SOCKET_FACTORY_CLASSNAME, SSL_CHANNEL_MODE, e);
}
Expand Down
36 changes: 27 additions & 9 deletions hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/performance.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -539,7 +539,7 @@ in Java 9, so if `default_jsse` is specified and applications run on Java
includes GCM in the list of cipher suites on Java 8, so it is equivalent to
running with the vanilla JSSE.

### OpenSSL Acceleration
### <a name="openssl"></a> OpenSSL Acceleration

**Experimental Feature**

Expand All @@ -552,8 +552,8 @@ significant performance benefit over the JSSE.
S3A uses the
[WildFly OpenSSL](https://github.com/wildfly-security/wildfly-openssl) library
to bind OpenSSL to the Java JSSE APIs. This library allows S3A to
transparently read data using OpenSSL. The wildfly-openssl library is a
runtime dependency of S3A and contains native libraries for binding the Java
transparently read data using OpenSSL. The `wildfly-openssl` library is an
optional runtime dependency of S3A and contains native libraries for binding the Java
JSSE to OpenSSL.

WildFly OpenSSL must load OpenSSL itself. This can be done using the system
Expand Down Expand Up @@ -596,19 +596,37 @@ exception and S3A initialization will fail.

Supported values for `fs.s3a.ssl.channel.mode`:

| fs.s3a.ssl.channel.mode Value | Description |
| `fs.s3a.ssl.channel.mode` Value | Description |
|-------------------------------|-------------|
| default_jsse | Uses Java JSSE without GCM on Java 8 |
| default_jsse_with_gcm | Uses Java JSSE |
| default | Uses OpenSSL, falls back to default_jsse if OpenSSL cannot be loaded |
| openssl | Uses OpenSSL, fails if OpenSSL cannot be loaded |
| `default_jsse` | Uses Java JSSE without GCM on Java 8 |
| `default_jsse_with_gcm` | Uses Java JSSE |
| `default` | Uses OpenSSL, falls back to `default_jsse` if OpenSSL cannot be loaded |
| `openssl` | Uses OpenSSL, fails if OpenSSL cannot be loaded |

The naming convention is setup in order to preserve backwards compatibility
with HADOOP-15669.
with the ABFS support of [HADOOP-15669](https://issues.apache.org/jira/browse/HADOOP-15669).

Other options may be added to `fs.s3a.ssl.channel.mode` in the future as
further SSL optimizations are made.

### WildFly classpath requirements

For OpenSSL acceleration to work, a compatible version of the
wildfly JAR must be on the classpath. This is not explicitly declared
in the dependencies of the published `hadoop-aws` module, as it is
optional.

If the wildfly JAR is not found, the network acceleration will fall back
to the JVM, always.

Note: there have been compatibility problems with wildfly JARs and openSSL
releases in the past: version 1.0.4.Final is not compatible with openssl 1.1.1.
An extra complication was older versions of the `azure-data-lake-store-sdk`
JAR used in `hadoop-azure-datalake` contained an unshaded copy of the 1.0.4.Final
classes, causing binding problems even when a later version was explicitly
being placed on the classpath.


## Tuning FileSystem Initialization.

When an S3A Filesystem instance is created and initialized, the client
Expand Down
Loading