Skip to content

Conversation

@jojochuang
Copy link
Contributor

No description provided.

Change-Id: I51b03ef075de994d118efec0c0077e030e37f5b1
@jojochuang jojochuang requested a review from steveloughran May 21, 2021 03:50
@steveloughran
Copy link
Contributor

This is one of those JARs where we've had most problems on specific linux builds and in a distro setup rather than the module unit tests.

What modules have you tested with this?

  • Have you tried the hadoop-aws and (especially) hadoop-azure test runs?
  • And on what platforms?
  • Any CLI uses (e.g. hadoop FS API calls &c)

Copy link
Contributor

@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this, I do need to some details on whatever testing you've done to make sure this goes through the code properly.

as an example, if the ssl channel mode is set to openssl for s3a, then all https connections MUST be through the native ssl libs, which wildfly will let us do.

fs.s3a.ssl.channel.mode = openssl

(Actually, I'm thinking we should add something to cloudstore to force the creation of connections through wildfly + openssl just so we can explicitly check that the wiring up is good.

@jasonwzs
Copy link

@steveloughran, is there any update on this fix? We are also waiting for the fix of this vulnerability issue.
What tests need to be done in order to approve this PR?

@steveloughran
Copy link
Contributor

What tests need to be done in order to approve this PR?

I'd like the hadoop-azure and hadoop-aws integration tests mvn verify to be run on a system with openssl installed, and for s3a, the auth mechanism switched to openssl so we can be confident it is all going through wildfly to native

    <property>
      <name>fs.s3a.ssl.channel.mode</name>
      <value>openssl</value>
    </property>

we have had major problems with wildfly and s3 SSL in the past, including sometimes hanging during SSL handshake (triggered by some Linux Kernel security changes)...verifying all is healthy is critical here

@jasonwzs
Copy link

jasonwzs commented Oct 4, 2021

@steveloughran , if we don't set fs.s3a.ssl.channel.mode property to openssl, will it still go through this wildfly lib in any chance?

@steveloughran
Copy link
Contributor

the default tries wildfly and falls back to jvm on problems; openssl means openssl/wildfy only

@steveloughran
Copy link
Contributor

rebasing for 3.3.5 with testing, as there is a CVE related to the one we ship. this is lower risk than for the 2.2 upgrade

@steveloughran
Copy link
Contributor

rebased into #5310 and targeting 3.3.5 so as to tick off another CVE

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants