Skip to content

Conversation

dongjoon-hyun
Copy link
Member

@dongjoon-hyun dongjoon-hyun commented Sep 23, 2025

Description of PR

HADOOP-19701. Remove invalid licenses field of hadoop-aliyun SBOM by upgrading cyclonedx to 2.9.1.

This PR aims to improve hadoop-aliyun SBOM by removing invalid licenses field via upgrading cyclonedx to 2.9.1.

Technically, SBOM scheme is going to be upgraded to 1.6 from 1.4.

$ head -n2 before
<?xml version="1.0" encoding="UTF-8"?>
<bom serialNumber="urn:uuid:82616609-099a-4b8a-bc63-9c4b39c554b0" version="1" xmlns="http://cyclonedx.org/schema/bom/1.4">

$ head -n2 after
<?xml version="1.0" encoding="UTF-8"?>
<bom serialNumber="urn:uuid:d33de7e3-2887-37ef-ab07-bd272ebe378c" version="1" xmlns="http://cyclonedx.org/schema/bom/1.6">

According to the testing, it turns out that the empty license field of aliyun-sdk-oss is considered invalid in the latest CycloneDX Schema 1.5 and 1.6. For example, this causes a parsing error when a downstream to try to validate it with Schema Version 1.6.

$ curl -s https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-aliyun/3.4.2/hadoop-aliyun-3.4.2-cyclonedx.xml | grep -C3 '<name></name>'
      </hashes>
      <licenses>
        <license>
          <name></name>
        </license>
      </licenses>
      <purl>pkg:maven/com.aliyun.oss/[email protected]?type=jar</purl>
--
      </hashes>
      <licenses>
        <license>
          <name></name>
        </license>
      </licenses>
      <purl>pkg:maven/com.aliyun/[email protected]?type=jar</purl>
--
      </hashes>
      <licenses>
        <license>
          <name></name>
        </license>
      </licenses>
      <purl>pkg:maven/com.aliyun/[email protected]?type=jar</purl>
--
      </hashes>
      <licenses>
        <license>
          <name></name>
        </license>
      </licenses>
      <purl>pkg:maven/com.aliyun/[email protected]?type=jar</purl>
The content of element 'license' is not complete.
One of '{"http://cyclonedx.org/schema/bom/1.6":id, "http://cyclonedx.org/schema/bom/1.6":name}' is expected.
...
        at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:135)

Since all published SBOM files of hadoop-aliyun are in the same situation, we had better upgrade the plugin for next Apache Hadoop release.

How was this patch tested?

After upgrading the plugin, this invalid empty license field will be removed like the following.

$ cat after | grep -C3 '<name></name>' | wc -l
       0

For code changes:

This is only a build dependency upgrade to bring the latest robust version.

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@dongjoon-hyun dongjoon-hyun changed the title HADOOP-19701. Upgrade cyclonedx to 2.9.1 to remove invalid licenses field of hadoop-aliyun SBOM HADOOP-19701. Upgrade cyclonedx to 2.9.1 to fix invalid hadoop-aliyun SBOM file Sep 23, 2025
@dongjoon-hyun dongjoon-hyun changed the title HADOOP-19701. Upgrade cyclonedx to 2.9.1 to fix invalid hadoop-aliyun SBOM file HADOOP-19701. Upgrade cyclonedx to 2.9.1 to remove invalid licenses field of hadoop-aliyun SBOM Sep 23, 2025
@dongjoon-hyun dongjoon-hyun changed the title HADOOP-19701. Upgrade cyclonedx to 2.9.1 to remove invalid licenses field of hadoop-aliyun SBOM HADOOP-19701. Remove invalid licenses field of hadoop-aliyun SBOM by upgradding cyclonedx to 2.9.1 Sep 23, 2025
@dongjoon-hyun dongjoon-hyun changed the title HADOOP-19701. Remove invalid licenses field of hadoop-aliyun SBOM by upgradding cyclonedx to 2.9.1 HADOOP-19701. Remove invalid licenses field of hadoop-aliyun SBOM by upgrading cyclonedx to 2.9.1 Sep 23, 2025
@dongjoon-hyun
Copy link
Member Author

Thank you, @slfan1989 .

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 52s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 xmllint 0m 0s xmllint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 60m 11s trunk passed
+1 💚 compile 20m 20s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 compile 16m 2s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 mvnsite 23m 22s trunk passed
+1 💚 javadoc 11m 38s trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 9m 20s trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 shadedclient 185m 36s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 49m 43s the patch passed
+1 💚 compile 21m 13s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javac 21m 13s the patch passed
+1 💚 compile 18m 49s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 javac 18m 49s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 21m 50s the patch passed
+1 💚 javadoc 11m 53s the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚 javadoc 9m 0s the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚 shadedclient 88m 52s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 511m 54s /patch-unit-root.txt root in the patch failed.
+1 💚 asflicense 1m 18s The patch does not generate ASF License warnings.
880m 54s
Reason Tests
Failed junit tests hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7990/1/artifact/out/Dockerfile
GITHUB PR #7990
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname Linux 3a691ff72698 5.15.0-144-generic #157-Ubuntu SMP Mon Jun 16 07:33:10 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / f944112
Default Java Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7990/1/testReport/
Max. process+thread count 2369 (vs. ulimit of 5500)
modules C: . U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7990/1/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@dongjoon-hyun
Copy link
Member Author

All tests passes except test4tests and unit tests which checks new or revised test cases. So, I believe this PR is ready.

@slfan1989 slfan1989 merged commit 5000a98 into apache:trunk Sep 24, 2025
1 of 4 checks passed
@slfan1989
Copy link
Contributor

@dongjoon-hyun Thanks for the contribution! Merged into trunk.

@dongjoon-hyun
Copy link
Member Author

Thank you so much, @slfan1989 .

@dongjoon-hyun dongjoon-hyun deleted the HADOOP-19701 branch September 24, 2025 04:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants