[SPARK-1520] remove fastutil from dependencies #437
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Merged build triggered. |
|
Merged build started. |
|
Merged build finished. All automated tests passed. |
|
All automated tests passed. |
Contributor
Author
|
@pwendell This is ready to merge. |
Contributor
|
Merged. Thanks! |
asfgit
pushed a commit
that referenced
this pull request
Apr 18, 2014
A quick fix for https://issues.apache.org/jira/browse/SPARK-1520 By excluding fastutil, we bring the number of files in the assembly jar back under 65536, so Java 7 won't create the assembly jar in zip64 format, which cannot be read by Java 6. With this change, the assembly jar now has about 60000 entries (58000 files), tested with both sbt and maven. Author: Xiangrui Meng <[email protected]> Closes #437 from mengxr/remove-fastutil and squashes the following commits: 00f9beb [Xiangrui Meng] remove fastutil from dependencies (cherry picked from commit aa17f02) Signed-off-by: Reynold Xin <[email protected]>
pwendell
added a commit
to pwendell/spark
that referenced
this pull request
May 12, 2014
Minor api usability changes - Expose checkpoint directory - since it is autogenerated now - null check for jars - Expose SparkHadoopUtil : so that configuration creation is abstracted even from user code to avoid duplication of functionality already in spark.
pdeyhim
pushed a commit
to pdeyhim/spark-1
that referenced
this pull request
Jun 25, 2014
A quick fix for https://issues.apache.org/jira/browse/SPARK-1520 By excluding fastutil, we bring the number of files in the assembly jar back under 65536, so Java 7 won't create the assembly jar in zip64 format, which cannot be read by Java 6. With this change, the assembly jar now has about 60000 entries (58000 files), tested with both sbt and maven. Author: Xiangrui Meng <[email protected]> Closes apache#437 from mengxr/remove-fastutil and squashes the following commits: 00f9beb [Xiangrui Meng] remove fastutil from dependencies
andrewor14
pushed a commit
to andrewor14/spark
that referenced
this pull request
Jan 8, 2015
Minor api usability changes - Expose checkpoint directory - since it is autogenerated now - null check for jars - Expose SparkHadoopUtil : so that configuration creation is abstracted even from user code to avoid duplication of functionality already in spark. (cherry picked from commit 73dfd42) Signed-off-by: Patrick Wendell <[email protected]>
markhamstra
pushed a commit
to markhamstra/spark
that referenced
this pull request
Nov 7, 2017
* Use a secret to mount small files in driver and executors. Allows bypassing the resource staging server in a few scenarios. * Fix scalstyle * Address comments and add tests. * Lightly brush up formatting. * Make the working directory empty so that added files don't clobber existing binaries. * Address comments. * Drop testing file size to N+1 of the limit
mccheah
added a commit
to mccheah/spark
that referenced
this pull request
Nov 28, 2018
Bump gradle-bintray-plugin to work with latest gradle
bzhaoopenstack
pushed a commit
to bzhaoopenstack/spark
that referenced
this pull request
Sep 11, 2019
Fix devstack trove deployment failing
RolatZhang
pushed a commit
to RolatZhang/spark
that referenced
this pull request
Aug 15, 2022
turboFei
pushed a commit
to turboFei/spark
that referenced
this pull request
Nov 6, 2025
…linux_x86_64.so to copy (apache#437)
turboFei
pushed a commit
to turboFei/spark
that referenced
this pull request
Nov 6, 2025
…" (apache#442) * Revert "[HDAP-54076][FOLLOW-UP] Fix could not find file ...libnetty_tcnative_linux_x86_64.so to copy (apache#437)" This reverts commit 720c28e. * Revert "[HADP-54076][SPARK-48420] Upgrade netty to 4.1.110 (apache#430)" This reverts commit 98f4ffe.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A quick fix for https://issues.apache.org/jira/browse/SPARK-1520
By excluding fastutil, we bring the number of files in the assembly jar back under 65536, so Java 7 won't create the assembly jar in zip64 format, which cannot be read by Java 6.
With this change, the assembly jar now has about 60000 entries (58000 files), tested with both sbt and maven.