Skip to content

[SPARK-12065] Upgrade Tachyon from 0.8.1 to 0.8.2 #10054

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

[SPARK-12065] Upgrade Tachyon from 0.8.1 to 0.8.2 #10054

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion core/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -270,7 +270,7 @@
<dependency>
<groupId>org.tachyonproject</groupId>
<artifactId>tachyon-client</artifactId>
<version>0.8.1</version>
<version>0.8.2</version>
<exclusions>
<exclusion>
<groupId>org.apache.hadoop</groupId>
Copy link
Contributor Author

@JoshRosen JoshRosen Dec 1, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey, quick question: why do we need these exclusions? AFAIK tachyon-client is an uber-JAR which shades all of its dependencies: http://mvnrepository.com/artifact/org.tachyonproject/tachyon-client/0.8.2

Copy link
Contributor Author

@JoshRosen JoshRosen Dec 1, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I guess we need this because the tachyon-client JAR still declares compile-scope dependencies on the artifacts that it shades and relocates, which will result in Tachyon client's shaded dependencies being pulled in. I'm a bit confused about why the tachyon-client JAR is published with a non-dependency-reduced POM.

This is corroborated by #8577, which needed to add additional exclusions because of this.

Copy link
Contributor Author

@JoshRosen JoshRosen Dec 1, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've filed https://tachyon.atlassian.net/browse/TACHYON-1380 to ask Tachyon to change the handling of dependencies in this POM for future releases. For now, though, I've verified that we don't need to change any of these exclusions because the client's dependencies haven't changed, so I think this PR should be good to merge.

Copy link
Contributor Author

@JoshRosen JoshRosen Dec 2, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, there was some slight confusion here: I somehow parsed "Distribution of Tachyon client module with shaded packages in jar" as "shades all dependencies", which isn't true, so there's not an issue here. My bad.

Expand Down
2 changes: 1 addition & 1 deletion make-distribution.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ SPARK_HOME="$(cd "`dirname "$0"`"; pwd)"
DISTDIR="$SPARK_HOME/dist"

SPARK_TACHYON=false
TACHYON_VERSION="0.8.1"
TACHYON_VERSION="0.8.2"
TACHYON_TGZ="tachyon-${TACHYON_VERSION}-bin.tar.gz"
TACHYON_URL="http://tachyon-project.org/downloads/files/${TACHYON_VERSION}/${TACHYON_TGZ}"

Expand Down