Skip to content

Conversation

@JoshRosen
Copy link
Contributor

@JoshRosen JoshRosen commented Jun 8, 2016

Spark's SBT build currently uses a fork of the sbt-pom-reader plugin but depends on that fork via a SBT subproject which is cloned from https://github.com/scrapcodes/sbt-pom-reader/tree/ignore_artifact_id. This unnecessarily slows down the initial build on fresh machines and is also risky because it risks a build breakage in case that GitHub repository ever changes or is deleted.

In order to address these issues, I have published a pre-built binary of our forked sbt-pom-reader plugin to Maven Central under the org.spark-project namespace and have updated Spark's build to use that artifact. This published artifact was built from https://github.com/JoshRosen/sbt-pom-reader/tree/v1.0.0-spark, which contains the contents of @ScrapCodes's branch plus an additional patch to configure the build for artifact publication.

/cc @srowen @ScrapCodes for review.

@SparkQA
Copy link

SparkQA commented Jun 8, 2016

Test build #60189 has finished for PR 13564 at commit d407846.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen JoshRosen changed the title [SPARK-15827][BUILD] Publish Spark's forked sbt-pom-reader to Maven Central [SPARK-15827][BUILD][WIP] Publish Spark's forked sbt-pom-reader to Maven Central Jun 8, 2016
// In the long run, we should try to merge our patch upstream and switch to an upstream version of
// the plugin; this is tracked at SPARK-14401.

resolvers += "Spark fork of sbt-pom-reader" at "https://oss.sonatype.org/content/repositories/orgspark-project-1124"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this permanent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, if we decide that the bits that I published here look okay then I'll push to central and remove this resolver.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please go ahead!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Let's wait for it to hit Central.

@ScrapCodes
Copy link
Member

Thanks for doing this. It will definitely save time, and some CPU cycles(energy) for all the first time builds.

@SparkQA
Copy link

SparkQA commented Jun 9, 2016

Test build #60216 has finished for PR 13564 at commit 57199a8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen JoshRosen changed the title [SPARK-15827][BUILD][WIP] Publish Spark's forked sbt-pom-reader to Maven Central [SPARK-15827][BUILD] Publish Spark's forked sbt-pom-reader to Maven Central Jun 9, 2016
@JoshRosen
Copy link
Contributor Author

This is now sync'd to Maven Central and has passed tests, so I'm going to merge it into master, branch-2.0, and branch-1.6.

asfgit pushed a commit that referenced this pull request Jun 9, 2016
…entral

Spark's SBT build currently uses a fork of the sbt-pom-reader plugin but depends on that fork via a SBT subproject which is cloned from https://github.com/scrapcodes/sbt-pom-reader/tree/ignore_artifact_id. This unnecessarily slows down the initial build on fresh machines and is also risky because it risks a build breakage in case that GitHub repository ever changes or is deleted.

In order to address these issues, I have published a pre-built binary of our forked sbt-pom-reader plugin to Maven Central under the `org.spark-project` namespace and have updated Spark's build to use that artifact. This published artifact was built from https://github.com/JoshRosen/sbt-pom-reader/tree/v1.0.0-spark, which contains the contents of ScrapCodes's branch plus an additional patch to configure the build for artifact publication.

/cc srowen ScrapCodes for review.

Author: Josh Rosen <[email protected]>

Closes #13564 from JoshRosen/use-published-fork-of-pom-reader.

(cherry picked from commit f74b777)
Signed-off-by: Josh Rosen <[email protected]>
@asfgit asfgit closed this in f74b777 Jun 9, 2016
@JoshRosen JoshRosen deleted the use-published-fork-of-pom-reader branch June 9, 2016 18:12
asfgit pushed a commit that referenced this pull request Jun 9, 2016
…entral

Spark's SBT build currently uses a fork of the sbt-pom-reader plugin but depends on that fork via a SBT subproject which is cloned from https://github.com/scrapcodes/sbt-pom-reader/tree/ignore_artifact_id. This unnecessarily slows down the initial build on fresh machines and is also risky because it risks a build breakage in case that GitHub repository ever changes or is deleted.

In order to address these issues, I have published a pre-built binary of our forked sbt-pom-reader plugin to Maven Central under the `org.spark-project` namespace and have updated Spark's build to use that artifact. This published artifact was built from https://github.com/JoshRosen/sbt-pom-reader/tree/v1.0.0-spark, which contains the contents of ScrapCodes's branch plus an additional patch to configure the build for artifact publication.

/cc srowen ScrapCodes for review.

Author: Josh Rosen <[email protected]>

Closes #13564 from JoshRosen/use-published-fork-of-pom-reader.

(cherry picked from commit f74b777)
Signed-off-by: Josh Rosen <[email protected]>
zjffdu pushed a commit to zjffdu/spark that referenced this pull request Jun 10, 2016
…entral

Spark's SBT build currently uses a fork of the sbt-pom-reader plugin but depends on that fork via a SBT subproject which is cloned from https://github.com/scrapcodes/sbt-pom-reader/tree/ignore_artifact_id. This unnecessarily slows down the initial build on fresh machines and is also risky because it risks a build breakage in case that GitHub repository ever changes or is deleted.

In order to address these issues, I have published a pre-built binary of our forked sbt-pom-reader plugin to Maven Central under the `org.spark-project` namespace and have updated Spark's build to use that artifact. This published artifact was built from https://github.com/JoshRosen/sbt-pom-reader/tree/v1.0.0-spark, which contains the contents of ScrapCodes's branch plus an additional patch to configure the build for artifact publication.

/cc srowen ScrapCodes for review.

Author: Josh Rosen <[email protected]>

Closes apache#13564 from JoshRosen/use-published-fork-of-pom-reader.
zzcclp pushed a commit to zzcclp/spark that referenced this pull request Jun 12, 2016
…entral

Spark's SBT build currently uses a fork of the sbt-pom-reader plugin but depends on that fork via a SBT subproject which is cloned from https://github.com/scrapcodes/sbt-pom-reader/tree/ignore_artifact_id. This unnecessarily slows down the initial build on fresh machines and is also risky because it risks a build breakage in case that GitHub repository ever changes or is deleted.

In order to address these issues, I have published a pre-built binary of our forked sbt-pom-reader plugin to Maven Central under the `org.spark-project` namespace and have updated Spark's build to use that artifact. This published artifact was built from https://github.com/JoshRosen/sbt-pom-reader/tree/v1.0.0-spark, which contains the contents of ScrapCodes's branch plus an additional patch to configure the build for artifact publication.

/cc srowen ScrapCodes for review.

Author: Josh Rosen <[email protected]>

Closes apache#13564 from JoshRosen/use-published-fork-of-pom-reader.

(cherry picked from commit f74b777)
Signed-off-by: Josh Rosen <[email protected]>
(cherry picked from commit 739d992)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants