SPARK-1949. Servlet 2.5 vs 3.0 conflict in SBT build #906

srowen · 2014-05-28T16:31:41Z

Kay Ousterhout mentioned that:

I had some trouble compiling an application (Shark) against Spark 1.0,
where Shark had a runtime exception (at the bottom of this message) because
it couldn't find the javax.servlet classes. SBT seemed to have trouble
downloading the servlet APIs that are dependencies of Jetty (used by the
Spark web UI), so I had to manually add them to the application's build
file:

libraryDependencies += "org.mortbay.jetty" % "servlet-api" % "3.0.20100224"

Not exactly sure why this happens but thought it might be useful in case
others run into the same problem.

This is a symptom of Servlet API conflict which we battled in the Maven build. The resolution is to nix Servlet 2.5 and odd old Jetty / Netty 3.x dependencies. It looks like the Hive part of the assembly in the SBT build doesn't exclude all these entirely.

AmplabJenkins · 2014-05-28T16:32:58Z

Merged build triggered.

AmplabJenkins · 2014-05-28T16:33:03Z

Merged build started.

AmplabJenkins · 2014-05-28T17:13:32Z

Merged build finished. All automated tests passed.

AmplabJenkins · 2014-05-28T17:13:33Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15255/

pwendell · 2014-05-28T17:47:35Z

Kay's problem was with linking against the Spark poms that are published using Maven, so I don't see how a change in Spark's SBT build could help with that (?)

srowen · 2014-05-28T18:37:02Z

Bah, I could be way off here. I see what you're saying but had wondered if somehow the SBT output was being used given the mention of SBT.

Coincidentally perhaps then it appeared that the SBT output included conflicting versions of this API.

Let me look again to see how this might occur with Maven, but when I checked today its dependency graph only has a conflict in the examples module, which doesn't matter.

pwendell · 2014-05-29T19:20:45Z

I just spoke with Kay, she was actually publishing locally with sbt rather than linking against the RC (which I though)... sorry, this could totally be relevant!

AmplabJenkins · 2014-06-10T11:27:50Z

Merged build triggered.

AmplabJenkins · 2014-06-10T12:10:35Z

Merged build started.

AmplabJenkins · 2014-06-10T13:14:18Z

Merged build finished.

AmplabJenkins · 2014-06-10T13:14:19Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15621/

pwendell · 2014-06-11T03:26:07Z

Jenkins, retest this please.

AmplabJenkins · 2014-06-11T03:27:07Z

Merged build triggered.

AmplabJenkins · 2014-06-11T03:27:14Z

Merged build started.

AmplabJenkins · 2014-06-11T04:57:37Z

Merged build finished.

AmplabJenkins · 2014-06-11T04:57:38Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15663/

pwendell · 2014-06-11T19:13:28Z

Jenkins, test this please.

AmplabJenkins · 2014-06-11T19:17:06Z

Merged build triggered.

AmplabJenkins · 2014-06-11T19:17:14Z

Merged build started.

AmplabJenkins · 2014-06-11T20:47:38Z

Merged build finished.

AmplabJenkins · 2014-06-11T20:47:38Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15685/

srowen · 2014-06-18T14:39:21Z

I think this is good to go. The initial test passed, but recent ones errored out. Just to double-check:
Jenkins, test this please.

srowen · 2014-06-18T15:08:19Z

Jenkins, test this please.

AmplabJenkins · 2014-06-18T15:09:53Z

Merged build triggered.

AmplabJenkins · 2014-06-18T15:10:01Z

Merged build started.

AmplabJenkins · 2014-06-18T16:40:17Z

Merged build finished.

AmplabJenkins · 2014-06-18T16:40:17Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/15873/

pwendell · 2014-06-21T21:27:48Z

Jenkins, retest this please.

AmplabJenkins · 2014-06-21T21:30:05Z

Merged build triggered.

AmplabJenkins · 2014-06-21T21:30:14Z

Merged build started.

ueshin · 2014-07-02T07:03:54Z

Hi, I encountered this kind of Servlet API conflict.
When are you planning to merge this? Or is there something to do?

vanzin · 2014-07-02T16:34:31Z

See #1271. Sean, maybe close this PR?

ueshin · 2014-07-02T16:56:37Z

Close this?
I encountered after the #1271 was merged.
I believe at least we need to exclude from Hive related dependencies.

vanzin · 2014-07-02T17:01:28Z

Ah, ok. I though you just had missed that other PR.

srowen · 2014-07-02T21:08:18Z

Yeah @vanzin these are actually different changes. Incarnations of the same general issue. @pwendell if you have a moment to have a second look, I think this actually works now.

pwendell · 2014-07-04T05:33:55Z

Jenkins, retest this please.

AmplabJenkins · 2014-07-04T05:35:52Z

Build triggered.

AmplabJenkins · 2014-07-04T05:36:01Z

Build started.

AmplabJenkins · 2014-07-04T07:37:16Z

Build finished.

AmplabJenkins · 2014-07-04T07:37:17Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16339/

AmplabJenkins · 2014-07-04T08:15:51Z

Merged build triggered.

AmplabJenkins · 2014-07-04T08:15:56Z

Merged build started.

AmplabJenkins · 2014-07-04T08:26:01Z

Merged build finished.

AmplabJenkins · 2014-07-04T08:26:01Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16343/

srowen · 2014-07-08T09:52:21Z

Jenkins, retest this please.

AmplabJenkins · 2014-07-08T09:56:06Z

Merged build triggered.

AmplabJenkins · 2014-07-08T09:56:16Z

Merged build started.

AmplabJenkins · 2014-07-08T10:39:12Z

Merged build finished.

AmplabJenkins · 2014-07-08T10:39:13Z

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16405/

…h of the same stuff that's excluded from Hadoop

AmplabJenkins · 2014-07-08T10:46:06Z

Merged build triggered.

AmplabJenkins · 2014-07-08T10:46:11Z

Merged build started.

AmplabJenkins · 2014-07-08T11:31:32Z

Merged build finished. All automated tests passed.

AmplabJenkins · 2014-07-08T11:31:32Z

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/16406/

srowen · 2014-07-13T17:08:34Z

Obsoleted by SBT build changes.

MapR[SPARK-960] Update Hadoop in Spark-3.2.x

* [CARMEL-5854] Data Read DML concurrent conflict * retry per each path

srowen mentioned this pull request Jun 30, 2014

SPARK-2332 [build] add exclusion for old servlet-api on hadoop-client in core #1271

Closed

Exclude old mortbay.jetty from Hadoop deps, and exclude from Hive muc…

95803e4

…h of the same stuff that's excluded from Hadoop

srowen closed this Jul 13, 2014

srowen deleted the SPARK-1949 branch July 13, 2014 17:08

agirish pushed a commit to HPEEzmeral/apache-spark that referenced this pull request May 5, 2022

Merge pull request apache#906 from mapr/SPARK-960

f4b6754

MapR[SPARK-960] Update Hadoop in Spark-3.2.x

wangyum pushed a commit that referenced this pull request May 26, 2023

[CARMEL-5854] Data Read DML concurrent conflict (#906)

f27f1d5

* [CARMEL-5854] Data Read DML concurrent conflict * retry per each path

SPARK-1949. Servlet 2.5 vs 3.0 conflict in SBT build #906

SPARK-1949. Servlet 2.5 vs 3.0 conflict in SBT build #906

Uh oh!

Conversation

srowen commented May 28, 2014

Uh oh!

AmplabJenkins commented May 28, 2014

Uh oh!

AmplabJenkins commented May 28, 2014

Uh oh!

AmplabJenkins commented May 28, 2014

Uh oh!

AmplabJenkins commented May 28, 2014

Uh oh!

pwendell commented May 28, 2014

Uh oh!

srowen commented May 28, 2014

Uh oh!

pwendell commented May 29, 2014

Uh oh!

AmplabJenkins commented Jun 10, 2014

Uh oh!

AmplabJenkins commented Jun 10, 2014

Uh oh!

AmplabJenkins commented Jun 10, 2014

Uh oh!

AmplabJenkins commented Jun 10, 2014

Uh oh!

pwendell commented Jun 11, 2014

Uh oh!

AmplabJenkins commented Jun 11, 2014

Uh oh!

AmplabJenkins commented Jun 11, 2014

Uh oh!

AmplabJenkins commented Jun 11, 2014

Uh oh!

AmplabJenkins commented Jun 11, 2014

Uh oh!

pwendell commented Jun 11, 2014

Uh oh!

AmplabJenkins commented Jun 11, 2014

Uh oh!

AmplabJenkins commented Jun 11, 2014

Uh oh!

AmplabJenkins commented Jun 11, 2014

Uh oh!

AmplabJenkins commented Jun 11, 2014

Uh oh!

srowen commented Jun 18, 2014

Uh oh!

srowen commented Jun 18, 2014

Uh oh!

AmplabJenkins commented Jun 18, 2014

Uh oh!

AmplabJenkins commented Jun 18, 2014

Uh oh!

AmplabJenkins commented Jun 18, 2014

Uh oh!

AmplabJenkins commented Jun 18, 2014

Uh oh!

pwendell commented Jun 21, 2014

Uh oh!

AmplabJenkins commented Jun 21, 2014

Uh oh!

AmplabJenkins commented Jun 21, 2014

Uh oh!

ueshin commented Jul 2, 2014

Uh oh!

vanzin commented Jul 2, 2014

Uh oh!

ueshin commented Jul 2, 2014

Uh oh!

vanzin commented Jul 2, 2014

Uh oh!

srowen commented Jul 2, 2014

Uh oh!

pwendell commented Jul 4, 2014

Uh oh!

AmplabJenkins commented Jul 4, 2014

Uh oh!