This repository was archived by the owner on Jan 9, 2020. It is now read-only.
Reorganize packages between v1 work and v2 work #220
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…es' into submission-v2-file-server
Author
|
Note that this is merging into #212. @ash211 @foxish @erikerlandson |
We still want to post them because in the future we can use these credentials to monitor the API server and handle cleaning up the data accordingly.
ash211
reviewed
Apr 12, 2017
| uploadedJarsBase64Contents: TarGzippedData, | ||
| uploadedFilesBase64Contents: TarGzippedData) extends SubmitRestProtocolRequest { | ||
| @JsonIgnore | ||
| override val messageType: String = s"kubernetes.v1.${Utils.getFormattedClassName(this)}" |
There was a problem hiding this comment.
is this to support compat across a span of versions? Maybe a new rest service with an old launcher?
Author
There was a problem hiding this comment.
This is necessary due to the fact that SubmitRestProtocolRequest expects all subclasses to live in the package org.apache.spark.deploy.rest. The deserialization fails otherwise. This is a workaround to keep the package names consistent for now until we remove this class entirely.
…server' into submission-v2-file-server
…server' into split-v1-and-v2-submission-packages
ash211
approved these changes
Apr 19, 2017
ash211
left a comment
ash211
left a comment
There was a problem hiding this comment.
LGTM though I'd rather merge the dependent PR before this one
|
Will merge into |
ash211
changed the base branch from
submission-v2-file-server
to
branch-2.1-kubernetes
…packages Conflicts: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/kubernetes/v2/ResourceStagingService.scala resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/rest/kubernetes/v2/ResourceStagingServiceImpl.scala resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/rest/kubernetes/v2/ResourceStagingServerSuite.scala resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/rest/kubernetes/v2/ResourceStagingServiceImplSuite.scala
foxish
pushed a commit
that referenced
this pull request
Jul 24, 2017
* Staging server for receiving application dependencies. * Move packages around to split between v1 work and v2 work * Add unit test for file writing * Remove unnecessary main * Add back license header * Minor fixes * Fix integration test with renamed package for client. Fix scalastyle. * Force json serialization to consider the different package. * Revert extraneous log * Fix scalastyle * Remove getting credentials from the API We still want to post them because in the future we can use these credentials to monitor the API server and handle cleaning up the data accordingly. * Generalize to resource staging server outside of Spark * Update code documentation * Val instead of var * Fix build * Fix naming, remove unused import * Move suites from integration test package to core * Use TrieMap instead of locks * Address comments * Fix imports * Change paths, use POST instead of PUT * Use a resource identifier as well as a resource secret
ifilonenko
pushed a commit
to ifilonenko/spark
that referenced
this pull request
Feb 26, 2019
puneetloya
pushed a commit
to puneetloya/spark
that referenced
this pull request
Mar 11, 2019
) * Staging server for receiving application dependencies. * Move packages around to split between v1 work and v2 work * Add unit test for file writing * Remove unnecessary main * Add back license header * Minor fixes * Fix integration test with renamed package for client. Fix scalastyle. * Force json serialization to consider the different package. * Revert extraneous log * Fix scalastyle * Remove getting credentials from the API We still want to post them because in the future we can use these credentials to monitor the API server and handle cleaning up the data accordingly. * Generalize to resource staging server outside of Spark * Update code documentation * Val instead of var * Fix build * Fix naming, remove unused import * Move suites from integration test package to core * Use TrieMap instead of locks * Address comments * Fix imports * Change paths, use POST instead of PUT * Use a resource identifier as well as a resource secret
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Move packages around to split between v1 work and v2 work. This will make it easier to develop submission v2 incrementally without removing what we already have for v1.