Skip to content

[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter #8543

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter #8543

wants to merge 2 commits into from

Conversation

@davies
Copy link
Contributor

@davies davies commented Aug 31, 2015

No description provided.

@SparkQA
Copy link

SparkQA commented Sep 1, 2015

Test build #41845 has finished for PR 8543 at commit 56e746c.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • class DCT(JavaTransformer, HasInputCol, HasOutputCol):
    • class SQLTransformer(JavaTransformer):

@andrewor14
Copy link
Contributor

andrewor14 commented Sep 1, 2015

@davies looks like the test needs to be modified.

@SparkQA
Copy link

SparkQA commented Sep 1, 2015

Test build #41891 has finished for PR 8543 at commit 666373b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor

andrewor14 commented Sep 3, 2015

Merged into master and 1.5, thanks @davies

asfgit pushed a commit that referenced this pull request Sep 3, 2015
[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter
b846a9d 
Author: Davies Liu <[email protected]>

Closes #8543 from davies/preserve_page.

(cherry picked from commit 62b4690)
Signed-off-by: Andrew Or <[email protected]>
@asfgit asfgit closed this in 62b4690 Sep 3, 2015
@davies davies mentioned this pull request Oct 30, 2015
Closed
asfgit pushed a commit that referenced this pull request Oct 30, 2015
@davies
[SPARK-11423] remove MapPartitionsWithPreparationRDD
45029bf 
Since we do not need to preserve a page before calling compute(), MapPartitionsWithPreparationRDD is not needed anymore.

This PR basically revert #8543, #8511, #8038, #8011

Author: Davies Liu <[email protected]>

Closes #9381 from davies/remove_prepare2.
ashangit pushed a commit to ashangit/spark that referenced this pull request Oct 19, 2016
[SPARK-10379] preserve first page in UnsafeShuffleExternalSorter
aa75b4c 
Author: Davies Liu <[email protected]>

Closes apache#8543 from davies/preserve_page.

(cherry picked from commit 62b4690)
Signed-off-by: Andrew Or <[email protected]>
(cherry picked from commit b846a9d)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@davies @SparkQA @andrewor14