Skip to content

Conversation

@davies
Copy link
Contributor

@davies davies commented Oct 30, 2015

Since we do not need to preserve a page before calling compute(), MapPartitionsWithPreparationRDD is not needed anymore.

This PR basically revert #8543, #8511, #8038, #8011

@SparkQA
Copy link

SparkQA commented Oct 30, 2015

Test build #44692 has finished for PR 9381 at commit 374ec3b.

  • This patch fails MiMa tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 30, 2015

Test build #1962 has finished for PR 9381 at commit fed4ab6.

  • This patch fails MiMa tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 30, 2015

Test build #44696 has finished for PR 9381 at commit 258d275.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@yhuai
Copy link
Contributor

yhuai commented Oct 30, 2015

Avoid of using mapPartitions can help us reduce the costs on closure cleaner. Should we create MapPartitionsRDD directly?

@davies
Copy link
Contributor Author

davies commented Oct 30, 2015

@yhuai I just rollback the changes, we use mapPartitions a lot, they should not have much different.

@davies
Copy link
Contributor Author

davies commented Oct 30, 2015

Since this PR just revert some patches, I will merge it to unblock #9383

@asfgit asfgit closed this in 45029bf Oct 30, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants