Skip to content

[SPARK-1912] Lazily initialize buffers for local shuffle blocks. #2179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 2 commits into from
Closed

[SPARK-1912] Lazily initialize buffers for local shuffle blocks. #2179

wants to merge 2 commits into from

Conversation

@rxin
Copy link
Contributor

@rxin rxin commented Aug 28, 2014

This is a simplified fix for SPARK-1912.

@rxin
Copy link
Contributor Author

rxin commented Aug 28, 2014

Note that this was previously fixed by @cloud-fan in #860. This is a simpler fix.

cc @cloud-fan @ash211

@rxin
Copy link
Contributor Author

rxin commented Aug 28, 2014

Jenkins, test this please.

@SparkQA
Copy link

SparkQA commented Aug 28, 2014

QA tests have started for PR 2179 at commit 66679d2.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Aug 28, 2014

QA tests have finished for PR 2179 at commit 66679d2.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@mateiz
Copy link
Contributor

mateiz commented Aug 28, 2014

Good catch, wonder why we did it the long way before.

@SparkQA
Copy link

SparkQA commented Aug 28, 2014

QA tests have started for PR 2179 at commit b2f0e9e.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Aug 28, 2014

QA tests have finished for PR 2179 at commit b2f0e9e.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor Author

rxin commented Aug 29, 2014

Jenkins, retest this please.

1 similar comment
@rxin
Copy link
Contributor Author

rxin commented Aug 29, 2014

Jenkins, retest this please.

@SparkQA
Copy link

SparkQA commented Aug 29, 2014

QA tests have started for PR 2179 at commit b2f0e9e.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Aug 29, 2014

QA tests have finished for PR 2179 at commit b2f0e9e.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor Author

rxin commented Aug 29, 2014

Thanks for looking at this. Merging in master.

@asfgit asfgit closed this in 665e71d Aug 29, 2014
@cloud-fan
Copy link
Contributor

cloud-fan commented Sep 1, 2014

It's much neater and simpler 👍

xiliu82 pushed a commit to xiliu82/spark that referenced this pull request Sep 4, 2014
@rxin @conviva-zz
[SPARK-1912] Lazily initialize buffers for local shuffle blocks.
92dba2a 
This is a simplified fix for SPARK-1912.

Author: Reynold Xin <[email protected]>

Closes apache#2179 from rxin/SPARK-1912 and squashes the following commits:

b2f0e9e [Reynold Xin] Fix unit tests.
a8eddfe [Reynold Xin] [SPARK-1912] Lazily initialize buffers for local shuffle blocks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@rxin @SparkQA @mateiz @cloud-fan