Skip to content

Conversation

@davies
Copy link
Contributor

@davies davies commented Sep 9, 2014

Aggregate the number of bytes spilled into disks during aggregation or sorting, show them in Web UI.

spilled

This patch is blocked by SPARK-3465. (It includes a fix for that).

@SparkQA
Copy link

SparkQA commented Sep 10, 2014

QA tests have started for PR 2336 at commit fbe9029.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 10, 2014

QA tests have finished for PR 2336 at commit fbe9029.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 10, 2014

QA tests have started for PR 2336 at commit fbe9029.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 10, 2014

QA tests have finished for PR 2336 at commit fbe9029.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 11, 2014

QA tests have started for PR 2336 at commit fbe9029.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 11, 2014

QA tests have started for PR 2336 at commit fbe9029.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 11, 2014

QA tests have finished for PR 2336 at commit fbe9029.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 11, 2014

QA tests have finished for PR 2336 at commit fbe9029.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 11, 2014

QA tests have started for PR 2336 at commit fbe9029.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 11, 2014

QA tests have finished for PR 2336 at commit fbe9029.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Sep 11, 2014

QA tests have started for PR 2336 at commit 7e4ad04.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 11, 2014

QA tests have finished for PR 2336 at commit 7e4ad04.

  • This patch fails unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to do something similar to what you did in #2338 here, i.e. do it only if this is local mode?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will rebase it after #2338 is merged.

@SparkQA
Copy link

SparkQA commented Sep 12, 2014

QA tests have started for PR 2336 at commit 1245eb7.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 12, 2014

QA tests have finished for PR 2336 at commit 1245eb7.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class CreateTableAsSelect(
    • case class CreateTableAsSelect(

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few lines prior to this, there was a comment

# CloudPickler needs to be imported so that depicklers are registered using the
# copy_reg module.

If this import is no longer necessary (was it ever?), then we should delete that comment, too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couldpickle is imported by serializers, so it's not needed here. The comments are removed.

@JoshRosen
Copy link
Contributor

This looks good to me.

@SparkQA
Copy link

SparkQA commented Sep 14, 2014

QA tests have started for PR 2336 at commit e37df38.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 14, 2014

QA tests have finished for PR 2336 at commit e37df38.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@asfgit asfgit closed this in 4e3fbe8 Sep 14, 2014
@davies davies deleted the metrics branch September 15, 2014 22:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants