Skip to content

Conversation

@gatorsmile
Copy link
Member

What changes were proposed in this pull request?

When running IntelliJ, we are unable to capture the exception of memory leak detection.

org.apache.spark.executor.Executor: Managed memory leak detected

Explicitly setting spark.unsafe.exceptionOnMemoryLeak in SparkConf when building the SparkSession, instead of reading it from system properties.

How was this patch tested?

N/A

@gatorsmile
Copy link
Member Author

cc @cloud-fan @kiszk

@SparkQA
Copy link

SparkQA commented Aug 16, 2017

Test build #80751 has finished for PR 18967 at commit c6cca96.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gatorsmile
Copy link
Member Author

retest this please

@cloud-fan
Copy link
Contributor

LGTM

1 similar comment
@kiszk
Copy link
Member

kiszk commented Aug 17, 2017

LGTM

@cloud-fan
Copy link
Contributor

shall we also update pom.xml and SparkBuild.scala to remove the special setting for this config?

@SparkQA
Copy link

SparkQA commented Aug 17, 2017

Test build #80753 has finished for PR 18967 at commit c6cca96.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

+1 for updating pom and SparkBuild.scala.

@gatorsmile
Copy link
Member Author

Since the changes made in this PR only affect the tests in sql/core and sql/hive package, I will not change pom and SparkBuild.scala, which also cover the testing in the other packages.

@gatorsmile
Copy link
Member Author

Thanks! Merging to master.

@asfgit asfgit closed this in ae9e424 Aug 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants