-
Notifications
You must be signed in to change notification settings - Fork 28.9k
[SPARK-32160][CORE][PYSPARK][FOLLOWUP] Change the config name to switch allow/disallow SparkContext in executors. #29340
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-32160][CORE][PYSPARK][FOLLOWUP] Change the config name to switch allow/disallow SparkContext in executors. #29340
Conversation
dongjoon-hyun
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, @ueshin !
|
Test build #127005 has finished for PR 29340 at commit
|
|
+1 I think this makes more sense. |
|
Test build #127013 has finished for PR 29340 at commit
|
|
retest this please |
|
I am merging this to get this in Spark 3.0.1. The test failure in Kafka will be fixed at #29343. |
|
Merged to master. |
|
Please note that there is a PR for the 170 |
|
Test build #127023 has finished for PR 29340 at commit
|
|
Test build #127021 has finished for PR 29340 at commit
|
What changes were proposed in this pull request?
This is a follow-up of #29278.
This PR changes the config name to switch allow/disallow
SparkContextin executors as per the comment #29278 (review).Why are the changes needed?
The config name
spark.executor.allowSparkContextis more reasonable.Does this PR introduce any user-facing change?
Yes, the config name is changed.
How was this patch tested?
Updated tests.