Skip to content

Commit 1c1b7ec

Browse files
committed
Add a migration guide.
1 parent 27c0073 commit 1c1b7ec

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

docs/core-migration-guide.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,10 @@ license: |
2222
* Table of contents
2323
{:toc}
2424

25+
## Upgrading from Core 3.0 to 3.1
26+
27+
- In Spark 3.0 and below, `SparkContext` can be created in executors. Since Spark 3.1, an exception will be thrown when creating `SparkContext` in executors. If you need to create `SparkContext` in executors, you can allow it by setting the configuration `spark.driver.allowSparkContextInExecutors` in Scala/Java executors or `spark.python.allowSparkContextInExecutors` in PySpark executors.
28+
2529
## Upgrading from Core 2.4 to 3.0
2630

2731
- The `org.apache.spark.ExecutorPlugin` interface and related configuration has been replaced with

0 commit comments

Comments
 (0)