Skip to content

Commit e807118

Browse files
tgravescsdongjoon-hyun
authored andcommitted
[SPARK-31055][DOCS] Update config docs for shuffle local host reads to have dep on external shuffle service
### What changes were proposed in this pull request? with SPARK-27651 we now support host local reads for shuffle, but only when external shuffle service is enabled. Update the config docs to state that. ### Why are the changes needed? clarify dependency ### Does this PR introduce any user-facing change? no ### How was this patch tested? n/a Closes #27812 from tgravescs/SPARK-27651-follow. Authored-by: Thomas Graves <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent d21aab4 commit e807118

File tree

1 file changed

+2
-1
lines changed
  • core/src/main/scala/org/apache/spark/internal/config

1 file changed

+2
-1
lines changed

core/src/main/scala/org/apache/spark/internal/config/package.scala

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1216,7 +1216,8 @@ package object config {
12161216

12171217
private[spark] val SHUFFLE_HOST_LOCAL_DISK_READING_ENABLED =
12181218
ConfigBuilder("spark.shuffle.readHostLocalDisk")
1219-
.doc(s"If enabled (and `${SHUFFLE_USE_OLD_FETCH_PROTOCOL.key}` is disabled), shuffle " +
1219+
.doc(s"If enabled (and `${SHUFFLE_USE_OLD_FETCH_PROTOCOL.key}` is disabled and external " +
1220+
s"shuffle `${SHUFFLE_SERVICE_ENABLED.key}` is enabled), shuffle " +
12201221
"blocks requested from those block managers which are running on the same host are read " +
12211222
"from the disk directly instead of being fetched as remote blocks over the network.")
12221223
.booleanConf

0 commit comments

Comments
 (0)